BsidesLV 2024 - Common Ground - Tuesday
Show transcript [en]
[Music] [Music]
[Music] he [Music]
[Music] [Applause] [Music] he
[Music]
[Music] why
[Music] h [Music]
[Music] w a [Music] [Applause] [Music] [Applause] [Music] [Applause] [Music] I'm just I'm just tring to give you something [Music] I'm just tring give you something okay I do I'm just tring give you something [Music] w [Music] [Applause] [Music] [Music]
[Music] [Music] I'm just TR to give I do you I'm just trying to give you something [Music] I'm just okay I do I'm just trying to give you something [Music] w
[Music]
[Music] [Music]
[Music] a
[Music]
[Music] he [Applause]
[Music]
[Music] [Music]
[Applause]
he
[Music] n [Music] a [Music] n [Music]
[Music] [Music]
[Music]
[Music]
[Music]
[Music]
[Music] [Music] [Music] a [Music] [Applause] [Music]
[Music]
[Music] oh
[Music] a [Music]
[Music] [Applause] [Music] hey hey hey [Music] [Applause] [Music] a
[Music] he
[Music]
[Music]
[Music]
[Music] TR oh [Music] hey hey hey hey
he hey hey hey hey hey [Applause] [Music]
[Music]
[Music]
[Music] [Applause] [Music]
[Music] [Applause] [Music]
[Music] [Music]
[Music]
[Music] [Applause] [Music]
[Music] w [Music]
he
[Music] h a [Music] o [Music] [Applause] [Music] [Applause] [Music] [Applause] [Music] I'm just TR to get this okay after don't I'm just try to give you [Music] something I'm just trying to give you something do I'm just to something [Music] w
[Music]
[Music] [Music] I'm just to give you something I do for you I'm just trying to give you [Music] something I'm just trying to give you something okay I do I'm just trying to give you something [Music] m [Music] w
[Music]
[Music]
[Music] [Music]
a [Music]
[Music]
[Music] [Applause]
oh [Music]
[Music] [Music]
[Applause]
oh
[Music]
[Music]
[Music]
[Music] oh
[Music]
[Music]
[Music] [Music]
[Music] [Applause] [Music]
[Music]
[Music]
[Music]
[Music] [Music]
[Music] [Applause] [Music]
[Music]
[Music]
[Music]
[Applause] [Music] hey hey heyy hey [Music] [Applause] [Music] [Applause] [Music] he [Music]
he
[Music] oh [Music]
[Music] [Music] track [Music]
Cody no one wants to sit next to you you're scaring people out
well the intimidation ended
someone
yeah see good morning everyone good morning welcome to bsides Las Vegas this this talk is painful learnings of applying Ai and security and we have our speakers today Anton and Chris um we'd like to thank our sponsors especially our Diamond sponsors Prisma cloud and vanta and our gold breaker srep it's their support along with our other donors sponsors and volunteers that make this event possible um for this for cell phones uh these talks are being streamlined live um except for Sky talks and as a courtesy to speakers in audience we ask that you check to make sure your cell phones are set to silent um if you have a question uh make sure to point at the
mic in the audience to the audience so that the people know where it is okay all right thank you and here are our speakers hi everyone um happy to be here to do the first talk kol and I it's the first time for us in bide Las Vegas we did give talking other bides and other places but um I want to do a a first intro kind of contract uh between us and you guys we will share the slides later if anyone wants I I just rigged it a little bit before the session so I need to redo it and upload to slides share and share it if at any point you feel that it becomes
a product Peach we say mob too many times because that's our company boo us okay loud like Boo something like this yeah exactly right this is not the point right we are not doing the product bech here we are going to talk about our research about our work what we did if you later want to know about it of course we'll be happy to do that um if you like it let us know take pictures post on LinkedIn make it fun and let's start so a little bit about myself and I will say ahead of time it's weird for me to Stand Here stand still but there's I can't take the microphone so it will be
weird I was born and raised in Israel if anyone has issues with did sorry um I live in Massachusetts in 2016 with my wife three three dogs three kids three dogs sadly two dogs since two weeks ago um it is sad I used to run long distance but if anyone here knows the weather in Massachusetts you can't run the entire year so half a year you're running and then you lose everything and I have I'm in the absc business since 2007 I was a developer turned to product manager and now I co-founded and I'm the CEO of mob if you want to connect with me I'll be happy to As Long as You don't send me
spam and bdr stuff and K yeah hello everyone I I'm really happy to be here it's a huge like it's very very good uh I mean like I enjoy it uh and thanks a lot uh for coming here so I I was traveling around the world for last like 10 years and uh I I am 15 over the 15 years in uh security and uh software engineering currently I live in Amsterdam uh with my wife and two kids and uh please make some noise for my wife she has a birthday today so while I'm speaking here yeah um and uh I am a co-found and engineer at mop so I'm leading the security research team
there we need to drink for that tonight yeah okay so we'll start with the agenda it's we have a lot of slides so I don't expect you to take pictures again we will share all those slides if you want to take pictures take pictures of K it's better uh we are going to talk about the problem what is the goal what we were trying to solve with with our research um the different steps of how we attack um attacked it how to not use AI how to maybe use AI how to yeah we can do the work without Ai and then the mix that we believe is um the way forward now from our perspective yes startup these days
everyone need to say AI AI was never the goal the goal if you want to solve a problem it's not AI the AI is maybe a way to get to that goal and this is how we um treated it so everyone knows this one right the lovely devops infinity loop that security vendors also want to say hey this is Dev SEC Ops in infinity loop but from my perspective Dev secops is broken and why Dev secops is broken is because you may have Integrated Security scanning into your pipeline you may be caning on every commit you went all the way in the ID are you actually fixing anything most issues that are found are being kicked off down the road
like an old can and are not addressing and the reason that I'm saying Dev SEC Ops is broken because the goal for devops was can we release product to the market faster higher quality everything best let's let's ignore crowd strike for a second but it did prove itself that it works right if Dev sack Ops was the same idea of the bring secure product quickly to the market I don't think we any of us made it because we find vulnerabilities faster we're not fixing them so why are wasting the money at the end we get to something like this all those issues that you didn't fix are piled up and I'm working with some companies that have
hundreds of thousands or even millions of vulnerabilities in their backlog why because 10 years 15 years they didn't fix any of them and now they can't right you need to come with a D9 and and clear all this mess so this is the problem and this is the most worthiest slide I ever had here so don't be worried the idea is there is so much work to do so very much work to do and people can't handle that right they they cannot over the years they decided okay we can't fix it so we won't fix it so what nothing happened yesterday nothing will happen tomorrow but there are Winds of Change and software supply chain is the thing if I am a if I'm a
software provider and I provide my software to another company and they provide software any vulnerability that I that I have in my code are now their vulnerability we are running on AWS if AWS has a vulnerability that's my vulnerability now and the the industry understands it between this and PCI 4 by the way that next year will be mandated to fix stuff um yeah you can't just sit on your hands and not fixing things anymore and that's what we were trying to do so our goal was to create automatic first party code fix we're not talking about third party not upgrading your library the code that you actually write that your developers write by the
way any developers in the room nice security people what else any CEO in the room okay we'll go to talk to you later um so what we were trying to do is basically minimize the meantime to remediate you see researchers veric code and all that nine 3 months to fix an application security High severity issue I mean 3 months that's crazy can we do it faster faster that was the first goal second goal we're developers in nature I mean I used to be still is we want to help developers get rid of that security pro problem to focus on what they are they love best what they were hired to do right build cool stuff um we want them
to build cool but secure stuff and we want to allow fix it scale and this is this was the hardest challenge for us can we actually allow a company let's say a big bank has thousands and thousands of SQL injection yes they do if you don't know they do I know um kind of scary I know someone can get all the information easily but this is what we're going to focus on because the other issues are hard to me to um measure and how we are going to do that in this session with AI uh who is old enough to recognize this guy cool okay back to C back to okay when we are talking about AI
first thing uh come into our mind is basically generative AI today is the narrative right like you have um all this Char pts Gemini all those models a lot of them and um like a lot of people say they really performs great on uh on such um tasks so first thing is very very naive approach which we also implemented to do our research and try it so the and the the naive approach here is like you have basically a template of the prompt and in this template you basically have um keywords right like I have xss vulnerability like xss you put the vulnerability you have uh in the following JavaScript code so here is typescript Java whatever you use
then you have the triple back tick and you have the code sample the vulnerable code sample and the vulnerability is in line uh the line of code fix it like fix it as um as ask for llm to do something right and uh here on the screen you can see like very typical xss injection it's reflected xss so um we take data from the location hre which is the URL in the browser bar uh and we concatenate with some text and put it into the body of the page right so if uh hiker put the script tag in the URL um you're going to have a trouble so let's see ah and by the way uh this is the screenshot from
um Char PT and for all of the slides here I'm going to use latest CH GPT uh slides was created like two weeks in advance of the conference so uh all all the prompts and all the responses are um pretty much the same today and I'll going to have some links also for the prompts like uh for chpt uh so um the answer from llm is very good in this case like you see it transformed uh what it was before the string concatenation it was transformed to a creation of uh new H1 tag and uh it set text content to the H so it's a good fix no vulnerability here is possible anymore um but you know for the sake of
research we want to try more than one sample right and uh that's exactly the same prompt as on the previous slide but this time we actually have uh reflected one click xss which is uh less common but still very common type of issue so we create uh a tag and we put hre to the window location search and see how we parse the location search in this case like we just uh remove the question mark after the URL and uh put it directly to HF and like you you you can recognize here is a again the typical problem you can put Javascript colum alert one and your code going to be one click vulnerable to uh reflect the TSS again
um let's see how llm fixes it and yeah like um here is like many problems there it's a little bit a lot of code so uh I I highlight for you uh things we are focusing on um basically uh um first problem I see is the llm uses the search param parer to get the something called URL it wasn't like this before remember like it was the just splitting the question mark so um something suspicious going on here and uh um the variable called safe URL like how they actually validate URL how llm actually validate URL in this case it just create a new URL constru so basically it doesn't prevent it from any xss injection or anything like that and
uh for the sake of like to being 100% sure it is still vulnerable I just copy pasted the code from llm directly to the example.com and exploited it in one second so it is vulnerable and um quick recap like what happened actually first of all llm ruined the application logic here because the website like the your code was rely on the fact that what what goes after the question mark actually will be the back URL it will be the part of the hre but llm decide that the URL parameter looks better than uh just question mark and so now we have broken application because probably backend renders a different way probably other parts of the application renders uh
render it in a different way uh uh and the second problem obviously it doesn't fix the vulnerability like as we've seen the injection is still possible so fail let's try to figure out like what happened it was exactly the same prompt right like it's exactly the same scenario what's wrong why one time llm was successful and the second time not and uh I I I don't know answer right because the it's llm it's a kind of black box for now and um but you you can see a really nice example is basically you have two prompts uh one prompt like they they are identical for the developer they are identical I need no JS static file server uh use pure noj no
additional npm packages and the second prompt is exactly the same the difference is only the uppercase n and a DOT between GS and node right like if you send this text to your developer they will produce exactly the same results ah yeah thanks um so and let's let's have a quick look how uh charp performs for this prompt the first example and like I removed half of the code so it's not the complete code sample you can go by the URL and check it out if you're curious but uh general idea like how many know JS developers here any yeah okay some some of you thanks um so this simple is like you create a HTTP
server and in a callback you just work with something called request. URL and it's uh basically what happens after the slash of your domain right in in the URL and uh obviously this is direct user input it is vulnerable for like I mean it it can introduce the directory traversal vulnerability but uh in this case llm did really good job it's a puff normalize which will move the do do SL segments to the beginning of the URL or remove them and then replace pattern to cut the do do/ segments at the beginning of the URL so sanitization happened here I I wouldn't do it this way I actually if I would implement this code I would do it
differently but it's a good solution it works it's not vulnerable but for exactly the same prompt with the uppercase letter and a DOT we suddenly have no validation at all like uh why why it's uh same for me it's the same text I need to say about uh also like different models May behave differently right because you have a tokenizer in in before the model you have different model architectures which may normalize kind of meaning of the prompt and uh make it better but at least like chpt which is one of I guess the most popular of today is making this mistake thanks um so short story we we started to do the automatic remediation
before Chad GPD came to life and suddenly in November or December 2022 came to life we were scared I mean do we have even a startup and I was going to Black at Inu and I told the team hey stop whatever you do the entire Team all three people stop whatever you do go and research it so we will see if we have a startup or should we close or is there a technology to use luckily it's not doesn't work um so we we did a research we started with with gpt3 we did a research uh started with gpt3 then three and a half then four also and validated basically we wanted to see how how good
is open AI in fixing code vulnerabilities we did we gave it an easy task we chose two easy two known applications web goat and um shop right uh we assumed actually that it was trained on them but whatever we scanned those with two SAS providers and we started to ask G um open AI to fix them we gave it decent prompt better than what most developers would probably do focus it on it and still the results will extremely underwhelming the the research is also published um on our site but the idea is that 29% of the fixes they were 104 I think I think something like that yeah 20 29% of them were good fixes not necessarily
following best practices but the fixes like Carol just showed the fixes do the work the problem by the way is if you fix something not best practice you will run the SAS skan again it will probably report it again because it won't know that it was fixed limitation in sast but let's let's give it that then 19% it didn't ruin the application which is good it actually touched the code in the right place which is good sometimes it even introduce new vulnerabilities though it didn't fix so you would say hey why is that not the worst wait the worst was that 52% were just bad GPD changed code in unrelated areas it did stuff like um validate domain create a
new method validate domain you look down there is a template of validate domain take an input and comment inside do the sanitization here so it didn't do didn't give you any help now this is just it's not just that right GPT well AI in general has some more problems with it some some more challenges that you need to address one is the context window the information that you give AI in order to fix something is usually small it's always expensive if it's not small yes I know there are some that can take millions and billions of tokens it's going to be extremely expensive and we can't build a solution based on that the other one
is let me put it here it's easier can you can you still hear me okay um the other one is parsing the LM output right you want to do something automatic with it not a developer goes to a chat and and and fix things as I said we were looking and doing things at scale so parsing it didn't work as expected you tell the llm hey generate Json generate something else and then large files the more information you give GPT or others the more chances it will hallucinate it will try to help you so much in areas that you didn't even ask for and it's a mess that you need to clean later so I'll do a quick stop just because
um you seem tired and we just started maybe we're boring but I wanted to make sure that you understand that we are not H it's not our interpretation we actually tested it and I'll show I like this slide because I'm making fun of us and and others um so for example we gave we asked GPT to fix not GPT this one was I think it's not today right it's mix TR I think um fix SQL injection here look at the code um there is no code change here there is just a comment or in this one we ask it to fix something uh the we ask you to fix hardcoded domain the code change is good it replaces it from HTTP
to htps Great nothing to do with a problem so you may say Okay Aon and koll you seem like fun guys but you're not smart enough you didn't build a good solution there are really good excellent um companies out there that build something with AI to remediate Let's shame them now without putting any logo because we like them actually so this is one now when you think about this one does that look like they fix hardcoded secret it's not now I'm not I'm not ditching them it's really hard to do those things and asking the AI to do it it's even worse um this one for example fix equal injection oh you just deleted the vulnerable line great problem's gone
right if you have vulnerable code delete it no problem then one of our favorite tools I couldn't mask this one because everyone would recognize it anyway but the ideaas and they are one of the best ones that we we saw um GitHub did the fix here why or why did they decide to delete this line no idea and we move on one of the things that when we started early on we knew you cannot fix everything you cannot fix things that require architectural changes for example fixing a weak cyer encryption is very easy let's replace it with a strong Cipher encryption B you just broke your application because you fix it in one place in one component you have 10 other
components that are still using the old encryption key uh um string so algorithm so you broke your application so it's not just us those are only the tools that we got our hands on um there are other tools out there I invite you to try maybe maybe they figure it out we haven't so far so let's review the goals that we had were we able to re to minimize the mttr yes but only if the person that use the tool knows actually how to write it in the first place so not so much can we fix that scale definitely no every fix needs to be vetted and verified and so on and so on so let's move to the next
step how to maybe use gen when we have a fix so first thing let's do custom prompt for every vulnerability and often for different code patterns because the same vulnerability the same issue type the same xss different code patterns will require different fixes second thing carefully pick the context you only wanted to give it the minimum that it needs in order to fix because as I said The more you give the more problematic the more chances that it will make mistakes so remove everything that you can and um well everyone that talks about AI needs to mention rag retrieval augmented generation basically the more context that we give the more information that we tell the AI this is
how I would do it if I was you Mr AI so try to do it this way the more you give the better the more consistent the results will be and last when it generates the the the when the AI generated the code we wanted to make sure that it's still compilable because we know it makes mistakes in the generation itself let's look at what kol just showed earlier right the same just different background same everything else this um the fix failed earlier to fix it now we told it fix it but you know what we told him how to we told the AI how to fix it maybe I'll say her we told her how to fix it
so we told um AI hey please fix it uh by by validating the URL right so we look at the results and great great success we finally finally cracked the code we figured out how to use AI for automatic remediation so what we need for that custom prompts the downside for every code pattern you need to understand it and you need to write different custom um different prompt not only that you need to understanding the code that you see now is a different pattern it's a new pattern and you need the new custom P prompt for it so let's think about that the second thing we mentioned is the context right I don't want to give
the AI all the file the entire file because then it will make mistakes so can we ask the LM to spe fix specific specific method instead of the entire file yes we can but the Tain flow for those understand static analysis how it works and I won't go into that because we don't have that much time but it goes from source to sync from the place where user input came into your application until it reached its final destination and it may go through multiple methods in a large file so you can't do that and sometimes it's more than one file so I can't give just one method to the AI and tell her hey fix it second the source of
the vulnerability can be even static variable in the class so you won't even see that as part of the method that you want to send the fix may require adding some imports if I give it only a method where will it put the Imports I want to take that and automate it into my product so when we were looking at custom prompts did we meet the goals that we said at the beginning did we minimize the mttr mostly yes we got to a place that most of the fixes were accurate it didn't require a lot of check but if still need to check every single fix again you may tell aan you don't look that smart maybe you don't understand
how to use this AI so we went to the company that everyone loves GitHub and GitHub in their article they said it and they were referring to their new code automated remediation tool gen can generate fixes for more than 90% of the vulnerability types which is awesome think about it it's awesome 90% of the vulnerabilities that you have the different types they can provide a fix not only that over two3 of those fixes can be merged with little to no edits sounds good right anyone doesn't think it sound good sounds good some say they don't and the reason is because the way I read it and I'm I'm a hater maybe onethird of the fixes you have nothing to do with them
and the other two3 you need to check them one by one so did we solve the fix at scale no did you fix did you help developers yes you helped them you gave them an answer they can check it two3 of the time you save them a lot of time but it's not at scale so we talked a lot and again I'm trying to be loud and noisy but some of you are slowing down slowly in their chairs so let's do a a small act um you know activity I'm showing you a code here anyone knows where this code came from I will send it I will give a mob shirt uh hat to someone no one so
that came from web goat now this code has vulnerability I'll make it bigger so you can see um basically taking a account name as an input concatenating to a SQL in SQL query and just executing it obvious squal injection right simple so what we did and this is by the way not from two weeks ago I created this one in a while ago but sorry um I asked chpt can you fix the SQL injection in this code and chpt did amazing part work it told me hey is this code is actually vulnerable to SQL injection due to fact that the account name is concatenated what you need to do is do a preper statement with the
account name great and that's what they did now the problem is that this fix will also break your application anyone has any idea I'll make the the problematic parts I'll make them bigger again do you see any problem
here what l no no login count account is actually the the good part the account name is the problem account name is being assigned to user ID if you look at the template as a schema user ID is actually an integer how could the AI know that it doesn't have access to the schema so actually the AI should have known that because account name is not surrounded by single quotes So it cannot be a string if it was um a string you would see here a single quote and you would see here another single quote but in and every developer not every most developers will look at this code fix and they would say oh my God it saved me
time commit if they have regression testing great it will break if they don't have regression testing production would break so this is a problem now I'll pass it to Chris again yeah um this is kind of a little bit um off topic for the presentation but I really wanted to share for you uh this little bit of knowledge uh we also mentioned the problem that you're uh that it's really hard to parse output of llm you actually ask llm sometimes oh give me Json and you may see like okay it's a perfect Json but suddenly at some point if you run it at production you will see in logs like fail to parse fail to parse invalid character why because
LM not always obey what you say it to do so uh sometimes you ask it for Json and it's not Json uh sometimes it also changes the unrelated parts of the code so here on this slide you can see this is the same code sample from the one of the first slides but here like the first line is tabs like the top character and the second line is basically spaces you know developers do that I don't know why but uh some sometimes it happens and we when we are opening a pool requests we don't want actually developers to see changes unrelated to their vulnerabilities so our goal is to change as minimal code as possible because it
gives Trust of the developer they can read it and understand why we changed it so we don't want llm to touch anything in the formatting and uh the trick here is like how we figure out how to parse the llm responses without asking it for Json or other formats we just basically add pipes to each code line L at the beginning the pipe symbol at the beginning of line and we ask like this tiny simple addition to the prompt keep the pipe symbol at the beginning of each code line and the results are really good it never mess up with the formating anymore and also it's super easy to parse it in Python so if you use Python
uh you just basically need four lines you split it by the new line check if the first symbol is pipe so it's a code line uh um and yeah again it helps to keep preserve original code formatting it helps to uh ignore additional text from LM because LM would often say like oh here is the fixed code for you triple backtick JavaScript new line then the piece of code and sometimes you don't have JavaScript you don't have backticks or like you know it it can be whatever and avoid Jason because Jason was a huge pain for us uh in pars and the llm responses so uh I hope yeah this is useful will be useful for you um and like we already
talked about two approaches today one is basic template which not works uh second one is like custom prompts and uh more sophisticated ways to query the llm to give the more more and more precise context but let's talk about something else like what if we don't want to use AI at all and it is actually implemented in in many places today so it's slow to implement but you you can see some of the famous companies here like mob uh es Lind um yeah and I I'm mentioning es L here on purpose because it's a code Quality Tool and all JavaScript developers are familiar with the with that but uh it has amazing feature of D-
fix uh and D- fix is basically if es see the smelly pattern the bad pattern and the code it just replaces it to the good pattern that's what we want for security issues um I also want to mention here open reite uh tool which is the Java um refactoring tool you can write your own kind of rule how to change the code and it will walk walk through all your files and change the code for you um and Sam grap of course they have automatic remediation like you can when you create an rule for Sam grap you actually can say this is the way how I fix it and what common for all of these amazing
tools um they they all work in the same way they basically pass the code to what called abstract syntax trees uh we're going to talk about it in a minute but all of you who is familiar with the code analysis you probably already looked at what is IST and uh second thing you you need to do is to understand what is your vulnerability and where is it located like pass the vulnerability report and you're going to need it anyway for any of those approaches right like because you have reporting from the famous uh s provider and uh you want to fix their signals so you need to parse their reports uh then it's kind of easy you
match the original code in form of IST to the report you figure out what's the code pattern what actually you need to fix right like what replace to what um and you need to apply the changes in some form to deliver it to the developers I mean you need to open a pool request create whoa whoa uh okay okay 10 minutes I see um so yeah you need to deliver it somehow to developers um first let's quick look at what abstract syntax 3 is about uh like our ultimate goal is to replace vulnerable line of code it's from first slide I guess um to nonvulnerable uh line of code in this case we want to apply don't
purify as a sanitizer it's a typical library in JavaScript to uh remove the dangerous text from the input uh and yeah I know this is super cumbersome but um like this is how abstract syntax 3 looks like it's uh basically you have a lot of nodes connected some edges it's a tree it's why it's sub syntax tree um and um basically each node represents part of the code in the code and each Edge represent the relation between nodes and here you can see the entire string is expression statement in terms of uh I think it's three seater uh parer um and assignment expression is basically something equals to something and on the left part you have object dot property
and like basically what you want to find is inner HTML which is reported by the SS provider s providers say this inner HTML is introduces the uh vulnerability to your code and you want to basically just take the right part and wrap it to the D purify so you just create another uh abstract syntax 3 node which is call expression and identifier is don't purify function and the argument is whatever was before you started to work with the code um so kind of easy uh and uh you you also need to far as the vulnerability report sometimes um it's straightforward I I really appreciate s providers who use the sarif format which is the facto standard today it's a
static analysis results interchange format and I don't appreciate the S providers who trying to introduce their custom formats and uh like it's not only about pars and XML custom Json protabs no it's also about like the data is missing sometimes you you don't have like code position or line position or file information uh yeah yeah yeah yeah I'm speeding up yeah um you also need from the report you need to figure out source and sync like what aan already kind of mentioned uh and eventually where is the best place to apply the fix and voila you have the um the code changes and you just render it as a g def and uh send it
to your developer um and yeah let's do some conclusions here the reviews uh basically it helps indeed minimize mttr because it gives solid advice it helps to fix it scale because it works similar to es lint which is I mentioned before but it's for us for the platform which is doing fix it's very hard to to uh maintain the data like the the res like the the code fixers uh for each code pattern and it requires a strong security team which we have uh um to produce the good fixes really good fixes which not break the code which not and which um like satisfy the standards today and it's really hard to scale unfortunately as you migrate from your
your check marks to sneak and you want to add new sus provider suddenly or you migrate from your JavaScript to your go I don't know and uh suddenly all the work should be done again um so and uh this is the final so it's it was the like ladder of three different stages and this is the last stage we have today it's how we actually believe correct way to use AI today we call it hybrid AI because it's a mix of two approach which is the pure algorithmic and the second one which is the custom prompting and I'm going to show it on uh on a n der reference vulnerability um which is reported by
fortify scanner and uh it may be the reason like some people say it's the reason uh for a crowd strike incident I don't know if it's true or not but um let's see uh and uh like you can see that's get environment variable it can get it's sh by the way not C++ but the idea is the same um get environment variable can return the null Point null if the CMD variable is not defined so the second line will break the code because CMD can be null and uh the obious automatic fix to do is wrap the vulnerable line to um if statement just check the CMD if it's not null and it's good but it's
good and easy for easy cases some cases are not that easy you can see here the vulnerable line is settings and settings is actually dictionary and inside the dictionary you have the object and object contains object field contains array and it's another dictionary so try to figure out what if condition uh for this line needed and uh instead of writing the algorithmically how to like consider all those cases and create the code which will produce the fix uh the if condition I'm talking only about the if condition here uh we can ask llm and actually for such simple tasks because LM don't need any context don't need anything like you're basically give it very solid statement what you want to do
and it produces amazing responses although um sometimes llm introduces additional components you don't expect like lse statement in this case it wasn't in the original code but you don't care because you actually can do the same procedure as you do for the source code you can put it to your I parer and just extract the if condition uh data and incorporate this to uh the customer code eventually um yeah I guess it's okay so I'll try to go faster probably have 2 minutes and if not whatever um were we able to reduce them in time to remediate yes we were um you don't trust us you can try it but you can build a solution
like this on your own that's why we showed also how to power the Json and all that uh yeah we were able to reduce the Min time to remediate and the fixes are following the same pattern all the time yes it's 2 minutes 2 minutes everyone um but the problem that we had before it was really really slow to add rules it's still not fast it's not that hey kol fix this it's more hey kol let's build the rule to fix this it takes a day two days sometimes now but it's faster to developer the fix and this is where we wanted to reach so from our perspective this is a well done this is
time you can clap if you want but we didn't finish yet so let's wait with that um summary okay so let's remember what is AI in general AI is a glorious copy paste right that's all it does it has a huge place to copy from it copies and it past it in a nicer place so as Lu lustra says and I'm probably butchering his name thinks of a model as an overeager Junior employee that blurts out an answer before checking the facts it knows where to call it it's not sure that it is the right answer that you want but hey I gave you an answer so great so what do you need to do it is a
great tool but it requires supervision and it need it's non predictable it's non deterministic you can't just automate the hell out of it it helps save developers time even with a very basic approach it helps but you can never trust gen I know that insecurity will have to say trust but verify I think that in AI insecurity you need to say verify and then trust and then verify maybe but you can't just run any questions
Yesa Solutions I keep thinking about false positives on just the static analysis detection that if you you know there's a I mean clearly you want that to be low but there's going to be some percentage of false positives and false in detction and then if you're try to Auto know correct these things what happens when so I'll I'll repeat the questions quickly everyone knows that SAS is notorious to having a lot of false positives um I got to stop sign uh but I'll still go uh notorious um for false politiv what is a false politiv usually it means that the code is bad but not vulnerable in most cases sometimes it's there is no problem in your solution
what is not broken you shouldn't good code you can't fix a good code but if the code is bad you should fix it anyway even if it's not vulnerable and the reason is because if you have copilot in your organization it learns from your pattern it will take the bad pattern now put it in nrow place because as we said it's a glorious copy paste fix the false positives yeah not only not only co-pilot developers love copy paste so um f fix it I mean the alternative is prove the alternative is doing the manual triaging persuading someone that it's a false SP if you're wasting a lot of time just fix it get down with it if
you can uh we will be outside if there are more questions because I'm getting the nasty eye here and we need to leave the [Applause] [Music] [Applause] [Music]
room hey hey hey hey hey [Applause] [Music]
[Music]
[Music] [Applause] [Music]
[Music] [Applause] [Music]
[Music] [Applause] [Music]
[Music] [Music] [Music]
[Music] [Applause] [Music] what [Music]
oh
[Music] oh
[Music]
[Music] oh [Music] a [Music] [Applause] [Music] [Applause] [Music] I'm just dring in I I'm just dring in [Music] something I'm just TR to something I to BR you I'm just trying to give you something [Music] a [Music] [Applause]
[Music]
[Music] [Music] I'm just to I I'm just TR to give you something [Music] I'm just trying to give you something okay I do I'm just trying to give you something [Music] o [Music]
m
[Music]
[Music] [Music]
[Music] d
[Music]
[Music] [Applause]
oh [Music]
[Music]
[Applause]
[Music]
[Music]
[Music]
[Music] oh [Music] oh [Music] la [Music]
a [Music]
[Music] [Music] [Music] [Applause] [Music]
[Music] a [Music]
[Music] [Music] [Music]
a [Applause] [Music]
[Music]
[Music]
[Music] he [Applause] [Music] hey hey hey hey hey
[Music] [Applause] [Music] oh [Music]
he [Music]
[Music]
[Music]
[Music] TR [Music] oh [Music] hey hey hey [Applause] [Music]
hey hey hey hey hey hey [Applause] [Music]
[Music]
[Music] [Applause] [Music]
[Music] he [Applause] [Music]
[Music] [Applause] [Music]
[Music] [Music] [Music]
[Music] [Applause] [Music]
[Music]
[Music]
he
[Music] h oh [Music] oh [Music] [Applause] [Music] [Applause] [Music] [Applause] [Music] I'm just try to get something this okay I I'm just TR to give you [Music] something I'm just TR to give you something I do I'm just TR to give you something [Music] he w
[Music]
[Music] [Music] I'm just TR to something I'm just TR to [Music] something I'm [Music] just I'm just trying to give you something [Music]
n [Music] w
oh
[Music] [Music]
n [Music]
[Music]
[Music] [Applause]
oh and
[Music] [Music]
[Applause] [Music] I
[Music]
[Music]
[Music] he [Music]
[Music]
[Music] [Music] [Music]
[Music]
[Music]
[Music] [Music] [Music] a [Music]
[Music]
oh [Music]
[Music] [Music]
[Applause] [Music] he hey hey hey [Music] [Applause] [Music] he [Music]
he
a
[Music]
[Music]
[Music] hey hey hey [Applause] [Music]
hey hey hey hey hey hey [Applause] [Music] [Music]
he [Music]
[Music]
[Music] [Applause] [Music]
[Music] [Applause] [Music]
[Music] [Music]
[Music]
[Music] [Applause] [Music] he [Music]
[Music]
oh
[Music] h [Music]
[Music] [Applause] w [Music] [Applause] [Music] I'm just to give something I'm just to give [Music] something I'm just I'm just TR to give you something [Music] m [Music] oh [Music] [Applause]
[Music]
[Music] [Music] I'm just I'm just tring you [Music] something I'm just tring you something I do you I'm just trying to give you something he [Music] w
[Music]
[Music] [Music]
[Music]
oh
[Music]
[Music] [Applause]
oh [Music]
[Music] [Music] a [Applause]
[Music]
[Music] and
[Music] n
[Music]
a [Music]
[Music]
[Music] [Music] [Music] n [Music]
[Music]
[Music]
[Music] [Music] [Music] oh [Music]
[Music]
[Music]
[Applause] he hey hey hey hey hey hey [Music] [Applause] [Music]
[Music]
he he
[Music]
[Music]
[Music] St [Music] hey hey hey hey [Applause] [Music]
hey hey hey hey hey [Applause] [Music]
[Music]
[Music]
[Music] [Applause] [Music]
[Music] [Applause] [Music]
[Music] a [Music] [Music]
[Music] [Applause] [Music]
[Music]
oh
[Music] h
[Music]
[Music] oh a [Music] [Applause] [Music] [Applause] [Music] I'm just I'm just TR to give you [Music] something I'm just try to give you something I do you I'm just TR to give you something [Music] he [Music] [Applause] [Music] [Music]
[Music] [Music] I'm just trying to I do I'm just TR to give you something [Music] I'm just try to something I do I'm just trying to give you something [Music] m [Music]
[Music]
[Music] [Music]
[Music] that's
[Music]
[Music] [Applause]
[Music]
[Music] [Music]
[Applause]
[Music]
[Music]
[Music]
[Music] a [Music] oh [Music] n [Music]
[Music] [Music]
[Music] [Applause] [Music]
[Music]
[Music]
[Music]
[Music] I [Music] [Music]
[Music] [Applause] [Music]
[Music]
[Music]
[Music] he
[Applause] [Music] hey hey hey hey he hey [Music] [Applause] [Music] [Applause] [Music]
he [Music]
he
[Music]
[Music]
[Music]
[Music] track [Music] St
[Music] he hey hey hey [Applause] [Music]
hey hey hey hey hey [Applause] [Music]
[Music]
[Music] [Applause] [Music]
[Music] [Applause] [Music]
[Music] [Applause] [Music]
[Music] [Music] [Music]
[Music]
[Music] [Applause] [Music] he
[Music] you oh
[Music] h oh [Music] a [Music] [Applause] [Music] [Applause] [Music] [Applause] [Music] I'm just try to give you something okay I do I'm just tring to give you [Music] something I'm just TR to something I do I'm just TR to something [Music] he [Music] w
[Music]
[Music] [Music] I'm just to I'm just TR to give you [Music] something I'm just try to give you something I BR I'm just trying to give you something [Music] oh [Music] w
[Music]
[Music]
[Music] [Music]
yeah St [Music]
[Music]
[Music] [Applause]
oh [Music]
[Music] [Music]
[Applause]
he
[Music]
[Music]
[Music] la [Music] oh [Music]
[Music]
[Music] a [Music] [Music]
[Music] [Applause] [Music]
[Music]
[Music]
[Music]
[Music] [Music]
[Music] [Applause] [Music]
[Music]
[Music]
[Music]
[Applause] [Music] hey hey [Music] [Applause] [Music] [Applause] [Music]
he [Music]
he
he [Music]
[Music]
[Music]
[Music] track [Music] hey hey hey [Applause]
hey hey hey hey hey [Applause] [Music] he [Music]
[Music] [Applause] [Music]
[Music] [Applause] [Music]
[Music] [Applause] [Music]
[Music] [Music] [Music]
[Music] [Applause] [Music] he
[Music]
[Music]
oh
[Music] h [Music]
[Music] [Applause] [Music] [Applause] w a [Music] [Applause] [Music] I'm just TR to I I'm just TR to give you [Music] something I'm just TR to give you something I do I'm just to give you something he [Music] w
[Music]
[Music] [Music] I'm just I'm just TR to [Music] something I'm just trying to give you something I do I'm just trying to give you something [Music] m [Music]
[Music]
[Music]
[Music] [Music]
[Music]
[Music]
[Music] [Applause]
[Music]
[Music] oh
[Applause] [Music]
ready
[Music]
[Music]
[Music]
l [Music]
[Music]
[Music] [Music] [Music] [Applause] [Music]
[Music]
[Music]
[Music] a [Music] [Music] [Applause] [Music] n [Music]
[Music]
[Music] the [Music]
[Applause] [Music] he hey hey he [Music] [Applause] [Music] a [Music]
[Music] he
[Music]
[Music]
[Music]
[Music] track [Music] hey hey [Applause] [Music] hey hey hey hey hey [Applause] [Music] he [Music]
[Music]
[Music]
[Music] [Applause] [Music]
[Music] [Applause] [Music]
[Music] [Music] [Music]
[Music] he [Music] [Applause] [Music] he [Music]
[Music]
oh
[Music] h [Music]
[Music] w no [Music] [Applause] [Music] [Applause] [Music] [Applause] [Music] I'm [Music] just I'm just trying to give you something [Music] I'm just trying to give you something I do I'm just try to give you something [Music] w [Music] [Applause]
[Music]
[Music] [Music] I'm just TR to this I do for you I'm just TR give [Music] something I'm just tring to something okay I do I'm just trying to give you something [Music] w
[Music]
[Music]
[Music] [Music]
[Music] a
[Music]
[Music] [Applause]
[Music]
[Music] [Music]
[Applause]
[Music]
he
[Music]
[Music]
[Music] the [Music] l [Music] n
[Music]
n
[Music] [Applause] [Music]
[Music]
[Music]
[Music] [Music] [Music]
n [Applause] [Music]
[Music]
[Music]
[Music] he
[Applause] [Music] hey hey [Music] [Applause] [Music] he [Music]
he
[Music]
[Music]
[Music]
[Music] track [Music] got [Music] hey hey hey [Applause] [Music]
hey hey hey hey hey hey [Applause] [Music]
[Music]
[Music] [Applause] [Music]
[Music] [Applause] [Music]
[Music] [Applause] [Music] he [Music] [Music] [Music]
[Music] [Applause] [Music] he [Music]
[Music]
oh
[Music] h
[Music]
a
[Music] [Applause] [Music] [Applause] [Music] oh [Applause] [Music] I'm just I'm just [Music] something I'm just TR to give something I'm just TR to give you something [Music]
awesome all right let's uh I guess let's do this good afternoon everyone welcome to bsides Las Vegas Common Ground uh this talk is how I accidentally became became a hardware hacker by Caleb Davis a few announcements before we begin um I'd like to thank our sponsors especially our Diamond sponsors last p and Palo Alto networks and our gold sponsors Amazon Flex track and Google it's their support along with our other sponsors donors and volunteers that make this event possible um yeah this is being recorded keep you know be respectful please um it's being streamed online when you have a question toward the end I'll come around with the microphone and please speak clearly into it and yeah let's have a good time here
you go here's
Caleb thanks all right how are youall doing today awesome well I'm just going to run you through uh hopefully a light-hearted overview of how I stumbled my way through engineering into Hardware hacking and maybe learn a thing or two on the way mostly what not to do um so yeah we'll just we'll Dive Right into and I got some time for questions at the end so you'll notice the uh the tents here uh you know things changed just a little bit instead of uh how we that should actually have a striketh through but um it's how I became a hardware hacker I think it's uh important for me to note that you know the guy that's in
the the other part of the heart in this photo is uh crucial to this story he and I were both uh electrial engineering students worked at a a restaurant together um worked at multiple different jobs and now you know we're back together again so um this is actually a funny picture when we were double e and we took a staged photo um if you can tell he's soldering with an oscilloscope and uh we're not psychology students or faculty so that's a fun little Kickstart into our careers so like I said I'm just going to run through who I am and then talk about it just a few examples of you know times that I have learned a lesson by mostly
screwing up and uh what that taught me about Hardware hacking and how I think it applies um towards the end of this I'll talk about why everybody should be doing Hardware hacking and you know the this the point of this talk is not to go too far into the Weeds on Hardware hacking find me any other time and I'm happy to to nerd out about whatever you want to all right so a little bit about me um my name is Caleb Davis I'm co-founder of Solace SEC cyber security Consulting um I was an electrical engineer um I guess I still kind of am um at the University of Texas at Tyler um did a ton of embedded development
before that lot of armc microcontrollers free R toss uh for for HVAC so heating ventilation air conditioning we we always used to say we would close our eyes and pretend it was lasers because it was not that exciting um but we did learn a lot along the way as you'll see all right so some one of the first things that I wanted to to talk about um hopefully you can see here you know this is probably the biggest nudge into Hardware hacking for me was really just being a terrible electrical engineer and you know bad at circuit design and and all of you know all of the things encompassing with that I think I've gotten a little bit better hopefully but
um you know one of the things that we often see when we're putting a a board together is that we'll have to you know modify that board or do some sort of rework right so you know i' I've got a few examples up above that I'll talk about but you know the things that we see commonly are the wrong uh component Footprints right if you lay out a part or if you have to design your own part um could be a huge problem if you get the dimensions off by you know even the slightest millimeter you could be talking about a short in a system um incorrect schematic wiring I mean if you're if you're having to Route
multiple schematics and you are reading tons of data sheets right it could be very easy to cross those wires and you know when you cross wires on a schematic it's not a big deal when you turn it on you see that factory installed smoke that I'm sure you know multiple people in here are familiar with um wrong component orientation this is another big one that we've seen where you know Screw Up try to send the board or the component to the back side of the board instead the front side get all confused and then now we've got to you know literally flip transistors over and then solder them upside down so we have actually done that recent too recently I
would say um and then the last thing accidental shorts um you know I've done this recently as well just throwing a via down on top of traces when you shouldn't and and what that means to uh your life and how sad it's going to be to fix that problem um so some of the learning opportunities I kind of alluded to it already but you know when you have when you have these uh issues that come across you can't I mean you can run a new board and you know wait that process and it could be weeks depending on you know your lead time for boards but something that I think is kind of inherent to what we do as as engineers
and tinkerers is that we try to fix it we try to root cause it and then that will inform our next design um my argument here is that you know a lot of those tools that we just did and took for granted kind of propelled me into Hardware hacking because I think that you know the first thing you do when you start looking at uh Hardware board you need something I do actually I believe you uh made a outrageous speaker request when you signed up for this talk oh good lord uh and uh I believe that you asked for the same rer that Nicki Minaj has yes okay so I have uh your 24 pack of
Dan the other half is on ice and an unspecified location great thank you uh but there's there's the on Ice part I've got your uh your dried crazy cranberries I got your uh your almonds here and uh I'm going to go out and get your throat drops and your fried chicken I think she said she has noou too delivered so just let me know where to pick that up okay right that's awesome man thank you the last person who had noou uh ended up uh not being able to speak in this track because she got food poisoning so be careful that great all right thanks for anyway that that's awesome thank you all right I'll fill in the gaps here cuz I
know y'all are probably trying to figure out what's going on um in the the cfp there there was a spot for like special requests for the speakers and I I'm pretty easygoing I don't have any special requests so I asked for Nicki Minaj's ridiculous set of uh Green Room requests and uh they obliged what's going on for for some reason excuse me yeah I I am really sorry but uh I we rece just received a season assist letter from uh Nicki Minaj's oh yeah it it turns out someone was put out it was Nicki Minaj I I didn't realize could patent such a thing I didn't think you could copyright it either yeah hey I'm I'm Chief legal
officer we have one um but yeah it's come to my attention that you've made unauthorized use of my client's copyrighted work entitled performance and security contract addendum for Anika Tanya Mirage AKA Nikki Minaj in preparation of an outrageous speaker request for your 2024 conference we reserved All rights and performance and security contract addendum for blah blah blah blah blah it's too much uh first published to the world in February 4th 2012 so give me a second to keep reading this uh demand that you immediately cease and assist the operation and delivery of the out regia speaker request but okay well wait a second wait a second so I think her speaker her riter says among the other things you
are bring the fried chicken right all right cool but it's at 24 and we've got half on ice and half here all right cool uh we don't have 24 anymore so all right okay we've desisted enjoy we are desisted we have desisted great have a great talk thank you so much y'all are awesome that's so funny all right well another example of you know sometimes decisions that you make can have real impacts on on how things work out um but yeah that that's hilarious I can't I can't believe that they actually obliged and I'm really looking forward to the fried chicken that they say is there and I very much doubt it is all right uh so getting
getting back the overall learning opportunities um like I said you know when we when we drop down the wrong part when we drop down wrong orientation Etc um we've had to depopulate too many parts to count and a lot of times you know that that's going to come in handy and I'll talk some more about the specific use cases where it does um but you know the ability to do some of that fine you know micro soldering like I I talk about later um is crucial right when you're talking about attacking a microcontroller that's got you know I think everyone's talked about you know depping capacitors or something like that for power analysis if not I'll talk
about it here um but that's a that's a huge part of it in addition to you know dead bugging Parts Etc um the other thing that we've done uh a tremendous amount of times um that I I think does come in handy is just cutting traces so you know if anyone's not familiar you know when you lay out a circuit board you'll have you know various layers and those layers will contain copper traces from one to another it's effectively just like wiring on a breadboard um you know a lot of times when you do run into those issues with like schematic wiring um we will have to get under a microscope cut a trace and we'll have to
sometimes even solder from that Trace which is what you can see in that picture on the left solder from one Trace to another um this is crucial and you you'll see in a second why um having the flexibility to do something like that on a board that may not give you the ability to to connect so um just quickly talking through some of the things that we have done um you know I'll just go through the uh the bullets here and I'll talk about some of these pictures but you know sometimes signals are inaccessible right a big part of Hardware hacking is you know leveraging signals uh an analyzing it decoding it um even you know manipulating it so you
know when when those signals are not accessible via some of header that you can connect to or some other means a lot of times you have you'll have to hack your way into that signal right whether that's scraping a trace and soldering onto it um depopulating a component or you know any any of those things are kind of in play for you know hacking those those PCB inner uh bus communication type signals um PCB man- INE middle it's the exact same thing to man- INE middle on a PCB you have to uh go through tremendous lengths a lot of the time um dead bugging if anyone's not familiar when you're taking a part or taking AP part off of a board and then
soldering directly to that part um so a lot of times you'll have like a like this at Mega part will have like a ball grid array under the bottom and it's very fine uh in between and then it takes a ton of very meticulous and prec precise soldering to actually attach to those signals and then exfiltrate the data from the board modify it redeploy it you know the world is your oyster from there um another thing that I I would love to talk about and uh you know one of the things I can nerd out for for a while on is side Channel analysis I kind of alluded to it um you know one of
the biggest things for like power analysis for example uh voltage rails if you're familiar will often have a ton of capacitors capacitors store electricity right so to effectively measure things like power or inject a fault um you'll often have to modify the board first before doing those types of attacks um and it just makes it you know a little more responsive to what you're doing um so depopulating components is crucial if you're even going to start doing side Channel analysis and oh just talking about the pictures so the top left is just an example of a BGA component you can see kind of the traces moving out to those vas those holes are called Vias um
that's a perfect example of if there's a signal that's coming from there that I care about it may not be exposed on the board if it's not like a a qfp which is a specific type of package um um so I might have to actually you know scrape that Trace down and solder to that trace or drop a probe or solder 30 gauge wire or something like that directly to that Trace so you know the all those steps that we learned for fixing the things can also be applied to this is is the point and then in the middle it might be hard to see especially for those in the back um we've had to do this several
times where you know you think that you cut the right trace and then you realize you did not cut the right trace and everything ceases to work um and then you have to literally repair a trace so you you know you cut up some 30 gauge wire you drop it back down and you solder it back together that's crucial too in Hardware hacking right you hack the wrong thing and you you know Hardware denial of service is not really super exciting um so you have to you fix it a lot of the time and then the the picture on the right I actually did this at one point this was for a uh uh clock
glitching I believe or false injection of some kind um but that's actually a surface mount resistor that someone had the this this was not my picture someone had the ability to pull that up and then place it on one side and then that opens up the ability to connect to both sides of that resistor which is really you know it's it's more difficult than it might seem all right so some other examples that we learned I mention that me and the buddy that I referenced um started in college which is where we made a lot of these mistakes early on um really a lot of that stemmed from no money no Mo Problems um you know we were broke
college kids and and we were trying to be electrical engineers which turns out is fairly expensive um so a lot of what we had to do there was you know try to be resourceful so some of these boards like the board on the left um we thought we were really smart and sourced a ton of components um from Mouser and then you know we were like you know the these capacitors are orders of magnitude cheaper than these I'm going to use these and then realized that they were 0 402s and we couldn't pay for picking place and we had to solder them all by hand so that's just to give you an example 0402 is the part in the middle
and that's a match stick so you know we had to go and solder all of those by hand whereas we could use something like you know the 0805 or even bigger um and do that exact same thing we were dumb and young and thought we were just saving a couple cents um and it turned out to cost a ton of time but once again the spirit of the talk is that you know that that gave us the ability to learn and you know we've soldered as low as o20 ons with the naked I and I think my partners actually soldered the 015 under a microscope before so you know I'll get back to you know what what
the point of this whole talk is but you know this example of just being you know constrained with money and trying to figure things out I mean I think that's that's part of the spirit as well of a hardware hacker because that's often your goal as well um all right so another thing that we learned is you know we we often had hostile conditions that that we started working in um just quickly talking about these pictures um we like I said we worked at residential HVAC we did it was called system extreme Environmental Testing um so this was a chamber where we put all the the outdoor units um it would run and Cake it with snow feet of snow and
it would be freezing temperatures and for some reason we would go and have to debug something that would break in there and we would literally have to debug in these conditions a lot of the time um and this is where you know we'll talk about learning um you know we we got the opportunity to learn with some complex tools that I'll talk about in a second but these were the Hostile conditions uh I I mentioned here that you know when you're when you're freezing and your hands are shivering and you are are having to do all of these things intricate at an intricate level uh with Hardware um it really uh lends itself to expedient root cause
analysis um because you want to get out of there as quickly as possible um so just to give you that that's me and SE that's one of my business partners up on like a big lift it also does the opposite where it was like 120° I think when he was doing this picture and then the this thing is called seat this chamber um so someone thought it would be funny to put a seat in seat um so someone took someone's office chair and threw it in the the snow room all right so let's talk about complex tools so the tools that we were using in those conditions um you know it it gave us the opportunity to go and
debug um with these complex things right so like a logic analyzer as you see on the left um really used for decoding signals I think is probably the most important tool to a hardware hacker um you know the you've got a big range you can buy cheaper logic analyzers and get by the the big thing is when you want to get into you know higher sample rate that you'll need or if you want multiple channels or if you want the ability to do analog signals relatively well um you can get up into the ,000 range with like the Sala logic 16 another thing that we used consistently was a portable oscilloscope so that's actually the
third picture um this is just you know looking at all kinds of signals and waveforms uh throughout you know we not only were we working on programmatic uh boards and looking at things like I squ C and spy and art things along those lines we were also looking at you know how discrete circuitry operated so you know we were doing doing electrical engineering things um up on a a forklift in the snow um with the the portable oscilloscopes and the last thing Universal programmers that's that that Seager jlink there um this is crucial is crucial for reading uh firmware from a Target or writing firmware you can attach you can debug on target you can
do a ton of things with that so those are some of the tools that we would use just to debug in in our uh conditions and you know a lot of times we would find ourselves um you know hooking up a Salia hooking up an oscope wondering you know what kind of ridiculous ambient conditions um are causing this problem and then you know I can't tell you how many times we realize that we plugged in the JTAG header upside down um so this is one of those examples where you know a stupid problem being you know you have something unplugged you've got something plugged in upside down you've got you know XYZ something um but you you throw
all these tools at it and you know really gave us the opportunity to learn how to run these tools especially in these hostile conditions all right next next lesson that we learned um one-way ticket to dependency hell I think everyone in this room has probably seen something like this at some point and hopefully it's not too triggering for anybody um but yeah the whether it's writing embedded uh C code or if it's writing you know some you know python requirements or whatever it is um will often run into dependency hell just as general software developers of some kind um so you know that that's the same for hardware for firmware um you know what the the point
here is that you know dealing with dependency hell and not shunning it away is actually something that's incredibly beneficial as well given that you know it teaches you more about your specific problem teaches you more about you know in in my case how the compilers actually work um the way that I'm you know linking libraries and you know understanding at a fundamental level how the the code I'm creating gets translated to you know machine code in in my case um and that to me that's crucial to a hardware hacker as well because you're you're starting from that point you know if I go pull firmware off of a Target you're starting from bik code right so the ability to get back to
something reasonable um you know if if you have an understanding of the not just the dependencies but the intricacies of the the code that's being deployed um it's crucial in in moving forward there all right bonus slide uh I did look up the statute of limitations and they have expired in Texas where I'm from um so I can tell you that a lot of times we had the opportunity to learn all about physical pin testing um slash b& um mostly due to our general forgetfulness um if you can see that picture that's actually the picture of the uh the facility that we used to work at there's a little cage on top that you might not be able to see that cage was
added because of my my partner that I keep mentioning um we used to just forget and we would have to break into the building full disclosure um so we would have to uh jump over the fence we'd have to tailgate we'd have to you know we would clone each other's badges back before it was cool um and then other physical weaknesses like you know we never really picked a lock there we'd probably get in trouble for that um but you know the air can attack is what we use consistently you know massive air through the uh motion sensor break into the room which is you know these are all the things that we did allegedly um and
you know it's funny now looking at all those opportunities and we we had no idea and now you know if you look at the list of things we legally get paid to do now it's the exact same list so you know that that's another Spirit of you know the the point that I want to convey today is that you know all of these things that are exciting and fun and you know we thought we were just kind of taking for granted as part of our job well not I guess breaking entering wasn't part of the job but the things we were taking for granted could also be applied to you know a different career entirely all right so now I get to nerd
out a little bit more um hopefully you know all the things that we talked about kind of come to a head when you go deep into the world of Hardware hacking and like I said the the point here is I'm not trying to go too deep in Hardware hacking I'm happy to do that with anybody at any point but you know you can see some of the same tools um just with these pictures that we're seeing here so I I'll briefly talk through some of these and and figure out you know uh or just kind of allude to uh what they are how how we can use electro engineering to to get to this point so
fault injection I guess show of hands is anyone familiar with like fault injection like voltage injection type stuff all right sweet well I'm definitely going to like go into a lot of detail then um so if you imagine like you know an embedded part is going to read memory at some point in time especially when it boots so you know a lot of these microcontrollers will have a voltage rail that voltage rail is powering the core right in order for it to read memory properly um it it has to have like a a good solid connection it's got to be fully powered when you get it into a state where it's not operating entirely well like sort of kind of a
brown out State um you cause it to do faulty memory operations so something that happens consistently um and across the board is that if you can change that voltage to the core at a specific time um when you're targeting specific pieces of the of either the bootloader or the way that the firmware operates you can actually cause an invalid read and sometimes what happens especially with bootloaders like St micro if you're familiar with that um they will read that and then say okay well that's not the thing that I thought it was I'm going to revert to this boot sequence right so just give you an example the the default is like level one right
where it's kind of a little bit of permission um kind of not if someone enables level three you can do this type of attack and go back to level one and then you open up a ton of different things that you can do to that Target just by doing some basic electrical engineering where you're using a multiplexer to literally switch rails pretty quickly right so I think you know fault injection or or any side Channel analysis is the the perfect harmony of electrical engineering and hacking
yep yeah sure yeah so I mean the in general Hardware hacking like the you know I think the point of Hardware hacking a lot of folks will say you know you have physical access who cares at this point and I think that's just it's such a misunderstanding of the capabilities of hardware and how you can secure Hardware so you know what what we're after to get to your question is firmware we're after sensitive information we're after um you know intellectual property all those things that we can take that firmware and reverse engineer it and conduct a broader attack at you know the infrastructure that's supporting the hardware device or we can you know exploit that IP uh gray Market attacks
are a huge thing with Hardware um and even just you know manipulate something if it's you know a critical infrastructure component right you can manipulate it uh and cause it to to perform errand operations whether it's you know flashing a a device like a End of Line run tester or you know some some sort of safety operation you can if you can compromise that you know it's a major problem right um so one other thing I'll talk about power analysis too because I think it's just it's awesome um so everyone's familiar with AES right AES encryption Okay cool so AES I think the for AES 128 if you try to brute force it takes like a billion billion
years or something like that I'm sure someone will correct me afterwards um with with correlated power analysis and certain conditions certain encryption modes you can get that to as low as 5 minutes right because what you can do is you can look at the the power analysis and you can look at the all the permutations of an AES key and there's a with with some boards there's a a property where you can correlate those two data sets and you can infer specifics about the the operation the cryptography of the system just by the power analysis right so you know to to me as an electrical engineer reading power and you know writing some code to
process the data that's trivial right um and then if you couple that with the the cryptography piece of it you know we're talking about Engineers can can break into things that they should not within with relative ease um and this is you know I I don't want to minimize like this these are microcontrollers that are everywhere STM microcontrollers nxp you know you name it they they're in everything and with the emergence of iot um you know it's going to be more and more of a problem and vendors are just now starting to fix it much less you know your your run-of-the-mill manufacturer that's trying to get to Market as quickly as possible me it's cheap keep as possible
so you know these attacks are not just in Academia these attacks I think are going to be more and more pertinent as as we progress um and as as these things become you know easier to use which I I could talk about more at a later time as well all right so what now um this my my colleague made me put this quote in there if Hardware hacking is cool then consider me Miles Davis I stand by that I I I don't know who Miles Davis is but Hardware hacking is cool um all right so the the topics and you know takeaways that I want to give here um the growth mindset and then celebrating small wins
you know a lot of times we would go and we we'd have to break into our our employer use all their awesome gear and fix all the boards that we broke at school um we didn't that seemed daunting at the time seemed like we were screwing up it seemed like um you know we didn't know anything about electrical engineering but you know the the things that we were learning I can see now were completely invaluable um we learned the the resilience of you know having a broken board and fixing it we learned all of the specifics of how um how to do that and you know all all the skills that we can now apply to our careers um
and you know the I think key to that is is that growth mindset and making sure that you keep track of you know even when it's terrible and you feel like you're not learning anything you're banging your head against the wall it's it's always progress right so making sure that you know everyone uh as much as I can everyone has that same mindset moving forward that you know it's for a greater purpose at some point um the other thing is that you know I went I think five years as a dou e uh not realizing the world of embedded Hardware hacking and you know I tell people now that if anyone would have told me what
you could do as an electrical engineer in the world of cyber security and Hardware hacking I would have never done anything else uh and I really mean that I think the just the the side Channel stuff alone is is so awesome and fascinating and you know there's a lot of really smart people that know how to do it but it's a relatively untapped space I would say so you know taking it and making it more approachable to you know good people hopefully that are making the world more secure um I I I would say why why do anything else uh yeah yeah um and then the last thing I think this is clear don't be
afraid to fail right failures where we learn uh all of our mistakes I hopefully I was a little bit uh vulnerable with all the dumb mistakes that I've made over the past 10 years or so um and you know realize now what what that led to now you know I'm uh I get to Hardware hack all the time and uh you know without without all the failures without all the challenges that that I face in my career that wouldn't have been possible so that's that's the biggest takeaway there and I think that's it so any any questions from anybody all [Applause] right in what uh circumstance you did the physical pen testing if you are into
AR King physical pen like breaking into a building yeah yeah I the the best example um we had a a building in Downtown LA and uh you know we we just had to go and we impersonated you wore wore the the nice three-piece suit and uh you know walked into the building and realized that they did have like security guards and got scared walked out came back around um and then we just literally tailgated and uh tailgated went up to the floor I was sitting next to the CFO um and just plugged into their Network and sat there for about 45 minutes until we got bored and then just left I mean it was it's one of those
things that especially me as a I don't know if youall know I'm from the south from Texas um just general talking to people is pretty easy um and people are too trusting I would say so I had a pleasant conversation with a lady that uh almost gave me just the Wi-Fi password just by me asking so you know that's kind of what you're up against a lot of the time is that you know I I'd rather me do it than somebody else but you know if someone goes to that those means of you know just getting access to your network it's it's pretty easy a lot of the time without all the crazy you know lockpicking air can type
stuff any other questions Y what do you think the risk is of people not understanding their Hardware I mean they've got great software they got great programs but they don't understand what's actually sitting on the board of their backbone or switches routers and stuff for actual physical components yeah great question so I I don't think people understand the downstream impact of how the hardware can interact with the broader ecosystem so the best example that I have with that is not like you know networking gear it kind of makes sense you understand that it's kind of the backbone of of a uh an Enterprise but I had one where you know something innocuous where it was a uh a water
filtration system and that water filtration system had API keys that were stored on Target and I pulled it out of like non-volatile memory on target through a uard interface and then I took those keys and then I attacked the API and then I attacked the API API had indirect object referencing and then I went from a physical device that probably would get thrown away to I take down your entire API for all of your customers right so understanding Downstream effects and you know how easy it is to actually reverse engineer something intelligible about a system I think is the biggest thing that people just miss because they they see it like I said they see it as you know you have
Hardware attack you could just pull the power plug denial of service game over that's not the case there's a lot more that that could be done the question
ohsh I mean we we've seen art like uart is universal asynchronous receiving and transmitting just like a a Serial interface but we've seen root on uart on Advanced devices that have no business exposing root level access to someone with just general physical access so that's probably the most egregious I mean we've seen everything from you know unsigned firmware that we could just you know I've been able to text the device um and change the upgrade server with an SMS message and then host a malicious uh firmware file and bypass their signature verification um to you know overwrite that firmware on a embedded device and you know lock it out do some crazy stuff with the io you know egre there's a lot
of egregious that that goes on out there so I wanted to actually kind of like reinforce the the the answer that you gave to uh this gentleman's question so I I too am an electronics engineer who somehow managed to get himself a role as an embedded penetration tester and yeah this is like this is they're paying me to hack what yep this is so cool so anyway yeah and a lot of threat matrices having physical access to something is usually considered like pretty low like if you you go ahead and calculate your risk scores they'll come out low but this is the caveat it's just like you said it's like so what'll happen is is a real world real world
attackers if they want to like attack your infrastructure whatever at scale what they're going to do is try to get a hold of one of your devices yep that's could be just buying it off eBay and they will attempt the reverse engine it to find things like what he just talked about like hardcoded API keys in the firmware in clear so if you didn't lock J tag then that becomes trivial to dump the firmware if you did lock J tag then you got to that's when you got to start jumping into things like fault injection and all that to get it out but once you do and you get it then it allow and you
or you find that okay there's hardcoded credentials in there and they're the same for every device like they're not unique per device and then yep pretty much like what I what he just described then it's like okay then things get really fun
yep yeah uh so for those of us who may have come in a little bit late would you mind sharing or going back to must have tools or capabilities for people trying to get in yeah sure so I think I I mentioned these because I think these are probably should be top of your list logic analyzer hands down is the best it gives you the ability to decode signals right whether that's in bus communication um or you know if it's something that's like an external signal sometimes that's helpful too if you don't you don't want to buy like a custom uh transceiver for everything like rs232 45 whatever um so a good logic analyzer I think is the best um
Universal programmer is probably second on the list where you know you imagine the same same sort of deal with the ability to just be dynamic and you know deal with multiple targets a universal programmer inherently does that where you know thousands and thousands of uh microcontrollers like like this gentleman mentioned with JTAG if if that interface is enabled you know you can use JTAG you can wire up to it or you know a lot of folks will just leave a handy dandy header for you connect to that dump the firmware and then you know you got a ton of stuff you can do from there uh last thing I'll say probably uh a universal well I say Universal
programmer that's more of a universal debugger but a universal programmer um to the extent that you can dump like external flash is another big thing where you know if you depop something off of a board you throw it down on something else you know a lot of times that's where you'll find like the API Keys hardcoded credentials all that kind of stuff and you know those Universal programmers are the same where you know different form factor you can uh exfiltrate information from you know memory or whatever it is so I'd say those those big three are probably the most I'll give you a bigger list if you want to stop by later hey for someone no sorry next okay um for
someone wanting to dive deeper into this uh what do you recommend for for example I I went to a coding boot camp there's a lot of like online resources for learning on your own how to code without going through an e program yeah what do you do on the hardware side of things like I wouldn't know what I'm looking at if I picked up a a logic analyzer or an oscilloscope today how would I learn that on my own yeah that's a great question because I I don't think I think it's inherently difficult to learn were hacking because you know you it's very easy if you if you touch the wrong thing and you solder to the wrong spot like
that factory insult smoke is real I've seen it a lot of times right so um it is more difficult I think that we're seeing more and more uh you know challenges like hack the box or try hack me are trying to do some more Hardware based things and there are kits that you can buy online um I actually if you see me after this I I can point you to some stuff I actually made a uh an open source uh you know learning environment where you learn some of these things the the barrier to entry though is always the hardware right so getting your hands on something that's cheap enough to understand the the premise and then you
know you can work your way up you know nice tools whenever you you know you earn your earn your keep and you understand like what's going on um but I would say you know there are some great books out there no starch press released um iot practical iot hacking I think and then the hardware hackers handbook those are two great books that i' I'd mention too yeah back hello um I believe you mentioned SMS earlier were you referring to a mobile device or just like an eded device with LTE or something yeah so a lot of these devices will be cellular for a number of reasons right um cellular is probably you know when something's not close to uh an access
point or you know you don't have long-term connection cellular is often used in a variety of Industries so you know the the risk to Cellular is is crucial right and I think that there are a lot of issues with um you know it's less so on the the cellular piece there are there are things that you can look at I think they're a little more complicated and then sometimes when you're testing cellular um you can get into trouble and like have black vans roll up to your house if you broadcast the wrong signals and like interrupt uh emergency uh response so be careful doing that um but I think one of the biggest things with SMS is just general
you know improper parsing of of data and trusting data that you shouldn't and that's that's the issue that I ran into it's like I use SMS as a vehicle to bypassed bad signature verification of firmware in addition to you know improper access control of you know elevated function like firmware updates so SMS is a vehicle you can attack that just like anything else but yeah it's it's super prevalent with Hardware
devices so this actually was a really good talk and presentation there's a um a field of systems engineering called anti-tamper yeah yeah and if anyone here hasn't heard of it I highly suggest you actually start researching it because there are jobs that not only that pay you to break into systems but pay you to develop counter measures to exactly what you're talking about here and a lot of that field of study came out of the fields of reli ability so to answer your question sir for C's components like we're talking about here you can get all the specs and schematics online and that's the first place that our adversaries in certain countries go to First yeah that's a great point and the
best way the reason why they are surpassing us is because we let it happen it's because everything is out there yep and that's how you all can obtain this stuff it's all out there it's all free you can get the spec sheets the data sheets you can pull it all and that's where you learn if you learn how it works and you can learn how to dismantle it excellent talk thank you yeah thanks yeah fcci .io if you're familiar like you know anything that has an FCC ID on it go look it up a lot of times you'll go you'll be able to dump the entire schematic the the testing from FCC it's just it's a function of of
exactly what he said you know a lot of this is in the open domain some vendors are better than others but you know just go Google this stuff don't read like an a thousand page data sheet figure out how to read read data sheets first um but yeah I I can agree more and then anti-tampering I know we're we're almost up on time um just to give you an example so I mentioned correlated power analysis is what I was talking about um some of these chips will do some crazy stuff like injecting random noise and current consumption which breaks that correlation so you know I I can't remember who asked but in in the spirit
of where we're moving as an industry we're seeing more and more of components like that that strengthen the hardware at a fundamental level that we need to see more and more the problem is you know it's bomb cost it's complexity it's Dev time you know that's why we're not seeing it as much as we should all right thank you I think your Fried Chicken getting cold thank appreciate it [Applause] [Music] a [Music] [Applause] [Music] [Music]
[Music] [Music] I'm just trying to I do I'm just trying to give you [Music] something I'm just trying to give you something okay I I'm just trying to give you something he [Music]
w a
[Music]
[Music] [Music]
[Music]
[Music]
[Music] [Applause]
[Music]
[Music] [Music] a [Applause]
[Music] n [Music] the
[Music] oh [Music]
[Music] [Music]
[Music]
[Music]
[Music] w
[Music]
[Music] [Music] [Music] [Applause] [Music]
[Music]
[Music]
oh [Music]
the [Music]
[Music] [Applause] [Music] hey hey hey he hey heyy [Music] [Applause] [Music]
oop just like a good time all right good afternoon everyone welcome to bsid Las Vegas um this is the Raiders the Lost artifacts uh given by youron I think I got that right um thank you for coming out we'd like to thank our sponsors uh Prisma cloud and vanta as well as Adobe drop zone and others it's with their support along with our sponsors donors and volunteers that make this possible um please keep your phone silence this is being recorded and streamed on YouTube uh if you have questions come around with the microphone please speak into the microphone so people on YouTube can hear you and with that let's hand it
off thank you so thank you all for coming to this talk I'm very excited to be here before we begin I would like to thank our sponsors um just kidding basically I managed to hack into this organizations and some of them did offer us generous back bounties and let let's see how we did it oh
sorry it's on not yet not yet cool so this is the organization I managed to hack it's more impressive right now I guess uh so let's see how I did it so first of all my name is y vital I'm a security researcher at paloalto networks I have about 20 years of experience in the cyber security space started off as a developer then moved to be a security researcher where have done vulnerability research mobile security back bounties and many other cool stuff in the past few years focus on cicd Security First at cider the creator of the OAS top 10 cicd security uh which was acquired by Alto networks have been working there ever since uh last year was the year exactly
one year ago I gave a talk called actions have consequences The Overlook security risk and thir party gab actions and one of the concept I raised over there is the permissive nature of the giab actions Pipeline and how very few organization do manage their pipeline permissions and today we are going to see the consequences of not managing your pipeline permissions so the agenda for today is a new novel attack path I found against public repositories on gab basically allow to run my my malicious code into your devices even the ones in this very room we are going to see some demos and of course we end our talk with mitigations and takeaways so GitHub action in a nutshell
is a a widely used cicd platform by GitHub basically code execution as a service enable you to build test and deployer code and because it's doing a lot of integration with your cloud provider or slack or J it needs secrets so a lot of Secrets is going on over there and gab allows you to upload artifacts as part of your pipeline now these artifacts are not packages they're not like the formal binaries they're mainly designed to share data between jobs or for developers to being up able to purist data in in order to debug the pipeline afterwards and stuff like that now this example here is um an artifact from Firebase uh from by Google and uh if the
if the repository is public then the artifact is public and well as well and you can download it so I had this hunch of uh doing Secret scanning on these artifacts because they're being compiled at a very sensitive environment uh and I haven't heard any chatter about it anyone that does that so I decided to take the most popular starled repositories on gam and try to scan them for secrets and it worked and I even have a name for it I called it artifact which is artifact packed with secets I found various secrets for various Cloud providers like class FM digital ocean COV GitHub and I did found some personal access token which uh the users used to
manage their repository push code ET Etc H but that is not the focus I did found additional tokens these are belongs to giab GitHub token and actions Rome token and this token used by the pipeline it's not like the uh previous token we saw it's explicitly used by the pipeline to interact with the repository and we will cover them in a minute so the way things work in theab action every time uh work for starts an Emeral short live token called giab token created and this is how the pipeline can interact with the the repository clone the code push code and do stuff like that now the permissions of the GitHub token is basically up to you you can
configure it inside your yaml pipeline like this and if you don't the default configuration from the repository will kick in as for the actions run time to it's a different type of token you can use it to access the cash system to upload a new artifact or get new crash entry and stuff like that now because secet scanning involves with a lot of false positives I wanted to be sure these tokens are the real deal they are legit so I decided to dig in and try to find out how they ended up in the artifacts from the begin with so I quick quickly find out the the immensely popular xent checkout I think it's the number one action on
GitHub it used to store the the credentials persist the credentials the gab token inside the local git folder uh by default this is the B 64 representation of the giab token and on its own it's not a problem but combined with the fact that a lot of users simply check out their code and upload the entire directory including the hidden git folder that's a problem because your token is leaked sorry moving on another problematic pattern I have noticed is that user that use the super popular the super lint Which is popular secret um sorry code lintel which is supported by many languages it used to print the entire environment variables to logs and when it comes to context of cicd where SEC
secrets are being loaded as environment variables probably not the best idea and these log were uploaded as artifacts as well and uh I reported this Behavior to the maintainers of the super lter and this Behavior was fixed so now it's the favorite part of my work hacking I got a lot of tokens different kind of tokens let's try to use them but quickly I found out there there is a problem because during the time of my research artifacts were only available after the workflow has ended which means the get up token already expired so by the time I try to use it and push code I always got the same error over and over again
401 uh because again the Tok is invalidated and there's no way I could wi win this race condition it was rigged basically so so from the majority of tokens I got I lost I lost the majority of tokens from the tokens I got I still got the actions runtime token and moving back this is the the decoded part of the jwd token I have noticed the expiration time of this token was 6 hours which is plenty of time to commence an attack so this is what I did a deep dive into the upload artifact action I wrote code that wraps the logic of the upload artifact I traced a victim's workflow I remember have six hours to do
so and swapped artifact with a malicious one and let's see a cool demo of doing exactly that
so on the right you can see I'm tracking the uh the workflow I've downloaded an artifact this is by the way a real Attack this is a project called schimoler a real project on GitHub with many stars and I've noticed the action random token was leaked over there inside the
artifact so I wrote a nice PC that downloads the the artifact extract a token and use it and I have swapped the artifact you can see remember I don't have any permissions to this
repository and just like that I replace replace the artifact with a malicious one
now just a second Now what's the meaning of this attack basically I could achieve an RC in this way remote code execution because if you remember I told you that artifact is used to share data between jobs so a job that tries to download this artifact which can be binary of course and execute it will get compromised the same way developer trying to uh consume the artifact and uh trying to see what's going on in the in their pipeline will get hacked as well now this T was pretty happy uh from achieving this rce I managed to use the the actions runtime token but then something truly H magical happened um as I was going through the
GitHub change log this is something I do very often I have noticed they have done a complete overhaul to the artifacts V4 and now the artifact is available as soon as it is uploaded by the job and I didn't have to uh basically wait for the workflow to end that means I have an opening for race condition right so I need to do the following to trace uh work for triggering identify the exact moment the artifact is available download the artifact extract token and create a branch now why create a branch because I wanted to prove I have right permissions and create branch and obviously I didn't want to push malicious code and create Branch does
require right permissions so this is exactly what I needed to do and I needed to do all of the above under 2 and 1/2 seconds you see the little animation that's exactly two and a half seconds um because that's the time between the artifact is available and then the job dies which means the token is invalidated but sadly I was no Neil to win any race condition I simply wasn't fast enough because the downloading the artif took a lot of time but then I had this really cool idea of why not using gab actions as an attack infrastructure because I was able to be much closer to the Target it can be triggered remotely which is something I
needed and it offers much lower latency and faster downloads so I did exactly that and wrote an offensive G workflow I present the repo Reaper and basically all I needed to do is to point this guy towards the potential repo and wait for results and this is the attack flow I needed to monitor pipeline 3 so Roto software that tracks when the pipeline is triggered pipeline can be triggered by a a contributor contribute code or U or a nightly build or something like that the pipeline uploads the artifact I need to download the artifact use the gab token in my case create a branch and six by the time the uh job ended and the token
invalided I don't care because I managed to use it and as a bonus because this attack was fully automated I sent a telegram message to myself so I know something has happened and it worked soon enough I got a lot of these messages meaning basically that the repo reer has managed to compromise the repo this is my favorite one by the way with me doing the dishes and this is the first race I won against uh project CLA by Red Hat I've managed to create a branch called Impala it's a very popular container scanning it has 10K stars on the GitHub just as easily I could have pushed malicious code and let's see a cool demo of the
real attack against clear so on the bottom screen I have the software that tracks the pipeline triggering in this case it was a nightly CI on the left side we can see project clear I did wake up at 4:00 a.m. to record this I do appreciate the effort and on the right side we can see the repo repair workflow waiting for the artifact to be available
in few seconds okay so the reer download the artifact created a branch use the token and you can see by now the job has ended that's mean the token is invalidated but I managed to use it nonetheless and let's see the branch we managed to create
cool so we see the new Branch Impala which of course wasn't part of the
repository so that was cool but I wanted to win more and more races and sometimes I lost depending on the how big was the artifact so I did some optimization some tweaks like extracting only the files I needed because artifact were compressed and sent a lot of requests per seconds um to identify exactly when the artifact is available it was crucial and I did some communication tweaks like disabling certificate verification and remove unnecessary headers and it got me to win this race against Ubuntu uh it's a component that ships out with every Ubuntu called ADC it's used to integrate with active directory and I've managed to create a new version of it called drun duck trust
me you don't want to download this version and in the aftermath of this attack I managed to basically avert a massive supply chain attack I managed to compromise Firebase by Google which has 1.6 million projects references on GitHub um as you can imagine if I take a look at your phone I probably see some apps written with the help of Firebase we already discussed about Ubuntu this is a major um distribution of Linux project lir Cyclone DX is a f famous famous uh scanner by OAS respectable security vendor I managed to compromise several repositories by Microsoft used internally by the their developers and AWS open search with a popular service by AWS now to the to the good news all you have
to do to mitigate this attack is basically change one line of code instead of using the original upload artifact you can use a tool I wrote a get action basically it's called upload secure artifact and what I did basically is integrate a layer of secret scanning inside the uh upload process of the artifact so if it found any sequence s is being uploaded it will block the upload and fail the pipeline you can try to use it it's fre it's free this is how it looks like found GitHub token fails the pipeline and some takeaways before we end this talk I can't stress enough how important it is to reduce your workflow permissions it can help you mitigate
against this attack and many others like command injection PPE whatever so do it uh integrate artifact scanning as part of cicd uh you can basically use my solution or any other there are plenty good open source out there and uh as as security Defenders we have to see the big pictures from code to cloud in this case because often the vulnerabilities will reside in benign features like like this one thank you very much if you have any questions
any questions yeah respon re or redesigned yes so gab basically said it's up to the users not to upload Secrets within their artifacts I think this response is somewhat limited because they can offer secret scanning as they do in other components of their ecosystem but uh they don't they also can uh change the behavior to action checkout not to prist credentials by default which is something that is lesser known by the users I wasn't aware until I started this research myself um the actions runtime token variable how did you find it um so going through uh logs I notice the environment variables and it is less so known by the way it's not uh formally documented this actions
runtime token but once I've noticed it I start to uh dig around and see how it is being used I I reversed a little bit the upload action the original action and I noticed what is the usage of
it thank you very much thank you [Applause] [Music] he
[Music]
[Music]
[Music] track [Music] hey hey hey hey [Applause] [Music] hey hey hey hey hey [Applause] [Music] he [Music]
[Music]
[Music]
[Music] [Applause] [Music]
[Music] [Applause] [Music]
[Music] [Music]
[Music]
[Music] he [Music] [Applause] [Music]
[Music]
[Music]
oh
[Music] h
oh [Music]
[Music] w [Music] [Applause] [Music] [Applause] [Music] [Applause] [Music] I'm just tring to [Music] give I'm just trying to give [Music] something I'm just try to something I do I'm just tring to give you something [Music] right [Music] [Applause] [Music] [Music]
[Music] [Music] I'm just trying toing I do I'm just trying to give you [Music] something I'm just trying to give [Music] something I'm just trying to give you something [Music] w
[Music]
[Music]
[Music] [Music]
sure yeah at the door little trap music doesn't you know add to the presentation all right good afternoon welcome to bsid Las Vegas Common Ground uh this is talk is titled uh free your mind battling your biases a few announcements before we begin we'd like to thank our sponsors especially Diamond sponsors prism cloud and vanta and our gold sponsors Adobe project circuit breaker it's their support along with our sponsors donors and volunteers that make this event possible these talks are being streamed and as a courtesy to our speakers and audience do you make sure your cell phones are silent just common courtesy uh if you have questions I will walk around with a mic and give it to you uh
please just speak in to the mic so the people on YouTube can hear you and with that here we go all right uh is this appropriate distance everybody can hear me fine perfect uh yeah so this is free your mind battling our biases uh I'm Dade I'm a staff security engineer I don't know if my slides moved so just somebody thumbs up perfect all right uh I'm a staff security enger for a fintex startup uh independent security consultant on the side I have a background in red team work at companies like Oracle and Intel which will be a little bit relevant later on uh and if you're interested in more about me you can find basically all my Links at ZX
da. de uh quick disclaimer uh this is not a technical talk if you're here for the latest buffer overflow in a poorly written C program this is not the place for you uh this is a talk about biases the unexpected benefits of being a beginner and changing the way we interact with our colleagues and peers and now I have to actually look at my slides cool now we are where we're supposed to be uh before we get started I want to take a brief informal survey I'm going to ask a couple questions and I just want you to raise your hand if the question applies to you uh have you recently done something that you later
thought was dumb have you recently refrained from asking a question because you were afraid people would think you were dumb have you recently been annoyed when someone asked a question that you thought that they should know the answer to and have you recently refrained from sharing a piece of information because you assumed that everyone already knew it uh it looks like we're all somewhat aware of our own biases at this point so that's great uh I I tried to pick some that were more likely to be common in more experienced people as well as people uh like who are more beginners are going to feel things so uh I'd like to start off by talk to the beginners in
the room uh the people who are new to the industry or even just new to their current job we're going to have discuss some common feelings and how they can be associated with various cognitive biases and then we'll chat with the more experienced folks and we'll end with a technique that I think can help us all battle our biases more easily uh we're only going to take a surface look at most of these because I had the audacity to submit this as a 20-minute talk before I started writing it uh hopefully it's enough that if you're interested you can go look them up yourselves uh I personally found a lot of value in understanding that several things that I
felt and things that I believed were common enough that they uh not only had names but they also had Wikipedia pages so uh I wanted to start with this phrase always be the dumbest person in the room uh I got this advice a lot when I was younger and I think a lot of business Guru type people will still give this advice today uh an alternative to this might be if you're the smartest person in the room you're in the wrong room the idea here is that we surround ourselves with people who are better than us and we will get better by proximity uh loosely uh I speak from experience when I say that this works really well if you want
to rapidly level up your own abilities but it's also really exhausting because you're going to constantly feel like you're behind all your peers uh it can work well though and we're going to explore a few unfortunate side effects of holding this belief about ourselves uh as a beginner we're not burdened by the curse of knowledge we know what we know we probably don't know some of what we know and we definitely don't know what we don't know uh we don't have years or even Decades of historical context around any given decision or around any given problem uh we look at it with completely fresh eyes and think of solutions completely unburdened from the shackles of reality
this can be a superpower if we're in an environment that'll let it flourish uh but it can also be a source of a great sense of Shame and disappointment if we're in a toxic environment I mean if we're the dumbest people in the room that would imply that we're the least valuable person in that room and if we're the least valuable person in the room then asking a question might just be a waste of everyone's time right I mean they have years of experience surely they've already thought of whatever dumb thing that I wanted to ask if we're the dumbest person in the room then when someone else says something they must know what they're talking about even if
we don't understand it it must be right right I mean they're the authority aren't they but what if everyone feels this way uh what if every one of us feels like we're the dumbest person in the room then we're all agreeing to whatever happens to be said regardless of if it's right or not we have created a bandwagon effect that just leads to worse decision- making in the long run I think it's important to to remind yourself that you're not alone if you have a question there's a good chance that you're not the only person who has that question or maybe someone else had that question a few weeks ago and they can help answer it for you which helps
you and it helps them to reinforce what they learned if we choose to not ask that question or to not attempt that new project or not commit to a project because we think we can't do it we're engaging in a form of self-handicapping if we stick to only doing the things we know we're good at and never attempting to do something that challenges us that's self-handicapping self-handicapping can help preserve our self-esteem in the short term uh by helping us to avoid perceived failures but it can also hurt our confidence in the long term by preventing us from experiencing meaningful personal growth uh when our minds are free of assumptions about how a system works uh
how something should work we're free to be curious and experiment we're free to ask questions we're free to try new things we're free to experience growth and development but we're also free to be wrong in fact we're probably going to be wrong a lot uh but being wrong isn't something we should fear being wrong helps us change the way we perceive the world perceive the problems that we're facing and helps us to overcome those problems when I was in third grade I did a report on Thomas Edison I didn't know all the things I knew about him today uh but to a third grade nerd he did seem like a good subject for a report on
historical figure uh one quote however apocryphal and paraphrased it might be has stood with me ever since uh I've not failed 10,000 times I have not failed once I have succeeded in proving those 10,000 ways will not work this quote captures an ESS reframing of the concept of failure uh reframing of the concept of being wrong being wrong doesn't mean that we're not successful being wrong is not it's just one stop on our journey to success it's a great way to reinforce what's right once we eventually figure it out there's another Concept in Psychology called uh the shared information bias which basically suggests that a group of people will spend most of its time and energy
talking about things that everybody already knows and spend very little time on the things that only a few people might know uh this isn't has some interesting business impacts if we think about it like if you're having a meeting and you want to make sure that the right people are in the room because you want to make the right decisions you want to reach some consensus and move the business forward uh but in business we don't really get the luxury of just sitting there and discussing the merits and shortcomings of every possible solution before we move forward it's also means that sometimes we're neglecting to make the best informed decisions uh selecting the right people
for a meeting is hard and relies on my understanding of what other people know which is flawed it leaves out people who might know a great solution but weren't included in the meeting or maybe they weren't on the email thread I don't think there's a one clear solution to overcoming this tendency it's going to be a game of balance because we can't just entertain every idea that everyone has before we make a decision we can't invite everyone to every meeting we could write documents and make them more widely available but we can't ensure everyone's going to read it and in fact probably most people won't read it so is there still value in writing it if no
one ever reads it uh if crowd strike crashes a Windows server in an airport and no one's around to take a picture of it did it really happen so if you're an individual contributor like me you are probably more inclined to scoff at the idea of having to write down every proposed decision the context the consequences Etc uh because the more Nuance that we understand about a problem the more we realize that we will basically never stop writing if we try to do that if you're a project manager an executive or someone who just really loves formal process you're probably very excited by this idea though and also very annoyed about people like me who won't follow your process but we
should definitely be thinking about how to overcome or rather counteract shared information bias and if you have tools that you've used to help overcome this I'd really like to hear about them after the talk out there uh one amusing note I realized while writing this talk uh shared information bias would suggest that every person in this room already knows all the things that I'm talking about and that's why they're here because they wanted to hear more about the things that they already knew about uh so to the people who ventured outside their comfort zone to be here uh I see you and I appreciate you switching gear I want to talk to the more experienced
folks in the room uh those of us who have put 10,000 20,000 hours into our craft uh those of us who have forgotten how much we know uh until we're randomly asked one day about some obscure problem and it all comes flooding back to us as we grow in our field we become more saturated with various biases even if we think we aren't biased or that we experience bias less than our co-workers that's a bias in itself called the bias blind spot we accumulate knowledge over the years and that knowledge helps us make informed decisions about our work that accumulated knowledge is why we're so valuable but it also represents a challenge for us as
well ah or behind this is where I wanted to be nope well this is where I'm going to be in a minute anyways uh if we've been in the same environment for most of our careers whether that's the same job uh the same company or the same role within the industry we're likely to face the status quo bias or our tendency to prer things stay the same cuz that's what we know who here loves Windows Vista right that's what I know that's what we're that's what we're here for uh so ultimately uh we become burdened by the curse of knowledge contrary to the beginners who didn't have the knowledge we do and that makes it difficult for us
to see perspectives of people who haven't been popping or patching shells for as long as we have even if those perspectives might be better than ours in some regards we're probably going to have a hard time seeing it because of what our experiences have shown us we also face confirmation bias favoring the things that we're familiar with uh favoring the things that align with our pre-existing beliefs and sub subconsciously leading us away from things that challenge those beliefs I think several of these biases help steer us towards decision- making that makes it difficult for beginners to be heard they help steer us towards the same decisions we've always made they help steer us away from anything that
challenges our status as an expert on a topic but I think we have to make room for beginners we have to actively encourage their participation their confidence to ask questions their sharing of ideas their ability to approach problems in new and novel ways sometimes we have to let them fail because if we know their idea won't work and we tell them as much they might not feel comfortable sharing those ideas with us again we also have to lead by example sometimes if we know the answer to a question it can be valuable for us to ask the question anyways by actively making the decision to ask the questions that we think others might have we are
encouraging a culture where asking those questions doesn't feel so scary or overwhelming for others we're helping to make sure everyone in the room has the same information and helping make sure that others are more comfortable speaking up when they have questions or concerns our mental models of how systems work are often biased by our experiences and by the knowledge that we already have in any advanced system whether that systems compr comprised of computers of people or some combination thereof it's surprisingly easy for our mental models to quickly become inaccurate by making a conscious active effort to free ourselves of the constraints of our own mental models we can look at things in a new light and
find interesting ways to improve them we can think critically about the things that we otherwise take for granted but making this effort is difficult it requires going against every impulse our brain is telling us it requires challenging ourselves at fundamental levels but there are exercises we can engage in that help these challenges get easier and encourage us to more easily slip into this Divergent way of thinking in 2007 Sir Ken Robinson gave a TED Talk that posed the questions of whether schools killed creativity or not in that talk he brings up this idea of Divergent thinking the concept of seeing a lot of ways to interpret a question which opens up a lot of possible answers
to the question he gives one particular example that I've found myself using a lot uh over the last 10 years or so how many uses can you think of for a paperclip most people in this room might uh come up with 10 or 15 PE most people will come up with like 10 or 15 people in this room are probably a little bit better at that uh you know maybe 40 or 50 uses for a paperclip uh but people who are really good at it they might come up with 200 uses for a paper clip because they're going to challenge the the very like notion of the question uh they're going to say who said the
paperclip was a conventional paperclip what if it was 200t tall and made of rubber uh suddenly the uses for the paper clip can expand dramatically by just suspending our preconceived notions around what a paperclip is uh this is in my opinion the essence of red teaming uh I think red teaming has nothing to do with hacking computers though that's the way that our industry has hijacked the term uh the actual skill itself that makes someone uh valuable red team member is their ability to think to think divergently their ability to look at systems and problems and think what if x was an X what if it was a uh when I got interviewed for my first red team
job one of the interviews revolved around a scenario in which I was an electrician in front of me was a light hanging from the ceiling and behind me is a light switch on the wall the light's currently on lists 10 ways to turn the light off 10 components of a functioning light and 10 ways to tell if the light is off and finally 10 ways to prevent someone from being able to turn the light off uh this scenario originated from a document titled jack of all trades which dates back to 2001 created by Pete Herzog uh its state of purpose is to teach Security Professionals to think outside the box and learn to use their knowledge in
different ways it puts people into scenarios that they're not likely to have a lot of experience in and then requires that they come up with answers based on those scenarios I think this is a great example of an exercise that helps to develop our Divergent thinking skills uh it has formed the structure of uh dozens of scenarios that I've used in the past tailoring these questions to be more appealing to the audience uh about 6 years ago I was visiting home and was asked to come speak at the local Career Tech Center about my career in security I have purple hair then not quite as many tattoos as I do now and I showed up in an all black outfit with
like this really long extra black hoodie uh I look kind of like I got trapped somewhere between a Hot Topic and the Matrix uh I talked to the kids about my experiences in school as well as experiences doing red team work for a large tech company I got to demonstrate The Perils of plugging in random USB devices such as the USB Ro rubber ducky as well as the USB kill killer uh to this day I'm very grateful for the generosity of the class teacher who let me destroy an old machine with a USB drive just to show it could be done uh I like to think that he also learned about the Perils of picking up random flash
drives that day uh but I also used the opportunity to give a talk to the students about Divergent thinking I gave them a scenario not unlike the Jack of all trades electrician scenario but more tailored to something that might resonate with them you have a test next Friday does it say that on the screen cool uh but the new Call of Duty also comes out that day how do you get out of taking the test how do you get your friends out of taking the test how do you get the whole school out of taking the test I gave the students a few minutes to jot down some answers to themselves and then asked for volunteers
uh to share some of their Solutions the initial answers were kind of boring uh I'm going to try to convince the teacher I'm going to stay home from school I'm like I'm going to convince my parents I'm sick that sort of thing um and then one student broke the Divergent thinking barrier and proposed that he would go around and break all the printers in the school because if if the teacher couldn't print the tests out then you wouldn't have to take it uh that's when the floodgates opened I think that uh the kids started to feel more comfortable sharing their more creative ideas uh one student said he would put raw fish in the HVAC system crank the
heat up and break the knob off I brutal but I appreciated it uh another student said they'd cause a car accident to take out a power pole nearby the school the morning of the test if the school had no power they probably wouldn't have students come in that day once the barrier of the conventional was broken the students probably came up with uh four or five dozen ways to get out of taking the test and I've never been so proud uh so I wanted to actually take some ideas how much time do I have left like three minutes three uh does anybody have an idea for uh how to turn the light off am I am I 10 ways to turn the light
off no this one okay 10 ways to prevent someone from or 10 ways to turn the light off anybody have an idea what throw throw a shoe at the light yes somebody said hit the
breaker that's what I like to hear uh all right on the flip side uh what about 10 ways to prevent someone from turning off the light any good ideas TI to a chair tie them to a chair small pox that will do it frame them for a crime and get them
arrested yes you yeah create a guard that that's like the the the answer that I would have expected to hear first uh which proves that we're at a hacker convention uh so uh I wanted to give a special thanks uh before I wrap things up I just wanted to acknowledge uh Toby kenberg who taught me what it means to be on a red team and rely encourages me uh to challenge my own assumptions and ideas uh Toby's the one who gave me the electrician scenario in the like originally and showed me the Jack of all trades uh questions and he offered me my first red team job I'm really proud to consider him a mentor even if we don't
really talk as much these days uh I also wanted to give a special thanks to Kelly Shortridge who inspires me and who encouraged me to explore early ideas of this talk uh as well as whose ideas have helped shape my own beliefs around security and challenging the status quo of the industry uh challenging our own mental models plays a like a key role in her book security chaos engineering uh which I highly recommend reading so to wrap things up I hope this is what you take away as an expert go out of your way to ask questions that you think others might need to know the answer to even if you already know the answer ask
to clarify acronyms ask to clarify assumptions that people are making you can lead by example and pave the way to a much more productive and informed team as a beginner be curious be inquisitive and don't be afraid to be wrong if someone says something that you think is wrong ask them to clarify don't assume that just because they have 20 years of experience uh that they're automatically right seek to understand why they believe what they believe finally and most crucially uh engage in Divergent thinking challenge your assumptions challenge your own beliefs challenge your own mental models this is how we become better not only at our jobs but as people thank [Applause] you right on
time questions concerns does anybody have more creative ways to turn the light off or tell if the light is off or whatever
yeah uh yes they make the PlayStation solar powered make it make it bike powerered so you have to exercise to play Call of Duty
[Music] a
[Music]
[Music] oh [Applause]
[Music]
[Music]
[Applause] [Music]
a
[Music]
[Music]
[Music] oh [Music] I
[Music] now
he
[Music]
[Music]
[Music] [Applause] [Music] [Applause] [Music] are [Music] [Applause] [Music] I'm just something I I'm just TR to give you [Music] something I'm just trying to give you something I do I'm just tring to give you something [Music] a [Music] oh [Music] [Applause] [Music] [Music]
[Music] [Music] I'm just in I'm just dring [Music] in I'm just [Music] something I'm just trying to give you something [Music] m [Music] w
[Music]
[Music]
[Music] a oh
[Music]
[Music]
[Music] [Applause]
[Music]
[Music] [Music] oh a [Applause]
[Music]
[Music]
[Music] the [Music] I
[Music] the [Music]
[Music]
[Music] he [Music] [Music]
[Music] a [Music] [Applause] [Music]
[Music]
[Music] now
[Music]
[Music] a
[Music] [Applause] [Music]
[Music]
[Music]
oh [Music]
[Applause] [Music] he he [Music] [Applause] [Music] [Applause] [Music]
[Music]
he he
[Music]
[Music]
[Music] back
[Music] hey hey hey [Applause] [Music]
hey hey hey hey hey [Applause] [Music] hey e [Music]
[Music]
[Music] [Applause] [Music]
[Music] [Applause] [Music] he [Music] [Applause] [Music]
[Music] he [Music] [Music]
[Music] [Applause] [Music] oh [Music] what [Music]
oh
[Music]
w [Music]
[Music] w [Music] w [Applause] [Music] [Applause] [Music] I'm just in something I'm just Jing in [Music] something I'm just try to something to BR you I'm just trying to give you something [Music] [Applause] [Music] [Music]
[Music] [Music] I'm just I'm just TR to give you something [Music] I'm just trying to give you something okay I do I'm just trying to give you something [Music] o [Music]
[Music]
[Music]
[Music] oh [Music] [Music]
[Music]
[Music]
[Music] oh [Applause]
oh [Music]
[Music] [Music]
[Applause]
[Music] oh [Music] e
[Music]
n [Music]
[Music] [Music]
[Music] [Applause] [Music]
[Music] a [Music]
[Music]
[Music] [Music]
[Music] [Applause] [Music]
[Music]
[Music]
[Music] a [Music]
[Applause] [Music] hey hey hey hey hey [Music] [Applause] [Music]
n [Music]
[Music]
he
[Music]
[Music]
[Music] [Music] TR [Music] back
[Music] hey hey hey [Applause] [Music]
hey hey hey hey hey hey [Applause] [Music]
[Music] a [Music] [Applause] [Music]
[Music] [Applause] [Music]
[Music] [Applause] [Music]
[Music] [Music] [Music]
[Music]
[Music] [Applause] [Music] oh [Music] w [Music]
he
[Music] h oh [Music] oh [Music] [Applause] [Music] [Applause] [Music] oh he
[Music] h
[Music]
[Music] [Applause] [Music] [Applause] [Music] a [Music] oh
[Music] I'm just TR to okay I do I'm just try to give you [Music] something I'm just trying to give you something I do I'm just TR to give you something [Music] n [Music] [Applause]
[Music]
[Music] [Music] I'm just okay I I'm just tring to [Music] something I'm just try to give you something do you I'm just try to give you something [Music] m [Music] w
[Music]
[Music] [Music]
[Music]
he [Music]
[Music] [Applause]
oh [Music]
[Music]
[Applause] [Music]
[Music]
[Music] oh [Music]
a [Music] alrighty good afternoon and welcome to bides Las Vegas common ground this talk is cyber harassment by Laura a few announcements before we begin we' like to thank our sponsors especially our Diamond sponsors Prisma cloud and vanta and our gold sponsors Adobe and project circuit breaker it's their support along with our sponsors donors and volunteers that make this event possible these talks are being streamed live and as a courtesy to our speakers and audience we ask you check to make sure your cell phones are set to silence be respectful and all that fun stuff um you do have a mic for crowd questions we'll go around and pass it on at the end speak into the
microphone so everyone can hear you and with that take it away all right let's see how this goes off of three hours of [Laughter] sleep uh yeah it's your fault by the way art in the back you started it last night all right so I'm LJ uh this is the second version of the Cyber harassment talk I started back in the day um one of the individuals that actually helped me start this is sitting right there that would be noobs he is wearing the masturbation Racing shirt that we'll get to later so with that um so who am I I am just a human like everybody else I am semi hug enabled I love animals if you've got cat
pictures please send them to me I love to see them dogs are you're welcome as well and uh maybe some drop bears so I have also been the lucky winner of Manifesto emails that could literally be put into a coffee book or maybe just a really great novel of someone trying to take every ounce of my life and blast it to the entire world um and after giving this talk the first couple times it's been really interesting how many times I've gotten a ceas and assist from random people who think this talk is about them so just for reference if you ever think this talk is about you and you need to send me a ceas and assist
you might want to look internally and figure out whether or not it is a u problem uh so as far as uh who I am and who I'm not so not a lawyer hired one uh one of the reasons this talk was born is it cost me $46,000 to get a protection order I am not kidding that is the normal average cost of obtaining one when it comes to cyber harassment um I'm also not a therapist I hired a seller one uh it's something I highly suggest anybody who's going through this or a child who's going through this um it's astronomically important for people to have somebody to talk to and so I would lean instantly on a therapist versus a
friend because sometimes you don't know what your friend's triggers are and you may actually cause them more harm by talking to them about things and it will end up not helpful so the definition of harassment in general is very interesting so I wanted to actually make this version of this talk much more um audience involvement so anybody here want to tell me what you would feel like was a cyber harassment to yourself I will give you my 2004 black hat bag if you tell me what you feel like would be a considered a justifiable protection order of harassment anybody willing to do it come on Art do it love you so and is very interesting
right if you feel unsafe if you genuinely feel unsafe about something that someone is saying to you you have every right to then tell that person to stop doing that you have every right to say I am not you know comfortable with what you're saying in the same sense if you have a friend who is posting things online or sending out emails or Discord or however many thousands of ways you can send people messages now if you find that to be uncomfortable you have every right to tell them hey what you're doing I feel is toxic and I think you should stop what people don't understand though is to then turn that into a court and a
protection order you do need a preponderance of evidence that that is doing that and that could be a lot of different things and what's very interesting is it depends on who the judges so my case was obviously very different than other people's cases I also don't know if the judge that I had happened to have that happen in their life and that's why they were much more you know listening and reading over my case where someone else may say this isn't a big deal so as far as defining harassment you need to Define that for yourself and then decide what you're willing to fight for and stand up for and that comes to yourself as well as
what your people around you are doing the thing I have heard the most throughout both of the cases that I assisted with as well as my own is that they think it is complete freedom of speech to go online and to absolutely blast somebody that is not true like if you were to walk into into a crowded movie theater or where we are sitting right now and one of you were to yell fire and someone gets trampled on the way out it absolutely is the person who yelled Fire's fault so people need to pay attention to what that actually means as far as who got injured from the words that were coming out of your mouth
or on text so this one's been a little bit interesting and it's changed um throughout from the first time that I gave this to now so from the first time that I gave this one of the big things is is when this slide was created by me I wanted to ensure I did not go into any details of how you would do this because I'm now just weaponizing anyone who's watching this one of the things that I never expected coming out of this is that what if it is someone who actually set up your network for your business or your home and then you guys get in some sort of a weird argument maybe you say
no about something you set a boundary and now they have all of your information it is damn near impossible to prove that at that point so if I had a time machine I 100% would go back to the ass hat that sorry for the whoever's listening if you're children um said that words are not a big deal and that you know the sticks and stones may break your bones that at this point is not the way it works um I know I meet with kids on a regular basis now throughout schools and Scouts and all sorts of different events that words are they would rather get punched in the face by somebody than have someone dog
pile them online so that is not a statement that should be made anymore if we've far outgrown that so this one is a new situation that came up um with me that was interesting and um now that I know you guys don't want to participate we'll just so imagine my surprise as I'm sitting at my house and I get served with a non-legal piece of paper that was super funny and it might as well have been in comic stance um and that person was now claiming that I'm the harasser even though I was assisting somebody with a case they are claiming I am the harasser so now they're going out and they're spoofing my phone number they're
creating emails that are just my name and they are going around and telling the entire oset community that it is me that is doing it the problem with that is once that lands you in a courtroom it is now costing you an exuberant amount of money to prove that you're not even the one that did that in the first place and it is a endless battle that unfortunately as we get further down this line that is what absolutely creates that helpless mentality and soul crushes people so that's the part where this community absolutely needs to start leaning on each other of the rule of if it's domestic we don't get involved has kind of gone out the
window when you are 100% aware that one of your friends or someone you work with is doing this type of thing and you don't open your mouth you are just as much at fault as the people who are doing it in my opinion because you are now not setting that line and that boundary and you are basically brushing under the rug where you can you don't necessarily have to cause a big drama llama over it but you can now turn around maybe to the person instead of having them spending their entire life savings in their 401k and assist them in getting them the help that they need to be able to get the case to where it
needs to be so these are Snippets from the emails that I was receiving and my lawyer at the time was receiving so you can tell from the bottom one it was somebody in our community there is not a day in the world that I will tell you who it is during the slide deck for many many reasons one of which is I hope they get help and they come back um they were actually really a good person um in the beginning so this is a situation where you're receiving these emails and what do you do with that what do you do with this information when you no longer feel safe in any way shape or form because
you are 100% aware that if your cell phone number is on your business card and your bill goes to your house now everything is out there so you're looking at these little tidbits of information and realizing that these quotes may seem like just words to some people these quotes are what a judge looked at and was able to issue a lifetime protection order for not only me but also my if I have a future spouse and children and not minor children my children in general so during the entire process of when I was gathering up all of those quotes all of the emails when I was trying to talk myself out of making a coffee book by just actually you know
redacting some things and submitting it to an editor I was looking at this going what do I do um there's many many many people in this community that had absolutely no idea I was going through it um and one of the reasons was I didn't think anybody would believe me I also didn't think that the words mattered or the large amount of stress mattered that was happening I didn't think that what was going on with my child was really something that people would care about um so I just kind of sat and idle and I stored everything I possibly could into a folder that I did not look at again it wasn't until I had a lawyer
take a look at it who stated to me that I absolutely had every right to get a protection order so during that process though that's where your money starts acre and once that individual found out that I was looking to get a protection order they were then fing I kid you not around 20 Page emails to that attorney costing me around $350 an hour to read these manifestos so that's where everything started mattering I can say if I didn't have a child I probably would have just stepped aside but the fact that my son was also re receiving letters my son was also receiving um text messages on Steam it became a I don't care what this takes
I'm going to continue to fight this and whatever credit card debt I have to go into I'm going to ensure that he's safe for the rest of his existence one of the biggest things that you have to be careful about when you get this protection order is the wording that's in it my original protection order was worded minor children so at any point my son would have had to go back and fight that fight again as he turned 18 so I made sure that it just says my child in that or children in general I can tell you when this court transcript hit my inbox I reread this over and over again just to be able to sleep at night
because it was the first time I felt like someone actually understood what I was going through and it was a complete stranger that was standing there like literally judging what I was doing during this process when I presented all the evidence at court he actually turned around pulled out 8 to 10 books laid them out on the table and had to find where the quotes and the threats and the harassment matched the laws at that time I got extremely lucky that judge was willing to do it he took a lot of time to sit there and really figure out what matched where not everyone has that luxury not everyone's going to have that happen so the fun Factor where I said
you might as well have served me with comic stance so the top part is what is an actual protection order if you ever have a friend or a family member that is trying to obtain a legal document to have somebody actually stay away from them it needs to look like the top piece of paper the bottom piece of paper and since I have moved I will say where I used to live is what the grand wonderful con County of South Carolina will issue for $15 and they will claim that that is a no contact order fun fact I was given one by someone who actually tries to drive me crazy and when I laughed with the police
officer on the phone I said this isn't a legal document he goes I know he goes it's hilarious that my tax dollars are used to serve people with that so what I did was I asked him I said how does one obtain one of these non legal no contact orders and it was go down to the county clerk's office you spend $15 and for the lovely tax money that we all pay a police officer will kindly go to their house and serve them with that so the reason my signature is on that is I just served it right back to him cu I thought it was funny so now we both have fun Merry Christmas so I'm the one who
laughed because that top one is something that again it took me an exuberant amount of money and time and evidence and getting it actually not forensically sound but provable that it was from that individual one of the things you need to look at if you're being harassed and you really want to have that is you have to have already established that that's that individual's email you have to already establish that that is that individual's account and then you have to on top of that make sure your lawyer is willing to fight and prove that they were actually accessing it at that time so it is extremely difficult to prove a harassment case that is exactly why for infoset
Community to if you know somebody's doing it you've got to reach out you've got to do something from a non-legal sense because not everyone has the money to throw down to do this so with that this is a no funding and lost hope case that I was helping someone with last year this one was a very interesting situation when it came from I joked around calling it spy versus spy for a while so it it's a situation where someone was a business owner on a small Main Street there's literally a 100 people in my town give or take old town and this person who owned the business every night would go home and would come
back and every morning every single device that it attached to her network that day would be blocked so she is a very non-technical person she didn't understand what was going on she's hot spotting off of things she finally calls me and says I know you don't want to get involved because I happen to know the individual that she was pretty sure was doing it she's I she was like at her wits end and so I go up and I look and I log in and all the Mac addresses are blocked so I go in I change every password I do all the things that they teach us to do you know mine is turn it
off and turn it back on again cuz I guarantee you that was not going to fix the problem so I go through and I do everything we're taught from a base level I go home that night I come back same thing the next day I'm up there and I'm actually working with her from my laptop because she was literally scared of what was going on this person was coming back and forth from her work front and back front and back all day W driving and again 100 people in this town I promised you weren't getting any unique IDs going back and forth here but every time that they were about 5 minutes out 50 ft out
or so it was really funny cuz her entire internet would drop and as they would walk past it would be dropped and then as they would continue on suddenly it would come back there is not a single police officer in a small town who has no Tech at all background that is going to understand what's going on and there is no world that I can stand there and even try to explain it to them because they now think that I am just crazy so she's going through this and back and forth she's taking photographs she has emails she has text she has everything that I would have assumed that I had she goes into court she has no lawyer she can't afford
one he crowdsourced the infoset community he was able to afford a lawyer that lawyer said that every single piece of her evidence had to be forensically sound as if it was a dang murder charge she tried she tried to get the notorized letters her and I worked on it short of somebody out of a movie you know running up behind him while he's on his laptop next door changing things on her network I'm not going to be able to catch this person nor am I probably going to be able to do that so she sent that email to the court saying she was done she absolutely was done she could not take it anymore um it was I actually
got her permission to say this but she did at that point need to seek help for an extended period of time somewhere else she could not even be in her home because of the amount that she was fearful for taking her own life and the amount of her Depression had gotten because she couldn't function just at a normal day-to-day basis so she sent this saying like I am done I am out once she did that she thought that's it it would get dropped they would stop however that person asked for that charge to be dropped but only with prejudice which means she can never charge that individual again with harassment so after that it got far
worse she can never take him back to court she actually shut down her business and moved her entire house her husband and her children to get away from it cuz she couldn't take it anymore then during my research of what was going on with my own case is when I came upon this case so this is where I really started having a big Outreach on going out with children and talking to parents and finding out that my case as an adult already put me into a state of depression that that my hair was falling out like there was multiple times that I know I couldn't really get out of bed in the morning but I had to cuz I had kids and I find
multiple cases of children ranging from the ages of 8 to 16 who have taken their own lives over cyber harassment so as far as her case it was a complete catfishing situation of a she was a little girl who had never had a boyfriend and there she was getting an online boyfriend who absolutely was her world and she thought everything was fantastic I highly suggest going in when I share the link in the next slide look at this person's what the mother has done because of what happened she has created such a foundation where you can go through and actually get their assistance to go out and have these same type of talks with the schools and ptas
and other kids functions and honestly even with your adult friends the fact that no one would ever think that a teenager would hang thems over having an online relationship is something that maybe from the 50s to the ' 80s we thought this is fine like this is just words what is going on now with the kids is that they're getting harassed in school they normally would come home and that would be a break for them that would be a mental break they hopefully have a good family or they have good friends or they can take a moment of silence to themselves that's no longer a thing when you have a laptop and a tablet and steam and various other
things my own son does not have Facebook for many many reasons he's a very sensitive soul and I just don't think that I need him to be dog piled on that he is on Discord he is a gamer the things that come out of his mouth on games I'm not very proud of but we have had discussions actually about when they're all talking like about each other in a horrible way ensure that your friends do think it's funny I'm probably one of the most sarcastic [ __ ] you will ever meet but I do make sure that my friends like are not in a state when I say it and if I do feel I've crossed a line I make
sure I check in with them so I do give him that lesson of it's fine to be online and it's fine to do that with games but you have to actually know who you're talking to and your friends and what type of influence you have on these people so this is the foundation that she has and the facts of what is going on this was one of the best things that I found during my harassment case because it actually gave me a situation where I felt like I wasn't the only one who's had this go on and I also wasn't the only one who had the feelings about what happened so it's one of those
things where you are in this state you're going through something and you say to yourself I'm done with this and you think you're the only one in the world that felt that way when you f stumble upon this and I ended up talking to the mother and I talked to other parents in the cases and some of them didn't want to be named for obvious reasons but one of them said it was fine to discuss it but it was an 8-year-old out of the Minnesota area that committed suicide over cyber harassment they literally felt as a 8-year-old child they had no other option because they felt like their entire life had been dogpiled online so that brings you to these are
the statistics of what is actually going on out there not only with adults but with kids and we need to get ahead of it we need to find some sort of laws that can wrap around this or some sort of groups that can assist somebody can go out and take what this mother has already started and take her to the finish line of you know creating more support groups of saying if you are seeking some sort of basic you want someone to get charged for harassment they may not get charged for the suicide unfortunately but they could still get charged for the harassment that is something that we could reach out as an infoset community
and be able to start helping people do there are things that are changing I don't know if many people remember the case but there was a case where a woman was constantly texting a man to kill himself he drove off a bridge she did get charged so it is coming around but it's people who then speak out of it and find out about it and they're willing to fight for it and I promise you it is ridiculously hard I get called multiple times per month on different people's cases and I sit there and I listen and it's extremely triggering but if I'm the only person that they have been able to speak to I would never turn off the
phone for that so it may not be something that you think you have time for in your life but make the time the suicide rate for the kids the fact that it is two times higher is something that a lot of people don't want to hear they don't want to wrap their head around it and so that's one of the things that find a way when you're going in to talk to your own kids maybe friends of kids adults I was there like I literally remember the final email that I got was that was the push over for me and the only reason I'm standing here today is because they're happen to be a friend standing next to
me who watched me fall apart while reading an email I cannot tell you what would have happened if they were not physically present when they watch my face go from a normal happy person to white as a
ghost so when the feeling helpless part catalog that evidence don't read it again I can tell you one of the hardest things about giving this talk because every time I do it I have to go back through mine I have to go back through mine I have to go back and look I have to look at what was said or what was done how much I'm willing to actually stand up here and say um I've had multiple threats that the entire manifestos will get released if I ever give this talk those manifestos have a wide range of falsified information in them ranging from that I am an extreme drug addict to I abort multiple children
there are massive amounts of false information in this that could be released but I'm still standing here and I'm standing here because of the fact that I know know that that person standing next to me that person who watched my face turn white that person who is willing to fight for me at that moment I want to be that person for someone else I want to pay that forward and I want to make sure that other people feel comfortable enough to pay that forward as well one of the other things that a big big thing when you're sitting in court that was very hard for me anyone who knows me I am a lot of a
human being I have feelings as the days are long and one of the things I had to do in court is you have to sit there and be completely emotionless and all factual what I learned later after that is that if I needed people to actually hear what was going on with me or what was going on with cases that I was working is I also had to be that way with other people if I was crying you would be appalled by the amount of people that wouldn't even listen to me and wrote me off as just being dramatic so I had to actually become this like extremely cold person if I'm working on this to be able to get this
information out there one of the other things that I did for myself during this process that a lot of people were unaware even exists is FMLA can be used for your own mental health I thought it was just for babies like I had absolutely no clue that if you were going through something in your life that you could take FMLA your work should be saying it but they don't you have to research it yourself you have to have somebody who tell you that it is something that CEOs of companies should put out they should absolutely tell someone if they are at a breaking point in their life it is perfectly acceptable to take FMLA take the time off that you
need and to come back I'm very fortunate that I have been surrounded by amazing companies that I have worked for that that was available to me without a fight at all nor should it ever be a fight but people do obviously have you know oh you're taking three months off or what is it for you also don't have to tell them what it's for the other part is the turning to someone's supportive there are a crud ton of people in this room right now that I have turned to during this that have really helped me out there are other people that actually saw me in a certain State and turned their backs on me and it put me in an extremely
different situation so I learned very very carefully who are the people I can go and talk to who are the people that I can feel supported by and who are the people that genuinely have no bandwidth to listen to what's going on and that is okay too that is one thing that I have learned that I have had to do for myself it is okay to tell someone I don't have the bandwidth to talk to you about this right now I'm not going to be the best person for you I'm not going to be the one who's going to help you through this but let me get you to where you need to go it's okay to not be able to be that
person for someone what's not okay is to make them feel they're just being an overdramatic crier in the corner and leave them there so the keeping yourself safe part is what is very difficult so when I say that the the justice system can work for you sometimes yes it can if you have money that's the problem right what do we do with that do I find a way to start a foundation for these cases do I you know start educating more people on how do we help each other so that the bank role is not that high I'm honestly standing up here telling you I don't know yet that is not something that I have been able to figure out over the
years the protective orders I do highly suggest getting them if you have what they call and I buter this word every time a ponderous of evidence you can obtain a protective order it might only be for 3 months your first round of protection orders you go into it is temporary if you can get a permanent one you do need to sit there with a large amount of evidence that is forensically sound to a degree of that judge what all of us sitting here considers forensically sound is going to be wildly different depending on the judge that's there and the lawyer that's there so hire an attorney that has a cyber background or has cyber training so that
they can understand what you're looking for as I've said before definitely look into getting that FMLA take a break it's okay it's okay to take that break your job will still be there without FMLA and if it's not probably shouldn't have worked for that company in the first place they're not there to support you the ones that's the hardest thing to keep yourself safe is do not block that person find a way to send those emails somewhere else find a way to maybe have a friend of yours buffer that account like when it came to the woman that came to me case with her network issues I was kind of funny about it I just went ahead and bought her all new
network equipment and kept the old one standing and let him keep hitting that one all day long it was great I added so many different devices to that he was so busy trying to figure out what to block it was really fun for me but sometimes so this is the biggest part of I redo the statistics at the bottom here because I really do want people to understand that this is not something that's happening to a few people this is happening to a lot of people around us every day and everyone's going to handle it extremely differently one kid being called overweight may take that extremely differently than someone who comes to me and says it or someone comes
to my son and says it because my son will constantly throw the joke of something along the lines of I'm built for I'm built for Comfort not for Speed so like there's there's some kids that can take it there's some kids that can't so really paying attention to that cyber bullying and what any human being who's having it happen no matter their age range and the biggest thing I have to drive home is speak up if you see someone doing something toxic online it is amazing to me on how many times I have had people tell me hey I saw something about you I didn't want to tell you because I was afraid I was
going to be the next Faceook Facebook post of them and I always say to them like well did you tell them to stop like what would happen and they would say oh I don't I don't want to get involved I don't want to get involved in that and that's everything ranging from somebody stating where I work somebody stating where I live uh I've had somebody uh lovely call me a rapist online trying to get under my skin that's uh also been sent to my hacker families um I have had that spread around town so that people come up to me and tell me that I can rape them any time like it has been endless for me on the harassment but the
amount of people that I have had to help me and support me through it are my Saving Grace and why I'm standing here and I promise you you have friends in your group that are going through a similar thing in one way or another they're just not probably speaking up about it because they don't want to cause drama but these statistics are extremely real and they're going on all the time one of the biggest things I did wrong was like the case with the foundation I continuously read and kept going back over bad in Washington DC itself has out every 10,000 people about 7.8% are lower that's 780 in San Francisco have every 10,000 the same number only about 6
about little over half half% 65 lawyer so what's wrong with that situation you see what I mean same number of people and you got over about 12 times as many lawyers in DC as here so uh uh so that that concerns me be careful who you get that one again as a lawyer CU there there's a lot of ambulance chases out there and a lot of lawyers should not even be they should be disbarred but they're being protected and that's why concerned anyway that's uh why I wanted to mention about laer it's a bad situation for our country right
now all right I can go back to drinking now [Laughter] hey wo [Applause] [Music]
la [Music]
he
[Music]
[Music] h [Music]
[Music] now [Music] [Applause] [Music] [Applause] [Music] [Applause] [Music] a I'm just TR to give something okay I do you I'm just TR to give you [Music] something I'm just TR to something I do I'm just to you something [Music] Co [Music] w
[Music]
[Music] [Music] I'm just try to I I'm just try to give you [Music] something I'm just TR to give you something you I'm just trying to give you something [Music] oh [Music] w
[Music]
[Music] [Music]
[Music]
[Music]
[Music] [Applause]
oh [Music]
[Music] [Music]
[Applause]
[Music]
[Music] n [Music]
all right good afternoon everyone Welcome to our last uh presentation of the day here in common ground uh we've got Harriet here who will be talking to us about why she's our oceans 11 a IAL so yeah look forward to that um before we get to that like to thank our sponsors and especi our Diamond sponsors prism cloud and vanta and our gold sponsors project circuit breaker and Sam grab um this is being recorded we ask that everyone keep their cell phones on silent to be nice and you know all those fun kind things uh when we do questions afterwards I will have a microphone I will give it to you you will speak into
it makes it easier for everyone to hear with that take it away and uh she's supposed to be doing a shot right now think that that's not a thing here right like I I'll they they've given me this very large cup of bourbon so I don't know about oh no what's that um so I don't know I can have a sip but you know I've got a few more talks this week so I'll I'll save it for them okay there we [Laughter] go okay can everyone hear me okay with the mic there okay I um if I hold it I'm liable to turn on turn into like a standup comic and no one wants that so
it's it's for the best that we don't go there um but thank you very much for sticking around until 600 p.m. the last session of the day um um I appreciate it um I'm a bit jet lagged at the moment my flight was meant to arrive last night I'm from Australia it arrived today so feeling a bit woozy but I think that means that I'll be like very candid and honest maybe and hopefully I'll remember my slide order but anyway I'm delighted to be here with you um so on your Oceans 11 team I'm the the AI girl but guy has a a better ring to it so so that's the title of going with so our objectives in
the talk today um are to hack the Casino um specifically the AI the artificial intelligence of the casino in question um so we're going to start with a bit of um open source intelligence and um and Recon we're then going to focus on the facial recognition Ai and we're going to put it in the perspective of AI security uh I'm an AI security person I run an AI security company in Australia um I've been working at the intersection of data science and cyber security for about 10 years now I started in Consulting uh then moved to a startup I worked in the Australian government and my PhD is in machine learning security and then a
year ago I started Mal um I'm also online in various places at Harriet hacks um the social media content I'm not really very good at but you can find me on X um Instagram Tik Tok maybe you don't need to check out all of them but um you can find me there we have a podcast it's a it's a whole thing um not to be totally cringy so why the Casino well it's Las Vegas why not um the casino is a really good case study or analogy for every large organization at the moment um who's grappling with this sort of tension of trying to adopt artificial intelligence very quickly but maybe not necessarily feeling um like they understand all of
the all of the potential risks involved and so obviously the casino is one of them and they have a lot of money so they need to care about risk um as do many other organizations so some disclaimers um we had this particular casino's permission um the attacks are real but I'm not going to show you exactly how to implement them um on you know camber casino's uh AI itself um and can I get a a show of hands in the room of who would consider yourself maybe an AI person okay and a show of hands for the Cyber people okay that's what I thought so AI people please don't come at me this is more geared at cyber people who might
not know that much about AI security um or it's meant to go sort of from a gentle 0 to maybe 80 um I'll be giving this talk at Defcon as well and it'll go into a bit more of the technical detail um but this bsides talk is a little bit more about the the theory side but but with some cool attacks um while I'm here we're talking about um camber casino's permission so as you can maybe hear from my accent I'm from Australia I'm from canra is anyone familiar with canra Australia yes I hear some some yeses for those of you who aren't it's our Capital City a lot of people don't know that a
lot of people think it's Sydney um but cber is our Capital it is our political Capital so it's very politically focused um it's a bit like DC but much smaller so we have one Casino um and fortunately Casino CRA were really um really accommodating and willing to work with me on this so C CRA casino is our City's best casino and our only casino but don't let that deter you they're they're pretty good um if you're listening you a great camberra Casino okay so um for the non-gamblers Among Us I think I went in with a certain set of expectations around how casinos would use artificial intelligence um but um I I found that they weren't necessarily fulfilled so so
let's just dive very quickly into casinos in general um so they make a lot of money so they need to care about AI risk right um they have a controversial history um in particular in Australia um some of our big casinos have been under quite a bit of heat lately for uh money laundering and not complying with their regulations I'm not saying this is true of casino cber but this is sort of the landscape in Australia at the moment there have been some Royal commissions uh around the world casinos have been um known to have money laundering as a risk you know of being able to move money around that's that's sort of the nature
of it so um in terms of you know using technology appropriately it's something they really care about given the the landscape that they're in at the moment um they use artificial intelligence um I thought that they might use artificial intelligence quite a bit for say detecting card counting or things like that um I I realized very early on that actually facial recognition and person identification was the most most important use of artificial intelligence um and there aren't that many providers of this kind of AI technology to casinos quite a lot of them um are the uh sort of Casino chip um and and card providers themselves it's it's a small landscape um does that matter well we'll we'll get
to that later anyway um but facial recognition and person recognition are definitely the most important forms of AI um and that's because C counting isn't illegal it's considered Advantage play um which maybe it sounds obvious I didn't know that um so you can card count but you shouldn't do it conspicuously or you can get thrown out because the casinos are able to throw out anyone they like um and so if you are um you know ostentatiously card counting and winning a lot of money they really don't like that um because the like sort of the the algorithmic bias in in the favor of the casino um like what I also found interesting was that the
in terms of hacking an artificial intelligence system um that can mean a lot of different things depending on how you actually Define ai so here's the first bad joke of the day so when I first told my mom that I work in AI she said Oh Darling why are you working in artificial insemination because see that gets a lot it doesn't get to laugh in a lot of crowds um but she works in the medical profession so for her that's what AI is and so for I think most of us we would know that AI is artificial intelligence but even so we all have a slightly different idea of what that actually means so maybe for some of the
Cyber people um as in terms of level setting when I say AI security um people often interpret that in different ways often people assume it means AI for cyber security um among the AI security folk um that I sort of work with um in this context AI security is the security of the AI system themselves but that term does turn tend to get muddied a little bit so when we think about AI security as a field it's more likened to sort of cyber security when you talk about the actual security of the of the system itself and in terms of the potential attacks and vulnerabilities in in AI systems um these historically have come from the academic field called
adversarial machine learning which has been around for about 10 years and the the slide that should be showing is an example of an adversarial machine learning attack in the wild so this is being able to add for example um specially crafted pixels to an image or specially crafted sort of material to an image that can prevent a model from recognizing it accurately so if you're able to add these perations to a stop sign um an AI system that's meant to be doing stop sign detection or you know object detection um can be essentially hacked you know so it's disrupted or deceived so that it can't actually recognize the stop sign so this attack sort of came to the with
this classic example which many of you may have seen before um but this is from sort of 10 years ago it's by this pivotal paper by Zing Goodfellow not to get too theoretical or academic but it's basically showing that you can add specially crafted pixels to an image um in this case of a panda to prevent the model from recognizing that Panda and instead recognizing it as a gim instead even though to a human Observer they no difference and these are specially crafted pixels they're not random there are lots of different methods to create these pixels so in a Compu Vision uh example like this you can see um that the the noise we're adding um is
is basically that that sort of pattern but all of these methods can be transferred to other fields as well or other domains that are a bit less obvious for humans to detect like uh like audio signals um you know things like RF um those sorts of fields so this is this this field of adversarial machine learning is um sort of the um the origin of of a lot of these offensive AI attacks so offensive attacks on AI systems themselves but when we think about actually hacking um AI or whatever has been known to be uh whatever has been referred to as AI in the past we can go way back and in the context of casinos as long as as people
have been using algorithms or as long as organizations have been using algorithms people have been finding ways to hack them so if we think of an example that we would all know things like email spam filters you know people have always been trying to evade those kinds of algorithms so that they can get spam through in the context of casinos a really you know cool example I guess is from things like random number generators um as we all know random numbers do not exist in a computer sense you have to have an algorithm that um that gives you the next number and so there have been some pretty high-profile hacks in the casino World um in in the
'90s uh for example this um particularly big one is from 1995 Ron Harris was employed by the Nevada gaming board whatever that that board is called um and in 1995 he was able to predict the next numbers um in in a game of Kino so that he was able to win $100,000 um and he was uh he was found you know that able to to go up to his hotel room and find him but there have been quite a few examples of these kinds of I guess hacks um in the casino context another one you could really consider a hack in a sense is is card counting itself cu the whole idea is that the
statistical um nature of of every gambling game we play means that the casino always wins right and this advantage that the casino has is different for every game so things like the slot machines it can be up to 25% uh advantage to the casino and the only reason that's limited is because of Regulation so that it's not even higher but in a game of blackjack it can go down as low to uh 0.4% depending on the rules of the game um and if you're able to play Blackjack according to perfect strategy and then include card counting so that's a way of you know sort of hacking that algorithm as well so that you know exactly the right plays to
minimize the um the fa in the casino so thinking in terms of the adversarial machine learning landscape so far so today in terms of the number of attacks that you could Implement on an AI system there's over 100 um and this diagram you don't need to worry too much about all the all the parts of it except that for every kind of attack surface you have in an AI sense whether you're talking about um a machine learning model like a convolutional neural network which is a type AI model that's really good at computer vision so recognizing images or something like a Transformer down the bottom which is very good for natural language processing so think things like chat GPT
um they have different kinds of model architectures therefore different kinds of attack surfaces and therefore different kinds of attacks that are more likely to work at different stages of that attack cycle um and these are just a few of them but in things like mid's Atlas repository which is the MIT um attack equivalent but for adversarial machine learning there are over 100 different kinds of attacks listed so that's the landscape of of adversarial machine learning um for for cyber people so I assume everyone here has seen Oceans 11 or you whatever the premise or Oceans 12 or 13 or8 um I I don't think I need to belabor it basically it's a it's a heist so I
thought what what better sort of narrative structure to have for a talk at you know in Las Vegas than to to try and commit a heist um and I'm not you know unfortunately I don't have any George Clooney or brid pit look alikes in my team um but the idea of this talk is to hack the casino AI that is most relevant to them which I found was the the facial recognition AI um to sort of get to my goal and what that goal is we we'll also discuss a bit later on so this is the process I went through so first of all we're going to interview the casino staff we're going to pick our
Target models we're going to implement the specific attack that I'm looking at which is called a distributed adversarial region we're going to disguise those regions in in different ways and then I guess we'll reflect on it and get some lessons learned or something um so the first step um we want to do a bit of bit of Recon we want to understand our environment so what I do is I tell people I'm a PhD student doing a Pap paper on AI security and I want to interview experts this is very successful I am actually a PhD student I've been doing it part-time for a very long time um like way way too long it's very sad um but people are more than
happy to chat to me about some of the most intimate AI security problems um people are extremely candid um I'm not saying that's necessarily true of of cber Casino um they were appropriately candid but I've been doing these kinds of interviews with lots of different organizations so Casino cber was one of them which was sort of perfect for this talk but over the course of the interview process which is for the PHD and also for the company so it's sort of nice jeal dual use there um I've already done over 50 interviews of 43 different organizations and it's all on the topic of sort of AI security and the potential vulnerabilities in their AI systems and
whether they know much about their AI systems or their AI security risks and as some you know initial um insights um 94% of those people in the organization could articulate how they use AI you know basically every organization is using AI at the moment but only 8% could articulate how they secure their AI so ability to secure it from those adversarial machine learning TXS I was discussing earlier so things that would disrupt deceive or disclose information from their systems um there's a really big gap there now that's not necessarily the the case for Casino CRA for for them they're they're at an interesting inflection point where you know big casinos so the the kinds of casinos that are in Las
Vegas um already Implement facial recognition Ai and many other kinds of AI um as per course like the the the building has been designed to be inherently um aable you know the the way that the architecture is you know you have to walk through certain corridors at some point that's to make sure that the facial recognition cameras can can find you the you know where you're able to park your cars um just the different Thor fars in addition to all of the sort of psychological tricks that are done to keep you in the casino um are there because surveillance is inherent in how you experience the casino um and it was pretty clear early on that facial recognition is the most
important way they do this even for things like money laundering or card counting it's not necessarily the AI that is you know detecting that but it's the responsibility of the dealer to be able to identify what's going on identify dodgy people and then alert the casino so that they can then put it in their facial recognition AI systems so this is by far the most important use of facial recognition AI um for those casinos that haven't been able to fully adopt um that kind of AI yet it really still relies on their people so you still have rooms full of people surveilling cameras in real time and trying to um hope that they catch the
right people based on um you know manual processes which is really hard and obviously this is um you know less likely to catch as many um people or entities as an AI system would but then um you know there's there's the whole privacy trade-off there too and just as every organization is you know trying to adopt artificial intelligence at the moment that's true of casinos um it's very expensive um and they really have to sort of make that judgment based on um you know how profitable they are so Casino camber is going through that sort of inflection point too and that decision- making process so we moved to step two and now we're choosing a victim model so there
are lots of different uh sort of surveillance AI providers out there um these are some open-source facial recognition models that exist just a few of them there are many um I could also create my own sort of custom model um but at the end of the day they're all pretty similar because they all are for image recognition and so they therefore all use this particular type of AI model uh machine learning model called a convolutional neural network uh and at the end of the day that's basically a model that's based on the way that humans are able to identify faces and objects as well um based on our brain structure so we have input data it goes through a sort of model of
of neurons and sign upses between them and then we're able to predict uh who someone is or or what they are um and so in a machine learning sense that's very much the same too you have neurons connected by different layers and there's a training process so that based on historical data You can predict future output so regardless of how um you decide to I guess customize your model um at the end of the day they all rely on the specific architecture and a very similar training process and so what this means from an AI perspective in terms you of how you actually um predict what a face looks like um it sort of compresses in a
mathematical sense into this space called an embedding space so all of the higher dimensional data that is captured About Faces um if we were to display it as a 2D image here or if you could imagine it in 3D it's basically like different clusters of features that are that are similar so in facial recognition for example it's very much based on the geometry of the face so how close people's eyes are what kind of shape they are relation to the rest of the face and so you end up with different clusters based on different features you're looking for and based on how similar the Clusters are um a model is able to make a pretty good
prediction the other thing that's unique about talking about artificial intelligence models versus you know a cyber system is that if you're thinking about a model um you you usually refer to its it's sort of evals or um you know lots of different factors you know statistical mathematical that that tell you how good it is at predicting something so here we've just got accuracy and these are a few examples of open-source facial recognition models and how accurate they are so they're all pretty accurate um they're all over 97% most of them are close to 100% of course how you decide to test this um is a is a question these are sort of based on um
research papers that that Benchmark these kinds of models um however uh the thing machine learning models is that because they all rely on that same convolutional neural network architecture you end up with these models that converge to the same kinds of features and representations um no matter who is building that model or or what organization is creating that process uh research has shown that actually if you're comparing uh clusters of models or similar models that are designed to do similar things they are 95% similar to each other at least all of the the numbers range from 95% to 99% so what that means is that if you take two models created by different organizations by different researchers
if you were to compare what they look like on the inside they are 95 to 99% similar and this is true of uh you know models created by companies that represent that IP there's only so many things you can change about a model if it's designed to do a certain thing so this is an interesting point but for us this principle of convergence makes it really easy to attack models because I can create a model that is almost identical to any other model and then create an attack and launch it um against that victim based on the surrogate model because the same data plus the same training type basically equals the same model so the third step is I'm going to
create what's known as distributed adversarial regions and this is the attack that we're implementing here so here's just an overview of some different kinds of adversarial machine learning techniques that have already been tried in in the wild so we've already got that that Panda image there the the one in the middle is these adversarial glasses so someone is able to um not be recognized by facial recognition or or person recognition by wearing um either glasses with this special adversarial coating or a jumper with the adversarial patent on it as well um the last image I like to show because it's funny but basically the the one with the cardboard box over the person is a a study done by Dara or or
rather Dara created a person recognition model to use in um sort of Urban Camouflage environments um and they asked a team of their Marines to try and hack the model and basically they were just able to act like not people and defeat the model um so they did things like hold branches and wave them over their heads and put you know cardboard boxes on them on their heads um so anything they did was that was a bit outside the norm was able to hack the model um which I like to show because I guess it's not really a sophisticated attack but these attacks do still do work um now is this attack a sophisticated attack I guess it depends
so this is an attack that I created um for specifically Urban Camouflage settings um so the research that I do um well I started when I was working with the defense department so a lot of it was sort of very focused on sort of military applications and the National Security context and so the idea is that you would be able to apply this specific attack to um to Urban Camouflage environments so this is the methodology for those people who are methodologically inclined or like a good diagram but basically at the end of the day I'm testing I'm taking an image of of something so a ship for example here um identifying a region in that image or
in that sort of that that video that is most likely to cause a misclassification and I test it by applying different kinds of adversarial machine learning methods to that region um testing different models and then from there I can apply to lots of different sort of case studies and settings and and kinds of objects and so the the point of this specific attack is really that I want to see how I can instead of having to perturb say the actual ship or an object that's being classified how I can add regions so distributed regions to this environment that prevent the model from recognizing what that object is without having to actually change anything about the
object so for example if I'm looking at a ship can I add adversarial Boys around that ship that prevent it from being recognized by um image recognition models and you can you can do the same for for planes um and for other kinds of military platforms as well and the the reason that you would like want to do something like this even if you're not actually um you know even if a human can recognize that maybe something there is a bit different is because in a lot of settings you don't always have a human in the loop or you don't have a human in the loop until a lot later in the process so it's more
about disguising something like a you know platform like this from some sort of automated detection so because I'm a researcher unfortunately have to end up with sort of graphs that look like this um as part of the the testing process so what we're really doing is we're testing the extent to which um applying these distributed regions to an image are likely to um reduce the confidence level of a model in what it's looking at so um for example if I were to um Place some distributed adversarial regions around that ship it would reduce the confidence that a model has that it's a ship by say 40% um and the we we did find that applying these regions to those sorts of
object decreases the probability by 40.4% but I'm applying this to casino facial recognition AI so this is a very unflattering picture taken of me um walking up to facial recognition detection and using this region I want to try doing things like adding jewelry design is maybe to be you know um could be better come on video hang on I downloaded it over here so this is um what this sort of what this ends up looking like so I have a demo here this is just um implementing the attack against one of these open source models if you're not a code person or you're not really familiar with machine learning code the point here really is
that you can do this in a minute or so so you don't need to worry too much about what the code represents apart from noting that we're taking this open source model we're applying we're creating the pations based on different um different targets this time it's me and my face I'm testing adding different um jewelry um around my face and I want to see if I can prevent that facial recognition model um fine tuned on images of my face as well um to prevent it from actually recognizing me so I know this is really exciting but you have to include a code demo right um so here it's shown that there's a match found so that's me testing the clean
image of me without any of those perations it was able to find the match that's that original image and then I'm going to test the new image the one with the adversarial regions applied and we're going to see if there's a match or if it loads as a spoiler there was no match so just does that add to the suspense I don't know um so I guess we could say that we hacked the AI right we were able to prevent it from recognizing me um we certainly want to test it we want to see the extent to which we're able to decrease the confidence of a model in predicting and recognizing me and the number that we end up with is like I
said before 40.4% so for the images that that model was tested on the confidence in being able to uh correctly predict what it should be was decreased by 40.4% so I had a real problem problem like trying to actually describe this as a hack you know I think the challenge in applying cyber security kinds of terms and methodology into the world of AI security is that machine learning models are inherently probabilistic so it's not like youd necessarily have a a binary answer at the end of the day or or you might but it'd be more scientific to test at multiple times using you know the multiple different variables and and different kinds of images so maybe I'm
able to prevent that model from recognizing me once but on the whole it's more about actually reducing the likelihood that it is able to predict who I am and the the tough thing about applying this to facial recognition is because in is because you're more likely to sort of bound the face so to look the model will only look for features within the sort of bounding boook in my face whereas other images are more likely to Ray on context for example um like that the the Water behind the ship um helps the model understand that that's a ship um so I kind of hope that it would be better like I hoped that it would you
know the results would be I don't know more shocking or more interesting or just more of a a hack or an attack um so I sort of I was a little bit disappointed I had to take a pause I did you know what you always do when you want to run away from a problem and that is I went on a trip I went to Europe it was lovely um and then I realized that the problem with my sort of disappointment in the attack was the same problem that I'm always encountering in my my work and that's that most of the organizations that we work with really talk about adversarial machine learning in terms of attacks and
defenses so all the different kinds of attacks that could be imployed against a model and you know versus all of the different kinds of defenses that should also be applied to that model but that's not really the right way of framing it because in cyber you don't really talk about cyber attacks versus defenses as you know that the the be all and end all you really talk about maturity and risk and so an organization has to understand their you know their risk their highest priority assets and targets and then decide the priority mitigations right and that's not something that is historically talked about in the world of AI Security even though it should be like that's the point we want to mature
to so the questions that we really want to be asking if we're trying to hack this facial recognition model is you know what what is the target what am I trying to steal if I'm bypassing a facial recognition AI in a casino context why is that and you know if we're sort of generalizing this and applying this to any organization that is a question that every organization needs to ask but if we think about it from the casino context it really means that you know some someone is possibly a money launderer or a card counter and you just want to do that and not be recognized and taken out of the casino and if that is the goal maybe go to a
casino that doesn't have facial recognition AI um is the first thing the next question that we need to ask in this context as well is like I prefaced before what does it mean to hack an AI model it isn't really the right terminology to use when if we're thinking about risk in a cyber sense and thinking about all the risks that a system might have in terms of you know the system as an attack surface um just so it's more about the risk that an AI system is going to cause a misclassification rather than the risk of an AI system in its entirety because many AI systems can do the same thing but they currently isn't any regulation
or really understanding there's certainly no requirement to have any research open source information um or close Source information about how robust those models are so just because a model can perform facial recognition pretty well um all of the different models we tested were extremely variable in how robust they were to um to an attack like this and that's because as I say here a machine Learning System is inherently probabilistic it's it's not deterministic like many of the Cyber systems we're used to dealing with um and the last thing as I've said before is what are we trying to protect um we're trying to protect the casino money their profit their brand reputation um all of these things but thinking about
it from the perspective of risk um because an AI attack an attack on an AI system is just part of the kill chain right it's not like we really think about um having one attack that's powerful and able to do everything it's more about all of the different ways that you could alter this attack depending on the um depending on your Target or the particular case study and the way I like to think about it is in terms of a stu net style attack versus a Dos attack uh because many adversarial machine learning attacks tend to fall into sort of a stu net style they're very cool they're very complicated they're able to you know deceive an AI
system in I guess a really cool way you know a way that a researcher can be like ahuh this is a cool attack right um but at the end of the day it's actually maybe the these sorts of attacks that are that are here um that aren't quite so cool to an AI person maybe but are actually able to disrupt almost all models um maybe not in the you know maybe I'm not able to get the model to think that I'm Angelina Jolie rather than Harriet Faro but I'm able to cause a misclassification almost all of the time just by disrupting it and that is far more likely to have an impact to a casino um or to an organ
organization um because even though that you know that that number that was 40.4% which is a bit disappointing if you're able to change that attack if you're able to think about all of the adversarial machine learning attacks that are out there and apply that as part of your existing cyber kill chain the success rate is 100% so this is the kind of you know thinking that needs to happen in the field of AI as well um all of the the other sorts of creative ways that you could apply something like a distributed adversar region in the facial recognition context um you know things like pimple patches being able to if you have access to the
actual camera and being able to stick like a clear piece of sticky tape with that adversary region in it depending on all of the different you know points of access you have to a system those are all things that could be done the other reason I really like the casino context is because it's all about surveillance and it's about surveillance by Design and as we're moving as a society to having artificial intelligence that is uh increasingly being adopted that's something that we really need to consider because our societies are increasingly becoming surveillance by Design um and whether this is a good or a bad thing is you know discussion for lots of people including you know sort of thinking
about policy and regulation but if we think about the casino as an interes
Related talks
31:06
20:51
27:37
50:12
16:22
23:19