Who Goes There? Actively Detecting Intruders With Cyber Deception Tools

we're going to get started uh we have Dwayne Daniel who is presenting uh a talk titled uh who goes there actively detecting Intruders with cyber detection tools uh please give them a warm welcome Hey everybody thanks for coming to my talk uh if you're going to have hallway track hallway is out that way welcome everybody else in I know there was a line in the bathroom so I'll stall for just a couple seconds while we get settled but if you want these slides and I highly recommend you get them uh I made a tiny URL for you I'm not sponsored by tiny URL at all but I do love them as a service forever URLs just

because I mean they're always trying to sell me something else all the time but I love them uh and you can customize your Ur again I'm not selling them um so part of me real quick I need to go deploy something I know that's a little bit odd at the beginning but I'm going to hardcode some access keys there for AWS and Commit This to this REO make a nice little update message and that should be live right now so back into main TF and this project yep there they are all right let's see what happens um we'll come back to that I'm Dwayne I came here out here from Chicago uh I love this area I

love Colorado a lot this was number three on my places to move list behind uh Chicago and uh Baltimore of all places but like I almost moved to Denver I got like this close um so hit me up on the Internet about anything but check out the security repo podcast as well Jason is a guest we're going to have another episode with him coming out next month so very excited about that um and anything else you want to talk about I love talking about karaoke and uh rock and roll if you know any local bands that are playing tonight let me know um I work for a company called getg Guardian if you didn't get a

sticker uh there's stickers in the room and I have more in my bag happy to give you some um we do this stuff as a company honey tokens is going to come into play and some of our research that we do but I'm happy to talk to you about any and all of that stuff as well out in the hallway we're not sponsoring this event unfortunately but still would be happy to talk to you about okay the Crux of this whole conversation is that we know what attackers want we know how attackers behave and just what does that look like in real life what is the actual implications of this well that's I'm going to tell a couple horror stories

and I always like to preface that with hey these are stories these companies still all exist that people might have got fired over it but no the overall these aren't that terrifying and if you are terrified just recite the Benny jzer at Litney against fear and you'll be fine if you want to talk about Dune later I'm a huge Dune nerd um so Cloud flare OCTA maybe you heard about this one I think everyone on Earth got hit by this eventually down the road uh there was this nation state actor who used the lapsis Playbook we think to get into OCTA um stole a bunch of har files har files are the uploaded actual credentials you give to OCTA like here's

my credentials um it's one of the ways you can do things anyway seals those OCTA does the right thing discloses and tells everyone hey you need to rotate all your credentials you got with us asap uh and everybody thought they did Cloud flare to their credit I love Cloud flare I use them from all my personal websites uh they're just an awesome front layer they do a bunch of cool stuff uh they thought they they handled it they thought they rotated everything and then it turns out they didn't a month after the OCTA breach uh November last year all of a sudden their jira their bitbucket and Confluence got infiltrated by these same actors and that's bad for a lot of reasons

because people tend to hard Cod credentials like we'll talk about later and basically at the end of the day glad Flair says we got this we rotated all 5,000 of our keys that sounds like a fun day work um Circle C uh maybe you got hit by this at the beginning of last year woke up right after New Year's and found hey all my builds broke why are all my pipelines down why can't anything run well because a developer was running Plex on a personal machine that was also doing work on uh never patched that Plex server through a Discord thing people like did spray attacks against Plex servers found hey this person actually works for OCTA and I can steal all their

credentials from the browser oh I can get into Circle CI now or no not Circle octo again this person work for Circle CI I can get into Circle C now hey I can start stealing credentials hey I now can get into all of these customer accounts and I can start stealing their credentials uh the same day they announced this that they were infiltrated and that they had to the only thing they really do is rotate all of their internal credentials and tell all their customers hey you probably should rotate all your credentials now that are involved with us anyway um they apologized the did the best but the same day that they announced that an

independent researcher out on Twitter said hey all my Circle C honey tokens went off something's going on what any might know about this and then like two hours later the news drops um Toyota this is a really interesting story uh not picking on Toyota or tconnect tconnect itself is a really cool technology I love Toyotas cars but they hired a subcontractor to take care of maintenance on this repository at some point in the past for some reason that developer pushes part of the repo into another differently named repo I would own a GitHub in public unfortunately contained a real data server key that had 296,000 rows of customer data on the Japanese website they apologized about this um but it took

them five years to figure this out and an independent researcher finding this repo out there and saying hey Toyota do you know this is here and I can see your real customer data and they were horrified and they rotated it of course and that's the story I read a lot of reports and a lot of information a lot of news bleeping computer dark reading every day and then all the new reports DB data do data devops report all these things and huge shout out credit to Jason's Arcanum security bot because last night in my hotel room I started throwing it all these reports that I haven't quite made it through because they 100 p reports I

don't get that much kind kind of time um and said hey summarize this for me give me top 10 list just build me top 10 list after top 10 list of what was the learnings of this and these two jumped out at me in a major way uh data do said the problem one of the major things is that we're still 63% are still relying on Long Live credentials by Long Live we're talking like a year or more I'm pull the how long do you think a credential should live what's like the the longest 15 minutes 15 minutes that's a little shorter than I would say but the reality is I I think if you talk to people like

um uh Veritas not Veritas uh Veni um people like cyberark it should live for the life of the workload and that's it and then it should just vanish and that should never be a good credential again uh unfortunately that's not the state of the world we live in quite quite yet I think we'll get there but then the bottom one code review limitations this fits hand in hand with uh some reporting or some investigation we did at getg guardian we asked uh 507 it professionals decision makers what are you doing about the problem secret sprawl 27% said oh we're doing manual code reviews to deal with that which leads to uh some other interesting data later but we're playing

this cat and mouse game constantly we're constantly having to rethink the way we do authentication the way we do authorization the way we do all of these things because attackers keep figuring out new ways in except they're not new ways in they've been behaving the same way for a couple Generations now and so it's so predictable we wrote this down and this is the minor or the the very very condensed superest high level of it but we know that they behave like this because they keep doing this and there are new playbooks that come out so like new I mentioned lapsis earlier like new F fishing attacks and new ways to do social engineering again thanks to the

world of AI uh but we pretty much follow the same pattern that's get in somehow that's escalate privilege that's steal credentials to move around laterally and then exfiltrate all the data and sell it back to you so really we're talking about this part but really I should expand that over to the initial access as well but for the scope of what I'm talking about cyber deception this is the box where we get them once they're in we need to distract them to do something else and then uh kick them out of the system because attackers they want these things yeah you can tie down and spend your time literally securing everything in your entire organization

to n's degree but really follow the path of what they're actually looking for data machine resources anything leads to those and that's the pattern we keep seeing over and over and over again anybody know why they want your data search with an N ransomware yay uh boo depending on which side of the fence you're on on that um yeah they want to sell you your data back they want to sell you your data anyone else that will buy it on the market and they want your machine resources why mine well not just Bitcoin all the other coin Monero and whatnot but to mind that and to sell it how many people have ever bought uh machine time on the

dark web yeah I know this guy was going to raise his hand um did you pay full price no you pay a super discounted price because you're buying stolen goods and people making hundreds of thousands of dollars a day doing the stolen good acquisition so we want your machine resources to either use it themselves I want your data to sell it so how bad is this problem that okay what what what gets you into this stuff uh sorry I skipped a step what gets you into this stuff um well that's where credentials come in you can find the credentials just laying around it's really easy just get in that database it's really easy just to get to that data source get to

those files uh how do you access a machine you use the token to get into the machine and there you go so how bad is that problem that that persistent problem of hardcoded credentials just laying around well we look at this problem at G guarding it's one of the things we do we look at I think last year the average was 113 commits per second that hit GitHub public we look at every single event there's in fact a fun feed you can look at uh called well api. github.com events don't try to keep up it is ridiculously fast and we only care about the commits that happen things that hit GitHub public and the is public event

which is something was Private now it's public that account for about 17 to 20% of all the secrets we find so anybody here has not raise your hand if you've not read this report okay just the people that raised their hand what throw out some numbers just yell them out we'll do prices right rules how many how many how many credentials do you think we found just last year in public how many 100 thousands I don't think Millions would be an answer they'd accept on the prices right but for sake of time we're going to give it to you I'll be a prize later um it's this many uh it's

12.77% it's a real person who made the commit it wasn't just a bot on GitHub or something uh so that's was like 1.8 million propono emails we sent out last year that actually got delivered so if you're working with Git make sure you're using an email that's reachable doesn't need to be your work email it could be any email you want but make sure you can at least get alerts to it because there are people in the world trying to do good trying to tell you hey this commit contains dangerous things you should probably fix it and if we can't reach you we can't reach you anyway this is a growing problem not a shrinking problem

we these are numbers from previous reports 10 million the year before uh not cumulative this isn't like we keep finding and adding to the stack uh we have documented over our fingerprinted over well over 20 million hardcoded credentials out on GitHub fingerprinting we take a hash of it then we take a fingerprint of it fingerprint doesn't go to anything but we can identify everything else off that fingerprint um that's a lot that's a lot and it's a growing problem not a shrinking problem well that sets the stage why am I saying all these random facts to you about how many secrets are out there in public and why attacker path matters like what since we know the attack path

we know what they're actually going to be looking for we know exactly what they're going to do there are very few attackers out there who are completely making it up as they go along from first principles those people exists I'm sure they do but that's not who we're reading about most of these ta that we're reading about are people that are following a Playbook they're following something they downloaded here's a script run at one two three and if you don't get money at step three find a new Target and go one two three so our goal really needs to be to waste their time on stuff they can't actually use that gives you alerts that

they're trying this stuff so you can react in a quick way that's the whole idea of cyber deception this is not a new idea in fact I would argue we've been doing this for thousands of years at this point uh sure maybe the Cyber part it's out of here but at the time that horse is pretty cool technology uh you got to remember all all cyber is just a fancier way of saying advanced technology um yeah we still live with this problem today how many people have ever been infected with a trojan virus on any machine they've ever had in their life some people never ran Kaza I guess or lime wire uh everyone else we had

this problem uh I blew up a machine once my first time I ever had to reinstall Windows scared the heck out of me then it was fun um but yeah we've been doing this so giv some something that they weren't expecting and a package where they thought it was something else this has been used against us but we know how this this methodology Works in fact I just read a report uh uh using AI theyve been able to like recover $300 million uh this is in bleeping computer this week and stolen funds because they found the bank accounts where all of these funds were going to because they strung the uh attacker off uh strung the

attacker along enough with AI Bots to eventually reveal enough revealing data they could actually act on it a really cool story I'm bleeping computer earlier this week uh this is the I think Kow kid part two strategy um they can think your shoulders out of joint and when in reality it's not you can swing up and knock him out uh maybe that was one of the Rockies anyway basically the same film um if you're weak where you're strong and strong where you're weak sunzo said it we don't know if sunzu was real we don't know if sunzu actually wrote it down or said it but there's this book that says he did so we'll just

say it's true but it's a good Tech technique look over here Meanwhile we're going to hit you from the other side in fact this is such a good strategy we did it in World War II uh everybody know this story anybody not know this story so I love this story it's one of my favorite stories out of World War II I'm not a huge War historian person but in 40 1942 the us mobilizes and we're getting involved in World War II we don't have enough tanks and bombs and planes yet those take a while to build they just do so we went to Hollywood and said hey Hollywood can you help us with this can we do something together they

like well we can build you things that look like tanks and planes and we have sound stages we can make you any sound you want and I said cool let's do that and that's how we got into North Africa we shipped a bunch of balloons over and from a distance they look good enough just enough to confuse raml and his people and said okay well let's go defend where they're Landing all this stuff meanwhile We snuck all the real stuff into another part of Africa and took it over this also happened on D-Day there's a bunch of balloon uh Bloom Battalion I think is what they're called there's a whole documentary about this just look up ghost Army wonderful

documentaries out there in the world including one made by the uh World War II Museum and down in New Orleans then we get to this turny point in history I am not a huge believer in the great man theory of History I think it's com a accumulation of a lot of pieces make these connections at certain times in history but this is one of those we can point to it's like yeah this is where the term Honeypot actually comes from uh I'm rereading uh cucko egg right now it's a great read but if you don't have time to read a book uh one make time but two there is a beautiful Nova documentary from 1990

called the KGB the computer and me that's a screenshot from YouTube it's free on YouTube and it stars the actual people who were involved in it like that's actually Clifford stole uh story goes like this he gets hired as a lab technician as an unemployed astronomer in Berkeley uh second day on the job there's like hey these these computers cost $300 an hour to rent 75 cents is missing and none accounted for um figure that out so it calls the FBI someone sto on their money the FBI hangs up on him because it's like 75 cents you're wasting everyone's time then he realizes through a bunch of research and it's a really interesting story um I'm not

getting into the meat of it but uh or the the all the details but um realizes this person's also getting into military installations like where nuclear weapons are and the Air Force is like hey this is actually a problem the NSA is like this is a real problem uh CIA is like yeah we should probably do something about this but the problem is they can't keep the attacker on the phone long enough to trace them all the way to back to where they they're going through a million hops to do this and it comes down to this uh telephone exchange in West Germany in 1985 it's so old of a operation that the only way to tell if someone's on the

line is to go through with an oscilloscope and literally test every connection in the room so how do they get someone to stay on the line well his girlfriend Martha said just put a bunch of fake data on a server like he doesn't know it's fake and we're talking about 12200 speed uh 1 1200 B modems um it's about 120 characters a second um just flood it with fake information and IT project showerhead because he thought of it in the shower like what how he's actually going to pull it off pull scripts conspiracy theories like just copy paste entire bbs's into these well-named folders and they just wait and sure enough they were able to track this guy back West Germany

turns out he'd been selling secrets to the KGB uh I only got two years in prison for it but there's another player in it that end up burning to death and then driving himself back to town under suspicious circumstances it's a really interesting story but part of it what he called this problem or what he called the solution was a Honeypot because it's sticky it's something that you want we think it's in there and like when the PO you get your head stuck in the Honey Jar honey pot well this idea spreads it's not just hey this is one really cool use case other people like well people are attacking our networks maybe we should do something about it and the

guy named Fred Cohen writes down the the detection our cyber deception all right just at that time the deception toolkit uh honey pots to TDK uh it's very technical I would not recommend reading this on a day where you're looking for Light reading but it's fascinating basically says that's and it's kind of Overboard in my opinion let's turn every system that's not currently in use into a trap you can do this you can still figure out how to implement this framework today but fast forward a few years uh there guy Nam Alfred eer come along and creates this thing called uh ballista ballista gets bought turns into C cybercop sting um shout out to Jeff

man who was the one that tolds me about ballista in the first place because I had never heard of it before he gave a talk at show meon and alt setcon this year uh Atlantic SEC security conference U but anyway Alfred yuger says this phrase at one point in his career to related to cybercop sting and I think it's one of those other moments where we can point to in the history of computer science and say yeah this is kind of that turning point it's no longer hackers who are used to be phone freaks who were just looking for free long distance so they could connect to other computers or MIT kids who built uh rebuilt cars on top of

roofs or hooked fire hydrant up to a uh drinking fountain which you can actually go see still on MIT campus not the car but the the fire hydrant um but hacking was this idea of like let's do something mischievous and fun and learning let's take it apart put back together see how it worked uh and this is kind of the era where it's like there always were the bad element but now hacking takes on the connotation of these are people that are financially motivated these are crimes that are happening so cyber cops sting makes a lot of sense to call it that fast forward a little bit uh and we get to austo honeypots have not gone

away they're only getting more popular there are all sorts of message boards out there and all sorts of um uh email threads going on about this that are all archived and he says one day he like well honey tokens they're a little heavy basically what if we just had this piece of information that shouldn't flow I've been calling these honey tokens and this whole subdivision of honeypots is born if anybody knows a Gusto let me know I would love to meet this guy he lives in Canada um but that sets off this entire other field of of thinking about Honeypot or Honeypot and honey token in parallel honey tokens I think get the modern definition like how I think most people

interpret honey token really comes from the way think does it inside Canary token Canary tokens are literally here's a little piece of information that shouldn't be touched uh and 19 or in 2015 they created the open source repo for their product and in 2016 someone added actually Nick added uh here are how you do AWS tokens with our system so now for the first time we've created a fake credential what I think of is a honey token and the way most people think of them I talk to today think of them a fake token for a system that if I try to use it it sets off alarm bells and doesn't give me access then we go to last year at RSA I

missed Kevin mandia this year but last year I only went and saw him honestly if I'm being real because Eric Idol was right after him and I wanted to get a good seat for Eric Idol for miny python I just saw John C this week as well so now I'm like one left to see that's still alive um honey tokens are your early warning signs he says this from stage and I happen to snap this picture and that's what it says right the very bottom there hunting tokens account bait which I think dials it back a little bit to the earlier of like just something that shouldn't flow not just these tokens but now we're starting to

get philosophical so what is a honey token well we just walked through how we got here and said that piece of data that shouldn't flow but what does it mean in the Modern Blue Team the modern Enterprise K guardian we we said this we've had to write something down and this is what we wrote down honey tokens are decoy credentials that don't allow access to any resources instead they trigger alerts that reveal the IP address of the user who attempted to use the honey token that's at a minimum and then honey tokens look identical to real credentials that's super important there's a couple points in history where they didn't and then it was very obvious and then nobody fell

for it and then it didn't work but onto how to build these things this is the most technical diagram I have in the whole thing but this is g g Canary our open source tool to build huny tokens it uses terraform and AWS because what you build is a real AWS token for an account that has zero privileges whatsoever if you go to deploy this I would recommend spending up a new AWS account just for these things just in just in just in case because they don't go to anything there's no credit card attached there's nothing they can do even if they they do get their hands on these things and then let's store that list of users in a list

uh and go build a little Lambda function and all it does is look for this S3 bucket that how Cloud flare will dump into anytime someone tries to use one of those credentials Jiggles the bucket tells the Lambda function hey time to send an email or an alert somehow a web hook something out to the public out of the box you can do slack AWS SCS and srid your mileage may vary it's open source is 2 years old um and we have not updated it since it's a warning but the basic idea of how to build all of this stuff and all those Source um code is open sourced I really want to stress though

that this is one interpretation of this idea this is one way to go about it it's the one that I'm going to lean into for the rest of this talk for the most part but I think it's important to keep that in mind for this next section because you can go build these yourself or you can just get them off the shelf if you want to go deploy these there's a lot of good reasons to deploy them which I'll go through later but the first is that the DIY you can just go build this on first principles but you can also do other thing off that first principle idea of data that shouldn't move why not have a password. text that

has a file Watcher on it that if anyone ever opens that file that's full of garbage you get to know their IP address and other information about them that's technically a haunting token what about fake email accounts as mandia said uh account bait just fake email accounts that if anybody ever messages that account something's gone wrong uh or somebody ever tries to access that account something's gone horribly wrong you can hide honey tokens inside of those accounts it's honey tokens all the way down at some points but get back on track to build this idea of a fake AWS token or similar um system token that is real but doesn't give any access but gives alarm Bells uh well you can do G

Canary like I talked about space huge shout out to them this talk wouldn't exist without this project uh space siren is the inherit how was they called the the descendant of a project from bit bucket called space crab which no longer is maintained uh but space crab which is the coolest name is basically how to build AWS tokens on AWS counts like the same way we do with GG Canary uh there was a problem with their original way they did it with enumeration it was pretty easy to tell if it was a honey token or not based on Public Information and that's why bit buckets like we're just not going to deal with this anymore and then space siren came

along and said yeah we'll just do this we we'll figure it out and they sorted it out so great great code to look through uh requires AWS noow and then Canary tokens itself if you can spin up a Docker container you can run it the problem with all of these like every other DIY thing in the universe that needs to perpetually keep running if it's going to have value for you is that you need to keep it perpetually running and there's the overhead of that and that's why there are commercial options off the-shelf options I should say um the first and this is where I send everybody if you are brand new to the concept of Honey token you've never

heard of this and you just want to play around with it go to Canary to.org for free oneof honey tokens if you want to like throw one of these out there and just see what happens I would start there because it's free it's awesome they're a really cool company uh they build basically honey pot as a service is their entire business model so it's not just honey tokens uh even though they called it Canary tokens it's fake um uh databases fake Excel spreadsheets fake emails fake PDFs uh anything that like sets off these alarms and say hey someone's tried to use this uh they even introduced credit cards not too long ago one of the

newest ones they have I don't think it's on the commercial I don't think it's free on the free list right now but it's coming it's in the canary tools they're commercial version is um CSS tokens uh which is a really cool idea because if someone's going to make a fake login page for you what are they going to do they're going to go to your real login page and scrape it and recreate that do everyone going to stop to think hey there might be a CSS token in here that if it appears anywhere else on the internet we'll set off an alarm Bell and let the person who created this site know that someone else has cloned

this of course not that's ridiculous who would do that well thanks thought that was a good idea um I love it I love that idea I just found out about that back at RSA anyhow I'm not I swear I don't work for for thanks um they have uh Canary tokens Canary tools which is their commercial version so if you want to scale up you want to throw 10,000 honey tokens out into the world and various forms and shapes they sell that again really cool company out of South Africa uh we have a honey token module uh my boss loves me talking about this part because uh it fits in with the rest of our platform so yeah it is that

implementation of GG Canary but we also cross reference it with all the other stuff we know about OS such as do you have actual hardcoded secrets in here um so if you have like 100 repos suddenly five honey tokens go off because you put honey tokens and everything you know exactly what repos were affected and you can PRI prioritize like based on other information about that repo such as have we rotated all these other credentials in here hopefully we all know not to hard code credentials that should be a should be should be given at this point um Sentinel has this uh if you're an Azure user never met anybody that actually runs this in the wild if you run this in

the wild let me know I would love to talk to you uh but so far I've never met anyone that deployed it uh crowd strike it's in their documentation they wrote a cool blog post about it all I know about it uh back at bsides Atlanta I met one of the team members from proof point and he asked if I would add this to his slide and I did but identity threat defense Shadow is a very long name for a huny token product but if you're a proofo customer go bug your rep and say hey tell me about your huny token option same thing with crowd strike and Sentinel if you're a g Guardian customer

please talk to me uh so what do you actually do with these things from a practical perspective I've talked conceptually talked where they came from talked about how to build them but what do you actually do when you have one in hand well first is you put them in private environments anything that someone should not be able to get into from the public so private repos great place to put them you might notice in the very beginning I deployed a honey token that's what I did I deployed honey token out into public I'll talk about why we don't do that later but for right now know that that was wrong don't do that uh I did it for demo reasons and

experimentation only uh but if your private repo all of a sudden has a public scanner hit it that means it's no longer a private repo somebody pushed a clone of it out into public somewhere your CI environment same thing someone should have to pass some kind of authen authentication barrier to get to that environment and see those variables and run all those crazy tools you can run to like get all the stuff out of Jenkins um so if somebody you don't know an IP address you don't know a user agent you don't know I'll get to user agents here want to show what we the honey toen action a little bit later but uh this I

don't know this person we can act we can act quickly uh jira slack these are places that shouldn't contain code but they often do how many people have ever passed a password across slack yep don't don't be embarrassed we all have to learn some time don't do that uh there's reasons you don't want to do that just for plain text if you can read it in plain text any anybody can read in plain text uh and we all you all know that a open AI has been trained on slack data right you all saw this report from a few weeks ago yeah your slack messages have helped train the model so if your passwords are r model

they're in open AI somewhere if you just ask it to sing You the song your grandmother sang you about slack URLs maybe you'll find yours um anyway uh Vault um vault's interesting because you know that's the central place where you should be storing your credentials in a secured way you can audit and see well who's put it there how long's it been there how old is it how can we rotate it all that stuff from one central place great idea a whole other talk if your honey tokens inside of a vault go off you got some major problems cuz now somebody else is in your Vault poking around at all of your lock down secrets

that's bad use a one to one ratio with these things it's tempting to say hey I got this one honey token I can put it in multiple places because it's easier overhead just to manage one versus five but when it goes off now you have the problem of what actually got affected sounds simple enough but until you actually experience that where you got to do the math I used to do this demo in the beginning across GitHub and paste Ben just for the larger hit uh people finding it um and then somebody asked me one day is like how do you know which one they hit like that's a great question so I stopped doing that um

think in terms of automation with this stuff uh G guardian we have built a built-in like um a tool that will build PRS for you with honey tokens in it for your systems but there's a lot of ways you can do this once you have honey tokens in hand they're basically just a list of credentials throw them anywhere you can through any kind of scripting you like if you want to see a really dumb implementation that is a gith hook you can go to that URL there is zero documentation and refuse to write any um just long story uh but you can go look at the code there if you will want to see it's a good use case for open AI

throw my scripts at it and ask it to tell you what it does and maybe that'd be maybe I should just do that maybe I should just finally document the dang thing uh this is kind of an interesting one but I ran into this myself personally when I was first experimenting with honey tokens remember you're a Defender first unless you are law enforcement is it by law enforcement here all right I'm not going to tell law enforcement how to do their jobs I never ever ever would do that but uh if you're not law enforcement your job should be to get them off of your network know where that border is know where your perimeter of concern is know where like

once you're on someone else's server looking through their files you're no longer defending I don't think there's cases where maybe that's true but just in in general you're GNA find out some really cool information just know that you should be kicking people out with that information not not going after them again unless that's your job this is a whole journey like everything else in computer science I met one person uh who came up to me and said thank you for that slide because I was literally ready to go deploy these on everything and that would have been like all I focused on for the next couple days if you do that you're going to burn out you're going to forget did

it and then you're not going to get any value from this whatsoever start with one repo start with one channel and see how it goes this experiment with one honey token over the next couple weeks next month if you put one honey token in one place per week you'll have over 50 of them deployed at the end of the year and you'll be that much more secure because what you're really buying with honey tokens and I don't have a slide specifically on this is you're buying time I mean that a couple different ways one you are buying time to shrink your dwell time because if someone comes in your system and sign trips over a flag and

says or trips over a honey token and gives you an alert within a minute of them being in the system well now you know they're in the system and the dwell time is going to be a minute versus the mean dwell time right now is like 14 days for non- ransomware and 17 days for I no 14 days for ransomware 17 days for non- ransomware according to sofos from last year so it might be a little shorter now but thank you uh but we're not talking um years anymore thank goodness in the industry it's no longer the average that you have a dwell time over 300 days which was the case at literally a decade

ago we're getting better but still two weeks is a lot of time to dwell do damage exfiltrate versus 5 minutes so you're buying yourself time that way other is if you're tackling these larger problems of like I know this if anybody got their hands on this repo we would be hosed and we got all these things got to fix in it all these hardcoded credentials all of these uh misconfigurations all of this other stuff that we need to worry about let me just throw a honey token in there right now up at the front and then I know if anyone's got it into it or not buy just a little bit of breathing room so time to fix it while you're

still being defending lastly what not to do I only have one do not and that's don't put them in public and I know I did earlier but the internet is constantly being scanned by good guys and by bad guys and by everyone in between uh I don't know how to count AWS in the good or bad situation so they just know that there's constantly scanning the Internet it's GitHub gitlab get Guardian uh tons of tools out there just constantly looking for new things new things to exploit um gitlab and AWS or not gab GitHub and AWS actually have a partnership and GitHub is doing awesome work these days on keeping people from committing certain types of API keys if it's

prefixed GitHub might be able to warn you that you hey you're trying to commit this to public but if you push it to a private repo and then make that private repo public it gets around that whole system and if you're using private it doesn't stop you anyway um if you put it in public though they're going to get scanned and that's how you know that something private has gone public so those are the two big use cases someone's got into something they shouldn't have had access to or this thing that should be private now is public and we got to deal with that either they either either either Situation's not good uh and something you should probably be

working on so let's go see that honey token from earlier and it has been holy crap uh it has been hit 92 times in the last 38 minutes uh about what I expected uh where are you Netherlands why another one's hitting me from AWS truffle hog expected that that's somewhere in America um that's get Guardian go all the way back to the beginning uh yep there's the first one I expected they do us internal we're not showing the IP address because they use some weird IPv6 address that's it looks better if we just say AWS internal uh but the reason reason they hit first is because get Hub immediately told Amazon hey someone put this here Amazon immediately did

something called attach user policy the specific policy they attached is a security policy that quarantined you so that email you got that said hey you did this on AWS don't do that we quarantined your account another good reason if you're going to go build these on your own build them in systems outside of your normal AWS account uh because these are get quarantined if they get popped you just don't want to deal with that whole it's connected to my main root user uh see truffle hogs in there someone on Amazon using B 3 good old bot 3 um oh yeah so what am I looking at here this is the uh G Guardian interface for

honey tokens I would show you someone else's system but I actually work for this company so I get free access uh that's why I'm not showing you st crowd strikes for instance um we're using cloud Trail and cloud trail can give us the IP address we're putting the little Flags beside it Go location and user agent and the action they called user agents super important because I can get a new IP address like that I have no idea how to get a new user agent without more work and again attackers aren't coming in working from first principles some maybe Advanced nation states are maybe they're after your billions of dollars in Bitcoin you have stashed somewhere I

don't know you but most of them are playing following a Playbook and as soon as they get kicked out oh I get a new IP address hey got kicked out again why they get kicked out again because they did the exact same thing from the exact same user agent over and over again Sumo logic I think even introduced blocking Auto auto blocking by user agent um a couple releases ago uh yeah so I'm just gonna clean this up I'm just going to revoke this thing real quick just to stop the noise 92 events though that's that might be a record um so in conclusion we know how they're going to attack it's always going to modify

somewhat here and there but we can keep sharpening the axe we can keep pushing back on them and one of the easiest ways to do that is just lay some traps since we know exactly where they're going to swing this is a really easy trap to lay either you build it yourself you buy it but set some false credentials out there that give you information about what they were trying to do who they are go build it yourself again think outside the box in terms of I can just put this here and if anyone ever touches it that gives me an alert that technically falls under the definition of Honey token data that shouldn't flow uh or just go use caner

to.org um can't say enough good things about them and if you're a guardian customer we would be Lov to have that conversation with you on it lastly again you're a blue teamer blue team I'm Dwayne I live in Chicago uh check out the security repo podcast we have awesome great guests besides Jason Jason East Street other people not named Jason Tanya Jenka uh that always gets a laugh um hit me up about literally anything outside of tech and with that I will uh open up for any questions like like last two minutes I [Applause] got were there any questions are oh and if you did not get stickers I have more stickers come up and see

me um for on premise environments what kind of strategies or or tools can you suggest for laying these types of traps and detecting uh when when these honey tokens get well that that goes back to the the architecture diagram and the idea of thinking in first principles with it so if you have an internal um um assuming you have written your own API internally or have at least known systems like they contain these kinds of strings so that's where the idea of like setting out file Watchers on a password.txt file that would solve that because one of the first things you do with living off the land is you go look for things that like that um go look for

desktop extensions that look like they would take you to a password and whatnot you got a comment Jason or you got a question oh um but yeah just the further thought is thinking those terms of um data that shouldn't move and setting up file Watchers that will provide some kind of information about who touched that file there are multiple systems on Windows and Linux that are pre-built to do that I don't know if they are off top of my head though but I have researched that bit uh so is there any like kind of like just off the T off your head kind of metrics as to how effective honey tokens are versus Honey pots these

days and has anyone gotten into like honey processes because a lot of modern ATT actors use like living land techniques where they'll use window apis or install software or anything like that really interesting question uh short answer is I don't know um I don't know how most of the industry would draw that that line between all three in a way that they would report on that um if anybody has that data it's thanks because they build entire honey pot systems versus Honey tokens it's not something that I have yeah I just I simply don't know the answer to that that's a good question though anybody knows let me know afterward let's go research that together I'm

going to ask your chat about this ask it right now all right um yeah I have a question about security researchers so when you put this honey PS you know you'll get some security researchers through bu Crow to hacker one uh how do you deal with them do you have to pay B yeah with our I can answer for our specific system it's called um IPL allow listing uh you ask them for like hey what are the I mean security researchers aren't going to tell you but uh as you identify them you simply say hey this IP range is obviously these researchers uh QA departments we allow that and with the tagging um so like what you wouldn't see

is like my home network wouldn't even pop up because it's allow listed so if I'm testing it from home I go through Amazon or some other machine somewhere off through another Network um that's how I personally do it but yeah you can totally account for that uh how you do that inside of Open Source that's going to be slightly different but that's there's ways to do that yeah you allow listening the answer but we'd have to know that range that they're coming from um but then again you can tie it in with a user agent so if it's this user agent in this range they're fine and yeah they will get popped occasionally from people just poking

around but then do they have a reason to poke around if the answer is yes then you reset the token you go back to go back to do your work this will be our last question um I'm actually interested in kind of that middle ground between true attackers and you people who genuinely have a reason internally to be poking around with that um how do you address say large Enterprises where you have like an inner Source type code base where you can say hey this is an education opportunity for a user do you do you have some way to say hey I see that Dwayne hit that honey pot or honey token let's talk to Dwayne about uh not

using random credentials he finds um yes that actually is one of the use cases but how you narrow it down inside your own network like all we can get back is what cloud trail can give us at least for ours so if you can narrow it down by IP address and user agent that way yes um but also yeah there should just be a larger conversation like why are you trying random things you find again if you're QA if you are a security team yeah okay I it totally makes sense but why is it someone internally popping this off and then you can start that investigation that's uh no I really haven't had that conversation too much

before about the education side of this um I think for that just the awareness training like this talk this Talk's available online as well not just from besides Boulder this is the most updated version but just passing around your team like hey know that we are just even telling the team like we put honey tots out there uh don't be trying those credentials unless you have a reason to do it there's no legit reason to test a Honeypot unless again you're testing it as a security person but anyway cool hopefully that answers everybody's questions thank you very much I'll be around so thank Let's uh let's thank the speaker [Applause]