Digital Intel Gathering: OSINT for Blue/Red Teams

good morning so everybody awake you made it day two for those that were out late there's also coffee and drinks downstairs when you came in to get feedback on any individual talk you can click on them on the website or to give general feedback you go to slash feedback when we like to thank the sponsors this morning for allowing us to have this wonderful conference here first up this morning we have ethan dodge he's a security engineer with specialization and Incident Response enjoys research projects and as an aspiring malware analyst Ethan thank you for that introduction hi guys doing so how many of you guys were here yesterday okay I thought it would be like a lot

less than people were just came today for an excuse to get out of work but seems to not be the case how many of you guys are going to RSA I'm not so i won't where's my hand I'm say I'm actually taking off on vacation for a little bit so um quick shout so I did not leave this on my personal page right here because I'm totally egotistical but the yellow and grey really mess with your eyes so sorry guys i don't know i get my coworker will give a talk with the same color scheme yesterday it was fine so I don't know what's what's the difference here anyway I'm Ethan Brian where time was

originally supposed to give this talk with me he's a good friend of mine a couple things came up and he wasn't able to make it please keep in mind that he participated in all this research with me him and I work very closely in several capacities and uh yeah he follow him on Twitter it's a believe its app Brian underscore where I'm so anyway so I work at a company called nuna health we are start up here in San Francisco we're just a few blocks away actually and i actually just barely relocated here for this position loving San Francisco so far only been here for a month though I'm originally in from Utah and I'm I'm really excited that I got a

couple of fellow you tones here in the room that came out for RS a shout out to Metta cortex insults on thanks for being here guys hmm all right so enough about me let's talk a little bit about Nina health where we work with government and self-insured I'm going to take this off the stand here we work with government and self-insured employers to understand and improve how people use health care that's the really vague term exactly what we do and it's what PR wants us to say because it's on the website in short weird we do data warehousing and data analytics for healthcare pretty much sums it up security is the foundation of our culture and our products and as a

result we like to be very transparent so if you have any questions at all about nuna our security whatever I'd be more than happy to ask them we got a couple other guys here that work here and they will that work in una and you could ask them as well we're also currently accepting resumes not necessarily hiring at this moment but probably will be in the month or two so I'd be more than happy to take a look at your resume and submit it um if you're interested come hit me up after all right sorry yellow and grey go off that this is it's a shame because it really is actually i think the complement each other very well when

it's not freaking out on the screen all right you get how many of you guys are familiar with the term o cent or OS int i personally don't like calling it OSN so i'm going to call o cent okay all right who can tell me what Oh sint stands for open source intelligence what is open source intelligence without looking at the screen right there all right yeah information available from publicly information from publicly available sources such as social networks public data records such as your property records or you're in a lot of states your case records like every time you get pulled over that is public and I'll some states have varying information that they share with the

public for instance i believe it's i'll get into those little bit later but i believe North Dakota you can actually see the last for the person social so that's fun if you're from North Dakota write your Congressman and get that fixed because it's kind of shameful anyway in leaked customer data there's also two there's a ton of other sources but the most common form of oh since I believe we do it on a day to day basis is on facebook who here has leveraged Oh since on facebook for an ex-girlfriend ex-boyfriend but you know maybe you're a little bit like this girl you know stocking you on on Google Earth yeah how many of you guys have had a significant

other like this all right uh you could tell details later all right y ou since okay why do we why do we leverage ossa a lot of people when they think of Oh sent they often think of intelligence company as intelligence agencies and companies such as like the NSA FBI CIA or threat stream crap strike those type of companies threat stream I believe is one of our sponsors anyway but private investigators detectives lawyers investigative journalists CREP Brian Krebs is a really good example of this he he's a I'm sure everyone here is familiar with his work he's an investigative journalist and he uses a lot of Oh sent criminal activity in law enforcement and also threat intelligence

right a lot of people have classified Oh sent at stalking it's not stocking it's called being an Internet detective guys okay but really it is kind of stock ish let's be honest so disclaimer so I am in no form there actually is a big difference between gathering open source intelligence and stocking somebody and I am in no form advocating stocking somebody because that is illegal so if you guys want to get into the differences we can later but a general rule of thumb is stocking actually Intel's physically following the person not virtually following the person like I'm Twitter otherwise we would all be screwed all right basic workflow for open source intelligence this is also

good excuse me this is a this is a general workflow for any threat intelligence got identify the source identify possible sources of Intel what are good some good sources of Intel for open source intelligence any ideas linkedin why do you say linkedin you could find so much stuff on linkedin are you penetration tester by chance okay awesome how many of you guys are penetration testers and abuse linkedin in a previous engagements okay thought there'd be a little more but there were let the record show that there were 100 hands that went up anyway twitter is one of my favorites instagram because of those careless people that well okay sorry sorry that was a generalization and very mean they're not careless some

people deliberately put the g enable geo-tagging on their on their posts and a lot of them have legitimate reasons I personally don't like doing that because it scares the [ __ ] out of me so anyway then you got to validate you gotta validate you make sure that it's not like a fake Twitter profile or it's not a fake linkedin profile or it's not it just make sure it's legitimate right and then then you got it then you got to automate it and we're gonna go into a little bit of automation if you're doing all this manually could get very tiresome the next step is to analyze so you could get a wealth of information

from LinkedIn and from github and from Twitter and Facebook and property records and tax Rick whoa tax records aren't public but anyway you get a wealth of information excuse me but doesn't really all apply right there's actually from all the stuff that you can get a very small percentage of what you're actually going to collect is going to apply to your target then determine the probability of it being useful to you while it may if it applies at the target is really going to use be useful little ply confidence right like I know for sure this person works at this company right or I'm actually really like only fifty percent shore or I'm sure I am one hundred percent

positive that this is my employ this is one of my co-workers and they're tweeting sensitive stuff about my proprietary code or or our trade secrets or something like that right and then from there you could usually generate new potential sources if they link to something or or it usually send it it's usually a very long process and you could be as thorough or as not thorough as you as you want to be you really could spend weeks just diving into into information then you enrich she had yet context probability and confidence level details and then you develop a narrative this person was here on this date and then they went did this and based on the

geotag from this instagram photo they were a part of this crime right that's the that's the typical law enforcement or if you're if you're involved in a corporate espionage case it also might be useful to look at the geo tags of people's tweets that might be involved in that I don't know the possibilities are endless right how many of you guys have ever used Mountie go you can't really see that melty go an email to go fans nice thank you that's like half the room I love Matty go for those of you okay for the how many of you guys have heard of mal to go okay okay so I'd say just a small percentage of people in

this room but we're going to go over exactly what it is just really quick so it's a link it's a it's a visualization tool for open source intelligence you can take an email or an inch or a person's name or a social security number or or a twitter handle or whatever you want and you could run what are call transforms on them wonder which are basically just scripts right in any language you want excuse me and and then excuse me I'm very sorry then you will slowly get a web of entities and relationships to your investigation some common terms I've already said entity entities transforms and machine machine so what an entity is is these these

three think these four things sorry right here are considered entities in melty go you'll see right here reddit com then you then you got their name servers right there those four things are considered entities then transforms is the thing that I that which is this I like to think it transforms the scripts and it's this transform that I actually ran to look up reddit's DNS servers thank you very much appreciate and then machines are running several transforms together to develop a big web and result how many of you guys have ever developed your own transforms for multi go we've got a couple whew okay it's very very easy pretty much you write this the script in Perl

Python whatever language you want and then you throw an extra few lines of code in yeah from montego chances python that's my personal favorite from melty go transform trans in port star they basically they just make a class or a module or whatever you call it i'm sorry i'm not a python guru i probably messing up the terminology like crazy right now anyway and you just you just import that and then you have to create a multi go transform object and then parse the arguments as system arguments and and away you go this is it's as simple as that and then oh yeah then you gotta call and say you got to call the object

and return the output so that it actually displays on Mount ego so little shameless plug here gavel is a Mountie go transform the Brian and myself we developed just out of curiosity and I'm not expecting many hands to go up how many of you guys have looked at it by chance okay 34 okay I expected fewer so so that's good what what it is it takes up court case records from individual states that's how I know that that's how I new information about North Dakota and what is provided in their court case records some states are a pain and they actually organize and file their court case records by county such as here in California and so you have to develop a

new transform for every single one of those counties so we're still in the middle development we only have Maryland and Delaware and I think maybe another couple states if you guys are interested in helping us with this initiative and getting all states in pulling their court case records into mount si go we would love the help hey go ahead and his up on Twitter oh it's a lot of fun you can get tons of sense of information and there's the link oh I'm sorry what was that oh there's the link github.com / Brian Wareheim / gavel I let him take the credit and put it in his repo sorry one sec I'm just gonna open my water

okay so here's an example you'll see again letting Brian take the credit here you'll see his name Brian Wareheim and we ran this court case we ran this transform against his name and we were able to see that he is associated with those that addresses and those license plates because he is from Maryland and Marilyn does give you the license plate numbers of the time that you were pulled over and whatnot this is dummy data so I'm gonna try and stock Brian I tried to convince and put the real date of it he wouldn't want to me anyway any questions don't be afraid to shout out questions in the middle I really don't care okay I

I apologize I'm a little embarrassed I'm not familiar with okay we have looked at a couple other sources such as now what was the name of the one that we were looking at it was is for private investigators and it had a name that rhymed I yes yep lexisnexis and we looked at a couple other ones yes we do we to be honest with you we kind of put this project on hold just because we had other things come up and we're a little more interested in stuff but we do to plant plan on on developing it further and we would love suggestions like that you'd like to help us reach out to us on

Twitter and we'd love to all right so I got a story for you guys now that you can tell because you can't read that I'm sorry I'm gonna have to go back to our to our creative people here how many of you guys have ever seen this there's a whole twitter account is there several twitter accounts dedicated to retweeting pictures that people post of their credit cards and some of them might be fake a lot of them are real i do believe this is this is scary how many of you seen this i wanted to shout out to danny Howerton meta cortex for this he's a he's done a lot of research on github he's actually speaking of east side salt

lake city in a couple weeks about his research done on github and I stole the screenshot from him so thank you anyway and it how many of you guys have seen this publishing sensitive files such as se shadow etsy password or SSH keys to github yeah that'll keep you up at night am i right man scary all right so here goes my story so um I was contacted by a university in Utah to speak on Oh sent and melty go this was for their IT security club I was going to so they were they were freshman to senior security enthusiasts studying Risa and researching information security at this university and they wanted me to do a

primer for them I thought how cool would it be to doc somebody and use Oh sent to embarrass somebody in that club right and who reveal all this info anyway so I contacted this girl that I knew that was in the club and I asked her following yeah I doubt you guys gonna read that cuz I can barely read it it's heavily censored just so you know just a warning I censored a lot of these to protect her and yeah I censored him so much that you might not be like I could be making this whole thing up but I promised this all really happened anyway so another coworker and i have been asked to speak on melty go on Oh sent

melty go we plan on doing a rather extensive demo and wanted to see how much info we could get on someone using sources openly available on the internet we were wondering if you would be willing to be the victim of the demo and she said yeah it's totally fine she said there's definitely a pasting out there with an old password hash of mine I put that in because that ended up being her undoing so we started with Twitter data because she had something like tens of thousands of tweets and she tweeted an average of like 20 times a day that is not an exaggeration so so it was really really good data and she did geotag a

lot of her tweets so we needed a way to parse through all that so we automated all this via Mountie go we identified validated season and then we're gonna end i'm going to show you all this process here this is the this is the code to the transform that that we use so this is how many you guys have ever messed with twitter's api it's it's fun who I think it's a good you api so this is the function that i wrote well I can't take full credit I if any of you guys are familiar with Justin seats I've taken he's kind of in a mentor of mine and I've taken a lot of ideas from him

and some of this code is his but I have modified a lot of it to my to my needs um excuse me so anyway so Twitter only lets you grab two hundred tweets at a time right for a request I mean so this is just going through grabbing two hundred tweets keeping track of the ID of each tweet so that you could go back and then when you go to this function which calls the previous function that it showed you the if we keep track of the ID then we make sure we're not pulling duplicate tweets right and then it puts it all into a list for us which is nice so we downloaded all of our tweets so they'll

let you grab two hundred tweets per API call but they'll only let you pull the most recent 3,200 tweets of a user so we were able to pull 3,200 tweets so if anybody so that this is just an interesting note so if any of your applications have tweets the of over 3,200 tweets for a person it's because they've been pulling them for a long time in saving them on their own on their own infrastructure and whatnot anyway so we looked at the geotag information we pull out it on a map what do you guys think this is this what this was home this is home so uh excuse me yeah that was pretty easy to tell right

there's like thousands of tweets just right there then we wrote this another part of this transform and this interacts with the Google Maps API how many of you guys a mess with Google Maps API so I cannot take credit for this part of the code at all Brian totally wrote this himself and it's a shame because it is definitely the coolest part of the transform what it does is it takes all the longitude and latitude coordinates that we grab from the tweets and it submits them to submits into Google Maps then Google Maps pops out a street address and what it does is it returns the top five street addresses that the person was tweeting from we

also made so in melty go you can apply confidence levels be depending on how thick the line is if the line connecting the two entities is thicker that that implies that it's higher confidence and so that's just what we're doing right here and again this is heavily censored so you might not believe me but I promise you this is her twitter handle and those are the top five addresses that she was tweeting from I promise you or I could just be trying to look like a badass but anyway then we found this that was a anyway we had a blast going through all her Twitter data and we were able to just a real quick side note

going back i will i will tell you guys that one of these addresses was her home another one was where she spent the most time on campus of her particular school and that's really scary in that we could and we did a map out the fastest route and all the possible routes that google map tells you how to get to those places and literally try and stalker right but that is crossing a line do not cross that line okay and we did not do that and and again we had documented approval from her throughout this whole process then we found this i don't know if you guys can see this and can you guys see

that and tell me what it is it's a pash is an unsalted hash and this is when it comes out to be we found a rainbow table online and we were able to crack it well the rainbow tables live a Kraken that was just a sha-1 unsalted hash and then we were like okay the scope of this we could get in big trouble if we go any further with this so i sent her another email and i said sorry to keep bothering you we just wanted to verify the scope of this engagement how far are we allowed to go we just made a significant advancement and are entering a bit of a gray area do you have any issue with us

attempting log into any of your online accounts then I further said that we aren't going to do anything malicious we're just going to get in take a screenshot prove that we were there and get out um and she said go ahead with the logins I have nothing embarrassing as far as I know hey exactly challenge accepted I like it anyway so have you guys ever heard of no um nope or there's another there's another site that does a similar thing I can't remember the name anyway you pop in a user name right here into the search and then it tells you all the sites that that user name is taken and she had a very unique username and it

was awesome so we went through that entire list there it was much more than is shown on here and we attempted login on probably about half of them because we ran out of time we got access to some stuff it was expired but still very sensitive right got into her reddit account or i'm just logged into my reddit account and blacked out the username you guys will never know anyway so um so yeah I mean this is yet so I'm a lot of you guys are probably thinking okay so this is cool he went and docks this girl but how does this apply to my job at this time right I'm gonna get to that because i really like i really like

finding applications to things and i'm going to go over that but first i'm going to go over exactly everything that we were able to find we were able to get her to her home address we suspected that it was we suspected what it was via twitter but then we were able to apply a hundred percent confidence level because we found it in in NC and her shipping address we're able to find approximate class locations based on where she was tweeting and what we really wanted to do but we never got around to was see we could actually determine her school schedule based on when what time she tweeted from which location we found her password that was that that transform

that i showed earlier where we found that the salt was a transform written by have I been pwned calm they have one that integrates with melty go be able to find her close friends she talks with the most on Twitter and Instagram job history linkedin facebook that's not really a huge deal um I was able to find several IP addresses that she uses regularly based on her reddit login history that's a big deal right especially a that I mean that's a little bit of an application right someone get into the reddit account of an employee and see the office I p and start attacking that I mean there's other ways to get the IP obviously there's no

problem yes so I I'm going to be as vague as popular as possible to protect her but she was involved in a breach of a company that did not assault their passwords I'm sorry what was that yeah um I don't know right she so to her credit get all of these accounts she says that she no longer she says she no longer uses but but yeah no it's definitely best practice to change that password and not use the same password across which she does not do any more now she had changed her ways before all of this anyway good questions any other questions no then we were able to find her birth date and where she got her

hair did we were all sort of able to find some stuff out on our family we got their addresses and their property records we were able to find out how many members brothers and sisters her age all of her brothers and sisters ages and all their names and we could have gone a lot further and done all of this on each of them individually and had a lot of data all right so anyway so let's get into a few use cases for the red team all right you penetration testers out there what would you use this information for social engineering fish all the things so this is we did try to fish her this is the sad part of

the story she didn't click the link she didn't fall for it so it's sad for me because i really wanted her to click on all right but it was a hats off to her man we use what we did we took information that we had learned about her based on her profiles and stuff and through all that into an email and tried to try to get her to click on the link someone also throughout social engineering right only you can prevent social engineering and she did prevent it so that's good so the red the red team cases are used cases are pretty obvious here right yeah you had mentioned earlier that you use LinkedIn data for investigations definitely also

a good source right so let's talk about the use cases for blue team okay and before I go any further i wanna i want to talk about Johnny Cash fans here I'm not a Johnny Cash man i just use the line um but anyway what so this is a very thin line and before you go implementing anything like this you definitely need to consult if you have a chief privacy officer a CSO CTO you need to consult other people you just can't start going and doing this on all your employees because there will be a lot of pushback and this is walking a very fine line of privacy okay so i'm not advocating that you guys violate any

about any other anybody's privacy but however I do believe that conducting Oh sent on some of your employees will definitely help prevent attacks especially social engineering attacks okay all right okay so we could use Twitter to see if there's if anything that they're tweeting about is potentially malicious if they're following competitors that's not really a huge deal right what are they talking with competitors on Twitter you know that might be able to Shady are they talking about your brand right instagram are they posting pictures about with their work badges on it that'll probably that would probably be accidental right I don't think anyone would deliberately well I don't know maybe but I wouldn't deliberately post a

post a picture of my work badge right but I could see myself doing it accidentally right passwords I talked about this in that is there a password or other sensitive information such a network diagram drawn on the whiteboard behind them in the picture that they're not aware of so there should what was the case was it the Trayvon Martin case that that the the attorney took his daughter took a picture on their lunch break and ended up incriminating him in some way and posted on Instagram I'm sorry that was very vague details but that's all I know but something like that right it could get you in a lot of trouble github again uh Danny Howerton

has done a lot of research on this committed sensitive file committed proprietary code how many of you guys have seen this oops I just committed our entire code base to github that's scary that's scary unless you're an open-source company than you do it every day but or committed company info like the directory I've seen that before one time I was I was a participating in a red Red Team Blue team mom CTF i was on the red team and we found one of the blue team's github repos and it had all of their phone numbers right there listen the repo right so there's some there's some really sensitive information and like I said if you want

to find out more on how you could harvest github check out medic or taxes talk in a couple weeks at beside salt lake city I think you've aren't even given the same talk at besides Boise as well right yeah then torque on right so I'm sure you guys can find it um facebook is any of the public activity malicious friends with competitors talking about your brand stuff like that same thing as twitter right brand monitoring this is where I think it could be extremely valuable doing Oh sent monitoring and this isn't necessarily targeting your employees this is more how many of you guys have heard of the term the tool scum blur how many guys have deployed and used

Scambler you I might be wanting to talk to you a little bit efforts in meeting scum blur for those of you who don't know is a tool developed by Netflix and basically you tell it to go search for stuff online and it will come back and tell you if it got any hits right and it's specifically meant to find potentially malicious chatter or leaked data whatever you want it to find it alittle triangle out to find find it correct me if I'm wrong but that's essentially the idea right yeah you can it will be you can monitor your name your brand I think of us nuna health is going to be extremely helpful for us

where we are a healthcare company we have buttloads of data very very sensitive data that I would want to know if someone is talking on the dark web about it right about nuna right I want to know right away so wait you could rate your employees most of the I would imagine that most of this stuff most of the sensitive stuff that they're going to post would be accidental unless you have a disgruntled employee or something right and I'm not telling and I'm not saying that we should sell our employees no you can't have a life on social media right that's not what I'm implying at all I think we should rate them and those that are a little more talkative

online and may they may have a little more potential to leak sensitive data we should we should monitor them a little bit closer right that's essentially what i'm suggesting who's your most active employer-employee monitor them a little closer all right um and then push it to your sim alert and correlate if you could get if you could get intelligence that there's an attack coming on your network or you're being targeted or an employee's being targeted by potentially you can prevent a lot of stuff yeah you can prevent us engineering attack you can if if you have reason to suspect that a employees going to be targeted for fishing you could pull them aside tell them to be

extra careful with their permission maybe even monitor all their incoming email stuff like that right so another shameless plug interrogator we Brian and myself and anyone else that wants to help us we're going to be developing a web application that will automate all of this it will be continuous Oh sent monitoring of your workforce or of any or for your private eye you do I wasn't monitoring of anyone else right you'll be able to visualize the relationships with a graph database much like melty go except in web form so that you could use it a cross-platform and we're hoping to get an initial version out sometime mid-2016 we were hoping to get the beta version released before besides but

unfortunately some other things came up a waiver not able to what real quick touching on what I was saying about Instagram looking for objects and photos Justin seats who I hadn't mentioned previously or sites I'm sorry if I'm butchering your last name he he has a very active blog at automating Oh CENTCOM and he actually talks about finding he he he researches more the geopolitical type stuff about automatically finding weapons but you could also take this article and the practices that he that he applies here and find other things there's a great article i would highly recommend it it says part one and it was released in january part two is yet to come out so

i'm really looking forward to it I'll tweet it out once it does come out so yeah and then just a few recommendations I'm sorry sir question just few recommendations i highly suggest following Jake Justin seats JMS underscore underscore pie on Twitter also the grukk you guys familiar with the grukk he does a lot of most stuff he's is very brilliant automating Oh CENTCOM that's Justin's blog belen Capcom so Bell a quick disclaimer about belling Capcom they don't they're not security professionals they are private their private investigators and they more focus on like geopolitical issues and and and using satellites to detect if armies are moving stuff like that right something's that you see in the movies

but it does give you a very very good idea of what you are able to accomplish and probably give you some ideas on how you could apply it to your job and save your companies but before anything happens right any questions yes yes it is very basic stuff and I'm sure as far as I know and correct me if I'm wrong please there's no formally released tool unless you want to pay big bucks to automate all this right the purpose that we're behind our interrogator application let me plan on developing is to make it more easily accessible to everyone we're going to completely open source the whole thing right gonna accept pull request anyone can get

edited and open to lots of ideas we're super we're huge advocates of open source software so the idea behind interrogators to make it more readily available to everyone and but you're absolutely right it is fairly trivial and simple thing things that we're going to be doing any other questions all right thank you so much guys give me a follow on Twitter hit me up on my blog or ethernet newton com I appreciate it Thank You Ethan they want to meet