Tim Tomes - Recon-ng and Beyond

all the red team and blue team speakers in bze Georgia Tim tones is the only one that slept in my bed last wow K the the worst part about that is he's is he's not lying the best part about that is it wasn't the bed he was in it was just one that he owned all right so that works out okay so uh here we're talking about o for ab ABC Recon NG and Beyond so I think on the program and on the schedule and everything else it was just called Recon engy and beyond that was a lie this is about how it relates to application security which happens to be my current field okay so who am I well

I'm the I'm the managing consultant at a software security company called enisum uh we specialized in any kind of software security whether you're talking about mobile web uh fit clients things of that sort um do a lot of source code analysis do a lot of dynamic testing do a lot of uh hybrid which is a combination of both of them and we also do a lot of sdlc integration type stuff so uh you're looking for software security type work or Consulting and vism is a really really good option and I run the commercial Consulting team there I'd be happy to have your business okay uh I like to consider myself a hacker looking out for users by training

at one point in time it was developers and then at other point in time it's it's you guys and basically what it's come down to recently just anybody that'll listen to me right I just like to talk about it I like to share my passion I got a lot of passion for what I do um and so basically anybody that listens I'm willing to uh to teach love to write code I spent a lot of time writing code been doing it most of my life even though I kind of came up through the hacker ranks um in terms of where I am as a professional um I've always found any excuse I could possibly possible to write code absolutely love

Jesus you can tell by my shirt and then I also typically drive fast get lots of speeding tickets and I talk fast too so that kind of goes hand in hand so caution there are some live demos ahead about 50% of the talk is demos anybody that's seen my talks before knows I do a lot of a lot of the Recon ngy stuff while we're in here uh offensive material is possible I don't think in this one we're going to see too much of that I know in past talks there's been a lot of offensive material I don't condone the stuff that you may see that is offensive however preventing it wouldn't be a whole lot of fun

because I wouldn't be doing demos and demos are obviously the more fun part uh a lot of fun for you when they don't work because then you get to make fun of me and then a lot of fun for me when they do work because then I get to show off some cool stuff um if You' seen me talk before you know that I don't look at Recon NG as an independent project it's not something I work on alone right I maintain the framework I maintain the API all the backend functionality that the framework provides but it's the community of developers creating the modules that really give it life uh and so in this particular talk I'm going to

talk about I'm going to cover some modules uh that were contributed to or developed by the folks that you see on the screen right here um so please give them a follow please give them a follow or a shout out on Twitter if you don't mind uh this talk wouldn't be possible possible if these folks weren't contributing their time and code okay so I've been talking about Recon probably or ENT I should say and Recon NG specifically has involved to ENT for about three years now um and through that time three years ago I was a much different place in a much different place professionally I was doing Network penetration testing and a lot of General Security Consulting and

now I'm solely in application security as a managing consultant and 's come a long way too uh the resources that were available then some of those resources don't exist now and then then there's some really cool resources that exist now that we didn't have before so Recon so the Recon and O in General's come a long way but now that I focus on appsi I take a completely different approach to how I conduct open source intelligence gathering and what I figured was is a lot of the reaction that I get from Recon NG is it's so Network focused it doesn't really apply to my job in appsc I get folks in the appsec field that

come to me and that's exactly what they say is ah I've played with it but it doesn't really apply so much to me because I just deal with application security uh well there's definitely a way to use it in a fashion that helps you there and I figure I'm already doing that I've got a pretty decent methodology I'll put a little talk together and I'll go and I'll start telling people how I apply Recon NG and open source intelligence gathering techniques within the application security realm U and then maybe that'll help out some other folks and uh make it the tool a little bit more useful uh overall so what good is ENT for an

application security assessment if you already have the source code so something like this is is actually really relevant to us at nvisium because we particular we have source code with just about every single assessment that we do uh we are very bug focused and not so much threat focused right we're out looking for every single flaw that we can find and trying to fix it at the root which is the developer right the developer writing secure code as opposed to being threat focused where Recon and Os itself is more of a is more focused on a threat Intel type approach and so this is a common question that I get um and I'll concede one thing right up

front that the darker the assessment and that when I'm talking about that I mean like white box versus Black Box the darker the assessment the more critical ENT or the more useful 's going to be in that particular assessment okay now this is this is not a zero sum game right because we're ultimately after the same thing whether you're doing Dynamic testing or whether you're doing static testing uh because ultimately we want our customers or our clients to make smart educated risk-based decisions and fix bugs right we want them to fix problems and also another thing about that is is most of us in the application security Community are doing Dynamic testing as opposed to just straight

source code analysis even network security firms are doing Dynamic testing web applications so a lot of the stuff that we cover here U are more on the darker side so the information is going to be more helpful but how do we help these companies how do we help our clients make risk-based decisions by showing impact right because impact really is the key to making those risk decisions if the people making those decisions don't fully understand the risk behind the flaw that we're identifying right then they're not going to make the best decision possible so static analysis shouldn't really change how we show risk right we can still show impact through vulnerability through the information we

find in ENT whether we find that vulnerability through Dynamic testing or whether we find that vulnerability through static testing the only difference is if we're doing static code analysis we have the cap we have more information we have the capability to provide a more actionable recommendation right we can provide code patches we can provide a better more educated recommendation on how they can fix their particular architecture to presentent that flaw okay so if somebody doesn't mind would you read that slide for

me ma'am sir I'm so glad you asked because that's what we're going to talk about today okay but no seriously um if you seen my previous talks you know that I approach a lot of a lot of my Recon NG talks in a in very like in a flow because a lot of times in network security you have to have a piece of information before you can grab another piece and then you got to take that piece and go get another one so there's really a flow to it um my particular methodology when I approach an absec is not that way uh it's fairly you can do it in any order that you want so basically what I've done is I've

taken the the way that I do it my process and I've broken it down into categories of information that I'm going to go after so what we'll do is we'll talk about each of these categories we'll talk about what we're looking for we'll talk about why we're looking that for that information and then we're going to talk about uh how to go about getting that information using a lot of the the uh resources that we have today so the first category information we're going to talk about is contacts so what am I looking for well I personally look for usernames and email addresses right and why do I look for those well because certain certain

emails email addresses and US usernames open up certain attack patterns that are very prevalent in application Security today so is as far as like if you see these down here at the bottom username and numeration which is what opens up the capability when we have usernames and email addresses that there's user enumeration there we can leverage that information in combination with these other things such as no anti- automation controls or weak password policy how many people here do absc assessments how often do you see the combination of those three vulnerabilities I see in just about every single application that I test just about every single one there's you may not always find using usame and

numeration and say like a login page but typically somewhere in that application pre-authentication it exists whether it's in a registration page whether it's in a forgot password page or a login page it's a tough problem to solve sometimes and a lot of companies accept the risk involved in that okay so you take usern username enumeration no anti- automation controls which is also quite frequent very rarely do I see anti- automation controls and even when I do quite often they're not done right anti- automation is tough to do right in a safe way and then weak password policies that's something we all deal with passwords are typically always weak I don't think I've ever seen a policy in

the past several years of doing applications to year to Consulting that was had a standard of more than eight characters so even if you have eight characters long with a very large character set you can still crack that relatively easily especially in an offline situation so the combination of those three things if we have usernames and email addresses there's a pretty good chance that we've got a win there another opportunity another TCH pattern is user enumeration Plus account lockout so somebody tell me what that equals if I've got user enumeration I got account lockout what type of attack can I perform on the particular application possibly who do somebody else user level denial service attacks

right all the time I see this all the time and anti Al another thing that and account lockout is one of those anti- automation controls right it's it's if it's not done properly it's anti- automation control done wrong which is seen HSC all the time account lockout is not is hard to do right and if it's not done right if I have a username email address which is a lot of times is a username I can definitely use your level denial service a whole bunch of people now that's not something we always do as testers but it's a good way to show risk because the bad guy very well may do that he may have a reason to conduct

denial service against you okay so how we going to do that well for those of you that don't use Recon NG it's got a pretty neat feature that I've built into it recently um I've got everything all the modules named in paths with dashes in the middle basically the information that you have or the information that you want to get to and if you do a search- contacts it's going to show you all of the modules within the framework that result in contacts if you do say search contacts Dash it will show you all of the modules which take a contact and turn it into something else so it's all about transforming data and this

becomes a really easy way to search for what you're looking for so if we go into Recon n NG and we do search- cont you're going to see a bunch of modules in there now the ones I want to focus on that are new in this particular talk that I haven't had before are the sales Maple module and the mail tester module anybody here ever heard of sales Maple not a single hand and that's pretty much expected because it's a brand new service uh who here has heard a pone list before should see quite a few hands there okay well one of the one of the co-owners of pone lists uh left the company when they got bought out by

another company called info armor a few years back and he went and he he was uh he himself was in the ENT field and he really was bothered by the fact that jigsaw sold out to data.com and we lost that really good information resource that was jigsaw so he's gone and created sales Maple to fill that Gap that jigsaw once filled so now we have a free and open resource where we can go and actually get contact information jigsaw was like we'll give you names for free for companies but we're not going to give you contact information he has said I want open access to all contact information to everybody really really cool service a really good uh I guess uh

um regeneration or you want to call it of Jigsaw so sales Maple is a really good module we're going to go through with that and he has specifically told me that I did this for the security Community right he saw pone list doesn't really cater to the security Community too much in terms of making it usable for the public and so sales Maple one of his goals there was to make it accessible to the security uh community and the mail tester is a module that validates email addresses so it'll go out it'll actually look at the email address to make sure it makes sense it'll look up the MX records toensure that a mail server exists there and then

we'll actually connect to the mail to the to the SMTP server do a a little communication there exercise the SMTP protocol and validate if that email address actually is valid so it's a really good way to now validate all those email addresses we've created using the framework so let's go ahead and see how those look bring this over so my code doesn't get in my way all right so opening up Recon Ng we're already in there okay if you do the search Dash contacts you'll see all the modules that result in contacts a lot of them there should look pretty familiar but let's load sales Maple actually sales Maple uses what uh uses a domain so we've got to add a

domain I'm going to go ahead and use my church's domain um because I've done some testing with them and they actually have a decent amount of information for us to look at so we'll add domain newspring.cc okay now let's load sales Maple and run it boom okay and it dumps out very quickly we get first names last names um and we get some uh some email addresses right so we're off when we're on our way let's validate those email addresses to see if they're legitimate okay so let's load mail tester and we'll go ahead and run it and we'll see that we've got some issues coming back for us right unknown response from the mail server email address did not exist

on the server so not every single smt SMTP server is going to actually give you the information that that email address address exists okay so what the mail tester module will do is if it doesn't exist and it can confirm that it'll remove it otherwise it it'll confirm it and leave it in there for you um it looks like here they've actually rate limited me didn't they fun I a sales Maple actually hasn't email validator as well that I'm going to be migrating this module over to their API uh for the time being this one does have a rate limiter on it but it'll be upgraded probably here in the next couple of weeks all right so where are some other

places how how are some other ways we can uh get to our contacts well we can scrap scrape exf data from files this is something I've wanted to do for a long time or Recon NG but I've just not put in the time or the effort to actually accomplish it um but now I've done it in in a thing called metac crawler and basically what we're talking about is is using Google dorks to find files associated with the domain pull down those files look through the meta dat and see if we can find actual usernames within an organization you know domain usernames um or just general information about the user that created that particular file and so I've got a module

that I've written to do that called metac crawler let's take a look what that looks

like if we look at the options for this real quick you'll see that it has this option to extract so reconnaissance OS typically passive right well if we're actually downloading files that's an active maneuver we're actually downloading files from their servers so immediately R off the bat by default you don't the the extract option is set to false and you'll just see which files are out there okay if you want to actually extract that metadata set that to true and it'll go out there and extract it so we can go ahead and run it first and see what it looks like without it and we'll see that there's quite a few files that they're hosting on on any

one of their web servers on that particular domain and if you'll notice you got resources. new spring.com images. um yeah it looks like those are the only two hosts that are there but we search for the whole domain so you you can you can narrow it down to just one app if you want to or you can look for an entire domain but you can get a lot of information back like this now let's go ahead and set the extract option to true and run it and we'll see here that it's pulling some pretty good information from there right that's sometimes the people who were last Modified by we've got some usernames in there got some information about the

computer name itself which is NewSpring in that particular case so it's a pretty good information coming back when it says resource not a valid file what do you think that means anybody want to guess the file doesn't exist anymore right you end up getting a 404 or something like that um so but in any case pretty neat stuff just added this this week it'll be in the master Branch coming up in the next couple of weeks uh but pretty useful stuff places we can get Conta s or contact

information is it changing up there hello there we go okay next thing I want to talk about is profiles all right so what are we looking for here well social media profiles right I'm lumping everything into social media profiles if you've got something out there that allows you to create a profile that has some sort of public facing information on it I'm just calling it a social media profile I'll lump it all into that one category right and public or private is important public obviously you can get straight to the data no extra work is done but just because somebody has a private profile a typically those are still innumerable you can in numerate private profiles but to get access to it

it's typically as simple as just creating an account and asking them to be their friend right pretty simple stuff um I would say that's in within the scope of a contract because anybody at any point in time can become friends with somebody on Facebook um so you're not I don't think you're breaching a scope there by doing that kind of stuff okay why do we want to do this once again we've got some attack scenarios that this unlocks for us if we have that information if we have user enumeration we got no anti- automation just like we did before but in this case say we have an inband reset password reset process uh how often do we come across inband

password reset processes nowadays for those of you that are doing upset testing it's the quantity is reducing thankfully I think now nowadays I'm probably around 30 40% of the password reset processes that I see are inband uh which is which is still bad but it's better than where we were uh a couple years ago okay quite often what you'll see is what's like the common thing that people use for an inband password reset process it's the common method people use security questions right I got security questions I'm going to ask you what your dog's name is I'm going to ask you where you were born what was the high school mascot right where can you

get that information social profiles right this is stuff that I see quite often I was asked to do an assessment about 3 months ago for for a Fortune 500 company which was actually inside the top 300 um this exact same attack they had a a single sign on Portal that was vulnerable to user enumeration and did not have anti- automation on it and then within their password within their password reset system it was all in banned all they did was ask you some security questions and what I found that is I could just continue to to use this uh user numeration vulnerability through the through the password process to enumerate all of their security questions I just took the security

questions for all the users I knew about started looking through the social media profiles gained access gained access to enough information to reset someone's password once I logged into the SSO system and had access to their main portal they had a built-in system that allowed you to look up the username for every single person within the company once I did that I took that username user that that user's username back to the same vulnerability discovered all of his security questions went and looked at him logged in as him then I have access to an admin's account now it wasn't his administrator account but it was his it was the user account of an administrator but guess where the emails

went when you reset his administrative accounts to something else his basic user accounts email address which I had access to and so using that particular approach I was able to elevate my own privilege within their environment through this vulnerability for those of you that are familiar with some of Rob Fuller and Chris Gates's talk that's low toone folks because if you look at username enumeration anti- Automation and something like an inband password reset process those are all three going to show up probably low sometimes medium on a report right but that got them completely owned so something to think about this information can help with with these types of attack patterns so how are we going to do this um search-

profiles got a bunch of really cool stuff in there um so I used to have this thing called the LinkedIn crawl module which did a really kind of neat thing it would use Bing to find find LinkedIn profiles associated with the company and then it would go to those profiles and if you if those of you that use LinkedIn when you go to your homepage you see that little column on the right that says people also viewed everybody seen that before well it goes to the profiles and it looks at the P people also viewed then it goes to each one of those and it checks which company they work at and then it does the same thing to each one

of those and it keeps crawling through looking at that also all the people within the also viewed column to find all these people that work for a particular company was absolutely Ely incredibly effective in fact it's one of the best contact or profile harvesting what it's it's ultimately a contact harvesting module but one of the best ways to do that so I re I I recommend you to look into it I had the module developer break that into three different pieces so basically now you have the being LinkedIn LinkedIn module that looks and pulls down a bunch of LinkedIn profiles then you have the LinkedIn crawl module that does all of the profile or profile parsing through

those through the also viewed column and then you have the one that actually goes out to each of those profiles pulls back the contact information and populates that contact table so pretty cool stuff anybody here ever heard of the name check the name Check website it's a place you can put in your username it'll tell you all of these different places whether your username has has been used and hasn't been used well we've had that module for a little while within uh Recon and it kind of gives you an idea to go look where people may have profiles if you knew like their handle or in some cases their actual name if that's what they use for you know what

they call their handle well michah Hoffman also known as web Reacher put together this profile fer module he does a lot of thread Intel work and basically he created name check on steroids like he's gone through and he's taken all of these websites some of them you know stuff like Ashley Madison others just social media sites that name check didn't cover and created a list of several hundred of these that will do that same check all right really cool stuff um a really easy way to enumerate profiles and then lastly a Facebook directory does anybody know that does anybody even seen does anybody know what I'm talking about when I say Facebook directory this is something I stumbled across

actually John Poland showed it to me uh his name is he's forced recon on Twitter a couple of weeks ago I had no idea that this existed I'm pretty sure most people don't anybody seen this before this is cool right you can look up anybody with a public facing profile you just find their name and you Whittle down in between these dashes until you get down to their actual names and then you have all the profiles for the people with that name that's pretty powerful I didn't know this existed until a few weeks ago I built a module around it that will go down it will automatically do all of that searching through you Whittle down to the actual

person pull all their profiles down and populate it in the profiles table within Recon NG very cool stuff the next step on this one and I'll probably won't get it finalized until towards Deron is I'm going to make it to where it's going to take each of those profiles and confirm that they work for the companies you have in the database and so that way you have less false positive poses but Facebook directory really cool resource for doing ENT on

folks okay so what are we talking what do we want to talk about next breaches all right what am I looking for when I look in breaches well definitely want passwords and username sets credentials credentials wonderful to have okay but also breach details and why do I want these things well first of all password reuse is rampant right how often I mean I'm sure those of you that are doing pest pen testing you see password reuse a lot in your environments um whether you're doing Network pent tests or whether you're doing web application pen test but even if passwords aren't being reused they're very handy for pattern analysis something I actually saw on the last five days if you saw say you did a

pone list dump and you had somebody's password from last October and the password was capital a u t mn14 what would you try right now capital s u m m e R15 right Autumn 14 summer summer 15 guess what got us into a Citrix box summer 15 on 35 accounts we only had 35 usernames and summer 15 worked for one of them incredible that kind of stuff works all the time so if we can do that type of pattern analysis see what you were using before we can make a pretty good guess about what you're using now and we have a much higher likelihood of guessing what your passwords are and then as far as the

breach detail are concerned Why do we want that stuff because we're looking for areas of weakness right A lot of times breaches are a result of some sort of web application attack okay so if we can get some information on exactly how you got breached perhaps we can find some weaknesses that are systemic through your environment or may occur in other applications because they typically go back to the weaknesses the weakness in a in a developer's lack of secure coding practices so how are we going to find this stuff well search- credentials right we're looking for credentials here that's where all the breach data pretty much lies within Recon NG is is within higher harvesting

credentials a lot of these you've already seen this is the free Pon list module here which will just kind of give you a yay or nay if that if that particular account has been breached um there is still the pay for Pon list modules there um some stuff that I may not have talked about before that's been in there for a while is the have I been pwn stuff anybody familiar with that stuff from Troy hunt yeah great stuff right he's got that same type of search where you can search for breach but he also has this really neat thing where you can search pace so he's got he's archiving all these paste from these different paste

websites and gives you the ability to search them so I have that built in here a module where you can go you can search these pastes and it'll come back and show you which paste has that particular string in it pretty cool stuff another place we can learn about breaches privacy rights.org really cool website and it's updated regularly a great place to go looking for for breach information the actual details I was talking about privacy rights.org here it is you go down down here and if you want to see say when was the last one that was yeah September 10th 2015 was that two days ago so that's the last time a breach happen say I want to see some

information on the H recent breach it's got ton of information in here they update it they keep it wellmaintained great place to look for details on uh on breaches if you haven't already been using that um and then another pretty fun one out at Lenny zeltzer website he he has this was company hack search which pretty much takes a bunch of hashtags from Twitter that are related to breaches and then searches Twitter with the company name or with the string that you give it kind of interesting I've never I've never got a ton of useful information out of it but it's really quick and it can give you some quick hits um and the another

thing is is just do General searching right your search engines can definitely help you in this particular area next category information I want to talk about is technology so what are we looking for here well we're looking for client and server side Technologies um what's important about that is if I know what client side Technologies you're using when I do when and if I do find an exploitable Vector I have a better I have a better understanding of how I'm going to exploit it SQL injection knowing what type of database would be good information command injection knowing what operating system you're using gives me an idea of what files I need to look for or what commands I can

actually run and when we're talking about client side uh client side Technologies you've got these really rich client side Frameworks like angular and meteor that are being used nowadays and if you can determine that those are in use there's whole other categories of vulnerabilities associated with client side Frameworks that don't exist when you're using software that's just server side that's just using a server side framework so really important to pull this type of information down also you want to look for outdated software and the reason that's important to us is because a lot of times that can lead to direct exploitation of the infrastructure without doing any work to find the vulnerability if we can

determine you're using an old version of Apache we go to cve details uh I think it's Doom or.org whatever it is we can see right then and there what type of vulnerabilities and whether or not there's publicly public publicly available exploit source code and then we're in um so looking at technology is extremely important as well so how are we going to do that couple places um anybody here ever use bwi.com basically what builtwith does is they spend a lot of time analyzing headers and and uh uh pretty much crawling the web trying to determine what technologies are being used for everything on theom Space uh I sometimes have trouble finding things off the outside the Doom space but within theom

space they do a pretty good job covering it um I'll show you the kind of information that that can lead to I think I've got it pulled up here that's the uh that's the was company hack thing built with.com here so if we do like nsa.gov it'll say hey they're using i7 right are there any do you have a zero day for i7 well hey here's where you can use it right um says where it's hosted you can kind of get an idea they're using aour DSS or maybe get something going on there a little bit about SSL talks about server side Frameworks using asp.net so this is a place where you can go get

some information there is using jQuery uh about the client side and server side Technologies in use they do a pretty good job of enumerating the technologies that are being used uh so it's a good resource to go get that information another place you can go get that information uh job postings right uh job postings is a third on thirdparty webbsite monster.com look at job posting you could also look at resumes or just go straight to the web page and look for look for something like this right here's nys.com com. careers if you're interested in a in a in a career in appsec you know you could just do that right there and send me a resume um but

yeah so that's a good place to go get it too right so next thing this is probably my favorite is going in in getting vulnerabilities so what am I looking for when I go out here and I use o s try to find vulnerabilities well first of all I'm looking for already readily exploitable vulner vulnerabilities and those of you that have seen my talks before you know the tech the information out there exists to do that um I'm also looking for Source cord leaks that may exist in news groups forums things of that sort and I'm also looking for searchable bugs and I'll talk a a little bit more detail about what the searchable bugs are here in a minute but

why do I want to why do I want to look for this information because this is like no or low like very low effort uh vulnerability Discovery right somebody else has already done all the babysitting of the scanner for those of you that have scanned web apps before it could be painful right they log out on you sometimes they end up getting a ton of false negatives because of that sort of behavior they got to babysit them pay attention to them make sure they're doing the right thing make sure they're not crawling endlessly right so all that work's been done for you by someone that's scanned all these public facing websites and then made them available to

you so low effort on your behalf you just run a search there's something there investigate it and it may be an AC of vulnerability that you can exploit and this results in some quick wins typically so how do we want to go about doing that uh first dynamically how do I do it there's a couple of online vulnerability scanners that I use Punk spider and a safea web uh Punk spider something I've talked about before they're scanning everything in the dotom space they may have added a couple of other tlds they don't do any G governments and I'm not sure if they do any edus either uh but they're constantly scanning the internet for vulnerabilities like active

vulnerability scanning SQL injection OS command injection a lot of the a lot of the stuff we look for very routinely and then a safer web is specifically built for the net frame work this is also a Troy hunt product basically what he's doing he's looking for configuration vulnerabilities in net so these are unique to Net Framework stuff so if you know you've already enumerated that you're dealing with net you can use a safer web to determine if there's some really quick wins if there's some vulnerabilities that exist there uh other places you can look are some V vulnerability disclosure sites for those of you that are familiar with what used to be xss they are no longer maintained

all that data has been imported and pushed over to xss posed which is is basically a crowd crowd contributed xss vulnerability website you just go there you can search for your domain see if anything's been submitted for that particular website or hosts to xss post and you might find vulnerabilities that way and then of course Zone H which is the uh the uh what is it the interface um what's the vulnerability I'm looking for somebody help me out here when you change the interface to make it look differently anyway those things get posted there all the time what's that the Faceook there we go that's the word I'm looking for all right so that's where they put a lot of defacements get

posted you can go there search for your domain search for a host see if any of your stuff Falls within Zone H um and then validate if you've had any defacements once again we can do search- vulnerabilities to find the the modules and Recon NG that apply to these okay here notice I marked out Target and said the app must be public facing a lot of times when we test applications we're doing it in like a uat or a staging environment something not production so we don't goof up an actual production application right well as long as the production app is public facing and one of these scanners has scanned it that information in theory should or could

still be valid in the uat OR staging website because a lot of times that code is exactly the same right so just because you're in one of those internal environments doesn't mean pulling that data for a p for the public facing version won't help something to keep in mind okay for static vulnerabilities well how do I go about doing this this is some of the coolest stuff that I've been doing recently um in terms of what's cool to me um stuff that's actually really helped out a lot and that I don't think is being exercised as much as it should be um and this is the idea that a lot of folks now are hosting

their code on third-party repositories like GitHub bitbucket um codx Source Forge stuff like that it's not just open source software being stored there anymore a lot of organizations major companies are now putting all of their software there in private repositories sometimes they publicize pieces of it but they're using these places now well this git Rob uh tool which I find is pretty cool actually you you give it the name of an organization it will go and it will find all the repositories on GitHub that belong to that organization it'll pull down all the file names of the files that are there and it will look for interesting things like secret token. RB and things of that sort or

what's this file right here for anybody that uses Recon NG I was looking through the code and I noticed this and I was like hey that's not very nice because there's one point in time where I accidentally pushed up my uh the database with all my API keys in it and that's it right so he's got he's got an actual uh signature in here to look for people that have pushed their Recon NG API Keys up to GitHub right so it's kind of a neat thing where you got all these signatures in there it's built in a modular modular fashion but the only module that currently exists is the one that analyzes file names so this is one of those projects

where if you like the idea of doing this kind of online static code analysis this is a cool project that you can contribute to another neat project is this search code search code.com uh this is It's a maturing project it's pretty new but basically what he's doing is is he's pulling down code for I mean just a ton of of these code repositories let me pull up the the website here because I kind of want to show you this search code so if you go down here GitHub big bucket Google code all of these he's indexing all of the public facing repositories for those and he's making all of that code searchable and then kind of like Google he's got these

advanced search operators that you can use to really kind of narrow down your search search by language all kinds of stuff and so using this we can use a sort of like a dork mentality there right to try to find vulnerable or interesting code in projects on all these different repositories some of the search functionality here doesn't work really well yet because of the way he's indexed the code uh but I've noticed it has gotten better better over pre over recent months so that's a project to definitely keep your eye on and it's still a place that I go and look um on each test it back up there all right here we

go okay so I mentioned GitHub and Google advanced searching um if I didn't it was a last bullet there well basically what I'm getting to here is is the is the idea of using Google dorks and GitHub dorks um to help find vulnerabilities right so GitHub Dorking right we got two different approaches anybody here heard of GitHub Dorking using the advanced search engine within GitHub to find code vulnerabilities probably seen some tweets about it every now and then when a new bug comes out everybody's quick to go to GitHub and be like can I write an advanced search string that'll find a bunch of repositories that are vulnerable to this yeah okay cool disclosure all right so the way you do

this there's two approaches that that that I use to do this particularly One is using that GitHub advanced search and then the other thing is using Google with the site operator of hub.com okay they both have their advantages and disadvantages one of the disadvantages is that with GitHub um when you use these advanced search operators it requires at least one just string that has to show up in the code so if you like use path and you're looking for a specific file name it won't just show you all of those files you have to list a string that's in there somewhere in most cases when we're looking for a vulnerability there's a string associated with it um so so it's not too

much of a handicap but is one of those things that you have to take in account when you're using it the on the other hand using the Google site operator makes it a little bit harder to focus on an actual Target so if your Target's a domain or your Target's a particular organization or or repository if you're just searching Google for it and you're already using the site operator to focus in on a domain it's a little bit tougher to do that U now you can so here's some examples here the first example is just going to give you every secret token uh file for for rails um within GitHub the second one will actually allow you to

narrow it down to an organization and a uh to an organization and a repository if you know that information okay so you can kind of do it but it's one of the disadvantages of using Google I like them both I use them both actually so I refer to the the Google git combination thing is goo get dorks don't quote me on that I'm not really trying to start a new word thing it's just something I throw around at work when I'm talking about it um the cool thing about it one of the advantages to that one is you can do it on more than just more than just GitHub right that applies to bit bucket it applies to

GitHub it applies to anything that Google can index any of those code repositories that search code is looking at Google's probably indexing so you can look at all those the downside is is I have noticed in looking for the exact same things the GitHub advanced search engine is more accurate and probably because Google's having to index everything and maybe it hasn't called it yet uh or maybe it's expired in their I don't know how they manage their indexes uh but the results of the actual GitHub search are typically more accurate and I have some and I've got some modules that actually do this stuff for you GitHub minor GitHub repos and GitHub dorks so what GitHub minor does

it's pretty cool stuff you just give it an organization name and it actually goes through and it pulls down all of the users underneath that particular company and all of the repositories so now you have the profiles the GitHub profiles for all of the listed Developers for a particular organization and that's going to become really key here in a few minutes also pulls down these GitHub repositories now when you pull down when you go back and you and you pull down the repositories you can go in and pull down uh gists and things like that too which we'll talk about here in a minute and then I've got a GitHub dorks uh module which will

actually take a pre-built list of GitHub dorks and apply them to all the repositories you've pulled down so now with in Recon NG you can leverage this advanced search functionality to try to find vulnerabilities and source code statically without ever sending without having the source code from them without ever sending a packet to their environment right now this isn't I mean a lot of times it's like well if I could access the source code why don't I just clone it locally and scan it right well what if your what if your target company has 150 repositories you're going unclone every single one of them and run a source code scanner against them all um if you're on a timed penetration test

and all you're looking is for some quick wins this is a really easy way to get some quick wins okay so it's all about you know what's the objective too and what's the scope I'm was going to skip the demo on some of these because I want to get in the content before uh before the time is up and the content is probably more important than watching the module just run You' be able to do that kind of stuff on your own as well all right so the next thing is sensitive information just sensitive information in general so what kind of stuff am I looking for when I talk about sensitive information well number one debugging

information has a developer done anything and left some debugging information somewhere development notes kind of the same thing as as as the the Developers for the organization use some sort of third-party um application to leave development notes somewhere and then out of band configuration storage I know a lot of a lot of developers will say well I don't want to commit my config file with all my passwords and API keys and stuff to the repository I'm going to keep it over here separately in a gist and I'll put a basic or no config file at all in the actual repo that gets pushed out okay why do we want to do this stuff pretty obvious right um first

we can learn about some applications that may be in private repositories so if we see some debugging notes we see some development notes we might find out apps that are private that we can't see find out some information about apps that are private we can't see um through just knowledge of their public exposure because when you have an organization like that you can create private repos in GitHub or you can create public ones and quite often you'll see you know a number of public and they'll have a number of private behind the scenes getting information about those private repos is important that's good information to have and of course credentials and secret keys that may be in those configuration files um

this is something that actually we'll talk about here in just a second I'll give you an example so how are we going to go about doing that well first thing that we want to do is look at pastes right paste bin paste. org all these different pace sites um great place to go get that type of information extremely hard to actually find the information right most of those pce sites don't have very good search capability built into them or things are happening so rapidly like past bin past bin only allows you to search for I think like the past 300 pastes or something like that so the way things are moving if you're not constantly cataloging that information

or constantly watching you're not going to catch it it's very difficult so my go-to is gists how many people here use Google GitHub gists I know I use is it couple folks okay cool thing about gists developers treat them like pastes right so instead of using past Spin and all these other places gist you know GitHub is safe that's where we keep our source codes so let's use gists well the really cool thing about gists is that or is that organizational accounts within GitHub cannot create gists when a user of an organization creates a gists by default is associated to his public account right so if they don't understand that if they're doing say some debugging or they're doing some

some configuration swaps between them and another developer and they're using gists within GitHub that gist is on their public facing interface not the organizational account and access controls that come there right so something I've seen recently on penetration tests is I just pull down I use that first uh GitHub minor module to pull down all of the Developers for a particular organization then I look through all of their gists I found passwords I found database connection strings I found API keys to third party apis all in their gists because they were debugging stuff or they were keeping development notes there definitely some stuff you want to look out for okay um and another thing you

want to look for in these things is you want to look search for keywords the kind of keywords that you would normally search for um if you were doing uh a file system search say you had command injection on a server I want to look for say passwords and source code well it's the same kind of stuff you can look for here just get a good list of keywords and search through those and I've actually built a module and Recon engy to do that as well so when you did that GitHub repo module on the previous slide it would pull down all the repositories and all of the gists that belong to each of those developers you enumerated use

this little module here it takes a pre- pre-built list of strings and it goes through looks at all of those gists and tells you if it has a hit there okay so once again we're automating this process of using GitHub to actually find this really really useful information all right and the last thing I guess yeah this is one of the last things I wanted to do this since I created Recon I've always wanted to scrape Google effectively and I've not been able to do that I stumbled across a small little script and I would I want to give the guy like attribution but I couldn't figure out who wrote it because I think I found it on ppin or something

like that but it was a guy that wrote um a parser for the Google front page that actually works it actually parsers HTML and I found out what the key was it's the user agent that you use if you use like a links command line browser user agent GitHub gives you the code in a very very Parable way right so by using that user agent I've now been able to effectively scrape Google in a way that's uh that's consistent I get back perfect results nearly every single time and so now I've got an effective way to search Google now that I have that effective way to search Google I've been able to incorporate the Google hacking

database into Recon NG okay so this is something I've wanted to do for a long time google the Google hacking database leads to all kinds of cool stuff there's ways you can find credentials in there source code um contact information you can get in there Excel spreadsheets different file types for all kinds of stuff those of you that have dealt with the Google hacking database know the power that's there one of the downsides to it there's a whole bunch of garbage in there too I mean there's thousand I think there's roughly 4700 different dorks in there and of those 4700 dorks some of them are just meaningless some of them are old some of them are stuff

people just submitted that don't really make sense or work so that's kind of one of the downsides to it the way I've written the module I'll show you in just a second we work around that a little bit but there's lots of sources for dorks you don't have to use the Google hacking database um Anonymous you a few years ago published their SQL injection Google dork list the one that they the one that they were using at the time to find SQL injection vulnerabilities and applications all around the internet um you can use your goog dorks with Google right you can use now you can now use the Google mod this G the Google hacking

database module to search for your goog um strings and then also custom dorks I keep myself like a list of six or seven that I use on every single test it's like I'm going to run these seven against every single thing and if it comes if it pops something back I'm definitely going to take a look at it and there's also the being hacking database um now that obviously won't work fully some of the strings will work for the advanced search operators that are common across the both uh but being hacking data base is something you can use with Bean to kind of do something similar so I do want to show you I do

want to show you this real quick let's load Google hacking database show info and you can see here I got a whole lot of options right basically what it does is I update the Google hacking database routinely I've got a script that does that for me when you load this module it parses it out and divides it by category so rather than search for all these thousands of of dorks you can just say which ones you want you can just flag the ones you want to search for so in this particular case say I want to search for error messages right because maybe I'm looking for SQL injection so let's set Google hav database error messages to tr through

and we'll run it and we'll let it go to town something I want to show you here hopefully it happens cuz people are probably asking well how you searching Google without getting um the uh the capses right well eventually if Google's watching Are you seriously not going to give me a capture this is like the only time they've ever not shunned me I'll run it again just to make him shun me here in a second as you can see I've already got hits right check this out got to hit here for warning cannot modify Header information if we go to this what are we going to find I I'll show you that here in just a

second if we go to this website yeah there's some information in here all right these are errors and then maybe down here there is some pathing information home cap repo yeah so we're finding error messages that are leading to code vulnerabilities this is a disclosure of inform information disclosure right I stumbled across this right before I walked in here today so this is pretty legit stuff now what just happened when this popped up Recon NG is capturing the capture request from Google presenting it to you and allowing you to answer it and then it saves the I'm a human cookie and then it just it actually increases the speed at which we can search so i n

gs l y if it's right Bam Bam Bam we keep going so now you have capture answering capability built into Recon NG Google can't shun you permanently anymore which is awesome right that's probably my favorite thing that I've added recently and it was also the hardest all right so in a nutshell that's the kind of stuff that I'm doing when I do over Source intelligence gathering uh for application security um you know I would hope that come on why isn't this working I would hope that you know this is some we've talked about a little bit of stuff at least glazed over some some uh topics here today that that you can probably take back with you for those

you you do the absc stuff you know i' I've talked to a couple of you once or twice I mean you see some stuff in here that might be useful to you even though we do absc stuff ENT still helps us we take a lot of this information we can start connecting it with these attack vectors and really showing some impact uh to the folks that we're working for U and that are paying us to you know provide them some uh uh actionable information so one of my final thoughts here um one of the really cool things about everything I showed you today none of it requires paid for API Keys that's something that's been a complaint about

past talks is well the API keys are cost a whole bunch of money and I got to go out and get all this stuff and it cost all this stuff is free none of this even requires you to go out and fetch API keys if you do appc it's as simple as this all right the other thing is help me build out build out these list of dorks whether it be Google dorks the the the GitHub dorks or even the search strings for the gists right if you've got something that you use and you're an absc share those with me we'll put them in the repository we'll grow that together let's crowdsource it also want

to give a shout out to the offensive Security Guys maintain the Google hacking database and actually provide it to open source projects like Recon NG um so that we can do stuff like this with it which is really cool for them uh they maintain all that stuff and uh pretty much for free and there's a lot of overhead built into it also quick note if you want any of that expensive like uh like Pon list stuff that comes in Recon NG um you've seen it in recent talk or past talks and you really want to tap into that data but you've talked to Pon list and it's just too expensive um I've got something that I'm coming up

with that may make it accessible to everybody in the community I can't talk about it a whole lot right now because I'm still getting it approved uh through some other channels but if it gets you know follow me on Twitter uh actually what I want you to do is send me an email so that I can contact you if and when if if you're interested I can contact you if and when it gets approved um what I would really like to do is take some of that expensive data that pist provides and make it affordable and available to the security Community uh and then finally if you like what you see here and you really enjoy appsec um

I'm teaching a course up in Charlotte um in the beginning of November 250 bucks a seat for two days of really kick butt web application penetration testing training I encourage you to check it out um probably over time already by a couple of minutes but uh if if you're okay with questions anybody got any questions sure on some of your U scanners I was wondering how do you deal with pays if if you get a pay wall do you to around those it depends on what the pay wall is so if the pay wall means you have to have a subscription and you're still and it's and it's a resource that has to be scraped um then a lot of times

you you have to go have your own subscription and then provide your username and password to Recon NG and then Recon NG will authenticate and then get to the information and crawl it and scrape it uh same thing with a the pay wall is API then you have to go out purchase the API give the API key to Recon NG and then it will use the API to pull that data down like Facebook to get to do the regular do you have to have a you have to have their account you don't even have to auth I ran into an issue with it today where it shunned me with a capture so I'm going to have to do some some trickery

with it uh that's why that module has not been released yet it's not been pushed to master Branch I'm going to do some uh some analysis and see if there's a way to integrate that capture into the framework so that we can actually use that resource but even if you can't automate it it's there for you to use through your web interface right and so if you've got some some very specific targets you can drill down and pull out some information on them so it's there regardless of whether we can automate it or not any other questions all right Tim questions from you for them I asked questions did you give out prizes I know you I know you answered a

question at one point in time you answered a couple of them and then we had some other people do you have any questions about Recon so called l a particular feature your L profile to enumerate everyone in the company what is people recently view people recently view okay all right

rub um oops sorry you got one come on for sure anybody want to take a guess at what the advanced the what the what the search string what the Google search string would look at for the module that scrapes exif data or metadata question again what would the Google search string look like in the background for the module that goes out and fetches files and then searches them for metadata what Google Advance operator am I using file Ty file type yeah that one that the guy in the glasses or one of those guys up there the guy in the glasses both of them have glasses figured out all right uh sure you put oh I'm sorry about that I didn't put

it up there did I there you go go apologize all right um all right one more so uh Tim mentioned two modules that will search for the rese two modules that will search uh for websites that archive um websites that have been compromised name one of them one two there uh um they archive breaches but they're not archiving vulnerabilities yep that's it Punk spider got a blue shirt in the back yeah blue shirt in the back is there anything else that's a giveway so we got one more talk this evening and we got got more giveways here this talk um start that up I'm not going to be able to do demos next week I'm literally going to have to

cut out all demos yeah let me turn these off

for

e

e e