Automating Web Application Bug Hunting

automating application security bug hunting thank you all right everybody here's okay check good in the back great my name is Jonathan I am a information security professional with the question mark and I am at kind of security and have a little project we're going to talk about today called intrigued and previously a bug crowd in rapid7 I'm Jerry gamblin on a rank amateur if Jonathan is just a professional question mark I am the professional security engineer at Kenna and you can you misspelled security that's that would explain it that's it yeah got it you saw the joke all right my blogs at Jerry gambling com2 many all right so I had a talk ready to go

here and I kept bouncing ideas off and Jake ran like oh what about this what about this and he's like oh I have this cool project that I've been working on that's 100 times better than all of these Python shell scripts keeps sending me that barely work so he is kind of like my mr. Miyagi so I decided that while I had another talk today about building an information security program that I would just work with Jonathan on this because it's probably way more useful for more people for him to talk about intrigue and talk about the data we get out of there versus me talking about some really really poorly written shell scripts but when you think of just just

to kind of base us out here when you start to think about application security everybody knows burp and burp is the gold standard why is that Jonathan I mean it just it is the tool that I learned on it's the tool that you probably learned on and a lot of the depth-first testing has to happen in a proxy so you know burp really is the gold standard it is the thing that I think of when I think of application security testing yeah every time I see a bug bounty submission it has a burp screenshot

so while we're just talking about expanding the data you can get out of your automation we're going to talk about a few of the tools that we love to look at in burb the attack surface detector allows you to automatically scan a webhost to try to figure out what is installed in there what frameworks what applications I haven't had a chance to use turbo intruder yet but yet I figure out it's a heard it's an amazing way to to Dass yourself just to show of hands how many folks have heard of turbo intruder ok a good number just a quick aside on that effectively what it's doing is holding the connection open and allowing you to send request after

request after a request and then collecting them all and on the way back in order to speed up the process of brute forcing a lot and so that's James kettle's project and so I would definitely recommend checking that out if you're doing any sort of web testing be careful on the server that you're testing on though you will knock it over and I'll use active buckets a lot in burp it allows you to give a AWS and a GCP key and when you're burping traffic through there it'll tell you if it finds a bucket that is either publicly open are open that your key can read and I see that miss configuration a lot you'll

go in and you'll stop it from being open to the public but if you supply a valid key it'll give you all the data out of it me yep go for it all right so with that said you know it it seems to us as we kind of sat down and thought about this a little bit we still have the problems you know with all the testing that's happening and and you know there's always more testing needed we still have all these problems of unauthenticated databases exploitable legacy systems exposed web vulnerabilities and misconfigured services so so there's something missing in you know web testing that isn't happening today and any thoughts on that no no I think you're exactly right we

we wanted the hope that moving a cloud would get rid of shadow IT it's just made it worse and there's you know is I think about the bug bounties and I think about all the tools and the sources that are available to us I mean there's a lot of them and this is just a I got tired of typing them out so I literally stopped but you know you get the sense there's just tons of stuff out there that's helpful in finding these misconfigurations these bugs the shadow IT so so again where's the problem what's going wrong and you know bug bounties really do provide an important safety net and as I you know as we look

across organizations that are running bug bounties we find that they're often much more secure on the external perimeter so you know this this is definitely a good step in the right direction but still there's still lots of things to be found even on organizations that are running a bug bounty because it's a great safety net but it doesn't offer a full coverage and that's the issue is when you have a bug bounty program or if you have any program it's only scoped to what people like that I've had a regular pin test you say hey if we look at my website and they spit in 30 of their 40 hours looking at your purchase path which

might be what you want them to do but it might miss some major vulnerabilities in your network so the tool of jay Krantz built and that I really like helps you get a more holistic view of everything in your environment yeah and this isn't to say that bug bounties aren't sufficient or they're not necessary rather that they aren't necessary they're not sufficient there's just so much attack surface that you really want your bug bounty focused on understanding the application and digging into the details of the application as opposed to sort of these broad problems and so you know there's really just inherent complexity in in the network today and so how do we get ourselves to more

coverage you know how do we get rid of some of these fundamentals and why actually is this so hard I mean rich mogul had a pretty interesting tweet on this just the other day the literally the hardest problem in security is solving those simple problems asset asset management vulnerability management you know some of the basic problems solving those at scale yep so I put this slide the previous this slide here's my favorite slide because if you think of this picture of San Francisco as your CMDB and I tell you can you show me everywhere on this map that you can get a good hot dog or they you can get a good slice of pizza how easy would that

be for you to do it would be pretty hard right like you could guess and you could try to try to figure it out and you could pick out the obvious places that you know but you would never be able to get them all right and that's the problem we have with cmdbs today is while you might get 80% coverage in a really really good environment you're never getting a hundred percent coverage I have not had an honest conversation with anybody who runs the CMDB that will claim that they have more than 80% of their assets fully covered inside that that tool yep and this is where we want to get right we want to get away from the hey let's look

at almost all of San Francisco for for your you know for where the best pizza is to get into something like this where you're just looking at a micro it's something small and really figuring out the minutiae of what you're looking for yeah visibility is really a challenge and so you know as we think about visibility and we think about finding problems there's really three keys that I want to talk about today I want to talk about that broad array of sources you know we saw that slide with all those different tools and sources and things on it that's super important to be able to get information from a bunch of different places but there's two

things that are really missing today and that's an ontology and recursion against that ontology hey Jonathan I obviously know it on Atallah jemaine's cuz I'm really really smart that word gets thrown around a lot can you just let somebody who might not know what ontology means yeah so when we say ontology what we mean is is a you know a set of concepts such as things you know entities really be relationships between those things being fleshed out and sort of built into the code and so you know when you think about brute-forcing DNS or you know looking for systems by scanning n map and map doesn't understand the output that comes out of it and you have to

interpret that output tell it that it is an IP address to be able to use it right which has to happen in your brain so can we build that into the code and can we have that sort of recursive process where if we find an IP address we know to scan it and if we have a you know a set of ports can we look into those ports more deeply so you know we're really talking about coverage here and I'm going to talk to you a little bit about personal project of mine it's called intrigued core and so you know it implements some of these concepts in the form of tasks which give us lots of

sources entities which give us that ontology you know an entity would be like a domain or it would be an IP address and it has this concept of machines which will give you recursion and this might sound really familiar multigo has very similar concepts right but it it lacks that ability to automate but if you're familiar with that that's kind of how that's a good way to think about it and so here's a set of sources tasks that are available in this tool today and you might say you might see some things that look familiar here showed an census probably very familiar and you know these are these are more the ones that are highlighted now are

more like the databases places where you can go search and pull information out right and there are tools that also pull from these sources spider foot is one that comes to mind a mass is another that comes to mind that searches from different sources but these are just some of them those are more like databases these are more like miss configuration checks so if you can't see it down below it's you know like Google Calendar check Google Groups check email brute-forcing get robbed which we just added today and how many folks have used get robbed quite a few ok cool if you haven't used it effectively what it does is you pointed at an organization in github it'll go

download all the different repos it'll look at all the users of that particular organization it will download all their repos and then it will essentially statically analyze it for things like secrets or leaked secrets and so it you know it tends to be a very handy tool but how do you know that you got all the different organizations that could belong to a given organization and so really like you know this tooling is built with the idea of pointing and add an organization and grabbing everything and so there's you know and today we're talking a lot about web application testing and here's somebody you know specific tasks focused on web application security brute-forcing credentials out of a URI looking for

security headers looking for focused content we'll talk a little bit more about that and even some Vons that we can run against an application but you know I mean this this is sounding more and more like DB Auto pone if you're familiar with Metasploit back in the day you could use DB Auto pone to just run every module against everything but the problem with that was you were kind of spraying vulnerabilities against everything and sometimes you'd actually knock services over so there's a there was a missing piece with DB Auto Pound and that was really understanding the ontology of what it was testing and so when we talk about entities we need to understand these different types of

things we need to understand these different entities and so you know built into intrigue is these concepts of entities you know a domain a network service and there can be different types of network services right you could have an SMB service you could have an HTTP service so this even goes further down but just to give you a sense of the types of concepts and so I'll just run you through this from a user interface perspective because this might help understand it it looks pretty good on the screen I'm not a front-end person so you can think of this as you know a pretty awful interface but it serves the purpose and so notice that

there's a task it's called create entity in this case which literally does exactly what you might expect it creates an entity and it's going to create an entity in this case of the type domain and we're gonna do it against yahoo.com and you can kind of ignore the machine in the iterations for now but understand that those are things that will give us recursion once we have a domain go do more things but for right now we're not going to do any iterations but we are going to enrich this domain as soon as we get it and so you know hey it looks it up you get your ipv6 you get your ipv4 addresses but

there's this concept of enrichment that allows you to completely build that entity out right and that's important for our ontology because we want to know that we have a complete domain we get all the MX records we'll get all the text records we get everything before we continue on with more stuff so we use enrichment and notice that this also has the ability to see enrichment here and it has this concept of scoping and scoping is pretty simple but it is built into it scoping basically says if we entered this ourselves its scoped and if it follows one of the paths that we trust meaning it pulled information from a source we trust it'll bring it in

scope and you can check to make sure that you don't actually go scan the entire Internet right which would be you know useful but probably not what you're looking for if you're only looking for one organization or one small set of things and so it keeps track of all this information nicely and effectively a graph database it's Postgres but you can think of it like a neo4j and it's implementation that i've set it up with and what that allows you to do is when you just look up a domain it'll grab everything and remember that domain probably is a load-balanced domain and it goes through all these different IP addresses so when you're scanning yahoo.com actually you're scanning all

that stuff or you should you know and if you you've probably noticed if you point and map it a domain it may scan one of the IP addresses but it won't scan all of them right so if we take this and we pull it away from a domain and we pointed at a URI right so we just do a create entity URI yahoo.com it goes out and it grabs all this information associated with that particular URI and it grabs a screenshot and the nice thing is you can iterate here in this interface and continue on with more tasks manually or you can have it automatically run tasks and you know hang on going back to that idea of how

do we now make how do we make sure that we don't you know literally throw everything at the wall you need really good fingerprinting and so there's this library from rapid7 it's called a recog a guy named John Hart built this and it's really good on the network side and this is just an example of a check yes it's XML yes that's terrible but it is free as in freedom meaning you know you can use this whatever you want however you want it's open you can contribute to it it's easy to extend it's relatively comprehensive on the network side but we're really working on the application side and there really wasn't anything else out there that was all of these

different properties so there's a library that I built called it ident that you know thrown a URL I just threw security beside that at just a minute ago it'll find the fingerprint it'll find some some various configuration information around it and you know hey it found Jack Daniel's gmail address Thanks sorry jack but we can use that enrichment process to match vulnerability so if we can figure out what software it's running it's relatively simple to take the CVE database and point it at that particular piece of software and come back with a list of vulnerabilities now I'm not saying yahoo.com is running this this is a different site but you can see if we can

figure out that it's patchy Tomcat you figure it's 6.0 1/4 we know that we can match that to vulnerabilities because the CVE database now publishes versions right which is nice it allows us to iterate through all the CVS find versions that are less than this and then match those to this site so let's put these concepts in action to take an organization right and again we're organization focused here broadly discover assets enumerate their abstracts and identify issues so one one quick decide before we do that there's this really great project called DNS grep that got released earlier this month and it's by this guy Irby Sam if you can't see it thank you to him and he

took the who's familiar with the sonar data from rapid7 3 quite a few folks ok so sonar is this project to scan the entire Internet and bring that data into where's it hosted at now I think it's hosted it open data that rapid7 comm Thank You Todd and that information is really great the problem is it's so big that trying to do anything with it notice the upper left here it'll take 10 15 minutes to get through looking for you know Urmi Sam calm a very small site presumably right yahoo.com will take even wait longer it could take hours to get information back so you know that kind of puts you in a tough spot when

you're on a test and you're trying to do this very quickly so Irby Sam built this it's essentially a binary search he noticed that DNS works you know backward to front you know everything starts to calm root and then Irby Sam is below that and then all the others are below that so you can build a binary search and a binary search allows you to effectively speed this process way up so he's doing in less than point zero zero two seconds so like milliseconds and so that's pretty cool we should use that and he released all the code it didn't go that stuff's publicly available you can go google for it if you have questions about it come up to

me after and I'll happily point you toward it anyway we built a task around that that task allows us to query that server for yahoo.com and just dump information about Yahoo's assets right and then create those entities in this in the application and effectively bring all that information together so that you now have a broad set of assets for an organization's specific to an organization but you know really again we're here to talk about applications how do we get to applications right we need this concept of machines because when we see a domain we should go search sonar exactly what we just did which will give us back addresses which we should look up which will happen through

the enrichment process and then when we get IP addresses we just scan alright and this is kind of pseudocode these machines are pretty easy to configure so you can kind of add in the types of things you want you know you don't necessarily have to do just this stuff but this will give us a list of applications because every time we get an IP address we'll just scan it right and if we get a network range let's just mask in it all right let's make it really simple and fast and so you know see you see very quickly you just get tons of information about the attack surface of a given organization all right and so I'm not sure which

organization this is this is just an example but it kind of gives you a sense you know there's a lot of attack surface out there for any given organization and because we do that CPE matching process we're able to sort of pull vulnerabilities out of this and so you know again that's mostly infrastructure level stuff now you will see some vulnerabilities oriented toward WordPress and you'll notice you said put a slide in for this there's a there's a lot more CVE level of vulnerabilities these days in the framework and so this you know this this ends up being a pretty valuable process if you're doing a bug bounty and matching to vulnerabilities and then matching

those back to exploit above one thing I'm not doing today that I will do is to say which of these are remotely executed remotely exploitable I don't note that today but it's relatively simple to do that with the cbss course so now now that we've got all these applications do you want to talk about this Jerry do the next one cool so there's a bunch of different resources out there that are very helpful in finding web or exploitable web issues and up on the upper left you see cyclists how many folks are familiar with cyclists yep quite a few good how many folks are familiar with payloads all the things over here I feel less but still some

some pretty good amount and the fuzz DB project yep cool okay so you guys have seen this before ultimately what this will allow you to do is now that we've got a good fingerprinting process we can actually take these paths and map them into the tool and again kick that off automatically so just an example of this because we can fingerprint asp net we can go look for trace ax D which still works yes right and in fact we can even pull out a better version because asp.net doesn't give us the version the specific version only gives us the sort of high level version which is not useful for vulnerability matching ColdFusion there was just recently I

think two days ago a ColdFusion bug that required a file upload but it's a RC and there's still quite a bit of cold fusion out there on the internet so you know being able to find cold fusion is a pretty useful thing that particular bug required file upload so I've added into ident the ability to check for file upload can we find that on this thing and I'm not spidering the sites by default but you could create a machine that spidered the sites automatically as well also if you're familiar with Arachne you can also use a rack and you to grab information on site and just do the scans so a lot of that was focused

on organization centric things yeah are you going to talk about this yeah yeah so we wanted to give you guys a take away and we quickly have been made aware that there's a way it's not a bug right because Google said it wasn't a bug it's possible to see if somebody has a public calendar how many people in here used esuite for their organization yeah so we quickly discovered with some help that if you have a public calendar it'll return a status code of 200 200 boom and if it's not public what will it return 404 so all you need is a list of everybody's email in your company and the intrigue module that Jake ran wrote

and you can quickly check to see who in your organization might have an open calendar that anybody in the world can see and just from talking to some people on some list it's gonna be more than you expect yeah so we promised the takeaway from from the talk today and we just this is the takeaway this was the one thing that we wanted to give you guys to take is to make sure you check your your public calendars so your calendars to make sure none of your users have them public so a little bit of a context on this to by the way if you go to the Sask Ubu calendar check module you can go to

github.com slash intrigue IO and you can find the intrigue code and that'll give you the URL it's embedded into that particular task one thing we're noticing and this isn't the first time we've seen this happen a broad array of SAS applications and platforms often end up building a config capability on top of existing user capability so so sometimes if your calendar was ever set to public and they added the domain level setting to disable that after the fact they didn't go back and disable all open public calendars that have been misconfigured maybe maybe unwittingly and this thing that we did a project on Google Groups yep where was very very similar they'd often been misconfigured

for the domain so all you really had to do is force browse to the domain and a lot of those Google Groups were publicly exposed that you know unwittingly more and more sasses are coming up with the AWS shared responsibility model yeah they're just not stating it as clearly so if you give someone your data you really need to to go back and a double-check especially those SAS administrators to keep up on the changelog notes to see when they change especially access features so that you know where you're supposed to be and ultimately like you shouldn't be able to bear force this you shouldn't be able to force browse to it no nonce nothing it

just gives you a 200 or 400 but that's not a good thing yeah so so let me give you a quick demo of the platform in use and so let me just pop to one of these and yeah so there's a bunch of things to think about so the question is you know is that the query is the issue or is that the case with all the different Google products and the answer is no probably yes yes well the intercept yeah again of course we would do that and you know oftentimes these these things where they have controls in place at the domain level are their way of handling this and anyone who is misconfigured

before those controls existed is on their own to fix that so it's a privacy thing it's not a security bug it's a Mis configuration and so

so so the question is you know what do we do about this I would encourage anybody who's running a Google G suite enabled organization to check their calendar privacy settings yep but remember it's not sufficient to just check your Google or check your check your domain level settings yeah you actually need to go check the individual calendars and that's certainly the case with this that was the case with the Google Groups issues so I just I would encourage everybody to understand that the privacy settings may be rolled out after the fact and you should think about things that have been exposed before those privacy settings were enabled we're about out of time do you want to show this real quick yeah yeah

let me just do a quick run-through so here you look at the yahoo organization so like what we were showing the screenshots of and you can effectively pull this from docker hub so you can just go grab it after the talk and you know like like we said you create an entity in this case we do yahoo.com I've already done that here so you'll see some things so let's just browse to the entities page this is all the stuff that's been found for Yahoo and notice there's kind of this interesting grouping here because of the way we do tracking of DNS records and IP addresses it lets us easily find load balancers we host things like that group those

together so effectively all these DNS names are hosted on the same set of IP addresses meaning their load balance together right and you know just kind of scrolling down here let's actually look at an analysis for you and let's just go app technologies right and so it's building out so this will take a little while to do an average organization of fortune 500 can take you know hours or days even to build out but the nice thing is you can sort of browse around what's running Ruby Yahoo looks like there's CC API Commerce Central right he's running Ruby cool there's a screenshot and if we want to learn more about this we can just run another task

here maybe you want to spider it alright and it'll give you some information here some configuration settings kind of dig in and then it'll kick off the task and everything is it runs in parallel so you you have the ability to kick off many tasks at once and let's just see what the issues where I'm curious what these were so it looks like it's found a bunch of s3 buckets those are probably all the same bucket but basically it sort of brings these issues together and and presents them to you so and this is a really really handy tool when I first had the idea for the talk it was really to put together a toolkit like this and

Jonathan just has this toolkit in a really nice wrapper he likes to joke about he's not a designer but the intrigue UI is amazing so like you said you can grab it off of docker and and run it tonight we're out of time we'll take questions offline we just want to say thank you guys for having us out today we've really enjoyed being here and talking to you about hunting for web application bugs cool thank you [Applause]