Web Application Penetration Testing on a Budget: Building an In-House Program

does anybody here actually do some web app testing oh that's awesome great so we don't need to be here then because you guys know everything right now so we are information security consultants we actually do work together on a daily basis I have about eight years of experience and information Riskin and IT actually I'll backtrack that about two years as information risk consultant eight years in IT mostly in apps app support and as well as on the project side of things myself are so as you as she just introduced I have been an ID for about ten years started as a software developer moved into audit from there InfoSec kind of like you know I

started liking this this whole thing and from there it started on to a couple of years in pendra but web application penetration testing you guys can hear us clearly yes / no louder louder louder that's good okay perfect thank you we actually don't talk that loud I think in real life but maybe we do won't even see screams yes that's true I do scream so this is gonna be a presentation about our journey we're gonna take a little ride with us and you're gonna see how we got here how we actually got here to b-sides was actually a challenge our mentor looked at us and said Natalya Herschel you're presenting at b-sides you need to submit

your slides but you should create them first so after about 10 minutes of breaking a chair and our mouths dropping and staring at him to ensure was he actually serious we realized he never jokes with these things and that is why we're here but back talk about two years first time I ever saw cross-site scripting I had no idea what it was I was like are you serious this could actually be on a website why why is this there how could they even let this happen I don't understand I was actually quite speechless and then I saw it again and again and again yeah I mean again xs/s was the first thing like reflected axises was my colleagues were

testing within my organization they used to get excesses pop up and I was very frustrated too at the beginning of it I said you know what enough of InfoSec I'm not gonna do this so it's you know that that actually initiated the whole idea of learning this putting a lot of effort hard work into understanding how to get those pop-ups what access means what clickjacking means I had no clue before that see what sequel injection means and you know so the whole exercise of learning and then you know coming to this point within a couple of years is what we're sharing with you guys so in our world we had a lone soldier he was

he had the qualifications he knew how to do webapp penetration testing but in our organization this was all out sourced nobody did it in-house he took upon himself because he's curious soon he wanted to see could he produce the same reports that they could apples to apples same scope same to us so here I want to ask you guys a question like how many of you guys actually outsource the pen test application to vendors okay how about how many of them do it in-house okay wonderful perfect so why are we here [Laughter] so anyways maybe then this is something that you've gone through yourselves or maybe you can find a little bit of a takeaway on how to maybe adopt something

within your organization or maybe we're just here for half an hour and you get to look at us either one pick and choose we don't mind so anyways he wants to do some apples to apples ensure same scope similar tools was he going to be getting the same results surprisingly yeah I mean like if you have looked at different vendor application pen test reports be it automated or in my experience I also looked at the ones that were manual for example where you know you find severities severity ratings the the greatings are skewed up because at times what they usually do is they'll pick up the report from the tool they would copy and paste whatever comes comes out in my

in my cases I have seen like tools gave out false positives we have to go through them validate those false positives which which is an exercise in itself like you cannot and in in in my experience these vendors have vendors as well as I I don't mean to say vendors who did the pen test these are vendors who provided the solutions to us as well like who share the pen test report with you guys so those pen test reports we reviewed on our end and in in this case what the tally is just saying the person who who mentored us was doing the whole full cycle on himself doing apples-to-apples comparison of one report which he took

from the vendor and the one pen test that he did on his own and needless to say they didn't exactly match in our case we noticed that there were items that were missed well he noticed that there were items that were missed even though there was similar scope so again apples to apples but the reports were different so looking at that why is that was it because he was more familiar with the application with this one no he wasn't he didn't have any he didn't have additional insight regarding the web app but it was just what we looked at what he looked at what they looked at and what the report provided the whole thing to regard the

thought of actually you know taking us together finding who interests our interest who's who's interested in learning this this thing who has the technical ability you know and and you know started our training so yep so the desire to learn were we interested after he starts showing us the cross-site scripting he started seeing that we're really really curious about things cuz we wanted to start seeing more we were looking at our own personal lives and we were looking at sites that we would frequently go to and be like hmm I wonder is this site really safe we started asking a lot of questions we didn't do anything without authorization of course but we started looking and by

looking we started looking at our own internal sites and we started with passive testing that way we weren't disrupting anything but we were gathering information we were learning from passive testing of course you're maturing the model and you move on to shadow tester and so forth also do note I forgot to mention and I do apologize you'll start noticing that there's a little magnifying glasses on certain items those are takeaways that we do have so somebody had mentioned that their presentation will be available hopefully for download afterwards and we do have takeaways so anything that does have a magnifying glass we have also put as a takeaway okay so starting with our training we were

looking at dvwa which is free we started looking at different daily incidents that were happening how what was happening with these sites and what would could we learn from it so we received a lot of homework but Allah also along with with reading Lee army we needed a couple of weeks of dedicated training time so if this isn't happening in your organization or you want to add additional training to it it does take a few weeks of dedicated training yeah and I mean most of you guys might already be aware of dvwa a month in a day the whole exercise of actually putting this together in a VM you know you when you give these challenges to your to your

team like teammates you know it's it's a technical learning experience each time he or she is actually going through the challenge of you know memory management within the VM you know Java is a memory hog different different applications tools that run use Java you tend to know that you need to upgrade your RAM you know allocate some ram locate more memory to Kali for example so so you know these day to day challenges that that you face will will will give you expertise and and you know you gradually go on increasing your your experience on on how to use these tools right we also realized that once we went down this rabbit hole there was no going back so

again once you can and see what you we've seen and not only did a piqued our interest and we are still here doing it but we've realized we started recruiting others and wanted more individuals ramp up and get into our world one one thing I would like to highlight here this is the technical skill assessment you guys can have it in your organization as well it's a simple sheet you know some bunch of questions that you send out to two individuals to do understanding interest and their experience you know they they'll come back and let you know that they know this probably they know incident handling very well they know network security very well they know web app and

test very well you may not even be aware that there are there are teammates who know about this and then you can you know ask questions based on whatever they have assess themselves on from there on you know get into the interest desire to learn you know putting in the hard hard work effort and giving them challenges day to day exactly so we started with maturing the program so we started again with a lone soldier who's doing this on his own in an organization that 99.9999 percent of all web app testing was outsourced to one person doing it for his area and now what we have is in a large organization we have a small team that is getting recognition

we started looking not only at our own internal compiling an internal list of all of our web applications we started doing planning sessions but we started getting recognition senior management was starting to take notice because they were looking at the dollars the dollars were other groups had huge budgets for pen testing and our group had a very small slim budget but then they started thinking well why do they have such a small budget and yet they have such a large list their list wasn't smaller than anybody else's maybe the same size maybe a little bit bigger but let's say the same size but our budget was very very small and yet we were completing these tests so you've got senior

management out looking at you which is a good thing and a bad thing because we kind of like to fly under the radar I'm quite sure most individuals here we're in security and IT we don't really like that recognition so with that we started realizing that we also needed to gain additional information not only did we have our POCs with our testing we had our high level details we had a report we also start seeing ok how else can we report this to ourselves how else are we gonna start planning for our next test because before it was just Hershel and I yelling at each other across the way it's going hey you know I got a test coming next

week you got capacity yeah sure sounds great to me let me start realizing we got to be a little bit more organized because this yelling back and forth doesn't help when you've got a multitude of other things this is only about 30% of our allocation we are actually also working on other work and we'd start forgetting so we thought we need to really clean up our program and mature and now we've got other people who are interested so we need to be able to track them as well again on the inventory of applications you will find using very tools what I just want to highlight here a few of them that we use I was Rick on ng to do

a DNS lookup on our on our domains what web I don't know if you guys are familiar with in Kali it's a free tool I witness to actually capture the the the website screen prints what it does is like all these tools together we put up in a script we created automated scripts so that they can go each time and you know grab the results you know Linux you can put bash scripts together and what happens is with the eyewitness you can actually have a full HTML report which which is very good for senior management like if you're sharing these reports reports with with business units to to let them validate the websites that they're they're having so you know you

can use these these are all free tools like you don't need to pay for them because we're all about free everything is frugal everything is free there is of course the little bit of cost which we will tell you all about but we're all about free which takes us to money see money everybody loves money we also want to keep it we don't want to blow it away or we don't have the authority to blow it all away because it's not ours so Kelly Kali's free why not use it laptops everybody here I'm assuming has a laptop personally and professionally quite sure if not really need to get one it's amazing burp license purp license it caused

about $349 us for the year and that's her professional that equates to about $450 Canadian at this time that was a couple days ago I have no control over the exchange rate so I do apologize if that changes um but that's pretty cheap right we're talking about frugal here next thing you know we need one insure had Hershel hadn't mentioned about ram and memory we learned the hard way ensure that that is upgraded we've had many times where our pro machines just crashed and just could not execute a test I'm sure you do have capacity for that so we also realize with time we had a quicker turnaround I mentioned that I would just yell over at Hershel Hershel

would just yell over at myself or on a call and somebody wants a pen test and we scheduled it pretty much right away or turnaround time we make it happen now you're talking about outsourcing this to a vendor you've got them insure do you have a contract are you able to do so do you need to approval for this when can it be scheduled then you also have to think about cost now how much is an vendor charge for a pen test I'm buddy have idea sorry three thousand eight thousand yeah right just an average Joe yeah yeah bending yes okay what about that yeah so again that was going with around our numbers of our contracts that

we were looking at and that was for an automated test it wasn't for a manual one for our contract so we're looking at cuz mama this is our journey and then again you know if the site fails that's a retest now also realize we had one site that failed seven times now I'm not quite sure that's normal for you guys but we have another one going head to head with it that was about six times and we were trying to see if it would actually beat it so if we were bringing him that money we'd have some nicer clothes here while we were presenting and probably real money be throwing at you so this one everybody knows but the security

testing methodology lifecycle we're not really gonna go through this in detail but if you don't know slides I was told would be made available but it's basically gonna go from inception you're planning you're talking to somebody gathering some information you want to find out you know what does the environment look at then you're gonna start your pen testing you're gonna get your findings you're gonna start reporting it getting your POCs and then you're gonna have your walkthrough don't forget your walkthrough you can't just hand over your report and say your site failed sayonara bye-bye they don't really like that we found out they actually want to know why they've failed because they want to remediate it or so they say these are

the webapp pen test tools that we frequently use we're not here to market work but we went through a bunch of tools testing them and we felt that burp is the one that we will use it has a bunch of extenders sequel map is free somebody said it's just like the safes security party-line is free easy quick so you know whatever is free bring it on Supplemental accessor you know I don't know how many of you use accessor we tried doing a bunch of experiments with accessor right to load payloads it doesn't work right now I don't know it's something to do with the tool but you guys can use it nikto gain an attack proxy and Ava's zap is

again an attack attack from C the only reason why we like work is the extenders they are amazing I will go I will get into detail for a couple of expanders well III and I would also say the reason why I also like Bert besides the extenders is I just like the look and feel of it I don't know why maybe I'm a girl OSI wasn't as sexy for me didn't really like as much so the look and feel did not work I found that just not something that I really would like to enjoyed using as well these are a few extra nerves that we use really handy is the logger plus plus why we say that I will

come to that but it captures all the logs along with the burp history version reporter software version reporter and vulnerability scanner these are the two new ones that we have seen that come up what these two do is like they will look up the version of like for example web server running will look for the CVS and then report back to you guys again CSRF scanner active scan plus plus co2 co2 is very handy and I'll come to that soon and these are all part of burp suite professional so again you're paying that one-time costs a year and this is all included you just need to go into the little store inside of it and you're not

paying additionally you're just downloading it so that is available for you if you're not aware so again frugal so we have a web app penetration test report that we end up sending to our clients clients always want to know first thing we get emails phone calls right away even before the ticket closes how are we doing is our site failing what's going on is it passing are we okay can you guys close it that happens to us quite often but what we do is we can't just let them know verbally we also like to send them a report we keep this for our own metrics senior management also likes it but we really really like it because we like to know

what did we do so we can go back at the end of the year and tell our boss and say look how amazing we are but we also like to give it to the clients so that way they can see it as you can see I think you can see it it's pretty basic it tells you the company um it gives you the URL the dates and we encompass the OS top ten not 2017 but this is the OAuth top ten that we capture on here and we capture it with lovely color coding you know right away whether the site failed and you know hello okay we're the finding was sorry that's went away and then we also started realizing that

we had a few company specific ones that started popping up so this is part of our maturing model Herschel was testing and he started finding some cores he started finding some hosts had her poisoning he was getting really really excited because it wasn't the first time he saw this so he was like you know what I really think we need to start demonstrating this tour mentor as well as to myself he's like we need to track it I think he want to track it for his own different metrics reasoning but we also like to chalk it for senior management and then we of course put our little notes because that doesn't always come with the Olas so what else did we

find what do we want them to know about and going back to writing the army we have our testers we have our passive testers we have our shadow pen testers people who are training we needed to track them as well now we're lazy we don't want multiple places that we want to be tracking all this information with will uncheck it in one place not saying that the client receives that information we of course crop that part out but we have one document where we're able to track all of this information and it is locked down a couple of things that I want to highlight here is the business logic failure one of the pen tests that I was doing here it was a

kind of employee directory and the trick was actually to go to the CEO and then you will get a dump of like everything so it's it's not it's just tricky you know sometimes it's it's has nothing to do with pen testing is just the trick that you put in so we wanted to track that as well and you will find that in your applications as well the configuration parameters here I Corps I this is a question to you guys like have you guys ever looked at get lab if you guys are using it how many of you use good luck within your organization so did you try running some searches within the get lab code like you know are some

ordered credentials for example I'd never mind a lot so you can use within your databases we found a lot we normally go shopping there yeah if you want to find the passwords so we also have a framework so again going with the maturity model we need to not only review pen stirs in our own org in our own group of how are they testing are these two findings how are they testing it but we also have to review external penetration tests reports that come in so I know quite a few of you here said you conduct your own quite sure you also review external ones chopin's yeah just a handful okay so this would also help us

so what we have done is just you know we didn't put all of them up there of course that's way too many the slide would be too busy and we'd be here all day but we decided just to put a four four of them up and cross site scripting reflected easy you know what did the tester do did they put in a payload was it reflected back do they view the source so just the steps that we would go through when we're talking to one of our colleagues yeah the idea was basically to actually come up with certain set of questions for each type of finding that you that you have like if you if you're using burp and there

are different type of findings that are listed you can have a bunch of questions to validate whether the guy has actually done his POC what I have done here is I'm just showing you guys that how can you leverage burp with other free tools available this is a repeater window of burp to a dvwa I don't know if you were able to see it's a dvwa link URL so you basically send it to the intruder what you do is this was what I was talking about co2 it's an amazing tool if you like command line a lot and what you can do is you can configure the parameters in co2 the whole command line is actually generated in the first

highlighted box you can set the options in the tabs which is given below if you want to do enumeration databases banner or what you whatever you want to dump and then use your love you know command line just put it in sequel map and then it will run so this is just leveraging the burp expect extenders with the free tools what I have done is I have put a link to payloads which will find in the references this is a dump of like four and a half gig of payloads that you can get and you can use it with with burp here what I've done is again the repeater window what I'm doing I don't

know if you were able to see this was too tiny about what I'm doing is there are excesses payloads within within within the folder that I just showed it's probably somewhere first DB sorry the is the first DB that I used and you can use XSS payloads loaded run it with the intruder and identify the string like you know with the grep match and you know it runs so there is a four and a half gig of database which it's really a little not made by me at but you can you guys can download I have a link in there and she's to blame I'm to blame um incidents um has anybody ever been blamed for an incident yay my people

that's great so we do have a pen test that we let individuals know that we will be pen testing but you know something goes wrong along the way wasn't us and the first phone call comes to our team our poor mentor he gets a phone call or an email and he turns around or boster and sounds like it's anybody testing now and they look at us we've been blamed for what taking down a site servers Oh network connectivity applications so when Herschel mentioned logs and he said we'd come back to that later that's when during the maturity model we started realized saying you know what we've really need to keep our logs because we need to clear our name we

didn't actually do this this wasn't us but nobody well of course would believe us if we said that I don't know why I think we look trustworthy but they don't believe us for some reason we have a wrap so log slugs are evidence always retain your logs we start looking at logger plus plus that is an extender within burp and we started being able to see that do note though if you close your burp state it is not saved when you reopen it you must save that each time before you close your births tape we learned that the hard way but the good thing is that you can run it through a converter and you can I produced that as

a CSV and that could be sent to over so again that was part of our maturity model and I think we still keep getting blamed for things and that will just continue for us this is a tool that I love this is again the link will have this tool it's a small database what it does is generates a report of for the management it's not really nice but you guys can use it if you want the source is open management thinks it's amazing though so but the database as well as the PSP report is available you can do sequel injection to this it will work the good part is um we also use it for

tracking who's assigned to each test you don't see that here because management doesn't need to see who's who's assigned each test and who's actually executing it but it is in the ticket and on the backend we actually track all of that for ourselves this is the logger tool that I was mentioning if by chance you don't have logger plus plus history saved burp has a history if you have saved the state sorry so if you have the burp history as birth stage save and you restore it what you can do is there is a tool called burp history converter I have that in the link again it's a takeaway what it does is like you can

the first section you see is the is the CSV converted file here and the second section you see is the XML so burp in burp creates the XML but if you want to convert it to CSV you can use this handy tool so what we would like to tell you is which we should have also told you how many how many individuals have sites that are hosts on so force Oh only one okay I don't know if you looked at your contract but you're also able to pen test that so definitely look at your contract but also look and see if they've deployed Akamai so um so definitely take a peek at that we do

have some Salesforce sites and within the contract our lone soldier was able to realize that we were able to test those okay and Herschel was talking about takeaways and that was the magnifying glass so we do some of these are things that have been internally developed and things that have not such as the burp history converter the dvwa we just want to make sure that individuals know if they don't have this already or or if they do but they need to leverage a little bit more information or want to do some compares are more than welcome to go and check it out the link is at codex PETA's this is the one where you can get the ones that

we created and the rest of them are here I'm sorry any questions for you we don't get away with anything yeah kind of so we always need to make sure so we have the two different logs we have the log or plus plus which we implemented afterwards two sets of books because you got to cook the books right um so uh no we realized that burp does we always have our state saved so when it was the mad scramble of clearing our name we needed to make sure that we could pull our logs from there but then we realized perp has an extender for logger plus plus and that was way simpler it's memory we want quick simple easy fast

and free I think we don't we just and we have one minute so we are getting kicked off the stage if you want to talk to us some more there's our stuff right [Applause]