AppSec Tales of Fails (and avoiding them)

Oh and we have another shortage one here but no less interesting and failures a knapsack importantly with poison again things grow in complexity as time goes on I think everyone would agree that there's no such thing as a non-trivial secure piece of software and trying to do what's best in order to think security in the programming process it's extremely important with that I'm gonna quit yammering up here I'm gonna let Rebecca take the floor and let's see what she has to say and I've got to fit into a very short period of time so I will do what I can but my Rebecca got a new application security testing for a new consultant for a

company called defense and all I know is that software so web apps mobile app stick clients pretty much anything I will and I hate what I have to do a bad job but sometimes the tests are set up so that you really have no way to succeed and I had like three or four tests in a row back in December where this happened most like man I wish I just had a way to tell these whites if you just do some different things we could have had a much better test cuz I hate watching people waste money so I thought well I'll put this presentation together and hopefully be able to point people to it

especially if I notice they're sending me down a crappy test Road and maybe I can fix the problem so hope all trying to love us up a little bit with what is application security

for about nine years it's been eight years as a software developer this initiative responsive forensics working there and I was also in the infantry in the Army for four years so personal application security and penetration testing are two very different things so my job is an application security consultant is to find new problems with a software right so I take a piece of software analyze it in whatever the appropriate method is develop exploits then on maybe I'll write Python scripts maybe our Venice wind modules and pass those off if we're doing that pan engagements then I'll pass those vulnerabilities and exploits off tooth and net pentene and then that way they can use those while they're attacking

somebody's networks but we're really only looking at one tiny piece of this big puzzle when we do that application security as opposed to that pen people who are looking at the entire network right so you have to do you have very different approaches but a lot of times the people who are running the test on suicide are the same people that want penetration tests and so they go in through with this mindset about penetration testing and that's over sometimes even cold I use with application penetration testing versus brother buddy so water so that's one of the big rooms today in the dermis taco work further than any internal back testers as third-party testers I used to work as

part of an internal team robota doing application security but you function very much the same as you do as a consultant right project teams they come to you to test the software you deliver a report you have exploits maybe disclose things to vendors have nothing so first off when do people start doing the application testing that should not be the first thing you do that location testing is so narrow and focused and it kind of involved some attacker knowledge of this internally development application so the odds of exploitation are significantly lower then you would have with most other exporters so first look at something like the critical security controls where the Australian signals Directorate controls those are

great places to begin and give you very clear steps I have an insecure Network how do I go from that and having a secure network so we're like number 17 or something so you have a lot to do before but there are some times when we shouldn't have software earlier so for some if you plan on selling that software you're asking net consumer to accept risk under we have right so if you makes a software and somebody is going to put their data up in the cloud and it's your environment if you get half minutes their information that's beautiful breather works if you like my least favorite things are important agents because I feel like by a motor complete

garbage the endpoint agents are some piece of software in the most certainly

to assume risk on all their influence through your vulnerabilities so I get a lot of mileage on people who want to sell that software for that reason there are clients if you don't know what to do maybe you come and ask me but you should probably look back with medical controls or some other framework before you do that because it's going to save you money because expensive so when I was a software developer the way I need to jump the application security was I came back from his hands fast I was like oh my god click our software's complete garbage but when I told my boss he didn't believe he said well you are all

developing secure software right [Music] one week then I'll come up with a presentation so that we have found that exploits all over our application remote code execution sequel injection cross-site scripting vulnerability oh my god I can't believe that our application is a live garbage but I was able to leverage that into me spending all my time it's true but yeah so sometimes proof of concept really fails so I've got a test before Nick I need a boner though these hands on their system and they wanted to all this in-depth testing on this application they bill well it was running Oracle WebLogic and I've done bad things how many people have had to work all that logic because there's

always zero days for Oracle WebLogic but here I was don't have X with the ruling for five years one so that was kind of a waste of money right you should probably be doing good owners only management first it will work a lot better for you so don't run into these things prematurely I had another test where they want me to really do all this work on their web app and the web app is running on Windows 2000 system because publicly exposed with me being beaten maybe this is not where you should be disappointing your security process but sometimes that's kind of where they need to be enough in-house expertise and you ask somebody else but before you go talking to a

third party about this what do you want to get out of the test should you just need a letter saying I have had somebody look at this and they say the these are the vulnerabilities of it that's up balancing you don't be in-house expertise but again it's probably much there's cheaper ways to do that or you can go buy a stool or something to stand around but I'll be more than happy to take your money finding all the bugs that's kind of what I like to do we're gonna be leaving stones unturned and software running out of time getting down that all the loan that knowing things because some mentoring with the changing abilities together let's show some

really cool exploits for the lower ones but again that's not right for everybody so maybe people are like look at just one of the big problems but knowing what she wanted me to go into the test is a big deal because otherwise I'm likely to spend a bunch of time working on something you don't care about and again next you need a Steven work and it sounds kind of stupid but I take my work with the most software development shops and I get sent on a software that won't even run so I had one test I got a start-up but it was like $150,000 application and they found vulnerabilities in the application because it never went to the buzzing

talked earlier so we do faucet and I've tested apps for wear anything that's malformed input is always a branch the entire service so when you send now let's enter a higher goal is to send out for min what it's a real problem if that breaks your application so at least knowing that I can like bond instead of an a right like that kind of level of testing or is put a seam up over there that's so a lot of times I tell people to development integration test and you will be space tested types of talks on that I've been and that's a really good way to make sure at least baseline level of functionality greengage certain parties another

difference between penetration testing and access this it's okay for me to not tested production penetration testers they need to test fraud systems because only the cross system it looks that exact way it has real users with that kind of thing so you do want that anything in scope but I don't want to even touch a proclamation if I can avoid it because I break stuff all the time right we're just doing buzzing all I do is send out form and book into your system so people come to me with questions like well why would you send them like that's not bad so I was doing test of some hotel chains and it started out testing their production system but

if you have any of you who have reservations work when you start reserving all the rooms at a hotel the hotel games but it gets really skittish you start wondering if there's like a conference in town or you like white white back to my food book the rooms my whole small so that was a problem you know that's a good idea not fraud incense and little did they know their non provinces talk to a third party systems production that says then I send a malformed time something in a micro so instead of one of the ballot Greek character time zone time Joe I said like a sequel injection camera when it crashed this third party production

service so you can really break a lot of stuff so that's why I always tell people for the love of God do not send me a picture production system because I will kill I did have grown up up test a couple weeks ago and they have these like optional arguments in protocol and I said the first logical argument crash the service second up to her mid crash the service so they just kinda finish the code there were no error traps on it which is good to know but again I was glad that it was not in production which kinda brings me to white box black box in grey box testing so when you do white

box testing like you give me the source code which is how I love to work because again I learned like all the problems and I'm going to do it as inexpensively as possible and in order to do that if you give me more information I can give you more vulnerabilities because I keep giving getting at the end of the test I'm just like there's swathes of this application and I know we just couldn't even touch so black box you give me no information so I've had people only do black box tests on their applications and they like require authentication or something like that and when we get into the snowfall talk a little bit more about why black boxes

Lily's great box of people French units at Scripps I'm really loved because I my buddy whatever my customs protocols so you feel like build out this grammar or you putz so if you can give me that past week I can build that out in a matter of minutes and then I'm just picking things up if I pull it but some people don't want to know right that's a valid thing where you have

but again burning piles of money our scope is really limited like I said only maybe like a single port on a single server is my entire scope so I had people before and again when you test them on production this becomes a problem so you might not even happen shewill users of these applications so this doesn't happen some fluid ran over HTTP the company knew it was a bad idea they're like well can you intercept and network traffic go out here and steal credentials it was like there any users and they're like well there's no users the application that may not be the best way to approach I've had people want me to go sit in

Starbucks on the off chance that someone will be on the Wi-Fi and connect to their system Oh creepy no that's really not a good use of my time right again I'm more than happy to go good for it but I wouldn't recommend it if you're trying to get something out of it we're going to get pensions they do fish that makes sense they're trying to get into these production applications so that's probably a pretty good one like it's a decent way to do Network does not happen and I don't really find a lot of unpatched machines is typically when people roll out software like that is the most secure point and that piece of software right right sometimes you have

new exploits come out but not often right the definitely the the kernels fully patched all of their like services they're running so at least I'm not gonna find these kind of bats the unpassed of ex-boyfriend decide pop things with no information there are exploits it takes me a long time to write an exploit so my dissing testing on IBM cognos solution a while back and it had a a proprietary binary protocol for its off the engine well first off I crashed that service a little hot so I wrote a fuzzy the script for it and I thought I think and was buzzing the application kind of like the native crafts under these certain circumstances well later when I

first had to capture the traffic then I had to analyze that traffic then I go right up cause I said it was like one to two days Center the buzzer catch a wrong turn a half day to day with more certainty good crashes then I have to go in like when debug and analyzes crash dumps so that took some more time then I can figure out exactly how to write this exploit exactly what was possible but it come with maybe for five days just never developed with single x point so it's gonna take me and you don't want me sitting there wasting days just ringing guests what's going on in their application so give me

all the information you can and I guarantee I will do a much better job much faster then again boss exploits we do penetration testing got a pretty good idea this exploit is going to cost this system I don't know that I'm just a stuff with this application and hoping for the best but when things like airplanes stop working like they want to in production to airports what I do for hotel chain so give me everything you can so the dresses for the web applications any API seven views even internal api's so I did a test that involved like some probably the application and fortunately they sent me like a diagram of their system so they're right box test and now

they will saying hey look I've got these api's in here what are the API endpoints on these just kind of work I know that they're supposed to be internal then at least let me take a look and see if I can get to them ensuring a high pub and they were like such benign stuff it's like arbitrary sequel queries read any logs in the system so big important things like that and this was up at AWS so I like Amazon Web Services so it's up to the trial now when I'm doing this I don't sit there and scan the entire fly out on the off-chance I've been a bump in your API and so this up consumed by

the application I'm testing I may never find it so that's why it's so important to give me all that information and even if I'm white box sometimes people want me to go like reverse engineer how to call your API based on your rails code I'm like I can do it but it's gonna take me some time to figure out what all these parameters or what they look like and again I want to do it as cheaply as possible external services if you can tell me if did you use octave and by the way doughnuts have Noctis systems they give you using their software it's not give me rights to attacking right you don't have the privilege of letting the

third party people so you gotta be careful of external services dangerous functions I was testing the bank account the banking system we need eight or nine months ago and they had this search page on there it apparently the search page the single credit so it can I run one search for any kind then again it had to be done in production so while I was testing this piece of software there was not one bank customer that could actually run queries against this in production which costs a big bubble great so all kinds of like fighters were going off but if you tell me hey look this is going to be a problem at least I can like realize

into very methodical dangle as stableman dangerous pieces things that may cost you money to run you may office right so I was messing a piece of software and it generated emails that I didn't know about so I had sent something like 15,000 emails so that helped us and everybody was just wondering that's the default things like that had another person who when I was part of an internal company so we did a lot less coordination they come up we said hey look we have to hold this phone system out can you just in like two hours take a look at it so I'm like kick in doors and on this application login with admin admin I run verb spider and in kind of

thing on there the reset it to factory default when you ran and get requests person so I read sent this entire phone system they asked me hey we have a presentation we're supposed to give like a demo in two hours why did you tell me that's going into it so I could be careful for fifty million dollars and then I find you a bunch of vulnerabilities in it and then you saying hey we're not going to pay you fifty million dollars we can admit you like thirty million dollars because we know we have to rework all this you know that easily pays for my maybe if you're gonna get it for selling me out if you have a bunch of apps and

you don't know which apps are good and which apps are bad I've heard of paper a couple of years ago that's something the sands reading-room about finding bad applications so you can do something like that or if they let me do it one word of caution if you have like five out maybe just let me decide how to spend my time we have met people who tell me you have to spend exactly one day for our application and you look at the first half like oh this is awesome look at steal this data and then you live in the next app and it's like fully passed out look like look the awesome me finding anything on your I'm looking stall on

Monday are probably pretty low actually if I can't do through force so you know just let me decide where to spend with kickoff and testing if you ask about specific tests so if you know that I want you to test our CAPTCHA install that give me summers spend my time if you say we allow users to upload Java serve with the surfer that's run on the server side that gives me stellar to spend my time trying to bypass protecting the team innovating I like to give people information early and often be aware over coordination so you want me to sit there watch you on the phone for an hour a day I can do that but I

will do correspondingly less testing and if I have to send you the email saying by pressing the system all day if I have to switch over to another system part of the day it makes me for my lips to do that so maybe go from that level after you do testing do write regression test for these pieces of software pass the vulnerabilities against down the stupid but some people think reports in this bottle away but network is not the same as that pack we have a really tiny soon so give me all the information you can because it will be funny vulnerabilities right so you try to drive that cost per key bank down so

these are a couple ways to do it hopefully it was happy or at least and are fun for people or at least a little under oh yeah that's pretty much it what questions do people so overall industry does so bigger companies that do a lot of software development personally I like DevOps shops as I'm sure a lot of other app tech people will want to like when I work with dev op shops I found vulnerabilities and they were like patching them as we go so as I'm like turning a result they're catching the stuff in live on production and it's not breaking everything which to me is amazing but if anybody want to talk about like the process step off

people go through that's been kind of my data for a while

harder correct issues or 270 I think people accept them quite readily I've done some other test work and really do some damage Financial Services and there's a lot of problems with a lot of things anything that was written before 2014 is probably going to be really bad like just from an architectural perspective so that's kind of one of my recommends I'd say that they accept it and then they accept the risk and move on with their lives because it's just this giant turd

approach be their friend so every time I'm talking somebody about this earlier and do a lot is DLC consulting and the biggest thing is to be very patient with the developers because they have a lot of pressure the security people are framing all your discussions with them being like sensitive children you don't want to scare them away because your your job is to call there may be ugly but I can talk with people out in the hall if you want about some strategies to do that

[Applause]

you