John Heasman - Doing More in AppSec by Doing Less

security teams mult multiple versions of application security teams which is what we're going to talk about today and I've been the ciso of a public company for 5 years now now during that 20-year career a lot of the time I've spent working with Silicon Valley based tech companies that put a real high premium on high importance on security and put their money where their mouth is frankly and during that time when I've given presentations or been on panels talking about absc what I didn't really realize at the time was that I was speaking from this kind of gilded position of having budget and headcount and room to experiment to figure out with an application security program what what

really works and and what really doesn't right but I know a lot of people in appc are not in that position you know maybe um you're just starting out in absec starting out the program at your company maybe it's a a program that's existed for a while but you're really struggling to get Buy in from from engineering leadership or getting push back from your engineers um maybe you're just uh completely confused when you see the Myriad of things that uh we tell you you should put in your application security program so we're going to tackle some of those uh topics today know a lot of application security presentations will will tell you uh we'll talk about

various uh Technologies and processes but they won't really tell you what you absolutely should do and what you absolutely shouldn't do so I'm going to try and take a a bit of a stance on

that I want to start by briefly discussing why we do application security at all you may may think there's a simple answer to that simply we want to make our software secure right but this is uh let's come up with a mission statement this is my application security mission statement um let's dissect it a little bit firstly we want continuous Improvement we want that in all aspects of our information security program application security is no different um tools technology processes our understanding it's all constantly improving and we want to take those improvements and feed them back into our program so continuous Improvement we want to reasonably minimize security defects um the lawyer in me made me put

the word reasonably in there right that's the acknowledgement that we don't have infinite sort Budget on on application infinite time or money to solve the problem so really it's a best effort with what we've got and then we want to uh do application Security in a consistent comprehensive and measurable way so we want to be consistent we want to run these processes over time and get the same quality of output each time we don't want that to uh vary wildly because it might mean we then uh miss things we want to be comprehensive we want to make sure that this covers all of our critical applications perhaps all of our um applications in in its

entirety uh and we want the program to be measurable we can't do continuous Improvement unless we can actually sort of measure how effective these processes are uh and feed that back in so this is why we do appsc but how do how do we do it well no application security presentation is complete without discussion of the software development life cycle the sdlc and you'll see a lot of sort of riffs on this but uh this is this is mine and we've got uh these six different phases here so uh it's pretty straightforward let me run through it real quick those who are on the same page applications typically start out with uh product management writing some

kind of uh spec or uh um defining the requirements of what it is that actually needs to be built and then we enter the design phase which is normally a collaboration between a few teams engineering product management um if it's got a um user experience then then ux will be in there as well and this is really the the phase where we design the thing and then the fun part engineering go off and uh build the thing and then uh we need to test the thing that might be engineering or it might be a dedicated uh quality assurance team and they may do manual testing or they may do automated testing and then we actually get to

deploy the thing yay the thing's deployed long live the thing and then 5 minutes later the thing is down and we enter the perpetual maintenance of the thing um but I mean this not we're not considering security at this point this is just um how Engineers would typically look at the sdlc right so how do we um how do we sprinkle on security let's sprinkle a little bit of that uh security magic so my first thought when I put this together is this is a lot of stuff right do we we really need all of this stuff to secure software in 2024 um I'm not going to go through every one of these uh but I want to call out that

really you can see in under things like development there's multiple ways to do things and you if you watch a lot of our application security presentations or readbooks where they they talk about these things then you might get pros and cons but what you don't really get a sense of is what do you need to do right there's a ton of stuff here if your application security program is just a a people relative to the size of your engineering org what what how do how are you supposed to look at this and decide where you should focus you know it's interesting an interesting metric to think about is how many application security people you might have at your

in in your infc org compared to the size of your engineering org and uh realistically if you have anything less than about 80 Engineers to one absc person you're doing pretty good by industry standards right often it'll typically be more towards like a 100 could be like anything you know could even be north of that 150 people hell maybe you're the only person working in ABS and you've got a whole engineering or so when presented with this where the hell do you stop like how do you like what what do you pick what are you going to double down on right when everything seems to be important so what we need to think about is do we need to do it all and there's

three kind of overlapping lenses here uh the first one is an adage that you'll get in every management book you'll also get it in every Engineering Management book but strangely it doesn't really come up in infc books and that is if everything is a priority nothing is right I think in many ways we like to think in infc that we're we're kind of special and everything truly is a priority right but this this um this saying is is so true you really can't focus on everything you've got to be intentional this second lens also from engineering is from a engineering blogger called uh Jade Rubik that talks about uh different personas in engineering to speed up the

velocity of an engineering team and one of these is subtraction ISM it's essentially just asking constantly do we really need to do this what if we don't do this what if we do this later what would the impact be so really like getting you to think about tradeoffs and this last one here is from an information security blogger um Phil Phil venes and he talks about knowing when to go for an a grade versus knowing when you simply need a pass so putting all these things together we can really kind of start to think about Technologies and processes as right we want the a grade we're really going to intentionally do this thing and we're going to do it the best

we possibly can or we just need a pass like maybe we're required to do this from a compl client perspective from a regulatory perspective we have to do it right but we're not going to kill ourselves doing it we're just going to do a best effort get the pass accept it move on and then lastly there's going to be a whole bucket of things where we don't have any real reason to do it it's not going to maybe it's not going to be a good fit for our organization so we're just not going to do it all right and in a nutshell that is my talk you don't have to do it all despite what

application security um bloggers and um books and presentations will tell you you don't need to do at all so let's go through a few of these things um going to do a Whistle Stop tour actually I should have said this at the start but my my Talk's in two parts first part we're going to do this Whistle Stop tour of a bunch of these tools and Technologies and processes going to share some thoughts on them and then the second part I'm going to share my version of how we do apps SEC and we're going to layer on these um processes so application uh inventory um firstly um a truism here you can't secure what you don't know

exists I'm sure we've all heard that before um in fact if you were to um poll a list of cisos over the last decade and ask them like what was your number one initiative it was probably something around visibility right just understanding like what we even have to secure so I'm a definitely a big believer that we need to know in absc what it is we need to sec here in the olden days when uh cavemen were writing um code on stone tablets uh and essentially we were building monoliths uh who here is familiar with the the concept of a monolith in application security okay a bunch of you for those of you that are not a monolith

is essentially a giant slabber code that has everything for an application to function in in one Reaper it's uh going to be the presentation layer the business logic layer the data access layer some giant cbase with tons of developers working on it now monoliths have fallen out of favor pretty dramatically over the last decade or so um largely because of the perception that they're really hard to maintain all of these Engineers working in the same same code base is a nightmare um ownership is unclear when you've got so many people working in there it's um release Cadence is slow because there're so many moving parts that you have to get right before you can deploy it um

now arguably some of those things have been solved for example in a repo with the code owners specifying ownership you know it's a really nice way that you can uh control who's um committing to that repo but um actually monoliths have one real major benefit which is you don't have many of them and you know what they are and you know where they're deployed and you know when they're deployed right so that means we can use Stone Age tools like Microsoft Excel for tracking our monoliths and um it it's it's not a crazy idea because these monoliths are not changing dramatically over time so keeping track of them on an Excel it's not immediately out of date the second

you write that Excel like it's it's it's going to be a pretty good uh source of Truth so that's um monoliths so in the old days keeping track of what you had was actually reasonably easy but enter microservices so as we moved away from monolith we we came up with mic Services where we essentially decompose different parts of that application into um domain specific services so you can see here on the left this uh picture I stole of uh microservice architecture where we've got uh you know a handful of microservices doing doing different things like the concept makes a lot of sense right these services are owned by individual teams for example a team owns

the data access layer and other teams owns frontend code and other team owns different parts of your business logic makes a makes a ton of sense but somewhere we kind of lost track of uh why we were doing microservices and we ended up with something like this predictably so we ended up with this nightmare scenario where we have not dozens maybe not even hundreds thousand of microservices in in some cases with uh and and and mapping the interactions between these it's a complete mess ownership of these in a lot of companies super unclear um and actually there's a really interesting quote here from the former CTO of uh of GitHub who basically says like we've we've done all this work to

move the microservices but honestly most things could probably still be a monolith in MO most companies so it's a classic case potentially of like Engineers like over engineering right so we end up with this nightmare scenario for um of microservices and then how do we keep track of this so in the microservices world how do we actually know what we need to secure um well that is actually not the question you should be uh asking the question you should be asking is who should own keeping track of your microservices right now think about the the orgs at your company it's obviously not going to be legal HR Finance coms anything like that really the only two

options are us infosec or engineering and I'll give you a clue it's not us your engineering org as as microservice is take takes root in your company your engineering org must be responsible for knowing what you have right and that needs to be more than just saying well go look in AWS at uh eks or ECS and you can see what we have no we need a better we need a better list than that need a better inventory than that right we need to know what what engineering Squad owns a service we need to know where the code is we need to know where it's deployed what environments uh we need to know where we

can file uh tickets for this thing we need a whole bunch of metadata and Engineering really have to be the ones that own that now infc we can partner with them it's a great opportunity for infc to be kind of stakeholders and demonstrate collaboration and all that good stuff but infc should Absolut absolutely not be the ones building it so my advice is um if you're struggling with that in your company go have a conversation with your engineering Leaders with your CTO and um uh Pro tip if you can make it make them think it's their idea then obviously you know you stand a good chance of success there um there are it's it's a solved problem

from a technology perspective there's uh commercial Solutions and there's really good open- source Solutions hat tip to an open source uh project called backstage which was uh created by uh Spotify for really having really great management of of everything that you have okay let's talk about something uh different let's talk about training um who here quick show of hands who here plays video games okay lots of you uh who does uh who plays ctfs okay still still a good number um who here likes uh mandatory HR and compliance training you sir a unique individual um now what if we took that mandatory HR and compliance training and gamified it you want to play that game no one wants to play that game

you're not that jazzed about it here in lies the problem with gamified information security training right it's Affinity bias we like video games we like CTF we like infc therefore we're going to push gamified training on you gamified security training Labs on you and we're going to on on engineers and we're going to expect our Engineers to enjoy that um Unfortunately they won't or at least a large percentage of them won't now some of you will probably thinking but we rolled out this gamified training at our company and for engineers and it was a wild success all the feedback was super positive right but the feedback you're getting is from the the people in engineering that small

percentage that like playing games and likeing for SEC and probably don't even need to learn anything from the training because they know it all already right you're not getting the people that really need to understand that knowledge to actually uh engage in it so uh it's a hard no from me on gam ofi security training how do we do uh how do we do training well um firstly be guided why you're giving the training so think back to that quote from Phil venables of do are we doing this just to get a pass like do we have regulatory or compliance reasons like PCI DSS uh for for example you got to train your engineers okay we have to

do this right um or or is training something that uh maybe organizationally uh your your your company is used to training and and loves it and and does well with it so maybe TR maybe training is the thing that you want to double down on and be like we're going to really put a lot of effort into this and make it great um if that's the case uh how do you how do you do that well firstly uh know the tools uh available to you at your disposal go talk to your HR team you probably have a learning management system that you can uh you can use um your your HR team may be able to help you with that they may

even have resources or budget to help you with that so have those conversations could help perhaps with um tracking completion and all all those pesky details that we hate to do as a infc um which frees up your time to them really focus on creating contextually relevant training yes I'm advocating that you write your own application security training now that may sound like a daunting task it's certainly time consuming but it'll be super relevant and anything relevant is is going to get more traction it's going to be more interesting um and probably technology the pace of technology in your company doesn't change so fast that the training needs to be updated that often so it's

an initial investment of time um and then you've got this great contextually relevant training you can roll out to Engineers I've had great success doing this um really using examples of real bugs that we've had um literally even linking to jira tickets in the training so Engineers can can go and see oh yeah this was a real bug um and lastly um you know those kind of trainings where they have a minimum time limit right that mandatory HR and compliance training where you've got to spend like two hours on the thing right how many Engineers or how many people in your company do you think are on their their multim monitor setup have got your training in one

window with that timer ticking down and then in the other window they're on social media or watching Netflix or something right um don't bother with any time limits build Goodwill with your engineers let them whip through it if they really want to just put a test or something at the end so you can you know make sure that they're they're testing out so you can show your your Auditors Etc that they they did it and passed all right let's keep going um threat modeling who here is familiar with the concept of threat modeling okay good number of you so I'll just go really quickly through this but for those of you that are not familiar um

threat modeling the premise is this we want to find design flaws as early as possible so really ideally before uh our Engineers start coding or at least early in the process so that if we have uh have to have them make changes these changes aren't expensive because the later we get in the Engineering Process uh the more expensive it's going to be the more timec consuming it's going to be to actually make significant changes right so you want to catch all this stuff up front uh catch these design bugs so it way it works is uh your engineers come to the meeting that you've set up and they bring this a beautiful data flow diagram like you can

see on the left and then you pair that with a methodology the one here I'm showing is uh stride um it's a an acronym for those different types those different classes of vulnerability but there's various different methodologies and also various different ways that you can draw those data flow diagrams but that's the general premise right but I want to back up a sec wait a minute how did we even get to that meeting where Engineers came prepared with this beautiful diagram right in a lot of companies there's really not the communication between the the product side and your information security side like how many times have you worked in a company where you've seen an

announcement about some new thing some new application like some announcement to the company this thing's now live and in your infect team you're like wait a minute has anyone even heard of that thing did we do a review on it right so what I'm really advocating for first of all is that threat modeling is kind of less useful if you don't have a rock solid relationship with your product team if you don't have a process to know everything that is coming up because if you don't have that then ultimately you're probably missing threat modeling stuff that you you should be ultimately threat modeling and secondly how do we expect our Engineers to uh produce all

of this material uh these beautiful data flow diagrams right um they they typically won't produce a diagram like this during their normal engineering work they may produce like uml sequence diagrams or other diagrams in their specs but they're not producing data flow diagrams so you're magically expecting them to not only know how to do this but also invest the time into doing this it that's unrealistic right you have to train your engineers then you have to persuade them on the value of this um and lastly a phrase that I've refrained from using until now but you hear in every absc presentation here it goes um shifting left sure we've all heard that essentially it means we're a small infos

team to scale security we have to make it the developer's responsibility right so you'll often hear you often hear of initiatives and absc where threat modeling groups are run by by Engineers but the problem is this what if the engineers don't know the threats like how usefully your threat modeling session's going to be so really uh well let's let's let's look at the guidance here so going back to what I said the the the first um the first point is really need to focus on identification and prioritization of what needs threat modeling this idea that uh a fewer really high quality threat modeling sessions on the things that are truly critical that you're releasing that's

way better than trying to um scale a threat modeling program and have lots of shitty sessions where essentially nothing really gets done no good action items come out of them and Engineers eventually kind of lose interest in them right so um much as I would like to democratize threat modeling and have infosec Play Take a more of a backseat if you're starting to roll threat modeling out in your organization infos needs to be really front and present in all of the sessions right and then it becomes a scale problem so you really need to be very focused and lastly we do threat modeling to identify threats and those may be in the form of design

issues or they may be um concerns about implementation but the implementation isn't done because remember we said we're doing this before the engineers write code right so the two outcomes you might get are tickets to go and change the design or tickets to um around concern about the implementation like when when when you've written this as an engineer like assign the ticket back to me because we want to review it like targeted code review so you really need to make sure that your actions are are tracked right and completed because if you're not tracking your actions and you're not completing those actions why are you even bothering with threat modeling so that's a really nice metric

you can you can you can um measure when you roll out threat modeling like what how many action items do we get out of every meeting and what's the completion rate on those let's keep going um static application security testing so sast has been around for a long time now um here's a diagram I stole to show how it works essentially we it it's automated analysis of code to find security vulnerabilities we um feed in our source code into our system it may produce um an intermediate representation uh it then takes a set of rules typically vendor provided rules or Community rules you can also in most add in custom rules and then it uses some

combination of those Technologies in the box at the bottom uh everything ranging from simple pattern matching through to more complex uh processes like control flow analysis or taint tracking so tracking where uh user input how user input flows through your application like in premise uh like like a lot of these processes the premise is good right it's a source of uh vulnerability now in the days before shifting left sast used to be run by security teams but remember that doesn't scale so we have to force it on to our developers now but in those old days there was sort of an implicit understanding that SAS wasn't that good the SAS would throw up a ton of falce positives and hey we're

on the security team we're used to this we we would do the triage and we would sort of quickly get to eyeball it and and say that's a falce positive Etc now the problem when we shift that onto developers is developer gets a finding and they spin their wheels they they spend time trying to understand hand it and then they they can't they they're like this doesn't seem to be a bug what am I missing and they come back to the infas team and infas then look at it and INF infas then say Oh yeah this is force positive developer goes away next issue for triage same thing happens they come back to infas oh yeah another Force

positive another Force positive another Force positive we're essentially training Engineers to think that SAS doesn't work right and that is the the big problem because a lot of these tools they'll throw out the findings and they'll tell the the severity what they won't tell you is confidence they won't say this is a critical vulnerability and I'm 95% sure of this or U this is medium vulnerability and I'm 10% sure of it if we had confidence we could at least like set a a bar and say okay anything that's less than I don't know some arbitrary value 80% we're just going to throw away and and really the stuff that we're really sure on is going to go to

developers but most tools don't work that way so we end up training our Engineers that SAS is effectively pointless and then eventually Withers and dies how do we get past this um okay that top Point another way of saying what I just said Force positives kill developer Goodwill right that's the biggest takeaway you need for for sast another limitation of sast is it can't find business logic bugs it finds kind of implementation bugs so imagine you're developing a banking application sass with just a default rule set is not going to be able to find a business logic bug that lets you transfer money that you don't have from somebody else's account into yours it's it's just not it

doesn't it's pattern matching in your code it doesn't know what your application is designed to do how do we make SAS useful well um we really need to focus on true positives so um take that rule set and uh over time just ruthlessly pair it down ruthlessly pair it down just so you have a few very well-defined scenarios with a super low Force positive rate right so Engineers are getting high signal because again Force positives will kill developer Goodwill let's talk about dast Dynamic application security testing so unlike static application security testing that focuses on code dynamic application security testing focuses on a running version of your application now normal normally a dask scan takes a bit of time

so you might kick it off in a cicd pipeline but it's not going to be synchronous you're not like the pipeline is not waiting for Des results to come in so often we end up just kind of scheduling Das scans against our Dev or our test environment maybe you know once every release or like once once a day or something like that um in the old days we used to build applications around a very page Centric model so uh depending on how long you've you've um worked in infc you may remember applications built around Technologies such as Java Ser lit pages and um PHP I guess we still use that uh acve server Pages even sort of

aspx superseded by aspects of um asp.net but it's a very page Centric model for building applications and what I mean by that is different pieces of functionality in the application were encapsulated by different pages login would be at a page checkout the various stages of checkout would be their own Pages everything sort of neatly um encapsulated in a page and this worked really well for dast because dast has has two real jobs it needs to find all the entry points into your application um and it needs to know essentially how to call those entry points with fuzzed data right it'll generate requests it'll look at the response to those requests and try and identify any security

vulnerabilities um so this page Centric model worked well because essentially it was easy to do or easier to do those two things we could just look at the uh HTML response that comes back we could essentially GP that looking for interactive elements like form buttons Etc and then we would get to learn like um of new pages that we should test and we could look at the form fields and we would know okay these are the parameters that we need to call that that page with so that model kind of worked well now where things went off the rail off the rails with d is about 15 years ago we started moving towards single page apps

so a single page app is basically um you know one one page in your browser that's uh using JavaScript a giant blob of java JavaScript to encapsulate all of your front-end logic and that JavaScript is is making uh making Dynamic requests to an API to to your application's API now that might be a private API as in it's it's publicly accessible but it's only designed to be used by your front end application or it might even be a public API as well an API that that you expose to to your customers but either way you've got this single page and the JavaScript making all these API calls turns out this was a nightmare for dast

and to be honest D is still trying to catch up uh it's a nightmare because D can't magically then determine all of those entry points it can't Analyze That JavaScript that you know giant blob of minified JavaScript it can't Analyze That to know exactly what the mpoints are exactly how to call them like what are the the um headers it needs to put what are the uh what is the the body of the request it needs to put and what is the sequence of calls that it needs to make right so dask really struggles with that how the industry um got around that is uh that most dask tools allow you to record macros they they allow you to

essentially capture traffic uh and then feed that to the dask tool like templatized traffic so dask can see okay here's a sequence of requests representing login I have to send this request first and then this one and then this one and here are the parameters okay that works but that doesn't scale right you're telling me I have to go through and record a macro for every piece of functionality on on the site and anytime anything changes I need to like re redo this process uh it's a nightmare so how do we make Das work um well that first point here D can't automatically go deep into flows without significant prep work who's going to be

doing that prep work well it's probably going to have to be us because we're the owners of the Das tool um and and that's time consuming um secondly kind of like SAS D won't magically find these hi value business context uh uh contextual business logic bugs so take the banking example again dask is not going to magically find that you can transfer money out of an account that you don't own um one possible Saving Grace for D is that if you have a dedicated QA team that the test software if uh they have ways of testing your applications in an automated way they've kind of solved a lot of these problems they've actually solved this idea of like how do we log

in right they've built specific tests automated tests that will test these parts of your application they'll figure out how to log in so if you're willing to put the work in you can kind of solve that uh problem of needing to generate macros for your Das tool by partnering with QA um so if D is something that you really want to go all in with um then you know that that could work go go talk to your QA team um something I probably should have said earlier as well is that when when I showed that slide of all the different Technologies any one of them done well can improve your security posture so this talk is less about like

perhaps which ones you should do and which ones you shouldn't it's which ones you're going to intentionally go all in for and try and get that aade let's talk about paved roads this is a concept that uh the security team at Netflix um have been speaking about for uh a good number of years now essentially um we need to come up with templates for secure configurations for for libraries and uh microservices we then want to incentivize our Engineers to use these things when they're building their microservices or when they're pulling in their libraries right and if we can incentivize them them to do that we're going to get all this great benefit because that these libraries are going

to be secure right they're going to be vetted by us um so how do we incentivize Engineers to do this well we make these templates really enticing from an engineering perspective we basically load them with all of the scaffolding that Engineers would need to spin up a new microservices project so we stick in all the configuration management code all the observability stuff that's required for um observability in your company we put login code in there we do the standard Secrets management stuff so it's really enticing for engineers because they have two choices they can go the non-paved roadway spin up something from scratch and spend like a whole bunch of time having to put all

that Scaffolding in or they can just Fork the the pave Road approach and just get coding on the thing they actually want to build so this is a rare scenario where it's done well it's this win-win it's this potential like Engineers will will will want to use the thing and you get all this security benefit uh so what's not to like well ownership and maintenance um can become a bit of a nightmare often you'll find that um you'll find that ownership isn't well well defined maybe this was a hobbyist project engineering leadership never bought into this you know somebody in company in engineering did it as uh in their spare time um and uh everyone

then forked it then that person leaves the company moves teams or gets too busy to maintain it and then what happens a new log for J2 comes out and uh um you know we infoset goes to the owner of the template hey we need you to patch this and they're like I'm too busy or no one know who no one know who who owns the template anymore so you really need to be explicit about that engineering leadership really need to buy into this concept and then tracking the um tracking template lineage in in cicd everyone's forked it everyone's on different versions of it um that can become a bit of a nightmare and then let's say um the expectations on on when

Upstream changes are pulled so let's say let's take that log for J2 example you've got templates you've fixed it in the Upstream template you you know where all the forks of that template are like everyone that's using that template and you go to those teams and you tell them okay you got to you got to fix it you got to pull in the changes from the the template but you know there's a good number of teams that uh last time they pulled in changes was two years ago or three years ago and there's like breaking changes and they they they're refusing to pull in those latest changes because they're like whoa this is a ton

of work like and and you know they didn't stay up to dat with those changes so those are kind of things you need to think through like um in short pave roads great idea but it's got to be it's got to have strong Buy in from from engineering otherwise uh it won't work okay um last process I'm going to go through I'm sure you're all familiar with bug Bounty um this idea of let's pay security researchers to find and responsibly report security issues in our code premise sounds absolutely um great you're any paying for um issues um so what can go wrong with a bug Bounty program well here's how it typically plays out you'll decide you want to do

bug Bounty you'll go talk to one of the three major vendors that runs the platform they'll give you a quote you'll like fall off your chair when you get that quote saying like why is this so expensive then you'll decide we can run this ourselves how hard can it be we just need to pull of money right then we don't need to pay the platform um so you you you start your program you publish something on your your uh um your site saying like send us the bugs and then you get deluged you get deluged with junk reports and you spend all day looking at these junk reports trying to separate the wheat from the chaff right

that is why these platforms exist they exist to do the triage and give you the strong signal out of all the chances out there that are submitting bugs okay and that problem is only going to get worse with AI if you look at um the C project for example there's some really comical examples of just like AI generated bugs that have been submitted um that unfortunately it's an asymmetrical problem the people on the on the triage team have to then spend time like understanding is this a real report is this bogus so running bug bounty work how do we solve this well you probably unfortunately need to spend the money and use a platform you need to

continually refine your scope to eliminate the noise if you get like findings not interested in um okay figure out paying it out or not but refine your scope so you don't run into that problem again um and make use of bonuses to really and steer researchers onto the parts here application that you really want um scrutinized um and lastly how do we think about effective of bug Bounty because number of bugs which is a primary metric it's a shitty metric like what does it tell you if you don't get any bugs does that what does that tell you does that tell you that your software is secure or how do you know are people even looking right so you

should really think about what the uh metrics are but in the interest of time I'm going to move on to part two this is just a few slides um we're going to talk about the power of true positives this is not um self-help for infc um though God knows we probably we all need it uh instead this is how my Approach doing application security security tooling considers the entire universe of vulnerability classes right if we think about running the program forward like the the the stages of the sdlc we are focused on that entire universe of like what could go wrong we're trying to catch it prevent it uh detect it as early as possible what I like to do instead is

focus 90% of my effort going reverse I like to um really focus on true positives those verified vulnerabilities uh in your production environment because you may not realize it at the time but they have so much power they have so much value for you understanding how to improve your security program so the true positive process now on the left here how do we get sources of true positives okay here's a non exhaustive list it's it's pretty straightforward we get them from pentests like you know here's a nice write up of a real bug we get them from bug Bounty once we've triaged them we might get them from ad hoc testing from from the infect team we might get them

from our QA team uh if you're working for a company that uh that sells to um government or uh uh other businesses you might get them from from uh B2B from from the from your customers so we've got a source of these vulnerabilities so what do we do with them okay we start from the true positives we figure out the uh root cause of those vulnerabilities now this is not just file a ticket for engineering and and walk away this is partnership with engineering to really understand what the problem was this is also not just read the read the pentest report and you know summarize it at the high level it's input validation or it's SQL injection

or or something else it's work with the engineer to really identify this is where the problem is in the code everyone needs to understand that infc engineering you then abstract that to the anti pattern what are the characteristics of uh that particular vulnerability that class of vulnerability and how widespread is this that's what you're ultimately asking because what you don't want to do is play wacka you don't want to just fix that have the engineers fix that vulnerability and then a month later you get the exact same vulnerability reported in code two lines below right you want to look holistically across all of your code all of your engineering teams and ask um is this vulnerability

is this class of vulnerability um widespread uh or or is it just ad hoc and that really helps you um prioritize so let's say we've got some um systemic vulnerability um some critical systemic vulnerability that we've abstracted to an anti-pattern well you of course work with engineering to fix that but then the final step of the process it's future proof with counter measures we need to take all of those um uh uh processes and technologies that we've discussed in part one of the talk and we need to figure out how do we use those to eliminate this entire class of vulnerability forever that is how you get better at absec by eliminating entire classes of vulnerability as they

come up one by one right and typically there's no Panacea you're probably going to need to use a bunch of those tools and processes to actually um solve this problem so maybe you need to add dedicated training to your engineers about that specific type of of issue or maybe you need customer static analysis rules or if you use Das or uh you know um you tune dask to find look for those types of issues or you work with your QA to hump for them as well um or you have a bug Bounty campaign where you pay double for those types of issue lots of things we can do rinse and repeat eliminating entire um classes of

vulnerability quick word on ticket tradeoffs um I really like this graph because uh it it basically shows let's say we have a vulnerability from one of those sources bug Bounty or pentest right it basically shows how long it will take for an engineer to fix that if you just copy paste put that in a ticket it's going to take a long time to fix that because an engineer has to pass through it understand what it's really saying translate it into contextually relevant information for them for the organization and then figure out how to fix it the alternative approach is you like us we could spend a lot of time doing that triage doing that Discovery writing

a beautiful ticket translating it from the the the language that the researcher used into words that are really meaningful for our engineering org and then proposing exactly what the the fix needs to be right but that's super time consuming for us but if you do that I guarantee your ticket will get taken care of and fixed like super quickly so there's no right answer of how much effort we need to put into our tickets but it's something that you all need to think think about when you're filing those tickets and you need to know like the the two ends of that Spectrum if you just copy paste and file that ticket it's going to be a slow like bumpy road

to get it uh get it done uh if however you really focus um on making those beautiful tickets tons of detail it'll get get done quickly but how do you scale that right so you got to think about that sweet spot on that curve that that works for your or one more thing on the correctness of fixes so um I think a lot of this is stating the obvious but who confus security fixes you may have other Engineers just giving a quick looks good to me or a quick thumbs up um and you may ask well doesn't retesting cover it by the pentesters right what I'm going to advocate for is that infoset really understand the bugs

and do some level of um validation on the fixes themselves ideally walking through the code with with Engineers wait a minute so you're saying we have to do this for all bugs of course not you need to figure out you know critical bugs obviously but figure out what works with with the bandwidth that you have on on your team but correctness of fixes is hugely important if you're not fixing issues correctly um then they will eventually be rediscovered and and it will um be even more work for for engineering okay takeaways um firstly I have unicorn sugar cookies for everyone I'm just kidding that PowerPoint template just just came with it um really just two takeaways um be super

intentional about how you layer security into your sdlc right remember are you just doing it for the pass or is it something that you're really going to double down on get that a grade um or do you just not need to do anything you just not do that thing and then um think about this concept of um running things in Reverse learning from True positives and really El eliminating classes of vulnerability one by one because that is a meaningful way to improve your application security posture thank you um that's all I got I'll be around for entire day would love to chat to you all about application security or anything else thank you [Applause]