The Software Security Testing Lifecycle: The framework testers need

hi my name is michael i'm an army green braid turned application security engineer i still serve in north carolina national guard as the senior enlisted cyber network defender i also work for a content creator for a company called security journal creating their application security focused content you can reach me on app apps guy mike or look up my company at securityjourney on twitter today we're going to talk about something called the software security testing life cycle a while back i was given a task to start doing research into application security testing to teach developers how to approach this big task that we call testing for security one of the big problems we ran into is i

did the same thing that everybody else does i went out to owasp i went out to all the open source resources that told you this is how you should be doing security testing these are the tools you should be using these this is the phase that you should be doing your testing and something that i came to realize is application security testing is very reactive and it's a single phase of a single step that we put inside our software development life cycle we wait until we get to the testing phase we do our security testing if we get any alerts then we go back and we try to remediate what we aren't is proactive and we don't plan for how we're going to

actually approach security testing that's what the software security testing life cycle is meant to fix so key thing what is the software security tesla lifecycle and i affectionately call this the swisstool because you have to have a good acronym for any good name you have swisstal integrates security testing throughout the entire software development lifecycle that's meant to be a much more proactive and focus your testing into the areas that need the actual security testing one thing that sets it apart from all the other different frameworks is it's not just about how does a qa team do testing or how do i do all my integration testing and my functional testing or my smoke testing it's very

specific of how do i approach appsec from the very beginning of my planning of my software development lifecycle all the way through to production why do we need a framework now i think everyone out here that's been in part of the oas community has gone out and seen that there is security there's a testing guides out there that talk about how do we approach testing a lot of these guides are really focused on how does a qa team approach testing and it goes and it talks about how do i do my integration testing and my functional testing to make sure all my parts of my application are working appropriately it leaves the security testing as this

very single step that isn't really thought about much and a lot of times people take a single tool put it in my pipeline and wait to get a bunch of alerts and then throw those alerts at the dev and tell them to go fix them it's a very reactive and very restrictive way to approach testing and it doesn't plan for testing in a proactive manner what it leaves ends up happening is we build up a bunch of technical debt near the end of our build process right before we're trying to get our product out into production and we're having to go back and remediate all these things we're finding at the end and we've basically

built this technical debt that's at massing major costs on to building your application now swisstal isn't just for the testing team it's not just for developers it's for everyone and the three categories here is it's for your developer it's for your tester it's also something we call product adjacent product adjacent is basically anyone that touches your application to get it from the design all the way to release whether that's a ux your production manager whether it's any of your qa teams everyone that's involved in the actual building applications needed to understand how the software security testing lifestyle works because then you understand that when we get to certain phases we're going to do certain things that might

change processes for everyone involved now this is also meant for any scale operation it doesn't matter if i'm just a developer but i don't have a qa team i don't have testers i'm one a team of three developers and we do all the tasks together from production all the way or from um ideation all the way to production well when i wear the tester hat and it's time to hit that phase and do that task that's when i need to step out of my developer mindset and start thinking about how am i a security tester and how do i apply the things we're going to learn here in the software security test and life cycle to

build a better more secure product the phases of swisstal include you need to know your security requirements you need to participate in threat modeling you need to build a security testing strategy you need to build tests and review automate tests and validate security findings and controls the way swissle is set up is each phase is meant to build and feed into the next phase so when we start in the planning phase and we start knowing our security requirements we're going to take those security requirements we're going to use them when we do our threat modeling we use the products from our threat model and we're gonna use them to design a security testing strategy we're gonna

take that strategy and we're gonna write our specific security unit tests and we're gonna do our peer reviews which are gonna enable us to do more efficient automatic testing or auto test automation in our pipelines and finally rather than just taking all the information we gathered from these previous phases and assuming we're good we're going to validate the findings we get and then remediate in a structured approach that allows us to focus in on the things that are true security alerts and reducing our false positives we're going to go through and we're going to break down each one of these phases and explain in detail what goes into each phase and how it feeds into the next

phase the first phase is you need to know your security requirements now when we start designing new software especially in the software development cycle everything always starts with define your requirements now as a tester i'm looking at this of what do i need to know about my company and my product that i'm going to have to test for throughout the rest of the phases of development if i don't understand the constraints and the security requirements that are applied to my application then i'm not going to be able to effectively identify what my company considers the security vulnerability to be and how i should test for those to validate that we don't have those vulnerabilities and that we meet

these security requirements there's three different types of security requirements you have your internal security requirements your external security requirements and your business security requirements external security requirements are things that are like regulations and laws that are being placed on your application you might fall under hipaa or you might fall under some other policy or directive that if i don't meet those requirements i could be held legally liable for not doing that or i could be lose basically face big fines lose customers massive things that can affect my application internals require security requirements are things that are dictated by my company so it might be things that follow our ethos we as our company we embrace the fact that we want to ensure

you have privacy it may not be a requirement that's legally applied to us but we believe privacy while you're surfing the web or why you're doing something is extremely important so we're going to build in security requirements for application that ensures you meet those requirements and then business security requirements that's really when we're talking about our cia triad right and when we talk about those is what are we what are the security requirements that if we don't implement we're going to lose customers we may not follow our policies it may not be directly against government regulation but if we don't meet these criterias we're going to lose business it's extremely important as a tester then i define and understand what

the security requirements for my application is so that as i go in and i start to threat model i can identify areas that i need to test to ensure i'm meeting those security requirements the second step we want to talk about is participating in threat modeling threat modeling is extremely important when we're building our applications especially when you talk about your secure development life cycle when we're looking at the entire what is the security of my applications here's my application here's everywhere that i need to put a security control to enable to prevent different vulnerabilities as a tester i should be involved in that process first of all as a tester i'm gonna have a unique perspective from my experience

from previous testing experience as well as i'm gonna be proactively thinking well how do i test for that how am i gonna ensure that i'm preventing that vulnerability how am i gonna ensure that security control is actually doing what it's supposed to do the more intimate knowledge i have of how the entire application works and how we built this threat model it's going to make me a better attester to the other phases of the software development life cycle it's also going to allow me to identify all of the different things i'm going to have to check in my security testing strategy whether it's the threats the vulnerabilities as well as the security controls i have to validate

i'm going to be able to list those out my security testing strategy to test later on when we're in the development phases now there's five phases to the threat modeling first you need to scope so what am i responsible for testing draw that's when we're going to create our data flow diagrams and i'm going to look at the data flow diagram from a tester's perspective we're going to analyze our data model using threat modeling techniques like stride we're going to mitigate i'm going to specifically might not be the one deciding which security control is going place that might be the security team as a whole but as a tester i need to see what security controls

have been chosen so i know i can test and validate their work inappropriately and the most important step in all of this is we need to do thorough documentation of our threat model because i'm going to take that information from my threat model i'm going to use it to build my security testing strategy now this is a typical diagram of how we build a threat model the first thing we do is we look at all the different components of our application now this is a very simple example but we're gonna look at all the different components and where data flows and all our inputs and we're gonna evaluate how the application works once we see how the application works

we're going to apply something like stride stride stands for things like spoofing tampering repudiation these are all different this is an acronym for all the different type of things that can identify looking at how my data flows in my application and identifying areas that i might have problems so once i take stride and i apply it to my model i really want to look at it from an attacker's perspective and the attacker's perspective is okay in this example i have a database in the background and there might be a possible way that there's an api that doesn't authenticate the user or require login so as an attacker i might have access to that api and if we don't do proper

authentication i'm able to get information disclosure because i'm able to get data out of the database without being authenticated you want to look at all these different areas and you're going to decide how what security controls were using to mitigate these risks as a tester i'm going to decide what threats are there that i need to now validate are no longer threats and what security controls have we chosen and are they working effectively that brings us to what a security testing strategy is i brought that in a couple times but let's clearly define what a security testing strategy is when we talk about it within the terms of swisstal it's basically it's going to be

a step-by-step list of all the different threats and security controls that have been put in place based on your threat model that you're going to test and exactly laid out how you're going to test those individual threats and security controls to build appropriate security testing strategy you're going to need to review your scope you're going to look at your data flow diagram for the different interfaces you need to test against you're going to define security tests based on the different threats and interfaces that you identified in your data flow diagram and then you're also going to define the security test to validate that your security controls are working appropriately the last step is once you've gone through and used all

that information from your threat model to create this list of things you need to test against you're going to select the proper people processes and technology to conduct the test what that's going to allow you to do is say these are three different ways i'm going to test this interface to ensure that my security controls are working appropriately and i have remediated and reduced that threat from being exploited going back to our original threat model where we identified three different threats we take this information and we're going to create the security testing strategy if you look in our left side here you identified the threat so threat one might be the system can disclose sensitive information due to an

sqli so there's some sort of sqli injection possible vulnerability because i have an interface that i'm allowing users to do input into that's going out to my database and retrieving information how am i going to protect against that well during our threat modeling we decided to use server-side input validation to ensure that the attacker is not able to do sqli against our database as a tester i'm going to take that information from the threat model i'm going to decide how am i going to test to ensure that sqli is not possible and that my input validation is working appropriately and we're going to create different steps that will be applied at different phases of the development process

to identify and confirm their control security control is working appropriately first we can write a unit test to confirm that the input validation on the actual security control is working appropriately we can use tools like wikipedia and or a fuzzer tool to auto in our automated test process to identify if we have sqli and we can also do at the very end phase a pen of the interface that the user is able to access there to ensure that we're not able to manually conduct an sqli now at no point here am i recommending any of these specific tools these tools that were chosen are specifically because they're open source too and a lot of these are on the oaus recommended list

that anyone can grab and adopt easily there's commercial tools that will work amazingly well also the important part is it doesn't matter if you're using a commercial tool or an open source tool just find something that's appropriate to your environment and start using it now the fourth step is we want to build tests and review when we build tests what we're talking about is we're building what we call security tests and the review is a security tester's peer review bill test includes two different things the first thing is what we call a security unit test and a fuzzing target and the peer review is taking that same concept that we use as a developer and

we're going to apply it to our security testing whenever i'm a developer and i write code and i push it to commit it's almost never where i'm like oh i'm going to make a big change in our product and i'm just gonna commit it to our main branch and no one else is ever gonna look at it because i'm gonna break something and i'm without a peer review there's a really good chance that i'm not gonna follow all the same constructs you need that check on learning basically with each other to keep each other doing good clean code we should do the same thing as testers as a tester i want someone that's going to peer review my testing strategy to

ensure that it makes sense that i've done the right steps and as we go through to start building the security unit test to start setting up our automated testing and doing our manual tests then i've created a plan that makes sense now let's talk about the different type of security tests we can build i already mentioned that there's security unit tests and plus targets we're going to kind of deep dive what those means because i'm pretty sure everyone's heard a unit test but there's a very distinct difference between a security unit test and a traditional unit test when we talk about security unit tests what i'm doing is i'm using a unit test to validate the

security control is working appropriately so if i as i'd identified originally in our threat model i have the chance for an sqli against our application when i build the security control which is an input validation to ensure only the appropriate characters are allowed into my interface then i also need to go back and i need to create a unit test that actually ensures that security control is working appropriately if you look at our example i'm using ruby just uh as a for you guys to know what type of code i'm using here because it's pretty easy to read and pretty straightforward about what's happening in the code example itself i have a class account and right here i'm

creating an account with my function create account and we're passing in a username and password and i have two security controls in here i'm doing input validation on the username i'm doing input validation on the password and in my input validation for the username i'm ensuring that i'm only using lowercase or uppercase letters it needs to be between five and ten characters if it doesn't meet that criteria it's going to return the fact that the username must be characters only for my password it's a little more restrictive is that i can use alphanumeric characters and it has between 8 and 50 characters long if it doesn't meet that criteria it's going to tell the user the password must be

numbers or characters only and then we create my password so this is a good example of a security control i've built but as a tester what do i do with this to ensure it's working properly now if you look at a traditional unit test you ensure that it's doing it functionally what it's supposed to do so in this example right here i'm setting up a new user account object and then i'm going to check that object given a username user and a password testpassword123 by the way horrible password just using it as an example at line five you're gonna check hey i want this to return true when i create my user with these two objects with

these two strings and the expected value once i create this user should equal to true now in this example this functional unit test is going to work easily because we're taking what i call the happy path now a lot of times when we do unit testing we're going for metrics and numbers we're just trying to get a bunch of green lights on the left side that said yep i did wrote one unit test for each one of my functions and they're all doing exactly what they're supposed to do and if you're being like doing that extra mile you may take one or two unhappy paths where i'm like oh if i just put a garbled number of characters

in here it doesn't work because it doesn't meet the criteria when we're creating something called a security unit test it should be a much more focused on how is the security control actually supposed to work and how would an attacker try to bypass that or exploit my code so in our next example we use the sqli security unit test i'm doing the same thing i did with my previous functional unit test it's like this time when i'm passing in my password i'm actually passing in an sql exploit and i'm seeing if my account will return false when it tries to validate whether that's a valid username and password pair now in this scenario based on the security unit

control i've already put in place it should return false the important thing here is i'm giving you a single example of one thing to test these should be as advanced as your imagination allows them to be you should try every edge case you should be able to use a file of the sql attacks against your security control be creative the same way attack would be one of the nice parts of this is this is really a developer task and when we approach security testing one of the most important things to do is either whether you're the developer or you're a security tester working with the developer the developer needs to be part of the actual security team

a lot of times we end up in these type of relationships where we have the security team and the developers and we're we're ahead right the security team is worried about security developers like i just have a deadline i had to get my functionality out um if my one tool that's scanning my product doesn't catch the security alert i don't care let me just get it out the door the better way to approach this is you need to embrace the developers as part of the security team and give them tasks like this to validate the security controls and start getting to think like attackers because to be honest creating these type of security unit tests would

be funds because basically i'm trying to exploit my own security control so i'm trying to attack my own security tool that i in place and the more involved you keep the developers in this and get them to embrace that defensive code mentality the higher quality your products gonna be now the other thing i talked about when i said about building security and test is fuzz target now fuzzy is basically the traditional idea of fuzzing is i take a bunch of random data i jam it in an interface and i try to get my application to do something weird it's either going to hang it's going to stall or it's going to crash in the more modern way we do fuzzing

it's not the best approach to jam basic data into every interface especially if you just pick random data that's kind of a brute forcing attack and it takes an extremely long amount of time if i try to test every interface starting with like zero to one to two to three all the way up just random long amounts of data to images to whatever it's going to take days to go through this type of testing and that's not realistic in a modern environment where we're trying to push code out every single day and in bigger industries you're trying to push code out multiple times a day in this agile environment fuzzing is still important but it needs

to be done purposefully and what i mean by this is we build these things they're called fuzz targets and fuzz target sits around my interface that takes an input and i'm basically able to say hey these are the different areas that i want my fuzzing engine to actually fuzz and this is the type of data constraints i want to put on how you're going to fuzz that target so we're not wasting our time doing a bunch of stuff we already know that it's not even compatible to that interface it helps us be much more proactive on deciding what actually needs to be fuzzed and it helps us think more critically about how to fuzz that target to get the best results to

reduce the amount of time we're spending fuzzing our application now the other side of this during this phase that we call build test and review is the tester peer review like i said before the tester peer review is that one chance i'm able to go to somebody else that's doing testing application it might not even be a tester if i'm a developer and i'm tasked to be the tester for this build i can go over and talk to a senior developer and be like hey can you look over my testing strategy that i built for this and make sure i'm not missing anything or do you see things that i should be doing differently do you think the tools i'm

using are appropriate for our application it's a really good time also to exchange information with this person and gain some knowledge in a different perspective because a lot of times when we're testing our own product we get kind of tunnel vision into like well these are the things i have these are the ways i always test this and it never diversifies you gain the knowledge of your peers to allow you to be a more effective tester so the things that you should be reviewing during your test review is your threat model to make sure you guys didn't miss anything your testing strategy to ensure that it was constructed well off your threat model and you chose appropriate actions based

on what was in the threat model the tools you chose for automation how are you going to conduct your pen testing and your pen testing tools and like i said at the very end here this is a great time to exchange information especially if you're two people working on a same company but maybe you're working on separate projects and this other developer's been seeing this other testers we've seen a lot of developers are doing this same problem every time and they might tell you hey we're noticing that we're not doing good whether it's input validation we're not preventing cross-site scripting well or we've been allowing i've been finding a lot of um things where we're exposing data that we

shouldn't be just exposing because we're not implementing our encryption prop appropriately this is the moment to gain that knowledge from either a more senior or a peer tester to now know that hey this might be something we're seeing systemically in our organization i need to be on the watch for when i'm evaluating my application the next step we talk about is automate security tests now as a tester everyone thinks of like oh your application security tester the first thing everyone thinks of is a pen tester you're going in there and you're attacking applications and giving feedback to the development team that's actually a very small fraction of anything you should be doing as an application

security tester specifically because that's the most time consuming part of the process as well as it's the human part of the process and anything we leave into the human part of the process is going to be prone to error you're much less likely to you're much more likely to miss things when you're leaving it up to a person to do this type of testing so that's where our test automation comes in now there's a problem with test automation too test automation is very restrictive in what it's able to do and find so that's why we use our testing strategy to identify what can we catch or should be caught during our unit tests through fuzzing as well as

through our scanning tools and what vulnerabilities might there be that we can't find using this different type of tools a very good example of this is dom cross-site scripting there is almost no security unit test or automation test this could be do a good job to confirm that you've prevented cross-site scripting dom cross-site scripting that's something that if you have an interface where you're taking user input and you're reflecting it some back to some way back to that user you should just manually go in and test and you should identify that during your security testing strategy that at the end you need to manually confirm that you've mitigated that threat let's go do a more deep dive into

automating testing why we want to automate testing it saves time it's going to be much more consistent than having a manually tester go through and do the process you need to ensure that it's being customized for your product and that's something you do during your security testing strategy you look at your product you look at your threat model identify what your application does and then you're going to modify how you run your automated tests for your application now if your application does not use an sql database you probably don't want to run every type of sql injection attack in your automated testing for your application because you're just wasting time if you do not customize your automated testing

scanning tools or your fuzzers for your application all you're gonna do is waste everyone's time and effort trying to get a more secure product out there but because you're wasting all this time people get tired of waiting for you to test their application they're gonna start pushing things to production while skipping some of the testing steps so make sure you customize these for your environment next thing is you there's a lot of multi there's multiple open source tools you can use for automated testing there's things like zap and burp well burp is also paid for product but the zap there's burp there's wipiti also known as elk3 all open source products you can adopt today and put in your pipeline to

start doing some of these scannings once again i am very critical that you customize these so the first time i use a product like with pd i just set it to scan everything and run it ran for like five hours and then gave me very poor results back that i wasn't able to action because i didn't customize how i wanted to scan for how my product was set up one of the nice things about automation is it's going to identify any threats that you missed during your threat modeling process it's not going to identify all of them but some of the lower hanging fruit that you might not have caught during your initial threat model automated testing does a good job

of finding now there's three different types of automated testing the first one i want to talk about is dast das is great because it's dynamic application security testing you're actually attacking the application and it starts out with you as a tester taking a security testing strategy you configure your das tool to work appropriately against your application your build server is going to build that application run that scanning tool and you're going to get a report and now as i'm gonna take this report and i'm gonna take those security alerts and i'm gonna move on to my next phase which would be i need to validate those security alerts are valid positive security alerts because what the worst

thing a security tester can do is take a huge list of alerts and throw it over to the dev team and be like all right here's a hundred things that popped red go fix them all and then if the dev team starts working and half of them are false positives you're just gonna end up with a bunch of angry devs that you wasted their time and they're not gonna trust you as a security tester going forward so not only you're wasting time but you're losing trust in the people that you're asking to help secure your product now the some of the limitations when we're looking at a das tool is the dash cool it has an exterior view of what's going

on inside the application so your application's running i'm attacking the tool and i'm waiting for some sort of response from the tool that's going to tell me i would accomplish something now that works to catch some low hanging fruit however it's very limited because there's a lot of attacks that attackers can take advantage of or exploits that attackers can use that will have an effect inside your application that doesn't always give feedback to the application you might have good filtering you might not give good response to the attacker but it doesn't mean they didn't find some way to get inside the back of your database and delete something whether they were able to get an immediate response from the

application or not the solution to that is what we call is that's so ios tools are the next step or next generation past what a das tool is so it's an interactive application security testing and what it does is during compile time it actually integrates what it calls agents inside the application they're going to sit inside the application on under the hood and watch for what happens when you run a das tool against it or you run a scan tool against it so that from the outside looking in it runs the same you're still going to attack the application the same way you did before however this time you actually have agents inside the application listening for what if any of

the data you're shoving into the application reaches it or any changes that take effect and the really powerful part of that is is that now instead of like the traditional fast tool where i'm just attacking from the outside and waiting in response i also have something inside the application that can really drill down what happens and i can get much more meaningful results from my scanning tool now one of the problems with ias is there's not a lot of open source tools out there and there's a lot of great commercial ones but they're not cheap they're pretty expensive so dash tools work really well then don't give as much feedback under the hood i ask tools are better but there's not a

really good open source solution out there and they're a little more expensive to implement now the last thing we talked about which is fuzzing now there's the traditional fuzzing where i just jam a bunch of data into all the interfaces or we have targeted fuzzing where we're going to go in as a tester i'm going to use my security testing strategy identify the areas that i want to fuzz i'm going to build my fuzzing targets i'm going to run my fuzzing engine against my application and i'll watch to see if it crashes hangs or delays i'm going to take that information from the report i'm going to use that information to identify areas that might have

built-in vulnerabilities that i need to go validate during my final phase when i manually test my application now that goes into our sixth step of the software security testing lifecycle and what this is we're going to validate our security findings and controls now this is the part that everybody loves and thinks of when they're thinking of application security testing this is where i get to go in and play the attacker i get to pen test or hack my application the important part is is i'm not just going in there and playing around like i would if i was a functionality tester or an integrate like a tester that's going in just doing some smoke tests basically just playing

around to make sure everything does what it's supposed to or if i can just randomly hack stuff what i'm doing is is i'm taking the information that i identified in my security testing strategy as things that need to be manually tested because security unit tests and automated testing will have would have a hard time testing for them and i'm taking the alerts that i got back from my secure from my actual automated testing i'm going i'm going to validate that either the security controls are working appropriately or any of the alerts that i receive from my automated testing or actual vulnerabilities the developers need to spend time fixing so if you see we've kind of created a

cascade of events to get here rather than just jumping at the end during my testing phase run my automation tool and have a pen tester come in and check some stuff out i used that security requirements to build a good thorough threat model i used that threat model to design a proactive and well-designed security testing strategy i took my security testing strategy and i decided what security unit tests and buzz targets i had to implement it during the build phase of my application as well as how to configure my automated security scanning tools to be able to give me the best results and then i took the information from my security testing strategy for the ones that i can't test

through automation as well as all the alerts for my automation tool and now i'm going to validate those different security findings and ensure the security controls are working appropriately manually based on that strategy so that's why swiss was so powerful that each phase is designed to feed into the next phase and ensure that we're testing in a proactive manner that's focused on what's actually against my application instead of shotgunning my testing techniques and hoping i catch low hanging free we go in and we do validating security controls and finding it's actually a very methodical process i don't just jump on there and just start all right well it said i have sqli let me just try to

start injecting input into these different interfaces and see what happens you actually have to set it up in the steps the steps is first you identify the threat and vulnerability that you want to test for so say we're talking about the dom cross-site scripting identify that i have an interface that has a possible domcr dom cross-side scripting vulnerability because i take input from the user and i'm reflecting it back into my html some some way now as a security control i put in a sanitizer to ensure that when that data got reflected back into my html that it should have been sanitized by input however my security unit test as well as my automated testing tools are going to

have a hard time validating that security control works effectively so i'm going to go in and manually test for that the first thing is i need to identify that that vulnerability exists or could exist second i need to have a deep understanding of how that vulnerability works if i go in and i just google oh how's dom crosstalk scripting oh if you paste this in that's the one way it works and i copy and paste it in it doesn't work and call it good i tested it i haven't actually tested it because you don't understand what's going on under the hood and how that vulnerability actually works in your application you have to have a deep understanding as

a tester of how these vulnerabilities work so you can start thinking proactive like the attacker would be like okay if this doesn't work well maybe if i put it in the iframe with an alert maybe i can get it to trigger then and now i start getting more creative because i have a deeper understanding of how the vulnerability works the third part of this and now i'm going to choose appropriate tools and techniques to exploit this vulnerability now if it was something like an sqli i might use a tool that generates a lot of sql injections for me and attacks against the interface so i'm not sitting there manually doing it myself it's still manual a test because i'm running

a tool against interface against my running application however i am using a tool to support me to do that to save time and do it effectively or it might just be me having to put manual input into some sort of interface or messing with the url itself without using a tool but this is the phase which identify if there are tools and techniques i need to use what are they the next step is we need to define our test objectives and this is where we define what i consider a positive test and what do i consider a false test so what is my validation criteria to say hey my security control works or hey my

security control didn't work because this is the response i got and it tells me that i have this vulnerability so now that we've gone through we've identified the threat we understand it we've chosen the tools you want to use and we've now defined how we're going to test it and what the criteria for pass fail is then we finally get to go do the part that everybody loves them the most we get to exploit that vulnerability in our code now we go in and we attempt to exploit it and either it's going to be successful or it's not and i must have to evaluate what the application does to confirm my result whether i had a

successful exploit or whether my security control worked appropriately if my security control did not work appropriately the next step is a really important one i need to identify why it's not so it's not it's not just enough to know that the vulnerability is there you have to understand that why was i able to exploit this why did my security control not work because if we're going to take that vulnerability and we're going to look at that security control and say hey you need to fix something we should have an understanding why it didn't work in the first place because if you don't do that root cause analysis during this phase you're setting yourself up to fix

the surface level problem that might have a deeper issue where if i was able to exploit it through this interface so i put on this controller a security setting to prevent it however the problem was actually the fact that on the back end of my server and the very like back end code i'm not doing my validation they might still the attacker might be able to get to that exploit from a different interface because we didn't do that root cause analysis and find out where the real problem was we could set ourselves up for failure so that's a huge step is once i've actually been able to exploit is taking the time to understand why and how that happened

and then here's how we should remediate it and the final step once it's remediated is confirming that the remediation worked appropriately and it's also confirming that you didn't break something else in the process of fixing that vulnerability so a lot of times we might be we might fix one vulnerability but we want to make sure when fixing one thing we didn't open up a different door by changing the way our security control worked we're editing or changing or adding a security control to something that we identified we need to make sure that when we fix or change something as important as a security control that's still effective so confirm that your remediation worked now swisstal those are the big six steps

to swissle some things you want to keep in mind is swissle is not meant for everyone just all right cool we have this new idea of how we're going to approach testing and for every application for everyone out there just adopt all the steps and just start implementing how does it make sense and you shouldn't do that for any framework that anyone pitches out there the idea is that swiss should be customizable for your environment and to be honest if you do nothing else do the first three steps you need to know what your security requirements are you need a threat model and come up with a strategy on how you're gonna test if you cannot do those

three things then none of the rest of it matters we all like to throw that dream tool in there that scans our application and gives us alerts but if we don't understand how our application works and the threats that face our application and the security controls we've chosen to use in our application then we cannot effectively test our application it's also incremental like i said if you do nothing else do the first few steps build small first step is identify what your security requirements are once you've identified your security requirements then maybe you add in okay now let's start doing threat modeling because you weren't doing threat modeling before increment it in steps take off those

bite-sized chunks this is that classic idiom how to eat an elephant one bite at a time right you have to get started eventually start with by identifying what the requirements are during the beginning of development now swissle is not meant to replace the software development life cycle or the secure development life cycle it's meant to complement it it's meant to answer the question of what do i do as a tester during each phase of the software development life cycle or what do i do as a tester during the secure development life cycle and you'll see that a lot of these things marry up obviously security development life cycle the thing we always start with education education is by far the most

important thing you can do educate your developers and educate your testers so they know how to secure your application because if you do it right the first time it's the easiest way and the most cost-saving way to approach development now we get into swissler we're talking about security requirements these are the same as the security requirements and the secure development life cycle as well as the just building requirements for software development when we get to plan and design that's we're talking about that threat modeling right and as a tester i know i should be participating threat modeling and building my security testing strategy so that when we start developing our application i'm able to build those

security unit tests into my application as well as start doing my peer reviews finally when we get to test phase what they call the test phase and software development life cycle that's when i'm running my automated tests it's not that i have been proactively thinking about tests all throughout this but now i finally hit the part where we're in automation and i have tools there to automatically run the tests they're gonna give me feedback they're gonna allow me during the deploy phase to do what the security development life cycle calls pen testing or what we call in swisstal validating our security findings and controls now swisstoll also works in devops because and i want to caveat this it's

devops because devsecops is kind of a misnomer i feel like the security is implied i shouldn't be developing and doing operations without security built into it however with swisstal when we look at devops when you're doing your plan design develop integrate deliver deploy production it's all the same process we're all just using different words to describe how we're doing these different steps so with swisstal during your plannings during your security requirements during your design is your threat model and your testing strategy develop and integrate that's when i'm building those security unit tests and i'm doing my reviews deliveries when we're running our automated testing and deployment is that during that phase when we validate my security fine is in control the idea

is swisstoll should complement what you're doing in your environment it shouldn't be an outside step that impedes what you're doing in your environment and it's adoptable whether you're using agile waterfall devops whatever your approach for software development the principles behind swisstoil can be applied some of the big key takeaways i want to talk about and things you need to ensure are when we talk about swissle it's meant to be a proactive and developer-centric approach to application security testing and the big thing i want to emphasize there is developer centric developers and testers are not sitting siloed in this process either the developer is the tester or the developers integrated with the security testing team to apply

how we're going to approach testing throughout our development cycle that's why the developers should be the one writing the security unit test they should be writing the fuss target because they've coordinated with the security testing team or the testing secur the security team to know what needs to be done during the development phase to enable a proactive already focused approach to security testing now swissle also enforces enforces a focused security testing that's driven by actual threats and security controls in your product so what it's not is a big shotgun approach where i'm just hoping to catch a random vulnerability that nobody thought of swissle isn't the i'm going to catch the zero day vulnerability in my application what

swissle is is i'm going to catch all the vulnerabilities i was able to identify early on and validate the security controls are working that i chose to put in my application because those are the real problems the zero day that nobody knows is coming yeah swissle is not designed to find that swiss is meant to set you up for success with what you already know you have and so you're proactively testing make sure you're doing it the correct way swiss still complements the secure development life cycle and the software development life cycle it's part of those processes it's not a it's not meant to replace anything that you're already doing in your environment it's supposed to be telling

you what should you as a tester be doing during those phases of development you already have in place and like all good frameworks swiss is made to be customizable for your environment it's not meant to be a you have to take the whole package as is process customize the principles that are laid out in each phase and apply them into your application development process in a way that makes sense now i appreciate you guys taking time to listen to me today for this talk about swissel you guys can reach me out at appsec guy underscore mike on twitter if you have any questions or follow-ups if you just want to connect i appreciate it thank you