Building A Modern, Scalable and Effective Application Security Program

well hello everyone thank you for having me it's uh it's a big honor today to be with you um first I'm going to start with this simple question who has never heard about the O Community okay we have some few number but I believe uh most of you guys are already familiar with the community and the projects its different guidelines and toolings and publication that all for free and the goal is to help developer build secure software in fact as of today if you check the official G repo of this community we have 1.2k projects and again it include free resources that can be used by Developers to uh build secure software and it's not the only resource

today we are starting to see more and more organization working under the same goal like the op ssf for example or Nest from the United States but also unseeing friends and Etc and again they operate under the same goal which is providing free resources to everyone interested by application security this is all good but in my opinion we still don't have a clear answer when it comes to this type of question which is simply how we can integrate all of these resources all of these tool in into our organization and you may get hundred of answers and many approaches on how to do this and in fact few years ago we had this uh de cops concept and how it

can help you to solve all your security problems and you may also came to something called the shift lift concept which is a uh an approach to integrate security within your project and there is also this research article from the IBM system Institute which claim that bgs bugs are six more expensive to fix on on the implementation phase than in the design turns out that this article is a big fat lie I mean you can check the fact by yourself and after some research we will find out that this research doesn't exist and the IBM Institute even if it exist it's just an internal program made for IBM employees and the last point is there is no data that confirm this

assertion let me repeat this again there is no data that confirm this assertion I mean I'm not against the concept of Shifting lefts it totally makes sense when you fix an issue earlier in your product the less likely it will propagate deep down in your code and create dependencies everywhere that you need to fix later on but why do most of companies today that have invested Millions on security tolling to achieve this deficit cops concept are still struggling with basic security issues into into their products and and why I don't want to sound pessimistic from the beginning of this talk but if we have some penetration tester in the room I think we all agree that it's quite few R to do

a security assessment without finding any uh security issue or anything to write in the final report also now with the the shift to Ai and llms and all this crazy discussion on um do we need to replace developers or something else as someone who work on a daily basis with developers do I have to maybe change the way I do my work or maybe uh find another job so this is basically what we are going to cover in this talk on building a modern scalable and an effective application security program a bit of background about myself so again my name is Abad and I'm application security engineer and educator I been working a lot and before on

penetration testing so I'm certified penetration testers since 2014 and I used to do a lot of security assessments and four years ago I switched to different role where I work on a daily basis with developers and helping them with uh their uh products by providing the right security requirement and doing a lot of Security review and acting as the subject matter when it comes to security topics I'm also a contributor of the OAS community and I had a chance to contribute to different project like the mobile security testing guide the OAS proactive controls and I'm also founder of klock Academy which is an e-learning platform on how to use this solution for identity and access

management and finally you can reach it by uh on LinkedIn so as you uh May notice I had a chance to work on both side of the industry and this is basically what it feels like and um the thing is you need to have or see think from two different perspective because as a security folks we tend to see everything as an important but we also need to take into the account the developer perspective because they have the their own workload their own deadlines to to fulfill and I believe this is the uh the way we need to approach this kind of topics when we want to integrate application security into the sdlc and this is basically what we are

going to cover in this talk so I'm going to present some of the latest Trend in application security and also provide you with some tips on how to scale your application security program uh especially when we are working on large environment and this talk isn't uh Silver Bullet I'm not claiming that it will solve all your security problems I'm just going to um explain the things that work for my own organization and showcase the things that work and also the limit of it and it's completely vendor neutral in case I mention any vendor's name it's more often like um uh a habit and it's not and it's definitely not sales page with that said um let's jump right to

it um let's first start uh with the global definition of what is an absc program and why do we need uh an absc program an apte program has an objective of reducing the number of vulnerabilities in your products over time by building a repeatable sustainable and proactive security practices that are embedded SE ly within your product life cycle and with that objective we do a bunch of activities a bunch of things the goal here is to combine and split the problem into different area uh instead of focusing only on a specific approach to De with application security and each um component of this of an absc program will require at least 1,00 to cover it in detail but in today

presentation we are going to focus on two main component which is treat modeling and detection simply because we are seeing um big change on these two component over the last uh year or even the last month okay let's start with detection the goal here is to be able to detect security issue in to your existing products or even new ones and here we don't want to rely anymore on manual assessment like penetration testing or manual code review simply because it cost too much and also because the development these days is more into um rapid agall approaches and chipping things more frequently um not like the old man where Le where we used everything um we script

everything and do everything um on a minory basis the goal here is to be able to keep up with the velocity of change and build on the existing process and integrate in all of these so how we can achieve this how we can Implement security testing when working with mod teams in order to answer this question I'm going to walk you through the history of software security testing software security testing goes back to the 30s where alen ma uh alenin and his team worked on uh the bomb machine in order to break the crypto system Enigma they uh they even made a move for this which is a a cool one by the way and it was this big giant

machine with hammers and cranks and wires everywhere and it was a big things back uh to to this period because it did change the course of events during the second world war and then around the' 70s we had a tool called lter which is which was the first static analysis tool and it did Lay the ground for many other tools that will take as an input your source code and point out exactly where you have security issues and Innovation they really accelerated around the 2,000 uh and here we have most of the tools that we um we see today and use today on the market and the the main issue with this kind of tool is that they are resource

intensive and they require a lot of uh uh requirement lot of memoryi resources and uh CPU in order to execute the uh security scan and it's even uh you need sometime to wait for hours or even days in order to scan your whole code base and get the scanning reports and that's why we have new generation of static analysis tools which can be used to run fast scans Focus focusing only on the important security issues so as a developer I can use this kind of tool to verify my merge requests before I am allowed to um push the code to uh the main branch and this is something that can be done in few minute or few

seconds parall to this we had other subcategories of software testing tools like software composition analysis with which aims to detect or scan your projects in order to detect what external dependencies are used inside your project and then verify if these dependencies have security issues and cve also we had test which is um automated penetration testing and here the approach is different from static analysis because we interact directly with the application by sending HTTP requests analyzing the respon resp and deciding from there if there is any security issue to fix and last type of tools is interactive testing here the approach is completely different and it combines both static analysis and dynamic analyses the idea here is to instrument your application

so we have deep visibility on the one time and parall to interactive testing the tool will alert you whenever you have security issues and this sums up the the uh Innovation we had and many different research we had regarding software security testing from the old days where we used to have big machines with wires and cranks Etc to the latest technology in term of software security testing with all this tools in with all this option we have now on the market which one is more relevant today which one deserve to be integrated into our organization do we have to put all or purchase all these tools or maybe we need to pick one depending on our

specific needs and whatever solution you may pick there is uh some success criteria that you need to consider before uh trying to deploy this kind of tools the first one is uh integration here um as a security folks we need to understand and accept the fact that that these developer teams and develop teams has already their tools they have already their pipelines and workflow to build the software so from a security perspective we don't need to change the way this team works and operates but instead we need to First understand the process and build on top of it in order to integrate in all of these then we have the uh implementation speed uh as we uh mention as I mentioned

earlier the development these days is uh um more uh focused on the devops devops uh life cycle and shipping things um at high speed so we need to be able to provide tooling that much the velocity of change we see in devops environments then we have the ease of views um as a developer I don't want to change the way I work I don't want to change my setup in order to install your tools um I'm already happy with my own setup and I'm not going to change it just to run security assessment so from the security perspective again we need to figure out how to propose a developer friendly solution that can be seamlessly

integrate into an existing environment then we have accuracy this is very important because most of the tools that we uh saw later on May generate false positives and this can be acceptable at some level but if we continue to push every time false positive and not accurate results this may impact the trust Foundation we have with our developer teams so it's very important to leverage tool or use tool that provide accurate results and then portability here the goal is to be able to run my security tooling basically from everywhere with there locally from the CI as a web service as a container image Etc

Etc I [Music]

think so we had the same question um almost six years ago and our decision or our choice was to start with Dynamic application security testing but down the road we uh notice that this is something very difficult to setup and scale and simply because it has a lot of requirement you need to configure account you need stable environment to scan and you it cannot guarantee a good test coverage because the first step of dynamic scan is to map all the entries of your application which is something very difficult when you want to scan uh a single page application and sometimes it will take hours before getting scan report especially when you scan a large application and when it

comes to fixing issues again it's very complicated because the developers need to inspect the code and figure out where he need to fix the issue this is why we decided to switch to static analysis and in my opinion it's one of the most used Tool uh today um by many organization simply because it's very simple to deploy the only requirement is the source code you can Alo achieve 100% code coverage because scanning the whole code base will allow you to spot on any security issue you can run at any time um and also in term of redition it's very straightforward in the report you will get exactly which line of code you need to fix uh without

having to invest much [Music] effort okay now we decided to go for example with sest the next question is where do I need to put or configure my tool and here there is different possibility and if you ask your security vendor he told you to uh use his ID plugin to run security testing and the idea is to be able to flag security issues while the developer is building his own code this in theory may sound interesting but uh imagine yourself working as a developer with a tight deadline you just need to finish this merge request and go home and on the other side you have have many tool andall on your ID in your

development environment telling you no no no you don't need to do this you don't need to do that and sometimes the results are uh inaccurate this is can be overwhelming from uh as a developer experience and also for the security teams it's not that easy to install and configure simply because uh developers use different technology use different ID and it's very possible to cover all these toolings so this is very terrible idea when you want to integrate security tooling next option you may have is cicd integration as of today Mo most of the vendors today provide plugins that you can install on your junkins for example in order to run security uh automated security scans but if you use this kind of

approach you may end up with pipeline that look like this and here you are just dumping bunch of tools to your developers generating hundred of results no one will review no no one will um be interested by this kind of U approaches and it will create definitely a chaos in your Dev uh devops pipelines so that's why we have now new approach again the old way to do just to sum up the old way of doing uh security integration either inside the Ci or in the IDE it doesn't guarantee you will achieve 100% coverage it can be easily bypassed uh on the Jenkins file pipeline definition for example you just need to switch the option that TR security scans

and it will disable all uh the scan uh that you have configured and also the fact that we have decentralized configuration it's very difficult to update a settings or made a change to the way you configure your scans and that's why we have a new approach which called pipelin integration pipelin security it's uh it's very simple the idea here is to um configure tools that will watch events from your secm tool and whenever for example you have a new code that has been merged your get repo this will automatically trigger scans somewhere that will verify your code instead of doing this uh directly in this CI and then it will provide isolated feedbacks directly to the person who did the

change and this kind of approach can be used to achieve 100% coverage from day one uh technically you just need to configure a web hook for each repo and then from there the security tooling will be able to to automatically trigger uh analysis whenever we have a change into your gate repo and the interesting thing about this approach is that you can manage everything in one place so once you get all your security tool in from the CI and have them in one place you can have full control on the configuration you can make change frequently etc etc and one interesting thing here is using this approach you can provide fast and private feedbacks so usually

when we run security scans we send the report directly to uh the mail address of a def team but this is not something we can do for example where when we scan uh when we detect secrets in the code you can't push all this uh the secret to a generic mail address with hundred of people here we can by watching the web hook we can detect who made the change who pushed the secrets and from there send isolated feedbacks and private feedbacks and also trigger other process to um for example reset the secret that has been leaked in order to implement this again you just need to configure web hooks that um will watch for events um that

happen inside your uh uh G repository and from there you will uh trigger uh workflow that will run security assessment here you have two main option either use custom script to code your workflows or use uh a solution that can handle this like KRA for example which is an open source solution to implement your workflows and the cool thing about this kind of platform is that you can reuse components into different workflows you can uh also uh Define these workflows as code AS yl file so it's very easy to update and uh maintain and this is basically our existing uh the workflow I'm using within my organization I'm not going to explain it um directly on the slides um

but this is something that we again and we can configure only as code so it's very easy to understand and also we can again reuse different components into different uh workflows let's see this uh now this workflows in [Music]

actions and how we did Implement pipel Security in our case so here here we have a simple setting file that contains um some parameters ready to to this specific project and here the starting or we start just by pushing a vulnerable code to our G REO okay so here we push the code and we commit the code to this remote get Repository and behind the scene we will have a tool that we scan the code and then detect if this new change contains any security issue here the code has been merged into into the remote brange and again behind the scene we have already a report a scanning report ready to chair with the corresponding uh

developers and it will be uh sent by email so as a developer I don't need to check on a different tool I will have the information straight away to my my mailbox and here for each security issue we point directly to the git repository so again we don't need to view the isue in any external tools we are just reusing the existing one to uh provide uh detailed information and specific information about the issue so here we understand that there is um SQL injection issue and Par to this we open a j tickets because everything is done uh using using tickets especially when working with Dev teams and inside the tickets we will have all the necessary

information in order to fix this issue

and

finally we have a Confluence page that contains that some simple secure code to reuse in order to [Music] fix each specific issue

and basically that's [Music] it so parall to this we tried to collect some metrics in order to evaluate the effectiveness of this approach and we did notice that um over time we had a significant um uh we had low um number of issues over time which was really interesting and regarding the time to fix or the efforts required to fix it issue we notice that it's also going down because as developers start to work on security tooling they started to get more and more familiar with um classic security issues okay let's move on to the next component which is Trad modeling most of classic tra modeling approach will require you to uh draw a diagram like this this is uh for example

in this tried methodology this means that you need to convert all your existing documents all your diagrams into something similar to this and I guess and we all agree that this is a painful task especially when you are working in a team with um huge workload and it's a very slow it's a very Road there is no better way to make security the worst part of someone's job and it's again it's very complex and we cannot use this kind of approach to review every change and it's difficult to scale and that's why we have now a new approach that try to leverage Ai and llms in order to at least automate the repetitive [Music] tasks it might seems um complicated at

the beginning to use this technology and Ed this technology into your existing workflows but we have today many Frameworks that can make it easy for us especially for someone who doesn't want to care much about what happening behind the scene like for example the L the L chain framework I'm not going to dive into the significant of each component but here the important one are first the llm llms integration component it will allow you to integrate with different llms uh like open AI clo etc etc and you can even change and combine these these these llms in one uh workflow and third component is document loaders is basically really interesting because it will allow you to enrich llms with

your own data data so for example if you have document architecture or security policies specific to your organization you can inject this um you can reuse this this uh this information along with an llms which is very powerful and last component is prompts prompts will be used to give instruction to llm agents in order to do a specific task for you and actually there was a nice research um that has been done to test how llms can be good at performing treat uh modeling so it the entry point was basic application called meal planner with a frontend API published behind the gateway and uh database to persist user data here the diagram was provided as C4

model language which is a way to represent a diagram as code in order to make it easy for llms to understand and uh use it as as an entry point so along with the architect diagram we provide also the project description and also a test user story to the longchain framework which is backed by different llms and instead of directly asking the agent to do the security assessment or the treat modeling um in one question here we split the the question or the task into two basic pump the first one will just list the data flows and different connection between the different components and the second we review each Connection in order to detect potential risk and the result of

this experiment was a bit interesting sometimes um some llms uh did provide relevant uh result to the scope but they were so generic here for example uh the mitigation is use https which is not something specific to this um to this uh use case but it's applied to any uh kind of architecture and also we had interesting result with different models like clo for example so here we were able to get volid threat which is floating the API uh Gateway and one interesting thing is that he managed to understand that we already have a security future to mitigate risk it was mentioned somewhere in the project description so we were able to understand this and use

it as a component to mitigate the risk and he did even challenge it because in the project description that has been provided in this study they didn't provide much details about how this future is implemented so he provided as um uh as a recommendation to uh review this uh this implementation and make sure that it will mitigate the risk also um last week the open AI security team released some Bots that they are using that they have been using to automate some of the uh most security tasks and one of the agents is called is the in C slack boat so this one basically act as an entry point to product team to decide if a new product

or a new future need to undergo a Security review this is really important because as a security folks we always get requests from different teams we don't know which one we need to handle first by using this kind of approach I believe we can focus only on the critical chains and the critical uh applications so what the future looks like with llms and um and trat modeling definitely this kind of Technology will reduce uh the workload spe especially when it comes to repetitive tasks I mean I think the um the road part of uh let's say trat modeling need to be done by machines not by uh by human and it's also uh important to customize

llm Frameworks and uh design our prompts but it's definitely worth it because it will save us hours and it will be more interesting to do this using llms than having someone um uh from security to do it every day and last Point uh in order to make a results more accurate and more relevant it's important to combine llms with rxs and also uh decision uh this will definitely help and prevent AI or llms from alysin eating okay let's do a recap in order to build scalable effective modern security program first regarding the scalable part I think it's interesting to not put everything on the CI this will all only slow down your process and it will be

difficult to manage it's better to implement external workflows that will just watch events from your uh existing uh Dev tools and then U send priv at uh uh security feedbacks directly to the corresponding team and this will definitely minimize the configuration effort on dev team side regarding effective it's very important to collect data and metric as much as possible because it's the only way to detect if you are going on into the right direction and regarding um the last Point uh I think it's time to invest more and more in llms especially on the IC field to automate basic and repetitive task this will allow us to focus only on things that matter and regarding the last point I

will I definitely recommend reading this boy uh this book life 3.0 which has been published I think six or five years ago um in this book we are trying to imagine how life will looks like in the at the age of AI and it bring some important question about how we can inject this technology into our daily lives um as an NPC engineer some of the question I ask myself do I have to maybe automate the most repetitive tasks and focus only as uh into the important one or maybe delicate everything and just chill and relax and the book uh explore many many other possibilities so I would definitely um recommend checking this book and why do

we need to ask this question today is because the earlier we know where we want to go the easier we'll be able to attend [Music] it again my name is my name is Abdu Samad and um it was a pleasure speaking at your conference and I'm looking forward to have your questions