An Introduction to Application Security Testing

All right, our next talk is by Daniel Alrich. Uh he's a Western New York native with a decade of experience in healthcare industry where he's worn worn many hats across infrastructure and operations, security, quality assurance, and web development. Born and western New York, he earned a bachelor's in information security from Sunni as we go and MBA from the University of Buffalo. feeling his passion for blending technology with practical innovation and this talk is an introduction to application security testing.

Thank you. Um that introduction kind of stole some of my about me slide here, but I see a camera there too. I tend to pace and walk back and forth, so hopefully I'm not out of the frame too much, but uh thank you for the introduction here. Happy to be speaking today. So my current position uh that I'm holding today is is uh manager of our performance testing and also application security testing for high mark blue cross field. I will say disclaimer none of the stuff I'm talking about today has any association with my career position but happy to share what I've learned over the last several years of being in the application security testing space and hopefully you guys

find it valuable as well. Um it's just some of my interests and hobbies here. Um we just welcomed a daughter into the family back in February. It's my wife and I in the picture on the bottom there. Uh we like to go travel and hike. Um so a couple of those pictures, one's from Utah. We got to cool stock canyons last summer uh before the baby came and then also do some hiking in Colorado. That's mount blue sky there as well. Uh the picture on the top left on the screen there is uh my view from my desk at work. We're in Senica One Tower now on the 36th floor. So I have a pretty

cool uh rocking space up there now. um I get to go to each day. So what we're going to be covering today here's a high level agenda. We'll do some introduction into the topic and and testing in general and then dive a little bit into application security testing. Talk about some of the program components from a high level. uh do a little bit deeper dive into a few of the testing types just to highlight and try to get a better understanding of um talk a little about triaging and reporting and then um you know getting started this any of the talk today resonates with you and you want to learn a little bit more I have a slide on some

resources that are out there learning resources and uh to get um started within the field also at the end I'm going to try to do a live cahoot we'll see how this goes I have some free t-shirts to give away so um we'll have a little uh game at the end of All right. So, introduction into application security. Um, apps are pretty much everywhere today as you guys all very well know. Apps are on your phone. You use web apps on a daily basis. And thread actors really do like applications. Uh, it's a nice way that they can find internet facing apps uh that they can find threats out there on and exploit those threats. But taking a

step back, where does application security fit in the grand scheme of things? You'll see this kind of rainbow chart on the screen here in the different the seven different layers of security. Application is pretty close to your mission critical assets out there. So, it's it's pretty important. It's it's one layer away from your company's gold or your organization's gold in the sense um and the only thing that sits between that is is the actual data level of security as well. So that's what we're going to be focusing on today is is the application layer. Um uh just by show of hands, just curious, is anyone um in a test engineer role today or doing any type of security testing in

their day-to-day jobs? Few hands. Okay. All right. Great. So this is more intended I'm I'm by no means an expert in in these topics. You know, I'm I'm I've been learning over the last year few years and and happy to share some some information. Uh the other thing to call out, where does application security fit into like the corresponding teams? Um and it's kind of in the middle of of a few teams out there and I tried to represent with this honeycomb chart. You got your DevOps engineering team that security testers will often work with configuring your CI/CD pipelines. You have your developers that you should be consulting with and trying to educate them on the vulnerabilities that are

discovered. You have your favorite governance, risk, and compliance folks who can set the standards and and policies as a guiding light. um vulnerability governance uh as well is out there to help manage the open vulnerabilities within an organization. And last but not least, QA uh the quality assurance organization and depending on the size of your company, these may be separate teams. It may be one person wearing the hat across all of these functions. Uh really depends on the size of the organization and the industry that they're in. If it's a highly regulated industry like finance or healthcare, you're going to have more dedicated resources to this type of of work. All right. So, what are the risks or or

real stakes of application security? So, when I was putting this presentation together, it was actually kind of hard to narrow list find a narrow list of of real world attacks that have been happened in the past couple years that were exploited uh from application security fails. So, I calling out a few here on the left hand side of the screen. There's some outdated software leak uh that exposed some um customers data from AT&T going back in 2023. Uh there was an SCA fail which we'll dive into in a little bit and talk about leaked uh cloud keys and secrets. This is very common. There's there's many many instances of this happening and we'll learn a little bit more in the

Verizon uh data breach investigation report in a little bit. Uh, and then to call a couple other different types. Um, infrastructure as code is is, uh, I want to say a newer concept that's been around for a little bit now, but securing your infrastructure and making sure it's configured in the right way. Um, Comcast leaked 36 million uh, users data based on leaving some AWS bucket leaked information out there. And then the last one, SQL injection uh, impacted United Health's uh, 100 million patients exposed. 100 million. Like think about the size of that. It's almost like one in every three Americans data was exposed in that breach alone. Like that's wild to me that that that scale which

could have been prevented or at least uh avoided um from fixing a a detectable issue um with with certain tools and processes in place. So the risks we talked uh uh financial pits. This is an obvious one. um when when a breach happens and you show up in the news, your stocks will typically go down. You'll lose a lot of investors, the trust with them as well, and not to mention the legal fees and and headaches that you have to deal with um after the fact and and all the work that should be done rightfully so to make your processes better resulting in a breach. And then what about AI? So, I I feel like I had to include something about AI

in this presentation since it's very relevant. Um, everyone's using AI in every every day. Um, so just show of hands like um who per I'm just going to go through a list of different AI agents. Uh, who uses um, uh, we'll go through each chat GPT. Any chat GBT fans? How about Grock? Any um, help me out. What other popular Gemini? Gemini. Yep. That's a big one as well. So So everyone has their kind of preferred uh, AI agent um, out there. And with it, it's very easy to generate code. Um, it's progressed a lot in the last year or two. If you try to sit down and build a web application from scratch a year

and a half, two years ago, which I tried, I I struggled with it um based on the code that was given. You try to do it this week and it's much easier to to spin something up. Now, whether or not that code is secure is another question. It's very easy to generate code, but is the AI at a point now where it's defaulting to generating secure code? I would say no. Not saying it won't be there eventually, but it's made it even more important to for this for these types of testing to be aware of since it's made it more accessible for for engineers to build stuff in the field. All right, to call out a few stats. Um,

like I mentioned, the Verizon data breach investigation report. Um, if you haven't heard of this report, Verizon does a good job and puts out a breach report each year and it has a good high highle executive summary. It even has some jokes and puns within it. So it's it's it adds some fun into the actual data itself. I'm not going to read through all of these, but a few of them called out. 30% of the breaches in their sample size involve a third party. So no longer is your company just your company. It's it's the partners you deal with, it's all the third party software you pull into your product. So it it's getting more and more challenging as

time goes on to control your your scope of where your uh threats exist with the software you build. Uh, another one that surprised me, 94 days is the middle time or median time to remediate leak secrets. That's over 3 months that threat actors have on average that if they find a key, they can discover what it's used for and try to access different things. So, that was wild um or surprising to me. And then, uh, 81% of the thread actors were external, leaving the remainder to be internal threat actors, which is a little surprising as well. So, not only testing your externally facing websites, but making sure you get the proper controls and roles um within your applications

and APIs even internally. Kind of that zero trust uh concept and mentality.

All right. Um so, taking kind of step above and looking at the program from a like a a high level, what components make up a successful ABSSEAC program? and then we'll dive into a little bit more into the testing and details of it. Uh so I thought of you know this f these are mine I'm sure if you Google you'll find different components out there but these are the fi five I kind of logically separated what I thought would make up a good appsite program. The first being the policies and standards that come from your governance, risk and compliance um uh department. And this is really just giving your developers clear standards on how to build the software,

what um repositories to go to and and giving them more of a playbook. Uh the second very important component in my mind is secure code training. If we get really good at finding and identifying vulnerabilities, that's great. Maybe even fixing them as well. But to fix the problem at the source here, the concept of shifting left, it's really to stop the systemic issue of injecting all these vulnerabilities and educating developers with proper uh training. Um there's a lot of training and a lot of responsibility developers and engineers have today uh and educating them from when they start or get onboarded uh is an important piece. Number three is secure SDLC process. So this is all

about integrating your tools into your CI/CD, making things happen automatically. uh and and ensuring that um the tools are integrated and not just with the CI/CD pipeline but also within the IDE. That's the earliest point where a developer can get some feedback on how their code is is looking and and um detect any issues early on. Uh number four is is uh application security testing. So making sure you have the right testing types depending on the application itself, the right level of testing maybe depending on how important that application is and the frequency of it as well. Uh which will this will be the majority of the presentation. And then last one vulnerability management. So

putting some effort around uh having like a risk register or an open vulnerability list that is managed and and known by the different application owners. Um application owners or product owners have a lot to worry about. uh this would be one more thing added onto their list of things they want to um at least know the risk. Sometimes the risk is accepted and it's not actually remediated. Uh but educating the application owners and product owners on what risk they have out there is an important piece. All right. Uh moving on to the software testing types. So before we dive into application security testing, I think it'd be a good just highle quick overview of the different types of

testing. You kind of split them up in two ways. There's functional testing and then there's nonfunctional testing. I'm not going to go through each one of these boxes, just to call out a few. For functional, you have the most lowest layer of unit testing to test specific pieces of code all the way up to user acceptance testing where you have a dedicated or maybe have a dedicated environment where real business users are going in looking at whatever changes are going in and signing off on those changes. That's very important for the business users to be happy with the product and therefore that they're getting signed off. switching over to nonfunctional. So, the functional is all

about what the app does. Nonfunctional is how does that app perform? And there's different um types of non-functional testing, probably even more than what's on the the slide here, but just to call out a few. Uh performance testing, making sure that the app runs stable under load. There's all different types of performance testing, stress testing, spike testing, but we won't go into that today. Uh and then another one to call out, accessibility testing. So this is a newer type of testing that's spawn up over the last few years and I find it pretty fascinating is ensuring the app is accessible for those with disabilities and there's been a lot of standards and tools even published to

help with that. So these non-functional testing are a little bit of a niche and and more of a unique skill set but you'll see more of these jobs I feel pop up over time as as companies start to realize that these types of testing are important. Um, I think that covers most of it on this slide. I'm just looking through. All right. Uh, diving into the application security testing types. There's there are more types than this on the screen, but I wanted to call out a lot of the main ones here, talk about them from a high level, and then dive in a little bit deeper in a few of these. So, on the lefth hand side, we have the

testing types. Uh, what is it? if there's ID integration or not. Um who it's typically performed by and then some highle key characteristics of each of these. Uh so who in the room has heard of SAS testing before? Anyone? That's the most popular one. If you've ever heard of security testing, that one's probably the one that your company's doing at minimum. Um this is all about scanning the source code for vulnerabilities. Um there are some level of false pauses with it and I won't go dive because the next slide we dive a little bit deeper into it. Um but typically this is in an early phase. It's automated uh and it can find different vulnerabilities within the

source code and it's pre-run time. That's an important uh factor to remember. Secret detection is a sort of a a subcomponent of SAS. you can kind of roll it up it but there's been tools that have come out that have gotten really good at detecting uh patterns of certain secrets that are leaked and and may be committed. Um so there so that has evolved in the last several years. SCA is a wide encompassing topic uh software composition analysis. You may have heard of things like dependency scanning or or even um license scanning. that's kind of all rolled up into SEA which we'll talk a little about a bit about as well. Um, typically this is

performed by the developers themselves or a centralized security team. It is dependency focused automated and it tracks a lot of open CVEs that are that are tied to the software components that are known and out there. The next one's a really cool one. I I think the team members on my team find this one the most interesting is DAST. This is dynamic application security testing. This is where you get to start playing around with an actual running application in a test environment. You don't want to test against production obviously. Uh so you're doing DA testing. Um there's kind of different levels of it in a way. I I call them out separately here, but D to me is more of

a level one. It's it's automated. You configure tools to log in and maybe run some business flows or or point and it's really good at trying a lot of different things quickly. And then you have more of a manual application penetration test which is more your traditional pen testing and you're you're trying more advanced techniques uh as well. So for the next one it's infrastructure is code scanning. This is all about the code. So as infrastructure has evolved over time, we want to be able to spin up infrastructure easily, have it uh source controlled and even um configured with different customizations based on your your end customers. So if I'm uh a server guide or a database

guide, no longer am I building that from scratch. I'm expected now to use uh different tooling like Terraform or or other open source uh uh frameworks out there to to spin it up and configure it in a certain way. Uh so what's important about that is it's great and it's easy to spin stuff up, but making sure it's done in a secure way and and having those controls. There's a lot of security tools out there that uh enforce it from a runtime or after it's uh deployed and can or detect issues. This scanning is a little bit pre that you want to identify as you're making those uh Terraform playbooks or modules or or infrastructures code playbooks and be

able to detect, hey, is my S3 bucket going to be open to the world if I if I write it in this way versus adding a new security configuration. Uh this is also where genetic can help you. Um, if you're new to some of these concepts, you can, you know, use it as a personal assistant and ask a questions as you go through and and and making sure it's okay from a security check standpoint. And the last one is container scanning. Um so containers can be pretty complex. I don't know if you guys attended the uh container talk earlier this morning. Um there is a lot that goes on within containers and making sure that your

images are hardened or the the packages that are pulled into those containers are um secure and container scanning is one of the more newer types of security testing I would say and there's definitely tooling out there that that handles this and this is more runtime focused and you can deploy it and you're running um either Kubernetes environment or other types of container environments. ments to find issues. So, a lot of different testing types. Sometimes you will focus in one and become an expert in one. Some others will just uh be more wide across and know the basics in each and then some of them are even developers are are expected or or trained to be able to do

themselves like SAS testing for example the whole left concept. All right, time to dive a little deeper into each testing type. So the format will kind of remain the same over the next few slides. Um so some of the strengths of SAS and secret detection early detection. So this is the most cost effective way to to fix vulnerabilities. There's a lot of time and effort that gets spent into doing SAS testing, identifying a vulnerability, writing it up, meeting with the right app owners to uh communicate it. Then they have to do their own research. Then they have to implement the fix and we have to retest to make sure the fix isn't breaking more things or or ejecting more

vulnerabilities and there's the whole release and audit and documentation portion of it. So as you can imagine one small vulnerability you can put a price tag on it from a pretty high standpoint. Um if you're in a larger organization that may have more red tape than than a smaller organization for example. Uh the other strength is it IDE integration. So a lot of the SAS tools that are out there have plugins um with give you live feedback. So you'll see ongoing processes and scanning and provide you lists of issues that are um categorized by their default severity. And the other great thing is a lot of this is automated through your pipeline itself. Weaknesses um SAS can be

particularly noisy if you haven't configured your SAS tool or if you're just using out of the box rule sets um and just the nature of how the testing works. SAS only has so much context about your environment and the way your application would run. So it can have a lot of high false positives and developers do not like false positives and neither no one really likes them but they're they come with the nature of of uh the tools. I've heard some industry standards around 10% of the findings or so are false positives. If you're if you're benchmarking around there you're about average. you try to shoot for a little bit better than than 10% for for

some of the um uh issues that come out. And then another weakness is depending on the type of scan. So just your basic scans may not be doing cross file or cross function analysis. uh for example, Senrep I think out of the box is very lightweight and it's very quick and it might not do as deep of a scan as for example uh like an enterprise checkbox tool where it's taking a lot longer to deep dive in and go through from file to function. You might be thinking why is it important to go from cross file to cross cross function into different files. Um, one thing at least my team team members always um are challenged

with is tracing back user input. So determining where this variable is coming from, what functions it's propagating all the way up to and it can be challenging to to trace that code back. Um, and SAS tools out of the box sometimes don't do a great job with that. Some vulnerability examples down there. These are probably pretty popular ones you guys have heard of. uh SQL injections, cross-link scripting, and hardcoded API keys or SQL detection uh which is a a subp part of um of SAS in general. And then some of the tools on there you'll see uh some of the names kind of remain from a commercial side. So on the slide, if you would have done

the same slide 10 years ago, you would have saw a lot less names on the commercial space. What has happened in the industry? This is common across a lot of tools and just companies in general. um these larger companies are buying other companies up to complement their product suite. So more and more companies are trying to be a holistic solve all your problems and more across their tool sets. Uh Jrog has has expanded theirs, GitLab has, Checkmarks has bought companies over the years and expanded out their lineup um as well. So I at least wanted to show there's definitely some other open- source tools out there. I just chose the the popular standard Senre. Um there used to be a

few more open source projects but it seems that Senrep has become uh the the uh kind of I don't want to say sole winner but some of them have gone down um and and combined into SER prep instead of maintaining their own. We're using Trivia and that's very good as well. Trivia. Yeah, I did hear that tool this morning. I I probably should have added uh that into the presentation. I was talking with someone and and not a tool I'm too familiar with as well. Anything to add? Yeah, I think on open source side there's some like language specific ones like go set is pretty good for go but it doesn't handle right. Yeah, absolutely. Yep. There's there's

language specific ones as well and that's where we'll find a little bit more in SCA. So I'll start there with with the tooling. Um there are there are some open source tools for software composition analysis and language specific ones. Greek transition. So, if you're in there writing maybe an Angular UI using Node, you've got your out of the box npm commands you can run. Um, an npm audit is one of them. I I forget the exact Python, but similar for pip packages. Uh, this is all about checking your dependencies and and seeing if there are known CDs out there that are tied to those dependencies. And these tools do a great job of pulling that

information in. And just because you did a scan today on your code and you don't change anything tomorrow, if you do a new scan tomorrow or at least check the the same packages, you might have more or less or there might be a difference in the number of CVS that are associated with your code. So this can this can cause some challenges with maintaining and and if you think about software from an enterprise standpoint the number of applications and the number of packages that are pulled in this data you can get very overwhelmed very quickly the data that SCA tools uh spit out. Um there's multiple components of it. There's the actual dependencies themselves. There's

your base images that contain dependencies out of the box and there's also um uh the license portion of it which we didn't really talk about at all. As a as a if you're selling commercial software, there are certain licenses that you're allowed to uh include and not include within your software. And I think that's a a big gap at least from my talks to other individuals in the in the field um is that that's that's something that hasn't really been figured out uh from an industry standpoint or even a tooling standpoint. So some strengths of of SCA is that you get to have some awareness into what licenses are out there and SBOM generation. So Sbomb's software build of

materials. This is all the things that make up your software. It's a great concept. Everyone should have sbombs for the software that they build and test. It makes it more transparent internally of what's included in your software. Um, and you can have a record over time and and do great things like SCA scanning with with Sbombs as well. Uh, there is some IDE integration as you code and pull in a new package. It can do a scan on that and make sure that there's no high severity CVES with it. There's also some automation through CI/CD pipelines with this type of uh testing and tooling. Uh and it's it's pretty quick at how it how it operates. Uh some

weaknesses is just because it's identified as CV doesn't necessarily always mean that there's a fix. Uh and and also contextual analysis can be difficult. So looking through all this data, you may not know if this CVE even applies for your software. that can be challenging from a security tester and as well as a developer trying to um fix the the open issue that's out there. So there's some noise in there I would say and some contextual analysis can be difficult. Some of the products offer like they put their own research on top of the CVS and then you can also combine it with SAS where your SCA tool and SAS can kind of talk together and then it

can determine or make a better informed decision or recommendation on is this truly applicable or not. Uh it's rapidly changing. There's new CDs coming out every day. There's new packages that are changing. Uh and then some of the vulnerability examples. Uh on the left hand side here we see uh log for shell. So who is who is around or in a security role for log for J. So that was like my first week as a uh going into my position uh being over security testing team and and after the first week or two I was like is this how it always is guys? They're like no this is a more unique situation. So I thought uh I

learned a lot from that and and but this is the type of testing that could be done to sort of detect um some of these vulnerabilities that are out there. The license violations which I mentioned and then there are some interesting ones as well. So so Olaf has a bunch of lists which we'll talk about in a bit. Um but there are some attacks out there. One in particular compromise of legitimate packages. So this is when um a developer will join anonymously to a project, have some legitimate contributions to that project, build trust up with the other contributors, and then what they'll do is then they will inject some malicious code after they're, you know, deemed a

maintainer or maybe an owner of that project. And everyone who's been using that project for years now, they may or may not be aware that malicious code had been injected. There was a really interesting story where a guy from Microsoft found something in like the Linux kernel and he he posted a research uh about it and he noticed some latency um differences for doing some simple commands and they were able to find and stop this uh from being actually loaded in and it would have a huge wide impact across the the world and it was a really interesting read. like those type of attacks are just so unique in in the way it will all go up to dependency

management. Uh name confusion attacks. We learned a little this a little bit about um the top level domain if anyone went to that talk a little bit. So a package maybe um have this one name and they might change like a uh an L to an I or a one or they might misspell it slightly. You might think you're downloading a package but you're really downloading a a different package instead. those some name confusion attacks as well. Uh tooling out there, like I mentioned before, I think a lot of the similar players are out there. Jrog, I think particularly does a pretty good job at um storing artifacts and being able to scan all those artifacts

as well. That's one tool I've uh used over the years. Had some success with as well. All right, going into Daston application penetration uh testing. So some of the strengths here is you get to find vulnerabilities that you can actually reproduce. So showing it in code is always great and it could lead to some false positives, but when you have a demo you can bring to a development team and say, "Hey, look at what I could do to your application or look what data I could steal out or look at this bad configuration that's that's out there running live in either your API or or web application." Um, I feel like it has a little more weight to it and it's and

it's more fun for the testers to actually go through and do. Uh, so it's runtime detection, uh, like I mentioned and it's and what's nice with some of the Dash tools is you can do real world business flow. So maybe you have a signup flow and you buy some products or or you have certain business flows you want to test every release. You can use these DAS tools to not only log in and and go through um but you can use it to actually use test data to fill out forms and submit things and go across depending on the tool you're using. It's automated and it has a pretty deep uh depth of testing. Some of the weaknesses

if you are doing those business flows now you have to manage users you got to manage test data you have to make sure that um the application is up in those environments test environments if anyone's worked in test environments sometimes they're not always uh behaving as you'd expect them or maybe not as functional as you'd hope when when crunch time comes to test. Uh so the vulnerabilities example here that there's some common names up there. privilege installation is a little new. And then file upload. I'll call that one out. Um that one's pretty interesting. I think uh some of the web apps out there have a lot of file upload functionality. So making sure that you're only allowing

certain types and scanning them for for malware. Um uh that's something that an application pen tester or DA could could discover and try out. Uh last weakness to call out. Uh this type of testing takes time and a unique skill. It's not something that can be done in in minutes or hours. It's it's more long uh typically longer than that. Um and it's a niche uh set of skills question. Yeah. Go ahead. All right. So So there's often confusion um you know around dash and I asked dash and I asked. Yeah. And um you know beyond having lower false positive rates. Yeah. Are there other typical use cases where you would prefer IAS over DAST and how

do you navigate the performance issues that cause because of this? That's always the concern with is so for those who are not familiar is like the next evolution of DAS where you actually install an agent that goes into your application and it will be able to listen to traffic and it will combine that with the DAS data itself and then it will hopefully give you less false positives. If you ever talk to anyone as an application, you say, "Let me install an agent where your app runs," they're going to be very hesitant. That's my experience. Um, no one ever wants more uh latency or or more issues that could go wrong. Um, so I don't think is maybe

as popular as some of the other types, but the intent of it is really to reduce the false positives. Um, so it's really just depending on um, do you care more about the false positives and and can you take that maybe slight hit in performance and convince your product owners or are you okay with just out of the box? Sure, we'll take the question. Just to add to the slide we had on the previous one. Yeah. Um, so for the burp sweet port swinger. Yeah. One of the we the downsides obviously you can use it for quite a bit with the plugin suite but what is some of them are very resource heavy resource intensive. If

you're using Burp plugins, absolutely I'll get down big time. So, it's just knowing what um knowing what plugins you have is important. It's awesome. Burp Suite Pro is like the tool to use for uh pen testing, app pen testing for sure. It's got like repeater. It's got all these uh great functionality features and a great community around it. But absolutely, if you're if you're depending on how you use it, you could go very slow and it can be ineffective at times as well. OS snap. This is another one to call out from open web application security here. Um, and yeah, I think that's about it from a tooling standpoint. All right, just a couple more slides to go here and then

we'll dive into the cahoot. Um, vulnerability triage and reporting. So now we've done all this testing. We have all these vulnerabilities and how do we communicate that effectively? Um, and then what type of reporting is used. So, if you've ever been involved in a third party test, you'll see a nice pretty report come through with colors kind of look looking at what's on the screen with severity differences and it's very organized. I think that's important for any type of testing to be done. Having a standardized format that prioritizes your issues because development teams don't have an unlimited time and budget. They want to focus on the most highest impact, the easiest to fix issues first

and they want an easy way to do that. They don't want to waste time on confusing reports. So the tools aren't perfect. Um the goal here is to reduce the false positives and and as I mentioned JAI can help learn these uh prioritization with risk factors. So don't always take what the tool says. There there are things to consider beyond the severity that's recommended by the tool like attack vector impact complexity uh privileges required to actually perform the attack. Um and [Applause] yeah the impact. So is it a is the application or attack going to expose something on your customerf facing uh core web application or is it more just an internal app that doesn't have

sensitive information right those can be the same SQL injection vulnerability for each of those could could be varying different impacts and and urgency to communicate another recommendation is use some of the standards that are out there so CVS scores are out there for a reason to help you score severity and Then alas top 10 has a lot of um categories. So who's who's heard of alas top 10 before a categorization. So uh it's a great way to categorize vulnerability or types or categories of vulnerabilities into the top 10 list. Um but they also have and stands for open web application security project. Uh they have a bunch of other lists they've published as well. So, not only web

apps, they have one for API, one for mobile applications, one for LLMs that recently came out, one for open- source software, which has some really interesting um uh uh categories and and I think those are the main ones that that I know of uh and have looked at before. So, last wrapping this up, reporting matters, know your audience. This is for any type of presentation. the information you give to an executive level is going to be very different than the information you need to be provided to the devs uh themselves to fix the um CV the vulnerability issues themselves. So last slide here is um getting started in apps what kind of resources are out

there if you're interested in this type of field or maybe want to play around with some of these testing types. Uh and this is not a fully inclusive list. I'm sure if you Google or use your favorite AI agent AI agent, come up with some new ones. Um, but so some free learning resources, OAS, I encourage everyone to go out there and look at not only the top 10 list, but some of the other projects that are out there. Maybe get involved. Maybe something sparks your interest and you want to help contribute to. Uh, Port Swigger has a web security academy. This is more hands-on training. Highly recommend this if you're interested in um application penetration

testing. And then there's some other tools out there which do a really good job of the offensive side. So you can look at code and try to look at a SQL injection and and and ways to look at it. But when you see it from the offensive side of things and kind of pair the two together, it's almost like another light bulb goes off on your head and it makes it easier to understand. So I think some of these tools have progressed a lot over the last few years in showing that offensive side combined with hey here's how to do the attack and here's how you fix it. And to me, that's the best way um I found for my team

members to to learn and train. Uh some appjack apps job titles. I just threw out a few. There's probably a bunch more out here. Uh test engineers, application screen engineers. I would say don't get thrown off just with the title. A lot of companies now are moving to more generic uh titles and descriptions. So, you don't really know until you meet with them and see it might be a generic like dev sec ops. Some days I feel like more of a DevOps engineer than I do an actual security uh tester or manager because we are we work with them so closely for the pipeline work. Um so so don't get discouraged just on the titles

themselves. Make sure you go to that next step of screening to learn what the position does. Uh there's some skills posted here and some certifications. I would say don't go crazy on the certifications. Um I've done a lot of interviewing where people have had certifications and maybe uh you know it didn't fit as well for the position. I think they're useful to learn specific knowledge. Um, some of them are pretty expensive to get as well. So, just be mindful and balance maybe uh your own projects and learning versus the certifications themselves. All right. Uh, so the last thing to call out here some skills. These are some important skills for a security tester um to be successful in their role. And

some of these are across all types of testing, but communication and collaboration. There's a lot of uh working together that goes on within the teams to confirm uh analytical and problem solving. A lot of times you have to dive deep to uncover a root cause. Root cause analysis is a very important part of the job in any type of testing role. And continuous learning as we all know things change daytoday especially with AI and Gentic AI coming out now evolving. Uh you just have to be open to changing and learning new technologies and ways of testing. And then the ability to read and understand code. So you don't have to be an expert actually writing the application, but knowing a

basic level understanding and what does this code do or using tools to understand that code. All right. So we we're at uh 340 here. So next um going to do a quick uh cahoot here and I'll give um some t-shirts here. I got some OAS t-shirts in different sizes and colors and we'll we'll give uh some t-shirts out to the top three um three participants in the game here. Go over and actually start this.

This works. Uh, looks like it's coming up with an error in one second here.

Apologize for that. I'm not connected to the internet. That would be why that was before. Try that again. So, while I'm pulling that up, any questions? So that's kind of like the end of the any any questions or things to share with the group. Maybe you guys have come across tools that have been helpful in the past or have an experience to share. See question in the front. I don't know so much if it's a question but a comment. Um and maybe you have an comments about my comment but uh on your slide about how to become uh good in appsac or something. Your number two was something about training developers or something absack program.

So Bill, a holistic program is important for secure code training. Correct. Yeah, secure code training. Exactly. Number two. And um so what it's been my experience so far is that um you can have the most advanced coders on the planet but many many times they are so overworked with their timelines are so tight and the organization doesn't give uh enough money to either hire enough coders or alternatively even provide testing tools secure testing tools underfunded is a is a big challenge we face in other fields. Y and um so it's very difficult to write secure code when you all you can do is like race to cobble something together that works. Yeah, that that definitely depends on

your um sprint cycles like how quickly and often you reduce or are putting out features or changes to your code and I I would say that's definitely a challenge we face is funding and and timing is priority today. developers have thousands thing to do and unfortunately vulnerabilities fall down here. Like I manage a performance team and a and a vulnerability team. When a an app team has a performance issue versus a vulnerability issue, the app team is way more engaged and more wanting to fix their performance issue versus a vulnerability. So fortunately sometimes it falls down. Um so hold on for one question. So if anyone wants to join the game here, we'll get started here in

just a minute. I'll take one more question. Uh you just have to scan the QR code. Let me click on it to make it bigger. Uh there we go. Does that make it easier for everyone to join? Um and then go ahead with your your question if you'd like and then we'll get started. I guess it's uh also more common. One of the things uh that I observe in the south was uh during the very early threat modeling and architecture. Mhm. there's hardly any attention paid to security provisions and uh issues like that. Uh I I teach software engineering basically say before you write a single line of spec and I forget the code spec that has

to be one of the top priorities. Sure. And uh it's a good point to make. Yeah. And it seems like that's where most of the shortcuts are, especially especially on the management side. Yeah. Yeah. Architecture. Can we get into this? Yep. Absolutely. Uh focus in architecture and threaten. I don't really talk about that. It's connected in a way to application security, but I left it out of this this speak since there was so much information already. All right. Looks like we got a bunch of people joined. We're going to start here. There's 20 seconds for each question. There's like eight or nine questions. Um, and there's a leaderboard and you're also awarded for right points

and how quickly you answered. So, those are kind of the rules and we'll we'll go through. We get questions get harder as as it goes as well. Good luck to everyone.

The first one is what is a key reason apps are targeted by threat actors.

All right. Apps often have exploitable flaws. That was uh an easier one here. All right. Daniel's going to lead with 902. Good name. I like that name. Is it you? No. That'd be playing my own game, I think. All right. Which of these is not a component of an application security program? So, we talked about five components. Which one is not part of it?

Right.

So fishing awareness campaigns are not application security. It's more general security awareness campaigns. Got that one right as well. All right. Moving on. We have a change in leader here. Are you professor? All right. Question three. Which of these types testing is not nonfunctional? It's like a tongue twister. I don't love the word nonfunctional either. Like when I say I'm a non-functional manager, it's like doesn't sound right. testing manager, right? Unit testing is is more of a functional type of testing. Correct answer.

MMCS is in the lead now. We've had a new leader each round. This is another easy one. Most

people have answered already. They're getting the hang of answering quickly. Okay. Yeah. All right. Static application security testing is the correct answer. All right. MCS is still up there.

type of testing scans source code before runtime for vulnerabilities.

All right, SAS is the correct answer here. So SAS is what runs uh before runtime.

What is the strength of daft testing? Couple questions left.

detects

runtimes flaws in the running application. That's the important piece here for DAS is you get to test against a live application, right? Uh two more questions to go, I think. Oh, three more.

Primary goal vulnerability triaging an appseac right as we rock stars here. Next,

which of the following is not Noah's top 10 list? We talked about a few top 10s out there. Which one is the one that's not a top 10 project from the awesome community?

Getting a little harder now.

All right, we had a few less answers there, but most people got it. Top 10 risks for Python. There may be a top 10 list out there, but it's not actively maintained by OASP. Good job there, 18 folks. All right, last question. What was the median time to mediate leaked secrets from the Verizon report that we showed in the beginning?

That's a guess. 24 days, 163, 24 hours, or 33 days is the answer here. All right, let's see our final winner here. Hoping it shows the top three at least. All right, so come up after me if you're one of these top three and we can get you a size and a color. Uh, I can't read all those names that quick, but MMC, who are you? Who are you? All right, so that concludes the talk. Um, we've got uh want to leave any secrets.

There we go. Yep. So, any any final questions, any comments, shares to store, stories to share with the group? We're just about at time. All right, that's it. If you want a t-shirt, please come up.