BSidesSF 2026 - Your Threat Model Is Lying to You: Why Modeling the Design Isn’t... (Farshad Abasi)

Our next speaker is Farshad Abbasi, who's flown all the way from Vancouver, Canada. Big round of applause. >> [laughter] >> And Farshad will be talking on the topic your threat model is lying to you. Why modeling the design isn't enough in 2026. We will do Q&A at the end of the session if time permits. Please post your questions to Slido. If you haven't already gotten the link to Slido from the QR code, you can go to bsidesf.org/qna. With that, over to you, Farshad. Thank you so much for the introductions. Let me just make sure I place this correctly. Just full transparency, I can't really see any of you cuz there's a light pointing right in my eyes. So, I just see a

really bright light and I just see my laptop. So, if you have questions, I may not see your hands, but then there's what you mentioned, yeah, the question and answer system. So, we could use that. But yeah, thank you for coming to the session. Happy Saturday. Good afternoon. My name is Farshad Abbasi and we're going to talk about why your threat model is lying to you, right? A bit of a controversial topic, but we're going to find out why that is. So. There we go. Everything works. Great. So, a little bit about me. You were asking, "Hey, you run two companies." Yes, I do. It's a lot of work. I'm founder and chief executive officer for Ward

Security and Eureka DevSecOps. Both companies help software developers make more secure software. One is a services company, one's a product company. I've been in the industry, I've been building software and helping people secure them for about 29 years now. I started building software in the dot com days, you know, got my comp sci degree, went and worked for Intel and Motorola and a bunch of companies and built software for I don't know, 14 years. And then in the 2000s, when And world still built software following waterfall methodologies 2008. I got a job at HSBC in the product security team. So that's when I pivoted as a from being a software engineer that was building software that may not

have been secure and learned a lot of lessons through that time. A lot of hard lessons. And then I pivoted into teaching people how to avoid those lessons that I learned as an engineer the hard way. But it was an interesting time. I was at HSBC for 9 years and when I joined they were building software according to waterfall methodologies. And my job was to build an appsec team and scale it. And you know, appsec at the time was like hey build you know, you got the different stages. You got requirements, design, building and testing and then at each stage you do certain application security practices. And I'd never really done threat modeling. So you know, as a as a product

security engineer my job was to help all the development teams do threat modeling and do it along across the world and we had hundreds of teams all over the world. So I had to learn it myself. So I went and bought the classic Microsoft threat modeling book. Read it. Made sure I understand how this stuff worked and they kind of threw me in the in the in the field as you will. So I had to learn how to do it, become a practitioner and then you know, go and teach everybody else around the bank and how to do this stuff. So lots of experience in doing that. I think this went to the wrong slide.

So yeah, I've been doing teaching and and practicing threat modeling for 17 years. A couple of quick things about me. I'm also the president of B-Sides Vancouver. So come on up. May 31st, June 1st is our B-Sides. Not as big as and glamorous as this one. This is this is one of the best B-Sides I've been to. But ours is pretty good. And then I also run the OWASP Vancouver chapter lead. And I'm a co-author of a really important project is the OWASP secure pipeline verification standard. Pipelines are getting attacked more than ever. We've been doing a decent job in the last 10 years as an industry you know, maturing appsec. There's still a lot

more work to be done and I know that first hand working in the field. But uh security is becoming a problem. And so, there's an OWASP standard is called the SPVS, similar to ASVS for application security, so pipelines. So, uh check it out if you haven't. First release came out in October, great standard. So, today I want to challenge something that most of us take for granted about how we do threat modeling, right? First of all, I mean, I don't know if I'll see the hands, but I'd like to ask a show of hands, how many people are doing some form of threat modeling? Okay, not too bad, maybe half of the audience from what I can see. That's

good. That's good. So, half of you are doing threat modeling, and I probably bet that you're probably doing traditional threat modeling, you know, data flow diagrams, architecture diagrams, taking things apart, you know, breaking things into components and functions and all that kind of stuff and interfaces. How many people are you doing Agile or evil story threat modeling or abuse case threat modeling while they're building the software? Let's see. Not as many. In fact, I don't see any hands. So, well, a lot of people are doing traditional threat modeling, not many people are doing Agile or you know, abuse case threat modeling, and probably none of you are doing reverse threat modeling, looking at the data that comes

from your scanners and informing the threat model, which is the main topic of of this discussion. So, let's get into it. Design intent is not production reality, right? So, all the two types of threat modeling that people do, the one that most of you guys do, it's all around design, right? Hey, we're planning on designing this. But when it gets built and when it goes gets deployed, it doesn't often mimic the design. So, threat modeling was built for a time that when what you designed was very close to what you actually shipped, right? Waterfall days were when I started my work at HSBC. Fine, you know, you did the threat modeling during the design design stage, and then design got

finalized, you had the threat scenarios, you went to build it, there was change controls if the design had to change, and you reviewed the change controls and you threat modeled them, right? That still didn't work. We tried that at HSBC when we did waterfall. It was okay. A lot of the changes didn't go through threat modeling. It was haphazardly done. And then by the time we got that right, the world moved into Agile. So then it was like bank, you know, 2010, "Hey, guess what? You figured out how to do threat modeling in waterfall. Let's figure out how do we do that at scale and really quickly within Agile." And then by the time we try to

figure that out, 4 years later DevOps came. It was a mid-2010s. Like, "Okay, now you got to figure out how to do it for DevOps and do it fast at scale." So, you know, it's been around for a long time and it's always been focusing on design. And that assumption held for a long time, but it doesn't hold anymore. Modern-day delivery is fast, is distributed. You know, and it's constantly changing. You know, DevOps and Agile, right? That's that's essentially the idea is that you're constantly delivering, constantly changing. But our threat models, they're still design-time snapshots, right? You create them once, you file them away, and you never reconcile them with what actually gets deployed. So that's a problem.

So, three things have changed, right? So we got basically delivery speed, infrastructure drift, and dependency depth. What that's what's changed since I started threat modeling. Dora, which is the DevOps research and assessment group from Google Cloud, in the 2024 report, they classified teams into performance clusters. So they had the highest tier, which they called the elite, and they deploy on demand. And meanwhile, GitLab did a survey of 5,300 people and they found that 69% of CISOs said that they're shipping two times faster. I was talking to a client last week. She's like, "Hey Farshad, like the way we did threat modeling doesn't work. I'm over the last year, we've shipped 300% more code than ever. Obviously AI has

something to do with it, as you probably guessed, but things are moving fast. And a threat model that was done at sprint planning can be out, you know, can be outdated by the afternoon. Infrastructure drifts, you know, there was a company Firefly, they did a survey of over 350 people 50 users and only 6% of them had fully codified their cloud infrastructure. The rest of it was not codified, it was manually deployed. So, that means 94% of those companies that they were surveyed had production reality that diverged from their what the code had declared or what they understood their design to be. That's not a good place to position to be in. There's also the dependency dependency

death problem. A typical Java app with 20 direct dependencies can have 200 plus transitive transitive ones. So, your threat model might identify dependency risk as a category, but it doesn't inventory which 200 libraries are actually in your running system. That specificity specific tongue twister specificity specificity gap is what attackers are looking for to exploit. We'll come back to this with a more specific example, hopefully not a tongue twister. Um and these aren't edge cases. You know, these are realities. Um and this is the baseline for most organizations including where I worked at HSBC. So, quick show of hands. Who has I already asked this, but let me ask it a little bit differently. Who has a threat

model for production system that has not been updated in the last 6 months? So, a lot of you put your hands up. You said you had done threat modeling, but have you updated in the last 6 months? Let's see. Bunch of hands went up. So, so basically you know, I expected a lot of the hands to go up because most companies, they don't you know, they do this threat modeling and they just forget it. Like they put the create a bunch of diagrams, they put them somewhere and they don't update them you know, often enough. And there's no judgment here because there's no survey data that says organizations regularly update threat models post design. I mean, I'm assuming I didn't see a lot

of hands because you're either really ahead of the curve or you guys are being polite or you're just assuming I can't see. So, probably didn't put your hands up. But the reality that I've seen is that people do threat modeling and they shelve the documentation, including a lot of our own customers. You know, we deal with a lot of organizations in fintech and health tech, and I see that many of them don't do threat modeling. If they do, it's been shelved off somewhere. They're not updating it. They're not referring it to it regularly. And and also in many cases, they're just not doing it at the right place and at the right time. So, oops.

So, here we go, right? Like the stats say in the state of threat modeling survey that was done, they say 88% of practitioners are using stride, right? And my beef with that is when you ask people, "Hey, how do you do threat modeling?" They're like, "I use stride methodology." Stride is not a methodology. Stride is a classification system for that it can help you think about threat modeling, but it doesn't have a process. It's not like, "Hey, do the step one and step two whatever." It's like you find the threat scenarios based on the categories, you know, spoofing repudiation tampering etc. And then you classify them. Um but, you know, fine. Most people are 88% say they

use stride. 74% say diagrams are mandatory, right? But again, a lot of customers we work with, they don't even have architecture diagrams or data flow diagrams. They're building so quickly, devops teams, agile teams that design documents and architecture are secondary. That's pretty That's pretty common out there. So, it's important So, if you're using stride, that's great. Some of you uh probably wrote the stride templates for that your teams are using. Uh but, uh I'm not here to tell you that that's broken, right? That's good. I mean, I still teach people how to do stride. I still uh help our clients do it. So, there's a there's definitely a place for the traditional threat modeling that's been done and even for

evil stories. So, we're not here to talk about that. We're talking We're here to talk about the gaps. So, the methodology's fine. The problem is where it stops, cuz it stops at design intention. It doesn't go beyond that. Hey, what happened after you coded it and you deployed it? Does that inform your threat model? We're not doing that actively. So, you know, it gets even worse. 52% of people surveyed have no management reporting, right? And and only 25% have a dashboard. Most organizations produce a threat model and then nothing, right? Our clients, the same thing. They hire us, we do all this threat modeling for them, we give them all these diagrams and it just gets shelved. At HSBC, same

thing. We spent my team did a lot of threat modeling and then, you know, there was a disconnect between the threat modeling team and the testing team or the people that were running the scanners. So, there's a lot of different problems with this. And the other thing is a framework like OWASP SAM, they say, you know, their highest maturity level recommends reviewing your threat models regularly, but then they say, e.g. yearly. I mean, so even the the aspiration is only annual. So, think about that, right? Annual is not enough. This needs to be happening on a regular basis. So, evil stories add value. I mean, evil stories there's a there's there there's something that you should be doing. Some

So, some teams go beyond doing STRIDE and they write evil stories or abuse cases during sprint refinement, right? If your team does that, that's great. Genuinely, it's awesome. It brings security, uh the type of thinking more closer to the development where it should be. So, now, could you write an evil story about transitive dependency risk? Yeah, you could. You could write one about I am role drift? Yeah, you could write about that. Could you write one about an accidental debug endpoint in production? Sure, right? You could write about those. You could say, "Hey, I'm assuming that maybe someone would accidentally deploy um some endpoint, I don't know, in production." Yes. Good threat models do anticipate these

categories, right? But it's really hard to dig into that at that granularity. And here's what the story can't tell you. Which specific transitive dependency is vulnerable right now, right? Or which specific I am role drifted this week. Whether that debug endpoint that got deployed into production um last Thursday is still there or not, right? This category is predictable, but the specific instance requires uh you know requires the whoops which am I lost? So they're lost. Requires evidence from the production. So you basically have to gather the evidence. So it's not enough to stop at design time. We should go beyond design time and gather evidence and bring it back and inform our threat model. So even the strongest version of design

time threat modeling predicts risk categories, right? It doesn't verify what actually exists in the running system. It's a bit of a disconnect. And that verification requires a feedback loop, right? So there's a lot of work in the industry to go forward, but there hasn't been a lot of work to go backwards from the results of the scans to inform the threat model. And that's the best case scenario. So let's talk about what actually happens instead. So instead, you got the you know once and done, you got us the compliance people, and then you got the no feedback loop, right? So I see these three patterns pretty commonly you know in client engagements. And I bet some of you have seen these as

well. So you know the there's the once and done, right? The team does a threat model. Usually they do it at project kickoff. It goes into confluence or SharePoint or they hire us to do it. We do it with them. It's painful. And then nobody opens it again. I've literally asked the teams to show me their threat model and they they have to search for it. They don't even know where it is sometimes. And and that's not good, right? The other one that I see is the compliance checkbox. Hey, we're SOC 2. We need to be ISO 27001. They say you need to do annual threat modeling. Let's do that. We need to do a tabletop.

They'll do something, but it's more to to satisfy the checkbox. It doesn't inform a single engineering decision. And the auditor you know the auditor never whoops this button is really sensitive here. Um the auditor never asked if the threat model was accurate, right? Did you have one? Great. You had one. So you pass. And and and then the third one here is there's no feedback loop, right? This is the one that hurts the most. The team, they run the pen test, they get great findings. Sometimes they hire us, we do a pen test for them, and then their report is just shelved. They just or they they make tickets. The tickets go into their Jira backlog somewhere. So,

there's really good findings, they're real vulnerabilities, but the threat model is never updated with what those findings revealed about the broken assumptions that they'd been made on the threat model, right? So, SafeCode documented these exact failures back in 2017. Eight years later, they're still the norm of how we do things. So, let's talk about why this persists. Right? BSIMM, I'm sure you've heard of it, right? BSIMM has constantly found that dedicated security staff represent about 1% of the development organization. That's a fairly low number. And um you know, I remember like HSBC, our AppSec team initially had about like seven, then later on I think we had like 21 globally. How many developers did we

have? Thousands. How many development teams? Hundreds. The ratio was really low. And And the management didn't quite understand like I remember when uh when we started doing Agile development, they came to my team say, "Hey, we got Agile developers are building stuff like using Agile methodologies. Can you assign a consultant to each development team?" I said, "Well, look, that's just not physically possible cuz how many development teams you have? How many people I have in my team?" And so, you know, that wasn't just us having that problem. And then later on in the mid-2010s or you know, in that 2010-2012 time frame, the concept of security champions uh came about, right? Um I don't know if Stanley's in this room,

but there's a there's a company Catalyst. They do a really good job with a product for security champions, but the idea of security champions was to uh address that shortage, right? Like so, how can we scale AppSec without having an AppSec person for every developer? So, it's a problem. Uh Security Compass found that less than 10% of organizations threat model 90% or more of their applications. That's really low, right? Not many companies are doing it, and the ones that are doing it, they're not doing it on 100% of their apps. And uh so, let me paint this picture for you. You've got a really small security team, right? 1% of the you know, development the the the staff is is is

handling product security. Um you've got modeling a fraction of applications out there, producing models that aren't updated. I mean, why we wonder why there's a gap between design and reality, right? It's challenging. So, BSIM 16 uh said that, you know, I think it came out last month. They said uh they interviewed 111 organizations, 91,000 applications, and the finding was that AI has overtaken all of the other forces uh reshaping security priorities for these companies in the last 16 years, right? That's pretty big. But, um this is what I find really remarkable. Um they still have no explicit activity about feeding scan results back into uh the threat models. Not one, right? That's still missing. Then they put them in separate domains.

Uh the structural separation between their So, they basically what they do is they put threat modeling under the threat intelligence domain, and they put the security automation scanning under secure SSDLC uh touch points, right? So, um >> [snorts] >> so this basically uh the structural separation between the intelligence domain and SSDLC touch point persists even as everything else is changing. So, let me show you what the gap actually looks like and uh with numbers. All right, so blind spot. Your model guides decisions while ignoring production evidence, right? So, this is the core problem. Your threat model says one thing, production says another, and the model never finds out. You're not informing the model from the production. Maybe

some people are, but not the ones I've worked with. So, let me make this concrete. And let's see. There we go.

So 6% right? 6% of organizations have fully codified their cloud infrastructure. That's basically uh based on a Firefly survey, like I mentioned earlier. 350 companies were surveyed um and 6% have codified it and the other 95% 94% of organizations have portions of their cloud environment that are managed manually, right? Outside of infrastructure as code. So, when you're doing it manually, do you even know what the architecture looks like? Are you even going to have diagrams for it? Probably not cuz most of them are, you know, creating the DFDs and and the diagrams aligned and the alignment with what they have in in their code, right? Infrastructure as code, Terraform templates, CloudFormation, things like that. So, 94% of them are doing manually, so

they probably have no idea what it looks like. It's invisible to pipelines, is ungoverned, and is prone to drift. Less than a third of companies continuously monitor for drift. So, the gap between what the code says and what exists is growing quite a bit. So, even as teams adopt more infrastructure as code, right? Cuz I mean, there's just a problem. If your threat model is based on your architecture diagram, right? And your architecture diagrams are based on your infrastructure as code. And your ISC only covers 6% of what's actually running, what exactly is your threat model protecting? Right? So um numbers. This XM Cyber, they found that 80% of real-world security exposures came from how systems are configured and deployed.

Misconfiguration, over compromised roles, uh you know, exposed storage, uh drifted architecture, right? All these are problems that contribute to this. Now, can you threat model these, right? Absolutely. A good cloud threat model should include uh misconfiguration risk, excessive privileges, storage exposure, and it should prescribe controls. Least privilege, encryption at rest, access logging. That's the correct way to do threat modeling, right? So, why does 80% of exposure still come from these categories? Well, two reasons. First, the controls prescribed by the threat model may not be implemented correctly across every resource in production. Second, configuration drift over time and the threat model is never reconciled with what actually exists. And this is what the problem is. The model says, "Hey, apply least

privilege right?" But production has 24 to 47 Lambdas and three of them have admin access that nobody remembers granting. So, yeah, 1%, right? There's that 1% over there. What is that all about? So, only about 1% involves code level CVEs, right? That's interesting cuz CVEs, they're in a I mean, if you have a CWE, CWE is a pattern. When there's a specific instance of that pattern, it becomes a CVE in a particular product in a particular area. So, you would think, "Hey, CVEs would contribute to a lot of threat scenarios." But it's actually only 1% of the problems were attributed to CVEs. Your threat model can predict the CWE categories that produce those CVEs. Injection, off laws, they're all

designable. 80% is also designable in theory. The evidence says that it's not being addressed in practice. That's the problem and that's the gap. So, your threat model covers design intent and prescribes the right control, but without feedback from production, you never know whether those controls are actually in place across your entire environment. So, let's talk about two breaches that can help you get get a concrete sense of this. So, Capital One is one of my favorite examples. I use this in a lot of my talks. Um, but you know, it happened in 2019. Most of you probably remember it. But, um, you know, uh, basically there was a it was kind of boring SSRF, right? Server side request

forgery. Um, it's a well understood problem. A competent pen tester would have found it very easily, right? But, what made this catastrophic was not the SSRF itself. It was the architecture level trust assumptions that were surrounding it, right? So, look at that middle card there. I can't see it, but hopefully you can see it. I can see it on my screen here. Um, the EC2 instance running the WAF had an IAM role with excessive privileges, right? And AWS has the metadata service there, IMDS version one. The metadata service was accessible from that role, right? The attacker used SSRF to get temporary credentials from the metadata service, then listed and read every S3 bucket in that account.

Um, there was an ACM paper by Khan, um, and colleagues at MIT. Uh, and then they put it perfectly. They said the EC2 instance functioned as both a security control and attack vector simultaneously, right? So, think about that. Think about that. The WAF was supposed to protect the application. Instead, it was the thing that got it exploit exploited it, right? The SSRF was detectable. Any competent pen tester finds that. But, the chain of trust assumptions that amplified it, the the over permissioned WAF role, uh, the accessible metadata service, the fact that WAF was simultaneously security control and an attack vector, that's not a code problem. That's a deployment architecture problem. Could a deep enough threat model have caught, um,

this at design time? Maybe. If someone had asked, uh, you know, what happens if the WAF instance is compromised? Like, if someone had asked that, and what can the WAF role access, right? But, the model was done at a higher level. They just said, "Oh, well, we got a laugh." So, they didn't really go down into like what happens if this happens something happens to the laugh. It treated the laugh as a trust boundary, not as a potential attack vector. Right? The deployment reality diverged from the design assumption, right? Cuz the design was like, "Hey, this is a laugh." But then reality, it was a EC2 instance running mod security, wasn't the dependency wasn't updated. You know the

rest of the story. So, the deployment really diverged from the design assumption. And this is the design versus reality gap that I'm talking about. Right? That So, this is a configuration gap. But let me show you about a dependency one, right? So, the capital one was a configuration example, but log for shell is a dependency example. So, same lots blind spot different vector. You all remember log for shell log for J, right? Back in December 2021, it was a lot of people's nightmares. Um, you know, it resulted in remote code execution in log for J via JNDI. Um, so, let's walk through this. Your application imports spring. Spring imports something that imports something else. And that imports log for J, right? Three

to five levels deep. Your developers, they didn't choose log for J. They didn't know it was there, but it was running in production with a CVSS score 10 vulnerability. CrowdStrike noted that one of the biggest challenges during the initial response was visibility. Organizations simply just didn't know um, you know, where this was running and that or that or that they were using it. So, the ones that had S-bomb responded in hours, they could they could search their dependency inventory and answer, um, you know, where are we affected immediately. Organizations without S-bomb, it took 2 weeks. Some never fully confirmed their exposure. Now, here's the kicker here's the nuance. A good threat model might include, uh,

you know, vulnerabilities in third-party dependency as a risk. And it should prescribe a control. Keep dependency scanned and updated, right? That's pretty standard in 2026. Hopefully you're all doing that. And that's the right control and SCA tools absolutely can find transitive dependency vulnerabilities. That's what they're built for. Snyk, Dependabot, OWASP Dependency Check, Dep Dep Scan, they can all do this. They all resolve the full dependency tree. So, why was Log4Shell a crisis? Two reasons. First, many organizations in 2021, they weren't running SCA back then, right? SBOM and all that the executive order came out and it changed a lot of things. So, that was one. Um second one and this is the one that matters for this particular talk is that

the organizations that were running SCA, they were treating the output as tickets. They found vulnerabilities, put out Jira tickets, prioritized them in the backlog. What they didn't do was feed those findings back into the threat model right? And ask, does this change our understanding of the attack surface for the component? Does this reveal something the model missed about what actually uh got deployed? The threat model prescribed the right controls. The tool existed to execute it. The evidence was being generated. SCA results were coming out. The feedback loop to the model didn't exist, right? Nobody closed that loop. And that's not a failure of threat modeling methodology. It's not It's not a failure of tooling. It's It's a

practice gap. The same practice gap we've been talking about, right? And the evidence existed. The question wasn't being asked. S- simple as that. And development speed makes both of these even worse, right? So, you know, the I remember remember I talked about DORA, the 2024 report from uh Google Cloud. So, they they categorize uh you know, the development teams into several elites, high, and medium, right? Elite category, they're putting out stuff all the time. Um high category, daily to weekly releases. The medium one, they're putting stuff weekly to monthly, right? So, the de- deployment frequency amplifies this problem. Um basically, the the top tier is deploying all the time. Even medium performers are doing weekly to monthly. If you update

your strap model at design time and your team is deploying on demand, how long is that model accurate for? Hours? Minutes? You tell me, right? Um no study directly qualifies strap model salience relative to deployment frequency. That I I haven't seen one. Um this is a logical inference. The math is pretty obvious, right? So, the gap is real, is growing, and we've seen two breaches in the so far in this talk that exploited it. Now, let me show you something you already have the data to close it. This is important. You may already have the evidence, right? So, here's the thing I want you to take away from this talk um more than anything else.

Um your pipelines may be already generating the evidence you need. You're treating it as ticket backlog instead of actually informing your strap model. This is the missed opportunity, and let's talk about that. Let's talk about how those signals can actually help you. So, you're probably running some combination of SAS, SCA, DAST, maybe IAST on your cloud infrastructure. Um hopefully all of them, at least one of them. Let's talk about each of these and how they can help you. So, SAS finds code level weaknesses, right? Sometimes it confirms that uh what your strap model already predicted. The model said you have an injection risk at component X. SAS found SQLI. That's a confirmation. The model was right. The

code just didn't implement the control. That's what happened, right? Like we modeled it, but the code didn't happen to implement it. So, that's a confirmation. That's an implementation gap. It's not a model gap, right? But sometimes SAS finds something in a component of code path that the model didn't account for. So, as an example, a developer added a direct service-to-service call that bypasses the API gateway, right? So, your architecture said all the service-to-service calls need to go through API gateway, but developer decided to direct do a direct call. This [snorts] wasn't in the design. The threat model never considered the specific data flow. Now you're on SAST, they find a vulnerability in that pathway um in that path. This is discovery,

right? This is not confirmation. It reveals um it reveals architecture that diverged from the design, right? That's pretty important to catch. So, I'm not really doing for time. Half hour, that's good. Um So, what what what about SCA? SCA finds dependencies the team didn't know were present. Sound familiar? Um so, Log4Shell, um SCA would have found Log4j. The threat model might prescribe scan and update dependencies as a control. SCA is the evidence that tells you whether that control actually covers your real dependency tree. What about DAST? DAST reveals runtime behaviors, endpoints that respond differently than design, off-by-path paths, off-by-pass paths that only exist in running application. A debug endpoint someone deployed to production and forgot to remove. Now, you have

modeled accidental deployment artifact as a threat category. But, yes, but but DAST is what tells you uh what actually happened, where it happened, and how um it's exposed. The model predicts the possibility, DAST provides the evidence that the possibility materialized. What about IAC, right? IAC and cloud scanning they find drift, misconfigurations, trust boundary uh issues um that didn't match the architecture diagram. These all sound familiar probably, Capital One. Um your model prescribed least privilege, cloud scanning tells you where that stuff might be, right? So, this all should be complementary and go hand in hand. So, all of this is evidence about whether your design intent matches your operational reality. Every build, every deploy you're generating is this data,

and right now, probably generating it, too, right? It's sitting in your issue tracker as you got tickets. Nobody's asking what it means for the threat model. Nobody's Nobody's closing that gap. So um this is the key distinction for everything I'm about to show you, right? So, there's a confirmation on that side, and then there's discovery on the other side. Confirmation is when finding a vulnerability validates a risk in your model that already predicted it, right? We identified injection at risk at this component. SAS found SQL live. Good. The model works. The code just didn't follow through with on the control. So, file the ticket, fix the code, and move on. But, the model doesn't need to change.

However, discovery is different. It's when a finding surfaces something the model missed entirely. Not a predicted risk that materialized, but a gap in the model's understanding of the system. An unmodeled component, a dependency the team didn't know about, a trust boundary that was drawn in the wrong place because the deployment diverged um from the design. So, another question. How many of you have ever taken a pen test finding or scan result and asked, "Does this tell me something about my threat model and what it could have gotten wrong?" Let's see a show of hands. Okay. Just a couple of people. So, not a common practice, right? Um >> [snorts] >> Yeah, I mean, it's uh these things are

challenging. So, uh let me just go here to just go back. So, I'm having problems with uh There we go. So, yeah, that's the gap. It's not the tools, it's not the methodology, um the interpretation step. The question that nobody asks, right? Capital One, nobody asked what the over-permissioned laugh I am role meant for the trust boundaries. This design assumed the laugh was a security control. The deployment made it an attack vector. Log for shell, nobody asked what transitive dependencies meant for their attack surface, the model might have predicted a supply chain risk as a category, but it didn't show Log4j was there. The findings existed, the questions didn't. Right? So, what does this finding tell us about our

threat model well what our threat model got wrong, right? This is the question that almost nobody asks. This is the entire gap. It's not a tooling problem. Maybe there's no tools today, but there will be tomorrow. I know a couple of companies that are trying to build one, ours included. It's not a methodology problem. It's a practice gap. Nobody in the organization holds both the scan and the threat model, right? Like at HSBC, I was in the threat modeling team and then there was another team that did the pen testing and the scanning and the two of them didn't talk to each other. So, the person running the scans files tickets, the person who made the threat

model, they move on to the next project like my team did, and the architect who drew the trust boundaries hasn't looked at the model since the design reviews, right? That's just the reality we're facing out there. So, let me show you how to fix that. And you don't need a new paradigm to do this. So, the practical extension is the six steps, no new tools required. You can do this in 2 weeks, right? So, let's talk about a bit of a solution in the 10 minutes that we got left here. Um what I'm about to show you is a six-step workflow you can implement Monday with your colleagues, right? You can use a spreadsheet, your issue

tracker, bring your threat model, and that's all you really need and a little bit of patience. No new basically you don't need budget approval. It's just a change in practice and I'm not proposing a replacement for your design time threat modeling. That's important. I'm proposing an extension of that. You add one thing, a post-build step that reconciles findings with the model, right? So, here the six steps. First, export your most recent scan findings. Take your SAST, SCA, DAST, whatever you're doing. If you run all of them, great. If you only run SCA, start there. The tools don't matter, it's the practice that matters more. Step two, map each finding into the component in your architecture where was

detected. This sounds simple but is pretty powerful. It forces you to connect your scan output to your architecture model. Many teams have never done this mapping. The scan team and the architecture teams don't even talk to each other in most cases as I mentioned earlier. Then step three for each finding ask one question. Was this risk modeled in our threat model for this component? Yes or no, right? And this is important. If yes, then you have the confirmation. The model predicted the risk and testing found it. Good. Check the box and move on, right? If no, you have something interesting, a gap, something the model missed, right? And that's where discovery lives. So, let's see the second half. So, in

step four, when the answer is no, this wasn't modeled, you ask the follow-up question, right? What assumption did the model make that this finding contradicts? Was the trust boundary wrong? Is there a component that the model didn't know about? Is there a data flow that only exists in deployment? Then you go to step five and you update your threat model. You add the newly threat, fix the broken assumption, add the newly discovered component, and now your threat model just got closer to reality. It evolved. And step six, not all gaps are equal, right? So, prioritization is important. A wrong trust boundary or a missing component is a systemic gap. That one finding probably means that

there are more like it. However, a one-off code error is something it's important to fix, but it doesn't change the model. So, focus your model updates on structural misses, right? So, the workflow in action, I'll just go through this real quick. So, let me walk through a concrete example so you can see how this works. So, you run your SE on your payment processing service, standard scan. The report comes with back with a CVE in Jackson Databind, right? Deserialization vulnerability. But here's the thing, your team, they didn't import Jackson Databind directly. it came in a transitive transitive dependency of your ORM, right? So, step one, export the CI the SCA report. Step two, do the mapping. Um you know, find um

the map it to the particular component. So, this was in our payment processing service component. And then in step three, ask a question. Was deserialization risk in transitive ORM dependency modeled in our threat model for that payment service? Now, your threat model might say, "Deserialization risk for the payment service was done." But, did it account for deserialization in a library that your team didn't choose? Right? Or didn't know was there. And doesn't maintain, right? The risk category might be modeled, but the specific attack surface wasn't. So, that's a no, right? And then step four, you look at what assumptions broke. The model assumed that the team had visibility into the dependencies, but they didn't, right? The trust

boundary was drawn around direct imports, and the transitive dependencies were invisible. Step five, update the model, add the transitive dependency attack surface as a threat to the payment component. Now, your model reflects reality, right? And then step six, prioritize. This is structural, not a one-off, right? If one component has unmodeled transitive dependencies, all of them probably do. This is an organizational blind spot, not a component level bug. So, one SCA finding, one question, the model just got smarter, and you know where to look next. So, here's a sort of putting it together. So, you have six vulnerabilities, what mapping looks like, right? So, here's what it looks like if you have a few of them. They've

got six of them. You just put them all there, map them against the components. Finding which component, was it already modeled? And if it wasn't, what is the gap type? So, by doing this exercise, and you can see this three of them are confirmations, right? So, in the in the re-rows where there's a yes, there's a sequel in the search endpoint, weak TLS, and a public S3 bucket. All of them are modeled, and they're all expected. Your threat model just predicted uh predicted them, right? But, there are three discoveries in the discovery ones those were things that your threat model didn't predict, and there's an opportunity for improvement. So, notice the gap types, missing component class, unmodeled deployment

artifact, wrong trust boundary. These are structural problems. They tell you something about your architecture understanding, not just about one bug. So, you can build this in a table on a spreadsheet, you know, take some time out of your first pass. It may take a little bit longer, but it will change how you think about your scan results. So, I think we have 5 minutes left, so I'm going to quickly go through these. Um Where it fits in your process, you can do it at backlog refinement, you can do it at pull request review, you can do it at release readiness. It depends. Um you know, this doesn't require a new meeting. It fits into your existing

ceremonies that you have already. And this is this isn't just a team-level problem, right? The frameworks themselves reinforce separation. So, BSIMM, like I said earlier, threat modeling is an intelligence, testing is an SSDLC touchpoints. No activity connecting them, right? Um BSIMM is pretty descriptive. It documents what companies do. So, this separation tells you something. The industry doesn't connect these practices. Even OWASP SAMM and um you know, the the NIST SSDF, same pattern. Even the European Cyber Resilience Act mandates both um that that doesn't require connecting them, right? Like the US executive order and the NIST SSDF, they say you should do threat modeling, and you should do vulnerability management, and all the testing, but there's nothing that

requires you to connect them. So, we're almost at the end here. Um Multiple companies are building stuff. I know threat modelers trying to build something, Sneak EV Clarity I, but they're all working in the forward direction, right? So, basically, they're all on the confirmation side. None of them are trying to figure out the discovery side. And full transparency, that's something we're working on. Um you know, it's not an easy problem, but uh you know, something that we're hoping to address. And managing the noise. So, I can hear the objections. They're like, "Hey Farshad, you know, there's a lot of noise. How do we deal with it? Uh but we already have too many findings. Uh you

know, do I feed more stuff into my threat model?" But, you know, you signal quality over volume, right? So, again, map them into components. Not everything is going to introduce a new threat scenario. Once you map into components, then you'll notice the themes, multiple findings pointing to the same problem. And then you also want to find out the novel ones. Like, was this component dependency or data flow in the threat model? If not, that's discovery. Cadence and ownership. So, start with a pilot. You know, you're the security champion. Take one application, one scan type in two sprints, and prove the concept. If it works well, do it at every sprint. The developer to security team can do it together. Then there's

different intentions at doing them at different stages here, right? Like, when you do it at release, that's your pre-release threat model reconciliation. You block on structural gaps, but then also you want to look at the organization. You also want to have that cross-application pattern review. Um you know, updating the organization's threat threat library. So, each of these have a different reason for being here. So, in final, here's your 2-week pilot, right? In week one, pick one production application, pull the latest scan findings, locate it in the threat model, run the six-step workflow, document what you find. Week two, share the results with the team, count the gaps found, update the threat model, and present to your lead, and

decide whether to continue. If you find zero gaps, I owe you a coffee. But, I think you'll see that when you do this, you'll find things that your threat model hadn't accounted for. And these are some metrics. Um gaps discovered, model freshness, coverage delta, structural versus one-off ratio. These are all um things that uh to keep buying to keep the buy-in from management, you need to show value, and these are good four metrics that you can use. So, I know we only have a couple of minutes but um there's a QR code if you want to connect with me on LinkedIn. I love talking about these things. Again, I've been doing threat modeling for 17 years. I

have a lot of people to thank for for paving the way before I got into this. Adam Shostack, there's a whole bunch of other people. It's a great field. Um there's even a threat connect now. It's a website you can go in and talk about threat modeling. So, there's a lot of great stuff, but the threat modeling that ends at design is not sufficient. That's what I want to leave you with, right? The goal of threat modeling hasn't changed. What needs to change are the inputs. Design alone is not enough. Your pipeline already has some evidence. So, do the two-week pilot. I encourage you. I challenge you to do it. And uh you know, link connect with me on

LinkedIn. Happy to hear your results. One application, six steps, and uh let's talk about it. Thank you. Awesome. Thank you so much, Farshad. Uh we have we have time for maybe one question. Why does threat model have to handle this? Threat model is already unfairly considered too heavy weight. Why not add this to config management or dependency management? >> repeat the question again? Yeah. Why does threat model have to handle this? Threat model is already unfairly considered too heavy weight. Why not add this to config management or dependency management? >> I don't know. Like um it's an interesting question. Okay, I mean why would we add it to config management? This is what you're trying

to do is figure out how attackers can attack your system. That's what the threat model's for. So, if you don't know all the possible ways that your application can be attacked, if you don't know all the threat scenarios, that's a gap. That's what you need you need to you need to I you need to be sure of the the ways that your application can be attacked, and there's probably lots and lots, right? Those of you who have ran the Microsoft Threat Modeling Tool have seen it. It's It's crazy. Like you put in the data, and then it gives you like hundreds of threat scenarios. I would never use that tool. But But the thing is that um you do need to know

what your high-risk threat scenarios are, maybe your medium-risk threat scenarios are. And in order to do that, you need to have a threat model that paints the full picture. If your threat model has gaps, if you only modeled what you were designing, right? Like I was saying earlier, design documents are never updated. I don't say never. There are lots of great people out there, say 80/20 rule, 20% of you are probably doing it, but 80% are not, right? So, what's the point of that? You threat model the design, you shelve it away. What does it tell you? If you want that threat model to work for you, it needs to be up to date and it needs to reflect

the reality of what's been in put into production. And it's not that hard. You run the scanners, right? You run whatever prowler in your cloud environment. It gives you all these, "Hey, there's a wildcard S3 bucket, right?" Map that into your threat model. What does that tell you? That's what we do for our clients, by the way, right? We've been in my company for 7 years. This is what we've done manually for our clients. But, you know, I'm hoping that tools can come out and automate this. I'm hoping some of you will take this away and incorporate these practices in your day-to-day. I'm happy to talk more outside if you want to chat. Particularly, this question is an

interesting one. Awesome. Thank you so much, Farshad. >> No problem. And thanks everyone for attending this session.