How We Scaled Our Threat Modeling Program In The AI Era - Javier Ortega Palacios and Juan Jose Lopez

Thank you very much everyone for Can you hear me well like this? Yeah, okay, cool. Thank you very much everyone for for coming first of all and for, you know, all the organizers to to have us. We are very excited to present this project to you. Um, so we call the presentation how we scale our threat modeling program in the AI era. So this a project that started in October 2024 at Twilio and basically transform how we build threat models in our company. We wanted to present the journey that we did, what is the the automation that we call TMA or threat model assistant and what is the how we did it and how can

you do it in your own companies. So I have here with me Juan Jose Lopez. He's the threat modeling program manager and I'm Javier Ortega, I'm the offensive security program manager at Twilio. First I wanted to set the base with the critical role of threat modeling. So why threat modeling is important and why we do it in in our companies. What is the problems that arise in threat modeling and what why it doesn't scale at today's AI pace and the solution that we built, the AI TMA or threat model assistant that we were we were commenting before. Last thing, I will leave some time for questions as well if you have of the presentation of the technology itself

and a call to action, you know, we want you to build your own TMAs. So so we will give you a bit of of examples and things that you can do to build your own AI assistants. So first of all, what is threat modeling, right? So threat modeling is a security process, that's the first thing, that is identifying threats, enumerating and assessing them and create counter measures for them, right? So you have a threat, for example, a SQL injection, right? That's the threat in a in a web application. The enumeration is that this is an important threat that you want to prioritize and the counter measure is to create parameterized queries, right? That's something that

everyone would agree with me. And there are three main frameworks. How many of you have used or heard of stride, for example? I see a lot of hands, right? So, you are in the right presentation. Um Mitre attack, pasta, and other frameworks are are widely used. At Twilio, for example, we use a stride, but you can use a a different one at your at your own companies. Why we do threat modeling, right? We say this is very important. Yes, Harvey, very very good, but why we are doing this? As a as a overall, I would say security, we are trying to do shift left security or secure by default to embed security in part of the our software

development life cycle. Yes, and we have heard this a lot of times. Put the security in the requirement phase. Put the security in the minds of developers. Uh put this in the design phase. Why we are doing this? Not only for security purposes, but we have a more robust system, but as well because it's cheaper. It's way cheaper to fix things or security vulnerabilities when they are in the left-hand side, in the requirements of design phase, than when they are already in production and this creates an incident for you. Now, what's the problem with you know, we have cursor, we have codex, we have Gemini, we have all of these great tools that they generate a lot of code, but

what is the problem with threat modeling? Uh you can see in this graph and this comes from GitHub, so this is public GitHub repositories. We have from 2022, I don't know if you can see it well, but from 2022 to now, basically, we have double the amount of code that we push per month, okay? Uh from developers' perspective as well, we have double the amount of developers or people that I sign up to create OSS or like open source software open source software in in GitHub. So, we have a ton of code, right? Um now, the question is, your threat modeling program, has it increased or has it doubled as well in uh you know, do you do the double the

amount of threat models? That's the the question. I won't give you the answer. Now, before the AI boom, so let's say 2022, before we had all of these tools, there was a scaling bottleneck. So, there was 200 developers per one application security professional, and this was already a problem back in 2022, right? Now, imagine all of those 200 dots that you see there that are software developers, imagine that they generate double the code. Now, we have the same application security professional, and they haven't done anything new to scale their your threat modeling program, they can generate the same amount of threat models. So, before we used to have a problem, now we have a big problem.

Okay? Um this is from 2023 from a company called Security Compass. They did a survey of different, you know, companies, different sizes, etc. And they said that 1/3 of the companies in 2023 thought that scalability was a problem. So, I'm sure that that number I'm not sure what's going to be in 2026, but I can tell you it's going to be more probably. Uh threat modeling general, the sessions take days, right? That you have to check the designs, take them with the developers, do calls. Uh so, it takes like days or even weeks to schedule and complete a whole threat model. The documentation as well becomes updated instantly. So, they develop a new feature, they need to wait again,

they are blocked, they don't like the the process. They're like, "Oh, these guys in security, they are really bad, you know, they they they they take days to to fix this." Or they take days to tell us these things. So, the documentation as well, you know, it doesn't is is not scalable, right? This this process. The commercial tools There are commercial tools to automate this, but there is no one commercial tool that fits all. So, there's going to be a way to, you know, put it into your own development process or into your own companies. And as well, they have a difficulty of difficulty of integration, and they have a cost associated with it as well, which

is not is not great. So, it brings us all of these new challenges that we were commenting, the amount of code and the scalability problems, but it gives us what a really good opportunity to fix all of these issues. >> [clears throat] >> Thank you, Heather. Well, at Qualio, we faced all those issues. We had a huge scalability problem. Uh engineering teams came to us asking for security review, asking for [clears throat] security help, and we needed well, you can see, maybe weeks to get a result and to get to give them feedback. So, we started thinking in a in a solution. With the boom of LLM applications, we started thinking in a in an automation to automatically create

threat models for our engineering teams, and this is how we started thinking in in the threat model assistant in in TMA. So, at the end of 2024, we started working on the project, trying to to think how to perform this and implement this automation. And well, after a few months in 2025, we started the test. And well, we we we started getting the the first results of TMA. And we we could see that instead of needing weeks to get a result for our engineering teams, they could get a threat model in just a a few minutes. Well, this is how TMA works nowadays. So, initially, the idea was to create an automation to automatically create threat models, but

at the end, after months of implementations, we created an an orchestrator. So, actually, we have a process for security reviews. So, engineering teams need to create a request for us. We create a ticket, we create the threat model, etc. So, at the end, TMA does everything for us. It's an automation to completely automate the security review process. The idea is to allow to create requests using the tools that engineering teams already use, the the tools that they know. So, if an engineering team needs to create a security review for their new service, they just need to create a request. They just need to send us and a technical document with all the specifications of the of the new service. We also ask some

security questions. For example, if the service is processing PII, if the service is public-facing, etc. We get all that information. We send that information to the LLM provider, and this is where the magic happens. So, the LLM provider, in our case, we are using OpenAI, put all that information together and using the context of the company, in our case, the information and the context of Twilio, and security policies, security standards, security controls, how to implement those security controls. We ask the model to create a a result. In this case, a a threat model. So, we get that threat model and we put all that information in a in a tracking platform. In our case, we are

using Jira. So, at the end of the process, in just a few minutes, the engineering team has a a new Jira ticket with the threat model and the necessary task to continue the the process. So, at the end, instead of needing maybe 2 weeks to get all all that information, engineering teams can have everything in just a a few minutes. One of the main components of TMA is the is the model. The LLM provider, at the end, the LLM provider is the one that is going to create the the result, the the threat model for us. So, it is important to select a model that works for us. So, this model need to have three main

requirements: vision capabilities, because we ask teams to send us the information in a Google Doc with all the information, all the technical requirements of the service under review, and usually these documents has, uh have um, images, data flow diagrams, sequence diagrams, architecture diagrams. So, we need a model and with visual capabilities to understand, uh, those, uh, those images, okay? Also, of course, we need a model with visual capabilities. Nowadays, there are different models, uh, with different capabilities. There are models for, uh, for reasoning tasks. So, we need to select one model that has these visual capabilities because we are going to ask the model to analyze all the information of the service under review and identify

the threats and the security controls to mitigate or eliminate those those threats. And third, uh, and this is this part is very important, we need to give the, the LLM provider, the model, the, the context of the company. Because if our company works, for example, with, uh, AWS, we need, uh, solutions and security controls related to to to that context. So, for in this case, for, uh, AWS. So, we need to add the, the model, the security standards, again, the, the policies of the company, all the context to create good results that apply for the engineering team of of the company. In our case, uh, for Twilio. Once we know the model, we need to

instruct the model how to how to act, okay? So, we need to create, um, a system prompt and we are going to instruct the model to act as a security engineer with many years of experience. Uh, we can set the, the purpose of the, um, of the task that is going to perform, the context, etc. One important part of this configuration is to, uh, tell the, the model how to send us back the, the, the, the result. In our case, we are using JSON, uh, JSON objects because this way we can get all the information of the result in a way that we can format to then create the the result for engineering teams. In our

case, we get the information from the JSON file to create a Google Doc with different sections. So, once we have that information, we replace the information in the Google Doc. So, this is a document that engineering teams can can understand. Okay? >> [clears throat] >> And once we have the model once we have the the instructions for the model, how we are going to ask the model to get the threats and security controls of the system? Well, we have we we use a technique that we named divide and conquer. So, instead of directly asking the the model to to identify the threats, we want the model to to analyze the service. So, it is very important and necessary that the

model understand all the elements of the service under review. So, we first ask to identify the threats sorry, the services that needs to be implemented, the existing services of the company that are part of the service under review, third-party services, users and roles, of course, the functionalities, the data flows, and the sensitive information. So, we ask the model to understand everything and put all that information together. Once the model knows the all that information, we can ask the model to identify the threats. Also, we use um a technique to enforce the result. So, once we have a first result, we ask again the model to take a a look at that result, to review that result, and to improve it.

And well, PMA was created to to fix the scalability problem. So, how we are fixing that issue? It's a prioritization. So, we we have created a system to identify services, the services that are under review, that are critical or not critical. So, we ask the engineering team some security questions. For example, if the service processes PII or if it's a publicly exposed or it's a control plane service that is very important for other services of the company, we mark that service as critical. So, engineering teams create the request, the service is marked as critical or not, and if the service is critical, only in those cases the the security teams are involved in the in the review,

okay? So, this this way security teams focus and work only on the most important services of the of the company. It is true that we encourage engineering teams to review the services that are not critical because this way they can be aware of the security of the of the services that they implement. But, we are fixing the scalability issue because security teams are only involved in just the most critical services. Here we have some some numbers, but as you can see, now almost all the the reviews are created by TMA, and in security teams are only involved in the most important ones. >> We wanted to to show you an example, right? We cannot put like a an internal

service of Twilio, so we took the library app from OWASP, that is a college library system architecture. So, you can see in the middle there is a web server, that there is as well a database on the right hand side, that is MySQL. The web service is an Apache web server, and there are three types of users. So, there's the students that can request books, they can look up books, and have like profiles, and they can as well log in, right? Those are the functionalities. The staff can, for example, modify those reservations. They can as well see the profiles of the students, etc. And the librarians are the administrators of the website. So, we gave this architecture diagram

directly to the AI, and we gave as well this explanation that I just gave you, right? So, the type of technology used, the that the students can do this, the staff can do that, and we just let it run, and we just took those results and put them in the presentation. So, there is no modification here. So, we just took three of them. There were more security controls and more threats identified, but these are the ones that were They were more interesting for us. So, the first one is related to information disclosure. It says unauthorized reservation data exposure via IDOR. The reservation endpoint accept non-numeric reservation IDs. So, changing the value returns other users' reservations, right? So,

this This can be a threat that is actually broken access control. It's very important and needs to be fixed. What is the recommendation that the AI gave? >> [clears throat] >> Per per object ownership check. So, verify that the user ID that did the reservation is actually the the logged-in user, right? That's a It's a simple check that maybe a developer forgets. The implement even gave gave an example of implementation of how you can do this, and he gave references as well to where does this attack come from, right? Like, for example, in this case from OWASP. Next security controls related to the denial of service. And it says transaction flooding denial of service. So, an attacker may flood management

system by sending a lot of reservations at the same time, right? The recommended fix is to limit the number of transaction requests per user or per IP, you know, to to prevent the abuse. And to enforce a strict limit. In this case, for example, it says 100 per minute per IP. Maybe it should be less, to be honest. But monitor for traffic patterns and usage to adjust this threshold. So, this is another of the of the examples that you you could think like, "Oh, do I need to implement rate limiting?" Or how do I need to do it? So, do it for example per user or per IP instead of you know, just you know, in a

different way. The last one I wanted to mention is related to elevation of privilege. And this session fixation enables privilege escalation. So, [clears throat] if we don't renew the session IDs when you authenticate, if one of them get leaked and you reuse them all the time for the same user, you will be able to authenticate as as that user. That's the the threat that they identified and this is the explanation that it gave. And the AI suggested fix is to regenerate the session IDs post login. So, always issue new cookies on successful authentication. If it gave as well an example of how to implement this in the code. So, on {slash} login success call res.generate,

send the set cookie with a new SID and invalidate the old. That was the most important important thing to do. So, we just need to just wanted to give you an example of how how this looks and how powerful this is and that this is real. This is available and it's possible to do and to encourage you to you know, to scale up with your your threat model program. Thank you. >> Well, now we want to show you some some metrics. So, nowadays we can say that the project is going very well. So, it's being used by all the engineering teams at Twilio and we have I would say fixed the the scalability issue. And we have some some some numbers for

you. So, well, the first benefit of using TMA for sure is the reduction in time. So, before having TMA we needed weeks to create a a result for engineering teams, but now using TMA they can get the first result, the initial result in just a few minutes. We say here 15 minutes, but it's even even less. And well, that benefit in the that reduction in time means more efficiency. So, it's more efficiency for both engineering teams and security teams. Security teams can focus on the most important parts of the company. So, they are more focused and also of course the engineering teams are not blocked. So, they don't need to wait for weeks for a result from

from the security team and they can just focus on what they do best that is creating do the implementations of the services for for the company. And based on that, if we are saving time, but we are saving for the company's money. So, we did some numbers and the second half of 2025 we have calculated that we have saved 400k for for the company which is a lot and well the automation still is is going on and getting better and being used every time for for more teams. Also, we created a survey for the engineering teams asking them how how satisfied I are they with the with automation. And well, so far they are very satisfied because

before having TMA they needed to create a Jira ticket. That ticket needed to be assigned to a security team. The security team needed to work on the on the review. And now they just need to execute a command and start the the process and they have a a result in just a few minutes. They will have the threat model and also the Jira ticket assigned to the right team. Can be the engineering team in the case of non-critical services or to the security team in the case of critical services. Also, more more benefit more benefits of using TMA. Right now we are working with more teams in the in the implementation and improvements of TMA. We are working with

the security compliance team at Twilio and now we have added information of the compliance standards that we use at Twilio. And if a team that needs a service that need to be needs to be HIPAA compliant, for example, we can add that information to the request and all the security controls needed for that service to be HIPAA compliant will be added to the to the threat model. What does mean? What does it mean? It means more clients an easy way to to verify if that if a service is compliant with a standard or not and more business opportunities. Well, we continue working on TMA. Right now it's one of the most important projects of the company

for security. We want to continue improving the all the capabilities and functionalities. We want to allow to use all the LLM providers instead of only using OpenAI. We want to create more integrations. So, one of the ideas is to integrate the the main tools and platforms of the company with TMA. So, we continue adding even more. Also, we are working with other security teams because right now we have we are orchestrating the standard security review that is the security review to create a threat model, but there are other types of security reviews in our company, for example, for the third party security team, for the privacy team. So, they are working with us to integrate all those

processes into TMA, too. So, they can automate their own security reviews. And also this year we are working on the project to make TMA open source because we want to share this this project with the community, with other with other [clears throat] companies and see well, how we can continue with all these new things using using AI and LLM and well, with more people working on the project, we are sure that we will improve it a lot more. And well, thank you. Thank you everyone. Thank you the sponsors. Thank you the organizers. We have some some time for some questions. >> [applause]

>> So, the question is if the if the person that is going to review if we have done a survey as well of the type of person that is going to review and

Yeah. Yeah. Yeah. So, yeah, that's a good that's a good question and we always let as well the you know the the engineering teams check what they have in their backlog and what they have already implemented in the services. So, when they receive the first threat model, they can say like, "Hey, I have implemented this. I have not implemented this. I didn't even think of this." Sometimes as well, they they are not that many hallucinations. We have seen that the reinforcement is quite quite good. It's a good technique to to avoid the looking the hallucinations, but it can happen. There are sometimes that it happens and if it says something that is complete complete BS, they we say like,

"Just feel free to remove security controls or or let let us know if we can remove a security control, right?" That's the the most important thing. We want to encourage the teams to to understand security and to do security by themselves and this is a very good tool to you know, to support them to start doing this.

Yeah. Yeah. As as the as the model improves as well before we didn't have that that many recent models are they were not as accurate as the model improves the threat model improves in general. So that's one thing that we have seen as well like improving from when we started in 2024 in October there were more errors. Right now is very accurate I would say. So so it has improved a lot and it just it's just getting better to be honest.

Well, I would say that well it depends on the model and also on the context that we add to the model. So as Javier said before so in the first versions of TMA we saw more hallucinations but it is true that right now we are not seeing any of them. Maybe just some confusion with platforms and things like that but using the technical of reinforcement and always trying to add more context to the to the model we are getting every time better results related to that. Absolutely. So I would say the most important thing is the context because if not the the the model can recommend things related to platforms that you don't use in your company. So the more

context the better results for sure.

>> Okay, thank you. >> [applause]