PG - Threat Modeling at Scale: More than shifting left

how's everyone doing today good yeah has bides been uh been fun for everyone so far yeah it's been I've been having a good time it's been enjoyable everybody here's awesome you guys are everyone's so nice it's crazy so thank you all for allowing me to speak in front of you and everything so should be good give it give it like one more minute get started all right okay are we good y good all right uh so my name is Troy um I'm going to be talking about threat modeling at scale and uh without further Ado let's go ahead and get started so uh a little bit about myself uh I love dogs beer science not necessarily in that order um started

my It Journey in the Navy uh I was in the Navy for about six years I was a fire controlman uh Tomahawk Tech uh from there I got out and I went to work for Costco uh I've been on the thread analysis team now for going on 5 years so it's been it's been a lot of fun uh my super supportive team is here as well so thank you guys um but yeah and then uh from there it's just been it's been a journey it's been a lot of fun uh real quick question for everybody who hear threat models has threat modeled does it on a regular basis oh wow awesome so please feel free to ask questions if we

have time at the end i' it's been going right about the 25 minute Mark if we can't get questions in feel free to find me afterwards and we can chat totally totally fine with that so just let me know all right so uh real quick what is threat modeling I'm not going into how to threat model or or anything like that but a quick quick overview um generally you want to answer four questions right what is being worked on so Define your scope of the of the engagement of the threat model uh gather information so this could be documents diagrams whatever uh and then usually you're going to build some kind of visual model of the application or system or whatever

it is your threat modeling uh you also want to answer the question of what could go wrong uh so identify threats vulnerabilities um sometimes you'll pair these things for risk within the threat model sometimes you don't uh it kind of depends on how you do it at your organization um and then uh what should we do about it so are mitigating controls your recommendations uh things of that nature and then uh for uh remediations you definitely want to think about you know what's going to work best for the business and things of that nature and then uh did we do a good job so uh validation of of the threat models through reviews with the project

teams or you know retrospectives or what have you all right so uh challenges of threat modeling uh really comes down to a few things here so uh lack of expertise so uh a lot of times devs are focused on you know Pro providing a solution that's going to work best for their customers and provide the most value right and that's what they're supposed to be focused on that's that's why they're there and uh so a lot of times security kind of gets put in the back burner or it's you know not at the Forefront of their of their building and stuff like that so um definitely you tend to see a lack of expertise with the

uh the developers that you're working with another is a lack of time uh a lot of times there's um tight deadlines you know uh you got to get things to Market and that can that time crunch can make it so that security becomes more difficult to implement uh and a lot of times everything else will get prioritized over over the threat modeling process get prioritized over Security in general uh but yeah uh and then lastly lack of resour ources uh a lot of times the uh tools that you'll be using are much more focused on making the build process easier helping the developers find you know maybe certain kinds of bugs that may not be security

vulnerabilities um tools sometimes don't support the active threat modeling uh and so having the right resources as well can be kind of challenging uh and then what does what does all this mean it means a lack of uh value when you're doing your threat modeling when when you're going through your threat modeling process uh when that happens a lot of times resentment can build between your project teams and the security teams and it just makes the whole process a lot more difficult all right so and then the big question that we're going to answer today how can you threat model and do it at scale right how can you perform thr modeling at scale so let's jump into it

so firstly is uh collaboration right um it's a very collaborative effort between your Security Professionals and your project teams so what I like to say is as a security professional you want to make sure that you're able to train assist and Empower your project teams to make the best decisions and build the best products that are the most secure right and as a security professional you can bring that knowledge to the table and collaborate with them throughout the the design process or the or even the early building processes right uh the while you're collaborating while you get this collaboration between the project teams and the and the security teams you're inherently going to have the

project teams learning about security so as the project teams learn about security they're going to start inadvertently or purposefully implementing those security those security controls that you've been talking to them about over and over we hope and start integrating those into the solutions that they're building and vice versa the security teams are going to start to learn what can hinder the business the most right so maybe there's certain mitigations that they learn you know makes the product work in a way that's counterintuitive right and maybe there's a different mitigation that they can use instead uh and then as they work with those business teams in the future they can put push those rather than trying to push something that's going to

cause issues and then uh overall with that efficiency is going to improve as you become more collaborative uh another so tools um like I mentioned before tools a lot of times uh it's it's hard to find some that are specific for threat modeling they are out there uh there's paid there's uh paid versions there's free versions but uh one of the most important uh aspects when it comes to the tools that you're going to choose to use in your threat modeling process at an organization is going to be asymmetric collaboration uh and what I mean by that is the tools need to allow you to perform duties or work or whatever it is that you're doing outside

of set time windows with the project teams or vice versa for the project teams to do things outside of a meeting with you and then allow you to go in and see what it is that was done what changes have been made what progress has been made things like that so that asymmetric collaboration in the tools is extremely important uh so you also have to take into account tools for uh diagrams right um with threat modeling you're making visual representations of your of your Solutions and being able to accurately depict those is extremely important because a picture is worth a thousand words and it's a lot easier to understand what's going on in a solution

when you're able to visualize it also uh supporting the collaboration right that I just talked about you got to have tools that can support that SO meetings recordings um for reference later uh whether those meetings are in person or virtual or hybrid um just being able to support supports collaboration uh another is building reports uh so having a tool that is capable of producing reports that are tailored for the audience that you're giving that report to so it could be a report that's more tailored for compliance it could be a report that's more tailored for maybe high level Executives um or maybe it's a more detailed report that's going to the project team and maybe other security

teams that are going to be uh helping in the process of remediation and things of that nature uh and then validation tools so with validation tools um you can use you know code scanning vul scans you can do pent tests and the idea there is that once those things are done you can look at the findings from those and see if they differ from from the threats and vulnerabilities that you sought when you were doing your threat modeling process right and if there's vast differences then you know that maybe there's things you guys are missing or not touching on in in while you're doing the threat models so it's it's a very retrospective kind of tool for

validation but definitely extremely helpful all right uh so procedures and practices so with this one there are two main questions that you have to answer when you're building up a threat modeling process I don't organization that being security team what should the security teams bring to the table is the first one uh expertise so their knowledge of security right uh they need to be able to give guidance so knowing what is secure is one thing but being able to guide the project teams to what the best option is that for both the security and the business to you know keep moving forward that's where that guidance becomes extremely valuable uh and then templates as well

at the end of the day the threat model is a product of the security teams right uh you want to own the template you want to own the document and you want to make sure that you're able to uh keep account of what is delivered to the project team and how it's delivered and how it looks what should the project teams bring to the table that's the second question time for one um threat modeling can be a very time time consuming process it's not always but it can be and the project teams and the security teams need to be prepared for that right when you get into a meeting and you start walking through the solution there could be a

lot of different connections different processes or functions of certain applications that you have to talk through and finding those those threats and vulnerabilities that could be posed on that solution takes time and so patience is is definitely a virtue when it comes to threat modeling uh AAS is also extremely important knowing who's responsible accountable uh consulted and informed and this is from the project team right so that when the security team has an issue they know who they can quickly go to uh to get it resolved and keep the flow of information open so that there's transparency uh and then lastly documentation and diagrams so to speed up that lengthy process that I talked about front loading can be extremely

helpful there so if you're able to get in-depth documentation and diagrams from your project team you can frontload a lot of the work so that when you get into these meetings that require a lot of time you've already done the heavy lifting and then at that point it becomes a conversation all right so I say I say here Define everything right and the reason I do this is is because the more speed bumps that you can remove in your process of doing the threat models the better the the idea here is is that you want to streamline it as much as possible right because when you're doing it in a large organization um at scale it becomes

timec consuming it becomes difficult and so either you hire more people or you make the process more efficient so what I mean by this is do you do the threat model make sure you have it defined as to what justifies a threat model what doesn't uh your intake method when and how are people coming to your team to request the threat model to be done uh checkpoints how often are you meeting with the project team when are you meeting with the project team what triggers it right uh and then what is considered an escalation so if there's a big disagreement between the security professional and the business on something right how do you how do you

resolve that quick being able to identify and have that defined how escalations happen can resolve those quickly extremely uh and then delivery and Post delivery so when you're you finished you need to know who to deliver it to and this is where those different versions of the document can come become very helpful because you can deliver the high level overview to Executives you can deliver the super technical one to the uh project team and other security teams right uh and then Post delivery so you've identified threats you've identified vulnerabilities and you know what you need to do to mitigate those things you're not as the person doing the threat model and oftentimes in large organizations you're probably not going

to be the one that is in charge of following those to remediation and so understanding how to hand that off to the next team that is going to track those and make sure that they come to fruition is extremely important to document and Define in a proc as a process so that the remediations don't get just found and then it's just white noise from there you you want them to get fixed that's the whole purpose of this as and you want it to be fixed before it goes to production uh and then what to do when the design changes right this is a huge this can become a huge speed bump a huge blocker is you'll be halfway done with your

threat model and all of a sudden they're like you know that one feature that we spent a week on yeah we don't want that anymore or we're changing it to this other thing so knowing what to do in that moment is extremely important and it requires a lot of communication um versioning of your documents is extremely important as well so that you can maybe go back to a different version that works better to then work back up from there to try and save you time um but at the end of the day remember that threat modeling is a collaborative process and so information that is being shared to you by the project teams is going to have fallacies

it's going to have omissions it's going to have um Fabrications we're all human we make mistakes right but identifying those things as fast as possible will help you to get past those so that you can finish your threat model and have it be accurate and not to mention when there is stuff like that you start to notice it really fast like as you're threat modeling things just will start to not line up it just won't make sense and you'll you'll do it on your own kind of you you'll see that something's missing so all right continuous Improvement right so as you're building up this threat modeling process you want to try to continuously improve it so what is

being done right uh reward that right so if the project team or the security team is doing things right make sure you keep doing those identify them and and point it out right and then what could be done better so maybe there's areas where there could be minor improvements made uh identifying those and taking action to streamline the process and make it better is important and then where are the repeating pain points so these are things where maybe you're running into blockers during the process at the same point every time right maybe there's something in the intake method that could resolve that or maybe you're finding the same vulnerability and it's going unfixed over and over again right

it's it's identifying those and getting it done as fast as possible uh and then retrospectives right this is where a lot of that work will happen is in your retrospectives getting the security team together talking through these things and finding Solutions that'll streamline the process all right benefits of threat modeling so so one one of many is saving time and money it is much cheaper to make a fix to a bug or or some vulnerability before it goes to production uh and it's it's a lot quicker too so using threat modeling to identify these things early will save you time will save you money it's it's extremely helpful uh and then cooperation and understanding so when you have the

security teams and the business teams coming together and solving problems together they're going to be more inclined to work with each other and be more open and forthcoming with information and then you're going to see improvements in security when that happens right because the business isn't going to be scared that they're going to get you know told no or become or get blocked on something they're going to be more inclined to come to you as for advice or just hey how do I do this better uh and then as as the threat modeling process becomes more prevalent in your organization older Solutions will start to fall off or if they have major architectural changes they get reviewed through the

threat modeling process and everything starts to become I don't want to say more secure but you'll have you'll have more confidence in your solutions that you're deploying that they are secure that they have less threats less vulnerabilities uh it'll also give you a much better kind of picture of your organization security posture and then uh I have up here improv pent tests and what what it what I mean by that is when you when you have the threat models and you and you've identified these vulnerabilities you've identified these these potential threats you're able to then feed that to the team that's going to be performing the threat the I'm sorry the pen testing what you're doing there is you

are focusing their their efforts right so pendis are expensive I think we all know that they're they're an extremely expensive Endeavor and so if you can focus the pent test on things that may be an issue instead of this broad just test everything mindset you can probably drive cost down and the quality of your pen test will probably go up you want to be careful not to get too specific because then they might miss things that they should have caught so it's a definitely a fine line you'd walk but it it can help in the long run all right so I have some uh some AI memes up here um what is the future of

of threat modeling well yes I think AI will have a play in it I don't think they're coming for your job though um what's probably going to happen is AI is going to be used to augment teams that are doing this threat modeling right they're going to be the AI is going to be able to see things that the human the human process can't see different maybe there's threat threats in one solution that ties into another that ties into another and that threat can manifest itself and an application that's maybe you know to to down the chain or whatever it's it's definitely going to improve the overall quality of the thread analysis but it's going to be

there primarily like I said to augment the teams and give them a better understanding of what's going on and just make their job quicker and more efficient I hope you like the uh the memes too they made me laugh and then this here this is a quote I'll let you I'm not gonna read it but you guys can read it um I love this quote it keeps me going keeps uh keeps me motivated and everything so I thought I'd put it up here but I just want to say thank you everybody for having me and and thank you to my wife and kids for being at home my wife's holding the fort down right now so so I could be

here so that's awesome and thank you to my team for coming in supporting me so thank you all [Applause] I think I do we have I think we have a few minutes if anybody has questions so how are you getting bu

developers and it's our developers

where have resources never

yes yes so is we what we shoot for is in the design process when there's a somewhat solidified design that's when we like to get involved without a solidified design you kind of spin your wheels a little and you'll do a lot of work that may be for nothing because the design will change so you want it to be close to to being done um but not into the build phase yet when you've got developers that are pushing back that that heavily though it's it's going to take a lot of of effort on everybody's part you're going to have to build out the process and show that it can be streamlined and then you're going to have to get

executive leadership to step in and be like this has to happen it's it's yes but you need to be able to show that hey we have a process in place that will make this quick like it's not we're not we're not going to take up more time than we need what you can frontload us with a lot of stuff that'll help and you know help shorten it

down like well know just back it's just like we we you we like to use the term funnel rather than blocker or gate and the idea behind that is when the they're building something they're eventually going to funnel down to a point that has to be they have to go through it right and one of those points is the threat model process they can't they can't they can't move forward until that's done and there's a lot of and rather than making it like a a a checkpoint in some list what has happened is a lot of other teams won't give them what they need to until they see a threat model from my team and so they'll they'll be like okay

we'll just go past you that's fine and then you get to the next team that maybe is setting up the network right they're like oh well we need to see the threat model and then they're like oh well maybe I should have listened and they'll go back so yep go ahead can you speak a little bit more about uh asymmetric collaboration yeah absolutely uh so the the like I've said like I said 100 times is and I I'm going to keep saying it is it's really really really really hard to get a bunch of Architects Engineers devs into a room with you at the same time like it's it's impossible and then you if you try to take into account like

Integrations of other tools that are being built other applications and now you've got Architects from another team and that team and maybe another team and getting them all into a room it's not going to happen and so if you have these tools that allow for asymmetric collaboration that allow an architect to go into a document make some adjustments leave notes and then move on with their day maybe they had five minutes and then you see you get maybe a notification hey somebody made an edit and then you go in and you see the edits you see the notes and then you can add additional questions to it you can make adjustments to it and this goes for everything from

reports to the diagrams to question docs you name it

yeah yes so we it's becoming very cumbersome um and we are in the even even now we are still trying to refine our process as best we can and so we have what's called a consultation summary and the consultation summary is designed to answer those one-off questions like let's say they're like oh we just want to integrate with that API I'm not going to threat model a single connection to the API right like that's that's a lot of effort for one simple function so a lot of times what we'll do is we'll shrink that down into a much smaller deliverable that just answers the question with basic security observations security concerns security requirements based on industry standards

things of that nature but yes that that's a good question yeah teams cannot

sayas what happen they they will be kept from going to production so they can move forward with testing they can move forward with qat whatever it is right but going to production if they there has to be a very strong reason that they would go to production with without having a threat model done so your management It's like got to be done model got to be done what's yeah what's crazy is I don't think I don't think threat model the threat model is even on the release Michaela keep me honest here I don't even think it is like I said I think other teams that are on the release to be able for it to go live won't do their

part until they see our threat model and so yeah can it can do we allow an app to go to production without a threat model we do but they have the back TR if they have to meet any of their requirements ands yeah so away yeah it's it it's definitely a very interesting Dynamic and I'm not saying it's the best but it's been working for us so far oh I'm sorry no no you're good yeah yeah it's it's they're roughly defined right now um but they're in the process of being much much harder much hard much harder stops like you you cannot proceed without having it but yeah go

ahead yeah yeah so how we Define it is are you seeing major architectural changes when you have those releases right are you introducing some new function that is going to create you know all these new connections and open up all these different Avenues of of attack vectors and all that if you're not like let's say your your release is just adding a data point to a connection that already exists you probably don't need to threat model just that that's that's probably quite a bit but if you are noticing that when you proceed it's going to create a lot of new connections maybe you're adding a bunch of apis that are all hitting these different points

and calling all these bases that's when absolutely you would want to proceed with a threat model so it's it's it's really it's kind of using your best

judgment yeah I see what you're saying I would say that if you can wait until a lot of it is done and in place and then maybe do a back fill on it because if you try to do it every single one and then like piece them together that would it that'd be very cumbersome and it you would start getting some resentment between the project teams and the security teams yeah all right that is all the time we have feel free to find me afterwards i' answer any questions you guys have thank you thank you thank you