← All talks

AppSec From the Ground Up

BSides Edmonton · 202441:2732 viewsPublished 2025-05Watch on YouTube ↗
Speakers
Tags
CategoryCareer
StyleTalk
About this talk
BSides Edmonton September 23-24, 2024 Talk: AppSec From the Ground Up Abstract: What do Starcraft, Red Queens, and Gandalf have to do with Security? Walk through the history of starting an AppSec program from the ground up, and hear how Clio went from 2 AppSec devs to a fully staffed Red & Blue team, the obstacles overcome during that time, and the systems built to support a 250+ dev org. Speakers: Gavin Miller 2024 Slides: https://drive.google.com/drive/u/0/folders/1ess6fUZNd9BbWK7pPBrh8UVE-7GXtMyG
Show transcript [en]

hi everyone morning welcome to the round of app my name is and I'm you know just here to basically introduce Our Guest um our speaker here who I mean I see I see has already familiarized himself so just a bit of introduction um we have Gavin Miller here Gaba is a senior manager of application security at PE he started from a pure developer but ground and he has been at the helm of implementing Cleo's application security program for the Last 5 Years you're welcome awesome thank you um yeah I thanks for coming to my talk and uh giving you giving giving me your time uh I greatly appreciate that and um since it's a smaller room like I

let's let's ask questions if if something's unclear if you want some clarity on something thing I as attorney not be familiar with I think these these small groups are yeah yeah move in even come move up um these small groups are some of the the best way to do these presentations because you can kind of guide it to where you want to go so um this is one of the few serious slides um coming up here to give you a little bit of context um Cleo is the SAS market leader in the legal vertical so we um we make legal practice management for lawyers uh and along the way we do things to help make them more efficient

to help give them the tools they need to practice law the company was founded in 2008 and so we've been around for 16 years that would make us a teenager uh some days our codebase looks like and behaves like a teenager uh and doesn't want to work or doesn't want to you know go to school um but bad joke um get ready um but we're we're building a um the we're building with the with the mission to transform the legal experience for all and I I'll get into a little bit of like that's why I'm there there's some really cool things that we're able to do and I think um we talk about government and it's in legal and

all that type of stuff as part of you know Security in this conference so give you a little bit of background I joined Cleo in 2019 um at that time we were an undisclosed valuation there were about 500 uh employees 100 to 125 of those were deaths and there were two members of the security team at the time with two products so uh what's that's about a 1 to 50 ratio which was is not bad um but that's you know I'm sure we're all familiar with that who who work in the industry it's usually security folks who are at the minimum um fast forward to today um we are a $3 billion valuation uh just

happened at a Series F round couple months ago um one of the largest in in Canadian history and especially in the legal uh Market we are 1,00 employees about 250 devs aim to be 300 uh at the end of the year there are now 19 members of the security team um and we have five products and unine services so we've grown I'm I'm strapped to a rocket ship and it some days it scares me of like how do we keep up we have um some of the most valuable documents in the world somewhat hard to monetize which I think is is nice for us the you know the people that are attacking us don't necessarily know what they're getting

access to don't tell them please um but uh this particular picture was um this was this is my driver's license photo that got uh memed uh Meed whatever it is uh that was after two days of very intense incidents uh and was the retake that was the best photo that we took at the reg uh I was just going to lean into that one um I'm going to come clean uh with you so technically this is my first apect job uh I was at besides probably 2018 2019 and I was I was listening to some of the speakers and and this is you know my invitation of like don't be afraid of getting up and presenting I was

listening to some of the presenters I was like I can do that I could do that and 5 years later Look at me now they asked me to say things like I'm an expert um but what it was is I came from that software developer background I did some did some blogging on security did a lot of research on on you know my own time getting into it I found it very interesting and ended up talking with uh the now CTO at Cleo and we aligned we we agreed on the principles that should exist within an absec program he said why don't you come join me and let's let's let's do this let's go on this

journey together and so that's um that's that's where we went or rather that's what this talking about um so kind of a historical looker um I want to I want to present it against a framework and this is one that we use for onboarding uh all of the members of Cleo but I think it works in a lot of situations and so I'm going to frame the presentation off of that of of context credibility and capabilities and we kind of look at that as a 30 6090 when you're on boarding into a new organization and and I think this applies in a lot of cases like security our job is not to go in if if you're new

to a role and just bll through and be like you're doing this wrong this wrong this wrong this wrong this wrong that's not that's not what somebody wants to hear unless I'm hireing you could do that as a pentester but uh even then you have to you have to understand the context of the business you have to know what is going on and and all those pieces so um going to start with context here by by talking about how when I joined Cleo what I did to build context into the organization in those first about year um this was the the developer that I joined with this was uh Chris Thompson who's here in Edmonton and P retired

whether that's my excellent management uh helping him out there or he was just done with me um but uh we decided to run the absec program on our own and we started with the uh audacious task of figuring out what do we do what is our purpose as an ABC team um Chris had started out as a developer that moved into security and he started to basically put in little bits of controls here and there but controls without a strategy uh and controls without a purpose you you kind of just end up playing whacka so when I got there um we one of the first things that I had to look at is what is our strategy

what the thing that we're going to look at that we're going to aim for um in order to run an effective program and what are the problems that we're facing and and the big problem that we that we had when when I first joined was we just didn't really know what was running in our ecosystem we didn't know you know all about the applications we didn't necessarily have the observability the visibility we didn't have a scene at that point so like all of these things were we we are want and if you're wanting to what's happening within uh an ABS program you're you're you're playing you're playing the Rumsfeld here's Rumsfeld slide uh yeah you you're trying

to eliminate the unknown I love the headshake he was you got one thing right the the unknown unknowns um the job is okay we've got a strategy you've got a build Contex what are the unknown on nose and how do we start surfacing nose and if if anybody is I know person in management the thing you don't want in a in a security group the thing you don't want in management is surprises and that's that's my goal my goal is to shed light and go find all of the surprises that are going to make my boss go paron swearing I'm G I'm G to swear if anybody's Le now it's like what the [ __ ] Gavin why didn't we know about

B um so that's that's my rule as I as I join I need to get an inventory I need to get visibility and I need to know everything that is happening within that code base and also know where all the hotspots are so that is what we decided to do and we decided to start with an application that um was a Starcraft reference uh so there was a tool that existed called foldl and as the name applies it Overlord all it lorded over all of our uh code bases it looked at from a GitHub perspective took inventory against all of the repositories we had and we we treated that as our core domain object and remembering I came

from an app or pardon me a software development background so I didn't look at any tools to do this like I'm sure some of you are in the audience being like yeah there's a tool for that and there's a tool for that I was like I don't know it's it's my first date um so we we've had this inventory project and I I co-opted it it was supposed to be for everybody and I said now it's mine um and what we did is we looked at all of our all of our applications from a repository standpoint it was there as a health check mechanism and we'll get into some of the details there um it was

there to look at you know what is going on in the code and we're absc we need to know what's going on in the code our first challenge though was you can't rip off blizzard you don't want to get sued so I made this my biggest contribution to Cleo uh this is the overlord Mass shot um and and it's actually proved to be quite valuable and uh for for our av in the background um we've turned him into knitted mascots where we actually so this is part of our security Champions program that we inadvertently build we made stickers we made these little uh knitted things and we submit you know if somebody submits a really good uh issue

to our bug Bounty program uh plug we pay up very generously um you know if it's a really good vulnerability we'll send them a swipe bag with one of these a couple of stickers a whole bunch of Cleo branded merch that type of thing we're trying to raise engagement we also do that internally with folks who have been major contributors they've found issues and it's that positive reward cycle um Tanya uh from she hugs purple she talks about that as well you have to reward the good behavior instead of running around and chasing people with a STI so we we tried to make a very um positive uh positive program so that we could inflect people people's behavior to do

the right thing um anyway coming back to we we co-opted everything here and this what you're seeing here is a a more modern day version of what we're Lord um and it's looking at a Noto yri vulnerability now the problem that we had brought this in to solve was we had it would take us about two days to go find an inventory and fix a single cve that popped up and between two of us two days that's a that's a glorious waste of time and this was before dependabot was around and um made some of these things easier GitHub didn't have a good security program at a time we've now integrated those things in here um but now we've we've got I I

can tell you immediately if we're running X xzs if we're vulnerable to you know the latest cve that's made the front page of hackers news and it's allowed us to have that full purview and context and it's solved that unknown unknown and the nice thing is is anytime a Vio comes online now it's automatically inventory so I don't have to go do anything um when when somebody starts up a new project and brings in you know all of the terrible code that we don't want to see we'll talk about more more checks that built on Overlord I I realized as I'm kind of talking through this this is basically a plug for Overlord which is close Source tool

but um here we are you can't have it um so that's what we started with was the context my my main goal is get context get visibility once you've done that as an absc you have to move on to the next stage of building credibility within the organization so you take that context and you have to apply it judiciously you can't again just go here are all the things that are broken go fix it that nobody likes that you need to be effective you need to be um you need to be tactful maybe political political suround word but you get my draft um so this is a slide I I stole from uh uh is it Margaret Sims uh at Shopify and

it's it's a great presentation um email me if you want the link to it or just look uh for for Margaret Sims Shopify um she was talking about compliance she's on the compliance side I stole this slide uh so all credit to her but when when we think about a security program a lot of a lot of classic security and this is what I talked to the CTO about when when he hired me um was we we want to build something that doesn't impede developer philosophy and a lot of security gets looked at as Gatekeepers and when you're impeding the velocity of the developers when you're keeping them from getting things done they don't like

you and so your job is to a build tools that can increase the trust in your program reduce the cost and increase the speed for the developers as opposed to the bottom one there if you're if you're getting in the way you're going to have a bad time and you're not going to be effective um an early version of Overlord uh was was blocking in the CI pipeline so if you had a if we had a cve that we discovered um that came in through our through our intake process and you were working on a PR to get out a feature that was completely unrelated to say you know the Noco giri upgrade we' block the pipeline said you can't

ship your change that's completely unrel reled to this until you go fix this and how do you think 125 developers who can't work on a Monday morning feel about that CTO yelled at me a couple of times it's okay I went to counseling I'm fine um so what we started to do is we started to move it into uh into the poll request itself and this is an early version again um we wanted to make the right things easy to do and the wrong things hard so um come over to this side side do as well um this is this is an early version of the Jack it's all on me it it just looks

looks up a web looks a PR and makes it moment B um you can see it's St look have license for any package that be break in it's looking at open issues he you you got go for do 600 issues lucky you uh the um you know the P sheets you widely use and then uh this one hope re mity and like asking a doubt to go value re mity like what what did that mean um so that one was no good either but you have to start somewhere and you have to evolve and um continue to practice this and I love I'm going to reference again Martin Fowler from U my days as a

sovereign year if something is hard you need to continue to do it because that friction you'll be like this is terrible I need filming I like better um remove the toil and and I'm working with uh a new security engineering team here that's just come on with VI the last few weeks and they have so much toil they spent all of their time doing toil work of like looking at blogs looking at automated alerts and like by let me turned on PO scaling firm just say that's failing and they're like well that's a really good idea that would have say this all just looking at stuff so figuring out ways you could do that I

inserted myself into the process I had an early version of this tagging me every time this went and then I will do the code review go okay we need to automute that we need to change that this is terrible this is expensive what we ended up with now so this is the modern day version of it um this like bringing in them through all of them the developer doesn't page you anything they nice uh you know they nice Emoji with with sunglasses and you can see line them it says pass automated review so again all of those checks that we did reasonable Cod quality is it the right license all the things can C them so that the El

doesn't have to think about the check so we just go here are the two or three things you might have to look at come talk to us if you don't know what to do but we've done it all for you this package is safe to bring in I had a developer come and and ask me and and complain about this like why do we have to do this why is this a thing we have to do it's open source can't we trust open source and this room laughs but the software developer laugh that's terrible so awkward um the software developers like why do we have to do this isn't the community would catch us and I like Po

me wor moving a Cleo is a part of the community that takes advantage of Open Source and makes you know and so it's our job to go look at and if that um that SSH you know OCD developer who was like oh that 0.5 seconds that I have to deal with getting onto my server they wouldn't have investigated and wouldn't have found that X exing tills um you know two-year hack thing so it's a part of all of us to to do this and if you can get developers doing it whether they know it or not um it becomes really valuable and so one of the things we did to make that easy you can see this

little uh fum over water pie so I don't I don't want a developer to go understand what reasonable quality is I want to point out here are the things that are weird and here are the things that are odd so we built out a custom C tool I'm not going to tell you everything p in there um but we find St like like this G bringing in uses it Val is there a base system for Sero into it that shouldn't be going e way um this has allowed us to to again catch things um not not major malware or anything like that but we've caught things that are not the way they should be so one

example uh a developer came me and like hey what are root certificates and why is this gem setting root certificates and it was not it was not delicious I don't think maybe it was but um we looked at it it was a dependency of it that it SE and said okay let's go talk to the original open source maintainer like do you know you're replacing root shs from 2018 then you was some sort of thing where one got roked and they wanted to bring in the latest and greatest so like it was best intentions they told us um but it was one of those faces where we could go fix the root issue and now nobody has to

deal with it and it's you know yay we contributed to the community but it's you know those types of efforts that again build those capabilities because we now have the context to operate on so it's you know it's a layer take approach you're you're adding more and more as you go and it started to get developers liking our tooling like this is this was feedback I got of like this tooling is great I don't have to review a gem this thing I used to have to do manually it just says I'm F to go and so we increased the velocity we increased the trust within our organization and within um within our program and everything like that and now

I can let developers run fast they just I Don't Care What They're bringing in I kind of do but um I don't have to think about it and again it's eliminated that toil within the program and it's something we just don't care about and so now we we've got you know we roll this all all of these things up and we've got this amazing system here that now does you see on the right hand side it does it does requirements check so we do we look at Best Practices we look at your dependencies we look at um your exceptions are they trending in the right direction it's turned into overlords become this like sock for us

and I'll get into a little bit more details there but it's also an inventory system we've got you know 100 repos who do I need to talk to if I need to get this fix into this service over here who owns that who's taking care of it his it good State and again this is this is something that we relatively set and forget I have to harass people every now and then harassment does you know still work every every here and there um the other thing too and uh that's my next slide um the other thing too is it allows us to so we raised a series a we raised $900 million earlier in the year

um it's a little bit of money and so people want to know what is our exposure if we're going to give you money um and so they want to know what is our what is our exposure from licensing who are we going to get shoes bu if we now put this in dust to your company um it took it took me a long time but you know because I'm not in the code as much but a a competent software developer um could have written this in in you know a couple of days and you can see we've got an inventory now of all of the approved licenses that we've said yeah MIT is good um I can't think the other one's

off the top of my head but um we we've got a list of all of the software licenses we're okay with and we thought some that we need to you know not know and okay our automatic inference algorithm doesn't know how to pick that up we'll go tweak that and and iterate it on it over time but this page didn't exist at the beginning of the year and we were able to build it in under two weeks and it allowed us to take a conversation that would have been a long time with people doing due diligence you never had to get invols wiring froming or funds they you you show you show them everything you become BSF um and I was

not BFF with them because I almost had 45 minutes done R and like hey that that was it that was you know explain these things what are they doing but security is now an enabler for the business because we've got capabilities that the business didn't have previously and again it allows us to be you know relatively speaking like we're generating revenue for the company and which company doesn't like a security team that isn't a cost center you just sorry again you able to interact that your other corporate systems that are use developers or your age professional I wish just in back and forth this simply the application and code stor or are you able to yeah we we

definitely could extend it I'm sure U it's definitely it it's very much a like it it's an Overlord it wants to take everything in and give nothing back um no we we could um the next stage here well so we're we're integrated with our OCTA system so the next stage here and and we'll get into it a little with the sock and this is our our pulse view we can start assigning things to people offboarding them all of that type of stuff but we could definitely start looking at other avenues of compliance um it's definitely been a very aback heavy tool right now um but we also we also integrate with our security engineering function ions and compliance

so like you're you're I'm giving you my road map here um we can take all of this stuff and push it into our compliance software and then we can always have audit auditable audit ready is is what we're aimed at where like okay the sock 2 compliance Auditors come and we go y there's the control and here's the data that got fed in this month um that's that's the goal uh and we have some of that built in so um yeah it's definitely like this can be you know kind of a a security Operation Center um pulse has something that we added in last year it's sound like an inbox you uh it's kind of like a stop um we were shuffling

all of these things into slap which is not great um it it work but you you're sing We need to know there we need to keep track of these we need to keep historical recurs for moding clients and so we shoved it all in here and then we started enriching the data uh and that's a different slide I don't know where it is it's oh it's further on in the presentation um we started enriching the data and so we can do c management I can drop in here and go okay what are the CDs I need to go look at today triage them go distribute them to teams give them a a head up on what they need to

fix and that's it it's it's made in very easy so that we can bring an an analyst in and now they're responsible for this side of things um I'm going to ask for a time check I don't know where I am how many how many minutes left 20 20 okay perfect we're right on time um awesome so and again um repository owners here pancakes being one of our uh contributors he can now go in me he knows I I know what I need to do I know what I need to uh look at for my repository so again we're handing out those capabilities we have the capabilities but we're handing out those capabilities as well to the developers

so that they can selfs serve uh credibility that side of things um yeah capabilities so now that we've now that we've you know hav done these types of things um and we've done we've gotten a really good foundation the programs working really well you're going to continue layering on it it's what we've done in the last little bit um but once you've got these in place you now have you you have credibility and capability that you can go in and you can do something like this um this was a comment I left on a PR last week where somebody was adding in SQL injection as an admin task and one of our developers flagged it and I'm able to go in and have that

um credible conversation I'm like hey I don't like saying stop to you but this is the thing you need to stop and what it does is it triggers a conversation now that in pattern of like oh okay why why is this not good what pattern did I miss what are the things that um that we're not doing properly in this code um and again you can you can be the security [ __ ] when you meet to be but you don't have to do most of the time but you've got the credibility uh and capability to do that it also gets you pulled into other things that you might not necessarily have perview in within

the appside goal um we had a appearance platform that's launched in um basically fun 2 and uh for those of you who who have you know done any threat modeling or anything like that we got engaged with this very early then we said okay here's what we're going to see we're going to see account table with so then people can J themselves in and try and site know that's that's will be expected in this in this new product that PO was launching um and the game of the project was as all exist now but um but also version all but make money mostly uh if anybody seen office face hings is a very lucrative business you take off one cent

and you make a lot of money from a transaction uh you try and get it right too so we saw this coming in and absc as as a group we like oh [ __ ] like we like prior to that you know legal documents are are not easy to monetize you can't monetize them rapidly from an attacker perspective but money coming into the system while hello uh and one of the key requirements of this product was you sign up needs to be fast this needs to happen in 15 minutes uh to go from not being able to collect payments to collecting payments uh it was our nightmare as a as a security group what do we do um and you know we kind of we

talked about okay what's what's going to happen account takeovers is the thing that's going to happen we brought this to the team and and I'm giving them all due Credit in this we're like we think there's going to be an account takeover poooa that's going to come and I don't think they're wrong when they said okay let's deal with that after we're done fixing this like you you guys you've got you've got the ability you go do it and I don't think they're wrong because we now have a very lucrative payments business and they ship software in six months and and this is that balance that you're always going to see as a security team of enabling the

business while protecting the business and we had a budget we said okay here is what our fraud budget is let's keep it below that and let's get as many customers as we can and we you know competitive things and trying to cannibalize our competition and blah blah blah blah blah but the atos did come um and uh we saw a couple of really cool ones uh ones that highlight so we we saw it come in they found uh if you're familiar a double orle uh numeration of time so basically what they R was to do was they found two or holes on our system and if you're not familiar with or Hole uh do you ask you

a question and it did through a yes Zone app basically um they had a list of breached uh emails and passwords from other reaches off from us from us is the camera we're safe and secure um and what they were able to do is they were able to take those emails and enumerate across and they got a yes no answer of does this account exist on CLE so that shortened their list and then they took the second list of the username and passwords and put it against the other or hole that existed in our system and they were able to then validate will this be a valid log so is it a valid user into the password and

that generating for them you know a funnel and a short list of okay we can now get into these accounts um we saw we we saw our metrics saw the um the encount uh takeovers happening and we're able to identify that and get ins sign Theus hacker and I won't fill there you talk about that more later uh what we didn't see was the double Wars they were in places that we didn't know existed and we're in this incident but like what is going on how is this attacker one step in front of us and someone on my team had the brilliant idea of well I wonder if they been the data from elsewhere they like no no no

we can see everything that's happening looking at the IP address like you know we could see it let's go back in time and you just see this massive Spike three weeks before where it's like oh there's the oral right there then they are using it and we have no metrix to notice it and everything like that so um that was one of the that was one of the fun attacks fun stories I can share more details there later um when we saw the inam takeovers then they came and they came to various creative meanss and again that's where you know security has to step in and we have to be Gandalf um and you know we don't want to be Gatekeepers for

the developers but for the for the bad guys we want to be Gatekeepers um so we we came in and we developed something called the behavior engine and sorry the text is a little bit yeah um this is my software all background coming in we're like we can s this F this is this is easy you don't need an expensive tool to do this let's just go find the really critical and white stuff and it's haer is going to be interesting in let's put a little bit of control in that and then we can s in their tracks so build something using it's the the okayand commanding to no no no chain of yeah there we go chain of mat powder what it

is is it's like down distance from last session if this is greater than 500 Miles give it a score of2 and each of these rules go through the once you know if it's the t0 or something like that um each of these rules fre for that adds out to once you PR it 3.2 on a threshold we flag it and we go somebody needs to look at this because this looks really suspicious this is behavior that is not noral for this account and that's a lot us get inside of the loop with the attacker and stop the THS from the account take over we can't necessarily stop it when it's happening um we if we

looked at it on MAR is too much data that's coming in to M loses but if we look at it when you're going to chose a pank Account Details for example we can go okay your your time zone of your browser is 12 hours outside you know lot in time that's probably something that's weird and you know it it has a s for few much too so this is again this is about uh I talking about the soft that we filed um this was this was what we built here to allow us to deal with it on a you know with impulse and and to then um it's words I'm looking for here to start data into this so This actually

isn't I didn't even take an outdate version we now we now go up to third parties and check certain high impact uh signals like IP addresses and things like that and that can that can give us a score and then we also uh track together all of the times that this verification has happened on this account and we go yeah you know what this there's something weird with this account and we can then go talk to our customers and go and you know take the time to reach out to them um one of the big things that we found from this is like we just have to pick up the phone and be like hey do you know that you're

logged in from India right now and they're like no I'm not and you're like yes you are um and and our our our lawyer our customer our customers are fantastic the ones that that kind of get it but there are some that just don't and and so they don't have MFA turned on they do after endall though uh but we have playing MFA you know we have we working with them like hey you know you need to you need to do this this is an ethical Duty you understand the data that you're dealing with like I'm given a talk in three weeks of like it's your job to do this you you have a bar uh you

know you pass the bar to do this um and and it's been it's been so cool to to have the you know the contexts credibilities capabilities that now the BAC team is getting into fraud even though you know while we're building up our fraud team and not something that like I never thought I would do when I'm starting an outside program but if you build the tools you build up all those things po you're legal to in there um and we learned a lot of lessons on the Fly I this Probably sounds like I'm polished and know what I'm doing the number of incidents I drop into and go oh [ __ ] I don't know what to do um all

the all the time um but you know you get good at faking being in control um and then the threats evolve because AI happen yay uh these these folks here um we have to we have to evolve with them so this was um this was our payments fraud uh ring that came in uh and these lovely individuals um are we call them because we had a group of human review we've done 9 hous and uh if you have a look at it kind of all start looking the same you swing your eyes we found a third party service that you can generate fake D me uh with an individual from a certain country um and you know it can

be a any state any Province any um you know any government ID that they want to submit and so now you've got your attackers evolving with AI and Photoshop tools and they'll put anybody's face on it to match the ID that they've stolen and generated and um you know the the work never ends for ABC um and and the security team this is a good place also to flug I'm hiring a manager uh security engineering manager come join me these things are fun um but that's the end of the talk uh I appreciate uh appreciate your time and um look forward to questions sorry what kind ofal P do you do you know what we we're primarily

GitHub uh right now we we all the pieces off that that is the that the evolution is to get INF those Fe but we're not there D question some of you slides you had the dependen Checker and you said initially you were blocking your developers but I didn't follow after that through subsequent versions how are you not one yeah so we we removed the block out of the CI chain so it was only if you were upgrading the package that we would post onto thank that's a great question we would post onto the pr and so we're no longer I I've removed everything from the CI chain because we're not moving so fast the vulnerabilities are not so damaging that

we have to stop you know stop the line for an automotive shop right hitting hitting the red button because somebody got hurt or something like that we're not that's not our uh our risk profile right so we can we can ship code while we go fix the cdds in Tandem and and just alert the developers when they're when they're involved in that yeah yeah that was a that was a a brain switch for me like when when I I couldn't get it I was like no but we can't let the bad code go to production and it's like it's already in production like you want the right fix to go to production and let's go do that not just stopping

everything um like how would you approach a all things you yours and specific detection to dis for yeah so my I think this is probably my again the um this first J um there there tools that you um and some of them are some of them are justs um that's that's be done and we we must build it but like whatever the environment is like I think it's a you can you can find the tools andure me because there's just there's just so much uh over that's um there is it it's theill wor of viy and and all depends on you know what bus are by how how easy is it to extract money from from the business uh and you

know how far is it to um to invin somebody to like in that security is 50 bucks per user per month um speak developers is does a lot of money he's a driv in value that you mean that's do whole this remain there um but that's not something when you're necessarily in a very thly G in security program somebody going be like sure have $300,000 I trust you um so you you got again kind of H those three points the FR one abilities um before this get trust that you're going to do the right thing when want give you this fun and you know some that long experience but when your brand new they're like

well you don't have no do you know how to spend money to to security when you've never done before

yeah thank you yeah thanks everybody