Bootstrapping Your AppSec Program

I must have scared half of you guys out of here I'm sorry hello everybody way in the back how's it going all right so who am I uh I'm David girvin I shouldn't be in Tech people made mistakes I ended up here I'm really not understand a background as a ship captain and an engineer and stupid things like that but somehow I hacked my way into Tech and I've worked for some companies like one password and we act purple and bit Discovery and now I am a senior product engineer product security engineer of architecture I don't know I have some stupid titles I don't really care about but it works so I have a very different view on how

these things should be and I think one of my values in the industry is I see when things suck so I'll be pointing out things that suck and making fun of everybody in different places if you're in sales I'm already sorry because I make fun of you a lot but those are my kids and my wife who are way too good for me and yeah some other weird things in there I don't need to read them all right okay okay what are we going to talk about uh we're going to outline building an appsec program uh with a different perspective on management integration I'll go over the cool tool stuff and Technical stlc and all that kind of

stuff that you need to like have in place but everybody talks about that stuff right it's not a big deal you can like look up any of Tanya Jenkins stuff and there's a million things on like how to set up your tooling and everything but I'm more interested in Social Engineering uh for good I used to do it for bad now we do it for good uh I'm not one of those like I went to college and was a good person I was a crappy person and very good at sneaking into places and figuring these kind of things out and went the other direction now I try to do it for good uh I have some

interesting thoughts appsec is sales if you suck at sales you suck at appsec so these are like skills we can all learn and think differently being a part of a team is super important to me and uh everything I failed at which is a lot of things so I I'm uh grossly incompetent um warning Concepts and thoughts are based on anecdotal experiences from building application Security Programs at places like one password and all these other fancy companies uh if you disagree with it it's perfectly fine if you need to email me and complain about my talk or any views I have that is my email address um if you find me abrasive and sarcastic but somewhat Charming you are correct so

and if you hang toilet paper over the back of the roll please leave now you're not welcome in this talk because you're a psychopath so what can we think about here syllabus this is what we're going to run through I don't need to read all this stuff right but um these are things that and you can grab my uh slides uh just hit me up on LinkedIn or whatever or maybe we'll put it on the website I don't know how besides Augusta works but um I have a bunch of this stuff in here and these are great blueprints like when you're thinking about it it just doesn't go for just appsec I use this for

everything because I build Out programs teams at our company I love hiring and that kind of stuff too so okay what is appsec application security or product security is a team program designed to protect all of your code you own that means everybody that has code should have an appsec team no matter how small the company you know unless you like Farm it out which is fine too um that includes the code your engineers write as well as third party party libraries and open source tools you use yes if you hack together a bunch of crap using every third party Library known to man you are still responsible for that it's also tools that you use you know

you're running all these open source tools there's vulnerability in there you are responsible it's a strong relationship with engineering and sea level stakeholders I know I look like a cartoon and someone who does not speak well I've spent most of my life chasing surf around the world I'm not like a normal human and I can still speak to the sea level every day and do it's internal sales you are selling to your engineering team because they don't like you because you're the [ __ ] who said your code sucks now you got to fix it and they said I spent seven months building it that specific way because I thought it was right that's a big hurdle to get over

uh we'll go through that don't worry uh it's a smart use of tools and testing uh I should have like bolded smart there because everybody throws every tool they can find especially open source ones that stuff and that's not really the way to go it's an understanding that if business doesn't flow there's no product secure this is a huge issue in appsec and we're starting to like mature in a way that's really interesting and we're seeing that we are actually a part of the business if we don't produce profit we do not get paid so we can't just roadblock everybody we have to like relax a little bit I can't just push our glasses up and scream

it's not old school grumpy gatekeeping if you're that guy stop it no one likes you at all especially people like me who come from outside the industry I worked in construction for years on the engineering side of things those guys exist there too and no one liked them there either it's not dumping a thousand volumes on engineering and saying fix it this is I I do Consulting with my company with it's called honey badger engineering for a reason because it's just ridiculous but anytime I get caught into Consulting that's happened they started an abstract program they said screw you I ran sassed uh yelling when someone something can't get fixed right away you have to be the nicest person in

the world or at least be sarcastic enough to convince people that you're nice building a wall and starting a war with engineering that is how you ruin a program immediately and I've picked up the pieces from multiple people who started it that way because they were way better at appsec than me but they weren't good at this side why spend money here yeah there's a lot of memes I'm like 12. so uh I'm really older than I look but uh 2020 the average cost of business affected by a data breach the United States is 8.64 million that's a lot of money especially since half of cyber security companies in my opinion are vaporware and they're put up

by VC companies they're just trying to get absorbed so if you if you lose money and you don't make money guess what you're doubles suck so reputational damage can be far worse than uh the original dollar lost I am huge on this so I threat model companies a lot of the times and do SWOT analysis for their space because I have a nerd on the side for business which is very opposite but my day job is very nerdy but I nerd out on like how business works and like you know uh the two big companies I've worked for which was one password in Red Canary I told both of them their biggest fear shouldn't be the data actually lost

which would suck I mean obviously it's if anybody ever lost trust with them um because people trust their password manager and they really trust Canary they literally have all of your Telemetry from every single endpoint you have in your company so um it's a different kind of way to start thinking about it um it's estimated that we lose 250 to 600 billion dollars of intellectual property being stolen every year that's why you go to China and you see all the BMWs but they're not BMWs and they weigh like 6 000 pounds it's very strange um hopefully your insurance mandates it I know insurance is a bad word but it needs to it needs we need it for policy

and Regulation and stuff not in the real world but in cyber security uh it's not 2004 and you're a damn adult that's my favorite line in this whole thing because I keep telling myself that every morning in the mirror I try to tell myself that all right first one let's go over hiring controversial take soft skills are more than important than being a Smee I know this is really weird I know I exactly it just uh if you didn't hear the the breath on the recording it was there um hot takes are so hot right now if you're a jerk nothing will get done at knapsec this is not pin testing I love pin testing I'm an offensive

security guy if I could just break crap and laugh all day I would I mean it'd just be amazing but that's not the reality your first hire in my opinion oh this is my opinion right it's I'm full of crap anyway so uh 50 50 mix of amazing soft skills and application security if you can find that one guy or lady or whoever is going to be um I don't want to tell your principal engineer but whatever everybody needs someone that's going to Mentor well you need that person to be balanced uh if they're just super technical but they suck at talking to people they will not Mentor well I promise you and if they're

not technical enough but they're really friendly well then they're going to look stupid in a meeting at some point if they're the upper range of people I've been that person multiple times high level of understanding of apps that's okay you know around the people that I work with I work with some of the best engineers in my opinion in the world I would say that I'm non-technical I'm obviously technical I code and I break and all that kind of stuff but I am Matt Graber is one of my colleagues I am not sitting there finding new dll injections from Microsoft before they hit the market and that kind of stuff um but you know it's I have a very good

overarching understanding of how security works because I'm a bad guy and uh and I loved that part of it um you don't need 10 years of Dev experience stop trying to find the Unicorn everybody wraps Up's like I want to get that senior web engineer who fell in love with security and works 10 hours a day and then studies for four hours a night and never talks to their family that's the guy that I want and I wanted to do it for a hundred thousand dollars a year that person does not exist and if you do find him guess what someone like dragos or someone else is going to grab him and pay him a quarter million

dollars a year and you're like dang it so don't look for that um you need to find people from everywhere you need to find good Bridge Builders this is a huge huge huge deal it's such a hard skill to find it's very easy to burn Bridges uh but you need to look for that in your interview process and and you know I'm the uh extrovert that adopts all the introverts in my company I'm like quarterly hackathons we all meet up in Denver I take everybody that doesn't have friends basically they have friends but not in the office right and I take them to the bar and buy them steak and make them get drunk and talk

to people and um that's also kind of like how I pair up people on my team if you get someone who's just this awesome software developer and they want their soft skills to work you need to buddy that person up with someone who's going to let them grow and they'll grow them too I mean I have a guy who's wrote half the code for simgrip on my team now I'm very thankful I stole him but he helps me with my like deeper python understanding and I help him not tell Engineers they're stupid it's like it's a mutual relationship and it's wonderful and we're good buddies if you just walked in I'm super sarcastic I don't

really call my Engineers stupid they're way smarter than me um appsec has insane salaries you probably can't afford it if you can then build a better team than me and and do the talk like right like I wish I could just pay everybody a quarter million dollars uh and this is just a plug for me funny sarcastic people can be great bridge Builders we work primarily with Engineers it's a different audience than traditional infosec and that's really true GRC people corpsec people in my opinion you want a different personality type there are more of the disciplinarians and in reality because they have to be they can enforce those things you know if you're downloading some sort of

malware porn thing on your on your workbox then you should be told you're being stupid right like that's NADA but you know if you're dealing with super smart engineers and you're like I just don't think we should use that Library you shouldn't be forceful you need to learn you know it's a different it's a different vibe yes that is literally I have a baby about that old I think she made that face to me earlier um and I never noticed that so hiring uh posing view hire Rockstar security developers fixed all the bad code so I have friends like Jim manicode and and all these like super famous abstract people who are all developers and

they're like no we don't need appsec teams we need real security developers and they go in there and they fix everything for everybody and that's their job and I'm like well one if you get a really deep code set like we have uh you're not gonna unless you're in that deep every day you have no idea what's going on there you know people should write notes in their code they don't so no one knows what's going on you're just trying to figure things out and you don't have time for that but two that does not scale at all um so even if you can find them it's very hard there's not enough of them so it will never scale like that's great

when you have 100 people right what about when you have ten thousand and eight thousand of them are developers I mean we have to be able to scale at appsec or you end up like all the other companies that get breached because they don't have web security right um what else do I write here I forgot they can easily uh become a block for engineering I see that all the time because it becomes schmee versus me it's like I'm smarter than you and I'm smarter than you and everybody's so smart and then nothing gets done and then everybody's like I don't talk with that team now because they're dick heads like that's your biggest fear that's how

you should go into appsec it's like we don't want we want to be the cool guys okay so this is a big deal for me uh leadership I'm really really really passionate about leadership because from coming from multiple Industries and coming into into a leadership position in security kind of right off the bat and being in Tech uh leadership sucks especially in security and I have some great friends of mine I've had some hard conversations with because I see how they lead their teams and not at our company like we're just friends and I'm like you're a dick like don't do that like you need to change things and they don't even get it you know a lot of people were just

really good at what they did and they got put into leadership and that's not necessarily the best skill set for that so um this is where many teams fail good security person isn't necessarily a good team leader if your team leader doesn't genuinely care about the individual's success as well as the teams you get shitty middle management and there's so much as far as the eyes can see that's the meme I should put there but uh man so don't ever forget that you can your HR team which is this is their job and they forget this needs to build out paths to success for individual contributors and people leaders such a big deal like if someone can't

get to principal and make great money the only way to make good money is to go into the leadership and people leading but they suck at people leading guess what everybody suffers for that so you know have hard conversations with uh HR people about this kind of stuff because it's important especially if you want to retain talent because Talent will leave in a heartbeat so my things servant-minded leadership this means your customer focused team you're leading your team so they're your customer so you're customer focused all the time um you're actually focused on your team members success and leading from the front doing the work you're the block person you're not throwing your team under the bus you know if your team

screws up you take the hit it's hard stuff you know it's it's really it's really difficult to get your brain around that stuff sometimes especially when they do something really stupid but um you know everybody around me was like and I'm not even the director right now at our company I'm actually like some weird shitty middle management thing um but I care about all the guys on my team I guess I'm the that the lead engineer or something it's like when we have one-on-ones I'm actually asking them like hey dude what do you want to do like where what what's going to be interesting where do you want to go with your career how can I help you with that

and even if it means them going somewhere else you know um these are real people you know it's not just about you and the company and it's it's a but when you do that and people see the sincerity of it they bust their ass for you not only do that but they do better work they feel more comfortable and there's not this like I have to get over this person on the team and I have to be better than them and blah blah blah it cuts all that crap out we have a very honest company I just turned on a giant pay raise to leave my company and go do a CSO job somewhere else and I won't because there's such

good leadership there right now so it's trickle down um humility and honesty uh makes up for not knowing everything gives team the chance to shine yeah oh man if you can lead from the front with I don't know that does anybody else know the answer to this I don't want to make the wrong decision here and you give someone else a chance to step up and like show that they knew something there it's such a big boost especially for younger Engineers you got guys in their 20s uh that means the world I've actually had guys slack me after meetings and be like dude thanks for giving me the chance to talk about that it meant the world to me

and I'm like cool yeah I didn't know and I didn't want to sound like a dick so I said I don't know um it's always worse to to try to be like know it all boss guy um honest visibility be open where you're crushing it where you need help such a big deal we have a very very my team is very mean uh in a loving way we're all actually tight we make fun of each other constantly but we're also incredibly honest with everything uh when we know we suck at something we're just open about it because we don't worry about getting fired because we're all tight we're not trying to throw each other the bus

um like when it comes to like some of my code I'm like my code is trash please don't look at that you can go fix it I'm gonna go run a tool and like make some speech somewhere and then we'll be good you know and they know that um taking responsibility for your failures and learn to care about other people which is so difficult have you guys met people have you ever driven a car people suck and you have to somehow care about them every day as the boss that is so hard to do like just driving from from Rome because we were evacuated by families were from Florida stay at my buddy's house to drive over here to do this talk

today I'm like I have my kids in the back and I'm like trying to hold it together like I'm gonna murder half the people on this road because people are driving like idiots and like I have the kids in the car and then I like always think about this and I'm like these are people the real people somehow have to care about them I don't know how to do that right now but when it comes to your team you know people are going to be butts and you just somehow have to be like okay well I'm bleeding and this is servant minded leadership which is I'm just going to kind of figure out how to make your life better it's where you

stop being a butt and I'm gonna tell you you're a butt so that's an important part of it too it's that Honesty part this guy haunts my dreams because the first actually both companies that came in there was this guy and this guy burned so many damn Bridges I have bought so many beers so many dinners had to talk about the stupidest thing why are some people into small dogs I don't want to hear about your dog but I will do this if I can build a bridge between you and my team to make sure this happens and then I have fantasies of kicking your small dog um sorry German Shepherd guy I just this

is my life every company will be different but here's how I try to place us um as appsec team we view our position as internal sales customer service a hundred percent similar to like modern I.T programs if you're IT company or like your it team does not see itself as internal sales to the rest of the company you need to go grab that director and be like you're 10 years behind this is how we do things now have good customer service um it's a a really weird thing we touch sea level that sounds gross I'm going to change that product managers Engineers managers I.T and sales and GRC I work with those people weekly that

sounds insane right you think I should just be like hacking code all day and running burp Suite until my fingers bleed but um no it's like we can produce a massive amount of value like we know vulnerabilities better than most people we can actually portray risk better than GRC can because we see the actual technical side of it as opposed to I mean there's technical GRC people but most of them are more compliance right so this is how you build that value this is how we become one of the most important teams because we're one of the most expensive teams in a company when you start building it out uh I do want my team to be loved by the

company it's really important for me my team is rock stars at our company like really like people when we fly into Denver everybody gets together people want to go out uh and hang out with a bar with us and last time we went to like open mic night and uh went and saw a comedy show stand-up show together took a bunch of the people from the company it's important I want people to think of us that way because it that's how you get stuff done uh we should add value to security and everything we touch and that's a really big thing if you're not actually adding value you know you can tell I have a business

background if you're not adding value you are not valuable oh that's a hard one to get around when you think like I'm just fixing stuff no you're just wasting money now if it's actually not valuable stuff so you got to be real poignant on on how you look at all those things yes this is my world I'm a threat modeler because I am a criminal so I know how to think this way and other people like that's a great skill set I'm like this is just because my brain's broken so uh absec path the normal one is always a good tool run tools sin volts two engineering [ __ ] about them not being ignored I can't believe these idiots

they don't care about us that's how appsec is or has been at least right that sucks um so threat modeling should help you find real areas of concern and exploitable vulnerabilities this is what you should start before you run a single tool this is everything this is how you should do this before you hire so you know who to hire so you know what kind of what kind of blind spots you have on your side what are your weaknesses you need to build out a procedure for threat modeling so others can learn and Implement you shouldn't be the only person threat modeling I have every one of my Engineers on my embedded teams connect threat model now and they're

good at it they're better than me they find all kinds of stuff um you shouldn't be threat modeling alone I usually try and have an engineer sales engineer or architect related thing with threat models I invite people in that normally wouldn't be there because they think differently a diverse mindset and a diverse group of people are going to find diverse vulnerabilities this is an awesome chance to build Bridges with engineering especially if they have never done this before they get so excited it's like Teach an engineer how to run burp Suite or take them into a threat modeling meeting give them a beer everybody's so yeah and then we can do this and did you know that we

keep plain text passwords and I'm like what the [ __ ] no no we don't do that and then I have to like you know reel it back in but you find that stuff with the engineers you don't find that stuff on I mean yeah you'll find plain text passwords on your own because that's dumb but you don't find the real vulnerabilities until you're like oh yeah yeah we SSH into that box and we all share the same password okay that's another time that my appendix burst when in a meeting you know and it's it's not going to happen though unless you bring them in so make it fun at beer food drink and black hoodies

give them black hoodies they think they're hacking just give them black hoodie like here this is our company black hoodie now you're a hacker sweet um and this is like my perfect threat model um but you know that's how it works government Overlord so yeah I made this big joke about the feds last time and uh and then I didn't read the schedule at RVA SEC and the FBI followed my talk and they were like I was like tell my FBI Handler you know I hope you got my Christmas present and the guy's on there he's like Dave I didn't get your Christmas present and you've been bad and I was like shoot so okay sdlc secure software development

life cycle yay this is what everybody thinks appsec is and it is this is this is our road map road map uh secure sdlc is implementing a set of controls to secure your software development life cycle your threat model first like I just said you can't secure what you don't understand pushing left is not always the right place to start I know if there's a sales person in here they're like that's wrong you need relationships built on Implement all these controls if you not you will get ignored you have to build those relationships first this is super hard but like I know every engineering manager I know our VP engineering very very well I have buy-in from product I

did all of that before I started even running tools or anything um you got to start with one control you got to plan out the next year and then you got to constantly adjust when people like throw a fit about it you have to be flexible many things won't work and you won't understand your work culture until you screw up and you piss a lot of people off but if you're cool and you're humble and you're like I'm so sorry you know I baked you a Turkey um they're gonna be like okay it's not that bad he gave us turkey you know whatever I don't know what you're gonna make them but after implementing control

step back and build a plan for continuous Improvement my KR is every single month continuous Improvement is a giant one there it's boring but it actually works these tools I don't care who you have you have Snick it's going to take you I know it's like sneaky sneaky I don't know you have that one it's going to take you a year to get that right don't listen to a single sales person they're all liars except ours they're good um any of these guys that sell appsec tools they take so long to tune and it's not just like the false positive stuff it's getting your vulnerability management process down it's figuring out what is false positives and actually can't be

exploited as opposed to just things you know that like oh that shouldn't be a big deal until that third party library that we had accidentally dropped something and now they can compound everything and now we're screwed it takes a long time to get these things so continuous improvements massive and it builds trust with your engineering team if they see you suck a little less each quarter on how you address vulnerabilities and how much time you wasted of theirs they're like okay those guys are doing their thing they're maturing it's a young team um what else did I put in there if a control is a blocking control start it as an education and suggestion control slowly move it to blocking to

mandatory make sure you have buy-in and written policy write your policies you need that's the stick right you don't ever want to use the stick but you're like no you can't write your own crypto dum-dum they're like why not I'm an amazing developer like it's in the policy I put that as number one you were never allowed to do that um it makes a huge difference but if it's a blocking control if you're putting something in your CI CD pipeline it's a tool and somehow it has blocking don't do it don't do it for a full year I know that sounds crazy wait to see how many times you would have screwed up prod before you have the balls to block it

really like threat model take your time on everything it's really important because that's a super way to like burn all that value that you made they're like you're just wasting our time now I had to have 22 Engineers this weekend roll that back because we couldn't get it out in time and then we pushed it too fast and you know it gets it can get bad step for each control I actually have these written down on my desk it sounds silly but I have to remind myself because I get ahead of myself and just run through things so you want to learn what are we going to solve technology we use what part of the company do we need

to understand uh enumerate plan get all the people involved and start putting everything together design look at how to implement the control in your company it needs to be a holistic approach don't just read anybody's book on apps that can be like that's how we do it think for yourself you're all intelligent people if you're an appsec there's no dummies here um you need to get your approval and your buy-in before you execute that's a huge huge deal and you need to figure out who actually you need that approval and buying from it's a big deal you miss that one person and you screw up their team and you jam them up you're going to

have a rough day right so execute do your plan right and then improve that's the other thing you figure out where they're pissed you go hey I'm so sorry I screwed this up please let me come back and fix it and make it better for you okay this is my sdlc plan uh that I like kind of I mean this is the basic stuff right these are the controls you want to implement um that's really small and I don't have glasses on uh but yeah so you know sras threat modeling uh you know you got all these different uh companies in here that you can just buy and do stuff with and they make your day suck every day

and then you have to have calls with their sales people every other week and you just learn to hate them sorry I zoned out a little um you know but uh these are the things you have to cover you know it's kind of hard I'm not going to go through all that there's a million talks about that but my slides are here or if you just hit me up on LinkedIn I'll tell you what I'm thinking first control boom should be holistic I like that word based on your company don't start with stats that's not holistic that just pisses everybody off SAS has is a very important tool that static analysis security testing if you

don't know super important tons of false positives makes everybody angry don't start with SAS that is a good way to just plummet your program and it's expensive um and if you do one uh find one that you can tune very easily that's I'm not going to like give Brands and stuff uh pick and control with low false except for Red Canary you can always use them uh pick control with low false positives for your first one this is huge writing policy to back up your Patrol like we talked about you need to be thoughtful set your tone for the entire security program this is so much bigger than you understand in the beginning that first

control is like if you can kind of go okay with it people like okay they didn't burn the house down yet you know but you will piss someone off cookies and beer help it's true okay don't there's a trigger warning don't get dazzled by shift left everybody knapsack shift left shift left shut up it's a horrible way to start you should shift left after you have a mature program sometimes you're better off shifting right fewer false positives the further right you go the farther left you go you have tons and tons of more false positives uh shifting left will still be a goal but sometimes it's easier to start right next controls different for every team holistic

approach uh you have a budget and a lot of code you could try saster SCA um I wouldn't say SAS I just put that in there I'd say you know try SCA that's a lot better all your third party analysis uh are you overwhelmed is it just you and you're like I'm drowning security Champions or slackbot my slackbot's gangster my slackbot will tell you how dumb you are when you do something and it'll also give you praise it's like hey good job buddy you didn't write your own crypto today um hired some folks yes we have a team now embedded security Engineers I'm very passionate about this we are one to one right now that's not

realistic for anybody I mean that's like oh one to three is fine you can sit through three 15-minute um stand-ups right and you can do that two times a week and you actually know what code is going on you know what issues the engineers are actually working on at that moment massive thing if you have enough people uh sort of fancy new product coming out if you talk to your product team do you know your product team if you don't know your product team get to know them because those are the guys that are like we're gonna build this thing and I don't know how engineering Works we're gonna plug stuff into it and then it's going to do these

things and then everybody's gonna lose their data all right if you become friends with them you can threat model or do sras right off the bat security risk assessments are kind of like old school you know but we still do them because they kind of make you look at at the engineering and the code and everything so it can be important um what worked for me for my first ones for this last let's get to go with Red Canary that's what this one is Dash and thread modeling that's how I started I love dast Das Das it's cheap no one cares about it anybody can buy burp sweet zap sucks um but you know it's

it's amazing run run desk you're actually finding real problems in production don't run on production if you can't handle it like don't just go ha and then you crash production uh it's it's great if you have a safe place you can run it locally or something like that um but it's a great place to start because when you do find things it'll be a manageable list like you run your even if it's just burp spider or something like that it'll run through and it's going to give you like what 30 or 40 things you're like okay I can one person can go through this and tell which ones are false positives and then you go hey

I broke into your system here don't do it that way but it actually gives you something real um and then the threat modeling side is amazing to start that way if that's your first control you're going to not only do threat modeling but you're going to teach each team how to do threat modeling and you're going to do it with them that one builds Bridges doesn't cost the company well it costs a little time so that is a real thing but you can do it and manual find some scary stuff got kubernetes how many of those spots you think are written as root I promise you a few of them are you want to see someone break a

container out go watch e in cold water it happens so you will find some really really good stuff that way and you build those relationships which are super important uh embedded with engineering like I said a minute ago I jumped ahead on my slide sorry uh when you're for a mature team and you're growing this is the most successful thing I've had if you don't have enough Engineers I have this thing that I call the security task force or so we're um we're a Red Canary we're all like bird names like we're the flock right so I keep trying to get them to be the murder Squad because it's the crows and they're in the black hats but it

hasn't caught on yet they just think I'm weird I'm like let's go murder Squad and they're like no David sit down um but whatever that's the other way to do it you can embed these engineers and they sit in on the stand-ups and they help with stuff and they're it's amazing when you have a security engineer in stand up and they're like hey how should we connect to this third-party API and the guy's like oh we should do that and secure you guys like oh hey I think if we do it this way we don't really have to worry about any security issues because that way can can be good or it can be a little off and they're like oh

cool thanks for making me not rewrite this entire API connection it's so powerful to have secure embedded Securities but not everybody can do that there's two of you there's three of you right what do you do get buy-in build trust you don't have to hire 10 security Engineers you can go in work your butt off a couple years in your VP of engineering is like those guys are solid they've been doing such a good job security task force get a two-week Sprint rotation with developers on the security team you will work on real development problems you'll work on backlog you know old Tech whatever you got a patch or whatever but you have these people for

two weeks let's all run burp Suite today let's just destroy some crap where offensive security but these are regular Engineers they leave with a security mindset they leave that two-week Sprint with like oh the security guys do cool stuff we could do cool stuff it's like you know it's like a advanced version of security Champions I hate security Champions it's the name of it bothers me it sounds like some real creepy thing like but that's another way to do this if you don't have the Manpower um yeah then there's the things and the words I wrote build trust have you seen how many times I said build trust it's amazing tools yay what do we want

all the tools why because we never ever ever want to stop looking at false positives for the rest of our lives oh man I'm like become a minimalist with tools I'm trying to lower it and lower it and lower it even the big name companies because I spend a lot of money I'm pulling out some of their newer features because they suck they're not tested they're giving me all this trash they're built they're burning my value right now they're burning my trust with engineering and I don't care what I have to do like there's some great you know open source stuff out there like Sim grip open source and even the Snick open source and like some of

those they may not be Ai and machine learning which is all Bs anyways but they can like you can tune them real easily so you can be like okay I know I have one problem these group of people keep writing passwords the wrong way and they're not using this so I can search for that you can start minimalizing this where you don't have to use these giant companies that just bombard you with crap all day if I have to sit in one more sales trying to re-hire me meeting thing I'm going to lose my mind but um don't trust the hype sales people can be sketchy I know I was one of them uh the less tools you do the job adequately

the better uh looking at 17 things a day is the path to the dark side that's how you turn into that guy from before build a plan before you acquire the tools this is so big that like list of things that's how I started and then I had all my team members there's a lot of us there's eight of us we're we're beast mode team because we got buy-in from the beginning so we had budget and I'm like hey we're going to take a control one at a time we're all going to look at and we're all going to compare and I built a little thing in Excel and it's like does this do this plus five points does this

do this plus seven points for each control so I could so I had to be more pragmatic because these sneaky sales vendors know that I like whiskey and they're like hey Dave we sent you a hundred dollar bottle of whiskey you know just for checking our demo out I'm like you sneaky and then it's even worse when you push code you know I've written a few rules for some of these companies and I started getting their t-shirts and I'm like oh I have T-shirts now I I could use them next year you know it's it's all it's a bad social engineering we're doing the good social engineering uh spend time real time testing tools I

spent like 30 days on each control minimum I got pocs from all the people that were supposedly good and I pounded into them I looked at everything we had discussions like very like straight up fights at one point over certain tools because people were passionate about this that are in appsec um which I encourage I want us as long as we're not being rude to each other to basically yell about things it's good um we tested all the tools and then we made it really really rational decisions and we're already changing some from our second year uh second yeah second year acquisition stuff so you know that's that continuous Improvement side you're not going to pick and people lie they

say things can do things that they can't and then you're like wow you're a billion dollar company and a three-year-old wrote your API and it crashes every single day cool I'm glad we gave you seventy thousand dollars you know it's it's insane you don't find that stuff out until later because they hide it in the pocs so be ready to tell people there's a chance we'll roll this over because we don't want to burn our value right um tune improve before you get another tool spend some time on it it doesn't have to be fast showing you know whoever's doing your budget VP engineering c-suite whatever CTO showing them how fast you're moving and how much

you've Acquired and burned through your budget is not show anything like it to us it does because we're like look at all we did if you get one tool and then you spend the next month making it perfect as perfect as you can before you roll it out you're testing it locally and making sure you're not just going to destroy everybody the people up top appreciate that because redoing something engineering wise costs a lot more than doing it the right time right so spend some time it's a slow process don't get you know worried about it and explain all those things up front for whoever your buy-in is uh no budget no toys yeah so how do we get budget

hey what's our time sorry on this one 3 45 to 4 4 okay cool uh we're fine how do we budget apps that program this really is the internet if you didn't know this if you don't work in appsec with third-party libraries when that idiot didn't sign the stuff for Ruby last year we're a ruby shop I had an aneurysm like I was like what am I gonna do we have all these customers we have to fix this things and Nancy can't use it because he didn't change his licensing and blah blah blah blah blah have backup plans for your third party libraries you know and have them written down I know that sounds crazy but I

literally have a list of what we have you know on a Excel spreadsheet because I'm in Tech and then next to it I have competitors that I can grab because I don't you can't memorize all this crap there's so much so I know like okay we just lost that because of a vulnerability we can't patch let's see who we can plug in there um it's work but it's it makes things go fast so sometimes you can't open source tools that's what you got to do they still cost they still cost money it's just Manpower they cost more for tuning for all these kind of things and I'm getting pretty burnt out on a lot of

the vendors that build off the open source and they're like oh built you this amazing GUI it's going to work better so you pay us money and then it doesn't you're like cool so you really have to decide like what are we going to build where are we going to buy it's it's a really tricky thing um but you can slowly show value which will earn a new budget but you have to be upfront about it be like I'm going to prove to you that we're a team that's worth money in this company we're gonna and and we're gonna do that here's my plan so you don't jam up engineering uh you plan everything out Implement

control slowly in fact it'd be super open with everybody that has buy-in on this it's massive deal you need to tell people because they're going to spend 100 Grand 200 Grand if you're at like Amex we're half my team because I stole them uh used to be I mean they spend like three four or five million a year on their security tools right so earn that trust and you can just be open from the beginning um you got to plan everything out you got to keep your metrics and findings remediation we'll talk about metrics metrics suck but you I have like a few things that help me and you have to build those bridges you really do and

you got to keep that communication I have one-on-ones with so many people in my company that are like 30 minute like coffee morning I have like this big espresso machine so I'm always like making espressos and hanging out with people and it seems like I'm wasting time I'm not when I need to get something done I'm the person that anybody in security I'm the whole it team will go to because I I care about people and we've built relationships and we're friends um now with some value you do these things you got to learn to communicate a business-based audience to a business-based audience if you can show Executives why financially it's good for your company you'll get a real budget

being able to communicate effectively with c-suite and board it's what uh you know that's how you can get an exploit cost the company how much money okay this is how it works that's that's hard to communicate if you're not used to it if you're just used to working with Engineers I career coach a bunch of people like as part of like my wife and I's Ministry is get people from poorer areas or you know places where they never heard that you can make real money in Tech into Tech and the number one thing I do and it doesn't matter if you're you know black white Latino like whatever if you're from that area I make you go

to Toastmasters before you even start anything and people are like that's weird I'm like no it's not do you know how many interviews I have were guys smart but he can't even say his own name because he's never had to like communicate things properly how are you going to tell the CEO that we're going to lose 40 million dollars if they don't spend 150 000 on engineering right now to fix a problem you know that's the difference between a hundred thousand dollar abstract job and a 250 000 abstract jog is being able to communicate that line that's the difference between a CSO and like someone who's going to tap out a principal and they're never going to

move into in into any of that stuff so it's huge and um like I used to do bids with my company and I worked across that and I know I talk like a dumb Surfer because I'm from Hawaii and everything but uh I know how to like get value get distress if something's really bad I can I can go through and tell people in a way that works for me and works for them and it's sounds silly but it's massive useful metrics God I hate metrics so you might hear have to build their own metrics yes you feel my pain bro they suck so much and I've like lost my mind because I hate Splunk I don't know

the language I'm always trying to like Google it on stack exchange like how do you make a pie chart and Splunk and then I'm like I hate this keyboard against the wall um okay this is what I do they're so hard but I figured out what metrics need to do I like hacked metrics at least for my head because I'm so pragmatic I have to like figure these things out so I can follow a path um they need to tell a story and the social engineering part What story do they need to tell for who you have to figure out what that person thinks and what they want from their job and I've actually asked people this

because half time csos will ask you for metrics and you're like I'm like what are you trying to get from this and they don't know they're just so ingrained in it that they have no idea what they're doing and you're like okay we really need to figure this out before I go and waste a bunch of time and Splunk and cry for the next six weeks um so now I make them I make my metrics tell a story and I make them tell a story that's going to fix the question that needs to be asked by whoever it is asking me for it and they have to like put their time into I don't ever anymore when someone's like I

want some metrics on that tool like no I'm not gonna waste time on that tell me what you want answered and I will build you a beautiful colorful story in horrible Splunk dashboards you know um but that that really is what it is and it's It's tricky because you know it can be the little things like well we had this many vulnerabilities last month and we only have this many this month and it's like did that really help us I mean it showed that somebody updated one package what's our risk there like what what percentage of risk do we reduce on the company like then you got to build a few more things because you're like okay I'm

now telling a story I'm showing the c-suite that we lowered reduce lowered the risk reduction by 15 this quarter because we knocked out 1500 critical vulnerabilities that couldn't be exploited but if something fell apart they could you know what I mean and this is all ethereal this is a little hard to like get in there and figure out but now you're showing like okay well if this is real we might have saved the company a million dollars okay you guys are worth a little bit more now you know we were ahead because we had all this fixed when that patch came out we could actually patch because we're on the correct version of Ruby on Rails right now

instead of you know six years behind because no one will change any of their code like those kind of things that's massive because I'd be like hey we patched this we wouldn't have been able to patch it and we would have got busted on this one it's like when um uh uh what was it called HTTP smuggling I found that at our company uh using burp and I was like oh my gosh we have to figure this out it like got in there with engineering and spent much time it was Amazon's fault stupid Cloud Player uh but then I called them you know and and had it fixed but it was like one of those things if we wouldn't had our

ducks in the row and we weren't organized and I wouldn't have already proved value when I called cloudflare they probably wouldn't like kick rocks but we were big dog it wasn't right Canary um but those kind of things are huge um so you need to have some sort of view on things education training is massive I'm in charge of this at our company right now for this quarter it's the hardest thing there is but I I'm so passionate about it but I'm also so burnt out on it because it's so tricky but um education training can't suck I don't know how to tell anybody does that you know like I've bought the big vendors and I've I've used our own and

I don't even know what I wrote on here actually uh social engineer security culture in your company that's so important uh onboarding from the beginning you need to do it uh be passionate about teaching I'm just running through this because I'm going to change it um yeah Okay cool so that that's all true still uh this is where I'm at right now with it though because I just revamped this again I have a company that gives the little training where the engineers can go in and we do a quarterly training and they have to do their apps like training and blah blah blah blah and they do it because I begged them literally they didn't they like threw a

fit in the beginning but I had enough invested that I'm like bro how many times have I got the bar tab for you guys like you're gonna do this [ __ ] it's gonna take you an hour a quarter um and then I also because the new guys don't know any better everybody comes along gets that training uh they have like I built out a special one just for onboarding to make sure we hit all the basics for our code base what like we could see them introducing you don't need to hit them with a bunch of stuff like that they can't use if you cannot be SQL injected don't make them train on SQL

injection um but now I built a couple ctfs that we all did and now I'm doing some stuff where I'm looking at all the metrics from there right seeing where we fail over and over what stuff we actually hit on I'm like okay okay well this team works for uh Linux agent we built our own you know and this team Works mostly with the Microsoft stuff and this team is our Cloud team they all have a security engineer so the next step for us is they're going to do lunch and learns with their team that are pinpointed to their weakest points so it's actually interesting to them because they know they suck at it every engineer knows they suck at what

they suck at like especially if you just get busted on it you know with multiple times with different tools so now we're pinpointing we're getting a little bit better but it's still a long way to go this is the hardest one if you can figure out how to train people and make security fun in your company and every company has a different culture you're going to win because then I mean my whole thing is to work myself out of a job I've already almost done that I hired a lot of people are smarter than me I'm like okay maybe I'll just go to sales make giant money now but it's the same thing with this stuff if I can make

all my Engineers into security people why would I even want to uh be around anymore I'm okay I'll go do something else I'll make canoes uh social engineering security culture culture good social engineering learns what is beneficial to both parties this is my sales background knows the audience has buy-in from all parties needed builds a culture based on trust and visibility servant leadership bad social engineering self-focused goals are you just trying to get stuff done for you if you're not adding value to whoever you're doing it to then it's not good does not benefit all parties involved manipulation zero visibility uses a stick instead of the carrot so I was going to be a physical penetration

guy I can break into anything in the world like I grew up first job as a locksmith I love all that crap uh then I had kids I'm like okay I actually want to be home for my kids but it's a massive thing the bad side of social engineering I'm very good at it I can walk in anywhere even with this big stupid mustache I when I was like 15 I snuck into tons and tons of concerts by just walking backstage and acting like I was supposed to be there and that kind of stuff that is the side though that I see in security they're just like oh man you know you need to do this and do this and

like there's like Dave told me to be nice to you I was like no you have to offer them something in return there has to be something there yeah I'm going to take you out and you know make you talk to people and eat good food and drink beer and stuff like that but that's not me getting him do it I'm always giving them value somehow and you have to figure out what it is for yours you know I've had instances where we fix code it makes their life a thousand times better because their code's more organized they have testing in place now and they're not getting jammed up when they're trying to push to

prod anymore because it's not a tangled mess that's huge value you just got to figure out what it's going to be for your little team that you're working with um yeah we went through all this stuff continuous Improvement I'm on this next quarter I'm running continuous Improvement I hate it because it's vulnerability reduction all that kind of stuff uh but it's so important one should you control can ruin your whole program uh constantly improve everything you implement uh why would anybody take it seriously if the team doesn't care which happens all the time right you have Sloppy code you have Sloppy tools you have Sloppy findings you don't know why are they going to respect you right you know it's like one

of those things like I'm definitely that guy like makes my bed in the morning even though I'm like the biggest Dirtbag I like lived in Vans half my life and I'm like I'm gonna make my bed in the morning this is gonna you know you can respect me now but it's the same thing you gotta like take care of your stuff um refine the obvious things first get a recurring meeting with engineering managers uh to critique controls huge get a meeting once a month please tell me what I'm doing that's making you angry and sucks I say it just like that and there was oh yeah this was my time to kick you right in the balls Dave

um but it's so important because you get honest feedback and they love it they're like oh you're actually you actually care about this stuff instead of fighting in like a stupid meeting about it um refined controls by studying your super sweet new metrics automate everything you can because working sucks um be customer focused to your engineering team and be objective just because it's your favorite tool does not mean it does not suck usually it's always my favorite tool that I have to like roll back I'm like dang it failed again and I failed many times uh some ways forward ah yeah easy stuff try to find the most uh obvious things first wait is that a different one oh yeah it was

sorry oh no that was it I didn't even know I hit it okay yeah so and that's pretty much it it's it's a weird talk right it's appsec without all the cool technical stuff and I didn't show you how many times I've broken something hacking people is a hell of a lot harder than hacking code but is the most important thing and that's why appsec Engineers are making more than just about anybody right now it's stupid money so uh I'm on LinkedIn I'm super open to this stuff if if you ever need questions asked or anything like that just I'm I'll I'll talk with you if you need like business stuff I have a consulting company you think but does

anybody have any questions oh whoa okay let's we'll go this way is that cool

for education uh yes if there are any training methods that work better than others in my experience uh no you know what I've the only thing I found is an individual approach on trying to figure out what helps people and and how to get them excited that security task force thing does more than any of the educational stuff if I can sit down like uh like our ctfs that we have because we have quarterly get togethers that are work because we're all remote um our CTF hackathon stuff goes people get so excited about that kind of stuff and then you can be like okay cool and like while you're there because you're actually involved you're like hey

let's do this I'm going to show you guys how to use burp real quick or you know like I'm going to show you guys how to throw a SCA scan on all your code real quick to look at your third party libraries like that kind of stuff goes super good and it's not it's almost like not teaching but teaching you're secretly teaching yeah so is that hopefully at all

what do you think people can get well yeah at my company they get behind the idea my CEO wrote half the code we're all security people at Red Canary I'm cheating most places no you're not going to get uh the c-suite of a non-technical company behind security but you can learn to communicate risk to them on and on financial level to the point where they're like we don't like this but we're scared to death of these guys like and it sounds bad but that's the only way that I've seen it really work and I've sat down in meetings where people hired me on to Consulting and I talked to the CEOs and I like went through some stuff and I'm

like I'm gonna take my consulting fee for this meeting but we're going to cut it off from that because you guys aren't ready for security and and that's gnarly because you think I just take money but I actually have you know some ethics but it is no point I mean I have kids I don't want to waste my time with people who aren't trying to like better their business right so yeah it's it's that threat modeling thing I threat model everything my wife hates it she's like you're such a damn nerd but I threat model everything like you know I would look at my audience see what their weaknesses are see what the value that

we can give them okay so maybe a dumb question but just to confirm there's there isn't like a Security Plus for developers there's just some kind of class or just in general like big developers this is how you security oh there's a hundred the problem is developers don't want to do them uh it slows them down developers want to create hit their kpis or KRS whatever and move on to their next cool little feature they want to build right um is if a developer usually has any interest in security they become an appsec engineer it's very difficult because and you're just giving them more time like at least all the developers I know work too much

and they have hard deadlines and if you're like hey we have this five hour course that you need to do or like Security Plus is like a 25 hour course they're like kick rocks nerd like that is not happening I'm building code I'm doing my thing I've been doing this for 20 years don't tell me what to do and it's it's it's a different mentality like you have to get them to care about security and then it's almost self-learning like it really is like I I generally think if you can hack that culture in there they stop uh caring about the little education you have to do and they start doing their own education does that answer you

kind of sorry none of my answers are like yay oh yeah uh these guys so I'm gonna go [Music] is one of you blue team interested in Blue Team okay here we go here we go here we go I don't want to give you something you don't want sweet those are for asking good questions and sticking around for the last talk it's like five of you awesome guys again I'm David thank you so much for coming I really appreciate it [Applause]