← All talks

We Need More Mediocre Security Engineers

BSidesSF · 202240:533.4K viewsPublished 2022-07Watch on YouTube ↗
Speakers
Tags
CategoryCareer
StyleKeynote
About this talk
Jackie Bow - Keynote: We Need More Mediocre Security Engineers The field of information security remains one of the most isolated - and at times, elitist - bastions in tech. We self-impose the highest cost of entry - be extraordinary or get out. Year over year the demand for security expertise and employees only increases, but our numbers don’t grow to match, and we’re burning out. It’s time to rethink how we talk about what we do, and how we invite others to join our ranks - and convince them to stay. Sched: https://bsidessf2022.sched.com/event/rjp7/keynote-we-need-more-mediocre-security-engineers
Show transcript [en]

now enough rambling from me um time for the real content to begin um our first keynote uh is we need more mediocre security engineers so a jackie of all trades master of none jackie bowe has spent time in roles such as a malware analysis reverse engineer security engineer and head of security at places like facebook patreon and the u.s government these days she can be fined scale found scaling security teams coaching and thinking about how to reduce burnout and increase compensation in the industry oh sorry compassion in the industry well that's important too you know that's right um freudian slip um endlessly curious and easily excitable she is currently the head of detection response at asana now without further

ado welcome to the stage jackie [Applause]

so many things

hi everybody just for the people in the back there's still like a bunch of seats in the second row right here there's some more seats in the front please come in and i forgot i can take my mask off that's so much better all right awesome yeah and there's some over here as well if you're waiting for a seat

great hi everybody i'm really excited to be here like really excited like i was beaming under my mask going up all of those escalators besides was the last in person conference that i went to and so being up here is really incredible for me i just want us to take a second to realize that we are here in person maybe look to the person next to you if it's someone you haven't met say hello

new friends new friends i also really want to thank reid tim and all the organizers uh it's really hard to put on a conference it's incredibly hard to put on one during a pandemic and i just really want to give them a lot of kudos for putting this together and getting this community together that i love and also inviting me up here to give you some of my spicy takes first who am i if you haven't met me my name is jackie bowe and i've been working in the security industry for just over the past 10 years mostly as an engineer in the realms of malware reverse engineering and threat detection i've definitely dabbled a lot

i've done cloud security product security corporate security devsecops as well as tours as the head of security and leading product security and now i'm directing threat detection and response at asana i love solving puzzles that's why i love this industry i love whether they're technical interpersonal or organizational i think of myself as endlessly curious and somewhat easily bored which is why i'm a security engineer which is why i love this industry but okay before we go any further let's talk about my talk title so the word mediocre kind of feels like i am just lightly slapping you with a you know white leather glove it feels like an insult just to say this word why did i choose it and why did i say i

want more of this in our industry well first i chose it because it's spicy i chose it because it would get you thinking it would get you maybe interested to come and hear me talk i also chose it because our perception of this word being so negative if we look at the actual definition it's of only moderate quality not very good pretty benign not actually that negative of a definition but our perception of it is pretty negative and i just want you to think about that as we go through this talk how we can kind of change what things are by how we perceive them so first i want to share a story so imagine me

a brand new security engineer at facebook i've been hired to mostly do malware reverse engineering to try to track actors using the platform to spread malware so i'm given an assignment i'm given a binary and i am told to find a signature that we can deploy to stop this spreading on platform so you know pop-up ida pro looking for patterns i am going to write a signature on clam av which is in my opinion still the best open source antivirus there is and i find a pattern in the binary that i can use to uniquely key write it in the hdb format film pretty good and i push it to production those of you who've been to facebook

classic campus know that there is a place called the sweet stop it has some of the best ice cream and i love ice cream so of course what i do after i push this signature is i go get myself ice cream and i'm very proud of myself i'm like you know there was a reason they hired me there's a reason i'm here i'm gonna make zuck proud this is great i get my ice cream cone and it's mint chocolate chip with rainbow sprinkles because i'm chaotic and i'm walking back feeling really good i get back to my desk and i see my manager who's my mentor and the person who brought me out to work at facebook

and he's like huddled over his laptop doing something and i'm like what's what's going on and he's like well the av tier crashed and i'm like oh and he's like yeah a signature was pushed and it's it's crashing the tear thing about clam av signatures if you forget a new line at the end of the text file it crashes just does not work rav tier also at this time did not have linters or things to check for this so effectively i stopped file uploads to facebook messenger for more than an hour but less than three ice cream ruined so why do i share this story mostly because i think all of us have stories like this some of us have

multiple i like to tell this story to all of my reports the first time they crashed production because the truth is we aren't perfect none of us are perfect and throughout my career this has just popped up again and again and the thing is no one is perfect this is in part why we have jobs it is our job security the fact that humans make errors i want to point out this quote from the dbir report thank you dbir writers uh that talks about uh the human element and that the human element continues to drive breaches this year 82 of reaches involve this human element whether it is the use of stolen credentials fishing misuse or a simple error people continue

to play a very large role in incident and breaches alike turns out ironically that systems built by humans are susceptible and vulnerable to humans themselves i know i know it's wild but for some reason in this industry we believe that as security practitioners we need to be perfect and in return we expect each other to be perfect infallible amazing hackers that always save the day and never forget to include the new line at the end of their files who have never reused a password or ever been a beginner at anything we set unrealistic goals for ourselves to know everything we pretty much just expect ourselves to be unicorns what's more as we expect that we're

doing security work not just in our jobs but we're also hacking on side projects we're doing capture the flags we're reading white papers we're keeping up with infosec twitter which have you been on there oh and also hey going to conferences on saturdays yeah basically we have this perception that in order to be a great security engineer or practitioner security has to be your life there seems to be this idea that by living and breathing and only doing security we're actually making ourselves better professionals and keeping the world more secure i have a hypothesis though that our extreme expectations of ourselves and each other drive burnout not excellence we have this idea that more is better

you know living breathing only doing security but just as noise doesn't make better alerts in a sim hyper fixating on perfection doesn't make us invulnerable to attacks more doesn't actually equal better what more equals here is burnout okay but what is burnout really you know we throw this term around a lot these days uh into like as of 2019 it actually has its own entry in the international classification of diseases icd-11. so burnout is a reaction to prolonged or chronic job stress and is characterized by three main dimensions exhaustion cynicism and feelings of reduced professional ability so wait cynical exhausted with some crushing impostor syndrome maybe masquerading as ego seems like we already ticked some of

those boxes and thank you to shutterstock for this content so what are the effects of burnout there's actually a lot of physical and psychological effects including all of these type 2 diabetes coronary heart disease headaches prolonged fatigue insomnia depression and i also believe that our work predisposes us to burnout more so than many other industries has anyone here experience burnout or something like it raise your hands nice thank you for your honesty so i myself is burnt out pretty much like multiple times it's actually part of why i'm standing up here in front of you i'm an advocate for preventing burnout and burnout prevention because of the mental and physical toll that i know that it takes

it's a big reason why i'm a manager now of people because i want to help others who love this field and who are passionate about security be able to do it for a long time so why would cyber security predispose us to burnout let's just kind of take a look at the past couple years first we have coronavirus which pushed everyone to pretty much do everything on the internet whether it was basic human connection care groceries jobs we saw the pandemic push the adoption of digital means an incredible amount we also had the ever fun log for j which a lot of us spent quite a few hours over some holidays working on and still are

working on now we had the colonial pipeline hack which caused a rush to gas pumps like a real physical implication we have the solarwinds supply chain attack which for me kind of shook me because the fireeye toolset was exposed and as someone who is a malware analyst like growing up as a malware analyst i looked to mandian and fireeye as the people who knew everything and for them to get their tools exposed for me who considers myself pretty jaded at this point i was shook and then you know there's massive geopolitical uh issues that are still increasing with huge cyber powers like russia so what are the effects of burnout specifically on our industry the first

and most obvious is we just leave we just stop working the second which is more insidious is we're less willing to train and connect with each other we're so busy working and attending that we we don't connect and the last is we stay reactive we only can handle what's coming up and there's things coming up all the time i like to call it the wheel of reactive hell this is where we end up when we're underwater with alerts and pr's and open rolls and new headlines and it's pulling us in all different directions it is the voice i hear most people speaking from when they're burning out i have a hypothesis again i have many

hypotheses we are stuck in a reactive loop of burnout and a bit of it is of our own design i want to talk about some numbers we have 600 000 open rolls currently and from microsoft's calculations it will be up to 2.5 million open rolls for reference the number of people working in this field is just over a million that means our field needs to grow by at least more than two-thirds to meet current demand we are no longer in the days of trying to convince stuxnet trying to convince people that stuck stuxnet is a real thing which i remember conversations with my parents trying to explain what stuxnet was we are in an age where cybersecurity is

everybody's business and we're not scaling to meet that we're not going to scale to meet these numbers and what's more do you know who's going to do these jobs it's going to be us our current burn rate is unsustainable we're burning people out faster than we're bringing people in i have some ideas though so what do we need to do we need to dismantle this security unicorn we need to challenge these perceptions of who we are as security practitioners and who can be a security practitioner so i have some ideas on how to do this so how do we dismantle it first we have to start seeing each other as allies rather than competitors second we need to tackle our tendency to

overwork and as well glorify over working and finally and most importantly we need to change our perceptions of who is worth hiring first let's talk about allies not competitors first off and pretty basically we're just not that nice to each other whether it is someone that's currently working in this field looking to get into this field our customers the general public our parents we're just kind of elitist standoffish we're more likely to challenge each other's intelligence than get down to working together we have this idea that as security practitioners we must know everything already off the bat and what's more we must be the only person to know everything there's this thing in cyber security

it's an affliction and it's going around it may get you it's this idea of scarcity this idea that knowledge can only be gotten by a scarce few that there is some artificial ceiling to the number of experts we can have in this industry this predetermined height that only so many people can get to and if one person gets up there another person can't join them this is something that i've seen out play again and again especially over twitter where people are berating each other for not knowing things for being less intelligent challenging each other on what they know and the issue is that we're not going to scale to meet the needs for cyber security

if we are alone in a tower and i think of our a lot of our anti-social behaviors come from this idea of scarcity come from this idea that we can't have too many people who are experts but the issue is we just can't scale solo we can't keep up it's not the time for towers it's not the time for creating your treasure piles of knowledge of opcodes and lording over them if we are going to scale this industry to meet the demands of an even more networked world we're going to need to drop the elite hacker stuff an interesting thing is social isolation and loneliness increase increases the odds of an early death by 25 to 30

percent in this same uk study they compared it to smoking 15 cigarettes a day fun fact another study found that an antidote to burnout is connection it's not leaning into isolation it's connecting with other people not responding to more bug bounty reports the thing is elitism doesn't make any of us better it is time to take all of the hacker tickets that you've earned at the chuck e cheese of hacker and turn it in for that stuffed toy or lava lamp at the desk greg because they're not worth anything here anymore also closer to home elitism is the enemy of diversity it's been shown when a group is in the minority it actually increases competition

and i wish i could say that this is something that hasn't affected me in my career but women right now we're only 24 of the cyber security workforce which is actually up from 19 in 2019 so holo to coronavirus for getting more ladies in this industry we could probably do a whole talk on that but i have firsthand experienced what it is like to be one of one or two women in a group of 30 to 60 men and feeling like we were duking it out for that novelty position of female engineer in fact my talk title was inspired by this incredible blog post by emily wenn who's a software engineer titled i need terrible female engineers

it summarizes that the competition and expectations for perfection for women who are in the minority is making the field intolerable so we add this on top of the already crushing pressure to be a hundred percent hacker all the time and we have a perfect recipe for everyone being the same coming back to our our tower and i want to go back to this ivory tower and talk about how disconnection makes us not see the forest through the trees and actually makes us miss pretty important things so again i'm at facebook it's around late 2015 and one of our threat intel folks comes to me and she says hey i think we should start looking at all

of these weird memes that these groups that we are attributing to russian actors are posting it's a lot of weird like controversy and like just lots of weird stuff we should look at this you know what i said to her is there malware she's like no no it's just posts and facebook groups and they're cropping up you know if it's not malware i don't think it's important because i was so obsessed that the only thing that mattered was the most technical thing which was reversing malware the funny thing besides the fact that she was talking about russian troll groups right before the 2016 election whoops uh it's also now that all of that hard one you know assembly and

malware reverse engineering skills that i learned now it's all about sas now we're all in the application layer and it's all cross site request forgery and i feel like a dinosaur so yeah let's talk about always being on and this thing where we just can't really disconnect from what we do and uh why can't we disconnect one is we love what we do i think for a large number of us this is also a passion this is also something that we are incredibly uh interested in but also we have to keep up with headlines things are changing all the time but like sitting here there probably was another you know confluence vulnerability or something happening

also security sorry [Laughter] security breaches could happen at any time in fact i bet there's one happening right now listen yesterday i took a day off and i'm looking at my head of security i did not take the day off i jumped into incident response see lest you think i am up here on my you know cloud of a perfect work life balance the other thing is we are rewarded for it when we catch those breaches when we answer the executive at 2am who has his hit twitter hacked because he's using the same password everywhere we get props for that and then it becomes expected oh you know security engineers are always working they're always on it's

fine you can ping them at 2am it's fine they love this this is the wheel of reactive hell we get stuck in this because there is always more work to do always more work to do we have to make peace with this we have to accept that there will always be another alert there will always be another cve there will always be something that's happening and that glorifying overworking or making it that it is the norm actually hurts us all it means we'll burn out faster and insidiously it means that we will block people out of this industry who have other things in their lives like children family or you know lesser things like mental

health happiness and hobbies so i challenge all of you when your partner or friend or parent or dog asks you when you're on your laptop at 7pm how much more work do you have to do you say you know it's okay i can stop and then maybe eat dinner because i know that's something i'm terrible at and especially for those of you early in your careers i encourage you to do this because these habits that you set up they will stick with you for the next decade

also i believe your best work happens when you have space to think when you're able to get off the factory floor and you're able to think about how the factory works when you can start applying your engineering brain to thinking about automations and how we can actually automate ourselves out of having to do that incident response out of having to triage those alerts you know that thing that happens when you take a vacation and you come back and there's that problem you are working on and actually i made a vacation of more than a week i don't mean that vacation i was like i took half a day off yeah you take a vacation more than a week you come back you sit

down and you're like oh yeah duh i could just do this this makes sense that is the magic of slack space in your brain our brains are incredible and they're at their most capable most creative when they're given the time to rest and digest and process what we're doing if we're just slurping down cyber security content reversing and coding we're likely to be caught up in that reactive wheel of hell susceptible to picking up the next piece of reactive work rather than knowing what makes the most impact i don't know who needs to hear this but this is something that i wish was said to me more often earlier in my career so i'm going to say it to you now

you can be a great security engineer and still have hobbies that have nothing to do with security nothing to do i'm not talking about loosely adjacent things i'm talking about nothing to do i will share mine i'm an aerial artist i hang upside down i think it helps all the instant response fall out of my brain excuse my language um i also am a devoted helicopter dog parent to these two uh these two muffins yeah that's what i do when i'm not doing security i also encourage you to talk to each other about what you do whether you like baking bread or you like hip-hop or you're a rapper or a dj talk to each other about what you do

that isn't security make it the norm that you can do other things and lastly my most important topic is this talent gap and how we're going to bridge it but actually i think there's someone who's better to talk about this first so i'm going to queue up steven here who has a message for all of us

how do we oh no av how do i make sound happen

can i have some support making sound happen

hey do you want to make a difference fighting bad guys and making six figures well do we have the job for you i got so excited

hey do you want to make a difference fighting bad guys and making six figures well do we have the job for you you don't have to be iron man you can work in cyber security hell yeah the us government estimates a labor shortage of about 400 000 people over the next few years in this industry here are the three simple things that you need to fill these jobs one need to have 10 years of related job experience two need to have multiple accredited certifications and three most importantly you gotta be a rock star does this sound like you no no that doesn't sound like me at all awesome come on down and apply at entry level

cybersecurityjobs.com [Music] we're waiting it's time to sell drugs thank you i highly recommend yeah give it up for steven yeah [Music] i highly recommend checking out his tick tock as zepho says he also does stand up community around the bay incredibly funny person so let's get back into it why can't we hire could it be that we're looking for unicorns okay i'm gonna say some things and all right why can't we hire we have this prerequisite that everyone needs to come to us already with an engineering degree in order to be valuable second when we actually interview people we don't test for the right things so degrees are a privilege it remains that a four-year university

or college is still something that while it's becoming more accessible is not something that is available to the wide socio-economic uh part of our of our country and our world it is still a marker of privilege to be able to attend a college right after school we still apply this idea that we live in a meritocracy and naturally those who are gifted or intelligent naturally will get siphoned into university or college and will get that engineering degree before they get to us and i'm sorry but this just isn't true and it keeps us kind of in this heterogeneous collection that we're currently in i heard someone clapping you can clap for me i like validation all right

so this is from an aspen institute study about diversity in the cyber security industry so it says while it's been noted that academic degrees do not necessarily imply a more advanced level of skill it has typically been considered a hiring prerequisite for most employees so we have little evidence that getting to us with a degree makes you better at the job but wait before we go too far lest you think i am railing on the education as a whole i think education is invaluable i consider myself a forever learner i love learning new things and i think the the ability to get educated and to go to college and to go get certificates and go to boot camps

is invaluable and necessary i think learning the principles of engineering unlocks in your brains the ability to automate all of this manual trash that we have to do right now i believe that it's the future of our industry i just don't think it has to happen before you get into the industry and i believe that availability to education should be a job benefit for any role in cyber security [Applause] right now you're all like oh clap for her she likes it okay um but let me take a step back and talk about how we have this area between people who don't have degrees and lots of job openings and there have been a there's been an

industry that has grown to fill this boot camps and certificates these courses which are usually very expensive sometimes promise people that if they go through them they will get a job which is misleading at best the truth is in this moment in our industry we do not have any standardized way to think about certificates or boot camps as a sign of proficiency this is a problem here's another quote from that aspen institute study there is little evidence about whether these programs have been effective in closing gap and increasing non-traditional candidates in cyber security it still remains that a computer science or an engineering degree is the universally agreed upon prerequisite and again my belief is that if we offer

education as a benefit not as a prerequisite we can start to fill that talent gap all right secondly let's say you actually get your resume into a recruiter and they're looking at it great you get scheduled for that on-site he gets your on-site and oh a lot of algorithms questions i mean it is just the normal cocktail joke that you have with people when you're like oh yeah you interviewed at that fang company did you have to balance a binary tree or red black no this is not how we should evaluate people for cyber security rules in fact this is someone who's a software engineer and he's making fun of the interview process the fact is we just don't know how to

evaluate talent and i wish i had the best answers for that but i don't i think it's something that we need to think about but you know technical rigor security is hard i'm talking about hiring people with different backgrounds and heaven forbid no degree or a degree in an unrelated field won't we fail security engineering is difficult it takes hard work and i agree i think security is a particular field that requires curiosity tenacity and a lot of stubbornness you have to love this you have to put in the work but you should be able to put in that work and get return on your investment but wait we have 600 000 open rules we don't have

time to train people we're underwater already that's the wheel of reactive health speaking people what do we need to do we need to leverage these boot camps these trainings these conferences these degree programs as expected parts of the job we need to hire people who maybe don't have that computer science background and send them to college while they're with us i got my master's degree nights and weekends while i was working for the government in fact the government paid for it in return for me working for them that kind of system works also we talk about this pipeline problem and the pipeline problem is this idea that people just aren't qualified when they get to us

and we talk about it as if it's something that will sort itself out or it's not our responsibility you know it's the responsibility of the colleges they need to teach more computer science it's responsibility of the high schools it's the responsibility of the middle schools the elementary schools the preschools it's lego's responsibility it's not ours the issue is that we're all going to burn out before this pipeline problem fixes itself it's all of our problem and so lastly just want to go through again what it means to dismantle the unicorn so first seeing each other as allies everyone in this room you're great you're my ally rather than competitors we don't have to quiz each other on who knows more

uh you know crypto algorithms or have you read their most recent headline we don't have to poke holes in what other people tell us they do to see if they actually belong to sit at the table we need to dismantle this glorification of overwork we need to tackle it and think more about how we can both live lives and be security practitioners and we finally we need to change our perception of who is worth hiring in the industry and so i want to leave you with a challenge just like in the beginning i talked about that word mediocre and how negative it is but just how benign the definition is it's okay to be a beginner

it's okay to not know everything it's okay to be curious and i want you to challenge who you think belongs in this industry and i think this is the answer for a more diverse and sustainable industry so let's lower the gates let's bring more people in and yeah let's make this industry what it can be yeah we need all of you and for everyone in this room who maybe took a seat and looked around and was like i don't see many people who look like me or you know i just don't know if i'm cut out for this you are you definitely are and if you need to be reminded this is my twitter and sometimes on it

but you can find me on other platforms and i will tell you you belong here you're great okay thank you

[Applause] still i'm still standing up here should i just keep talking who questions does anyone have questions do we have time for questions what are my dog's names oh yes okay so i have a two and a half year old pomeranian named luna who is chaos incarnate and i have a almost 16 year old terrier mix who is ornery and curmudgeonly i love that that's that's the question where do we oh that is such a good question where do i think mentoring fits into all of this i would not be where i am without the mentors who believed in me especially when i had no belief in myself like when i was a grad student

and i was sitting at the csoc at mitre trying to decode a pcap and i just had no idea what i was doing my mentors believed in me and [Music] i think yeah mentorship is one of the biggest gifts you can give someone

yes oh do i have advice for people who want to be better mentors just do it just try like yeah be okay with failing be okay with not knowing everything just sit with someone doing ride-alongs is incredibly valuable especially like as a blue teamer what you do is more of an art than a science so having people watch what you do and learn from you is mentorship in itself don't overthink it

can i can i take more questions i feel like yeah okay they're nodding yeah yes rather than we will hire a security engineer we will hire somebody with a clue who will just start throwing first throwing stuff at say five people i love that like a cohort so the question is like what about a program where instead of just hiring one security engineer you hire five people and just kind of throw them into the the fire and yeah no i love that idea i think that is kind of like what we what we do with internships except our internship programs usually only uh accept people from software engineering backgrounds so i love the idea of like having some kind of

you know little test or people can do on their own time like you know can you can you use google which is probably one of the most important things to being a security engineer if they can you know pass like a little uh like a little mini capture the flag bring them on totally i think that's great

okay thank you [Applause]