Doug Leece: Detection Engineering

[Music]

[Music] this is our second home for b-sides so i i want to have a big shout out to the folks at bow valley as well they've been measurable you can't see them because they've got a set of screens behind it but there's actually actual people sitting there just like at a telethon watching us and making sure we're looked after when something gets wrong you know we've got people stepping up and you know somebody can't make it all come in all it is we have people moderating channels now it's been all we could hope for i think from the b sites and i think calgary should be very proud of what they've done so i originally had a completely different

talk planned and i have some research in the works and that is not this talk uh we've got some gears going on this but really i'm going to talk about pretty much what the hell i've been talking about for the last 10 years but it's now got a brand new name i feel vindicated so as a derby a couple of years ago but before we get into that let's uh just you gotta have the obligatory about me and mark put in there grumpy doug i am indeed a self-proclaimed grumpy security guy and i am the original info cynic professional but there are some of you who are running around probably not wearing them but have

infocentic professional t-shirts as well that's again another inside joke but it's more about the longer you're in this the more you the more you know people need our help and the more frustrated you are with their unwillingness to accept it that's the story behind that so um yeah mr millman advised me earlier i used to buy electronics or i was like yeah i forgot about that whole part of my life so i've been doing this a long time messing around with electricity because it all led to being a rock and roll star at one point so you know that was kind of my thing it's going to be a rock star i'm still waiting i've sold dozens of copies of my

records online so it's a good thing i got a day job but yeah i've been at this a long time electronics then got into telecom into it and then into cyber so i'm not sure if it's a requirement for them to have guitars but if those guitars need a new home i'm sure i can fit a couple more in because i lent a couple out but oh yeah i also co-host a podcast with uh my friend tim mccreet and it's on risk we don't hack nothing it's all about risk management because way back when i went to school business school first because i figured if you're going to be in the music business you should know

something about business and i heard mick jagger went to the london school of economics so that was enough for me so um and no hacking i'm out yep sorry man this is about running your business and running a security program but it definitely involves hacking trust me otherwise you don't know your stuff works so i'm getting to that and i'll speed it up i mentioned mark shakulas shout out to him as well he's one of the other gang of five that started b-side to say gang because there's lots more but five of us wrote our names on the paper um so yeah he's uh one of them and uh yeah so it's great to see everybody here

you can find me on twitter but i tweet nothing but you can see some of the people i follow which might be interesting to you so that's about the only reason to follow me sometimes i post in linkedin usually about hey there's another b-sides and you can always find me at gmail and that's it you know i'll usually answer usually not quickly but there we are so i've got that whole boring part out of the way but it's been great to connect with everybody and for those that i've never met in person and you see me joking with everybody that's pretty much how i am all the time so maybe we'll hang out it'll be great all

right so what is detection engineering i went to derby about derby con sorry uh second last one i think and it was all on purple team but they weren't really calling it purple team anymore they were calling it detection engineering and i thought this sounds right to me this is what i've been talking about for a long time and i'll show you what i mean as we got through it but what is detection engineering so we were joking about it earlier this morning and i could see a fellow mcs ent4 on the call you know there was a big kerfuffle about that because it was a microsoft certified systems engineer and it's like they're not an engineer i

don't know it's like but you do engineering things so what is detection engineering well it sounds to me like it's something to do with building or thinking about detections and yeah so we're really going to cut into this deep because there's a lot in the words i dug around and literally these were done an hour ago so this is not off the top of my head i've been thinking about this stuff like i said for a decade or more so this is just sort of amalgamating and trying to squish it in 30 or 40 minutes for you and hopefully keep it light because it's been a you know two long days your brain's sore trust me there's no source

code analysis no squinty print all good so detection engineering identifying the threats before they can do significant damage now just let that sink in and uh let's go there and i did not do all the work for the ctf mark was the guy that put most of the flags in for the first couple years so i miss him dearly all right so detection engineering nice short it's great elevator pitch but and trust me i lived this for personally and i was astounded at how they couldn't see the value and what i was proposing but i'm used to disappointment i've got two ex-wives and four children so we have uh one of the thought that calls he's

rather go oh my god that guy anyway um when you try and sell this idea you're gonna hear stuff like well we already have we've got arcsight we've got qradar we've got nitro we've got sentinel we've got azure we've we got equipment i got cisco i got narctel it doesn't matter what the equipment is everybody's going to have to have equipment that's good you have equipment check you know i'm not sure if that's supposed to be a check box but i would uncheck it if it was so we have the equipment yeah so we don't need a security we don't need what you're talking about and then i would have nothing to sell actually ken that is not true you could

sell detection engineering services and i happen to know you have a rock star in your stable so trust me you're never going to be out of a job if you've got people like that now we don't have we've outsourced the security to vendor x so we don't have that risk anymore and when i heard that coming from what i would call a seasoned security professional and again i want to be a full disclaimer i've worked for some of you on this call i've worked at places with you guys these opinions are my own and my own alone i'm not representing securednet and enbridge suncor o valley college b-sides proper if you hate what you're hearing here

sign up for caffeinated risk and send us hate mail like uh this is what i this this you're just getting straight ahead doug not any corporate spiel here uh which is you know probably explains my career trajectory anyway that's uh and yeah adam you you covered that off brilliantly assumed risk transfer to third party is trans unfiltered it's the best one yeah it's like that broke a long time ago do you know what after you cross that 50 line oh man your attention to even concern about that filter it's just like out the window good enough so without source security we don't have that risk anymore you could think that again what about if and let's insert a scenario what about

if that third-party underwriter doesn't pay your claim now what are you to do because that's never happened right oh that looks like an act of war oh because russia came after me it's an act of war right well if that's the case then wouldn't that mean that i could call up my friend justin and say hey send your troops over to deal with that guy yeah so it doesn't you know just no it doesn't work like that so there's this there's this problem with that a whole equation and there's a couple really good books on that one of them i'm going to recommend for you two of them actually if you really want to get your head

inside of that problem with the military but trust me that cyber insurance thing not that it's a scam but what if it doesn't pay out what if the third party that you have doing your security actually goes out of business uh oh now what are you gonna do so you're not outsourcing the risk you've made it somebody else's to-do list but you're still the holder all right as long as we're compliant we're okay yeah and so i'm not sure that i would uh okay i cannot watch this this is too good all right i'm holding my hand in front of you i'll just block out the tweets okay there we go now you have my

undivided attention and i'll read them all after as long as we're compliant we're okay no let me put it this way if you secure things even if you're not compliant you can probably have a pretty good reason why you haven't met that yet and why you might deserve either an exception or an extension or something like that but if you're still holding up your paper tiger hoping that's the that's all that's standing between you and you know economic ruin i think you missed it okay and we had a pen test last year we did okay well i don't know who did your pen test was it me well or was it you know somebody really good

okay and what did you tell me i could look at because here's the problem with a pen test the minute that you say go in and pen test our stuff and no offense can but the salesmen get involved and the project manager get involved and all that kind of stuff and i've run my own business and you can't just let guys like me go and do everything you'll never make a freaking diamond they'll never finish the job so pms and sales are important but that means your scope is whatever you've decided your scope is and that may be fine but if your real problems over here it doesn't matter okay so i guess i do have to move this so you

can see my clipper thing all right so then i get this other argument well where's your ring so i am not made saying for a second that a detection engineer is equivalent to a professional engineer at all that's not what i'm saying here and this is where i think microsoft and the mcse had a valid point when they said you're a microsoft certified systems engineer you're doing systems engineering we don't build bridges because oh yeah could you imagine your average id department building a bridge or a house oh my goodness i wouldn't go in it but that's how it goes yeah red hat certified engineer yeah that makes sense tim so yeah so is that a misuse of the term no i i

looked up the term because dictionaries are great argument winners miriam webster they seem to know a few things about dictionary if that's a skill i don't know if it is or not activates or functions they're doing the activities or the functions of an engineer and that's it so yes if you're doing activities and functions it's just like i'm not really an engineer but i play one on tv that's okay you're still doing the same thing and i was talking to somebody who's pursuing getting their png based on something to do with that very concept that they've been doing engineering work for two decades and they are very good at it they've just never gone through

the formal training and it's a depending on who supports you on that you can actually get certified with a paga on that i'm not we're not going down that rabbit hole but i'm just saying you're doing engineering things so when we're doing detection engineering we're taking an engineering approach to building detections when i'm talking about detections i am specifically referring to the ability to identify that something is occurring not protection although a protection mechanism may be put in and say i will take the data from said protection mechanism to create a detection but it's actually after the stuff's seeing what are you going to do with it all right and i also want to highlight

two more things it's useful to people because we've all had sims full of data that we couldn't even there's so much of it you couldn't index it you know you press a button and it comes back an hour later well that's going to help when we're already in and out the door and we're implying that these are complex things we're not teaching a 14 year old kid how to make fries at mcdonald's these are complicated problems so that's i don't have a ring but i do this kind of work so am i engineering i think we are you know happy to debate that and i don't have a ring i'm not saying i am an

engineer i do detection engineering work all right so it turns out that i've tried to you know like the cheesy meme of one simply does not so one simply can engineer i should make my own yeah yeah be made out of a guitar string or something so uh this seems like a pretty reputable source engineergirl.org some of the best engineers i've ever met are women so i'm taking this as good in fact just a shout out and this was completely unplanned and spontaneous the two founders of our ctf company are u of a engineering alumni like how cool is that eh so way to go ladies so yeah engineering.org what are engineering principles so these are things like

the ideas and the concepts that they take to a problem because every problem is going to be unique if i and mark and i worked together a long time in a couple of different outfits and one of them was a consulting situation and you go work with a customer in a consulting engagement and you do what you can for them and then somebody like ken puts you on the next job and i'm with a whole new group of people a whole new group of problems my mindset doesn't change in the fact that i'm still thinking in certain ways but what's going to work for this company isn't going to work for this company over here so again we're

engineering solutions that fit this company not it says on my checklist that you need things i got nothing against checklist but with my coach my team now i say you know checklist discourage thinking and when you're launching a rocket ship that's a great time for a checklist because they're the time for thinking is over is it on is it not but when you're just going through and designing a security program saying do i have these things on a checklist i think we're doing a disservice to the program and to the company we're serving so we need to do that in a more meaningful way so back to that first one is it useful to people

you put in this thing and it's on and it meets all the criteria that we said we'd meet but i'm not seeing how this helps me with this other problem yeah it doesn't do that yeah that's an awkward conversation i've had a couple of times so let's not avoid that so yeah right on any time let's spread that around there's a few of my quotes like i never heard that so this is another one that i'll drop a little later so what problems are we actually solving and that's a trickier question and usually we're asking the wrong people on that and i'll get into that what do we have to work with so remember i said we're doing detection engineering

so that implies that we've we've made it here after something's already been put in so trust me i was certified for a long time on checkpoint netscreen which later became juniper i've got elapsed ccna all the ctf stuff's running pf sense yeah i think we're close enough that that's not flag anymore all right um i don't care what firewall you use i wrote my first firewall using ip tables and a bash script and curtis rempel telling me the things that i'd screwed up now there's a blast from the past they work so you know firewall is a firewall to me it's blocking the traffic i don't want it's allowing the stuff that i do yes there's

some cool stuff you can do 90 of the time nobody ever turns that stuff on so what problem am i actually solving don't ask the text let's move it up a notch what do i have to work with oh we have some firewalls good what brand i'll take those my ferrules are still iptables yeah it works fine that's what's going on under psns under the covers and uh when you rip open a lot of these heavy duty ones they're running vsd under the covers that whole nokia checkpoint thing that was all just bsd boxes with no journal file system but that's another separate your story all right now the other thing this is why venerable grey geniuses are valuable to

your organization and i'm not just saying that because i am one uh the gray part i'm not sure if i'm genius and it's certainly not verbal but um the great part for sure what went well what went badly okay because these are things that we bring oh my god yes i can't look let's not get started on krempel and his crazy vacs experiments but those are great what went well what went badly because we've learned we are i.t professionals in this business you've learned you've seen other organizations even in your own organization if you've been the same company all along think about what worked what didn't those are all things that we can put into this thing of what

are we solving for an engineering what problem are we solving what do we have to work with if you don't have any budget it's no good saying well you're going to need the next greatest thing in the world right so i did find the one cheesy obligatory meme that we've all been here and i love this one probably because it takes me back to when i was on the other side there's what the engineers do this would have been me and shakula putting it together oh my god this looks like webb cbs to me i seriously think so okay everybody moving on there we are yep yeah and how the customer was built yeah so but what this means is

we miss this one at the end okay and in our case as the detection engineers as the security professionals the senior security professionals in the organization and the juniors don't don't worry i'm talking to you too i'm not talking down it's like watch for this and maybe you guys can help move this and there's a piece for us on the end of that too okay but this is really all they wanted and this is what yes and of course i've seen the updated version with the roller coaster and how they were built oh my god yeah we just bought disneyland it sure doesn't feel like it still looks like a bumper car so this is my favorite

because i work in process control uh cyber security and stuff excuse me um it's it's different a little bit and i'm going to get into that a little more here in a sec but i'll often hear engineering engineering or it's something to do with physical fenestration or some other even if it's process management and stuff you go yeah that's kind of it's got issues oh but this is cyber so it's got to be different so i'm working now with an ex-marine and it's been great i gotta say that so it's it's on tape now so if you can send stephen the link um it's been great to have what i would almost call a peer to work with finally

where he's kind of like i don't care it's all security to me and of course being a marine they're totally mission and this i was trying to say this 10 years ago already our job is not to answer the phone while on camera here we are so i know one person that isn't watching right now then because they just phoned me either she's funny saying my fly's undone but you can't see it first that's all good uh the um cyber securities the securities mission is to ensure the organization's mission succeeds okay that's it we're not supposed to be all of those other things we wind up being that through sometimes our own willingness to be nice because you know we're we're the

nice computer guys we're here to help and we are here to help dammit like we really are i wouldn't be doing this this law for this much frustration in my life if i wasn't thinking that it was going to make a difference somehow at some point so uh here's where we are so you know we get that the organization's mission is just whatever they do that could be bacon cookies that could be pushing oil down a pipeline that could be making electricity go down it could be making planes come and go it could be getting people on and off the planes and taking money for the plane fare there's a mission there's a reason you're doing

what you're doing and the only thing that's different with cyber is we've added a microprocessor and data and sometimes we've got connectivity and i say sometimes because my brother

i'm shutting you guys off again i'll undo the paper when i gotta switch slides all right so my brother just changed jobs but he was working up until recently at a fish plant and they have a fish cutter this thing costs three million dollars and it's complicated as all heck it's made in iceland at all places i guess they know a lot about frozen fish there too and because he's a pretty sharp fella himself and he's well into the computer world even though he's you know got a whole different career in front of him he was the guy he became the computer guy for this really fancy fish cutter and if i walk up to said fancy fish

cutter with uh rubber ducky that's the thing that's the one right the the hack five one with the usb anyway some usb with the shell and out it goes and in this case there's nowhere to go because it's air gapped they don't need real-time stats on their fish cutter they look at the fish coming off the end of the line you're still out of business it's not working they got to fly the guys back from iceland but in this town this is the only thing going on so the minute that fish plant is not operational everybody's temporarily laid off till the fish cutter works again like it's that bad so you know cyber effects can be shown

anywhere so it's not compliance it's not the eps but i'm here to tell you it ain't easy it's seriously not easy if anybody tells you this is easy and i got this figured out you should hire the next guy because that is not not how it works at all it is hard okay so what are we going to do to change this because i really want us to move into we we're all familiar with the problem space but i think we it is the rubber ducky thanks kelly we got to do some science maybe we're making meth maybe we're not but we got to do some science but here's what scientists do even when they're making meth

like that's a bizarre show of ever saw but see how this punk kid turns into a pretty rigorous guy and he got specific about cleaning and everything because you know the the protagonist was so meticulous about stuff but he was a chemist he was a world-class chemist and we're how are there are we world-class practitioners do we even know how well our firewall works we're often surprised when we thought that was cool that's a lot we check we see if there's data coming through and that's kind of the end of it and you know was it newton's third law of thermodynamics everything tends towards chaos we almost never go back to see if it's still working as well as the day we

put it in you know and this is my favorite [Laughter] this is my favorite we almost never think that it's something else so i was working with some guys a while back and they only come up with the craziest stuff so god bless them you know alternative hypotheses were explored regularly and my running statement was when you're hearing hoof beats you should be thinking about horses not zebras and i stole that from chris saunders but that doesn't mean we shouldn't consider what if it's zebras because sometimes what if you're in africa well then it might just be a zebra because that makes sense the context is there but when we're looking at this what i'm saying

testing for alternate hypotheses yep see it's going off is that alert going off because that is the normal login activity for this whiz-bang third-party virtual agent that you've got on there that does all kinds of stuff to get you lined up on the network i swear to god i was looking at this and i could have i thought for sure we were popped you know i'm watching this thing come in and it's run get this run in powershell who am i what's my ip address who's the members of the domain administrators who's in the group we go only red teamers run that only red teamers and the vpn client that we have like i got nothing right it turned out to be

this is completely normal because the next thing you got to do that alternative hypothesis so how many other people are checking for domain admins oh everybody okay so though it's no longer and this is what i'm getting when it becomes such a common event that would be bad somewhere else but it's completely normal in your environment that's that alternative hypothesis that it's actually just the regular horses it's not the zebra so you know you gotta you gotta think that through a whole different way so we gotta do the science now but is it a computer just a big calculator because when you look it up these computers have been going back a long time and they were just fancy

mechanical ways of doing math this goes back to the 1800s already and yeah maybe in 1964 that was true that it was a calculator but big inexpensive but now they're attached to kinetic devices and i'm not normally a huge bruce schneier fan as and you should read everything he does you know but click here to kill everything or click here to kill everyone highly recommend that because he points out this breakdown now of how these computers so this is a radio and a computer strapped together nothing bad could possibly happen there with mobile microprocessing and data that's an issue okay and we've got the obviously in my world you know we're attaching these to heavy industrial things

my good friend and co-host works at a place where they have very very very large vehicles attached to computers to do stuff you know computers are attached to everything kinetic and when that goes wrong something's there so another key saying you know hardware eventually fails and software eventually works the firmware is just another fast version of software that's running in there somebody programmed it stuff's going to get out of band it's going to break and they're attached to kinetic devices there's going to be problems most of our businesses now wouldn't run without computers and you know some of us might remember when you used to take data off a piece of paper and write it

down and you put it in a folder and put it in an envelope you mail it to somebody you write out checks and you'd send them and the money would come out of the bank and the banks closed at three in the afternoon except on fridays like our business world our life has changed and everything's down to a web page and clicking a couple of buttons and that's the whole business so it's going to be very difficult for us to tell as a human looking at a canvas all day is there something funny about this transaction or is this legit you know when somebody shows up but it's it's different when it's in the physical world we can we have a lot more

senses to work with we're really away so it's not just the computer anymore it's the way we run our business it's the way we run our lives in most cases and when this stuff gets loose you know the the personal pictures of people and the texts that shouldn't have been leaked and all that you know it just ruins people's lives there's just a lot of bad things that can happen with this and we're not the genies out of the bottle and long gone so we're done we can't fix it so we got to fix the problems with the computer security so we need a plan so when we go back to our very first definition of detection engineering it

said before they can do significant damage and prashant nailed it earlier on he said resilience for the whole nine yards right yeah adam just said yeah we're waiting on implants because what could possibly go wrong with that you know but there's you know a number of talks on hacking medical devices and stuff so let's let's not explore that i want to explore how we would do detection engineering first thing is we got to do it to get to the significant damage so so if i'm going back to what the customer originally wanted here the only significant damage i can see here in the final solution is if this tire itself breaks and you fall down and hurt yourself or the rope

breaks and again that doesn't look very far should probably what are you gonna do you know a couple days off and i don't know lightning strikes the tree and electrocutes you you know we're getting pretty far i don't see a lot of significant damage there but even then what i did right there is i was doing threat modeling so what you need to do to figure out significant damage is you need to look at what is it going to take for that business or that mission to function and function is a big thing there's probably three buckets you want to put that in there's a tomorrow function like usually people can survive a day outage but you know what do they do the

next day so that's one so there's that immediate resilience capacity you gotta have and then i need to be ready in a month to be able to what run payroll again run versus sure okay but i got a couple of weeks to solve that okay i need to be meeting x number of commitments over the year those are all different types of obligations that we're going to have to have so okay i love it so there we are we have lots of gaps lots of breaks but if we work with our organizations we can start to figure out how to mitigate the significant damage we're not saying we're going to the significant stuff the stuff that's

going to take you out so i'm doing this for my friends at enbridge and i'm shouting out to guys that have helped me on this rod beaudry steve griffin steve hengo nicole kill guys are all real important to this getting this message across so collecting hope is not a strategy i stole this from steve williams they were taking over canadian pipelines that hope is not a strategy well collecting hope is not a strategy in the log monitoring business just get everything in there so i did the threat model on the tree and i came up with what lightning could land and it could take you out that's kind of a thing pretty far-fetched but possible

you could break the rope and it's like so we did that real quick now obviously your business process your mission the things that are going to keep you running is going to be a lot more complicated than that tire swing i sure as heck hope so or how are you making money at that can i do that because i could use the rest then you got to look at the sources that you're getting the data it's no good saying to your boss i will start telling you how good or secure you are once you get me this that and the other thing you got to work with what you have what if there's no money i've worked in

places where there was no money you're lucky you're getting paid and we you know there's open source software there's somebody's somebody else's computer there's a lot a lot going on and once you start getting the risk the threat model figured out and you figure out your sources now you're going to build out some prototypes you're going to test them and i'm suggesting you even use ctfs i'm not kidding then you've got to go fix the things that aren't working how many of us have tried to take in logs from source only to find out it's not being parsed properly it's not even being sent or all of the logs you've got coming in are there but they're all from the same

source which is the load balancer you put in front of it to make sure that nothing came through we forgot to click that other button so you know there's a lot of things that you're going to go through so it's not how it's going to work so you've got this piece here so i see a lot of talk about risk and now we're going to i just tune in you'll hear us talk about it all the time and i'm not pushing the show i'm saying this will get you started on thinking about this risk is not what you think it is for most of us and here we're talking about resilience rather than perfection can we survive

the next day and we've got to remember that every dollar that we're taking out of the organization for the next whiz-bang security thing is money that's not going on something else that may need it as well so how do i make this happen i gotta start a cultural change somehow my favorite is tim mccreed's risk acceptance memos the minute you get somebody to sign to say i already know that there's a thing so i want to go back so this was the one piece i missed and this is the risk when you're doing your detections and they come through and they find stuff there's going to be things that it doesn't catch you have to make a decision at some

point about whether that's good enough but it's not the detection engineer's decision it is the owner of the risk the person who's actually responsible for that part of the mission being completed that's the person that has to say yeah i am okay living with that because i'm not spending 200 grand to fix it i can solve it for ten thousand dollars is what good enough they did their acceptance so the final thing and they're giving me the hook but i'll go over and probably won't cut me off but it's gonna be short i promise this is where we come in as b-sides as the community all right we need to get our workforce ready to deal with what's

coming at us and i'm not shaming anybody there were people hacking away at this thing what two o'clock in the morning i heard i finally packed it in about 11 30. it's like yeah they're still there and i can tell from the launch people doing stuff but look at all this red compared to the green those are the solves they missed versus the ones they got so if they are nation state russian hackers that get it right on the very first try they're not they're masquerading themselves very well here they're hidden amongst this but you'll notice what we've done we've broken it down and i'm not spying on anybody and it's all anonymous don't get worried about this

what kind of steps along the attack cycle and so there were all these different flags that were built in you know where you are in the attack cycle and we can start to see sort of how it is and you'll notice that recon is one of those things purple where you got to spend a lot of time and you do for sure they tend to spend a lot of time lining up the shot when they're taking it so we got to get them ready now how do we get people ready please not another fishing training please i've seen them they're no good so but i don't have any money so we don't have money for a full-blown cyber

army to compete but what if we had a militia instead what if we had our whole organization trained to work like a militia okay if there is a fight they know what to do when they got a gun behind the door and out they go you know so they murdered a guy in aj's tabletop i gotta ask later okay so you think militia the first way you get through this is you encourage awareness through empowerment and i worked for a fellow and he put together this brilliant campaign called trust your gut and i was so impressed by that because it was saying to people you're smart you're mature you know when something's not right so trust it

instead of panicking going oh it's computers i don't know try to bring them up so we ran hacking 101 three times courtesy of bow value whole valley they did a wonderful job again shout out to zinji and james to set that up and refund it it's not that expensive but it took us a little while to get her done and you know we had 20 desktops and people are hacking away into running end map and they're learning stuff for the first time we have people that have never done these things and they're there and it comes back to some old saying i'm not sure where i learned it but if they knew better they'd do better

if we teach people how these systems work even at the most basic level this is how they find you on oh well maybe i should not put that on social media then let me just take a look you know there we are okay we're raising that awareness but now it's a tangible thing because they actually did this i'm thinking of a hacking 201 where we teach people how to spoof email yeah i think that'd be kind of cool somebody mentioned the table tops yesterday uh when i first heard about this over lunch i said you know i think that's brilliant because i've sat through enough table tops that i never want to do another one but i love

the way you're going and like i said this sense of humor has been keeping me entertained for 30 years plus so i'm all good but something that engages the people and i heard i just saw in the chat i don't know the details but somebody got murdered yesterday in a cyber event in a tabletop that's going to keep your interest it's like breaking bad for real so yeah i have no idea this is not an endorsement applied or nothing check it out though it sounds really cool i've heard nothing but positive things about this gamification of the security process and finally the cyber range of you know what we've done here we've built this to guard what we are

trying to protect so yeah we picked on you know critical infrastructure oil and gas that sort of thing so believe it or not nobody got there again this year so we'll probably retire but you know there's an actual modbus stream controlling the blinking lights over there and so we want to watch those sorts of things um what is the cyber range make sense for you what we're doing here doesn't make any sense for banking so you probably need a different kind of range but where you can extend that further is if you're building out a system that's for your environment you should build out another one and turn it loose throw a couple cali boxes in the network there try some

stuff see what can be broken in see how it responds so prashant made a quote about the fainting goats and plc's a little earlier i don't know what happens when you run nmap on 11 against that thing does it actually fall over well did you even test it no we just think we're scared it's like we'll test it maybe it doesn't matter not as tough as nails oh you know we found out believe it or not isa has a whole program now where testing the crap out of the front end of these things is exactly what you do before you get it stamped so there you are so i went a little over time not too bad by

my standards but you know i was a month late for my first birthday so that's okay but um i will answer one question because he and i go way back do you find that there is still a gap where a detection engineer is great at the core but there is a disconnect at the higher stack with the application 100 so back to the ball or the tree swing yeah they don't even know what they're building they're really good at the nuts and bolts but they don't understand i would rather go the other way personally i'd rather take some people if they want to get into cyber like what are you doing oh i run the

databases i used to be an accountant and all that let's grab some of that not everything it's not going to work for everybody but those that said yeah i want to do this pull them in so they're maybe they're way one step above militia and i heard that cisco ran something like this for a while and it was successful for them that they would take people from different departments and they would swap security people out for those and it was a twofold one it got the business i i hate that term but i'll use it in this case whatever else is going on in the rest of the world they've brought them into what goes on

inside of security but to mark's point it brought to security people that are wicked with packets and you know fuzzing and protocols and this and that and they know how to do stuff and they can do it at two o'clock in the morning in a data center put them out where they're actually having to listen to a customer flipping out because nothing got delivered well now we're getting some synergy and we're training that getting that this is part of your mission understanding into the people that are delivering the technology that's making the mission happen and i think collectively then we're going to be able to design better detections but the point is you really need to start designing them

you can't just click a couple buttons and hope you actually need to look at what you'd need to know if it's going to work if it's under attack or if it's safe how would i know that well i tested it i attacked it and this is what it looks like and when i don't attack it it looks like this okay and final one which word help analog security cannot recommend that book enough because he talks about getting faster once that alert comes in you've really got to work with your team so that they know what to do when the fire department shows up nobody's looking at a manual to figure out how to hook up the hose to

the truck they know what to do that's where your incident responders need to be as well at that level and the only way that's going to happen is if they're trained for very specific scenarios and they've got a step not a step-by-step checklist thing but this way the decision criteria they understand how to move and how to get through so hopefully that helped out uh it's been a great conference i'm really happy to see everybody here and hanging in there i know we got prizes to give away and all that good stuff but let's hang on let's go do some good things and let's go catch some bad guys all right and if you are a bad guy

hopefully you learned how not to get caught this weekend talk to you guys soon eh bye bye