Security Automation Steam Engine Time!

[Music] hey everybody josh marpett here uh we're going to be talking about security automation today and uh it's a fascinating topic and it's one i've recently gotten involved in pretty significantly recently being the last i don't know 10 years so uh let's start off okay good uh sorry discord said your stream is still running we just paused this preview i'm like ah what does that mean hang on i gotta read it you know it's like a user you gotta read an error message occasionally what an idea all right so this is me uh i'm the co-chief vision officer of red lion uh i'm a digital forensics guy i've been around the block i'm not gonna read this

to you dear god uh besides delaware is uh my conference along with janice paulson uh and i'm also on the board of b-sides dc i've been on the board of b-sides las vegas uh hackers for charity i'm an ex-cop ex-fireman x-horse dentist no i'm not joking and yes my business partner scott lyons will tell you about it in detail if you ask especially if you buy him a beer been around the block more than once patent-pending on a blockchain storing forensic files patent and like i said just just about done everything in the in the history of what you can do in this world although i've never made astronaut so i've been firemen cop but not

astronaut one of these days nasa better watch out okay all right so today we're going to be talking about security automation we're going to be talking about definition history types successes failures the psychology of it uh budgets uh does it really help what's what is the steam engine time josh talks about and then we're going to do some wild speculation and hand-wavy stuff okay so we'll we'll talk about that and we might even get to what the next lecture is going to be on although since i don't think i added it to the slides maybe not we'll see we're doing this sort of on the fly just to be clear if i told you that i

finished my my slides more than 10 minutes ahead of this talk i would be lying so it should be interesting let's let's party all right so what is automation in the first place like where does it come from automation of course is how to automate a process and we're not talking about a single task unless that task is also a process but how do you automate a process uh using electronic devices software code something whether it's a robotic arm run by a raspberry pi uh welding a car together that's an automation of a process using a piece of hardware whether you're automating by writing a script for four hours to take care of a 30 second job

never ever get that return on investment but hey you automated it okay uh as every good sis had been knows if you have to do it three times it's time to automate it right so that's the the idea of automation we're taking some task on process and we're turning it over to a mechanical or electronic or software-based solution i think we've defined that pretty well now actually gartner goes on to define it even more they have something called soar you might have heard of this security orchestration automation and response these are ideas that they're using primarily for socks security operation centers they're saying when you pull in the alerts from your seam your security incident event management system

when you pull in all the logs when you pull in all the data when you automate the ingestion of all that data collection and ingestion i should say and you take all those inputs and you process them and then you output alerts notifications dashboards reports whatever your your visualization of choice is you're automating that or you can automate all of that all right so we can orchestrate our security by orchestrating all of the different point products all the different tools we can automate it by ingesting automatically processing automatically and pushing out visualizing automatically and we can respond now this is where it gets interesting we can respond by oh gosh somebody reported that they lost a laptop here's what we now have to

do and it invokes the process it invokes the uh the system that you perform when a laptop is lost and then you go through all of these different steps all right cool but where is the decision so so in the military they use something called ooda loops and they're decision loops i'm not going to get into the definition of them but they're decision loops and if you have a a drone with a hellfire missile for that drone to fire the hellfire missile it requires a human to be in the loop to actually push the button okay or make the decision to fire the missile are we keeping people in these oota loops and these decision loops

of ingestion processing and visualization are we are we as humans making decisions this is where it gets interesting when you automate something should you keep a human in where do you keep the human in and i'll get into that later i'm sorry i'm getting ahead of myself but this is really fun what decisions need a human decision what decisions can be automated these are where these questions start sort of perking up if that makes sense this is fascinating stuff so we actually so red lion my company is actually a compliance company and we're like soar is nice it's a really cool acronym it it's an actual word it's neat it's nice it's wonderful we don't like it so we've got a couple

of choices and honestly i'd love some of your feedback we thought of a couple because honestly we want security and compliance orchestration automation response and risk evaluation because that's what we're doing we're evaluating the risks we're automating the collection of data the processing of data the orchestration of the point products to collect and process that data and we're automating the response well can we do that for security and compliance yes along the way of the collection processing and visualization and response the decisions made on that data can we perform a risk evaluation and even a threat evaluation to determine is this something we should do right now or should we wait a little bit and let a human in the

decision loop so we thought of score spelled a little weirdly but you know still a word we thought of actually just splitting off the compliance entirely so compliance orchestration automation and risk evaluation instead of response so i'm curious what you all think and i'd really love to hear your thoughts either in the uh post talk q a room on twitch comments on youtube comments email me call me twitter at me whatever i'd love to hear what you think okay so please your vote matters not as much as it mattered a few weeks ago just to be clear but it does matter all right and for anybody that didn't get that joke it's just after the

election right now all right so uh typical security automation you purchase all these point products and you want to automate them well what kind of point products are you buying and what kind of integrative systems are you buying i mean nobody here knows about nessus and nobody here has ever gotten a pen test where the low-cost pen test vendor barely scrubbed the nessus logo off the front of the report right more than i like to admit it's gotten that way vulnerability scanning asset discovery if you've never heard of rumble oh dear god go to rumble.run it's amazing if you need any help with that yell we're a reseller and i'm not trying to sell you i'm just saying

i can absolutely give you some advice on that event correlation elastic.co active directory examination if anybody remembers the uh the old bloodhound that's still around i think it's still useful uh anybody here use it still uh incident management security uh jp bourget a good friend of mine started it he actually just sold to i think swimlane uh for a third party security assessment prevalent a friend of mine rob used to work there uh for ticketing systems you've got jira for sims you've got oh elastic.co sorry so for event correlation and sims same thing but for identity and access management system sale point one of the better entitlement systems i know of but it's not cheap let's be

clear so you've got all these point products they're all collecting data they're all doing things with the data they're all managing data uh which of them are automation systems really only one or two of these are automation systems uh elastic.co down in the lower right is a sim so it can it can automate does it well depends on how you set it up none of these products out of the box are automation systems all of these products have the ability to automate but none of them out of the box are automation systems they are effectively point products and i don't mean to insult or impugn any of them they're all good products in their own

way but they are designed for a specific purpose so how do i automate them well i mean for those of you that are coders you're going hey they've got apis i can api them into something the answer is yeah of course you can have fun building it it's going to be unpleasant because by the time you normalize the data by the time you get all the data in one place by the time you figure out what all the different parsing patterns are for logs it might be a few years later and your boss might have gone what happened i sent you to do one little integration project and it's four years later you know i stopped paying you after year

one what happened you know that kind of thing it gets ugly fast so you want to be careful with that integrations are not easy trust me but they are possible all of these products i believe all of these products have api integrations so you can do integrations but be careful how you do them they can get ugly fast all right i didn't see that meme my business partner scott lyons dropped some memes everywhere in my presentations he's actually done it before that i'll start a presentation i gave like literally the day or two before and there's different memes in it just so he can get me to stop and laugh uh one time i was taking a drink same

time i switched the slide and he actually got me to choke and cough he said i was worried but i still counted as a point bastard so why is automation so important and this is a psychological question as well as a money question because automation is important because people are expensive they cost a lot of money we can treat our workforce we can not hire certain people we can uh make their lives more efficient we can do all sorts of things if i don't have to have somebody pouring over the log files every day okay so i can save a lot of money and i can save a lot of head count i can use that headcount for really really

appropriate uh things that are not tedious are not onerous or not whatever okay and it'll automatically reduce my costs and that's a good joke because it won't i was honestly expecting him to change that slide to be honest with you so this is fascinating he didn't take this one but he took the one before all right whatever automation isn't going to automatically reduce your costs it is expensive difficult not fun like i said the integrations can be incredibly difficult so why do we do it it's the same reason that we we chase outsourcing back in the 1990s and smart shoring and right sizing and name your fad and if you've ever seen the gartner hype

cycle you'll notice that these things go as fads and then they come down and they come back up again to where they're actually useful and although i'm the hype cycle's a little weird to me but it works uh automation and soar right now is just coming to where it's actually useful and so there's quite a few companies out there that are building automation going we automate everything i'm not sure i believe that they automate everything let's put it that way automation done intelligently can reduce your costs increase your efficiency increase your profits automation done not intelligently not so much but i mean do we need to do it can we just do it the old ways

yeah not really thank you scott security technology markets in a state of overload this is straight from gartner by the way thank you gardner they said that basically there's too much pressure on budgets there's not enough staff and there's too many dear lord and there's too many point solutions how many point solutions if you had to look at your company your organization your entity your agency whatever it is and go all right how many point solutions do we have for just security for just compliance for everything in that market for for all of our technology needs how many agents are on my workstation right now oh my god there's probably five or ten agents on your workstation

from different products there's 70 different point prod products we went to one company that had 150 points for various point problems and 17 agents on every workstation and they wondered why they were having technology problems it was it was a massive overload so they all were fighting i swear to god every single one of these point products was fighting with everything else you could practically hear the you know the brave heart style screams at going across the motherboards it was ridiculous so i love the memes so it's it's really awkward to uh to do this it's very very difficult it is a a really really tough system to have in place you've got so many different point

products uh budgetary considerations especially this year how many of you have had your budget slashed this year dear lord god almighty it's ridiculous okay how many of you have had your budget slashed not allowed to hire anybody and told we have many many more problems we have to hit there's uh over a couple a quarter million covet 19 scam sites out there there's more ransomware coming out practically every day uh and we can't even go on site to fix things i mean not that i'm complaining about working from home too much but still it's awkward at times so we've got to be careful of these things we've got to figure out how to make our dollars stretch okay

it's it's difficult and automation has that promise we will make your dollars stretch we will make things work better we will make your single pane of glass who's heard that before we're going to give you a single pane of glass you'll have all the information you need it'll be in that single pane of glass how often has that been true not as many times as the sales people would like you to think i think now is it possible yes if you're smart about it okay yes if you're mature enough as a company to do it yes if you're very clear on what you're going to get what you're going to pay for what you need

and you work through that with the the vendors there's a lot of good choices out there as to what you can get but there's also a lot of bad ones got to be careful of these things all right so one of the questions we always get asked is will this actually help you know can we automate security can we automate compliance can we automate these things to a point where it'll make a difference the answer is maybe if you're mature enough to take advantage of it yes so here's the test if you tell the head of i.t or i.t security pick one i don't care here's a problem here's a point product solution to fix the problem

and they go that's fantastic go buy it you're not mature no i don't need spam calls in the middle of a presentation so if they say that they're not you know you're not they're not mature all right [Music] and it's effectively it's not gonna happen but it's important because people are actually expensive and it's hard to get them to concentrate on little things all these log files flowing by everything going on oh well i have a sim that filters everything for me sure and does it make the decisions for you and it doesn't pull it in from every single device does it pull it in from all your different places all your different locations does it perform the scans does it do the

the compliance control testing what does it do well it filters the data that's cute i mean that's that's viable that that's a part of what you need you do need that don't get me wrong i'm not dissing that you need that but it's a part of what you need okay orchestration allows consistency in our responses it allows us to understand that when i am answering or responding to a to an issue a lost laptop let's say let's pick on the lost laptop i'm going to respond to the lost laptop the same way that rando will respond to the lost laptop the same way that scott lions will respond to the lost laptop the same way that uh a blind will

respond to the lost laptop okay we're all going to be consistent in our responses because we have a playbook in front of us what we're supposed to do you know we just had a discussion in one of the other discord chats here at b-sides delaware about uh checklists oh auditors love checklists i'm like yeah so do pen testers huh you know so to system administrators no the answer is no the yeah they do the good ones do anyway why because it gives them a consistency a process you know i started in this business in help desk support okay telephone help desk oh dear god that was i'll just call it fun and leave it to

nostalgia for for the reality there but one of the things that i got asked was what separates a mature technical support organization from an immature technical support organization and i said being methodical if you're methodical you can have a flash of genius you can have intuitive thoughts that's great but being methodical is what separates the mature organizations from the immature organizations a mature organization and system administration uses a gold image uses a an ami file uses a docker image uses a configuration management system an immature one builds what they need for each individual server for each individual workstation for each individual whatever a mature pen tester works through systems all the way from top to bottom they you

know an immature pen tester goes and i'm sorry i'm gonna piss people off but this but an immature pen tester goes i see a problem and goes and exploits that problem and breaks in and gets all the information and exfiltrates the information it writes the report and goes yeah and they didn't do their job but they broke in you're absolutely correct they broke in but i need them to make me more secure well you found one problem and fixed it yeah great what if there's 20 more right next to it did he find them well no well then he's not done his job and that's the issue we we have to become consistent we have to

come as an industry we have to come to the point where consistency is important we have to come to the point where we are professional where we are mature now for a lot of people they are don't get me wrong i'm not dissing you okay what i'm saying is that as an industry we are supposed to be professionals you know does a plumber walk in and go ah there's a broken pipe and replace that section of pipe yes they do but they also try to figure out why the pipe broke oh the pipe was old okay it's been pitted they see it's been pinholed and then all of a sudden it pop great fine they get

it okay but they try to perform a root cause analysis they try to be have a holistic view of what's going on as an industry we have to do these things because otherwise we're not performing risk evaluation and threat analysis properly if we don't know what the risks are and if we don't know what the threats are we're not working properly as an industry it's it's really just that simple i again i'm not trying to denigrate anybody but as pen testers as incident response people as forensics people as compliance people as uh uh uh whatever as anything assistive administrators any part of it i.t security compliance investigations whatever we have to be professionals and we've been working towards that for

a lot of years back from when you had cobalt cowboys to when we had our you know 1990s shadow runners to now when we mostly wear suits and ties except when we're home or on conference vision of course being home is most of the time now apparently pants are now optional i i don't know these things anyway but soar score core whatever you're going to call it and i still want some responses on what you think what should be we are really working on being professional being consistent being who we need to be to be sitting down at the adult table and the uh the sore score core whatever you call them these are the way we do it because it

not only helps automate our systems it helps us concentrate on the important pieces that are business aligned i'll i'll get off my my high horse i apologize so this is my my hand wavy time all right and if soar is done properly it saves money time and effort it it puts everything on rails and and if you read that list development devops security compliance etc it puts them all on rails uh and anybody that looks at that list and sees something weird well that's because i put it in there deliberately just to check if you're listening so pizzaops is actually fake uh but if anybody comes up with something that is actually pizza ops i think that would be

hilarious anyway so if done properly it's amazing it helps significantly but you have to be mature enough to do it you have to be a a in a serious enough frame of mind if you will to get it done properly it takes planning it takes effort it's not something that's done overnight this is not buy a new scanner put it in we get results the next day this is the result of planning for years and years and years ahead of time the result of uh either buying an entire suite of products in one shot or building your own automation and your own data pipelines if you will from product to product to product to visualization from ingestion to product

to processing to visualization i should say and if you don't do it properly you're gonna waste a lot of time a lot of effort you're gonna piss a lot of people off and you're not gonna get the results you can you're not going to get those efficiency gains you actually can realize so what is properly and i'm going to expound on this just a little bit okay i guess i'll end a little early that's fine um for a small company a large enterprise or a fortune 50 they're very different for a small company automation might mean just having a very good ci cd pipeline let's assume we've got a small code shop automation might be we have a great ci

cd pipeline and we can actually kick off a build by an automated process every night and everything works beautifully and we are as a software development shop we can concentrate on writing code not the code promotion not the you know did the build work not the let's do the tests like we've built out our unit testing our regression testing our load testing our our everything we've got all our test cases the testing part's automated oh my god do you know how much time you've just saved us as a code workshop as a code shop dear lord okay our cicd pipeline i commit some code it goes through a static tester it goes through a dynamic it gets built the

built environment is is in our test environment it gets run through a dynamic testing engagement a a tool or whatever and then it goes out for the test suites that we just talked about so i did this backwards sorry and for a small code shop or code building company that's amazing now other people can write code they don't have to write and do everything else that makes their life better that's an efficiency improvement that is massive and you're like well of course we use ci cd i have jenkins you know i have all this stuff what like is this so special trust me there are so many so many so many companies that do not okay

there are so many companies that don't have ci cd down pat i i've i've got to tell you the story oh yeah so this is a few years ago i was sent to a company i was working for god i forget who i sent to a company that does uh do you know store brand credit cards uh uh rando you know when your kids are big enough uh brenda do you have a daughter or i forget i'm sorry nobody can hear me a five-year-old son yes i do you have a five-year-old son if you had a daughter and she's get to about 12 and your son two i don't know they go to claire's and they get like a

300 credit card you know that you don't have to have any credit they'll give you a 300 credit card sure okay any of those little little tchotchke shops in the mall you know if you still have malls around you well this is one of the companies that ran those private label credit cards they're called they sent me over there like things are not working right there was a problem here and a problem there and that kind of thing and and randall i'm going to i'm going to quiz you on this you're going to love this scenario so i go over there and the uh their main code guy oh good his audio froze i'm not going to

be quizzed on anything where'd you go josh the credit card companies got to him josh where'd you go man it's not me

oh

should i just sing the jeopardy theme until it wait for him to get back uh all right while we wait for josh to get uh figured out uh brb oh

i'm like you're kidding right no you're you're joking no so i went over and i started his machine up and all of a sudden things batched this guy was running the entire credit processing system on his workstation and i know that just about everybody listening to this stream just went okay that's not proper ci cd that's not proper automation that's not proper any damn thing hey uh josh the only reason we went uh is because you just froze for like three minutes oh crap i'm sorry whatever that whole story was uh we said it was because of that okay well i'm back uh am i my back properly yes yes you're fine okay good so um it was still saying live that's

weird okay so did you hear the the the denim of the story the end of the story uh not really okay so all the credit batching stopped like just died it didn't happen that night so i'm like what the hell's going on here so i look for the host name i'm reading the code and i look for the host name that that the stuff is running on i'm like which server is on i don't even know nobody knows because their lead code guys on vacation can you guess where it was running at his house on his workstation right there in the office whoops because we sent a patch out and shut every machine down because we

updated all the machines and his machine you know his machine wasn't on so uh yeah uh the guy this uh wait did i lose the whole story uh yeah essentially like you ended with uh randa i'm gonna quiz you later and then you froze and i was like well thank god i don't take a quiz now well you took it anyway damn it basic the 30-second version of the story um and also it was a credit private label credit card processing company uh i did an upgrade on their machines i i i spun a patch out because i'm like oh god you guys need some patches really badly so i was doing in waves i spun a patch out deployed a patch

across all of their workstations and servers upgraded everybody and they were all shut down that night i said turn your machines on the next morning this one guy was on vacation so he didn't turn his machine on none of the credit card processing batched and it turns out that he was running all the credit card batch processing on his workstation in his office because that way he could know it worked and he could remote desktop into it from outside to check it if it went wrong well that's just convenient i mean isn't that a wonderful idea was he doing it over vnc or something straight rdp which he had coated into the uh into the firewall fun

fun side story on vnc if you google my name one of the google uh results is a thread that i put my at beginning of my it career asking how to vnc from the outside to my company's server oops we'll be uh taking that down and uh you know saving it for blackmail and then just just just it's right on google like it's one of the last links that's horrible um so uh you know this guy like like that's not ci cd that's not security that's that's horrifying but he's like but i automated it right on my machine to process everything perfectly and i can log in remotely to check if it goes wrong now this was a small company like 40

people but still i'm sorry that's credit cards have they heard of pci okay they were doing plenty of transactions it's but they were doing saqs self-assessment questionnaires and anybody that's in compliance just cringed right there because there's so many things wrong with what i just told you okay like horribly horribly horribly wrong uh let's just put it this way i i very specifically said to to my boss at the time i'm like i will do the security issues that you're that they're having problems with uh when it comes to their compliance i'm walking the hell away because there's no way i'm getting them compliant without rejiggering everything and the only guy that knew their code

was this one guy and if he got hit by a concrete truck they were out of business and if i went in and said you have to do this way this way this way he would have left and i'm not putting them out of business i'm saying you've got to send somebody else in so they did and they went out of business like seriously not a joke they'd literally let this guy run their system for so long he was the only one with all the keys to the kingdom so sorry anyway so that's a small company how about a large enterprise okay for a large enterprise you know you automation can be amazing you've got to make it so that nobody's

indispensable even in that small company we've talked about some of the basic points nobody's indispensable okay we've got to make sure that if you get sick if you get coveted if you get hit by a concrete truck uh uh you move to florida get sick and get coke whatever um you get hit by the bad drivers down there that you turn everywhere in florida it's crazy uh sorry kevin i got a good friend kevin johnson in florida and he i i bust him on the driving down there all the time but uh if you get sick get hit by a concrete truck we've got to be able to pick up the pieces and keep moving

all right uh you can't have something that is done in one place and nowhere else and nobody knows about it's not documented you can't have uh you know automation has to be so that it's reliable resilient uh you know gets around blockages and contingencies and emergencies the whole nine yards but so that's common to small large fortune 50 doesn't matter that's common but how you define properly implementing automation is different for all of these companies for the small company a good ci cd pipeline with backups of code and a backup programmer done they're they're good they're in good shape right for a large enterprise you've got to make sure that you're meeting your compliance requirements

you've got to make sure that you're meeting your security requirements you've got to make sure that you're doing things in a sensible fashion again a good cicd pipeline uh good test cases that you have tested automatically on the end of the code and i'm picking on code but it doesn't matter with security it's how are you doing your phone scanning how are you doing your asset management how are you doing your configuration management how are you doing your change management are these all integrated because if they're not i'm sorry you're not doing it properly okay and it's just that simple and when you get to fortune 50 you know fortune 100 fortune 500 whatever when

you get to the big guys you know what i'm saying the big companies you have to make sure that not only your security and your compliance are automated but your risk evaluation your threat analysis your business impact analysis on change you're adding more and more pieces when you add more and more pieces yeah you're bigger companies there's a lot more sort of slop in the budget but when you add all these pieces it adds up fast this is a lot of people working on these things the more data collection processing and visualization pushing out the data to the people or places responsible for it that you can do the more consistent you can be and the smarter you can be about what

you're doing and the ramifications of the decisions okay look uh we're going to be doing a series of these lectures on security and compliance automation i think it's fascinating we're going to so so disclaimer time redline is studying it pretty intensely we're heavily involved in standards making as well as product building so we're this is like something that that's really got our attention if that makes sense uh we think that right now is steam engine time for automation if you're not familiar with that term rando are you familiar with the term steam engine time do you know what i'm talking about when i say that uh no i do not as a matter of fact so if

you're a terry pratchett fan he wrote the discworld series and a lot of others and this is not just from terry pratchett but i love this uh he has this one book which talks about you know steam engines were built at three different places at the same time or four different places i forget and that's actually a true story uh in in you know regular world history when steam engines were built they were built at i think three places at the same time and all the inventors had never talked to each other they had nothing to do with each other it was not that they stole the idea from each other although they swore that you must have

stolen it from me no no they they just steam engine time like it was time for steam engines to come out so they did and if this guy had failed well there were two others coming up the coming up the road if you will and they again they had nothing to do with each other say different models different designs different ideas but they were all steam engines and they were all being built all at the same time right now it is time for security and compliance automation you've got a half a dozen companies doing it this way half dozen companies doing it that way five dozen companies doing it this way it's a very popular way whatever

some companies are automating compliance some companies are automating security some companies are automating a lot of this all of that some companies are automating your checklists your risk register your uh your third-party surveys that you send out your questionnaires that you send out oh god please stop with the questionnaires okay just to be clear some people are automating your vulnerability management program you get the idea but the idea of automation of your the chunks of security compliance and risk are all really happening right now okay so you've got a lot of different companies doing different things in different ways with different mindsets behind it so if you're looking at automation you need to decide what you want to automate

and why before you say we're going to pick an automation company you look good let's use you you're cheaper than them well they might be very different solutions the one guy might automate just von scanning and the other company well she might automate vol scanning and you know compliance uh half of the compliance controls that you have so yeah they may be cheaper but that's a better choice it's going to give you more value or you might go i don't care about the compliance controls i've got another company that does that that's fine your use case your need your desire for automation orchestration response risk evaluation etc needs to be very carefully uh formed and

shaped and understood before you buy any of these products please god stop buying point products start buying things to integrate the point products you already have and start buying systems integrated frameworks that start giving you information you can use in an actionable format that will give you the way to do something move your company forward in an efficient manner that saves you time and lets you get some sleep i know that stuff that's illusionary okay so if there's anything i can give you as a takeaway it's this i have not gone too incredibly deeply into the technicalities of this what i have done is tried to give you an overview of automation for security and

compliance as a field it is a field but it's a very very widely varied field know what you need buy only what you need and get a good price all right thank you and if you have any questions there's some contact info um i'm happy to answer any questions from the twitch stream or the youtube although i'm not actually don't have it open right this second sorry but if somebody has any they can throw it to me in the post talk q a in a couple of minutes or you can ask it on twitch right now and i'll try to answer them live there's the only one we have is from beep boop beep uh just a comment from them they

said it's a slippery slope to have written a checklist to capture testing items if not done correctly you'll just turn a pen test into a qa test that's a really good point and this starts you know this this was going back to the discussion i was having with w mackie uh which was great stuff and i enjoyed that very much by the way about you know compliance people like checklists and penetrate unlike penetration testers like checklists the good ones you know you you've got to understand that when you're testing something for maturity and i'm having this discussion on a standard that i'm i'm working on um and i can't talk about it i'm literally no i think i can i'm i think i can i'm

on a cmmc working group uh we help to build a lot of the pieces of cmmc it's a lot of fun but it's fascinating discussing what does maturity mean uh um um and and i'm not going to go into the details i can't i am under nda but i will tell you that it's it's fascinating discussing what maturity means and how do you measure it and how do you determine from what pieces of paper and what pieces of evidence and what you know interviews and all that kind of thing can you get the picture of maturity and it's a very delicate as you said yourself it's it's a very delicate difference like for example a a full red team pen test

okay you know full scope red team test your red team engagement shall we say it's going to get you a hell of a lot better results than somebody running you know hitting the big red button on on uh you know not arcsight on any of the the pen testing frameworks okay on the other hand you know the criminals have now decompiled cobalt strike and they've got cobalt strike and github all over the damn world and they're like it's good enough i'm getting into half the sites i'm looking at and it's just the big red button so you know again you've got to weigh your options very carefully so that was a great great comment thank you anybody

else if not i will be in the track one post talk q a and uh coming back in about 15 minutes to introduce james corbett for his uh for his impact talk [Music]