BSidesCharm 2023 - Complexity for complexity’s sake: The bane of cybersecurity programs

foreign

[Music] hi everyone I have four o'clock so I'll go ahead and get started um so hi I'm Nikki Robinson thank you everyone for coming today I'm going to be talking about complexity and specifically complexity when it comes to security tooling so let's get started oh maybe there we go uh so first up all thoughts feelings beliefs are my own they do not reflect my employers now that that's out of the way so uh by day I'm a security architect at IBM I worked on the ciso remediation team so I do everything from incident response vulnerability management everything in between um I'm also a professor of practice at Capital technology University so I teach research and design so I'm also an icit

fellow for this year 2022 to 2023 I also recently wrote a book called Mind the tech gap which was really fun and some of that uh it's the blend of I.T cyber security and human factors which uh shock I'll give you like a little preview I'm going to be talking about today um I also co-host the Brazilian cyber podcast uh with my good friend Chris Hughes on LinkedIn live I have a DSC in cyber security and PhD in human factors and one of the reasons why this topic I've really wanted to talk about for a long time is because of my research in human factors and in Psychology so that research really led me heavily into

lots of different areas in cyber security very specifically how quickly things get complex and and how difficult it can be to manage environments so

so I wouldn't be a good researcher if I didn't start with my problem statement so I'll start there but then I want to talk about how really how we got here how did we get to this place of complexity in our security tooling environments and not just security tooling but it's sort of cyber security programs in general there's a lot of complexity when it comes to our programs whether you're on a small cyber security team whether you're an I.T professional that's also the cyber security team or or working in a large organization uh then I'm going to talk a little bit about what that means in regards to human factors and then hopefully providing some good feedback some good

things to take back either to your own organizations or your own teams or your own tools uh to try to see if you have some complexity in your environment and maybe see if you can reduce some of that and improve efficiency of some of the security tools that you might have in your own environments and then I'm going to introduce a cyber security tooling and maturity model sort of especially for smaller to medium-sized businesses where it can be really difficult to try to determine what do you even need in your environment what do you need to sort of go forward and make sure you're getting the right picture so I'll introduce that near the end

okay so why are cyber security environments so complex and why tooling specifically is so complex it's really no secret a lot of environments are moving to cloud or using hybrid Cloud environments they might be using you know AWS and Azure or AWS and gcp as well as managing on-premise systems too they may still have pii or IP that they want to keep on premise or or specific environments Legacy environments maybe they're not ready to move to the cloud so that problem statement alone of having Cloud environments and integrating that into your organization starts to add complexity that that's where that picture sort of starts because when you're managing one environment it's pretty easy to sort of say okay

this is what I've got I I think I know what I need let me go from there but once you're managing multiple environments simultaneously especially if you're a small team if you're a developer with complex development environments it becomes really difficult to start to get that picture of what you actually have and then what do you need and then it starts to be okay well do we also need this tool do we also need that am I actually seeing all the vulnerabilities in my environment am I not so those questions start to come up and that's when new tools come into play or you know hey let's rip and replace let's rip out a tool and put a new one in place that

that may or may not solve the problem uh I.T and cyber Security Professionals overwhelmed um I think especially as a security practitioner I can say this for from my vantage point too there's a lot we have to look at and not just a lot we have to know right there's certifications in education but when it comes to security tooling the things that we need to do our jobs whether you're in a sock or you're doing incident response or digital forensics chances are you're using I would say maybe five tools minimum probably a lot more than that and it's also possible that there are other tools in the environment that maybe other team members are leveraging for the same

thing but they're seeing a different picture than you so they might be leveraging other tools seeing a different picture but you have to come together and figure out wait did you find this vulnerability did you find this so security tool sprawl becomes not just a problem for managing the environment because you have to do upkeep on those tools but it also can potentially increase the attack surface too all of these tools need access to other things so not only do you have this connectivity between devices in Cloud environments but then if you're providing administrative access or opening ports or you know providing additional access to these security tools that are containing all this information uh solarwinds anyone

it becomes this big pile of information that if compromised could be pretty detrimental to the environment too and then I just wanted to hit on a couple of buzzwords digital transformation and zero trust you know there's zero trust tools now there there's there's tools for lots of different reasons and it's not to say that one tool will fix everything because I think now we need lots of tools to do our jobs we need lots of tools to see the entire environment I'm going to get into that in a second uh complexity and depth so I wanted to take that down a level just from multi-cloud environments to anyone that's managing Legacy systems and new systems or had to potentially

even pull Legacy applications or Legacy OS into Cloud environments and are probably in transition to using newer or supported OS and application levels so now not only are you managing multiple Cloud environments or potentially a cloud environment in on-prem but you may also be supporting multiple OS levels multiple application levels that adds to that complexity it takes you down another level of oh we're not just supporting or looking at vulnerabilities for for all seven anymore we're looking for rel7 and rail 8. I need to make sure my patch my patch management tools I need to make sure my vulnerability management scanning that I'm seeing vulnerabilities for both but now instead of having eyes on one maybe one OS level I have to have

it on two three four five different levels a complex development environments if you have anyone that's running a development environment you probably have prod Dev and test you may have other environments too you might have a Sandbox you might have other areas that are specific for you know different types of applications and so those amount of environments continue to grow so when we're talking about security tooling not only do we need a window into those systems but we need to then parse that information and see it at lots of different application OS and other levels too so it becomes really complex really quickly and then looking at user accounts versus business accounts you know did someone

open maybe a cloud account on their own uh for the business maybe they're running some sort of environment in there but then there's also business accounts that they're using for other things so now you've got lots of cloud accounts that you're managing across lots of different areas that you need to have visibility into and you may or may not so this migration uh when we're talking about digital transformation it's not really just digital transformation it's really we're changing the way that we manage environments you know it's not like we just need a vulnerability scanner and maybe antivirus or EDR we need a lot of different tools now we need um which I'll get into in a second

but talking about sassed dast um we're talking about pipelines now cicd pipelines there's a lot of additional components and things that we need to be able to see logging logging is still a problem because now we have all this data but how do you parse it do you need to get another tool to be able to actually parse this information that you have so that's where the the problem started to happen we saw a need for new types of tools different types of tools to help give us a better picture into our environments because we need that data we need to see what's going on and if we can't get into these multiple environments it makes more difficult

hence another tool or another tool you know if you have AWS and Azure you may need to have separate security tools or separate security tool solutions for each environment that might play in their own cloud environment but that means two sets of cloud Security toolings Solutions that you're managing at the same time uh the needs of business are changing so quickly I mean we've all seen how quickly development is taking over a lot of what we do there are a lot more devops and devsecops environments the way that we managed Legacy I.T infrastructure is very different from how we do it now and some of that means that the original security tooling that we had in place is

a good start but now we have to integrate other tools probably open source tools other things to help give us that picture and then I highlighted Shadow I.T and Legacy systems because I think those are the bane of all cyber security problems but but especially when it comes to security tooling it becomes really difficult when you're trying to focus on what tools do I need to see in these new environments these new application and Dev environments versus how am I still looking at you know I hope not but a server 2008 or 2013 or how or 2012 how am I actually seeing am I actually seeing vulnerabilities on these systems and do I need to continue

looking at them while I'm also trying to figure out how to manage infrastructure as code so it's very two different two different sets of problems that we're trying to solve with lots of different tools so I wanted to take it back up a level because security tooling can mean lots of different things um it could be everything from you know looking at a vulnerability scanner to what do I need to look at for IAM how do I look at my different policies for user accounts because those are vulnerabilities too if I don't have any sort of lease privilege in place or how do I actually get a picture into what's going on with my accounts and account

management so managing current infrastructure while also trying to tune and manage tools for current threats current iocs indicators a compromise trying to look at specific threats or specific iocs for your own business or your own domain it can be really difficult to manage that current security tooling infrastructure while also trying to constantly tune looking at alerts tune alert student alerts so it just becomes increasingly difficult to manage the other is the I wanted to bring up because when a business is evaluating security tools The Business Leaders may be looking at security tools for a different reason they might be looking at it for what kind of reporting capability does it give me what kind of

visibility can I see quickly yeah what kind of Graphics can I see what kind of reports can I get versus you know security tools that we might use as practitioners to help us get a better picture of what's actually going on and we need a different view than a business leader would so how do we get the right tools to help make sure that our Business Leaders Executives and directors are getting the right information they need while also giving us practitioners the right information to be able to remediate vulnerabilities to find them and hopefully hone that information going forward and then integrating threat intelligence you know that that's a big um it's a big step for a lot of small

organizations to integrate threat intelligence but it's becoming more and more important to integrate those types of tools that type of Intel maybe even a program into an organization so that you're getting more context into your own environment instead of saying okay I've got you know 10 000 vulnerabilities what do I do with them what's most important how do I prioritize and that's really the most important piece of this is that how do I get the right information so that I can prioritize because there are so many vulnerabilities released every day whether there have scores or not whether they have patches available or configurations available to actually remediate them but that adds another additional layer of complexity that I'll

be getting to in a second too but it adds to that picture of okay I need to see all these vulnerabilities but now that I have all this data how do I prioritize it what am I going to do with it which takes me to a vulnerability scoring so uh I did both my DSC and PhD around vulnerability training and vulnerability scoring mostly because I think it's a really interesting topic but also because vulnerability management can be a really complex topic and for a lot of I.T teams or development teams you don't have a ton of time to sit and worry about what does a nine versus a 10 mean and what do I fix first you know if I have 5 000

critical vulnerabilities what does that mean to me you know I can't remediate them all what do I do first so that's one of the reasons why I think this is a good one to sort of dive deep on because this is just one area of security tooling and really one area of one type of security tool to sort of highlight how complex this can get when you start to dig into what does this information actually mean when I'm getting so much information from so many different tool sets so we have now for vulnerability scoring which is great by the way we have a lot of information uh it's great you know we have CVSs is is a place to start it

gives us an initial score but it is it doesn't provide a lot of context it gives us a sort of a baseline but like I said when you're in an environment you have 10 000 critical vulnerabilities you know where do you start now we have the exploit prediction scoring system or epss which again is great it's another enhanced uh ability gives us an enhanced way to look at is okay a vulnerability is a critical or a medium or low is it exploitable if it's exploitable okay should I remediate that first probably if it's externally facing should I fix that first yeah probably but that helps give you that picture then to sort of Whittle down just from a

CVSs score but now we also have the Kev another great tool available to us known exploited vulnerabilities now we have a really great open source library of known exploited vulnerabilities that's awesome now I can take this information add even more context into how I prioritize vulnerability management and Remediation but depending on the tooling that you're using there are other great product tools that provide scoring or maybe enhanced context like the tenable VPR the vulnerability priority rating which again is like a heat map and it helps show you where you probably want to focus your remediation efforts but now we've got heat Maps we've got lots of different scores we've got open source libraries that we need to look at

to see if this information is applicable um and then we need to have the context of our own environments where we're probably managing multiple patch level system multiple patch levels we probably have risks that we've accepted for certain things certain ports have to be open okay but now I've got externally facing versus internal systems now I've got pii or Phi if you're in the healthcare industry and items that can't be remediated so all of those layers all of those individual pieces start to stack up into building this picture and it's literally just to ask the question should I remediate this first that's it there's nothing to do with threat intelligence or anything like that and and so once you start adding

all these other security tooling EDR and all these other components into that picture it becomes a lot of information really quickly um and I know that once you're managing 10 15 different tools it sort of becomes okay I've got vulnerabilities in 10 different dashboards am I getting the right information is this one is this tool showing me something that this tool isn't um so it can it can become really confusing really quickly so I wanted to give just sort of a glimpse of security tooling landscape this is by no means a complete picture because it's changing all the time and we need to see different information all the time especially as I think development environments continue to get

more complex and more mature we're using lots of different types of of tools and and ways of building code like um without infrastructure or without as much infrastructure having to manage operating systems things like that that will hopefully help at least some of the infrastructure management component when it comes to vulnerability management but when we're looking at security tools and starting to look at security tool sprawl it starts to become this okay I need EDR because I need active monitoring but maybe I'm not convinced that I'm getting a full picture from this EDR tool I want to also add some specific type of malware analysis or detection or I want to look for something else specifically you know I

used to use Malwarebytes in combination with other things so it's not unusual to have one or two different solutions just for one of these areas logging any complex environment or any environment where you have multiple administrators or multiple developers hopefully you have logging coming to one place but it's also possible that you could have logging going to a couple different places you might have a couple different solutions for logging tools hopefully not because that would be really complex but it certainly happens and then when we start to add in the picture of asset management configuration management risk management how are you actually managing if you have open risks or you know you have items that you can't remediate are you

storing them somewhere are you saving them in a database or are you keeping them in a list somewhere and then when you're adding in incident response forensics if you have a small team if you have a big team and they're doing their own forensics work you probably won't have to worry as much about the you know the sort of forensic tool landscape but if you have smaller teams where you have to have all these tools in one environment it can become really difficult to manage them all at the same time and not just the amount of information that's coming in but something as simple as oh did I update that am I using the latest version of in

case or is my attenable Nessa scanner up to date so it's not just the actual seeing the data that you're getting and configuring the information to get the right picture but it's also the management and infrastructure of those tools okay so I'm gonna hop out a technical for a second and hop into human factors and some of the psychological things that I think are just as important about talking about the technical complexities of security tooling when you're managing 10 15 20 different security tools I start to think about mental workloads not just managing the infrastructure managing the data you're probably reporting on that data and you're probably making decisions about that data so that might be just one piece of your job

too so if you're like me if you're like a security architect or if you're a security engineer and you're managing lots of different tools you're probably not just doing that you might be doing incident response you might be looking at sock alerts or working with the sock if something happens as like an escalation point you might be working with management providing reporting and data so you have to be able to ingest all this information and be able to pull it out to the right people and giving them the right context that is a lot of information to take in and to make actionable decisions on um and I would say security people are really great at doing that but it's also

I think one of the reasons why we see a lot of burnout in socks and you know in cyber security in general we have a ton of information we have to take in all the time make decisions and get it right we can't get it wrong in cyber security because if we get it wrong we might miss something an incident could happen you know something could go wrong so it's really important that when we get that information that we act on it properly context switching this is one of those things that I know we do we do this all the time outside of cyber security right you go from one app to another you go

from one tool to another you're looking at maybe let's say you're looking at EDR and you're looking at different types of alerts and things like that and you're like oh let me see that vulnerability report was coming in that should be done uh right now let me go look at that and then you go look at the vulnerability report and then you're looking at email and then you're looking at slack and um then you look at Facebook I mean you know how do you have the time um but context switching essentially means that you could be missing things right I could be missing a really important Facebook post if I'm constantly looking at EDR all the time

no no but um but it can be a lot when you're context switching to potentially miss something and if you're missing something that could be uh could be like an alert in EDR that could be malware or or maybe um you know you're not quite sure but I don't really have time because I also have this report that's due at 3 pm and I have to send that in um it could be missing something that could then turn into an incident so um it's really important to for us when we're context switching to to not miss something and and uh it's more about I think awareness of what context switching can mean and how difficult it

can be when we're doing this all the time metacognition this is my favorite psychology word uh I love this concept too because metacognition is how we think about thinking it's really just meta um but how do we think about thinking how do we make decisions if I made a decision how do I go back and look at that decision and determine that I made the right decision how do I determine that I had the right information at the right time if I've got 15 different security tools I probably have the right information but did I have time or the right rule sets or the right configuration in place to be able to get the right information

to make that right decision and if I make a wrong decision can I go back and look at that and say hmm do I need to tune this security tool to help me get the right picture next time um so metacognition again it's one of those things I think it's important for us as security practitioners to get time to be able to go back and say do I have the right picture did I make the right decision there based on the information I had at the time or could I have made a better decision if I had tuned this tool in a different way so so thinking about it that way uh perception I'm going to talk about

this again you'll probably hear me say this like four times but perception my perception is that this tool doesn't work we need to get rid of it it's it's not working and so we're gonna rip and replace ripping and replace tooling means especially for the people that are doing it not only do you have to completely decommission any systems that were managing that or supporting that if you had vulnerability scanners you've got to pull all that out now you've got to bring in an entirely new tool set up a brand new infrastructure and configure it all of that configuration time adds again additional time additional complexity you could be missing something because you're also trying to

decom the old systems and bring new systems online because in the time that you're ripping and replacing you could be missing data and you might have to do both at the same time you might have to manage both tools at the same time while you're bringing the new one online again that adds to complexity and time spent on One Security tool problem versus maybe the security tool landscape where you've got 10 or 15. so perception my perception is that this tool doesn't work is that reality is that really true or is it that we haven't had time to properly configure this tool because we've got 50 other tools that we're managing and I haven't had time to to

configure this tool in the depth that it could you know I think of things like Splunk um that tool can take a ton of time to for configuring uh rules for configuring alerting but it works really well and it can work really well and there's lots of tools like that that when you can spend the time to really configure them properly they can work really really well um but a lot of times we don't have the time to sit down and configure a tool to even 50 or 60 of what it could possibly do uh so we're into we end up not with the full picture we could have and then you know maybe to management or

executive management it looks like well but we're not getting the right information or I'm not getting enough information this tool isn't working and maybe that's just perception maybe it's reality but maybe it's just perception it versus business versus cyber security so this is the whole reason I wrote the book because um we all need a different picture we all need to see something a little bit different to provide context we all have different goals you know I.T or development they might have slas they may have if they're working in agile maybe they have two week Sprints that they're really worried about in cyber security we're worried about the next ioc or the next exploitable vulnerability and how that may impact

those teams and so sometimes those are really competing goals and it can add to the attention or misconception about how the security tooling is working or what we need to see everyone might need a different picture but again sometimes that comes down to tuning it could come down to how is this tool actually configured to report the information to the right person you know am I sending a 300 page vulnerability report to a developer and asking them to just fix it all you know that's probably not a good solution um so how can we tune and potentially fix some of these things to make that picture better for the actual audience the people that are actually consuming

the information and again that just takes additional configuration time sometimes time we don't have alert fatigue you know this is one of those things that has been studied really heavily especially with socks um because and not the socks that you wear um but for socs um still one of I love that acronym anyway um so socks that that is their job sock analysts they're constantly consuming information looking at alerts Network traffic um EDR alerts or or is this malware did someone get fished was a file downloaded to someone exfiltrate data did someone they're constantly looking at all these different streams of data um when you see alerts all the time you start to think that's is that really an

alert should I really be is that really anomalous it's probably not that bad um especially when you see you know 99 of the time the traffic that you're seeing is probably okay you might miss that one percent because it's like oh my God I'm looking at alerts all the time all the time uh there was a study there was a study done on alert fatigue and it was on essentially how long a sock analyst could look at a screen and get the actual context from the alerts that were coming through and I think the cap was at like 15 or 20 minutes of the amount of time that you could actually sit and stare at

the screen and get the right information out of it and that'll be dependent on the person right like some some people might be able to take that information in for longer some people for Less um but imagine staring at that screen for eight hours a day every day for years um alert fatigue is a real thing you know if you're constantly looking at alerts you may miss something and it's not you know it's not anyone's fault it's just how we as humans operate um which is why job rotation I think is so important for socks you know not having them constantly look into alerts all day provide them uh something else to do you know six month rotations or

something like that to help give them a different picture before they can come back to to looking at alerts um and then decision paralysis so this is one of those things I just started looking into recently because uh one of the things we do in cyber security in general but especially when we're taking in all these streams of information from all these different security tools uh we have to make a decision based on this information when we have so much information it can be difficult to make that decision to say is this anomalous Behavior or is this expected because if you say this is anomalous behavior and you go ahead and start with an incident and you're wrong

you might be getting executive management involved or you might be getting a customer involved in an incident that you know may or may not be happening but then the opposite can be a problem too if you make a decision too quickly without the right context you could also make the wrong decision in the wrong way so decision paralysis is this sort of oh my gosh I've got so much information I'm not sure what the right answer is because now I'm taking in so much information it's hard for me to to sort of make that decision um I think the human factors piece or and the psychological piece really of human factors um and human factors for anyone that

just hasn't studied it in depth I see a lot of research around human factors as security awareness training um and and that's one piece of it how do people learn but the other piece of it is uh design how do we build tools that are effective and and help people be more efficient um understanding engineering and psychological practices together so it's one of those reasons why I think we need to sort of bring human factors into what we do as security practitioners one because I think it can help make our jobs a little bit easier but I think understanding big problems like security tool sprawl and complexity even just being aware of some of these things we

can build them into our programs and add a job rotation into socks maybe to help reduce alert fatigue or reduce missing you know potentially major alerts okay so to come back out of human factors a little bit um I I asked this question because you know when you start to think about how many tools you have in your own environment uh the real question is what do you what do you need what do you actually need you know do you have tools in place that you know maybe you're getting information from but maybe you don't log into that tool that much maybe you don't really need it that much or maybe you're sort of getting the information from

somewhere else um but you have to have some sort of vulnerability scanning in place you have to have something in place to be able to understand uh what what vulnerabilities do I have in my environment that might be multiple tools it I think especially in very um in multi-cloud environments or in again in on-premise and Cloud environments you might be able to extend a tool from on-prem to Cloud uh maybe not you know it's going to be dependent on the environment but to really help with that prioritization piece because without it it's just a ton of vulnerability information uh that you're staring at and trying to figure out how to how do I fix what do I fix first

endpoint detection and response some way to contain a machine some way to contain a an endpoint that you suspect might be an issue or have an issue that that's something that you really need you need some sort of alerting or capability to get a good picture of your environment of sort of what's going on or if something might be malicious uh real-time monitoring so that you can have some type of continuous monitoring in place so whether that's with a vulnerability scanning tool you might have you might have it with EDR some sort of continuous monitoring and alerting happening uh you might have to have another tool for that depending on what your environment needs are

um and then I should have put asset and software management at the top because that really should come before your vulnerability scanning tool but where and how are you managing your assets and your software libraries you know do you know what you have and and where it is because without those it it's really difficult to get a good vulnerability management picture but it's also really difficult to determine not only what vulnerabilities do I have but what do I need to fix first

okay the other thing I wanted to bring up that again adds to this sort of big complexity picture which is open source versus paid tooling so chances are you've probably got a mix of both in your environments there's a lot of really great open source tools out there to help if you have one picture with your sort of maybe your vulnerability scanner or something like that you might have another tool in place that might help give you better context or better information um but multi-cloud environments you might need to have external Tools in both sets of environments not always there are plenty of tools that can play in multi-cloud environments but it's certainly about maybe finding those

tools instead of managing maybe lots of tools across multiple Cloud accounts or across multiple different types of cloud environments and on-prem and then open source again it's another great potential area to get a better picture of your environment but you're also again potentially increasing that attack surface depending on what type of tooling you're using so if you're using something that isn't being patched or updated frequently it could be its own vulnerable uh entity that you're sort of bringing into the environment too so it's another one of those you sort of have to weigh am I getting better information from this versus increasing my attack surface but it's another essentially component of how do I choose

the right tools what do I actually need in my environment and what's worth it what's worth it to have in the environment so I wanted to introduce this this is um it's pretty basic but hopefully a good starting place for anyone that's got you know maybe massive Tools in your environment or lots of complexity in your environments but to think about you know sort of where you want to start um and once you've sort of gotten to the top I'll go through each of these but once you've sort of gotten to the top to come back down to the bottom and start to evaluate again you know we do continuous monitoring for vulnerabilities for Network traffic and

and alerting why don't we do continuous monitoring for our security tools you know why don't we continuously monitor them and see if they're working for us if they're making sense for us or if they're configured properly just like anything else you know configurations can get changed or may need to be changed depending on how the environment is you know I always use something as simple as if you have um you know Windows 10 and windows 11 you may need to change if you have a vulnerability scanner in place you might have to change your policies to make sure that you're accounting for any of the changes in Windows 11 versus Windows 10. Powershell monitoring you know that

may change depending on what version of Powershell that you have in the environment so again it should be this continuation this iterative approach to security tooling where you're you're sort of asking yourself is this still valuable and if it's not valuable why is it because I haven't had enough time to spend on it or is it because it really doesn't meet our business needs anymore you know we're moving to multi-cloud environments we need a tool that works for both this just isn't going to work anymore um so it's really just sort of about you know asking that question and making sure that we do this over time so starting with something like EDR right that would sort of be the first place

you want to make sure that you've got something on your systems when you bring them online and making sure that they're being actively monitoring actively monitored some sort of vulnerability scanning of course and then some sort of network monitoring maybe that comes with EDR maybe it doesn't maybe you need a separate tool for that um compliance reporting and Frameworks I've seen a lot more especially for environments that maybe go through regular audits maybe they have assessments or they're beholden to specific regulations they might have to have a separate compliance tool or framework something in place that they're actively monitoring there are plenty of vulnerabilities scanning or vulnerability tools that come with that but again it's just additional

configuration that will have to go into place um and then Sasser dast if you have complex development environments and again that's not all inclusive there's lots of other application testing and like I said CI CD pipelines and things like that that you have to consider and then you know are you getting to a place of talking about threat modeling and probably using the tools that you have in place to use for threat modeling or potentially getting threat intelligence Tools in your environment to again expand the view of the information that you have but to also give you context for the information that you're getting and then hopefully getting to a single plane of of glass or something like xdr

which is extended detection and response but hopefully giving you some some single mechanism some or maybe one or two areas that you look for information versus 10 or 15 different places that you have to look for information so hopefully that's the end State you're getting to a place where you have this sort of Consolidated tooling and monitoring um knowing that that's not always possible but hopefully that's a good end state but then always going back through that process and saying okay am I actually seeing the right vulnerabilities in the environment am I not um and and going back through the steps

okay so my last bit here um that I'm going to sort of talk about recommendations and this is just these are questions I ask myself all the time too um but uh when we're talking about complexity and this is complexity both from the technical side and from the you know people in human perspective too take a comprehensive inventory of your tooling that's open source and paid tooling you know what are you paying for how much are you paying for it is it working for you but also you know what open source tools do you have an environment in your environment um and I mean you know really talking to you know if you're management talking to

your security people and saying hey what are you using like what are you actually missing are you missing pieces that you're trying to fill the Gap with open source tooling are we not providing you the right budget or tooling you know to sort of go back and figure out what you need um you know versus trying to come up with Creative Solutions or building tools to help give us that picture um so what do you have versus what are you missing this can be a this is actually a really difficult question to answer because you don't know what you don't know um but I think that's why like conferences like this are great or actually like talking to vendors and

seeing what products are out there because um sometimes just having a conversation with someone can give you a better idea of oh I didn't know that was a thing let me go check that out and see if I can use this and maybe it'll give you a better picture into your own environment vulnerabilities management is one of those things that especially in like I keep mentioning complex development environments but that's because vulnerability management is getting more complex it's increasingly complex because without the right Tools in place it might look like an application is secure you might run a tool and it's like oh yeah this is fine um but it might not be you could run

another tool and get 150 findings and it might be because you didn't have the um the right tool for that specific application or that specific library to actually see those things so um so determining okay do I have the right tools for the actual application infrastructure that I have um and then the big question are you really missing functionality or are they not fully configured I really recommend this this is one of those things I do all the time where I go back and I look at tools and say one am I getting the best picture and two and like if I'm missing information can I do it with this tool is there something I'm missing from the

Management console or from the actual visibility it has to different systems maybe because my asset inventory isn't quite right I'm not getting a good picture of what's actually going on the to actually determine the risk in my environment so I highly recommend taking a look at your tools seeing how well they're configured and spending some time with them um and I say skill sets uh because something like Splunk I am not an expert in Splunk by any means that is something I would need to take time to really really learn I understand basic rule sets things like that but I think to configure a really complex tool like that I would need training I would need

to go and spend a week and really dive deep into it to understand how it would best serve me and my environment so um taking that back to to the teams you know do the teams have you know if you're using like a vulnerability scanner do they really know how to configure it or did they sort of inherit it and they're managing it you know configuration and operations and management are sort of two different things so if you're handed a tool you're not quite sure how to manage it it could be worth it to go take that upscaling and say I'm not quite sure I need to go figure out how this tool really works

how it's actually configured because that might give a again another clue as to um why it's configured that way maybe I'm missing something and maybe how to configure it properly um and then for practitioners uh I think having really good conversations with management about saying you know this is what we have right now this is what I'm able to provide to you as you know an architect and engineer this is what we have I can't give you this information or I can't prioritize vulnerabilities because I don't have this this is something I need to be able if you want a better picture or um you know you want to see vulnerabilities in any sort of

prioritized way not just criticals and highs to help open that picture up and the same for management you know I encourage management to actually go talk to their practitioners and say um okay this is the information you're giving me is this everything is this really what we need do I need more information am I missing something here um I I think those conversations help to alleviate some of the perception that I was talking about before perception can be one of those things that especially when it comes to secure tooling or choosing the right tool or not I think it can be really challenging to make sure that you know management understands that there might be things

under the covers of the actual tool that they don't get to see because they just see a report or they just see a dashboard they don't get to actually see all the configuration and time that goes in to make sure that that tool gives the right information so um so I encourage those sort of iterative approaches to those conversations because the information you need is going to change over time too it'll probably change day to day but hopefully at least starting those conversations can help to give at least a picture of okay this is what we really have this is what we don't have and this is what we need versus this is nice to have

um you know and hopefully that again will help to bring that understanding back to from management to practitioners on you know I know I can't have everything but if you really want everything this is really what we need to give that idea um so I wanted to um I really wanted to leave plenty of time for for questions um so that's really my presentation on complexity and security tooling and happy to answer any questions that anyone might have

thank you I think Dusty oh oh hi

[Music] that's coming down

steps to say here's how yes totally um thank you for that yes that is totally one of the reasons why I wrote the book because uh conversations between developers and security or it and security I can say this because I was I.T operations for about 11 years before I got into security so I've had those conversations from both sides of the fence and I know that they can be challenging or that there can be tension and friction between teams mostly because you have competing goals you know a developer is worried about their two Sprint two week Sprint cycle or they're worried about you know hey I have to deliver this on this date and you know maybe my bonus or my pay is

dependent on this or my job maybe I'm worried about my job if I can't deliver this this that's I'm worried about job security and so as security practitioners I think that's one of our jobs when we're working with developers or or it in any capacity is to lead with empathy that's always one of the things another piece of that psychological component but to listen to what's actually going on with them and so when we provide a vulnerability report to not provide them that 300 Page report I like to start with the top five that's that's usually my recommendation is start with the top five and say Here's the five things we have to fix in the next two sprints this is what we

have to do now because if you overload them with vulnerabilities it's sort of like what what am I going to do with this you know I don't have time for this this is going to slow me down it's going to break my product um all these things and so I think if you can open that conversation with hey here's the report but here's the top five things we need to do and I'm going to help you do that um yeah so hopefully that answers your question yeah thank you yeah and

like that's I'm sorry I couldn't hear

[Music] foreign

no that sounds awesome that sounds like something I need to go to a week for training for no but that sounds awesome I love that if it's tool agnostic or if you're if you're able to actually build that type of analysis in I love that because um I guess it would depend on the solution right if you're able to have the right skill set right for people to to be able to do that and know that you're not missing information yeah I think that's great and you gave me something to research when I go home so thank you

[Music]

starting point awesome yeah thank you any other questions yeah

yeah yeah so I yeah thank you for the question yes I do think that we're going to get to a place like that I think that that is one of the problems that I think our industry is trying to solve there are a lot of vendors out there that are trying to solve this problem knowing that you know hey to do my job I have to have 15 or 20 Tools in my environment and that that's not a feasible solution because we are humans there's only so much information we can take in and actually digest and act on that's the other piece right is I can digest a lot of information yes yeah they have a ton yeah if you go to

the website they have a ton of different degrees and um like I said different labs and um so yeah I I mean I really enjoyed my time there that's why I'm still there um but yeah they have a lot of cool degrees Yeah yes

yeah so I think the human factors conversation from both the management perspective and the practitioner perspective is really important I guess I focused more on the practitioner because I am a practitioner but yeah I think it's it's both ways right um really effective management especially when it comes to a situation like this isn't to say you know my security practitioners didn't configure this properly this is their fault you know it's really about the question did I provide them the training and the Skilling to understand this tool or did they get hired and I told them they need to figure it out today so yes absolutely I think it's it's sort of um has to be from both sides

um and the this idea of perception this is something I just started exploring in the last year or two when it came to research because I think perception and I don't have any proof behind this yet but my my hypothesis is that perception affects a lot more in risk management and in Burnout and things like that that we're seeing in the industry then we are aware of at the moment you know if that perception is my security team is doing a bad job um that's going to be negative for everyone right versus saying what's actually happening on the team let me ask around let me ask a few questions you know did do they know how to manage

this tool no can I send them to a week for training and fix the problem great I don't have to fire someone and hire someone that's bad for everybody um so yeah so yeah absolutely both ways yeah any other questions no all right well thank you all so much I really appreciate it um thanks [Applause]