BSidesATL 2018 - Vulnerability Management 101: Practical Experience and Recommendations (Eric Bryan)

everyone good afternoon transsexual coming out if you saw my name in the program and you decided to stay a real appreciation thanks so much come on I am Jerry Brian security engineer with technology solutions so I've been a 19 for about 18 years and in security prevention of those years my background is in telecommunications and technical support and about 10 years ago I started working as a private investigator on the side and it's of digital forensics and that got me really hot and heavy into the security side I currently specialized a risk management compliance consulting and of course vulnerability management so much become what's called ability man it's defined as the cyclical process of identifying classifying remediating

and/or mitigating four pillars a vulnerability is defined as a weakness and information system security design procedures implementation or internal controls that could be exploited to gain access to information or an information system so why is more ability management important because the sans needs to set so they have a document called the top 20 critical security controls these are a simplified list of actions to help an organization and assess it'll improve their security posture specifically here in number 4 continuous on a great assessment and remediation the point which is to continuously acquire assess and take action on new information in order to identify the owner abilities or mediate and minimize the window of opportunity for attackers it's also

important is there's a constant stream of threat intelligence routes is coming from many sources we'll get to some of those sources here a minute and failing to keep pace with these just allows for concurrency location for example there's a zero-day vulnerability published assuming that it's not published in a responsible manner everything of information is available to all parties it wants and then would have a three-way race between the attackers to weaponize and exploit the vendors to develop and deploy Center updates and the defenders to implement defensive measures whatever those may be so my context here practical experience is with the reach every client of ours up in Charlotte this client has about 45,000 devices and

they're running everything from Windows cisco juniper Apple Android and countless applications on these devices so when we started the hallmark of the formative management program was very very reactionary okay so the scenario an existing vulnerability was discovered so what do you think happy ran around with the hair on fire like everything was all crazy so we pursued the vulnerability until it was remediated or the risk was accepted so our next scenario there's possibly applicable vulnerability so something's out there might apply to us it might not so what happened what kind of crazy once again hey so again panic mode was activated here we go and again everybody's here was on fire and you tracked the vulnerability until we reach

some kind of remediation in this process it was ultimately effective it ultimately did exactly what we needed it to do but it was incredibly inefficient instantly exhausting everybody involved especially the security team who had to facilitate the whole thing so from the experience here I came away with three primary components to vulnerability management the first of which donor ability discovery we'll talk about that first vulnerability notification so taking the information we get from disturbing our abilities and passing it to those responsible for mediation and finally remediation verification so ensuring the remediation be completely so the first goal when we started this program wants to improve our vulnerability discovery but the first step first thing we did was deploy

a bonus game so rapid7 next bonus will be used and that helped us identify existing assets vulnerabilities and configuration issues and then the first thing we did after we got stood up completely implemented was run a discovery scam of all internal assets now the client operates a / 8 network so nearly 17 million IP addresses so this thing took weeks and weeks and weeks to run away we had an IP address for all hosts we had an OS fingerprint that was for one does devices we're gonna be right everything else was gonna give it safe and unusually Windows devices it would give us the host name as well and then use that information with your

create for an interest and so we would create a Windows service account and it will scan these devices and come back with information useful for mostly Windows and it would analyze system configurations open ports and protocols and create a software inventory trees device for those Prudential for valid even still do discovery scans to help identify unique broke-ass as the pocket so the next aspect to improving vulnerability discovery is reviewing newly published bonerboys and to do that it's peeking from any sources first the national vulnerability database or NVD which is a multi-format downloadable vulnerability listing we get this database on a weekly basis and compare with previous weeks and look at the difference between the weeks this was

especially fun the first time we did it I had to go through years and years and years of information there trying to figure out what applies to us and what we need to address now secondly the United States computer emergency we're spreading a scene for us sir that's the division of Homeland Security and they provide notification services and also becomes critical next up vendor specific alerts so Microsoft racks Tuesday falls in here as well as those from Cisco VMware Oracle etc least on a regular schedule and/or s corner abilities are post the next source was research and aggregation sources Brett post CSO online and sidon are a few examples here also it's open to account if you think

that we found out through our penetration test we have performed and finally such a medium so Brian Krebs here to eat in your service if you want the latest greatest security information Twitter is where you hang out so thanks to these sources we've identified some vulnerabilities so what now the next aspect for us to consider was resource allocation how can we prioritize our resources in a way that reflects the specific needs for organization first we consider how the vulnerabilities report by the industry so the common vulnerability scoring system or the cbss and that's an open framework for communicating the characteristics and severity of software vulnerabilities he uses three primary metrics he got face the intrinsic qualities of a

vulnerability simple aspects of the vulnerability that may change over time and environmental aspects specific to an organization's environment you use the ten point scale as you can see here and cbss version 3 introduced dynamic components and that allows for adjustment of the score based on these environmental factors we considered this but found this method of calculation to be too cumbersome the connivance enemy for us to use and also it was very subjective so two people looking at the same vulnerability might come away with different scores for the same thing so we've got implicit home calculations look like this but it feels a lot like this ma'am am i right so we devised an ultimate simplified

calculation scheme the adjusted scoring system my acronyms need some work the purpose behind this was to take the to account to CBS s store and factor specific to the organization there are three factors we considered with a maximum simple score of 10 so we would take this 10 points combined with to CBS s 10 points and average that to get an enterprise a simple score so the first and largest factor we looked at was external accessibility get any of the assets affected be reached from outside the organization if just one could they've got four points but if all the assets were internal only you guys next up data sensitivity so we have all three since

their data types here in our environment so PCI thi you see how we gave weight because compliance bhi this grocery store also operates a pharmacy so we had to consider that as well and PII personally identifiable information you can see they were stored like this in the mutually exclusive so if the system had data types all three only the height would take affecting finally which I'm you to account prevalence in the enterprise so if you were to implement this elsewhere you would need to look at the size of the organization and adjust this we have an optimum scale for us was fifty or four devices three points twenty to forty nine to twelve and eleven twenty

got one point so by implementing our judges scoring system we go from this to this much simpler right so let's take a good look and a practical example of our adjusted scoring system so this is one that actually happened this clown DeBarge uses ice and the next is the supplicant so tons and tons of devices with this it's a twenty 1766 38 in connectors RCE so we had all these devices and they were internally under them to be accessed from outside the organization directly so and they were within pcs co and waiting more than fifty devices so we did our math here and came away with an adjusted score of 6.9 so it knocked it down from a high to

a medium which you know what's the huge value in that but it helped us over everything else better now we have adjusted score we need to determine if the vulnerability should be would be addressed in the time so that's the 10-point scale we'll talk about a second ago low remediation should not require medium high and critical they must be addressed medium within 90 days and plan critical within 30 days so now we're starting for our ties are vulnerabilities know anything and would determine the timeline for remediation what do we do and then so we need to take the information that we got and present it to those responsible for a mediation so this time has about half a

dozen administrative teams to address vulnerabilities at different levels and we needed to find the best way to get this information to them we limited the vulnerabilities presented to ten thirteen per month some of them had lasts less than ten some of them had chosen 60 so we needed a way to condense that down to make it easier for them to address and not try and kill themselves trying to address them all at one time so to get to the extent we would start with an expose report super helpful I topically bottle deviation with details and we've combined these reports with other sources and create two documents now the person was a patient and a spreadsheet overview give them an at a

glance view of that team and all my abilities I found that was helpful for them just to kind of get at first glance this accompanied to do all right the next one was a word doc so we laid out all the information that was in the first document but got down into the weeds about the sea bees and the suggester remediations and basically all the information that they would need to address it we wanted to give it to them in this document all right now that resembled the reports we scheduled OTV so to be dispersed in vermont we would meet go over the documents and we would involve the administrative team and their manager and director we found that the higher up

would win program was for potty treatments and so then we would have a follow-up meeting two weeks later and I can't go over those two documents TEENhood updated put their planned course of action in their response document we found that there were four possible responses they could give us not applicable to our organization you throw it out move on now a false positive can be disregarded applicable can be remediated within 90 days for critical or high and 90 days for medium and finally applicable but cannot be remediated in the creative center time frame order so when we got to these we would do a risk acceptance plan so why don't we do a risk acceptance plan the

ideal with the risk acceptance plan is for like all information necessary to senior management so that they could sign off on it the i-x Department does not accept for instance the is Department does not accept risk more time the is Department decided to separate that's all senior management to do so at no point should you say shrug your shoulders say yeah I guess we're okay with that one no you got to run it up through senior man nights I have it inside out so what constitutes senior management a good rule of thumb here is the person most likely to get fired if it goes sideways so what goes into risk acceptance plan so we have all their documentation and

we have our descriptions of the vulnerability so that includes the CBE's suggests remediation what could be done to fix it your platforms your scope how many devices were affected and what platforms so operating system and also what applications would be affected here and you basically know the layout for worst-case scenario manager and after all the bad then you can lay out the communicated controls that includes your audience a feels with our walls your AV software network segmentation physical access controls in my personal favorite listing you don't have application when you look into it so let's talk about remediation so the network kept under 30 thought and final component remediation verification ensuring that remediation is completed so we discovered

vulnerability and our mediations been reported by the administrative team that's fantastic that's what we were looking for so in these cases it's up to the security team or whoever's facilitating to obtain evidence so what constitutes evidence if you found the vulnerability with a thorn ability scan then you can do free run the scan if they say they fixed it easy enough no problem if it was discovered outside your goal of ulnar abilities it makes it a little bit more difficult but if you don't have any proof that you're vulnerable so how do you get proof that you're not so patch level screenshots help a lot here as well as vendor reports so that's like your Microsoft

MBS a Microsoft baseline security analyzer also this client uses f5 so load balancing software and they have their own tool where you can export the configuration into this and it will spit out here's all the packages that are missing here's your configuration is used that was helpful to us as well so we wanted to make sure that remediation could be verified by any means available let's take a look at a few examples and one instance next pose notified us that many windows devices were missing the critical facts not surprising said the administrative team acknowledged said they were working on it but a while later they confirmed that remediation was complete next folks can confirm this there's a

problem there were some devices that weren't being scanned they get stood up more devices in a different subnet that wasn't being scanned and it was partly on them for not communicating but partly on that's what I do on our discovery scams which we do now and another case vulnerability was discovered outside of next better Soho another animal the administrator team was notified in their response was not applicable we're not using the configuration that would be vulnerable to this however we discovered later that it was very much little and this could have been prevented by having multiple people check it because the first time we had one guy checking it the second time we had a different guy and you know

having them work together from the outset what are prevented us from missing this so now what we've discovered my experience here undergo some recommendations for anybody wanting to spend up for mobility management in your organization first of all bring in a consulting firm to perform an initial assessment based on the sands top 20 in critical security control we talked about a minute ago these items include number one your hardware inventory so inventory of unauthorized and unauthorized devices number two inventory of all not authorized and unauthorized software and number four continuous vulnerability assessment and remediation oh three of those we talked about already a second recommendation stand up your vulnerability scanner is compatible with most if not all

seriously doubt that the vulnerability scanner will be compatible with all devices in your environment your discovery scan first like I talked about earlier and it's recommended for your first pass cause it's minimally invasive and relatively quick that would take some time to complete but an important discovery scan will be worth it to get an understanding of what assets are out there and then using this information you can do your credential scams where you get more information about the devices based on the size of your organization you want to break these down by IP range or a geographical location organization unit or even asset type next recommendation break the Vice hardening Sanders the CIS benchmarks are a good way to do this and

they provide specific guidance for a hardening assets based the operating system is running and these provided for a wide array of devices so Linux IBM Microsoft operating systems so every flavor of Linux or most flavors of Linux every flavor of Microsoft and also I oh s switch firewall around and also function devices as well some vulnerability scanners like next Bo's and necess can run a specific type of credential scan against the device and can come back with a report that details pass or fail for each item Nessus has this cool all kind of capability where you can import a config file from an iOS device and then we'll come back with a report showing you the

pass/fail for each device there credibly beneficial for evaluating security of whatever device to that you can evaluate next recommendation hire a professional penetration testing firm if you need one I know one the focus here should be on externally available assets anything that may be a if any way that is part of your compliance process and anything with sensitive data I have these recommendations are a little bit too resource and sensitive for your organization consider software as a service and these are great because they don't require infrastructure investments and you can focus your time on using the application instead of deployment or administering it or maintaining everything so our key concept here the key to information security program

especially vulnerability management is to build it around goals and strategies not a tool not a set of tools but a mobility scanner is incredibly insightful if you've never used one whose pain went up their skin with it you understand what I'm talking about but introduced under donor ability scanners often miss things I want to say open I mean always the u.s. cert and MPD that we talked about a minute ago there's super robust give you tons of information but keep possibly tell you everything you need to know about your organization bend your notifications equally valuable but they can't account for existing configuration issues so ones approach to vulnerability management must be comprehensive cover as many areas as possible concerted all

teams working together towards the goal of a mobility mitigation customized specific to your organization and here's where the invested scoring system helps also consistent the same for all teams all owner abilities and all circumstances if you do decide to undertake this there would be times when you say you kind of shrug your shoulders and want to say oh that's close enough or that's not something we need to worry about but the more consistent and regular your process the higher your likelihood of ultimate success also changeable so initially constant changes will be required to your program like you've heard the phrase no battle plan survives first contact with the enemy vulnerability management very much like that and inevitably your plan is going

to require modifications kind of first attempts so in conclusion vulnerability management could be a powerful tool for reshaping a nice department engaging in a cyclical process of identifying vulnerabilities and notifying persons responsible for addressing them and tracking for mediation to completion has the potential to vastly improve organizations any question cool listed there I appreciate you guys inviting me to speak