Does Patch Tuesday Really Matter Anymore?

this is going to be a history lesson for some of you and I'm looking just judge we gonna be the case for others it's gonna be a little trip down memory lane right so we're gonna talk about you know what is the order how do we get to Patch Tuesday why does this thing even exist for me to destroy it here today so number one if we look at vbscript and it's kind of funny because I I saw this post earlier today on Twitter's about you know a year 2000 was so great all these things that occurred you know as anything this sounds like yeah that shitload of vulnerabilities that were going on because of VB spur if you look

at these just show hands real quick as I go through this who remembers Melissa all right the I love you virus code read ninda why was vendor name Nimba named min min don't write admin backwards right plus you know all these occurred in the year 2001 and it was like one after another after another we were just getting hammered with this [ __ ] non-stop and guess who's getting the big black guy in all this yeah exactly it was Microsoft so Bill Gates came out with this memo in 2002 early 2002 oh yeah January and it was basically a cease and desist order to all of his developers stating that basically they were to go through and do a code review

no matter what the product was what product group owned it period no more development this one offer like six months all they were doing is security architecture and code review they were doing this [ __ ] that they should have been doing to begin with right secure coding so from there what ended up happening is this hodgepodge of various patching products feature packs solutions being brought in from Microsoft and they got really confusing really quick how many of you worked with SMS back in the day how many of you know what SMS stands for not short message systems management server was Microsoft's earlier version of SCCM how many of you guys use SCCM know what

it is okay you're gonna get some more exposure to that because that's kind of what a lot of this is but you know we had everything from the Windows Update web app critical update notification the SMS feature pack automatic updates the Windows Update agent and then finally Windows Server Update Services ooh and that was like in a two-year span how confused do you think that marketplace was so all of this ends up leading into Patch Tuesday in October 2003 where Microsoft said that basically were going to give some consistency because all this [ __ ] that was going on wasn't work and it gave the consumer the users the enterprises no structure no format how

many change control meetings do you guys like to attend my number on that is it right and we're going to talk about that later too because I think there's ways for you guys to get down to one a lot of them around patching so basically Microsoft brought this on now over the past 15 years now can you believe it's been 15 years that we've been asked this yeah over the past 15 years a few of the vendors have other vendors third-party vendors have also got on this right but not everyone and even of those that have kind of followed into Microsoft's you know Patch Tuesday some of them don't even do it on we basis some of them it's every two

months all of them is once a quarter some of it some of them it's by annually so even that gets a bit confusing right so 15 years later we still can't get this stuff right I told my wife I wouldn't cuss in this presentation because I'm on campus now this is all information gleaned from a an article that was just published what is it June so a couple of months ago you know the only 60 percent of organizations over the past two years have suffered a data breach and their statement is that it was basically for vulnerability which they had attached at 60% they hadn't passed yet they do the vulnerability was out there they knew it was applicable to

their environment they knew the criticality of that vulnerability and yet they still weren't passion half of those or another study from the pony bond Institute said that they were hit with one or more data breaches in the past two years you know half of the organizations and that I think if we build affirmative Ito there was like some 2,000 organized worldwide not even that 30% or 34% of those said they knew their systems were vulnerable to the attack prior to serving occurred in that big bridge so we have some horrible horrible statistics here coming from you talk to your organizations around the world so 86% of the vulnerabilities reports paying with patches last year so they

had these vulnerable systems they know that they're applicable to their environment there's patches for 86% of them again they're still not being applied you know all this is just saying we have this compounded problem here so clearly these zero days are really not as much of an issue as they were before right yeah zero days used to be that oh [ __ ] moment and you know we're in a firefighting mode immediately and we were going through this on a very fast clip recurring almost monthly with one vendor to the next but how many zero days have we seen this year it's been very very few right so what is the problem first we have to come to grips with the

facts the other 90% I like to call so I hate to disappoint you guys but Microsoft really is really is in fact Microsoft over the past 15 years has become very good at what they're doing they have the largest most complex in the Evolved sock in existence and if you want to argue that with me and start talking about fire Ari and some of these others I will put your [ __ ] and check like nobody's business there are billions of sensors that they are monitoring and aggregating data from every single month there's nobody who comes anywhere close to Microsoft's Intel when it comes to security threats and vulnerabilities that exist and it's not just their products they're looking

at everything so let's get into some of the statistics here so when we look at Bonner ability reports and this is basically taking from-- see details comm you can use miss minor whoever all the data is the same right it's just which feed you want to consume from but i I pulled all this from CDE details last night so this is all the data that exists in 2015 Microsoft accounted for 9% of the vulnerabilities that were reported by the top 50 vendors by NIST Mayer CD Telecom 2016 only 7% of all those vulnerabilities were attributed to Microsoft in 2017 last year 8% so in reality columns are in 9% to date for this year so in reality no more than 10%

of all those vulnerabilities reported have been attributed back to Microsoft and when I say Microsoft I mean anything that has a Microsoft name associated with that particular executable environment whatever so you know Windows 7 Windows 10 2016 2012 you know 2008 office Internet Explorer edge you name it if it has Microsoft's name associated no less than or no more than 10 percent so what does that mean when it comes to all of our patching efforts well most corporations today use wsus for patching right right show then how many of you guys have wsus to patch your Microsoft so it is a very capable solution I'm not going to take away from it at all it is you know for

SMEs small to medium-sized businesses a perfect thing you know essentially you just synchronize it with Windows Update comm you know hold on the updates approve which ones you want fire them out use your group policies to determine which endpoints you know get those policies and which patches and you're pretty much done right so it is a very capable solution for being able to bring all those updates inside the perimeter and being able to control the flow distribution of it right so it works it's a whole lot better than that hodgepodge of [ __ ] that I showed you earlier now for the traditional enterprise and how they're looking that patch management it's a lot different

right because there's a lot of complexities that go into distributing patches when it comes to having five thousand ten thousand a hundred thousand systems globally how many of you guys work for a large-scale corporation so you get what I'm talking about you know the pains this is kind of that whole oh

your windows inside our build ran into a problem and needs to restart

damn it I forgot to put my uh well I forgot to put my cell phone

of instances and you tell might like people once you build enough data in the RSA Archer applying an update from RSA is a huge undertaking and most people only do it once a year so it doesn't matter how many batches they come out yeah s AP is coming the same way agreed

another good point so what happened where'd I get pissed okay let's try this again and uh mother very clean okay okay so patch release cycles so what a lot of enterprise organizations go further this is actually part of one of the articles and I wrote for ice a journal for ice soccer it's one over the other but this graphic window security tips is one my old side stone hasn't touched a year but this graphic kind of shows what the typical enterprise has to go through on a monthly basis you know first we're going to have week one where where we have those update notification channels and that's really important for you guys making sure that as you're

expanding your year reach into third party vendors and starting to manage those software updates on the more aggressive cycle make sure you have appropriate communication channels established with those vendors whatever mechanism they use for alert notification to their customers it may be an email make sure that email is not going to a single individual make sure it's going to some kind of distribution list or ticketing system internally so that you have you don't have that single point of failure essentially some of o will have an RSS feed just a blog site whatever the case may be but make sure that you have a means of monitoring that communication channel from that vendor determine the implicit bility you know

that's when the first things that you have to do is the applicability and the criticality rating of each one of these see BES and knowing if it is applicable to your environment what the criticality the risk is to your environment right so how many of you guys have an inventory system within your environment for hardware software come on show of hands okay how many of you actually trust the information within that database okay just we're all on the same page here working from the same book and then it goes through a limited pilot and this is normally you know your virtual machine environments your test labs IT information security those kinds of folks people that can fix

the issue themselves at because birth then they go into pilot testing this is typically a very limited number of users with cross-referenced representation different business units because what anybody want to stop why do we use different business students different software they may even have different Hardware right I may have tough books out in the you know oil rigs off the coast right there may be some incompatibility because those tough books have different drivers and software that handle the keyboard and display right I have different hardware drivers for this MSI laptop with its keyboard with its display graphics etc yeah that may conflict with the particular update so make sure you have a good cross representation of

different business units and will have different applications and potentially hardware problems as well so that's tends to be about five to ten percent of the population if you have you know five thousand users and you're a farming you're probably going to have you know distribution to around 500 maybe 250 we three you're going to expand that to the larger user acceptance typically about 20 to 30 percent addressing any issues that are going to identify that maybe some issue with your own builds with the software configuration it may be an issue you have you have to take back to the vendor whether if that's Microsoft or someone else you know these things do happen and then lastly full-scale deployment

steady-state controls in place and then you simply allow the rinse and repeat right because by the time you get to this point guess what that's Tuesdays happen all over again you start this process it's why it's so difficult for most organizations to be able to stay on top of their passion because guess what do any of you have a main headcount in your environment an FTE responsible for nothing but patching Patrick period that owns your patching process I see two three he's like gosh we put that on top of there are other responsibilities right that's how most organizations operate large-scale you know enterprises yeah they do have dedicated patching teams evening you know because they're

not only patching Microsoft and third parties they're also having a patch internally developed applications as well so it's this constant struggle to stay ahead of the game in what happens when you have a zero-day that blows this [ __ ] out of the water yeah what happens when Intel takes a crap all over your environment so oh yeah so the bigger issue like we talked about earlier is we have this 90 percent concept right have I found that into your mind yet 90% of the vulnerabilities that exist in the world today have absolutely nothing to do with Microsoft so we have to fix that issue but we still have all this other stuff that we're having to manage on a

day to day basis right we can barely get through all of our Microsoft's before we're dealing with this now when I talked with organizations and I asked why are you not bringing your third party updates for all that other 90 percent into the same process flow I hear the same things over and over again and that's what you just solved there we don't have the time we don't have the resources and we don't have the right tools I get that I get it I've been in these ones I've been doing this [ __ ] with patch since 2003 when Microsoft did release a feature pack for SMS now I've been dealing with SMS in SCCM since 1998

yeah and my career took this shift into infoset starting back in 2003 looking at patch management frameworks for customers how were they going to bring that capability into their environment using the tool sets that they already have so one of the ways system center updates publisher now I didn't see a lot of hands go up when I asked the question earlier how many of you have SCCM and you're in bars there there are several hands out and I'm suspecting just because of roles and responsibilities some of you may not be aware that there is secm within your environment you may not be responsible for but most organizations today that are Microsoft's friendly shops that are I would say

above a thousand seats probably have a CCM as well it's almost free for you to bring into your environment now System Center updates publisher while it has that System Center brand it's part of the system's family that whole suite of solutions system center operations manager configuration manager virtual machine manager etc but this one it's really about extending and enhancing wsus which everybody's pretty much using right now what it does is basically brings a lot more flexibility and control by being able to integrate all that third-party updates so the stuff really fills that void but we're gonna get into some of the issues that I do have with them but you can see here some of the vendors

that are included in that so I've got an environment here and and I lose my mouse what happened so let me just bring up excuse me updates publisher real quick this product is essentially twofold so it can handle third-party partner updates for integration with wsus but it also allows you to patch your own in-house develop applications if you do have those dev teams as well so as long as you have any of the data around the application anything from being able to identify the registry the binaries the version of those binaries etc you can say don't look for based upon this applicability and then use this binary to update it and then use this information whether if it's from the

registry the file itself a directory structure or whatever to be able to say yes it has been patched now you can get all that integrated into wsus and be able to push it out the exact same way you would okay well you can also bring in catalogs from partners so if I click this URL I'm not online what's that with current branch yeah oh yeah yeah so we're getting Oh trust me we're gonna get to all that okay I did connect to that earlier that was actually a pretty slick process for guest Wi-Fi did you see that with Saint Mary's I normally never use guests with Wi-Fi abuse myself but anyhow long story there Wi-Fi when you register you have

to put in your mobile device number and then it sends you a text with your username and password that's me that's pretty slick I've never seen that before in all the corporations I visit so basically here's the catalogs that are available and scup does anybody see an issue with this my supporting software vendors sorry lots of vendors yeah I know give that man an ice-cream see you at 12 uh yeah we're talking Adobe Dell HP Fujitsu that's it right this product was released in 2011 it's had incremental updates and one major update not too long ago about well yet the number of vendors did not expand for whatever reason Microsoft has not been able to garner the attention of the

software industry and their partners to be able to publish their updates as a catalog for consumption and Scott but remember you can use these and just flip adobe you could probably make some traction with at least keep in reader an acrobat up-to-date all right if nothing else it's good for that be able to bring those updates in no HP Fujitsu you know being able to keep some of their driver BIOS updates those kinds of things so it doesn't serve our purpose but also for being able to work with your internally developed applications so this is one of the solutions that may work for you just keep that in mind that it is available to extend your existing wsus environment

so continuing on now enterprise he capable third-party patching you know this is where you end up getting into some of the vendor based solutions there is a cost you have scuff is free you can download it right now it will work with any W or Windows 10 system or server if you're working in that environment you do have to have the administration toolkit forward to install in here your desktop OS but it is free these are not so much but look at the number of vendors we have four vendors that are in the wsus collectors you know from the third party or from the Microsoft Park or solutions that are out there there are a hundred and fifty

plus they're pushing over 200 at this point it's a very competitive space so they're all competing because their catalogues that have the vendor updates that's their their secret sauce that's their goal right so the more vendors and applications that are in the catalog the more competitive they become and they get into this way tour so to speak with each other but because of the way that they work in some restrictions Microsoft has put on the wsus api's these updates are secured by nature because every update has to be signed before it gets published into wsus so it will use that's either self sign or CA provisions certificate to sign that update before it gets injected and this all happens

automatically behind the scenes as part of your set up most of them will also remove the scuff requirement now that's very recent only in the past year year and a half for most of these vendors before you were having to inject it into the scuff catalog and then it would update from there or it would actually use the api's from the scuff catalog to do that pass through integration into wsus that's no longer the case today so it does remove that requirement and gives you basically a solution that you just plug in to wsus or through wsus it will update your SCCM catalog so to your point both the scuff product by itself and these third-party vendor

solutions will natively integrate into SCCM regardless of what current branch version you're on so i test regularly with you know 1709 1811 whatever the case so for 70 yeah so you still let me let me table that one but yes i will answer your question yes but i'll show you in just now so some of the main players in this space you may want to take notes how actually i'm hoping we're able to publish all these to the d size site but avanti my former employer i started back when it was heat software heat and LANDesk both merged informed the Avanti Brennan and Landis owned the chaplet product heat on the patch link product so these two

fierce competitors ended up joining and basically the reason I put a bond to get the top of the list if you had these two fabulous products and not I'm not gonna talk [ __ ] about Shapland even though I was on the inside I did all the time man but yeah they were both really good solutions so what what a vodka did was put all the parts on the table kind of like Apollo 13 right said okay let's bring home this is what we get to work the best parts of both solutions put them together now it's called a baktun hatch I believe but that's what we're what Sarah this is formally the Secunia product Secunia it still got some of the

automation in there but it's a little bit more manual in regards to you have to kind of review and approve the updates but it gives you a lot of control flexibility they do some stuff that Avanti doesn't do as far as their own proprietary criticality ratings of the updates you know so that you can kind of prioritize your spend and focus based upon the criticality of the update from the vendor and the applicability and criticality to your environment so that's kind of a key if you need that kind of guidance in your patch management then you may want to look closer at what Sarah patch my pcs kind of bubbled up from almost the Soho small

office Home Office kind of user to SMB now they're getting into that mid market space so if you guys are like less than let's say 2500 users this may be a really good solution for you to look at in extremely cheap I'm talking a buck a seat right to be able to bring third-party patching into your wsus environment and using it to push those updates up manage engine I highly recommend against reason being and I love that we're in Senate owned a [Music] security consulting firm based here was doing a pen test for one of their customers that customer happened to be a managed engine user managed engine head well it was it has a

web front-end so through their land and expand you know escalate privileges and continue to give it to other systems they end up on this web server and start to do their pen testing on it and discover five honor abilities in the managed engine product that led to anti-authority system access across the board on all the men that dungeon clients in that environment I'm sorry but if you have five vulnerabilities and two of them essentially give you root access on every client agent sitting out there you do not belong in the security business I'm on camera saying that get out okay so SCCM third-party software updates from these centers you're going to get full catalog integration process

automation complete based on management and recording and that is what we're going to look at right now and I love this new feature in PowerPoint because I tend to do a lot of click through demos for stuff like this so that no matter what the environment I can survive just here in PowerPoint but this is kind of like a live click for scenario but did you see how I clicked in through that I love that that's a whole new section from this PowerPoint deck I just found that last week okay so once you've done the initial setup with whatever bender now I'm using the slides that I had for my heat software slash avanti days but

it's the exact same way now this is the SCCM environment it may be new for some of you guys but just to show the level of integration there's not a whole lot with the wsus as far as you know what you're able to do like I said it's good for the SMB space it will push those updates out group policy and you kind of said forget it approve the updates that you're done right you have a lot more control flexibility applies to reporting and capabilities inside the best CCM and if you're not looking at and you are kind of that environment you know I would even say 500 systems enough really look at deploying secm not only for software

updates you're going to see some of the other security relevant information here in just a minute but those of you that were questioning your software or hardware inventory I'm not saying that CCM is the best but they've been out this [ __ ] for 20 years and it's pretty damn solid if you keep your clients healthy and I'm not talking about the client agent because even it is self repairing now but your WMI repository your admin dollar shares you know those kinds of things then SCCM is a very solid solution so once you've done the initial configuration said which vendors you want to bring in which products you're interested in patching what's going to happen is it's going to

pull down those catalogs it's going to do some dump the information figure out what's in your environment already from their perspective what is it and at that point you can say ok yeah I want to start patching you Acrobat you know flash layer etc so at that point all this stuff starts showing up inside of the SCCM admin console right next to all of your other updates so you know as you scroll down there's Adobe Reader Apple washer iTunes now we're into Microsoft Update see you tell that by the name and the bulleted IDs you know they're obviously Microsoft so here we have the exact same you know interface I customers would be pushing out the typical Microsoft updates not

with all these other third parties and I'm talking like a hundred and fifty a hundred eighty different applications that you can bring in whether if it's to SCM or bringing it into wsus and just pushing it out from there so I can select a bunch of those updates give it a name I can use deployment templates inside of that CCM they'll still be applicable target my collection go through the rest of the deployment wizard I'm not gonna bore you guys with that now automatic deployment rules right good part of what we need to do is have scale we're talking about bringing in a hundred and fifty different applications he may only take about 12 15 of those yeah because they are the

most important to you and Russ assured you're 12 or 15 important ones are different than is right so that's why those but you can bring in automation to be able to help you adopt all these new versions and applications that you're going to patch but that doesn't mean you have to have right so how can we automate this SCCM makes that possible with automatic deployment rooms and you may choose to push out particular updates I'd talk to everyone in the environment you may want to only use automatic deployment rules for your pilot or maybe just for your phase one you know your IT and InfoSec users because it to break something they can fix it or push it out to your pilot

because at least you can automate that part of the patching process the rest of it you may want to have a little bit more hands-on control so here's some of those automatic deployment rules best buying when it was released at the laste departments flash player has been superseded or expired should be in there the vendors Adobe they go ahead and push it out and you'll see which collection you're going to push it to except for so I set up these automatic deployment rules based upon whatever selection criteria I want and as soon as those updates are released I've targeted a specific collection inside of SCCM for them now some of the other things that we can do is compliance baselines I'm a

huge fan of compliance baselines inside of SCCM I use them for everything from the operating system to match the hardening of Windows 10 to CIS or you know what have you whatever controls we need to and being able to have continuous monitoring of their compliance with that particular base on and making sure I'm controlling the drift I know what that image looks like on day one from either you know that gold image DVD or you know that task sequence in SACM whatever process I'm using I know on day one what it looks like but I have no clue on day 365 yeah unless I have a solid inventory in a way of being able to monitor the actual

compliance or the baseline compliance so here I have my baselines create a baseline now I'm actually going to use software inventory information to see if my systems are complying with that baseline or not select pad G software updates I'm gonna expand it you use Adobe Flash Player select whichever updates I want say okay now each one of those updates is a CI or configuration audio that goes into that configuration baseline I take that baseline target the systems I want to now I'm actively monitoring their compliance with those items and I'm done in like five seconds right I can assign administrative categories here I have my patch management security and that just plays out later on the reporting which

we're gonna get to you here very quickly we get through this bit so now I've still you know provided all my selection criteria which collection all desktop and server clients you see the date I can generate alerts if X number or X percentage of systems are not compliant by that's specific time frame and then how often I want to evaluate those systems for their compliance so all this is fine and dandy I need reporting I need visibility so I need that to happen two ways I need to see a particular endpoint maybe I have a compromised system I've taken an offline I need to you know start going to prove my process one of the things that I can

do is look at their compliance with that particular base on local right so here is the actual SCCM client agent properties I go into the configuration select that particular baseline that I targeted with those CIS and I say you report now I can look at that report locally and determine is that system apply with that particular baseline now this is just to look at a local system but there's some detail in years as I scroll down you can see the compliance information severity all the CDE data about that update which is really helpful if I selected coffee paste it into Google lookie there CD details monitor now I know exactly what's going on but I can also take that

particular CD ID plug it right into the console and now I can see all the relevant updates that are applicable to that CBE detail right so now I realize that it's not just Adobe that's impacted by that particular update but Google Chrome is also vulnerable would I have had that bought that Intel invisibility any other way so that's basically that now the built-in reports give you that bird's-eye view so we just looked at one system now what is that collection looked like that I targeted or my overall environment how secure is it based upon that update and as you can see here there's a Adobe update information now here's our Microsoft updates so I'm getting like a bird's-eye

view of the entire environment based upon that selection criteria so also some customized reports this is a community driven one that plugs right into SCCM now we get a lot more detail and that's pretty much it as far as the integration with secm any questions on that before we start to wrap up okay so wrapping up basically just kind of recap and where we're out what we've covered here so you know ultimately all these things matter because when it comes to software vulnerabilities and these updates that are being released unfortunately not every threat actor operates on Microsoft schedule therefore those 90% of applications out there that are being patched that vulnerabilities are being published they're not

necessarily happening on Patch Tuesday schedule so we have to be able to bring tools processes automation good thinking into that and be able to be much more reactive because we can't sit down or sit on our ass and wait until Patch Tuesday to be able to do something about it now all the automation controls that you just looked at the way that we're able to integrate into wsus in to SCCM think about how you write your next change control request and is it for that moment in time that particular update or can you theorized how you're going to standardize your patching approach using these tools and processes and get a change control that says you have a

blank check if it fits these parameters that you specified in this automatic deployment girl or this group policy for this particular update classification criticality etc you get a blank check from your change control board to let you move forward with automating as much of that process as possible that's what's going to help you address a higher number of bond abilities much more of that 90% without having so much of the overhead and expense for that management process so we talked about who the players are if you guys need help evaluating this obviously I'm here you know as a resource I have no problem answering questions if you want to do deep dive like a proof of concept we can

talk about that later so don't want to interrupt or delay Dan brother is next so don't change this channel don't go anywhere od Oh was there a change speed oh I'm sorry okay that's a diol thank you