Size Doesnt Matter Metrics and Other Four Letter Security Words

testing every everybody have a good conference so far whoo give it up to besides Philly crew I think they've done an excellent job and I've learned a lot while I'm obviously ER I learned about the AP arson at work I learned about how to hack myself how to ask for permission to hack I've learned about social engineering I learned about PowerShell and hacking outlook I've learned a lot more than I'm probably giving this presentation but thanks for coming to see me so today I'm gonna talk about size doesn't matter metrics and other four-letter security words and it's really just about a metrics program before we get started I want to make sure the disclaimer comes

out here everyone can hold hands or bow their head and say the prayer of the information security professional the information is presentation solely the opinion analysis the presenter no employer past or present endorses any of the information or opinions contained within and I give credit to my my buddy there Ian for making it a prayer so you should always say that before you present any information a little bit about myself 17 years in information technology with numerous roles I've done application development any mobile application developers keep your hand up if you've done it on Palm Pilots okay yeah before you had an app store and you can monetize everything that's when I was doing mobile application development I'd

have that done enterprise architecture and I've also an information security I'm a native son of Philadelphia and alum here at Drexel University Temple University's hold a cissp I'm a director in Philadelphia today just so you know a little bit about me my interests are beer poetry technology and football so if you do one of the social engineer me buy me a beer tell me a poem about a robotic football team that uh that is gonna reduce the number of head injuries in human football so that's really up to you guys it's a little bit about a little bit my agenda today it's really about you are here really identifying where you are in this space in terms of

where you are in your information security metric program and to start with and then once you identify that so where does that actually leave you what are you gonna do about it I'm gonna talk to you a little about size and why does not matter and then what you identify it doesn't matter what the important things are actually going and I'm gonna bring it all together so you guys can actually see an implementation in progress so you are here I remember going through the King of Prussia Mall if you got in nobody from Philadelphia King oppression was pretty big but I remember going there got the bus traveled all the way into to the mall

and I walked in and was so expansive I was on the third floor somewhere and I didn't know where I was I me and heard around so I finally got to one of those giant directories with a big board and had the arrow and the little dot said you are here where was that I mean I was just you know I'm on the third floor I'm suspended in the universe I have an XY and Z coordinate and yet I didn't know where I was and I think sometimes we write an information security program it feels like that you have this really you know I have all these blinky box tools is but you really can't identify or tell

this story so everybody comes up to me and says are we secure you know do you do we have a protect against ransomware yeah do you want to you know are you doing a good job yeah I am what did you do with the money Jim what did you do with the money because that's what they're really asking it in the day what have you done with all the money you bought every blinky box you you have all the great toys you have MSPs you have sock you have all these these cool things but they want to know what you did with the money and how I translate that is really do you have an effective

information security program and how are you telling that story so for purposes this presentation I'm gonna have a company called Sunoco we are the leading distributor of in Globe Christmas decorations in the United States starting since 2006 so everything I talk about is gonna be through the lens of the snow co company we're actually a leader in the southeast I don't know why that is but it's kind of strange I guess it's they don't get enough snow so when I got through this job in a snow Co they handed me the endpoint protection program they said hey Jim this is here you go you can do the endpoint protection for the organization they said look 100

percent cover you have on hundred percent of these endpoints covered this is a great job they just handed me the keys to the caddy and daddy's gonna go through it go for a drive he doesn't even you know just make sure you don't run anybody over or hit any any guardrails and you'll be okay so I started asking some of the engineer's questions I said hey where'd you get that number from and I said I don't know give me one give me a second they start typing away to talk about this number and they come back the next day and they said well the desktop engineering team gave it to us I said how they get it come back the

next day oh it's based on the software distribution package and it's the number of installs well is it accurate I don't know so I start peeling layer by layer back and 100% quickly became 50 percent this sound familiar to anybody so what's what was the problem here well installs right when you install something you still need to maintain it so just because you installed something doesn't mean it's still there there's people that uninstall it deal with Ella very privileges it uninstall it there were services that just weren't running I think there was a couple instances where it just stops starting altogether no one no one actually investigated but I don't really fault them because I mean what

ends up happening is you're doing your day job and you're on the hamster wheel task coming at you new projects new requests all day long and you don't have enough time to do it so a lot of people go out and buy a security event and incident management program you want just every log in the organization it was do everything you need to do but that's you know through one single pane of glass and it's information overload you don't need to answer every question you just need to answer the important ones is my security program effective and so sometimes it's like finding that proverbial needle in a needle stack watch your hands so I got it I got it I

got the answer the question Jim the answers big data that's what we need is big data if they answer your question is big data then the question is stupid I mean that facetiously but you're answering the question is my information security program effective okay so in order for you to prove that it's effective you need a team of data scientists a team of analysts of maybe some engineers and you cantí won't be able answer the question is my security program effective I no offense of data scientists are just expensive so I'm going to talk to you about size how big is your program how big is your program you know I don't know but yeah at

anybody have fans of ghostbusters not the new one oh well ghostbusters I had that scene with Ray and Egon where he says hey we got a big problem everybody goes how big Egon neiguan goes paraphrasing Egon says if you see this Twinkie here that's all the psychic enough to add all the psychokinetic energy and I in the New York area it would be 35 feet long and 600 pounds he added some context to it right so how big was it kind of get an idea of how big a 35 foot 65 foot Twinkie is but how big is the Twinkie compared to a thumbtack everybody anybody not have a Twinkie about this big but thumbtacks

but the size of your thumb right they call them thumbtacks but how do I know well I could use a ruler I can measure it but still it's Winky looks a little bit longer if you ask me what about 747 everyone knows the 747 is bigger than a Twinkie at least I hope so but what if it's a model huh let's measure it so I'm here to talk about size doesn't matter what's going to matter to your information security Metro program is value is providing context how big is it and now I'm going to talk a little bit about how to actually do that so you are here you're owning that space you know where you're

at now you have to actually put a plan together and it's pretty simple I offer two two phases this one put a plan together to execute on that plan rinse repeat always repeat so the first step here around the plan is just don't reinvent the wheel this is real about inventory I think every every talked about in from information security management is really just about getting that initial inventory to understand what you're actually protecting they begin with so I'll talk a little bit about what that means also about once you have an inventory how do you prioritize it if you have a million questions to answer or different questions your board want you to answer

how do you actually go in a prioritized manner so you could you could do the right thing and be more secure fast or more effective faster so before I get to this slide there there's one convention I forgot to call out the four-letter security words are actually any reference to any framework or outside resource I think a lot of time people get overwhelmed with them because just it's great yeah I have a framework now what do I do so I'd make some call outs to them but I don't specifically get into them because I think it's simpler than sometimes just using framework I think you could do it a lot quicker without a framework but I do make

reference to them here so don't reinvent the wheel we're talking about inventory inventory your applications what do they do inventory the infrastructure they sit on are they in on-prem in the cloud are they in the DMZ are they in in the country maybe there's regulations in another country have to worry about right having that inventory becomes a blueprint for your roadmap and then you have these users these people that are using the applications what do they do with it who owns the information right getting that getting that that information really becomes the foundation for your blueprint and you should spend a lot of time on this 4-letter security where call out here the CSC 20 used formerly

the sans 20 the top 10 20 critical controls number one is inventory you're authorizing all the authorized systems cybersecurity framework identify asset management phase number one simple stuff it's already out there you can use these tool sets because but now you are here you didn't rent the reinvent the wheel you are here you have an inventory now what'd he do prioritize and you can prioritize when they method you use I think this is pretty effective but data classification how sensitive is the information the organization as a restricted intellectual property EPA chai PII and what is it really location where is the information Linda clone but you want to you want to identify where these things are because

these are going to be inputs to how you go after the asset so you want to protect and then critical processes so you know how important is this information your business if it went away could you still operate and if you're in a business continuity role or disaster recovery role you could just put your recovery point objective in here and your recovery time objective and you know you already have that information if you don't this is a good place to get started if you start answer yourself these questions and say Jim I I don't know what you're talking about you know you could start there by identifying where the gaps are in your program and a lot of the 4-letter

security word call out here CSF covers it under respond and recover and the CSC 20 covers it under data protection so some of these things are already there but you don't have to do this I'm going to show you a little bit about how how I've done it in the past how I do it today I don't I don't go overboard with it so application inventory here it is an Excel spreadsheet I took my application name Salesforce I have an ERP undisclosed name SharePoint and some some weird FedEx postal application that's sitting in the corner of the data center no one knows what's happening but I have mole on here and I've classified them I put information

labels against them I've talked about where that information actually exists I talk about what happens when the information goes away how important that is my organization and I've assigned a priority label seems simple it should be I mean size doesn't matter it's an Excel spreadsheet with some information on it so for the sake of argument here I'm going to talk through my company that I used to work for Sunoco and these are the labels we've actually end up using so the labels actors I pick for for ranking we're a classification department and job level so under classification I have a 1 to 4 rating whether something's really inside the organization versus whether it's outside our restricted information from a

department perspective we've ranked at certain functions maybe a higher risk than others and job level and this is interesting because I put the VP's at the top but they usually are they have that they set the policies but a lot of times in real organizations they also the most accessed information than the most permit in terms of what they have access to which is scary sometimes but you can use whatever labels you guys want for this you want wider numbers bigger numbers I'm just using this qualitative analysis this isn't quantitative in nature there's no data support this is how you rank your information now you rank you're risking organization so I'll show you how rocks walks through pretty

simple so these are fictitious use cases here for snow CO and I've talked about when my rankings were you know they become numbers pretty quickly which they all flip over the one four to three ranking and then I multiply them together to get a an aggregated risk score of what I what I want to use my prioritization now you guys can use whatever you need but here tell me that Jane was in research director has access to compliment it got confidential information probably intellectual property so maybe for an Apple working on project Titan an open secret right but this is the priority this is the use case I want to go after now if you have

a lot of assets to maintain no priors Asian makes sense but if you have 50 assets on or assets maybe prioritization as make sense but what if there's hundred assets in order to put protection on them takes a lot longer right you have a hundred systems but they can't have zero down zero downtime so now prioritization still makes sense to say I'm gonna take these down first to put it on there or whatever protection scheme you're going to put on there just make sense so far it's pretty pretty straightforward so I shouldn't matter should be just common sense so now we're gonna talk about executing and I put it through these lenses here the first one is efficiency it's really how

well does a performance how well do I scale I bought my blinky box I bought my agent I did whatever and now I want to deploy that in my environment have you deployed it effectiveness how well does that control perform you put it in there what's it doing next to pieces are more of like a contextual more like painting the outlines of what you're trying to describe efficacy how well well does the control you're putting in place perform against others or having compensating controls potentially and then trend how things happen you know how are things marking over time it was this way today it's this way tomorrow this way the next day can use those snapshots in time to

really tell a story I'm gonna walk through each one of these give you a give you a little bit of an example so efficiency how was it perform I have a target of Y and I have X deployed so Jim I learned fractions in like third fourth grade man you came in here to touch me teach me fractions numerators and denominators no I came here to tell you it's simple size doesn't really matter here you know here's my snowcone one thousand over two thousand fifty percent deployed I've deployed it 50 percent in the organization but the point here is make sure you can go to bank with this you know make sure you can take this your CIO CSO CIA SOC

whoever it is make sure you find out that that information is valid make sure it trusted but verify right the mantra of information security professional so make sure you actually sit there and verify this number so you could take it to the bank put controls in place to make sure that if anything that water falls into that number is affected your numbers should reflect that as well now effectiveness how well does your control perform this is really the heart of it this is value how much value is the control you're putting a place providing and the way I look at value is really three three lenses may be it may be the fourth in there but it's really

differentiation howhow is this differentiated from anything else you're doing can I tell a story gonna answer a question right and these these vary by control type and a lot of times you know at the fore will ever security work all out here is there's consensus metrics out there but you're gonna get this from the vendors themselves I mean in reality when you buy something they're gonna sell you on number and you know what the value proposition is what number what dial to take a look at to do that and you can lean on the community as well there's people out there but we'll provide a lot of those effect efficient the effectiveness metrics out there

the ones I picked out just you know there's buckets and buckets these things but I said all right let's take a look at incidents per endpoint what's the velocity or how often is the service being utilized and utilization metric what's my mean time to remediate how fast when I find something am I actually taking care of it and then percent automated remediation I invested all this time and money in something but 70% of the time the team's going to happen to remediate the viruses themselves well that doesn't sound like I'm getting a lot of value right it could be good indicator all right efficacy this is probably the the more difficult or most difficult to capture when you're talking

about a metrics program this is how well does it perform versus alternates us versus them you know that them could be you know it could be competitors it could be industry metrics it could be another vendor know not a lot of Clearinghouse information on this but the example I'm going to use here is this around like maybe have a managed service and you want to take a look at your uh oops I have a definition here there you go simple simple math again size doesn't matter X is y is the vy x weigh a hunter and you're gonna get yourself something and i'll tell you how much you're better or worse than somebody else so in this case i'm using

that service example i have a service provider that's providing remediation for my endpoints when they find a virus and today we're doing it 30 oh the vendor we have does it thirty minutes to remediate the meantime to remediate that's great news new service provider says we could do it in 40 minutes while we're doing 25 percent better all right we'll stay put for right now but it's a good metric you know i thought about a little bit about this i mean there could be ways you could even you know say you want to compare your endpoints the event point a endpoint technology a and endpoint technology be honeypot yourself after you ER gets the if everything's

going on a per user per month basis from a service perspective do have from with one half from the other doesn't cost you anything different you could just sit there and on e pot yourself and say alright well this is a little bit better let's let's let's put all our coins in this basket so it becomes interesting when you start taking these metrics is you make better decisions and then trends metrics over time X of 0 x1 I've captured it today I've captor tomorrow I've done this I pour up really and changing over time it was this yesterday now it's this what's the difference changing over time percent was it a large rate Febby scared my logs went up

a thousand percent Oh should be that should scare people unless it's one and two logs you've done right so some of it provides a lot of more context in terms of the information you're looking at so now I'm gonna bring it all together this is I'm gonna build a dashboard based on some of these metrics I pulled together the first step of building a dashboard is really what story you want to tell what's the narrative the lens you're looking through if you're scientist maybe this is the hypothesis you have right you're gonna have the scientific method here's the question I wonder debunk so what store do you want to end up telling I'm gonna be pitching out to

my seaso and I want to tell them I want to tell them helmet the highest covered risk population how much coverage I have there I wanna talk about overall coverage because everyone's gonna ask that question and I want to find out what the impact is to the remediation time so how long has it taken action remediated endpoints when you get in these situations so a friend put coverage I took the Karen coverage what it is today what it's been over three months and I've taken a look at the change since I've started so here at Sunoco I've taken this job now I'm telling a story about how well I perform since you've hired me same thing for the

meantime to remediate here's my caring time my trend and how much it's changed since I start size doesn't matter just simple metrics here so now with this is the part where I'm gonna actually pitch you like you're my see so so this is my dashboard I'm pitching out my best foot forward well mr. C so uh since you've hired me 100 percent my finance users have endpoint protection we've identified those as the highest risk population and nothing to worry about we've identified we validated that information we made sure that endpoint is up to date running if it goes out of bounds of coat and reinstalls we are covered no problem now in the process because we have such a

low population 99 92% of the endpoints have been protected and over three months we've increased by ninety eight point eight percent 988 extra endpoints we've done we've done a I feel with an excellent job and while we were doing that we had a thirty time mean time to remediate and well three months ago is twenty minutes so she decreased our ability the remediate viruses but I can tell you the story is I've dedicated all our resources to make sure those endpoints were protected so when viruses did happen we're just gonna respond on this fast seems like a pretty straightforward story make sense you guys so when to start now this is the only ancient Chinese proverb the

best time to plant a tree was 20 years ago second best time today just start today if you're really mature in your program you know take a look at it again yeah if you're at the planned stage you know get your inventory your assets if you're at the prior taste stage just you know do it and if you if you need to execute you know if you've got one domain maybe it's endpoint protection maybe one take a look at I think a lot of people have their vulnerability programs take a look at that next talk about what a vulnerability means what it means to have a host and a DMZ that exposed the risk you can start today and

then rinse repeat so wrap it up you are here make sure you do a reality check identify where you are in that space-time put some context around what you're doing make sure you understand where you are own up to it baseline yourself size doesn't matter value does context does you don't need the next whiz-bang sim next gen data protection threat analytics platform in the cloud that's gonna solve your organization just answer a couple questions I'm in the Clio is my security program effective what have I done with the money answer that question and make sure you tell your story no one's gonna tell it for you I know that no one's gonna tell your stories and make sure

you tell a story when someone asks you how well you're doing good no increase things by ninety eight point eight percent since I've been here I'm rocking it that's my story but also knowing when things aren't going so well and there's anybody noticed that I my four-letter security words none of them are four letters cuz size doesn't matter huh all right with that all open up to any questions comments concerns do you guys have any metrics programs in your organizations that are working not working metrics that are tough to communicate tough to solve any stories you haven't told or ones ones you have told no one's been happy with them all right well if you do

have questions hit me up on Twitter talking after this but thanks I really appreciate it oh yeah question that's that's a good question I think the question if you didn't hear it is how do you it when when you're creating a program how do you actually go and figure out what the right thing is anyway after I think by doing the inventory so I I normally would start with the application and finding out where the data lies so that planned step becomes key because everything else pivots off it so we'll use vulnerabilities vulnerabilities as a as an example in my career I remember the one time I create I had that big program that big report that says you have eight

million vulnerabilities and none of them impassioned forever and everyone just looked at me like so when I took it back a little bit I said okay well what matters let me omit to it let me take a look at the applications and the infrastructure that sits on so I was able to say ok let's do it this way how many applications have information sit in DMZ highest-risk how many have financial information how many have PII on them so I start carving up those attributes on it so when I when I came back and it wasn't about what they wanted to hear when I said hey you give me this report it means nothing to me what do I do I said all right well

I'll tell you what you know that PCI compliance where I was trying to get well these systems over here are unpatch I don't deal in Boehner abilities I deal in hosts I don't done you know in the infrastructure you know you have 40,000 vulnerabilities but it's for servers you have for servers that deal with your credit card information that are at risk and that's how I explained it to them so I think that lens of that plan and identifying what attributes make sense so if you're in sales it's you know if you're dealing credit card PCI if you're in medical you know you're dealing with HIPAA so it's a pH I so I think it's

really about taking that your lens and then you have a vernacular to speak to people about but you're right they won't know they will not know it to say they'll give you zero advice so you have to kind of create that that LexA kind of vernacular to talking about it yeah when you do that plan Steptoe becomes like a network and information that we're not a real network with an information network you know who it is what they have access to what systems they use you know when they last use it right all of a sudden it's just like it becomes a heat map and comes from pointing right at you I'm scared of this

one and that's the one you can actually go after that use case anyway good question

yep yeah I've been involved in I don't know if I've been that successful but I think I'd like to be in my mind but um but yeah I think gaining the trust to begin with so once you say you're gonna do something that that just the first metric the the efficiency that's the first one to go after I gave you money would you do with it because a lot of times you know you buy a project you know you spend money at the end of the year now your next year you have to go implement it and they I've been had people breathe down my neck before going well I gave you the money to do would

you do with it I think just staying on that and building this trust to say here it is here it is here it is here it is in terms of translating that and getting more money it's a little trickier I know and I've been looking at this myself the National Association of directors as a NACD I don't know if anybody familiar with these resources but if you get the information from the NACD they should have a list of metrics that the board cares about and I haven't I think you have to pay that's probably I haven't got access to it yeah but if you get it access that information you could actually take that and say okay this is

this is that pivot I need in order to talk the way the board talks so it's probably trust between you and your seaso but see when your board your CEO is probably more like more the business language to really risk and risk if you have a mature risk program talk about an in plain language hey Suzi loses her laptop our reputation is is over any question that's a good point maybe never never go out without an ask at the end hey I've done XY and Z but you know what I could use I can use one more person Mikey's mom or you know I'd use this over here consultant for a little bit help me out any other

questions any other comments experience you guys have cool I appreciate you spending your last your last time with me thanks [Applause]