Operationally Focused Pen Testing

next up is Greg Anderson on operationally focused pen testing thank you all right so definitely looks like the audience is thinned out after lunch so how many people in the audience are either pen testers internally for their organization or performant as an external party yeah and my entire team got it all right what about procures of pen test we have any of those the audience people who are going to all right not a whole lot i'm surprised all right so this talk is going to kind of help you guys work together i hope it's kind of got a dual audience to it i'm hoping a lot of this information can be used by both parties pen tester side to

make you better pen testers help you deliver better products as well as on the consumer side to help you expect more from your pen testers so a little bit about me right now I work for certs risk and vulnerability assessment team the team's job is performing pen tests for Department of Homeland Security federal local some state and other municipalities previously to that I've been in IT Help Desk since pretty much high school done network administration domain administration security analysis and previously a little bit of GRC work to in the nuclear industry it's my follow me information if you'd like to know more so we're gonna talk about pen testing a little bit in here but why pen

testing right so we have pretty much a decade of high profile data breaches and at least the last five years we've seen a lot more of data breaches which has caused this real increased observation awareness of cybersecurity a lot from the general public side even more so than the IT side as normal what we've got now is governance framework starting to address cyber Sarah pentesting it a little bit of a high level they're at least mentioning it now regulations and compliance are starting to demand more there's been a lot of reference to PCI there at least the regulations that are saying adieu pentesting on a regular basis so that's at least a good thing and then even mass

media coverage it's like less some of those last few data breaches especially target you've actually seen people in the news talking about well when was the last time they had a pen test done so it's really kind of interesting to see that kind of progression towards pen testing in just you know general public but why pen testing who really cares why is it important though current risk analysis methods are a little bit subjective believe it or not so we have things like probability likelihood impact and even threat actors some of these things can be subjective for the people to perform the ascent forming the risk analysis their biases can interfere with it and in other times it can be

downright volatile so you might not consider hacktivism in your risk analysis mechanism but then your CEO make some statement on the news and you've offended you know all sorts of people that now want to target your organization from a hacking standpoint that was never considered in your risk management profile before and even on the worst side of that sometimes you can deliberately alter your risk analysis formulas and those sorts of things by skewing the inputs to reflect the output that you want so long story short objective risk analysis takes very disciplined and mature organization to do it effectively and one of the things that pen testing can do is help you validate these conclusions that you've

made about your risk though ongoing cycles of performing your risk analysis validating with pen testing addressing your risk analysis going back to pen testing again and help fine-tune this maturity over time on the other side demand has created sort of this low supply of quality pen tests that are out there in the world today we see a lot of things like just validated vulnerability scans passing for compliance mechanisms yep check the box we did our pen test in other cases we see scope being so tightly constrained that it doesn't reflect reality at all but again you get that compliance check box this does happen a lot in pc i don't mean to beat on PC I know it's been on a few slides

here but a lot of times you'll see well that's not our prick payment processing systems so we're not going to address that during a pen test but in reality it does provide an attack path into your payment processing environment so it should be addressed another thing here is their technology focus as opposed to business focus we always forget as technology professionals that technology only lives in an organization to serve the business so we need to address these things from the business side as well not just patch this fix this and there performed in a vacuum right it's just a pen test it's a report it goes into someone's file somebody might look at it someday but there's really no

interaction with the rest of the business on a lot of these things though the end results drastically decreasing the business value of the pen test there is generally a negative perception especially in compliance space we have this really high cost service that we're providing and nobody really cares it's the same cost as the other check boxes in our compliance process and a lot of times there's this adversarial nature with the pen test right you're coming in to test our ability to run do our jobs is the way it's viewed a lot of times and at the end of the day we're just not meeting the expectations on the previous slide of what we intended for a pen test

to be so a bottom line there's a bit of a disconnect right we have what we expected pen test to be to help us validate risk but then we're not living up to those mechanisms right so where do we go from here how do we bridge a gap how do we reconnect our expectations as to what's actually being delivered but I think first and foremost we have to identify what we do well because as pen testers there's a lot of things that are done very well and we should keep doing those things i'm not saying anything's changed there but we have to have that self analysis to actually say okay we have areas that need improved so let's

look at the proving those and here's a kind of breakdown of some of the things that we do from if you look at kind of the pitas or other standards the breakdowns are the phases of the pen test so on the left we kind of our good side which is pretty much we're going to see a lot of the technical stuff right recon on our Oh sent capabilities handing an enumeration our exploitation these are things we often focus our time on our own a professional development on we do really good at those things but on the other side of like our methodology we've got pre-engagement capabilities actually scoping these engagements out so they provide value to the business post

exploitation where we actually kind of go after the crown jewels we actually address what matters to the business from a technical standpoint and then actually conveying this information so it means something to the business so really the rest of this talk is going to kind of take a detour from the where most pen test talks go and really address the right side of this the methodology here where we're focusing on so are the soft skills that make pen testers good so bottom line pen testing is more of an art than a science there's a lot of things we have to fine-tune it's not just plugging into a formula or with a lot of automation stuff we've

been hearing I think someday we can automate pen testing but right now humans have to do it so we got to kind of fine tune those capabilities pen testers are great at technology we've always been good at tech and we always seek new ways like attending this conference to learn new things about technology at the same time though we need to spend the same amount of effort fine-tuning our business acumen that we actually do these things better for the business because again this is a business service not technical service though here's kind of how this is going to break down for the rest my agenda slides like 10 slides in here's what we're going to talk about for the rest

of the talk here so we're going to talk about sort of that pre-engagement phase that we just looked at we're going to talk about goal-oriented pen testing to help drive that along for the business we're going to look at what the objective of the pen test actually is and kind of refresh our view of that as well as look at how we report on some of these things let's talk beyond technologies to the business and look at who our audience actually is without further ado a little bit on goal oriented pen testing so this isn't a new idea I forget his name now I knew it when I wrote the slide deck but this is

a concept that's been out for a while I just haven't seen it got a whole lot of traction but goalie and goal oriented pen testing is really just define a an objective a goal for the pen testers that's measurable i used the smart slide here which I'm sure everybody hates because it's the same thing you see for your performance reviews making your goals you know specific measurable and actionable but there's some credit there's some good things in this right especially when we're looking at the pen testing standpoint because when organizations tell you oh we want you to come in and hack us okay what does that mean or we need you to fish us or to show us you

can exploit this it doesn't provide a whole lot of understanding where do we manage expectations what exactly do you want us to do so instead of those kind of high-level statements here are some examples of how we can actually leverage goal oriented goals in our pen testing though demonstrating the ability to exfiltrate PII from the HR systems from an external network attack within the confines of your two-week engagement that's fairly clear no it doesn't specifically go into a lot of the rules of engagement type things that you'll talk about but at least you have some type of focus now and at the end of the day this is going to help you manage a lot of your expectations in reporting

another example here compromise the domain admin credentials within a 30-day kind of reconnaissance and fishing campaign there's your scope down to something you understand no what we can do kind of takes which the boundaries around what's expected and then again we kind of a last one here just gaining undetected access to a secured network this case an ICS network within some red team engagement that takes a longer period of time kind of under underscoring the fact that you know we want this to be somewhat stealthy do those sorts of things over your 90-day engagement helps bring everything together so what do we do to get there this is all well and good but how do we

actually define it so again this is pre-engagement this is talking about your scope with your business executives or the at least your points of contact within the organization and you're really going to try and focus on identify why they're looking to perform this pen test in the first place right there's a reason they're somewhere and it may require some fishing not that kind we want to actually just kind of kind of sounds weird going from technology people but havoc on versation with them try asking some open-ended questions not just yes/no questions I listed some examples here but there is no template for this what do you hope to gain out of it kind of get them thinking about the process

order your concerns one of the things I like in here is ask them about their strengths instead of their weaknesses right everyone loves talking about their strikes when's the last time you were on a job interview and I started telling you how the company sucked ask them about their strengths first and that's going to give you insight into where their weaknesses might be and then you can outright ask them after they're comfortable talking about it what are your weaknesses and then what are your security concerns this is often it's worded as what keeps you up at night I know a lot of CIOs that sleep very well so that might not be a valid argument

question to ask but why would this be important so we kind of talked about a little bit but it's going to give the pen tester a clear objective of what's being required from the pen test manages expectations you know what's supposed to happen the organization knows what they want we're going to meet in the middle and we're going to deliver exactly what's expected to the organization it's going to help them validate their risk so if you're having this conversation with them at the beginning during scope definition get a feel for what their risks are are they going to give you their risk analysis probably not but through some of those questions you can start to identify they're putting a lot

of value here we should look here or security vulnerabilities and then use that in your report so they can use it later to validate their own risk conclusions and if nothing else it's going to make you reporting a lot easier pen testers hate reporting I hate reporting if you have a template already of what's being expected out of it right to the expectations your jobs almost done before you start especially for the executive summary portion of it all right this section redefining the objective so we want to look at the pen testers objective during the process here so I hear a lot of times talking about pen testing especially to the non-security specific audience mostly

more the IT operations personnel about a pen test being you know we we want you to come in and find all of our vulnerabilities nope not going to happen use a scanner scanners are much better at finding all your vulnerabilities than a human is going to be so the job of the pen tester is really to find a viable attack path I don't care if you have a hundred vulnerabilities if I find the one that gives me access to your organization that has some impact to your organization that's the one I'm going to use I don't care about the rest of all and then I want to use it to demonstrate the business impact so like I said once

you're in use it to go out and touch something right see what else is out there go after the crown jewels and then identify and measure or identify measures to eliminate or minimize that attack path though often these are technical attack paths obviously so how can we minimize that how can we make that attack surface smaller that way the organization's risk is ultimately lower and then reducing business impact is the end goal right if we can't eliminate the tack path how do we mitigate risk so of this analogy here or this list here most of the time these two things are addressed by pretty much every pentester right we get an attack path we get in

and we tell you how to fix those things that got us in what we tend to slack on is what's the business impact of what we did and how do we reduce the business impact though will address those two things here so in the post exploitation phase it's all about business impact Carlos Perez actually has a great website where is slow again for the website is actually shell is just the beginning I think that's a great analogy for this because it's it is it's the beginning now is where you can actually demonstrate what cert the service you're providing demonstrate what value it has the wrong answer here what's you know why does the attack path matter well we

got domain admin and high-fives and we're out you look really ignorant when you do this sort of thing because domain admin really doesn't resonate well with the executives in the organization they don't care okay great how does that matter to me um it's just it's just too broad wrap it around in context that actually resonates with you organization so instead you're going to want to think about okay we got domain admin credentials where do they work obviously in the domain what can we do with the credentials what impact does this have on the business of what system did you gain access to what value does that provide that could be you know your email systems domain controllers other

system HRAs systems anything where you can actually find valuable data from your scoping conversations that might allow them to understand that risk a little better at the end of the day how do we measure it though right business impact means so many different things to so many different organizations but one we often rely on is dollars and cents easy we're going to steal money from you or you're going to get a bad reputation etc etc this is losing ground I think though I now to talk about this yesterday a little bit where we're seeing the stock market trend differ from way it used to write so target home depot or some examples of this where

yeah they took a huge hit financially from stock trading after their breeches but then it rebounded not that long after that far beyond where it had ever been so if you're talking to the CFO about well you're going to get bad press from this they're gonna say who cares it's temporary we're going to rebound customers are still going to come here bye-bye so if you want to use dollars put it in the context of their Incident Response their capabilities that one time fee that's gone so I just read a statistic recently on fishing costing just fishing that we're not talking data breach costing organizations roughly 3.7 million dollars a year so use that number instead of focusing on their

stock market performance right and that 3.7 million is their Incident Response their lost productivity their containment through his recovery and all those sorts of things that number is going to resonate well with the organization versus you're going to get a bad reputation that kind of ties into production so if you know lost productivity some other things are up time so if you're or uh yeah uptime or if you think of like amazon shipping organizations those things if their control systems that you may have compromised during a pen test go down they can't ship things anymore that's a big deal those are things that need to be looked at one that's getting actually increased popularity search

engine optimization very valuable to people you go into some organizations and tell them that new ciphers are going to increase their search engine hits they're going to get placed higher on that issues fixed already so put it in context that they understand learn the business don't just assume you know everything because you hacked in and relate the technical findings that you have because they are very valuable we're late on to what matters to the business though get in their heads a little bit why is it important I guess we kind of touched some of this so it gives a lot more credibility your technical findings because a lot of times they don't understand what you're

talking about I was listening to the crypto talk going I don't understand any of this but I'm sure it's great in creates a driver for remediation right if you can put this in terms that they can understand and they can use now they've got a little fire lit under them to actually get these things taken care of rather than a report just sitting on someone's desk somewhere so our last section here was actually two in this one a reporting face so we often think about technology so much and this is a quote that I put in here specifically for who it came from and the organization that I thought was valuable but too many think that the solution to

most problems is a technology control rather than people in process and I thought this is actually came from a like a workforce development article talking about you know people fresh out of college always want to use technology solutions for everything but the fact that it came from the VP of core core security who makes you know you can criticize it all you want there are flaws in it but one of the most powerful pen testing platforms on the market so they do some of their own vulnerability analysis the fact that this pen testing VP is fun saying that you know there's people and processes involved in this too that kind of resonates with exactly this topic here where it's not just

about technologies that goes right down the pathway of we've kind of in pigeon holed as this technology service but in the end that's kind of our fault we fo is so heavily on technologies we have to get better at that kind of business side of things as well and these technical findings that we have are often linked to larger problems their symptoms right you went into the doctors offices that I feel sick he's like well yeah you've got a cough yeah you're probably not going to go to that doctor anymore you want to know why you have a cough all right these are drivers there's human driven processes underneath these things that actually cause the symptoms that you're

finding in a pen test so we actually need to become familiar with these understand the operational deficiencies that actually lead to these findings so as an example right so a pen tester takes advantage of some telnet varner ability the system happens to be listening on telnet and ssh our tip end up and the vulnerabilities two months old that's an important part so our typical recommendation is going to be well there's a vulnerability there's you should apply the patch and you've got two redundant services listening so disabled one if you're not using it that's great that fixes that one problem but it only addresses that one problem there's probably other systems in their environment that you didn't compromise

that have this same problem so you're not looking at why this problem exists in the first place a better way to do this would be well of course apply the mission patch right you need to do that however do that soon and then look at why was that there in the first place let's look at your patch management program maybe you should reduce your patch window down to something 30 to 45 days you're not having two month old patches that are out there in the wild they're missing patches that are out there in the wild and then maybe in another 90 to 60 days you probably have some configuration management issues or you're not reviewing these things if you

have both telnet and ssh on in your environment and then later to a year mr. 90 days to a year establish a baseline configurations do some analysis on those periodically check for Veet deviations to the baseline and then use your instant response program to respond to those if any deviations occur writing this roadmap of services like this or ideas like this is drastically going to improve the value that you're providing as a pen tester

but it's difficult right we're technical people we don't always understand the operational side of things in organization so how do we get there well I like to think of it in the context of you're all smart people what would have stopped your hack right if they would have patched this two months ago or a month before you got there that would have stopped your hack they're thinking in those kind of concepts of what human actions may have gotten in the way of me moving forward in my attack path and then how do you make the full recommendation well there's tools out there to do this though the critical security controls are very accessible to a technical audience they're all

tactical controls and there's mappings of those things that exist to the more operational controls like NIST in ISO so if you can digest the technical critical controls and understand those all oh the mappings to what operational controls in an organization would address these technical findings and then kind of make the full pathway for them so go from your technical critical controls to the nist 853 your ISO standards and then write that up into your recommendation so that way you have kind of this full path of the environment you understand being heavily technical as a pen tester to something that the organization can understand operationally and actually implement that will address broader broader vulnerabilities than just the

technology had why is this important so we're addressing beyond that superficial patch right patching this system there's 20 other systems out there that need patched as well it provides them a roadmap so instead of just saying here's a list of things you can do here's a prioritized list of things that are going to provide you ignore value over the long run than just applying these patches that we found and it's going to provide them a way to address vulnerabilities that you didn't find that they didn't know they had or that aren't even published vulnerabilities yet right you can stop zero days that's an amazing idea right good cybersecurity hygiene this is allows you to do can

stop zere it is and you're going to identify more than just that preventive control so often we think in fix it prevent it from ever happening we can actually identify those detective and response mechanisms that can go forward into helping them have a better security posture at the end of the day it's going to significantly increase the business value you're providing so if nothing else you might get paid more to pen tester all right speaking to your audience so this is kind of a big one so this is my opinion so who read your pen test report right executives the managers of staff who actually implement stuff and then that technical staff who actually goes and implements things each

of these audiences need something specific they're different out of their pen test or out of that report however most pen testers right there reports as if they're being read by other pen testers they want to look cool and that's fine no that stuff's fun but at the end of the day that's not who's reading your report in most cases we have kind of the three audience here so we have executives so the reports section that they get makes sense would be the executive summary their primary concern that you need to communicate in the executive summary is going to be the business impact and how do we move forward right what impact does it have on us where do we go from here one

approach Express business impact in terms of matter to the business so like we talked about dollars and cents if it's appropriate supplies production up time SEO whatever the case may be that's going to be different for every business then briefly summarize your findings in terms of operational and management controls not hatches right you should not read about ms 08 06 seven and an executive summary your findings should be tactical they should be here and now right this is what we found completely factual help relate the findings to that goal of the pen test like we talked about previously so if your goal was very specific how did your findings how did your attack path relate to that goal

I'm trying to keep the Garvin jargon and CBE SMS numbers like I mentioned those should out of the executive summary those are for later and focus on remediation right we want to address talk about the findings but we want to focus on moving forward like I said so focus on remediation something strategic something that they can really grasp a hold of and start holding people below them accountable for address the big picture talk about programs not systems if in the next section so managers our staff is your next audience their primary section of the report should be the main body to report that should be theirs should have a nice flow to it shouldn't be interrupted by plethora of

tables and data and outputs just main body that's concise primary concern there is going to be remediation right again we want to know what's broken but more importantly they want to know how to fix it summarize your IDs you're finding IDs title severity etc they need to know that information but they don't need you know a book about it address critical and high priority findings first right that's what you're going to do even if it's just patch it right that needs addressed first if it's critical then start looking at quickly and operational stuff so like we had on the previous example start looking at maybe reducing your patch window if that's appropriate start evaluating for changes

looking at your configuration management program to see if it can be approved those quick win type things then long-term you do have strategic type goals in there for them so incident response to changes in your configurations that way if anything does change oh you have a separate another port to enable now go in there and identify that through your instant response program then your technical staff so this is again this is a section where pen testers tend to do really well at these should be in your appendix though they should not be part of your main report these are something that could be torn off your report and then hand it out to the people who actually

need to do things technical primary concern technical findings no surprise there you should have the detailed attack narrative in here student shouldn't just be a list of findings though some context around what you did why you did it in how it impacted them things like the tools you use the commands you use the host you targeted where'd you gain a foothold where did you move to after that something that's kind of nice to see that I've seen in some pen test quartz is the validation this is how we actually found the vulnerability we exploited then if they come back to it later down the road they fix it they patch it they can use that string to

actually validate it on themselves that it's been fixed and in other important things give credit where credit's due there's a lot of things that slow you down during a pen test so you have credit don't make it just all the bad stuff about them it's really going to help that relationship if you're actually saying good things about them in a report and your detailed list of findings I'm not going to tell you what you need to include in that but Taylor your severity descriptions or recommendations to the audience don't just make it copy and paste that's fine as a basis but a critical vulnerability on one system might not be critical on another system it just depends on what

it's performing so tailor it to the people you're actually talking to in the system's you've targeted and then again the tools commands method you've used to actually use the fight to find that why is it important who really cares it's just a report though it's the product deliverable this is the proof you actually did something during your pen test and you didn't just sit there for two weeks and each audience gets much greater usability out of it right if your executives have something to hold your managers accountable for and your managers have something that they're looking their technical staff to do and it's concisely written they're all getting value out of it and they can use

it going forward again adds value to the business and you as a pen tester right they're getting value on it out of how they can use it you're getting value out of it could you performed a better service you can increase your billing rates if you'd like to so some key takeaways from this before the pen testers in the audience security transcends technology it's not just about the exploits there's actually a good conversation yesterday about the the 12 months or something before somebody who said they've actually fired an exploit it's all about usage the operational processes they're using the same passwords those sorts of things in an organization broaden your skill set to understand the business there are so

many other aspects to how a business runs in the technology they run on at the end it's going to make you a better defend tester right if you're thinking about these operational aspects what human things can do to get in the way of your engagement and is positioned you as a partner instead of a supplier of just a service right you're there to help them you're there to collaborate with them either understand their problems to help them get through it and it helps them long past the engagement right it's not just fixed this and we're going to go away if you provide them that roadmap you provide them as operational ideas they're going to have more things digest

for a longer period of time to help them over the long run and at the end of the day they're adversarial nature that we talked about you're coming in to tell them all these things are doing wrong nobody for their to partner with them to collaborate with them going to start breaking down the adversarial nature of the engagement and again make you more of a partner from the business side so the consumers of pen tests so you can kind of change how you think about pen tests if you're thinking about it from just that technology standpoint think about it from the more operational standpoint if they're fine if your pen tester is finding certain things in your

organization why are those things there why are those technical things there you can provide you a lot of insight into your risk analysis that you've done in the past to help you validate what you've done is it accurate does it reflect reality and then demand more of your pen testers right if the customers are demanding pen tester is to provide more operational in sight well guess what pen testers are going to start providing more operational in sight right supply demand works ask for sample reports chop around until you get what you want from the pen testers or somebody that can do it for you but big thing here is just focus on the programs that you're doing and how these pandas

can actually advance those programs that is the end of my time so questions comments concerns thank you very much I appreciate being here you