Hacking the ROI: How to maximize your value from a Pentest

talked on hacking the ROI I mean how to derive the most possible business value from your penetration test and this kind of this talk really arose from the fact where as a penetration testing consultant I've been engaged with many clients and on both sides of the spectrum I've noticed efficiencies where their money wasn't getting the maximum impact it could have gotten from the test but first off a little bit about me I'm an attacking penetration consultant throughout their security I'm based out of Charlotte North Carolina and have a background as a full-stack developer before getting into security and I spend a lot of my free time developing various security tools that I later find out

that someone else has already made so they usually don't see the light of day but that's the much the dismay of my wife by the way but other than that I've got a little bit of an alphabet soup as well but let's move on let's talk about the ideal world for a penetration tester you have loose rules of engagement an amazing attack chain your cross the CDE is getting domain admins popping shells all over the place but at the end of the day does that really translate into value not necessarily because it really depends on why you were engaged and why the client is paying all this money for you to pull this off if you just give

them a just a cluster of a report it's not gonna really get that much value no matter how bad you have a number how many findings you actually identified so this really goes goes on to why I'm giving this talk and the first step of any engagement is going through scoping and identifying what your objective is and the first step of scoping is you've got to determine your motivation whenever you in engage in a penetration test the motivation should always be clear that is the first question that absolutely needs to be answered because otherwise you're basically just ordering like one pin test please and at the end of the day it's you may not get

what your actual objective was so it may be closing the gaps in existing very mature security program but it also may be for HIPAA or financial compliance and this is something that needs to be identified very early on within the blue team in the the client side as well as needs to be communicated to the red team that's being engaged and this will really help you identify your goals so your goals are that subset of the motivation of a couple of things that you really want to accomplish when you're performing an assessment and this will ensure that you are really getting that value the like motivation and goals are probably the two biggest things that

if you can identify you get the most bang for your buck but goals that may include testing a new product you're like a medical device company or you're rolling out a new piece of software into production or testing segmentation for PCI requirements or just in a retail environment so a lot of the times we see clients that don't have a clear motivation or clear goals going into a penetration test when they get the report it kind of put their hands on the air like all right what do we do now and that's something that I really want to see change in our industry of having really clearly defined motivations behind getting assessments and this kind

of translates into what are the actual needs of your security program some clients will come in and say hey I want a penetration test and we asked have you ever had a penetration test before and they're like nope this is the first one and that kind of raises some red flags because sometimes a vulnerability assessment or a simple scanning program would be get them the most value for their dollar because identifying your those root cause deficiencies in your environment will really help you kind of mature your security program to a point where having that manual testing will yield a lot more value to you but this kind of goes into the red team and blue

team responsibilities on this subject because as a blue team you need to understand what your program maturity is if you just started vulnerability scanning and just rolled out your patch management program you're probably still in the very early early phases of your program and may not be ready for a full-out penetration test maybe test parts your environment or just get a simple vulnerability assessment but as a red team it's really a responsibility to advise them and identify deficiencies in their program and if it's immature make those recommendations that will guide them in the right direction to really get what they need and not necessarily what they want because there are people in our industry as you all well know

that are gonna try to upsell you they're gonna try to sell you a lot of stuff that you don't need that will ultimately yield you with a bunch of paperwork that you don't really know how to decipher take action so this not falling victim to unnecessary purchases is a really a key tenant of this of knowing where you're at having a red team that will advise you on what you need as well as a great example of this is let's say you have an environment with 10,000 hosts couple different domains and you really want to get a penetration test of that environment well that's gonna be very very expensive so if you really look at

it you wanna identify the root cause deficiencies inside your environment and that may not require testing all 10,000 hosts that may require only testing a subset of your environment that falls under your entire security program that will help you get an idea of where you're failing and what is really causing your issues and so that you can identify that root cause and remediate it across your environment so that's probably one of the biggest things that we see is that clients don't necessarily understand that um testing taking sampling of their environments can sometimes be just as effective if not more effective than testing the entire environment but granted this is in a subset of cases because sometimes you

may need to test the entire environment and that's where your motivation and goals really come into play next I thought I really wanted to touch on this because I feel like communication as IT people we know that sometimes we are not the best communicators and it's one of those things that I feel that everyone could do a bit better job of making sure that information is disseminated across the team across blue and red it's really one of those things that I feel like could be improved the most upon and that really falls on to falls under like information about the environment when you're conducting these tests really knowing your environment knowing your sensitive assets um which include your

highest risk assets in your environment that basically will cause you the greatest business risk and from a risk assessment you should already know these and these should be communicated to your blue team members to your red team and so they can identify where they need to take it easy where they need to be careful also where if they compromised would cause you the most harm because of that way they will know what really matters to your business and what will show you like where the deficiencies in your security program are putting your business at risk furthermore ongoing issues in the environment or absolutely critical to be communicated across again your blue and red teams because these ongoing issues I can't

tell you how many times that me or a teammate has been blamed for an issue that has been ongoing an environment for the past year and then that causes the entire test to come to a halt we are currently billing at that time and that leads to wasted dollars on the client side so having this communicator having this information communicated to the red team will let them know where the environment is kind of a little finicky where it's soft and where they can really keep an eye out because this will kind of keep that that communication line open if these issues arise and finally the whole tenant of this is sharing is caring guys it's

really really important to share as much information as possible because the more information that is shared between the two and even internally the smoother all of these assessments are going to go and the more people are gonna learn the more they're gonna just overall just grow inside your organization and as a the red team that you're engaging and as we move on we're gonna go on to testing and this is the Optive required test in uniform by the way but on testing I'm gonna touch light on testing because uh it's all self-explanatory you know it's where we pop shells and do the magic but there are some things about testing that are not really not really touched on a

heavy enough in my opinion the infrastructure of your tests is critical um defining your methodology is one of the single biggest things I think a red team can do to improve the quality of their tests because any red teamer knows that they have their specialty you know whether it be wireless web applications networks it's something that that's their bread and butter and they're great at the others they may be they may be good or they may be mediocre at and that's something that we really need to identify as red teamers to increase the quality of our penetration tests as a client you should really ensure that your the red team that you're engaging they do have a defined methodology

because that will give you an idea of what's being done what you're paying for and what is the quality that you can expect from essentially the service that you're purchasing on whether it be your internal red team or an external Assessor finally having open communication channels within basically between the red team and the blue team during testing we'll just have the whole testing process go smoother because they can let people know of ongoing issues anything that arises during the test whether it be critical findings or stay host being down it's one of those things that time wasted is money wasted so anything that you can do to her to eliminate wasted time is is amazing so

I'm going to move on to the most important part of any test or sorry I got a little bit ahead of myself but we people forget that when they engage a red team that the red team is goes off and does their thing but they forget that the red team is actually they are the V attack experts coming in the the offensive consultants and these consultants they have a plethora of knowledge that your blue team may not necessarily be completely knowledgeable in so they're really here to help you know as red teamers we're here to help improve the security of the the environment and this is where questions come in like what controls can I

implement that will basically do to make your life the hardest and where to implement multi-factor off is is a good example of those as well as how can I get attract the assets and patches that really matter so that will really deter an attacker inside my environment as well as detection and in regards the PowerShell and finally what information security processes can I implement in my environment or what do you think is is deficient at this current point it's it's one of those things as I encounter a lot of clients will engage us and then they'll kind of set and forget and do their own thing and then forget that you are really engaging these people as

consultants inside your environment um even if they're an internal red team they are still getting a view of the environment that you don't necessarily get to see day to day if you're an engineer you you work on your specified work that is directed by your security manager if you're a security manager you're busy managing all of your resources but with that in mind now I'm gonna move to the most important part of any penetration test and that's reporting because what people tend to forget is that a penetration test be essentially what you're paying for is a really really expensive set of documents because at the end of the day this is what ghen is going to give you the idea

of your risk profile inside your environment this is really directed to the the red teamers writing the report in order to really make that report impactful you have to remember the motivation as you go through the report the motivation is will guide the entire the the tone the focus and pretty much everything in the report because that client is really trying to solve this one issue and if you don't really focus on that on the report and they have to spend time deciphering what you're trying to say and what really matters to them then you have not done your job and that is something that I see a lot is that what the client wants

is not necessarily prioritized in the report writing and that's something I think that a lot of people including myself could be better at next with the motivation in mind we got to remember the audience with any report the audience is going to be generally three groups you're gonna have your seaso your executive level your security manager and your engineer so let's consider the the executive level is really concerned about risk right they're concerned about their overall risk profile that their that their IT assets have to their business and what what did you do as well as what can they do to remediate this risk that you identified and a lot of the times this has to be succinctly

stated because they're gonna read probably the first couple pages in the report to really get an idea of what was done and then begin taking action and that's that action is going to probably be delegated to that security manager and that security manager is going to go on to read the the stuff that's more related to the root cause deficiencies and inside those environments what all the vulnerabilities were related to where their program is lacking if they their patch management program had fallen through on some assets it's uh it's one of those things that on people tend to forget that all three of these levels have different have different job responsibilities and they have different

resources available to them so with that in mind the security manager is going to look at the board abilities that came in on the impact that the testers had identified inside the environment and delegate that to his engineers and those engineers it's absolutely critical that they can go through the report and see what the vulnerabilities were they can find them inside the environment they can understand what those vulnerabilities are and then proceed to remediate them in a in a realistic way and that really is so critical to give them a solution that is that is actionable because a lot of the times there are there are solutions that you can recommend for a vulnerability that are just not feasible

inside the business whether it be a service being open to the Internet that's a business critical system they they cannot take that service off the internet that's and I know that all of y'all probably have experienced at one time or another it's like why do we have this here but at the end of the day it's all the business and that's the the key tenant of this talk is is that this business needs to generate value and sometimes you need to accept a high-risk vulnerability in order to keep business keep the business running and that's something that really needs to be kept in mind when you're providing these solutions on vulnerabilities and moving on this is something that I know that

testers get really excited um if any of y'all are a red team or any of y'all red teamers in here then you know it's like going through an environment and you're like popping shells you know it's just exactly you kind of get yeah you're doing that and you're at your keyboard it's it's amazing you know it's having having so much fun but the thing to keep in mind is that findings cost money everything you find is gonna cost the client or your organization money and that's why you really need to act and when you're presenting this to work whoever it be whether it be the executive leadership of the security manager um really kind of take put your

tone in the in the way that you're helping them and that you are optimistic about helping their security program going in there and saying yeah man you're totally effed is that maybe the case but you got to phrase it a little bit differently you know this is this is about it this is about being professional so you really gotta weigh that presentation in the report of giving them again actionable things that they can look at you like hey this is the top thing that you need to implement you know you need to implement multi-factor off on the parameter like first thing and giving them a prioritization and lending your expertise to the client is is absolutely

critical because this is what they're paying you for at the end of the day is to give them that perspective and moving on in terms of giving them a perspective it's it's one of those things that again I don't see many clients do and they definitely not enough do it is y'all know the old adage give a man a fish he eats for a day teach a man to fish he eats for a lifetime and that's something that really needs to be kept in mind is that giving clients information that they can take and implement in their program and having them prompting them to ask questions of you of what could deter you the most like they were like I

mentioned in the testing phase what did you see in my environment like how did you do this or how how did you identify this asset it's one of those things where they have insight into an attackers mind and are able to get that information and actually get defensive strategies back which i think is probably the most valuable part of a penetration test because that way you can see what attackers are looking for what they're exploiting and what you can do to make their life difficult and finally again I can't stress this enough if providing the actionable recommendations um I put this here again because it's it's another thing that I feel like is is not

accounted for necessarily enough because you have your you have your default solutions you know it's I know that you probably have a reporting platform that has your solutions to your vulnerabilities right sometimes you kind of look like a doofus when you say hey you need to implement a solution they're like we can't do that and it's it's very much about molding a solution for a client helping them come up with a solution and working with them because in as a client it's it's about again asking a question of that tester it's like hey we can't do that how do you think we can MIT it like reduce our risk on this particular finding and at the

end of the day it's these questions in this reporting process that will generate the most possible value for a client in their penetration test and that is all I have for today I can credit my wonderful wife with these amazing slides because I do not have a design bone in my body but luckily I married someone that did but now we'll open up the floor for questions um so feel free

[Applause]

and you know this part of our I you got to talk to HR on how you can maximize their profits what key points would you hit on when you're trying to get a job and you're trying to talk about so are you are you trying to break into the pen testing field so um basically talk when you're talking to HR they probably won't ask those questions but if you're in an interview probably with the hiring manager um if you if you get to that point in the interview just talking about like you're interfacing with clients and what you what you can do to advise clients and be kind of more of that consulting aspect and kind of just

providing I would say providing guidance and helping client clients in the right direction is probably one of the the biggest things that they'll see as an asset in you is like a penetration tester

where do you find the most effective as far as

could you repeat the question one thing that I know that we do on our reporting is that some people some I mean I've worked at other shops in the past have not done is one is a manual validation of findings to reduce false positives not having any false positives in the report makes people very happy but um a to is have it making sure you have that report review call and sitting down with everyone and reviewing the report with the red team so that questions can be asked and kind of get more feedback on it but also providing a robust executive summary it's probably one of the biggest things that that I've seen clients really appreciate is that kind of really

identifying the risk aspects and so in providing them guidance on what they can what actionable steps they can take from top to bottom I'm rather than just handing them a list of findings and being like hey here you go good luck kind of giving them a an idea of what to remediate first next and then kind of further down the line

yeah exactly so it's kind of doing the interpretation for the client because as an attacker you wonder I feel like um or at least an attacker understands the the risk that actual looks like a spoiler bility of a vulnerability a bit more than just the cbss score you know so it provides a bit more of a like you said a qualitative assessment of their environment rather than just hey you have X number of high-risk findings um X number of medium findings especially executive executive summary if you see that there are certain things that they are doing that are doing well make sure that you mention that so they can see that they're giving us a return on their

investment in things that they are that they are doing and doing well and too many times if you don't have a good penetration testing team that comes in our company they don't they don't give that feedback so a lot of times the executives don't see that so the only thing they're seeing is what's wrong they're not necessarily what's right I think that's something we got to keep to remember that when we do these reports absolutely on the subject of risk exploitability do you find value in things like the threat like dread scoring way you know things like you know probability of something actually being detected and then exploited do you use that in your methodology um I do not how

could you elaborate on dread score anything could be valuable um subtract scoring is a is something that microscope Microsoft developed it's basically it's like an alternative to cbss mm-hmm but it also takes into account things like whether there are tools available the amount of school there is the amount of skill that is required is there and things like that after yeah yeah I know I definitely think that on what you're the dred score could is definitely a is definitely a good measure as well as um a lot of the times we tend to kind of come up with our own CMS cbss scores based on what we've observed in the environment so if it's um let's say you have a cross-site

scripting vulnerability but it's like behind authentication you need add admin in order to access it then the actual risk associate with that vulnerability is much lower than an unauthenticated cross-site scripting for example so it's it's really one of those things that I think as an industry um we can do a bit better of defining the wrist behind vulnerabilities a bit more quantitatively rather than what do basically have to do now is you have a base score for a class of vulnerability and then you kind of gotta weigh weigh your options individually you know in a way

[Music]

um well in in the beginning um I know that on my team we in the past have simply kind of gone through on the the processes recommendation and then worked with the client up to a point with the report of recommending what they can do but I know that now on other companies as well as I think that Optive is starting to actually deploy people out to help clients fix these vulnerabilities because sometimes clients aren't equipped to deal with a large scope maybe a large score of an organizational problem and that having like external consultants really help guide them in order to implement it correctly so um I know that Optive hasn't done in the past but we are

starting to do it but I have been at organizations that do do that and that is a something I think that clients have found really valuable is kind of getting that that knowledge sharing and training their team of how to implement it correctly and run it correctly

all right um no one has any more questions I really appreciate all y'all's time and hope you have a good rest of the conference