diff -q 3rdpartyassessments internalteams | grep qualified assessments

sounds good hello my name is Kelly odonnell and I'm going to be talking about the differences in third party um assessments and internal teams and how we might be able to get uh some quality assessments out of that um so here's the agenda I laid it out kind of like a pentest report um just kind of be trying to be a little creative but uh the big thing is is we're going to really review the goal of a pentest and and discuss that and and talk about that everybody you know I have uh peen up there you know I see that so many times come across like you know do you really know what you want is a is a pen test

this girl sitting on the floor uh testing her the how skinny she is you know I mean there's everybody has a different um definition for a pentest so we're going to you know talk about uh that a little bit in this talk so again the reason and how this talk came about was uh was driving home from Derby con and reflecting on the whole puppy mill pent test comments out of the um Opening Ceremonies no the panel talk and it was like well we're not doing anything about it we just keep talking about these poor pen tests that are occurring and we we don't we don't do anything about it so again we're going to discuss the differences of

third party um and internal testing teams and you know what we might be able to do to achieve uh quality pen tests so uh who I am I'm a father uh cissp cism uh Sands uh yeah blah blah blah on the Cs uh a president of the Charlotte Metro Issa yeah all right one member where's the other guys come on oh two members yes all right so uh I'm a red team manager uh sometimes I think I'm uh a gang prevention manager with some of these guys or you know a troll collector because they love to you know troll me all the time so it's a shame days is not here because he'd be doing it right now

but uh graduated from High Point University um undergrad andrad um when I started my career it was at RJ Reynolds and I started at the physical layer polishing fiber optics um then I led you know that led into doing a little bit of network admin you know actually plugging that fiber optics into Network gear and then going from the network gear then starting to configure the network gear did the CCNA way back in the day all that jazz um so from there went on uh to move into um uh security kind of got thrown into doing firewall Administration actually built the first um checkpoint firewall for RJ Reynolds and um you know just because I had uh dealt with a firewall I

got that job so and then from there I just went into a different area of security and uh really opened my eyes especially when I started taking like the gpen and you know what you could actually do so and we're going to talk about training and and and that so little disclaimer this doesn't you know this is my talk it's my thoughts it's my opinions it has nothing to do with my employer which I didn't mention anyhow so here we go I mean um what is the goal of a pentest you know what what should it be and in my mind here we should be a identifying true business risk um you you know there's there's the

scanners there's the vulnerability assessments out there that that identify vulnerabilities but after you exploit it's what can you do after that it's what can you get so and that is going to really determine the true business risk because that's what's going to actually show up or outside your network if you have those gaping holes so and so how how do we really um Define what the true business risk is and you know how do we find it and and what should we be doing and I and I say this because there's so many times when the lines of business come to me and they say I have this server it's in development and I want you to pent test

it before it goes into production I was like yeah no uh maybe a QA test that is or maybe a vulnerability assessment but um you know and I give them this example I say okay there's this web server it's got a data classification of public and it's in Dev and you want me to test it so my my Engineers go ahead and test it they exploit it all the other systems in that Dev environment are irrelevant that so they can't they have to stop the test right there so yeah great I can exploit it uh so what's the business risk to that it's E I can exploit it and it's public data probably pretty low right

it's probably overall that's going to be low let's move that same box and the same scenario into production and now my guys do the same they use the same attack they exploit it but now they get to do Post exploitation from there and then they pivot they go deeper in the network and they find say a box with 100,000 credit card numbers in it and they can extract it out through that box what's the data what's the risk now that's probably going to be high for that box to just be on the network and be able to be exploited so that's why when and things change too so when you go from Dev to production you have IPS

that change you have accounts that change you have just things change when you move it from Dev to production so you really need to be testing in production so again uh goal should be to really get to post exploitation and you you really need to learn about the um applications and the systems and and what the business goal is what are the crown jewels of the business what are they trying to protect so a couple uh methodologies that that we use and we have to are a large Financial uh institution and they push down nist so you have to use the nist Frameworks and everything and and a nist is great but it's not that robust and and we'll go

into next slides we show um what I'd like to use and and several presentations today have mentioned it is p test uh penetration testing execution standard where several of the elite uh um Engineers within the community have put this together and any and a lot of folks can go ahead and add tools to it and and it's just it's done right so um here's nist as it looks it's very basic do you got your planning Discovery attack and Reporting um but let's take nist and P since pest is really robust and let's just merge them together let's you know let's take this and merge it so under planning let's do the pre-engagement and initial threat modeling pieces and I think uh

pest has threat modeling but I like to uh break it out into planning and Discovery for threat modeling because you should know a little bit about the system you you know you do your pre-engagement you're getting your scope you're you're identifying that you should almost have an idea of what you're going to do from a threat m perspective you know you look at it from an internal uh perspective maybe like Edward Snowden you know threat or or an external you know hacker trying to get in from the outside there's many different ways right so you should probably try to look at what the engagement is and apply that and then when you get into Discovery you actually

might find other systems and other um applications so you probably want to do threat modeling again you know you're going to do your int intelligence gathering you do your threat modeling again you're going to do your vulnerability assessment piece there that's going to lead on into uh your attack phase and uh your attack is exploitation and post exploitation and and if you look you got your attack and the arrows going back it it should just almost be like a revolving door until you've you know really completed a thorough uh a test you know between uh attack and and uh Discovery just keep going going around until you have exhausted that and then when you're all

done you you know you see all these arrows pointing into reporting and this is the reporting piece is you got um executive summary introduction methodology findings recommendations and conclusions and um that is I believe from Sands and maybe even offensive security I've I've blended it so again we should be doing uh in the attack phases the exploitation and the and the key here to this talk really is the post exploitation piece uh and where we can start delivering true business risk or real business risk so um here we getting to you know third party value and um you know third party has has a lot of value I'm not saying that you should go one way or the other with internal and

third party you should do both really so um the in the third party value you can get an outside perspective and a different set of eyes if you've been testing something over and over from an internal team you probably should get a third party to come in and and test that at some point in time um they can also help reduce unimportant tests since this costs money so if you have lines of business that like to take advantage of internal teams and just throw assessments their way you know and you know they're really not high priority you could actually say well you know what we're kind of booked but I do use this third party but you'll have to pay

a and I bet you they'll re-evaluate that really quickly um they're also good for um HR and payroll assessments a lot of times you won't want your internal employees uh finding out the payroll data so if they pop that database and can see everybody's payroll you might not want that kind of information out there and you might have a uh kind of a bidding war on hey I should get paid this so excuse me I'm trying to go fast because there's another talk and then resource reliability you know if the third party has someone that leaves the company they're going to get someone on that assessment that can pick up right away um and and so they're

they're really reliable uh when you're dealing with a large fi there's a lot of policies in place that um if if someone's ready to leave and they're going to go outside the company it's a standard policy to let them go right away so it kind of puts you behind so anything that has been um you know queued up on their computer you have to get access to you have to find out who's emailing them I mean there's a lot of different things to think about there when you let someone go or if they leave so here's some of the internal uh value you know you can spend a lot more time on assessments uh as um an internal

team and we typically give like our uh Engineers three weeks to do that's typical three weeks to do an assessment one for report writing and you PR in the beginning of that you probably got about almost like a week for scoping out everything so you're really looking at like five weeks doing all this um they can perform uh better regular testing um they're just there on site I mean it's just easy for them to pick up and and do it or they're just constantly testing things right if they've got downtime they might be using burp we found things just out there on our our Network um that we we can go ahead and fix right

away we didn't need a true assess uh uh engagement for it um because we do no notice testing too so um and then keeping sensitive data so on the flip side right so uh I talked about the third party value with the payroll systems well on the flip side you have intellectual property you may have certain things that you don't want a third party to know so you'll that's a good piece to use you know an internal team for so they can keep that inhouse whatever they find out of that that stays inhouse versus maybe uh a third party leaking that or if they store their finding somewhere that doesn't get popped you know you had hacking team

being Reckless with some of their passwords and all their data and stuff like that and I've heard horror stories with other assessment teams and I won't go into it but uh the way they treat their data so as a third party you really should be using a lot of encryption and maybe not some of this Cloud stuff out there but that's another topic um and and an internal team can really uh work directly with other internal teams and make blue better right I mean uh constantly uh we're not just assessors and and I should be wearing my uh purple team shirt up here today probably but you know our team really is like a purple team we're

helping them be better because we're identifying and then we're helping them fix and or ident at least identify for themselves cuz a lot of times we go in and they don't even see us so um you know and then we see okay where can we uh make tweaks to some of the uh detection systems to so it can get identified um and that's the continuous instant response um we can also quickly validate new threats we've done proof of Concepts you know spun up we had an um where uh blue coat came out out with um a CVSs score of a five and the internal team evaluated that and said yeah I think this is more like an eight

or nine maybe even a 10 and they stood up a proof of concept right away and if they they could send a link in and do a 407 redirect out in your credentials were actually going out because they the blue coat thought to pass along your credentials to the proxy server out on the outside so you know you quickly we were able to stand that team up and work on that right away so um another good um is counter surveillance you may not want your third party doing count counter surveillance you may uh but as a big fi we want to keep that in-house and uh if there's any bugs in any of the um uh conference

rooms or executive suites we kind of want to keep that to ourselves um and then for executive profiling yeah they would probably be a little nervous if a third party were to do that cuz some of the things we find um and you know you had dashley Madison uh um database so we'll we'll actually go through that because that can they can be they can end up going being a target from that you know so and then another uh uh point for an internal team is U the the team that I have they Mentor quite a bit and so they're they're adding a lot of value to the blue and and helping them if they

want to ever get onto the offensive side you know um they're really doing a lot of mentoring there so here's a question for you will third party um call it a pentest even if it's not we keep hearing check Mill or I mean we keep hearing the puppy maill pentest and we also hear uh the uh check mark or pentest and some might uh there's a there's a lot of them out there that will educate the the users but I think there's a handful also or more than a handful that they may not they may want that money and they may might that contract might be very important to them that they will just listen to the customer and if the

customer says I only want you testing this little piece and it's in Dev they may call it a pentest you know I know you and I think we see that I've seen it firsthand question isn't that supposed to be capital P in that was in my first one you missed that you came in late but yes you make a good point yeah we get a lot of I said it earlier we get a lot of requests that come in and uh folks just don't even know what they're they want and they capitalize peen like it stands for something so yeah so and and they don't know what they want and that's that that that's a

problem we need to educate and so will the uh testing company pass up on the contract if the Rules of Engagement are uh restrictive probably not most of them won't because it's money and dollar signs um and and if the Rules of Engagement never allow that post exploitation are is it a pen test I mean some stuff might be but uh you know we should General will of thumb try to achieve post exploitation so we do have some third party pitfalls like I just mentioned so it the customer is sort of always right when we get into this so it they don't want that friction um and we have and then that can lead to a checkbox

pentest um and then uh will you know will they pass up on the assessment again I I said that in the previous slide or if it doesn't allow the post exploitation so there's some pitfalls there okay add one there too is if you a client has had a a pin test or what you call or not and you come in and try to explain to them to know that's not really what it was you can't sort of throw them under the bus with kind of what they they paid hundreds of thousands of dollars for that wasn't that right so right it's tough face at some point yeah yeah so usually you come in and you've seen what

another company has done before and it's hard to for you to tell them that really wasn't a pentest that was more of a vulnerability assessment if that maybe it was just even a scan thrown on a report yeah because I think there's I think there's even differences between a the vulnerability scan and the assessment and I think we should take be taking our assessments towards exploitation right you exploit and you stop there and because you validated it right that's validation you validated there's a real and so that that's the part of the assessment there a scan just says oh here look here's all these uh false positives or negatives or whatever you got to go research it I mean that's

that there's no value there we have to deliver value so all right so we so here's you know a little bit of um why this came about this talk and I found this slide and I started reading it I'm like wow question breeders and pet stores and this was really for puppy pentes but this actually kind of works for our industry if you just replace some of these words here I mean if you question let's say we question the company performing the pent test right so instead of the Breeders we we put in uh we substitute that for the company we urge governing organization the next one urge pet stores right to support shelters we so urge governing

organizations to support quality testers we have to do something and then I've got some slides later on on some recommendations and maybe where we can go from this and I don't know what the real answer is uh but we'll get there educate family and friends about puppy mills and we probably should just add educate family and friends about puppy maill pentests right I mean we should we should really educate their um share links posts tweets uh and stories to raise awareness I mean that's what we're doing now we're we're out there we're getting out there we do that we we tweet about these things there's uh what it's Evan Booth he's got pentest fail right is that him yeah pentest fail

and Evan Davis Evan Davis sorry so yeah that's what I meant boo going right now other EV yeah the other Evans jeez too much too much of this thank you um so you know we try to get out there and share links and posts and and talk about our our experiences uh talk to legislators and authorities and here I I have uh talk to governing bodies you know maybe it's PCI maybe it's not I don't know but we don't have anything and PCI at least has some teeth right now so I don't know um inspect breeders inspect pen testers and the companies um organize meetings uh and educate uh to decrease the puppy maill pent test

again add puppy maill pent test to that and uh the last one no more profits for puppy meal pentests get the existing organizations to support them basically so I mean this slide kind of fits you just got to replace some words um and again I'm you know I'm kind of tired of hearing about these uh companies that are performing these types of tests why can't we get companies out there that only perform perform quality so quality tests and there's a there's good companies out there that do this so so here's some internal pitfalls and there's probably a lot more but this is what I came up with and Dave could probably help me he's sitting in the

back but um you know we have uh some internal pitfalls our HR and and uh recruiter our recruiters have a hard time screening really good candidates um audit I don't know how the third parties I don't know what you have to do for audit but man we have to jump through Loop holes from a big fi and an internal team and it's like why are we even doing this should we just you know have a third party do this I mean there's a lot of value for us to do it but audit holds us back um culture right I mean the culture doesn't does not typically match pentest ideology I mean really the for an internal fi team it it just

doesn't I mean I can go on and on but um and I got to get through how's my time um you know you I got you know probably good all right well I've got a lot more so and then there's uh I'll try to go through this you guys want to be here all all right um and then we have that potential turnover because of that uh the bullet ahead of it yeah I'm I'm trying to get through this you want us to keep interrupting or you want me to get through this I hear you uh and then uh permission memos right I mean we don't really issue permission memos I mean we have but

they're interm permission memos we really want to push towards an uh a policy and we'll start talking about how training is not really focused uh for internal teams the training that we have out there like Sans you start looking and Sans leans towards third parties everything all the tools they have are more third parties and the permission memo they don't talk anything about an a policy saying hey this team can go ahead and do what they need to do and have that get out of jail free card and that's a lot of work because you got to work through your legal as well when you get this in place and they want that um language correct so you know training

just because you know you know because without it you kind of look like that so so some recommendations um you know might be that we need to start licensing our pentesters or having a governing body that uh can ensure the quality pen tests um and then also we all need to change a little bit of the training mindset on the focus for internal uh pentester so we can have internal pentester representation you mean just because I have Security Plus doesn't mean I'm nin exactly there's not boot camps out there right and and also what I'm saying is that there there's there's okay training and I and and there's good training and Sans is good it's not great

right there's other things out there like offensive security and stuff like that but what I'm saying is Sans and all these all the others too they're all doing I don't see anybody focusing on internal teams it's all the all that is focused on third parties and the way the course is structured and there's no boot camps out there toest exactly right I mean you got Security Plus you're ninja right exactly having the Security Plus Perfect come see me uh so I don't know what the answer really here is is it a governing body you know that can say um you know this is a legit pentest company and and all the others are just out there trying to

get work and you have this Consortium of uh pentester companies that do actual work and have and are proven that's the thing they're proven so and I don't know maybe it's uh you know like an Angie's List I have that up there so we just call it Bob's right Bob's pentest list or something I don't know um so yeah the is full I mean you've got PCI DSS and you have qsas you have PA qsas and uh isas and as asvs and qirs and pfis you have all this but you have nothing for pentests and I would question why in the hell do we need an ASV why do we need someone saying hey they're qualified to perform

a scan it's automated this is automated let's get that out and put something in where it's an authorized pentest vendor right I mean it's a liability legal terms what they're doing they are liable for those things that they scann the purpose of report scan something you should have patched they should have told you about since our pentesters they'll need to absorb the liability of missing an exploit yeah which we all know that's so easy to catch every possible system so you want to put your liability Now by having that authorized and improved person yeah well and and and that's time based though too right so it they have to they should come up with it they should

sayon policies yeah but those aren't going to show everything I mean you're going to see the vulnerabilities but you're not going to know what's valid and what's not a lot of times got a question so I think it's important say that they need to EST this who is well there's a PCI Council and there's members of it has no teeth it well PCI has some teeth them only for merchant bank or Merchant um Merchants right right yeah five or six CPAs that are rocket weci DSS has no nobody get they they yeah there's fines they do and they're fined when they get sued they care but no nobody cares it has no teeth make it half teeth that's how you

make it halfth well they're a lot of companies are getting fin because they've ignored it I I was an internal Isa and the first week on the job once I I got there and I didn't have the ISA kind of classification we started getting fined and guess what they sent me out to become an Isa money makes people do right right so anyhow moving on so here's a summary of the differences I know that's kind of small to read I'm not going to go into all these I really covered a lot of them already in the presentation and I know we have another presentation that we would need to get on our way um so in conclusion again I probably

stated it a couple several times I'm tired of hearing about the puppy M pen tests I think we need a government baring to ensure the quality or we need something because we need to remove the check box um and uh you know I mean PCI has the ASV we talked about this for vulnerability scans I think we really do need some sort of authorized scanning vendor but this does need to be higher than PCI I agree with you you need an I can you do you need the community to say that these guys rock and these guys right yes amen yeah so we need to come together and figure this out I think definitely so any

questions yes how do you know all this just working for small Regional Credit

Union trolls you missed the you missed the slide with the trolls yeah so working anything else you question no okay I think we have another presentation so yeah we can um wece that but thank you Kelly references thank you thank you all right we know you're solid know you're