Red Team Panel Discussion

buddy for listening to the Badgers of a crate Thank You Keith or your introductions I'll try to keep mine relatively brief so I'm not going to say what companies I work for now something to do with hammers I did cross paths with him at a large banks it looks like Sauron down the street although the mic closer to your mouth I had called up closer all right you got it get really personal so I am a manager right now underneath what kind of i have i have vulnerability scanning pentesting war games and what is some people call like a red teaming but more of the red vs. blue can you see me I just pwned your box did the blue

team actually do things some people call that purple I will hold off on any of those comments before i get into that but those were around the military i I to kind of explain how I got into this role I was a corpsman and fmf corpsman so what that means is I was a medical person for the Navy I serve many years with Marines the line units as well and in part of that I spent many years triaging large pink so large que haces of mass bodies figuring out how to figure out which one is priority take that to what we do now we're looking at a mass amount of casualties which one's priority right that's what red team

really does in addition I also served in the National Guard and had a nice little time in Iraq with that I come with the idea of also the op for red teaming approach were truly the idea the red teaming goes in there and we'll hit that in a highlight but I have certain stances on particularly the colors and really the effectiveness bringing out to the short start what to bank of america I've before that have also worked SCADA systems emergency operations for Jacksonville North Carolina where camp lejeune is for six years I've seen pretty much everything from the fire departments from power systems to Swift to ATMs to wireless to all kinds of craziness and we all need red teaming or

offensive security is probably a better word for it and I'll stop my rant because I think I'm losing myself in here no you're fine the early in the morning that it's it's not enraged man so the first question in both you've already touched on it is what is red teaming there's a lot of debate about what that is there's a lot of confusion about what that is some people think penetration testing is red teaming some people think vulnerability scanning is red teaming some people claim that they're doing red team in and they may not be doing it so from people who are actually managing red team's can you now first of all let me say this GDS

they do it from an external perspective they're doing it where companies are hired they're hiring GD has to come in and do it they don't have to deal with the political aftermath of whatever whatever is left from the time you deliver the report whereas Matthew Becker's managing a team from an internal perspective and it they're different it's different management strategies i guess but there's a lot of overlap as well so from your perspective Keith what is what is red teaming so it's whatever the client wants it to be ideally it is a no-holds-barred no rules so interdisciplinary so whereas a typical pentest very siloed is we're going to come in loud with our scanners in fact we think the client is probably

whitelisted RPS our IP addresses so that we don't set off alerts and we try to get as much Brett as possible in a true red team assessment we are we're going to cross all those silos we're going to leverage the network the application the social engineering the on-site access and we're going to decide where we take it and not let them know and so we're where the siloed pentest tend to be loud and noisy and in and you know we're coming you don't get a true perspective on your ability to detect those those activities and and you know a gauge how well your team can detect them and then if they can how are they going to

respond to them so the the troop in test I think touches on that aspect that is not often this often sort of glossed over one of the aspects that defines a penetration test as a penetration test usually is that it's Tom limited do you typically have time constraints on a red team effort from since we were you know being hired incoming externally that the time constraint is usually driven by the budget constraint yeah now if it's if it's required you if it's part of like a see best certification there's a defined scope that you know it has to be what it has to be but if the client is engaging us just to voluntarily then then they

can define the the time by the budget and in some cases they the no-holds-barred thing is is flexible as well you know they may say we are just interested in technology so you can look at our applications or network etc but social engineering is off the table ok Matthew what's red teaming from your perspective so read to you in my perspective will pull away all the all the cool metal of magic pieces basically it's trying to understand really a full perspective of all the amount of money you put into everything and how secure your business is I really look at it is if you go back to the history in itself red comes from the red versus blue kind

of more of the Cold War type of approach where militaries would act as the as the opposing force using there's tools tactics and techniques the people the way work the equipment they used and trying to see how they could defend and actually interact and there was also the White team we're doing the same thing really around business I think the one of the shifts is we spent a lot of time on information security but really what we're doing is we're testing to see these in IT itself has pretty much become like the lifeblood of everything we have our business runs completely on everything everything from our water from the the power lines we've seen all

that headline news that stuff's been going on for 20 years plus but we're driving with the idea and businesses are saying it's great let's get rid of people we don't have to handle them we've got these machines they're forgetting these machines are actually what's running their business it's what's making their money it's what's keeping them there it's also making sure they're getting paid every day so to me that's what Red Team truly is as we get into the intermix of the colors and those pieces i think the pen testing does actually is a facet but it's just like organized crime they're using computers now just like a gun it's just a tool to get to a means

to finish something if you want to take into the criminal element there just use it to do a crime there's an N stole there's a there's a human part and also this is one thing I think should be put into red teaming when we are talking about it and even some of the pen testing the behaviors of how they're acting this is also going back to the purple color we're looking to see how someone actually did something and compromised it how the script looked on an nmap or whatever whatever technical pieces you're actually looking at and seeing what that reaction is so it's the full gamut of looking at people process technology and more of like a business

security or how am i doing after all the money I've spent one last thing if you don't have the Volm scans and the pen test and all the light stuff done before then don't even buy our bring and red team it's going to be too easy they're going to wake make it one way or another anyway if that's what they're brought for they're looking for no holds bars they're looking for social engineering they're looking for going after badges they're looking for technology signal waves psyops you could bring all kinds of cool things and that's psychological ops that's a propaganda that type of stuff you could have all that in there but if you don't have some of the basics

you just wasting money by the way if any of you have any questions at any point in time even though we're going to reserve questions or they'll boot some time for questions at the end if you have questions and you're like hey I want to ask something feel free to raise your hand jump in its Kelly I'm kidding yeah and please don't take this as anything being wrong anyways approach really the whole thing is just explaining it to other people who are not in this who are understanding the pink in the Reds and the hacker you walk into a room and you say I've worked six months on us great project and it's going to be all the money I've spent and

my merits and these are all the metrics I'm supposed to have to have this running with the the 590 keeping it up time and you've walked in there and said I'll [ __ ] you didn't do a lot of stuff there's a whole there's a hole there and I just shut it all off kind of pisses people off so the goal is really trying to help people instead of getting all scary as a booty man yep no no you're fine I was just thinking I've heard you used the term business security before because it's not just about information security anymore I'm hoping you will elaborate on that some in a little bit so I'm kind of curious show of hands who

is in offensive security today right now who's employed making their living doing that okay and go ahead and put those down and who is interested in being an offensive security at some point in time excellent okay so that makes sense is there anybody in this room that's not interested in being in from our offensive security not interested this if you've been in a while yeah that's some point you're like I don't want to do this anymore yeah but I don't know what else I'm gonna do right so yeah yeah we've got the cave picked out we're going totally off-grid right so um where was I going with that primitive skills were out there with fire now yeah so

business so I used to work under matthew up until very recently and he used the term business security whenever he's making the case to management why red teaming is an important activity because not all businesses have an appetite for Reggie mean they might say they do but when it comes down to it they're like oh yeah we do but you can't touch the production systems okay well that's not red t me okay well yeah you can touch production systems but don't take them offline well I can't make that guarantee it may go offline I mean there's no guarantee that I can't do that so at the end of the day you either have it might

be a penetration test or it might actually be a vulnerability scan but business security encompasses more than just the IT and Matthew was alluding to this and I don't want to steal anything that he's about to say but it gets into the physical security aspects it gets into the people which is quite honestly still the weakest link in any organization the human element and it's unfortunate but I don't think we're ever going to replace humans entirely unless we go full full on matrix style and plug people in and we're just using them as an energy source to power the computers you laugh but a man it's not that far of a stretch how would you know if you

weren't there already right now you might you might actually be plugged in today yeah now we're get into what is human and what right that's all done a copper totally different panel discussion right so I'm gonna got a question yeah sure

yeah I'll repeat the question I'm sorry I started thinking about it but yeah he's asking is there a common pet like is there a path to win every single time

so I'm gonna take a step back there is an exact exploit that I would say is a technological piece I think as a whole and every cissp and I have been also on this if talked about it we talk about Sunzu everybody from ova sunzu art of war someone will say know thyself right sounds easy right no one has system records that right there is the first problem they don't know what's important for a lot of them they have a lot of protocols that are unsecured and it's a business need and there's a lot of that's how it was run so don't fail just keep running a lot of businesses have continued to drive with an older model

going back to the putting this technology underneath it just forgetting that I don't know 1970s we're just turning into color television it's new it's a technology we're running on things are our forefathers and I've got these are people some of these people here of like wow this is new technology if we looked at this years ago it was like whoo someone's going to go shopping online it wasn't an idea now it's every day and we're pushing societies collectively in that I'm going back to basically the security there not knowing how to understand what is their heart versus their arm how do they cut off and stay alive that's the biggest problem is they don't understand the inside the

outside we're still arguing about things of the MM hard outside squishing inside don't worry about it the adversaries are just going to on the outside Lance actually had the old model of the castle they still think we're in a castle on a global scale that's the problem i see collectively is one exploit i don't personally have one atop my list okay ya know I from a technology only perspective I would say no there's not really you know in the the early days of the internet and security it was common to attack the network perimeter you had firewalls that failed open and so everybody focused on you know patching that fixing that and then so the attacks

moved up to the application layer you've got sequel injection cross-site scripting if started to do a better job patching that and so it keeps you know moving up the stack so I would say if there's anything that we we find that's common across all of them it's pushing beyond that technology stack into the layer eight you know the humans the social engineering and if i were to back you know back into the technology stack a little bit it would be administrative interfaces that are exposed externally that just have a simple user name and password for logging in which is not a vulnerability in a in and of itself it's um you know it's not going to be picked

up by your volunteer probably but it's it that's where we can then tie those two together the social engineering where we can fish them for credentials and then go back to that administrative login interface that should be locked down with two factor authentication or even ideally you know behind the firewall where requires VPN

so the question was I just want to make sure that and make sure the people back there understand what it is and these guys as well you've heard a lot of problems with active directory and you want to know if that comes up as a path to success for own in the organization

active directory is definitely a target a very rich target because everything's managed from there right yes yes I was trying to figure out where the voice was coming from yes oh really yeah it's a credential question right the Bulls down to password strength or i can let one of these guys saying oh you're doing [ __ ] you go it's a a lot of times you don't even need the password if you got the hash you can pass the hash attack it's it's an attack where you take the hash and you feed it to another system and it just okay yeah you're so-and-so come on in and once you do that to a domain controller if you have a domain admin

credential then you've got the entire domain and once you've got that it's pretty much only organization so to speak you own that system you own that Network now you own that entire company and if you're an ethical hacker then you put it in your report and if you're another kayaker you too bad and malicious things to them right so the teams that these guys are running our ethical hackers they're trying to let the organization know that these problems exist before hand let's see I've lost my place the so I'm kind of curious what pushbacks do you get from management whenever you when when you start talking about red team efforts and Keith I don't know if you do you

experience pushback from management because I mean they're coming to you and they're saying hey we want you to do this and then you you say okay well here's our plan do you get pushed back from them not so much once they've once they've engaged us they've already internally committed to it I would think the only the only push back we might receive is they start off with pie-in-the-sky ideas you know we want the the full red team experience no holds barred you tellin price and any well yeah you look cheaper yeah exactly yeah yes faster cheaper yeah what about what about an internal red team perspective what kind of challenges do you face from maybe maybe

the piers that you're working with in addition to management above and you can talk broad you don't have to talk about current area which approach is the easiest approach so I get the luxury unfortunately you get to hey here's your report have a nice day call me if you want I get the luxury of here's a report here's a bunch of extra work that you've already had and you already had a your performances in your evals already based on how much you're doing to build things into production now I've slowed down everything or depending on many companies they have this this this green this great idea that we're going to gonna stop businesses from putting

things out in production without any defects yeah you try telling an SVP who thinks that they own the world and that their or their job is the reason the company runs that they're not being able to push out their big project they've been boasting about so um that can definitely be a challenge I will pause there however and I am going to use a word that sounds weird in the red teaming and my own little aluminum hat NSA I am if you haven't dug that up it's an old book that the NSA put out for information assurance methodology in essence basically what it is it's the best way to try to understand the worst-case scenario in your pre

assessment of finding out what that person cares about ask them for to negate this problem you need to understand the business you need to understand their pain points their problems their scare the more you can figure out how to make your exploits correlate to that the easiest they'll be able to digest that pill otherwise you'll just say cross-site scripting over there that's cool what does that mean let's go take out your business we also have lots of talks on social engineering if you can social engineer and SVP to be like I understand this is going to take down my business I could go to jail that stuff that helps not always I've also had challenges sometimes of just leadership

and it's also goes back to the colors and I want a pink team versus a red team trying to explain sometimes the management why you need things like get-out-of-jail-free card because they think collectively that their enterprise and there are instances as the overarching laws and anything within as does for brick-and-mortar pieces covers you it doesn't I mean would you feel safe without going without at least some paper saying I'm supposed to be here we would never do that no question

and there's one other compounding piece of this thing called CNN and Fox News that just scares everyone so we're hyperaware media side well it does play into it because depending on where the background of the law enforcement they and being with my my military background in millet mehreen core some of them when they turn in transition they want to be due diligent could be a good thing but sometimes it adds to it so taking the law back and I'm going to make sure I try to answer the question and if I didn't answer please ask al in a different way when I'm going back to the get out of jail free card in our aspect

looking at a bank you have the idea that you can go into that bank and you can go and play with like their ATMs because it's got their logo on it right however once that turns in once something goes into it it becomes monetary money it becomes United States law enforcement takes that as a different approach now you're doing money laundering in other places in there where does that that blend into how they're going to incarcerate you because they don't understand and you're now breaking something that's now federal law versus a civilian allowance is that where you were going with the question so in that that is also some of the challenges you have to explain to management that it

isn't the myopic or the very short I've been arrested and I have a piece of paper it's what's going to happen afterwards when they start prosecuting you that you have actually illegal backing that there's something out of it in addition they also have to understand why you're incarcerated there's no red teaming going on so that money they spend on you just wasting away so hang on an internal red team they have a title of a red team they have a manager to manage the red team they hire people with the skill sets to do red teaming they still have to make a case internally for being able to read team the organization there are cases of

those yes that's so interesting okay

it once you've been doing Red Team you long enough you probably that's going with them it it seems I because you know and if they're putting forth the job descriptions they're hiring the talent they're seeking out there doing the interviews they have the funding it seems like they've already made the decision to employ the the people with that skill set that do those things against the organization so they can identify the weaknesses before the the people that are on their payroll identify those issues and they end up in the news some of those are and nothing against any of the instructors and and the end the piers sands there you look the marketing they say I need pen

testing and red teaming get us one cool we got one pci osc p we need one I don't know what it's going to do but we need one and they don't understand really what that held it in place in their business what exactly it is information security is still some companies are still like having problems of the old antivirus we put an antivirus in there now we have resource problems maybe you should upgrade that resource instead of saying turn off the antivirus got a question over here

yes I'll actually let you search your wife sure I mean yes that happens sometimes well we we try to point out where those weaknesses are and what they're getting for their money and and then ultimately you know they make that decision so I'm not sure what I'll stand to that well it sounds like it's going back to definition though and I had it and I didn't ask it but so what is a penetration test what defines a penetration test as opposed to say a product assessment or a vulnerability scan how do you differentiate penetration test from a vulnerability scan or vulnerability assessment right so we've touched on the the no-holds-barred access in the look go

ahead oh sorry yeah and the interdisciplinary nature of it what your I think another key difference is instead of you know going a mile wide and an inch deep you're you're we're going to do reconnaissance we're going to find figure out where we're going to focus and so we're going to maybe miss you know we're not going to look at something over here but we're going to we're going to identify this is we think this is a promising vector and then we're going to see how deep you know we can take it and we're going to try to do it as as quietly as we can just like an actual attacker would so you're going to

that that's the so you're talking about red team unit that's sorry I misunderstood well the question was what is the penetration test and so hang on yeah I got one thing to add here I've heard it explained this way the difference between a penetration test and a red team activity would be post exploitation what do you do after you've popped the box so to speak right because at least in my experience and probably some of the other people sitting in this room a penetration test is tom limited and you have specific targets and you're not really generally allowed to touch any targets that are satta scope I see a few nods going so that's more or less a penetration test

the vulnerability assessment would include like a vulnerability scanner which knows about some common problems and ways to identify them and it generates a mountain of information and a vulnerability assessment might include going like manually going through those in determining if these are valid vulnerabilities or are they false positives within your environment I think do we agree on this so far okay so a red team exercise would be once you're successfully penetrated a system what other systems is that system connected to what things of value or on that system and how could you use that to further damage the organization does that sound about right very much no yeah I think so so you're speaking more to

once you've once you've gained that foothold you haven't you've proved you can pop the box right so that would be a successful penetration that's that's a pen test so now we're pivoting from that point and seeing how he'll help how far we can go and what can we find what can we get and then once we get it so that's your ingress find the data and then we're looking for a sustained to egress and can we do all of that undetected undetected so that make sense okay do you have anything to add to that because there's there's a lot of confusion about the definitions I think still and there's still you could even and I saw

the question there still you could even with the pen test there's the application pen test which is myopically around just the application in itself and then you're looking at the full scale of how integration of the pen test and what you're talking about could almost be goal-oriented pen testing because it didn't have the physical and the other pieces but it's getting at least deeper in the dive and whatever terms use the challenges really is just making sure it's common consistent English or for them to understand it that's really the crux you can call what you need whatever is easiest way to explain it it's just if we're not explaining that we can't explain really the value of it right at the end

so what so what you're heading on is risk appetite what is their risk appetite which is what I was talking about on the pre part of trying to figure out their worst case scenario understanding their businesses the process is the technology that run those businesses what you're talking about is the [ __ ] is exactly right and not to negate that resources and making sure this this low hanging fruit are taken care of but what we're trying to do if we're collectively the red teaming pentesting ends result of impacts is we're really trying to understand what the business is what is their cares if it's just dachstein a bunch of information out of there seems to be

important because they're droning people for it whatever but if they're worried about actually like I use this system that happens to push out inventory to cross 15 15 states that right there is the stopper it may not be I've docks their their HR stuff or not there they also have a lot of this liability especially a lot of the fortune 250 companies they have this money thats it aside just for the oh crap happens so that might not be it the business driver shutting down their business finding out what shuts them down for a bank it's not the stock market what they worried about is a bank run no one having money in it that means they've lost all trust they

spent how much money marketing on it they spend how much money on getting spotlights for her like the NFL spot how much money is that cost that's there in there's that's the impact if I'm able to get a Twitter account and cause a mass media and cause a run and you spent how many millions of dollars on marketing and trying to get those contracts and the slots you've lost that money and the additional money you have to do to react to it yeah go ahead sure oh that was the NSA I am it's it's still kind of floating around I think it's excited started to die off there's also a spider into I think they have a red

teaming module as well I haven't taken that one unfortunately what was the stand what was the standard depends on the industry it really depends on each each industry each company will pick their own framework and model that they're kind of adhering to not saying you can't use that as the best practices honestly for what for a red teaming and what I would do is I try to understand all of them because HIPAA Sox basil which may seem if you talk to most bank people the technology is a very small piece of it but you're talking about liquidity and credit and having no people there being able to pay the bills kind of bad for them I'm not familiar

with it standard so it's like so yeah but yeah L isn't yeah read it understand it Sakura is a good start yeah

I would agree with that so I think I try to remember it was somebody from the NSA or somebody spoke recently and said that you know everybody thinks that that using zero-day vulnerability holes is is not going to protect you and I think the other thing goes back to the interdisciplinary nature of these red team assessments what we what we try to do is take two three four of those those medium risk rated vulnerabilities that you're you're probably more likely to risk accept and just ignore and then chain those together and and that's that's how we get in that is actually key it's the storytelling it's these myopic one small pieces that start one thing that pivots to x y&z that this one

device that says all only three people log into it but it's connected to an upstream or downstream to something that happens to me I don't know the omni-channel of everyone's driving because they want to have everyone's customer from wiping your butts and knowing I need toilet paper to tell your frigerator to literate I don't know whatever scenario case they want to come up with but yep hey why not putting we're putting web servers and toaster so why not

if if the customer can define it that's that's the goal that's the best way to do it right if they can't you try to help them get to some sort of go if they still can't then for me personally I'm trying to back down and you know it's not even a goal-oriented penetration test at that point it might be we can look and see what is there and we can report on that right if they unless they have the appetite to just go full bore and go ham on the stuff then then you go ham and report what you find right I don't know right right right yep all lifelock and that and the class

action suits are becoming extremely common now the lawyers are all over that so hang on one second we've got a little bit of time left and there was one question that I think is probably on some folks minds so skill sets when you're looking to hire people what skill sets are you looking at what are you looking anything in particular or just what are you looking for and people that you hire for your team so you because it's interdisciplinary you know it's a red team we're not looking for a you know somebody that's got a you know one particular skill set you know we're looking for people who are somebody who's good in network somebody who's

good in application somebody's good at social engineering and specific you know maybe windows gaya Linux guy but so a variety of skill sets that can work together I think you know we're looking for people who have a really deep knowledge of their domain so because one of the things that we're trying to avoid is detection you can't hire somebody that just is going to come in with default settings and yeah guns blazing they have to know the the limitations of the tool how to tune it to to fly under the radar low and slow what tools they can't use it all because they're going to have signatures that set off alarms that kind of stuff Matthew so in

addition to the technical stuff you highlighted things I look for particularly is people who understand psychology psychology of themselves what is that carrot that gets them up every day being in this fields for 20 ish years everyone asks you how you got here how do you hack I want to be information security I want to do it I tell them if you want to keep banging your head on the wall and you're bleeding and you like that that's your passion keep going for it that's what I'm looking for because it is it's when you're arguing with your bosses about trying to articulate risk and saying this is important and they're going to push back you're playing

devil's advocate I also look for people who understands processes looking for holes and things because really what we're doing is we're looking for the technology to stop a process to articulate something for that so one of the questions I'll ask is what makes a good information security professional it's one of the common questions I ask I'm not looking for all someone can just tell me that I can do a cross-site scripting enjoy a pop-up box because the executives don't even know what that pop-up box is let alone how to turn the computer on still it's not going to help them all right and it goes actually the question and I'll keep this brief but what is the success of a team or a team

one you've got the client so even in an internal team part of the job is your pedaling and your looks like a consultant hey I'm here to destroy your stuff will you let me come to destroy your stuff it's really what was going on you have to understand their goals and making sure they get a report in addition especially for the inside you also have the other metrics how many defects you found how what's your rotation of operational like you did 18 assessments this week do 16 more those are the battles you're still fighting because you're still getting equated to everyone else how many buttons did you push so trying to figure out how to tell

that and success the story the one last thing is also making sure being working in what we're doing diverse skills diverse thought especially when you get more than two people together so you don't get group think they start getting their they're different individuals you have to be able to understand them their skill sets and how to empower them to grow and that that's a key yeah and I would I guess add so when when part of our interview interview process gds is we we throw up spin up a intentionally vulnerable application give you 24 hours to just do a quick pin test on it and then another 24 hours to write a report and deliver it to us and we get on the

phone and you play the part of the consultant and we play the part of the client and so one of the things we look for is how you deliver it how you do the risk ratings how you you explain the finding and I think this goes to the the goal oriented question as well and and how do we what do we go after and and how do we present it so if you can get domain creds that's great that's going to impress the the technical people but the business guy that owns the application and where runs the businesses can be like what what does that mean so if we can we can get in and use those domain creds to pull

out health records or you know something sensitive and show that as evidence that you know than that that helps them understand so when we do these these interview app pin tests I look for for those type of somebody that can explain the vulnerability in business terms and also risk rates it appropriately so for example you touched on you know just cross site scripting and showing that you can pop up a box that's great so I get reports all the time where somebody rates cross-site scripting as a critical right well it turns out in this particular case all they could do is reflect it back at themselves right so what you know what's what's the business

risk there and and maybe it's still a valid vulnerability but maybe you want to bump it down a couple notches maybe it's a medium or and then describe explain that scenario walk me through how the attackers actually going to leverage that to get to the business assets not the technical assets hang on one second well but because the question always comes up certificates and I know that's kind of it's a fun discussion and that's had it almost every conference I've ever been to but is there any minimum level certificate requirements that either of you have of candidates you're looking for there's not for us we don't have a requirement it's it's definitely a bonus I think the the OS CP

has has a lot of prestige for us a genius yeah it's it's a certificate produced by all offensive security if you haven't heard of it you should check it out the PCI standard now lists as one of the qualifying markers of somebody who can perform an actual assessment that they consider good Keith mentioned a company or an organization out of Europe crest yeah see best see best yeah and I don't know a whole lot about them but they do one for one with the offensive security Oh SCP certificate remember their entry-level penetration test so yeah it's it does have quite a bit of prestige right now and with good reason it's an incredibly difficult certificate

to achieve and then we have a few in the room here so if you want if you want to talk to people there are people here today that can answer questions about it go ahead Matthew oddly enough other certifications you just IT related actually helps as well so if you happen to be a ccie in those type of areas because if you're talking to another ccie already built that rapport you already have that technology and you have to understand the systems as well so don't negate any other certification and just think that you need to get a security certification you need to understand and deep dive into all of it and understand how the interconnect so

what I've heard so far is experience is probably the number one thing you guys are looking for so you're not going to typically hire somebody right out of college to do this line of work unless they've been unless they're just really really good at what they do and they pedal you know they've studied it since there was a kid here yesterday by the way in Doug's exploit development class 13 years old I don't I don't know how well he was doing but I I'm imagine he did fairly well in the class I think he probably set back there and pwned all the old farts but 13 years old by the time if he continues on this path by the

time he's 18 let's say and he's willing to go you know he's looking for a job there's a pretty good chance he could get on with you know pretty pretty what would be take most people 15 10 15 years worth of work experience to get into because he's our he started so young right and he's doing it at a time whenever it's all just seeping right into his brain question back there we would be very cautious about hiring anybody that's got black hat experience we you know we deal with very large financial technology healthcare companies that are going to do background checks we do background checks and some of our clients do even more rigorous but background checks so

that could that could become an issue as a blanket statement i would say i would agree a lot of the being and inside you're stuck with the other HR regulations i do think though that also because of the legal system collectively and determine of pet black hat it also could be situational basis it could have been that someone did something that wasn't really hacking but it's equated to hacking that some lawyer said i'm a hacker prosecute having at least that conversation it also goes back to the other question about information security and how their demeanor is how they seem to act themselves how collectively their ethics portray out of it because going back to what he was

talking about that the the challenges if you already have someone known as a black hacker unless you're kevin Mitnick they're going to be scared yeah I would agree with that it's like I said we would approach it very cautiously the challenge maybe you know even if we believe you're now a white hat we've still got a you know you've got to be marketable to our clients so we would have to take that on a case-by-case basis and then I wanted to touch back on your comment previously on not hiring out of college we actually do hire some people directly out of college I don't know how many any of that applies to anybody here but okay good but typically those are

people who live and breathe it and do it outside of the classroom it's you know it's they're not just going to show us their their class you know their grades you know we ask questions like you what come you know what network do you run at home what you know what is your involvement with with local campus or you know organizations if you've if they've started you know their own a wasp campus organization you know really a high level of dedication that they live and breathe it outside of the classroom besides it involved in visa odds yes yeah that too so I'm going to plug a couple of local organizations if you're if you're young and you're in

college right here in Charlotte there is a no Charlotte chapter there's a student owasp Charlotte chapter at UNCC Charlotte hackers anonymous otherwise known as chaha is a good group of folks to get involved in there's several members here today gosh oohhhhh about Charlotte is a 2600 I know I'm forgetting a few Carol Annika Carolina con is a great conference happens in raleigh every year have been going on for a very long time I'm sorry buzzers anonymous this is new to me farese there is it that's something that's the after god nah yes yeah no but there's a lot of opportunity and it is a great way to demonstrate your to increase your skills and to

demonstrate to potential employers that you're you're not just you didn't just go to school and got a certificate or a degree you actually care about this stuff you live it you breathe that you sleep it you eat it you're passionate about it i think is probably the word that sums it all up your wipe the blood off your off your forehead and keep doing it right keep banging your head on the wall why do I keep doing this I don't know you put in eight hours and then you go home you'll know when you put in another eight hours and then you sleep phrase for fun yeah yeah who does that by the way who just does this stuff

non-stop hands up how long have you been doing it for what age did you start just everybody shout it out I don't 818 huh yeah so young a lot of you started young 1018 eight anybody younger than eight no I was seven myself one anyway yeah I had a UH anyway it was a Commodore 16 anyway good times I had no storage anything I had to write I had to rewrite it every time I wanted to run it and we can talk about that later let's see any other questions okay let's do that one there cuz I saw his hand first black shirt yeah

so the question was in case did everybody hear it okay I'll repeat it the question was is what is the advantage of using a red team either an internal red team or an external red team vs a company doing bug bounty programs like bugcrowd or hacker one or any any of the other flavors that are out there today so I would take it as controlled cast from a business perspective hiring a red team you're able to actually have a point of contact and collectively get some scope and understanding of what your concerns are the bug bug the bug grub project definitely has it has its value but it's more of swatting flies especially from the executive present but thing because

now it's who finds things rather than I'm trying to control and worry about something and maybe my worst-case scenario and that also really depends on how the red team pitches it and the scope of the assessment how do I take on it I guess I'm still trying to process the question so are we talking about true red teaming that's being crowd-sourced like the whole interdisciplinary or

so the company or you're speaking about his company called sin app okay I personally haven't heard of it I don't know if either of you have I have any I haven't heard of that service okay so it's kind of hard to say if we don't know that much about it but what I gather so far from what you're saying it's a crowd source type activity where they have vetted penetration testers Assessors red teamers of various sorts and I don't have an hour so the companies we come to them much like a bug bounty program I imagine they say we are interested in your services and then how does the work become available and how do the Assessors say I want to work

on that and how is it awarded

right so based on what you've said so far it actually doesn't sound that different to me the only difference would be from an internal perspective if it sounds like it does a lot similar it would compete with what GDS is doing in a lot of ways for an internal red team vs cin app you don't have to dedicate resources to it on a continual basis in order to have an internal red team you've got to have people on salary on staff but to Matthews point it sounds like you lose a lot of control right like whenever when things go sideways they they have somebody then go ring the neck of ya and say you you messed

something up you're in trouble now right I don't know how that would work with the company that's crowdsourcing that they would have to deal with sin app the company and I don't know what the but on the other side thinking about it there would be other value because now you have a collective of a larger audience of different skill sets so one other challenge is making sure if you have a small groups it has been there for so long one how do you keep a black box approach mean they've been there inside don't look at anything don't look anything don't look at that IPO now I want you to attack it right as well as

keeping skillsets growing while most corporations they say they give training but they don't have an unlimited budget you now have a cross of most people who are digging and having that cross spectrum so I second season without value in there but from a large business piece it's really back to that strangle who do i strangle for liability yeah I don't know what like how how coordination I guess would work as well like you know internally with our team we we've internalized we know what the skill sets are of the different team and and with the tools we have available to us so from a crowdsource perspective I guess that would be it not so much a

concern but at least a question I would have is is because there's a lot of coordination is required so how do they become aware of what's available to them oh I found this vulnerability here but I need you know some help the poppet or maybe they don't even know they asked to ask the question so I guess that would be a question for me but then I'm yeah go ahead now I have a question for him so have you participated in the Senate exercises else intact okay after I was ok I'm kind of curious who's in charge of the assessment like who would be the decision maker ultimately if it's like so there's managers here that basically

get to make the decisions and then in both cases they report to a customer of some sort if you got the crowd source I mean you're logging into a website you've got instructions or some kind of work order that's coming through do this do that but whenever the tester says had I found this can I go a little further how I'm kind of curious how the decisions are made on the fly ok

okay it sounds very similar to me it it would probably be a call savings advantage for the customer would be the big driver I would think but they lose the the throat to choke were there any other questions I saw some other hands I have one other one on the surface intact piece so would they hire someone who is more from the college level of that approach because it may be a value if they could get into that pool showing their commitments similar to the same thing as what we do here the more we show our skills the more kind of we get those pieces that might be an avenue of someone trying to get into the industry

of red t mean that approach rather than trying to do the HR I have to have five or six years experience I can't even get past the filter

no thank you we've got a little less than 10 minutes and I don't want to run over because we have another presentation but I saw your hand next so

yeah that's actually an interesting and good question so I I did that myself my degree was in environmental engineering so I was out inspecting smokestacks and worked for a state government and ended up inheriting a token ring novell network so that's how I jumped into technology and then when I went to bank of america you know i think a large organization gave me an opportunity to get my foot in the door and technology there and then have a lot of internal teams to become exposed to and and you know pick and choose and get hired you know internally transfer that way so i went from there to internal audit technology audited bank of america and

so that's how i got in i don't have anything really i mean i I could I can attest to transfer from a medic 2i t is how i did it basically long story short i went from being with marines running sick call handing out medications the doctor would leave went to Lakehurst New Jersey and they said all you get is a blood pressure cuff and I was bored so we had something else to do I think a big organization that has lots of internal you know a variety is one way and that's the route I took also you know it goes back to the passion living and breathing it doing it on your own in

the evening so even if your current job it doesn't really give you those opportunities if you're if you're dedicated enough about it then you can get in the door that way we hire network administrators that don't have security background we have our programmers in fact I think that's good they bring a good perspective I think what you're doing now communicating that you're interested to people who are either doing it or can help you get into it that's that's key I mean I'd almost anybody that's doing this has they had to get started somehow somewhere and it usually boils down to knowing somebody and they interviewed you in some way shape or form and they were like I like

you I like your skill set I want you on my team see you hand back there

that is a very good question I'm going to turn that over to Keith yeah that's a good question so I would I would look into I so I would ask for an example of a report a redacted report because again it goes back to you want to see that they can understand the risks we can tailor the risk ratings to to your business into the actual scenario and we're not going to just run a scanner and take the the boilerplate out we so GDS we you know try not to get into into sales mode here but um we we pride ourselves in highly customized reports and weeding out false positives so I think a good place to start would be in

addition addition your normal and what is your experience what is your background but get an actual example of a report and we customize our remediation recommendations to your technology so you know if if you're using asp we're not going to have a java recommendation you know in there or just a generic boilerplate recommendation so that that almost assumes that the customer is educated enough to be able to make that determination and and to assess the different companies skill sets that's a incredibly good question I'm going to turn that back on the audience from a moment cuz there are other people doing this work how anybody willing to take a stab at answering that question the question was how does the

customer that the services of all the options of vendors that can do this work

yeah does that answer your question you offer services such as this don't you as a vendor yeah correct you there's there's a good bit of Education involved even when the client comes and says hey I need your services and then trying to get to the bottom of exactly what they need is is a big part of it and it goes back to the confidence for that for that that customers paying the money if you can't explain your services to them how good are your services and nothing against it is a hard thing that's why I have such passionate about the color challenge

but it's not

yep yeah I think that's the point the so the the customer come in to you they're driven by compliance you mentioned that earlier they don't really know what it is they want or need and so right yeah

absolutely yeah that's that's a big big problem and a big question how do you vet the person's skills right is that what you're saying so there yeah I think if you can another question you might ask is I'm not sure exactly how you would ask it but try to gauge how much repeat business and long-term relationship they have because if they're just running basic scans and saying yes to everything that the client asks for they're probably not going to be around very long I mean they're they're projects that come to us that we we say you know no we're not we're not suited for that or we steer them in a different direction we because we think

this is you know here's what you think you're asking for but really what you want is this I mean it's not to sell them it's to steer them in the right direction and in some cases they'll ask us to engage a project and you will be like you know what we can answer that in just an hour you know we don't even need to bill you for it and so that bills that long-term trusted relationship so if you can get you know get some referrals and try to gauge long-term relationships that's another I think we has an ask the question yeah let's go green shirt right here

let Matthew so this is um right yeah the question is do you want to just give it keep giving the same reports over and over again and do you feel like you're just kind of rinsing and repeating or do you want to collectively improve its really what the crux is the red teaming is as a self going back to business security really we're just testing the rubber meets the road we're seeing if everything else is done all the processes all the training everything else back to why they spent all that money if we're not figuring out some way to learn from it it is I mean yes it's job security but some people are still

learning how to change their password 20 years ago chi dia rinse and repeat so I definitely see there's value in it for every single engagement no but it's going to be kind of it's a continual learning piece the balance that was always trying to keep making sure that you don't taint specifically we get into red teaming full going and pen testing trying to do reactions that that synergy between the red or your offensive and the defense it gets too much and they're muddling and there's collusion or there's assumptions already made which makes it challenging which goes back to that having an external red team gives you that whole other view and normally they'd only come in for a year and then

wait three years if you're doing a full cuz it's going to take a while to do a lot of this remediation if you're really doing a full scale you said scripts though you're not talking about just screenshots and repeatable steps you're like tools yeah man

that's so the I mean if you're employed internally though and you're in your goal is to secure the organization that you're working for your responsibility is to help train the detection and response capabilities that's why you're there that's that's why they're paying you yeah so I saw Lance's hand up when we're hiring people um it can be usually we try to provide two or three references and we have some good relationships with a few where you know usually we can find two or three and we try to find references that are specific to the industry you know for the client that's engaging this so it's yeah there's some that say no and don't want to be but it's not hard to find two or

three all right and yeah we're going to we ran over we're gonna I'm going to stop this now because we've got another presentation to go and then we're going to do lunch appreciate everybody appreciate you guys coming out and show some time and your experience in your expertise with the group here yes yes thank you for having us thinking take a quick break and get back here as quick as you can because we got some