Red Teaming vs. Penetration Testing: When It's Worth Doing

all right guys um so I'm I'm Dan Cannon I'm from North Korean security and uh I'm gonna moan all of you about red teaming um because I don't always do it but what I do want to do with an audience who can't leave the room and you don't have to agree with me uh fundamentally I mean I I've been a penetration tester for over a decade now um done my fair share of red teaming been part of engagements and done a lot of work with different companies of different sizes can I just get an understanding of the room how many people here are penetration testers panda perfect okay so no one can tell me I'm

wrong so in that case let's start with a little bit of History then about what uh what we've always done inside the security from an offensive security perspective so it once upon a time yeah computers were invented everything was Happy Days nobody had any vulnerabilities because no one could figure out what they were and nothing was connected to the internet because before many of your time uh the internet wasn't as pervasive it is now uh and yeah there was there was this no security testing done systems were left at risk um and realistically they weren't attacked that often because they weren't connected to a huge amount of things and then as things started to become more

interconnected systems started to rely on each other we realized that we needed to do a bit more bit of vulnerability scanning we needed to take things uh carefully we needed to have an appropriate approach to cyber security the word cyber probably wasn't as bastardize as it is now but we needed to actually understand where are the vulnerabilities what's interesting what's likely to get attacked and what's not likely to get attacked and how do we fix things how do we prioritize things how do we rate issues everybody familiar with good old cbss scoring of zero to ten and yeah yeah vulnerability scanning 101 is yeah 10 means bad fix that patch that turn that off if you can and we're

just trying to focus on Direct visibility this is all about just have a look at a broad landscape and turn around and say that's bad that's okay that's all right nothing's ever good because otherwise we'd all be out of jobs so nothing's ever good okay then time testing came along and penetration testing was all about actually simulating attacking computer systems it was actually about turning around and saying look this is connected to the internet we can attack that this is on an internal Network and you're assuming it's trusted so it's vulnerable and you've let that you know vulnerabilities rise up to number 10 out of 10 and it's a big problem but you're not fixing it let's get someone coming

and run a penetration test and tell you exactly what's wrong with this for everyone apart from the one penetration tester in the room apart from myself it's all about connecting to a network and being able to turn around and identify machines identify software and vulnerabilities Ryan did a great talk on Half-Life exploitation being able to figure out how you can leverage those attacks and those vulnerabilities as a hacker and then demonstrate systems and games the whole system's compromised data is in Jeopardy and your red lights should be flashing everything should be going on and for years that's that's what that's what the approach we took that was a proactive approach to cyber security penetration testing was

simulating a hacker now the problem with this is that this is looking at a security function or a system you might not be able to see it but ultimately this is turning around and saying I have a thing can a hacker attack me or I have a web app can I hacker get me or I have a network how secure is it great for getting visibility of that thing widget Network environment whatever it might be awful for getting a good idea of what an attacker might actually do if you're a penetration tester and you are told to attack the most secure network in the world that's what you're going to attack fact that there might be an unlocked

door to a data center right next to you that's out of your scope you're not allowed to identify that no hacker in their right mind would come at you this way when there's an Open Door over there the fact that there might be a wireless network that has no password that has the immediate access onto a network you're not allowed to test that because of that data scope this is the down part of penetration testing you are looking at a specific thing and it's the scope and we've got to stay in it otherwise we just you know devolve into hackers who are doing damage but enter red teaming that's where we are now the idea is that

we use the tactics of attackers and we don't Target systems or environments or things we target companies we target an actual real world approach of how a hacker would Target a company because we don't know how the bad guys are going to get in until we have a look at the whole landscape there is no point saying your company website is amazing no one can get onto it and just ignore the fact that developers have also put a website up next to it that gives immediate access onto a computer system there's no benefit there red teaming is the idea of looking at how real world attackers are going to gain access to companies and how they're going to compromise

environments and it's really good and it emulates real world threats and it takes the idea of ttps those tactics techniques and procedures and it replicates that in an efficient and ethical way now the problem is as cyber Security Professionals and with a room of not a huge amount of pen testers in here I don't know how big your egos get when you do work and I don't know how much you fancy yourselves as spies and bad guy hackers who are on the right side of the law but there are a lot of people who can turn around and say I'm gonna do a red team and then they do a pen test and they'll be like I'm going to do a red team

engagement and then they'll break into a building plant a Raspberry Pi and then they'll do a pen test and that's that's one of the big flaws we have there are dozens of companies out there who will turn around and say it's a red teaming engagement that means there's no rules we can do whatever we want as long as it's not anything illegal but there are no rules we don't have to break into the computer system or anything we can turn around to be super cool spies break into the building in a dead of night plant a reverse uh access but yeah essentially a pie that has a 4G dongle attached to it and we will get into this system by bypassing

your physical security because that's what cool bad guy hackers might do except in any in any real world scenario does anyone think that hackers are actually breaking into buildings in the middle of the night to plant a pie onto a network the core fan to their VPN like I mean hands up does anyone actually think that's what that's what hackers are doing putting that much risk into being identified that they're going to break into a building and they're going to stand talking to your receptionist and holding a box and tailgating their way in or holding 19 coffees or whatever you might have heard about with the Fantastic ways of social engineering that's it's not it's not realistic but

it's fun and that's the down part of the industry it's so much fun to do it you get great stories but it's not realistic and that's why I like to stand and whine about everybody who sells red team they're the minority who are doing it well and they are doing it with really qualified and really experienced professionals but there are a lot of people out there who turn around and be like right get the balaclavas on get the Raspberry Pi break into the building I have a colleague who spent four hours in a uh in a Janitor's Closet in a room waiting for the the office to lock and for everybody to leave to then burst out

and do a pen test you know I myself have tailgated into buildings and then texted the clients and sort of be like so I've shown the weaknesses of the physical security do you want me to break down the social engineering Foo or do you want me to sit here and wait for everyone to leave and do a pen test it's the problem we've got is the terminology in the cyber security industry is so fake and so misused and so misunderstood that we end up with people selling vulnerability scanning as pen testing red teaming is pen testing no security is vulnerability management because if you don't look at it you can manage it absolutely fine um and it's it's a real challenge

so when it comes to Red teaming and what's not red teaming we really need to Define what's going on and this is a conversation that's happening with a lot of different companies that are penetration testing consultancies red teaming companies everybody hates everybody and everybody disagrees with everybody and there's lots of conversations going on to try and get some form of standard of when one company says I'm going to do a red team another company who says I'm going to do a red team we'll do a similar thing and not one company is going to reverse engineer a piece of software come up with a custom binary exploited through a phishing campaign get onto the system

leverage their lateral movement through God knows how many domains escalate up to an Enterprise admin and then boom show all their work and another company is going to put on the balaclava walk in and plant a Raspberry Pi that's the problem we've got and the difference of red teaming and not red teaming can be seen as how complex and how long-term people perceive engagement so red teaming we're measuring weeks or months it's got to be like kind of realistic attack scenario you need to understand who's the victim who is the target what's going on where are the potential entry points is it a web app that's been put online 19 years ago that people have

forgotten about that still exists spoiler got into a computer got into a company through that exact exact way they said they had 19 different websites we found 47 of them one of them had a great login page that said the login will be four one four four two and that worked but it's not it's not a red team because it's it's just it's not really focusing on real world attack vectors it's just turning around and doing attack surface and exploration and then going through a pen testing approach it didn't have any tools techniques procedures of a real world attacker but doing a DNS enumeration scan we've got to think that ultimately the key outcome of something like a red

team engagement is not just to be able to say I've owned the network I'm the May admin I'm Enterprise admin if you know the difference between the two of them but to turn around and say here are the gaps that The Blue Team have here is the fact that you've got a sock that you might have invested tens of thousands hundreds of thousands some bigger companies Millions and here is the fact that they did not see a sophisticated attack walk through their Network and be able to compromise all the systems whereas for something that's not a red team we don't have to worry about that the vast majority of my career if a stock Rings me up and says

are you attacking the network ask them they go hell yeah I am that's that's the job that's the engagement I'm attacking this thing over here it's all about and you actually identify an exploit vulnerability and then just demonstrate it can you show how an attacker would actually rinse their way through a company or a Target or a client and be able to get access to the Downing information when people are talking about the crown jewels realistically in a pen test crown jewels are always just domain admin because hand testers like to feel like they're the best whereas in a red team engagement it's about what works for the company it's about understanding what is a client going to

actually be concerned about how is it going to damage them if they get attacked the fact that if anyone remember the talk talk hack from 2014 there's not everyone ultimately I mean talk talk got hacked in 2014 really easy attack it was SQL map um used as a smoke screen while uh while they also pillage data from other systems on the network but the damage to the company was actually reputational the damage was the fact that two years later someone knocked at my door and said have you thought about moving to talk talk and I laughed in their face and shut the door I was like you can't protect anything you can't have my personal data or financial data the

damage is reputational now it's something that's not real world any penetration tester would have said there's SQL injection there fix that you're done in a red team engagement we can understand that damage of someone being able to exploit this is reputational It's Financial it's the fact that you're you know the CEO is going to have to go on TV within it ludicrously updated computer behind her and being able to justify why you're still be trusted with computer systems so when we think about that we've then got to deal with General common misconceptions from a client perspective because from a technical perspective and from those within the in the industry we've got an idea of what we think a red

team engagement is it's about replicating the bad guys it's about being that super cool super Elite hacker from a client perspective those people who are actually purchasing red team engagements we've also got to worry about the education event because clients are I'm not going to use the adjective in my head clients just don't know what they want some of the time and they don't know what the words that we use mean but when it comes to Red teaming some clients just saying red teaming is pen testing because there are companies out there that say we've got a red team or a tiger team or a defensive team or a pen testing team or a security team and they

just use the words interchangeably so red teaming becomes pen testing pen testing becomes vulnerability scanning and it all just leads to disillusionment of where any value is red team is also also just a fishing campaign because red team is all about trying to trick people into into doing something it's about demonstrating human risk so it's just a fishing campaign there we go send someone a link see if they click on it later editing engagement it's so unless you're going to do anything whether that's generally just efficient um but what a red team is is it will make them more secure so the common misconception going will it make me more secure is that actually no it won't it's

going to highlight all the gaps that exist which is going to highlight where you're not looking it's going to highlight that you built up barriers over here to protect a system and you've left the door wide open in the back so it's really important to understand what's going on so when we think about any security testing especially red teaming we need to get the basics right before we understand is it even worth doing it because replicating the tactics and the techniques and the procedures and the tools and everything that an actual apt group uses is really good if we're dealing with a mature company but if you can turn around and just bypass a file because

it's badly configured they don't need a red team if everybody is following every phishing link that you send them and giving you usernames and passwords everything that you don't need a red team engagement you need normal awareness of what's going on if you've got software that is just again talked or having ludicrously outdated software and really vulnerable where is the value of someone turning around and going okay so the cool ninja bad guy hacker is going to hit you with a 20 year old exploit great it's not valuable because these are Such trivial ways of doing it and password security raise your hands if you believe that there is a company out there that does not have at least one domain admin

with Summer 23 as their password it's it's gonna happen I used to work at a company with 80 000 employees and the amount of time I'd see the year or the month or just keywords you just die a little bit inside but it means that there's no value in a red team engagement you need to get the basics right clients needs to understand how do they get security to a decent standard before they think about investing heavily and understanding how an attacker would get them and smart device security you know we all know people who are just going to plug stuff into the network or connect God knows what to the network there needs to be

that awareness of what's going on I dealt with one company who absolutely nailed this one year we did a regime engagement because they insisted on it ruin them within three days the next year they invited us back any time we touched anything on their Network plugged anything into the network did anything not only did they know they could pinpoint the geographical location of exactly where it is and be there within five minutes to turn a go how you doing anyway that they had the basics right we had five minutes to plug into a network stand find that there were no outdated systems no vulnerable software start password spray and get nowhere and then we had to unplug and leave because we

knew there was a countdown timer of how long it would take they had understood Year One everything's awful and they had put the effort into getting the basics right so so red team in the scenarios lots of people like to hear sort of War Stories of how things have gone and I'll try and come through as many as I can um but there are other team engagements that I've been on that uh that highlight the the good things and the bad things at Red teaming and lead to an understanding of why some things have value and something really don't we're going to talk about a global pharmaceutical institution where we ended up with domain admin and had

access to computer systems on three different continents we'll talk about a UK financial institution with a cset who was really proud that he'd identified the attack vector and blocked the IP addresses and then devastated when we told him that that was six hours ago and we've migrated from that it might have been not as well have been two years ago and we have talked about the global infrastructure institution if you got time who had a multinational uh site and they just had no visibility what they had how they had and their active directories and it shows that it's all about the basics so red teaming this is one of the first red team engagements of the

company I was working for at the time tried to do and they were obsessed with the idea of ocean um and being able to go outside of the box of a pen test and really focused on we should we should do flyer drunks we should go and turn up to their site and give people leaflets to a free thing to a new restaurant if they give us their username and password on a website didn't work it's not realistic it's not useful it wastes time waste money and quite frankly again what real criminal hacker is turning around and coming up with a really nice word art A5 leaflets to turn around and go there's a new Indian

around the corner follow this QR code and give me your password who who is going to fall for that and that's what companies are selling we had to argue with the sales team they were foolish didn't win the argument we had to do it anyway wasted time wasted effort But ultimately what happened with this one the pharmaceutical company we ended up having to chain together multiple zero day vulnerabilities in a web app um to then be able to gain access to a computer system and you can tell that the stage two and three really came into its own of red teaming because not only did it require really technically gifted Consultants but it required the ability

to turn around and go a team of four or five people was not enough this could be done by anyone a little bit of ocean understanding who your target is by time we started to get into the technical work this needed a fully fledged capable team to be able to shift around different people with their different specialisms anyone who says they can do red teaming with two or three guys they're either fantastic phenomenal guys or that ain't red dealing that's just fantastic you need to be able to have those really key skill sets and this was like I said zero day web vulnerabilities followed by an absolute hot knife through the butter of someone's active

directory to be able to hit three continents and dump shed loads of data now the value of this one was that again as pen testers and I imagine a cyber Security Professionals some of you have an idea of what other people need and what they should think and what is secure and what is important we rocked up and thought okay the main admin Enterprise admin access to computer systems you're a pharmaceutical company access to all the drug recipes and everything to be able to do things the key about our red team engagement is that in a pen test that would be great we should we could show all the different vulnerabilities to get that access in a red team engagement we got

to work with the CSO who turned around no no no no no no no no no the logging system about how we make drugs that's the key thing if we lose access to that we don't know if we've made paracetamol or cyanide we don't have any ability to guarantee anything that we've done so that meant that we could really turn around and go okay how would a threat actor actually impact you as an institution they try and gain access to that because at that point you will pay any amount of money to get it back so it's all about trying to figure out what does a client actually want instead of what are the cool things to do

for a financial institution we spent ages again stage one getting told by other people this is what we need to do for a red team we need to do a fishing campaign we harvested credentials you think great still can't log on to anything they've got IP blocking to be able to get onto any of their systems we can't do anything there's no value in just turning around and saying your users are not aware that they shouldn't be giving out uh usernames and passwords and when you do password uh credential harvesting overfishing you end up with a lot of people that are just putting fake passwords in any way as penetration testers uh the company I was working for

we had a phishing campaign set amongst us identified it within about five minutes and a company of 40 people gave away apparently 3 000 different user credentials because we just blatted the uh the landing page that we've been being tested against with absolute gibberish and you can imagine as an attacker there's no value there unless you can get someone to really give you an ability to execute something to log into something to achieve something there's no thought there's no anything and this is a key thing of client thinking it's just a phishing campaign it's being able to get people to give you passwords there's value but I don't know if you can really map that to a red team engagement so that's

not a lot there this is the one that we ended up having to go do a physical uh physical site visit um it was again not not overly realistic it's the idea of a USB jump drop it's a key thing that a lot of companies in the UK do they'll go and do a physical USB drop or they will break into a building and plant a Raspberry Pi and it has value to it because it lets you get onto the network but ultimately we then went through a pen testing methodology and we got domain admin and yay happy days we've got the main admin at no point during this red team that was sold to a

client with an idea of can you secure your company did we have any real action that took any effort that wasn't just a generic penetration test we did we did a fishing campaign followed by give us access to a computer and and get onto it there's a lot of cool stuff that happens in there and there's a lot of Teslas or companies out there that will turn around and go yeah but it's really cool and it's different from just the pen test but with a completely custom payload that we created and when they reverse engineered it they could turn around and say the guy who made it's called Steve because every parameter is a variation of the name Steve Stevie

Stevo Steve everything like that and it it took time it took effort and companies do that but there was really nothing here that happened that wasn't just a penetration test so there's no real value but finally this one this one was really interesting this one was where we turned around and went right let's let's actually do a red team engagement they had millions of pounds invested in their stock they knew what they were doing they were on it they could identify problems and we went down and assumed for each scenario and that's when you know a company is taking this seriously they don't care how you get in they want to know if someone gets in what can they

do can they get to our crown jewels because no penetration test ever starts with an assumed breach it starts with where are the vulnerabilities we don't care where the vulnerability was let's assume someone like someone onto a system someone opened up a thing someone downloaded a trophy let's assume something happened this took a lot of effort and there was so much work that had to be done to make sure that an actually capable soccer team that had all the tools that they needed they had EDR systems they had network monitoring they had IPS systems could turn around and not see what was going on and it ended up having developers spending days trying to bypass EDR systems to be able

to leverage vulnerabilities that we knew existed so that we can we can do anything about and it's again that kind of premise of a pen test will just fire and forget if a pen test tries to exploit a vulnerability the EDR stopped it great cool okay you've got protection in place with a red team indicating there's a lot of work that goes on to really try and think if you've got the Russians the Chinese anyone in your network they're gonna put time and effort into being subtle and into being quiet and they're going to understand what you're doing and so it's all about trying to make sure that you can actually provide that value of this

being a really mature client who knew what they were doing you'll wanted to test the stock and they are they're the best red team that we've been we've been part of because quite frankly they have that defensive side of the of the coin and it was a real test against their systems and their people and their processes not just do we have vulnerabilities we can point it quicker so one of the objections ultimately everybody turns around and says it's too expensive it's not good enough no one's it's not realistic I've got to give you access that's not okay and assume breach is not realistic the amount of people who talk about this new bridge are not

being realistic because surely no company ever has anyone ever do anything wrong and have hackers get into the system um and then we've got to turn around and deal with the challenges of everybody gets excited about red teaming and everybody then ends up selling the physical intrusion the physical red teaming whatever they might try to call it and people just revert back to that physical break-in where they just get to feel like really like exciting spies and hackers and bad guys and standing covers we're gonna quickly through this we then also got the last problem that charlatans in the industry exist and I'm sure we've all seen our fair share of snake oil salesmen this is one I had to

deal with when I was an internal employee at a company this was a company that were trying to convince me they had the skill set within their team to conduct a red team against us even if you convince me that you can do red dealing and purple teaming but you have none of the prerequisite skills to do any testing I would turn around and call you a liar um equally if you turn America you've got red team is who can't do purple teaming which is working with a blue team so red teaming with notes and timestamps and documentation I'm going to turn around and call you a liar but non-technical people look at this and go there's some skills there that's

some really cool stuff the social engineering this guy this guy's amazing he can run the team that's fine and it's a problem that we've got to deal with so is it worth it quite frankly for the vast majority of companies no no it's not we should stop selling it because it's not worth it for most people vulnerability assessment needs to be done as a minimum for most companies and as much as techies and cyber Security Professionals and Assurance individuals we might hate the fact that we've got to go back to basics clients are not where they should be companies are not where they should be and it's cool to sit here at the peak of

the pyramid and go I'm awesome but when you do that against a company that it's just awful it's like beating a five-year-old at a game of pool where they need a stool to stand up and see the table they don't even know what they're doing there's really no value to it and we need to get people to a point where we get the challenge as pen testers we get the challenge of attacking companies that are sufficient and good and have defenses and have the basics done and a cyber Security Professionals we push the idea that from an offensive security perspective there is value here if you're at the right part if your ftse 100 company who knows what they're doing

not I don't know Nando's who just turn around and be like oh you've got like four computers per per restaurant great there's not really a huge amount of value in any event not to listen to Nando's I mean they're great um and that's and that's where we go why we've got to think all the terminology and security is confusing to everybody including Security Professionals who are in really complementary areas of European and cyber but clients then just have it coming from all angles where they don't understand what's going on they don't know what they should be asking for they don't know how to protect themselves and and as professionals it's our job to be able to

turn around and talk about Crypt analysis from earlier and how that can help vulnerability research really and how that can help penetration testing or vulnerability scanning and how that can help and not revert to okay this is really cool this is really fun this gets money for the company let's go for it and just tell them that balaclava meant that we were actually about guy hacker and that Raspberry Pi is a permanent access device or whatever phrase We want to come up with and instead really focus on trying to improve standard of companies within the UK so that's my winch that's my moan I said it with red red wine I like to whine about this kind of thing and say that

everyone's doing everything badly I've been guilty of it as well when we get excited about all the cool things we can do and how we can help people but ultimately we've got to really think in cyber security loads of people are doing it wrong loads of us are doing it wrong and we need to really make sure that when it comes to the general goal I like to think that everyone in this room is here because we all want people to be more secure we all want to help people or we all want to show off how clever and talented we are but again being a five-year-old at a game of pool that's not it's not a challenge

it's not impressive let's let's try and upskill everybody else and let's try and improve everything else so that we can turn around and actually do something interesting that's me thank you very much

for any questions yeah one or two if there are any questions [Music] do you think [Music] so red teaming it's all about the weight of the injective and penetration testing it's trying to identify all the different attack paths and vulnerability and outset is all about here right here is everything well so ultimately when you've got a red team engagement and a client thinking they're going to be more secure after the fact you've pointed out one route to that crown jewels as you pointed out here is how an attacker can have the least resistance to get what they need so when they fix all that and think they're secure determine about yeah but I didn't say everything else

it's okay we we found a route that an attacker would take so it's about the crown jewels for that kind of identity yep always against Applebee's you know the difference against any individual plus yes and no typically they're against a company or a system that they have but there might be individuals the determine and say I want to know what kind of um what kind of attack surface surrounds me what kind of information there is about me it's referred you know doing an Oaths into being able to turn around and say is this person a risk when the timber I remember the Ashley Madison hack yeah so when Ashley Madison which was a fair dating website got hacked and

everybody's data got uh got compromised that became a thing with a lot of directors and a lot of individuals turning around and saying I need to know if my data is on any of these things because if it is I now represent a risk so there are times where you do it against an individual but most of the time it's a system or network or activity is because they are companies yeah wealth management organizations focus on that kind of thing if he's here's a person of high value they might get attacked thank you very much guys foreign