Next-Gen Deception: Exploring the Evolution of Social Engineering - Stephen Bondurich

sorry for the brief delay there technology can be hard thank you for joining me today welcome to nextg deception exploring the evolution of social engineering my name is Steven bonrich I'm a penetration tester and Lead social engineer at 7x I have years of experience social engineering conducting real life engagements and observing how people respond when they are under real simulated attacks and today I'm here to talk to you about what the bad guys are doing to evolve their approaches and how AI is going to play a role in that already and how it will in the future so real brief what to expect here I'm going to be going over the different attack vectors that we are seeing real

attackers use um later talking about how artificial intelligence is going to be integrated with that I'm going to talk a little bit about how we defeat multiactor authentication in some of our attacks um and then finish up with a little bit of guidance on how to protect yourself so I want to begin briefly with this though because this talk doesn't mean anything if it's not a real problem that we're facing and I'm throwing a lot of numbers at you right now but I want this to just illustrate that social engineering yes it is still happening and not only is it happening it is happening more and more uh how many of you have done training

and received a test fishing email by a show hands pretty much everyone in this room how many of you have received a fishing phone call where somebody called you on the phone and tried to attack yeah not as many of you attackers are kind of keen on that too and that's why Vishing is becoming much more popular uh we're seeing a huge increase in that year-over-year recently because a lot of people are getting used to being suspicious of emails but they're not so much trained for that from the phone call side of things um and that kind of gives way for vising to be a much more effective approach to attacking does it still work um yeah it

still works and not only does it still work it is the most effective way for attackers to get an initial foothold in a network or establish initial access it's really common it's still happening um and that's why we're here talking about it

today yes so fishing is receiving a social engineering attempt over email fishing is when you receive a phone call and what you will see later smishing that's over text message um apologies for the graphics This was meant to be displayed piece by piece in in a PowerPoint um so I'll slide this to the side for now but we all know what that is it's a QR code right they're everywhere advert Ms check-ins parking tickets uh you go to a restaurant you get one of these instead of a menu can't be the only one that gets bothered by that but they have this perception of being a trusted technology and that what makes them so effective for

attackers we've all been trained off of fishing emails with hyperlinks so when we see one that has a QR code instead of the link it it doesn't align with that Paradigm in our head of what we're trained for what we're expecting in a fishing email and furthermore when we scan it we scan it with our smartphone so we have this idea that we're not putting malware on our computer we're just doing something on our phone all of that taken together kind of lends this to be a much more effective way than using hyperlink because attackers have realized that people are suspicious of the hyperlinks not yet so much suspicious of the QR codes the other

interesting thing about QR codes is that they can evade some email filtering Solutions right A lot of times a filtering solution will pre link it'll see where that link takes you where the URL goes to and it'll do some reputation checks on the web page not a all of them can do that on a QR code so this just gets sent through like any old photograph um and the filtering solution is none the wiser to the fact that there is a link in this email device code logins device code logins is this new technology that Microsoft has put out uh that's basically Divine designed for devices that don't have an easy to use keyboard like a smart DV a printer iot um to

enable them to sign into a Microsoft 365 account and the basic way this works is that the user from their Smart TV makes a request and receives from Microsoft a special device code that code then the user takes and goes onto their computer puts it in the the Microsoft website and then from there completes all their normal signin steps username password MFA on their computer when that signin is completed the smart TV now has access to a session on their account that's kind of what the general flow looks like uh down here you can see I have an example of what a command line tool right not just iot devices the hackers have sort of appropriated this

for for their own purposes um and this is a code that I was able to receive what does this look like from an attacker's perspective the attacker generates a device code on their attacking machine they send a victim that device code and a link to Microsoft's website the victim goes to Microsoft puts in their device code logs in and that gives the attacker the login session on their machine here you can see two examples of what you might see in a fishing email that uses device code login right here is one where it gives a bit of a pretext and it gives you the link and the code itself um that's one way you can receive

it a much less transparent way of receiving that would be something like a Microsoft team meeting invite where you go to click the join the meeting link and it brings you to the device code login page you got a meeting you know you're expected to be somewhere under the you know crunch of time um and maybe you don't think about it too much you just see a meeting code you put that in complete the login and that's how the attacker can get in like that it is powerful and it's powerful for a couple of different reasons the first of which is that everything that the victim does is on Microsoft's legitimate website there is no fish page

um there is no malicious URL involved this is all Microsoft's own infrastructure and they're doing it all themselves the attacker doesn't have to relay credentials the attacker doesn't have to beat MFA um it's all being done in a trusted space you don't have any malicious Pages involved um and that can make alerting on and detecting a device code fish a lot more difficult because there's nothing malicious being sent in the body of that email besides maybe a link to a Microsoft website um there's also some very interesting attacks path paths that are possible with defeating InTune through device code log and we've been able to steal the session cookies and use those from the command line to

authenticate register uh our own computer into InTune and adopt all of the InTune policies and suddenly we now have a fully InTune compliant device that we can use to access a browser session um and escalate further from there okay so pivoting into smishing this is fishing over text message right how many of you received a text message that you thought very obviously was a scam right and you think yeah right your package is being dropped off at 3: am. um you know it's almost makes it worse that so many of the smishing messages that we receive are so shoddy and poor quality because then when we receive something that's half competent we're more likely to believe it because

it doesn't fit in our conception of what a real social engineering over text message looks like um and they're effective for other reasons too right we're familiar with getting automated messages right we get suspicious URLs we manage appointments and deliveries um we access secure functions right so we're kind of used to getting some um traffic over our text messages uh and that can kind of make us more prone to believing a social engineering attempt you also can't filter text messages the way that an organization can filter emails coming through there's not that spam filter in place um and although it's possible in a limited capacity spoofing of a phone number is also an attack Vector that

attackers can use to impersonate uh the message coming from a different number here's an example of one that actually works you might see something like this um asking for a login because your authenticator session is expiring um or maybe you get an automated it help desk message that is related to an error that you're having on your account right the goal with smishing is the same as it is with a fishing email it's just the delivery is a little bit different and therefore our perceptions and our ability to be trained on them are a little bit different as well I have used both of iterations of these messages in real social engineering campaigns and they do work they are very

effective okay pivoting now into Vishing which is social engineering over the phone um there's two goals that an attacker can have have when they call you up they might most transparently try to induce you to do some sensitive action reset a password reenroll MFA you know give them bpn access change your financial details what have you a less transparent goal that attackers might have is to gather information from you and these are much more difficult to defend against the attacker will call you they will impersonate somebody that you trust or should trust like an IT Help Desk person trying to help you out with a potential breach and they might be looking for your personal information yeah

potentially financial information they could be probing about Trade Secrets they could simply be looking for you to confirm or deny a bit of information related to your company they might want to know what the processes in your company look like how does MFA reset occur what's the password reset phone number how do we get in contact with the help desk how are identities verified these are all things that an attacker could call an average employee and kind of just sus out this information try to get a feel for how things work um and this all is part of a larger reconnaissance effort to then use on the help desk so they can call up the

first and only time and know exactly what they'll be asked they'll know exactly what information they need to provide and they'll know exactly what to expect in terms of identity verification and this lets them be prepared this lets them be ready when they go back and try to induce one of these actions it's going to make that attack a lot more effective because they using real people to get information that might not be publicly available so why is it so effective we can spoof caller IDs I can call any one of you and impersonate any phone number uh that I wish the extra interesting part about that is that let's say I do my Recon and I find out the phone number

of your direct report your boss if I call you and I spoof your boss's phone number it's not going to show your boss's phone number on your phone it's going to show your boss's contact name whatever you have that saved as that's what's going to show up because your phone thinks that's the number that's calling um so you know you pick up your phone you're going to see a name that you know and trust and the color ID spoofing is is very effective for that reason uh again you can't spam filter with this doing open source reconnaissance um enables attackers to have a lot more information about a person when they get on the phone with them to inform the

attack and to make it more accurate and effective uh and finally it's just it's personal and it's human right you're not just reading text on a screen you're interacting with a human being and they can if they know what they're doing respond to your emotions they can respond to how they think you're perceiving the nature of the phone call and they can adapt and they can build off of that there is no one script for a vising phone call because every person treats it differently um and a skilled attacker will know how to respond to that person and steer the phone call in the direction that they wanted to go in a hot new Target for the help or for

Ving is the help desk the it help desk is um a location that the attackers can call in and gain access to a lot of sensitive account functions right if their goal is to take over an account they can call the help desk and look for something like a password reset or an MFA reset and you know this can be effective for a couple of different reasons compared to just calling the employees themselves um you know the help desk is there to provide services to people and to create that seamless frictionless experience to help them get on with their day um so sometimes they're willing to compromise a little bit on security if it means just getting

a person um to be able to do their job again this is particularly the case with thirdparty outsourced help desk companies who are quite literally paid to provide a good help desk experience and therefore in a sense incentivized to just get the user on with their business um perhaps at the expense of following their secure procedures uh the third party also just doesn't have the lack of vested interest in the company that an employee of that company might have um and furthermore it it you know it's not that hard to find the verification info for how the help desk is going to work um it's out there you can do ENT go on the dark web talk to

users you have an Insider at the company that helps even more um and indeed we do see this not too long ago MGM Grand Casino their helped us SC Vish they saw outages at all applications across the country um and it was just as simple as somebody calling in the help desk asking for an MFA reset compromising a a system administrator's account and escalating and pivoting from there some of you might remember this this was back in 2020 uh a teenager Ved the help desk for Twitter and took over a a very privileged admin's account uh and then ran a Bitcoin scam and if you can believe it I think that person made about a million dollars in Bitcoin

posting stuff just like this they had like Obama they had Apple they had malamore I mean it was just like crazy so now we're going to Pivot a little bit into how AI is helping the attackers get along with their business right so in the days of old you might have expected to see a fishing email like this Shock full of Errors typos and logical flaws how many of you think you would have fallen for something like this if you received it this morning honestly we like that uh how many of you can maybe think of somebody you know who you wouldn't feel comfortable putting this email in front of them yeah yeah and that's why it still

works but obviously it will succeed a lot more if it's more convincing and that's what we have ai for I can make a free open AI account go to chat GPT and ask it to cook me up a delicious fishing email but it won't do that popular GPT models um and AI llms that are out there they do have some ethical guardrails that are in place designed to prevent against this sort of thing but is it possible do you think for us to outsmart the most sophisticated Machinery ever devised by mankind and immediately after the first message reframe the context of our request could it be that simple yeah it's that simple and here's the email that It

produced for me what's also interesting about this and particularly devastating about it is that it can use very domain specific information related to the targeted service I I go a step further by using something not as widespread as Amazon but ibx health insurance basically ask for the same fishing email and you'll notice that it it spits out a fishing email that has very detailed information that only somebody who has an ibx account or is an ibx admin would no it knows the exact steps it knows what the UI is going to look like it tells the people where to click that adds to credibility in the email that adds to familiarity in the email um and overall

having the ability to have ai write very sophisticated emails lowers the bar to entry for attackers um and particularly concerning is that this is helpful to foreign adversaries who spent their whole life getting good at malware developments but never quite truly mastered English they don't need to they can put together the the one piece of the puzzle that they previously did not have the capabilities to use um and get a very convincing naturally English sounding email put together then we have audio deep fakes right this is more than just text to speech this is more than just Siri reading back to you from your phone audio defs are hyper realistic in the sense that they can capture an

individuals's intonations speech patterns through cadences um and they don't need that much data to be trained off of unfortunately due to technical difficulties we don't get the beautiful rendition of my voice um being cloned by an AI model but I will tell you this model was cloned off of just 10 minutes of listening to me speak in the office um and it was very very effective again these are effective because of Recon and ENT it can spoof caller IDs generally if you're getting audio deep faked what you might expect is that you get a call from a voice that's familiar to to you and they're in some sort of urgent situation they're in danger they're you know in a

financial Harm's Way and they need something desperately and they need it now they can't accept a call back they want it now um and the way that we get our and you know a common Telltale sign of that is that they're looking for a payment that's not traceable they're looking for wire transfers crypto gift cards um that's a very big Telltale sign of an audio deep fake attack they also have photoo video deep fakes um one of these is a real human one of them is AI generated based off of that human you want to take a guess who is who right side's the fake one and nobody knows that better than security company no before who hired the fake one on the

right there um as an employee this was actually a North Korean actor um he passed four rounds of video interviews a background check using a stolen identity his references this real man's references were called and then about 25 minutes after they give this guy his laptop they see malware being spread all over the network and he got flagged pretty quick so we're proud of no before for that but it it just stands to show that even a security company they can be duped by how authentic these AI video deep faes are all right defeating MFA so it's great that we can use AI to get username and passwords but we still have MFA so let's talk about some common ways to

defeat that uh for push notifications right sometimes you might just get lucky you might give them a push notification log an attempt when they're working right maybe the start of their work day maybe the um right after lunch when they're kind of expecting a session to refresh um and they might just to prove it or you could fatigue them by just hitting them over and over and over again with push notifications until they just want it to shut up and they accept it so companies saw that and then they said well what if instead of having them approve one push notification we give them a six digigit number to punch in that's six times the amount of TAPS it's

got to be more secure but it's not um it's just as trivial to create a page that you can capture in real time the MFA code after capturing a username and password um and realiz that into your genuine login attempt and then the latest one is number matching where two digits show up on your screen and you got to punch those in your phone that stumped us for a while uh until we developed some API calls in our fishing web pages that would allow us to after a user puts in their username and password on the fishing site go here we do the login attempt we get a two-digit number on our screen we send that user the

two-digit number into their web page they put that into their phone that gives us a login session and then we tell them to go away to make them think that the login actually completed um and that everything was working as expected so what can we expect out of the future two months ago open AI released the realtime API that is the first speech to speech AI model that can listen to words coming in and generate speech coming out um that it uses a set of preset voices so we can't use the voice cloning on that right now um but that's a huge advancement because it allows you to talk to Ai and get an AI

voice in response my prediction is that the next step is that the technology behind a real-time API will be combined with something like a worm GPT that is a GPT that does not have those ethical guard rails in place so you can tell it to conduct a fishing or social engineering exercise with very specific goals combined with the technology of something like 11 Labs voice cloning um and suddenly you have this model where any voice in the world can speak to a person and dynamically adapt using AI to what they say to conduct a social engineering operation I think that's the really the last piece of the puzzle that we're missing right now and I think

that's what we'll be able to see technolog is already there it's just a matter of the bad guys getting their hands on it so how can we stop this regular training and reminders of course are key but not just about the old stuff we need to include the new stuff we need to educate users what the new attack vectors are look like and what the future is going to hold for them they have to understand what information is risky to give away over the phone what sensitive actions are um should not be performed over the phone without outof band verification we want a culture that Embraces the reporting of potential security incidents we don't want people

to feel afraid of asking questions device based controls are a great technological control to prevent or mitigate risk against um unauthorized device logins and then of course regular exercises like receiving those fishing emails getting those phone calls those are Paramount as well as far as AI right we know now be skeptical you can't trust caller ID because that can be sco that can be spoofed um a good way to verify against that is to say I'm going to hang up and call you back because you cannot call back a spoofed number you will only call back the real number um the best piece of advice that I can have that I think is maybe the most novel is to plan

a safe word with your loved ones or maybe with your organization that in the case of a real emergency they can repeat back this safe word to you um to prove that it is them something that an attacker through no amount of ENT would be able to acquire um or ask an offline question questions say you know uh what did you have for breakfast this morning um what's the painting on your bedroom nightstand you know something like that where um only true true people that are themselves would be able to have that information um and just know that this is happening and it's going to continue to happening so it could happen to you it's going to happen to your

organization um and to be ready for it that's all we're thr on time now so I want to say thank you and I'll be taking questions out in the hallway