BSidesSF 2026 - The Great Credential Caper: How to Perform and... (Christo Roberts, Dan Hollinger)

We have an incredible talk coming up. Dan and Christo. Uh the talk is entitled and I have got to read this cuz it's many words and I am old and can't remember things. The great credential caper. How to perform and then defend against the nearly impossible to defend. Please give them a huge whooping round of applause B-Sides San Francisco. Guys, over to you. >> [applause] >> Oh, we need the mic back, Sam. Sorry. Can't get this off. All right. Well, we're going to be talking about account takeover attacks, but there's a lot of different type of account takeover attacks that that people can do like um like um keylogging or info stealers or shoulder surfing or even those OMG cables. I

don't know if you know about those. They look like an Apple cable. They plug them into their laptop and and they um they have a Wi-Fi card built into it and they can sniff out passwords that way. Um but that's not what we're going to be talking about. We're going to be talking about the easiest to perform attack, which is taking these previously exposed usernames and passwords and using those in a kind of a brute force way. That's credential stuffing. So, this is a bigger risk than ever because there are so many re- um breached passwords out there in the world. We'll get into the more of the detail of it, but it's the it's we're going to show you how how

massive a problem this is, how hard it is to defend against, and then I just spun up Claude code and and and told it to just hey, go evade the bot detections that are out there. Uh the the the captchas and and whatnot, and and it figured out in a minute or two. So, we're going to show how that works in a live demo. Um I was over at church this morning praying to the demo gods that the live demo works. So, hopefully that that that that follows through. And then uh we're going to show some defensive countermeasures, how you should think about defending against them. Yep. Good morning, everyone. Thank you for coming to join our talk. I'm Dan

Hollinger. I've been a product leader for over 10 years now across early Cloudflare, uh Zscaler, and identity security at Vezza. You know, I look forward to diving into this with you guys and Christo. And to highlight my favorite musical is Hamilton, but I have been listening to a lot of K-pop Demon Hunters lately. I don't know if anyone else has had that experience, but So, I have not listened to K-pop. Maybe I need to check it out. I'm Christo. Uh about 25 years in the industry, last four at uh at um at Cloudflare and seven before that at Akamai, but um the customers that I support are these large social networks that I'm sure you guys use and a lot of

these other companies that are doing Super Bowl ads, that kind of stuff. My favorite musical is Tommy, especially in context to credential stuffing. You know, he's he's deaf, he's dumb, he's blind, but he's still through brute force and and um gumption, you know, he's still able to to win and just like these attackers. And and and is that still your favorite musical? >> No no no. Uh as you might have seen the other night, I made my own musical for B-Sides. It's a really interesting story. So, that's my new favorite musical. So, check that out. Hope versus Anon. So so I want to highlight, you know, you've probably heard lots of theater puns and musical

puns over these past few days. We didn't create any new puns. We created a new musical. Like that's where AI is taking us. But I'm not going to sing. So, I don't we're not going to do that. But okay, so the big problem. So, Cloudflare put out a blog post a couple weeks ago actually, which was timely because I came up with this idea in December and uh this this was referenced in the blog post. So, we see about 20% of the world's internet. So, I think it's a good proxy for the entire internet if you think about it. We have a tool that goes to look at Troy Hunt's Have I Been Pwned database that's always getting

loaded with the latest um drops of username previously compromised usernames and passwords. And we saw that um 41% of logins were using some username or password that's been previously used that was exposed on the internet. And then we took that data and we went a step further and looked at uh the week around Black Friday and saw that of those previously used um compromised passwords that are on Troy Hunt's list, 95% during that week were used by automated attackers. So, the the big takeaway here is that not only are people reusing the username and passwords and they're getting exposed and leaked in data breaches, but then bots are now firing them up and using these left and right to just try and get

into different accounts. When they get into an account, typically they they you know, it's a green light. They they that one worked. Then they go and sell that username and password on the open market or the or the dark web either way as soon as they can in an automated fashion. So, that's kind of the the nature of how these things work. Yeah, ultimately we want to highlight and you may have attended previous talks where captcha solvers are a business. There are platforms out there you can again essentially get captchas solved for one or two dollars to cover a couple thousand captchas. You know, this has always been a business. This will always be a cat and mouse game. And so arguably

this has been automated for for quite a while and is only going to get more automated and and something to consider. But even with the captcha solvers, the this would be like an API call on the fly to go solve the captcha for you. It's even easier now with AI because you can do it in line like we're going to show you. So, something to stress is just the sheer volume of passwords that have been leaked. So, this is 16 billion with a B. In all likelihood, your password has probably been leaked from one of your your primary accounts. Uh another thing we want to highlight and take away from the slide is this is only 6 months old.

Uh this was happening in June 2025. Ultimately, this is happening almost every day. Not all of them are public. Some of them are are private. Your data has likely been exposed. We highly recommend taking steps to address that. So, I I uh just did a little bit of uh Googling myself on the open web and came across a torrent site and I was able to just download a about 10 GB file in a couple of minutes, a bunch of plain text uh usernames and passwords. Uh this is not that list, but this is a similar list, but uh the point is that they're out there. They're easily grabbable and um it's not hard to find.

And then, you know, that just gets the stuff just gets layered on more. This was what? Like um a month and a half ago and uh another 150 were exposed. And uh so so there's just so many out there. And this one I found because I looked my own email up like you guys all should. You know, it's another key with the key takeaways here is you should look yourself up and and your people around you, your loved ones, and whatnot to see what's out there. But um I found myself in this CarGurus dump that um I I leased a car 2 years ago and uh used CarGurus to do it and um and and there I was. Um

so, it's out there. And here's me looking myself up, motokristo@gmail, and um I used to be Well, I still am a a Detroit Tigers fan, but back in '84 they won the World Series and you know, that's my password right there, '84 Tigers for that that site. Um I didn't pay for this particular service. I paid for a couple others to try them out. Um but theoretically, I could pay for this and get the full details and and see them from there. Um So, zooming out a little bit of what credential stuffing ultimately is uh is if you look at kind of step zero, we have a whole bunch of human users out there still using uh credentials the

same password, the same login on all of their sites. Um these are being used across multiple accounts. So, ultimately all it takes is one breach for these legitimate credentials to be found by malicious actors. Those malicious actors will then set up automated attacks across all of the common services, email, banking, credit card. And the goal there is once you take over that account, you can use it for illicit purchases. You can start to scrape PII or additional personal information. You can start to take over other accounts because you're starting to form that web of a relationship for that individual. So, it all starts with that kind of weak password, the reused credentials, and forms a life cycle that ends up with a a

breach or with that account in trouble. All right. How many people here have a username and password that they use for more than one website? Yeah. Well, the people that aren't raising your hand, you're lying, you're being lazy, you're not raising your hand. Everyone's doing it. So, time for demo. Live demo. This is a list of 1,000 of very common um passwords from some security researcher. I forget his name. Sorry, I should just give him credit. Somebody I want somebody to pick one word out there. We're going to set that as the password and then we're going to crack it. What? I heard dishwasher. I heard dishwasher. All right. So, we're going to try this

live demo here. Okay, we're over here in and we're going to type in dishwasher. D I S H W A Okay, it worked. Okay. Okay, that got set. So, I'm over in the terminal window. I just put Oh, my demos are all on Cloudflare and so I just pushed it out to Cloudflare's KV store. So, it's there stored as a password. Now, we're going to hop over to Claude code here. And um if this works right, does it say node? Yep, okay. So, I'm just going to run this one script in Claude code. And the cool thing about Claude code like Claude co-work and others is that it now uses Playwright to open up a browser and and

act like a human. Um so, it's going through and I I created this to go So, even if you chose a word at the very top of the list, it looks through these 1,000 in a in a random order. So, it won't necessarily find it, you know, really fast that way. But um this works 100% of the time. Now, I know if captchas are here or you're rate limiting, then it would theoretically slow this process down. This is really just to illustrate how fast and quick it was to use Claude code as one example to um to to um you know, do this process. And um even if people were using captchas, we talked about how they're going to solve

them and we're going to show even easier how they're solvable. Or if they're using rate limiting, um since you would you typically if you were an attacker use a botnet of res- using residential mobile proxy IPs, um then uh you know, you wouldn't even get rate limited because it'd be different IP addresses and different signatures on every request. So, that that's that's not necessarily the answer. it's just you know, rate limiting and and um um CAPTCHAs and whatnot to slow things down. Um Longest this took was 70 seconds. I think we're up to about 60 seconds here. Uh Come on. Oh, it stalled. Wait, what happened? All right, demo gods did not deliver, but I got a video of it right here, so

we'll just show it there. All right, so skip to the end here, but um the idea is that uh it will solve it. We're going to skip forward in the video to popping open, solving it, solving it. And I I I can promise I worked in the driver. So, you know, it'll it'll it'll pull out the password, it'll tell you how long how long it took and um and whatnot. Okay, so that was the demo for that first demo. We got another demo coming up. Um okay. So, the other interesting thing was the first time I did this, this is Cloudflare's dashboard reporting for bots. So, we do a score of 1 to 100 whether or not a request looks

legitimate or not and then gives that to two customers to use as a tool if they could if they want to make laugh rules around it like treat automated traffic different than human traffic. And at first it looked totally all like automated traffic. So, I went back to Claude and I said, "Hey Claude, you're looking automated. That's not cool. You look like a human." That's all I told it. And then in a minute later it said, "Oh, I'm instead of using Playwright as as my engine, I'm going to use Playwright stealth mode." And then it did it again and now all of a sudden it looks human. So, the point is people are paying for these bot protection tools.

You look in the very middle green dot there, I know a little bit of eye chart here. Um but this is saying that this scored an 89 with close again closer to 100 more human-like. And so, now all of a sudden these, you know, is are bot scores you know, garbage? No, they're still aid to method as a signal in defense, but it's not doesn't have the same power that it used to have now that AI can solve this stuff so much easier. And this one just happens to say malicious and 10 over on the right just because this is one of the failed um tries, that's why it says that, but uh but still this would get through, you

know, with with the right the right guess. So, how does this affect the real world? Yeah, so ultimately this is not just theory, you know, this is happening to some of the the biggest companies out there that have security teams, have bright people. The goals is or the ultimately there enough holes in the system that even these companies are are running into to problems. And even if they are not directly breached, there might be exploited credentials from other incidents that attackers are then bringing to a PayPal or bringing to a financial service. So, these things are still happening uh daily, weekly. The the risk is there even for big companies to little companies. So, highlighting a second example of of

DraftKings, you know, was hit by a credential stuffing attack and account takeover. Ultimately they had to pay over $300,000 to kind of handle the the fallout from this attack. So, that these are this is happening live. This is not completely theory and is happening now with an ease and a scale that we we likely have [clears throat] not seen before in a pre-agent world. And and I wanted to call out here that this happened to them back in '22. It's not that they didn't learn their lesson. This is really hard to defend against. They could have the best tools out there, but they're still susceptible unless you want to cause all this friction and and make customers do it

you know, 2FA. And of course for corporations this is different, you know, YubiKeys you know, and and other keys like that are excellent and and authenticators. There's a lot of other tools in there, but my point here was you can't um you know, if you're um your customers are the masses out there, you don't want to install a lot of friction in their user experience. So, that's why it's really hard to do MFA even though it's it's smart to do it when you can. Um it's just not always available. Uh so, the the name of the game even last year when I was speaking at BSides on mobile and residential proxy IPs was about find trying to see if the requests

were automated or not. And if it was automated, it was most likely bad. We weren't really living in the agentic world yet. And now the game has completely changed. So, now the idea is to try and figure out not what's automated, but what's authentic. That's the key. You're trying to see what is happening with your traffic and how to treat them like are they authentic? Are they are they doing things you want them to do? Or or or are they are they being sneaky about stuff? And and and and the authentici- authenticity part of it is now cheaper and easier for attackers than ever. So, it becomes um you know, as we all know, the the mice just got a

hell of a lot smarter and the cats trying to trying to figure out what to do about it, you know? I mean, it's a cat cat and mouse game, but um it's uh now all of a sudden the mice are kind of winning the race right now. And of course this is because of, you know, the agentic AI boom, but it's not just that AI helps us do what I did as an attacker and spin up a botnet and and and, you know, an attack that way, but also um you know, you now we have to deal with agents. And you want good agents doing good things. So, you want authentic agents coming to your site and

buying things or I don't know, making bets on FanDuel. You know, you want to allow that traffic because that's still money. And so, what do you you also have to kind of think about that angle. Like even if something is an agent, you know, how do you know if it's a good agent or a bad agent, you know? And and what are you going to do about it? And this really all started um Anyone here know about the Ralph Loop? Yeah, a couple hands. Okay, let me explain everyone else. Okay, then you guys that know about the Ralph Loop might not know the history of it, but this guy Geoffrey Huntley, real interesting character, drives around Australia in an RV all the

time with his family. He was hanging out with his 9-year-old about a year and a half ago using Claude code to build things, to reverse engineer things. He's a quite a mad scientist guy. But his son they were they were playing a game and his dad kept having to go back to the laptop and and uh and and check on the code. And and his 9-year-old son said, "Dad, why don't you just put that in a loop?" And Jeff said, "Whoa." So, he came up with the idea of just telling the tool what to do, what the what the output should be, and then the tool like those of us that use Claude code know

dangerously um skip permissions, you know, that's the YOLO mode. You know, it you know, the tool will go figure it out like so what I did here with with the demos is I just said, "Go act like a human. Go go solve these CAPTCHAs." And it figured it out and it solved failed failed failed failed failed like 2 minutes later it figured it out. So, it's this Ralph Loop is the is the big um kind of monumental shift in not just programming world, but also attacking world, too, you know? And that's And then and then coming down from the not that we had enough theater metaphors yet, but I look at it as like coming down, you know, on the wires into

the into the into the show into the play to make it even more dramatic, the claw. You know, I got my Mac Mini in my I left it on my backpack over there. I was going to hold it up, but you know, we're all buying those and and um and and it's not that you need open claw. Well, for those who don't know, open claw is AI that does work for you instead of you typing into something and getting an answer back reactionary. This goes out and does things for you like make your bets on FanDuel or, you know, pay Dan 20 bucks on PayPal um for helping me out here in the presentation. But um

uh the idea of open claw is now the this mass I mean um Jensen Huang talked about it as being one of the core things for Nvidia moving forward. And uh so so this has so much more exposure. That's what I'm getting at here with open claw. And so this Wired article from last month talks about how um using this tool called Scrappling, this is a Python library that's been around for years to solve CAPTCHAs. Now all of a sudden it's combined with open claw because you want your claw to go out in the world and do things for you. So, now all of a sudden these CAPTCHAs are more solvable than ever. All right, demo two is actually doing

that, solving the CAPTCHAs. And this time we've got four different CAPTCHAs. Not live. We got four different CAPTCHAs. First, you know, I'm going to eat my own dog food. I work at Cloudflare. I'm going to solve the the Cloudflare turnstile CAPTCHA. It's not a call it um not not a CAPTCHA, but um um all right, so it spins up a browser here using Playwright. There it goes. It's got to type in it. Now this time we're using a known username and password from like a database breach. And I just really just wanted to show that you can automatically solve these things now. So, that was the Cloudflare one. Next up we got hCaptcha, which is

like an open source alternative. Um And then this too is just a checkbox kind of thing, but I just wanted to show that it works on different tools. And then after this one will be Google's V2 reCAPTCHA. Now it works for V3 also, which is the latest, but the V3 the one doesn't look good in a demo because there's no visual element to it, but it works. Um the most interesting one will be next here. I don't know this this um G test I think it's called the fourth one, but I don't know I don't know where they came from, but but it's the idea holds true here on the fourth one of it being like a a puzzle like that needs to

get solved. So, this didn't solve every time when I did it on on um in my testing, but it did solve it sometimes on the first try like we'll see here in the video. So, they put in that, they got to click the verify. G test. I don't know that where that comes from, but but see pull it over. Look at this. It beat 99% of users. >> [laughter] [snorts] >> I I think we're going to need to invent proof of love. Proof of what? >> Proof of love. >> Love? Okay. >> Yeah. >> [laughter] >> That's how we're going to move Is that your start start off idea? Is that what you're going >> we'll see.

All right, so so we need a hero in our story. And um you know, Troy Hunt, he's awesome. He's been collecting all these usernames and passwords from breaches in the past. He's been doing it for years. And um and he hashes them all so you can ping that. That's what Cloudflare does to to to to run that on the fly if someone types in a username and password on a website on Cloudflare. And we give this away for free, too, by the way. Um it will go it'll hash that username and password, both of them, and then go compare it to his list and then send back a signal to the origin or even to

the front end there to the user if you wanted to set it up that way that says, "Hey look, this username and password have been previously compromised. You probably need to go change it." Um you know, but as a as a customer of Cloudflare as you know, a business, the business can decide if they want to enforce that or not or how they want to enforce it or maybe some customers they might do a step up. And you know, if it's a valuable customer that, you know, you're afraid something's going to have happen to them, maybe they get you know a different um action than a than a mid-level customer, I guess, something like that, right? So, you can choose

different levels using this tool. So, by the way, I get mentioned it before, another plug, go to this site or go to this other couple of the other ones that are out there and check your own username. You know, check your own password. You can look at the enter in your own password and see where your other passwords have been compromised, as well. So, go check that out for yourself, do it for your loved ones. I think it's something that we all should at least be aware of, you know, like maybe you don't care about certain websites, but other ones you do care about, and it's worth it's worth taking a look at it. Yeah, so to highlight, on

one hand you have to be your own hero. Be your own hero for your family, your grandparents. You know, friends don't let friends use bad passwords. Um moving on to kind of the corporate side, there are plenty of vendors out there trying to resolve this problem, as well. You know, many with bot management systems, uh threshold scoring, all of the systems that were targeted towards automation first. And so, some of the concern is each of these are going to have to evolve over the the next few months in order to start addressing the authenticity problem, and not just the automation problem. And so, there are a wide range of of options here. Um and if

you go to the next slide, the main thing to highlight is even these are not perfect. You know, again, it's the the Swiss cheese model that ultimately they're trying their best, they're filled with smart engineers, to play this cat and mouse game, but the bots are going to be bypassed, CAPTCHAs can be bypassed, all of these systems will continue to evolve over time. And so, on the note of uh Swiss cheese theory, I want to take a quick poll. Who here loves onions? >> [laughter] >> Onions, okay. Who here loves parfaits? Oh, I thought everyone loves parfaits. I didn't know what parfaits were, by the way. I had to ask Dan. Does anyone not know what parfaits are?

Okay, we got to tell Dan. >> All right, so a parfait is normally a breakfast or dessert of like yogurt, granola, and fruit. It's got layers. If you guys don't get the reference, you know, onions have layers, parfaits have layers. Defenses should have layers. So, ultimately any kind of defense you're looking for for both automation and authenticity should be looking across these four layers. Ultimately at the password side, so that's the supply. Looking at the request layer, so what can we do to start to understand if this is a bot? Is it coming from a a residential proxy? Is it coming from a VPN? You know, behaviorally, what's going on when they're signing on to this

account? Is there anything that we can detect from an anomaly, from a different region or the like? And ultimately a new agent layer. What do we How are we confirming authenticity that this is an agent acting on behalf of a real user? Where did they get those permissions? So, walking through these each one by one or very quickly, um going into the password layer, step here is just reduce the supply as much as possible. So, let's eliminate or get rid of reused credentials with a password manager, passkeys, you know, using uh U B key and and Titan and the like on the enterprise side. You know, obviously, if there's uh the app level, how do we prevent these weak passwords

from being entered and used at the the beginning? And ultimately, reducing the sprawl of passwords to begin with. So, these this is coming in at the enterprise level with all of the IDP providers and SSO, um and then other privileged access management solutions. So, how can we just shrink the amount of bad passwords out there available to attackers? Because they're going to get them, if not now, then later. Then moving into the request layer, kind of three main things to look at are detecting and challenging automated traffic. That's what we've been good at these past few months or past few years. You know, ultimately finding ways to slow down that velocity. So, none of the

demos had a rate limiting, none of them were looking to throttle credential stuffing. Those are some easy or low-hanging fruit to to move forward with. And ultimately, attacking or disrupting the the infrastructure that they might be using. So, is this a a known VPN? Is this a known proxy? Is this a a virtual machine? You know, finding ways to to reduce that. And ultimately, on the account layer side, you know, starting to look at behavior across sessions, looking at abnormal behavior both at pre-login, uh potentially a way forward is post-login. You know, the behavior of an attacker once they've gotten into account is very different than a legitimate user getting into their account, checking their bank

credentials versus immediately changing the password. You know, different steps and things to to look out for that apps can start to identify those patterns at the account level, and then ideally take steps to increase the multi-factor authentication or increase the friction over time for those users. And then to bring it home, the agent layer is I think where we're still defining a lot of things. So, how do you define trust despite giving them the keys to the kingdom and letting them automate as much as possible? You know, this is where we're looking at signed requests, uh proof of possession. You know, how are we authorizing this agent? What policies are they running from? You know, strengthening the access

decisions, either again, based more off of context and not just a a token. And then supporting better controls across an industry or a company. So, MFA, better training, usage and and rules around agent use. So, to start to kind of summarize, really the it's might be the end of the line of using human as the trust boundary. So, ultimately, human plus automation is what we used to be tracking, but credential stuffing is becoming much more human-like, the much low lower cost. We're now with agents able to disrupt the CAPTCHA as a service industry, um and it's always been distributed and always been autonomous. But I I think that that shift is now occurring. Bring us home. Yeah, so

I I kind of got to apologize a little bit, because there are no silver bullets here. There's no really answers. I mean, that's why the talk was titled the way it was. This is more of I think a discussion and something that's things to think about and and and you know, each business might be different in how they approach this. You know, the tools are out there, but how do you how are you going to use the tools? So, that number one thing is like, does it look authentic or risky? Again, I want to just We have couple extra minutes here, so I'm just going to explain this thing this new thing that Cloudflare came out with that we announced is

hashed user IDs. So, what that does is it takes any username, hashes it on the fly, and lets you look at analytics on it and what it's doing, and you can decide, you know, what if that activity is good or not. But my question back to my team and and you know, for all of you out there is like, so okay, so you can see Dan moving around the website, but how do you how do you know it's authentic traffic? Like we use the example of like um I forget what they call it, where if you sign in five times from five different locations in in a half an hour, does it mean that you're unauthentic or are you using a

VPN? But anyway, like why would a bot even sign in five times within a half an hour and not just once and and move on? You know, um in my musical that I put together, I used Cloud to write the lyrics and all, but a lot of it was my own thought. I said in this presentation, too. And they talked about um this idea of like, you know, Hope there, she gets an alert late at night seeing 10,000 logins in like, you know, a couple of seconds. So, she and and they're all from different IPs. They all look completely authentic if you just look at the heuristics at the user agents and the IPs and the and but all

of them, 10,000 of them, went to go reset the password right away. So, you know, maybe one user logging in going to reset a password, that might be a common thing, but you know, 10,000 of them at the same time, okay, I can see that as being an alert for an attack that's happening right now. But you know, if it's I don't know, on a per user basis, I'm not quite sure how to find out or or understand what is authentic. You know, I guess it depends on your website and and your your application and your expected behavior, right? But it's things to think about and and discuss and and and figure out how do how to use

those tools. Um yeah, and so yeah, does the user behavior look expected? That's kind of the same thing as authentic. Um and then if it's an agent, then what do we do about it? You know, if it's um is it authentic? Is it is it doing good things or or bad things? And it's it's sometimes it can be hard hard to tell. So, you know, it it you know, good agents would definitely be automated. You look automated, you know, unless they you know, use some techniques like like we did here to to to not look automated, um so to look like more like a human. But you know, what matters is um you know, whether it can it can the the

agent can like prove somehow sign off on itself saying that I'm definitely this this agent and I mean and I and I mean to do good things, like some kind of maybe a cryptographic signature of good agents that are people track to to keep track of them. Um so, I don't know, that's one idea. But yeah, and and and then you also need to make sure that that agent is bound to some specific identity. Um Yeah, so that was the end of the talk officially. Um I do have um a couple of small fun things we could do, like I could pull up a browser and we could have someone put their hand up and take

their email and and look for passwords they have. Anybody want to do that? >> [laughter] >> Didn't think so, but I've got a couple of tools. You want to do it, Andre? All right. All right. All right, let's do this. All right, so um One of I we we had a reveal earlier, but DeHashed is a service I paid for, and I think it's kind of pretty interesting here. So, let me hop in the portal. Let me get my glasses on. All right, so here you can see that you can look up by email address, username, password, even like VIN number. So, if you saw some VIN number on somebody's car, or you could even look up their

license plate. I mean, that's how Uh there was a big data breach from ParkMobile. I think most of us here probably use ParkMobile's app. That ties your VIN number to your username and to your password. So, that was a big data breach that happened last year. So, you can, you know, look people's passwords up based on their license plate. It won't always work, but it but it but it but it is a a method. All right, so what am I going to look up here? An email address? All right. A N D R E Y R E Y at cloudflare.com. Yes sir. All right, so it is a corporate one, so it's probably not part of a data breach

data breach, but we'll just see what's out there. All right, give us a sec here. Okay, no document matches found. Come on, give me a personal email. Uh it's uh Or your kids Are your wife's or something? >> [laughter] >> Jake Jake N N as in Mary? Jake.n@gmail.com Like that? Yes, sir. Okay. Yeah, yeah, no identity theft at B-sides, please. Okay, there you go. I got two You got two. Now now these aren't going to tell us necessarily what Oh, there it is. Jason Who's Jason 545? Oh, that's the username. Never mind, not the password. Okay. So now we get passwords here. Now these are what I guess Cisco type 7. I don't know about

that one. But if it was um um um SHA-1, which a lot of these older ones are, and let's go look at MyFitnessPal, what's in there? Um a username and a date of Okay, so at least you have like a you can go to MyFitnessPal and guess passwords, but um um but uh if it was a hash, like one of these older hashes, then you'd go over here to um a site like this and drop in the hash. I mean, obviously this all happens automatically, but then, you know, it will try and solve the hash for you, and even if it can't, there's a service here. You could put in your email and and give them some money, and they'll

have professionals go work on solving the hash for you and spit out, you know, whatever's there. So my point here is that there are services out there that that are that are usable, that are you know, white, gray, black. I don't know how you want to describe them, but yeah, they're out there doing that kind of stuff. So we will end there. But first, before we end, uh see here. So back here. All right. Uh some fun AI slop here, you know? I mean, at first I wanted to put Dan there as Hamilton and me as Aaron Burr, but Dan said, "No, no, you're you're the you you started this." I'm I'm the Aaron Burr of this relationship. Let's be

fair. So I told So I told Google uh Gemini flash 3, whatever the latest nano banana 2 thing is, "Hey, just swap our faces. Don't you think that's simple?" But um it had longer hair with Dan's. Oh, yeah. So So Okay, this is the first one I started [laughter] with. And you see how Dan's got the longer hair, which is which which which looks great and all, and I'm all, "Hey, make me, you know, Hamilton, but give me the longer hair." Ah, come on. So I said, "No, no, no, no, no. No, shorter hair, shorter hair, shorter hair." Uh okay, that's a little shorter. Okay, more short More shorter, shorter. Oh, wait, come on. Now you took Dan out. You

You swapped Come on, what are you doing here Google? You know? So then uh then then I said, "Oh, well, put my head over there. Make my hair a little more gray cuz it's not really that black." And then it did this, and then yeah, so then I ended Oh, and then it made my our heads way too big. I come on, dude. AI is not perfect yet, but you know, you know, you get the idea. So So that's that. Well, yeah, we're I think we're ready for some questions. Any Any questions out there? >> My own? I'm on. Hi. Guys, that was awesome. I I'm also a big Hamilton fan. So it's been an amazing morning. Okay, if you do

have questions, you can go to bsidesf.org/qna. That's Quebec, November, Alpha. Uh and that will put you into the Slido system. We do have some questions for you already. So uh from the top. Um when measuring bot attacks, can you differentiate signals between AI bots and a human using a tool like W Fuzz or some human-written script? Okay. So uh how to how to determine if it's AI or not, basically, with with bot scoring? Yeah. Um there are some methods. I'm not quite clear on them all. Um I know Cloudflare has working on a couple of them. Um but nothing's coming off top of my head. Dan, you got anything to add to that one? Yeah, I I mean, you're typically

trying to look at past behavior of of of that bot or again, behavior of how it's interacting and how quickly it's interacting, but all of that is kind of what we're highlighting here, kind of old technology that's not really built for for the next phase. I mean, so so Cloudflare does have I think we have up to 700 800 bot signatures, and we have it stored at radar.com. You can go Google and look it up and see those. So these are bots that are kind of honest about what they're doing, being good good about it. Um and uh and so that's a way based on user agents, frankly, but also we look at the user agent plus the ASN number to make

sure it's coming from the same company. So we do a number of different checks to make sure that it's that it's honest about what it's doing and it's not just saying that it's Google bot, but coming from a different IP address. So there is that kind of like bots, you know, telling you that they're bots. So that's piece of the piece of the story there. Um Yeah, there was something else I was going to add, but I forget right now. But I don't know. Let's uh yeah. JA4 Yeah, well, JA4 is another JA3 and JA4 are fingerprints-based. Those are based on uh the SSL handshake. So you can very quickly see if there's a a botnet um

attacking a site cuz you'll have all these requests for a specific JA4 hash, and you know, that that would match up to the same uh web server and and and and TLS or SSL handshake, you know, even if it's uh you know, spread across, you know, multiple IP addresses. So yeah, there's techniques like that to determine whether it's um an automated bot or not, but I think the question was more focused on is it an AI tool or not? Um but uh I will say that a customer did come to me recently and said, "Hey, um we know Cloudflare, you guys say that Perplexity is a is a is a is an AI tool that's not honest

about what it's doing." And it's true. Perplexity can be pretty sneaky about it and not be as open and honest as the other ones. And but my customer came to me and said, "Are you seeing the same thing with Grok?" And I I did the research on it, and I found out that it's not even on our on our list of 700 known um AI tools and bots like you'd think it would be. And I asked our product leads, and they said quite frankly, the X team didn't really want to want to be on that list. They don't want to They They want to kind of play in between a little bit. So it's no

clear answer, but I just hopefully give you some data to think about. Yeah. I love this. All right, there's some MFA-related questions. They're kind of similar. Okay. Um is MFA the solution? And do you have any insights into how automation is affecting the MFA layer? Yeah, that's interesting. Automation the MFA layer. So um you know, there's the good approach and the bad approach. You know, the good guy approach and the bad guy approach. Like you maybe I want OpenClaw to get a text message to the laptop and have OpenClaw read the text message and actually put the code in. Maybe I want that so that my OpenClaw can do things for me. So that I guess that could be on the good

side is it I'm sure it could it could it could happen that way. Um from the bad side, I mean, sure there's like SIM swapping that can happen. So I mean, like I said earlier, um MFA is great if you don't mind having that customer user experience, but MFA is also not perfect. You know, um so yeah, there's plusses and minuses for it. I don't know. Do you have anything else to add, Dan? Yeah, I I would say not ultimately the solution cuz again, the use case of sometimes I want my agent to be able to do as much as possible. And there's also just the added friction. I'm sure everyone's kind of felt that of anytime you're using an

application now, you're getting the email, you're getting the SMS. They're almost defaulting to it if they're in a slightly riskier regulated regulated business. So I think MFA is part of the solution, probably definitely not the solution period. Okay. Um stolen authentication tokens. Okay, thoughts on that? Is that outside of the talk? Uh yeah, yeah, a little bit. Yeah, it's it's So if I if I got it right um man-in-the-middle attacks can steal tokens. They can, you know, log back in from a different browser, but look like it's the same exact session, something like that. I think that's how it works, right Dan? >> Yeah, either session hijacking or kind of SSO token hijacking that work across

different apps. Yeah, definitely, you know, fascinating topic, but a little outside the scope of this talk since we're really focused on exposed usernames and passwords and people logging in with those specifically and you know, that type of an attack. That's what we're focused on with the credential stuffing. Yeah. One more on MFA. Yeah, how effective is MFA when dealing with credential stuffing? I know we kind of covered some Yeah, yeah, we we covered it, but yeah, just double click on it. Yeah, I mean, yeah, it's it helps definitely. I think, you know, one thing I wouldn't didn't do too much research or talk through too much detail is this I mentioned it once here of the step-up approach of of

looking at what, you know, don't treat all your users the same way. You know, maybe a user hasn't logged in a long time, maybe you force him to do 2FA, but if they've logged in last week, maybe you don't. I don't know. I don't know what the pattern is there, but there is some you know, you don't have to treat all users the same. So you you can think about, you know, when you want to add friction and when you don't. Amen to that. All righty, any more questions, friends? You can pop your hand up. You can put the questions in at bsidesf.org/qna. You are waving your hand. Can I get to you? Andre, go ahead. I'll I'll repeat

it.

Yeah, that's That's interesting. So Cloudflare actually got the sequence analytics API tool where um you don't want to allow someone to go straight to a checkout API if they didn't look at a product page first and put a product in their cart. You know, Andre made a great point of um the same sequence analytics could be used for figuring out what that user is doing, and do they look authentic or not, you know? Like you know, Akamai had this thing where um people would type in a username and password, and they would look at the the keystrokes and make sure that the keystrokes are not uh all all the same uh space apart time-wise, right? Not all like 2 ms 2 ms 2 ms 2 ms.

So it would look like more like a human typing it. Now I thought it would be really interesting to go further. I don't think I've ever seen a tool do this or not. Um and they also track mouse movements, and they also have an SDK you could put on a mobile phone, and they use twist the mobile phone around the geome- the accelerometer and the uh geometric thing on it will will will change will will look like more like a human, right? But I thought it'd be more interesting to maybe even match up how Andre past types his password in and have it follow his pattern so that you know, more like it's like him doing it,

not even some other human retyping his password, which I thought would be really an interesting way to do it. Um I don't know. This is fascinating thought, but yeah, to your point, I think, yeah, watching the user move around the site and and giving it a confidence score, basically. If it went straight to the change his password right away really quickly, super high confidence score that it's doing something bad, you know? But you know, if but if it I don't know, had more of a human pattern to it, that would make more sense. Yeah, I I I think that's going to have to come in kind of just as a we assume a breach will happen.

Post-breach, let's track what the behavior is, what the previous behaviors were, and there should until again agents up the intelligent mouse game oh, do some fake work first. You still might have to pair it as much as possible to the signature of the real user cuz ideally you just have more usage data of how they're leveraging it versus when the bot comes in, they've gotten through the front door, and maybe at some point they'll learn to stall, but at least initially you're in that first wave we can hopefully track to that and again throw in some MFA or additional friction to just reduce the cost or increase the cost for that attacker cuz ultimately they will go to lower-hanging

fruit of an application to then try to to make their next move. One last thing I I uh I got a bunch of these free glasses to give away. No branding on them. So, if you see me out there we I think my brother's up here too with a bunch of other ones and we got a bunch more of these neck neck gaiters, too. So, if you guys catch me over there I'll give out whatever freebies I got. Nice. We love free stuff. Thanks, gentlemen. >> [applause] >> I have one last question for you. Can't hear you. That is an incredible hat. What is the story of that? >> My my daughter made it. Yeah, it's she

we bought it when we were at Yosemite last and she colored it for me and I was usually trying to call out my kids. Hey Hudson and McKenzie if you are still on the video, but yeah, she made it for me and yeah, that's That's super cute. I love that. Cool. Big round of applause, please, everybody. Thank you, Chris Donahue. Thank you.