Faces in the Fog – Unconventional User Enumeration

All right, this is Faces in the Fog, identifying users through unconventional means. You may notice that I am missing Justin. Um he was the one that proposed this talk and this idea to me and then promptly went on vacation um as soon as we scheduled this for Bides. So, welcome. You get me today. Um you get me to talk about a one of the most exciting topics in application security. um at least in my view, right? Like there's there are some uh some alternative views that are out there. But our story here actually starts uh with this little string string that I've put up on the screen here. Uh a single quote or one equals 1 with a hashtag after it. You

may recognize this and it has nothing to do with user enumeration. I'll get there. Um this is my introduction into SQL injection obviously. Um I used to work for a little company called Omega. They made hardware and there was a click of death thing that happened. I wasn't associated with that, but I was a developer on the website and this little string cost me a week of my life because somebody put it into the username field on the login screen and the system decided to load all of the user objects into memory based upon this being sent to the database which meant that the system crashed because it didn't have the memory. We had Java out of memory

errors all over the place and we couldn't figure out what was going on. But this was one of my introductions into security in the modern era. And after, you know, having this experience moving into a penetration testing role, we started to test applications. And I lived for five to seven years on this strength. This paid my bills. This was in every single or pretty much every single report that I wrote for years and years. 90 to 95% of the applications that we tested had SQL injection problems somewhere in the application. And yes, it's gotten better over time. But my argument today and where this research is coming from is that enumeration flaws that we see in

applications as testers are as common in the applications that we test nowadays as SQL injection was 15 to 20 years ago. uh we did analysis. Part of the reason that we wanted to talk about it was the number of vulnerabilities that we find enumeration specific when we're testing applications is through the roof, right? Every single application that we test pretty much, you know, with a with a few outliers actually have problems with. So what does this go back to? This goes back to the basics or the crocs and socks of application security. We talk about authentication, we talk about authorization, we talk about auditing. And as a gray beard in the industry, it seems that I my job is to just talk

about the dad topics and the gra, you know, the crocs and socks of application security, which I'm okay with because these are the features that may not be super interesting, but are exposing our applications, our organizations, our users to risk that we probably don't expect. All right. So, that brings us to this idea of enumeration. Um, we're going to do a short run through of what it is, where we actually see it, some of the new techniques that are out there for identifying it, some of the research that we've been digging into with new tools, new availability. Um, but realistically, it's basic, it's uncovering authentication tokens that are used by the application to identify users. the most common uh you know

username or email address are the two that pop up over and over and over again. Sometimes we get other tokens that are associated with it and it you know give access to certain portions of the applications. We'll talk a little bit about that, but realistically it comes down to identification, authentication tokens, authentication flaws that are in the majority of applications that so basics of enumeration, we're just like I said, we're identifying users, you know, there's some number, there's some, you know, identity that's associated with these people or with these accounts that are in an application that we're testing or that, you know, we actually want to find out more information about them. Um, so enumeration occurs when an

application reveals existence of that unique piece of sensitive storage and whether or not that's username, a password, a name, an email, social security number, right? That was one of the the initial vulnerabilities that I actually dealt with was social security numbers. Um, but it's not explicitly a security issue. Uh so you think about something like Amazon what do they care if they have enumeration on products right or even like a number of different items that are in the database. Yes usernames might be sensitive user uh ids might be sensitive gooid values but not all of this needs to be protected in the same way. That is part of the problem that we're running into is that we don't

have the same protections built into place for those sensitive tokens that we necessarily we may need as we're moving forward. But anytime those tokens are used for that security decisioning, those controls that enforce some sort of authorization or authentication the application, that's what we're going to be doing. Um exposure allows for exploitation here. Whether that is through something like um credential stuffing attacks are fairly common nowadays uh or you know just straight up brute forcing. That's a common uh method and we're going to talk about that. So enumeration types, there's three or four different enumeration types that we run into. First of all, explicit and this is probably the one you're most um well okay. Yeah, it it's one of them that

you're probably most familiar with, right? Um a call that reflects positive user existence and usually these are on registration forms or other places like that, but you can browse directly to a resource, get back a positive negative response on whether it actually exists. So an API call like a restful call like this with an email address, hey, it's going to tell me this this exists or it doesn't and it's so that I can register for the application or do something else. public profile pages. um these it you know for especially social media applications what do we care right like this is the whole purpose of a social media application this becomes a bigger you know a bigger problem if it's a it's

a loan application or something else like that that data shouldn't shouldn't necessarily be exposed um testable valuables values here what you would expect email user name account numbers account ids um sometimes they're iterable right that gets us more into authorization and insecure direct object reference or bola access. Um, but there are there are there are times that that is used for authentication and is innumerable ID contentbased. So these ones are explicit that means like a direct call not a lot that goes into it. We get a response back very quickly. The content based are more what you are traditionally exposed to. You think about the login screen and it's going to give you some sort of user not found or

you know login failed right like different messages on the screen specifically that built into the content of what you are viewing. So examples uh so I've got that for the login screen. Forgot password is another place that we see this often. Um and sometimes it's the flow of what's going on but normally it's just some sort of content response that's in the application. And then we have airbased. These get to be more nuanced. And this is where we're seeing more and more of these actually pop up. It's not uh you know, we all know about the login screen that we shouldn't actually be giving messages that are um explicit, right, as far as whether that user exists, but we

often forget about the error responses that turn around and give us that same sort of information that we complain about a a specific component or object that we're looking up. Um, so if it says 404 not found versus 403 unauthorized, that's enough for us to know that that object actually exists in the API or in the application and we can use that for authentication tokens. We can start to actually gather that data back up. Um, so I've got examples up there again the same kind of like IDs that timing attacks get very interesting as well, right? So this is the the timing type that we're talking about. um the the way that we actually construct

applications um can give indications based on how long it takes to respond on whether or not the object actually exists. Think about a login flow. Um if the user exists and we're doing a brypt of the password, you know, we're creating a hash there and that takes three or 300 plus milliseconds to actually perform and generate that hash. If we don't do that same condition when user does not exist, um there's enough time difference between those two items that someone could actually start to identify what is there and what exists. Um and again, like this is stuff that we've seen. Same thing with like uh edit profile screens and other places, 403 responses. if they do the lookup first

as they're pulling data back and the database lookup takes longer than it is expected uh when a a user exists versus doesn't exist that's a positive reinforcement that we can actually enumerate and put things together. So testable variables anything that triggers those expensive operations. All right so locations places we see it this is where we're going to get into a lot of the examples. Um, first one that I have up here is an example of a login screen. Um, and all of these, right? Like, uh, funny enough, like we we test applications, we look at source code quite often. Most of these examples that I'm putting up on the screen are actually all from the same

application. Um, and it was not one that was purpose-built. So, um, sign in with credentials. We've got not email. The problem here is the email required versus invalid passphrase. um when the email actually exists and then it's in the database in this lookup there's a little bit of code snippet that we're missing there. It returns an email a required message um if it you know the messages are different basically if the object exists in the database and it does not. That's very simple. Um you'll see these sorts of messages on login screen quite often. Couldn't find that email. If you're a member you'll need to create a new account. Um and you know that's you know incorrect password right

like that that sort of contentbased enumeration is fairly easy to detect. Um we see it in scanners should be able to pick this up but funny enough we won't actually see a lot of scanners that identify enumeration in this manner because it does require some context and some you know some forethought into what's actually going on. Um capture triggers can subtly indicate a real account as well. uh again you know some of the timing and other things that run into that hey if the account exists then we actually require the capture um or we don't or the flow itself hey if the account exists then we ask for an MFA token or then we ask for some additional

piece of information uh my initial experience with this was actually at um Zans as I was helping build their pentest team uh the web developers decided they wanted to onboard everyone in the all of the customers all the accounts that were live into the new web application that they had just built for web and online banking. So they decided to build a form and ask people the one piece of information that they knew about them in order to register that single piece of information that they asked for for customers was the social security number. Um and all they were doing was confirming all the social security numbers that were valid for accounts. And then on the next page they

would ask account details. Um, so all we did was went to the social security website, said, "Hey, give me all of the Utah based social security numbers, like prefixes that were issued from 1960 to 1980." And ran that through a brute force tool and were able to identify out of a million requests that we made, something like 300,000 of those social security numbers were valid for customers of the bank. And again, this is just the account login, the account registration process that we're looking here. Um and that goes to this sign up flow right unauthorized access like this has some other role issues that are associated with it. Um but you'll see the validation of the email right like

okay it's required number one password is required but then it does this decrypt hash down here and if the existing user is there then it gives a user already exists message. Simple enough. Um create password. This is and again they they we call these all different things and this is part of the problem that we see with the with enumeration in general and the reason that we find it is developers forget that we're identifying users in other places besides just the login form. So create passwords, user signup, forgot passwords, what what have you. All of these different locations end up being end up showing existence when they don't intend to, especially to users that aren't necessarily authorized to

know what that list looks like. So in this case, right, like we've got, you know, test legacy Rachel is the username that we're testing here, but we get an error back from the API that the username is invalid because it's not available. There's a little bit of context that there and again I think this is why we haven't written tools around user enumeration up to this point is it is very context application specific, endpoint specific whether or not we actually want to run these tests. All right. um existing user. I mean, this one's pretty, you know, easy, right? Like it's already in use or status is pending. Now, in this case, this was found on an account update screen. So, I

already had to have an account in order to actually execute this code. Um, so on the account update screen, I would drop in a new email address and say, "Hey, I want to use this." But when it already existed, it would tell me that was a problem. Um, and you know, again, like from a threat perspective, whether or not this is something that we're concerned about, it's one thing if this is a social media application, you know, it's linked in, you already have their email addresses, they're sharing it. It's another when it a is a dashboard that banks use for loan applications and you as like the the the application developer does not want the

customers to know who the other tenants that are using who the other competitors are that are using that that same platform. That's the sort of risk that we're going to start talking through and thinking about these screens. All right. forgot password send off email. Um yeah, again, right, like you know, it all kind of comes back to the error messages that we're sending back. And we're going to dig in a little bit here shortly to the responses and what they should what they should be, right? Um whether or not the email can be updated, whether or not we can process those chains, what have you. Um that yeah like when when we built out this list of just

like number of items that we'd found across theationcape it was was rather eye opening to us as well just how many places it can exist in an application that forget um yep ballot accounts delays timing everything else and then it's the second that we start to deal with other sensitive tokens EIN in this case right uh We large cut large applications that service multiple organizations probably don't want to you don't want your social security number you don't want your business EIN exposed to attackers in this manner because it means hey I know that this is a place that I can find data about you know Seth's company uh because I looked up their or the EIN

here and it told me that it's in use and it's already been registered with this with this um with this service. So account lookups uh you know error based enumeration is pretty common API wise. Um we don't always get the error messages back across like this. A lot of time it's it's suppressed nowadays. Um and I think this is part of the reason that we're seeing this from a location perspective. We know that the error messages shouldn't show up in our React application, our front-end application, but the APIs are very explicit about what they return because we built requirements for them to be explicit to the developers. Um, and you know, just inserting yourself into that

flow, you can actually see a lot of this as it comes back. All right, account lookup. Um, account lockout. Uh, this is one that people often forget. So in the case of this code that we're looking at here, okay, we've got a user exist and user password reset requested at and then it returns like this password reset email has already been sent. Now, this means if I'm actually looking for a positive negative response, I have to make two requests, not a single request to actually build out my list and perform the enumeration attack because I've got to send it once and then send it again to see if I actually get that uh that message coming back to me. If I don't

see it coming back, then I know it doesn't exist. It's going to take a little bit longer as an attack. I just doubled the amount of requests that I that I have to make, but it still is, you know, that's more than automatable. Um, login. So, account lockout, this becomes a little bit more expensive as an attacker. So, that one's double. Most account lockout schemes will happen within, you know, 5 to 10 uh attempts to log into an application. and the user will get locked out or you'll start to get messages saying like, "Oh, the lockout, you know, you only have three more attempts." Something like that. The second we start to give those messages

back, it is confirmation that the account exists. Um, you know, an account that doesn't exist. Yeah, if I try it 10 times, I don't get locked out. All right, that's great. I'm going to move on to my next target. Go. All right. So, from an identification perspective, how do we actually identify enumeration? uh this is the traditional po approach right I'm going to identify the possible endpoints I'm going to send in manipulate the known valid values see what responses come back um and then I'm going to check the response for those enumeration indicators whether it is timing attacks whether it is the feel anything across the board that we've talked about up to this point um I think part of the reason

that like like this gets again to be difficult is that um there's not a standard like a one-off approach to actually do this it it it is very contextspecific and if you look at burpuite that's what I've got up on the screen here there is no specific tool or specific target that actually burpuite uses to perform enumeration um or to positively identify enumeration. And there's tools that we can use for brute brute forcing that are built into it, but there's not even a plugin um a BR plugin to to do that analysis for us. Now, there's some of the new like Burp AI that stuff that's coming that may actually be helpful from this context. There's the authorization

uh the O analyzer and a couple other plugins that will give us some of these indicators. They've built in a number of presets like usernames and you know email address generation that we can use but again it it we were turning to intruder at that point and we're just using it as a brute force tool. There's no reason there's no real knowledge about the application to identify those those users or those email addresses. So we turn to static like let's turn to the code let's take a look there but it's very much the same process. We identify the end points in the application. We think a little bit about what the response is going to be. We're going to

look at the error messages that are coming back out. Um, and then we have to analyze all of these flows. Look for timing attacks. Look where look for password hashes. Look for secure comparisons. Look for database lookups to actually figure out whether or not we're going to be vulnerable to this. Uh, what we found over time is that the best approach is to look statically and then test dynamically. That's very common in a penetration test, especially the open box ones that we do. Um, but it is it is still a timeconsuming activity to step through each of these and actually like you know to make these calls and we want to speed that process up. So um you know if we're looking st

uh statically we're going to look at you know this is out of a Django application. We're going to look at all the URLs that are being called that are being accessed. Some of these may have some sort of enumeration in it. Um, from a general swath without experience, you're probably going to call out the login page. You might call out register and um, you know, what else would we call it there? Maybe the change password, forgot password, reset password flow. Uh, those, you know, are fairly common. and we know that those are going to be an issue. But I'm there might be other places that we're forgetting about just because we don't know what this

application does, how it's handling those tokens, and what else could actually go into it. Um, so if we look, you know, we start to do that trace further on, we're going to jump into I think this is the login flow right here. But you do see account lockout there. Oh, you know, you failed this number of login attempts. Okay, there's enumeration right there. And then below that as well, you've got uh differences failed login versus invalid username that's being sent to the the rendered page itself. I'm still going to have to trace further and further and further. It's be it's going to become a little bit more um of an intense project to actually get through

it. Um this example comes out of a different one. Oh, this is the one that I think I copied it in. Oh, nope. Yeah. So about halfway through this one, there is the compare sync that's there where we're actually generating the token um the hash token or the the hash value for the password um from brypt but we only do that when the person exists. So it's a timing attack, right? um you know uh and this application itself was actually safe from all the other enumerations that we found but the fact that we it wasn't actually uh doing this timing comparison when we're you know when we create the password um if the person doesn't exist it was enough of a

a distinction even over the internet that we're able to identify okay so I want to talk about modern ways to actually do this and identify locations where enumeration can actually exist Uh it's all the hotness, right? Like the talk before me, they were talking about pentesting, like red teaming with LLMs. Uh we're going to use we're going to do the same thing. Um but we're going to use it to build context about enumeration and speed up our identification of the of innumerable endpoints and maybe values that we should be concerned about. Um, so instead of, you know, trying to do this manually inside my brain, looking at the code and doing everything, I write first

thing I'm going to do is actually browse the application. I'm just going to browse it with Burp Suite. Um, I've got that preloaded up here and I'll show it here in a second, but we, you know, we browse like a vulnerable application. I know that has a multiple locations where enumeration exists. um exercise all the possible functionality where authentication could happen, where it could possibly be be a problem, and then I'm going to export that session and I'm going to send it to an LM with a prompt that's using reflection, using modern prompt techniques and ask it what it thinks. Um, now I've done this with multiple different LLMs. um everything from deepseek like the reasoning LLM you

know the newest versions of chat GPT to uh Gemma Gemini and others I found that the haiku models uh from anthropic are actually the best at the identification and even spitting out details as I want them so that's what we're going to use in this case um and so let me just run through this so the first thing that I did here was I actually ran through so this little application Anybody that's been to my talks or has seen this before like probably recognizes the t the vulnerable task manager that we use for all of this. Um log in we can do whatever right like this is just an a purpose-built like vulnerable application that has a whole

bunch of vulnerabilities that are to it. Um the next thing that I did is I actually jumped over to here burp suite. I took this vulnerable application. I dumped out all of the the session itself, which is just just XML HTTP requests and responses uh that I'm interested in. I want to know, hey, is there enumeration that actually exists? And then I swing over to actually writing a prompt, right? Um I've got a system prompt that's laid out that's using reflection. Um, if you want to talk about AI and LMS and how to actually use them to identify vulnerabilities, whether it's in code or whether it's in running instances, uh, like we can do that at a later date. Um,

I am going to post this up on, uh, my GitHub here shortly. So, Seth Law is the g is my GitHub, uh, handle, um, if you want to play with this on your own. But what we're doing is we're using reflection. We're doing analysis. Um, and then I'm asking it basically, hey, I'm giving you an HTTP request. I want you to analyze it for the possibility of user or email enumeration. Let me know what you actually find. And I'm only looking at the request. I'm not even looking at the response in this in this using this tool, right? Um, yeah. And then, you know, realistically, I'm just pulling apart that XML. That's all that this is doing. But the meat of

it here is, hey, can I use an LLM to actually speed up this process so I don't have to spend as much time thinking about enumeration um as I did in the past. So, uh I'm going to kick this off here. Take a look. I'm like, do I need to pray or, you know, sacrifice a goat to the demo gods really quick first? But here we go. User enumeration. So, it's parsing through all of these requests and you're going to come back with this like possible user enumeration. Okay, I've got no there. Project details. Yep. No direct evidence. I tag this with justification because I want to know what's actually happening. Um, no, no enumeration. I

went to the upload page apparently. Nothing that's direct in any of this so far. And again, I'm just looking at the request. I could expand this out with with the response as well. Um, and most of this, you know, looks like it is, you know, fairly valid up to this point. Now, I get to this. Okay. Download profile pick. Now, this is where it gets interesting because when I designed this application and I was thinking about enumeration, I didn't even call that URL out as one that was possible. But it is saying yes, look, it's possible user enumeration because in the URL I've got a three. I've got a user identifier that exists and I can enumerate through and I can

use that to populate my list of possible user IDs. Okay, forgot password. Yes, it definitely is saying like, hey, look, there's like, you know, whatever, right? like it looks like there is the possibility here to do user enumeration and all I'm doing here is I am refining the list of endpoints that I I as a tester am going to go jump in and actually start to look so yeah like you know so we're stepping through here there's 40 requests or so um again I'm using anthropoc model I'm using AWS bedrock in this case right to actually do this analysis um there's some interesting postp parameters in here where I'm like doing injection and other things. Um, but you know those download

profile pick, forgot password. Okay, manage groups, right? Like doesn't look like there's anything in there. Um, but it does say that there is a refer header has something with Chris in it there. So, it's pulling things out of the components that could be interesting to me over time when I am thinking specifically about enumeration. Now, again, I get to the profile URL profile 3. Yes, user enumerization is possible there because of that public profile like we talked about in the past. Reset password searches may or may not necessarily be there. Um anyway, okay. So that is the demo gods worked that time. So thank you for sacrificing the chicken whatever it was. Um that gets through the list of 42

items that I had in here. And honestly this took me the script took me probably 15 20 minutes to write to actually you know refine this analysis from a enumeration perspective. That was you know time well spent. And again initially like I went through and okay that was the only thing I was looking for. But if we jump back over to the presentation here that brings me to actually I attacking the exploitation or attacking the enumeration and being able to pull data back out. Um in you know traditionally what this has meant is just been like straight through forcing like I showed before. All we have in Burpswuite is intruder that we can use for this but we do have to have a very

specific payload set. Um it's automatable but it it is also easy to detect. Um the cons here at bot prevention is that we start to run in to large key spaces, right? Like yes, Burp gives me a list of, you know, 200 400 usernames that I can use, but how valid is that for custom applications? Yeah, it's not very valid. I'm going to have to go and do some research and figure out what's actually going on there. Um, but you know, it's effective over time. This is an example of enumeration of various tenants or various customers of this application that we were looking at that actually led to different websites that were implemented on top of this

platform. And because they were only using three character token values to define where those dashboards existed, it was very easy to enumerate who are the other customers of this of this organization. And again, these are competing customers. They probably don't want to know about each other. And it's all because they're just using these very innumerable uh values that are there. Um and again you'll see that response like the length response that we're coming back from. This one's you know pretty easy to determine actually. Um same thing here like building a list of usernames right like we're taking this uh list that has we had to actually prepend this and do a little bit of OSN to figure out who

worked for this organization. build a list of possible username and then we have like invalid accounts that exist as well. Um and it all came back to the response timings, right? Um how long it actually took for it to respond when it did exist. So this is the flip side like the timing attacks I talked about before the um we thought oh it's going to take longer for a valid account to come back but in this case it actually took what five times 10 times as long for an invalid response to come back when it aired out as opposed to a valid account would come back almost in instant the problems that we have with

exploitation like I was saying building a list of possible objects like if it's gooid values or something out we got to figure out a way to actually retrieve those the key space for email addresses is huge. Anybody ever written a reax for email here really easy to capture all the space that's possible. Um bot protections actually prevent a lot of this from happening and gooids make enumeration you know so it's nearly impossible again gooids are you know random. So from a solution perspective, we you know instead of depending on what's already out there, we want to extract valid identifiers from the application in some way uh when we're doing the testing so we can we can get

this response and we can build these tools quicker. Um OSN here is your friend. We can use LMS to to build a list of those custom domains, uh usernames, whatever it is that's actually there. Um thankfully right like both Google and Microsoft are very very helpful when you want to enumerate email addresses. Um you know an organization exists you know the format that they use. Um anybody actually have like their own email server that they host anymore? Does anyone want to admit that? Okay. Yeah. So everybody's using either Outlook, you know, or office.com or Google Workspace. I guess there's some, you know, outliers that use iCloud or whatever Apple it is, Apple has, right? Um, but as long as you can

identify that this is where the email is going, it's pretty easy to actually go and say, "Hey, do these actually exist?" I don't have to attack your application to get this list. I can go to Google or Microsoft for this. Now, you know, there's some bot things that are out there that we're going to, you know, deal with, but it's pretty easy to build a list of valid email addresses using services. So, the solution here is you've got to think about payloads, right? It's very easy and like the easy button is just to turn to web buruite or whatever already exists in seclists or you know those different buzz databases that are out there but they're usually

not specific enough to be really effective in exploiting and attacking enumeration endpoints. we not we want to put some thought into that actually before we turn to intruder before we use an LLM to actually go and identify those valid users and then we can get to credential stuffing and some of the further attacks that are there um you know one of the things that I love about enumeration is this is typically the gateway to full exploitation of an application um and it's it's required to know like to take over an account and the first thing we have to do to take over an account is actually identify who it is So we can do that to actually win.

So I took that same script um and we modified

it to extract users from that same um application um from that same session to actually see what sort of user details are already included in that. So, in order to do this, instead of just sending the request, now I'm sending the request and the response, and I'm telling it, hey, give me the users that you identified in this. And you'll see, you know, some of them come back with nothing. I logged in with this Chris account, so you're going to see that come back over and over. Um, but there may be like other places that I can start to glean this data. And we don't often think about this when we're giving data back to users. I mean, like this is

all sensitive data because it is being used to log into the application. Um, now demo gods are going to curse me because I am using LLMs on the back end or maybe I've overloaded Bedrock, I guess. Project details. I did want to, you know, take a second and look at, you know, the different ones that are coming back. But you'll notice, okay, on the upload page, not only was Chris Oh jeez, sorry. All right. So, this is what I'm running here. I didn't realize that. Yeah. So, I'm doing analysis on I'm using an extract users. Very similar, but but now I'm asking the LM to give me back a list of the users that it's identifying. And you'll notice it's

even pulling things out of uh like tokens that exist, like cookie values that I didn't see before, but there is a user ID that's associated with that one. um the upload screen. For some reason, there's Crisp, but there's also this admin account that got flagged in some of the response. So, somewhere in that um in that HTML, it's being scraped out of there. Now, this doesn't mean that I couldn't write a email reax or a username reax and to do the same thing. The nice thing about the LLM is I can just talk to it directly and say I don't have to give it like an exact specific pattern. I can just be like, "Hey, I

want to know about users. I want to know about any users that you find in any of these pages. Um, there we go. There's Seth because that's the down that's that profile pick that came back, right? The name of And these are these get interesting because there is a Seth account. The name of the the JPEG itself that's being returned from this is Seth.jpg. That's where it's pulling this information from. Um, I'm going to start to push this into other specific places. Now, you know, some of these may or may not be misnomers. Info.tm TM christm.com that's valid but because it is using you know it is looking at the full request and response I know that

that you know that email address doesn't exist but it's still going to pull it out because it's seeing it as potentially a valid a valid list. Let's see if we get anything out of this. All right. And then you know we get to places like manage groups. Hey, it found a drop-own list there of all of these usernames that had been registered with this application, right? Um, fairly interesting that there is, you know, different values that are associated with it that are in the database that I didn't even think about, right? That manage groups wouldn't be a place that I would necessarily tend to to get this information back out to actually pull pull the data. Manage

projects is the same way. So it's finding all of this and again you know this was this is a process that probably would have taken me an hour or so to comb through all of this data to look for these to do IDE searches to look for things within Burpswuite itself to identify these values but like the LLM's giving me that access very very

quickly. So where does that take it take us? Like if I am a developer, I'm creating an application, I'm trying to prevent this from happening. This is going to be a it's not an easy process, right? Um what what is the most common way that you think we would mitigate or eliminate enumeration? We're going to make this more interactive now. Tired of talking. Any

ideas? Remove all the error messages. Yep. Yep. That's I I mean definitely that's the number one thing that we should actually see, right? Um enumeration is hard. You can be like uh you know Google and Microsoft and just not do it right. Shift the risk to the to your customers. Um because you know we can enumerate all those email addresses. That's great. uh they do give uh very specific options to to help with that. However, one of the things that that you can add to your arsenal there as captures, obviously MFA, um any of the 2FA tokens, like the second that we start to introduce some of that, it'll it reduces the overall risk of

enumeration, especially during that authentication flow because we are going out of band. We're using a separate device. We're we're using SMS. We're using some sort of email that we're sending to clients to actually validate that person as opposed to just the email and password which ends up being in the breach databases that are everywhere. Um it does really come back to the code itself, right? Like how we handle the error messages, how we actually remove them, how we you know we're using generic error messages across the board. Like there's no reason for us to be that specific. most most users of an application, you know, I I don't know about you, but when I put in the wrong password two or three

times, what's the first thing you're going to do is hit forgot password, right? Like, we're not going to mess around with that anymore. But I know we're at a tech conference, so like that's probably not the general sentiment. A lot of people's will just like try their iteration of okay I did spring 2002 now I do 2003 2004 2005 until I hit the right one right spring when I last reset um secure timing comparisons. This is where it gets really difficult um because we just have to account for okay yeah we're hashing a password we're doing a lookup but we also have to account for hey when that you know the positive negative responses that are in

there we're hashing things when we don't necessarily care about it but we're starting to do secure timing comparisons of the different values as they're coming out. There's not really a good way to catch that outside of testing it and looking at the code and thinking um and that goes back to this, right? Like the compare sync that's here. Um and whether or not that actually exists, we actually, you know, we should just be doing a beat no matter what if we want to get around this one or asleep, which is crazy, but putting asleep for two or 300 milliseconds throws that out of the window and it means that the an attacker could not necessarily

Um, and I know that it's I like we know that it's difficult, right? Uh, but this is still a focus area. I would like this to get better. Um, we're going to be, like I said, I'll be posting the, um, the actual code that we're using for that LLM analysis. If you are testing an application, you're interested in enumeration, um, feel free to use it. Feel free to jump in to Wasach Appsac or other places. Um, and, you know, start the conversation because it is a hard problem. I'd like to see it like the number of them actually reduce in my reports. I'm getting tired of talking about it. So, instead I come and talk to

more people about it, right? Um, but if you do have any other questions, I'm Seth Law on social media. Um, I host that I'm one of the co-hosts of the Absolute Absac podcast. Um, Justin is Phantom Traveler on social media and he'll be here next time, I promise. Or I'll send him instead and he can talk about it cuz he gets all up in arms about this as well. Um, and you know, you can find us at Redpoint Security. We're also here. We're, you know, we're sponsoring as well. Um, and love to continue that conversation. And yeah, but otherwise, that's everything that I've got for today. Are there any questions before we go ahead and

Yeah. No. Um, I mean, and and I mean, this is partially because the questions that I'm asking aren't, "Hey, tell me how to exploit this. It's like the simple, if you look at like the prompts that I'm I'm pulling into it. Pull those back." I've never had problems, especially with Haiku, and maybe it's because there aren't guard rails that that are up there. The way that Bedrock is, you know, situated, it's basically just, hey, it's an LLM that you can send a system prompt to. Um, and maybe I'm I like maybe we're getting around it because the system prompt is, you know, you're a highly analytical agent specializing in both security and functional review, right? Like I'm

telling I'm not telling it, hey, I want to hack this site. I'm saying like I'm doing this from an analysis perspective. Again, like some of those strategies they use from a red team perspective to get around the prompts or get around the the safety guardrails that are built into that. That's probably what I'm running to or I'm not running to.

Yeah. Um you'd be surprised that probably cost me about uh less than a cent especially against the haiku model. If you're doing that like 35 haiku 35 sonnet they're fairly inexpensive. The second that you get to the reasoning models, you know, the price like 10 time 10 10x that price. The new sonnet model, the 371 is a lot more expensive. Again, it's got that reasoning capability. the like the one trick here that like we can talk about as well that you see that reflection that I've got up there um is like that turned out to be the key to actually getting back the data that I wanted to instead of having it going to one of the reasoning models and having

it do its own reasoning just telling the telling the LLM, hey, I want you to reflect on your initial output and rethink what I'm asking. Um ends up being incredibly effective. So if you if you're doing model or prompt engineering for apps tasks, you should be using

reflection. Uh yes, one of the problems that I have is so if you notice the output was very structured, right? One of the problems that I have with the reasoning models is they just ignore my structure requests and it'll come back with like it'll get to the fact that there's a justification in there and it'll decide that in that justification it's going to start telling me about authorization issues or the possibility of SQL injection or other things that I didn't ask about because it's trying to reason out, hey, I'm a, you know, I'm a security agent or whatever it is. And this might just be like me like building this prompt specifically for you know

certain models as they came up and the reasoning models need a new approach to that. Uh but like I've had it it's so we've used them it's just been more difficult to actually structure and again I'll push the code up. You can play with it. You um you can try it out with the different reasoning models, even the ones that you can download. Like I said, DeepSync locally or what 53 or a couple of those that you can run on via Oama. It's fairly easy to do. Um yeah, and you know, let me know what your experience is. Come to Wasach Appsac the local OAS group and we'll continue the conversation. So, all right, there's no more questions. Thank

you so much.