Unwitting Hosts: How Residential Proxies Increase Risk

[music]

Thanks for having us. [applause] I'm Darren. This is Blake. Um, continuing the theme of uh mystery. I work for Telus. He works for not Telus. Um, no more specifics will be provided now. Uh, yeah. and we're talking about residential proxies. Um, many of you have probably heard of before. They're not new. Um, this is not really a novel threat we uh discovered, but they have been um at least in our data kind of surging in popularity or at least commonality. Um, so we figured it was it was worth uh you know describing them and and and presenting um some detection work that Blake did that's uh actually open source now. Yeah. What what are residential

proxies? Well, they um enable network traffic to appear to come from a different device um or perhaps different network, right? Um so that's what a proxy does. The residential part um refers to the fact that they frequently run in um you know home networks, right? You know people's houses, their local small or home office um network and they're often installed legitimately by users. uh they have a variety of um you know marketing and sort of promotional campaigns to do this. Um in some cases they may actually pay users uh to install them. So sort of a uh you know revenue sharing model type thing. Um and they're often resold. So a lot of existing uh previous talks and articles

about more the you know uh economic or b business side of residential proxies. So feel free to um go seek that out. That's not what we're going to go into here. But I just wanted to um plant that seed if anyone's interested in this topic. Um, and then why are they a problem is the other question. For for one thing, uh, threat actors like to use them to anonymize their traffic. And if these devices that are running them are corporate devices, you know, maybe with a uh VPN or some other um way of of, you know, being part of a enterprise network that can make it look like uh an attack is coming from your network,

which generally isn't desirable, I would if you uh maintain the network. Um so that's one issue. Uh and then they can also result in reputational damage. So again um if you run like a service provider for example um again having attacks come from your service provider not a great look. Uh but beyond that they can be more concretely dangerous because um residential proxy functionality is often part of applications that perform a lot of other malicious things, right? Um, so one of the ones we look at, we looked at, for example, has the ability to pretty much change any settings in Chrome. Um, so that's bad, right? Uh, and there's a lot of other similar um types of malicious technique

that may also be bundled in by applications that purport to be just residential proxies. All right. And Blake is going to take it from here. Yeah. Thank you, Darren. Um I am going to talk about some of like the technical specifics of um residential proxies how they behave and like how we are actually detecting some of them in in uh real network data. So you know this is kind of the cartoon picture. We have four main entities. we have, you know, going from left to right, we have a a proxy user, and this [clears throat] is somebody that could potentially just download some sketchy VPN on Android and are using it. Um, or an actual threat

actor that wants access to to a specific network. Um, we have a proxy broker, and this is kind of a service that's I, you know, I think its main purpose is to connect proxy users with actual like proxy victims. Um, the proxy victim is some interesting, you know, computer on either an enterprise network or your home network. Uh, you know, like Darren was saying, like these can sometimes be bundled with potentially unwanted applications. Um, but I think the the thing that we've seen most notably is that the the users are typically unaware that they have this residential proxy software running on their laptops. Um and then finally we have the the destination server and this is what the

the proxy user actually wants to connect to. Um you know the the area in yellow that's the the enterprise network. That's what we have visibility into. So we can't really see the proxy user talking to the proxy broker. But we do assume that we have some kind of monitoring um infrastructure in place on the network that we control. So we can see the proxy broker to the proxy victim to the actual destination um going like top to bottom. So this typically starts off with the proxy victim initiating a TCP handshake. Um then some authentication material is shared between the victim and the broker. Um and at that point the kind of the circuit is set and the proxy user

can start sending traffic through the broker through the victim to the final destination. Um our focus is going to be primarily on TLS. So you know we can see the TLS handshake the TLS client hello make its way all the way through the diagram and then the server responding. And so we see the TLS server hello um on the way back. This isn't super important. I just thought it was like kind of interesting. Like in this in the session setup for a lot of these residential proxies, they do authentication through plain text um JSON web tokens. So you can see the actual like user identification data, you know, the issuance, the expiration. um probably vulnerable to some kind of

replay attacks if you can um actually observe these. So um we have an open- source project called Mercury. It's it's on GitHub. You can you can download it, explore it. Um its primary purpose is to process pcaps and and monitor live network traffic. the um you know Mercury is built on high performance uh native Linux networking. So AF packet v3 uh we regularly monitor you know 40 Gbit per second networks without dropping any packets if you give it enough hardware and RAM um you know we we've pushed it to to monitoring like 100 Gbits per second without any any packet drops. So it's it's pretty performant. Um, and like the main goal is just to observe all of the

interesting packets on on your network, pick the ones that are interesting, and uh write out the the interesting elements in in JSON. So, if you have a TLS session, we're going to report a ton of information about the client, hello, the server, hello, the certificate. Um, but then all of the encrypted application data records we we ignore um because they, you know, generally have less less information. There's an extensive schema. So I I think our schema has between like500 to,600 elements like a bunch of different network protocols and interesting features of the network protocols. It would be great if you're interested in this kind of thing if you could like look through the schema. If there's like

a protocol you're interested in and it's not there, that's kind of a best case scenario for us. like send us a message and we'll, you know, it will either be in our backlog or we will want to prioritize it because we're we're always looking for new things to to extract. So from the point of view of um Mercury and residential proxies, if if you're monitoring your network and there is residential proxy behavior, you're basically going to get two records that have the the structure of of what's on the screen. And so these are both representations of the TLS client. Hello. Uh Mercury has a fingerprinting system that extracts a lot of the interesting cryptographic parameters

that the client supports and makes a fingerprint. Um but then there's, you know, the the normal network five tupal stuff and then some interesting metadata about the the TLS um actual yeah client hello. The the thing that's most important here is the random knots. So the the random knots with in the client hello is basically there so the the client and the server can negotiate um ephemeral keys and provide protection uh against replay attacks. Um but with a residential proxy like the really interesting thing is like you're not supposed to see repeated random nonses on your network. So that typically says that there's like a severe entropy failure somewhere. Um, which is obviously bad, but it just

doesn't happen that much. So when you do see repeated nonses, that's a really good indicator that something funny is going on. And in the case of residential proxies when we see uh repeated nonses and then we see this pattern of you know an external host talking to an internal host and then that same internal host talking to another external host. Um that's kind of like a tailtale sign of residential proxies and the repeated knots is kind of give us the information to to make that like high fidelity detection. Um in the in the way that we use mercury it's typically in like a big data setup. So we're bringing a lot of the mercury telemetry back to a Spark cluster. And

in that case, you know, there's a relatively easy way to to do the detection. You just group all of your network five tupils by the the random nonses, aggregate the the flow keys, and then do that filtering logic based on the pattern of external to internal to external. Um, and that's what we initially did. Um and then very recently like about a month ago um we released some code within Mercury to actually perform this detection online. So if you if you're running Mercury you can either run it on a pcap or you know network interface. If you do d- network behavioral detections it will turn on this bit to start looking for residential proxies. And if it finds a

residential proxy, it will populate um a new analysis object and you'll say, "Oh, it's a residential proxy." And yeah, we we assign probability scores for a lot of detections for different reasons, but for this one, it's always one. Oh, yeah, I think that's it for me. All right, thanks Blake. I'm back. Um so so that was the uh the main like super technical part of the talk. And then what I did was um I got curious about how prevalent this type of behavior was in the data sets that we have access to. Um so one of them that I I looked into is um telemetry from a agent that we make called uh network visibility

module. Um to be clear Blake and I do not make this but Cisco broadly. Uh and uh yeah it's um pretty useful telemetry. It's um like I said, an agent that runs on endpoints uh Windows, Mac OS, and Linux. Um so for that reason, you might think of it kind of like a endpoint detection and response type thing. Um but it's not. It's uh more network focused. So it actually sends us um telemetry that looks kind of a lot like IP fixed net flow uh but with process information added in um for the processes that made those network connections and then their process hierarchy all the way back up the tree. Um so anyways that's NVM. We

have quite a lot of it. Um, for the month of data that I looked at, there were about 180 billion uh telemetry records. So, decent sample size. Um, and yeah, uh, we focused in on two um, known residential proxy services um, Infhatica and Rave. Uh, Rave is a helper application that claims to facilitating like watching streaming videos together like uh Netflix parties or whatever. Um, but what it actually does is um residential proxy services. So, you know, bit sneaky. Um, and then Infhatica uh officially is a residential proxy. So more people um I would say are aware of that one. Um but it's still pretty prevalent. And then um yeah, basically the the stats we found were that from those uh

180 billion events that we looked at um there were 5.4 million for Infhatica and RA. So pretty large. Um, now interestingly this was from about um 3,000 different organizations that we monitor and it was from just five of these. So what that tells me is it's not very prevalent but where it does exist it's um extremely uh prolific I would say. Um there's a a few um aspects of the um the data that I put in here if you want to threat hunt or write a detection rule for these services. Um that's not a Mercury rule if you don't have that uh available for you for whatever reason. But um in the interest of time, I'll actually skip

through those and get to the um final slide uh which I thought was interesting as well, which is that we uh we went through and analyzed all the the remote host names uh that these two applications talk to um in that data set we analyzed. And um we we boiled it down to um a little over 6,000 uh distinct host names. And then we compared that um well we ran like thread to intelligence look lookups for this using virus total. Um and we compared that to a a sample data set of 6,000 distinct host names from just arbitrary NVM traffic. um and they were about five times more likely to be malicious according to virus total. So I think that's another

um pretty stark example of the risk of these residential proxy services. So, you know, if any of you are out there uh trying to perhaps convince it to uh not let these services run in your network, that might be a good stat to uh bring up, you know. So, yeah, that's pretty much our talk. Um I think we might have a little bit of time for questions. I'm curious if you guys know of any open source uh lists where that identify known proxies and known residential proxies or if uh you have any data yourselves that you're willing to share. >> Talk to me after the talk. There there is some data that I can absolutely

share. Um we we get a lot of the um this picture like a lot of the proxy broker IP addresses. I have several thousand of those that I see and and they they do change, but they they don't change as frequently as as I would have thought. Um, but I I I personally maintain a list. I'm sure there's some open source list and I feel a little embarrassed for not knowing it, but I I could get you some of the lists that that I maintain. I was wondering if you had advice for people that run destination servers that are getting hit by this sort of traffic. like wasn't really covered in the presentation, but if you have any like

flagging identification device when you aren't actually sitting in the network >> like if if you're being attacked through a residential proxy, >> I'm not sure there would be really specific techniques that I'm aware of at least. Um I don't know I I I don't think so. I mean, I I think from your point of view, you could see that it's coming from a corporate network or residential network, but is is that because like there's malware running on that residential network or there's a residential proxy that's using that as a a foothold? Like being able to differentiate those two things? There's not a lot of clear patterns in the network traffic. Um, so like probably your best bet is just to complain to

whatever company that's sending you the >> Yeah. >> the attack. >> Yeah. I mean basically from my perspective if you were looking at it from the like end victim side I'm not sure it would be much different to you than like being attacked by an AWS IP right you know or some other provider that rents access. So certain certain providers definitely have more uh malicious traffic than others I would say but yeah I I think again it's it's like a hosting provider from that perspective right >> I run a destination surfer that's also been attacked by these kind of things and um it started hitting when I put uh like a git uh git forge software on

there so that seems to be correlated to like AI scraper bots that are trying to train their um their gather training data, you know, without any kind of uh >> rate limiting on it. So, I'm wondering if you uh if you know like of the unscrupulous users of these uh how common that is that they would be like AI companies trying to to gather training data. >> That's a that's a really great question. So, I I did see some like report that AI companies were doing this in the destinations that I see. I I haven't seen them, you know, I've actually seen a fair amount of traffic going to sites like reddit.com, which I'm I'm guessing

that would be related to to that piece, but that's still like a minority compared to some of the financial like footlocker.com is like one of the biggest destinations that I see because I I don't know. I guess there's still like people that resell like cutting edge sneakers or something. Um, yeah, I see a little bit of it, but not as much as I would have thought had it been like a really prevalent problem. >> Yeah, actually um on that note, and I didn't include it in the talk because I have a hypothesis about why this is, but I'm not um necessarily able to prove it with the data I have. But I uh I see a

lot of traffic to e-commerce websites, so Amazon and the like. Um and separately I read an article saying um that a lot of uh rather unscrupulous e-commerce sites use residential proxies in order to do like price comparison dynamically like automated price comparison. And so that may be why that is showing up in my data. But again, I wasn't able to actually make that correlation myself. So >> as as far as the uh open source lists, I know like firehole proxy is like one of the lists on GitHub. It's not great to be honest, like from what I've experienced like a lot of the hosts on there might be dead, but it's like it is a open one. Uh spur data is like also

pretty good, but that's a paid feed. Um, I'm just curious, uh, have you had any luck trying to entrap some of these clients by something like a Jaw4 fingerprint on the TLS? And >> I like this question. So, um, no. Uh so the you know most most fingerprinting techniques they're they're really good when you um if it was like malware [clears throat] creating these connections or like a specific version well malware that doesn't use the default TLS library on whatever operating system. So that's pretty rare to begin with. Um, but even in that in that niche use case, in the case of residential proxies, like the vast majority of things, even even the ones that are going to like the sketchier

websites, um, they're typically coming from like normal browsers, um, or other type of things. So, when you look up like a a network fingerprint, it's going to it's going to say, oh, this is like a Chromiumbased application. Um, so it doesn't really give you a lot of information uh to tell you about like the client software and it and it's mostly because like the residential proxy, you know, it's just passing through traffic. So it's really like what is the what is the proxy user and like based on the fingerprints that we see it seems like the proxy users are either using things like normal web browsers um or they're they're using things like curl which is

again you know the the fingerprint doesn't give you a ton of information there. So you can differentiate those cases, but it's it's not like you can come to the the conclusion that like, oh, you know, the these sessions are all malware that are coming through the the residential proxy. >> Well, thank you again to Darren and Blake. [applause]

[music]