Netsec is Dead(?): Modern Network Fingerprinting for Real-World Defense

Good evening everyone. Welcome to our final session in theater 14 today. I have the pleasure of introducing Vlad. Take it away. Thank you. Hello everyone and welcome. Uh I'm Vlad Lucian. I have been in cyber security for over a decade. Uh now my journey started at Samsung Electronics. Then I moved uh uh to a company called Avast which is a vendor where I helped to found and manage the IoT lab which was the lab focus with focus on the IoT and network security threats and now I work for Elio uh where uh focus is the same as mine which is cyber deception that is you honey pots and honey nets as well as mass exploitation and network

reconnaissance which is usually uh can be a first step in the attack kill chain. With that said, you probably uh can see that I like network security and I like networking and networking is simple for some people. For others it is absolute nightmare. What I meant to say is that networking is simple. In 1969, somebody came up with the very first RFC or request for comments and everything was peachy since then. Except networking is not so simple. In uh 1981 uh the very first RFC describing TCP standard was published and uh that means that the TCP was around for longer than many people in the audience have been alive. And with all of those standards

and extensions, requests for comments and amendments, networking is simple where it works and not quite so simple where it doesn't. And with that said, wouldn't it be nice if there was a way where you can actually create a characteristics of connections and clients and servers without going to the wire shark or dumping a pickup every time? Which brings me to fingerprinting which is just that it is a characteristics either of some specific connection or server or a client or just the pipe between point A and point B. With that said a lot of people like to talk about taxonomy. So let's go over taxonomy of network fingerprints. uh one way we can put fingerprinting in

two different buckets is to describe them as active fingerprinting and passive fingerprinting. Active fingerprinting is where you create some sort of new connection. You introduce new traffic into the system and you create fingerprints based on some replies. One of the most popular ones would be end mapap and its OS discovery and passive fingerprints as opposed to active. They don't require any new connections or any new traffic. You can just sit on the data pipe and create fingerprints for the connection based on the existing traffic. In this presentation, we will be focusing on the passive fingerprinting because it is not noisy and you can go back home into your company into your enterprise network and start fingerprinting all the things

without uh introducing any nodes or uh any noise or changing your network stack. Another way to describe fingerprinting is to classify them as the fingerprinting that happens at the start of the connection or which or fingerprinting that happens continuously. At the start of the connection, you usually take some uh handshake for example TCPA handshake or TLS client hello and you create fingerprints based on just the data. Continuous fingerprinting on the other hand happens continuously during the lifetime of the connection and it can change over time. Uh it is basically some flow-based analysis. In this presentation we will focus on the start of the connection uh because usually it is quite deterministic based on the data

and doesn't change over time. And finally you can put different fingerprinting algorithms into buckets based on the OC layer on which they exist. For TCP IP fingerprinting, we put them in one bucket. And for L7 fingerprints like hash or J3 or J4, it is different bucket because it works either on L6 or L7. We will cover both uh of those buckets. Now when we speaking on the thing uh about fingerprinting, there are quite a lot of different algorithms. All right. uh from very first OG like Quesso and PF to the modern Neon FP4CP and also from J3 which is OG of the Thank you uh OG of J of the TLS fingerprinting all the way to the more

modern one like J4 and J4 plus. You can see there are actually J4 and J4 plus and that is because J4 is an evolution of TLS based fingerprinting uh uh J3 and J4 plus is its own suit of different fingerprinting algorithms like uh fingerprinting of HTTP connection or uh connections or SSH connections. uh J4 is released under the same PSD3 close license just like J3 and J4 plus is released as proprietary Fox IO license where source is available. So new mileage when using those fingerprints may vary. In this presentation, we will cover PF and MONFP for TCP IP fingerprinting and J3, J3N, and J4 for TLS handshake fingerprinting. When we talk about all of those fingerprints, it's important to

have a quick refresher of the OC uh seven layer model. Anybody sees a problem with this picture? Okay, I see a few hands. Great. So, uh, one of the layers is SUS and specifically in this case, it is the layer that somebody add called zero trust. Uh the reason I'm showing this slide is that a lot of people here are at the beginning of their network security journey and are trying to learn and if you go to your favorite search engine it is possible that you will find or see seven layer model described with more layers nothing against zero trust it is we using it in our company but it is does not belong into the OC seven layer

model so OC model just describes diff or tries to abstract different layers of network communication starting from the very first which is bits on the wire all the way up top to the application layer where application data lives. In this talk we will cover TCP and IP which is layer three and four and also uh we will go way up into the presentation layer uh which is responsible for encryption and also a little bit sprinkles of application layer. Now in many cases uh people refer to L7 as actually uh L5, L6 and L7 which is fine because application layer and everything that happens in application it's very interconnected. So let's move to the TLS handshake. Usually we have a client and

the server and they just want to exchange some data to figure out which way to encrypt underlying traffic. So they negotiate TLS version on ciphers and so on and TLS handshake fingerprinting is basically based on the client hello for the client and server hello for the server. With that meet J3 the OG of TLS fingerprinting. uh as you can see on the slide it is a long string which is the IRC that everybody uses and this string is just a hash MD5 hash of underlying J3 string or J3 row hash right and as you can see it has five parts in total so let's go over each part the very first one is the the XML

value of the TLS version. So 771 is TLS 1.2, 772 is TLS 1.3 and so on. Then we have cipher suits. It is just a list of cipher suits that client is able to use. And following that we have the list of extensions. Again it is just the extensions of uh that client is able to use in the communication uh in the TLS level. Following that we have elliptic curves also a list and finally elliptic curves point uh formats. With that said, here are three J3 fingerprints of my Chrome browser. I g them just by going to the web page that gives me my J3 fingerprints and refresh the page a few times. And you

can see that I got three different hashes, three different J3 values, which is quite a problem for fingerprinting TLS traffic where every time you connect you get very different hash or fingerprint volume. So why is that? Let's take a closer look at the J3 strings of all of those three connections. And you can see that the first value, the TLS uh version is the same. The cipher suits are also the same. They have the same values and they ordered exactly the same. Going back to the uh fingerprint row value, elliptic curve point formats are also the same and so are elliptic curves. So let's forget about them for a second and let's also get rid of the row

hashes. Obviously if you put different strings in MD5, you will get different values. What is happening here? As you can see, the TLS extensions are exactly the same. They have the same values, but they ordered differently. And that is because Google Chrome a few years ago started created a new feature called TLS extension shuffle. They did this to circumvent fingerprinting of Chrome browser which is really helpful. The thing is if Google knows this uh attackers also know this and they can implement their own randomization of connections which make this hash quite hard to use. Another small issue that I personally find with J3 is that it has no human readable part in the hash itself. It is just an MD5 hash of J3

string. So the first lesson learned if you're doing network fingerprinting and the algorithm that you use for fingerprinting has some row value always store this because if you build your own IOC database with your sweat and blood and tears and then a small new feature and small new shuffle will change all the values of your fingerprint without those with those row strings. you can actually just go back in time and start shuffling them as well or ordering them. Uh but without that you are kind of out of luck. With that said, there is a small refresher on the J3 uh of J3 fingerprinting called J3N. And the only difference is that they actually started to shuffle sort the TLS extensions to

also fig help with fingerprinting of Chrome browsers and all browsers that started to shuffle the TLS extension part. With that said, meet the new kit on the block called J4, which is the evolution of uh TLS fingerprinting created by John Al House in 2023. And this fingerprint actually has three parts. The first one is human readable and then there are two hashes. So let's quickly go over the fingerprint. The very first value is just one character and it represents the way uh this TLS handshake happened. T is for TLS, Q is for quick and D is for DTLS. The following two characters are here is human readable uh TLS version. So 13 is for TLS 1.3, 12 for uh 1.2 and

so on. Then we have an SNI type. D stands for to domain and I stands to IB. So basically based on this character you can quickly see hey this connection here actually happened uh to the domain and if you have your web server you can actually quickly distinguish hey all those connections are actually happening to the IP address as opposed to domains maybe they are not really uh your proper clients. Following that we have a cipher count which is just the count of ciphers that client supports or advertises. And then we have extension count. Finally we have first application layer protocol negotiation volume which is quite helpful because here you can see hey this actually this client uh

wants to communicate over HTTPS version two. Oh sorry HTTP version two. Then we have truncated SHA 256 hash of cipher suits this time sorted. And finally the last part it's truncated SHA 256 hash of extensions which are sorted and also signature algorithms in order in which they appear. Just like J3, J4 has has J4 row which is the row value which is used to create those SHA 256 parts. Right? So in here it is actually consist of four parts. You have human readable part. Then you have uh the part with all the cipher seals, the part of signature algorith uh hash uh the part of the extensions right there which are sorted and finally you

have all the signature algorithms. So it is quite useful if you want to go back and see what is the most popular extension algorithm or cipher that is supported by all the clients in your network. With J4, you can do a lot of cool things. For example, if you use Archime, which is an open- source DPI utility that you can just deploy across all your network, you start collecting J4 hashes and then see, hey, there there are a lot of clusters of multiple IP addresses that use some specific hash. In example here, it is actually NRO uh clients who are trying to enumerate your network parameter. And also there are multiple detections that you can do uh in the J4 space. For

example, the first value here is J4 hash. The second value is J3 hash for the exact same thing. Depending on the sample or the blog post or the source of your data, it is either uh cobalt strike C2 communication coming out of your network or trickbot or windows https metrop. So if you see this hash in your network, uh something must go wrong. Going a little bit deeper into the rabbit hole of fingerprinting. Let's go to the TCP IP fingerprinting. Just a quick refresher here is TCP header and TCP data. TCP header is between 20 and 60 bytes depending on the number of TCP options. And oh boy, do I like options. This is just a quick refresher on the

TCP three-way handshake. You have a ser client that tries to connect to the server. You have seen server replies with syn and finally you have the final act and usually TCP fingerprinting happens at the very first part of this connection because it has the most interesting data and most characteristic about the client. So meet the OG of TCP fingerprinting uh POF where uh it's so old school that all is spelled as zero. It is created was created by Mahal Zaliffski all the way back in the year 2000. And on the slide you can see the typical output of PF2 tool. And below that you can see the raw signature we which is uh PF signature right it has

quite a lot of parts but they are quite simple. The first one is the value uh of the IP ST. So four is for IPv4 six for IPv6. uh then there is a TTL value uh which is basically uh number of hops left. So the TTL of the packet that arrived and also some guest value based on the typical stack. So if you see 120 obviously you're going to guess hey the TTL is probably 128 and there are eight hops between you and the client. to moving to another value from the IP part of the TCP IP. You have IP option length because remember IP bucket also have IP options. During the TCP handshake, you should probably see uh

zero. Then we have MSS or maximum segment size and window scale and uh window size and window scale. uh in newer version of PF this value of window size can be represented as a multiplier of maximum segment size. Finally we have all the TCP options in order in which they appeared in the packet and uh IPs. So in this case DF for fragment and ID is present. And finally, we have payload size class in the IP uh in bucket. During the sin uh TCP 3-way handshake, you probably should see uh payload size of zero because there are still no data. Uh and uh zero is for no data and plus is for any amount of data. So if you see

uh TCP and shake with plus there, something is going wrong. And on this slide you can see typical detections list of detections of POF. And maybe you now start to see the problem why PF is not so widely adopted. Majority of those detections because POF stands for passive OS fingerprinting. So majority of those detections were created trying to determine which operation system is running on the other end on the client. And with all those new interesting algorithms with HTTP and all the cookies and all the headers, it become not as used anymore and was not very used for cyber security. Unfortunately, one of the biggest users of PF uh is Cloudflare. They released the POF BPF,

not eBPF, BPF compiler all the way back in 2016. and they used this compiler to create the rules to basically drop the traffic from the weird devices with weird PF values and also as a part of their antids protection. So meet the new kit on the block, Meon FP created by Ken Webster in 2024. And from the name Mon, uh it is quite fast and also very simple and I do believe we can do with more simple things in cyber security. It has just four values. Everything is human readable. The first one is window size. Then we have TCP options as kind numbers. Then we have um maximum segment size and finally scale window scale. Going back to those uh TCP

options in order in which they appear. In this case we have maximum segment size set sucks permitted we have time stamp and no op nope. So it is used usually for padding and the TCP header. And finally, we have window scale. Here's another another example of very common mon fingerprint with all the options. Any guesses what this can be? Which network stack produced this fingerprint? Anybody? So based on the uh window size being a multiple of maximum segment size and the fact that we don't have an eight in the TCP options which usually is used uh by Linux operating systems because they use time stamp option in the TCP stack. It is Windows and in this case it

is Windows 10. Now this is the same fingerprint right except the window size is 65 535 and this time we actually have uh TCP option 8 which is the time stamp so in this case it is an Ubuntu I know how it paints arch users here not to say anything don't worry you are able to fingerprint your network stack after the present mentation and here is almost identical fingerprint and the only change is maximum segment size and that is because it is the same machine connecting to the server through mobile uh network of Verizon. You see Verizon actually needs about 72 bytes of overhead for their communication. So you naturally your maximum segment size gets

lower. What happens when you try to use our Windows machine and connect through the different proxy servers? Obviously you have a smaller window size and you also have a smaller window scale. All right, here is a different proxy and you can see that window size right now is even lower but also the scale size is zero. So this is a trap. By the way, if you look into the TCP options, you will see that there is an eight there which suggests that it is actually a Linux machine connecting to the server. And the thing with proxies, especially public proxies that you can find on the internet, is that they work a little bit different because when we fingerprint

stuff, we usually fingerprint at the site of the server to fingerprint the client. But when you use the proxy, you actually fingerprinting the proxy itself and not the client. So just remember where you fingerprint and what you fingerprint. how VPN tunnels can affect the connection because usually let's say you have a client you have the server on both sides you have a network with MTU maximum transmission unit send set to 1500 which is the fact of the standard and you have nothing weird going on between point A or point B. So when you use VPNs, you just encapsulate the underlying TCP stream into something else which creates overhead. So going back to our favorite Windows 10 machine,

this is the fingerprint and this is the fingerprint of the same machine that actually connects over OpenVPN. So you can see that in this case the overhead is very not nice 61 bytes. Then this is the fingerprint of the same machine connecting over the wire guard which creates a overhead of about 80 bytes. And finally you have the fingerprint of the same machine connecting using I v2 which uses 100 bytes of uh overhead. Now remember different implementations different libraries and different network setups can create different overhead. I think a lot of different mobile networks will have some sort of semi-unique overhead and semi unique semi-unique MSS. Uh and if you actually look into the uh windows

size part of the fingerprint the very first one you can see the Windows network stack at work because Windows really tries to set the window size as a multiplier of maximum segment size. So you can see it here that it is just a multiple of this value and finally here it really tries to make sure that it is some nice multiplier of uh the MSS and the reason for this is that in the RFC and in the specs it is strongly suggested to use multiples of MSS. Now here are three very common fingerprints. You have Windows, you have a buntu, and you have an iPhone. Let's put them aside for a second. And do you see anything weird

with these two

fingerprints? So, if you have a public IP address, I can guarantee you that you receive connections with those two fingerprints every hour of every day. So let me help you. The bottom fingerprint you will see connections from academy for internet research and also French search and top fingerprints you will find connections from shadow server rapid 7 sonar and binary edge. Anything any guesses of what this may be? The first one uh um you will also find those connections. Don't worry, you always will find those connections from thread actors. The first fingerprint is uh c characteristically a zmap and the bottom one is mass scan. You see scanning the internet is hard. Scanning internet on the scale is very

hard. And those two tools actually writed their own routines to generate sin packets very very fast. And because of that they don't have many TCP options because there are no expectations of actually using the connection. You use sin scanning just to find and knock on all the ports and find which ones are open. Which brings me to the network reconnaissance killchain chain. We have first stage is usually port scanning because you need to find hey what is actually open and then you have service enumeration and discovery. You want to figure out hey what is running on those open ports and you can do it in many different way. You can use the same server same IP address. So you have one

stage and this stage basically does both port scanning and also service enumeration. Uh which will result in having two different sets of fingerprints. You will have one set of fingerprints used for fast scanning and another set of fingerprints used for service discovery depending on the network discovery engine that you're using. This is by the way fingerprint set of fingerprints used by shadow server. Then you can be a little bit different. You can split this routine into the multiple stages. The first one being the port scanning. And then you offload the data from the first stage. Take all the open ports, all the IP addresses and move it to a different server which will be responsible for service discovery. In

this case, you will have two different IP addresses with two different sets of fingerprints. Uh this is by the way showdown. And as you can see they actually tried to use different sets of fingerprints. They created their network stack for fast scanning to use uh TCP option of uh maximum segment size and they also randomize the window size uh which is quite interesting because it really helps to identify shoddon scanning servers because they usually produce about 38 to 60,000 of unique fingerprints. So if you have some IP address and it is not possible to figure out what this is based on the reverse DNS or static IP lists or something else, you can try fingerprints. But what about the

attacker? What happens after reconnaissance? Usually uh bad guys like to do something we call initial access, right? And they move to the exploitation. The thing is advanced actors can afford infrastructure or can afford to rent a botn net to do the scanning of the internet to find targets but a lot of them prefer not to do that because it's hard it's expensive you have to deal with a lot of things so they use something else. Who here uses census or shutdown? Okay, great. Who here blocks census or shutdown? Okay, that's great. So, what happens when you don't block senses and shutdown or other scanners is that you get yourself a free attack surface management with one catch

is that the public the data in this attack surface management accessible to everyone. So all the bad guys need are actually go to public uh scanner and get the data for targets to exploit. Which brings me to actually combining the IP blocking and fingerprint uh based blocking. Uh who here uses IP blocking? Just out of curiosity. Okay. The same people who here uses uh fingerprinting blocking or blocking based on the network fingerprints. Okay. Amazing. Please find me after the talk. So um here you can actually use uh combined approach of blocking known stuff or very very bad malicious stuff based on the IP addresses. Uh for example uh census is quite nice. It will just give you all

the IP ranges that you can block. Uh third actors uh not that nice. you have to figure this for yourself and then use fingerprint based blocking for blocking fast scanning and fast scanning activities uh just based on the fingerprints. So you don't block the IP address, you block just based on the network fingerprint and that way you are not affecting any other traffic that can be benign coming from the same IP address. uh you can uh actually look for combining IP blocking and fingerprinting and you will find a few blog posts about this. Now you like fingerprints or you're excited about fingerprints. How do you gather them? Well, with MONF FP with TCP fingerprints, you can use Rustbased

standalone tool and just deploy it to all of your machines. And right now we are working on publishing the Archime plugin. So you can use it in Archime and we starting to work on the Wireshark plugin. So you can use it during the trainings or workshops etc. So coming soon trademarked with J4 and J4 plus you can use again plugins to Army and Entop and Zeke. If you like DPI uh you can use vendor specific data for example AWS cloud floor uh front and finally you can you can get the fingerprints from the reverse proxies like cloudflare and fastly unfortunately they do not support mpb yet but if there are people in the room who are huge enterprise customers

of either cloudflare or fastly wink wink you know what to request Next. So how can we or where can we actually gather fingerprints? This is very simple example. You have a client, you have a server and you want to fingerprint the client. So you fingerprint at the server side. It's very simple. With reverse proxies, it's a little bit more complicated because you cannot actually fingerprint at the server side. Otherwise, you'd be fingerprinting your own reverse proxy, which is just not as fun. So, what you can do is start fingerprinting at the reverse proxy part and forward the fingerprints in the headers downstream where you can capture them. Now there are many ways to skin the cat and now there is on-prem

infrastructure there is cloud infrastructure there is multi cloud infrastructure uh there is hybrid and if you're a startup your infrastructure is a laptop and your brain uh fingerprinting the brain waves is out of scope of this talk. So depending on your need, you can start fingerprinting at either the router or if for example you have a huge set of unsemble scripts to manage whole infrastructure, it makes sense to just deploy them on the server and forward it into your seam. So how can you capture some true value from network fingerprinting? Well, the easy path is usually to write more detections. This is just an overly simplified example uh of detecting fast scanners. This is uh sigma rule that you can push and

start creating new detections. In the real world, you probably would have a few stages like enrichment and trying for things to make sure that you have proper data and you don't have too many detection because trust me, if you actually publish this to production, fast scanning happens every second or every few seconds. So, uh you will be perfect uh vendor that generated 1 million of alerts that are not that useful useful. What is more interesting uh in my opinion is to do something with this detection and this is an ELK example where you can actually write the rule to see if something is scanning your infrastructure and push it into your routers into your block list and start

blocking this uh actor. So this is the example of zero first patients uh first patient uh detection where basically if somebody is scanning part of your infrastructure you detect it and you start blocking on the IP layer uh across the whole infrastructure. So once this actor moves to other IP ranges or other parts of infrastructure uh they already blocked. And finally it is quite fun to use in thread hunting. Uh if your platform supports wild cards you can use wild cards in fingerprints to start searching for interesting things. And in this example well uh there is absolutely nothing interesting happening. Nothing nothing to see here. Finally, I would like to actually thank Ken Webster, creator of Neon FP and

acknowledge his work as well as Luca Derry who is the author of open source and top DPI. John Alhouse, Jeff Atkinson and Josh Atkins, creators of J3 and now John House is working on J4 and J4 plus. Mihal Zalefski, creator of PF, uh NDIC, who is the main person behind Archime, which is amazing, as well as all contributors and the open source community as a whole, which is extremely helpful uh to actually achieve something in modern text. So one thing I would like you to live with is that fingerprints are just another tool in your arsenal and it is up to you to extract the most volume. You can start detecting things, you can write detections, you can use it for

IOC's, you can use it for blocking. It is up to you because remember ultimately the security of your network parameter and your company is in your hands. So that's pretty much it and I think we have uh time for questions. Thanks everyone. Um I don't have any questions on uh Slido. Does anyone have any questions from the audience? I can have you step down. Okay. Otherwise Oh yes. One moment.

Uh so great presentation. So I have my question is can you use network fingerprints to allow list internal applications? Uh the last part sorry can you use network fingerprints to allow list instead of block the internal applications that you use? You can absolutely do that. Uh so you the thing with network fingerprints is usually you will have to write some kind of ebpf filter because ebpf is amazing and you are the owner of the network stack in this case. uh I I have mentioned a few examples where you can just block fast scanning, right? You can absolutely actually use it uh as a part of XDP or express data path in Linux kernel and you can start blocking everything that

does not pass your filter. So you can absolutely set hey I have three fingerprints which is the Windows and Linux. Everything else gets dropped. If you're more into chaos and cyber madness, uh you can actually start to reply with sin a to every scene that you receive on which you are not listening. Right? So you have one port 80. You can start replying with sin al to every request that uh that is using that is basically has a fast scanning fingerprint and you can drop the scenes that are coming for scanning your actual infrastructure. So you can absolutely do that. Yes. Any more questions?

Hi, thank you. Hi, thank you. That's a great presentations. So my question is like since you walk through the OSI model, have you seen value in combining passive TCP IP fingerprints layer 3 and four with TLS fingerprints layer six and 7 to improve detection accuracy especially for stealthy scanners or C2 traffic? Uh yes uh I was afraid that this presentation was too long. So you can absolutely combine TCP fingerprints with TLS fingerprints. For example, you can use wild cards in both JF4 and Neon FP and you can say, "Hey, if I see something from uh TCP stack that is kind of weird, which is non-standard fingerprints, very low MSS, no window scale size or very small window size and

you see something uh like hey uh in J4 there some something like with cipher uh count where you have the value do that is just counts the number of ciphers supported by the client. If it's very high, you can start dropping those connections. So you can absolutely do that. For example, uh in in JF4, you you have just up to 99 uh cipher uh supported cipher algos, right? But what we have seen in the wild is that some attackers try to be funny and they support or advertise that they support like 150. So in J4 it will be like just 99 because it's kept at 99 but uh it means that is something weird because

normal clients and normal browsers they support like 20 25 60 but once you go very high you can start say okay this is weird and then also you see that hey this is TCP fingerprint of is either very weird or is an IoT device and you can start dropping those connections. So we can absolutely combine that. Now with that said I think we have just uh two minutes. We also doing a network recon uh workshop which is uh in person on Thursday May 1st uh where it is basically a two-way conversation where we can have uh deeper talks about specific parts of network fingerprinting. So feel free to check it out. And then there are two uh codes for

Q&A and feedback. Hey, thanks everyone and that wraps it up. Um, there is a happy hour I believe starting. Um, so if you want to make your way out and they are showing movies tonight, so we do have to clear out the theater. So I'm going to start hurting you all out. Thank you so much. Thank you very much. And thank you. And if you want cool stickers, just find me.