BG - What Reaction to Packet Loss Reveals About VPN - Sergey Bratus & Anna Shubina

try again hello everybody and you better probably not okay it works it works okay so this is a talk about something I've been curious about for a very long time since 2007 actually about whether interesting things happen if you interfere with an encrypted channel in a really dumb boy in a really dumb way that doesn't assume knowing anything about the channel so I was curious about this until since 2007 and then chill DARPA cyber fast track actually financed this and we did some studies with with the company called milk curd and found out that indeed there are some really interesting effects going on even if we don't understand what is it what exactly is happening this talk is about really

simple ways to interfere with encrypted traffic basically you lose every you just you drop every nth packet and you end up with a signature it's as simple as that and our educated guess is that the reason this is happening is that there are all kinds of interesting behaviors produced by the fact that the packets go through multiple queues and the more complicated the system of queues is the more interesting the signature becomes when you start interfering with the queues we have some suggestions as to what to do to make things look a lot less interesting and a lot less finger printable this is this is how it looks if you take if you're looking at fast traffic so these are two

signatures of two different ciphers Oh using ipsec and SCP file transmissions going through IP shock as you can see the dots here which we will explain in a moment or cluster cluster than completely different places the clusters are of different weight this is the same stuff with the openvpn we drop each fits packet suddenly there are signatures so these are timings of a hundred packet hundred of packets added together as you can see a s looks completely different from Blowfish this isn't visible with the unarmed I this is still working progress and unfortunately we were not able to and to explain a lot of things that they're going on but we think that this is important because

this is something that has not yet been explored and discussed and you should also regard this as an invitation to join us in this exploration so how did they discover this effect in 2007 Dartmouth College hosted the Democratic primary debate so normally this should not affect normal humans so there are crowds of people online there are crowds of people on campus and the network is somewhat more congested than usual but you don't expect much to happen to affect you personally except my ssh stopped working and they didn't start working even after the television crews left the campus so what happened was so what happened was that the network people decided that we were going to be

really helpful to the television crews and to give priority to the streaming video but it turns out that the cisco algorithms for a quality of service are not as good as you would like and that when and that when configured improperly when your ssh gets really low priority on those they start doing really bad things to traffic like as a sage because what they do is delay each end packet for example and tcp doesn't work well with that and ssh works even worse with that you basically cannot use as your ssh anymore it took a while for our sysadmin to solve this problem and by this time I was curious what else can you do if you

just drop or delay each end packet for example can you allow through normal web browsing but totally ruin anything encrypted and make it so painful to use as to being impossible to use and the answer is yes what we've seen in the wild suggests that the answer is yes so let's talk about packet timings by the way I talk more at these conferences but this is really Anya's word from start to finish all I did was write a few shell scripts so I am basically just enjoying the support role here the vocals so there has been a previous work that looked at our round trip times and what they can tell about routers about the

network there is a talk at DEFCON 11 by Tony Katawa called a fashionably late what your RT te tell us about your network and the idea there was that you ping your router as fast as you can and then you look at the round trips and by our filling the queue of the router you can actually tell what how deep that Q is so you see the sort of a sua pattern in the rtt when you fill the queue and then when it clears and then you feel again it'll fill it again I so there is this periodic pattern that tells you what router it is what size of a queue it's in so much as what size excu is

this has been used for packets for network scanning so there was the port bunny in kernel scanner that was much faster than n nap at the time and map since then improved this was done by Ricky Ricky labs by fabiane Yamaguchi and Felix limner and the little port bunny you know think bunny is their artwork so what that did was it looked specifically for that saw pattern when some buffer on some hop on some router good saturated and eased off and so they were able to finish their scan without losing packets to the over filled cubes and they could do in you know 15 seconds what the what the nmap took tens of minutes to do so these are

known effects I woods last-known is how the advanced features of TCP interact with file transfers so this picture is from Stewart Cheshire's a study of how the combination of the delayed ack in the negev algorithm in tcp caused this kind of packet timings so this was done with a TCP probe and a transfer of a large file would just stop for 200 milliseconds doing nothing in the middle of the transfer then ramped up then again then again do nothing and why is that it's a combination of two behaviors first the delayed ack so when you imagine an interactive session session you colonel doesn't actually ack a segment coming in right away because the colonel things well this looks like an

interactive session right so we'll wait for a hundred milliseconds a 200 milliseconds till the application actually gives us some data back because say SSH echo back every character right so we'll just wait and if the application doesn't send us any characters then we'll send the ACT back off to the timeout and if it does then we'll send that data will that act back and so you'll save a packet that way so this is the delayed ack and this happens when only one packet arrives if two packets arrive of the second one will be act immediately and then there is also the Nagle's algorithm you don't want to send short packets uh to the other side if you

already have packets in flight and they have not been acknowledged then and you have a short buffer then you just wait again 100 to 200 milliseconds until either an act arrives or more data is given to you by the application so when that happens you send the larger packet if an egg comes then you send whatever you have so both sides then have weighed behaviors the colonel actually sits and waits hoping to save you a packet or an ACK and this causes the bulk transfers to get completely screwed up like that I mean imagine you're waiting half the time that it takes you to transfer the file and this is just one example of the

interaction between the layers like the colonel and the application layer now if you add VPNs to this right VPNs have their own optimizations and queuing and timeouts so if you control the Gateway say you are in our isp ah then you just basically start eating up packets and you can tell by the timeouts just exactly what kinds of combinations of cues are on both sides so this is important you can profile these things and you can finger print these things so if you are an ISP or if you are an organization that controls its own border router and you have this encrypted stream crossing your border and you don't know anything about this stream you don't even know how it

started you could start interfering with it and by watching its timings learn about what that stream might be and that you know in if you are actually if you actually have a duty to protect your water now of course if you're an evil ISP or an evil gateway you can use the same thing to try to link streams of unknown origin to see if they come from the same kinds of software stacks and you could try to find out things about those strings and one thing that are on a found is that you can actually tell the cypher of the enclosing VPN pipe if what you use inside that VPN pipe is TCP and ssh that is to say protocols with a

rich behavior and rich interplay between the layers so and that you can do only by measuring packet timing because crypto libraries have their buffers so the experiment is actually really simple you transfer a long data file and you drop each fifth data packet just like that and then you time for example a hundred data packets together and this okay this absolutely dump thing it gives you a signature if you get the signature just for dropping each fits packet and aging a hundred data packets to the timings of a hundred data packets together and by the way you see my contribution to the stickler for free right yes and you see my contribution to this right there on the screen that

iptables line signatures from a fast network so for different ciphers you can see they look hugely different by the way if you repeat the experiment you get that you get the same proportions of dots in each spot so there's our tender for hundreds of packets together so well maybe I'll serve as the designated pointer right so there are like three clusters and don't blow the green line is the same traffic if you don't drop when your packets so if you don't drop any packets than this openvpn traffic looks exactly the same for all the ciphers but once you start dropping them they start looking hugely hugely different so this is a very this is a

normal gigabit network it works best one on fast gigabit when there is no contention so this is basically what you get the timings that you get 400 packets every time when you don't interfere and when you start interfering you start seeing that these hundred packet timings actually split up into clusters and the clusters a look are the clusters are differently populated so if you look at this as a sort of okay let's just project everything onto the y-axis and make like those dots that you're getting are actually stripes and you color the stripes according to the intensity you know this the intensity is the side of the cluster you can sort of see this as

a spectral picture and well you so can you well you can actually see it but there is this cluster there at that arrow more about the years in the moment yes AAS is actually kind of special as it turns out so yes is actually the closest thing to the unencrypted traffic at least an openvpn UDP so and in other places we looked at was also that for some reason the closest thing to an encrypted traffic so the scale is different there but these are those two red lines below for AAS are the same two red lines as order for the unencrypted traffic with each fits packet dropped so this is for openvpn so what we did with

openvpn we configured it to use a ASC bc which is its normal method and then normal configuration and then we're configured it to use no encryption the null cipher and so again are the green line is the undisturbed traffic the red line the red lines here are the traffic that has been disturbed with every fifth packet being dropped and therefore you get all of those retransmission mechanisms in place over the openvpn and over the tcp/ip and the ssh are all p UDP i'm sorry UDP are playing into this so you see those two clusters are due to some interaction between the layers and then one of those clusters splits if you are starting to use aes it's as if you

are applying you know a prism to a rail light and then you know you get a which of those three clusters that were observed on different ciphers and no encryption no the unencrypted one is at the end so as you can see the proportions with a sr closest except for this extra thick green thing so there is something to see here first even if you don't use any ink Shin at all you just use the native cues of you're still seeing the native cues of your nav p.m. even with UDP right so the open with openvpn aula regia gives you some interesting behavior just by itself right and then if you switch it to a cipher from the null cipher then

you actually get very different sizes of those clusters of four different ciphers so just because you have a file transfer and this packet dropping behavior your cipher shows statistically isn't that interesting I mean you're not learning anything about the data being transmitted but you're learning about the pipe and so you can link say two instances of the pipe to each other for example well obviously openvpn is a by the keys or a good case because it has a complicated behavior so IPSec has a simpler behavior but we still get signatures even there if we use a CP to trance transfer the files so the same experiment drop each Fitz packet time hundred packets together but we're not

just with that in this case a lot of ciphers start looking a lot more similar than they look for the openvpn so think of this architectural e right openvpn of course is a user land process that gets it packets from the pseudo interface the phone interface and so you know it your packets get out of the colonel and then get written back to the colonel that's quite a bit of cross-layer interaction at store and forward and buffering and that sort of thing now ipsec is in the kernel right ah it's you know between your second and third layer however they put it so in order to elicit some behaviors you need some other a store

and for schemes in the world some our ba other buffering schemes being involved but you know even if you're on a VPN how do you transfer a file do you do an ftp no you don't because that's you know you're cursed if you do right so you use SCP right and you use another layer encryption and that layer of encryption and that another layer of reliability if you have that gives you that behavior enough to fingerprint Lee of the pipe what happens if you use netcat well congratulations so the green stuff is SCP the red stuff over there is net cats right check the signature is gone so you can't see anything anymore so just like

usenet usenet get through usenet get through IP second nobody will be able to find out with cypher you're using there are no signatures and that's not cat with UDP transfer with UDP transfer I mean this is the simplest protocol can that can ever be right we've eliminated the signatures chaining mode doesn't care so as far as changing chaining mode is concerned this this fingerprinting algorithm doesn't doesn't can't tell the difference the results are the same as far as it can tell this is the same cipher and also notice this is ryan dale this is a s right slightly different ok so now when you actually go to slow networks you end up with a lot messier

results which you will probably have to run through Fourier transforms in order to actually get something useful out of that that mission but you can see that the periods are kind of different than the time that the writing's going on and just that it's a lot harder to tell what these things are so um it's eight really it really seems to be about the cues so say if you look at the VPN that is a user Lambie p.m. such as openvpn right you have your tongue device and your tea you have one layer of TCP or UDP that goes into your VPN then it goes into your VPN transport and then you know it's whatever a application that

you're using to transfer files so in each and every layer of this your eliciting some result ah now I will return to the slide for for a moment so if the block sizes are the same if the chaining mode if the queueing of those blocks is the same as is the case with rondell which you know became a s then some things don't matter and you get the same signature but basically you know this is a really nice way to say that these things are implemented very similarly whereas when you have other things such as SCP versus netcat or say openvpn versus ipsec they are quite different and they show that way some things that which right did not work I

could great hoops for delayed packets because this is how iska ruins your ssh it delays and Pakatan you're screwed a misconfigured cisco we are unfortunately if you take a fast network and you delay the packets then your packet arrives after the end of the transmission and you you're basically just losing that packet and if and the Sun slow traffic is a lot harder to get to figure out and fast local traffic so random drops don't work you basically get they really mixed up picture as if you were dropping and different different patterns so the the first packet in an or the second packet in n or something like this you get the summer you get the sum of different

pictures so what happens there is that a dropping and packet gives you a slightly different pattern depending on the N and if you drop packets randomly then you get a mixture of all of these and a there isn't really much to do gives you the same pattern just shifting what do I know if you're so and the last thing so may this be a warning to all of you so don't if you actually want timings of your terrific you can't filter packets one capturing because you end up with bogus time stamps as it turns out so don't do that filter them after you capture yeah BPF is killer so you get you end up with packets in the wrong

order and with timings that don't make any logical or any other sense to you whatsoever it stands to reason because BPF is its own virtual machine right that runs over the colonel buffers runs that byte code that your TCP compartir dump compiles over those buffers and it does introduced yet another layer which is more complex in its buffering than any other but when we first saw that we were like no our result is lost you know all is it's the end of the world pretty much or the end of the project and then I called a few people and said well we were like how come our packets arrived out of order that they were captured in

there something please yeah and they're like nah you shouldn't be using VPN don't do so don't ever do that capture first and filter after so what's going on we don't actually know that so there might be the timings of how long the cypher takes to do encryption it might be partially the cipher block size or it might be something about the packet fragmentation well the Rhine Dale versus IAS suggests that block size and what you do with the block matters although if you do just about the same thing then you know it doesn't fragmentation there have been attacks on ipsec using packet fragmentation and we know they work they actually accelerate a lot more than the

cipher so that's probably a thing so we've made a few profiling attempts to find out exactly where that time is lost during the algorithm and there actually are interesting patterns that emerge but we're still not ready to make any claims about this oh let me act as a laser pointer again how am i doing at the laser pointer not very sharp so on top on top of this there are this is without packet drops and on the bottom there there is a district with with packet drop this is an SSH transfer instrumented and on the x-axis are functions hey this is magic or I can play the cat and like go jump to get the

word she's really pleased with yeah Eddie Howe oh well thank you so anyway so you see there are some function there are some functions that give you a significant spread of their timings and again this is what the ssh does how it reconstruct the zone stream so when you are hit with a packet loss pattern then you see that those are timings spread into several clusters so whatever was taking time and whatever was varying now there is a lot more and its values discreetly isn't that interesting I mean the fact that we saw the clusters that were tight enough well now we are seeing that with just a standard SSH and it's you know read so

many bites off of the stream routine and so the functions are a packet read packet read sequence number packet read expect and that sort of thing so you sort of see that the a cluster of that that was really tight spreads and it spreads discreetly you know it doesn't spread all over the place as you would expect will say you know a normal distribution or something like that so we're triggering some cross-layer interactions yeah we don't know which ones yet so you saw this nice straight green lines which occur at unperturbed traffic and everybody kind of assumes that all ciphers look exactly the exact same way on a VPN right in timing wise but just because you don't see a

signature it doesn't mean that you're not going to get it if somebody does something as dumb was dropping each phipps packet of your traffic also if you're really concerned you might want to think twice before using SCP inside your VPN or like whatever the double encryption algorithm and the streams and side streams yeah so TCP over TCP is again a known bad thing to do so when you run your VPN through ssh people tell you to not to be because TCP has its reliability it's a packet loss mitigation mechanisms and when you play those to make two copies of those mechanisms against each other you get much worse that they then the delayed a curse is now go that we

touched upon so usually they say SSH is a poor man zpm but it turns out that even if you don't do tcp over tcp even if you do a tcp transfer over open VPN which uses its transport which uses UDP as its primary transport unless you've configured to use TCP you have a very clear finger printable behaviors very clear interactions so double encryption considered harmful and you know if you're sure of your pipe and you don't want to disclose anything about your activities whatsoever including what cipher you use or what software you use then your best with having just one encryption layer that's the sort of paradoxical take away their and this is work in progress so we're going to most

importantly we're going to figure out what to do with Sloane networks and exactly how much information we can get in a real situation as opposed to a situation with a really fast frustration well we were able to fingerprint a connection from boston to san francisco just by averaging the the pattern in fact how much time do we have okay well we can we can talk about that slide why not we have 20 minutes hmm okay well apparently not

so what we want to do is a prototype VPN for being unfair printable despite those things the hostile the VPN to use in the hostile environments other people are working on doing that in real hostile environments right so you have quite number of projects of that kind but we're looking specifically at timings and specifically at queuing so you know eventually they might be able to pick it up I and well there are things there are thanks I'm being thanked for the scripting and milk Ward has done quite a lot of network setup for this and our students Max and evident user have done a lot of experiments so that we could actually confirm that the effect exists

and is not just a fluke this is the theropod challenge coin the sabra fast-track challenge quiz for whatever reason it because I kept on it as you can see ya so the cat impersonations so thanks to much thanks too much for making this possible if you are ah and this is this is it thank you for listening [Applause] [Music] questions let's let's get to your microphone okay so I'm assuming you did play around with the block sizes I mean as witnessed by AES and reindl as you noted sure sure the algorithm to did you mess with the key size at all to see if there's any difference between like AES 192 256 or 128 okay and they'd work okay

take the take the year but did they like in the case of a s i guess i'm curious if you change the key size was that still very similar to the null cipher then i'm trying to recall if you have a comparison i don't remember i can look it up for you later if you're gonna publish work that like has an entire table that'd be awesome oh yeah that will happen for now we have a gift hub account with the with the reports and the scripts and that's public look in on the shoulder actually there there are there are not there are very few results there are still so there are no i don't i don't think this made it mm-hmm so I

don't think the difference Savior sales made it we definitely tried them right I remember we had the zoo but was this other Christian any other questions okay thank you very much we will reconvene in here at 3pm next talk will be at three o'clock you