ATGP - #radBIOS: Wireless Networking with Audio - Richo Healey

if a the lapel oh that is this yeah we're good okay awesome uh thanks for coming out um so yeah this talk is called red bios it's basically about how i go in together and put your thing so this is like the inevitable like Who I am slide I do some rough stuff and we're come security force right I'm gonna see how many times I can make him do that I I've done a bunch of things in the past but like most of it has been software engineering just like solving different problem with software I'm going to apologize now I managed to get on parks like before even getting to vegas and so if I wonder off topic or I say things

about making any sense please feel free to like interrupt me because my head feels like it's full of kokomo anyway so right by us um this talk is not about like groundbreaking research in like audio encoding or manipulating data it's much more about how like despite knowing nothing about this I managed to get it to work on specifically I managed to get into work while like actually decomposing all of the parts in it subselect and now like talk reasonably confidently about oh do things like excellent rating data and so the backstory is a couple years ago drag austria families like malware samples for this thing he was calling bad buyers um yeah he was positively convince to

his nation state know what the media lost their collective [ __ ] and made things like this but Sir he made a bunch of like fairly plausible sounding claims about it but there was a lot of controversy because like he wouldn't come up the samples and then he did and the samples weren't real and they sent him hablar Annie way it was a long story and there was a bunch of stuff about it but so in the middle of this whole debacle my good pal snare starts tweeting these and so in case it's not clear that is a BIOS chip on the face of someone doing something radical and so I was drinking with dominic and mike at

shriek on last year and they planted this like horrible idea in my head to build a thing that looked and behaved a little bit like bad buyers and call it rad buyers and then come and talk to some chucks about it this talk is very explicitly not about bad virus and like burnt asking about it afterwards I kind of like I'm not a malware reverse engineer and like I don't even want to have opinions about what oh I mean it's one of those things that like until there's concrete evidence is kind of pointless to debate so anyway this is like what I mean this is like the bulk of jaggers's claims about like what bad virus

actually did so it was able to exfiltrate data via audio specifically that were going off to either a key material or documentation I it was able to infect other hosts with audio which was the claim that caused everyone to kind of like hiccup and do a spit-take it was completely platform agnostic so I didn't care whether you're on a Mac or Windows machine or you had like Phoenix buyers or some other manufacturer you had you AFI as well as like you didn't care what platform you're running on top of it was like file system agnostic and finally it branded highly inside of you efi or at least as far as like launching infected to target your horse machine

which is a series of bold claims like each one independently is kind of a solid effort so these are the two things that I kind of like looked at and thought I can probably do this at the end of the day if I actually found like UEFI bugs that I could infect it with audio I would not use it to exfiltrate data I would use it to infect people anyway it's not important so cool [ __ ] rad right so I'm actually going to do this talk in the most bizarre order possible I'm going to do the demo now because the demo runs during my entire talk and then hopefully at the end I will successfully move to an ssh

key from that machine to this machine using nothing but audio I will preface this by saying if anyone has a dog or just really doesn't like high-pitched noises or if anyone in this room doesn't gonna have a bit of a headache by the end like probably mention it now I did actually so I gave this talk in Hamburg one time and a guy that I work with is literally bleeding from the face by the end I'm like almost totally convinced that it wasn't my fault but it's kind of hard to say with actual certainty so now we do some lie packing and this will go well like it always does so I actually have a shell on my demo laptop which is

obvi I'm not planning to excellent rate it over my she'll die so that it'd be super Nate sorry one second live demo is my right so I haven't necessarily I'm not planning on actually excellent rating the data with it because that would be kind of [ __ ] his Torah is swapped cables between two machines is miserable as is trying to type like this bail I do have mail come on you can do it i believe in you this is not the failure mode i was expecting for this talk this is what i get i decided to use my phone tethering instead of nakhon Wi-Fi because i figured that had a much better chance of

not screwing me ah ok so I sweet I actually had the funds to big perm once ok so what I'm going to do is so it sits on top of this platform program station I have this script called slurp and so slope you just give it a file and it adds it to the local database of like stuff that you want excellent rated um so I'm going to run this sunday that's sh k I seriously considered using ed 5519 case because they're like this lung and it'd be crazy easy but I was in a good mood so I did it with an RSA key which is actually like fairly sizable and so now on this laptop the one that

I'm presenting on I'm going to run the server which is called soundstation for fun honey reasons i'll get to in a second so this is going to start listening on these three frequencies and then on this one look if I remember how to computer okay sorry this machine is now like broadcasting huh that's new I [ __ ] love live demos whoo okay so what's happening here it Nate ah so two people independently said that they're going to [ __ ] with my demo and I'm starting to wonder if that's the case all right now until either start working or I will do some like spectacular [ __ ] to try and make it run anyway one second I

swear it on this ball all right [ __ ] this yeah yeah get bak William

[Applause] yeah well I'm damned airs anyway sir oh [ __ ] now i'm catching that exception oh [ __ ] i know i probably have some Python i care about at least the pit isn't it oh god okay so is my connection Fox like really this high swear to god alright just take my word for I'm starting it on this computer it's not over ssh it's doing your thing it doesn't matter now because I caught the exception right let's get back to the talk uh as I'm going to drive into it actually it has this like unique property fault tolerance in that even if it pulls over a thousand times if it works even once

it'll work let's just see what's gonna anyway it's not important Denver's son so as you probably gathered I built on top of this thing I wrote a while ago program station answer ground station is a distributed graph database everything in it is content addressable which is kind of eight kind of have some like awkward API constraints but the interesting thing about it is that when I originally wrote it my goal for the project was that if you take two machines near one another they will automatically like share all of their state and then if you take those two machines and take them to two more machines you're going to have this like graph explosion but what you wind up

with is a bunch of like shared information amongst everyone who's ever interacted with the graph and so it's almost entirely written pipe in that's very few external dependencies and kind of conveniently at this late stage in the game I deliberately built in a really modular fashion because I wanted it to be possible to kind of extend it with I like new types of data to put into it and most specifically like more transport protocols like audio and turned out later um so anyway when I first heard it I literally was just screwing around with here I had for a decentralized network I kind of in spite of the fact that I get a little bit mad

every time anyone says decentralize because there's nearly always not actually the thing that they want I thought I wanted to decentralize thing I also explicitly wrote it because the company I worked for migrated to zero and it's a piece of [ __ ] so I replicated jira into ground station and then I heard a thing so that everyone in the office if the ground station knows where near each other it would like sync up the geodatabase I did all of this to avoid interacting with JIRA and honestly i recommend the experience so anyway it does a couple of other kind of neat things out of the box one of them is that I mentioned it was content

addressable and the reason for that is the object database i'm using at the very lowest level is actually get so uncool property of it is it if you point it at a git repo and then bring it near someone else it sinks get wrappers which turns out to be really cool to have like 200 software engineers for example anyway it has a shiny web-based i used to have it in this talk I took it out because it was sort of meaningless and waste of time but Sir ground station there's kind of three core components it has an object graph has protocol drive is and it has transport drive is I'm gonna talk a little bit about the first

two because it kind of doesn't make sense without them but obviously like the main reason we're here is transport so like under the hood it's a graph database and so what that means is you have nodes and edges and the edges are basically like a property of the nerds and it's everything it is mutable so the only operation you can do on the graph is to insert more data and I mean this sounds limiting but it turns out there's a lot you can do for example if you want to update a nerd you just like pushing garden has a reference to the one you're superseding and you just put the new date or in that and so what this winds

up looking down [ __ ] awesome at Kane or dot so anyway this is like a really you're a very very simple graph I'm the only operation you can do is like put more stuff in it right and so as you're adding things each node has a reference to its like logical parent and that's how you traverse the graph you start at the top and you just like keep walking parents until you get to a thing without parents um this is called property that if you lose data or you never got it in the first place because you were using a janky audio transmission mechanism or whatever the pots of the graph that aren't actually often by that missing

nerd are so traversable and complete which means that for a lot of use cases if you suspect that you're only going to have a fraction I'm still like reason about it meaningfully and so I just kind of like to try and break up so it hopefully makes a bunch of sense so this is the source of truth which is like the object whose name you would refer to to talk about this whole graph and this guy is like the root which is you're the most immutable thing is there really nothing you can do about it now so anyway that's like an overview of a graph database as far as critical driving scariest this like thin layer

around the guitar DB basically so that they have some notion of what your underlying data is actually meant to look like I they handle doing things like presenting the data to the user as well as like taking data from the user and like munging an into a format that ground station can understand there there actually a bunch of these now the happiest moment of my life is when somebody wasn't me contributed 12 ground station which I was knotted in expecting and so like get is actually technically a protocol driver i guess but someone wrote like a message board and so you like post them and you take your laptop for a walk when you get home it has new

posts in it and they're all from nerds who do this it's not good it's not important but so this is the protocol driver I just interactive with that you hopefully just saw and this is like a really thin example like all the notes how to do is read an arbitrary file shove it into the database and then when you eventually run a triplet driver can excellent rate it to somewhere else to like we're finally getting on towards like actually talking about transport the first transport protocol I wrote was kind of naive although it it went up scaling car than I expected so the assumption was that all of the nerds are on like a homogeneous link layer Network

so you can at least like reach other peers and your broadcast traffic will like find its way to someone and so what I did was I set every nerd up to just like shriek UDP broadcast traffic indiscriminately and if any of the other nodes like heard it that had enough addressing information that they could negotiate a connection they connect sync up and do this like hilariously complex handshake that I devised for like optimal sinking of the graph database and then eventually like both parties now have all the data prey um came like what does this actually have to do with hopping air gaps I couldn't find an actual bunny hop photo but he's like so

close so first of all I'm going to preempt QA every single time I've done this talk which is why didn't I you sound burtom which is this thing the Linux kernel which kind of natively tries to do this organ your radio or whatever there's another thing like mini something anyway so they're a bunch of there are a bunch of projects that sort of do this they're very black boxy like it they're almost entirely inscrutable Sam Burnham turns out to just not work very well as being Linux Pacific we're trying to defeated my works everywhere I go can you radio I'm pretty sure no one really understands except whoever wrote it I'm like dulla thing up there no one

told me about it um but I mean the the crux of this was specifically that I was coming into this knowing basically nothing about audio except that like I'd seen a sine wave before I'm like I I wanted at the end of this to be able to like at least like ask reasonable questions even if I didn't have all the answers sir I found this thing called quite matte which was actually the thing that I got screwed by um this [ __ ] Kate Murphy I actually ran into a couple of years ago at pea-sized runner and we hung out for like two hours before we put together that she was the one who wrote quite net and I was going to dry

by oats which was like an exciting coincidence but so anyway quiet net is basically the same received protocol that runs on two machines it like sends data from one to the other but it's like almost entirely written a very screwed up a pipe in the only third party dependencies it has this pie audio for actually like driving the sound hard way like taking your array of samples and actually like shoving them out of the speaker and numpy because it turns out that like math is fairly expensive if you do an amuse learn Python and we'll be doing some math um so this is an example of the quiet net session and I would love to say that I like had to try

really hard to get it to fail in this way unfortunately it's just like fairly unreliable but for like very defendable reasons like if you're using this in its kind of intended purpose which is like I type things into my computer and they show up on your computer I mean like reading this you still know that I was trying to say hi there or something close to that anyway um and so the reason that this corruption very specifically happens is that it uses an encoding called psk31 and so I've obviously like trim this a little bit but looking at it you can very quickly see a couple of things one is that there's no repeated zeros in the middle

of any of these code words the reason for that is that it separates code words with two zeros and so it would be impossible to tell the difference between them the other is that it's like very very obviously optimized for English text the shortest message is a space and the second shortest is an a so from the ground up it was very deliberately designed only for migrating like text that looks a fair bit like English around I at one point one of my very early prototypes actually base 64 encoded binary data and then transferred it with it don't do that anyway um so yeah it has some nice properties slow particularly one of the things I ran into a lot is like how do I

synchronize the two streams like how do I get the two computers to agree on where in this like stream of arbitrary data we are and it turns out the decidua link scheme that doesn't rely on like fixed length words and constant sync is really really convenient you just like drop into the stream and then the first time you see two zeros here at the start of a word and then you just like keep reading awesome right so I like took quite net and I like hacked on her two scripts until I could like very reliably move like a kilobyte of data overnight and then I started thinking well they're probably a better way I might actually

have to go in a bit deep in this no one's going to be terribly interested in this and so I asked myself how we've into warrior and this is kind of what I came up with and this isn't like a perfect pipeline but like in the general case you probably find yourself doing something that looks almost exactly like this so at the very top you like reach out to your audio hardware and you say hello I would like a stream of frames and it's possible to peyote or seg faults but if it doesn't you get a stream of frames and it's awesome so then once you have your stream of frames you like take some frames and you do a

thing called a fast Fourier transform which turns your frames into points and at that point your points now represent like how much how much activity there was even given frequency at that time and then you take your points and you walk along them like with a sliding window saying like hey at what time was there this turn that I'm very specifically interested in and that you to bits finally once you have your bits you can in my case I have the [ __ ] out of them which I will also get to to try and turn them into symbols and finally you turn the symbols in two bites and then you have data so simple why didn't

I just do this nip sorry all right I'm going mad I swear all right anyway so I fourier-transform i figured i'd skip over the like give me frames thing because it's like you import pi audio and say give me frames um so a Fourier transform this is like that we can pin your definition it transforms a signal from its original domain which in our case is time into the frequency domain which I mean I sort of get it now having done this for a while but like the the actual thing that it lets you do is ask the question was this turn present in this set of symbol in this set of samples I'm like programmatically kind

of manifest is just this big array of floats so this is the like fourier transform gift like sort of helps me understand it a little bit I guess so this is transforming the wave into like array of flirts and then it'll play again and I clicca this is literally the first time I piece and so if you do it again you transform back from whatever domain you're in back into the main of time right and so it's a transitive is the mathematical property for this anyway it's not super important the important thing is that you FFT the [ __ ] out of your view frames and you get a bunch of points right and so this is

what your points look like after you extract the intensity of a given turn and and so once once you have your points you can start walking along them and saying like what is the average intensity over this window right I'm so like as you slide along them for like this set it might make sense to use like three is your threshold this gives you a modicum of hysteresis protection as well as just protecting you from people in the audience making hideous noises actually so that was the other thing I forgot to mention is that at least one Joker was working with an air horn this morning I so y'all might you get nirvan as well as an egg so anyway you like

slide along your window of frequencies and you transform them into bits I'm sorry like now you have an array of bits and so like the naive idea would be to say well like if if the turn is there it's a one and if it's not there it's a zero like that's sort of how binary data works and this works okay if you know for a fact that you're in a stream and you have some way of like independently sinker in the stream with the other end otherwise like you're peeing written bad Python or whatever and crashed is just going to transform your stream until like megabytes of zeros which is almost certainly not really the thing you want

and so I fed up with this for a while and I brought some really really Jenky stuff that sort of worked in the lab and then I called it a day and then two days later I was like okay this is not okay I should do something more professional um but Sir the reason I actually go node selected in the best place was I talked that my husband and Dominic spell game shrink on 14 I'm gonna try and avoid digging into it to date because it turns out you can't old for almost exactly an hour about it comfortably but the notion is that you have lineally isolated Hamming cards and so this is an example

of a having card you can flip any bit in this message and unambiguously work out which message was actually meant to be which is really good right like if your data might get flipped on the wire you can like take it in on packet you obviously like you pay somebody head in like how much data you can transfer like I have to send six bits to transfer like to which is not great but one of them um but Sir the the interesting thing about Theory codes are nothing you the interesting thing about their research was that they split this into two sub cards right and so if you flip like any two bits in this message you can still

unambiguously work out that it didn't belong to this set and so the reason they'd built it was a defense against the packet in packet attack that Travis could speed developed where you basically just like send valid wireless frames in IP payloads and the first time something bit flips that like reads aren't until it finds a valid looking header and there's like right petter data I love header data and you get to inject packets into into a network that you're not privileged on huh and so their proposal was that we incurred header data with one sub card and paler data with another sub card such that even if it gets bit flipped you still tailed it like you ought not pause this

payload as a header because I probably in padley and so this is kind of like a you know an example of what this actually looks like in practice like I flip this bit at the end here and and you basically just like look at edit distance most people do this with look-up tables another early prototype of this so often they give the talk this guy the poor request saying like hey your C++ program that runs for like three days to generate these cards is no but said three which is a satisfiability Silva can do it in like nine seconds and it was like so fast but I just like took this entry implementation which happily has five

bindings and literally put it in my thing and invert the SATs over for every single packet we shouldn't not to be optimal you just generate look-up tables um but Sir I still like I was on the fringe of knowing things about DSP at this point I'd like I'd processed some signals in my time but I still I still like didn't really really know what was going on but I kind of looked at this thing and I was like wait let me get this straight I can get bits wrong and they'll still be right amazing it's gonna be impossible but with that um so I literally just started trying to belt nails in with this like amazing hamma

and so this is what one of my first attempts look like you might be able to see where I'm going with this I so you remember my psk31 slide where it was like you can't put interior zeros I just stretched everything I just made it all way longer because it turns out like the biggest something is the easier it is to find this actually did what um one time I did this I'd murder 240 bytes in 45 minutes um which like I agree is [ __ ] all don't want to do that however like I one of my first memories I spoke about earlier moved a kilobyte urbanite um and so like sweet I nearly picked up an

order of magnitude Apple's still useless but you know I took exactly the same process and to be fit the method of picking these ones was incredibly scientific I wrote a program to just run it over and over and adds add ones until it successfully got the same data I call it yellow science so anyway this is like what that actually looked like on the way this is just some like protocol analyzer app for my phone it looks like moss someone in QA will probably point out that I should have used moss technically they're right at this point the data rate would've been significantly better but so like you know I I had all these like janky

mechanisms for moving data across an air gap but I like I didn't really solve the problem of like doing it reliably or interpreting the data afterwards or like doing it in a knot lab condition where I'm sitting there and I can very carefully press return on both machines at exactly the same time because they kind of hinging on this assumption that they're synced up but at the same time it's sort of worked in the lab so I was like ah must be close now as I mentioned earlier I ground stations first implementation had this like to make a bunch of UDP noise and then establish the TCP connection with the other end anyone who's dealt

with TCP news is like sort of noisy on the wire but I was like well like connecting from one machine to the other I'm like sending some data with audio isn't crazy hot it sort of works like just doing it in the other direction in like a synchronizing coherent fashion ought not to be too hard I can just write a bsd socket interface and like TCP sockets and it'll be awesome um and like all I had to do was change all the constants from like import socket to like import audio whatever and like just call it socket thing and I don't have to change any card hey plus so that doesn't work um audio is really hard duplex is

crazy difficult as I discovered um one thing yelling or what one thing yelling and me make a bunch of noise like I am at the moment is like reasonably straightforward but two things just trying to negotiate what they're even going to listen on when they don't know anything about each other is a nightmare um I'm sir I kind of like I fed up with this for a while I actually like I had a bunch of reasonable stabs at it right like I really wanted this duplex thing to work um and as it turned out I sort of wanted to work for bad and stupid reasons but like the thing that I wasted the Mar Simon was reinventing wheels

like the best part is that at no point in economy just like braid the TCPS back to like see exactly what it does like it the UM but Sir this was my first attempt where I like I wanted to have this like a TCP like thing where you like send a packet and then your other and accent if you're paying very close attention you realize that like a TCP tailored is gigantic and TCP ack is tiny but in this instance I'm really a bitwise so that didn't work very well in as far as I mean like it sort of worked but my data rate went back to the like kilobytes and a sort of order and so like this didn't

get me anywhere but then I kind of figured well like what if we take the retransmitted perch like instead of actually waiting for an explicit ack what if I announced myself and then wait for my like I make the announcement turn every like five seconds I'll sorry and when my PG is it they just stop making a bunch of noise on like some random frequency and I just keep an eye on the spectrum and the thing that like gets really loud after I announced is like probably the thing that I'm interested in sir also I like start listening on that and then the second I have that thing i will start barking and then we've like sort of negotiated frequency

maybe it'll work it was really difficult to optimize for I also did some pro Yolo science every single time I've given this talk where I like look around the room I'm going to give it in and try different frequencies still the transmit rate is good so anyway I kind of tried this thing and then instead of trying to act individual payloads like everything in ground station is request remand you like there there are no informational message is the only time you talk to the network is if you want someone to do something so I thought like great I'll just like keep sending messages until I get a valid response to them and that k that way i avoid like needing explicit

acts I just like wait until someone up and like definitionally is got it because they responded to it it also doesn't work but so I kind of went back to the drawing board and started to think about like why I actually wanted duplex communications the reason i implemented them in the first place in ground station was because i was thinking gigantic databases like some of the get wrappers that operates on have like gigabytes of data because they sometimes a designer checks in a Photoshop file and that's the end of the goodness of your get rapper but like in this instance there was like never going to be a case where these databases were like big big I mean if they're if

they're that big and you're trying to move it around with audio like I think you just need to look at your life and your choices but it also had a couple of like the duplex thing actually broke some things that I kind of felt r innate when I was first thinking about this so one example is that like as it stands like this machine is making noise and kind of because I'm doing a demo like I'm targeting this one but actually like anyone in the audience could like run get this random SSH key one of the things I've been sort of playing with the last few days because only just occurred to me was actually just

recording it on my phone and then like playing it back later instead of trying to like positive real time um but so we like yeah my to Blois thing was like complex and error-prone at my favorite a complete monstrosity to debug because like I didn't even know what frequencies it was gonna pick and year I could blog them that I was often recording the wrong thing and the handshake took forever because when you're transmitting like ordered bits a second as it was at that point there's a lot of sitting um so I I started like I went back to the drawing board and felt like how how can I make this work without doing this

really hard work which is difficult and I went back to thinking about like the way my graph works so earlier I mentioned that the graph I'm using has this property of content addressability which if you're not familiar with it ordinarily like when you think a database you think of a thing that like probably has some like identify a tailored pez right and you can go to the database and say hey I i have this ID I want the dollar associated with it give it to me and you get to pick both of them in a content addressable database the name is always a digest of them and so what this means is you get given data

without any like a priori knowledge of what it's meant to be called and you hash it and then you just shove it into the database like with that name and so when you're encouraging this you actually stick the parents inside of the body as you know an array of names of parents and in this way like the entire message is kind of like self contained in that if you flip any data it'll actually hash wonky and it won't wind up there right so like in this since this is a I couldn't get exactly how to try and describe this but like so this guy here like has the hash of its parent stored inside of this body and this one

got corrupted on the while like a bit flipped or whatever but that made a hash entirely differently and it kind of like lands outside the graph conceptually which means that if you just like keep trying all of the [ __ ] up packets we were just like not land in the way and then when you traverse down from here like you will only hit the good path so I was like okay cool like instead of trying to be really really clever in like optimized for always getting the data only transmitting data that my pian aids and like doing all this smart stuff what if I just like yell incessantly all of the data that I have I'm just kind of

like assume that my pay will hang around for long enough to like probabilistically get it all um which like again like it has it has some neat property I mean one it works great but it also means that like you can actually have like essentially a data fountain and like going near it means that you get a set of the data and if you come back the next day you'll get some more and so forth so anyway I cranked the hemingway back because unfortunately like with consistency comes just like more data that you need to transmit and I went really overboard with the like just add more bit solution to all of my problems so I went the

Hemingway back to the point where like transmitting messages was feasible again sounded like Yelling's i junked up the payloads really aggressively started yelling them and just think everything into the database whether or not it necessarily looked okay or not and this actually sort of worked um and so I went into this hill climbing mer I was like okay I feel pretty good about my mechanism for moving the data around but like I'm sorry I feel pretty good about my mechanism pre carding the data but like my fur Phi layer that I've kind of yellowed together is kind of [ __ ] like what else can I do so I decided to actually read a paper and I had sort of

valid reasons for not doing this verse incidence the biggest one was that uh you know sort of like dumb reasons I was really really interested in actually kind of like deliberately not invented hearing the whole thing like building it from the ground up as much as possible because i figured it's like extremely difficult to build like not impossible but very very difficult to build something yourself that you still don't understand but so I kind of like I think I'd roughly reached the limit of what I could achieve like inventing Meyer and signal processing algorithms so I started doing a little bit of reading especially i started actually reading up on things that I'd like her dog for the

math looked really daunting in the first place Oh unambiguous encapsulation protects me from bit flips in the sense that if I have like a well-formed message and it's a bit wonky I can like fix it but it doesn't help me at all when I have this like long stream of bits that is like your orders of magnitude bigger than individual message and I'm like great like where does the message stop um I'm at various points I tried your this like monkey we know anything where you like slide along it and then you see like at what point do you have the most valid messages and like that's probably the correct offset but it had huge problems and it meant

that like if you drop a bit in the middle you like have to go that you have to keep trying this and it was like a lot of work and I just kind of didn't care for it like I I kind of got the feeling that someone had done this in a way that was less junkie than that um so I went back to like what I was actually doing to key the data on the air so what I've been doing so far is called frequency shift keying right you basically like make a turn on a frequency whenever you want to send a bit and like as long as you've agreed on what the frequency is

in the encoding scheme like you can transmit data great and so I'd read a lot about phase shift keying which is like unambiguously got it the correct thing that I wanted like that is what I wanted to do but I saw I read a bunch of papers and specifically I read some implementations of it um which are difficult to find native Python which was kind of like I just didn't want to c++ monstrosity um but I kind of I go to the point where I like I fundamentally agreed with the author that it worked but it was still like mostly unclear to me why like you take the signal and then you delay it and then you multiply it by

itself and the signal just like comes out and yeah so I kind of looked at it and I decided who's going to be easy to build a smarter FSK than it was to actually like wrap my head around psk so I kind of went back to the drawing board and looked at like the way my FSK worked and so as it stood I like had a single bit of information which is like is this started on or off and I was encouraging bits into that by saying like being with my one bit which was kind of unpleasant for a variety of reasons so I can looked at him like well I have this like entire

spectrum to play with and I'm using this one arbitrary frequency that worked pretty well in a conference room once like I can probably do something smart of that answer what I did was I separated out into three frequencies right so I have 19 k which is the headache hiest one I found so far which denotes a 117 k which tuners essential which is basically like this is the gap between two messages and 18k which is a zero and so once you take your string with data and do your f of T on it again you wind up with something that looks like this so this is just a couple of floats that I was printing

and it basically shows like I how prevalent each of these three frequencies are at any time and so this is before I kicked off the listener all right sorry before I kicked off the I think I called it a Shrieker in the card base because I thought it was funny and so this is like basically what background noise happens to look like in my hotel room last night but there's once you start it all of a sudden you get these like gigantic i'll use it's like great there's a signal and so one of the interesting things is that I your because of both like my technique my poor choice and frequencies and the fact that I'm not kind of doing any post

processing to clean up the signal they all jump up like in this instance like the third column which i think is the zero but in this case is like intuity I mean like I bowling numbers you'd say this is probably it they're like oh pretty high as well um answer like one of the first things I was doing was just like looking once above a threshold got me nowhere I had to actually take the data f of T it find out if anything was about the threshold and then we do the sliding window dance again to see like which one was consistently above the threshold um so anyway I was like I did this and I got like another order of

magnitude improvable on my bit rate as well as like a much better accuracy like that the Hamming wasn't needing to soak up quite as many things but like my hamming code rib Goldberg machine was still doing a hell of a lot of work and I like kind of looked at that as like the next thing that I could plausibly optimize and so this actually came up when i was talking to oz minutes me this year um because I filled him in on all the stupid [ __ ] I done in the year since he gave me a bad idea um that like yeah this is kind of like how I was having these bits before right like I i take

the bits and I feed them it in many instances like I mean this this medium is analog and the way I'm interacting with it is naive it's pretty common that like I just like I have indeterminate pits and like traditional signal processing kind of like well so it turns out there are two schools of thought on this this is all ausman telling me that I'm an idiot so I I smuggling was like I invented the thing so typically when you're processing signals at some point you have to make the choice between hard decisions and soft decisions and what that boils down to is like how early in the pipeline you take your array of floats and turn it into like concrete

bits and I took the third door because I didn't know there were only two so what I did was like when I had bits that I just like wasn't sure about like it really could just go either way I just like stored a third value and I let the Hemi card suck that up because it meant if I resolved all the bits that I was pretty sure about and then result the bits that are actually like guaranteed to be wonky I wound up recovering a lot more data and then I mean the process we're doing this is pretty easy you tried both of them whichever one has a closer edit distance what value valid word is almost

certainly the thing um so anyway does anyone have an egg no one amazing is anyone bleeding okay so I have a hunch that this is going to not be working just quietly alright well that's definitely failed it is very loud so the reason for that is actually honestly a little unclear to me used to do the clicking so the clicking isn't the thing that it's listening for the clicking is like so if you if you imagine that you have like a waveform like looks like this when you start a speaker you want to like kind of like ease into it basically like you want to like be exactly and that's that's what it is it's basically every time I light up the

speaker it pops

I mean that that wasn't me if it started before him I I actually I would totally serve more than one person didn't even threaten just flat-out told me they were going to so like yeah I I would actually believe that in any case I'm efficiently evidently I wrap the wrong thing and rice half so i'm just going to write this off as a failure I'm sorry guys is the first time the demo has failed yeah i mean i can it oh [ __ ] let's just do this again um really Oh needing crash awesome love psych hold anyway sir um I can make this stop I don't this is sort of pointed at me in my head really hurts

now um sir I'm gonna regret doing this in a second um okay because I don't have another SSH key handy so I'm going to use my real one actually no I'm not going to do that [Music]

[Music] so I let's do neat ok so I wrote an object into the database so this is where Sir but that's an a I did music once um great uh how loud is my volume sir so this is what it actually wants upsetting like a low I oh um yeah it actually is does anyone here which frickin Modi it is definitely Hume using anyway so I'm going to tell you in theory I could have pulled out a pot um sorry yes anyway I should totally do that like if it sir I mean like I I did like learn some stuff from doing this make some conclusions I mean like the first one which probably goes without

saying is this is a really stupid way to steal things from spines you actually like maybe sure use this for a research project um I kind of like from the ground up all of the components that I built in this are deliberately really sort of reusable so like if you're building something with this or even like toying with the idea or you just like you're in like [ __ ] you you're an idiot I can do this better would you probably can shoot me an email about I would love to like chat about it and see if we could work together or something um oh do isn't as inscrutable as I kind of felt like it was when I picked this

up I'll buy to be clear it's still like pretty inscrutable um I I spent so much time shotgun debugging with the the signal analyzer on my phone and the best part it doesn't have a zoom thing so I had to like take screenshot something like it yeah it's not great um sorry fun fact do you guys want to see bold lines and it's it's not bold line is broken on my computer and it makes me angry but like the hill climbing approach to this actually turned out to be much more reasonable than I use mean to me so when I picked up this project like especially the first few weeks were just like misery and sadness

because like nothing worked ever all computers were bad and so I kind of like I got the sense that this was going to be a very binary thing right like it either like works flawlessly or doesn't work at all um but like once I got over that hop of like actually getting data off the wire you're doing something with it and it having at least like something to do with the original data even if it's not bit forbid identical once I go to that point like the the optimizing for local Maxima in like some specific field and then like picking the next thing that seems like a bottleneck actually worked a lot better than I was

expecting to and if you do this you will sir get used to coding with a headache um one time I flew from Melbourne to New York and worked on this on the plain truth so I have sand canceling headphones uh yeah every was miserable it also turns out that a plane is an acoustically shitty environment for this kind of thing and then I have to do the inevitable like greets thing ultimately I would probably never done this research if if snare didn't tweet a bunch of stuff about rad buyers um Michael husband for being really really gentle with my feelings when you told me that everything I done ever had been invented years ago Mike and Dominic for

doing exactly the same thing but they weren't gentle with my feelings Kate Murphy point at which i like borrowed a lot of both card and ideas from I'm finally so I went to contact this year and I met dragos I was half expecting to glass me and he didn't do that for taking the piss out of him for an entire year so that was pretty name um oh yeah this is like crabs station is up quite so I actually I didn't want to welcome this in the last couple weeks and I didn't want to release it before the talk not it's not sweet air today but I was like Aaron whatever all like publish it off the things so I'll push that

today quieten it is up there and like you can play with it as well as like samer to guinea right here if you want inscrutable things or you're smarter than I am you can work in your radio cool I guess I had some time for questions does anyone have any uh yeah sir it was all these things that like I kind of looked at two um actually forgot to put in my slides now I mention it I probably have content for the next next ten minutes so one of the things that I actually wanted to do was I to do like six FSK so like my my hamming codes were like six bits wide and so I could

completely eliminate the like we're in the stream problem I am if I can send like all six bits at once the problem that I had was that I as I mentioned earlier like the frequencies all like bleed over into each other and that's really not a problem when I only care about the strongest one like the strongest one is probably the thing I was trying to send um but with all six bits like all of a sudden I actually like need to care about like the individual values of all of them and so like the bleeder that just like obliterated all of my data I got the sense from like the rating than I did

that sk was gonna have similar kind of problems in the sense that like there was just like fancy post post post processing that I didn't do before to be meaningful that's not to say it kind of artwork that's kind of like why I didn't go too far down that road did it to be honest I kind of like I go to the White Rose making real ground with FSK I'm particularly like FSK is really really easy to debug like you just look for the hot patches on the spectrograph so that's why I kind of just like the system with it

yeah so I I kind of like i intuitively knew that numbers that are multiples of each other is probably a terrible idea um it was a little unclear like to what extent I had to go overboard and I saw I pick some random numbers and they worked it was like a plus um so the other trick would that was um I was using shitty laptop speakers of mics and when I was trying the six FSK thing I kind of had the sense that like if I could spread the frequencies out like far enough I'd get away with it the problem being that like there is only there is actually not very much usable spectrum between the

audible like the audible band and like the point at which these speakers are just like not capable anteriorly producing noise anymore and so that really limited my options again if I was trying to do their name thing which I kind of consistently was so I think from memory the human audible band kind of ends it like order 14k or something they're a bunch of audio people in the room who probably want to correct me yeah sorry 15 to 20 yeah um and so like these machines like I mean to be clear it's a little unclear as the problems the speaker or the mic but like anything about 20k was just a complete write-off basically impossible to recover data um

so I didn't go too crazy with that I tried it unlike a bunch of laptops like basically any laptop that someone let me like execute arbitrary card on um i had a stab at it results were pretty similar I mean I was a little bit surprised I'm like I'm not an audio file per se but I like I really like music and so I have a like reasonably nice set of like doesn't my desk and so I tried this with them because i was like maybe they'll be good but i mean it turns out the a like studio monitors are kind of really aggressively optimized for the audible spectrum because giving people headaches really delicately is not super important

to their business model i guess um i guess at the end of the day that the takeaway i had on like what's because i'm using is that like everything I'm doing is sort of like naive and sledgehammer enough that it didn't really matter um I it up with for a little while because curious know if I could actually fingerprinted machine based on its own your signature because I thought that would be kind of cool I I got as far as like yep that's a computer wasn't as simple as I was kind of hoping yeah so I mean that's cropped up a bunch of times um a lot of people have told me that I need to jam listen to dubstep because no

one will ever know to some extent it's kind of like I mean it depends exactly what your plan is as far as tagging it the way I've implemented this it's like fairly tolerant to background noise accepting this one bloodbath demmer it is very reliably worked in a lot of rooms like while I'm talking and answering questions and whatever in the past um I think it particularly the way that I'm doing it like you would almost certainly need to do something kind of fancy or like psk where you you treat the music as being like a carrier wave and then you modulate that yeah with the data I mean like doing what I'm doing by just like lowering the frequency so it

like happens to be in the music I I don't think it would work I mean basically all of these things are like it is definitely possible like whether or not it's within my reach or it's something that i could make work and then except weeks reliably is like probably not sorry so the question was was i tempted to implement what was it v32 yeah so um kind of so I read there's a there's also a Linux module called acts 25 which is another one I actually under so I cut a bunch of a relevant stuff on this talk but there is a project called Byzantium which is trying to build distributive mesh networking which was actually the

people that contributed the other module to ground station because they wanted their their use case very briefly is like you're in a disaster affected area like all of the infrastructure is destroyed but everyone still has laptops it would be nice to be able to talk to each other and so they built a message board on top of it so that like I put some like I put a person's message war being like my girls have escaped and someone else puts worse in the message for do you like who's guards are these and then someone who knows both of this happens so like ferry between his boat and the graph sync up and like now I

know who has my goats um and so they going to contract from or bounty or like a grant from some organization to try and make their thing work over ham radio and so I worked with them on the original like acts 25 implementation um and so that was one of the things that I picked up nose like this will be easy it turns out that like once you're on the other side of a ham radio like your oak but your signal is already so much better attenuated than like portable noise in a janky room with eckers and misery my sense is that they're not like even if you implemented it it would still kind of early get you as far as

they like protocol decoding phase like I don't think it would have helped me very much as far as actually like pulling pulling signal off the wire in the first place but I could definitely be run okay I'm nearly out of time I've got time for like one more but three minutes I've time for three more but there are no more questions amazing chosen thank you happy [Applause] um