Asymmetric Warfare and Corollaries in Network Defense

so i'm going to start out let's see it's a small crowd which is cool i'll try to project i'm not going to make a big deal about it but this should be good how many of you are network folks what i mean by that is you've configured or you know how to run a router or routers switcher switches firewall firewalls network services dns those kinds of things so this is a network kind of thing that's not disrespectful towards the coders or application security guys because we'll actually talk about a little bit of that but it's just going to be touching on the edge of it so based on that how many of you know what flow information is network

flow how many of you collect and or use network flow data okay so i'm going to just tell you for just a minute what network flow data is and this is a simple way of saying it when you have traffic that you're sending out on the internet or transiting on the internet network flow data pretty much gives you a few really critical things source destination address source destination port time stamp you'll get protocol information you can get other types of information in it as well but generally speaking and i'm telling you just straight up it's it's not payload data this is not pcap type payload data network flow data is just kind of like a

network telemetry type data that's not user related but it's transit service provider type related um that being said i'm going to kind of talk from that mindset and i'm going to mention a little bit dan thanks for the intro and some of you folks know me and i've known so many for years and years now but but uh i work for the utah education network we're our service provider for about one point five almost six million i p addresses it's as210 on the internet we run six to seven pairings uh internet border routers and it's a fairly decent sized network in utah it represents 41 districts school districts all libraries all charter schools connections to a dozen higher

institutions and so it's funny it's just big enough to be interesting right but it's not too big to not be able to see so we are a service provider from an internet service provider standpoint i tend to i have the opportunity to go to security conferences that are service provider related security wise so some of the things that i tend to really enjoy from our conferences here you know saint or hear def con or even some of the stuff that's involved with the 801 group there's not a whole lot of relation with some of the ways i perceive security that's not to say it's special it's just sometimes a little different with that background being said i was at

a conference a few months ago how many of you know what twitch tv is and who wants to just tell us what it is it doesn't matter who okay so who has a guess on how popular it is huge like i'm telling you right now we went to kind of like a beer and gear kind of social type thing a few months ago at one of these presents or conferences and i was just shooting the bull sitting down enjoying kind of the crowd chilling a little bit and i found myself sitting next to one of the twitch tv backbone engineers which was nuts i mean nuts and we're sitting there talking and generally speaking and this is just kind of open

information i don't think there's anything crazy here but uen you know one point almost 6 million ips 41 districts all that stuff we probably see a benchmark um flow somewhere in the high teens the high twenties gig um backbone-wise but um and we'll see high-water marks that jump up into the 40s but if we go if we get up into the 30s and start leaning up there there's something wrong um either start a school or there's a big update for apple or something like that but generally speaking when we move out of like high teens mid 20s high 20s there's something not good going on so i'm sitting there talking with you knowing about that now

about somebody who's used to seeing 20 30 40 gig of backbone traffic in a service provider environment and over the course of a couple hours the guy mentions to me how they distribute that service how they deliver twitch to their customers which is crazy i mean it's crazy and it's interesting like we would tend to think about core networks and backbone networks in a certain way but when you start talking to like major global networks that are top five consumers like commodity wise of internet bandwidth it gets to be a bigger deal and their architecture is starting to be crazy so i'm telling you this information just kind of as a secondary thing but we were talking about some security

things and how they actually scale to address security they don't scale vertical they scale horizontal that's an architectural thing and so they had an 800 gig ddos sometime in the last few months 800 gig i've seen campuses tip over 40 i mean just they're done so i'm like holy crap man how much bandwidth do you have the ability to transit he's like we're about 1.2 tera that's nuts so i'm like how do you mitigate it how do you you know deal with that kind of an issue and he said we just eat it they just eat 800 gigs so it gets me thinking about certain security implications and that's kind of the place where i'm coming from that's why i

asked a little bit about your background and wanted you to know kind of where i'm coming from this is my march madness bracket okay and there's a point to that this is today's schedule for games and this is today's schedule for b-sides right so we're going to get this done uh this talk happened by accident i'm telling you straight up it wasn't a bad accident but it wasn't a close call either uh i'm on the committee with b-sides somebody invited me i don't know if they'll invite me back it's been really cool but um sean asks they send out cfp the call for presenters or we started to get ready to send it out and

nemus put together the code for it and they're like we need somebody to test i'm like okay i'll test it because i can type like the wind blows you know so i just typed in a talk that was a crazy idea that i thought would be cool and it goes like that didn't work try it again so i'm like fine and it went through and he he's like okay cool we're good to go and it was just one little blip and seth knows this and whomever else on you know the weight of the conference kind of things all right cool cfp's good to go we'll roll it out just that and sean's like hey thanks jt by the way

looking forward to your talk i'm serious that's how it went down so i spent a few months thinking dude i made that up that is that is that's bull this is bull so so it's a bit of an accident but i want to tell you something i don't necessarily know well i can tell you right now i had no real idea personally like professionally about the implications of trying to wrap my mind around trying to feed a line of bs or to talk and where the truth may be and the fallacy may be and all that so this thing about asymmetry the abstract is kind of funny you know asymmetric warfare and correlation network defense it just sounds like

somebody made it up but um asymmetry literally means that you're attacking a target you're intending to do harm or inflict damage on a victim in a few different ways there's this kinetic way kinetic is a force way you know where you might be shooting them or dropping bombs on them or whatever and then there's the idea of cyber and we could totally belabor that point all day but this is an idea of asymmetry you might have a target you might say hey i'm going to attack one aspect of their critical infrastructure from a cyber standpoint digitally if you will but i'm also going to attack them kinetically and that's what traditional asymmetry means well even though it's an accident i actually

did think are there multiple modes of asymmetric threat from a cyber standpoint that would actually be something you could try to identify and look for as a network defender and i think that there is um i also want to mention a disclaimer if you think i'm up in the night i probably am but if you're interested in talking about things a little more just hit me up because i i think it's kind of cool and i do think there's something to it in fact i'm pretty sure there's something to it and i'm sure i'm not the person who discovered it i can tell you that right now so the proverbial needle in a haystack and i

know it's been a little challenging av wise with this room but there's the needle on the haystack and here is just literally an exercise for me to demonstrate to you sorry dude oh okay for me to demonstrate to you what it is that i'm trying to get out here so if you look this is output from my terminal and i did a couple things real simple um just basic and maps i targeted triple w and org right that's where i work and i did it from two locations one of the locations was just right off the hot spot on my phone and the other location was off of i have a few vpss and this was like my san fran

vps so i just didn't end up that was all i did and it was really easy you can see what came back tcp 80 and tcp 443 i know that the font is small but i promise you that's what it is and then i jumped into a flow collector network flow collection uh box that we run and i started looking at things now there's something that i learned a few months ago and the software guys and uh in the room might understand this a little bit better but there's this concept of the manifolds does anybody know what a manifold is from a software standpoint how about from a security standpoint anybody heard the term manifold

okay so let me i'll just tell you what it what a manifold is a manifold is exactly what that middle part says where it says destination victim i've got a command that's saying nf dump dash big r and then it's got like some file locations and they're actually time stamps starting at the beginning of the day in the middle of the day and then i've got some filters that are saying hey i want you to look at the host which is actually the ip address that resolves the triple w main.org and so there's a ton of traffic that comes back from that i mean you'd imagine right i'm talking it just will scroll and scroll

and scroll and scroll but then i started to build manifold the manifold is moving down to the second line that i looked yeah again at host you know 236.11 as it resolves but then i also added to the manifold this idea of and proto tcp i mean a lot of traffic still in fact based on my nmap scans and what i actually know just on the side these are probably roughly the same you know besides whatever other like icmp type things or little things that people are looking for that get blocked but the manifold keeps uh growing and i go to ad proto tcp import 443. so it's interesting one of the things that i've

learned along the way and i didn't necessarily know that that's what it was called but as you're trying to isolate things from a security standpoint forensically with network flow data you're actually building a manifold of everything that's normal and everything that you expect to see and then it starts to get interesting the things that you would expect they go away as you build your manifold or build your filters and the things that you didn't necessarily expect or start to be odd start to show up as your manifold as your filter starts to close them out yeah yeah it's a filter of normal yes sir that's exactly what it is thank you okay so then the next thing is and this

is where we start to say oh jt is behind the scenes like he actually knows what he's doing because i did this on purpose to show you all what i'm talking about or what i think i'm talking about so moving down source attacker i've got another command it's exactly the same with the others but then i sat there and said hey show me the host which is actually the ip for that san fran vps that i told you about yeah i want to see proto tcp i want to see port 443 and next thing you know total flow count summary total flows four total bytes blah blah all i saw were the four hits that were just part of my initial basic

nmap scan and the time stamps correlate too if you look at it but anyways so there's there's a difference between what's going on at the top with me coming off of my hot spotted connection and what's going on at the top with me coming off of my vps in san francisco and that's the you don't actually know that was me you wouldn't know that it was me that has got an a nature of asymmetry to it right it's coming off of a hot spot at ip location it's called temporal locality it's coming out of a temporal locality in one place in a temporal locality in another but behind those two different temporal localities there's one actor and that's me so that's kind

of the point of what asymmetry is talking about and that's the point where i'm saying if this sounds nuts and it probably is um so yeah i did it on purpose and it's only to illustrate the point that this is an easy way for us to understand what the crap i think this is what i think we're talking about and you might sit there and say there is a jekyll hyde nature to it that we see the guy that's acting you know totally normally coming off of a hot spot or coming out of some area in his country you know and looking at things normally but what we don't see is that there's the same actor with a second side to his

face that's coming out as maliciously and trying to do damage one part of it's friendly and open the other part of his obscure and closed um we're just about there who knows what the internet of things is who wants to help us out with what it is oh man well i didn't plan us to cut to the chase but that's awesome so let me give you what i think it is i think you're right so so the internet of things cool cool gif i stole off the tubes of you know a network connected fridge network connected music whatever whatever that is down there anyways i mean if you just jump back five or ten years

most of us would just sit there and say yeah we've got more devices that are ip connected in our homes than we ever had and we probably expect we're going to have more devices that are i connected in our homes in the future than we have now um think about something too for a second would you and we're still we're going to stay on this slide but like you got to link this router at home that's doing wi-fi you got a tv a tivo that's recording the tv shows that you care about you might have a few other things like the nest controllers on the wall that are taking care of your temperature and things like that these things are ip

connected um so when was the base image or firmware for that device compiled four years ago five years ago last stable right was more than likely not current because they're getting ready to deploy to a production base of tens if not hundreds of thousands maybe even millions of consumers so they're deploying a production base of software that is already behind and when was the last time that you or not to offend anybody in the room but when was the last time that your in-laws or your non-technical uncle or aunt or whomever packs the firmware on their home wi-fi device their router they probably haven't now how many of you in the room remember whether you've read about it or

heard about it or whatever i really don't care when there was talk about highways um you know like that you drive on with your car being built in such a way so that they could also be used strategically to land airplanes in the event of a war you remember i remember so think about this paradigm i'm going to tell you that i think it's still here i don't think that the highways that were being built in such a way architecturally to land airplanes are you know still what's being used they may very well be but i think that some of these things are baked into the cake when it comes to allowing for the use of

commercial civilian critical infrastructures to potentially withstand a asymmetric exchange and that's just me being crazy but um but think about it if that clearly makes any sense that's what i'm proposing to you um how many of you know who dan gear is black cat 2014 luminary guy he gave a talk called uh cyber security's real politic nuts but awesome i would download it i would read it highlight it underline it and understand it because he's going to touch on some of these things i think um i i can't stand well i'm pretty happy about drama and i tend to be dramatic myself but i'm not big on drama that really rolls us up and gets us riled up and i

don't mean to say jt you're seriously telling me that you think the modern day equivalent of us landing airplanes on highways because airstrips have been bombed out is now the internet of things being exploitable because of old code bases that are widely deployed and readily exploitable in case of national need well you know that's kind of what i'm telling you but i don't want to get too nuts about it i'm not saying we're all going to burn and die but it has brought me to this when sean was like hey dude i need somebody to the group to check cfp and see if it works and i just spewed my special brand of bs into

that form and it didn't and i spewed that special brand of bs again with a little more lipstick on it and it worked and the function check was good and then he's like oh by the way i'm looking for your talk it sent me to thinking and i am not sure if that accident or close call that wasn't intended to be a talk has got me thinking it was either a close call or it's too late so the talk's not really about asymmetry because i don't think that matters i don't think asymmetry matters i think if you've got a threat that's coming in from two or more directions that you should mitigate each one of

those threats as though they were an atomic threat not an atomic threat in like an atomic nuclear sense but an atomic threat in the sense that it is a singular threat that you should take care of regardless of what other threats line up against it and may have a bad actor behind the scenes uh orchestrating it against you asymmetrically because that doesn't matter it doesn't matter i don't think it matters um i'd recommend if you're interested in this kind of thing or if you you think that it's interesting there's a gentleman by the name of dave meyer um he was a distinguished engineer for cisco and then brocade poached him and brought him over as their cto

and he talks about this concept called robust yet fragile software side of it i think seth and i might have talked about this before not stuff is that uh the more robustness or richness you deliver in your applications the more fragile things become think about it there's an inverse to that complexity you can't build in complexity and expect it to remain hard it won't a great talk he's probably given that talk more times than he's wanted to or but you can find stuff out there on him robust yet fragile dave meyer dan gear mentioned it got a few nods if you haven't read cyber's real politic i would recommend it and even if this is

just thought type work your g-whiz not to get you too far away from the day-to-day tactical stuff that you do including myself but this is good thought work type stuff i think here's another one how many people remember that book that was called uh yeah man that's the one this is very much akin to that line of thinking but it has been updated and it is kind of the internet of things thought with a current mindset paul henning camp then if you're interested in flow related type stuff you know security as you would look at it from like a traffic or like an internet traffic flow standpoint this is based out of carnegie mellon university

in you know partnership or part of the software engineering institute and the network security association but there are some really killer free tools out there at tools.net.sir.org and if you're like hey man i don't need to do that myself but i would like to read some cool things from an analyst standpoint um there's some awesome documentation along line with it it talks about network security analysis and things like that and so i'd recommend that too um yeah this is kind of you guys know what core business process is right this is kind of the idea is i don't necessarily know that the true concern should be about knowing who the bad actor is that's trying to

attack us asymmetrically or who owns the botnet because seriously it's the same thing it doesn't matter in my mind it doesn't matter um i don't think the core business process um is the trivial many i think it's the critical view we tend to get lost in the weeds um forgetting about the critical few and playing with the trivial many and it just doesn't matter so anyways that's my talk

questions or anything or you guys want to go play seth go ahead i was gonna ask you what have you learned from this experience i'm not gonna test the cfp like webpage again man that's bad news so what kinds of things oh yeah i don't you know what you wouldn't actually be able to detect it um so imagine if you're trying to identify a botnet that had two of thousands of hosts that were attacking me quote unquote and imagine you were actually seeing both of them and somehow correlated them and i don't know how you would necessarily go about doing that um from a traffic standpoint unless you employ other types of tools that cert organization they talk about

actually capturing things at a non-sampled rate for flows which most people don't do or capturing full pcap data and then looking through those things they're trying to correlate them with high level tools but then imagine that you know second bot of the two dropped off was patched somebody mitigate the threat or whatever and next thing you know it's gone i don't really know and that's where antonio seth i've kind of given up on it because i think asymmetries exist but i don't think they're important or i should say i don't think they're as important as addressing atomic threats to the core business process asymmetr asymmetry is a cool concept and it probably exists in fact i'm sure it exists but it's dumb it

doesn't really you know have anything to do with our day-to-day tactical in my opinion i'm willing to be wrong

i'm saying like the the the threat surface is small i i wouldn't i would i would pay more attention to you know good firewall policies vendor agnostic i really don't care what vendors you're on i should i don't um logging um making sure you've got good patch schedules and some change controls i'm talking about the you know meat and potato stuff that we all think is just lame and not flashy i would totally worry about that stuff that's meat and potatoes uh lame and not flashy before i would even start to carry about asymmetry like i seriously just thought it was killer a lot of this information i was seeing and picking up and wondering and putting together

and then i submitted the idea the thought just whimsically on to the cfp as we were testing it out and it was more thought exercise that's all it is seriously i'm not trying to waste anybody's time it's just thought exercise but yeah

yeah like a uh yeah like a smoke screen i agree i i think it's important i don't think it's the kind of thing that we would be this is like tla type stuff that's what i think

is

is

is

here yeah brings us back to the importance of i think the end result of your thought exercise here yeah bring those faculty in terms of