Network Situational Awareness with Flow Data

[Applause] good afternoon everyone um this is a network situational awareness with flow data uh some of you may have seen what may seem like a part one okay this we can't hear you I can hear you all right all right days okay this again so restarting here uh Network situational awareness with flow data uh some of you may have seen a part one type of talk about this and I wouldn't really call it a part one and a part two because previous talk was like really in depth on really what makes a flow uh you know the termination conditions all kinds of really indepth stuff that uh at the end of a day like this I could probably expect to have to

like wake you up at the end so that you'll know to leave because the in-depth stuff kind of gets in the weeds a little bit so instead today we're going to talk about a little a little bit more interesting stuff uh some pretty pictures and whatnot um I am Jason Smith um I am a big car freak uh I like to take things uh maybe junkyard stuff uh just you know bare metal and kind of throw it together um you know love to weld and so like the car you see right here it's a a replica of an old Lotus Super 7 uh that's a car that as of you know last Tuesday I just

got it licensed so I have a license plate now and I can actually drive that on the road and put people at risk um because I'm I'm quite sure that's what it'll do uh I I say I'm an iot Enthusiast on the slide but it's only really a buzzword uh to kind of catch attention because it's really more microcomputing things like that I don't care about having hair dryers uh talk to the internet and stuff like that I'm more like to you know just make gadgets hook them up to cars hook them up to whatever I can hook hook them up to and make them do funny things originally this car was intended to do all kinds of

Nefarious uh activities using like laners for CBS and hack RFS and stuff like that but all of that's super illegal uh especially when you extend it on over the course of a city so I didn't do that uh I'm also a an amateur fry cook I guess you could say big Chick-fil-A fan was pumped about lunch today um and so in the process of going to chickfila about two times a day throughout the week um I decided I would buy me a pressure fryer a big steel crank on the top and you crank that down you cook all kinds of food in it uh pretty much anything can be uh you know dipped in egg uh thrown in some flour

and then cooked at uh 360° or so and and and be better I think um I've I've worked for a few government agencies uh state and federal um a few different DOD groups used to work for the state of Kentucky uh moved over to firey after doing some work with those guys um and have been firey for almost three years I guess since November uh work with the best team ever um I can say that and get bonus points because they're in here I'm I'm always pulling little Hy Jinks so I have to make up for it by calling them the best ever um a Shameless plug for Applied network security monitoring which it's right here someone may win

and there's a copy there and there's probably copies in in most of your pockets sure you all love it but uh one of you will win it today if I don't forget to give it away and uh so the you know the book was written by analyst for analyst and I guess the them say about it because I mean it's been out for a while but 100% of author proceeds go to Charities uh rural technology fund Kaa you know hope for the Warriors autism speak few others Chris Sanders is right there I'd point the laser at him but probably get him in the eyes um but yeah there he is face palming so what are we talking about

today um I'm going to try to go through this more or less quickly because normally uh when I talk to people about flow data it kind of slows down as as they glaze over and I realize I should probably speed up instead we're going to keep it exciting though today so what is flow data we'll talk a little bit about what it actually is just kind of a a basic um summary about it uh why is it really important you know why should you come away from today and say man that was a good talk I should probably set this up in a couple minutes um and then some flow Basics uh you know just kind

of you what is a flow what do we call flows and things like that uh we'll talk about how to collect it um and and sometimes more importantly how not to a lot of people get themselves in situations where they'll set up this nice flow solution they're like yeah I'm really doing it right and then realize like they have to like you know clean it out every day or every 5 days or every 30 days and you know when I think about flow data I think about indefinite collection uh where you never delete it where it's easy as cake to just add on storage and that you know a little bit of storage goes a long ways with it um

we'll use it for situational awareness and some examples talk about some detection and response uses and I'll show you again some tools of the trade so what is not on the agenda and can be found at that link uh from a previous talk in Nashville um I've went in again to in-depth uh flow descriptions uh you know all the little details about you know a lot of the fields you know the difference between version 5 Version 9 Etc won't talk about that today because gets kind of boring uh the data type comparisons you know what what are the difference between flows and J flows Etc um any kind of broad collection methods I'm not going

to go into details about those instead I'm going to go into the colle ction methods I like the most and that I recommend the most there are some that aren't great there are some that are really excellent um I also not go into any analysis methods with silk now you'll see some silk output but it's just as an example uh by comparison of other things hopefully in the end you'll see that it's extremely cheap to set all this up so what is a uh a flow record um so it's easiest to compare it to full peap because everyone knows what full peap is it's you know the endall uh data type that oh you know if I have peap I

have everything and you know I and I would say that's true but usually it's a matter of uh of really parsing it uh Chris Sanders uh gave a talk um about you know the efficacy of data types and he gives some numbers by comparison but you know sometimes it's like to really do ad Justice you have to compare them you know one to one that if you aren't already a really serious expert with pcap you don't know the other tools by which you can churn through it and automate you know actual um parsing of it uh because pcap on it you know on its own won't do you a lot of a lot of good

if you're just saying I'm going to do network security with pcap and wire shark because that's not going to go long ways uh so flow data would be something akin to the uh phone records that you might get from your phone company if you somehow still get mailed phone records um whereas a you know pcap data is like a full recording of the phone call um now across an Enterprise consider you I want to record every phone call for you know 100,000 people uh versus I just want to see the records of what 100,000 people kind of did one you wouldn't be able to do anything with easily one you could do a lot with

especially if you turn out some statistics and had some tools to parse it uh so that's kind of uh the approach that we're going to go with today on really how do you parse that stuff you know how do you parse these huge amounts of uh data when I say huge amounts of data um the comparison might be something to like Mount Everest um versus uh you know a mole hill a mole hill that's about the height of me still not a super tall mole hill but if a mole came out the size of me you would be pretty scary um but literally you know so 29,000 ft is roughly Mount Everest um and uh so you know I would be you know

0.01 to 0.02% of the size of Mount Everest so the storage requirements so this is that little car I built um this is a big bah 75710 peap truck carrying a lot of data a lot of peaps um this little katum which mine actually is a little heavier but the R500 weighs 1,4 PBS it would be kind of a comparison if you stacked six of these trucks and then you compare the the little tiny car there uh so you know even you know picture in your head how Zippy the car is compared to all these trucks and it makes a cooler Point even though it may not actually be accurate performance- wise um when you consider

the actual uh time that you know the dwell time of an adversary you know we talk about average times of you know 140 some odd days in an environment uh it really helps to you know actually have a long uh you know just long period of time that you can actually look back and I don't know many people that have 140 days of pcap data uh especially any you know environment of any kind of decent size um if they do it's some sort of uh zany index stuff that's going to take a lot of effort to actually you know go retrieve and a lot of time to actually you know break that out and look through

whereas with uh pcap you know if you or sorry with netflow uh you can just kind of say oh well I know the time period that I need you know I may have two or three years of flow data let me just look at that time period you know grab some stuff out of that and maybe I can take that and go to some you know metadata some bro data or something like that get more context um and maybe make a a conclusion so flow data itself you know what is it uh as a summary uh well we often call it session data um we call it flow data we call it net flow a lot

that's proprietary term own by Cisco uh they own a lot of different little net flow terms version five version nine there's a lot in between some are skipped basically um but uh the basics are that it's this 52 so you have uh you know Source IP Source Port Des IP desport protocol sounds pretty simple and it really is as simple as that you know there are more fields that are associated you know packet sizes things like that for the entire record U but for the most part when people think of flows they think of just this five tupal uh we we associate the times uh for which the records start and the records end um and uh you know

that's that's essentially it we'll stack them up if termination conditions are met um termination conditions might be you know timed conditions uh they may be configurable size conditions um a lot of times you know an analyst will see you know a series of what looks like some sort of Beacon or whatever and really what it is is you know just an extended communication where a bunch of terminations termination conditions are met um you know I I associate you know this this flow generation the flow building um usually like an assembly line where you have this uh uh this person who these these packets are coming in and and he or she says oh well this has a unique uh you know sip dip

pair of some Sour support desport and protocol I'm going to put this in this little basket here and I'm going to really think about it when other packets come in to see if those are unique or if those are not unique they are unique keep throwing them in this basket until a certain time uh you know passes where either they don't show up or certain time passes where they've continuously showed up and I need to you know the basket's kind of full by my configuration ation and I need to get a new basket so when basket is a flow record uh so this is our our engineer uh and say he or she because this is Silence of

the Lambs and you know Buffalo Bill had a uh you know a thing where he wanted to be you know kind of either one so he puts the back packets in the basket and uh as a new new unique record comes along he's actually handling a lot of Baskets at this point so General in and collecting Network flow data this is the fun part um assuming it goes correctly for you it doesn't it can be an annoying part but usually it's not usually it's pretty easy uh these tools basically when you just kick them on even in their default State they'll start doing something for you uh silk uh eventually gets a little complicated but we'll talk about that so

in general you'll need uh either some sort of router that can export flow data a lot of people say yeah we we have a lot of that and I recommend you to take caution against doing that because a lot of these these devices will send you sampled data and sampled flow data won't do you a lot of good um it's going to be poor for statistics for the most part um it's basically the best thing it will be for but won't be great at that it will definitely be terrible for any kind of instant response so you really want unsampled flow data and if you can generate off some routers you may find it's it's a lot easier to uh you know

take sections of the network and send it from specific spots you know label them with certain sensors and it would be nice but usually that's not the case and you won't have control of that instead usually I recommend setting up sensors that have software um software flow Solutions uh like f probe or yaf F probe being a version five flow generator and yaf being a IP fix uh flow generator again there are multiple types of flows you know ipix J flow s flow um you know net flow a bunch of those U but we're not going to talk about those because basically a lot of those are vendor specific the flows we'll talk about today are IP

fix and net flow version 5 so how do you collect it specifically well you can uh collect net flow data uh using you know any kind of sender which you'll send it but you actually want to you know take it in and actually parse it and do all kinds of things I like Silk the most that's what I kind of I guess grew up using uh Argus is great in its own ways and NF dump is awesome you know a lot of people use NF dump and NF sin which is the the visualization aspect of NF dump but we'll talk about silk and other tools today um there's also some general log aggregation Solutions um you I don't always

recommend those and I'll talk about why in a minute um but you know a lot of people will use uh um log stash and Cabana and whatnot um and there are a lot of tutorials out there probably more than anything about using elk uh to handle your floats and it's it may be great great in very small environments um but you'll see that some storage issues occur some speed issues occur um there are some there are some other issues um despite all that you know I guess Argus is is already installed in like security Union and who who went to security Union conference yesterday okay so you know quite a few right there um so a lot of people may or

may not use Argus I'm not sure because you know I've heard many times that you know there's you know the thoughts about you know maybe that maybe broon is good enough and you know maybe people aren't actually turning on Argus and things like that um but hopefully I'll convince you that a a Pure Flow solution is something you should really have um and I think the next numbers will show you why who actually uses a flow solution right now awesome so far the best numbers yet 90% uh to anyone watching the video that can't see the audience uh so the uh flow data tool comparison so we'll talk about raw Solutions like Silk um compared to uh you know generic

log Solutions like uh elk uh so basically uh silk has a you know a compressed binary format that it uses across this flat structure um essentially what you have is as as new flows come in they kind of just get uh thrown into uh just some sort of directory structure by you know time and date um and essentially when when you call a query it's calling out that directory structure as you call that query uh so once you actually get to using it you kind of see the correlation between how it's even set up uh you know visually you know when you just LS through that directory and when you're actually calling uh those queries um

silk can can be used pretty quickly to examine any data from any time and I put an asterisk on that because you don't really want to say oh I want all the data for the last year at once CU what do you going to do with that in the first place not a lot um and uh I mean even for for trying to narrow it down you won't do much with it um so you I say you know stay to a week's time if you can uh you can draw a lot of Statistics in that week it is not riing on indexing uh that can be good or bad it's not relying on archiving to say

that's good or bad um because you know a lot of the speed that you get from uh tools like uh uh elk uh come from the indexing um the uh the cons would be that's flows only so you know a lot of people are like well you know you say I should have a flow solution but I have all these other data types and I would say that's that may be a case where you want to consider a lot of storage um and actually using using a solution like a generic log parsing solution or something like that uh to handle your flows you're probably going to pay a whole lot though I mean if you can

you're better off pivoting from a real flow solution to that other log collector um it is also not very extendable uh in its uh basic easy setup specifically silk so silk we've made it pretty easy to set up but if you want to go crazy with you know you know using uh python with it and using all these uh you know trying to make apis or whatever for for tools like chive um you're probably going to have a hard time um without a little bit of extra work but it's not impossible I mean if you're developing your own tools I mean you may find that it's it's actually quite easy it's also uh not all solutions are

create equal again I mentioned three just now they're going to be different for everyone I use silk all the time CU I find it easiest I don't think the other ones are as easy to use they're learning Cur is a little more difficult but that's my opinion uh your log Solutions obviously there's several front ends you can use we talked about uh you know Cabana right now and it's h it's pretty awesome you can make all kinds of pretty graphs you can pretty much do whatever you want to with it uh it's pretty extendable uh tools like Splunk are like working with Clay you can pretty much do anything you want to with it and that's pretty awesome uh but

the downside again is storage uh reliability things like that um in the end you know Splunk might cost you a whole lot when you're sending tons of flows to it um there are some tools though that have flow specific um pricing models uh so even some uh uh Security Solutions like um I guess Q radar uses a different kind of flow solution that they uh they want to charge differently for and it's basically all because a lot lot of complaints happen that say oh we have all this flow data but you're going to charge me you know an arm and a leg for this stuff and there's no way anyone can do that and it's true I don't know many

many companies that are able to do that and they usually end up doing this internally instead so what is the big difference um this is a you know a pretty junky chart but you know you see this little strip of red up there and that's that's silk compared to elastic search on storage retention uh the difference being that elastic search uses 28 times more storage uh than silk for the same amount of data uh you know I have the the asteris down here that says uh you know all this was created simultaneous generation on a pretty beefy box um and all the collection was from both Solutions and both are compressed in this case uh still 28 times difference

I'm not you know an expert with elastic search uh like some people are but I'd say they would have a hard time you know whittling that down to anything that's reasonable uh screenshot there of uh you know just kind of some of the the output also that was 4 million flow records in case you're curious back one bit 4 million unique flow records so how do you get started with flows um again you know the actual number of raised hands for people watching uh was you know maybe 15% in here which is which is pretty good um and maybe had some shy people so like 20% being optimistic um but how do you how do you actually do all this stuff so

again we'll talk about silk um it was uh made by the guys at Carnegie melon um it's an awesome tool excellent uh documentation all kinds of great things you'll use yaf to actually generate the flows yaf is different than silk but it's also made by the same guys um and basically we will you know kind of show how to query those uh with different filtering and statistical tools we'll talk about flow bat uh it's the front end that Chris Sanders and I developed for silk um flow bat's pretty awesome and then it adds a lot of features that you can't do in silk uh namely quickly making a query you know that's the first thing uh that's kind of a frustration

when you're at the command line you're like oh well let me add this thing into the middle of this partitioning switch or something like that and you go to that and you know you you try to type it in maybe you mess it up because you don't really know what that switch is because there are tons of them uh well Flowback does a lot of things like autocomplete um and actually allows you to use the mouse with it so um it also removes a lot of the learning curve for junior analysts as well as teaching you how real silk queries are to get started uh it's pretty fast um we just have two little scripts which

I'll I'll kind of demonstrate uh here in a few minutes uh you know in the end it takes about 13 to 15 minutes to install both silk and flow bat uh and have them up and running to where you're running queries and doing whatever you want to uh so that's pretty quick uh you know and it only requires you know maybe 10 seconds of total user time then it does do the rest of the installing your so silk uh again it's made by the guys at the CT Network situation awareness team it's two Suites it's a packing suite and it's an analysis Suite the analysis Suite can do a lot of Unix type things so I recommend when you're

using any of these tools that I'll mention here you know in the next few minutes that you don't try to pipe a bunch of raw output to a bunch of Linux type things because then you're just going back to this raw uh you know these strings and whatnot you're not using the binary format that it's optimized for and uh you know if you want to look at the documentation again there it is so the actual collection architecture for generating flows here we have a netflow version 5 uh F probe sensor uh you may have a router that hopefully is doing unsampled flow generation and you have this yaf sensor that's doing this IP fixed stuffff it's sending all these flows to

this security server um be it you know whatever you've built maybe it's security onion or whatever that has uh the silk packing Suite on it specifically a tool called RW flow pack that's all installed with this package uh RW flow pack says I'm going to store all this in a way that you could probably read it and the analyst will either SSH to that and run some commands directly or Flowback will be installed on that security server and then the analyst can just go there in the browser to get started as a review we have flow source which is some router or some switch or some sort of software generation like the app or fpr again

installed with silk on a box uh we have a silk server it's some sort of security server that uh you're sending all your data to then we have the analyst workstation uh the workstation will probably just browse directly to the uh that server and access flow bat uh flow bat will again be installed on on the silk server a lot of times we install silk uh the analysis packages as well on that Silk server um because it helps a lot to kind of speed up the process for installing flow bat and for the analyst to actually say oh well now I'm used to flow bat I should try to use silk as well because they kind of Step It Up

from being just the routine analyst looking at Old flows to saying I can automate all this stuff um the actual silk installed just run these three commands I assure you you won't get any m we just run them without question um when you run them you can go get the mail or or go get a drink or use the restroom or whatever that takes 5 minutes um if you have a non-typical network layout or something like that you'll probably have to uh modify silk.com or sensors. comom but for the purpose of testing at your house or you know setting up on basic sensors or anything like that um you should be just fine running these uh all of it takes

air dependencies so again just run these and afterwards you should be able to run some queries and so here is kind of an example if you have epilepsy close your eyes that's a very serious warning well that didn't work so here I am running some stuff silk onab box. sh and I play with cat little bit I'm looking at the install for flow bat I get my shoes on I'll go get the mail uh they play with the cat some that's a Bingle cat named Cheeto after the cat from Cheetos the food well so we're done now so that was H that was like five minutes um maybe a little less that was on a a clean uh updated the btu box um

1404 specifically so if you make an issue about newer vuntu you can join the crowd on GitHub um as far as the uh RW filter outputs and and kind of what they look like and different things like this I won't go into this too much but basically you have some basic stuff and say let's look at the beginning of the current day and the current time uh or you might say you know a little more in depth and and I've I've placed these on multiple lines just so that it's a little easier to read and doesn't look too too garbagey um but basically we're looking at like where any IP is is from China based on you

know whatever Max mind or whatever you're using for your your country code map and we only basically send out uh we want the output to be just the five tupal and so what you kind of get is something like what you see at the bottom here this bottom uh output is actually from a pcap conversion which is several lines I mean you have to convert the pcap uh you know pull it down convert it um do what you need to do and then throw it into RW filter which it's kind of a pain um but you'll see that we kind of optimize this a little bit in that case it was like some SD bot

infected device um so an example of rdb stats you know here we're running like a a top 10 um bites transferred by protocol um and then a top 10 sip dip pairs uh combination and the outputs being things like this so RW cut is just your raw flows uh you see you know the the flags that they were involve the btes uh or the btes in combination the records uh things like that and as well as the 52 RW stats again is the statistics uh tool for silk it basically does all these calculations on every flow record that's been in that whole big thing and it says oh how many records were there you know per field

that you specified you know in this case I specified an application field uh which is kind of unique toh to Silk and yaf that yaf is looking at the uh the actual packet payloads and determining you know oh is this DNS traffic or is it HTTP traffic or what is it uh so we you know defined application here and those are how many records for a certain time that we had per application RW count takes bins of time uh in this case it looks like we were doing 30 second bins and it says how many records were in each one of those bins how many bites how many packets uh so in this first bin

in the first 30 seconds we had 1380 bytes uh for that bin you'll notice the 0 27 it's because sometimes these bins are split up you know maybe the entire communication was in 42 seconds well in the first 30 seconds it's going to split that up as much as it can so flow bat uh what is flow bat well it's a front end for silk flow basic analysis tool you can go to flat.com learn a lot about it we have a lot of documentation Now by a lot I mean just a little bit but it's pretty uh pretty understandable and you can kind of go through examples on flat.com uh and then you know again it's super easy to set up

uh but you know if you want a demo sometime you can you know email me or what not I can show you a demo if you don't have 15 minutes to set it up on your own uh so how do you get started with flow bat uh you know I'll kind of run through the same kind of procedure as silk um it's just as easy it's just a script basically um and then if you have some issues it's probably because you're you're not actually reaching out to uh uh that server that it sets up so we just run these and this is going to take a long time by comparison so like almost double the time so you'll

probably like need to get a drink uh or like go fix you know prepare some food or whatever so again epilepsy warning Close Your Eyes in this case I'm running the you know flow bat installer it asks me a couple questions it needs uh privileges not to install anything bad in this case I'm installing oh this is terrible I'm installing uh or I'm I'm cooking some some chicken so I'm a big fan again of frying food um this is uh kind of a replica of of Chick-fil-A chicken but I think it's a little better for their their nuggets and uh if you if you have some pretty quality chicken breasts it's actually quite good uh for a chicken

sandwich as well I would say it's better maybe it's just fresher in the end I don't know or maybe I put too much sugar in it or something like that but it's pretty good um so basically what you have here is H um you know a a regular non-spicy chicken mix and well let me check it's done so I only got halfway done with my chicken brine um and that took you know 7 Minutes uh chicken brine takes a little longer than 7 minutes I'll show you the recipe for that at the end it's like a it's like the third giveaway that everyone gets though so flow bat Basics um this is kind of the meat of the conversation not

that it's going to take another hour but uh kind of the takeaway um so what you see here is that I'm looking at all records for the current day you know again my little animation's already kind of done U by comparison you know before you had to kind of think about what you're doing specify you know more than just partitioning switches you had to specify input switches and output switches specify RW cut and all these other things that I know because I love silk but most of you probably don't know just right out of the box you may not know about this protocol switch but if you look into our help you'll see you know

the most basic examples you'll see some of the things I'm showing you today um but this is essentially silk you know it has auto complete uh you'll see a lot of stuff if you just put dash dash on there um but it can handle a lot of different things this this is pulling all records for that current date um here's another example again we I showed you the second example for silk which was saying you know any local cider and then any uh Chinese IP address be it Source IP or desk IP um here I'm showing off the uh the query Builder which is usually probably what people go to first thing because they don't know all of these

commands um for the uh I guess the the Zen mode at the quick query um but over time you see that it's building a query at the very bottom in red um here I I say NECC on the additional parameters because at the time we were missing an NECC field but it built a query as well so you can kind of learn what silk is actually interpreting um and and saying oh I know what I need to do so maybe you'll know what to do next time if you decide to like automate this or something uh this um again I I specified a week on this one but but it came back pretty quickly um and you can uh you

know you know go through the pages and whatnot or you can reduce these uh um different uh reduce the table down to whatever you want to see in this case I'm going to go down to just the five tupal um and usually I like to see starting time at the beginning um and then if you wanted to you can also sort by starting time and do things like that so here we're using stats and again I'm going to use the query Builder even though you can do this through uh just using a quick query at the little Zen mode command line there um but I'm just going to do one day um I'm using RW

stats we're going to look at the top 10 uh Source IP desk IP pairs by bytes so it's going to count up all the bytes and it's going to say oh how many work for each one of these and so there you have it that went really quickly at the end I apologize but uh here we'll look at a graph an outbound data graph so you know when you want to specify if it's inbound or outbound that may depend on you know what you've set up for your home Nets um or your IP blocks and you know how they're range in sensor.com silk.com but out of the box they should do uh uh you know any kind of local IP addresses uh

um just you know they're already set up in there um so I specified types being you know out and out web and uh any kind of outbound icmp and I get this graph it doesn't really look really great it's because it's looking at bins again we talked about bends of data um that was looking at 60-second bins I want look at 10minute bins so it's stacking everything up so it should split this up a little better and something like this may give you a good idea of situational awareness you know maybe for that total day or you can have that like a rolling time um so in this case I saved it as you know some 24-hour outbound data uh

and and we'll put this into a dashboard that I don't see Martin holti in here he loves dashboards so here we are throwing this into the uh the dashboard and it's already kind of done um so it saves all these uh results and allows you to you know per those per query those periodically uh so in this case I've said oh every hour I want you to update this so that I see something new but you could tell you know every five minutes show me something new or whatever you want to do just keep in mind it's running a query so if your query is kind of dumb and takes uh way too long if you're like yeah update this

monthlong query every 5 minutes then it's going to do a monthlong query in silk every 5 minutes so just be cognizant of that and you can take out packets or records or bites or do whatever you want to there so here we're looking at nonstandard ports uh now you know how do you kind of tell what non-standard ports are um it goes back to app labeling uh so yaf said hey you know these look like HTTP but maybe it's not on Port 80 well if it's that then then go ahead and tell me that so here I've added this additional parameter um it's a plugin called appmatch doso you can make your own plugins but they're moderately

difficult to do uh and I also said that you know I only want packets that are kind of confirmed communication so it needs to be like a return thing so uh here you have uh where I've done a account of all of these are a top 10 of well now I've lost it but you have application and destination Port down here so what that showed was that there were five distinct uh destination ports that were used that were not um you know typical like 443 so in the next example you'll actually see some of those uh that were that were non-typical according to uh to you have so there you again you have another like one non Port

80 HTTP Port things like that so identifying Services uh you know I talked about you know app labeling and it's pretty awesome you can just use ports you can just do use whatever you want to um but there there's a kind of less false positive prone way um and so here we just use app labeling only uh we basically make sure that yaf uh yaf itself is doing all this and I won't go into too much detail because it's a just a big long silk query is the the bulk of this but at the end I said hey show me Source ports and applications and you'll see that uh um for instance like 902 is

is is actually just uh that was VMware um but a lot of these you know 1800 which is actually flow bat itself and it says hey that looks like some HTTP stuff so yaf is able to see those things um because it just does a little bit of of analysis at the packet level um just at some headers so if you want to analyze peap files you know I mentioned you know it's like four or five commands of you know pulling this peap down doing whatever you need to do with silk um but it was a lot of commands to remember M uh in this case if you just have a pcap that's setting on that flow bat server um be it

you know in some peap repository or whatever as long as you have permissions to do this uh you can just specify the pcap and you know tell it whatever protocols it wants to look at um kind of look at whatever you want to and it'll do all the conversions you know on the flow bad end and it'll do everything it needs to do and then it'll also output everything for you so you know if you had some sort of uh um you know peap for like you know this an ex some exploit kit traffic uh you could easily turn that into uh you know some flow data um without actually having any kind of overheads so this will be kind of

temporary um but it'll also store that output rwf file uh into a temporary file so some flow automation uh so basically we looked at flow bat which is is not really an automated thing so you know a lot of people ask about apis and we're like well flow bat's just a front end for silk uh so wers run on silk like well we still want an API so I guess they just want like a proxy essentially you because you can just do everything through silk if you want to um a lot of these commands again that's why we include those commands in flow bat you can run any one of those and get the same outputs in silk if command line you

obviously won't be able to drag and drop anything or do any kind of sorting easily um but you'll get the basic results from that and over time you may learn from that you may say oh well well maybe I want to pull some sort of uh you know list from the internet or something like that and I want to store it as a a set and I want flat to have a dashboard that references where any Source or Des IP is from the set of bad guys um so your script on one end is pulling everything from the internet making the set list Flowback you've told hey the set list is located here show me all

results and you've saved that and put that in a dashboard um once you get to doing that it's pretty fluid uh the only downside again is you have to know how to script that out and that takes a little bit of silk knowledge first uh you can you can also expand uh all of that deployment into the analysis pipeline it's another tool by the guys at c um it's it's pretty awesome but it's not as easy to set up uh or at least we haven't made it as easy to set up yet you know maybe that should be a focus um but we don't get a lot of requests for it even though it's pretty

awesome it'll do its own uh watch list reporting uh it'll do all kinds of like tunnel tunneling detection uh things like that so like if it sees like um you know any kind of DNS requests that are like abnormally large or like they're occurring too quickly or something like that or like it sees the icmp occurring too quickly from a host or you know things that just don't make sense it'll make an alert for those things and you can send it to uh you know any kind of log solution that you have um and make some alerts out of that um if you want to know more about it there's the link uh there's also tools like flow plotter

which I'll talk about in just a second not in huge detail just because you know that was kind of a part one talk uh flow plotter is being being augmented now to to remove all Google visualizations which you'll see in a minute so what is flow plotter it's just uh when you're running an RW filter command or any silk command uh you can basically pipe all that output to flow plotter and make kind of I not say any plot that you want doesn't do sanky diagrams yet Martin again but uh again it's it's just a this a little bash tool that it's pretty reliable uh and it's open source you can see what it's doing so it's not doing

anything weird um but again it's it requires Google visualizations and Google visualizations uh they require that you don't actually download all of their libraries but they need to be sourced remotely it's part of their user agreement uh so here's a just kind of a splash of some graphs that are available in flow plotter uh there you at the bottom right you you have like a jiggly Force directed graph um Martin had a visualization talk um last uh I guess two or three talks ago he said that uh animations that aren't needed shouldn't be there um but I like the jiggle that occurs with the the nodes and whatnot uh so it's there it's it's not incredibly

useful you can't just like throw something out there and be like man yeah see I see you the bad guy because he's connected to all these other guys here it's never going to going to do that for you instead what you want to use it for is you can see some like color coding that that I'm doing there you want to use it to show another colleague or manager a network operations Guy this is kind of what I'm looking at because if you just show them a table of data they may not get that if you just show them this this blob of stuff they won't understand what they're looking at but here at least you can convince them that

you're right even if you're wrong what you see in the middle is kind of a different tool alog together it's just an asset modeling tool so basically you can't really see what it's doing um but basically it says assets at the far left and then it breaks out into uh web servers and mail servers and all kinds of other servers and it'll show you everything that's on your network that it can actually detect and say this looks like this should be you know a mail server and maybe you'll have duplicates in there because you know maybe you know some guy's running a security server that's hosting a lot of stuff and so maybe it'll show up in

multiple different trees um but it's pretty cool and that you don't it doesn't require any options it just says hey you've thrown me all this this flow data and now I'm going to throw you a lot of assets here and some may be wrong or you may have too many but it's unlikely they'll be very wrong it's possibly likely depend on your environment they'll be too large um and in that case I did I do have a thresholding involved um the bottom left you see a bubble chart which is kind of a three variable chart um that allows you to see packets versus uh bytes versus records um that alone isn't very useful but you can do things like uh

distinct desk ports for a host uh compared to distinct desk uh IPS compared to whatever um kind of whatever you can think of U which is the hard part because in anything you do with flow data you have this answer that you think oh man I'm going to get this cool answer and I'm going to like really like rule the day but the question is really hard like you have to be able to word the question in a sentence uh so if you can't like just say to yourself in you know just English say oh what am I looking for you know I want you know hosts that have reached out to four or more distinct destination country coats

and that's a sentence but if you're just like I just want to you churn through some data and see the bad guys you're not going to find that and I'm not going to use any kind of h u hype words like hunting um but uh you know that kind of stuff requires prerequisite knowledge on you know how to approach it you can't just find the bad guys you know point to it the bad guys are in there um the rest of your stuff you have you know your typical line chart and that's based on like the bins of you know packets or bites you know over time and you have bar charts and whatnot uh that they work

pretty pretty well if you throw them into a dashboard or something like that as an ey frame um but you can just uh if you want to use like you know Python's really simple uh HTTP server to just host these and you know people can look at them and make your own dashboard uh another summary of silk um and almost done here uh it's extremely fast it's really easy to manipulate stuff if you're really good at scripting um again it's a lot of data but they offer tools to whittle down the data as well um it's got awesome documentation uh and again more tools are coming out to to actually analyze this stuff um I'm

updating flow plotter to get rid of all the you know very happy colors and make them more um you know web what 3 four 5.0 whatever we're at right now um to make them look more hip um but it also again I'm pretty pumped about the different options you have I'm basically freeing it up to where you're not limited to bites packets and Records but you know anything that you want any kind of like distinct counts or anything um it makes it a little more challenging to figure out what your input is but your output is all that more valuable uh flow bat again is a graphical front end to Silk uh it can install super easy not going to give you

a seizure again uh to show you how easy but uh you can rapidly pivot from data I didn't show all the examples of you know you can click a time for instance say I want the previous five minutes of records and it's going to put it you know this huge long query out there that you know you would be punching in numbers just to figure that out on a a calendar whatever but it'll just you know decid all that for you it has some visualizations and whatnot dashboards whatever you want and again it's by analyst for analyst and so I use it um pretty much all the time uh Chris uses it a lot of people are using it now and

so when people say we want a certain feature in it we take it seriously uh not seriously enough to probably work on it immediately or stop eating to to fix something but we take it pretty seriously actually to fix bugs we we jump on those pretty quick so you have flat.com uh you have the source if you want to go look at that and in conclusion you should probably be using flow data it's super easy to uh to set up if you ever really you know dispersed environment maybe it's difficult if you don't have any security posture at all like you don't have a tap or any sensor or anything like that you may find it's

pretty difficult to set up your own uh your own flow generation just because uh I mean you probably should have those things I doubt you have alerting or anything if you don't have those um it's really easy to collect stuff from it uh makes analysis much easier um for you know pivoting from large data sets to the pcap you want and it's minimal barri barriers to entry um here is your first giveaway which everyone wins uh it is the recipes for that chicken which I swear to you is the best non-spicy chicken I've ever had um I assume this is recorded um and then I appreciate it very much um there's my contact uh if you want to go buy applied

network security monitoring you can get it at Amazon and and basically feel good that you're donating to charity um also when you buy it at Amazon go to smile.amazon.com because that is a a percentage of every purchase you make goes towards a charity of your choice it can be rural technology fund which would be awesome but it can be whatever charity you want it to be uh not a lot of people are aware of it but it costs nothing to you and uh it doesn't change your Amazon you know browsing or anything like that it's just free money for Charities free at Amazon's expense so that's it any questions

yes any other questions um when you use a Pryer is it it sounded like it a pressure fryer yeah it's solid it's very solid it's steing you use a regular fryer it will if you use like a a pressure cooker it will explode like like it's a lot of terrible videos and a regular fryer is okay and this will work in a regular fryer yeah yeah thank you it'll take longer is there a difficulty if you have to use like gluten free FL gluten- free better it makes crispier breading okay yeah or you can mix Vodka with your marinade and it'll like reduce the gluten for regular flour okay yeah thank you

so broon logs for one like the con logs themselves and they're actually quite different as far as we're looking at unidirectional flows when I use like ipix or fpr um other tools like Argus it's another flow tool that looks at bidirectional stuff broon logs are bidirectional like broon logs are pretty awesome in their own I never recommend someone to just get rid of them but like I guess when it comes down to like if it was one or the other I always recommend just because again the storage uh that that that broon will require for the same amount of data um now granted broon actually requires Less storage for what I've seen at least than just churning a

bunch of flows into elastic search or something like that and part of it's because bro actually you know there's a little more data to it it's people will say oh bro is like flow what it's like faux flow it's bro fux flow it is uh that's tongue twister um but broons are are awesome I'm I'm definitely not bashing brons like if you already have like a bunch of bro logs and they're going to elastic search or something like that then maybe you'll say well yeah I already have bro you know broon data and maybe it's already good enough but ask yourself how long how far back you can store those bro those broon LS and you know if you can't store more

than like 30 days either maybe store more of them or consider a flow solution because flows take up you know no space At All by comparison of you know any kind of raw text like that CU again like broon there's no like special format they're just strings right so it's stored in in a pretty substantially larger format um if you compress them you have to deal with the compression things like that um but again you know if you throw into elastic search all that is compressed you know on the way so good question anybody else yeah this is I'm just wondering you know this is spicy you be releasing as soon as I get it fixed up I

will um I work on it I'm pretty diligent about it but it's not perfect yet and I don't feel confident to go to a conference with it yet but maybe it'll be its own talk I don't know I'll trick you into listening to only that do you have uh yeah yeah it's you won't be able to buy it there it's this pressure magic uh um pressure fryer and it's a big crank on the topic still crank so it doesn't explode use a lot of oil have more questions or any more questions Put negative connotation on that okay well I appreciate oh no I have giveaways you're supposed to remind me you did not you all

lose yeah right so we have the this dualcom tap um who knows the two most common flow types that is it net flow version five and N you have to fight for this whichever who can race to get it it's up to you everyone clap yes yes congratulations okay applied network security monitoring mediocre book by two mediocre gentlemen um let's see right I've been caught a Ruffian no less than three times a day in talks uh so yeah thank you um I guess name three raw flow Solutions Ryan Sans those are three flow types that's good enough because no one wait a minute what was the last one NF dump that is true sorry Ryan you

win it is not autographed okay well I guess that's it I didn't mean to keep you too long so have a good uh evening