Quick Picks from Bro/Zeek logs

So I have mentioned Zeek because they have been calling Grow for the past 20 years and they have been dealing with the renaming of the project because it was giving a wrong connotation because whenever I would say, my colleagues said, oh, I'm going to broke on this. And they're like, wait, which conference you're going on? Broke on? Like, is it? "No, it's not a conference with Bro, it's the Bro Idea System and I work on that and that's how I'm going to Bro Conference." So they have been planning to rename it and they actually announced the new name of Bro NSM this year, BroCon 2018 in October. This is November, yeah. In October and they renamed it to Zeek. So I might be

like talking back and forth between Zeek but whenever I say Bro or Zeek, they both mean the same. Jumping to the top. Since it's a FHIR talk, I will be really quick and if you guys have any questions, just raise your hand and I will try to answer it in my talk. So what is Zeek? Zeek basically is a network security monitoring tool that is actually, I define it as a passive scanner. So if you have it on your network, what it does is it just passes the business to your network, whatever traffic is going on on the wire, It will capture the traffic, it will try to analyze it, parse it, and it will produce logs in flat ASCII format for you. And they're human digestible. They're flat

files, and you can just go through the logs and see what's going on in your network. The good thing about BZ is-- Apart from the normal PCAP capture solutions we have, a PCAP consists of everything. Each and every packet you have seen on your network will be dumped into a PCAP file. It becomes hard to analyze it when it grows in size, right? And you have to be very thorough in the wire chart filters or DCP dump filters to actually pick the packets that you we want to see. What Zeek does is it does a pretty neat job of analyzing the traffic, analyzing the application layer protocol, what is in the traffic, and then it

logs that protocol in the particular log file. So for example, if I'm getting a lot of HTTP traffic in my network, what Zeek will do is it will produce a file called HTTP.log and log everything, each and every packet that it sees is an HTTP packet to that log file. And it's like flat-as-P file. So it's pretty easy to digest. And if you have some centralizing tool where you are monitoring or educating your data, it's a really good place to educate the logs and search through it. So that's what it is. It just nests the traffic and it produces a whole bunch of-- Currently, it has more than 50 types of protocol parsers. And it

has been in development. And whenever there's a new protocol coming up, they start writing the protocol. And whenever there are some protocols that are very new, like industrial protocols, which Bro doesn't support yet, it will log it as an unknown protocol. So if you want to implement something, if you want to see what exactly is logged as an unknown protocol, there is a file called weird.log that Bro generates apart from the non-normal conventional protocol files that you can actually see through the weird.log file and see what kind of protocol is not currently filed in Bro. A quick overview of what Zeek does. deployment. So how you can deploy Zeek in the network right now? Zeek has the capability of running into a clustered environment, so it's

not like a stand alone, but if you have a very big organization where you have a lot of servers and a lot of clients and you want to load balance the traffic, you can run Bro in a clustered environment as well. So at UD, what we have is we have 10Gbps uplinks and downlinks. We are monitoring just the north-south traffic. We are not doing east-west anything with Bro. We have four physical boxes and each box is running a bro instance. Not a bro instance, each box is running almost 36 bro instances and all of them are working as the worker mode. So they all are the workers. So we have like four bro sensors. They

all are workers and then there's a manager which is another physical box and all the logging is done on the manager. There's another instance called Logger. So when you run in the cluster environment, it has the rules defined for your physical box, whether you want that box to be the worker or you want that box to be the manager that is aggregating all the logs from all the 4 Pro sensors. And we are basically monitoring our 10 GPS uplinks and downlinks. Basically all the north-south bound traffic, like we are doing the internet to the network, internet to the UDN, internet to the internet traffic. We are not actually wanting anything internal, but we would really

like to because Grow is so good at detecting some things and picking up some things automatically. You do not have to be like Grow, Grow. to deploy Bro, you just have to deploy it and monitor the files. So it's pretty neat. And we will be talking about some of the quick detections that Bro does, which are really neat and they are right off the bat. You do not have to do any kind of customization to get that kind of traffic. So why Z? First of all, it's a great open source, free tool. And universities like free tools because we do not get a lot of budget to install very commercialized tools for our environment. So

we have Grow as just because it's free and open source and people like it and we do not have a full packet capture solution. So it's really great. So if you do not want to have a full packet capture solution because of the size of your network, because of the storage requirements, because of the infrastructure and maintenance requirement, it's really decent to have Bro because it locks everything, pretty much everything by default and if you want some fields which are not getting logged currently, you can actually get that. So Bro is a full protocol parser. So even if Bro logs pretty, Bro logs very few fields, but if you really want some fields extracted out and get logged, you can actually do that as well. So, for example,

we were having, we recently deployed a TLS fingerprinting where we we really wanted to fingerprint the DLS clients that were all connected to our DLS servers. So all the fields were getting extracted. But the SSL.log file that Bro generates, it has very specific fields that Bro thinks that is valuable for a user or valuable for an analyzer to look at. But when we were implementing DLS with the fields like the hash mapping, like how the, if the client is advertising that these are my algorithms that I support, what does the respond back? So those were the intricate fields that were extracted by Bro, but were not logged, and we wanted those fields. So we knew

that they are being extracted, we just customized code to actually use that. So it's really good, and if you do not know what exactly fields you are looking for, you can ask the pro community that, for this DNS protocol parser, do you extract these fields? And they say, yes, we extract it, but we do not log it. And they will give you the right pointers how you can go ahead and get those fields from the traffic. So coming back, YC. So it's dynamic. It's full-path capture solution. It has really strong scripting and logging framework. So I have already mentioned the logging that it has more than 50 different protocol parsers and it can log each

and every protocol in its own file. Like DNS.log will have all the DNS traffic logs. HTTP.log will have all the HTTP traffic. They recently put support for Windows logging as well, so they now log RPC, DC RPC logs, they now log NPL logs, et cetera. And apart from the learning framework, they really have a really good scripting framework. What does that mean is they have their own scripting language. They call it Grow Programming. And it's pretty easy to learn because there are so many scripts out there that you don't even have to write a script by yourself. Somebody might have already written it and you just have to pull it down and install it and

run it on your cluster. So yesterday there was a talk about the Bro Primer, which actually is not, which was not a talk, which was a class that actually described how you can get your feet wet in Bro, like how you can install it, how you can run it in the bare minimum, how you can configure it to run on the tap or sniffing board. So if you guys were there, you know how to run Bro, but if you missed the chance, there are a lot of tutorials out there that can actually show you how you can actually install Bro and get that working in your environment. and I will talk more about the scripting framework later on. But moving ahead, I have Blue running now. So

a lot of times what people think that it's an IDS like Snort, that for example, if I have Snort running, it will produce the alerts and then I will get alerts saying, oh, somebody was trying to hack my network and I will go ahead and look. Bro is not actually designed to be an IDS. It is designed to be a network security monitoring tool, basically. So if you want to look something, you have to search, you have to put it in your log. So it's not like-- so a lot of people have misconceptions that I had Bro running and it didn't didn't produce any kind of alerts for me and I didn't know what it's

doing so I just kept it running. But it's not really like that. Grow and Snora are two different moves to target different problems. Previously they had Bro has signature framework as well. So if you are very pro-smore, and if you like to write signatures, and if you like to detect things in the network, you can absolutely do that. You can absolutely do the exact same thing in Bro as well, because they have strong scripting and strong signature framework as well. But it's really meant to be the monitoring meant to be a monitor of the network. And I will talk about some of the quick things that it automatically detects. You do not even have to

turn some plugs on or turn some switches on to have those kind of detections. So some of the quick things that it automatically by default comes with, like it comes with the scripts that detects those kind of detects automatically for you. So for example, some of them are like scanners Bloq comes with a scan script that automatically detects different kinds of attacks that are in the following slides and how you can actually go through the logs and search for those kinds of activities on your network. So we actively detect scanners each day on our network and we almost block tens of thousands of IP addresses who are just scanning our network. Because scanning is the first part of the refund, right? Like when a pen tester wants to know

about your network, the first thing it has to do is to scan your network. It has to collect all the information on how the IP address of your organization played out, how many servers you have, how many HTTP servers you have, how many DNS servers you have, and how he can actually, how open your network is so that he can attack or exploit the And that process is very noisy. It cannot be very subtle in that process. So if you can detect scanning, you have like blocked their first step of doing or pivoting in your network. So Grow really does a neat job in detecting scanners. There is a scanNG package that is in the

bracket. That is a customized package that is written by one of the people from Lawrence Berkeley National Lab. What the package does is it not only detects the normal scanning attempts, but it detects some of the highly innovative scans. Like, the scanner is not really in math, but if it is doing a subtle scan of each and every server, you might not be able to pick it up in your network traffic if you're going through the traffic. But that scan and package, and this stands for next generation, so you have four different kinds of scanning detection automatically built in in Pro. So it detects knock knock scan, it detects landmine scan, it detects low code

throttling. Those were the scan types that are very subtle in your network, and if you're really looking for that, then we will be able to pick it up. So that's a custom package you need to install it in Bro to detect those kind of scans, but Bro comes with a standard scan, standard average scan and core scan, and we will be looking at the examples later in the slides. Bro automatically detects the SSH brute forcing, HTTP password brute forcing, and SSH injection attacks as well. It's building scripts. You can disable them if you are having a performance impact, you can automatically disable them, but it is by default enabled so that you know what's going

on in your network. IOCs, Bro has Intel framework integration as well. So if you guys are a big fan of IOCs in your network, like if you have the lists of bad IPs, if you have a list of bad DNS domains, if you have bad hashes for the files, if you have that Intel data and you want Bro to detect that in your network, then you can actually do that using the Intel framework. You just have to enable Intel. Intel framework is by default disabled because of course, it needs the Intel input. So if you do not have input, there is no point in enabling the Intel framework. So the first thing I did when

I got into the university environment is, university is pretty decent in sharing the Intel. So if you have the university network called REN-ISAC, they share Intel pretty decently. We get really good Intel feeds for free. being a part of the university, an educational institution. So I was thinking that we are getting all this cool Intel fees, so how can I integrate it with our NSF system so that whenever it sees that kind of IOC in the traffic, I should get notified and I should know what's going on in my network. So Grow has great support for Intel framework as well, and we already blocked the IP addresses that are reported by different sites as malicious,

like more than 85% with more than 85% So we have not only just as a packet sniffer, but we also do active blocking on the alerts that we get. We have a fake Googlebot detection using growth scripts. I will talk about it later. And then the OpenVAS and ZMU scanners. they are open, they are the scanners that people use and you can easily find them in Prologs and I have examples for that as well. Moving along, so those were the very basic that you can automatically pick up from Grow logs and I will have slides to show you how you can do that. But apart from those basic use cases that we came up when

I started working with Grow, we had some advanced use cases as well because when I was analyzing the log files, I saw so much information that was so useful in the network that actually resulted in some in some really high level or advanced use cases. And I think I have mentioned five of them. The first one was detecting firewall misconfigurations. So our broad sensors sit between the routers and the firewall. So it sees everything before it hits the firewall. And then firewall filters the traffic it wants to, and then it hits to our servers or our clients. So we had a use case recently in PaaS that we have some restricted subnets. we do not

allow SSH into because they are our servers and they should not have SSH code open. And Bro was seeing internet traffic and all of a sudden, I was just going through the SSH.log file in Bro and I was just looking into these subnets and all of a sudden I saw an IP address. It was from the restricted zone and I was like, "Wait a minute, why Bro is reporting SSH service open on that IP address?" So I went back, I talked to my manager, I asked him that, "Okay, do we allow SSH in that subnet?" And he said, "No, we should not and Bro should not be able to see the SSH connections to that

IP because we are blocking it." So how Fireball works basically is if I have a rule saying this is the subnet and this subnet is not allowed, like drop all the connections to that subnet, Fireball will just block the send. So if Bro is seeing a SYN packet and it is getting dropped on the firewall, the client would never reply back within SYN and because client never got the SYN, right? So the connection will not establish. And then since the TCP handshake is not complete, there is no way to figure out that it's an SSH connection because if there is no data transfer, then how Bro would know it's an SSH connection, right? So I

was getting all those connections involved as SSH connections at the SSH.org. that meant that there was a successful TCP handshake and some data got transferred and that's how Bro detected that it was an XSH connection. So I was like, that okay, if Fireball is blocking the same packets to their IP address on port 22, then why is Bro reporting that XSH port is open? So it was a dead stop, it was a Linux dead stop and it was of not use and nobody was using it. So I actually went back and I asked them that "Do you have this SSJ service open?" And he said, "Yes, we have SSJ service open." So we went back

and forth and we figured out there was a misconfiguration in the firewall that actually leaked the packets into that subnet. So that was the finding that we had that, okay, now we were thinking the firewall is doing that, but it is actually not doing that. So the advanced use cases were like, we were able to determine the firewall misconfigurations using Roam. Enumerating servers and services, there's a really cool log file that Bro generates called knownservices.log. Whatever Bro sees in your network, and if there are servers on your network that is responding back service, it will log that server as an HTTP server or an SSH server. So we were able to determine how many DNS servers we have open to the internet and there were

very many because other departments had their own DNS servers who were open recursive resolvers. So we had a project of like shutting down or of narrowing down our attack service by shutting down the services that are wired internet accessible. So you can easily find all the servers and services that are on your network and who is serving what on your network. For example, we have 500 HTTP servers, while there are 500 HTTP servers on your network. Who is monitoring them? Who is watching them? Stuff like that. Policy compliance, there's another log file that Go generates called software.log file. It logs all the software that it sees on the network. For example, if you're using a

browser and if you have browser plugins enabled, Firefox Firefox does a pretty decent job of advertising all the plugins it has and with the versions as well. So, Bro detects that and Bro logs it into the software to log files. You can easily find out the clients running old versions of PHP, running old versions of Adobe, running old versions of Flash. And if you have a policy in your organization saying that you should not be running PHP version 5.4 or less, then you can actually pull out the list of all the IP addresses who are using PHP older version and we can actually give them the notice saying, "You have either upgraded or you need

to down from our network." So we did that as well. Few more advanced cases is malware detection. Of course, if we have a system, we want to see the network traffic from that system going back and forth for determining and what kind of software was that server using, how many ports were open, what was the communication look like during that period of time. So we do a lot of malware triage using Brologs and then recently we came up with a project called fingerprinting of unconstrained systems. University of Delaware has a lot of clients who are unconstrained. We do not control what goes on our students' Mac or what goes on their phones or iPads. So

if you want to fingerprint them, then what software they are using, we have done that using Chrome. There was an actual real talk of almost an hour in last year's that I presented on just on the fingerprinting of unconstrained devices. So if you guys are interested in how you can fingerprint your devices and come up with an inventory of the system, then you can go back and check that out as well. It's a pretty long talk, so I'm not going to go into the video because we're already running off the time. So fake Googlebot detection. These are now some of the slides I have that will actually show you some quick fix from the logs.

So Googlebot's-- what actually Google does is Google crawls. they have the web crawlers and usually you have robots.file on your web browser to allow Googlebot so that if you are running a web server you should be recognized by the internet force. So what people were doing that since if I'm running a web crawler, your robot.txt file will block me because I'm not Google. So what I would do is I would fake that I'm Googlebot, so please allow me. And robot.txt file allows Googlebot based on the user agent. So if the user agent has Googlebot somewhere in it, it will get allowed. So we realized that some of the characteristics of Google for running the web

crawler is Google always uses that that side of the block, like 66.249. If you reverse the NSS, like if you look into the DNS name of the type address, it always ends with the googlebot.com. And it has the user agent, of course, as googlebot. So these are from the notice.log file. They're automatically getting detected. Bro has a script to detect the web crawler activity on your network. So what we did was I just wrote a command line that get me all the docs that have a web crawler in it. And it has Googlebot as a user agent. And just exclude the IP address range of the Google that Google uses. And then just print me

some of the feeds. And I realized that these were the three IP addresses that we got, or two IP addresses. who had a very weird domain name, like teriya.com, and they were using Googlebot as the user agent string. So that means that they are faking that we are Google, but they are actually not Google. So we directly block those IP addresses because they are just faking out that they are Googlebots, but they actually are not. So you can detect the web calling activity on your network, which is suspicious, which is like why they are saying they're Googlebot when they are not, right? The second thing is quick wins. These all are based on notice.log file.

Notice.log file gets automatically generated by Bro if you're running Bro on your network. You do not have to enable anything or do anything special for it. So the scanners. So one of the slides talked about quick wins previously. So these are the scan attempts that we got by Bro. So the not-no scan, max-cato scan, and error scan. We absolutely blocked all IP addresses that are imported by MonsterDoc file. And not only that, it also listed the criteria of detection. So for the first one, it says that IP address has scanned a total of 12 hosts on port 8080, and the distance was around the one mile. The backscatter scan, it says that, you know, it

gives you the basic information that what port got hit by what IP address and how many times. So these are just some quick wins. SSH and HTTP attackers, you can see that the first attempt was detected as the HTTP brute force attack, second was as an injection, and the third one was faster, as we get it. So these all are just looking into the notice.log file. We haven't done anything special. We just ran Grow on our network and we just found all these things and we actively block those IP addresses periodically whenever we see that. And it's very cool, like it says that the IP address was actually guessing the password and it was seeing 42 connections. So it's not like somebody just trying to get into the server

and it failed 42 times. It was actually someone trying to brute force an SSH password and 42 different connections. Lastly, I was talking about the Intel framework. That log, Intel.log, will only get logged when you have Intel framework enabled. So since I have talked before that really use Intel framework a lot. That one was for the Intel address. So we have a list of blacklisted IP addresses and we tell Grow that if you see that IP address, just log it as an Intel.log file and we will work through that. So we have automated scripts written that whenever there is a log generated in the Intel.log, we pick up what was the IOC and whether we

want to block it using DNS blacklisting or using our IP blocker on the borders. So that is because of our IOC detection and we blocked everything that reported as 85% on regular confidence as a established IOC. The second one, we have a block that also automatically gets logged by Bro. what it actually tells you that Bro has the ability of parsing the protocols, right? So if something is not complying with the TCP/IP protocol stack or the RFC of the protocol, it will log the tag activity as the weird log. So we get a lot of activities weird like Bro saw SYN packet with data or Bro saw a data packet before SYN. So those all anomalies are logged in the

weird log file. So that is also a very cool log file that you can keep looking at and you also block all the IP addresses that are listed as bad ICMP checks and connections. So I have listed all the things that we actively block on and there are a lot more so I couldn't like put together everything in just 20 minutes but these are some quick wins that you can actually have right on top of Grow. Like you do not even have to get these kind of information. And this is very basic attack activity that might be happening on your network and you might be blind to it because you might not pick up some

of the things, but this is always detecting on your network. This is getting logged as the weird or anomalous activity on your network. And lastly, there's an http.log file. You can do very simple command line control, A lot of times scanners do not go and bother changing their user agents. So many times we have found that OpenVAS is scanning your network and they have not even changed the user agent. So if you just grab an OpenVAS and HTTP.LOG, you can actually see the OpenVAS scanners getting hit and you can actually actually block the IP address. So the first example is the OpenVAS scans and the second example is the ZMU scanners, a lot of sophisticated

scanners, they try to mimic some legit user agents like Mozilla, Firefox, or Chrome or Safari, but if they do not care, if they don't bother, they just scan the network with the default user agents and you can actually pick that up using the HTTP or log file as well. So these were some really quick things that you can get out of the grow logs, right? Just looking at the log files, and you can really You can really sophisticated stuff as well using scripting and signature framework as well. So that was it. 27 minutes, so I didn't do bad. Still have three minutes, so any questions? Go ahead. So all this activity you're seeing hitting against

this age and all this stuff, it's all hitting publicly accessible assets or is it hitting stuff that's inside, deep inside the network? They are hitting our clients as well as things that are behind, that are internal to our network. But since our grow sensors sit outside the firewall, so if they would have been placed inside the firewall, firewall might have blocked those attempts before even hitting grow. So even if it's just a sin attack, grow would be able to see it and grow says that this person is doing a sin attack on your network. So that's a good thing, right? You have layers of security because firewall eventually might block it. But just to know

that somebody was trying to create the password or someone was just trying to do a SYN attack is useful information because that might be trying something else in the future on your SYN network. So if that makes sense. Or it's just the internet. It's just the internet. It's just a lot of traffic we are monitoring. We are not monitoring the internet traffic. So then when you block it, actively block it, is it scriptomatic or is it holding those IOCs in it?

IOCs, you actually load in Bro. Bro automatically, when there's a hit on that IOC on the network, it produces the Intel.log file. We have scripts that goes through the Intel.log file and whenever there's a hit, it picks up that IP address and we have another script that blocks it on the border. So when you say we actively block these IPs, right, do you take those IPs in manually or you have that script? It's an automatic script. Yeah, it just pulls out all the blacklisted IPs that got hit on your network, it puts it in a file, and then our different block lists, they just pick them up and they just add them in their configurations.

It's all automatic. It was just one time manual, you have to write scripts to pick that up, and that's after that, it's all automatic. You don't even have to go ahead and change anything in the future. Is there a way to exempt any IP address that you want? Absolutely. So there is a... If you go to the Intel framework, there are the file list dot profiles, where you can not only find site location. So we have all file listing done. So when I started that, I accidentally blocked the Google SPF IP range. and I was thinking that Bro was picking that up as the scanning attempt and what was happening was Google was trying to

send emails using SMTP and it was saying there was an SMTP scan and Bro picked that up and Bro blocked it and I was like, "Oh wow, it's detecting all the activity and it's blocking it." And some of our admin said, "We have delay in the emails. "The email started at 8 o'clock "and we are getting it at 10:00 AM. "Why is it getting delayed?" And I was like, "Oops, let me check." address and the complete Google SPF range got logged. So then we learned the lesson that we should bypass the Google SPF range, Microsoft SPF range, and the cloud ranges so that they do not get accidentally picked up as the scanners and get

blocked by the road. So has really decent support. Can you also bypass by the FQDN or is this IP address? You can block it with IP address, you can block with the CIDR, you can block it with the FQDNs, and you can block it with the-- You can buy it. I don't think so they have regular expression support. I might have to check that back. But you can absolutely wirelessly wirecard. If you do not want to block anything with the google.com in it, you can use astray.google.com and it will wirelessly. So whatever Intel IOC support bro has, for whatever type, it has a wireless for it. So even for the hashes, if you have five hashes and if you have wireless, file hashes, you can

actually do that with the hashes as well. So whatever it has as a support for blacklist, it has a support for white list as well for that type of Intel IOC. And if not at bro level, you can whitelist the things at the level you're blocking. So we have a centralized whitelist that is our second layer of protection. So bro whitelist is perfectly fine, but just in case something happened and bro didn't pick that up, we have a central whitelist that is sitting right before the router BGP calls, so that whenever there's a BGP call, it will go to the whitelist and see if that IP address is listed as the whitelisted IP address. and would dice the null route for the right-wing address. If you have a

both whitelist and backlist entries, which one would take the precedent? I think the blacklist would take the precedent over the whitelist because you have both files. You have both-- You're talking about-- Yeah. What would take the precedent? So you said there is a blacklist and there is a whitelist, right? So you can use both at the same time. Right. So which one would take the precedent if you have both entries on it? So how it will detect the internet IOC is based on the . So if you have an IP address that whenever there is a traffic pattern, it will go through the list that you have. If you have it in the blacklist, it

will put it in the internet log file. But if it checks into the whitelist and it is a whitelisted IP address, it will not log it. It will not log until it's explicitly mentioned that it should be logged no matter what. So that you can actually log and see how many IP addresses that are in the whitelist are actually getting hit in your network and is picked by Bro. So Bro has a very versatile logging framework as well, so it all depends on what you want to log. So you can enable and disable that as well. It is in the Intel framework, if that makes sense. Yeah, so it doesn't really block anything, just like...

Bro does not block anything. We have to fix up and Bro just logs it. All you get out of Bro is logs. Rho does not have anything active unless you are using the packet broker. They have IPS mode as well. We do not use Rho in IPS mode because we are pretty scared to use it in IPS mode which we are not 100% sure of. But whatever it logs, we take actions on the logs that we get from the general file group. And that is all the manual and automatic. Things that I have showed in the quick fix, we just pull out the IPs in the log and we just put them in ML routing.