Intrusion Detection in the Clouds

minimal electronic music for those of you who know what that is uh Josh presented at Defcon 18 on the topic of building your own security operations center using open source tools uh please stick around after the session as Josh will be spinning during the break to celebrate his career change farewell [Applause] party no I love the security industry I'm not leaving hold on a second okay uh I just had a weird technical glitch of course just in time so we'll see if this works um second all right so I'm as you just got I got this uh intro so that's one slide I'm a security analyst here I get to review domains review um delicious

behavior from malware and other cool stuff like that and uh I've done a lot of work in the security industry for many years I'm going to be talking about building an intrusion detection system to protect uh cloud services or Cloud servers and in my demo hopefully which works will um it will show uh protecting a uh a web a web server like hosted on HostGator right so Cloud hosting is really cheap cheap and it's convenient and it's easy to do an inexperienced user can pay $10 a month or less and have a server up and running in just a few minutes the problem is that security is really limited uh just the providers that I'm working with today they they

limit their their offerings to providing documentation on how to secure your server or how to report of a compromise of some sort so what kind of people would even bother with what I'm about to talk about I do it you might be interested in doing it uh it's kind of fun but I bet that most likely most of the site owners out there are not going to be interested in doing this so this could be something more that you do because you're hardcore geek or maybe you um you're providing as a service to some other people and why bother doing this I found it to be really fun and uh a little trying at times and security is

important the looking at data from fish tank which is a site where people can submit uh fishes that they think are real uh and people vote on these fishes to to say these actually are real there were a total of 3,000 sorry over more than three million fishes um submitted and of those verified there were uh under just under 2 million just last month there were 28,0 46 online verified and of these online valid fishes for March there were uh TW over 12,000 unique domains and those attributed to a little over 2,000 unique asns I tried to do a little research to see if I could attribute those asns to free hosting providers and um and

compromised providers potentially compromised or actually purchased domain names for uh more targeted fishing but that was a little bit more work than I was able to do but this is just a list of the of all the asns involved just kind of scrolling through it it's not that exciting to see um but I hope to one day be able to put that into some kind of beautiful graph or something uh additionally compromised sites they or sites are compromised just to compromise them for a statement of some sort these in these situations the compromise is much more obvious but in the fishing situations the content is typically buried deep in the site or Mau is buried deep in the site and the site

owner is not aware of this this until they're put in a blacklist or notified by a third party in those cases they usually uh from what I've seen in most cases they remove the content they don't fix the initial Vector of comp sorry they remove the content but they don't fix the initial Vector of compromise so using Wordpress as an example WordPress is a Content management system which I'm sure most of you or all of you are familiar with there are plenty of vulnerabilities for past and current versions and there always being updated there's a site this is from a website you can just go look at the vulnerabilities additionally there are plugins for Content Management

systems and those are also exploited in my demonstration you'll see that I will exploit a uh a plugin and I'll show you observing it and not observing it you can scan for things very easily uh WP scan is a great WordPress scanner and I will show you some of that too um before I go into more I'm going to give you a basic super high Lev overview of what it looks like from the average visitor to the attacker so you have your hosting provider and this is super basic got your ISP these are these are clouds they're networks and you have the internet in between and when the uh person the visitor wants to go to your website they

just they just go there that's the really simplest way I could put it now let's show the attacker or the hacker version you've got your website you've got your ISP maybe they're not ISP I mean if they're smart they're proing but whatever they're coming from some place and they just uh they're there and they go through the internet and visit the website so I'm going to show you uh hacking a plug-in called the revolution slider or slider Revolution I call it rev slider it has a bunch of names it uh there was a vulnerability that was presented a while ago it's been updated now but you could get do a local file inclusion attempt and get file out of

his server use um exploiting this vulnerability I'm going to be putting in this this string here which is either's domain it's my name there's nothing really added except the web WordPress but uh I'm going to be pasting this command and it just basically is an attempt to get the wp config file which is going to have my database username and password which would be a great start to a compromising a website so here's my if I log into my jp.com my WordPress admin page you can see I've got rev slider down here at the bottom this version is the vulnerable version so um this is a a just a visual of WP scan scanning my server uh this anyone

can do this they can see what version of Wordpress I'm running which I didn't highlight but you can see at the bottom I've got a plug-in running I've got rev slider it doesn't say the version but it's um worth a check I mean it takes a second to check it so I'm I have a video here but I have some demo actual demo stuff I will be doing soon but because of the DNS the way I had to set the DNS up I had to do a video for this part so opening uh Safari or whatever there's my WordPress site this is the attacker the pasting in that command that I showed you and what they should get is a popup

asking you to download the wp config file and they actually get an error message they get an error message from Mod security mod security is an Apache module that provides security features so that when it hit this jp.com this is actually at HostGator and HostGator doesn't say they are running mod security or anything like that so that's but that's cool mod security protected the site but we actually have no visibility into this at all um if a attack was performed and it was successful we'd also have no visibility all right let's talk about the neighborhood just for a second I'm going to talk about where this domain lives in HostGator just to give you kind

of some context of maybe why someone might compromise your website just some quick information this is some DNS traffic for jp.com all that's hopefully me um it's not much it's just me hitting the website you can see uh there's a IP address that 97 that's the last octat there uh that's what we're interested in we want to know who else is sitting on that IP address CU you know I pay $10 a month for this host and I have a graphical user interface to access it I don't have root access uh it's shared hosting um here's another view of it just real quick kind of um from robex it's a website you can see there's

jp.com and I've got my IP there anyone asking for where's this website It'll ask query name servers and DNS all the way up until it gets to the HostGator and HostGator says oh it's right here at 97 which I I own HostGator I own but who else is at this address well that's a lot of people a lot of domains not people but this is a listing of everyone sharing that IP address that's a lot of like I don't know what what else is on there I don't know if anyone's going to want to Target that or Target the actual IP address to see what they can find uh just an end map scan of that IP address

you can see there's a lot of ports that are open and filtered a lot of room for exploration at the very least now looking at a basic how things been for a long time uh you usually have if you have a network and uh you have full control of your network you maybe get to you have a router you control or maybe you're just in there but you've got intrusion detection systems which are monitoring for actual taxs and malware on the systems beaconing out all sorts of stuff like that you're protecting your mail server your web server your active director server all that stuff that's great you can go look at logs you can go do an instent

response on an actual machine when you see a compromise rise but um as we all know the the it's easy to set up websites and also the security perimeter perimeter is eroding quite a bit so people are moving things not just websites they're moving things out um so that they're easier to deal with now in my configuration I am actually going to be doing uh you have the internet and I have my intrusion detection system is going to be running on a virtual private server at Lode Lode is a place I'm paying $20 a month for a Linux server which I have full access to I don't have it's not a hardware machine it's a

virtual server I'm going to be running some things on there and sending traffic over to my HostGator server and then later on I'll be switching it in my demo to another uh provider as well um I don't know why I didn't click the button for this little arrow diagram because that's pretty obvious here's the intrusion detection system it's running Apache I'm doing some proxying which I'll get into in um the hope I don't get killed before I talk about what I'm doing to mitigate my proxy stuff uh I'm running IP tables I'm running fail to ban which will add some things to IP tables I'm also running snort there are I think there's actually much better

stuff like cotta I prefer but I'm using snort for this TCP dump for packet capture and I'm using some other stuff maybe I'll get into that uh looking at my linode control panel this is where I log in on their website I've got a Debian system and it's pretty it's pretty minimal not very much memory not much space um I don't have much bandwidth but I imagine if you wanted to do this for lots of websites or lots of servers you might need more uh oh forgot that I added that that was my uh a record for line not anyways I'm using an open proxy open proxies are extremely dangerous for the internet they uh I'm also mitigating it with IP

tables uh with an open proxy you can have your attacker classical hoodied person and they can connect into your open proxy which are constantly being scanned for people are always scanning Pro proxies they can connect into it and then use that to connect out to another site and then do whatever they need to do and it looks like it came from you so they're really dangerous uh this is my uh my first rough attempt at doing this I have some other plans which I get into at the very end uh the configuration uh just a little bit High a little less than the last high level overview but this is still a thing the user says where's jp.com internet says

it's over here at 97 that's HostGator this is the the way it's set up right now and then they go there uh the version I'm doing is they ask where's jp.com and the internet says it's over at1 189 that's actually Lode my LOD server with the IDS and they think it's there so they go there and then it carries on over to host skater now the public DNS how I set this up I had this is where I purchased jp.com and where I set my name servers the name servers are as you can see are host Gator down there and you just go in there and you change it to uh to Lode my where my IDs is this is the one

change that the site owner has to make uh after this you wait 24 to 48 hours although sometimes nowadays it's almost in it will update and then anyone asking for this domain will say it'll say it's over at LOD but of course the files are still at HostGator as you can see here I have this is my control panel for HostGator um zooming in we see that we've got this add-on domain and when I do the configuration you'll see how I have it as an add-on domain it basically creates a subdomain uh my files my WordPress files are stored in this public HTML domains JP so let's put an intrusion detect system in there I'm going to do a demo

of just kind of real fast how well maybe not real fast but anyway I'll just do a demo uh so I have a uh a little text document to kind of guide me because there's a lot of typing and I don't want to make you suffer um so if I on the top top window I'm going to be SS I'm going to SSH into my I have some scripts and I'm using public key authentication so I don't have to type passwords a whole bunch uh oh I do have to type it a couple times at least sorry okay and I'm going to do root because it's just a little easier so this is me logging in to the Lode host

I'm just going to show you some things like uh I have uh if I look at my Apache config Apache you can configure sites that's I mean you can configure as many sites as you want but looking at my default config I have this right here I'm listening on Port 80 I'm listening on Port 443 uh 443 is for my uh where I look at my actual contusion detection results uh down here you've got this is the proxy I'm listening for anything jp.com Port 80 you can modify this to do other ports of course because https is obviously a thing um then this proxy pass right here it says anything going to this domain send it over here this is

my subdomain I have I have a server whatever it's name absent faith.com and J.S faith.com is my add-on domain sitting at HostGator so this is going to send everything over there uh quit that I'm going to show you uh this section here this is the proxy configuration um this is what I did proxy request on this makes it an open proxy this is the extremely dangerous thing that you shouldn't do but uh I will now show you how I have IP tables configured if I look at IP tables which is a software firewall uh I have nothing in there right now I'm going to add some rules and I I won't go into great detail

about it but I'll just tell you this is an output rule to allow all Port 80 traffic to 97 that's HostGator I'm going to add one for the input chain and I'm sorry I'm not going into more detail on IP tables there's a class I'm sure you could take um and then I'm rejecting I'm rejecting everything else that is going to Port 80 and I'm going to demo oh I should probably type the whole command I will demo using it as an open proxy and failing and succeed maybe slightly succeeding um so I'm going to start Apache because I don't have it running because I'm still a little wary of this whole setup uh

uh all right patches started now I'm going to run um I have a script I wrote to just kind of run TCP down but it's going to show it streaming in the top window right there so that's cool showing streaming it's not showing me uh some traffic so it's not too noisy um you'll see some stuff moving as it goes but uh now this bottom window is me on my system right here this system I'm going to tnet to jp.com this is connecting to ID to the IDS system at Lode and then I'm going to try to use that to go out I'm actually going to go out to the HostGator address so I will

see success so if I tell net and then I got to paste something else in here real quick before it cuts me off and one more thing um so I'm trying to connect to that subdomain and we wait a second and you can see down here here's here's some WordPress content so that was it connects on through but now if I try to use an open proxy I tell net to this system again and I'm going to try to proxy off over to Yahoo uh I want like the index page from Yahoo and I just wait and it's going to it's going to kill my connection is just a second because I didn't allow it um I

got some server information but so people are going to be able to scan my host they're going to be able to find that there's no Pro there but they can't use it hopefully um I'm sure there's always a way around something so I'm stopping TCP dump on the top and now I'm going to show you the rule I have this is a snort rule I only have one enabled because I don't want to make it a big mess uh if I look at my local rules file I have down at the bottom here I've got this uh it's looking for this content right here this that I'm going to paste in the browser it's where I showed you

before and if I look at I'll just show you real quick this directory this is where it keeps snorts when it's running it's going to write to a binary file a unified file and that file will eventually be picked up by another program which maybe you're familiar with called Barnyard that's going to put it into a database so we can look at it um looking looking in there I've got one file but in a moment it's going to put a another file called whatever. You2 with a naming convention so I'm going to start snort I have a script I already wrote to start it so you don't see the whole command so snort running and uh I actually need to

connect in again to the IDS on this bottom window here because I'm going to run TCP dump because you need to see the packet capture after so TCP dump it's a script this I had the script ready to write a file called host gator. pcap to to a directory which I'll show you in a few moments all right so now I'm going to open up um well first I'll show you we're going to log into this is the intrusion detection system I'm using snorby to show the alerts the snort alerts if I look at my I've got a lot of these alerts because I've been doing a lot of testing um but you can see in a

moment you can see I've got uh the last alert alert was actually yesterday because I was doing some testing last night so 418 right here um opening a new tab and let's go to if I uh go to my text file we're going to visit my domain we're going to see HostGator so this is sitting at HostGator we just passed through the ads but now let's go ahead and paste in that entire command this is that one I showed you before when I do it I get that mod security error so minimize uh fire Fox and I'm going to stop TCB dump I'm going to stop snort and I'm just going to show you real quick in this verog snort you can

see that there is a YouTu file um it's got some data in it 607 bytes so I'm going to start Barnyard barnyard's going to grab that file and it's going to put it what it found into the database so starts it up and you actually see it gives you a little notification it found something um I can go ahead and actually stop that right now cuz in there so we go back to the browser and uh kind of have to go back that way so I click on the alerts here you can see I've got a new alert here 222 so we've got visibility into an attack we see that there's there's the git request this is what I typed in I

know it's kind of a mess um hopefully you're familiar with reading stuff like this but you can see g w p- admin and the whole thing is there but you don't know if it was successful you don't know that mod security protect ected you so for that you actually need to look at packet capture so on the bottom window I'm going to exit my IDs I'm back on my local system and I uh I'm going to copy over that pcap real quick to my desktop uh okay so looking at it in wire shark I'm going to filter on HTTP cuz clean out a little bit of a mess there um right here at the bottom let follow with

stream you see there's the get request this is the response not acceptable so we know this wasn't successful we know there was an attack you know you can set up your IDs to give you emails or whatever you want if you want to know when attacks are happening although I wouldn't recommend it if you're using a lot of rules but there we can see that it happened so that's awesome uh I'm going to I think I'm going back to the slides um okay so now we're going to do another hosting provider with that does not use mod security I bought a host on digital ocean $5 a month I have a Linux host with uh I have root access I've secured

it as best as I can I didn't do mod security this is how it's going to look I'm just it's the same as before the changes are made at the IDS not at the registar uh and it will automatically point it to this new server as I just pointed out it won't be changing this is at digital ocean this is my control panel I have a a I set up a DNS Zone it's a if I were to set my register to point to this it would say this is where jp.com is but this is uh this is an internal basically internal for this situation uh that's my a record the 32 is the address so let's do another demo this

won't take as long but we're going to see a successful attack this time and we'll see the visibility into it so first of all I have to change oh first I'm going to flush IP tables this is removing those rules I didn't actually have to remove those rules I could just add more keep adding more I could do maybe a million websites I could do a lot of websites though but uh maybe 10 but I removed them and I'm going to um I'm going to add IP tables rules for the dot 32 address that's the digital ocean address adding to the output chain the input chain again and then I'm going to add the reject which will reject everything else

um looking looking at it you can see there's my so 32 I'm allowing out in and reject and then I'm going to modify my Apache uh sites file where I'm proxying oops actually need to modify it okay so down here where I previously had this do the subdomain I actually got to put the IP in or you could do a domain if you're doing subdomain but put the IP in it's got to be it's a relative P or it has to be absolute path um in terms of it has to have HTTP col SL slash so I'm putting the IP there and putting it here and it has to this has to follow a certain

convention so that's the change needed there then I have to restart Apache to take those changes into effect and now I'm going to log into my on the bottom window I'm logging into digital ocean so I'm um now I'm going to get rout access real quick I'm going to start oh actually before I start aache I'll just show you what it looks like this is my Apache config file it's running Apache as well which is serving my files you can see I've got this anything looking for 192 this address right here Port 80 uh send them over to this directory where I have my WordPress files uh exiting that if I look at uh look at that you can see there's some

WordPress files right there that's what's going to be served to the web user web browser person um I'm going to start a pachy because I'm not running it right now on my digital ocean host and now if I let's see here I'm going to go ahead and start snort again up at the top window this is on the IDS start snort and then I'm going to exit bottom window I'm exiting digital ocean I'm going to exit I was exit a route there I'm exiting now and I'm going to log back into my IDs because I need to start TCP dump and um I have a script to write to a file called digital ocean. pcap so that's

running and you can see if I go back into here I mean the last alert you're going to see is the last one I did because I haven't done a new one yet um but I'll just go ahead and let it load and see uh okay so you can see that is my last alert 2:22 p.m. now I'm going to let's go to jp.com this time we're going to see digital ocean and I'm going to paste that same thing into the same local file include into my browser bar and we will actually see a popup asking us to download a file so here's this popup we let's go ahead and get that file and um I'm going to minimize

this and then here's this file I just downloaded if I open this up I'm sorry about the black text I just Chang that but this is my WordPress config file so someone got it there's my password there's my WordPress database username this is a great start for an attacker they've they're getting there um but as a website owner I have no idea that this just happened so unless I'm using the intrusion detection system so I'm going to stop TCP dump I'm going to stop snort I'm going to start Barnyard and it's going to grab that You2 file there was one that was written it's going to start grab it and I should pull type the whole command

and it'll it sees there was an alert and it added it to a SQL database I have running we stop bar yard we go back into this and for some reason refresh doesn't work on that page I mean it does but it's um you can see there's a new alert 2:30 p.m. and here's the G request looks just like the the HostGator one you you you don't know that it's successful you need to actually have packet capture to see if it successible I think that's a failing of intrusion detection systems not providing a little bit more than just that get request but I'm sure they can be modified um then I'm going to grab that pcap

file from I'm going to exit on the bottom so I'm on my local system again I'm SCP that well hold on okay down here is my digital ocean peap I open this up in wi shark and we'll filter on HTTP clean it up a little bit we can see here's this git request right here I'm going to follow the stream uh and we can see there's that git and here's the response the and this is the actual the file being downloaded with my password and username so now I know as a site owner who's running this service hopefully I I know that someone got my file I need to change my database username and password

I need to fix whatever may have let them do this it doesn't say rev slider although you can guess by the get request up at the top um hopefully I would uh Wipe Out the entire website and start over now going back into the slides uh I'll talk about some potential problems with this let me take this back uh it kind of kills analytics a little bit the statistics uh on the site which is frustrating if you you're a site owner running a just a basic site or I mean usually you're not doing analytics on other services or maybe you are but talking about website specifically here you want to know where your users came from or most people do

um you can see at the very top my line out host is the main visitor the other two are me doing some testing from different IP addresses so uh if you were providing this as a service you might have to create some kind of analytical thing or some statistical thing running on the intrusion detection system uh this is a little video I just I'm showing the uh Apache log running on uh it's running on actually the intrusion detection system so we can see visits to jp.com are coming from U certain IP address and we can see the uh attack when I think I do the attack again yeah I do the attack again and we'll

see it being sent back um to the Lode host 189 so it's kind of it's a little weird I'm sure there's a way to programmatically deal with that but I haven't dealt with that yet um other protocols such as https uh I have experimented with putting a CER using a certificate on a on a site and then putting it on my IDs but I wouldn't recommend that if you're doing this as a service because then you man in the middling people and causes some concerns um but https it's going to be you're not going to be able to see it anyways you could just let it pass through most of the attacks or if not all the attacks on

a web on a on a web host is are going to be HTTP SQL hopefully you're not sending data you know my my SQL data or any database data in the clear you have hopefully have a tunnel which you wouldn't see anyways so basically just for this purpose you're watching HTTP other services you might be running you could be of course um it could be other stuff and and I actually talked talk about in a moment how um how how that might be addressed uh so the open proxy uh which still worries me quite a bit I'm using IP tables to mitigate the attack attempts I'm using some other protection mechanisms such as fail to ban this is me getting

blocked when I was doing some stuff um uh I blocked for like 10 minutes that was automatically added to IP tables uh so that's that's another option for protection um my future ideas into this topic I would like to do this but I just haven't had the time where I maybe set up a a little bit more robust Cloud Network um and it's running a name server and and then it sends you off to a router Router goes pushes you right through the IDS there's no proxy involved that way you can send any kind of traffic uh without setting up a Apache config file or something like that and then I just thought of this

today when I was watching that Cloud talk this morning which is pretty cool uh I was thinking it'd be cool to watch uh the um connections to to things that like like skater systems and stuff like that there there are honey pots for that I think it's called con poot uh you could do you could put an intrusion injection system inside the stream connections to honeypots and then those would send data to Honeypot server you could have alerts so you would see the attacks you would see the alerts I don't know it'd be a cool uh threat intelligence feed perhaps so that's something I may be working on in the future and other than that I've I've got

nothing else so you have any questions you considered how this scale because for example if I if I have hosts at let's says right and use elastic L Balan then that means that would have to be L balancing yours correct how been yet yeah I I think you would have to load B load balance the um what what oh I'm sorry I was already ready on the answer um how would you load balance this if it was a in like AWS and you're doing um consider how this play out an environment like AWS for examp that has bu well I haven't really thought too much about that yet but I think I would imagine hopefully I wouldn't be using

the proxy option I'd be using the router option you can put throw as many machines in there as you need to um like yeah yeah that's go ahead you IPv6 um I didn't use IPv6 hosts in here so uh these I assume I could proxy that too I haven't it's still I'm still exploring oh I didn't I didn't block in off IPv6 no no I suppose I probably should yeah any other questions oh go

ahead it was my um first attempt at trying to make this go and I was thinking it's got to be possible to have an intrusion detection system to watch stuff from anywhere going into one location um the open proxy is the only way that I could do with software that I I mean there's other software I could buy software it still is basically an open proxy but to allow connections from anywhere anywhere at all without having to specify I I'll allow these connections or these networks in a file of some sort or in a in the IP tables I had to I had to do the open proxy oh yeah I think I looked at that too I

was just going with what was on the system I was trying to keep it a minimal installation but I know there are better options this is um more of a proof of concept and the beginning steps of research but yeah definitely other other options like that like mud

proxy okay I guess that's it for questions thank you very much