Cruising Ocean Threat With TheHive, Cortex & MISP Without Sinking

hi everyone thank you for having me here besides littleman I'm going to present a talk around miss cortex nd hive and my name is Adi as Bruno mentioned and on the leader of the high project so to start our talk we'll begin with an overview so about the hive projects basically I've used the project with other people for two reasons the first reason is because we work at a computer security incident response team I lead one at a large financial organization and since I've started working in this field 10 years ago I have been kind of getting like mad because I was thrown under a large number of security events that keeps going on and on and I'd time

I didn't have the right tools in order to deal with cope with that mess massive amounts of security events including of course false positives and try to drive down to time to detect and react so as a token token of appreciation to the community which helped me and helped with the other founders of the hive project do their work thanks to the large set of open-source tools out there we create behave in cortex which I'm going to speak about shortly so how we set up in order to drive the time to react and detect we decided to try to explore ways in order to collect as quickly as possible in as cleanly as possible alerts and events from

different sources try to automate all the tedious tests initiative tabular one analysts do on the daily basis for example analyzing malware analyzing URLs and things like that because a lot of the work that a security analyst does might involve copying in passed in from different interfaces and scripts for example if let's say someone reports suspicious email to me and of course it will be locked back and forth oh how can you please send me the full headers for example of your email and then I find a link in the body of the email I take that link I copy it then I go maybe to virus stole and I paste piece it there but maybe when I do that I should have

thought that I should not be doing that because at the at the time and I we received the sisters email I'm not sure it's a targeted attack so maybe by pasting I would see them for me sending the file to versatile or send the link to versatile I'm leaking some data that attackers might be observant so I'm given like I would say tips that I have found them in my network and also as I've seen this field bro I realized that there is no on say it's not a one man or one woman so it's a large skillset involving different security analysts from different perspective reverse engineers malware analysts and singer correspondents and so on so they need to

collaborate all together so we created tools to help with the cyber threat intelligence I would say like collection but I would say in the hive in cortex this is just a little part of those tools because we rely on the de facto standard for that which is miss our core I would say a value would be a rather innocent response in digital forensics and we'll see later in some use cases so created - - cortex we also provide a set of libraries I'm going to mention later and also what I call fears which basically take some others from some source for example your IDs of your museum and then inject them 2d hype and there is one special

theater called sign ups which I'm going to talk about they're currently the hive core team is six members baked it with a large C cert community we we are not commercial shop of course so we don't really track who is using our software but we can count more or less more than 50 large and similar organizations throughout the world that are using actively either the hive cortex and or both so we created the hive internally in our team in October 2014 and we kept finding it using it on a daily basis we had 12 year analysts using it and once it was good enough for us and decided to share it with the community we didn't want like just a

proof of concept and call for help we wanted something really stable that works for us on a daily basis in once we were satisfied with it we released it on November 2016 16 under a GPL v3 license as Miss and a few months later released cortex which is an analysis engine which was first embedded in the high and we extracted from that for reasons I'm going to mention later if we take the sense six steps process which could be familiar to I would say even junior incident handlers so you have the basic step which you always have like preparing your team to deal with the incident response in digital forensics then move on to detection and

identification from there to containment eradication recovery to get back to business and then lessons learned which often than not is something that we forego because like we are busy or we don't want to document things properly so if we check those six steps process I would say that the hive more or less covers all of them because it allows like exceeded the team to was they receive an alert right Hindi hide try to like identify it liberation by that cortex also like contain things for example we can use the cortex responders to perform actions like for example block this IP address which is malicious at the proxy level or open this ticket in JIRA to ask the IT team or the

network team for example to do some action and then you can also go as I would say as far as eradication but I would want to recommend that for automated ways at least not if you don't know what you're exactly doing again through cortex responders and move on maybe but to recovery by again using the responders to say I'll block some IP or some user account and things like that so cortex is an analysis in response engine it covers the four steps from detection I would say uh notification up to more or less recovering well this is used for IOC storage indicators compromised but also will share in with your constituency and we are your other

peers in partners so in a nutshell d hyb is what we call assert a security incident response platform for those who fancy buzzwords Carter came up a few maybe months or years ago with a new fancy word for that what security orchestration automation responds well so it allows you to collaborate in real-time it has like a Twitter like feed which you can monitor to see what's happening in the platform and just by for example if you see like in your Twitter feed like a new task that has been created just by clicking on it you its belongs to you you can start working on it and the others will see immediately that you are the one who is for example to

win reverse engineering malware it has also fully customizable dashboards to allow it to track activities for example you can say what are okay top ten types of I you sees I'm ingesting am i doing something with them do I have the tools to do something with them or you can for example say what is I would say regarding my SLA is to my constituency what is the maximum time I'm taking to deal with a specific would cases I would say fishing cases or again one of my top ten used analyzers for example let's say virus or was among them so that will allow me to say okay so next month maybe I need to add more

money because I'm exploiting my virus total quota it by itself I would say it's more or less useful but it's better to use it in combination with a cortex enemies so cortex would be used by the hive for analyzing at scale of circles of different nature for example you see like hey take those 100 euros submit them to virus total passive total or take those 100 files set them to both kuku sandbox and Joe sandbox at the same time it also allows you to do response for example as I said block this IP address at the proxy level open this ticket in the architecture system and so on it can also leverage miss for CTI

fractions we will cover that later it has multiple authentication matters lateral you can mix and match so that ad or two api TS TLS certificates and so on it has also went hook support for automation and this just a few of the main features of the platform as for cortex as I said earlier analysis response community has 101 analyzers so when we release it back in February 2006 17 we supplied 16 and almost all the others have been created by the community and contributors they thought they are all open source but of course like outside the virus analyzer which is open-source requires a subscription to the commercial service some are completely like free like file

info which is kind of I would say file static analysis analyzer it has several models for example only tools and things like that it's completely free and to develop the analyzers you could either you go the easy way using Python because we provide a library to ease up that or you can go the hard way choose another language for example Russ tell concert in Russia created I would say about scent AMA lasers in the gold language and the hive can plug itself with multiple cortex instances for example you can have more cortex instance within your network for your internal services and maybe interacted with a few external one like faces proton virus from doing our

analysis and things like that and you have another cortex is sitting on the outside which is dealing with your sandbox because we don't want to I would say detonate malware that can get off hand and escape from the sandbox then maybe impact cortex and which might result in impacting your beehive in instance and your internal network so slot for opposite operational security reason you have got two cortex instances and you can cortex can leverage miss for additional analysis possibilities for example it can query multiple missed instances to say hey do you have this IP address and if you have the IP address can give me the events which it contains that contains the other IP address and

of course I think many people know miss or use miss so in my opinion is the de facto standard for trade sharing and also it's the opinion of some out say big names like a forester in there q3 report of yeah of this year they have compared miss different commercial tips like a cryptic IQ anomaly threat questions and so on and the adoption they found out that the adoption of myth is by very very large in comparison to the commercial Internet is and moreover the the support is quite good in comparison to the commercial alternatives so they said like yeah if we don't fear false you should use miss so it allows you to collect your threat

intelligence whatever you put behind that word let's not get into the debate now sanitize the I UCS for example yet and disseminate them through like sharing to multiple missed instances but also to different other tools such as D hide it allows you also Internet to do correlation and basically we use it when you use it 300 of the hive cortex in this we used to store our IOC s it supports tagging galaxies objects tucks and misses as a tag and much more can be also highly automated and as I said earlier the hive can import events from Miss and export event from Miss and can do both at the same time 8-pin for example import from to

export the three-and-out say do both on to others and it's also tried to integrate it with cortex for example if you are only a missed shot you don't care about doing in select response you just wheel it through your your like a thread analysts something like that and you're only worrying about enrichment and things like that you can still use miss in cortex so you can call cortex from miss and in which the dataset from an architecture point of view so both the hive and cortex used more or less the same stock so the back end is written in Scala using back and play framework and the front end of both cortex and the hive are just basically

making restful api calls to the backend and currently we use elastic for storage on both dehyde and cortex but we have plans to move away from that for more serious limitations we have encountered on the road with elasticsearch we are considering moving I would say around q2 next year to a gravity and the analyzation responders are usually written in Python but again you can use any programming language language that are that is supported by like Linux 150 feature of cortex it has bareback or role based access control and in role based access control you could like have multiple organizations within your cortex you can have for example I say I'm a large t-shirt I have operations in North America I

have in Europe and in Asia and say okay I have my North American c-cert which has let's say a coda of virus of what it can make 500 queries per day to virus total and for Asia I can give him like 204 Europe 300 and I can like for each organization I have my specific configuration but I can also diverge caching for example I can say if you receive the same request for the same observer ball for the same analyzer for example someone trying to analyze an IP address in Asia using virus Odle and one minute later someone from North America doing the same thing you just served them the report from the test they is

they donate my pants fresh data so you can save up your money in your current recorders and it can be called of course from the hive and from different tips and miss for example if you are using a SERP called the MS tool which is a commercial alternative it has already in the configurational cortex so it can leverage cortex to make queries in enrichment and you can also like if you are still here is still like doing like you know command line scripts and things like that you can just call cortex for P which is a Python library and cook your own script in order to query cortex and do response and analysis and it has also

web UI but the way the way is mostly used to configure things and to do quick assessments of observable it's not really used for outside scale production environment as for the workflow so everything in behive is a taste so an investigation equals a case you can create the case out from scratch bit kit also like clear case out of complex you can have multiple top plates that can import an export from multiple needs already hard instances let's take an example you can have a fish in top plate you have a math spam campaign template you can have a deed of template and so on and you can create cases or in the situations out of alerts you receive

directly in the platform so either fed by alert Cedars or directly from Miss as Miss events once you start a case you create tasks for example you have a task called identification at Aston communication with my constituency a task reverse-engineering the malware tests out save recovery or lessons learned and so on you can group those tests we touch the rope so for example let's say I have a group called communication and I have a task publication internal communication another task external communication and so on and at least can contain 0 or multiple observable so ops overall IP address URL file halves and so on that you can analyze using 101 analyzers and shortly in a few weeks

more through cortex and you can customize outsid analyze results using report on tests so we supply report on pace for dogs 101 analyzers but you can customize your own yeah just like bootstrap code that you can change to your liking and basically report a place of two sorts you have the long report we will show an example shortly and a short report so the short report just give you an idea it's like a quick assessment and has a colorful microwave dangerous blow each informational and Excel etc and and again while analyzers are specific to observable so for analyzing observables responders can be applied at any level so at the alert level at the cave level

at the task lover but also the observable level so imagine for example in a task you write something like block IP address blah blah blah and you have a responder that it can be applied to a task log so just click on the responder and we understand which is within the task load and it will perform the action and report back to you whether it's successful or not we can have also have for example a responder at the other cleaver so you receive alerts look for example I use your report institutions email you preview the alert or shoot another scam so you don't want to create a case for it works on something just say to your user in your e so you don't

have to write anything you just like click on the bottom of the responder and we respond automatically to the user so like hey forget forget about me today it just a scam don't bother me so as for the integration of the tree we can think about that as a triangle so on top you have D hive which will receive others from different sources for example you have cyber threat intelligence providers seems emails as I said suspicious emails coming from your constituency for example in our team we have developed that plug-in in Outlook where users can just like when they spot a suspicious email they highlight the email click on the bottom the body sent that to a

mailbox where we have a script that's corrupts the mailbox of Python scripts and any new email it will lock packages as another and send it to the hive so from there we can do our analysis and everything respond in beehive and just click on a responder respond back to division so we don't have even to see emails again we do everything from behind and once you have like I would say the alerts and you start to win cases analysis so you can as I said we leverage cortex to do your analysis on a respond and if you have missed you can plug your miss or enemies if you have access to would be hype chill receive if it's an export also

cases to share back with your peers in partners and misspent cortex can't talk to each other so cortex can analyze like has analyzers to query Miss and Miss can leverage also cortex to enrich the data set one feature we had in cortex one that is not currently we we remove it it from cortex two because there was almost no one using it cortex had the ability to use all the miss extension models so that's we have dropped that from cortex to but if the community wants it again we can put it there once with some motion of synapse so as I said like there are figures there are some open source feeders out there for example

there is a one we created called digital shadows to TX which if you are a digital shadows customer you can just plug your incident Locker the threat intelligence incidents from digital shadows right into D hive if you are a fire inside customer you can do as well it's lvu circular desert in Italy who created an art theater for that as well there is zero Fox 2 th in others that we have a special I would say theater called a synapse which is kind of Mayer theater the idea it's provided by us is we'd like people to like we provided a framework to cortex to ease up development of analyzers that's what we have one in one today and we are trying to do the

same with other feeders so in synapse you can just design I would say your input today it's support exchange in office 365 but also clear rather as a sip and so you add your inputs you have a workflow or design it within synapse you say ok when I get this I do that like it is then did that and then it has the connector 2d hive in order to feed it others and as for events so when you receive events from this they will show up in the other explain of the hive I like this I hope you can read a bit the the slides but I will share them later publicly so you see them like this and

you have several icons on the right hand side where you can preview the others and decide what to do with them so here for example it will give you one we have the description we have the different miss stats the different type of observers that are within the the alert and before importing it we have to create a case complain so for example here we have a case template probably you can throw it with it called miss event with different tasks so for example in different groups we have identification group called containing a task called initial assessment then in depth analysis we have another task group called communication internal then Pearson partners and other and so on you

can add like security TLP PAP which is permissible action protocol so permissible action protocol is basically what you are allowed to do or not window veneer investigation so example you are suspecting you are under the attack by a as I said threat actor you shouldn't be doing action on your network that would give the tip them off that you are looking after them so we got do passive actions so you can sit the PAP as read it has the same color code as TLP traffic light portable and basically if you design things well on your analyzers and responders they will not work if you are doing something that's active on your torso Hey this action is only for pap amber

and below you are in pap red you cannot allow do that GOP is also important for analyzers for example you can set viral file submission to GOP Green and if you have an observer that's GOP amber so you should not be sharing it publicly so if you submit it to by our subtle the attacker even with anyone watching a verse Allah will see that you have shared that so they might change the tools tactics and procedures so basically if you have the virustotal analyzer set to TLT green when you try to run it or not tell the ender of servo it will not work so it's like I said a safe word so once we have created your

Miss event template once you import the alert it will just give you import other tests and different complaint formats but you can overwrite that to your liking so once we have imported the alert it become a case like this with the different tasks and so on I have like the observables for you one in this example 10 different tasks I need to work on you can see on the right side the real-time stream which is like the Twitter feed you can just open like this feed in a specific window so you don't look you have any more and just have this little window in your big screen and whenever new things happening they will work as in Twitter you see a

timeline and you said hey this is a task for I would say I don't know analyzing this malware I am free I'm not doing anything now so I just click on it to start the task and everyone will see that so that's to ease up collaboration and if you see a lot several bottles up there like for example you can merge cases for example you start to investigation and down the road order is relevant to the same thing so you just take case want east to and merge them as a new case to collaborate you can like share those cases like a share one because it's already coming from a Miss instance you can share it to other miss instead as well

so the counter will increase and you have the responders that apply at the case level so here decide you observe walls so you can like I have statistics in filters and so on and for example here we have so someone on the Miss instance I'm connected to has updated and evened with new things and I've already were started working in the case so you don't have to do anything you see that little heater and icon there so they have added new observables so you receive them automatically in your case you don't have to do anything so once I have my observers I can move on to analysis for example here and I have an IP address

and I had a little earlier mean I've already seen it in another in a different case so I just select the analyzers I want to run to for example here on this shield lookup and also a virus or gate report and then I can see like in the Twitter feed Twitter like feed that I've started the analysis and once the job has finished it it will show up there and I see all the mini reports showing up but here the digital score is very high and but VT does not know anything about it so when I go to the say longer report of dish it I can see there there is a show of certains

because it has found new observables so if I go there I see there is an autonomous system I saw an email address so I just click on that and import them again in my case so it's like enrichment poutine and so on as again the interaction between is in D hive is like as I said whenever you created a case out of a miss event and that miss event is being updated you would receive the new attributes in the miss even automatically in your case by default every one one hour that you can and for example down the road you are working on your case and you discover new indicators of compromise you said okay that's that's cool lets me share

them back with the original order of the mists event as I would say talk and upgrade later and a collaboration between teams in incident response so probably you will not have the rights to write to that miss event because you are not in the same organization as the other end so that's not a problem for the height it will figure out that I need to create an extended event so extended events are nifty feature of mists which allows you to extend an event out of an existing one by adding new data to it even if the original event does not belong to you so here an example for example I was analyzing on that case and I found a new

domain name that is highly suspicious so analyze it it's using 40 word it tells me it's a malicious website and also it's there are at least three related euros in virus form so I still like that here I can say okay nifty I tagged it as I you see so it's an indicator of compromise and I share it to miss so I exported to dismiss instance I see that it has been successful and from there I can see that now I have two events so the bottom one is the original one belonging to this circle organisation and I don't have white rights on that organisation and the new one I have created which is on

top and it's basically if I look up at that event I see it's here it's extends even 485 and if I look at it again I can see that it has only one indicator of compromise or attribute this is the domain name I was investigated so it extends the original event and again this is very important so let's take again the example of someone contributing a miss event it contains 120 attributes they are not in the same organization as me so I said hey that's cool let me create a case start investigating so when I create a case I have 120 observable I move on to incident response and then the original outside order had 30 new attributes from their

beam I had 3 automatically 30 new observers in my life case I do an incident response and I found 5 observed ones from my side so this is mine I would say you know I would love to share them back so when I try to share them back they receive nothing on the other than because they are all observers they are not IOC's but when I had I will see here 3 you know I try to share so they will submit to in your attribute so the height will only share Iowa sees it won't share for example observers like your internal email addresses so it doesn't make sense it might even might be that from a confidential

confidentiality point of view and there is of course more to it so the hive can model monitor the connection health to all the myths and cortex instances it connected with so it would tell you if one connection with a Miss piece disease framing it even try to authenticate there to test the connectivity right does the same for cortex and you can have several filters to tailor your integration with miss because more often than not if you begin in this field you say hey let's let me start with finding some miss instance out there public one or or or things like that one famous one is the circle missed instance in if you have ever tried to connect to is look at at the

Miss is this off circle it is full of garbage and when you start doing that in the hive you end up like when you first synchronize the miss circle instance would be hide you might end up on one hundred thousand events and it would be swamped you call it a day and you abandon cyber security many go like grow potatoes so to avoid that we have put a different set of filters so you can this side you can only import missed event that have like ten attributes or the miss event that had are dated less than one month or you can even tag whitelist so for example anything that contained that I would include you can also

blacklist organizations so for example I say those organizations are just contributing rubbish so I just like ignore them altogether and so on so the upcoming features is so in queue to 2019 as I said we are moving away from NSTIC sure to grab Eadie we also share sightings witness and we will add support for miss objects as well and in 4.3 so this is like a road map I hadn't signed it with my blood we will add taxonomy such as a tech support so to drive the trio it's pretty easy you can like try them in combination because we now provide a combined eight training VM containing misty hide in cortex everything is connected or we can transfer them separately and

there is additional software from bless you from the hive project in this project so for example ale from the circle lead lets you luck its its analysis information leak framework lets you submit whenever you detect something send it to the height automatically we provide a set of libraries to integrate with 20 interact with the restful api of both beehive and cortex there is a large feeders as i've already mentioned and from the community there are like for example an email theater provided by deserving merchants from the science IEC there is a simple script to interact to bridge clear runner with the d height if you don't want to use sign ups and as i said once telecom has a set of analyzers

we can Ingo so that's it for me thank you for interesting they have I don't know if you have have time for questions any question lady no okay thank you