Threat Intel Analysis of Ukrainians Power Grid Hack

that's my voice okay hi everyone welcome my name is neo Yoshi my talk is black energy power it's gonna be a story about a power grid hack and kind of going step by step anyone likes stories okay most of you if not all of you so that's great before we start just a little bit about myself I'm originally from Israel I used to serve in the Israeli army in the intelligence course I cannot tell you much more about it or I have to kill you but I moved here to the u.s. 15 years ago and worked for multiple vendors specifically in user behavior analytics and threat intelligence I also worked in a SCADA networks which is relevant for a

story today just by a raise of a hand which one of you is actively working in threat intelligence which one of you has threat intelligence teams within their organization okay some of you all right so that that's a good audience for me to kind of hopefully help you learn one or two but threat intelligence so we'll start with the acronyms little quiz what I of C stands for one two or three or four indicators of compromised we're gonna run it fast kill chain which one is the second phase of the kill chain anyone weaponization TTP stands for technique techniques exactly technique technician procedures which one is not a threat Intel feed that's kind of hard but

Splunk I will speak about that and last question HMI stands for [Music] sorry number two correct by the way we have giveaways at the end those with the right answers will definitely get it but so are the others so you're welcome to care okay so what we're going to do today is I'm gonna tell you a story I'll go over a little bit on those acronyms for those who are not familiar with them I'll talk about a term called pyramid of pain and tools used to investigate threat attacks and then we'll go over specifically the Ukraine power grid hack step-by-step how it was done getting to the technical details and then talk about some conclusions who might be

behind this hack and answer any questions you have by the way I'd love to do those talks interactively if you have any questions during the talk just raise your hand and ask or comment or whatever so indicators of compromised indicators of compromised simply put our artifact within the network either network or host that mean being somehow suggesting there there's an intrusion so few examples of indicators of compromised IP addresses URLs emails all of those things can help us kind of put the puzzle together and understand the attack who is behind the attack and where are we within the kill chain now indicators of compromise are not associated to any specific security attack model but here you can see a the

kill chain model as an example killed kill chain model has multiple steps that attackers will go in order to successfully execute their hack each one of those indicators can be associated and collected either by computers or human being to the kill chain so for example IP address could be related to command and control server md5 could be the hash value of the malware email address could be related to the delivery method and so on and so forth and so those help us to understand the kill chain and I'll show you in my talk today a specific incident and how we can work the the kill chain using the indicators pyramid of pain by the way I heard it in

some other talk someone was calling pure love pain why does what is pyramid pyramid of pain is trying to aggregate all those indicators okay and put them into kind of buckets the bottom are hash values IP addresses they are relevantly relatively sorry simple indicators you go higher the pyramid you get into domains and infrastructure based search our network and host those are more sophisticated and at the top you have tools like malware pieces and TTP like I mentioned earlier standing for tactics techniques and procedures which is basically a behavioral indicators those are trying to identify the adversaries who are they what their motivations what's the methodology and what's the mode of operation now the reason it's

called pyramid of pain is because the higher you go on this pyramid the harder it is to provide countermeasures so for example hash values it's relatively easy to block too they're using endpoints you can create signatures for specific md5 but it's also easy for the adversary to change a house hash value of a malware just by adding a little bit of a code into it right same thing with IP address IP addresses are relatively easy to change for the attackers but also relatively to block for the defense ur if you go higher it's harder and you go into tools and TTP's it's much harder for the attackers to change the actual malware they using but it's also very

hard for us the vendors to identify those pieces and that's why they call it the pyramid of pain it's painful the more you get higher in the pyramid so like in any other investigation and by the way I like to I like to see cyber investing creations just like any other crime investigation like a murder case you have a detective you have evidence you have a suspect and you're trying to find who murdered well where's the body and who who is the the killer and so the tools were using obviously as you know sand boxes for malware network analyzer to identify lateral movements and and communication back into situ and any other observables such as password

keywords the alleles configuration file registries keys all of those helping kind of to put all those pieces together and solve the mystery now if you don't have all the information you can use what we call enrichment tools what's enrichment tools enrichment tools are external repositories like virustotal and others that already have some history related to a piece of malware or an IP address and you can add information and get more context around the evidence that you find on the scene log files log files from your environment can help you cross correlate the information from the outside sources and outside sources could be vendors either commercial or open source that send you indicators that are additional

kind of an information you can look at to those indicators you gathered by yourself make sense great so let's look at this specific incident that we're going to investigate today the incident happened in Ukraine it was a power grid that the incident happened at December 23rd 2015 you can see here a brief description but three main points that I want to clarify is that this is the first time in history that someone brings down a power grid first cyber attack that creates a power outage and gets power out of citizens it's very coordinated this specific operation was three different sites going down at the same time simultaneously in addition to a call center a power grid call center that was

attacked as well and lastly the impact two hundred and twenty five thousand people lost their power so let's start going down the pyramid of pain remember starting from the top tactics what is the tactic obviously in our case is going after critical infrastructures in our case its electricity and bringing down the power the question is why and to answer that question there's another pyramid I like pyramids by the way it's called the Maslow's hierarchy of needs anyone familiar with that great so a lot of people are familiar so as some of you know this one tried to kind of match the motivation what motivates people with their needs and at the bottom of the pyramid there are the critical basic

needs of people such as food water and electricity and so obviously the attackers were after a basic need and I claim that it's not only the power but also the security and safety of people that was compromised because when you're after a power and people don't really know what's going on they're also insecure and and it's it's basically attacking their safety in order to understand the techniques how this was done I need to talk a little bit about what we call ICS which stands for industrial control systems anyone familiar with that okay great again this one is a general term for control systems in the production industry and critical infrastructure industry that includes multiple pieces

such as PLC's which stands for programmable logic controllers and art use which are remote telemetry units that are the glue between the equipment in production or in critical systems in our case it's going to be turbines breakers transformants that's the hardware piece and the computing network and so in order to take off the power it is critical to for the hackers to understand how ICS network work now HMI you remember from a little quiz stands for human machine interface and this is how it looks like basically there is an operator sitting in a power grid and has control on the turbines in the breakers they can manually turn off some of them turn on the others and basically

monitoring the process unfortunately the protocols are very proprietary within this world and multiple vendors have different protocols and different applications so it's from a security perspective very hard to monitor and make sure there's no intrusion into those systems in other terms within this kind of world is SCADA SCADA systems are the again general term for all those computers and networks behind this environment and you can see here that there are multiple layers that the attackers had to go through the starting from an external network unfortunately this one was not air gapped but there was a VPN that was about was supposed to protect the the SCADA network as well as Ethernet to serial communication line and the

attackers had to go through multiple layer before they ending up at the HMI stations which has the capability to create this power outage okay so going down the pyramid of pain we're looking at tools now and the main tool used within the Ukraine power grid attack is black energy anyone familiar with black energy yeah so black energy is a relatively old malware family that has involved evolved even through the years and the story of black energy starts 8th 2007 at that time it was a very simple Trojan specifically used for denial of service attack moving on 2010 black energy version 2 was a much more sophisticated piece of malware and that point it has capabilities for bank fraud

and spamming and then the piece we're going to investigate today is black energy 3 that's the one that has been used on the Ukraine power grid attack that specific one has a modular architecture it has plugins that he can load and each plug-in has a specific function within the kill chain and we'll go into details there also black energy 3 has multiple ways to be installed including a dropper just a simple dll or as part of an application malicious installation the second tool used in this attack is kill disk kill disk like the name suggests is main purpose is to wipe out a hard drive and the reason it has been used during this attack was

twofold first to remove any evidence in other words to kill the black energy piece and and other evidences second to render the SCADA Network unusable so the attackers idea is that after we make the attack we will make sure power cannot be restored from the SCADA Network from a host perspective the attacker demonstrated variety of capabilities there were two main methods for them to actually access the SCADA host the first one is they managed to steal credentials okay and after they stole the credentials they managed to create a VPN tunnel to the SCADA Network and at this point they can recreate a remote desktop a session just like you know you guys using a remote desktop RDP or TeamViewer

they can just see the HMI and operate whether in the operate as a remote user and the second one is they crafted a customized malicious firmware and injected it with the in the SCADA Network to one of the PLC's so they would be able to execute what they want to execute and render the SCADA Network unusable now in parallel you can see here on the top the black energy piece kept on maintaining persistence and communicated back and forth into the command and control so you see that there is you know few redundancies on their communications that they kind of have more than one backdoor they they could have used so far so good any questions all right so I need to

start speaking slow where I'm not sure I have 50 minutes talk we'll make it so now we're getting into those pieces of evidence the small ones domains IP addresses those are the basic indicators and just as an example this is an IP address taking from the site and this couple of things you can learn from an IP address first of all you can learn the geolocation so we found out that this is in the west side of Ukraine but then you can look at domain tools and find out what was the domain associated with that specific IP address at the time of the attack and it it turns out to be a donor list of domains

one of which was a govt that you a domain which suggests that this specific attacker was after Ukraine government govt for government and ua for Ukraine [Music] the other thing we can do in an IP address is to use a threat intelligence platforms and enrichment tools like like I mentioned earlier like virustotal and now we're starting to get a better understanding of what these IP addresses are associated with for example in our case part of the kill chain is the command and control you can see here kind of a circle of all the attributes black energy as expected is the Associated malware piece to that IP you can find out what historically the ports have been used with that IP address and

other attributes and related indicators this is a very interesting visual that we found on one of those variants and that specific variant found within the site was modified to support proxy servers the fact that someone modified the proxy server suggests that the attackers had some kind of a reconnaissance and some kind of an inside information on the fact that there is a proxy and even found the exact proxy URL that needs to be used now obviously proxy is essential for this piece of malware to communicate back into command and control right so again going on the kill chain we will see that this was a very thorough thought process from the ethical perspective so here's just a

list of what have been accomplished additionally to still in credential the attackers managed to create a VPN tunnel to access the HMI they even disarmed the UPS system so the internal computers won't have any backup once the power is out and eventually created the final attack so let's go over the kill chain step by step and see how the indicators help us solve the mystery for those of you who are familiar with the kill chain process this is a slight modification of the traditional kill chain it's called the ICS kill chain and it has actually two stages the first stage has step by step what what the traditional kill chain looks like and the second one is

the ICS attack itself so because of this CAD kind of portion of it there are additional steps necessary for the attackers before they can attack the the SCADA Network and by the way you can see here it took six months from the very first intrusion to the actual power outage so six months the attacker managed to stay under the radar without being discovered which is kind of scary right think about it all right so let's go stage by stage I'm starting with stage number one the attacker decided to weaponize a Microsoft Office document with some attachment you can see here this is a decoy document obviously it is in Ukraine I kind of understood that

this is some type of a political article don't speak Ukrainian but the idea is to to add this one and obviously the the embedded Mallory didn't will be a black energy dropper so the delivery is going to be spearfishing so emails are being sent into personnel within the power grid that that have that use the the same network and those guys are opening it and once they opening the attachment there are going to be encouraged to enable a macro now if they end up learning a macro this is going to compromise a and exploit the specific Microsoft vulnerability you can see here the article of this vulnerability and the way it works is that on that specific version if it's

not patched it's going to allow remote embedded pieces to the PowerPoint and you can see here on the bottom here that those pieces are coming in from an external source now obviously that that piece of information is not an image it's an executable and it's the dropper for black energy he considered the INF file manages to change the extensions from gif to gif dot exe and at this point the black energy installer has few challenges the first one is as you know windows 64-bit has a validation for digital signature process so a driver cannot be installed without a valid signature unfortunately they don't have the digital signature so what they did is the black energy is changing the boot

configuration to allow temporary signatures so at this point it can install it however when you change this there is this little text on the next to the tray that says test mode and in order to stay under the radar and make sure that this is not showing up they specific malware piece we're using is running a patch for user 32 DLL Mui which is a user linguistic patch and that allows them to change the text to whatever they want and they choose to mask it out so it's not visible now the last challenge is that a UAC access control user access control pop-up will come up unless they're using some kind of an API and they choose to

use the windows application compatibility API it's a shame that database that comes together with the with the dropper and at this point they also masking this so everything is on the background at this point the driver has one kind of a task which is to inject the DLL into a user space process specifically svchost.exe the driver doesn't have by the way any rootkits boot kits very lean which makes it very has a very hard to be detected even if you have rootkit scanners you won't find it and at this point it looks for a disabled driver available the server disabled driver replace it with the malicious one and start it add it into the registry and Windows takes care of

the rest and at this point the DLL is up and running okay we're here so the deal of the main deal itself is useless for the attackers because it has like I mentioned earlier a modular architecture he doesn't really have any functionalities and so at this point the main DLL can use a list of a library of plugins and each plug-in has different type of capabilities you can see here a few of them password stealers screenshots takers update malware's the destroy system one which is the kill disk and the upload is pretty simple the DLL itself has three kind of API calls to to call those specific plugins as necessary and here at the top you can

see the command and control servers that actually been used during the attack you can see this fool your by the way and anyone here can spot any threat or pattern between those names here yes they are all from Jun good job actually you're gonna get a shirt not only given me yes that's true apparently those hackers are our fans of Jun Jun so let's look at the specific dll used within the you korean power grid hack the first one was stealing files very important function of any Trojan file or a Trojan malware this specific one was looking for private keys and think about the VPN access that makes sense they also try to store some password and

get some system information and very important portion of the the attack was the dll that responsible for network discovery because as remember as I mentioned earlier at this point they need to start looking for the SCADA Network and start exploring how they're going to execute the state the second stage also PS exe AC who was familiar with those it's very powerful was embedded with done with that DLL which allowed remote executions okay so basically when they are establishing a foothold within the environment they're now are moving into stage 2 stage two the using the credentials in order to disable the UPS system just a nice kind of things they thought about make sure that after this operation there's no ups

usable in the environment the second thing and that probably took most of the time is developing the firmware that is very unique to those specific sites and I can tell you that between the three sites that have been hacked they had to develop at least two separate femurs because the sites themselves have been using multiple vendors and so that probably took time and probably involve them actually setting up a replication or is simulation of the same network they are packing in their environment because like you know like we do we want to create a product test it and only then deliver it to the customer right that's what they did only that this specific customer probably was not happy the

product worked as planned so they're sending the the the firmware and at this point they're ready for the attack and the attack is going to be simultaneously three things going in the same time first the using the TeamViewer in order to access remotely to the HMI and flip off the breaker now to using the kill disk in order to remove all evidence and render this data unusable and basically wiping the entire hard drive and then in addition to that they are launching a telephony denial of service which which is basically phone calls going into the power grid call center so people that lost power won't be able to call and report it back which obviously added

more chaos and panic and successfully they're doing it and where is it from Jim so the reason the reason it's from Union and the group that that eventually this whole operation relates to is called sandworm which is this huge creature from Dion that was being treated as a guard and and and the history of the group is very interesting and and by the way this is attributed to eyesight which actually this couple of search security search companies that work on that but that specific slide is coming in from eyesight and the history shows that that specific group historically is after governments and critical infrastructures which kind of makes sense but why Ukraine and and this

is kind of my interpretation actually my wife is from Russia and I'm from Israel but we know one of the we know a little bit about politics in around Europe and you probably heard about the the cream kind of crisis Russia is in the East trying to kind of pull Ukraine to to her side the West European to their side and to me this one might be related to the this crisis between Russia and Ukraine [Music] so this is a timeline of the actor and couple of things to mention here so first of all those guys are from Russia they are the one who wrote or rewrote black energy and they have managed historically to find multiple zero-days

specifically on Microsoft Office not only PowerPoint but also Microsoft Word and they also among that pockets and I think there are a hundred power gates specific $100 targets that were investigated that associated with this hacker a lot of from Paula Poland Ukraine but also some in the u.s. so no this is not from the power grid outage but I guess what I'm trying to say on those photos is that it could have been much worse all of those the the water facilities transportation gas and even nuclear plants use very similar network structures to the one that was used during the Ukraine power grid back SCADA I mentioned earlier is something that is is implemented all over and someone who

worked within the SCADA I know that those are relatively legacy systems they're using XP or sometimes even older Windows services and systems they are not being patched because those guys are obsessed about touching the system they want the production to keep on working they want the power to be on so they're really shy away from patching those things and they relatively unaware to the security I would say challenges right and and to me that's a red flag what happened here is the power outage for few hours could have been much worse [Music] so let's talk a little bit about how we can solve that so specifically as I mentioned the the specific a vulnerability found within or used

within that specific attack can be patched so this is a Microsoft security update that could have blocked that specific hole that that was used during the attack and you can see here that they also thanked eyesight for the investigation and and though today it cannot just go and execute remote oil II code or remote oil e-files however that's not the only way someone can intrude into a system right there are other vectors you can hide behind installers you can use other exploits and so just added another reference to June or you can read the subtitles if you cannot hear that

so the sleeper must awaken so I really believe in that change if you want to make if you want to improve your security posture if you want this not to happen you have to change something right that's the definition of insanity right if you keep on doing the same thing I'm gonna expect other results that's not gonna work and so you know I'm we are big advocates in in trying to help that from a threat Intel perspective specifically I came up with the list of obviously another pyramid that's the last pyramid for today I promise and I was trying to look at it you know from from a kind of a network layer perspective how we can make it

better specifically talking about that incident and and SCADA networks in general and how we can add layers of security in parallel to maybe add some policies so the first thing I want to say is that Newark has some policies that the energy specifically power grid needs to follow and the u.s. might be in a better position did that specific one incident and I think air gapping is the great solution for it but Newark is kind of it's a regulation and I'm not sure it's followed a hundred percent I'm actually sure it's not follow the hand percent everywhere behind that to some kind of an awareness and kind of self understanding of what layer's within your network are the weakest link and so

I kind of looked at it from different levels data and application level one thing about HMI it should be a very black kind of blog or set of applications that needs to be in an HMI so application whitelisting might not work everywhere I think on HMI it should work because this certain specific of application you should know should be there other applications shouldn't so that's one thing enforcement of password is obviously always important and as you saw it was part of this skill chain stealing passwords segregation of duties this is very important make sure no operation or not hm I has full control on the power grid you might want even to segregate in the network itself and make sure that

you can take off some of the networks if being infected without interfering with the other parts of the network look at the specific iosys indicators of compromised from that specific event and other events and implement patches that are relevant for those specific vulnerabilities if you have a specific CVE that we publish or any other vendors publishing use those and obviously use other indicators like IP addresses and md5 to to be used within your firewalls and your endpoints from an architecture perspective I believe a gap is something that should be considered when using those sensitive operation networks if not adding at DMZ and adding higher-level obviously the VPN and was not secure enough in that specific

incident so make sure you have extra layers of security over VPN and again I can tell you from experience that most of the vendors will ask you for a VPN to support the equipment so you know people like from Honeywell or from you know Yokogawa or any of the process automation providers wants to be able to support it and the users wants to be able to get some support from their vendors but as a as a network security guy you want to make sure that the there's few levels of multi-factor authentication and another Security's measures make sure you backup all the files remember what happened after the power outage the entire files were basically removed make sure you have a backup so

you can restore back your configurations and as I mentioned try to limit the access obviously the last part is more towards the human behavior and I'm coming also from user behavior analytics and that's most of the time is still the weakest link make sure that you train the operators the users kind of to understand what to expect in the network and also from an ir perspective create workflows that not are only relevant for your corporate network but also for your scheduled network so one of them could be this disconnection of part of the SCADA or moving fast from automatic to manual mode and finally as a security engineer or threat intelligence engineer keep on investigating for new yaaro

rules and you not rules that you can implement within your addresses and your endpoints so to summarize we spoke about the pyramid of pain and we said that it the higher you go on the pyramid the harder it is to provide countermeasures countermeasures we spoke about indicators in general and how those can help us identify the kill chain and specifically in our case the ICS kill chain on like like I always say this is just one example in Ukraine this can happen here this can happen everywhere and by the way it can happen on any type of Industry right critical infrastructure is just an example and so indicated investigation is part of a continuous process threat intelligence

needs to to have within each organization and finally make sure you implement those the sleeper must awaken so this is the resources that used during my research and that brings me to the end of my talk thank you we have time for questions any questions all okay yes please

okay so the question was how do we approach serial network based in to as as opposed to IP based that's a good question so the what what happened today is there's a trend where you basically encapsulating those serials protocols they lately legacy serial protocols and starting to putting it over IP address there are certain additional component of security that you need to do with those because you cannot keep your network with the same security measures after you move from a serial based into an IP when you add the IP you also add potential additional vectors in order to attack the scalar network and so there are specific products in the market that are aiming specifically to secure to use

issues related to what they call a SCADA over IP okay any other questions all right thank you so much Shane everyone just come here there's a couple of