Two Notify All

thank you for having us besides Augusta we'd like to start off with a question do any of you want some real housewives of Silicon Valley no great well our presentation is nothing like that so we're gonna start off I'm have my friend here mr. Daniel and immitis start us off if you have any questions feel free to ask because your microphones aren't being recorded if you ask there's a question we're going to have to repeat it so everyone can hear it it's not that we're slow or that we're just slow right alright thank you like slide deck thank you everyone for coming out this morning first off can everyone in the back hear me just a quick check cool awesome

thanks guys so as I stated previously my name is Danny this is David and we we're first Isco fellows so for those of you who are not familiar with Cisco belts we are part of the security business group within Cisco so I'm sure many of you guys are familiar with Cisco where you know kind of things for routers we make endpoint a VA we do email Tomah tree we have an endpoint antivirus engine we obviously come out with snort we just came out looks north three I think two weeks ago so that's really exciting yay for multi-threading and we have a number of security products like that so as a palace member extend of our job to

feed through all these different limit resources such as these to help find some adversaries and threats to various networks that's kind of one of our really cool things about Cisco is we have so many different products from the Rocky wireless access points the endpoints to the firewalls it kind of gives us a really cool viewpoint into certain networks if we have all these things in place and that allows us to be able to comb through all these so trees and kind of string together some of these attacks a lot easier than traditional computer security products which may only have an antivirus type function or a firewall or just email symmetry so as you can see we have a

number of things we poor all this analysis to make some really cool reports and some really cool findings so we can tell you about now but first before we do that we have some giveaways and we give ask a question and give away an item does anyone know or can anyone guess what technology none of these none of these socks were none of these cases while and today I have to do with I'm going to give you a clue it's the opposite of Wired who's the first one to say that who's the first one to say Wireless raise your hand you sir you win some stuff a wireless indoor panel congratulations who are who are the

sponsors of these items we would like to thank them we think you're a luminaries unnamed company one so let's start with VP and filter if you haven't heard of it where were you it was all the rage in May it was such a rage I had to spend half my May in the Ukraine well that sounds like fun it really really was so a VPN filter was a piece of malware that we discovered on Soho routers Soho being small office/home office routers and Danny will walk you through kind of the the workflow of this interesting and multicolored diagram yes so for those me who are unfamiliar with it this was an advanced piece of malware that was

actually a little different and it was targeting these Soho routers I won't array since we believe this was so particularly effective is because as you guys know most networks don't actually monitor these inch devices there is no antivirus for your router there's really knows a precedent for people to start checking these routers as long as the experience what's supposed to do they kind of just seem to forget about them and they way in a background but the very important thing is that they route all of your traffic so if you were to hypothetically compromised this device as was done in this case they would have unfettered access to everything I think some cases they really actually able to use this to

move into the network picking add additional access to like workstations servers and things like that so as previously saying here's kind of the work pool we came up with um this was well wait wait we didn't come up with this the adversary came up with it we just documented it was just yeah we wrote about so we originally found our sample actually on virustotal at the time it was hanging up there I think for what a year or two and it was just never it wasn't being flagged by anyone so they just assumed it was benign and then once we start analyzing this it seemed that they were exploiting these routers think some sort of automated fashion to

this day we still actually know how they were being compromised but just based off the breadth we're assuming that they could be doing something like just scanning for default admin credentials through something like show then there was a number of vulnerabilities existed for these types of routers such as micro Tech and as we kind of mentioned before people seem to forget about these edge devices so they're extremely vulnerable and after never patched it presents an opportunity in the weakness taking entry to that network so once this router KK's exploited through some sort of unknown means and then exhaust is first name and this first stage is extremely persistent and can live past reboot so even if

something were to go down or if it gets out it can spend right back up again and regain exit Network once actually starts there and then choice and findings command and control server and in order to get a sucking stage implant downloaded so one of the things that was actually really unique about this is that it makes a request actually to fill the bucket and actually downloads an image so if you were a kind of a network admin you make say maybe it's just one of my employees who's kind of you know doing some kind of social media stuff or it's just maybe if it's from a peer department that could be legitimate your purpose and it would never really go

recognized and then once it downloads that image and the EXIF file data they actually have an IP address embedded in there so it looks like a normal image it's something that would ever raise suspicion but within that metadata excellent they were then able to obtain that IP address if that were unable to work for some reason maybe they have some filtering in place to block these social media sites and then has a hard-coded IP address they can try to communicate to in order to download that sucking stage that stage that was hard-coded was call to know all da so that would then allow them to find when after a big servers and then it would then try it down with

a second sanction implant which would thank communicate via tour and then would have a number of gangs and they've x-ray VAR some of them so let him talk about that but before we do that the title of our talk today is called to notify all is actual joke from the c2 domain to know all which it's nothing when we uh when we took a look at it and it's kind of brazen for a for an adversary to have a domain like to know all has a c2 domain I mean it's kind of its kind of ballsy we we thought it was a joke at first turns out it was a joke so we we named our talk to notify all as

an Amash to that but some of the modules that could be downloaded by a second and third stage involved a lot of man-in-the-middle activities particular particularly the ability to apply it as a cell strip and force us to sell downgrading on most websites or going through these routers so even if you thought you were connecting to a legitimate secure site it would force to download and then be able to inject a javascript payload into your response causing the adversary to be able to exploit a router another tool that was used in place was a binary replacement tool so if it looks like it was downloading anything that had an exe of any type of binary it could actually

replace it with a Trojan version that would then execute when it got to the victim machine so these were all about compromising lateral movement in these small and medium-sized networks so this is obviously very bad so we wanted what we were hoping to do here we're kind of uh kind of give you a bunch of short overviews of things that you may have heard about and press but talk a little bit about things that you not have known for instance if you Google VPN filter and look in May of 2017 you'll find there's a lot of people who have cue naps QNAP sans devices that were talking about it repeatedly and they thought it was a crypto minor the

footprints of BPM filter had been pretty much available for three years but nobody ever actually took time to look at or put it together what it actually meant and that was something that actually completely shocked me when we looked at the scope and the depth of this attack how people had talked about it for years but nobody said oh this should we should look into this specifically everybody assumed it was part of a mirai botnet turns out it wasn't so that is BPM filter also as Jenny mentioned a little earlier one of our biggest problems with this was there's not a lot of good forensics capabilities for routers meaning that if you found find a router and you lose

power on it the first the second stage would just go away which music we'd really have to take a take heart router apart and basically solder JTAG on and download the firmware download the the memory while it was running be able to analyze the image and it's a point we even had a plan to play power cable and splice or funding generator a running battery into it so because I'm plug it from the wall and it would stay stay up and we'd be able to move it back to our office for further analysis we never actually had to do that but we did have a plan for it and if you want to take a

look real quick these are the countries that were affected by BPM filter I'm not sure if you're a fan of geopolitics I know I'm not but it looks like there are certain certain places this is get hit more in other words for instance it doesn't look like Panama really is that affected kind of a little bit maybe but not right Huber's got no VPN filter that's strange let's just stare at this for a second ponder what this could mean well we're dumped Andre so are part of the are part of this investigation was to find clients that were infected by this and this actually I when I was originally giving this assignment said no problem easy peasy

we'll have them all for you tomorrow turns out it's a lot harder than we thought so we actually had a brilliant data scientist named Nate Winslow who came up with a way of doing a time frequency analysis and mapping the behavior to decibels so on your x-axis here though that's a logarithmic lookup in decibel with frequency along the bottom of time so we were able to build and if you look at this you can kind of see a rising line that sticks out right in the middle throughout throughout the site this is how we were able to figure most most clients using not only Cisco internal telemetry for partner telemetry and third-party feet as well we were

able to put together victim list of machines that were infected with it and we would see repeated patterns like a machine we were sure would be compromised with like the states - of this attack it would power cycle for some reason whatever reason it seems about every three weeks a device with an aunt with power cycles so you know you you would get used to it going off and coming back on now when it came back on it would act like it was stage one for a little while in some cases as little as two hours in some cases as much as four days before being promoted back to a second stage attack and this is what

allowed us this this graph is what really was a breakthrough so we were able to take time frequency analysis which you don't hear a lot inside will work the guy that used the guy that did it for us is a geologist it has his PhD in geology and he loves time frequency analysis so here we go we got to use it in cyber so just a hand that that's kind of another technique that we use really effectively with this campaign but could be used for other companies or other researchers is that what we were trying to measure is to frequency and the rate at which something would make a note so if something baking's out every

looks 8:30 sucking or maybe except every 6 hours it kind of get a map of this always occurs at this time tone and you can kind of run some analysis a case or any other domains that we see are being requested and take seem frequency or with the next same kind of timeframe like that and if so could that potentially be maybe a similar domain excuse in another operation by the next same group or it could have maybe be some sort of backup domain to kind of help unravel more of the mystery behind it in this discover for additional infrastructure any questions anything you ever wanted to ask about VP and filter but never had people to ask we're

right here I was hoping would be you oh I think what's not a picnic sniffer we believe we believe that our instrument and point distribution capabilities and models that we don't have the code for yes sir our JTAG interface is usually disabled until so routers it depends on the router sometimes if you get lucky and it's a strangely manufactured router put together by spare parts it probably isn't but it's one of those things where if you've hit a brick wall it's this is only only thing you have to move forward so you hope they don't

so the question was if we know the purpose or kind of intent behind this particular campaign if it was financially motivated and it was motivated by maybe some sort of espionage for a nation-state oh wait paper thanks thank you yes we believe based on the code that we found in the malware we found we believed it based on the amount of effort that the adversary went to confuse attackers we also believe based on a second and secondary and tertiary follow-up modules that this was a an implant designed for espionage so think about that 500,000 small home routers compromised for an espionage like adversary which would mean why attack the company if you can just grab

people's credentials when they're at home yes sir

the question is when we found the type map was there any reason that it seems to be heavily focused around Italy and Greece I to point out again we did not corroborate with the adversary on this so unfortunately any sort of thing about you know move us behind that would just be our speculation and what we've talked about my expect lane just because but we are not being a Masseria we do not know what you know what they were trying to do now if you wanted to like get a beer later and hear our wild theories right there's this whole thing about aliens right but that doesn't seem to have any any supporting evidence whatsoever I'd

point out the aliens theory has no supporting evidence however it doesn't snuffing before seeing it

that's funny you say that that is exactly what we didn't that was kind of the point we were we were going for here it seemed like they were attacking everybody but broadband infrastructure there's a were question I think so last one so believe so when we couldn't think some tree strokes we believe it was associated with an OVH server campaign in France if you guys see connections to an OVH server that uses a let's encrypt certificate you should be weary and that's just in general that's got another thing that's not just for this case okay all right so we're gonna give away you something else now who knows what sicko smart install this alright well I'm gonna have to ask

the simple question then who knows that cisco is all right cisco makes an operating system what is it called iOS who this is the first person somewhere over here you win some blue team handbooks because if you have cisco gear in your environment you should probably know about blue TV i don't think that's a good advertisement but and then what's not nora spider company oh it's not endorsed weather hopefully i'm sorry all right so what is this goes french salt I gotta be honest I've been in this business for 20 years right I've been doing security for as long as I've been able to drink beer and if you told me 20 years ago Cisco would have this thing

where you could basically deploy it once it was set up all of your Cisco equipment are you walking in or touching anything or even knowing passwords I would be horrified and I would be shocked and the younger somewhat more malicious version myself wouldn't not be able I could not continue myself because I would then be able to set up like a quake server not quake 3 we're talking about the original quake right I could not wait for that however turns out that for some reason we have his capability if you can read the set up there but it's a plug-and-play configuration and in the image management which means that hey if you're a really good attacker and you'd

like to backdoor this iOS image you can do that and it will get deployed to everywhere on your aasa or ASR 9000 and it would be almost impossible for anyone to find it so it's a art install plug-and-play configuration image management provides zero touch and as a security person to phrase zero touch should make you upset because zero touch also it synonymous with things like password lists default login all right it's no touch it's not good to touch deployment for new switches so we're not just talking about well I like what mikrotik fire loss here or Windows boxes you should take off the network we're now talking about Cisco switches that there's not a lot of good

for instance information in the world about them you pretty much have to call us to do forensics and also no one knows what they look like when they're doing bad things we have so many protocols on our routers and you're like oh god it looks sending out something weird on and you need to keep for it no no that's that's cisco discovery protocol that's that supposed to be there and everything you know is that this affects also the routers so well does me go after the switches some people you have no sense behind your firewall so it couldn't be a little bit safer but if you have a big Cisco router working at the ISP level that

could also be vulnerable to the same type of attack and as you guys know if you start going after ISPs and things like that it can cause a lot of damage very very quickly this is my favorite sentence of the entire thing you can ship a switch to a location place it in the network powered on but no configuration required on the device now I don't know about you but for a long time I did penetration testing where I would do things like ship switches to places and give instructions for people on how to plug them in claiming I was from their supplier and they would do it but it would I would always get burned

on the having to provide home support right like I would have to walk them through iOS CLI stuff it's not fun but then you bring something like this and I literally feel like I'm the GRU I can just ship something to you you plug it in I only just there's no that in between stuff so if we haven't if we really haven't sold it here Cisco smart Saul is a big deal for us at Cisco right there's all kinds of capabilities you can do with it and you can read them but some of the things we've seen we have literally seen people log into routers change running config change their boot images change the configure script with

a script though TFTP down and config and the scary part about this is we didn't have any prior knowledge of this we literally just went through showed on to find these and the rule of thumb is if it's on showed on its exploitable say it with me if it's on showed on its exploit on that's really what we'd like you to take away from this talk how did it begin so we'd like to talk we'd like to talk about sexy stuff like hard reverse-engineered like you know bloody in the network knife fight with an adversary that we spend a lot of time on this what's none of that we have a partner go hey do you know what these IP

addresses are we were like no they're like there's Cisco smart install devices out on the internet do you know what Cisco smart install is and I was like yeah I mean it's a smart way to install so won't we feature lovely feature they then briefed us on what was being done with these are like oh dear oh dear we have to tell somebody so callous itself doesn't handle these types of vulnerabilities we have to go to a group called Peter core a product security group who handles all of Cisco's product security vulnerabilities so obviously we walked in a very respectful manner to Peter and by that I mean we ran with hands in the air

but screaming thank you anything strange I would but I'm just not feeling I'm not feeling manly enough for that okay well so this started in 2017 and peace sort did an advisory on it and they're like oh this is bad like Capital bad you shouldn't you shouldn't you shouldn't run this anymore so this guy Matt only if you if you don't follow him on Twitter but do like rage fests every once in a while this is a great guys fall we try to get a picture of him green raging about this but he refused service picture taken for this presentation but he even at the point and remember all of us work for Cisco so he was like come on this is

they turned this off this isn't a big deal and then a bunch of new stories came out about how there were people with default route in Cisco and you would expect us to go no no what default route it's just some miscommunication we went dead away and went it's worse than the fault route there's a lot of things you can do with this you can't do with route turn it off right now this is kind of a big case we went the head was I'm not sure if you guys heard about this one but actually this happened in Russia I believe or if they actually had a number of Cisco routers there was a big I subpoenaed unnamed country expands

nine time zones and they love to use Cisco routers as some of their backbone ISPs as a result we woke up one morning and we discovered a number of Cisco routers are being exploited via this picture that was embedded in some of the products as a result they were then able to gain access to a number of these devices and they basically deleted their router config files for those of you familiar with that they basically renders the router completely inoperable and this was a very big deal and that this was something that we were kind of yelling into the void and I know this is sometimes other people you know who are doing open source works discovered this

really krish critical vulnerability and you're trying to tell people about this the whole series of this and that if you don't do this it's going to be the end of times and he's just falls upon deaf ears and so someone finally hears you thanks the wrong audience and then what they do is they what they used its owner ability or this feature if you will and they can use this to basically shut down internet routing for an entire country for I think what was that half a day a day where there was just you were not able to access the Internet you were not able to do anything and then so you can imagine if you're some sort of business

and you have a downtime of 24 hours that can cause a substantial amount of money and this was something we're thinking you can fix it they had to try colder ISP but then the ice pay start getting flooded with phone calls about saying hey my Internet's not working and it's just perfection wakes us really back cycle and the worst part was we would get our outreach team would talk to reporters about this and we'll get described as a vulnerability as if there was a patch coming well it's protocol abuse there was no patch coming this is this is exactly what some engineer we have not met but we're currently looking for attended to have happen so while a

lot of people were waiting for a patch to make their smart install work well we kept telling them you have to turn it off there is no working well and active exploitation do you oh by the way the gentleman that's tweeting in the upper right hand corner Omar Santos is one of the leads in our product security group so if Omar says something is bad by definition a bad thing any questions about smart install

the question is if it was so easy to use zero touch why would this one develop that capability allowed to fall into wrong hands it was developed to work on intranet devices you were never supposed to have these devices facing the Internet Capitol internet it turns out a lot of people were even against our best guidance it was kind of a not best-case scenario you know how like a lot of people want to push best use cases this was a not best use case scenario so I think the original idea was that sometimes you have these big companies and they have understaffed IT department store they have people who aren't as familiar with proprietary cisco command line

interfaces or things like that something wanted to provide some sort of solution that they can give to the a professor where he or she can things just plug in that router and in about five minutes they can get that up and running and it wasn't again this would we're great if you're on the intranet you know your internal network and your hand behind a firewall again at that point you're pretty safe it was just unfortunately some people were taking this device that was developed for an internal resource and using it externally and that's where they've been presented this issue that was a great question and for that you want to get a yes certificate to Safari

Books one of my favorite blood providers they're awesome I used a long time I was training on the plane I was learning just about before you did congratulate

all right so I think we have time for maybe another story or two so the next thing we want to talk about was coin hoarder for those who are unfamiliar with this thing we're gonna do a quick overview and then we're going to tell some of the wealth of stories we also have a dance that we're going to it's an interpretive dance of how great Bitcoin know and [Applause] corne Horner was the internal name we had for an investigation that started a little over three years ago or a little it was made public about eight months ago there was a joint research effort between Talos and the gentleman and then Jeremiah O'Connor who was actually the

hero of the story and I wish he could be here with us today but he has to he's off being awesome with Bitcoin elsewhere he he actually brought me this investigation that says hey check this out I wrote a machine learning tool that will take domains and look for phishing variants of them like you know Apple comm right it would be a three and seven P so you ap 3le comm so fishing where specifically well on me I can't spell anyway yeah but so we were trying to figure out exactly what they were doing and we watched these guys and we have we have partners all over the world one of the partners we work with you Ukraine

National Police and their cyber unit and they they heard about this and true story at the time it was not illegal to steal Bitcoin in the Ukraine there was no law yes Bitcoin that because it wasn't it wasn't a real thing let's never I was never developed before yeah we cared about TV because I I'm old and I still don't get it so this is one of the funny things is at the time we actually released this report I think it was actually you know February there was a famous story about the Bitcoin pizza and for those who are familiar with it some of the original Bitcoin people will reminisce about the good days when they

had all these bitcoins but they were essentially worthless and one of the funny stories is at one point some guy actually bought two Papa John's pizzas for 10,000 bitcoins so at the time he thought this was a really good deal because he and think this is going anywhere and he's like I might as well cut my losses and see what I can get now fast forward to to mean where you guys know a Bitcoin was worth what approximately sixteen thousand dollars per Bitcoin times ten thousand so there is internet being internet decided it'd be really funny if they made fun of this guy and create a Twitter account and they tweeted up a value of those two

peaks every single day and this was the value of those Peaks us on a day that we released our reports yes hey hope you enjoy those Peaks us how good could they be wait we don't have Papa John's fans here so this was a previously mentioned they developed these algorithm to help look for these people squatter domains - what would look like a legitimate service and originally the idea was can we use this on some of our products such as endpoints to look for weird connections to a URL can we run this over some of our emails on the tree to see hey can we use this to potentially find you know these spearfishing domains and block

them before the user ever has a chance to click on them and then what we do is we kind of just have one of our internal clusters and we said all right looks to turn this algorithm on and see what finds and it'll kind of give us our output into another directory so we're kind of like what that run in the background I think for a little bit and we came back and we saw that it was hitting on these weird stuff in Google and we're like whoa that's weird Google's clearly correct and we were kind of you know wondering what could possibly happen and then we noticed these ad works and this is where they

were actually purchasing so for those of you who are unfamiliar with a Google ad works I can do a quick to an overview so I'm sure everyone's familiar with Google and maybe you want to order a pizza so you go into Google calm and you'll type in the word pizza and then it will give you the top results based off their ranking algorithm well one of the things you can also do is purchase what they're calling an Edward so what that does is that basically promotes your search engine or your search website to be at the very top of the page so if you were to type in pizza and I'm in Augusta and I own the local pizza shop I can try to

promote my website in order to get more business what they were doing is they were buying these works such as watching or Bitcoin exchange or things like that they would try to purchase the actual legitimate one and then different variants that could have been affected by users so when they click on this they were kind of preying on them and assuming that they were can actually look through everything they're just going to say okay top result click here and that's what they were doing and this is two of the examples were able to discover in the wild and these those are the actual ones that Jeremiah were able to discover right if you do some lookups on

it you'll see quickly registered to hundreds of thousands of hits and like less than a day that's a pretty that's an unusual indicator for us doing analysis and I should know that this was just from our open D in Santa Fe which is less than 5% of all Internet activity so just kind of keep that in mind that this is less than 5% and that was still Andy said that wrong it's up to as much as 5% of the internet this is what real site looks like no I said I got that wrong this is the seat this is a phishing site this is why these work so well this is the fishing site so you're

probably wondering how we were able to do this we worked with local law enforcement and we were able to watch that box very closely and we we came up with something very strange the people who are running this scam did not seem to know anything about code servers or what at one point they even hard-coded they're a home ISP IP address into a config file for a firewall rule so it made it very easy for the Ukrainian police to find them but the code itself was all automated all they needed you to do was click on one of those ads and go to one of these spun up sites that looks real and it would transfer all your wall

information out we figured some by the end of it or something close to 50 million dollars worth of Bitcoin in February Bitcoin money well it was stolen it's worth I think eighteen by eighteen to nineteen cents now but at the time Bitcoin was doing well I'm talking about the eighteen and nineteen cents it's at least $20 tipping cents so you're wondering how how could we stop this right and Cisco art we couldn't we had to go to Google and this is a result we actually got Google to stop accepting AdWords for crypto currencies while they tried to work out a solution for this at this time they don't really have a good solution they're working on it any day

now so if you guys want to collectively join me and holding my breath it will any day now be ready to go again don't hold your breath don't tell anyone someone at speech told you hold your breath waiting for Google wall die so who's heard of Olympic destroyer did it destroy your Olympics experience now I didn't really destroy him was a live experience and now it's a problem as I'm not a big fan of the Olympics misses with my normal television schedule but then Dan had actually just started working at Cisco when this happened one member of our European two members of our European outreach team were researching this and found it and that were able to publish

it quickly long before anyone else knew it was the thing and it had some basic very similar workflow to BPM filter but different targets it would inject a win logon dot exe process with either WMI or PS exec and it would be able to scan an IP subnet or do ARP lookups for other machines at it it would then compromise it and move on a little bit destroyer could do a lot of what all the other malware that we've been talking about you do it's still credentials from the browser it would steal stuff stealer stealer I'm gonna have to ask me about that I think that's supposed to be systems failure well I like stealer stealer

right that makes you think that you're gonna go to a football game with the Steelers first up like the Pats and they're just gone the Russians have stolen the Steelers pass so it would actually patch one log on and show discovery credentials it had a wiping capability much like Nietzsche where the black energy malware from 2016 and it could run PSS Zak and connect two additional boxes with it pretty standard stuff and nowadays we have a more deep belief that these are all part of a larger campaign being perpetrated by a particular actor so the one thing to know about this is it was so particularly effective because it was a worm that release in a wild traditionally when you

have these sorts of operations they actually have some sort of person or a group of people interacting with these things to try to grab some information this one that kinda exists they let go wild and then they basically just stepped away from it and let them do its thing and the next way I was just able to continuously try to go from one computer to the next computer and it was just wiping the Master Boot Record or to basically make the system inoperable and that brings us to bad rabbit now looking at this gif do you get the feeling that this is a mischievous and/or bad rabbit

yeah yeah that's this my I don't do graphics my weight okay that could we talk to the graphics we can barely do the spelling so this a very similar propagation that would create a fake file called info pub that and it would scan port 139 internally so it would get in originally via via spearfishing and once it would get a foothold it would be able to scan internally uh 4139 looking for additional credentials or opportunity to laterally move anybody know what the name of ms70 no 101 is eternal blue that's what we felt the rabbit should have been blue but apparently we don't know what we're talking about but after being able to scan for everything it could internally

it would then use a eternal blue to try to move laterally so well this is interesting we have one more thing to talk about which is actually funny and we'll be able to take questions on this after the presentation who's who's heard of ccleaner alright ccleaner who is amazing I'm going to tell you a true story that we tapped around a little bit about in the media but we never ever really were real internally we have a host-based endpoint two lamp runs on your host that looks for all this type of nasty endpoint malware okay we actually we're in the process of rolling out new functionality and you know if you've ever had to write software

functionality means bugs lots and lots of bugs so we got a piece of yeah we got binary from a partner that's like hey we're running your beta version it keeps flagging on this we looked at it it was signed correctly and it was from a vas as part of the ccleaner it's all family so we went like most normal people would alright where did our total break they spent three days and I cannot even begin to analyze how much manpower was spent on trying to figure out what about our tool was broken it turns out our tool was not broken at all it turns out our tool of sewing is exactly what was supposed to do that ccleaner binary

was indeed backdoored so you have knocked us all over with a feather and there was some really colorful language there may have been a chair thrown anything stop a whole team for working 72 straight hours to find a book a so sorrowful logic and multiple cases being escalated to us for being like why are we blanking on false positives there was any time to say there's a kerfuffle going on but it would be minor there was a bit of a misunderstanding but when we finally figured it out it was actually pretty easy to understand we were able to work with partners to get the the list of C to process the copy of the c2 server was

downloading and we found that while stage 1 was prevalent the second stage that would download you'd have to be on a specific whitelist from a specific domain and those domains were high tech companies like Cisco Microsoft you had to be from one of those countries to get the vaulted domain thankfully us being Cisco we were able to get a second piece of the malware pretty easily for additional reverse engineering so just in the future if you're writing a PT's don't don't naturally include companies that have their own built in apt research teams as people that should be able to get second stage malware downloads but that's another hearing over there the the story that i want you to take away

this is that we built the tool to flag this we deployed it but we did not trust it the first hint of problems we said oh well it must be a tool mistake and we spent 72 hours and who knows how many man hours trying to find that mistake if you don't trust the tools that you're using you shouldn't be using them and that was a lesson that we had to learn even for ourselves any questions yes ma'am

[Music]

well there's a look on the bus

does any remember the story that I've asked gave for why that happened apparently ccleaner was bought from a standalone company the integrated to have a product line and they said that the guys who did that we're now leaving to start their own new company to do the same thing and they bugged CCleaner to advertise for their new company that's so crazy I came to say it without really laughing I mean it does look like I'm laughing oh boy am i laughing but you're absolutely right ccleaner is something that traditionally could be bugged and that was around the beginning of the third day when people started asking well what if it's ccleaner and not the known good binary

that we've been but for the first day we actually had no idea really what the binary was we just knew as a binary and it was a false positive and it was until being the third day that people started going what is this and somebody said it was ccleaner and I said you know what was funny about the D&C hack the bad guys just ccleaner to clean up after themselves and that point we went oh dear I said the one that was a little exactly how it was said - oh oh dear so I was think the one thing is I don't want to harp on the best so much because any sort of large company could be

vulnerable to these types of attacks they were is just unfortunate of not having you know a proper network plan or network security plan in place at that particular moment um this kind of happened to a Cisco this could happen to a Microsoft so I wouldn't say I would inherently distrust any one particular product but the only thing I will say if you are some sort of system admin is that when you're creating kind of your gold image that you're gonna use across multiple different workstations just ask yourself is to software that the end user actually needs or it's just something that could be cut out or most people they're going to need maybe Microsoft Word or they're gonna need

some certain you know outlook products but I picked something like this that you really don't need on there you shouldn't be having it on your corporate network or on your corporate workstation I think they expend the best way to think about things and that goes across multiple things in multiple different products yes we actually saw that technique used earlier that year in the me doc Nietzsche case where for three hours the update Center server for the me doc accounting software was redirected to a box in OVH with a let's encrypt certificate so if you're following along at home great OVH and Nana let secret are not your friends we got five minutes left she wants to ask

any crazy thoughts anybody want to hear about what I think about what's in the area 51 no area 51 is great they have bad cheeseburgers no thing 60 links he loves handling I don't want to say it's aliens but but any other questions comments suggestions will also be available right after this talk if you want to talk to us on one oh yes sir

but you got to understand the question was what would you do about applications in the future to keep him safe right and that is an actual great question and here's the problem and both of the ccleaner and the Medoc case the application that was distributed were good it was the update functionality that pushed down a bad update for both of them so if you're looking to keep a manifest of well tested and known good binaries by checksum your that's only as useful as a still a new update comes out personally I'm a paranoid person so I would disable auto update on applications in general I would have a isolated sandbox like environment where updates are downloaded applied and

tested I wouldn't just test for operations I would test for suspicious Network behavior you're looking for anomalies Network behavior after that after that passes a soaking period bundle it and push it via whatever update message you have for your organization and everybody there's so many different update methodologies you know take your pick but I would make sure that I am the one distributing the auto-update not an external vendor once you have somebody that can go from your box through your perimeter defense grab a piece of code download locally and execute it you've created basically your own back door there don't create your own back door so I think next what we would like to do in every particular

ideal case however I wasn't with most doors and businesses or you don't really get to have that look safe so I think is try to do your best to work with your management with your CTO a traffic policy in place to say hey if we want to start any additional software you need to be known that there is a potential security risk there's always a greater exposure because of Excel anyway what we're trying to uh say that if your boss comes in and tells you you need to install this right now 99% of the system admins are gonna say okay cool may expose things where we kind of want to our bus or at least I won't do my best

to protect some of these newer or level people so that way infant those fall down you're not the one holding the bag you can say well my management told me to do so and at least done you know you're not the one who is stuck at the end thank you for coming like I said will be available right outside the door for additional questions comments if you just want to talk about other stuff we'll be right outside right outside the door thank you very much for coming and we love feedback [Applause]