ADVERSARIAL MACHINE LEARNING : THE CYLANCE CASE STUDY - Adi Ashkenazy

thanks rychelle for the intro as Michelle said this is this is the silenced case study it's a partner of adversarial and she learning or attacking machine learning it's a hobby project that we that we didn't publish and a couple of months ago and in this project what we wanted to do is to see if we can take next-gen endpoint protection product by a company called sons now part of Blackbird studies and somehow finds an interesting way to attack its core mechanism now the you know if there are any and I'm going to assume that there are lots of offensive cyber practitioners in the crowd as you know bypassing a single control is not necessarily that that difficult

unfortunately but unfortunately but in this case we wanted to target specifically the core mechanism which is machine learning right the basic kind of promise where the flag ship product is based around detection using machine learning now a bit about us and I'm gonna say us and we quite a lot in this in this talk it's not because I'm insane that's at least not the indicator my my partner in crime and colleague Z knees right there and he has a category five stage fright so I'll be taking this for the team and he turns over background it's very similar we both have a very kind of heavy offensive cyber security experience in government service up until 2016 we then spent

around three years designing and developing an automated retina solution commercially and we now run our own consultancy right here in in Sydney now it's I think at least as important if not more so to say what we're not we're not a experts we're not who she learning experts last time I touched machine learning was about 15 years ago in the uni before AI was sexy and and this is actually important for two reasons one this is a great mechanism for me to cover my ass so everything I say in this talk which is not 100% accurate there you go there you go and this is actually important because what we were trying to simulate is how people that do have

offensive cybersecurity experienced how they kind of approach those new technologies with being domain experts Colin alright so the things we're gonna cover in this talk are as follows we're gonna talk about why this is important why at least we think this is important why sounds specifically out of all potential companies I'm sure that they were not def thrilled about this research what are we looking to achieve we're gonna talk really quickly about AI and classification problems for people who understand really quickly and we're going to talk about then how we kind of approach the approach of problem try to reverse the thought process of the company it's specifically the product itself and we'll see if we found a cool

bypass for the product a bit about the publication itself on the feedback and then really really looking forward for any questions you may have now this this would kind of viral and we got lots of feedback and lots of questions from from our peers and from reporters and one of the main questions or the first questions we were asked was why why did you choose this research and my my instinct was always why not like why wouldn't you want to do something like this it's challenging it's you know curiosity and as we say the office every day you get to fire up iodized a good day so there's the challenge aspect but also I think this is extremely important

because and I've been on the other side as well of developing products there's tremendous amount of marketing dollars that gets spent to educate the market to educate decision makers that are not necessarily very well aware of the different technologies and the efficacy of different security controls and I think it's really important that the kind of independent security community also provides its own independent view of that issue and also we have a specialist a specific knack for anything that is a bit overhyped and I do believe that AI machine learning have been a bit overhyped in recent years so why why silence why on earth would we choose sauce so which is some some people posting things that with Joe sauce

because we were paid by the competitors and this was a hit piece and we timed it perfectly before the financial results the reasons were way more earthly and practical first first and foremost it's an AI centric product and we were looking for something like that second it's consistently ranks on in the next-gen products and actually the most important one was that we could actually buy it I don't know if you ever tried buying an enterprise-level product but while you are known a company that does cybersecurity or they can Google even see that you're a cyber security professional they don't want to sell you anything all right then we'll try and hide kind of security by obscurity so

the fact that we could actually buy songs on was one of the factors now a bit of a disclaimer whatever I say here does not mean that I think that's on this is a bad company or their product is not any good or anything like that it especially doesn't mean that I think that the competitors are any better and I've heard of sales people using that to sell other products that's not where we come from now I will say the same time in terms of motivations that their marketing didn't help and all these are quotes from their website they're actually using a silver bullet right there in the testimony and they call like if I take any malware and I changed

one bit they call it a zero-day attack because it's an unknown malware that's not a greatest use of the term that I would I would use myself all right so what are we looking to achieve now as I said by passing one product once that's a kind of an entry level mission or challenge for anyone who's entering the field but we were looking for kind of a Fivefinger plunge to the heart of the product we were looking to achieve a universal passive by us now this term was actually a strawberry this later but Suns came out with a with a PR they said this is actually not a universal badness now we kind of made this up this is a this term

very personal part because this it's a bit of HIPAA Sh I think but what we meant here by Universal is that as an attacker I will only have to have one type of treatment to any malicious payload in order to make it undetectable I won't have to work hard for every new payload what I mean by passive I think or kind of tree or an intuitive it just means that I don't I don't have the intention to attack a park itself I'm not gonna install it I'm not gonna you know remove the the kernel driver nothing like that I would just drop a file which also means that I don't need any high permissions of any of any kind

and hopefully I'll be able to bypass the product at the same time what we'd like to prove and I think we have them self is that any new mechanism any new mechanism that you introduce as a security control also becomes part of the attack servers and machine learning is no different and that's this becoming pictures it's very likely that you're going to see more and more things like that all right now volunteer anyone raise your hands please you're at the end what is that well done everyone please but you're not done yet what breathe you know what breathe this is what breed of a cat and I turn it for you right okay so I'm gonna make this

educated guess that most of you in the crowd we're also capable of identifying that this is a cat and the reason is you have this amazing brains that have been trained to classify animals and the way you have been trained is usually by you know your parents or carers and books and movies and they were pointing at animals of saying hey here's a fee I pay here's a dog and your brains tried try to understand what you're seeing and you started extracting all kinds of features of these animals and you got to a point where you can classify them in a very high level of accuracy this in a nutshell is is classification problems right the ability to look at two

different things and kind of give them a label out of possible options the reason you didn't know what type of breed of Ted this is is probably because you're not trained for it if you've set for maybe a few days and you look at different types of cats that it's probably something you would also be able to do now also know that you're capable of identifying this as a cat although this is a bit of funky cat in a way it still has a bit of shampoo because I didn't wash it properly that's my cat and you were able to compensate for these different problems in the image because again that is part of the capability you have it

can compensate for all kinds of features and you're still capable of calculating the end successful result of this being a cat now there are two types of activities in classification problems that are important for us to understand the first one is very very difficult that's kind of secret sauce it's lengthy it's resource intensive that is the trained part the objective of the training part is to build a brain the building ability in order to do so you need to have a large data set training set lots of cats and dogs images for example or in the case of cybersecurity and malware lots of malware lots of benign files the next phase after you have the data setting you constantly

want to refresh that is to be able to do what's called feature selection that can be done manually or automatically but you want to identify those properties that are most meaningful so now if I go to the cat and dog example it's fair having the fur is that meaningful and identifying between cats and dogs not really right both have first but if I tell you that it you know only one of them oricon can extract a property which says does it act like a douchebag that's like 100% cat right one artisan so we want to aim for those meaningful features of what we're looking at once we have that and that get is an essential part of the secret sauce we do

the training I'm not gonna go into that because first of all I'm not an expert and second there are millions of ways in which you can your training will tweeter this black magic right now and at the end I have a model the model is really just an algorithm or mathematical function which is capable of taking a subject saying cat or dog as beautifully exemplified right here by our volunteer that went home it's amazing anyway so the second activity is in itself because if acacia that is very easy to deploy you can have lots of different brains like here you can have them for example as endpoint protection products and the only thing that happens

really at that phase is you get one sample for example an image of a cat or a file you extract the features that you are previously defined as important you apply the model which is really just looking like a function or algorithm and you have a decision and confidence for example I'm 92 percent sure that this file is malware that is kind of in a nutshell the concept of classification problems and how it is applied to cyber security now I always had a problem with that and when I was told about this about I think it was about four years ago but by one of my colleagues there were two things that bug me the first

thing that bugged me was that in many offensive cyber operations you actually use legitimate false and you know there won't be no more because you don't use it but let's put an aside because because that's not the topic of the talk another thing that really bugging me is kind of the naive aspect of that a dog is really fine with being a dog a cat is super fine with being a cat he's probably on the couch looking at the dog and thinking that silly Beast you know so they're not trying to deceive anymore there's no real competition in that sense in that sense it's 100% natural but what if we took something or someone who is malicious and intelligent and has

a motivation to deceive us and that is the human operator humans have that capability not all are malicious fortunately but they can be and what it instead of cats and dogs we wanted to have birds and human beings that is our classifier and I test you as human beings to full classifier that tries to see if your Birds or human beings now for example I'd say you know I'm gonna give you $1,000 if you're able to full such a classifier that is based on image recognition it's gonna take a picture of your face from your head and it's gonna tell you if you're a bird or a person now it all depends on how well

this model has been built right what features is it looking at for example if it's only looking at the of feathers you have that safe can identify feathers and you wear something as silly as that costume you have a fighting chance of fooling the model and moving for it unless there are delays or security out there to look at other properties so with that kind of mindset let's start talking about our plan now a lot of technical people present company included have an intuition and instinct immediately installer product start you know fire not for first engineering tools and start looking in the code and trying to understand what he does and how it's an option we sometimes do it

but actually I think this is kind of something that works in our favorite it can be better so this something is what I call the offset paradox OPSEC stands for operational security or how well a company keeps its secrets so for example how well the silence protects how it doesn't from the world they wouldn't want anyone here for example to understand how their product works and if they did so their product would be very secure and if it was super secure it must sell well but the reality is otherwise right in order to sell the product they have to tell the world how it works marketing so you know sponsored sessions and booklets and brochures and

everything and anything that tells the world how well the product works and also tells how it works too people are not supposed to know there's productivity which makes them choose sometimes programming languages which are easier to reverse there's the legal department that has this crazy fetish for pennis files done like crazy from the day the company starts and all of this will work in our favor in this case and the way this is manifested is in what's called a suit which is a fancy military term for open source intelligence I call it everything you can google basically and there are few things that are really really useful in the innocent and they kind of as I said

that it's a result of the object products in my mind so parents are a personal favorite for me and spent a few hours just reading through sounds as patents and the reason I like patents is not because of the way they're written and it's horrible to read patents but they do provide you with kind of a high level understanding of what components are in a product and how they have the peripheral processes work now in addition to what you would usually see since it's dated you can see how the thought process evolved within the company itself you can actually almost here you know the co-founders and the chief scientist thinking and kind of laying out their vision for their product and

how it should work and why it's gonna solve lots of security problems so we thought tons of interesting information into patents I'm not gonna go into all of that of course but one example i didn't want to show you comes from the world of marketing as I said there's lots of content that gets pushed out there to explain why the product is so awesome and this is a smalls and in the center bottom center is a small excerpt from I be sure that you can of course find online and it gives us two kind of valuable pieces of information the first one is a name the main product or the main module that I am after in this case

it's called sounds infinity they gotta love those Dark Lord names for different products but we just been looking at this for sure I know that I'm looking for infinity that is your classifier and when I read the description I can understand also that this classifier has different models probably for different file types now if we map that to the actual folder where the files have been installed which is on the top right because two files that are kind of interesting one is called solace host infinity model that is about net assembly anyone wants to guess what what that file might be and infinity model don't worry the model amazing the model the machine learning model we may we may

be after which is in the infinity product and there's another father than its goal sample scoring to PE now I need a wild guess that the two actually stands for a version P is not pretty much just that's portable executable which means executable and another wild guess which says that this file somehow helps to score different samples that I'm trying to understand if they are malicious or benign so this is again just a small example but we constantly could use that type of information when actually looking at the code going back and forth and matching and saying oh this must be that oh that must be that and that really helped us shorten the amount of

time needed for this tremendously so now for now it's time to fire up your favorite reverse engineering software now these are all Dalton assemblies so a life was was definitely easier we did look at the drivers well but that was less interesting and as we can see we have the Silas host infinity model far from before and in the resources we can see that it has a file that's called infinity model dot model which not surprisingly enough is the encrypted model data at the same time we can find in one of the interfaces a key called about called the values I'm decrypting Sciences intellectual property I think that is there is a you know to make you

feel bad or is a warning sign in a way you have the encrypted model data and you have a key what is the next logical step the creep they're modeled with the key right so fortunately for me I have my colleague with me at that time and because this was an infinitely stupid idea and it was infinitely stupid because what are you going to see if you're gonna the Crick the moment it's just a bunch of numbers without context still going to be meaningless and hopefully not hopefully but thankfully I was prevented from going down that rabbit hole for for a couple of weeks so instead why don't we why don't we build our own classifier so we have this

powerful interfaces that are provided to us by science and what we can do we could in write this engineering masterpiece these five lines of code took us a long time and what we were actually doing in these lines of is we're taking the extracted encrypted model file and we'll have better let Silas do all the heavy lifting they're gonna do the decryption for us I'm not gonna feel bad for using that key and we're going to use that sample scoring file to actually call a function which is called from role compute score and what that function does is it takes any file that you give it and they just give you back the score now this provides us

with two two things one is less critical but it's nice it just shows you gives you a good and easy mechanism to test the score any sample with our own executable and two this is now a starting point we have a threat to pool in terms of dynamic debugging which is what we want to do we want to understand the process unless so the actual data in the model itself now before we can before we can actually do that we want to recreate the PDB so we want to extract the intermediate language and then reassemble it and there's actually an anti tampering mechanism that prevents you from from disassembling doesn't assemblies and if you can see

here there's there's one property called suppress I'll DSN now it sounds very scary it gives you that message box if you try to disassemble that the doclet assembly but apparently the way it works is there's kind of an honorary contract between the disassembler and the assembly so if you just change that property you can that that's done like you bypass this amazing thing by the way there are other software's that you can use to reverse this assembly which just don't look at the property it's foolproof and there's what we did actually we just changed one character at the similar itself so we would never have to touch this again so that was really useless I don't know why they

bother and there's another mechanism in place which is alpha station now obfuscation is annoying I think we hopefully can agree on that I think it is more problematic when you're trying to the static analysis because the code is mangled and their names are problematic and lots of go twos and all that stuff but if you have the capability to dynamically the bucket that it becomes a bit of a nuisance but not more than that so at this space we have our executable which is a starting point to debug the program we have PDP's which allows us to look at code that looks exactly like what you see on the right hand side and we can start

understanding how the hell this magic of classification with machine learning works I'm going to show you with three main functionalities and this is the first of them I'm not going to throw lots of lines of code at you I'm going to talk about what we saw and the abstract concepts that we saw because I think it will be more helpful and the first thing we're seeing is a kind of a lengthy parsing process or functionality so for every executable there's a parser the parsers role is to go through all the properties in the file and load them up in kind of a meaningful and useful way now when I say properties you can see some examples here some of them are

really straightforward linker version number of sections in the file all it's kind of all kinds of things that sounds for their own reasons after their own research thought that are interesting now as I said some are straightforward others are calculated for example entropy and there are other more elaborate properties and the relationship between property types and properties values is one too many that is an important point so with later versions for example and the volume it's one-to-one doesn't it be going to be one but with other things like string is it could be property type of strings will have many different values so that is kind of the first phase now what's also important to understand is that this is

not the features that you saw from before the way we understand it at least the properties which are human readable and I can articulate what they are they are different from the feature vector that scientists use which is actually a completely abstract concept it's an area of 7,000 elements numbers basically you can explain this as X or Y and we'll get to that well almost automatically so how do you build this feature vector how do you go from properties to those abstract features so this took an immense amount of cathain to understand like immense and what we understood at some point that we're looking at is this table of tables which converts from properties or property types to features

so for every property type that was parsed before there's a table just like the one in the middle which has different value ranges so you know if we're talking about later versions so for every potential range of values for the liquor liquor version there will be a sequence of actions that need to be performed I know this part is hard if it's hard for you just imagine imagine what it was for us so again for every property type there's a table and in this table for every potential value of the property there's a sequence of commands that need to be executed what are those commands increments and decrements of different indices within the feature vector let's see an example

so let's say I already harvested the property type CLR common language runtime doesn't matter what it is it is a property in the file let's also assume that the value was 2 so once this is parson this is now being processed we will fetch the right type type handler which is the sealer property handler the table you see in the middle and then when we actually want to treat this value we will find for example that the following these 2.0 and then there will be in this example three different operations we need to execute one is to increment index 100 in the feature vector 1 it's a decrement 200 and the final one is to increment 300 this

ladies and gentlemen is the the kind of the process that goes from every file and it can happen hundreds of thousands of times there are thousands of property types but there can be hundreds of thousands or even more property values and for each one of them fetch the type you understand which is the right kind of type endler and you change their values feature vector accordingly now as I said I can't tell you what what each element in the feature vector is because it doesn't have like a classic or human readable meaning now that we understand anyway and many students in the crowd do you ever used in your algebra do you like it did you ever think you're gonna

use matrix multiplication and you're really work I did so you may find yourself 15 years from now actually multiplying mattresses yes just so you know alright so this is a of course I'm kind of fast-forwarding a process which was the main process of perversity this product we didn't get this formula some exactly formula for anyone but we kind of went from this point step by step but we tried to understand what the hell were seeing we understood that the feature feature vector which is here on the left it was kind of ready it was ready for processing because we didn't see anything else changing in the feature vector but then we saw all kinds of thinking of multiplying rows by

columns and summations and we're writing everything like on paper and trying to figure out what we were seeing and then what point were like matrix multiplication could it be so yes that's exactly what's going on in here and the different constants you're seeing the A's and the B's and the C's in the different matrices these are all numbers that were kind of built in the lab right that's the magic sauce of Silas death these are the numbers that kind of represents the machine learning the model that they built the brain which is why we will never release those numbers that is their intellectual property now in between that's after every time for example when this vector is multiplied

but that matrix worries you what happens when I multiply one by seven thousand by one point two fifty six what's the result what type of metrics is that what's the rows and the columns yeah some more studying yeah yeah we had to Google then to say don't get like alright so uh so anyway once you have that the interim metrics resolved there's the hyperbolic tangent function that's a plow to apply to each and every one of the elements I hope I sounded smart by saying hyperbolic tangent function like we didn't know where that is either like we wrote that which was like 1/1 is that the natural yeah powered by - and then we Google that and

we realize that that what we're looking at so apparently and this is there's lots of guests as a speculation in our research because no one ever confirmed that from science but what we were thinking we're seeing here is an approximation of a neural network and those different functions in between the multiplications are what's called activation functions and activation functions are functions that try to simulate if a neuron would fire in an artificial network that is the best of you know the best of our understanding that is what we see and this is kind of the the crown jewel of the model now at the end of all this these metrics multiplications you multiply this by

that and this by that and then you have a matrix which is one by one and that is the score so minus one means super malicious the bad you know the baddest malware you can find and one means the most benign file on earth so at this point we're kind of high-fiving each other or thought we were done had another coffee and we we kind of looked at the code to see what happens next and then we had this one so this is the actually the final the final call and on the on the score of default so after you're done with this calculation of whether something is benign or malicious you actually have a

white listing mechanism white and black listing mechanism but it's not not your run-of-the-mill Y Costilla mechanism you don't look for hashes or file names or anything like that it's really a kind of a cool sophisticated white listing mechanism and all trying to explain as best as I can so the first thing that happens is that the your vector from before it gets reduced to 3,000 approximately $3,000 now for the sake of the example and explain this let's assume that it was narrowed down to only three features so now I can think about it as a point in space right so this is this is my sample right now it's a point in space and each one of

the XS is really just one feature now what songs did is they in their lab I guess the research and they found all kinds of families that represent good in the white list or bad families of software so other than my sample there are a few tens of points in this three-dimensional space each representing a good type of software and if my sample is close enough there's a threshold if it's close enough to any of these other good software then it's considered probably a permutation and therefore that the score is reversed to fully matt9 or fully malicious if it's on the black list that's what we understood by looking at the code now scale that back up of 3,000 different

features and it's the same thing is the same calculation just in the 3,000 dimensional space another three dimensional space so that's that's the mechanism oh but what we were really interested in at this point and this is I think a big part of understanding of product and reversing it the hopefully the feeling it is not just understanding how it works technically but understanding the thought process behind it and we were sitting there and wondering why the hell would they neither whitelisting matters and the answer is probably reality because in and you know in in the movies and books everything like works on the first go and everything is easy reality is not like that and what we started

speculating was that they probably had the product deployed yeah it was still an early version of the product they probably had a few misses false negatives are kind of fine that's not so bad false positives are a disaster for any company full disaster because this means you will start the leading or quarantine all kinds of legitimate files which means you can really mess up normal operations of the business so what do you do why they say mechanism right because he can fix quickly all kinds of things and also there's another option we thought but I think it's a combination again and we're speculating which is it's a mechanism for hostages yes but also there's probably some kind of limitation

to the accuracy of the model and there's some point that you want to say okay it works so well for the 90 something percent but there are some exceptions I'm gonna solve it with something like this so whenever in mind let's move forward now this is this is the actual list of the they call those points in space they call them centroids it's based on another technology not talk about it but we actually saw there's these strings and we kind of tried to see if there's any meaning anything we can actually extract from these and I was looking at then I've been a gamer from kind of the age of three which was about a hundred years ago and I probably

immediately looked at me and said I'm a game like I'm 100% the game and it kind of piqued our interest like why would they have this specific game what's what's the story behind this and and we googled it anyone here played what could they give her yeah so I didn't know this game I'm a bit embarrassed because I am an active gamer but maybe it has to do with the fact that you basically soccer with the cars which sounds like that not my finish I've soccer with cars but but all the thoughts will come back to roughly alright so at this point we're doing like this right we have two options to attack this this product we

can either aim program whitelisting mechanism which we now kind of understand how it works and the way we can do it is by trying to think how we can craft an executable to have the those 3,000 features such as it's close enough to whitelist anatomy that would be cool because that would mean that it would be identified its 100% benign the other option we have is to attack the core mechanism itself we've seen how the properties get sparse we saw that the process we understand it we have that very long weird function and that's also an option so we contemplate the whitelisting option for like a whole to me I think and we realized that we don't

have like every kind of Twitter yeah so we say okay let's go for thee let's start with a core mechanism and see what we can find and of course anything we can infer from one to another would be helpful so we go back and we look at the the different properties and you know how they're parsed again and we're trying to see if there's anything that stands out and there is there is something that kind of stands out and that's the size of the type handler for Strings and I'll recap because that was like at least 20 minutes ago so the type handler is that table that knows how to handle each type of property most of them that their size

was like 4 or 8 because these are the potentials bodies right for Polly's 8 values that's kind of the range 12 but here we have 850 4,000 different values that need to be handled in a certain way what does this mean so once we need we understood how the mechanism works we understood that what this means is that some of us have identified eight hundred and fifty four thousand different strings as having the meaning like being meaningful enough so that they want them to impact the feature vector now in a it's called in a normal world in a normal PE the real part will be just a few tens or a few hundreds of Springs

that would match this string so it shouldn't be should it be any any problem now just to close the loop at how it works whatever I need a string comes in attached they use something is called murmur hash oops something's called murmur hash and they have in the tyke on their table there was 850 thousand different hashes so I don't have the list of the actual strings they look for but I can take lots of executables and hash their strings and see if I get matches and and this is this is how we can tell them the hash is compared to the hashes in the string and if it's der the feature vector gets manipulated simple as that

so what's her perfect is that space we're saying the Pali we're saying strings have the potential for disproportionate impact on the feature vector why there's so many of them each string that will be found each and every student fundable will do something to the feature vector will increment and decrement all kinds of things we don't know exactly what the impact will be but we can understand that there is a potential for disproportionate impact now at the same time we look back at the whitelist mechanism and and we we try to follow again the thought process so last time we spoke about the whitelist mechanism we said that this is probably a mechanism to kind of you know ha fix

or either just help with accuracy generally speaking but if this was you like if you were working there and you had this big problem with a game such a big problem that you had to add this the centroid called rocket League what is very likely that you would do the next probably go back and try to fix the model right maybe you know fix the problem for now but now let's go back home to the lab now we have time let's fix the problem and we started looking at the rocket League we actually bought the game for this and some other games that was a great excuse and way to play them a bit just make sure that like

sanity checks and we looked at the different properties of those Fossum look at the strings and you know we realized that the strings that you have in games are really kind of unique and if if songs came back and tried to Penry train the model with lots of binaries from games there's a good likelihood that those strings are gonna have you know an interesting impact on the future of vector Y because do you know of any malware that actually use strings from games is that even logical or possible if you write them or would you ever consider taking strings from games not likely which is why it is very likely that those type of features if we go

back to the feature selection process those features are likely to rise and have a more meaning in the machine learning model so if we scraped the strings from rocket League and similar games and we carefully inject them into malicious file we have a fighting chance of achieving a bias that's kind of our thesis and what we also noticed by the way which is kind of a side note is that some of us didn't really look at all its attacker economics there are some properties that are really hard to change but to change strings that's you know that's easy that's the easiest thing you could do so since we're I don't want to say lazy I'll say

efficient we're really efficient people we said before you're gonna carefully inject strings into the the executables why don't we just slap them onto the end of the file and just see what happens like it would never ever ever work like does not work in that way so let's have a look so what we're gonna see here this in this video we're gonna drop three files maybe cats the famous potential harvesting tool we're gonna use Samsung which is a pretty infamous ransomware we're going to use their regional celebrity the the one-up one a try paler so first we're gonna drop them to disk as is and solace is gonna rightfully so detect them you can see it on the right

hand side there they are okay so they've been detect and once we try to dump them to this they were immediately detected as they would what were you doing now is we just the only thing we're doing like passive Universal we're taking strings from rocket League and we're slapping them all to the end of the fathers the only thing we're doing we're not touching anything else so we're creating our own kind of version we're gonna dump them to this and before before this went alive some researchers like we had a peer review and to say this would never work then I'm clean because the behavior would be so malicious that it would be stopped by

other mechanisms so we decided to do with the demo like by fully running lima cats we're now we're gonna run samsung we're gonna run when i cry they're all gonna be running together encrypting this data by out of our vmware a machine and yeah we'll see what kind of response we're getting so actually it is being encrypted as we as we speak as you can see so we can look at the different flaws we can see there it have been encrypted and yeah not a lot of response from anyone this would work we try this like with hundreds of them always running at the same time no problem whatsoever all right cool so let's move on so at this point we're

starting to get excited and we're like okay with you know responsible adults we need to be methodical about us and we need to summon the hordes the different malware is an intestinal kind of largest set so we look forward for some kind of thing that has a bit of credibility and we end up on a website of the Internet Center for security something like that and they release lists of the worst malware for every given month and we took the list for May 2019 the top 10 were s'mores and we as you can see the table here these are the result the score before was 9 out of 10 were detected we have nothing to do with this

stuff and then we treated them again in the same way does the springs for Rockley and the score after as you can see not only are they just benign they're like super benign they're family family and friends so this is like almost a complete flip and then we said oh this looks like it's working oh my god and we decided to go and test this on 384 samples that we obtained from the zoo github repository and when you get 100% unfortunately we got that's close to 90% but I think the most astonishing number in this sample is that the average changing score and again from minus 1 to 1 is 1 point 6 to 7 so it's

almost a complete flip that's crazy it's way more than we ever thought that we would achieve now there are a million ways to optimize this we could have achieved way better results but as I said we are efficient we didn't want to go for that this is a hobby project like we we had a real job to do so these are County results now we're actually in discussions and at that time with with a cybersecurity reporter called Kim Zetter she's very well known about the fallibility we discovered an RC and kind of these we saw the results and we thought this might be interesting a lot of people and we kind of pitched it to

her and she said yeah like everybody hates machine learning hype let's do something about it so um so actually she contacted solice about a week before the publication they said that you know thank you it's it's not a big issue we'll be able to handle it I figure they said in a few hours if I recall correctly and we went live on the 18th of July this started on Viacom headline it for a few days and then went to other publications and on July 21st sounds respondent formerly they contacted us immediately of course we gave them samples and everything we didn't make this information public at that time to give them time to fix so we didn't say

exactly how to do this but generally speaking that this exists and their response was kind of iam what you would expect they played it down a bit and they said first that this is not this is not a universal bypass like we made its up like I don't know how you can say that it's not because we made it turn out basically and then they said something they had actually lists of mitigations that made sense to me but then they said that this is going to really release the fix is going to really released in the next few days poured that the long list of litigations now I've been involved in commercial software development for around 20 years

and I was like no no way in hell you don't fix something like that you can do huh sticks but you don't do full fix so we were waiting and waiting and waiting checking the product that we have which is the smart AV home product like every few weeks we made contact again which came on the 14th of August we said look it's the bypass is still one percent there there PR came back and said fixed across all agents we said we're stirring this right now and it's not fixed so then they said yeah I know it's not fixed it the whole versions will be rolled out later now this was again we published on the 18th of July

as a this morning this is still exploitable on the home version they claim it has been fixed on the enterprise version but we have no way to verify that we asked them for access we has to help them without they fix this but we didn't get it in response in those so that's kind of it and happy to take any questions who's in charge of mic yes we did any work we tried portland's that that was a homage for my younger boy likes for tonight yeah it worked as well

[Applause]

yes oh thank you just a quick question was it the wife is to make it is for actual core mechanics for the core mechanism was exploited exploited so the whitelist mechanism and there's a bit of confusion about it also it with the publication the whitelisted mechanism provide us with the hint of what type of software may have been retrained alright but we didn't attack the white listing mechanism it's actually like homework if you want to attempt if you have enough information to go on that but I'm actually I'm very very sure that there are many many other ways to attack both the core mechanisms and the wife is luminous oh that's a good point that you're

making yeah if anyone has their Enterprise version I'd be happy to have a look

[Music]

[Music] well as I said before we are efficient so no actually again this was as I said this was kind of a hobby project and we when we saw this result we're like we know we can continue this there's tons of things we could find like like you said their weight of different strings we can find probably most of the string in the table in the first place just like taking a large number of binaries and hashing their strings right many ways in which this can be continued but we just stopped there because we had no more I'm just waiting to see that top of yours and not not immediately we didn't again they should see we

didn't go back and look at all the different type antlers and we took as I said look we have offense of cybersecurity experience and usually what we had to do always be really sick I'm not joking I'm really efficient and once you find something that works you exploit it you move forward I think there may be other properties that can be abused and exploited I think that strings specifically are just so easy to abuse and they even didn't even look at the location that they look at the number of strings so it was just kind of ripe for the taking

so I'm going back to the offset paradox right how would you make this fire human maybe had tons of anti tempering mechanisms you would not tell me what's the name of all kinds of things the names of the Assemblies would be different you would have other layers in there to stop me from doing all kind of compensating controls within the product itself there many things you can do but what you saw is kind of that that was that was it you know and I think this is my way this is this is a problem in detection and prevention software generally speaking I think the sometimes they fall in love with the core concepts and the science and you see you know you

see headlines like math defeats malware and all kinds of kind of thing like that it can be helpful and you can if you use it in a proper way but I don't think they spend enough energy on attacking their being their own worst enemies attacking their products like crazy day in day out and spending enough money on that and again this is what we try to simulate right how would normal attackers try and tackle this would they be able to find the bypass would it be strong by us and the answer in this case was was this and by the way we didn't spend a lot of time on this research I think it was about 30

hours in toll and the technical aspect writing it this up way more like way more so I think it's reasonable for attacks to be able to achieve something like this or or take something like this as a project if they only asked me I would have helped them there are many things they can do look first of all you need to fix the obvious spice when strings needed you need to give them a bit less of less of a weight I think they need to take more late approach and what they do like not have just one Oracle the machine learning you know engine that says this Malaysia so this is benign they should combine several mechanisms together I

mean III feel that sometimes that witness you know with the next gen wave of endpoint protection products they're throwing down the drain twenty or thirty years of cumulative experience of how to battle power and I didn't think you should replace it I think you should add to it I think it should be another mechanism it shouldn't be oh here's something in here that just solved everything you know that's my approach something I don't know why again they said it's fixed in the Enterprise version I didn't get a chance to to see if it wouldn't it would withstand the second all slit-like maybe maybe I would look at it and the two of us look at it and for a few hours

and say oh you know here's another there's another way I don't know but like I said I don't think it's plausible to be able to release something so major and you know a few days over the week

well that is that is again that is the hotfix type of thing I can still manipulate the thing is that I think that what what you just said now for example that's a classic dispatcher now between a tiger in defender right I made my move now you made your move now I'll make my move now you'll make your move right today let's bail and you working its office right so you're battling this day in day out at hotfix a hotfix it's pretty easy that's what I think they did but it just starts this arms race which is inevitable in any type of new mechanism in this case it's a machine learning mechanism but you can't mean

they even say you don't think that okay you know I solve it it's it's gonna be a an arms race this wondering where are you oh okay yeah I was following makings of they're in tweener because when a silent wondering which particular QDR meddler thank you for sharing this year so I was just thinking from a legal perspective maybe you have the fine grain you may you know that you're not supposed to be the first engineering they actually become the legal team athlete you know talk to you about these were in maybe you're not supposed to be doing these are fortunately not but look they did contact us they were super nice they said we are researchers also like you

you know we're happy to get any information that can help us improve our product as I said before I don't think it's it's actually a well engineered product I don't think you know anything bad about them but no they didn't try to scare us off and I think it's a good sign at the end of the day these kind of things while they're not great for you know somebody googling solace and getting dust this is not great for them maybe but I think at the end of the day it makes their products more more secure and I mean how many times did people find problems with simatech our products thousands of times right so I think it's

a big issue so the slides are not yet available I mean really I'm willing to make them available online by the way we did release the blog post before like on the 18th of July which kind of describes the process we will update this now with additional information and we will also include this his presentation

the main binary you just literally run the command strings and the main binary that's it now we did we didn't prove it a bit because there were like a few megabytes of strings I think it important either ruggedly and we wanted to make it more lightweight so we then filter out we hash them we use the murmured hash we hash them and we just so which one matched those in the table and we we've filtered it down it's 60 kilobytes of strings from rocketing that's it again we can add those four for time you can add those for many you now have an excuse to run looks like tons of games to buy tons of games we took we restrict

all the strings from Rocklea we hashed each one of them where the murmur has the same as they're using and we check to see if they're indeed in the in the table in the tiger and all those that were there we chosen the rest we threw out the garbage basically what did you the strings actually funny you should mention no I saw something again we were fishing we didn't have time for all of that but we where is that where there are other names here that I are interesting like you can see for example I don't know what sounds suspect this looks like something maybe it's their own product maybe they're you know identifying the wrong part now we did

try strings from other type of things which I like from chrome and things like that it didn't work no true true look I don't know why they I don't know what they had initially the problem with with the games I think games me have some you know things like they have Internet's all kind of net connectivity features and maybe imports it look like malware I don't know it obviously had some kind of problem with that but I can tell you exactly why