Is this binary Naughty or Nice? How Google leverages ML and Santa to detect persistence on MacOS

We have our next talk for today. Is this binary naughty or nice? How Google leverages ML and Santa to detect persistence on Mac OS by Kristen Smith. Let's welcome her to the stage. [applause] >> Hello everyone. My name is Kristen, but you can call me Krispy. And I am here today to tell you a story about how we being the security team at Google went on a journey to develop a machine learningbacked solution to a rather noisy security problem. Hopefully, if you found yourself in this room right now, you have some kind of interest in detecting bad things with machine learning, Mac OS security, or maybe I've just misled you completely and you really like Christmas. Whichever one

brought you here today though, I am going to do my best to make sure it's worth your time. Before we dive in, I want to better introduce myself and my co-workers who I collaborated with to get this project off the ground. I'm Krispy, a security engineer on Google's detection and response team. At the time of working on this project, I was a member of Google's identity task force, and my area of specialty was the abuse and theft of credentials, tokens, and cookies from disk. Next we have Sard Miy who is a software engineer on Brainor. His team utilizes deep learning and large-scale data analysis to improve authorization and risk detection at Google. We use a

lot of tools that his team develops in our security incident detection and response efforts. And finally, we have Kathy who is our identity task force tech lead. So let's get into what I'm going to cover today. First, we're going to take a look at the shortfalls of traditional antivirus software when it comes to detecting file access based malware. Then, we'll take a look at how we as humans might approach investigating a malicious execution before walking through how we can build a machine learning model to do this for us. Naturally, we're going to want to take a look at how our model performs. And if you think it did well, you're in luck because I will show you how you can

generate your own logs to do something similar in the privacy of your own home or at scale on a corporate fleet if you'd prefer to do that as well. Next, we'll drill down on how we can use everything we have built to tackle the problem of detecting persistence on Mac OS before finally talking about all the problems with this with what we've done and the hopes and dreams I have for this project in the future. As a security engineer, a significant part of my job consists of codifying detection logic to catch badness on our corporate devices and network. Google has billions of users, so gaining access to court machines is extremely appealing for attackers who have an interest in

our users, their data, and the company as a whole. For this reason, we have to make sure we're monitoring for a broad range of attack vectors. But this is easier said than done. A significant area of interest is compromised via malware. My specific area of focus is identity. So everything credential and authentication based. At Google, we have controls in place to alert on potential malicious authentication activity. But there's a problem with this approach. In the case of stolen credentials or tokens, we have a gap in our knowledge. How did this theft occur in the first place? While detecting credential abuse is extremely beneficial, the intervention occurs later in the attack chain than it ideally would.

There's a great range of antivirus software that can be used as the first line of defense in stopping malware infecting a corporate fleet, but it's not perfect. What it is really effective at is signaturebased detections and known bad heristics. This covers cases where malware has been seen categorized as bad and in turn a detection has been developed to look for the same bad thing happening again. Sophisticated malware however can evade signaturebased and behavior specific detections but it still needs to interact with the file system. Traditional security tools can struggle here because they either look for known bad files or that specific attack pattern. And this is the problem that myself and my team have had a crack

at solving. My design process started off the back of a very high level brief. Design an MLbacked solution to detect the access of sensitive credentials on disk. During the development process, I realized that file-based access detections could be beneficial for other sensitive files and not just credentials, but we'll get more into that later. But this was a pretty open-ended request and I thought a good place to start would be to look at the problem in a way that I as a security engineer would face it and try replicate the same logic with machine learning. So let's look at a scenario where we as security engineers have received a signal for a potentially malicious execution. Also if you are wondering yes

I do all of my security investigations in comic sands. We have a Chrome extension for that. But let's start with the log entry. Two specific things here are interesting to me. What has run and how it has been run. We can see that the file has that an executable has run called https disk and it was executed from the user's temp directory. And when it was run, a couple of arguments were passed in on the command line including a username. My next question is what happened when the executable ran? Here I'm looking at a snippet from the execution tree of the host which allows me to visually trace its parent process and see some telemetry about the executable. It's

flagged as very rare. And if I hovered my cursor over that question mark, I would be able to see that we have exactly one record of this binary executing on the fleet. And over to the right, that blue symbol indicates that the process has net flow. It's not pictured here, but I also care about what file operations have occurred as a result of this process running. Looking at this log, I'm starting to get a little bit suspicious. That being said, we work at Google and if I have learned anything here in my four years, it's sometimes engineers just do weird stuff or it can certainly appear that way in logs. For this reason, I need to ask why might someone

need to run a program like this and is it plausible for this sort of activity to be legitimately workrelated? looking at some information on the um user. Most of it's redacted for privacy, but I can see they can they work in technical support and engineering. While they could need to run a script like this for their job, it doesn't immediately strike me as workrelated or benign activity. I'll solve the mystery uh for as to what happened. Spoilers, it was us. Well, uh it was the detection and response team uh in collaboration with our in-house offensive security team. Th this log entry was generated as the result of an exercise that we ran to test this

detection amongst others. What this binary did was read the user's SSO ticket, their machine certificate, and use the two to authenticate and send web requests impersonating the user to an internal service. In essence, this is bad and this is the kind of behavior that we want to detect and mitigate. I'll keep this one brief, but it needs to be mentioned because it's so significant. Looking at this issue, we are looking at the malware problem from the file access perspective because it's fundamental to detect the kind of badness that we are after. But file operation logs are extremely high volume. We are talking tens of thousands of file accesses per machine per day. So when you scale that up to a corporate

fleet, it is absolutely huge. And of course, it's not really feasible to investigate every single one of these log entries one by one. So what are we going to do about it? We need to apply the same logic a human would when investigating to a huge amount of logs. So let's do that together. We are extremely lucky in the security team at Google as we have tooling and agent teams that support a lot of our security endeavors with features that they build for us. One of those teams is the security endpoint agents team who have implemented logging agents for our internal versions of Windows, Mac, and Linux. For the purposes of this talk, I'm going to focus on Mac OS as the

agent we use is actually open source and I'm going to introduce you to that super handy agent and that is what is going to give us input for our model. Santa is an open- source Mac OS security agent which was pioneered at Google but is now maintained by the awesome folks at North Pole Security. And it was from these wonderful folks that I blatantly stole the title of this talk. Because the whole idea of Santa is it will tell you if a binary that is running on your machine is naughty or nice, as well as give you the power to decide what you want to do with it. I'll talk about some of Sans's other nifty features later on,

but for now, I'm going to show you their file access authorization feature and how we can configure it to create input for our model. When file access authorization is enabled, Santa will log all file access events that match a policy you specify. And these logs include the timestamp, the process that attempts the access, the file being accessed, and the action taken because uh blocking is supported as well as the policy that has triggered the log or the action in the first place. Our next question is what what do we care about monitoring? While the sky's is the limit here, these are some file access based threats that we are interested in monitoring at a high

level. Threats requiring right access such as malicious launch agent or demon rights on Mac, establish persistence upon system start or session login. The modification of the SSH config across all platforms can also be used to establish persistence. From the read operation perspective, we're worried about info stealing. Think access to browser data like browsing history or session tokens or the xfill of sensitive tokens and cookies. Now we've got an idea of how to get the logs we need we need for our area of interest. And going off the investigation from before, we know what information needs to be considered when deciding if a file access is bad or not. So it's time for us to make model.

So what do we need to do to get our data into a form that can be interpreted by machine learning? We have identified three aspects of the log that we as humans use to make the decision. But what do we need to do to transform this data before we even think about how we're going to search and compare logs to each other to decide which ones are bad? We need to get the data into a form that can be processed. And the bulk of where this is going to happen is at the featurization stage. And we've actually done the first part of this in the investigation stage as we've identified the features that we need to transform.

Those being the file path of the sensitive file, the command that accessed it, and the user that ran the command. To do this, we're going to calculate what's called an embedding for each feature. We're going to use the same method to calculate the embeddings for the command and the file path. And that is going to be a multilingual deep learning model called RETSSIM which stands for resilient and efficient text similarity. The core idea of RETSSIM is to convert text in this case file paths and commands into a numerical representation called an embedding. The embedding is a vector of numbers that captures the meaning and structure of the original text in a way that is meaningful for our

model to process. In essence, the closer the vectors are to each other in this numerical space, the more similar the original text was. The first step is to process the raw input text. Unlike many traditional models that process text by words or subwords, RedSim uses character level vectorzation called REVC. First, chunking occurs. The input string is split into manageable pieces or chunks each 512 characters long. The REVC character encoder then converts each uni-ode character within a chunk into a compact 24-bit binary representation. This turns the text into a dense numerical format ready for our next step. Next, a small transformer model processes each chunk to create partial embeddings. The transformer model, which is a type of neural network, analyzes

the relationship between characters in each chunk to create a 256dimensional vector for each one. To create a single unified embedding for the entire file path or command, an embedding averaging module is used. It combines the partial embeddings from the previous step to produce a full text embedding. This final embedding, which is also a 256dimensional vector, represents the entire input string. And this is what we are going to use later to calculate the similarity between two different logs. But we have another embedding that we need to calculate. And this one unfortunately is not as straightforward. Before we go into the nitty-gritty of our next embedding calculation method, let's get back into the frame of mind of

a security engineer investigating a sus a suspicious signal. A lot of the time when we're investigating potentially anomalous commands on a machine, it does end up being an engineer doing odd stuff. Not necessarily bad, not necessarily good business practices either, but as a whole, not malicious. Be it project work, debugging, formal research, or just poking around, sometimes the logs can just look suspicious, but they aren't in reality actual badness. But if I were say investigating an alert for someone who works in sales, who has never opened a terminal before and is now running curlpipe bash, which spawns a command, a process that accesses a sensitive file, that would be more interesting to me. My point being, sometimes user behavior

will deviate so significantly from what is considered the norm of someone in that role or team and consequently that meets our threshold for investigation. That's not to say we want to treat every command being run by an engineer as benign, but the context is super handy when we're trying to build up that big picture of what's happened. That covers the first two points nicely. And the third one, the historical behavior, we're actually going to cover during our search method. We're going to use a framework called facade to get this user embedding. Facade stands for fast and accurate contextual anomaly detection and it's a high precision deep learning based anomaly detection system that we use in

a lot of areas at Google. We call facade a generalized action context framework meaning that it uses two sets of features to produce a score. Context features and action features. A context represents the featurization of a principle at a point in time and an action represents the featurization of a resource at a point in time. These features are given to facade the facade model to produce embeddings for the context and action which are combined to give us a score. The model is trained to produce low scores only when the access is similar to benign accesses in historical data. Context features model relationships between principles which in in our case are employees using a peer graph. The

graph is bipartite consisting of principles and peer attributes which are shared entities like meeting ids or employment groups. An edge connects a principle to a peer attribute they share. For example, an employee that attended a meeting. The features for any given principle are computed by finding all other principles reachable within two hops of the graph. The resulting feature is a list of all other principles who share at least one peer attribute with the starting principle. In this calendar meeting example, an employees context feature is simply a list of all other employees who attended the same meeting as the original employee. And this is what captures who they are interacting with. and via what shared context.

Actions represent a principle interacting with a resource at a point in time. To compute the features for that resource, facade looks at past actions that acted on that resource and it considers the principles who acted with the resource before the current actions lookup time. The principles are collected into a weighted bag of tokens where tokens are principles and weights indicate the recency as well as frequency of the past interactions. In this picture, weights are configured to count the number of past interactions. Now, we have built out the logic for how we are going to calculate each embedding for the log entries. But what are we going to do with these embedded logs? Our embeddings are sent to a scan server

via RPC requests. This server then performs an approximate nearest neighbor search at a very large scale. Scan organizes the log embeddings into optimized structures. So when it receives a new log, it quickly searches the index for the next nearest neighbor. And it does this using hybrid coarse and fine grain search. First broadly filtering the data, then precisely checking the closest candidates for speed and accuracy. Scan returns the distance between the query embedding and its next nearest embedding. And this is what is going to give us an anomaly score. The higher the score, the further away this embedding log is from the next most similar one. And when we're taking into account all of the factors that we've talked about

in the embedding stage, this is how we decide what is bad. Finally, our entire architecture looks something like this. Our online scan server searches through the search data set to find the nearest neighbors. The data set creator runner is a daily chron job that first executes the training main and then the merger main. Training main reads the logs of yesterday samples featurizes and writes them to storage. Mergemain reads daily data sets of the last six months and merges them into a single data set. The search data set will automatically use the latest created search data set. So the big question is does this actually work? And while it's not perfect, I can tell you it has found

some interesting stuff. Going back to that exercise I mentioned earlier in the presentation. This is a distribution of all of the logs on Mac OS that we received for the week the exercise was running. This data has been aggregated and we're only looking at entries with a distinct user identity command and file path tuple. And these logs are only for a subset of sensitive files that we have decided to monitor on the fleet. So over the oneweek window there were 33,382 525 sorry 382,525 unique tpples and this is the distribution of how um the scores they received on the model is distributed. So we can see most of the volume is concentrated down the left hand side of

the x-axis which is good because we are expecting most of these logs to be benign. And then we can see we've got a little bump. And it is hard to see on the diagram, but the tail does keep going um right up until that top score of 0.35. During the uh exercise, we ran five different scenarios, all of which created multiple log entries, which are included here. The uh first executable we ran was the binary called HTTPS disk, um which we looked at in the sample investigation. And those logs had an average score of 152 out of the 33 million odd logs. Uh we had another variant of this script which got a um average rank of 11 which was great. And

we ran a few other scenarios that didn't score quite as well but still ended up in the top 0.015% of an most anomalous scores for the oneweek period. But I want to show you how we can take all this logic and apply it to solve a specific problem. the detection of persistence on Mac OS. And this is where our handy logging agent Santa comes into things. So first of all, how do you establish persistence on Mac OS? From the file access perspective, malware can write or modify a property list or P list in the user's launch agents or launch demons directory. If malware writes to launch agents, it guarantees execution as soon as a user logs in. And if it writes to

launch demons, it ensures the malicious process starts immediately on system boot even before anyone signs in. In both cases, the property list sets the run at load key to true. And this ensures the payload is um executed automatically by the operating system. So since this is uh file access based, it's a great candidate for us to use for our model. The full Santa documentation is available at North Pole security north pole.dev. And believe me when I say we're just scratching the surface of what this agent can do. It is very impressive. And if you have an interest in Mac OS security, I'd strongly suggest checking it out. But specifically, we're going to be utilizing the file access

authorization feature. And this is going to be what will monitor our directories of interest. Firstly, we need to install the sand package. as I was just doing this on one Mac for development purposes. I did it via homebrew, but the docs do have instructions if you need to deploy this across a fleet with MDM enforced. Next, we need to generate the main configuration profile. This will control how Santa behaves. Uh we could directly embed our rule logic here, but I prefer to keep it in a separate file just for readability. We'll then deploy our configuration. Um we'll need to manually install the mobile config. Um, and the next step is to define our file access

authorization policy. And this will involve writing another property list that we link to in our main config. Next, we want to verify that our logs are actually being paused by Santa and that the rules are active. So, we can do that via a handy command line tool. And then step six is the fun part. We'll run a suspicious script that modifies the launch agents directory and we'll get our log entry. And by this point, we should have what we need to put into our model. The installation step isn't remotely interesting, so we'll skip straight to profile generation. The awesome North Pole security folks have this handy config generator, so you don't have to write all the XML by hand,

but if that's your thing, like, please go for it. Um, [gasps] but I'm going to leave most of the keys set to their default state because I only care about monitoring one specific file path and I don't need to do any um blocking actions at this point. So what the generator is going to do is give us the XML file based XML based configuration and we need to decide how we're going to set the key value pairs. Since what I'm trying to do is pretty simple in terms of monitoring. I'm only setting these two fields. The first is the path to where I am storing my policy and then I'm just updating the um interval second timer to 30 seconds as

is is advised during development. The next step is optional um but I've opted for the more verbose logging by setting the log type to JSON and specifying an output file for my log events. After I've hit generate the mobile config the mobile config will download and I can install it via the system UI and it should come up as a system profile in settings. After adding full disk access I can verify that the Santa demon is running by running Santa ctl status. As things stand right now, the watch item section is empty because I haven't actually added a new property list file to parse the rules from. To do that, we'll need to make another XML

file. And this is where we define our rules for monitoring the launch agents and launch demons directory. Here we specify the file paths that we want to monitor and set the audit key to true as we only want to log and not block anything at this stage. But you absolutely can configure Santa to do logic based block events too. Next, we can confirm our profile is being paused correctly by running Santa CL status again and confirming that we have an active rule under watch items. Now, let's make some logs. From the file access perspective, all I need to do is run a binary that writes or modifies a launch agent to simulate what we are

trying to catch in malware activity. I have a very basic Golang program here that writes a property list called super duperus.p P list to the user agents launch directory. Here you can see I have run the program. A file has been successfully written to the launch agents directory and our logs have been written to the file we specified. They contain the user ID, the file path of the file that was written and the process name of the file path that the executable of the executable that wrote it. I removed several section of this log just to fit it on the slides, but the configuration, this configuration option of Santa includes heaps of useful information like the uh

signer as well. So, let's put it in the model and see what it gets. Since I ran this program on my personal Mac, I've substituted the username and file path of that of a random Googler that does not work in security engineering. And I ran the data entry through the models data from two days ago to make sure it was as fresh as possible. Looking at a 24-hour period, there were 7,886 total unique combinations of user processes and file paths for rights to the launch agents directory across our Mac fleet. The top score of the day was approximately 0.439 and our log got a score of 0.4357, which was the second most anomalous score for the day out of nearly 8,000.

Well, this is a good result. I won't pretend that it's a perfect catch all and that it will solve your problems. [gasps] First of all, there is a distribution shift issue when new binaries are deployed across the corp fleet because initially a binary that touches a sensitive file hasn't been seen before and the model will flag it as really bad and it takes until it's been executed on lots more hosts for the model to be retrained and go, "Oh, actually this isn't that weird. I'm not interested in it anymore." Um, sometimes it's not particularly clear while a log was flagged as highly suspicious. And when that happens, I like to look at the next nearest

neighbor for that log entry to try reverse engineer why the model has given it a really high score. A lot of the time it is obvious, but sometimes it's not, and it can just take a lot of digging because you don't get an explanation unfortunately. Um ideally we would train the data um train the model on data from on each OS separately but this whole process for running everything is quite expensive. So at the moment we're training it on one data set that involves our three platforms. Um and the model and pipeline cannot handle every single file on our host being monitored at this stage. So filtering still needs to be applied to manage it. It's it's good from the

perspective of you can identify sensitive system level files that everyone has, but this doesn't help when a user downloads something sensitive and then a that isn't um configured to be monitored by our agents and then a info stealer touches that file. So that that situation is not covered unfortunately. The next point is lacking the context of execution in the leadup to the bad file access happening. Um the model currently evaluates suspicious activity based on a single log. But thinking back to the investigation example, it is really helpful to know what's happened in the leadup and it doesn't take that into account at this stage. And then finally, this last point is there isn't an executable integrity check. It's just

going based off the um raw text of the file that is being run. Although in the future I do want to include at least the hash as an embedding to um determine if a executable has been seen before on the fleet. All right, and that just about wraps it up for me. Um if you have any questions about what I've talked about today, I'd love to chew your ear off. Um that QR code's linked to my LinkedIn profile. You can contact me there or you can send me an email. Um but I just wanted to say a quick thank you to um the security endpoint agents team as well as the folks at North Pole Security because

they are absolutely fantastic in the work that they do for us and it means I get to go away and do really fun projects like this with their support. So thank you.