Introducing LMDA: Enhancing Lateral Movement and Data Access Identification on Windows Systems

[Music] All right. Thank you so much for the introduction. I'm not going to waste time uh in our introduction. All right. You want to teach it? >> Yeah. So who we are a little bit about usberg is a leading provider of cyber security and various services. Uh we've been named a leader by forester in both 2022 as well as 2024. We recently been acquired by Level Blue uh to be essentially providing the largest pure play cyber security MSSP uh within more or less the world. Uh today we'll be talking about how we can enhance our investigations. As we know, we typically start off our total incident response life cycle with gatherings forensic artifacts followed by looking at them

and then going back and gathering more especially in kind of larger environments. The idea behind Lambda and LMDA and what we're talking about is to help and simplify the initial gathering phase and kind of help you jump straight into your IR investigations. The idea is you gather as much as you can at least the critical triage data sets that you typically use very early on in your stages and how you can use that data set and work through it in a normalized output. I don't know about you folks but usually when I'm looking through a bunch of different artifacts I take a lot of time trying to normalize that into my master timeline. Uh so now this way I

can work with one simple output um or kind of individually all in one normalized form. Lastly, we'll be going through some examples of how we can use uh visualizations to help our analysis as well as how we can go through it from a reporting requirement stamp standpoint uh to our stakeholders. Now about you folks, that's me uh when I can enhance my analysis. Uh the agenda as we already kind of talked about we'll be going through an overview of LMD or lambda. Uh we'll be talking about some forensic artifacts. Uh what we do consider as part of our initial triage poll. Uh we'll be going through some of the finer definitions and how it's implemented. Uh for those of you that

don't use veloc frequently, don't understand VQL, don't worry. We're not going to be diving really too deep into that. Uh and then lastly provided some examples how I can use this data set from visualization reporting standpoint. Now for those of you that don't know what Velociraptor is but are fans of Jurassic Park, fortunately this might not be the best place for you. Uh Velociraptor is a tool that is open source. It's currently supported by Rapid 7. It was built by DFR practitioners meant for DFIR at scale across environments. It really does make the overall acquisition and analysis process a lot faster especially in larger environments. >> All right, thanks Kosia. So now that we know what we're going to talk about,

let's u dive a little bit in depth of what lambda actually is. Right. So when I say lambda, it's it stands for lateral movement and data access. And uh in any DFR investigation like one of the common goal goals is to identify how many systems were touched by the reactor. Stakeholders commonly asked like what is the impact? What's the total number of impacted systems? To answer that question, we enumerate lateral movement. And when I say lateral movement, it's basically when a thactor compromises like an initial system in a network, they typically move to uh imported systems like ad file servers and that movement like all together in the network is termed as lateral movement. As simple as that.

All right. And the uh whenever th actor moves to these certain or any systems like they leave traces of their activity. So uh if they use explorer like we can use certain artifacts to parse explorer and uh get the evidence that third actor was there. And similarly uh one of the main investigation goals is to identify what data could have been accessed or exfiltrated by thactor or thactors. And to identify that we uh parse data access artifacts. Again by data access I mean uh whenever tractor compromises system uh one of the major aim is to exfiltrate sens sensitive data. So uh basically they uh traverse to certain systems via explorer or may use remote management

tools like win SCP, screen connect uh any disk to exfiltrate that data and again like the activity that they conduct leaves traces behind. So we can uh parse that activity and prove the evidence of data access or exfiltration. Let's take a minute to see like how we can actually get that information. So we use forensic artifacts and forensic artifacts are nothing but Windows features that are presented by Windows to enhance user experience. So for example, if you use RDP or remote desktop protocol to like remote into certain systems, uh if you right click on that icon uh in the toolbar, you can see the commonly accessed or um recently accessed systems and that's there to

enhance your experience so that you can like quickly traverse between systems and this feature is called as jump list. uh we can use jump list to parse the data present in the jump list and uh prove that a certain user at certain time connected to source B from source A. So that there is the evidence of lateral movement taken from Windows features uh and in this case jump lists. So that's how we uh use Windows features to and there are a lot of Windows features that can give you a lot of useful information in uh DFI investigations. All right, let's take a look at what artifacts do we use in Lambda uh to identify lateral movement and data

access. Right, beginning with Windows event logs and u just to note like this is a small representation of what we actually include in lambda uh like we have a lot of artifacts. Um you don't need to take picture of this like it's uh all listed in the uh EML file or the code that is on GitHub which we will get to uh in ending slides. Right. To begin with, we include um login, log offs, disconnects, connects from uh security as well as RDP event logs. We also include the PSX service install. So that may not be a remote access feature. I mean it remotely remote interactive access but it is a remote access uh and commonly

missed uh when analyzing uh like system event log. So we do that for you. We also have Windows remoting or PowerShell remoting. Uh again a commonly abused Windows uh feature by the Raptors to connect to remote systems as well as uh SMB events or server message block access. It's like connecting to a network share from your system. So that will include uh information in these SMB event logs which again can prove lateral movement. Now there are obviously other artifacts to be used to identify lateral movement. Some of these also identify good access. It's a bit of a twoprong sword. Uh some examples that we do include include within LM specifically uh would be various registry keys as it relates to

the usage of RDP as well as bounding shares through SMB. Uh we also use LNK and shell bags particularly just to notify you as an analyst whenever somebody is accessing another share remotely. Um then we also do know that recently uh thread actors love clearing logs at least I've seen it pop up a lot more in the past few weeks and few months with particular thread actor groups. Uh but the UAL is one of my favorite artifacts at least when it comes to lateral movement. uh we only consider four roles specifically at least when it comes to LM. You are welcome to add more in the YAML file definition. It's not too difficult but we only consider file remote web and

FTP. Uh file is going to be really the most useful, most voluminous for you. The reason I love it is when the event logs not available, uh you can generally get a pretty good idea of what's going on in the environment via the file, but it's only SMB access. So you do have to kind of consider that. uh another source of lateral movement that we have seen in the past few years prop up uh specifically using RMM tools legitimate tools like team viewer screen connect splash do alterra uh win SCP putty uh open SSH uh so every single one of these tools usually leaves uh their own specific uh forensic artifacts that we can use to identify their use whether

that is a file that is created by the actual application whether it logs into the application log whether it leaves some remnants within the uh registry hive. Uh so we try and kind of get a more generalized idea of usage of these tools just so that you as a practitioner as an examiner can follow up uh when you're doing your deeper dives if that's something that you so wish for data access specifically. Some of these we've already mentioned uh LNK files uh shell bags we previously discuss discussed jump lists uh which can be used to identify data access. We can also uh include MRUs uh browser access for local filebased access uh also uh mention of wcp in terms of we've

seen high prevalence of it using for used for data xfell as well as just local file access as well for transfer purposes. Uh so these are all kind of French artifacts that we do consider as part of our initial uh data capture. >> All right. So uh let's talk about why we decided to build uh these artifacts. Right. So before lambda I used to run my hunts one by one and analyze results one by one. So it was very efficient. It was not uh I was not aware of all the different artifacts um and in like high tensed or fastpaced instant response like certain times we miss uh commonly used artifact. So that happened to me happens to

everyone. Uh again, not a good look. And uh lastly, I used to spend time cleaning outputs from multiple formats, multiple artifacts, one by one. Uh that was me a couple weeks ago before we invented Lambda. So again, uh these were the pain points we came across when uh performing DFI investigations. So that is why we built Lambda. As discussed earlier, uh investigators may not be aware of all the different artifacts that there are, which there are a lot. Um again, you do not need to spend time normalizing results from multiple artifacts. Like it's done for you in one single place. All you need to do is click buttons. And uh lastly, but not the least, we

include or present results in a timeline format. So it's easier for you to just plug and play results from lambda into your master timeline or use this as a master timeline because it includes like the most useful info and uh clears out or filters out the noise or extra information which you may not need to prove lateral movement or data access. All right, so that is me. Uh again like I'm not that old but yeah uh my life was much easier after we started using lambda. All right uh let's take a peek under the hood like how lambda actually works. So like discussed before we uh have like multiple artifacts into uh two configurable hunts. Uh and when I say

config configurable you can exclude or include artifacts as you will. So we don't we do not need to use all the artifacts as we discussed. Uh we present normalized results meaning uh results from like LNK jump lists are in a format that is uh not differentiated. So like uh it's better if I show you how the results look like instead of explaining in further slides. We do have master sheet which contain all uh results from all the artifacts as well as uh individualized results. So in case if you wanted to focus on let's say LNK files only you have the ability to do so. All right and like mentioned before we have it in a timement format and under

certain columns like that's where the normalization comes in. Um let's quickly take a look at columns that are available in LM output. Uh we have the time stamp obviously the description column uh will mention if it's a remote access or access coming from PowerShell or SMB um security or like network access etc. We do have source destination system as well as IP. So in certain artifacts like you only get a system name and in certain artifacts you only get an IP. So wherever there exist an IP or system like that information will be included. So we don't need to like go back map what system belongs to which IP uh and user account obviously the most

important one is direction. We bifurcate or uh tag each entry uh as inbound or outbound. So you know like if an access is coming into the system or going out of the system. So again very useful in determining like if it's uh coming out or coming in for data access uh again starting with the event time. We have the user account uh who may have accessed that data that will be in the path column. We also include file size where applicable. So in case if there was an exfiltration event uh you may want to calculate the total size of file or like file size of all the files that may have been accessed to like compare if this matches

to the exfiltrated claims by thread actors. Uh we have the time type column which is of importance which I will be explaining in further slides. And uh for both LM and DA we have additional information. So if you needed like extra info like uh what is the type of access from security event lock whether it's a network access remote access interactive access that is in there. So yeah uh most useful info presented in normalized format.

Now let's talk a little bit about how um it's built at least more of a VQL perspective. Those of you that feel a bit uh haunted or find this a little daunting, uh VQL or velociraptor query language is an SQL like scripting fun language. Essentially describes what the binary itself does on either the server or an end client system. Um really what's shown on screen here is as Falcon would say the meat and potatoes um of the artifact definition, not to be confused with a forensic artifact. Ironically, philosopher likes to use artifact definition to describe an particular action that you're taking on an end client. Uh here we're showing you how we implemented the notebook section.

So if you've used Boss Raptor before, um when you initially sent out a job for a data collection as part of a hunt, uh there is a tab on your uh task that outputs the results in a table. uh typically for every single source uh as defined within that definition you have an individual cell that has a table and then the next cell table for every single source. Uh what we did is we essentially hijacked uh the first notebook cell to just add a very giant change statement uh of each individual source output as it relates within our larger artifact definition uh which essentially allowed us to create that master sheet that we're talking about

that contains all of the results from each individual source output. One thing to consider when working it and this is really applicable for every single artifact definition within the velocraptor the notebook usually always has a limit 50 statement per row. Uh so if you are working with this just make sure to find and replace your limit 50 statement to either you know a user account or a host name and IP uh that you will want to go ahead and filter on. Um as otherwise you're definitely not working with the full data set. How this looks like in the actual notebook uh this is the darkened theme version. So it's not the default look where it's

just white background. Uh but as you can see the column names that Falcon was previously discuss discussing. Um, and then if you wanted to kind of go to the individual output of a particular artifact, let's say terminal server registry keys, you just have to scroll down or press CR+ F and it'll find it for you and it'll show you the output particular just for that one artifact source. Uh, you know, in the event that, for example, if it wasn't from server registry keys, if you wanted to just like look for your RDP events, you just scroll down to the RDP section. Um now let's speak about how we can use this data set um for visualizations

particularly like node edge graphs. I don't know about you folks but I love a good visualization. Uh we're going to be going through two separate examples two separate text acts of you know one a more simpler and and click and go way another which is going to be a lot more difficult to use at least right away but it does allow for higher customizations I see. So wish uh the first is graphana. So what you really want to do is go ahead and pipe your data over to elastic um and then from there set up a connector between elk and graphana um to the desired index that you so want to visualize. The alternative format which first we'll

go under and then we'll double back uh is going to be using Python or like Jupyter notebook to visualize that data set. Um, so Velociraptor does provide a Python module called PI Velociraptor. Uh, that allows you to interact with the backend API of your velocraptor server. Uh, you can issue any EQ query as you typically would from, let's say, the actual, uh, GUI interface. uh then we can you know use the Python module within something like uh Jupiter as an example um along with some other Python dependencies uh like bokeh to go ahead and visualize your your ultimate node edge graph uh that you wish to take a look at. So let's see how it actually

works um at least not from a demo perspective but from a screenshot perspective. Uh from the graphana piece your first task is going to go to your server events. Um, we did create a custom elastic close updater uh or uploader that's a bit different than the one that's typically out of the box. The only reason is because the column names uh with graphana are a little bit difficult. Uh so it just kind of presets all the column names as you so wish or need it. So you can just import it into Graphana right away. Um it also edits the default naming convention um of a velos wrapper index when it's being piped over to elastic. Uh so this is an

example of what you would expect to see should your connector work correctly. I'll have the visualizations there at the very end uh from an index management perspective. Then you go ahead and launch up your graphana. Um you go ahead and connect to elastic. Give it a name whenever you want that index that you want to visualize to be. Make sure your URL connector is set up and all ready to go. Um then you specify the actual index in elastic that you want to visualize. Um, one few things to point out. Just make sure that the uh the time field is set to ID. That's part of that custom elastic close updater that we have to

make. Um, you see the green check mark. It means you're all good. You can go ahead and click building a dashboard. To add the visualization, select the one that you created uh just previously a second ago. Let's say RDP here. And from here, as long as you have node edge graph selected, you want start querying your data similar as you would in in uh velocraptors gy, but now you can do it with like a node edge graph if you want to see autom based off of a user account, a host name or IP. Um specify time ranges, but yeah, they have more visualizations. You don't have to use the node graph. Um it's just available

here for you. Although albeit some of the column names um are not necessarily going to work nicely with our custom flows updater as it's built for the node update. Uh but you can use just any of the normal elastic uh flows updators for that. And here's a just a more finer example of just using graphan out of the box for that. Uh for the Jupyter notebook perspective, there's a few tasks that you'll have to take first in order for you to actually even be able to run commands against your server. First would be to create your Velociraptor uh configuration file, your API configuration file, which basically just contains like the URL and certificates that you need to connect to

the backend API. Uh you can either create a new user for this or specify an already existing user. you're specifying as an already existing user for your API config file. You just have to make sure that the user itself does have um API client rights. Um otherwise the user won't be allowed to authenticate. Uh from there it's as simple as installing PI Velociraptor. Um here's an example of a query command that you can use to run a uh VQL query against your server. This is usually like a really easy simple test command that you can check to make sure that you have a connection. will just give you some general information about your server. Um, now from the

Jupiter's perspective, what we'll be using is more similar functionality of PI velocaptor Philip Pandas that allows you to send a query and it returns it into a pandas data frame or rather into a data structure that you can import into a pandas data frame. Uh, from there really the world is your oyster depending on how you want to work with that data, massage the data. Um it's really up to you from a visualization standpoint. You can take the pandas data frame, you can pass it over to networks and then pass it over to bokeh uh to create a visualization such as this. Um this is using just test data. So it it looks very uniform in terms of the

connections. Uh this is not necessarily a real life example of what an visualization graph across the network would look like. Uh but the kind of key point of this using Jupiter is you can do you know whatever you want at the end of the day the data set's your own. Uh if you want your edges to be you know colorcoded per user if you want your nodes to be larger because of their in andout degrees it's your wish. Um use cases. Why would we even want to do this? So for those that like visual things uh this is a good alternative. Um, in other perspective, if you are very f fresh into a IR workflow and you don't

have a whole lot to work with, you don't really have any indicators of compromise, the client doesn't really know what's going on, um, and you don't know where to start, but for whatever reason, you decided to start from lateral movement, which can be a very big mess to usually start from. Uh, but you can do some quick analysis techniques like degree analysis of your nodes, right? Figure out your large in-n-out degrees. uh basically if a system is seeing a lot of like SMB lateral movement that could be indicative of SMB scanning um and you know it's an example where using degree analysis to pinpoint those high in degree systems uh for you to take a

closer look at would be useful multihop detection so as we know and Falcon pointed out earlier thread actor moves from system A to system B elevates credentials then uh you know tries to reach another system to to to stage data or run something like arclone to just excfiltrate it right Okay. Um those are defined as like multi hops. You go from one system to another. So there's some cool multihop detection or rather algorithms that you can use to identify your multi hops. You can constrain it to you know the initial session of where the user kind of starts appears out of nowhere thin air and kind of track that user um as it goes through systems uh

for you know identifying those multihop uh sessions within an environment of your data set. a few things computationally expensive that are really probably not applicable in most cases especially kind of high tense IR because they just take too much time to set up. You know you can do like graph um simplification via click analysis. Uh you could there are some really cool papers out there for like graph neural network implementations for uh anomaly detection. Um I recommend some of you hopes to read it because they could be very interesting. they have a really high uh success rate in detecting anomalies. Although again, very competition, expensive, and if you're, you know, coming into a hot IR um and

you have to follow thread after steps, you're not going to take the time to set up all that infrastructure, import the data, go through it. It's going to take way too much time. But if you're, for example, working um you know, continuously with a client for for for years, you already have that data set. Something to think of. And that's uh that's how visualization works. All right, thanks Costa. So now that we know like how LM works, uh let's take a look at DA or data access aspect of lambda. All right, so uh like Costa mentioned uh lambda sorry DA is very similar to LM in uh VQL structure wise. Uh we do have normalized results similar to LM but

like columns will be different as I discussed in uh previous slides. Again, uh we do have individual sheets if you wanted to focus on s single artifact. Right? So now that you get results from DA, uh let's see how you can put it in a format that is easier for you stakeholders uh and anyone who will be like viewing the uh data access results uh to be better ingested. Right? So we have a NXL template uh on our GitHub. Uh I will start with the overview tab. So like beginning with the coloring legion, we have a certain predefined color codes to make everyone's lives easier. uh if for example uh a certain row or certain cell

contain keywords like password or credentials uh it will be automatically highlighted with uh respective colors and that is like the uh conditional rules baked into u the excel template which are easily configurable uh and again just easy to differentiate between what could be a potentially sensitive access or a sensitive data. Uh next we have definitions for artifacts that we parse u in case stakeholders or non-technical audience wants to know where the data is actually coming from like they have the ability to do so and understand like what is happening behind the scenes. And lastly for the overview tab we have uh certain review methodologies u if you wanted to like make sense of the data that will be uh coming from

velociraptor results uh you can like for example follow tier three. So this will lead you to u something like if an employee is accessing data of the working hours. So let's say like there was access to uh sensitive files on a certain day at 1:00 a.m. that is an anomaly uh that the employee may not be expected to do that. So again like that leads us to a potential incident. So that's how you can go with the data. Right? For the scenarios tab we have listed uh certain scenarios that may or may not generate forensic artifacts. uh and these are specifically uh related to data access. So in case like a stakeholder does not understand like why

sometimes forensic artifacts are generated and why sometimes they are not. This is a very good example uh in like tab uh to explain like how thread actor actions generate artifacts and how sometimes it doesn't. And lastly, uh the meat and potatoes of the Excel tab is the u data report where you can uh just copy uh results from DA artifact and paste it into the Excel template. And as discussed previously, the uh path column is configured with conditional rules. So again uh keywords will be auto highlighted with uh respective colors and you can change them according to your will. Um importantly we have time stamp in UTC because that's the uh only time stamp in

VFR1 and but we also put a local time stamp it uh as is is in as standard time but again it's a very simple Excel formula you can just click on this cell here and uh add or subtract how many hours you want u easier to identify it in local time obviously we have the host name the user and the path of the data access the type uh generally results from LNK jump list and browser access will be uh file access but results from shell bags and mu registry keys will be folders. So uh one thing to keep in mind and uh most importantly the time stamp type column. So for artifacts like lnk jumpless and

browser access uh this time stamp like right here will u portray the actual time of access but for certain artifacts like shellbags and mu the time stamp won't be actual time of access it will be the time stamp when that particular registry key was modified so important point uh to keep in mind when you present to stakeholders so that you do not say something which is not true. Right? So that is how you can uh present the uh uh results from LM via visualizations and uh from DA via the uh Excel template. >> So what did we talk about today? What are our key takeaways? Uh so we talked about some forensic artifacts as it

relates to lateral movement data access. uh some more rather tier ones not tied to the Windows operating system but as it relates to you know remote access tools. Uh we talked about how we can you know take one single collection step in the beginning of our initial triage uh IR life cycle and how we can take that data and output it into one master result file. you know they can use your master timeline just call data from it as you so wish or add on to your master timeline from there in a more generally reasonable format uh such that it is already all normalized so you don't have to go ahead and you know normalize all

your other data sets um you also can visualize that data uh and however so you wish we presented one example of visualizing into a node edge graph uh and likewise you can use that data to uh analyze you know report to your tech lead, report to any other additional stakeholders um that you still have. Uh you know, where can we kind of have you folks contribute? Veract is an open source community. Uh so, you know, there's always going to be additional Windows artifacts. They're discovered especially as Windows adds on new features. Uh so if you folks have an idea of a Windows artifact that you want to, you know, go ahead and parse out or

create, uh there is something called the artifact exchange on Boss Raptor that you can go ahead and contribute over into. Sorry. Uh we also only talked about Windows. So there's obviously other operating systems out there, other platforms. Uh so you can make kind of the same exact idea but for a Linux environment, for a Mac OS environment. Uh lastly, we've presented a very rudimentary and on the GitHub I have a very rudimentary example of a visualization. Uh but the really the whole idea is that you know everyone likes to visualize their own way. There are other procedures, there are other uh tools out there available. So, if you folks have one that you like to use or

have one in mind that you would like to try out, go ahead and contribute back to the greater environment or community. For those of you want to pass a fishing cast, uh that uh QR code will take us over to uh the our our GitHub account which is listed on screen as well. If you don't want to scan the QR code, >> u we'll get back to this if you don't have a chance to. Uh a few shoutouts. uh some folks that helped us inspire the way that we wrote the actual VQL artifact control on screen. Some of these folks are pretty well known in the veloc community. Some of these folks have made some pretty

cool artifact definitions available in the artifact exchange as well. So something to uh you know give a shout out to. >> All right. Uh here are the meme sources like yeah the hide the pin hat my favorite. That's my representation when I work. Uh our testing included the uh second to latest version of Velocraptor. Uh the latest was updated uh a week before. Uh and then for Windows machines like we use Windows 10, Windows 11. Uh again like if you found this a little bit helpful like we have uh a whole lot of thought leadership coming out. We continuously talk talk at different conferences. uh we will be presenting this uh lambda as well as like another

tool that I have at sector or sector uh black hat Canada uh like uh in couple days so if you are there uh catch us up you don't have to sit through our presentation yeah say hi uh again you can follow us on LinkedIn or uh Twitter whatever we have presence in uh certain countries like level blue uh again like Costa mentioned is a world's largest MSSP MSSP. All righty. Thank you so much for joining us and uh again uh feel free to reach reach us out on LinkedIn. Uh we are easy to find and uh thank you so much. We are open to questions. [Applause]