The Log Rings Don't Lie: Historical Enumeration in Plain Sight

Okay. Uh good afternoon. This is uh the 2 PM talk on the blue track. Uh the log rings don't lie. Um it's about logs. I think that many of us who are familiar with logs, we use it to our benefit, but also how can this be used by the attackers point of view. Uh let's welcome uh Leon Proco. [applause] Uh thank you all. Uh thanks for being here. It's I'm very happy to again be part of uh Besides New York City. Yeah, for those of you that haven't realized, yeah, I will be speaking today. I guess my badge tells you. If that doesn't tell you, there's a couple of slides that will will let you know. But yeah, uh my

name is Leon. I'm a security researcher, cloud security researcher at Exa Force uh where I uh my main job is to find new sort of ways on how to utilize cloud or how to use utilize cloud uh resources as attack uh uh as attack vectors and then try to find new ways on how to detect them. Now uh in this uh in this topic we will talk about how logs can be used as an enu as an enumeration method. This is uh I I should preface by saying that this research is not complete. This is just one part of the of a large research that I'm currently doing on how to use logs as different as a way to commit different

sort of attacks like xfiltration enumeration uh command and control xfiltration infiltration. So this will be just one subset of that and then you can also ask me questions about the rest of the the research that I'm I'm currently doing and as I said as part of not finished research there are a couple of stuff that I'm aware are not included in here that I will try to talk about but of course just you know bear me bear with me on that part. Also, this is a part this is part of an ongoing uh research trend that I've done for the last couple years where I take different uh cloud features and I try to weaponize

them. So using uh what the target will have so either defaults or what the target is currently uh using and turning them into the attackers into attackers tools. So basically a sort of a living off land with uh with cloud features. Now cloud has each cloud provider and uh yeah another preface is that we were talking about the main cloud providers here AWS, Asia and GCP but the concepts in here apply to basically all the other cloud providers. This is just a template and each one of them has if I'm not wrong the last time that I saw AWS alone had about 480 uh services but correct me if I'm wrong on that one. So we're

talking about 400 plus services on each cloud provider. Each one of them or at least not most of them each one of them will have their own logs. Uh each cloud provider will have a main logging uh service and then they will have separate logs for each service that they will be using. So uh the job of defender incloud is to actually find out what is the best way to to get into that and as you can see I'm also getting uh giving some examples there that you know are just examples like uh DNS logs uh storage logs which do exist on all of the cloud providers but as I said those are just a

small subset which boils down to the each cloud provider uh AWS for example as I said too many logs two main services that you usually use for AWS are cloud trail and cloudatch. Now cloudt trail is the API the AWS API logging service. So due to the fact that in cloud you manage everything through the identity and you manage everything through the identity through the AWS API you need to wait to actually log those that's what cloud trail does it will log all the events that are being done on the AWS API against specific privileges that you do have cloudatch on the other hand is a service that just collects logs from other places it's not the only way to

collect logs you can just put them on on S3 bucket or any sort of storage. You can just forward them wherever you want. Cloudatch is helpful. Very sorry due to the fact that you can also query them. You can easily access the logs with a caveat that it does have a retention period. So you will and also a higher cost but you kind of get what you you know you either get a cheaper uh more uh permanent log source or you get uh more expensive and uh less uh less retention time with the caveat of uh of being able to query dashboard and everything. Now this is where the easy logging basics stop because we go on with GCP. GCP does

have the equivalent of cloud trail which is the audit logs and the audit logs uh contain four types of logs. We have the admin activity system data access and policy denied. Uh the admin activity are basically each API accessed uh by a specific identity to do a specific task. System events are just stuff that uh Google is doing on your behalf. uh data access are just uh I probably should have also put that one here. There's also one type of log in AWS which is called the data event logs uh which are uh events related to accessing specific data like accessing files in uh in a storage uh downloading uploading deleting everything. This is the

equivalent for for GCP and some more because I also saw some as we will see later I saw some other stuff on the on the data event and each uh each GCP organization will have some restrictions by default implemented and you can also implement your owns which they call organization policies. The last uh uh the last uh type of log will detect uh everybody that tried to to run something and was blocked by by them. Uh each of the audit logs will be applied on projects which are the equivalent of accounts in a in in uh in AWS. So places where you can put your own resources in folders which are groups of projects. Uh

billing accounts are basically for everybody that uses Asia subscription but not necessarily subscriptions are just uh a grouping of payment methods that you can that you can add on uh on Google so that you can uh apply uh basically pay as you go subscriptions on each on each of the of the projects or resources that you do and organization which is basically for the for the people that come from Asia or AWS same as the AWS organizations or the uh Asure tenants the the whole grouping of basically everything else. Other types of logs in uh in uh GCP uh we have uh resource logs uh and the resource logs depend on what type of resources you will have. You have

security logs, you have ser uh service specific logs as I said the resource logs. Uh you have uh transparency logs which are actions taken by uh again actions taken by by Google on your own account. And again as I said these are the only ones that I've tested. I there might be other ones. Sorry. You know this is at least as far as I' I've gotten. Then we got Asure. Asure for those of you that has has had the unfortunate you know uh yeah of of using is split into three parts three main licenses. You have the subscriptions the cloud aspect the infrastructure aspect of Asia you have the entra ID the Asia ID for

those of you that can still continue the tradition and uh the M365. Those are the three main licenses. All of them will do have uh logs of their own. You do have the your normal activity logs in in Asure. Then you have the resource logs again as we said. And then you have metrics which are just if if an instance has a high CPU, low CPU, if it's up, if it's down, you have M365 perview audit, which is basically uh the perview is uh a service that you use to collect all of your logs. And then you have the the uh audit logs. Uh uh in there you have uh the security system logs. Uh M365 comes with a lot of

products. Uh some of them are security related like the DLP, the MD, EMD, the the uh defender for for uh one drive and so on. And then you have other service specific which are not security related logs and a bunch of onra ID logs that as you can see here and uh all of them can be again configured uh based on what you want for this case uh we will use for entra ID we will be using the signin logs only again for reason of time but yes you have a lot of them uh a lot of logs that you can use now So bless you. Now uh as far as enumerating logs with sorry enumerating

resources enumerating cloud rel uh using logs this is not a new technique. This is has been used. This is actually something that kind of as attackers we think about doing. We are limited into an infrastructure. We want to get more information. We don't have access to get more information. We use what we can what we can get. Now logs contain information. Okay, logs contain information about different stuff that a specific identities are doing on a specific resource on a specific part of the cloud. Can as attackers utilize that? Of course we can. What can we get from logs? We can we can get a list of identities. We can get a list of sources where they

come from which can help especially when with cases where you have geoloccation uh limitations. Uh you can get uh failed attempts which means what you are or allowed and failed attempts of events which means what the identity is allowed and is not allowed to do. You can get information about their endpoint through the user agent. You can get information about other resources on cloud through the input and output parameters on the cloud. So on the on the logs so there are different stuff that you can that you can do in in them. And uh for this specific reason for this specific case we will be using four types of logs which are the AWS cloud trail logs the

Azure uh cloud audit lo activity logs the entra ID uh audit logs and the GCP audit logs. Now uh and yeah uh one thing that I did uh mention a bit previously is that uh for each cloud provider you will be having retention periods. Uh each cloud provider in the absence of you having a seam of your own will be allowing you a specific amount of time to have logs stored on their uh on the account or basically their own version of of that. And they do have uh specific I'm sorry specific retention time which comes by default which is usually 30 to to 90 days depending on the cloud provider and then it can be uh uh you

can use another cloudnative solution to extend the the uh the retention period. Now uh as far as cloud trail logs go what can you retrieve from a cloud trail log? uh each log and not not just the cloud trail usually will be outputed in a format that you would want. AWS by default does have uh JSONs but you can output everyone in in a format that you would want and uh uh for each one of them uh as I said you you can get different informations. Now in a what can we get from the uh cloud cloud uh trail logs? We can get the activity being performed. we can get the uh credential that they are using. Now

based on the format of the credential you can guess what type of identity it this is. Is that uh a permanent identity or an IM user or role or an IM user or is it uh an identity with temporary credentials like an IM role. Uh you can get the the identity's name. You can get each each cloud uh each cloud trail uh log will have a section itself uh where they contain all the resources that were affected and their resource types and the way that AWS uh manages them is through a format of AWS double uh two double quotes the service and then the the type of of resource which uh makes it easier for you to sort of create a

graph even collecting those. Then we have the second part. We can get uh information about uh the account that this operates into. You can get uh information about again uh the identity that is executing that is the identity again using temporary or permanent credentials. You can get information of the IP. You can get information of the request and the response parameters which are the input and the output of each execution. almost DJ executions, some of them do not give you outputs by default and some of them, especially the ones that are being done in the back end by AWS will usually show null in one or both of them. We have seen a lot of

cases of those especially the the one that comes to my mind is the quarantine policy where the policy is applied the execution is being done by AWS and the request and response parameters will be null. But if some execution is being done by an identity, for the most part, you will at least have uh request uh parameters. Then we have GCP logs. for the GCP logs you can get exactly the same type of information that you would get before just that right now you do see different types of identities that are being that are executing the uh uh the event in uh due to the fact that in Google you can have uh permanent credential users which

are called uh service accounts you might have that as the identity executing or you might just have a user that just logs in to the uh authentication to the to the API and then just executes that. In our case, we have a user at domain.com which uh if we continue down that down that line, we can get again a list of of names that uh that can be can be retrieved. Again, we can get the the event being executed. We can get resource information and we can get uh and we can get the the res uh the the response of the execution. Again same sort of information as we we retrieved earlier and this continu continues again

on the Azure we again get the UPN which is basically the the user's email the yeah the user's email in cases of uh a service principle it will all if I'm not wrong it will be the client ID correct me if I'm wrong on that one you can get the tenant they're operating on you can get uh network information you can get the events you can get the status if it succeeded or not now we will see uh the reason why I'm continuing to talk about the status is because we have we will see a technique on how to actually programmatically enumerate privileges using using logs. Uh yes. Uh now uh one other thing that we can do

uh as we said before we manage cloud through the identities and we manage cloud no we manage cloud through the through the API using identities which means in order for us to be able to manage a specific part of the cloud we need to have those permissions to be able to do that. Now in all cloud providers there are two main groups of permissions. You have the API permissions and you have resource specific permissions. Both of them both of the permissions can be added and they can be removed. What we can do is we can go and search for events which do have those addition and by the way I should have also added the

the removal in here and we can again create a graph of our own of what a specific identity is allowed and not allowed to do and what resources they are allowed to to access and what resources they are not allowed to access. So in case we have an identity that we are not sure what privileges they they might have just looking through that will give us uh an idea what I said before uh due to the fact that we have uh statuses of uh responses for the specific task that we do uh we can go and search through all the logs and then try to find out which ones of them are allowed or not. Sorry hopefully

you can hear me here. Now whenever you are searching through logs usually they will have a specific limit. It might be a thousand it might be a bit more you can read at once you actually okay then sorry yeah I thought you could hear me. Yeah. So uh I will try to okay I will try to move it like this. You will be able to retrieve those logs. Now if you go and retrieve the logs what you can do is you can list them and by by default each log will be ordered reversely from the newest to the oldest and then you can go through all the logs for each identity and then say is this

event being allowed or is this defend being this event being denied. If that is denied and the uh the event is newer and we we are you know uh the identity is not allowed then we can just go for permission denied. If the event is allowed then we can say that this event is allowed. So right now we can just go and query and do uh go through a loop and query all the identities and all the the permissions that they will have and then see which identities does have each permissions. Now this comes with a big caveat the fact that not necessarily somebody will execute something that they have access to and we do have cases

of uh permissions being temporarily given and then later later removed. So I might be given access to a specific type of a permission I execute that that is removed I don't execute it again the previous example will just show that as permission allowed though that permission is not. So it does come with a big caveat but for the most part especially for permanent or for long-term permission giving you will be able to actually find out what an identity will be allowed to do and what not. Secondly we have the sign-in logs. Uh each signin log very sorry here each each sign-in log will contain some information regarding the identity that will try to sign in. Now different cloud

providers have different methods and different services that do manage sign in into cloud and uh as a you know as an extension they will also have different logging for for that uh for that purpose but for the most part what you will be able to to get from that is if the identity is using uh a specific mechanism like an SSO if the identity is u the name of the identity again the the endpoint information due to the user agent. Uh you will be able to find out if they are allowed or not. And from the error field you will be able to find out if the that identity uh just failed attempts because they just

typed mist you know mistyped the password or if they were logged if uh if the if the identity is not allowed to login anymore for example due to the fact that they are uh they left the company. We will be able to find out all of that just by by quering through through logs. Now uh going again through each cloud providers uh AWS has three ways of managing and just one event that we care about uh for managing user signins into the into the AWS console. The first one is through something called the login profile. uh AWS IM users right now in AWS are sort ofish in deprecation feature but people use them a lot. Uh IM users are credential

are identities with permanent credentials. uh if you want that user to have access to the to the management console to the GUI the browser to have browser access you can give them what is called a login profile which is a fancy name for saying you can just give them a password and that's it and then they can just go login and then you can also have you can give them uh some sort of MFA when they when they log in second one is through a technique called uh federated credentials uh for the people that have used uh the net SPI AWS consoler, you kind of know the technique, but basically the idea is you get a set of

credentials through a specific API which is STS get uh federated token if I'm not wrong get federated key and get federation key and then you pass them through the sign in through a signing process in AWS documented signing process in AWS and they will give you back a URL which after passing you to be able to get uh access an identity going through this process do does not have to have a login profile to get to get access to the to the AWS console. And the third one is through the identity center. Basically uh whether you can connect the identity center through an IDP like octa or just using that to login using that you can

just uh login through an email a password or some sort of uh other factor authentication and each one of them will have different fields that will be you will be able to detect uh in the in the in the cloud that you know what type of identity this is is this is for uh login profile. file using an IM user and authenticating with a login profile. Two of the main things that you will be able to notice are one the fact that the AR does show user in here and secondly each authentication through the login profile will have a field on the additional data of login to. This is where you are allowed to log

in. Basically that's that's just saying that this is how the credential managed to to access the management console which is different from a federated set of credentials. Now for the people that see that if you have ever used pu you will see that imported dash is actually an event done by pu. PCO does put imported dash on each session that they get. But that part there imported attack user can be anything you want. This is what AWS calls a session name. Basically, when you get a session of any sort, you assume a role, you get credentials, you get some sort of temporary credentials. You can put any name you want. That can literally be just attack user and it's going to be

fine. You can see again that the principle is the imported, but the AR in here says federated user, which this user is not. This doesn't have any sort of uh on-prem to cloud uh uh connection. This this infrastructure doesn't have any on-rem to to cloud connection uh to to go uh to go and authenticate stuff. It's just one API call that allows you to do that. And as we can see again here, you do not have the event the the field of login to because this identity did not technically go through a normal authentication. they were given an authorization URL to go and access the the uh the uh management uh console. They did not go the their authentication

process is just the permission to execute the call. That's literally what the the authentication per uh process is to them. And lastly, we'll we have the identity center. Now for the people that have used uh AWS and have configured the identity center, one thing that you will know is that this can be altered of course but by default the identity uh the each uh identity or all of them together will be given a role and that role will have specific permissions for you to to access the management console. By default they do have a format of AWS reserved SSO underscore administrator access underscore some alpha numeric characters as I said that that can be anything but

one thing that you do see differently from uh just a role trying to access something is the fact that in the end the name will be the uh the name part of the email of the identity that is logged in which means this role here can be pro can be given to anybody on the on the company. Bion is the one that access this because Bion through the identity center through an au authentication method was authorized temporary credentials to access the the uh the management console and they were given this session with this with this role here. And again as I said before we do not have a login to field because again we did not go through the normal

authentication process that the login profile will go to. We went through another mechanism which technically is a service in AWS but it's it's not the service that allows you to login through through the uh account user and password and the entra ID uh entra ID uh one field that is important or one field that is different from onra ID is the additional details now additional details contains information regarding the uh the output of the authentication from a specific ific identity and you have these are the ones that I've managed to find so far and I'm pretty sure that there are more I just you know these are the ones that I've managed to to collect through all of my my tests

that I've done you will have uh a complete login MFA accepted MFA not accepted uh the the user you will have uh a message for the user checked I you know remind me again which means you can understand the flow of somebody getting you know logging in into into an an infrastructure and for the uh GCP uh now when you log in through a user in GCP you your events will actually show as Google workspace events but strangely they will be part of and I should have added that sorry they will be part of the data events which kind of weirded me out because the data events technically were supposed to be data related uh data access related

events, they do store those in there. Uh, one good thing that they do have differently from others and the reason why I'm not showing all the fields in this case is because I'm just showing you differences between one and another. You can get the same type of other information from all three logs. What I want to show here is what other information you can retrieve from uh provider specific logs. In this case, what they do offer is just uh login success or failure. So you can just filter out whatever identity you want and just see if they are allowed or not allowed and for what reason they weren't allowed to to log in to the uh to the

organization. And yes, in the end, as I said, as a technique, now I'm pretty sure some of you might be thinking, okay, what why the hell did he just come up with all of this? As I said, as a technique, this is known. And to my knowledge there is even a mit attack uh technique just for log based enumeration. What I'm trying to show you here is how you can achieve that uh that technique on uh specific uh providers in cloud and how you are able to to do all of that and what information can you retrieve from each one of them based on authentication mechanisms or access mechanisms that you will be uh having on

uh on those cloud providers. And uh if you are thinking about how do you detect or prevent this detection in this in this sort of technique is not that easy due to the fact that a lot and I mean a lot of identities especially programmatically will be allowed to access logs and there are roles and this is one part of the research that I'm working on there are roles that specifically allow you that do not that are not uh monitoring related that do allow ow you to list specific kind of logs due to to write to read or write specific kind of logs. So uh you will be able to achieve a specific task with just minimal access on on the

on the uh cloud providers and very sorry if I brushed through that you will be able to achieve everything through just this much events. For the people that do penetration testing in cloud, you will know that cloud is annoying except for Asia, but that's that's another thing. It's annoying when it comes to enumeration due to the fact that cloud uh handles permissions in uh whitelisting matter. you are not allowed anything that you then you are allowed to do stuff which makes it really hard because either you are just going to just fuzz permissions and good luck doing that when I'm very sorry the permission is the right permission which means you will be tempering something or god

forbid deleting something uh so you will need to find out another way when you are not uh when you do not have access to know what kind of permissions you have and this is part of the research that that I'm doing. And as far as preventing goes, the best way to prevent, I'm very sorry to say, is the is a list privilege. So just be aware of who you allow to to read logs. It's not easy to do. And I'm one of the people that just say it's not easy to do that, but it's literally the only way to do that because otherwise you will be allowing uh somebody else to to read the logs. You cannot just blindly

you cannot just blind the defender by not logging. You should log because that's that's you know uh information important information for you. But what you can do is quite literally just go and uh be aware of who can read those logs so that the people that need them can keep them and the people that don't need them cannot abuse those. And with that, thank you very much. I guess it was a shorter uh presentation than I expected. Uh thank you. I don't know. Do you have any questions? [applause]

>> Yeah.

Yeah. So,

>> yeah. So, for those that didn't hear that, the question was, if I'm not wrong, how common is for threat actors to actually utilize those in their day-to-day uh process? It's basically their day-to-day process. So, a threat uh threat hunter, threat, sorry, threat actors or threat hunters. Uh threat actors, I don't know of any threat actor, at least at the top of my head, that does utilize that. uh as I said it's not easy to detect due to the fact that if you are allowed to specifically do something and you have done that before uh it's kind of hard to to find out if it was you or somebody else that did that. To my knowledge,

there isn't any there isn't any threat actor that that did that. But I would presume that some sort of log enumeration, some sort of them uh is being done by by threat actors even when it comes to just debugging the output of a specific task that they that they did. So I believe they might be doing I just don't know at the top of my head.

Okay then. Thank you very much. Thank you. [applause]