GT - Can Data Science Deal With PAM? - Leila Powell

my name's Leila and I'm a security data scientist from panacea and today I'm here to talk to you about Pam Pam is privileged access management essentially these are the tools and processes that allow security teams to control the assignment and use of administrative privileges so why am I talking about Pam today why is it so important well it's one of the top five critical security controls and actually went up in the charts and the latest version up to four four five but that's not the only reason a Pam is one of those things that's kind of captured the interest of the board it can be notoriously difficult to get the board to buy into security and the

importance of it but for Pam the concept of a super user who can wreak havoc on your business critical systems because you effectively given them the keys to your front door is one that really grabs their attention it's easy concept to understand so now security teams find themselves thrust into the spotlight after longing for some attention on the board for many years and now they're trying to deal with Pam under time pressure and under scrutiny so today we're going to reframe Pam as a data science problem and see what we can do with data analysis to help the security teams deal with this challenging area today I'm going to take you through a number of areas first of

all Pam panic why is it so hard secondly we're going to look at how to be a bit pragmatic with Pam it's a massive area so we need to bring down the scope a little then we'll talk about how to do Pam in practice I'll talk about what data we're using what analysis were using and the benefits to security teams are getting from this and finally I'll recap a bit before we finish up so Pam panic the way to understand some of the reasons why Pam causes so many challenges is to first of all start to look at what good privileged access management looks like so let's have a look what you need to do

to do good purpose access management all you need to do is determine approved access paths probably install a password vault or one-time password system to reduce risk exposure monitor you to push access everywhere at all times identify access patterns outside of those approved notify we educate users probably also their bosses we work out exactly who have access to what when and remove all privileged access you deem to be unnecessary and all the while do not disrupt business as usual so faced with this kind of situation security teams confine themselves effectively in their own version of a horror movie so why is Pam so challenging many reasons but here's my top three first of all in other areas of security

the goal is to drive things that I'm to zero so vulnerability management another challenging area you want in an ideal world zero vulnerabilities never gonna happen but fine that's the goal in Pam you can't aim for zero privileged access you have to have some privileged access so people can do their jobs so people can patch service or do what other changes are necessary so it's particularly because security teams have to walk a line they have to find out what is the minimal acceptable amount of privileged access so that people can get their job done while still keeping risk low and this leads us to the second reason why Pam is so challenging working out what that minimal acceptable level

of privilege accesses is really hard first of all because it's really hard to know who has access to what the way permissions are assigned in large organizations can be infinitely complex you have the hierarchical nature of Active Directory you have groups nested in groups nested in groups you have companies growing by merged in acquisitions you have different systems across the the kind of sub groups within the organization different areas of the globe people can get local access assigned directly for a particular job and never get it taken away as one see so said to me we're really good at giving people extra access so they can be efficient at their job which is not very good at taking it away again and

this brings me to the final reason well of my top three why Pam is hard if we look at how permissions are at the moment you basically got this kind of Jenga tower that's tottering around that all business as usual is built on top of but from the businesses perspective it's kind of a if it ain't broke don't fix it situation everyone can do their job right now so why are we going to try and pull some of those Jenga box out and I hope the entire thing doesn't fall down the reason for that is from a security perspective it is broke having all these additional permissions that people don't need is a massive kind

of attacks if it's a massive security risk so unfortunately we do have to go and play Jenga so how can data science help security teams trying to do this first of all we can help with that visibility problem you know I said it's really complex to understand who can actually do what so by going getting datasets from across the organization linking them together effectively we can give some much much more complete accurate and timely visibility on the permissions situation we can also start to look at how we can use data visualization to allow security teams to consume this knowledge better so if you just have a list of permissions it's quite hard to understand what that

really means what the impact in other areas will be but if we switch to something like a graph visualization suddenly people can consume a lot more information a lot more effectively and finally we can look at decision support this is where we move on from saying hey here's a really clean accurate nicely visualized set of data go and do your work - saying hey how about we run some more analytics and try and provide you with a prioritized list of areas to address so it's really starting to take away some of the manual work so if this is a data problem what our data challenges the first one is what I like to call do you even log and as with the

first thought this morning I'm gonna just refer to John Hall's talk on opening the track yesterday to tell you why it's really hard to get data and get data where you need it to be and I'm not regoing to say any more about that the second challenge is the variety of datasets that need to be combined to have a good view on Pam as you'll see later I'll be talking about things like event logs Active Directory HR data identity access information and things like a asset inventory or CMDB finally when we talk about privileged access management we're not just talking about devices this is about application access to databases and applications as well each of those layers of the stack has

many flavors we've got windows linux unix mac OS we've got different flavors of database we've got all your in-house application gooeys that we have different ways of logging and different formats we've got to try and get all that day which is kind of problem one but then we've got to understand it clean it analyze and bring it all together in some kind of care and picture so you can see really where the risk lies and finally we've probably got multiple ad domains as well so that's just another another potential issue to contend with so as you can see the scope of dealing with Pam from a data perspective is potentially absolutely enormous so we have to be a little bit pragmatic in

terms of how we're going to approach this so the two kind of pragmatic principles we're going to go with a start slow and start small so start slow if you think about the three V's of big data volume variety and velocity you've already got a bit of a variety problem as I already talked about so you've got enough of a charge to kind of deal with that so the suggestion the approach we're taking and our suggestion is forget about velocity for that you don't need streaming streaming log events first of all why not just focus on getting the variety right pick an update frequency if your analysis that's suitable something like a day would be

fine the second reason for forgetting about kind of streaming and velocity if you like for now is there's really two phases of applying security metrics when you're talking about the basics of having good cyber hygiene the first step is where the security team first gets visibility on an area so there's a new analysis program there's new data coming in and effectively that's kind of like lifting a rock and looking under it you know it's gonna be horrible so there's gonna be a lot of cleanup to do the second stage is after all that cleanup you kind of just trying to maintain your situation you know you've lowered your risk everything's looking good you're met you're happy with your metrics and I

just have to keep things ticking over so when we're in the first situation which we are as soon as Pam comes onto a radar for most security teams you're going to focus on cleanup which means you absolutely do not want anything like real-time monitoring because it'll just be a bunch of noise so here's yet another reason to just you know start with say daily refresh of your data because slow and steady is gonna win the race in this case so how about starting small as well what I mean by this is focusing on a few systems that are important so look at this very complex Venn diagram here you've got a systems you care about

on the left and data you can get on the right and basically where they overlap is where you should start doing your analysis so an example of this might be suppose your company has I'd know a few payment systems that run off Windows servers and you know there you can get the event logs for those pretty easily then focus on that focus on doing this kind of analysis around Pam on those servers that's not to say however you shouldn't start to request the date you'll need to expand that to the Linux servers or to the databases or the application layer themselves because you'll probably be a long time coming but the point is don't wait to you have

all the data to solve Pam across your entire organization before you start to do something as many of the as many of the other talks of demonstrate there's a lot of power in just doing a POC first of all and proving value so systems you care about this approach we take is to focus on business critical systems for this now of course there's other aspects to provide access management there's things like there's always a risk of something like if you you know if we if we put low priority systems out of scope there's always the risk that attacker might come in through your aircon system or your IOT toaster and move laterally across your network but we're not we're

not addressing that here we're securing our kind of crown jewels if you like and we had a talk earlier that kind of talked about lateral movement detected that so this is not what we're looking for here but they're still a bit of a problem how do I even know what my business critical systems are if I had to ask you all to go back to your organization today and get a list of all the applications databases and servers that were business critical how successful would you be because in my experience as security teams I'm working with they just simply don't have access this information and this makes their job even harder how can you protect

things if you don't even know what you've got so trying to trying to rely on a kind of static CMDB or inventory that's often has to be updated by hand is really problematic you get situations where you know you might start to look at something analysis we're doing and people will say hey why is everyone logging into this production server and then you go to a team and they say the dev server it's just misclassified so I mean you can't get anywhere with this now building a kind of smart inventory is an area that could be a talk in itself so I'm not going to cover it in detail I've just linked a blog there by one of my colleagues but here's

a kind of quick intro to it so you can see what I'm talking about essentially when you're doing data analysis for other parts of security program you'll be getting data about your devices and if you combine that data particularly from tools that have a different perspective on devices you can start to get quite a rich picture of what's on your network so for example vulnerability scanners will go out scan a range of IPs and might find things that weren't even registered in your device inventory' in the first place then you've got a network based data you can start to see what's talking to what maybe you can start to infer the importance of machines based on traffic

to you and from them then you've got things that sit on the endpoint maybe now you can get an accurate read on what the operating system is maybe get something like a MAC address you start to get more solid identifiers they don't change all the time like ip's do essentially this is a process you can go through to start to build up a better inventory or challenge the one that you have to work with in your organization at least when you have this you can start to go to a business owners application owners and say hey what's this server that sits in your region and your team is logging into on a daily basis what does this do how important is

this so assuming you've got your smart inventory or at least you phoned a bunch of people and cobble together a spreadsheet we can now move on to Pam in practice Pam is a many-faceted topic so what we've been doing is working very closely with our early adopter customers who we really considered to be development partners and we've seen the areas of Pam that they wanted to focus on a prioritize first and working with them we're trying to understand how we can use data analysis to facilitate that so the two areas that we're working on together is evaluating kind of what's going on now so who is logging in to these you know business critical devices

we've identified and then discovering the art of the possible where possible is normally bad which means who has who has administrative permissions on these key devices so let's start with what's happening so kind of entry level data to be able to solve this problem you're going to need your event logs so I'm going to focus mostly on how we're doing this to for devices just in the interest of time and not reading out a bunch of lists but just keep in mind that you know Pam covers all the other types of logins as well not just not just device stuff so for Windows you've already had quite a few mentions of Windows event sort of

log forwarding NX log of things like there's ways you can collect this data in previous talks you can also use syslog to bring these events in once you've got this date you then need to identify the events that you're that are important to you and now case it's successful logins so with these we can use Windows Event IDs so clearly pull them out for syslog it's a bit more complicated you have to start to pass the event message itself but anyway once we've got that we need to define what access is allowed so we know who's logging in we know which accounts are logging in to use key devices but are they doing it in the appropriate way or

not so if you remember I said right at the beginning that when the key things are Pam is to define what allowed access paths are how people should log in from which count and from where and this is really something a security team has to put in place this is all about policy so this might be where security team has installed a kind of password management vault and then you know that only the accounts managed by that vault should be used to access those systems or maybe there's been there's a jump box has been specified or maybe there's a certain type of account we should be using whatever the rules that the security team is decided upon we can kind of

codify those and then classify all the events the login events that we see as kind of good or bad the other thing we can start to do if we bring in a little bit more data it starts to ask who so you can see what what accounts logging in but who actually owns those so kind of next level data we can go and get identity and HR data and go back to that system inventory or Excel spreadsheet whatever you're working with to see who owns and maintains a system the kind of things we can do with this from an identity and access management tool we can start to map actual humans to the accounts that are we can see

logging in this is valuable for a number of reasons first of all again one of the whole ways of doing Pam is to re-educate the users to use the approved paths so now we know who they are and we can send them in email be like hey you should be logging in this way we can also find out from HR database who their line manager is who the accountable person is now you always need this right if you're gonna drive changing an organization you need to have someone's ring up and be and say like what the hell is your team doing this is always important for things to actually work in practice within a business the other kind of angle we can

go at is trying to see what team the person works in or we can go back hopefully to our system inventory and find out what team is responsible for maintaining that device and if the person that's that has that account is not part of the team that's administering that system why on earth are they logging into it you can also use this the other way around as well and say look at everyone in the team that is responsible for managing this Linux server do they all have the right access or one day when they come to do something critical will they find they're locked out and this is particularly important when you're making changes because remember we have

to preserve BAU so what's a security team get from all that kind of piece of analysis we're gonna get information about all the login events to business critical devices by unapproved Access Pass they're gonna know the person that's using those unapproved routes they're gonna know they're accountable manager so you can start to produce reports and sort of retrain people and they can start to validate if peoples on teams have the correct accounts and permissions to allow them to do their job the second phase is looking at the art of the possible so just because you can't see and someone logging in it doesn't mean they have administrative permissions that they shouldn't so now we actually want to go and see who has

local admin rights on these devices they took in a need for this is groups in the cat's local admin permissions the ad domain of those groups because you know in a multi domain environment you can't assume that the group name is going to be unique so we need to check we're kind of querying the right data and also ad group membership data so in terms of the group membership data we're just going to and queer Active Directory direct directly and get a list of groups and you know you have a group membership list and you can kind of walk the hierarchy if you like to find out for each group what at all its members local admin permissions for

Windows turns out to be fairly straightforward to go and get many vulnerabilities informational vulnerabilities that you can use to retrieve the list of local admins all these things like OS query or you can use endpoint protection where you can query agent to find out properties of device like who has local admin permission it turns out doing this for Linux and UNIX is considerably harder you still use the the approach of going in via an endpoint tool however we're now trying to pass out the sudo as file the location of which differs depending on which distribution which version of Linux or UNIX you are using and it's very complex it's not just a list of who

has permissions it's got a complicated syntax all of its own where you can say this group can do this but not this and this user can do that but not that and this group and that user but not that group and that user and you can define aliases for groups and users within the file so this is like a really messy kind of data data cleaning problem but assuming you get there in the end we can combine those two sets of data somewhat like this we've got the data from the local admin or Suda's file so we know which group is directly assigned permissions and by looking at the hierarchy we pulled out of ad we can see

who else has inherited permissions and if we do that for multiple devices we start to get us in there looks like this this is just this is just an example but it's very much representative of what we see in real data and although this is not the most complex example I've seen that would just be basically millions of lines it's pretty messy right you can see that there's only free there's only three devices here and already we've got an absolute kind of chaos of lines and group memberships the kind of things that security teams are trying to clean up are as follows this is just a simple kind of example to show a few specifics

you've got directly permission to account here this is this is gonna be bad because it means to have any visibility that this account has admin permissions on this device you need to go to the device itself and the same is true if you want to revoke that access the other kind of problems we see are accounts of multiple paths privileged access so because of kind of group nesting and multiple group membership certain accounts have multiple ways to get the same permissions and in in real data we've seen examples where one account can have twenty to thirty different routes to local admin on the same device this is a problem problematic because it's hard to manage

and also if you think of that cleanup mode I talked about every one of those paths is almost kinda like a wire and a bomb that you're trying to clip as you go along we are exploding you know business process somewhere else because you suddenly shut off someone's access the second they needed to to be able to do this is a good start what the security team gets in this kind of analysis is they can see all the paths to local admin on business critical systems and they get a kind of list of all the permissioning problems that they want to that they want to remove and their location the challenge here though is the scale right how do we find all those

permissioning problems in this but for every system remember there's only three devices here and without impacting other systems so if you think what we're doing here is using graph visualizations to say for these given devices showing me everything that has local admin and you might see a group that doesn't look very important yeah we'll just take that away but out of your view of your current kind of you know data scope there's some system over here which you've just shut off access to and tomorrow you're gonna get a bunch of angry phone calls that no one could login to their production database it's a real challenge here which is one we're still working on we

can't show all the the entire graph of permissions in one go because it's simply not human readable but we need to come up with a way to try to give context about what else is happening so the next steps on this and things were working on at the moment is tight trying to help further help security teams untangle this essentially was trying to take them away from having to manually view all that lovely graph data and this is where we get onto this decision support start part which is a work in progress one of the things we're looking at is a what-if analysis can we provide effectively some kind of sandbox where people can play around with the graph

without actually touching the permissions themselves and they can see the impact of changes they make we can also start to look to kind of standard graph theory we can apply path finding algorithms to find shortest paths we can find examples where between two nodes an account and a device there's many many different connections so we can highlight these these can be removed first and we could we're also investigating I also want to look into if we can see whether using techniques like clustering of users and groups and the ways things permission can identify outliers for so we can start to see if certain things jump out and basically we can prioritize these so security teams

have a bit less manual work to do so what should you take away from this talk first of all access to data by instrumentation is essential check what you can get your hands on first ask for more but don't wait to get started don't boil the ocean that's probably my number one piece of advice for any data science problem ever try to focus on say daily reporting or whatever time scale seems appropriate for you and focus on your your business critical systems try to demonstrate value early and this will make it much easier if you to justify getting more data and finally Pam is essentially a bit of a data jigsaw puzzle you can do quite a bit with only

a few pieces but the more pieces you put in place particular in terms of being in business context around application or device criticality identity access management information HR your device inventory the rich of the picture gets and the better the security teams are able to prioritize and clean up this challenging area so I'll leave it there you can find me on Twitter I'm around this afternoon and happy to chat and also if you have any feedback or suggestions or comments on how you're doing this yourselves be really interested to hear them thank you [Applause] does anyone have any questions we can take maybe one or two um so you were looking at path finding algorithms if

you looked at minimum cut no no yeah so this is really this is kind of the next steps to a tilaka but have you talked at all to the bloodhound guys no so I think I think blood time is an interesting one because they're they're doing something although it has parallels it's quite different so I guess they're looking really specifically around in the active directory and kind of hierarchy we're just looking at membership basically they're looking I guess in in a lot of detail around how you can add users to groups how you can move across the active directory tree and elevate privileges so I think that's not really the kind of angle we're coming up but

certainly some of the customers we work with use blood towns in parallel with what we're doing here yeah and what were you using to visualize the graphs this one was I have two checks we're trying out a bunch of different bunch of different libraries so I think this one might have insight escape which is an open source project I can check with our own alfrid enemy team yeah we figured out all right Thank You Leila thank you guys [Applause]