GF - Security and DevOps are really Best Friends - Emily Gladstone Cole

with that let's let's get started awesome okay I can I can hear myself y'all can hear me okay super all right well first of all it's a thrill to be speaking it besides Las Vegas Wow thank you all for being here to hear me talk about how to make sure that operations stays friends with security the almost inevitable survey at the beginning how many of you have done some kind of operations or IT or sre type of oh great good show of hands how many of you are in here because it's a semi dark space that's away from all the rest of the hustle and bustle okay I thought there might be a few of those excellent

and I know that we have least one person who's here to cheer me on so thank you all right well I hope y'all are going to learn something today I'm hoping to have this be more of a dialogue if people have questions please feel free to go ahead and raise your hands we do have someone who will be moving around with the mic if that seems relevant if no questions Hey awesome just you know have fun so why am i the one who should be talking to you about ops and security well I started out in ops I was assisted min I did some IT stuff I used to take care of gosh um I ryx boxes

solaris boxes Dec boxes and then eventually moving on to Linux and FreeBSD and more Linux and more Linux after but I'd done that for a while spending a lot of time dealing with operations incidents I moved over and started working security incidents and did some security incident response I've done some security research as well and now I'm back on an engineering team I am working for a gauri we do email security and I am working with some DevOps and infrastructure folks to help get our culture and our more security aware and bacon more security so on to the interesting stuff okay so when I started writing this talk I submitted the proposal and I said okay

let's you know let's frame the discussion of dev and ops around the C is critical security controls super great you know most people have heard of them it'll be pretty easy well right after that I started at this new job which I have three months today and I started working with their infrastructure and I realized that there's a lot that just doesn't actually apply so the C is security control number one is all about hardware assets and number two is all about software assets and I started thinking about hardware and my company is all everything is hosted in AWS so we don't really have any hardware so I guess maybe instances or VMs or possibly even

containers can be referred to as hardware but then you get to things like AWS as lambda functions you know serverless is that hardware is it software I I wasn't quite sure and I decided that I kind of needed to reframe things so you know I had to kind of throw out that top and say alright let's let's think about assets some more even more than my dad who was a financial adviser and money planner had spent his whole date talking about assets so um I'm gonna be talking a lot about assets here and I just I realized I had to completely reframe what I'm talking about so out goes this agenda so the update of agenda I figured give at least

a brief intro to DevOps and sree and I'm gonna talk about some of the principles of DevOps and SOA movement and ways that they align with good security practices and then I'm gonna spend a bunch of time talking about assets how to define them how we want to track them monitor them what we want to do with them then some time I'm on least privileged and vlogging I have some additional material in case I talk really really fast and I get through all this stuff and y'all don't have any questions that is more about DevOps and what they want so standard disclaimer this is my opinions and it does not necessarily reflect those of my employer although if

I have anything to say about it you will and non-standard disclaimer I do hope you like Cap photos so core principles of DevOps and the sorry number one is it everybody shares the on call the doves the operations folks maybe even some of the QA folks depending on how much or your team is in your model is number two practice empathy understand where the other people are coming from and work with that and then finally automate everything you can you know write write everything as code if you possibly can and that will make things easier and we'll talk about pets versus cattle so on call back in the day you know the ops team was the only ones that were waking

up in the middle of the night and devs páginas kind of just say um no um it compiles it builds it's not my problem anymore and they they weren't suffering so even if something alerted every night because of a constraint that had been built into the code the Ops couldn't deal with any other way it was no this is this is an ops issue but with everybody sharing the on-call responsibilities we kind of from an Operations perspective they people kind of got I don't I don't want to say tricked into it but it got convinced to do it because devs wanted to have more ability to deploy code and not just sit there and and do the code they wanted to be able

to deploy for testing and then if they were deploying for testing then why not deploy to staging and then if you're deploying to staging why not deploy to production and then operation said hey folks if you're going to be the ones deploying the protection well maybe you should be the ones who are waking up in the middle of the night so that happened and it gave the ops team a better idea about what the devs were up to and it gave the dev team a really better idea about the challenges that the ops folks had been dealing with and it started bringing the teams together so that's kind of what we're we're going with the empathy which is

just you know understanding other people's perspectives and those those shared experiences of being on-call understanding the the real-world constraints that maybe didn't show up in the works for me it works on my laptop type of situation really helped start this this whole philosophy of coming from a place of understanding and support rather than having this adversarial thing or possibly even just a waterfowl model or waterfall model where things get tossed over the wall between dev and QA and then from QA to ops and they don't really interact back and forth and give gave the ops team an opportunity to do something that I've spent a lot of time doing which is writing code and so that brings me to

automation obviously if you automate it you're only gonna have to do it once and if you're a lazy sysadmin or a lazy security person you probably really only want to do that once and then you can move on to something that's more interesting and it started with cfengine to allow you to manage systems and users and config files and even software deployments and then puppet came along and there's ansible and salt and all kinds of tools that help you manage all of your infrastructure and even nowadays you can create instances or containers as code in your automation system so everything that you can you can do there is is part of the part of the things

that you don't have to manage manually and I just realized that I managed to delete my pets versus cattle slide I'm sorry about that I will just tell you about it so the basic idea was that either you can treat your systems as pets which is they get a lot of attention and they're they're all special you know their names and you're friends with them and you build up a relationship with them or you can treat them like cattle which you know get raised in a big herd you don't have as much of a personal one-on-one relationship with them and in the end they go off and do whatever the next thing is which is maybe feed people or

you know whatever it's it you don't have a relationship with them they're just they're just there to provide a service and one cow is much like the next cow but you know anybody who has a pet like pet cat maybe might say well you know no it's they're not interchangeable you know this one matters and that one matters and they're different so the the thought about pets versus cattle in an automation and in operations context is that you want to have your systems be more like cattle that you don't have to spend a lot of time and energy individually on each one keeping them up and patching them if you can patch all your systems at the same time after

you've done your testing to make sure that the patches are compatible with all the rest of your software then you're much better off than if you have to manually log into ten twenty a hundred a thousand systems and patch them one by one so treat treat them like cattle instead of pets and it's obviously it's also really great for for a security perspective because you don't want to have too many special things that that look different from everything else the more that you have everything the same the more you can use basslines and know what is what's supposed to look like and what it isn't so that's that's one benefit of having the the cattle model versus the pets and

of course you're always going to have one or two servers that you have to take special care of but if you can you know do as much as you can to get minimize that the better so these are some books that I have found are a lot of fun of learning about DevOps and uh sorry I do have them in the references slide at the end I am always happy to talk about books I have a bunch more that I've enjoyed and that seem to be valuable so there's plenty out there to read okay so on to assets what even are assets so um I had to think about this you know there are so many things that in this personal

formulation of mine that I'm calling assets it's not only the physical services servers the appliances the VM is the instances the containers lambdas even things like s3 buckets cloud IPS load balancers weather cloud or physical I'm calling domains also assets I'll get back to that a little bit later some more but also SSL Certificates and potentially you might want to treat your software as assets to things that you things that you have that you you need to keep track of for you know expiration dates or other things so it's um one of the things that you think about with all of these things is you're paying for them it's some way or other you you

bought it initially or you're paying a fee for it being up so that's that's kind of my framing on this so that's a quite a few lionesses and cubs and you want to be sure that you know where all of them are I mean if you're a ranger in a Conservation Park that has a bunch of lionesses and Cubs you're not going to want to let the tourists go drive around on Safari if you don't know where all the lions are because if you do maybe that you know they could get eaten then that would of course be bad you don't want to lose any tourists or have anything dangerous happen to them so these are these are some things you pay

for you do definitely want to attract them so you want to make sure that you're paying only for the things that you're using if you've done your Bluegreen deployment which is the the model where you have one set of systems that is running in production and then the other set of systems that's running the new version and you swap and now the one is in production the other one isn't well now you've got a bunch of sets whether their instances or containers or servers or whatever that used to be production they're not anymore well do you turn them off so that you're not paying for them anymore you know do you immediately roll them into the next

version you want to make sure that if you're not immediately using them for something else that you turn them off so that you don't get charged for that you also probably want to make sure that your you've got your the you've got as I said you've got them all shut down when you don't need them anymore another reason to keep track of your assets this you want to make sure that you're working on the right environment I mean some people are lucky enough to be able to pen test against their production environment most of us have to do our testing against staging to make sure that the customers aren't actually impacted well how do you know for sure

that you're testing on staging you you know if you're like me you have a lot going on and you can't necessarily keep up with everything that the the infrastructure team is doing and what is production and what isn't now the ops side engineering side will often say oh well we can just go to our cloud provider and look it up there or we have this cloud tool that will do all this kind of stuff but that tool is probably not going to give you any way to show who actually owns the asset and it's probably also not going to say what the purpose of the asset is so unless you have really informative hostnames you

don't know for sure exactly what it's going to do so a centralized asset database can help you find out all this stuff and it's a good place to track your assets just having it all in one centralized place rather than going from spot to spot to spot so story time I had my first incident in my new company was there was a system that was getting that was running TCP dump and all I had was the IP address I didn't know what the host name was I didn't know why it was running TCP dump should it be and I didn't know who owned it because all I have is the IP I went and I realized hmm I have you

know 580 WS accounts even with the command line it's still a nuisance to go in and look through each of those accounts and try to find that IP I went to my cloud tool it wasn't really obvious I hadn't ever logged in before they'd given me an account but I hadn't been able to play around with it couldn't find the IP there I had to finally go to our sim and I was able to find out what the hosts name was only in the sim because of course DNS lookups didn't work from my laptop in the office to the AWS infrastructure I would have found it a lot easier and a lot less it would have been an immediate non-issue

if I had been able to see that this was a testing instance that some of the folks were doing to try to reproduce a customer problem Hey yeah of course they're gonna run TCP dump they want to see exactly what's going on and how the packets are flowing if they can do that on their their test environment so that's one case where it would have really really been useful for me so I know this isn't an easy problem even the NSA is having trouble with it and I think that's that's pretty rich given that Rob Joyce who is the former head of the NSA's tailored access operations team actually give a talk at enigma usenix enigma a couple years ago and one

of the biggest points in his talk was hey please make sure that you track all your assets it was it was a really good talk and it's basically hey get these basic things right but hey you know that's I guess it's a lesson and sometimes people don't practice what they preach so how do we help so this is an exchange that happened a couple weeks ago and somebody had been communicating with their the team that they were working with to do some pen testing and they had found only some of the assets and so they knew they had more surface to explore and then somebody else found 120 percent well that's awesome so what kinds of things do you do so we

like to collect data right if you have if you have DHCP logs if they're sending into your sim if you're using DHCP that can be a good way to find assets and you're already getting that data the scans that you're doing to discover vulnerabilities or just find out what's out there may be useful in building your asset inventory and you know just gonna hey you you can help them by giving them this kind of information to make sure that they know about what all those systems are and what they do so assets assets should be tracked and monitored and so I just talked about some ways that you can help opps track and monitor their assets and one of the

things that's really hard even for people who have got some kind of asset monitoring is how to update that asset database because a lot of places it's tied into their automation tool and if you don't boot up the automation tool correctly if it's shut down if it's misconfigured if it's not running then that host isn't gonna check in and they're not going to know about it so we still want to know about these things because we're still paying for them because they are still part of the attack surface for our company so we definitely want to do that we're so we are scanning we're discovering new assets as I was saying you are probably doing these anyway to

find vulnerabilities to to find systems that you didn't know about and hey why not share and the other the other point is time is back into DevOps with the dev team now being able to create instances or lambdas or whatever just as part of their work they're probably not as good as a traditional ops person in knowing how to harden a system even thinking about okay if I'm going to create an instance do a my starting from a secure am I just pulling some random instance in from the marketplace which could have who knows what in it I mean it might already have a Bitcoin miner built-in probably not but it might and you want

to be sure that people are doing everybody dev and ops is doing the right thing there so it's good to good to do some scans and help reinforce what they're already working on okay so other thing you want to you want to do as you're scanning is you want to look for outdated rules I don't know about you but I don't get told every time there's a new deployment that involves switching from a host group a co-host groupie or from you know software a to software B and so sometimes there you know there can be a deployment where that'll happen and my firewall rules are suddenly out-of-date they're pretty good at telling you if they need new firewall

rules but they're really not very good at telling you if they want you to turn off the old ones say they don't think about that so scans can help you figure out whether or not you still need the firewall rules you need you need the view PC configured this way or that way or the load balancer pointed in this direction or that direction so once again that will help you save money and they'll help keep your infrastructure more secure and it'll help remind your operations team that hey they should communicate with you as well as with the devs and you know maybe maybe firewall rules and load balancer rules are other things that you can manage like assets

and track in your centralized database it's that way potentially you can well I'll talk I'll talk more about life cycles in a bit but if you if you have an an estimated life of your firewall rule you can you have a built in check of hey I know it's time to make sure that this rule is current still or can I turn it off obviously we're doing scans to find vulnerabilities so some everybody knows about our plate I hope spent too much time on heartbleed um so you probably do have some automated vulnerability scanner but as a security person if you have to go and say hey I got a finding dear dev or ops or dev ops or SRU team

please patch this in your infrastructure then that's putting a lot of burden on you you know using the dev ops spirit of automation if you can have it automatically open a ticket for them so that they will patch their stuff that's even better and then the the ideal state if if you can convince people to do it is have all the infrastructure the containers the instances whatever automatically updated all by themselves so that your dev team is just using the latest thing automatically and your your CI CD suite which you can use to automatically deploy a build and test and make sure that everything still runs right can just do the right thing and you don't have to touch anything

yourself automation for the win okay so obviously tying into vulnerabilities assets are things that should be updated this means things like life cycles as I was talking about for firewall rules earlier but also for Hardware type assets this also means of course doing those patches that we are talking about and doing your best to avoid tech debt so asset life cycles if you got hardware it gets old it dies you ideally you want to replace it before it starts having failing parts and yeah I I know a whole bunch of people who are still running Ubuntu 1404 probably about time to upgrade to 16 if not 18 and you want to think about the life cycle of your

operating system as well just in case people aren't other things to think about lambdas if you're not using it anymore anybody who is using them in prod you just kind of set them up and there they go and they they keep running and you in put stuff in and hey if you've updated and you're using newer lambda you may not turn off the old one but if you've got a lifecycle for it then you're automatically prompted to turn that kind of stuff off so that's that's useful domains and SSL starts as I was saying they do have an inherent lifecycle of a year or two years or whatever and I you know your your DevOps teams should be

monitoring for that but it's a security team is probably going to be the one that gets yelled at if they don't get it renewed in time so it might be a good idea for you to build in some kind of lifecycle awareness and as I was saying if if you've got that centralized asset database if you can put that lifecycle field in then you're you're automatically doing the right thing and making sure it gets renewed in time so nothing bad happens there so some things about how you sell patching in lifecycle so obviously tech debt is bad you don't want to I don't want to have things building up and building up and building up because if it if it does then

something it's much more likely that there will be a software vulnerability if they haven't patched it or if they haven't updated to the newer version of whatever major major tool it is and I kind of think well okay folks I keep making these tickets for you to patch your vulnerability as well maybe a little bit better would be if you build in some automation to automatically do that I'll just I'll stop bugging you I'll stop coming to your stand-ups and saying hey folks now there's ten things that need patching yeah yesterday was ten now it's twelve and you know getting leveraging opposites desire to just have everything happen automatically with with minimal interaction is going to be

a really positive thing so let's see let's move on to lease privilege so you don't you do want to control the use of admin privileges not everybody needs admin or you know root access to your AWS account not everybody needs to be able to log into production I know that the security team probably does want to be able to in case of incidents but you probably want to restrict the dev and the ops folks who can and run things on their one one thing that I have have noticed is that well we're all kind of a little scared of the legal team coming down on us we're all a little scared of auditors and if you cut

down on who has admin privileges you can help them reduce that fear and you can also sell it to people as Haig if you're if you don't have the permission to do that if something weird happens and I'm not even gonna look at you as a possibility because we all know deaf folks who would just kind of log in and start making changes on things if they have the ability to to try to get something to work better and then it's not controlled and your auditors say well what happened here and you're like oh looks like it was Joe again Joe come on stop doing that and you know those privileges are also I think things that you probably want to

regularly audit I just started at my company and it looks like nobody has audited that kind of stuff for awhile and so I'm having to start from zero and I really wish they had looked the things even every six months if not every three months so can help you can help your other security folks out by setting that up as something that is regularly checked on blogs I mean I think we all really do like logs and while we may not like looking through the logs for interesting stuff it's it's really important to have them in case you need to do some investigation if if they don't already have if the dev ops side doesn't already have

big fascination with the logs and being able to debug things if things go wrong well that's kind of unfortunate but it gives you the opportunity to preach the gospel of visibility and debugging capabilities so I don't know if anybody remembers Ren and Stimpy but I thought I'd throw that in there so if we help law ops to can you know all the things that we all want to have logging to our centralized infrastructure you know sometimes your ops team will find that they want to use those logs as well now it is it's easier to give granular access you can set up your sims so that some people can only look at these kinds of logs and not that

kind of logs or these kinds of events and that kind of events and just just a reminder that I always throw in there when I'm talking about logs I always start with the outbound traffic because it's it's the most fun so next thing data protection just a couple of notes there because it's been really a problem lately so some real-world examples of data protection failures and things how you can make them more relevant for your ops team so your your company's code and github I think you can also treat it as an asset there are so many things that you can find there that maybe your ops team hasn't even thought of the security implications obviously you're not going

to make all these things publicly available if you think about it but there is the hackers will look for all these things dome mine that's a cloud security company check some AWS API keys into github in a publicly available repo just to see what happened and they found that the first time that somebody went and downloaded them was three minutes later so if that gives you some ammo to help convince your ops team that they want to they want to take a look at these things and try to prevent it so if you've found some of that sensitive data a reminder to you just because you may not use get day and day out if they

just come it over it then you just go back one commit in the commit history and you will see all of the information but if you actually go in and remove the commit that's the way that you can clean the history out and the data will no longer be there obviously rotate the credentials truffle hog if you want to look into finding stuff in github Dylan Airy I think is how it's pronounced I talked about this at besides San Francisco you gave a good presentation about it and truffle hog and some things that he's working on I actually have that link to that in the in the notes I mean nowadays AWS themselves are looking

for AWS API keys in places like github and they will also reach out to you but I think I'd rather I'd rather find them myself rather than waiting until AWS gets around to scanning it so s3 buckets I do think that they're assets as well there is no way in the AWS UI to tag them unfortunately so unless they're named to reflect what they are and what their purpose is and their lifecycle and so on and so forth you may not know my company is just barely ten years old and I found an AWS bucket that had been created you know about a month after the company started and they hadn't been thinking about permissions quite as much as they ought

to have there wasn't anything in there that was super secret or needed to be secured but there was also no need for it to be there and no need for it to be out in public view nobody had ever gone back and looked at all of the buckets but fortunately now at AWS there are a few tools you can just use the s3 UI there's Amazon Macy which will actually monitor not only permissions on the buckets but also some of the contents which is kind of cool so with that we are doing pretty well on time does anybody have any questions at this point or shall I talk more about ops and how to win them

over good question yeah it's coming just going back to your point about automations work in the hospital so a lot of people are afraid of automation because there's a lot of possibilities that things might break especially if you automate patches move do you have any suggestions on how to actually make some friends yes so how do you how do you help people overcome their fear of patching I the great question my best suggestion is to if you can build a a test environment you know a little isolated Network I know it's good it means that you have to have one of this device and one of that device and so on and so forth but if you

can just pull one out of the rotation and test your patch on it and you know make sure it works that's that's really the only thing that you can do I mean I I understand it's so hard in a hospital network because you have you have the requirements of not only all of your PII and pH I that needs to be secure but also sometimes the tools just don't get the updates themselves and you don't even know if the OS update is going to be compatible with your tool and that's that's important you know if you don't have an ongoing support relationship with the vendor which is would be the best thing is making it their problem

you know that that they have to comply with all of these you know various compliance frameworks and HIPAA and so on and so forth then yeah that the best thing that I can recommend it's it's not necessarily gonna be easy but is you've you've got to test it yourself you can't just you know you can't just roll it out anywhere ever and hope for the best not in that situation yeah anyone else okay beeping all righty so a little bit more about what operations people tend to want from their security teams I did an informal survey of a bunch of folks that I know who work in operations and you know asked my my own DevOps team at work

what what it is that they want so here are some stuff that they came up with they said okay transparency if they don't know yet who's on the security team and what they do then they are going to be less likely to just think oh well I should check with the security team if there's something going on and I'm not sure what it is or I don't know what the implications are of making this change to our infrastructure or to our application or to our system I passed colleague of mine we had a lot of fun doing a security where we would brew up some pots of tea and we would invite people from engineering and

operations to come over and just hang out with us and be social and just you know interact on a more personal level and that just gave them more opportunities to get to know us and learn a little bit more about what we were about and you know maybe ask us questions in a less less pressured environment and you know if they didn't want to ask something in front of the whole team hey you know they knew who we were they knew where we sat they could come over and talk to us and so one of the other things is they want to make sure that they the ops knows why the security changes are being recommended

or are being made and they want to really understand what the security team is about and what their main goals are because they they don't they don't want to be antagonistic with their security teams but also they do want to understand stuff they're you know they're curious you know just just like security folk so next opps would like you'd be realistic which is to say yeah there are some security changes that are really important and they have to go in but at the same time there are some some things that are going to give you just a little bit of win and they're going to impact productivity they're going to impact an engineer's workflow a lot so the more

that you can do to help the understanding of what's going on and why and how it's not going to make their work harder the better things will be I don't know how many of us said hey just have said in the past to our colleagues hey don't work from the coffee shop because you know public Wi-Fi is insecure or hey we're all going to this big security conference maybe you should you know put your phone into airplane mode for the entire time you're there that's not realistic for a salesperson they're gonna want to be available for their customers so you know try to try to match what what is a realistic threat model if somebody doesn't have any

private company data maybe there's less of a risk of them getting getting hacked at a security conference the other thing is if you're making it easy for people to do the right thing it it will be more likely for them to do it so respect I was going to title this empathy but talked about empathy before so if we all know that security team is supposedly the team that says no a lot and the more that you can explain why something is happening I've found and just like treat them like smart folks who want to learn and want to do better the operations team will understand they will though will cooperate if they understand they'll buy in better and you know when

you're starting out to make things happen starting out with small changes to prove that you're not you're not going to just break everything just for the sake of breaking things can also be a big positive I know that kind of goes against the the model of patching vulnerabilities where you go for the biggest risk possible and mitigate that and then work down the line but sometimes you've just got a got to build a relationship before you can start coming in and giving people orders so the more that we can know their priorities and help them out and we can help frame what we're trying to do the better it will be for everybody as I

said Ops does want to learn new things you know they a lot of folks in ops and dev nowadays DevOps sre have come in from non computing backgrounds and so they have they have lots of questions and they've also by getting as far as they have proved that their ability to learn and so you know they they'll love hearing hey these are all the details or I found this really neat hack read about this I'd like you know if you want if you have any questions talk to me about it that's a lot of fun I have personally found that taking the time to reach out to people and teach will pay off later because the other thing that that goes

on I had to throw out this Swift on security name it says well you know Ops really is doing a lot of security work the other thing that the ops wants they want jobs everybody's hearing about how security is a lot of fun and how there are a lot of job openings and yeah there are also a lot of job openings for DevOps but you know sometimes you find somebody who's really curious and who likes to learning new things you can start feeding them security knowledge and then you can hire them and they'll be on your team and they will already have relationships with ops because they've been there and that is going to help them out a lot so final thing if if

these other things work to motivate your operations team give them shiny things I I do prefer to call it a reward rather than a bribe but you know if you just want to be blatant there you can call it a bribe everybody likes stickers how many stickers are there floating around here cube toys also fun although I think frigate spinners are a little bit dated chocolate you know rarely goes wrong I will just say though booze is I mean I know pretty much all of us here are probably you know drinking a fair bit this week but some folks don't drink so maybe find out a little bit more about them before you buy them a bottle of whatever alcohol

there are some really awesome ginger beers and root beers that are craft and fun that you don't have to give people alcohol so with that I'm kind of saying you know there's these are a bunch of things you can do to work with your ops team and help them succeed you know some of them maybe even most of them hopefully all of them are things that you are already doing if not these are something more things to think about so thank you very much if you if you have more questions now B I see a hand over in the back corner there awesome do you consider I am policy to be an asset and do you let your dev ops team

because kind of hard not for that dev ops control I am today yeah functions is to make boundaries and yeah yeah yeah I'm pretty much anything that you can that you can audit I think you probably want to consider it as an asset as you're starting out obviously you're you're probably not going to start out with a security person or somebody who's thinking about oh I should make sure I review this you know once my you know I have two developers who stayed and then there's the one who left they're probably thinking more about how do I pick up that person's workload than they are about how do I you know turn off all of the

access that they have so it's you know as as things evolved doing things like you know making sure that the root access to AWS is the main one that is able to set I am policies that's a that's useful you know there are tools AWS config now it will actually help you audit your iam stuff so that when you're when you're trying to make sure that people are only doing what they should be doing on AWS that that they have they do have some limitations yeah it's if you're if you're lucky you can potentially even check that stuff in to your source repository and then yeah you're definitely treating that as an asset you know just only thing I can can

say is you have to first sell them on the necessity of restricting the control and then audit and review and review and review to make sure that you're that they have the right access that they need I mean it's a philosophical thing whether you give them possibly a little bit more access than they need and make sure they can get their job done or give it to them in dribs and drabs until they are able to do everything that they need it also depends what stage your company is in if you're in a bigger company I was probably a little bit more tolerance for starting small and then adding permissions versus if you're at a big

company you're probably going to want to start with more and then cut back gradually as as they are convinced that they don't need this or they don't need that and you can justify it just let's wait for the mic just would you characterize customer data as an asset as well and if so what's your point of view in terms of devs treatment of that versus security ops tree specifically in the realm of integrity right so I think yes that customer data is is an asset you can't track in the same way that you do track everything else I mean the contents of a database you know if you're if you're lucky enough you're your asset database with your customer

data is probably changing all the time and growing because you're getting more customers so yeah you you definitely do need to track it it's another one of those growing pains type problems I've found is when you're when you're starting out you don't have any restrictions on who can do what on AWS on who can access the data on how it's able to be accessed but yeah then as you go you have you definitely have to pay a lot of attention to it you have to get yourself a way to think to to control that and I I was I was torn about putting customer data into this talk but it doesn't really fit in this

formulation because it's such a special thing and there are so many additional regulations around it you yeah there's there's no easy way to inventory it but you you definitely do want to control the the who can get access to it how they can get access to it I you know I be happy to talk after about this because I don't think there's I don't think there's the one way to do it but there are a number of approaches depending on what your situation is and what your industry is and how you how your I mean even how your security team is tied into engineering etc etc that you want to think about so anyone else

no all right well thanks a lot folks it's fun [Applause]