Compliance Without the Chaos: Building It Right Into Your DevOps Pipeline

All right everybody, our next speaker is Verun Gernani and the topic is compliance without the chaos building it right into your DevOps pipeline. Um Verun over to you. Thank you. Thank you so much. Um the bar and chill out space is already open right but don't leave um until I finish yapping through my presentation. Okay, before I start, I was like trying to play with the, you know, presentation and trying to connect. How many of y'all saw my jokes show up on screen? No, a couple of you guys did. Okay, so I can try and have them again. So, anyways, my name is Von Gurani. Um, this is my second time speaking at Bides and I'm super nervous as usual. Uh this

talk is titled compliance without the chaos building it right into your devops pipeline and I've highlighted the compliance word in yellow and also the devops word in yellow that means that we'll try to combine these two concepts together and look it has compliance in the name and it's Sunday it is 2:15 p.m. It's post lunch in a movie theater. Seats decline. So if you are awake, that's an honor to me. Anyways, so thank you for being here. Okay, give some quick introduction. I'm a security engineer who turned into an accidental compliance therapist. I started off my career by writing Android and iOS applications. And when I got my first real job, I was doing ISO 27,0001 vendor

security audits. If you ever done those audits, you know it's painful. It's something that you want to do. And I was like, this is super I can't do this through my career. So I quit my job. I came to the US, got my masters, and then I got my internship. And my first assignment as an intern was, hey, look at our sock two controls and uh figure out how can we pull evidences. I'm like, "Oh my god, I'm I can't like sit through another like 3 months of doing compliancy work." So, I wrote some scripts to like pull evidences automatically, put them into a zip folder, and people loved it. Apparently, Python 3 was not something that

compliance teams knew existed. So, yeah, that's fine. Um so kind of pivoted my career to figuring out how can I do things that can help GRC teams um feel less painful when they go through audits. Um, so my goal in my career now is to make engineers hate compliance less because as an engineer I hated compliance being on that side of the room where you have to answer walkthroughs, answer questions which are like so obvious and take time off my dev sprint. Um, so yeah, I don't want people to go through that. So I'm taking one for the team. Um, quick disclaimer, I'm not representing my employer in any way. This is my own time. It's my own dime

and whatever I speak and present here is no way related to the work I do for my employer. All right. These three beautiful words letters GRC governance risk and compliance they mean a lot. Uh there are entire businesses built on achieving the right maturity across these three practices. This is security adjacent. I would not call it complete security. But in this talk I will want to focus on the last letter C compliance. Okay. So before I get deeper into what compliance doesn't bore you guys. I want to know how many of people from the audience are from the complian who relate to the compliance team. Like show of hands quick. Okay. How many of you are like

security engineers like threatel dev? Okay. infrastructure folks raise your hands. Oh Okay. Sorry. Um Okay. And how many of y'all work for a security vendor? Okay. Okay. Good. Less people. Thank you. Okay. Um All right. No, that's good context for me to put things on the slide. All right. So, if you're a compliance team, u you are pretty familiar with what's going to show on the screen, but for those who are not, just some quick rundown. compliance folks are under a lot of stress because they are put through multiple security I would say threats not the security threats but like security based threats which is like a 4% fine if you fail your GDPR audit like

if you if the GDPR organization finds problems with your systems the worst thing that happen to you is a 4% fine and then the finance teams comes after you saying that hey look we are on the stock market. So we are subject to the socks audits. So either you comply with our sock security controls or we are screwed. You work for an organization, you have board of directors, they want to see the security maturity um with the NIST cyber security framework. So you got to put all your controls and maturity into the NIT CSF. And lastly, the sales people walk in and say, "Hey, look, we need a sock too for selling our software to other

businesses. So can you help us achieve this beautiful certification? Broadly for a compliance team there are two work streams they focus on. One is about achieving compliance and second is audit. Let's talk about both those work streams in brief on what these teams actually end up doing to understand how we can actually help their processes become better for achieving compliance. They have like a kickoff meeting which involves partnering with engineering legal internal audit. Then they do a gap assessment what things exist and what things don't exist. For that they work with the product teams. They work with remediation of those gaps. Then they test those gaps and then they go for an internal certification um internal validation and then they get

through a certified process during an audit. This is happens like once in 6 months depending on which audit you go for but usually compliance teams have multiple audits every year. So this next life cycle happens any number of times depending on the scope and depth of your organization. There's a kickoff, there's a pre-assessment before an audit. There is sampling that comes through. Uh what sampling is just okay there are like 100 things that we want to test. Auditors who come from an external entity, they want to see like a sample of 5% or sample of 25 or sample of 30. Then comes validation of that samples and then there's a close of the audit. This is

like a very broad like life cycle. Uh I don't want to like um undermine the work the compliance team does but broadly this is what happens and again partnering with teams is a very big aspect of being in a compliance function. So again engineering legal audit there's engineering is on voluntary assessments sampling as well you have to partner with engineers to see what can we get from the systems when the things are sampled during the validation phase as well. there's engineering involved in some way and then in close there is a legal internal audit who come and do the program management work. So if you look at these two broad processes, engineering teams play a very

big partnership role for compliance folks and usually these people don't like compliance because they compliance comes and takes things off their plate because they have to do and go and build features versus trying to achieve compliance for compliance teams. So our goal here is to figure out it's compliance without chaos. So how do we make lives of engineers less chaotic and get compliance folks to reuse a lot of the tools and processes that engineering and DevOps teams already have to go and achieve compliance to the very deep end. Okay. So let's look at this DevOps. It's a very popular infinity sign diagram which is common across engineering and devops cycles. So it starts from the code process. Engineers

write code send it through a build pipeline. Some kind of testing happens. The release, deploy, operate, monitor side is usually blurry and overlapping depending on which tools you use. But broadly you have some kind of compute and some kind of logging and visualization. um these names that I've put here are just for representative purposes and just like give you an example how things would work but yes that's the tools that usually engineers interact with to go write code and deploy it okay so to understand how can we actually help engineers hate compliance less I want to take an example of a typical build process like right from writing code into GitHub. Engines write code, they

push to git. Git has some webbook triggers that probably triggers a genkins build. Jenkins builds the applications, runs the unit tests, builds the docker image. So we are going into the containerization world as well. Um then the docker image is prepared stored in some kind of registry. during the deployment process. At the latter end, uh, Spre deploys the application into a Kubernetes compute infrastructure. So, and the Kubernetes stuff manages the scaling, load balancing self-healing redistribution of pods, you know, and clusters. Okay, let's look at so the last part of Kubernetes. Let's look at this part in detail. And like the reason I'm going through each part of the technology is because before we can help like DevOps and engineering save

their time, we need to understand what tools they are using and where can we inject our compliance injection so we can extract that data right from the source. So let's go bottom up. There is the Kubernetes infrastructure. This is deployed on some kind of compute. This could be like a EC2 instance, a bare metal host, a virtual machine. And K8 manages the control plane which actually has clustering on top of it. And then each cluster has a bunch of name spaces. Um, and when an engineer deploys a container through the entire process that we saw, the container falls into Oops. Oops. Oops. the container falls right into the top of the name space. So whenever we deploy

an artifact, we need to make sure that we are embedding compliance right into the pipeline and it does checks that scrutinizes every aspect of deployment right from like the core to the final execution with the name spaces. So I'm trying to like put compliance in like two parts which say that we have compliance in the pipeline which makes sure that the incoming deployment is secure and then there's compliance of the pipeline which says that the underlying infrastructure is compliant. So at each level of our Kubernetes infrastructure we need to ensure that the below layer is secure and compliant. So for example at the cluster level you want to limit access to the clusters. On the control plane level, you want to

make sure that MTLS is enabled from a service to service encryption. On the compute operating system level, you want to make sure that you have the right settings for your EC2 instances. If you're using like a bare metal host, then you have a lot of baseline checks that you want to perform at the compute level. So, not just the deploy, but right under the hood, the infrastructure needs to be compliant as well. Okay, let's look let's take a look at how things get complicated under the hood. And this is why engineers say things like, "Oh, it depends if this part is in scope or if it's secure because infrastructure does get complicated depending on your

organization. So on the left we have like completely unabstracted infrastructure. In the center we have some abstraction where you don't have any of these problems of compute but you have cluster as a service. This is your AWS EKS and GKE. This this center one what it looks like and then you have name as a service. This is like a little less common but then you have some vendors who offer namespace as a service as well. And then your engineers don't have to worry about what's under the hood. You can choose tools that can abstract the compliance and security problem for you. Okay. So CIP compliance in the pipeline would look like just at the top

level and then based on your organization infrastructure the compliance of the pipeline will change. Okay. So I just introduced like two broad concepts. Compliance in the pipeline compliance of the pipeline. Compliance in the pipeline is like your you know TSA pre-check. If you're clean you fly through. If you're carrying a suspicious object or a suspicious YAML file, you stay on the ground. So if you have compliance checks baked in in the pipeline, you will have those kind of majority. Secondly, compliance of the pipeline. Make sure that the DevOps tools that you're using are secure and compliant. Now depending on your organization and vendor scope, that controls can differ and change. Um so compliance of the pipeline is like

ensuring that you know your brakes, your mirror, your seat belt is is fine before you actually take a drive. Um and like yeah so if you talk about like the car example um it's like having the traffic lights work having the roads paved rightly for making sure that the roads are fine on which your infrastructure is running. Okay. So from a features perspective for CIP we'll have features like automated scanning policy enforcement SAS scanning and audit trails and for controls on the compliance of the pipeline side we'll have access control again audit logging configuration managements and the tools that we use they need to be secure so tool security part would be included in cop as well.

Okay, going back to our example, let's take those tools, put them on the right. So you have CIP, you have COP and then you have all these compliance controls that will come on the left side. So you can imagine for a security and a compliance team, it will get complicated because they have to see that sure our compliance of those tools exists plus we are using those in a compliant manner. So I want to like separate those two things out. And to explain these concepts, I want to take one example for each and and talk about how would it look to have CIP in Kubernetes for a config management control. Okay, Kubernetes configuration management. We'll take an example which

says um there should only be used uses of authorized container images for your containers when you're deploying it to your namespace. Now this is a very simple control which says I'm going to read the entire paragraph but you're going to use a designated registry my registry.io for storing your containers and you can only use containers that are deployed in this particular registry. So docker container goes inside into the name space. Okay. So what can you do? If you want to help the compliance team or if you're the compliance team, you can make use of this existing tooling write some additional logic to it and come up with logs that can be used during achieving compliance phase

and the audit phase. So in this example in in K8 anybody can deploy and technically pull images from anywhere including untrusted risky sources but we want our engineers to deploy and pull stuff from my registry.com and another registry.io. So the second line of that code whenever this is going a little deep into the Kubernetes architecture whenever a container is pushed into a namespace the Kubernetes admissions controller controls what goes in to the namespace and the clusters. So for every time like a pull is made for those containers admission control knows okay is this the right thing or no and it creates some kind of logs for doing that. So think of admission controls control as like a

nightclub bouncer. If your container is on the allow list on the guest list, you're getting in the club otherwise you're not. Um so what you can do is write some OPA rules. OPA is open policy agent. There's a plug-in called as OPA gatekeeper which is specifically for Kubernetes that talks to admission controller. You write these rules like a very simple like rule. Um you can and if you're not u an engineer on a compliance team you can use like an geni tool to write these rules for you. So you write these rules put them into your OPA gatekeeper that so OPA has two parts the policy part and the data part. The policy is where you define what's

allowed what's not allowed. The data part is which kind of like generates all these logs. Okay, so you've written those rules. You have an unreal content that gets pushed into your namespace. It triggers the admission controller. Admission controller talks to OPA. OPA says, "Uh-uh, that doesn't look right." And it creates a decision log. The decision log goes into some kind of logging platform that you probably use. And from there, you have some kind of compliance tooling. That's the new magic stuff that you have or if you don't have this is a time you go and build those things or you acquire them through some vendors. The goal here is to have the center part the data

bucket which has all the decision logs coming from this side coming from the admission controller through opa policies into data base. This could be a data lake or like a postgress DB or some kind of like blogging platform that you might use and then eventually you have some kind of compliance tools that you probably might end up using can consume it. If you're in the compliance world you might assume like oh my god I have to make sure that all this is built out. Trust me most DevOps teams already know how to do the two third parts of this slide. Your job starts from collecting this data and actually putting them through the controls. And I

just showed you how like you have these controls, you can translate them into actual actionable items that meet each control for compliance. Okay, any questions you can go on slido please post on it and I'm very happy to answer towards the very end. Okay, let's look at a next example of compliance of the pipeline. So in this we want to see how do we make our spin deployments compliant. Okay there are spica spker is a CI/CD tool which helps engineers deploy applications but spker itself needs to be deployed. So as an infrastructure organization, you will have two choices either to self-manage it or just to like go and buy like a vendor who does spre. But if you choose to deploy it

yourself, which is most DevOps for folks want to do things their way, they might use something called as hailard. Hail is a configuration management tool for spinre. It helps engineers deploy spin into the organization. The configurations are stored in a directory called h and using oh sorry back okay using hailard you can actually go and deploy things like cloud driver front 50 orca fiat all these are spica components which help spin run functionally correctly Okay, so they use hailard but hailard in itself does not track approvals. Hail just helps engineers make changes to spin. And if you hear the word changes and this is like a change management example, you know that you can use data

coming out of hailard to support your change management controls for compliance. So hail has the time stamp. It might have the name of the person making the change. It can also have the log entries of describing what exactly changed. So you can use that data and map it to your change management control and have this automation set up right from a DevOps pipeline that helps meet your compliance control. Okay. And now I don't want to go through all the tools but you can do this for particularly all tools that your infra depends on and you can help compliance teams understand infrastructure through its own codebase but also help them save time during audits and look here's the thing you

don't have to like know how to configure everything it you can take the help of like geni to do all of that like LLM are pretty good at making you write those reggo policies so you can like work very closely with your DevOps team like this is what you need to do to do it right. Okay. In every slide I keep talking about the compliance tooling part and honestly this can be like a separate conversation in itself a separate talk in itself but I do want to like talk something about it because it's important on how we structure data for compliance. Honestly, compliance tooling is a lot dependent on your organization, your compliance commitments, your

infrastructure blast radius. But broadly, I'm going to go through like these three parts, which is evidence collection, audit logic, and continuous monitoring, which should be a part of your compliance tooling. Now you can go and build this yourself which I'm honestly a big fan and supporter of but you can also go and procure these tools from like a vendor which you might use and these vendors are getting mature over time. Okay. So let's say you want to like build yourself or procure a vendor. What features should you look for within these toolings? So for evidence collection you remember we have this a bunch of data we're collecting from all these CI/CD tools. It should have an

ability to query and filter large data sets to generate compliance trails. So it should have ability like plug into a database or a data lake or into a logging platform and ability to collect that data and transpose it. It should be able to chain audit events. So let's say you're talking about change management. You want to track it across different tools. So your tooling or your vendor tooling should be able to chain these events together so you get a holistic compliance view. It should also be able to scale up and scale down evidence artifacts based on the compliance framework you're choosing to view it for. And this is super important because not all compliance frameworks are built the same

way. Some are a lot more detailed than the other. And you don't want your tooling to just have one kind of evidence because if you go into an audit, you don't want to show the auditor everything in your books. You want to scale it down to what specific they want to see. So for example, if you have like a sock two, they just want to see probably a screenshot of something that is like, oh, you have a vulnerability management program. Sure, you're you're good to go. But for PCI, you want to see exactly each vulnerability on when it was triaged and when was fixed. So depending on the framework, you want your tooling to be

able to scale up and scale down. Think of think of it is like when you have to travel like a local flight or one day return flight, you have a travel backpack. But if you're flying like an international vacation for like 3 weeks, you have like two suitcases. So you want to be able to like choose your travel companion based on your travel needs. Similarly, you need to like be able to scale up and scale down. Okay, but you know good toolings you with good tooling you can get like good artifacts but you don't want to end up like chasing artifacts that no auditor asks for. So you need some kind of logic built into it and

this logic is very specific to your infrastructure and your organizational needs. And look, most companies don't have one audit per year. They have a sock to renewal coming up. They have a PC assessment. They have a socks um interim audit coming up. So you have multiple audits in one year. So you have to have logic that can support the scaling up and scaling that we talked about. And good tooling will be able to slice and dice the evidence that can be reused. So you don't have to make multiple queries into infrastructure but with one query you should be able to fetch that information chop it up and then be able to match it to different

controls. It's also not about this tooling is not about just collecting evidence but also mapping the right evidence at the right time. So depending on which audit is coming on you should be able to have the right artifacts for that particular audit at that time. Some audits are time based, some audits are based on historical data. So your tooling needs to be able to understand this and play with it and not let these controls be a part of like a customizable as a part of the tool. It should be baked into the tooling itself. Okay. Continuous monitoring. Honestly, this is like a buzzword in in compliance world, but if you do it right, you can

build good compliance muscle. Your compliance student should be able to run like checks every hour, every day, every week based on your own needs. Um, it will be not fun to get like a failure 2 weeks before the audit. You want to catch the failure as and when it happens. You should it should also be able to like freeze and have like point in time assessments because let's say your CISO wants to see on a Friday afternoon, hey, how does the compliance posture look like? I have a board meeting on Monday and so you want to be able to have a point in time assessment. So not just continuous monitoring for the right compliance attitude but also having

ability to grab a snapshot uh for the point in time assessments and you can present that to your leadership and look nobody wants bad user interfaces. So it should be able to provide this into a dashboard that you actually want to use. All right, this is the shameless plug where I have written about this last year and if you want to go hands dirty and roll out your own audit engine that's how you can it's a three-part series but I feel the third part is the most relevant one you can try and build out your own compliance tooling internally and see how it functions if you don't want to go and buy a vendor tool because it's not compatible to your

use cases all right uh that's it that's a summary we spoke about how hard compliance gets the devop devops tools that people end end up using compliance in the pipeline, compliance off the pipeline like two separate heavy lifters or compliance teams end up working with an example of each um with the end saying that your data is key and that data can be used to mature compliance tooling. So yes and the compliance tooling is um the heart of your compliance team in this case which has evidence collection a good audit logic and ability to continuously monitor your security compliance posture. All right that's it and uh thank you so much. Great job Vern questions especially

loved your jokes. Um thank you. one long question. If anyone else wants to ask a question in the time it takes me to read this out also note J I summarize maybe no excuse me I need you to use the mic if you're going to ask the question.

Thank you. Um in your examples you're using pulling audit logs of the actual deployments and authorizations as part of your change pipeline which is awesome. But frequently you then need to take that information and map it back to all the original source datas. And you kind of implied it by using chaining and it'd be cool if you gave a lot more detail. Okay. So it all Okay. If I go back on so you're referring to I think this slide. Sorry not this one. the next one where I talk about collecting data and then this slide right like how do I chain events from spiner up to genkins that's what you want to know so you you do that by

actually building serializers right that's a lot deeper into compliance tooling at this point I would love to nerd out for the next 30 minutes about this but in short when you comp when you're collecting data from multiple sources your compliance tooling has some kind of serializer serializers built into it which is like okay on 27th April at 246 uh Vun pushed some changes into git. How did it translate into Genkins? At what time did the artifacts get built? All this is being tracked and being serialized into compliance tooling. So all these tools don't have to talk to each other, but you collect all their data into one data source. So what I'm hinting to compliance teams now is that

start building your muscle in a compliance data lake. Don't rely on security data links to figure out what is failing and passing. Your data link should be able to collect all those events. You don't want to duplicate data. Of course, you can always like bounce off of your SDLC data and logs and know security logs, but create the pass and fail status in your own database or data lake and that's how you'll be able to like actually change that data because you own like compliance should own that data. That makes sense what I just said. More or less. Okay. Another question for you. Uh, what can people on the engineering side do best to support the compliance team with this

process? Love that question. Thank you. Um, be more empathetic and understand that they don't want to put you through those problems, but they want to actually like support you to do less work. And just what you saw if you have things like this in your organization where you are using some kind of tooling to build your tools give access to this data to compliance folks and ask them look here's what we have what can you do with it can you map this to some of your configuration management or change management or access management controls like I know for spin you have something as fiat which manages authorization if you use the logs from that to prove that

you know the access to this container was just done by people who were truly given access. It's part of the access review process. You can prove that control and prove it at scale. So yes, like take these tools to your compliance team and tell them that this is what we have. Can you please reuse this information versus coming to us with your questions? Yeah. Thank you. Nice. Thanks for round of applause everybody.