The Current State of Infrastructure as Code (IaC) from a Security Standpoint by Albert Heinle

okay hi everybody is my mic working and everything perfect okay well thanks first of all to the organizers of this conference it's my first time here to be honest and I love it so far so it is a great work that you put together and thank you for welcoming me today here as the speaker um yeah I'm going to talk a little bit about something that I am very very passionate about namely the movement of infrastructure as code and I would like to talk specifically about the possibilities that it brings to us as Security Professionals and the insights that it can provide and sort of the the the way it is um changing a little bit our deployment landscape but

let's start with an opening question so what is everybody's understanding actually of infrastructure scope just no wrong answers just you know anyone terraform terraform okay what else anable okay cool so I hear tools I hear tools not principles right so and that is like sort of my the the general understanding that I have that the world has about infrastructure as code and I hope we can learn something a little bit today about that infrastructure as code does not necessarily mean infrastructure as code as many people understand it then we will talk a little bit about uh the gray areas that many infrastructure as code tools have and infrastructure as code security approaches okay so we're going

to talk a little bit about the gray areas that are there I mean this movement exists since about 2015 you know and it's cannot catch up with decades full of it experience as we yeah as we actually have and uh one of the things that I would like to also emphasize today is the absolute great chance that this gives us to perform some rigorous security testing I know it is late in the afternoon you know and if you are forgetting everything else I'm saying today those are like kind of the main points I would like you to take home okay so okay let's talk a little bit about the evolution of infrastructure as Cod so software

deployments as they are sometimes done still today and done in the past right so everybody of us has worked at a place where there was a server set up by somebody who quit 10 years ago and people say Don't touch don't touch we don't know what happened there but this thing works okay so this is unfortunately something that almost cripples the ability to innovate to move fast to do anything okay so and this is like sort of where um the the the pain really come from that led to the development of trying to codify a little bit of infrastructures and in my perspective and you know I like that disclaimer of these are opinions of my

own and nobody nobody else's so there's actually the two main things that led us today to enable us to do what we call now infrastructure es namely codifying anything we want to do in an infrastructure having it inside a code base having it part of the change management that is enable a for example by like you know git or other change management systems of your choice so two items are first of all Cloud right so you don't need to go anymore to your it department and say I need a server in this wack by in two weeks or something like this you can just provision them although you know in principle it's still the same it's just more automated

than it was before okay and all those actions happens through apis that means there is like less of the Personal Touch involved and less people involved and uh so there's no physical machines on premises needed and the other item that is very important is the containerization so anybody who has written Docker files here okay I knew this would be the majority this is wonderful so you have now truly reproducible environments you literally write I want to install this I would like this operating system or this base image and you you go step by step by step you basically create a documentation what is inside your container okay so this is basically your new bomb or nowadays the term ibomb

infrastructure bill of materials has sort of come come out of this uh movement okay and what you can do now again with this and I cannot emphasize this enough you track those changes that you do inside there with uh inside a code repository right I remember when I first time used Docker and I had an intern I asked them to write some um Docker files they were like well why am I not just spinning up a container and do the changes there no because the problem really then is then you lose the ability of immutability and then you're back at sort of this case okay so you don't want this so now we actually have

this ability to say this is our infrastructure this is how it was built it doesn't matter who what personnel changes we had in the last couple of years we can do this and those are the two changes and or revolutions inside our technology space that enabled us to really codify what we're doing with our infrastructure okay so so now the question is what are the new possibilities that this gives us and now let's take like a 10,000 feet bird view on this okay so we can keep it inside a codebase and what is also important is we can move actually faster we don't need to think anymore oh what happens if I do changes in this machine or who did

changes in this machine last time we have actually a full sort of audit log of changes available to us and right in front of us and uh previous speaker here was also the the one who talked about um moving fast and breaking things right I mean the to me actually there's also this perspective of if you are able as an organization to move fast and do it not because you have to move fast because you need to get a project done but because you are really confident that you can actually move fast that is actually to me a biggest sign of security that you can have because that enables you to do small changes fairly

quickly and we all know that fixing security holes are a accumulation of a lot of little tasks that you know you just have to do certain settings you have to set and uh things like this so we can now also uh using this conization uh thing um it enables us to create sort of a very very prod likee environment on our local machine so whatever you're doing with your containerization orchestration in prod you can emulate sort of almost almost almost one to one with the docker compost locally this means you will introduce less deployment bugs you will uh you will create less issues once you put those things in okay and then we can also given this we can of course reap

all the benefits of that code always had compared to infrastructure so people can create PLL requests people can review them properly so we have actually a possibility to bring this software stability and I always like to use this example of blue screens I mean we all remember blue screens right so software used to be like very unstable until we actually did software development properly and has static analyzers look over our shoulders and things like this we can actually reap all those benefits and apply them now to actually infrastructure and uh as the last point I mean I know I keep repeating myself the immutability of it is actually a concept that um allows us to uh yeah to

actually have these very stable environments that we have a clear idea what is happening inside of okay so now let's talk about the 37 flavors of IAC Omar so when I asked my initial question here there was like terapon right this is like this is like an initial reaction that many people have but there's there's different flavors of how to actually create your infrastructure so first of all there is some Cloud native scripting possibilities so there is from AWS it has its cloud formation um um what is it called Microsoft has its aure resource manager Google has its Google Cloud deployment and those are all usually yaml or uh except Microsoft has this bicep format that I don't know did

anybody actually ever write any bicep FS no thank you that's that's what I thought uh then there's of course those internal external Frameworks that try to be like okay well I mean we are partially dealing with hybrid Cloud we're part we're dealing with like sort of mixtures of everything so then terraform came around or pumni which directly uh is working like a a code library that you can use inside your Java code or your C code then there's anible Chef puppet and the list goes on and on and on because there's a lot of uh those um sort of tools that have spun up but those are like I would say the most prominent ones that are out there

okay so those are like Frameworks that people are using for one reason or another okay so now the other thing is that and this is the other thing that I was kind of missing when when I was asking a question kubernetes where does this all fit in right so kubernetes is a container orchestration um well tool SL methodology which allows you to basically create an environment of containers and give Specific Instructions how they interact with each other almost like firewall rules they can create a whole ecosystem of containers and in a very minimal case if you're using Cloud plus kubernetes you can actually just tell your cloud provider set me up with a kubernetes cluster and I take it from there right

so so your terraform file might be very simple so this is this is how kubernetes fits in all of this um thing and one of the questions that very often come when it comes to infrastructures code and specifically the tools uh what should my orc use actually right so so there's like this is the the overarching question because there's just so much available uh the the space is very crowded so of course short answer is it depends you know like it really depends on what you're doing and the long answer is like it's more like a journey it's not like a destination you you should usually if you are migrating towards it you should do it step by step from

starting to containerization to moving everything into your code pipeline um just as little decision guidelines though if you're using um if you're using anything that is just Cloud you can go ahead use the native apps use the terraform use whichever one you would like right that is that is not a problem if you're doing anything on premise or anything that is in a non- major cloud provider okay because many people are trying to use nowadays different cloud hosters and more simple providers in AWS Azure for cost eff effective reasons and things like this just use something like anible which also has capabilities to set up machines directly and create certain playbooks okay so those are like

sort of the decision guidelines okay so this hopefully answers uh or gives you a a slight overview of the infrastructure as code world as it exists today any questions for now in between before I go to security yes

oh yes of course oh thank you thank you this is a this is a very good question so okay say you have a server right and somebody in your team has say SSH access rights to it and they exercise rights they do a change inside the server live and you have no trail of it you have no audit trail of it you don't know what they did but somehow it fixed your current problem they were fighting a fire or something like this this means at this point This Server SL container is now tinted with something that you know you cannot really reproduce immutable means you do all the changes in in the code and then deploy them over

there and then make sure once it is up there nobody touches it because then you are not losing your sort of AIT Trail or the way how you can then reproduce whatever has been done there before in order to get to a working state

yeah yes yes yes and then that is sort of the goal of the the the whole the the whole premise of of the infrastructure is code movement although in most places it's still the case that you know you spin something up and then people still do those games this is I what I'm describing is also more of an ideal scenario how it should be done versus how it is actually still done there's you know know there's still a little bit of a gap I hope that answered your question

yes

absolutely oh absolutely absolutely this is this is definitely one of those things that uh so in case some people didn't hear this uh it's it's like it it reduces the amount of tools that you need to use or that your team needs to know because you can go by the um what is it called by the things that your team already knows okay um as per the uh evolution of security testing with IAC so now okay now that we've been talking a little bit about it um let's think about what if we do true IAC and I'm not just talking about the resources that we acquire from our cloud provider I'm also talking about the um

the configurations inside containers let's say you spin up a container with a with a postgress you can actually keep that configuration file also inside your code repository so that means you have a full visibility what you have been configuring what you have been doing uh out there so there's a ability now to do actually what we call white box penetration testing so assume now you have like this total overview of this is how my application is built and this is how my security goes and this say it's like a little visual of like okay you have an idea that you have like a wall there but there's like a little hole there let's make this like you have a

SSO but there you allow one path with basic o or something like this that you allow in your infrastructure you would actually see this right because you can look inside the configuration files you know how these things are configured you can find the little tunnel that that you allowed for certain special applications all those things you would usually not be able to find with what we call the blackbox penetration testing and I would always like to use the strategy video game example out of this so this is all the good old Warcraft 2 right imagine you have like this this map that is not totally not explored but you have only a limited amount of time limited amount of

resources and you have to explore with whatever your little orc friends are knowing about whatever you can attack right so so you have no idea what is behind but you try to gather and you try to uh fill out this map so this is sort of a Brute Force approach and this actually will lead obviously tosses some results and very common attack vectors but it doesn't give you this very clean view of whatever is there and now the question is of course why didn't we do this all the time well in this traditional model the hey this is the server don't touch sort of scenario right actually to get to those configuration files to find all configs

for your postgress instances for your web service it would require the IT team to actually SSH into a lot of servers gather a lot of files from you this was infeasible this was actually simply not possible although it would have been the cooler thing to do of course right H and um given that uh we can now move all of this stuff into a code repository you can literally copy it locally and and run it and then also do security scans on those configuration files locally enables the feasibility of of this approach right and uh yeah and in the end you also have of course since this is is also a journey let's say you

have nowadays A Cloud setup but people didn't use necessarily terraform or anible or what not to set it up you can actually get nowadays also snapshots from your cloud and translate them into IC and then do those uh config configuration scans and this is one of those things that I would like to communicate today this is a great opportunity to have a full overview on what you're doing and where your risks are openly in front of you for analyzing either automated automatedly or via uh yeah via manual review right so this is like sort of the what I think is the greatest Revolution for from I from a perspective of security which I provides okay so now one thing I would like to

add here is unfortunately right now we actually do have a little bit of a false sense of security when it comes to IAC and um yeah and its capabilities so first of all um let me give you some examples anible as an external framework compared to a cloud native one if you install if you have who uses Dynamo DB okay one over there two um so Dynamo DB set up in AWS if you do it via anible in the documentation you will not find a way to encrypt it at rest although a AWS has perfectly capability of doing this so you have to make sure that you're actually studying the capabilities of your cloud provider not

of your tool so this is like one of those things that are very important and current I scanners many of them are not uh necessarily figuring out all those nuances right so most of them Focus also adjust on terraform and cloud formation so for example if you use kicks out there you you would and you look at the anible scanner uh it actually only looks at AWS Google cloud or Azure if you use anible to do playbooks to install stuff on servers you are actually absolutely uh not covered there but you you get this false sense of security okay I have my IC Code scanned so even uh yeah this this also comes under the

point that for those suppored tools those uh scanners are not complete and the other thing that I would like to mention is we should also not forget when we're talking about IC is is not just this top layer of terraform or kubernetes files it is also what is inside the containers I said the web server configurations the database configuration the streaming server configurations everything like this should be in your codebase and also should be scanned okay so and this is what mainly is ignored and if you're looking at the term of container scanning as it is being used today by many providers out there what they're doing is actually a cve enumeration because they're looking at the uh

software that is installed there but not actually at how things are configured and if they are done right in there if a mongodb for example has its default non- authentication mechanism disabled for example right so anyway just to uh because I'm almost out of time uh let's talk about like how this sort of can change the future this actually enables us to create this concept of self-healing infrastructure so as we have for code we have a good way of sort of the process of creating and fixing bugs and uh and and and uh iterating over the code base we can now do the same thing uh over um yeah over infrastructure um and open source

container orchestration contains Enterprise level um mechanisms so such as failover and other things that usually people would do in a very makeshift and manual way in the classical uh scenario uh scanners can reveal then if you use those mechanisms or not and predeployment testing like in the software world will make infrastructure most still so that's the sort of the goal that we're having with this okay so uh then the other thing that we are this would enable is if you're looking through compliance Frameworks and yes no I'm not a CPA but I've read the Trust Services crit IIA of sock or Hippa or whatnot if you're looking over this a lot of things is actually hidden inside configurations

you cannot necessarily ATT test that something is done right but you can at least say something is done wrong if can find in configuration files uh a lack of authentication mechanisms or some encryption add resters or encryption and Transit um um sort of parameters missing and things like this and then you can have obviously with this infrastructure Scot movement the change management uh address so this is actually something that we can almost put on autopilot in the future at least the technical part obviously not the HR things that sock also Encompass and things like this okay so now let's uh drive a little conclusion and I stayed I think within my time frame so uh I promises a shiny

future we are there almost there okay but you know just just uh keep this all in mind I mean this is this is actually something that will enable us to do things very right and even smaller organizations to use Enterprise level mechanisms okay so then be aware of the current limitations of IC tools and scanners um and uh you can if you start the journey today you can actually reap the rewards very quickly and uh on our blog we actually have a bunch of articles that provide you with a lot of examples of where things can go wrong with current ISC tools and how uh to scan things and if you would like to see

some example reports uh and and uh some some uh ways how to misconfigure certain um services on your infrastructure you can actually check it out on our on our portal with some example configurations that we provide there okay uh other than that thank you very much those are like my uh handles and yeah any

questions

okay

yeah okay well that's that's of course a good very very good question so I mean the the the short I mean how I have done that in the past is like you focus on the functionality that the server needs to provide right so it it gets certain calls from other external services and you slowly take the functionality out of it right so you go function by function by function and you keep it alive but you you get less requests in it and once it is dead you kill it so this is this is sort of how I would uh I would go about it

yeah

so what you have just described should only be a Breaking Glass mechanism okay so you were saying that somebody actually just goes in there fixes something quickly because heck something is going extremely wrong so this ideally this shouldn't happen too often right but if you do this literally every step of the way you know you what are you doing you executing certain commands right on the command line so you keep a track of those commands and then you put them either into the docker file or you know wherever you have it in order to do those uh specific fixes now it also always depends so now actually I have to roll back a little bit there it's also

it's like is this a database prop is this a problem within that Linux system or things like this so usually if you have for example a database problem then it and and you have to basically manually go in and something goes wrong um you have to also look if you're applying data database change mechanisms properly like for example liquid base or something like this so you have to make sure that every tool and every process that you use has some form of traceability right and as long as you have that you know like this example that I said at first copy those commands in the in the into the docker file that would only work if it's something on the

operating system level or something like this but make sure that even for your databases or any change that you're making it has a certain um sort of tracking mechanism right and that is sort of how how you get around it but it you know you're you're right this is This falls under this sort of gray area if uh you have something exotic going on that is not very standardized out there yeah yes so what what you described here it sounds like the same sort of techniques we would already be using inspect sof [Laughter] yes should we anytime somebody begins some by saying something is sofware to find something this could very well be I absolutely absolutely sofware to find

networking yeah sofware to find radio yes I I I I absolutely agree with this so it's it's it's like this sort of and also all the problems that come with it and also all the processes that you are usually working with if you're using software should be applied here and that is like sort of the the change of thinking that you know need needs to sort of happen because butly Dev not a shipment fromst yeah yes that's also very fair yeah yeah you mention code scanners and other Cloud tools GP how do you build in conjunction with um your I a work so usually okay this is this is also a very good question so how

do you use code scanners today I mean goes goes back to like how do we do codes Cod well you have a Jenkins or you have a GitHub actions or bit bucket pipelines whatever you're using for your cicd pipeline you have this sort of testing area right so and in this exact same manner you create a pull request it automatically spins up one of those scanners and the scanner either gives you a green light or a red light and then either you fix something or not and then it goes over to manual review but yes so so the the co the principle should be the same as for for code nowadays I mean if you're thinking about

it historically when we wrote code we would create it code it throw it over to QA they would push some buttons in the hope to find something and obviously they always didn't find everything and it came back so the same we actually do sort of with infrastructure today right we put it on a test environment the QA goes in clicks around oh seems all stable let's let's ship it so and this obviously we still need a QA but you know this assistance of additional tools that can actually automatedly look inside of all of this is sort of the uh yeah yeah just further to that point though a lot of the stuff in your pipeline today for your code yeah is

looking for things like vulnerable modules don't think that's going to happen much here you're going to be thinking more about tools that look like Dynamic analysis and static analysis tools look in detail at what you wrote yes but you wrote not what you called in but what you wrote yes exactly exactly yeah so this is this is basically what we're doing at our company so yeah yeah oh

yeah yeah

but if I'm setting up an infrastructure with I need some sort of you know things need like yeah used oh those are many many many um so I mean generally I mean so it it really depends how you're doing it right so so yeah you need to have the the repository ones right that are you many of those are actually provided you by the cloud providers directly so you can actually look into those like I don't want to know advertise many different ones but yeah if you want you can come after the talk to me I have I have a bunch of lists here right here excellent well thank you so much Albert yeah