Fawlty Towers by Charles Bain

hi everyone I'm Charles from North Korean security I'm a penetration tester and a trainer and I've kind of brought a talk to you today all about infrastructure as a code how we can manage it how it's changed over the years and a few issues around some of the key products that are out there to manage said infrastructure as a code but first of all we need to establish what environments are in organizations we're all aware of when we're kind of in and out we have the three kind of key ones we have development pre-production and production and they all come in a variety of different risks information use data use and they all need to be taken into consideration when creating these environments so for example development are quite often just spun up for the likes of as it does but it doesn't intend to develop pieces of software to develop infrastructure you're very much there to try and have an understanding of what you're trying to achieve When developing your overarching infrastructure let me have pre-roduction which is kind of that middle Gap in between that full deployment that full y running service when it comes to the infrastructure that you are using and this is more when it's going to have maybe replica data we're going to start to see traffic flows occurring to be the same as if it was to be in a true production environment then we have the likes of your final piece your production environment this is very much where customer data will flow in and out you've got the kind of correct service lines being given out and the risk is a lot higher if it goes down being the key factor for this particular point of production it's that concern that's going to fall it's that concern that to get it back up and running to that continuity level that you're looking for for that business line is going to take days it's going to cost so much time so much Manpower and so on whilst if someone to turn around see yourselves and say the development environment is going to fall okay great we'll we'll just spin it back up another time we'll we'll go from there and then pre-products like oh it's a bit annoying but again it's not a huge loss it's fast reduction we're very much looking at trying to speed that up infrastructure has changed very much over the years for example the 1980s we've got Apple three we've got ethernet networking Contour eyes and we've got a classic windows 95. you know everyone has that kind of golden operating system Windows 95 is right out there for me and again you've got this this take of the first modem was used in 1996. and then we start to progress we then start to change we then start to be able to manage attributes objects and all sorts with the likes of Technologies of active directory we're now supplying updates to our operating systems using Windows update per se technology is really really changed dramatically Each decade and then it's just completely accelerated when we hit the 2010s we have the likes of terraform we have Azure we have ansible we have Google cloud we have Docker we have all sorts International infrastructure is no longer kind of useful in a sense of putting it into a production environment especially with Technologies are out there we don't want to have that approach of if it dies it really is dead we only know one person who can fix it we want to avoid that we want to try and move away from that and this is very much where these Technologies can come in you have terraform and ansible fantastic infrastructure as a code pieces of Technology and then in 2020s we've got the likes of cloud 365. you know this is very much a case of the Pinnacle of reusing someone else's computer all the time we can directly log into a machine that's hosted in Azure that's a normal desktop and that can be linked directly to an endpoint now it's kind of this weird like thing client hybrid approach that we can take and it's effective because if that endpoint is destroyed you've immediately got that understood running up in the cloud It's very effective there's a loss factor is just replacing that particular device and again the kind of Icon at the top right there is bicep that's azure's kind of new infrastructure as a code language which allows you to spin up multiple different tenants and all sorts being Microsoft thing obviously why it's already surrounding technology terraform wasn't good enough for them but it's very much there is there for you to take advantage of color move forward with but this whole transition this whole 40 decades work of wear it's very much kind of favored towards the clown and it's not stopping the stats are very much agreeing with this so for the likes of Google they say that 41.4 percent of Global Tech and Business Leaders plan to continue investing in cloud-based products even if there is economical issues it's a non-stopping force currently we have Excel Euro Escape have very much rated it as 1.2 trillion dollar area and public kind of usage as from September 2022 they're not small numbers it's their thrive in business it's something that we really need to pay attention to and then we've got G2 I've estimated that 51 of it spending will shift the Cloud solution front edition on-prem by 2025. but throughout all this there's this kind of masking pattern that the majority of people's infrastructure will completely shift to Cloud but how do we manage it how do we manage this effectively trying to make it kind of more in tune with what we're lined up to we don't really want to pay someone to spend like many many days going through these configurations one by one and going very good because that's no different to on-premise infrastructure you know we're still applying the same management tactics for some reason you know we're still taking that high touch maintenance approach when this is we've already seen we've already gone through four decades worth of research and development why is this style of management still the same we need to we need to adapt we need to change it and this is very much where the likes of infrastructure as a code comes in we can code out we can map out through the likes of yaml files Yara rules and all sorts on how exactly you want our image structure to look we no longer have to sit there waiting for an item to Bone to a disk and then to watch it install and go great now we've got Windows updates I'll see you in two days we can streamline it we can make sure that's effective and we think back to the likes of development pre-prod and production environments well that's great because if ever a production environment follows and we've got infrastructure as a code slap it back up point it to where you want to run them like build this for mate then we'll continue service and that's very much the kind of dull here and red hat very much defined as management and provisioning of infrastructure through code as opposed to manual processes that we just keep slogging on about we don't want to have these manual process anymore we want to make sure we're up and running as soon as possible but how and I've kind of use this as a process it's very generic process that I can put up on there it's going to be different depending on what product you've chosen some of them like to take an extra step of initialization but first we need to of course create the code which can be a really nice outline within the sake of a yaml file detailing what operating system you want where to pull it from so whether that's Azure AWS maybe you want it from the likes of vagrants because you just want something headless just completely spun up straight away there's that availability there so we need to create that code we need to make sure we kind of Foundations have been drawn up to ascend like some certain degree past that point we need to plan and test it so terrible we've got a great module where it's test where you can actually type in terraform space test and it will plan it out for you and it's it's fantastic it will see whether it's actually going to work on AWS Azure whatever it is your target is and then as soon as you're ready you can execute it there is a complete pre-check of how your infrastructure is going to work and how it's going to look as opposed to building it again that's not quite right we're not getting Network traffic from A to B and we don't know why or this host is completely different and we have no idea why well we've had a different 90 admin over there and they perform the installation but it's a bit different because they do things differently there is none of that there is no archery and there is no difference it's just completely uniformed exactly how you like it it's completely outlined to exactly how you like it but again we're looking to make sure that's ready to go we want to make sure it can be deployed smoothly again the biggest Point want to stress here is continuity is the infrastructure is code in a production environment and afterwards we just want to play an old simple to play it we're going to make sure it can go up and it's ready to use well that's great that's very much some of the Technologies that's used to outline it but we need an engine we need something to drive it and that's going to be down to the likes of something called a provisioner so if ever you've had the joy of playing with infrastructure as a code and not using provisioning with it it's kind of like buying a really nice car and then not having something any roads to drive on it's it's useless it's absolutely useless so with this in mind it's a case of you need to make sure that you've decided your provisioner I would argue first how do you want this code to be point and shutout is that the case if you want it gone to the likes of a local instance of esxi maybe proximox is it something more on-prem that you want to make utilization of in this fashion or are we doing it to the likes of traditional cloud services like AWS and azure either all is absolutely fine just need to decide how that's going to be pushed through so for example we got vagrant up at the top which is a great provisioner to be able to push through to local hosts so if you've got virtualbox and you want to play around with infrastructure code you can deploy multiple color instances you can deploy fantastic element environments just to have a little play around with what's currently available and then terraform being one of the primary Provisions that you use across multiple different parties it's its libraries are fantastic it's vast and it's adaptability is great and it also has a health checking function as well so if ever your infrastructure changes you'll you can get it to set up so it notifies you and it'll automatically correct itself if you configured it properly as well that's a resilience as well as continuity kind of all under one Hood it's a fantastic way to move forward with it all but then you have the likes of configuration management so we provisioned it we can say spin up three Windows servers two minutes service over there great well then we'll do that but now what well we want to do a little bit more kind of finite configuration management well using the provisioner we can configure but if you want to go even more sophisticated this is when you start pairing up Technologies so terraform and answer I don't know why they looked as opposing Technologies because they work really really well in harmony if you have terraform supervision your Windows host your Linux host and you go and so here's a Playbook that we've used and we know that works go ahead and run it build it for me straight away because you can get terraform to call ansible straight away and continue forward and a great thing again the ansible is this item potent so if it needs to make a change I'll make a change otherwise it won't and then it will recap you at the end to say what's been changed what hasn't which is awesome all right if you want if you create that Baseline you get all you want to be re-hitting that Baseline then there's nothing more that we can ask for and it's agents so I was sick and tired of installing a software where you have to try and get either whatever they want to call on sidecars agents whatever the label might be you don't have to go through that process of trying to install a multiple different hosts and to try and make sure everything's communicating well it communicates over SSH can I log in and if it's if it's the likes of Windows it looks for winrn our right management kind of service line if it can communicate on both of those you're good to go you can install whatever you like and then configure it incredibly quickly but the two key files of understood that we need to be aware of is your playbooks which is the yaml files that will outline exactly how your operating system will look and your inventory it needs to know where to point to but again looking back to your provision now your provisioner is ultimately your inventory so you can say it's a terraform create me a file of all the IP addresses and categories what's being created and you just push that for example Phil we have our section web we have four web servers here okay ansible the only web how do you make sure patches have been applied to nginx it's a really smooth kind of transition forward I'm forward approach to infrastructure we kind of applying a CI CD pipeline for two production environments which is very much the kind of goal of this talk anyway so this is very much what a very basic ansible kind of yaml file will look like so what we said is we've gone provisioning Kali Linux and install tools and the tools have been put into a particular list that I put into a provisioner configuration which I'll show you in two moments and then it's a case of we want you to apply updates and and then go systematically again through that tool list to make sure everything's installed correctly that's it that's the whole file just in case you think that I've got it short that is the entirety of the file just to spin up Kali Linux our local machine using virtualbox through the likes of vagrant and we have vagrant here just listen out the tools to install as you can see we want to make sure that is installed sliver half scan Covenant C2 and that's my framework nice and simple as you can see it's config.vm provision we've just pointed it straight to unscore deployed our machine answer all you do the rest like it's a really nice and effective way to deploy that particular infrastructure but not all teams are equipped to deal with the CLI fashion match landline interface approach but the resultable tower which is kind of going to come more of the key part of this talk essentially managed you can create users you can create organizations and you can make sure that everything is smoothly locked down to whatever provisioner or cloud service you want if you've got a huge team must be beneficial if you've got a team of under 10 I would say it's not worth it but from the management perspective of what needs to be run to check in the states of these jobs you're not constantly having to look at the processes on your machine and see what's been running what hasn't you can just simply go into jobs or projects and just see what's being run where from and what files have been used so we've got a little glimpse of basketball Tower here where it's going to be logged in and it's going to look at the likes of your inventory which is really key part of answer because remember this is the part where we're going to point it to the relevant areas on the hosts that you can touch that's really nice and simple we can very much add in-house we can specify variables and so on and then immediately we've got our inventory there to call upon for projects well that's no different to when we're looking at projects we can quickly add one and detail it through exactly what we want to call it what inventory it's going to call upon and what actions it's going to take next but it's not just local access that we can configure like in this one we've configured it to 0.2 GitHub so if you've got your code management using the license gitlab or GitHub whichever way you kind of go about that you can point ansible Tower directly to it and it can pull collaboratively down to then run these environments it's really nice way to go about it but of course with every product there's the faults there are boundary issues and we need to make sure we can address them we need to make sure that this is not going to be something that you're subject to especially considering of the whole talk I've spoken about using this in production environment I don't want anyone to go away like cool downloaded it we've been compromised like no that's not what I'm here for at all but the three key things that I'm very much found with ansible Tower is the API abuse so during the this is an authenticated attack it's not something that can be done authenticated so you will need to already have a valid user account compromise you have to complete this but as you're going through the creation of um organizations within ansible Tower you can upgrade a particular user and you can make sure that this user has super user rights and as you can see the part kind of highlighted its adjacent requesting it using the post method and it's just a simple variable of true if you modify that there is no server-side validation and it will continue through creating that account you've suddenly upgraded an account from what could be a standard user to see the user writes where you can create as many projects playbooks change inventories it can cause a lot of damage you can even spin up brand new infrastructure if you wanted to which is the part that really interests me all right just something spin up anything kind of going everywhere what if you're coming from an unauthenticated perspective well unfortunately it's not as interesting but what you can do is that you can connect over the likes of the websocket service line and perform information disclosure so here we've got a request going out to ansible Tower over websocket on the response we can see what jobs are running don't see any other details so that's about as interesting it gets unfortunately with this attack but we can see what jobs are running we see that it's got project ID of 56 and it's pending okay we can kind of determine the value of that particular ansible Tower Service depending on what what how many jobs are running potentially but that's about as far as it goes in this one but it's just something to consider we were trying to keep information as close to obviously a minimum then we have the likes of ansible Playbook misconfiguration so when you install an ansible Tower it they tend to point you to the usage of Docker or of course using ansible to try and deploy their own product which is great you know but the default settings within the within their own inventory file it's quite ridiculous so you've got the case of you have the host names you've got you've got the credentials which is the far more interesting part so it has postgres SQL database which you can have direct connectivity to and it's just kept with those credentials unless you actually go in and change them you'll quite have to find default deployment we'll keep those credentials as you're saying with our web portal it's asmin admin and that is very much kept by the phone for something that's as we discussed earlier is four decades worth of work and I kind of stuck with default credentials it's something that just needs to be addressed and needs to be fixed the bit that I found quite interesting is what if we found ansible on a compromise host that can communicate back with ansible Tower so for example let's say we had a misconfiguration with pseudo rights and a standard user was able to run an ansible Playbook due to his service designated by Artful Tower is there something we can do with it yeah absolutely we can go ah