BSidesMCR 2019: Navigating The Red Forest - Derek Price

I had everyone and so we're gonna go through navigating the red forest which is helping you understand how you can secure forests with Microsoft's es a and so Who am I so I'm Derrick price my security consultants and CC group I've been there for seven years and started off as a junior working way up starting off with getting things like CRT eventually come in getting CCTV our team leader so very interested in infrastructure which is where a lot of this sort of comes from a lot of people a lot of test involves testing Active Directory so it became quite an interesting thing for me to sort of like look out and investigate so the agenda

is to go through you know what is Microsoft's red forest or ESEA got design and we've got how to build and all the gotchas that we've found along the way and how to use privileged access workstations and their design deployment and not a little bit security 101 so the main reason and that idiots exists is actually due to a lot of issues that people have and when they're on enter so anybody put you hands of your pen tester in the room okay quite a lot of you and so how many people have got da off an intern or job very quickly but a lot of the time you know you end up with you know da credentials and you need to be

you look how many dimensions choices there are and there's an absolute there's so many and and a lot of times there's no simple fixes for them and a lot of time you spend talking to the client about how they can secure it but they can't change that their forest and you end up talking to people and having these sorts of questions and people that don't understand that directory or they put groups in groups within groups within groups or the director looks like a complete mess and people have left and people have made changes to the directory so that it can't be changed in the future that people got the reversal encryption and all sorts of crazy things

turned on so Microsoft came up with the yes a a which is what this looks like so what you actually have is you have your normal forest which stays as it is but you actually build another separate forest which becomes your administrative forests which or yes ei what this allows you to do is is to have to keep your forest exactly as it is and Microsoft have absolutely used this in their own infrastructure so that it Microsoft are quite good at sort of doing what they say and what they do what they recommend so they have actually built this and it allows you to keep your forest as it is and just have this set one next to it

there's a bit of a better diagram here and so you actually build your forest completely separate and you have what we constitute as tears now with tears what you end up with is as tier system that works like this so you split all your infrastructure up into into tiers so in these tier 0 you have your domain controllers new parameter and you also have your your exchange servers and your tr0 and then your tier one isn't normally the rest of your service and then you have your tier 2 which is normally your workstations now what you actually doing here is actually splitting up all the infrastructure in your actual having different sets of credentials for each one so your tier 0

is my monitor maintained by set of credentials you have your tier one with a set of credentials you have a tier two and none of the tiers could talk to each other so your tier 0 account can't log into a tier 1 and your tier 1 can't log into tier 0 so you can't go up and down it's all keeping it all and sort of separate which allows you to have a set of credentials for each one on the separate forest now when you're coming to the stage where you want to design and you want to build this infrastructure you need to have some

[Music] [Music]

all the temporal differences are mine [Music]

[Music]

okay so and so when you build miss you have tire putting some some design considerations so you're gonna build this on all new infrastructure you don't want any of this infrastructure to the old legacy kit so you want to build this on new hardware needs to be in a secure location the recommendation for Microsoft is to use hyper-v and a big cluster you need to have separate storage cuz you want to keep everything away from your your your your old domain and sister to make sure nothing's ever attacked anymore going to in science business only you're not going to do main join this new infrastructure until your old domain you're gonna create a new a new forest and you know have a

heavily sits very heavily segregated you don't want something to come in and attack your new shiny new administrative forests and so it's gonna be heavily segregated it's also gonna be heavily hardens a lot of people will have issues with hardening still stop people doing their day-to-day job or make things like take longer but on the red forest implementation you want it to be as secure as possible you also need to have the latest operating system so you need 2019 and it also needs to be the 2016 functionality level and 26 is a must-have due to the way and the all the all the infrastructure and all the trusts that you put in or or in 2016 and

2016 and one of the biggest design considerations is that you have to have it it has to be a one-way trust I've actually seen clients try to implement the red forest and they put in a two-way trust which means that the legacy olds main it's all happy but very easy to get into allows you straight into the red forest so you must make sure it's always a red part it's always a one-way trust now one of the biggest things that you could do you need to be aware of this as well is you don't want to be put in a red forest into infrastructure that is already and already has have lots of issues because you're still gonna get

compromised so you want to be able to fix it I saw the low-hanging fruit to start with so the things we recommend to clients is that they you know remove the legacy stuff like open shares didn't mean oh they're must deploy lapse kerberos thing and you must be known to change or you and Kerberos tickets to you know use your accounts to longer and possibly contract get rid of NetBIOS over tcp/ip and ll M&R get you patched up today but once you've got a client in this position you can then help them sort of like deploy the red forest the way to build this and is to use Hyper be 2019 it's got and it's got some nice new

features in it it's one of the best parts this is it it has a proper two node cluster so it's very simple to build and very cheap to build so you can should probably build a red forest and get all the infrastructure in place for around about 10 to 15 grand and because we only need two servers now and you can also employ lots of features and there's things like shielded VMs so that they are encrypted on disk and help prevent flight baroque administrators and get access to virtual disks or taken into other environments you also going to need things like separate storage and plugged into there we're going to build everything on a guarded fabric and give

extra to give us the security so once you've got all this infrastructure in place you can then start building and the only things you're actually required to do a red forest is to start with the domain controller and also server in there so that's it even you've got these two bits infrastructure sitting in it into crazy you can go into the just entire administration section if you want to but a lot of people just tend to implement the basics with the domain controller and also if you didn't want to do the just-in-time stuff the only stuff you've got to add is an SQL Server SharePoint Server and a Pam server with all the management stuff components installed so

not too much more so how does the red forest actually work and so you have this sort of concept on the screen so it allows your admin forest to be on one side in your production for us to be on the other and you have so they basically the privileges on that so you have your account in your admin forest and this maps over using the shadow principles so you have your your tr0 account and you can map that over to a domain administrator or a different type of tiered account on the other side that has admin and certain parts you have your tier 0 user the tier 1 user and your tier 2 user and these are all

mapped over you can sort of see I don't know how big that picture is other sort of quite good so you can see here you have your solid min and you have your and a lot of shadow principles here and we've got all the admins here and they're all mapped over to these objects here so these have all been mapped over and go over you can sort of see it a bit better in here so the sole admin here has had its privileges mapped over to the production Forest so you can see here that they've actually got in the production forest allotment but in the pocket do the throughout my slash all you can see that the domain admins is

all tacked there with the five one two five one nine and everything that's in this sort of diagram here so you can see the SIDS at the end there have been mapped over and which is using the shadow at the shadow principals and allows the ability of you to now have full control over your production forest from your administrative Forest so now you build this and new production and this new the administrative forest you know how this you is it's just sort of sat away all secure and hardened but you're still using your laptop and to to access it so Microsoft came up with and privileged access workstations and we're gonna go through what it's a Paul mind you need

one then go through why you should use a dedicated workstation why an administrative VM is bad and these have content regarding happening so you've got your nice forest all built and it's all hardened and all really good and this is the sort of extra thing you can do so the current threat environment most of our organizations is an attacker kept Sano or at the user's workstation the privilege escalates and they under the assets so what we recommend is is that you have a workstation that's in a position of you know what you want you want our separate workstation connected to your new red forest network which is a secure and you do that using and this

for the guts of where stations that's how a normal attack would happen so what you can do here is you have an administrative VM which is what a lot of people do now in administrative vm if I was to compromise your user vm your user host does the compromise your user host that would give me indirect access to everything on your system so if your administrative VN lives on there I can get straight into a mystery to VM pull out the credentials you're an attacker system what we absolutely recommend you do is you have a user VM so your host you open your main host your booting up is active administrative system and it has a VM on top which is your users your

user VM is having on to the simultaneous use so you've got your administrative host and your user VM if something is to compromise your user VM they get your low-level credentials they can't go anywhere else the sort of stuff that must have got a lovely zero day in hyper-v which you've got being a problem system is going after you in divide zero day for hyper-v so with the administrative host you're in a situation where that host that underlying code is completely firewalled off and everything and you have everything sort of sort of there but they have this user VM and it means that if that gets compromised you won't have and you know you can't leave the can't

break out anywhere they'll get some user creds and that's all now and once you have that up and running you can then need to get a lot of hardening done on the underlying host you want this you want this administrative her to be as hard as possible and have little ability to get anywhere so you can use things like credential guard and so you'll want to use connect regards to protect credentials and so that are in the slot in memory and you can do that using the hypervisors built-in capabilities and you can also you know we also need to go down the lines of hardening so the biggest thing is all about getting your hardening really good

you don't want something to be able to get onto your administrative host and compromise anything go go in that way so a lot of the things you got to do is lots of hard in lots of baselining and Microsoft actually have a list of hardening guides book of justice workstations with all the GPOs in them and you can also obviously employ a blocker so that the only things you should be able to really run is remote desktop and any powershell scripts that you need to get your privileges and lots of GPOs make sure the lights get listed and one of the things that one of the admins that who-who ever seen build this and they have to do they actually went

through and did all the lul bins mmm attacks and they manually went through and create gpo's and a blockers to stop that from happening so once you've got this you now in a situation where you have your privileged access workstation you now have your production and you have you put your production network which has been left alone and you now have your your red forest what you then have to do is you start to you want them to be in a position where there's no administrators left in your production forest so you can start pulling people out you give everybody their public assets were stations are new set credentials in the red forest and it

should get to a point where the only accounts that are left in your administrator many administrators group in your production one are the default administrator view which is gonna take in a safe for emergency brake lights and you probably your exchange a account which will go into wider exchange you'll need to stay so you've now got your if it if you know if your attackers on your network start doing all that usual attacks responder and the staff it's getting to the point where they've got some credentials and up onto a box and they do you know that I'm going off to a domain administrator there's actually no domain administrators left or if there is there's not going to be any

credentials flying around for them and they'd have to one of the big things about the red forest as well is that it has the ability to do the time to live on your credentials is really small so when you activate your account so you can say it's in public for four hours after four hours that's it's calm and that allows people to very granularly get their access complete the task and then all of the credentials are expunged from the system so in it when an attacker gets on they're going to find it very hard to move around and also as the tears can if anybody wants to compromise anything that's his sort of lockdown enough in a way that if you

were to get on a tear like the work station and you did get creds that lasted maybe 20 minutes you only be able to get on to your work stations you would be able to go any higher where I was a lot at the time when you're attacking if you know me just bounce straight up - and so it's in terms of main admin so one of the things that you a client will normally ask is you know what sort of walking are what sort of gotchas are there for the red forest and so the way exchange works is it has something called Universal groups that get created when the system gets built with exchanges Universal groups they

don't shadow well and I know some of the some admins that I've seen do it I've tried but it doesn't shadow well at all and enterprise admins doesn't shadow well either and so a lot of the time they can normally promote themselves on the production for us to do the enterprise admin tasks and then put it back in the safe and change password the officer server needs to be standalone on your red forest and so recommend you keep that separate and don't have it don't start poking firewall holes and into your red forest area otherwise you end up in a situation where and you start to put holes on something that's quite secure and so the

low bins is hard and manual and there's so many logins and sort of things to go through and harder and that it can be quite and quite time consuming and also need to make sure you put people into the protected user group so the pity people on the production and on the production on the red forest sorry get point strategies group so they don't use - don't leave credential flying around and also one of the biggest things which I think of few admins have come a cropper with is the shadow principles that you're using to make move York remove your credentials so that they work and shadowing over users it requires a functionality level of twenty

twelve on your production for us we had a few issues where a user or an admin decided to put all this and said all the symmetry so intercept for structure but is one of his domain controllers was 2008 and which meant is folks matter was 2008 and it got really weird and one login to sound systems one of the big things as well that's coming through is Azure and jaw isn't it's quite hard to in so to get into your red forest and what mic supposed to coming up with what a better way to do that soon and and that it's actually did it end of my talk so has anybody got any questions yeah and so obviously if there's not a

lot of people done it yet which is sort of the reason for the talk and not a lot of people have actually invented yet haven't seen one actual working example and it was a learning curve there was deficit issues and by the protected user groups wasn't put on and so there were leading credentials lying around on servers and a lot of people have to have to build it and make it slightly insecure to get stuff moved over to start with and see the problems with firewalls but I've seen seen some very good so the good thing about the far the red forest is that you build in you build some completely new to the side so

you don't actually have to disturb your forest at all to start with so you can build build it get all the laptops up and running so you can basically have everybody doing it but they can also they can you know they still happening with their old sets of credentials and then what you slowly do is is you have to unpick all the tech debt that you've got in your production for you do say you met group administrators and you see there's 45 domain admins you know that's gonna take a lot of picking we normally see where you know I get the service account in there so we create the service account in 2013 and never

knows what the password is for it and it's also running like exchange and it's ruining all sorts of stuff so it's a time consuming process to unpick but what you should end up with is your production first days as it is very nice and very stable and you just slowly unpick the administrative signs of it so you prefer so we take all your users out there in there give them the privileged access workstations and let them let them carry on and then you can start don't get the service accounts you know start with the easy ones like backups first a lot of the time you'll probably find out that most of the domain administrators don't need to be

diminished administrators it was just easier that's what a lot of people used to do just go that doesn't work let me just provide that to a domain admin how it works we're all sorted so you could push that out to you know to get to do these lots of nodes to go through those and then to start moving stuff over but when you move over you should literally be sat with your production forest which is still probably got issues but you should have the administrative side happening from over your on to your secure side and when people are moving you know when things are going over there's you know they'll be no domain admin hash is running around I'm like

responder or you know let gonna ship in it and an open share somewhere and if you've got laps employees you know if a user gets on one box they shouldn't be able to then migrate to you know 20 30 other boxes then get stuck on one place and trying Icicle a will be a lot lot harder especially when it's time to live I stood one up so we haven't come across one where we've been Tessa who helped build build them but I've stood on top the lab environments and you know played with the features and seeing what doesn't work what does the Shadow Copy like Exchange is and there's no shadow very well and what we haven't seen I

said is not a lot of customers or people that do becoming a lot warmer getting a lot more people talking about it when I go to do in the system all I've seen this or people start to become while aware of it but it's been out for quite a number of years and it's something that Microsoft sort of pushing a bit more now it's definitely this is not a lot of detail that most of the people that have built this having to build it with you know very little documentation and Max was like yes here's all the information here and what we what really want you to do is to pay 200 grand for them to come in

and do it all for you make it look all lovely yeah so to actually build it it doesn't actually take that long because you need your built in something completely separate so find some tin and so ordering you know kit takes about a week it's a stand it up you and you actually need a domain controller and what's a server and to put you know a cluster in one location and the cluster in another location so once you've actually got that sort of stuff done you've made your forest you you know you don't know the DNS and all they are they getting it up and running and then you've put that trust in so that your production forest trusts your

red forest it's it's not actually element to get into implemented you know you create your accounts so all nice simple process the hardest part of the red forest is to under the hip all the legacy stuff that's that's lying around on the network and that's the hard bit so you could tell you you could probably build a red forest in a month and have it working you know to get all the privileged access workstations sorted distribute them to everyone and give everybody the credentials the tier zeros tier 1 to tier 2 accounts get all pacifies in place but then you've got to then go right let's start removing the domain admins from the production forest

let's move all the users that have got village assets were stations were we're left with 20 service accounts that you know probably one of them breaks payroll or one of the HR system relies on them but it's also an SQL Server password so it's all the tech DEP that you've got to unpick there's an hardest bit of a building it should entail on that's on yep it's a nice shiny tip [Music] so so that is a harder solution to employ or launching at you are once you get started you've got to be you've got to want to do it so the point the point where I've seen most places are they're rolling it out and even there you know

the grow adapt to help desk to help desk up early Jessica were stationed with the tier 2 account at only intuitive account and they've been break sleep quite well you it goes it does curb a lot of bad habits as well so people that are like not liking the hardened workstations and stuff like that but once you've solved got you you group several pop in the Union you've got you two zeros given you even log on to your domain controllers you've got your tier ones that logs into all your service people get more accustomed to it and then you know they've got the per the traffic were station and their admin console is their house that one's going to a VM to do it

so they aren't doing it all it's you are gonna need somebody that does understand it to help implement it well once you've got it started it's not it's not there's definitely an increase in if they need some some bright people at the top will help and design it but with the what did you say with that can you bender coming in the previous example you better go in it because because everything's obviously off to the sides they they can they can be given domain up in the container to do the work in infinity but you can speak to most products that you get in now don't need the main either they say they do but you can also do know you can do

store you role based access control stuff a bit better you aren't going to need a couple of admins that understand this you know through and through and builder and debit and so I've been a Windows administrator before and I got open with him and I lapping but never labs not on picking a a very big domain but I'll played around with it and it didn't it didn't take the crazy amount of time to do it once it's done it's pretty much self sufficient because you know you got all you always stuff happening on one side okay I thought the places I've seen in employing people aren't people struggle to their start to understand it book the people that are

then there now there's like second nature to them so you pillage upsets workstations connected to your admin forest you can you probably increase the see yeah so you can probably increase the security I do feel justice workstations in general so you could take it if you everything want to do the red forest you could probably take the privileged access workstation concept an employer because is quite a nice concept that you know your admin host and you use a VM which means you know you use it gets compromised they can't break out anywhere and go and get admin credit so you'll probably increase the security of that and the pillage SS workstations is like a separate part so people people

can't just build the red for us and not use pod SS web stations but it's sort of like asking for trouble there and but you know you could definitely just the pillage assets which station is a full hardening guide on Microsoft's technical articles and they have like a level 1 to 5 and level 5 is a privileged access workstation and it's got all the Greek policies in there and then it's good to you the bases just roll that out and it's got all the guides on how to orbit is that answer your question [Music]

so with you read the forest you saw that you're making like a Bastian forest now so you do know all the extra hardening you're doing approved access workstations cause I thought that sort of hand in hand with it and and it's it is just that pointer I want my trust in with you and you're doing it in a way that you're making it more secure which is why Microsoft like ordered them read for it so it's not just like the one waitress that you're putting more a lot more effort into doing it you're also doing the tiers so you put one my trust in and you just have you get to some domain admin side

over here but what they have to done with the preference got concepts like you're adding it tears in so you got your tier 0 for any domain controllers you tier ones via servers and you to choose view workstations it's an approach so the the functionality that you need is that shadow principles and you come in 2016 which is why the red forest has to be a 2016 because a lot of the features to do all the Trust's and the put and they and the shadow did the shadow principles only in 2016 and so that's that's the thing that comes along but it's more of a concept and more game admins not to be lazy anymore you know

don't just log into everything with da logging you know make sure you use your tier 2 accounts and there's a lot of orders then you'll have to do the start to say why did you log into that domain controller with eater to account or why did you try to do that because it's less than credentials manner and so it's more of a concept and lots of other things so there's not to me and you stuff it's just a functionality side on one on side yeah questions yeah we're going free yeah so that's the principle that most people are following is to fit is to make exchange treat it like a domain controller because of these these these

grew these special groups that creates that don't move over and they just treat them up it'll make the chore you probably can get da from exchange sir yeah and so you can probably pull out the logs using your conventional system if they're gonna poke a firewall hole just to pull out your logs should be the end of the world so you wouldn't need to produce it so you've got you've got all your log it happening on one side well there should be no credential you should use Windows Event log there have been some doubt and you see logarithm would sit on your production side not in your red forest you could you could you could build a separate

same instance if he wanted to but it's got to eventually talk to a stock so it's got to be a little bit compromised on file rules to allow you know stuff to get out to get to where it's at your location as long as you've you know lock down the source destination and ports it should be shrinking the attack surface to like it's already going to see well finish thank you very much yes we thing to do certificate stuff as well yes as I said in like one of the things is to make sure your clients not got all the general you don't want to build this right amazing thing was something even still like you know ms 1710 the entire

network because they're just gonna go around attack the whole network and go I didn't need to may not me because I could and everything all you know a shared password or a horrible open share the red forest is a bit like open fix open shares you need to go away and make sure you you know you get away with all stuff but you're right you want you want the low-hanging fruit to go away and then sit even make it even harder so it was to get like a beacon on a host