BSides Berlin 2023: Evgenij Smirnov - Bye Bye NTLM

thanks a b thank you very much uh welcome to this session um big shout out to the organizers for making this possible to the sponsors for yeah making this possible as well um let's have a quick show of hands it's uh really hard for me to see those hands but I will I will give it my best it's not on slides but let's uh let's have it it uh three questions real quick who are you here who of you here are running in let's say Enterprise Computing environment that does not include Windows of some kind great so this talk is not for you so from the remaining uh 99.9% um who of you are running an

Enterprise gr Computing environment that includes windows but does not use ntlm to your knowledge for authentication right let's lower the bar a bit who of you are running an Enterprise grade Computing environment that does include windows but does not utilize ntlm in tier zero wow then you've come to the right place what are we going to be looking at today we have a lot of ground to cover we have 30 minutes to provide solutions to problems that cropped up over 30 years so we'll do a quick recap of why ntlm is bad for your health if you are from the blue Corner if you are from the Red Corner then ntlm of course is a

gift what Microsoft is doing to uh help us out of this mess what the rest of the world seems to be doing in terms of soft Ware and Hardware vendors uh that should be aware of these problems as well and for those of you who are Defenders or admins what you can possibly do today what you must do today once you get out of this conference and get back to your office or your home office or whatever uh and what you as Defenders and or admins should look at doing tomorrow my name is yeni I come from Berlin uh it's all over my social media handle so I can't move away without breaking my whole social

media profile I'm Microsoft MVP with not for security cloud and data center management I spoken at conferences before mostly Powershell related stuff I run three Microsoft themes us themed user groups out of Berlin so if you're into Microsoft Technology give me a shout so a quick recap how nlm works for the one person in this room who doesn't know how that works um basically it's about a user on a client computer which ideally would use Windows because ntlm is a Microsoft proprietary technology uh introduced in Windows NT that's where NT in nlm stand for to to replace Lan manager which was the Authentication Protocol before which sus really really great time uh and tlm in the first issue

sucked also but will get into that so a user that is logged on to a preferably a Windows client would like to access some resource let's say fil share on a server and for some reason they NE negotiate until until M to authenticate that user um we should all be using cus for that nowadays but there are scenarios where inm will get negotiated even in modern operating system so let's assume they have negotiated Ana or if you are from the red team you have used some protocol downgrade technique and coerced the server to be using ANM against you so uh the server says well User it's good prove who you are and of course it

does not send the user the string saying prove who you are but it does send the user a bite array which we call a nons in cryptography the user is happy to apply and they will let's see if this cursor thing works yeah uh and it send sends the server a package of information uh saying who's the who the user is the nons this bite array that the server initially uh send it and uh a cryptographic uh encryption of these nons by a private key or it's a symmetric one using an md4 hash of the user's password so now if uh if both of these communication Partners were in a work group then the authentication data flow would

be over because the uh for for authentication to occur the server would have to know not the password in clear text but at least uh these md4 hash uh of that password so the server would be able to redo the encryption it knows the nons it knows the username it knows the password hash so the server would be able to verify if uh this encrypted um if this encrypted uh data that the user is sending uh is is valid for that particular user nothing fancy here and that's what this protocol was actually uh designed for it's a challenge handshake type protocol but enter Central authentication uh active directory domains or Windows n domains even so the

server does not know the user uh the Server doesn't even know if the user exists so it will go out to a domain controller who ideally should know that the user exists and also know the password of that user and send the domain controller this whole information package the domain controller should be able to verify if that password is valid by redoing the encryption again now the domain controller does know the nons it it knows which user uh is meant for authentication so it should be able to redo the encryption and um compare the uh result to what has been supplied by the server and if that checks out then the server sends a positive me uh the

the main controller sends the positive message back to the resource enriched by the group IDs the user is a member of and that Ena the server to provide authorization for the user basically look if the user would would be authorized to uh use that share that uh he is requesting and if everything is okay then the access to the resource is granted so that's how it works there are problems with that uh first off you see [Music] arrows on the right side of the resource and you see errors on the left side on the of the resource at no no point in this data flow was the user um obligated to prove their identity directly to the domain

controller ideally that would have happened uh on interactive log on but it's it's not necessary in Windows so there is a divide here and there is nothing in this authentication flow to ensure that that uh the server or the resource being divided by the red line is actually one in the same entity so a person in the middle is perfectly possible if you manage to become that person as red tea you can talk to the left to the user and you can talk to the right but you can do different things from what the user has intended to the second problem ntlm does require a password for every authentication nlm does not use any other authenticator than a password

so we stuck with passwords that have have to be typed into a sort of login form as long as we are stuck with ntlm then the cryptographic function the the outer function the encryption function is solid it it's even solid by today's standard but the password hash that is uh serving as key for that encryption is not very up toate up to date anymore right md4 also known as NT oneway function n WF it's crackable as long as the password is has reasonable uh size reasonable length so relatively weak cryptography next thing again um the resource will be sending all these data package to the demain controllers so a replay attack could be theoretically possible because if you

capture a packet um containing ntlm authentication you have all that information that your disposal could use that for other uh nefarious things but Microsoft tried to amend that and uh this of course happens at the cost of the domain controller knowing about every authentication session that has happened uh uh against that domain controller so it basically has to keep track of every nons that has been issued for some sometimes uh for some time which puts an additional load on a domain controller in active directory that is um uh doing ANM authentication and of course as you can see here uh at this um data packet it does not require oh sorry here's the cursor this data

packet is uh completely reproducible using the password hash so with ntlm knowing the password hash is equivalent to knowing the password in clear text in terms of providing valid authentication uh which leads to the pass the hash attack which is widely known in the community both on the red and blue side so pass the hash will be possible in your environment as long as resources and domain controllers both support ANM so problems with that um the first reaction of course would be okay let's get rid of that and go full curus we will look at what's possible in a couple of minutes but even if you go full curus in your active directory based environment ntlm is

still bad for you Microsoft had uh made some design decisions as they introduced active directory into the world and one of those design decision design decisions they made is if you use passwords then the arc4 algorithm in cus is cryptographically equivalent to NT oneway function why did they do that they thought people will be uh will take servers with accounts on them and promote them to the main controllers so local accounts that only have ntlm cryptographic materials will have to be converted to active directory accounts uh and to provide curus authentication for those uh we'll just keep the hash because we do not know um the clear text we can't re can't reencrypt the password

to provide a cus hash using a different algorithm so we stick to using the same algorithm right which means that uh if you know the nlm hash but the resource or the domain controller will only accept curus Authentication you can't of course do pass the hash but you still can do overpass the hash which means you request the TGT from the uh from the key distribution uh distribution center from your domain controller but you use the nlm hash for that and just tell the domain controller more or less I'm simplifying uh that it's the arc for uh cryptographic material out of that password and the domain controller will will usually be happy to oblige then hearing that the knee-jerk reaction

would be okay let's let's get rid of nlm completely in the authentication flow let's give every user a smart card because ntlm cannot use smart card so if we if we give every user a smart card then they will be stuck with using caras that is correct but if you enforce use of smart cards if you do not enforce use of smart cards they still have the password uh even if they do not use that password they still have a password and they still have nlm hashes of that password flying around but if you enforce the smart card usage we're talking about Microsoft here they are known to put backwards compatibility above both usability and security for

decades so uh in the Microsoft implementation of curas there is an ntlm supplemental credential Field in in the ticket pack which contains Tada nlm hash of what if the smart card usage is not enforced then it's the intm hash of the actual password if you enforce the smart card usage it's an nlm hash of some very long and random strings like passwords that computer accounts uh um uh give themselves but but if your domain has been around for some time this cryptographic material is static that does not change no matter what password policy you have in your environment so you can can't do uh can't can't usually crack this uh thing because it's like 20 250 characters long and

completely random but you can still use pass the hash with it if resources will Exel nlm authentication and you can still do pass overpass the hash with it if your uh domain controllers will accept Arc four for TGT ISS so um check this attribute in your domain with a very comprehensive and short name uh if you're using smart cards and actually enforcing uh their usage so there are a couple of problems that we have to take care of until M has can be harvested I expect to be at least a dozen people in this room who know uh more about harvesting hashes uh than I have uh uh ever uh heard of but there are many ways to harvest and reuse uh

reuse uh those anti-lam hashes directory directly as long as the environment supports nlm authentication so what can we do to a orate that we should look at restricting harvesting possibilities we should look once a hash has been harvested at uh restricting usage and of course and this is an overarching topic that is not in scope of this session you have you should always look at restricting impact if the account has been uh taken over in this uh in this manner right so the second one is uh this uh condition that arc4 hash is actually ANM hash so the solution approach for that should be getting Ark four rid of Ark four in your curve r that will also help reduce

the scope of the Cur roasting attack the Cur roasting stay remains possible but if you only have a uh cryptographic material in those uh curb roasting uh tickets or curb roasted tickets then the roasting part The Brute Force decryption will take uh much more time than if you have arc4 which is basically md4 andm has performance impact it's not enough to um to Institute a denial of service attack on your domain controllers by providing ex excessive uh anlm authentication but it's measurable and it's uh all the more measurable if you work with virtual machines and your environment is somewhat saturated CPU wise then it could well be possible to even measure the impact of moving away

from nlm and towards kerberus uh on your domain controller performance last but not least that uh is something that people moving to the cloud had encountered if you start using Cloud only clients then it's absolutely possible and has been for some time to authenticate from a cloudon laptop that's only joined to uh Azure ID or anthra ID as it's called nowadays to on premises resources using curus you have to have a line of sight to to to a domain controller to do that at the moment um but it's possible to use that to do that using Cur so you could have a cloud joint client that is perfectly capable of accessing on Prem resources unless those resources are dead set

using ntlm because ntlm will not be possible from a uh client that is not the main joint uh or at least where the authenticated user is not the domain user because the cloud does not know the user's clear text password nor does it it know it's anlm hash it knows a much more complicated hash like thousand thousand um hashing F function passes over that hash but that is no use for basically ntlm so these is the problems that you will encounter if you try to move away from nlm uh in your environment and there are solutions for more or less all of them there are scenarios I I uh mentioned that earlier where we actually

do seem to uh not be able to avoid until am at this time this is of course an interactive offline log on because Windows uh is not capable of caching kerberus cryptographic material at this time nor would it do you any good because Ker's tickets are limited in lifetime so if you support offline log on to a domain join system then you are caching ntlm uh credentials and uh uh so ntlm hash has to be generated when you log on interactively where you don't have any active directory domain to authenticate against then you stuck with ntlm for the time being no uh way around it at this time we will see in a minute what

Microsoft plans on doing authentication in a work group no matter if your client is the main joint or not if the resource you authenticate against uh is a work group server not knowing about the domain not knowing about active directory or at least not being member uh in that active directory then you're stuck with ntlm because that other system also only can do nlm uh for the time being and there is a use case that is uh getting quoted uh rather frequently if you start accessing Resources by their domain uh IP address rather than by their DNS fqdn you're stuck with ntlm that is not entirely true anymore and has been uh for some time but we'll look

into that as well in a second so this stuff has existed for over 30 years what Microsoft has been doing to help us get rid of ntlm in Windows authentication until very recently was drawing attention to the problems that are there showing us how we can detect nlm usage in our environment you'll have to do that at some point but uh Microsoft uh implied we should start with that I'm uh I do not agree with that that is not what helped people and organization I consulted for uh actually move away from inlm but uh there has been zero guidance and zero technical assistance from Microsoft to actually uh make that move there are at least these two

blogs in the Microsoft blog space that I know of that start with part one getting rid of nlm part one then they described in great detail how you do the logging and the monitoring and all that data correlation and then they promise part two which never comes the first one is uh from net pile uh and it's it's 15 years old I think and the second one popped up uh fairly recently so um keeping the fingers crossed here that part two may actually surface one day but it's not uh there yet but of course every time you have Microsoft uh on premises uh doing a security assessment doing a consulting or even uh your favorite uh favorite

Microsoft consultant I was that person until a year ago um uh they will all tell you move move away from nlm that is bad for you until very recently but uh the team around um Steve cyphus who is the person responsible for Kus at Microsoft actually started doing development to help us to remove anlm from Windows there has been a talk by Steve at the the blue hat conference which is Microsoft internal conference uh and um one of the videos got published you will find that when the talk gets uh on YouTube you will be able hopefully to uh read that URL uh they describe what they are going to do to help us get ANTM free and uh what

is going to happen in windows are three things they are taking up developing the aurb protocol it is a sub subprotocol of kerberus that's been in the um RFC proposal stage since July 2007 it's nothing that came from Microsoft it actually came from MIT like uh the rest of kerus but this stuff got proposed in 2007 and um didn't move move uh forward much uh ever since but now Microsoft putting some weight on that scale uh to get it to become an official RFC uh keeping a finger keeping fingers crossed here so what will IAB uh be able to do every resource that is capable of C cus authentication will also be capable of uh proxying the cus packages

that are not designed to the for that resource and this moment the uh user has to get a ticket granting ticket from the main controller NE line of sight to that then using that ticket granting ticket it will request the service ticket for a certain service and then and only then it will go out to the resource itself uh and uh submit that service ticket to get access to the resource so this all uh communication flow will be proed through the resource so especially in Cloud scenario in V in a VPN scenario clients will not need line of to a domain controller to do cus authentication but only the line of s to the resource that

they plan on accessing anyway Happy Days the second part removing uh removing dependency on ntlm for offline and work group scenarios right every Windows system at some point in the future will get a local KDC a local key distribution center implemented on that system which will allow us to actually do a curus authentication in local scenario and the great thing about that is that Cerberus does not contrary to anlm uh rely on passwords only so you will be able to do um uh uh like smart card certificate or Fight 2 authentication on a work group on a standalone Windows system keeping fingers crossed here and this is in the works already per service nlm enabl you you

might have that one application that you need for the next two years you know it goes away in two years but it you need it for for the next 24 months and it only can do until am and the programmer who developed that is dead and took took the source cat with them into the grave right I mean it's it's a normal scenario right um even at Microsoft there are there have been tools at Microsoft where a certificate has expired and nobody KN knew uh where the SCE C it um so um if you have one such service you will be able at some point in time to only enable ANM authentication for that particular service but disable it

globally on the system providing the service great stuff the first step has been taken already um in my opinion it's uh it's at the wrong end of the scale but it's probably that what was easiest to implement you will be able very soon to turn off nlm authentication in SMB Windows File and print sharing protocol if uh that is not needed for accessing that uh protocol so this is what Microsoft will be doing to help us get rid of uh intellum they are doing that but it has not surfaced out outside of their uh development environment yet the second takeaway from this video is they did some uh Telemetry analysis on until authentication of uh large Enterprise

customers and slightly uh smaller Enterprise customers that agreed to give them that information and they broke down they broke down the ntlm usage by the condition causing that usage and uh I mean it's to be taken with a grain of salt of course but in their Reckoning 52% of ntlm authentic apption today come from applications where someone at some point in time hardcoded use using ntlm into that application espo was part of the Microsoft development uh offerings for the for the last 20 years the Urgent recommendation to use negotiate as uh the authentication uh interface in your applications if you're a developer dates back to 2010 and you still find application manifest even source code where ANTM is hardcoded as the

authentication package to use espo will fall back to ntlm if it can't negotiate Kos so nothing should theoretically break if you change uh ntlm to ESO in your applications and of course that thing with IP addresses rather than fqdn service addressing that is more on the implementational side of things it's more the admins who uh make their lives easier but not relying on DNS because they don't like the person manage managing DNS and because everybody me included keeps tweeting about DNS being the source of all problems uh out there right so like zero days since it was DNS uh te off calendar but since uh Windows Server 2016 uh I think is one of the cus that

came out shortly afterwards it's possible to uh to to use IP addresses in SPN and you're not enabling it globally uh you must set a registry key on a client the client must be a Windows system because other curos implementations uh still do not allow that uh and um if you set that registry ke on a client and if you set a uh IP address based SPN on the service account or computer running computer account running the service then you should be able to obtain a service ticket for that service using IP address of the system providing the service so that's what the rest of the world is doing basically depending on uh who is

your software provider of choice nothing at all or maybe reluctantly replacing that ntlm by espo making a smoke test smoke test failing and then replacing is back U uh because they have to meet shipping deadlines so it's on you to tester your uh application providers uh if you detect NM being hardcoded in that application now what can you do as admins Defenders of the systems until both Microsoft and your application vendors fixed this stuff several things first and foremost go ahead and disable the previous versions of the protocol if that is not in place already and nlm V1 enables the attacker to do a simple package replay Lan manager restricts password length to seven characters so if you

have a 21 character uh long password only the first seven characters will be used and to make matters even worse they will all be converted to uppercase um yeah so any environment that is semi modern in terms of operating systems and uh development Frameworks being used should not be affected by disabling landan manager and nlm version one your mileage will of course vary as is always the case in it but this is this is uh like 40% of the attx surface reduction of nlm just kill the older versions ntlmv2 sucks uh as well but uh it does not suck nearly as badly as the as the other two try to disable Arc four in curus

Microsoft tried to force that on us in I think October uh or September CU that didn't go very well uh October or September last year uh that didn't go very well um many uh many environments that actually hardened Keras in terms of encryption protocol had to reenable ark4 for some time at least until the Cu uh got rolled out to all the main controllers and all uh affected systems then they were perfectly able to reenable Red disable Arc four in cerus but um absolutely look into this because as long as you have local password based authentication and cus accepting Arc for then hashes can be can be harvested on your end points and maybe even servers and hashes anti-lam

hashes can be used to for overpass uh the hash attacks put interactive admin accounts in the protected user group that restricts their uh ticket lifetime uh to 4 hours but for interactive administrated Administration that should be enough and this is the only method available to you today to basically effectively disable nlm for one particular account authentication policies I could talk about authentication policies for a week uh I I have actually done so in workshops with customers uh it's it's it's the best feature in terms of Windows security that's been published since sliced bread and it's as old as protected users it's came came out with uh 2012 R2 nobody using that and this is a visual Circle because nobody using

that nobody nobody wrote about that and because nobody wrote about that nobody nobody's using that and uh so it goes on then is there is something you can absolutely do restrict pass through nlm authentication in your environment that is the next topic and I can't repeat it of enough least privilege in all system so reduce the impact reduce the outcome if you can't get rid of the uh cause if you already uh hybrid with anthra ID absolutely take a look at moving clients into the cloud because they will able to do curvs to your on Prem resources uh if you set it up that way but they will not be using nlm anymore so getting read you you're

getting the drift right getting rid of ntlm is nothing that you can do by pulling a lever it's a journey what Microsoft said about the journey I quoted that already start monitoring for now and we'll um get back to you when we figured it out you you'll drown in logs under way we will see in a second uh how how many logs that will be what really worked in my experience is start by taking tier zero and admins in general out of the equation get rid for an of nlm for your highly privileged account Harden services and service account in term both in terms of nlm and ceros uh Arc for enable cus in your applications

whenever possible there are those applications that could use ceros but you you have to tell them to do that least privilege everywhere and then you are at the point where Microsoft suggests that you start your journey then when you have done all that homework you start monitoring for identities at first to determine what users are actually using ANM in your environment because that is the lowest hanging FR we'll see that in a second uh and then when you're at the end of that investigation then start monitoring the whole signal chain and if you're done with that and know what's going on in your environment then you restrict the network authentication and for the moment you

would be done with what is possible so disable nlm V1 and LM if it still breaks your stuff in 20 23 then you have other problems than sitting here at a conference you should be looking for a new employer isolate admins from enm protected users cus on authentication policies red forest red Forest Microsoft confused people by uh by um uh declaring the EA um management infrastructure for deprecated if you're on premises it's still valid take care of services and service accounts use fqdn everywhere yes you can use IP addresses or host names in SPN but you actually really shouldn't you can put fqdns everywhere where a computer has to be addressed and then chances are your

application start using uh curas by all by themselves use UPN this is actually not necessary in most cases but it helps verify that spns that you would be using to request service tickets are actually in place SQL service accounts not granted rights to set their own SPN and cus authentication policies go ahead look into it if you have questions and you will uh hit me up on Twitter we can talk about that you'll have to get to know your environment to achieve this um do document those systems that you find not being cerous capable at the time and start pestering the vendors or your purchasing department uh about upgrading those but there are plenty of

systems in the typical environment that could absolutely use kerus but are not configured to use it exchange if you have an load balanced exchange organization chances are that kerberus um kerra um infrastructure is either not in place or outdated and they fall back to nlm SQL clusters are potential candidates for ntlm DFS if you if if if you use a standard wizard for uh for creating a dfsn uh uh Nam space chances are you end up using NM at some point Network printing Network printing is is a real problem uh where ntlm is concerned RDS connection Brokers the first Microsoft idea of uh having an RD RDS form could support curus the current infr structure cannot if you have a load

balanced RDS firm and least privilege for all systems so now last minute here are the logs you will be drowning in to monitor uh ntlm usage in your systems you would want to know what user authenticates VI inlm to what resources on what servers and what client application uh has caused this um uh authentication flow right to get to this information you actually need three sources they have been part of Windows since 2008 you could get client user and server not knowing which application accesses which resource by only monitoring the DCS these are the uh data fields that you will get from uh from your DC you could get a client an accessing user and the actual resource being

accessed by monitoring the server and if you want to have the client application you have to monitor the clients that's why I say you do not start monitoring the whole signal chain from the get-go but you start by monitoring the DC make sure that no uh highly privileged users come up in those logs after you done remediating and then you could start collecting server logs and in the end you could start collecting client logs that may provide you with further information and as last step of that Journey at least as as far as it's possible today you put all the servers that you cannot remediate today on the exclusion list restrict the ANM authentication in your domain and work

through the exclusion list together with your purchasing department Arc four get rid of Arc four and curus that's the best thing I could do uh for you help also helps with C roasting and when your exclusion list is ampty you're almost done and all that you have to do is wait for Microsoft to provide aurb and local KC and then you can have anlm free windows thank you very much

thank you very much of um you are perfectly on time don't worry at all that was great we will have a break now of half an hour our next