← All talks

Encrypting Your Infrastructure Without Getting Fired

BSides CDMX · 202522:5438 viewsPublished 2024-10Watch on YouTube ↗
Tags
About this talk
A practical guide to implementing full-disk encryption in enterprise environments using LUKS, Clevis, and Tang—open-source tools that enable network-based key escrow without traditional key recovery servers. The talk covers threat models from opportunistic theft to sophisticated physical attacks, demonstrates TPM limitations, and explores defense-in-depth strategies like Shamir secret sharing and quorum-based decryption.
Show transcript [en]

ESP so we're not going to do that uh so uh I'm happy to be here from Austin Texas and uh this is an awesome conference and I'm loving uh be uh Mexico City it's just great I'm talking today about uh tanging in cleis and encryp your infrastructure without getting fired because we don't want that that's not good and uh I'm uh I've got a whole bunch of certifications and whatnot I've been in the tech industry for more years than I care to admit see the gray hair and uh if you uh look there at the bottom there's a a cute little image that I grabbed wholesale off of Wikimedia Commons and uh that's says cleis there we're going to be

talking about a little bit about this uh open source project set of projects called tang and cavis among a bunch of other stuff today but uh that thing on the outside is a cleavus and in the middle there's the the block that's a thing called a tang and if we uh look at that thing there we've got a pin as well so we're going to get to all of that stuff in a bit uh but first of all uh a question so who here in the audience is interested not yet in cyber security infosec but is interested in entering infosec cyber security okay handful of hands all right next question who here in the audience has

been involved and employed in infosec cyber security and very closely related fields for at least five years uh Sino anos all right rookies keep those hands up uh rookies check look at these hands lunch is coming up so now you know who the bug and who's brains to pick so there you go um so yeah infoset careers we talked a little bit about that and also look the rookies out there I can't stress enough how important it is to get involved in the infoset community this conference that you're here at it's all volunteer run this is a labor of love yes we have a bunch of sponsors here at this conference that are that are

footing lots of the bills you paid a little bit there look most of this is being paid for by the sponsors including Delo thank you very much for hosting this uh but you know this kind of thing right here we've got does not happen with volunteers and and it's all about volunteers so look you want to get involved in this uh I've done a little digging here since I've been here and in the local area there's this thing called the no trust security Meetup no trust Meetup check that out you can check that out at meetup.com and look you really want to boot trap and get your career going in infos uh besides cdmx look if

you look at my shirt here it says besides Tampa these days I live in often Once Upon Time I lived in Tampa Florida and when I lived in Tampa Florida uh at one point I I ended up somehow on the board for Tampa uh besides Tampa rather uh it's more accurate to say that I was a Lackey for the actual smart people running bsides Tempa but that's beside the point you want to get involved you want really help out you volunteer for bsid cdmx next year and look the volunteers right now they're going nuts they have not got much sleep recently after this thing ends they're going to party tonight and they're not going to

want to think about this for about a week or or two or three something like that but go on BB side cdmx website and I've already looked there's a way to do contact you like set a note on your calendar in about three weeks from now you click that thing on bite cdmx and say next year I want to volunteer I promise you you reach out 11 months ahead of time they will find all sorts of nonsense for you to do so uh great way to get involved and look another thing here is uh in Austin uh I'm on the board I'm the vice president of the an organization called the information system security Association it's a

worldwide organization it's different from organizations if you're familiar with organizations like is square I'm also member is square or isaka uh Issa is totally independent of any certification not it's all about the broad aspects of information and Security in general and I just a little diing here apparently Mexico is one of the maybe the only Latin American country that does not have a a Issa chapter so hey you want to get involved in the community go start an Issa chapter here in Mexico City it's high time that somebody did highly recommend to go check out that organization it's great stuff so story time so uh let's say that uh you are involved with security for a set of

retailers a set of stores and these stores let's say stores are called their convenience stores let's say they're called uh I don't know let's say uh uh uh yeah okay work manager uh UMO do you know say that's the name of your your stores you know okay I'm trying to say a uh like 7-Eleven so you get your stores and uh your stores you've got your normal point of sale thing happening in the front of the cash register you got your posos but then you've also got in the store youve got some sort of inore infrastructure in the back you've got some sort of server you know it's probably some sort of Dell HP Desktop kind of thing but it's running

in the back and it's running some critical stuff and you've got it in some sort of cage it's somewhat secured and that's great so uh you know you've got that and then burglers go into one of your stores and uh it's nice that you've got that nice little cage protecting that piece of Hardware but they've got a crowbar so uh they grab the the the thing and you know they're just common thieves they're they're looking for whatever and they already got the gra grabb the cash register they're like oh bonus item computer and uh then they go sell it on wherever and and uh anyway your company gets all sorts of excited when the data that was on that ends up

on Black Market and while you're doing a pretty good job keeping all your credit cards and stuff off of that server you had some stuff in flight or whatever that that was there temporarily it's bad you've had data loss you've maybe lost personal uh customer information it's bad and and you're thinking well you know no store would ever do that sort of thing uh look I mean at this point uh uh in America Chick-fil-A is running kuber NES clusters in the back of their stores people are doing this stuff so uh it's bad so in short what if I told you that bad people want to steal stuff so what would happen if stuff you

had valuable stuff you had was stolen from your data center or uh your Cloud infrastructure or as we were talking about here your Ed Computing kubernetes cluster not be good Data Theft is very expensive now how expensive it is to your particular organization that really comes down to your particular organization I could sit there and and grab off the internet from Gartner and whatnot some statistics that claim how much money is lost in each particular uh Act of data theft you know I mean it really comes down to your particular circumstances and and what is going on for your business as to how much a particular data breach or rather a bit of data theft would be for your company

but uh it's generally bad it's something you at least want to mitigate how much you want to mitigate it well we'll talk about that in a little bit but you definitely want to do at least some sort of mitigation so you know in the case of our example with our our convenience store uh in the back you know if we had had data at rest en cryption on that server everything would be cool uh you know they they'd have grabbed the server and uh all they'd have is an encrypted hard drive and that's it uh you know Common thieves no big deal I mean data at rest encryption it protects you from common thieves what does data at rest

encryption not protect you from you know Common thieves crowbar what does it not protect you from uh it doesn't protect you from ransomware I mean it's great that your your data at rest is encrypted so you get nailed by ransomware and congratulations now you've encrypted the stuff that also has been encrypted by The Ransom rare on top of it oops not bad um so you know it certainly does not protect you from actual hacking Talent like people with serious computer hacking skills um you know just doesn't work so Linux root partition encryption in the case of we're talking about Chick-fil-A Chick-fil-A's got kubernetes clusters they're not running Windows they're running Linux so uh you've got

your Linux root partition and uh with the encryption on it you've got a Chicken and the Egg problem with disc encryption uh wellit look you've got this problem whether you got windows or Linux uh or whatever or Mac OS you you got your your data at risk uh you've got your encrypted data on your hard drive but at some point you're going to turn off power and then you turn on the power on your computer and uh somehow you have to do this decryption stuff uh and if you get wrong you've asked yourself uh oh you've crowd stried yourself if uh no offense to FKS running crowd strike but it's been in the press a whole bunch of

people have a whole bunch of down time right now because of that um and you know alternatively with this kind of thing hopefully you've got a backup pass phras sort of thing uh you could manually enter your pass phras that's great on a work laptop that you're handing an employee is an alternate means of decrep thing you know if they know what the P phrase is or whatever you could do that that's not okay when you've got store clerks who probably aren't allowed to know what PES to decrypt their inore server maybe the manager is allowed to know I don't know but it's just a bad situation so you're option would be like an SCC server and

there are ways to do this there you can look online uh there are various people that have demonstrated how to do this sort of shenanigan with say SSH uh and you've got your thing connected to an SSH server and that's actually holding the decryption key and and that's how during boot it connects to an SSH server uh it grabs the key and gets it back that's great I mean unless of course um you know that particular server that you're using there gets compromised that's bad you've also got this thing called TPM and we'll be talking about the TPM uh some more who here has actually played with TPM in production uh okay you guys are lucky consider

yourselves lucky there are no kids here right yeah okay it's down out across uh so um but but not all those Roses in this world you know okay we've got the TPM The Trusted privacy module we've got uh the The Trusted privacy module is theoretically uh tamper resistant so it's this ship that's uh on your system you know and it contains a whole bunch of stuff we'll get into it but in theory it's supposed to decb your hard drive the magic that's on this chip these days it's not necessarily on a discrete chip it might actually be part of your overall chip set on your system uh and and that's better when it's not on a

discreet ship but it's still hardly foolproof so a few weeks ago at uh my local meet up in Aus at my local de meet up in Austin uh the guy that runs it was doing what's clearly the beginnings of a preliminary talk he's going to do where he's demonstrating how in his his spare bedroom he's got not modern chips you know not the latest and greatest in nanometer you know and Manufacturing for chips these chips from a few years ago but he's demonstrating a proof of concept where he's he's got the chip and he's shaving off bits of the Chip And then he's got a laser beam that he is flicking at parts of the Chip And the

idea there's a lot of trial are doing this sort of thing but the idea is it eventually you hit the right spot it actually flicks the bit on the chip now I'm no you know expert on chips and secur okay I know some stuff about security but I'm no expert on all this sort of stuff but man you start flicking bits on chips like that you know you've got your quote trust and privacy module uh you could theoretically flick enough bits to convince the trust and privacy module that hey everything's okay give me everything it's cool so uh that's not good um so yeah lasers lasers are the future and and look I mean if this guy

is doing this at his back bedroom with some you know ulous technology the whole point is is that there are actual people with talent that can do this for real on modern technology and probably way more accurately uh so more practical uh variety of circumvention here I'm gonna actually show you a little bit of a video we're not doing audio forget that uh but I'm going to show you a little bit of a video for YouTube here in a second here and with this guyy going to demonstrate uh is he's bought some nonsense he's got a raspberry uh uh pie Pico and he put some code together and he's going to sit there and take this

thing and stick it on top of the discret TPM chip on a laptop and as the thing is booting up he's going he's his little piece of of Hardware that sticks on this Chip is going to lift off the decryption key is it send over the boss off the pins of that chip you know back to your system so that Windows bit Locker can decrypt your your uh your disc he's going to demonstrate that he's going to lift the decryption key off and folks once you've done that done that it's trivial at that point to yank out the the SSD or hard drive or whatever and sck it on the Linux computer or whatever and he's going to demonstrate all that

so here we're gonna look at this video very quickly uh and and I'm going to show you a link at this at the end of the uh uh presentation so here he's got the laptop takes off the cover here and just licky split he's got his Raspberry p Pico thing boom on top he's got the decryption key uh oh and my goodness that was 42 seconds there chose on his uh on his clock and he's going to talk some more about Locker anyway it's a cool video highly recommend you go check it out after it um so network based decryption I mean clearly the TPM is not all what's cracked up to be it's not

what you know Microsoft is claiming it is in terms of it's the the end will be all it's GNA cure all your problems it's G to cure cancer you know fres your breath all that kind of thing uh clearly there's more to it than that and uh I'm going to demonstrate and we're talk going to talk a little bit about this thing with network based disc decryption What If instead of just having like an escro server like I was mentioning before what if instead we can actually use the Network as one of the things we have uh in terms of authorizing who we are and proving who we are and we're actually using some cool cryptography

and talk about a little bit we're going to use this thing called uh The mallum relia Exchange and it is just a variation on uh DIY Helman and it's going to do decryption without uh escrow as I mentioned so little bit about Dy Helman who familiar with Dy Helman couple of hands so uh who here has gone on a website in the past few days yeah okay nobody's raising their hands I don't believe you anyway uh D Helman is whatow freaking everything on the internet every time you use htps you're using D helment and in short uh again something I grabbed off of I totally snagged off of uh Wikipedia gay Wikipedia uh in short what we've got here is both of us

have a both both parties have a p private public key pair and we're going to uh trade our public keys and we're going to do a whole bunch of math and uh at the end of the whole thing we end up establishing a shared secret using this the the public private ke pairs that we both have we establish a shared secret across the internet or whatever the inere network and and the shared secret uh uh never want uh had to travel across the internet that shared secret is used uh as an encryption key for an encrypted session Diffy Helman It's Magic and and it's what powers our world so that's normaly Helman this mum Raa thing is a

little bit different so you've got the cleavus client so that would be the thing sitting on your big Iron server maybe it's that uh dell or HP Desktop you've got sitting in the back of your convenience store uh but it's your quote server and then on the other hand you've got a Tang server now this thing is really lightweight I mean it's called a server technically it is but there's really nothing to it and uh so the the change server there's it public key uh then your big Iron server does some magic there the client um generates a new public private key pair and then comes the best part apparently the the the laser thing doesn't work on this

coose screen uh then comes the best part math happens but the best part here is at the bottom the client discards this private key because it never uses it again so again there's a variation on Dy hel not the same thing and uh so that's the first piece that's when initial trust is is set up on this particular step you know the initial setup uh anyone who's ever used SSH you initially connect to a server and it asks you hey this server's fingerprint is whatever it is it plops a whole bunch stuff with a cool little um uh set of characters and a pattern kind of a thing that would be the equivalent with SSH of establishing

trust and that's what we're doing here in this step The Next Step would be what happens every single time the server boots up so again you've got your bank iron server it generates a new public private key pair for a different purpose this time and we do some math uh the Tang server does a math and sends some stuff back and this is enough for the quote Big Iron server to reconstitute its encryption key for its boot volumes and presumably maybe the rest of the volumes on the system and what's Magic here is nowhere is the ention key story it's all well it's all in the world of math and we're talk and look if you

really want to geek out with the math I grabbed this presentation uh from Red Hat actually I know red hat is in the building maybe there's some red hat people here please don't Su me I totally RI this off your presentation uh but uh here's some cool L Tech uh a tech math uh that explains this in more detail and if you're really into discret math geek out with that really cool stuff so the Tang server I mentioned you know that's a very small thing we've got our big Iron server running cavus the Tang server really really lightweight thing it's a lightweight https server it uh can be run on something like a Raspberry Pi in this case this is a Raspberry Pi

zero uh it serves up Jose which is a variation on Jason Json rather with encryption and uh you know it'll do this whole mum rail thing so that Raspberry Pi zero I mean that's enough horsepower to do a whole uh um Data Center and then some by the way I'm G to say this at the beginning guys you know I I speak quickly and I'm American I try not to be an ugly American but uh at the end we're going to have a QA session we're going to have live Dem a little bit here too but uh if at some point you're com if you've got a question save it for the end place however with the exception if

you you're pretty sure that I'm saying something that a significant minority of folks in the audience don't understand please do shout and and let's clarify that so pleas please do anything that seems you know if there's language barrier please do shout out and uh let me know I'll be glad to cover that in the middle here so cleus so we talk about the D Ser cus is the thing that's running on your your big Iron server it uses a lux slot we'll come to what Lux is in a little bit here and you can uh still decrypt and manually decrypt things with a different Lux slot uh uh Lux is cool we're going to see how that

works a little bit as well so that's cevus we on Lux Lux is the Linux unified key setup who here has ever played with and is familiar with Lux on Linux oh come on somebody has to have played with Lux okay handful of hands so uh Lux is how the easy way to do encrypt on Linux and it's frankly the preferred way to do easy encryption and decryption so you've got one master key for a volume that a lux volume and then you've got a bunch of slots specifically you've got eight user slots with Lux and here here's an example of uh how this looks in short you know you've got your master Kitty and Slot one when you set up an initial

Lux volume is going to contain a pass phrase that you entered when you set up the Lux volume that pass phrase when you you enter the the pass phrase correctly that will uh uh decrypt the master decryption key which is then used to actually decrypt the the storage volume or the block volume in question and LU provides extension you could actually have multiple pass races if you want that sort of thing so if you want to I don't know why you want to do that maybe do in case you lose one of your past phrases or something but you know theoretically these past phrases should be very long they should be like at least sentence length they shouldn't be

ascendants that can be found in a book you know all that stuff so with CLE you set up CLE on the boot volume to automatically do decryption cleus will use one of those slots but if you notice slot one still contains your your pass rays that you can use to decrypt it so for some reason technically this don't work you've got a backup option to decrypt the volume in an emergency we'll talk about ways to avoid that kind of emergency happening in production but that's always nice to know that that's there for you so uh you've got your tank server and theoretically it could be on the other end of a VPN this might be a

good option if you've got in the example the convenience store um but in that case that also means that if your your VPN or your your network connectivity is down that also means that your store server can't come up so maybe that's not okay it could also be something that you hide remember the the tank server could be running on a Raspberry Pi zero if you really want to get crazy there's no reason why that Raspberry Pi zero couldn't actually be in your drywall without any way to get through it without busting your drywall you could hide this thing very well if you want to Brit Security on this Tang server thing it has a set of public and private

key Pairs and much as you do with your web server you probably periodically want to rotate those pairs now you can replace the pairs the the key pairs that are handed out for establishing new trust relationships but uh you want to hold on to the old Pairs and and tank server can handle having the old pairs stick around even though it's not handing them out anymore because you'll need those to decrypt any of the servers that establish trust with those and you better not be throwing out the old pairs until you're sure nobody's using it you know just cryptographic maintenance stuff all right now this is where I was going to do the live demo but we had

some problems uh trying to settle all this up so I'm actually going to do a uh we're gonna I don't know how many of you familiar with Bill O'Reilly but live later and uh yeah so uh we're going to try to do that cut over only once with the a uh because I I don't really trust it so anyway if we're gonna do that you'd see all of this awesomeness and you're just gonna have to imagine you saw it you're gonna see it at the end it's gonna be great so what this solution protects you from it protects you from common thieves as we've covered uh but you know it doesn't really protect you with uh from

sophisticated people in scills you know ultimately they have you and your data if the people with skills are the ones doing the burglary if it's not just a thug with a crowbar if it's someone with Talent from people at this conference with a crowbar uh oh so defense and depth strategies clus has the ability to require more than just one tang server you could do this thing called Shamir secret sharing which allows you to do really complicated or at least really cool cryptographic stuff where you could have or you know you could have both tpm2 and tank you do all sorts stuff with this uh you also have a quorum of Tang servers which is really really cool

so you could say you've got five T servers out there for some reason you could make it so that in order for the system to actually pull off decryption you need at least three of them so cool stuff but with great power comes great responsibility ultimately you know all of this is a business decision frankly all this is risk in of itself we're trying to mitigate risk and yet we're doing freaking risky things so uh I talked a whole lot of trans about TBM earlier and uh now I'm gonna un PP TPM a little bit this is I if you guys get this reference this is a thing especially in Halloween people throw rolls of toilet papers mild uh vandalism

is pretty Harless mostly uh so in untp UNT being uh TPM you've got TPM version 1.2 which was the first thing that was massively deployed and my goodness it was horrible it's still out there and you can actually some TPM modules you can downgrade them to 1.2 if you need to for some reason it's weird and like each implementation is wildly proprietary and bizarre uh tpm2 on the other hand is definitely better uh it's not nearly so bad but again it's it's still partly yeah so TPM you know you've got your bit lock bits your ufi bits and you've got your secure boot um it's small and uh it is a small computer that's sitting on

your motherboard well actually these days on your chipset and the apis for for TPM will interact with that very small computer that theoretically we should trust if you as an individual want to trust it I mean that involves a whole bunch of corporations that set up the TPM if you say go look at the SP from for TPM and you look at what's involved with the specs with TPM you might not trust it at all because you're just looking at this the spey spec documents and they are freaking huge and complicated but that said lots of people are looking at them maybe it's secure uh you know I'm not familiar with any horrible breaches with it maybe I'm

wrong about that uh the TBM though is very very slow but again who cares it's only used for for you know automatic decryption on boot stuff like with a a bit locker with windows so you can also store open SSL Keys most people don't realize this there's actually a module for an open SSL where you can store your keys on it and this is hand you say if your server is no breached uh well if you were using the DPM maybe they didn't get all your SSL keys if you're doing this kind of thing maybe there's value there you might want to think about doing that kind of thing uh it can validate tampering I mean PPM that's

kind of what it does with all secure Boot and UEFI uh and that that's what bit Locker is doing uh theoretically can detect that and this stuff also exists on Links at this point and again how good is this you know seems to be okay um and and but look I mean they're getting too deep into this TBM the way it works is it's doing a bunch of what it's called measurement and the measurement bits uh are you know doing a whole bunch of math and then the results of these individual bits of measurement are set to sent to the TBM module and it it's based on all of that but the TPM decides whether or not it should give up

the goods um whether you're using TPM or whether you're using Tang or cleus I mentioned earlier once you have physical access you're kind of screwed you know I'm going to do the demo of this but this is probably the right place to do it so at this point this is zy so I've got two raspberry pies I think we've got them back here so we've got our our simulated Pang server it's running on Raspberry Pi and then we've got our uh Big Iron server that's running cleis and I'm going to demonstrate uh right the video that hopefully you're going to see on the screen if this works okay uh the video is going to have the um Tang

server um rather the the cleus server your big Iron server uh that's what you're going to see on on bootup and here let's uh let's unplug this are you ready over okay so what we're looking at right here folk are we live what we're looking at here folks is uh this is what happens when you're you're not getting access to your Tang server it's a little zany back here I've got a keyboard I was going to demonstrate what happens if I type to enter all of the uh to enter the P phrase actually let's try it right here