CG - LibreSSL - Moving the Ecosystem Forward - Brent Cook

thanks for coming everybody thanks to be sized Las Vegas for for hosting me this is really excited this is my first talk here at b-sides and I think it's just thrilled to be here I've really enjoyed a lot of the talks have been going on this week and I've had a lot of fun with the Metasploit booth as well if you want check it out.you behind I have a chance to try out the Metasploit of all three CTF what you've got running get a shell within five minutes and you win a free t-shirt so on with the show so this talk is Libra cell pushing the ecosystem that the crux this talk is we're going to be

talking about Libra so cell I'm the affordable maintainer for Libra so cell for the OpenBSD project we're probably talking about not so much the project itself but some of the effects its had on different operating systems different sort of programs within the sort of the security tool ecosystem and the ways that things have changed over the last four years so this is kind of our agenda we're going to be talking a little bit about introductions libras to cell if you've never heard of it before I'm going to tell you a little bit about it talk about some of the things that I work on which is mostly around portability across multiple operating systems it's kind of an unseen unsung job but

it's I kind of enjoy it partly because I'm a glutton for punishment we're gonna talk about three different stories that sort of mentioned things that are difficult with in the portable realm and ways that we've addressed it with in Libra SSL and we're gonna be talking about ways that operating systems have you all to make things easier for us we're gonna be talking about our core random which is the interview generator format for Libra so cell I know entropy generation may not be the most interesting topic to a lot of people but it's a really fascinating topic especially when you're trying to be secure about it we're gonna talk about the ways different operating systems approach it in ways that we've

sort of worked around deficiencies over time we talk about environment variables which everyone I don't know this is one here like environment variables I know you do I know you do and so we're gonna talk about some things that can be bad about environment variables and ways that we've worked around them and some other projects that you may be interested in learning about so Larissa sell its four-year old Fork of openness to sell it came about right around heart bleed when heartbleed came out the OpenBSD team started auditing libras are open SSL and said oh there's a lot of bad coding here we've been just kind of blind taking that faith that this is well-maintained that this is secure

we're going to start ripping out all the obsolete broken code out of this system and we're going to build our own version called libro so sell a lot of stuffs been deleted in fact I think what is it boy Lika moves in a half million lines or so that we've deleted over time we've added a lot of new features as well we were one of the first SSL Forks to have crypto crypto support like Chacho 20 we were basically one of the first to come with a better api for open ssl as well we have a api we call lib TLS which is a simplified api that makes a lot easier for software developers to not shoot

themselves in the foot when they're writing secure software when the other kind of neat things about Lee versus Allen this is something we've evolved over time is it actually has full software compatibility with open SSL 101 102 and 1:1 so you can basically write software for any of those api's and it they all work with open SSL directly we work very much in lockstep with open BSD team every six months we put out a new major release we have about five or six people that work on the core pieces of Libre sell over time it kind of is a rotating crew one of the neat things about the open SSL or libres or self development is that anyone within the

OpenBSD project can contribute to it we don't really have sort of this huge separation between who's like an SSL developer and who's not actually generally within the OpenBSD project the last person to have touched a file gets to become the de-facto maintainer of that file so if you ever want to accidentally become the NFS maintainer commit some code to it and then you get to win that as well it also has some interesting features that that were added to Leber's to sell like privilege separation and isolation that's specific to open BSD like for instance it uses the the pledge command quite often to automatically lower privileges that are used within the project and we'll go

into that a little bit later so one of the things that we approached Woodleigh bracel specially when we started the original fork was we noticed that open SSL doesn't trust operating systems very much it influenced own memory allocator which is one of the reasons why Hartley was exploitable with an open BSD because it basically a circumvented a lot of the memory protections built into open BSD so one of the things that we did first without Libre cell was we got rid of a lot of the kind of duplicate implementations of things like memory allocators or things that don't trust the operating systems to do something right in places where the documentation was wrong we fixed that as

well in fact a ton of work has gone into Libre cells man pages I know that doesn't sound like super sexy either but I think Libre cell has some of the bests OpenSSL man pages around and if you just take a look at the number of commits that go into the main pages it's pretty incredible we've also tried to remove all the broken api's and all the unsafe api's this is sometimes a give and take because you might find that within the software ecosystem that a certain API has used a lot because OpenBSD is an operating system into itself it has about five six thousand ports that it needs to work with and this is the base

SSL layer for it so generally we we find out really quickly if we remove a bad API and someone's actually using it a lot of times we actually work with those upstream maintainer and fix their software upstream so that they're not using a bad API as opposed to just continuing to let the status quo move on so really one of our goals isn't just to make levers to sell a good project but to actually make the entire ecosystem of a better project by working with upstream maintainer x' working with other software and making them better as at the same time it's a big effort it's a huge team effort so how did I get involved with this I worked at a company

called boundary and I worked on our agent we were owed an agent and see that basically monitored network traffic and sent it all up to the cloud to be processed and all the software that we had is part of our agent you could configure it and build it like this you've ever built software on Linux you know configure make install it's easy right except when you build open SSL you had to create these huge monstrosity zuv patches and huge like crap to turn off all the broken stuff and you know her no heart beats no engine no Kerberos no SSL v2 it's basically they kept so much backwards compatibility that they just kind of paralyzed and never moved

forward so I didn't like this and it didn't happen to have separate built steps for everything so really my interesting her cell was really just to make it easier to build software it wasn't I wasn't really into the security part initially later on I got into that but effectively that's the reason why I got more less recruited by the OpenBSD team to work on this project right when libre cell came out originally they said it's only gonna be out for open BSD we don't care about any other operating system and so it's kind of a dare I said well I wonder if I could make it work for this boundary agent you know and thanks I sat

down for a couple weeks and I wrote you know I like Auto conf and not to make I don't know if anyone else who likes those things but they're not too bad you a little bit of em for scripting and you know you're off to the races are sort of and so anyway I was on vacation with my family in Japan and Theo de Raadt starts emailing me and says hey you want to work with us and make a portable version of libros a cell and my family was all jet-lagged and you know they were all bouncing around so I just would go down to this cafe the Sakura cafe and iki brocro and and I would just hack all

night when we were in Tokyo and I worked on you know basically talked with those guys and we ended up you know deciding to meet in Slovenia in a couple months so basically I'm just talking to these random people on the internet and they're like hey come to Slovenia and half with us and I've never been to Europe before so I just you know decided to go and if people know I'm usually not a very adventurous person I don't do things just on a whim but in this case I went out there and I met like 70 or so developers we've hung out in like a co-ed hostel and you know did a bunch of

hacking with with both Bavarians so it was a lot of fun we got our first like five or six releases out that week I learned how to use CVS which I don't know I hadn't used that like since college or so but we got it all hacked up pushed it out and then we got our first well okay first of all I just you know talking a little bit about Libre cell again we support a lot of different platforms we support Windows in like four or five different incarnations my ax Mac and West FreeBSD hp-ux Linux netbsd openbsd Solaris we actually support some weird stuff like sorting sand MIDI picks which is also another Windows layer

we had three different build systems we have the OpenBSD source tree which is how normally gets built Auto conf and we even support see make even guys ever you see make before it's a pretty cool project I really like ninja especially doing ninja builds everyone the open B team is really jealous when I can build Lee versus L in like 25 seconds and everything else takes you know five minutes this is kind of gonna circle back to that first week where we sat down in ljubljana and got those first patches out those first releases of libres to sell out and that is the idea that making things a pile is easy making things work correctly is really hard and basically

how can we make a portable version of Lieber's to sell that doesn't drive us crazy doesn't require a bunch of F destined defines how do we make it so that we don't have to have a ton of patches all over the place to make this thing work one of the approaches that we took to make this stuff happen is we actually have this concept of I guess you'd call them intermediate headers within labor so sell what it basically does is whenever labor sell try student is anyone here a C programmer just just to kind of start all right so not going over too many people's heads on the whole include things but the idea here

is that they're our definitions of the way functions look within the operating system in these headers and what you'll find is that a lot of times these vary from operating system operating systems sometimes there's a function miss seeing sometimes the definition is a little bit weird sometimes there's a whole subsystem that's missing from an OS whether it's Windows or Linux or whatever and so you have to fix it and so how do you fix it either you write in your C code a bunch of little you know conditionals like if it's Windows do this this is not Windows do that or you can actually intercept all of the includes that happen within the built system and and include your own

version that automatically fixes all these system calls for you so like for instance whenever we would do time related functions within liebherr SSL includes timed @h but it actually includes our own specially modified version of time 2h which has all the missing pieces already incorporated into it and then we have this this kind of interesting code here I'm going to point it out so what this basically does is it uses an interesting little function that you may have never seen before or a directive called include next and on every compiler except for visual C include next will automatically include the next header file in the path so it basically allows you to create an interceptor that grabs that that sits

between the reel header and your the header that you want to modify and okay lets you mutate the effect of including that header and of course these other little things are kind of a weird hack of a trick so if you include sort of a relative path you can trick Visual Studio into thinking that's actually a different header altogether and and get around the limitations there but they don't have include next so that's really how we do it we we basically rather than coding around like say windows missing F open or F open not working properly we actually define our own version of POSIX F F when that acts like POSIX and then redefines what F open is inside of

windows code to look like the POSIX F open this basically means that the core code of larissa cell has know if def windows in it and that's kind of a goal of ours is to make it easier for developers to develop code without having problems where one section is has one behavior another second is a different behavior and just comes again this week spaghettified mess so that's one way that we've been working to get the operating system to sort of be the same ideally there's nothing in there at all another way that we have managed to make portable Libre cell work well within different operating systems as we do sort of runtime detection of operating system attributes for instance

a key function that we use within libres a cell for doing entropy generation is a function called arc for random now arc for random is supposed to be a very secure random number generator and really easy to use it doesn't even have a failure mode there's no way for it to fail there's no return value you can give that isn't supposed to be completely random and unpredictable now occasionally people make mistakes and implement it badly and so and that's the case then we actually have code within libros a cell that can detect like for instance if you're running netbsd before version seven arc for random was broken because they you know actually used a copy from open base different like ten

years ago and then updated it now eventually we worked with the net BST team and updated it so that they did have a working version of arc for random so there's actually code within Li versus Zelda automatically patch in and out arc for random depending on if the underlying operating system is sufficiently good another thing that we do is we continually monitor upstream code it's really nice that a lot of people are open source so we can actually see if things change over time and in fact as part of kind of our libres itself porting plan we've seen a lot of operating systems adopt things that we've sort of been pushing and it's been kind of rewarding to see that

happen in reality we talk about three different stories of this kind of ecosystem push happening in the real world first one is around clearing memory and keeping secrets so when you have a crypto library oftentimes it's very important that things like the secrecy you decrypt don't hang around memory normally the operating system will have like a free commanded let's you free the memory back to the operating system itself but that doesn't necessarily mean that goes away someone else could reallocate that memory and get that secret right back so you want to make sure that you clear things securely and it's actually surprisingly hard so in 2014 libres to sell when we started working we found a lot of code like this inside

of open OpenSSL basically calling memset now mmm said is a c function that's supposed to clear out memory however the compiler can be really smart about it according to the standard the cloud can say if no one actually ever reads the memory then why bother clearing it at all so it connected detect if it's like for instance if buffer never gets read again after it gets written to then if it can actually optimize that code out and a lot of galata compilers will do that as soon as you internally in the optimization so how do you how you stop the compiler from doing it well compilers are going to compile so sometimes what yet to do is you have to

trick them and to make sure that they're there they're often optimizer can't tell that you didn't read it later so you know for OpenBSD 5.5 we implemented a function called explicit b0 which calls be 0 but calls it with one level of indirection so basically the underlying function can't be seen so it doesn't know if you redditor or not how it turns out this is defeatable easily by something called link time optimization where you basically after you compile the code when you glue it all together with the linker the linker can go oh well then no one really did ever read this code after all or read this memory so it optimizes out remem set there are explicitly zero

so how do you fix that well net bsd tried to fix it by adding a bunch of void star pointers and making the mem set function itself sim that could be replaced this is interesting in that one it makes it so that you can actually replace mem set with any function you want so you have a nice sort of a hook for all the secrets within your program another neat thing about this is that it doesn't always work either so back to the drawing board something else that we tried actually here I'm just talking about this you basically the biggest problem with this is it creates a nice little rock gadget that you can use within all your other

code something else we tried was was this so basically creating a weak function what a weak function is if we were basically giving a hint to the linker that this function could be replaced at any time at runtime with another function so it basically prevents the link time optimizer from being able to do its work and prevents it from from working however there's still an arms race that you have to do it turns out that the latest versions of GCC and LLVM and all that kind of stuff will still look to see if if if this works or not and will well potentially replace it with with their own versions so one of the things that we did to

continue sort of pushing the arms race forward is try to convince everyone first of all to this is the right way to do it but also creating a way to see go forward a way to actually test it let's see if we can go to the place where we're testing it mmm I didn't clue slide I'm testing it but the way that we actually ended up having to test this and so like how do you know that the compiler optimized something out or not without actually building the code and testing it in the first place the way that we figured this out would was we wrote something that used signal handlers and created an alternate stack

and would change the memory in the alternate stack and then what signal signal handler and check to see if the stack changed under the covers and then if that happened then we know new if the compiler optimized the code out or not one way that we work around the the compilers themselves being extra optimized is we actually have a separate library that we link into Libre cell that has all the optimizations disabled if you do that that for now convinces most compilers that you shouldn't optimize this this code out something else to kind of to note is that at least as a 2016 we didn't have any explicit be zero or explicit memset inside of G

Lipsy G Lib C 2.24 did eventually adopt this they ended up doing their isolation in a completely different way because there's a little bit of not invented here syndrome amongst a lot of different operating systems but at least they sort of adopted the API and sort of accept responsibility for for implementing it now there's still problems with this there's a recent use next paper about debts to our elimination still being a problem within a lot of secure software so strangely enough this is still a problem in 2018 with different pieces of software and different compilers something to note here is if you read the current Linux manual for it it actually still says explicit B 0 can

decrease your security if the compiler is determined enough and decides that you're all your data stores in a register then you can't optimize it out so it's kind of what those damned if you do damned if you don't kind of things something funny to note is that if you do some windows it works perfectly and has for like a decade because they had a secure zero memory function for a long time they recognized this as a problem and so when we added M supposed to be 0 forwarding at the windows was a piece of cake and you'll find this is actually a continuing theme in the windows thing so arc for random entropy is hard but why

does it have to be so hard so prior to Libre so existing OpenBSD or openness elf did a lot of weird stuff to create random numbers it would do a little bit of stack and do a little bit of PID it would try to get things from good places and you know dev random and stuff like that and it failed it would always fail kind of crappy so there was a lot of problems where there was just a lot of extra complexity and the entropy generator and OpenSSL in fact that's kind of a continuing trend actually recently rewrote their entropy generator and it's just as complicated and just as prone for failure something we did in Libra as

a cell we actually simplified it down to this fact our entire PRNG system inside a liver cell fits on this slide we took out the ability to seed it we took out the ability to add entropy and said darn it we're gonna trust the operating system to do the right thing and always give us random data so we just call our four random buff whenever we need random bytes now you might ask well what if it does fail well Ark for random is defined as not being able to fail so if it can't generate random numbers it actually will crash your program there's no other alternative to it so we'll talk a little bit about the

primitives that go with that so in 2014 when I was doing my intrusion Allah port my first thought was well suit I need an ark for rent on the works a lot different operating systems where am I gonna get one well there was a there's a program called leap Lib BSD you might have seen it before it's actually still shipped with a lot of operating systems and I didn't bother looking at how it worked so anyway I incorporated it and assumed that the up screen developer had to implement arc for random well but then someone wrote a blog about my port and told me how insecure it was and how terrible was and then said basically you

know this guy doesn't know what he's doing and so I wasn't looking at it and yeah it calls get time of day and get PID and mixes it together and that's not great because it creates very predictable entropy States almost Debian level but not quite and so anyway it was nice to be able read a blog post to be semi internet famous for a little while but we went and fixed it so take two well I found the code from OpenSSH and I thought well those guys know what they're doing right I'll use their code but turns out they used open SSL random number generator so it's kind of a chicken and egg problem so screw it I'm

gonna use this guy this thing called Lib Audrey that looks really good and complicated I don't know but if you look up liberati you can look up their github page it basically says that like your eyes will bleed and monkeys will come down from this guy if you use this for anything they actually need security so you know what do I do so anyway this is actually kind of cool the OpenBSD guys actually came to me and said hey we'll help you figure this out we will build hooks into our random number generator that allow you to hook into OS intrinsics and write your own portable versions in fact we even got the OpenBSD team to write Windows code

if you look in the OpenBSD tree under lip live crypto there's actually versions of arc for random for like six different operating systems in the OpenBSD tree that and those are explicitly there for anyone to use within their code or within you know their own portable libraries and we put that there intentionally hoping that people would notice it and would say oh here's how we can generate entropy random numbers well and in kind of a predictable and non fallible way whether people actually pick those up or not is an open question but you know we tried our best one of the things that our random does it's also important is that every time you fork a process it

automatically clears out of state that way if you if to give a child Forks from a parent you don't end up cloning the crypto state and you don't end up like safe you were both generating an RSA key or something like that you don't end up with the same key simultaneously so one of the things open beastie has is that you can allocate memory that never is inherited by your children right it's just a simple in map flap and in that flab in map flag and and so that's how they do it but it turns out that other operating systems don't have a reliable way to do that another thing the open BC added was a

call called get entropy and this is a call through the operating system again a call that cannot fail and it returns up to 128 bits of entropy and that's supposedly all you need to proceed a random number generator we gave it out there again to hopefully other operating systems would implement it and unfortunately there really wasn't a great way to do it in any operating system please not in 2014 so some of the things that we did to implement ones that would work well you see this good entropy Linux Tutsi there's a lot of sausage in there or I guess sausage-making going on in there under the covers what it does is if it basically can't get random numbers

from Debu random which is supposed to be secure on Linux it tries deprecated syscall if that doesn't work it falls back to the most crazy well it's really a best effort it's better than pids and get time of day because it actually looks at like you're a SLR it looks at things that are pushed onto the stack it looks at the entropy that comes from your memory allocator it looks at a lot of different stuff but in the process of generating this you know you know better fall back it's you can see multiple orders of magnitude slower so you don't want to hit this if you don't have to but we started working with Ted so and got

something like get entropy added to the Linux kernel but unfortunately they decided to implement something a little bit different than get entropy normally open BSD requires that you not ask for more than 256 bytes of entropy at a time and the whole reason for that is so you don't end up with the exhaustion of your entropy generator over time it took a while for others to adopt it and about the only OS that didn't quite just adopt it exactly like we specified in open BSD was Linux and that's because a lot of people decided that they needed to create Monte Carlo simulations and they needed you know lots of weird little tweaks and what if I need more 256 bytes

I don't know we need to make it generalize for everybody so intimate ended up adding something called get random does anyone here know get random no ok Adam knows get random so anyway a committee got involved a bunch of people started making requests and we ended up with something called get random get random is ok but it has a lot of failure modes and we get a lot more ways to exercise our fallback code in that particular case if you follow like say Python or Ruby or a few different languages that end up being used in like systems not in social systems programming but like early in the boot process you might have noticed that

there's been some bugs that have come up with get random in early entropy generation and that sort of stuff just because it can fail or it cannot fail and it might hang might not hang and so there's a lot of issues with it however turns out that windows had a built-in way to do this all along and so you call grip gen random and you get a perfect random vision generator at least one that we're currently trusting the OS to to provide it for us like I said before we have eight different implementations of get entropy again they're all in the OpenBSD tree you can use them I believe freebsd linux net bsd OSX and

solaris have all implemented since then their own versions of get entropy and so that basically means that the lis rest of cell code does not ever call any of these fallback functions if those other versions are available and that's actually pretty cool because that means that now these kind of secure easy to use crypto primitives are now available in a lot of operating systems partly because we push libros to sell out into the ecosystem in 2018 where's arc for random now a lot of people think Oh Lee versus L all you guys need is get get entropy to work well actually I don't want get entropy I want our current which is really the the user space API

that you want for generating random numbers we still have some blacklists OSX at least 10.12 still had a are to 911 had a non-complying arc for random they had some problems with reseeding freebsd i thought worked very recently but I tried it again recently and it still fails all the fork tests that beastie 700 Solaris a lot of them now actually provide are for random which is really great I like the fact that now that's actually available unfortunately and I include that yeah we tried to get this get arc for random specified as an actual standard and because a lot of people objected the fact that it had the word arc 4 in it which implied that you

should use art for it doesn't actually use that algorithm anymore a lot of folks kind of balked at it just based on the name unfortunately guys stuck in committee and so it sort of sits there for the next last four years and and it's not really moving forward so hopefully it'll just become an ad hoc standard and everyone will implement it here's that here's a here's a one of the peer and G bugs that recently came up and on the mailing lists with regards to get intricate random or what was it called get random yeah with it with regards to the Lennox PRNG having early initialization issues now all hope is not lost there's actually a GFC patch

that's current for that's sitting in the patch queue for implementing our for random maybe about twenty years after we've inflated and OpenBSD it'll show up but we'll see if you've ever implemented Linux from scratch it does include it as a patch by default which is very nice so one final story of ways that we sort of moved the ecosystem forward is around environment variables so if you ever used openness to sell in anger before you might have noticed that there's a lot of environment variables that changes behavior it's not really a great idea for libraries to be tweakable by the user based on environment variables so there's a lot of issues with different OS vendors like Red Hat

patches they're open SSL to be able to respond to Red Hat specific environment variables there's a lot of problems with with processes that raise privileges whether or not they look at their own environment variables or they forget ones that are that are unsafe is it kind of an issue especially if I like a set UID binary the user controls so under the covers open SSL uses a kind of an ad hoc version of a function call is set you GID basically that the the idea behind this function is if a process is ever escalated privileges is that you GID should be set so basically if you've ever escalated the route maybe you could come back down to a regular user if you

have an Escalade double process it needs to always have is that you GID set because that will mean that you should not trust any of your environment variables basically that that trust has been breached what we found is that this function is broken on a lot of operating systems most of the BSD is a Mac OS implement is a Cisco almost everyone else implemented as kind of an ad-hoc Lipsy function which had a lot of actual time of check time abuse functions you look at this function here this is a simplified version that was an open SSL basically check to see if my user ID is my effective user ID and my group ID is

my effective group ID but you can actually be changing both of these things while you're running this test so it can actually retails open which is not a great thing so one of the first things we did was like how can we implement this in a way that actually is 100% reliable for instance on linux GFC had this thing called get ox file which which you may know there's three different sets of arguments get passed to a program there's the program's name of the the argument lists like our Gavi there's also something there's something called the auxiliary vector and what this is is the list of operating specific arguments that get passed to any sort of elf binary that have things

like a random number that seats my memory allocator or what is the base vector of where i need to like start loading libraries one of them has is a tea secure which basically says is this a set UID a binary or not so that seemed like a really good alternative to not having a syscall however it turns out that get ox Val when it fails also returns zero so you can't tell digit of failure and actually having like a privilege escalation or not so eventually they fix that in G let's see two so you actually have error checking at this point but you ended up with code like this which also wasn't that great and wasn't necessarily the right

approach there's also ng ellipses something called secure get a and B which actually it turns out is insecure a side from its name because it actually has the same time of chat time abuse issues that we just talked about a second ago that we're in the underlying open SSL library so we'll go down a little bit deeper and we'll check to see if it's even possible to override it so it turns out you can because it also sets an internal variable you can actually just reference it and change it on the fly so you can actually tweak your programs to be up and down and and and change where this Lib C enables secure function or

variable is so fun stuff other operating systems during the process of doing the porting actually found a bug in FreeBSD s implementation and that ATS secure always lied and the reason why they did this actually was to speed up threads P threads in their linux emulation layer it took them only two years to fix the bug but they did eventually and there's a security adviser so it's really the only free/busy security prize ever happened in my name but I'm pretty excited it turns out like five other people also reported it so glad they fixed it so it turns out because there were so many bugs in it set Juji ID for different operating systems Mac OS doesn't work

pass the fork Solaris didn't work before 11.1 hp-ux was actually perfect which was kind of weird but i'm happy for those guys i'm a except so many failure modes yeah like it's basically unix written by aliens so they have so many different ways to like escalate privileges and like it's very not Unix II so I just didn't want to get into it so basically because they're also broken we implemented is that you JD is yeah everything is always unsafe never trust environment variables so we set that waited for a year waited for someone to scream at us no one did and we deleted all the environment variable usage out of the entire library so so that's good that we

actually use this approach quite a bit now this one was kind of an accident because I had actually had a bug in my cafe Kerr script where it automatically assumed everyone was broken but because no one screamed I was happy about that so some other interesting things that we've done as far as sort of moving the the community forward is this is kind of a controversial decision when we first pad out libres to sell we called the open SSL version version 2 and initially that was just kind of a funny joke and that we were saying oh they'll never fit out version 2 so we'll go ahead and claim it for ourselves and they'll have

to jump to three even if they ever get to two we estimated it would take them 25 more years to get to that point so it wasn't gonna be a real problem but it turns out that a lot of software under the covers uses open SSL version to determine the features are available like rather than saying do I have this function it says oh is open a cell version greater than one oh that too dot B or something like that and so this caused a lot of random software to fail over time to fail to compile now mostly this was like the job of our porting team to fix this with with patches but hope our real hope was

that software developers would stop using up selves SSL version as a signal and what instead actually checked see like as a function implemented is it not implemented but no one ever did and in fact to this very day we continued to push patches up to upstream projects nginx is actually a current one that we're working on right now where they just trust open SSL version to mean something and it doesn't necessarily mean anything at all something else that we've been working on a lot lately has been open SSL 1.1 api does anyone here actually code against open SSL and kind of know the differences between 100 and 1.1 so the big differences between the two are for

open SSL 1.1 they decide to make all the c structs opaque and what that means is that you can't access them directly from within your C code services if I have a an SSL context or something like that and I look inside to get like the version I can't look at it anymore and instead have to use an access or function to get access to it now you might ask well why did the open SSL team decided to change this from I can get access trucks to not access trucks why I actually found out I went to the International cryptographic module conference earlier this year and I asked rich salts why they actually did this

and it turns out four-fifths compliance so because with its compliance you don't want to have to change your binaries you don't want to have to change your API for for people's perform clients testing when you have opaque when you have open structure like this every time you want to add in them remember it changes your ABI by hiding it like this it means that they can actually push out new 50 releases or that haven't been the whole thing recertified they can just recertified one object so that's actually the underlying reason it was for Phipps compliance now this wasn't we were advertised that much but I was interesting seeing all the really really distraught government cryptographer

cryptographic software writers worrying about the openness l1 at one jump so for for libres or so we don't care about that stuff well will full Fitz I mean partly because it's a lot of work and not a whole lot of gain from our point of view but the other thing is we don't care that much about a BIS and api's well I said they're wrong we care a lot they care a lot about not breaking people's old api's so we try to keep every old API available that's why I Libra sell actually works with 101 code all the way to 1:1 code because we actually incorporated all the api's at the same time so code libras to sell basically

supports all the API simultaneously now another reason why the OpenSSL team decided to make opaque structs is because every time they change the size of their structs are added in a new element or something like that all code had to be recompiled that's because they never changed the version number in their libraries if you have open assault 101 it's always called open SL 101 even if they add a function or remove a function so that causes problems when you have to tell everyone to recompile li vs l actually bumps the library version number every release by at least four or five because our philosophy is every time you change the API you should bump the version number

so while Lee versus L is currently at version 2.8 I put out the release notes for that today the underlying library version is like 1.57 or something like that so we're like major version 57 because we bump version numbers but the neat thing is you can actually have all the versions of Libra so installed simultaneously so you don't have to recompile your code you can actually keep everything pointing to the old version the new version things like that I've worked with a lot of Linux distributors and other folks and said hey why don't you just incorporate some way for your package manager to install multiple versions of library no one wants to do it I don't understand why

but yeah that's that's the breaks there are some brave Linux distributions that do incorporate Li versus l and when we put on your release they just recompile the software but you know what updating 100 packages on your machine isn't there a bigger deal anymore maybe back in the modem valid days it was but think about like your normal like Debian distribution or you know following Ubuntu what do you have to do to install like 4 or 500 packages just after the main install so yeah you know update update another thing that we've been working on a lot is software configuration management so we've tried to make it so that it's easy to incorporate Lieber's to sell with different software a lot of

works been going into the sea make sub system we get a lot of feedback we finally have a lot of Windows users I don't understand why but a lot of folks use it and one of the main reasons is actually because we were producing an open Acela exe which is very useful turns out some people said if you put that in the Windows Store I would buy it so we may have a new revenue source for leapers to sell in the future as well just just compiled some meeseeks so that's really the gist of it I have lots of more stories to maybe talk about if you guys want to sort you know you know

ask me after the talk or something like that I also have a lot of different other projects that we can also talk about a little bit offline Lib TLS which is our simplified abstraction layer on top of open SSL it basically makes it really bulletproof to to get a secure application running within see we also have go bindings we have Python bindings and I forgot a couple of other languages but those are the main ones we have a net cat so you guys like net cat you guys are all hackers right so so actually Libre cell shifts with net cat but it's a special netcat that knows how to do TLS as well now you might have

used what is what is the one that comes with the Wireshark is it called in cat yeah and in that cat I forget anyway so similar to that if you compile it with like enable net cat will actually install it for you so if you want to use the cool enabled netcat we have to use it for all of our testing because it actually has every feature enabled so you can enable OCSP States stapling within that cat if you want you can do all kinds of crazy stuff we have a whole bunch of other cool stuff - we have an acne client which like just lets you do blessing crypt with the see it uses

Lieber's to sell under the covers we have a special open vs DHCP server I've got a portable version if anyone's interested in it that lets you basically it's just a simple HTTP server but the idea here is that it the main goal is it supports really good privilege separation so it actually like one process that's in a sandbox that can't do anything but conserve files another one that can really just bind to a port and then like a third one that handles all the configuration management real ad which is the gateway load balancer same kind of thing and I'm also the maintainer for open and TPP well you might wonder why an interview client would be here the entropy client

actually doesn't trust NTP servers by default it can actually ask random TLS servers if they also collaborate that the time roughly matches what the NTP servers are saying so open ntpd by default confer instance talk to Google look at Google's headers over TLS and see what time it says from that point of view and see if your netcat if your NTP servers are lying to you or not and will automatically desync if it turns out that they can't reach a consensus and one of the biggest problems with that reasons for that is because NTP itself is not really a secure protocol and there's been a lot of work on like a secure version of NTP but it's never

really been successful yet so anyway that's my talk thanks for coming guys and let me know if you have any questions [Applause] thank you quick question on filing when you run out of entropy crashing so if I don't want my program to crash do I have to rap I can't remember what function with R and bytes to have to wrap it with like said long jump or what are my options so the interesting thing about this function and of course I open-source you can check it out is it actually only ever seeds itself at the very beginning so whenever you first start your program it will either seed or it will fail and basically after that

first set of entropy it's now stored within the state of art for random and it uses chacha 20 I believe to actually produce the rest of the byte stream that ends up being used for the real random number generation so it'll crash right at the beginning or no that's right you either completely work or you completely fail there's no in-between like I might fail at a day or two or like that do you think there's some cases where blocking could actually end up getting the entry pool filled up yeah absolutely so our I'm get entropy is supposed to block it needs to block until it gets the data that it needs or if it can't it

can't can't be succeeded then it will fail inside of open BSD it's not a problem because open BSD always reseats its random error generator from the bootloader into the kernel and whenever every boots it saves the last state at the random number generator in a file that the bootloader knows how to read on the next reboot so it's basically kind of a cyclic issue in Linux it's not so much the case I will block until it's got enough entropy from I don't know network disk activity stuff like that but this is really a problem and a lot of like cloud environments where everyone boots and there's like no entropy coming in to the systems and so

in that case currently Libre cells implementation will fall back to that kind of crappy 67 per second you know fall back which under the covers is really still asking Linux for entropy anyway so it's kind of it turns out it works maybe due to a quirk in Linux what we really like is for and other references the influent get entropy to have some way to reliably see their intrument generator on boot I don't think anyone's really solved that problem one of the one of the hesitations at least I've heard from the likes community is because the kernels the boot loaders the ellipses the init RDS all those kind of things are all maintained by different people no one

really wants to work together to come up with the solution so that's the crux of the problem right there hi thanks for the talk and did you leave SSL guys do anything with the asn.1 code yeah one of the first things we've been doing is we have unrolled all the macros so one of the biggest problems is that like the C code is all meta programmed and it's kind of crazy and that you can't tell what's implemented where because you have to follow four or five different macro levels just just to read it so actually joel singh actually unrolled all the macros in all the asn.1 code to just just to make it readable in

the first place in addition we've been doing a lot of work to make a bit more memory safe we've removed a lot of the allocations and deallocations from that code we've had a few foibles along the way we tried to simplify things sometimes and have failed especially with regards to like certificate validations date time is another issue that's interesting in that when we're looking at the different versions of dates you know 32-bit 64-bit dates and that sort of thing we didn't want to add 32 bit compatibility code to libres to sell just to have again the we don't like F deafs all over the place and we really wanted to sort of try to get

everyone on 64-bit time T's and right now Libre cell has some workarounds for 32-bit code but they're very minimal and we throw a big warning whenever we compile on a 32-bit system that doesn't have 32-bit time T that hey this will stop working in 2030 ish and you better fix your system before that time high so this is less about like libra asses out portability but like TLS 1.3 just got ratified yeah and like libri SSL compared to like open SSL and boring SSL kind of tends to be more conservative with implementing drafts of that's right implementations I think that's great but now that it's been ratified you know and OpenSSL has it or in their

beta and how what does that timeline look like for Libra asses okay are you are you waiting for opening cell to get out of beta 1.1.1 or like are you trying to match compatibility with OpenSSL we're just kind of waiting to see what happens I would say like a lot of open source software a lot of that's written to scratch a personal itch so a lot of the folks that develop Libra SSL do use it and like say some business capacity may they build like routers and stuff like that on it but no one internally really has like that it's that they need to scratch for for building open SSL or I'm sorry I sell one or TLS one two one

three compatibility but we are having a hackathon next I know March February or something like that and maybe we'll get together and they're gonna own you that is zero round-trip time you know you got to get it correctly yeah I guess it kind of depends if Sun wants to sponsor this and maybe let me take a six-month sabbatical or something like that yeah well I don't know Kaylee gave me a weird look anyway yeah so so yeah everyone kind of depends occasionally we do sponsor people to just tack on things and fix it up it's it's in their own possibility

for your for your random number seating have you ever worked with the hardware random number generators so the random number standard random number seating trust the operating system to do the right thing if your operating system uses say Intel's you know random number generation code or arms or anyone elses we're going to trust the operating system to do the right thing I forget if open BSD itself actually includes it but it certainly doesn't solely rely on it into its entropy generation I would say that most folks that use like say cpu generations they still always whiten it with some sort of other source and don't trust it as the sole source make it so that you know when some random thing

comes out about it just doesn't work or maybe maliciously unintentionally whatever we're not sort of have all our goodies in one basket but effectively that's kind of for the implementer to decide and that involves something we're trying to be opinionated about in Libra so um we did run a Tantra P when doing it was kind of a cloud-based solution running on a hypervisor we ran to a couple things we ran into older hardware versions of the VMS where the things that linux used to generate and Trippi weren't available during the virtual chipset yeah so that was a bummer and then we've learned that if there's a lot of programs executing because of what you pointed out earlier that they

use random numbers for loading the program's you could also run outta entropy so it's kind of important to keep an eye on that in your program right right and what I think I think one of the advantages of the arc for a random approach is that one you'll never see a individual process use more than two hundred six bytes of entropy because that's all it really it only grabs 128 bytes to seed in the first place for our for random does it once and it's done so your program shouldn't run out now obviously there's the forking issue which could be an another problem and there's not really a good way to get around that right now I think it's

important that the program is blocked rather than you know failing open but I just made one other thing to note about arc for Rand is that if you never call it you'll never it never gets seated either there's actually a mutex around it and it checks to see if it's been seated once before or not and if it never gets called then it never becomes a problem any more questions who's got a couple more minutes left nut oh we're late relate to some of these questions who would you trust from an audit perspective is there an organization that says we looked at these 32 random number generators then we we generated a tree and random numbers from each one

and verified which one seemed more truly random is there a source that you would feel trust able on that idea well I'd say that a lot of folks trust Daniel Bernstein for a lot of this kind of stuff he's sort of our you know our oh I don't know what the word is for someone that everyone trusts but anyway that's one of the reasons why we use Chacho 24-under underlying byte stream as far as the rest of the trust ability I think the answer is don't trust anyone in particular just trust a lot of people at the same time and hopefully the bad ones get canceled out by the good ones that's all I can only suggest at the moment

what do you got time for one more question I was for the previous persons um if you're working in certain fields you will be told who to trust by your customer yeah that's a good point you they they have a there if you want to talk to them all right well thanks everybody [Applause]