Charting the SSH Multiverse

Hey everybody, come on in. Grab a seat. Put move into the middle. Make a friend. I am John D, your ring master or master of ceremonies for the afternoon. Soon we will have our first speaker of the afternoon. As you can see up here in threetory tall slides. Oh my god, this is so cool. In the IMAX theater, HD Moore. In fact, let's go ahead and bring him down now. I think he's ready to talk. give you guys an excellent trip through the SSH multiverse. With no further ado, HD Moore, introduce yourself. Please speak right into the mic. You'll be fine. Thank you, sir. Cool. Thanks, John. Hey, I can I can hear. Um, wow, this theater's huge. I

feel like an ant, but uh, thank you so much for coming. Um, this is my first time being back since 2011. I was here for the first Bides and then the second one, and then I took 10 years off from public speaking, and I'm finally getting back to it again. Uh, thank you very much and hopefully I'm not too rusty here, but I'll do my best. Um, so just quick intro. My name is HD Moore, co-founder, CEO of Run Zero. Uh, most probably most people probably know my work from Metas-Ploit that I did for about 15 years. Uh, so today I'm going to talk about charting the SH multiverse. Uh, the idea is that there's

a lot of SSH out there and until about 2023, SH was pretty boring and then last year everything exploded and this year it's uh, continuing to get lots of exciting new vulnerabilities and exposures in SSH implementations. So if you haven't been living under a rock, SSH is the second most common admin protocol in the whole world. It is more popular than RDP. It is more popular than anything else except for web interfaces basically. So if you look at everything from like showing graphs and so on is either a web interface or it's SSH and then there's everything else way below that. Uh so it's the most common uh admin interface. It's not a web UI.

It's enabled by default in pretty much everything. It's part of every major OS. It's embedded in every type of thing and IoT device and server and even mobile. So SH is pretty important and we want to talk a little bit today about kind of the shape of the ecosystem, some of the exposures, some of the research been happening both, you know, recently and then even more recently and then some of our recent findings. So first off, um, a lot of folks think like, oh, SSH, if you have public key authentication, well, you're good. There's not a whole lot you need to worry about. But there's a lot of stuff that happens in SH protocol that happens before you even get to the

authentication stage. Um, so even before you get the banner, you have the TCP IP, window size, scaling factor, etc. that stuff immediately tells immediately tell you the kernel version of the system. Now if you take the server version like opensh whatever and you combine that with the MSS scaling value from the TCB um IP traffic you can actually narrow down exactly what distribution it is pretty quickly. So even with an unadorned SSH ban open stage 7.4 you can generally nail it down to exactly one of a few distributions with maybe basically one packet in a single connection. Um now once you get into the server version it gets a lot of fun. Some dros like

Abuntu, Debian, they actually put the specific package version into the banner as well. So you know whether it's patch or not because it's either running the the right package or it's not right. So you actually get really granular security information from that banner as well on the distributions that put the package name into the version. Now as you start going forward in the protocol, you start doing handshakes, ciphers, kitmax, key exchange, things like that. You start getting a lot of like really interesting combinations. You start doing really detailed fingerprinting about the configuration of that device. Now you know, okay, it's not just a real device. It's a real image running this thing from this one vendor because of

the combination of ciphers that are enabled. Then you get what's called the pre-authentication banner. Now, it's not the same as the SH banner you typically see. It's actually kind of like a message of the day type thing presented to the screen. And you probably haven't seen a lot of interesting ones if you're just a normal Unix admin. Um, but if you start looking into the world of IoT and embedded devices, you find some crazy stuff. So, for example, um, anyone here, show hands, know what this little logo is? It's probably hopefully not too hard to read this threetory thing. Ah, okay. At least one person. Anyone else know what that logo? too. Awesome. That is an

Axon body camera on the internet. Uh, and it tells you exactly what it is because even before you authenticate to SSH, it says, "Hey, I'm an Axon body camera. Here's my MAC address. Here's all this information about me." Um, so it's amazing how much information you actually get out a lot of devices pre-authentication with SSH. Then lastly, even in the authentication phase itself ends up being really leaky. So, if you're using something like the Duo authenticator, uh, you can actually figure out whose phone number is using the authenticator just by starting the initiation of the authentication flow. and it'll actually tell you, oh, look for the passcode or the the text message to this ending phone number. Right? So,

if you need to confirm that, okay, I think this um server belonged to this person or company. That's a great way to confirm that is just to start the authentication process, be like, yep, that's exactly the the link, the URL, the whatever uh that I expected to see. So, the idea here is that you actually get a lot of useful data out of out of SH even before you authenticate. Uh another really fun u uh quirk of SSH is that when you authenticate through a public key, uh there's two stages. First you say, "Will you take this key?" And the server is like, "Yeah." Then you said, "Great. I'm now going to sign my session ID with that key to prove I own

that key." But that first part you can do without actually having the private key of the said key. So if you have a public key from somebody else, like let's say Gotan's public key, you can then ask a server whether it accepts Geotan's public key even if you don't have his private key or her private key. Right? So the idea is that um if you're trying to figure out whether a device is admined by a bad actor, as long as you get a copy of their pub key, you can now find out every server that they have access to and the internet is small enough now you can actually mass scan the whole thing in about four hours of

shamble plus Zmap, right? So when the XC back door thing happened last year, we went off and scanned the whole internet trying to find every machine that geotans pub keys from GitHub would actually let you log into. We found thousands and they're all false positive. So that's actually what led to building this whole toolkit which does a really good job of like randomizing keys, inserting random ones, things like that to make sure that false pauses are actually kind of narrowed down as part of the process. Um, but since then uh a lot of folks will uh well there's a couple other campaigns we helped out with a frontman was tracking one of the Mariah botn nets and the actor in that

case was pushing a hard-coded pub key to all the machines that are popping. So we went and looked across the whole internet for what other machines accepted that key but not on the Marai targets but on their home network basically basically finding their home development machines. So you can do a lot of really neat ops stuff. You have someone's public key and where do you get their public key from? You get it from GitHub, right? You go to anyone's username on GitHub and end keys in the end of the URL and you have their public key. So um same thing with like launchpad.net. Uh you can find fir public keys in firmware, malware, things like that. Um so the funny thing about

all this is like it wasn't really intended to be this way. There's a comment in the open state source code saying is this actually okay? Should we be doing this? But it's just part of the spec at this point and it's kind of too late to fix it now. I'll just head up the clock is not running down here but I'll try to keep an eye on time. Um so SH itself when you get into the protocol itself it is horrendously complicated. It's basically just a big state machine though. So as you start going through every state of the protocol you say first your version now my version now key exchange in it now new keys handshake and now we

actually do the service request then we do service accept then we do authentication. Uh you're just kind of going back and forth through the whole protocol. What's really neat about SSH is that all the different stages of the protocol are incrementing message ids. So if you're doing something at the very beginning in the clear text phase, those messages all are always going to be within a certain window like message one through message 30. As you start getting into the pre-authentication but post encryption, those message IDs go through a different range of IDs. And then post authentication, you now start getting messages above 80 and then going 80 all the way to 250 depending on vendor

extensions. So theoretically you can limit the state transitions by saying if it's above this number during the protocol exchange, we don't need to, you know, we can't we can't accept a message beyond this state. And that's really important because people screw it up a lot. Um, lastly, SSH really is more than just a admin tool. It's actually a secure transport and a secure authentication layer. So, keep in mind, just like TLS, your encryption actually happens before you authenticate. So, with SSH, you're doing a diffy helmet, handshake, and then you're actually um, you know, agreeing on a key, and all that fun stuff. Then you're authenticating over that channel, often clear text authentication of that channel for

passwords. Uh, public keys, doing like proof signing and stuff like that. Um now the neat thing about this is it's similar to TLS but it's also different in a really important way which is TLS. What happens is both sides negotiate and agree on the strongest uh algorithm shared by both sides. In the case of SH however the server must pick the first preferred cipher from the client that's acceptable and that's actually what enabled the terapin attack last year is that someone was able to the the attackers were able to or the researchers were able to basically truncate the packet so that only the cipher they wanted was chosen instead of the one that the client actually

preferred to be chosen. Um and lastly, once you're actually authenticated in SH, now you can do a million things. SH is not just a shell, right? You can do concurrent shells, you can have master control sockets, you can do uh file transfers, you can do TCB porting, unix socket, agent for 40, X11 display for all in the same protocol all in the same session all at the same time. So there's a ton of cool things you can do in SSH once that initial control channel is created. So when you look at like the scope of how people admin things in the internet and how you actually secure transports, SH is effectively another version of TLS in a lot of ways. it it

actually fits the same kind of niche if you will for creating that secure handshake but it's very different in the sense key management is not CA based for the most part while you can use CAS to manage keys they're generally not global CAS they tend to be enterprise specific CAS instead so uh TLS is kind of like a you know if you wanted to call it a uh a a dodgy version of TLS where you hardcode all your thumbs it's not too far off from the truth so let's go into the ecosystem a little bit so we all know that open is the big thing right but where did open sage come from well go all the way back to 1995 and Tatu

Yolan created SH version one as like freeware. U he continued development which eventually became the proprietary version of SH which is now tectia. Uh but then Bjorn and Granville forked Yoland into OSSH and then OpenC forked OSH and OpenSSH and then the PKIX SSH guys decided they didn't like everybody else and they created their own version um with Blackjack and Hookers or at least with X509 inserts and then Dropair created their own separate tree way after after the fact. So you have a couple different lines but the majority of the um opensh and sh we love and use all originally came from tat's codebase and it's been modified since then. Now if you go look at the internet like what

is actually exposed on the internet? Well it's almost all opensh like vast vast majority like more than uh 90% something like that is just opensh based shas on the internet. Um and then the second most common of course is drop air bear which is common in embedded devices. But the thing is like even though it's the most common implementation it's not necessarily the most important in a lot of cases. there's a lot of excuse me very important equipment that runs obscure weird hand-coded version of SSH that are horribly broken and that's really what I want to focus on today. It's like what's all the other stuff out there. So yeah there's fun things you can do with

opensh great research but how do we shell everything else for fun. So there's lots of cool stuff you have like all your ICT gear your firewall networking storage stuff and you also have all your application specific SH demons. Um, so move it has its own FTP, sorry, SH engine that licensed from a third party. IP works. Globalcape has it own SSH engine. There's all these kind of like one-off vendor implementations. Garrett, for example, usina um that are very different from anything else out there and they've got their own kind of quirks. So, don't want to go into too much depth here, but effectively uh there's some standalone versions of SSH that are not libraries that are kind of

like their own thing like Wolf SH, Cisco IOS, LSH, PKIX, etc. Then you have a whole bucket of SH libraries that are used by to embed SH into applications themselves. So you've got lib SSH which is part of everything. You've got the Go X cryptosh wrappers things like wish and things like that around it. And Java, you've got Mina, Python, Paramiko, Erlang, they've got an SH module that's been pretty popular lately. Um but if you start looking to just opensh itself has been fragmented a million different ways. So if you look at um you just diff the code for Apple's version of opensh versus like stock bog standard open bsd opensh p1 right portable um there's

almost no changes in Apple they did a really good job of only changing a very few things to enable things like the pks helper the event logging endpoint logging things like that and just didn't muck with anything else which is fantastic so when you're getting opensh on a Mac it's actually not terrible and it's good to use now you start going further down look at debian look at ibuntu you're looking at like 30 to 50 patches depending on the distribution on top of basic SSH like these made these lib xc utils back door compatible because they link to systemd and from there link to xz and normally ssh does not do that. So when you're getting

opensh on things like debunching red hat and so on you're getting a very different version of it. You're getting a whole bunch of extra code and different behaviors that aren't actually part of the standard open source or opensh implementation. Uh, and then PKIX is like kind of like uh, you know, uh, Wonkyville. Basically, they really hate the Open Sage developers as far as I can tell. And so instead of copying their changes line for line, they like write the logic slightly different just to be spiteful. And you can feel the hate emanating from that code over the years by doing the diff. You can see like, dang it, I need to copy this change, but

I don't want to look like I copy the change. I'm going to do it this way instead. Now, the funniest part about this is like, so I went in diff like every implementation for like eight months last year and have been keeping up on it since. and PKIX often times they get the logic wrong when they copy the change. So you'll see a changes like if not this do that and then you see PKIX do the exact inverse of that like that wasn't correct. So far none of those have turned into security vulnerabilities yet but like I probably wouldn't use PKX at this point given that it's all spite and wrongness. Um now the one that's the far worse of

course is Microsoft Windows. Now Windows it was really exciting finally shipped OpenSH by default with the platform right and that was that was cool and exciting. Uh the downside though is they didn't ship good SSH, they shipped their SSH and their SSH was opensh that they'd like monkeyed around with. So we presented on it last year, but effectively they backdoor the most important crypto function in the entire codebase by making it carriage return line feed compatible for a crypto comparison. They made it no longer timing safe. They made it a one bite out of bound memory read. Like they managed to screw up literally the most important three lines in the entire codebase. Lines that had been written by Damen

Miller 20 years ago, no one had touched since. They went and mucked with it so it could parse log files differently. It was amazing. So, uh, I don't have strong opinions about uh, well, I probably do, but I don't I wouldn't recommend using OpenSt on Windows as is, put it that way. Not only does it have a lot of different code, they they ripped out all the safe part of it. They pretty much ripped out most of the secure part of it, too. Uh, but even worse, it also sends telemetry back to Microsoft of the client versions that connect to your server and vice versa. So when telemetry is enabled in SH which is on by default

and someone connects to your SH server on Windows, it's going to take the client version which is whatever Mac OS or Linux whatever you're using and send that straight off to the cloud and say thanks. So Microsoft sees people saging into your servers by default with telemetry which seems pretty awful. So try not to use SH built into Windows by default. So last year was probably the most exciting year in OpenSh history since the CRC32 exploits and X2.C and all that fun stuff back in the day. Um and there's a great reasons for that. So first off, um Fabian Bomber at Rurer University Bochum came up with a really neat way to break uh channel integrity

by basically truncating the packets in the client side abusing the fact that if you can do some protocol layer stuff, you can actually make it use a different cipher than expected and downgrade the protocol and fantastic work and that became the Terapin attack. Um then of course there's Gotan, our favorite internet contributor, right? Uh so Gotan was um a persona who was probably likely the product of a state actor. They built a nobody but us back door and that the back door could only be used by used by somebody who had the private key that corresponded with the public key in the back door. So a nearly perfect backd dooror. Um they targeted the systemd

patches that are in ibuntu uh Debian um Red Hat Fedora etc. but didn't actually target the normal open stage builds kind of you know as it was. So it wouldn't have to Mac OS for example. Um and they did a fantastic job of going through and getting their code all the way to shipping basically and getting it out there. And it was only caught last minute by Andreas Frund. And you probably saw a lot about this last year, but uh it made it basically into the rolling releases and it got caught right before it went out to millions of people. So, uh part of this work last year is we decided to go figure out like

can we go find Gotan's home server by t taking their public key and then scanning the internet for it and that kind of led to this work and the answer is no. They've got really good opsec. Um so they're they are well funded and managed to not basically leak their identity even after a three-year campaign which is impressive. Uh the other fantastic work last year is by the qual threat research unit. uh these folks came up with an amazing way to abuse uh a regression in a previous signal re-entrance uh uh fix basically that leads to straight up rce pre-authentication in opensh like one of the very few in the history of the project uh it's really tough to exploit

due to the timing so theoretically it takes about 6 to 8 hours to get a shell on a device running a 32-bit version of oneu and then some unknown numbers of days or weeks or months to pop a 64-bit machine given the ASLR component and then solar designer found a very similar version that only affected Red Hat builds because Red Hat made their own patches different which had its own basically uh issue in that sense and that one was a little bit less bad because it's limited to the priv user. But the the most incredible thing about the qualice bug is it actually was remote root code execution in it default configuration of opensh. Uh that's just something that hasn't

really happened in 20 years. So another one that's really fun that I love last year is the Watchtower guys. If you haven't read their blog post they're like a thousand pages long and they're hilarious and they're just like trashing a vendor continuously. It's amazing. So, I love their love their blogs. Um, there's a company called Move It for getting a lot of people ransomware. Uh, they also make a file transfer product. Uh, and they have a they had the weirdest bug. Uh, they had a bug where if you put a file name into the bytes of your public key during the handshake, the server would then read that as a file and then authenticate against it somehow. And

it's like, how do you even get there? Right? So, not only could you use a UNCC path to like make your SH server turn around and connect to whatever path you wanted to and then give them whatever contents of the key, but they were um the uh uh Watchtower folks were able to turn that into full-blown rce by having it parse a log file that had a key embedded into it in a funky format and things like that. But the really messing the messed up part about this whole thing is the vulnerability wasn't even in move it. It was actually in a third party IP component called IP works. So, a license a library licensed by lots of

other products. So, we never heard about all the other vulnerabilities and all those other products. We only heard about the movement one. But man, those are probably still vulnerable. Anybody who stopped paying their SDK fees, you know, last year is probably still shipping a vulnerable version SH server for their Windows products. That kind of takes us to where we were last year. So, we created this tool called Shamble after we realized how messy everything was. And we went through and just basically threw it out the whole internet for fun and just watched it rain shells out. And we went and figured out some of the bugs and found ways to make that better. And then

in this research, I'll talk about some of the improvements we made to and some of the newer stuff we found. But the idea is that we wanted a tool that um spoke the SH protocol really poorly in really bad ways and that your normal tooling couldn't really accept. So until about I don't know like five six years ago probably when parameo got uh much better um it was really difficult to write tools that spoke that Sage protocol without writing C and probably creating your own vulnerabilities in your client side code trying to do these things right. Um a lot of my friends would work on like oh I'm going to exploit this S sh basically accidentally self-exloiting

themselves because it's so finicky to get it right. So, it's great having um these great libraries like you know, Python, Go, etc. that support the low-level protocol side. But some of these libraries weren't quite good enough yet. They really couldn't get you low enough for the protocol to find the types of bugs that we were finding. So, we built kind of a not just a custom, you know, SH enumeration tool, but basically a custom version of SH protocol stack that allows you to at any given state of the protocol say, how about I just get a shell now? How about now? How about now? How about now? And eventually just start dumping things out. So, maybe halfway through the

authentication process, let's ask it if we can do a port forward. now halfway through um or maybe pre-authentications, ask it just to open a shell, things like that. Uh we also found some neat ways to do timing analysis different platforms. We had to turbo brute force public keys. So you can grind through millions of keys per second on a couple machines just by um spraying public keys out them because they don't actually limit them. So lots of fun work, but um I'm going to talk more about the new stuff here. So with Shamble last year, we found about 110,000 just open shells on the internet by just throwing it the whole thing. About 27 million devices with SH open.

Uh this is mostly IPv4 only. about 14 million would negotiate a full SA session and then 110,000 were just straight up shells one way or another when it's all said and done. Um so about about nine different unique vulnerabilities popped out of it. Um basically ruckus APS it was an older version of the OS but easy way to get a shell remote route. Uh Digi Transport ICS gateways just straight unauthenticated remote uh Panasonic Ethernet switches are still vulnerable as far as I know. So if you have one of those, throw it in the dumpster. Um, and realtech ADSL gateways have never been able to get a response from anybody over there, but like every net bit and like

there's this huge pile of equipment that all built on this real tech SDK. They're all vulnerable still remote route, but just by default, and there's thousands of them on the internet and no one cares. So um, that's nice. Uh, separately, there's some neat tools that are based on the Go SH library like soft serve and goss that we also found some RC bugs or authentication bypass bugs in. Uh and we actually got scooped by one of them, the Goggs bug. Uh another company, Sonar Source, reported it like two weeks before we did. So they managed to scoop us on the disclosure there. And then we also started uh digging into like OpenSh for Windows and a couple

other products. And this is where we found like the out of out of bands memory leak and the weird back door in Windows OpenSH. And then where for ion network service access points, just kind of an oddball kind of legacy device, they'll actually do do port forwarding out of the box even before you log in, which is fun. So you just like create a session to them and start just proxying through it and start hitting everything behind the device, the network and uh eventually it'll time you out saying you didn't log in in time. Like great well I've already owned your whole back of network so it doesn't matter now. Um and another fun bug that popped out of this

was the unlimited public key testing. So taking the entire uh GitHub key database for example of 117 million users finding someone's like drop air server and is jamming all 117 million keys through that server to figure out who actually owns that box. So if your public key and your username is is available somewhere, you can find out someone else can find out whether you've got access to a particular machine and as far as know drop never responded to emails and that does doesn't care. So you can still do that. Um so going into more recent research so uh the call folks again dumped out yet another cool bug and this time they found a way to do man attacks

or machine in the middle attacks against open sage clients and they did that by actually causing SH client to run out of memory in a very like tricky way and that caused a particular error condition to hit and so on. Um, they also found a pre-authentication generalized service that caused out of memory in the server side by stacking ping messages in the protocol. So, you think these like this code's been around for a very long time and yet we're still finding really cool bugs in it. Uh, but only because the Qual team's looking apparently they're the only ones capable of finding cool bugs like this. It seems like them and solar designer. Um, the Go stack

actually had another interesting bug where the default GoC Xcryptosh stack uh would call the authentication handler every time a public key is received. that includes every tested key, not the full key. So you could take a big list of keys and each time you send it to the server, it would call this function, cache it, and then later on when you did your real authentication, sometimes it would use the wrong key in the wrong state. And you can basically authenticate as a different user entirely by sending their pub key, but then authenticating with your private key with a different key entirely. Um, so this is a fun one and actually mostly the only real commercial case I know

where of application effective was a netapp telegraph agent. uh and is partially mitigated through o an OS update or sorry a library update but you can still shoot yourself in the foot if you do it the wrong way. Uh another great project that has really taken off last year is Hanobach has this project called bad keys.info is anyone aware of it? Okay, a couple awesome. It is amazing. Like he's like hasn't really been like you know marketing the hell out of it but is it's really cool. Includes like an entire uh scanning toolkit for SSH, TLS, DKIM, etc. It can negotiate SMTP. It can basically pull keys out of almost anything and does really cool both

cryptographic analysis of the key for weaknesses for all kinds of fun attacks like format. And then it also has a giant database he's created of known bad keys that have leaked. This includes like all the old bad keys repos like that we built at RAP 7. Uh some other like SEC list repos things like that. And then more recently he took the entire fordet public dump that got lead to the dark web scraped all the SH keys and private keys out of the dumped Fordet configuration files put those into bad keys. So if you want to know whether your Fortnite devices SH keys got leaked, you can now just query the back the the bad keys blob list and find

out and the whole thing is super efficient. It's like a 200meg blob when you download it and it's all done through binary search and kind of a like partial hashes and things like that. So really cool project and we're making use of it now in shambble as well. Um but the more fun bug recently was the Erlang OTP SH bug. Now this is great because like so I'm not a huge fan of AI for the most part. I think it's kind of a waste of time. Like I don't need to talk to my spell checker that often. Um, but the way that uh a lot of folks use AI is kind of funny. They're like, "Hey, we

are able to like get AI to generate an exploit for the Erlang OTP bug." It's like, "Guys, the exploit was in the unit test that was committed in the fix. It's right there. It's literally the same." So, the this particular bug is funny because it's a state machine bypass where if you send an exec command prior to authenticating, Erlang would just run arbitrary Erlang code. So, it's remote code execution in the Erlang sense. Like, you don't even have to like create a binary payload. You don't have to do anything tricky. You just say like, "Hey, run some Erlang buddy." It's like, okay, it's great. Um, so what was really weird about this though is like, why

didn't we find it? Like, we tested this exact scenario a million times. You could not figure out why we missed this, right? Um, so we started digging into it. And the funny thing about this particular bug is, uh, a lot of folks were exploiting it before the diffy helman handshake. So their exploits were actually being sent clear text after the key exchange, the ke in it, but before the key exchange. So ends up we missed it because uh RSH stack was waiting was waiting for a reply to the channel open request before it would send the exec request. And in this case the earring code does not send a reply at all in that case. It basically just doesn't

send anything. But if you happen to send a channel open then you send an exec. It just runs the code and it's completely blind. It's like almost like a blind SQL I it's a blind code exe code exec in this case. So it's tough to test for because there's a lot of devices that have the exact same behavior but don't actually just randomly run your code. Like all the Cisco IOS stuff for example works the same way. If you send an exec a channel open and an exec you don't get a reply but it's not hopefully running your code. So the real world impact is like nobody really uses Erlang SHD as it is in the wild. There's like three of

them in a given day on showden. There's like 42 over the course of 3 months. So it's not really important in that sense unless you look at the products that embed it. In this case the net comp server of most recent Cisco installs all the Cisco IOS XC stuff is all based on Erling and there's a lot of those. Um, so there's about three or four thousand internet that have 8 port 830 open or 2022 or 2024. Typically not port 22. That's usually the different stage implementation. Uh, and of course in this case Cisco said there's no patch till May. So good luck guys. Um, hope you figure out how to turn it off. But

if you'd like to patch your own uh, conf servers, it's actually really easy. You take the the exploit off internets and instead of telling it to run arbitrary OS command or whatever, you just tell it sh stop and will actually kill the sh demon entirely. So you can quickly self- patch your own overlaying boxes just by telling it to stop that h team which is great. So cool. Uh so we did a big update to Shambble. It's all live right now. So if you go to github.com runzering shambble with two s's you can get the latest stuff. We added automatic bad keys info block list lookup. So as you're doing scans it'll say yeah it's a known bad

key or not. It'll tell you whether it's got like a bad crypto key or it's using the old debop random generator things like that. Um we added a bunch of new off bypass methods including ones related to the Erlang ones. It's off by default because it's a little bit buggy. Uh we add a much wider algorithm host key support. The next big update it's going to add all the postquantum stuff. But we're almost almost there. I'm just trying to get all the go tip changes merged in. And uh yeah, the experimental blind exact stuff is there if you want to enable the checks for it. But the nice thing is you can take shamble and

it's a binary that runs in any. You can just throw it at any network you want and it just randomly range shells out which is fun. So over the last few months we've been getting random emails from folks saying, "Hey, I just ran this thing and like this random quantum table library gave me a shell." I'm like cool. Is there a known CV? No. Okay, whatever. like just there's so much garbage out there that sometimes it's help just have a you know a vacuum cleaner for extra shells out there effectively. Um what's weird about this work is that you generally don't want to do all this type of scanning against open sation drop air because we already know what those bugs

are like those implementations are pretty solid. So we added a a new option called skip versions and you give it a reax of like you know open sh pipeline pipe drop error etc and it'll skip all the really like time inensive checks on those platforms and just do the really interesting checks on the the lesser less common sensation implementations. So this means going from you know hitting entire 108 in your internal network in like four hours to about 30 seconds because there's very few devices out there that are these oddball weird ones and they're the ones that have the most likelihood of returning a shell. So uh we did kind of a a re well I did a

rescan last night. So, if all your alarms went off last night, it's because I ran ran shame against everything on the internet again. Um, what's different from last year is interesting. We went from 27 million devices with port 22 to about 22 million. But most of the ones that are no longer online are all honeypotss and tarp in tarpets. So, there's actually more real S8 servers online than they were last year, but less total port 22, which is interesting. So, up to 15.4 million. And of the 110,000 random shells we got last year, I got 48,000 last night. So, we're making progress. not quite there yet, but it's it's half what it was before. Um, so this is

excluding like the normal missing patch like openization old things like that. Um, a lot of these bugs, the top three ones are effectively use of hardcoded to reuse host keys, different kinds of authentication bypass, and of course pre-authentication port forwarding are all super common. Um, so here's a list of the checks that currently in shambble some of the new ones down here are like the vault exec skip off the bad keys block list. Uh, shambble's super easy to run. You just it's go you just compile it, drop it wherever you want to, whatever else. tell it to go shambble scan whatever it spits out a big old JSON file and you can just grab shells

out of it. You can also tell it to do everything from like give it a list of passwords, give a list of users, telling it to give it a giant list of SH keys to go hunt against and it'll actually try to like try all those off keys in all your machines. So really use case for like your blue team like let's say you had a SH key for a public key you use for a bunch of automation and you got rid of it and you want to make sure all your servers no longer accept that public key but you don't actually want to put the private key in something and try it. Well, you can give a shamble

just the public key side of it. tell it to scan your internal network and it'll say, "Oh yeah, you're actually still using these public key over here." So, it's got some pretty practical purposes for just internal security. You just give it all the keys that should no longer be used in your environment, whether it's a user that's no longer there, whether it's a hard-coded service key, and you can quickly get a list of all the devices that still accept that key. Uh, and the neat thing about Shamble is it actually let you do a very specific interaction. So, you can say, I want to scan this network and when you get a shell, drop me into interactive

shell and automatically run pty set l environment ldbug all and then run the shell command. And you can give it whatever commands you want, right? So you can tell it to run, set crazy environment variables, all kinds of things that are really pain in the butt to do through open directly. The interactive shell is neat because you can do TCP port forwarding. You can send arbitrary signals to the backend process. You can make it seg which is a lot of fun. There's this great service that when you uh pre pre-authentication, it gives you a login shell, but you can tell at that point you can set it a signal and if you send it the signal

segv, it goes ah I died. It's great. It has the funniest error message ever. So you can also send arbitrary strings to it. If you want to like command, you want to just stuff random garbage into authentication prompts, you can do that through the send command with a bunch of hexedit stuff. But yeah, it's a lot of fun. Happy scanning. Um, and one tiny little bit and then we'll go to questions here. So, uh, bonus. So, part of writing shamble is we also worked on a library called Xcrypto, which merges the old Zcrypto work from the Zmap guys with Go Tip and the basically the the dev version of Go and a bunch of Shambble tweaks. And this gives you a

single library that can speak TLS, sorry, SL3 all the way through TLS 1.3. and then very soon all the postquantum crypto stuff all part of the same library. So if you look at how other tools do it like nuclei for example and nuclei supports three totally different TLS methods and uses all three independently as part of it scanning because it tries to cover these you know backwards and forwards compatibility including shelling out the open cell command line directly. We don't want to do that. That's crappy. So instead we're going to put it all in one big library and let you just call it and it'll do the right thing depending what you do. What's also cool about this it uses the

um permissive ASN parser that came from the Zmap side. So if you have a certificate that's absolute garbage, but you still need to parse it and do things with it. A lot of parsers will just blow up, say, I can't read this thing. In our case, we'll we'll parse it anyways and then let you pull the data you want out of it, even though it's like a broken ASN format. And you can find a lot of really hard to find fields in that data as a result. So you can find that project at X Crypto. We're really hoping to start PR request to other major security projects to start using this thing because it's a much nicer way to

do it than shelling out to three different libraries. And thank you very much. All right, let's hear it for HD.