Hacking $100K Gas Chromatograph Without Hardware Vera Mens (Vulnerability Researcher@Claroty/Team82)

Sorry.

Hello everybody. Can you hear me? I hope so. very excited to be here at besides Brazil. I'm very fond of besides did some presentation in other besides events. So very excited to be here as well. I'm Vermen a vulnerability researcher at team82 and today we are going to talk about a very cool research that I did and I hope you will enjoy the talk at least as much I enjoyed doing doing this research. So today uh title is how we hacked 100,000 cars from autograph and without having it and how you can do it too. So let's start. Uh prior to talking about the research, I would like to introduce myself. H I'm Vermens. I'm a vulnerability researcher

at team82 by clarity team. uh you're probably familiar with the team and if you are not uh we are the teams that uh performing a lot of vulnerability research on OT and IoT devices and our goal is to find vulnerabilities and uh disclose them to the vendors so they can could be patched. uh and uh after the uh vendors patched uh the vulnerable uh vulnerabilities we are uh going to uh talk and uh talk about them in a presentation like this one or publishing the blog about it. Uh so we would like to uh the community to erh uh to be better at security and uh be more aware of the vulnerabilities. Of course our

goal is also to uh disclose those vulnerabilities and uh make the make them patched. Um as uh for me I mostly do embedded devices is because this is what I most enjoy doing and me and my team also uh participate in ponton competition which is a very cool hacking competition you are if you are not aware about it you are welcome to read. So let's start about uh the research but prior we need to uh get some context uh for the device we are going to talk about today. So we are going to talk about gas chromatography. We are not uh going to uh get deep dive into what how it how it work. It's not our business

here, but we want to just understand what it does. And it mainly its goal is to take a mixture of gas into some oven and output the uh discrete component it in its gas. So it's used in chemistry and pharmacy and some other industries. And as for us researcher is just a computer that we can hack and exploit. So this is what interest for us and as for me I'm not aware I'm not a aware of what a gas formtoraph is right I'm a security researcher. I'm not a chemist or in some in any way. So for me the first thing I need to do is uh read the wall documentation to get the context uh for

uh this research as well. Uh we today we are going to talk about the research and uh it's and how it's all started. It all started when my boss asked me uh to perform an analysis on proprietary protocol that works with this device. This h from photograph like something that I never heard of before and uh like we saw right now what it do and uh this is how it looks like. Uh and uh the specific uh gas forgraph that we are going to talk about today is a emerstone rosemount uh gas for photograph that is uh that is somewhat uh one of the smallest models that they have but it's still pretty big in size.

Um I don't have the advice. This is in the title but I would like to h I would like you to look just uh for a second in the internals just to uh get impression on how it works. uh along with as a device there is a software client which operator will uh install on his Windows computer and with this client uh we can monitor the device configure it and maybe see some logs right uh for uh almost any physical device that we have we are often have some uh client that we can install on our computer so we can be able uh to configure it. So this is uh our ecosystem here in uh this research. Uh

just a second. Uh so let's look uh at it in high level. We have our uh lab technicians that sitting in their labs and uh they installed the uh the configuration program on their Windows machine. And we have the actual lab uh in which we have the actual uh gas chromatographs device and uh we have a a protocol in this case a proprietary protocol uh that was developed by Emerson and uh that protocol is a secure protocol it's encrypted and runs on TCP port 10,000 and this protocol H the technician will communicate with the device via the uh configuration uh program of course u see some logs and perform any operation that industry needs to

perform. So this is how the this setup is looks like in real world of course. uh so our uh goal here and it's how this research even started is to anal an analyze the proprietary protocol as it with many proprietary protocols this protocol is not documented. So when we need to analyze the protocol that we are uh that communicates between the program and uh the device uh the only the only way that we have and can to be able to understand the protocol, we need to somehow to emulate or simulate this environment in our lab. And uh the optimal pre- requirement for protocol research are that we will have the client software and we have the device

and the capability to see and uh and alter the network and the communication uh between the two. If we are uh able to see what's going on and we have uh the uh actual program and the device, we can look reverse engineer all of the components and somehow with time and many effort to understand what is going on. Uh and it it takes time of course because it's an undocumented protocol. So we have to be very patient. So the the one thing that's very easy to acquire of course is the software client. It can be downloaded from the from the website and it is easy download uh can be easily downloaded and installed on any Windows machine and uh

we can already start with understanding the basic capabilities of the device. uh and that's what we did. It is a easy easy step. Nothing uh nothing too complicated is happened here. Um but we don't have uh the actual device and we somehow erh need to emulate it and let's see why we don't why we don't have the actual device. Uh the thing is that uh those devices are pretty expensive and uh as you can see the price here it's a little bit uh bigger than I'm allowed to spend uh on my research and since we couldn't just buy this for two weeks research we decided to emulate it and h we we Before we starting to emulate the

device, we need to understand what we want to emulate, right? Because we have the actual device. What p what it performs? It uh it gets a mixtures mixture of gas and gives some components about h discrete components about its mixture. But we don't know we don't need that, right? uh we don't erh we don't care about its uh chemical components and abilities. We care about the brain system and the main binary that operates the proprietary protocol and uh for that we only need the firmware. We don't need the actual device and luckily for us uh Emerson published its firmware on its website. So we just can download it and extract it. And luckily for us it was a

Linux file system right? We we all familiar with Linux file system. It quite easy uh to uh take some binaries out of it and maybe uh try to run it on a another machine. Right? So um when we see a Linux file system, it tells us that we are probably will be able to successfully emulate the uh protocol um emulate the protocol and connect to uh the software client that but we need this in this phase we need to understand what a binary we uh we need to emulate right we need to find a binary that uh implements a protocol And it's not that easy or straightforward because we have the device that is that is developed in

some in at emerson right and it's packed only with the binaries and in structure it's needed to operate in in specific industry. It's not a general computer when I can Google where a specific file res reside right um in this phase this is quite tricky phase in which we need to find the actual binary that implements uh the protocol that we we want to allize and it's good it is good practice to find to start to find this binary in in this file Because in Linux and in Linux and especially in a not general purpose machine the first binaries that need to be always operating it will be reside in any files. So after a lot of searching we

did find it and its name doesn't look like any something interesting. H but this is uh this is how it looks like. We find the uh the init uh script that in its file that in from the start of the file system and in some of the in one of the scripts we see the h the actual the actual binary is executed. The name of the binary is XTPD. Nothing that can some somehow hint us that it is the main binary that will implement the protocol. But after some errors and we are able to find it another erh sorry I'm trying to is there some problem the next slide it's got it's not I'm not

sure that I'm getting

Yeah, just a second.

I'm not sure that we see the correct slide. I will just wait for a second. Oh yeah, that's cool. That helps. Thank you. Yeah, that's cool. So the question we are about to ask here is can we analyze the protocol statically without emulating the protocol? We have the binary, right? We can reverse engineer it, put it in IDA and just see how it behaves. Um technically we can do it of course. Uh, I have a technical problem here because it seems like I put push forward the slide but it's not working. Maybe some someone can help me.

Yeah, I see that. I can I I Yeah. Okay. Okay. Now we are good. Yeah, cool. So the answer is yes. Theoretically we can reverse engineer the binary and uh find out uh find out how the protocol works. uh if it's matter of time right but eventually all the code that implements all logic is within the binary and by sitting on the binary for hours and hours reverse engineer it we are able to get the full logic and in some uh project of ours it's the only way to do to go in case that some sometimes not every device can be emulated and somevices devices are have to be researcher statically and that's what we do. So theoretically yes it's

possible but we rather have the emulation working so we will be able to manipulate the protocols to see uh how it works. So we be will be able to fully analyze it which will take a lot of a lot of less time if we will do it dynamically and not statically. So the next question is how we erh perform that emulation. So let's go back for a second in for our context right our goal in this research in that phase is to analyze the proprietary protocol of gas for motorgraph that we cannot buy we cannot buy it because it's super expensive for our research and we have no other option to go other than emulate

it and the emulation is possible because we have the firmware on Emerson erh uh website which is not encrypted and can be easily extracted. We have downloaded the firmware and we have extracted it and found the main binary that we that implement the proprietary protocol. Now we have now the situation that we have is that we have the Microsoft client that we have downloaded from the website and we have the firmware. We found the the main binary that implement the communication between the uh client and the device and now we are in the phase that we are want to emulate the wall settings the wall setup. So we will be able to see this communication in

action. So how do we emulate this? As I told before, we saw that the fware is basically a Linux file system which is very convenient for us in the case of emulation because we know that we can erh now be set some uh Linux machine right we can uh copy the fware file into it and perform sage root command. We will talk about it later but first let's see some caveats that we have. The thing is that most of the and IoT devices comes in nonstandard archite architecture for in this uh specific case the architecture is power PC it's not that common to have in general purpose computer so if I would like to put it in some Ubuntu uh I will

need to find the computer that runs power PC which will be quite challenging for for ARM today we have a lot of options right we have a Raspberry Pi and we have uh even the Apple that may that uses ARM as its architecture but for power PC and MEIPS it's still a challenge and uh for this specific emulation we will need to use QMU for those who doesn't know who doesn't know what QMU is is a just an emulator that you can install on your host machine, general computer, whatever or any other and it will run instruction from another different uh architecture which is super useful for OT and IoT research. In my case, I have my

Mac on which I have uh Ubuntu I have an Intel Mac and I have Intel Ubuntu on it as virtual machine and on top of this I have QMU that will run the um the machine that will run the Power PC instructions. So what we are going to do is we we are going to install power PC Ubuntu with QMU and after we will power on that Ubuntu machine we will copy the file system into it and truth to the file system. So we will be so we will have the file system of the gas forgraph working for our emulation. Let's see how we did it. Okay. So, this is the uh actual command I used when I powered H

the Ubuntu Power PC. I'm just downloaded the Ubuntu power PC from Ubuntu website and I uh search for a board that will implement the machine that will implement the power PC architecture that will be supported by QMU. Uh this is an important part in emulation and uh in my case this machine was a Mac 99 which is a Macintosh really old machine uh which power which was on power PC. Yes, Macintosh back in the days uh it used power PC architecture and this board is uh supported on QMU. So for in in my case I just run the Ubuntu on a Z board with QMU and uh after installation phase I have an Ubuntu power PC uh installed

uh with emulator. So after some time after some time I have uh my setups going on uh I have I have you cannot see it it's in a gift that h probably uh cannot be shown well but here's a situation that I have uh my client my uh client that is installed on Windows machine and I have uh the emulation and h it's all uh working and h it's all working I mean uh the basic h the basic startup of the communication working and I can start to uh investigate the protocol or am I uh after some time and I understood that and I said it earlier that protocol is encrypted and uh and now I need to await

to see somehow the communication. Uh so that's the that's the situation we are at now. We have the program micro uh Windows program the client and we have the emulated uh device in the QMU and we have uh the communication channel uh which uh is encrypted on TCP 10,000. uh we will have to patch uh patch the machine to disable the encryption and uh in our case we were uh really lucky because the the client supports and the client and the device supports both encrypted and decrypted uh communication. So we all all we needed to do is to lie about uh our version in the handshake uh of the negotiation between the client and the server and uh to have our h our

communication totally decrypted. So now we are at the phase that we can uh see what's going on. We have our client and we have our device and we have the communication that we can actually see what it's going uh on between the two uh parties and in this in this picture we see uh the actual client and uh we see here the wireshark the IDA and the client both for a couple of hours or maybe case uh while h I'm analyzing the protocol. So the protocol is rather simple one and uh just second the protocol is very simple and I would like uh just to cover it in in its basics. So we have our h

header which is some sequence number and common type common type and h length of the data and the data is just embedded on uh the bottom right after the header and just to illustrate how the protocol works uh I will uh give an example of uh one of one of the h commands like see uh one of the commands erh to to change level of a user as communicates between a client and the device. So in this example we have our header h and we have our h common type which is lemon hex like seen here and we have the data length and after that we have the actual username and the level we want uh to be changed to. So

very simple protocol nothing too uh nothing too complicated. Uh the next phase in this research uh after we have analyzed uh protocol uh my team leader would like to know whether the protocol is secured and this is phase in which we start to look for vulnerabilities. Um and let's find out if the protocol is secure. Right? So we have here pre-authenticated remote code execution and let's see how it works how it was discovered and you can track it online with uh the CV if you want more information. I would like to uh before uh talking about the specific vulnerability just I would like to uh tell about common procedure in which I like as a

vulnerability vulnerability researcher is looking for uh for vulnerabilities when I looking at and in this phase what we have and what the the steps that I'm going through is I have the binary that implements protocol and I want to see how secure it is. So uh the most of the job that I will need to do here is of course reverse engineering since the protocol is proprietary is and not documented. So no logic flaws can not be found online. So right here I have my binary loaded into IDA and I have some understanding of the protocol right because I have analyzed it before and now I'm ready to find actual variabilities. So the first thing that I

will go after is the logical ones because logical ones are much easy to exploit than the memory corruptions. If after some time I cannot see uh I go through all my uh attack vectors and I cannot see a possibility for a logical bug, I will go straight to the memory corruptions of course. But since the exploitation of memory corruption is much more complicated, it it you know OT system is still much easier but it's not that uh that easy to exploit in terms of time like logical bugs. So I've straight going to find some logical flows and the main logical flows that I would like that I love to start with are pass reversals and command ejection.

Commanding injection are is the flows that really easy to spot because there are used h they use a lot of strings in in it and only with eyes and just look at the strings table we can see some hints about the exploit exploitability of uh this flow. So we found a one common injection and just a second h we uh found an OS common injection in uh in this procedure. The procedure is called forc calibration. This is a common command uh this is the number of the command as you remember uh we have a common number in our header of the pro uh in the protocol and uh let's see how it's implemented.

I it seems that I have another issue.

Uh we are seeing you right. Uh Vera, are you listening? Yeah. Yeah. Everything's okay. Only my cell phone get I I cannot. It's it's it not seems that I going forward with the presentation. Okay. Sorry. That's okay.

[Music] Check if you you you're now it's working for you. Please, it's not it's not working. No, I pressed next slide and it looks like it's not moving forward for me. It's not. It's stays on the same page.

Please, if you if you could remove the this the presentation and put it again maybe. Yeah, sure. Yeah, sure. Sorry for No, yeah, understand that stuff. Yeah, we we are doing a live session. It's not so simple to do that. And then the yeah that stuff happen

maybe if I present my screen it will be better rather than the presentation. Sorry if I present my screen maybe it will be better rather rather than the presentation. I can share my screen. Maybe it will be better.

You are

You're listen, we're trying to put you on the I think that the presentation is not working so well. We are not seeing the maybe I can present my screen. Yeah. Yeah, you could do that. Yeah, let's let's try that. Let's try that. Yeah.

The power is Connect.

We could try again. Vera, please if you could. Yeah, we we remove you and and try to put you again to see if it's working.

I think now it's okay. You you are only on mute. Try to unmute your your Okay. Are you seeing now? Yeah. You see you see the presentation. Yeah. Yeah. Yeah. I think everything now is okay guys for whoever stayed. Thank you. It's it's for those who stayed. It's seems that it's really interesting. So thank you very much for staying. So let's go to the command injection vulnerability uh that was found in the first calibration erh command of the protocol and h as I said uh those h vulnerabilities are somewhat easy to find relative to other vulnerabilities like memory corruptions and why that is because we can look for uh system calls or some strings that

will indicate that a bash uh call is executed and you can see in this uh code snippet that this is exactly it. We see some command that is uh constructed in some way and later it fed to system uh function which will uh execute in an a separate shell the comment that is forward to it as an argument. So that's cool and uh in our case we only what we need to validate is is whether we can um we we can as an attacker uh somehow uh to uh to provide the input to H to that command. And in case of forced calibration, this is the only common that we found within the firmware that an attacker can

uh bypass can send to the device and uh be forwarded to the system library function. So this is how it looks like. This is how looks most of the command injection uh vulnerabilities. So we have some uh bash executable application and we have that input that is controlled by attacker and uh this is how uh it is the command is looks like. And now uh for example for a reverse shell we only need to uh to do is um in instead of some file that will be opened by data application we providing netcut application that will uh start the reverse shell uh so it will be executed on uh the device. So here how it looks like just with

Python we sending we a proprietary protocol that we have analyzed and know how it works and uh we open the socket and uh this uh specific command does not require any authentication and we have our reverse shell on the device. So this was pretty cool. So uh we can start about anything else. H I will uh we don't have enough a lot of time but there is something that I would really like to show you guys and girls. There is something that we as researchers can understand only from reading the documentation right and I would like to uh showcase this example to you. We have here an instruction of a reset administration password and here

is what we can read from that documentation. We have uh the administrator password that can be reset by calling the uh support and it cannot be uh reset by the support without a factory reset which is not a good practice right you wouldn't like to forget your password to call someone and it uh to call some support and they will be able to uh reconstruct your password right because if they can maybe someone else can to do it as well, right? And this is what happening here. We are reading this documentation and the red flag should be uh turned on in the head of security researcher something uh that something's very suspicious because if the support

uh technician somehow knows uh how to generate password maybe it's a generation process p is resides on the device itself. And uh long story short, it is we have that CVE. Um and we do not have the actual time right now to go uh and see the actual implementation of it. But uh we do have a blog and at clarity and you are more uh than welcome to uh look at it and far uh find more details. So I will just uh go uh forward with uh with the actual technical details and just uh show you this slide which shows that the actual key that will with which the password will be reset is just a MAC

address which is not secret information. So uh if you have your hands on some device and you see its MAC address which is written on the sticker on device itself you can uh log into it and bypass authentication and this is how it looks like in the protocol which uh authentication is successfully bypassed. So uh we have our uh RC and bypass which uh seems that this solution was successful and this is a great time to thank Emerson. So they patched the vulnerabilities promptly and we enjoyed to collaborating with them and they released a new version of the device that if for some reason you didn't update it so go into it uh soon. Uh so this is a vulnerabilities

that were uh were discovered during this research and if you like our uh this research and other ones like I said we have a blog with a lot of good stuff and if you have uh any question about this one you are free to reach me out in LinkedIn and GitHub or ex uh I would like to answer any questions. So that's it for my side. I was said that I have only 40 minutes guys.

Are you missing us?

Let me Are you listening us? Yeah. Yeah. Yeah. Uh we have a first we are they are requesting your linkaging if you could share here in the chat. Yeah, sure. Yeah. And f and again first thanks for the great uh work that you're doing and you present here for us. I think I have a question for you. How many months or weeks that you uh work for this kind of what for that that that that gas chromatograph? Yeah. Yeah. That gas chromatograph uh research that you do. How many weeks? How many weeks or or months that you work? Okay. Uh so this is a good question because uh it looks like when when you perform this type of

research the time it takes it can somehow tell about a time lapse of some other research another device. This specific research was like six weeks maybe two months. H I think uh the most the most of the research went to emulation process which is not an easy process. uh we didn't talk about it much but uh there is a lot of dance to patching the binary so we it will be able to run on a simulation application like QMU and not the real device and it can take some time and reverse engineer the protocol which can also h it's a pretty uh tedious work if the binary is stripped. So I think it had been for two months,

two months of work. Perfect. And and a great work. Any questions? Well, thanks a lot again for your support here in the team of H2 from Crow Chief.

I can hear you.

Okay, guys. H I'm going to go. I think uh this is if you can if you hear me uh the presentation this is my handle in all social media. Oh Vera it's okay. Sorry. It's okay. Yeah, we have an issue here in the internet that that you are presenting, but uh we really would like to congratulate you and then we will move here to the to the next presentation and see that. Thank you. Bye. Bye. Bye-bye. Thank you. [Applause]