BSidesWLG 2017 - Tom Isaacson - IoT: How to fight the tyre fire

yeah I'm Tom and this is my talk so I'm gonna put this slide up I have no idea what it means I just I love this slide it's just gibberish you can you can look at that while I introduce myself so yeah my name is Tom I don't work in for a sec I'm a developer so this is really talking about IOT from a development point of view rather than just hacking stuff one point I'm I'm not here on behalf of my company I haven't put my company name on the slides it's not that hard to find out who I work for some of the things I may mention in this talk may possibly pertain to current or

previous employers products please don't run out and start tweeting about this stuff because that would just be embarrassing if you want to try hacking a product that I may or may not have worked on please go ahead you will need to buy it which is great because that pays my salary so feel free to do that so IOT it's the standard slide where we have a definition for IOT the Internet of Things is the network of physical devices vehicles home appliances and other items embedded within electronics software sensors actuators and network connectivity which enable these devices to connect and exchange data that one's from Wikipedia you can find hundreds of these definitions online bollocks its

IOT is just a buzzword that somebody's made up recently what we're talking about is embedded devices with network connectivity this is not a new concept network devices herbs or embedded devices been around for a very long time depending on how you define embedded device it's usually acknowledged the first embedded device was the Apollo guidance computer which was in the Apollo Rockets first flew in 1966 obviously developed before then a lot of the devices you hear being hacked things like webcams the first webcam was the Trojan coffeepot in 1991 again not really a new concept routers get hacked routers the first router was called the nicknamed the bread truck in 1976 these aren't new they for a long time what's the problem well

the problem is I have teased a bit of a tire fire it's it's causing a lot of problems it's not going that well what will easy about IOT that means we had to define a new buzzword for it what does it mean when we talk about IOT well firstly the numbers webcams routers those kind of things they've been around for a long time but the sheer volume of products we're talking about now I got these estimates I don't I don't quite know what definition of IOT they're using I'm sure you can make the numbers go one way or the other by changing the definition but yeah this year they reckon eight point four billion IOT

devices connected the internet right now that's a big number last year was 6.4 billion so that's an increase of 30% 31% estimates 2020 twenty point four billion devices that's an awful lot of devices so that's one of the Tom's is that one of the things that really defines IOT is the sheer volume of devices we're talking about longevity IOT device is expected to last quite a bit longer than other products most one I'm not sure you've all got a mobile phone most people tend to replace their phone every maybe two years if you're a real Apple fanboy maybe one year if you tend to drop your phone down the toilet six months I don't know IT devices are

expected to last a lot longer you think about something like a washing machine you really don't expect to be replacing a washing machine more than every maybe 10 years if you're buying brand new some products are expected to last even longer than that these aren't your typical cheap throwaway products they're expected to last for a while pretty much the definition of embedded devices they have minimal or non-existent UI some of them some of them have more you think about some of the definitions of IOT include electric cars they often have got a big UI and then you've got things like the light bulb where you've got just nothing at all it's harder to interact with the

user without them making that decision and finally [ __ ] security there's security on these things is terrible now when I talk about [ __ ] security what am I talking about well I suspect you've all heard of the mirror I bought now there's sort of the big news that IOT is rubbish which didn't really come as a surprise to other people roughly 200,000 devices in the original botnet since the source code release it's actually grown although it's not much use anymore it's sort of fragmented because people are using different bits of it but it was able to generate these enormous DDoS attacks firstly on crabs secondly on din big numbers 623 gigabits per second one

terabit per second London these are these are big attacks and all Mirai was was a set of default credentials for some 60-plus devices where these devices somebody had hard-coded in this particularly since hanjo Ching Maya I think devices all had the default username password on telnet so first of all they left telnet turned on and secondly they had these hard-coded passwords I don't think you could actually change them even if you wanted to and the other thing that made that bad was these world white-label devices so they weren't selling those devices themselves they were selling them to other manufacturers to either rebadge or put into other products mirrors kind of been a gone we've now possibly got

Reaper based in partner Mira it's using those same default credentials they've added a bunch of other attacks for routers and some surveillance cameras kind of high-end webcams these are a little bit more advanced using things like buffer overflows and those products most of these have already been patched but very few people are upgrading the devices partly because of that minimal you are you difficult to tell the device has an upgrade available unless you're actually logging into it and having a look estimates our repo a bit vague at the moment somewhere between ten twenty thousand devices and a million devices depends on who you ask hasn't yet been used in the wild I would guess it's pretty much guaranteed to go

off when RIS risky business goes on holiday because that's usually when things turn to [ __ ] there's this other one had you may had Jimmy's really interesting it's a more sophisticated implementation that has a better control command control process but the really bizarre thing is when a device gets taken over by the hajime botnet a message appears on the terminal says just a white hat securing some systems which suggests that maybe someone has actually written a botnet to go and capture these devices before anyone else does so they can't be used by the bad guys or it's a bad guy just pretending we don't know so had you made something like 300,000 devices also not yet used

so it may turn out to be malicious we just don't know so why are these products associate well I'm a developer I work for a company a company has a budget company has resource developers to work on things you have a choice do I want my developers to work on security issues or do I want them to work on making team products adding features fixing existing bugs you tend to go with the new models and features it's as simple as that there's no real benefit right now to fixing security there's no real legal requirement there's nothing that says you can't ship this product because the security shed some of this is being discussed in the u.s. in Europe but it's

not really clear how this would work yet it's not really feasible for every single product to go through some sort of government certification process maybe it would be a voluntary thing it's difficult to say there are some things the FTC is currently suing dealing I don't know if it's going to be successful or not over there insecure routers and webcams it's possible that that might be enough to scare the industry into making things better I doubt it because the rate of which they're suing people is just it's a pretty good gamble that you're not going to get hit there's no consumer interest in security you can't sell a product with securities of feature charge more for it and expect

people to pay for more for it you can put security on the box as a word and I'm pretty sure marketing people do that all the time with no real justification for it at all oh this is a secure web cam it's exactly the same as the other web cam but it costs 20 bucks more there is I think Australia and possibly some other countries are discussing this idea of a securing rating scheme so it's similar to when you buy a washing machine and it's got the thing on the front that shows you how much water it uses per load would be some idea of a star scheme or something like that saying when you've got one star because you've

managed to turn off telnet 2 stars because you provide updates or something like this 3 stars because you've actually used a hash something along those lines and that would encourage consumers to actually go for the the more secure products maybe that might work there from is companies drop products they end-of-life products companies go bust well they get bought and then shut down and there's no real way of maintaining support for a product once the company who made it disappears maybe you could have something where you put the source code and the keys for that product in escrow and if the company goes bust you can then get access to it so you can maintain support

it's not something I've ever seen done and the other thing is some bits of hardware you have the ability to install your own firmware so open wrt is a pretty well-known open-source version of linux that you can install on routers so you can wait to set up the router the way you want to do it but there are some limitations have a hardware allows you to install we'll talk about that a little bit so in my slides actually said that oh well top 10 for IOT was new it's not new actually I realized it was actually posted up there in 2014 and obviously no one's read it yet but let's go through it anyway it's still relevant so number

one insecure web interface yeah I'm not really sure about the ordering here insecure web interface seems unusual to put a number one because a lot of IOT products don't have a web interface but I kind of see where they're going with this attack uses weak credentials captures plaint exponentials or enumerates accounts to access the web interface pretty obvious stuff the reference is up here this was written in 2014 they referenced the eyewash top 10 from 2013 I've updated the references to the the most recent 2017 month which I think was finalized last week so you can talk in there talking about injection cross-site scripting cross-site request forgery that the standard web interface problems well though I'm not I'm not

sure how many IOT devices where you get things like SQL injection a lot of IT devices don't really have databases on them so again I'm not sure how relevant that one is on a product I may or may not have worked at in the past our web interface is just based on the standard busybox web server with some CGI scripts with some shell scripts for the CGI stuff it's really really basic stuff and I'm quite sure those shell scripts have got bugs in security bugs in and if you really want you can have a look at them and tell me number 2 insufficient authentication and authorization I kind of think this is where Mirai starts to

fit in having telnet open with default credentials that's that's a pretty good example of insufficient authorization

that yes and you can see how that is going to be an obvious problem you need to fix insecure network services attack users mobile network services to attract the device itself or bounce attacks off the device so this is pretty standard stuff unnecessary open ports UPnP obviously is a problem exposing ports to the Internet and then you get these weird ones like Wi-Fi access to the network what I'm talking about here is in particular there was a product couple of years ago called I kettle was an Internet connected kettle I did you often hear people say why on earth would you make something like this the question to Prince Prince Tamar on why on earth

would you buy this what possible use case do you have for an internet-connected kettle and it did occur to me earlier after seeing the earlier talk about the light bob iot light bulb maybe all of these products are being bought by pen testers who are writing conference talks maybe there's a business model there I don't know the problem in the eye kettle was you I think there was a mobile app or something and you enabled the kettle to connect your house Wi-Fi by giving it the SSID and the password the problem was the kettle didn't check that the SSID was using the same level of encryption it was previously so you could use a stronger signal with the same SSID knock

the couple off the house Wi-Fi it would then connect to your Wi-Fi but you didn't have encryption turned on and it just gave you the password for the house so there you go you've got creds pretty stupid bug did occur to me that I should probably check my source code guess what say bug it's it's an easy mistake I didn't occur to me that you might have an SSID without the same level encryption turned on so you just present the password to and I wouldn't be surprised to find there's a bunch of other products out there that have the same bug in and a part of the problem with with embedded devices is you're not

relying on existing software like Android or something like this you're writing this stuff you're self-sow the same mistakes get made over and over again okay not as dumb as leaving a telnet turn on with default credits but I can imagine this is a common bug lack of transport encryption integrity verification this is really about the data being passed possibly between devices on your network so maybe you've got a bunch of light bulbs and they talk to each other or something like something along those lines the problem here is that the solutions we have right now for encryption most of most of them are relying on internet access they're all built for websites and things like this and the problem is

you can't guarantee that these products are going to be connected to the Internet there are lots of situations where you take an IOT product and you connect it you've got your IOT lightbulb you connect it up on your house on a local network because you're a security person you don't want to expose these to the Internet and then you use your phone on the same local network and they never connect to the internet which is fine except if you're relying on being able to get upgrades or new certificates when the old ones expire you've also got the problem of some of these products get made and then left in a warehouse for a couple of months and then you get them

out of the box and all of a sudden they've been in a box for five years and the certificates on them have expired and then you connect to the internet and they don't work so you can't just make those same assumptions about products that you can on the Internet Darry there are even more complications when you actually want different products made by different manufacturers to connect together I'm on a Standards Committee for some domain specific stuff where we're talking about different manufacturers making products that want to communicate together in an encrypted way how do you manage that especially when those products may never get connected to the internet it's quite difficult and if you really want to know

about this stuff how much Ryan's talk from Christchurch hexagon it's worth it because he explains Pekar with emoji which is impressive in itself so moving swiftly on privacy concerns obviously a major one the data that gets collected by these devices is it being stored securely now I said there's no legal read requirements for releasing IOT products but stuff like this is starting to come under a legal framework so especially in the EU the GD P R which kicks off I think on the 25th of May next year so not long now has these requirements for things like user consent and see you know normalization of data interesting there's a legal obligation to notify the authority in a

debated breach within I think it's within 72 hours which is really interesting because especially we look at a case for uber they covered up their data leak they don't want to tell anyone about it they're now in the Europe anyway if you're collecting data on European citizens which you but uber obviously do they would be obliged to tell the authorities within 72 hours of finding the breach and the fines for this stuff are pretty impressive 20 million euros that's 34 million New Zealand Dollars or up to four percent of the annual worldwide turnover whichever is greater now there are companies that are making more than 35 million dollars four percent of their annual turnover so

these spines can be really significant maybe this is what starts to really kick off the privacy process but this is just the storing of the data online this does not affect the devices themselves in secure cloud interface an obvious one probably not as bad as it sometimes is obviously to connect to the cloud you have to be on the Internet so you should have access to updates most of the cloud providers are doing a reasonable job of providing security you stick with stuff like edit less it's not too hard in check your mobile interface this is a little bit more complicated I put Bluetooth in here because an obvious one when you have a mobile app

you want to be able to connect your mobile device to the device you're talking to you might be using Bluetooth and I'm being very frustrated with the amount of information there is about how to make Bluetooth secure what's best practice the Bluetooth SIG of which we all remember released this could launch studio quite recently which helps you to configure your Bluetooth products not so much phones and stuff more like when you're making an actual embedded device that's got Bluetooth built in and they had a webcast for this thing and I actually asked and look at the question does this help me check for security best practice no is there a security best practice of Bluetooth that

you publish no so I mean they kind of invite this stuff to themselves the best guide I've found is actually published by NIST at the end of that it's kind of out of date because Bluetooth 5 just means it's being released insufficient security configurability I'm not sure this applies to IT devices sorry IOT device is a huge amount you tend to have the security they come with that doesn't tend to be a human out of configuration in their insecure software and firmware again I'm in somewhat two minds about this one if you completely lock down your device with secure boot and encrypted firmware and things like that it means that no one can then ever be

able to replace the firmware with something they genuinely want to use and that's kind of one of those ongoing arguments if I've paid for the hardware don't I have the right to do with it what I want and I suspect this one's gonna go on for a while poor physical security I mean this is an obvious one I remember Matthew Garrett hacking a DVD player and key week on a couple of years ago by basically putting a file on the deep on the SD card and it it read the file and turned off the security and God knows what so that's the other was top ten for IOT it most of its kind of obvious but it's good to have it written

down somewhere so it's really important to be able to get firmware updates for these products as I mentioned earlier Reaper is including those nine attacks for devices where they don't that these these issues have been patched but a lot of customers just aren't applying the patches and so these devices remain in the world and they're still vulnerable so you need to be able to update the firmware the problem is as I said a lot of these devices don't have some sort of UI so they're not just gonna be able to pop up miss saying please update me often the user has that the owner has to go and log into the device something to be able to

see there's an update available so the obvious solution there is to make the updates automatic well that's all very well but you've got to be very careful when you're updating devices in the wild and if it's a washing machine okay the washing machine knows when it's actually doing the washing cycle if it's just sitting there not doing anything at all then it can probably do an update itself without too much worry but if you're talking about things like cars industrial control systems you can't just update them just like that it needs to be some sort of synchronization there so it really depends on the device you're talking about it needs to be tested on real Hardware variants now

this this is something I think a lot of people don't quite understand when you make a product and you start manufacturing it you don't just stop products constantly evolve probably the biggest thing we have to deal with is changes to NAND so a few years ago all of the fab switch from 42 nanometer to 21 and then more recently they've started me switching to 19 so the chips you're using in the product go end-of-life and you have to replace them with new chips and sometimes it's a new generation of flash and you need to upgrade your kernel to be able to get that support in for the flash drivers and that kind of thing hardware evolves there was a product I

may or may not have been involved in a few years ago where we actually upgraded the CPU and then we slowed down the clock speed because the new CPU was faster than the old one so the two products looked the same it sounds nuts but we didn't want customers going into a shop and buying two products and finding one of them was running faster than the other it doesn't make any sense to them just because one of them has been on a shelf for six months and one of them has just come straight from the factory so you end up with this sort of legacy support of a whole bunch of different hardware variants in the field

there was a case quite recently where a company called lockstate have these internet collected blocks which were recommended by Airbnb and they did a firmware update and bricked a whole bunch of them brick them to the point where they had to be returned to the manufacturer so they could fix them sound like a funny story but I'm pretty sure that what happened was they had a bunch of different hardware variants in the field they tested the update on some of those variants didn't realize it was going to brick the other ones and did the update over there and this is a problem that has to be dealt with if you're talking about say you are

a company that's been going for a few years you ship maybe 30 different products you have got between one and ten hardware variants of all of those thirty products and then you want to do a software release across all of them that's quite a bit of testing that has to be done just to make sure you're not breaking them that doesn't even make sure that there aren't any bugs in the software so it's a big overhead the download path needs to be secure and obviously if your connection to the Internet to be able to get the downloads you can use TLS to verify who you're talking to you can use certificates to make sure the security the software has

been signed but as I said if that products been sitting in a warehouse for five years and the certificate bundles expired then maybe it can't even connect your server to download the update with this fresh set of certificates in there's there's also case quite recently the logitech harmony link logitech end-of-life this product quite unexpectedly and the rumor is that this was because they used in Equifax root certificate and the roots to view it was lost and suddenly they had absolutely no control over the security of their product and the easiest thing for them to do was to brick it I don't know if that's true or not but I can see exactly how that happens you choose a

certificate authority and suddenly you find out five years later that they were all dodgy and you can't rely on them these are all problems that are going to be coming up and the other thing is the update part needs to be secure supply site attacks are happening quite a bit especially this year there's being ccleaner where a signed version of this offer was put up on their website which included malware medoc was the accountancy software that contained the petia malware that was released in Eastern Europe Minter linux mint transmission which is a BitTorrent client all of these have been basically shipped off a server or a website containing malware and again if somebody hacks into your

servers and it enables to put out some sort of malware choice offer how do you stop that happening especially if they've got access to the build server which contains the signing certificate for your software it looks valid but until someone actually checks you don't know it's happened now I did read a paper a while back I thought I couldn't find the reference where someone came up this idea or having multiple certificates you sign your software we say three different certificates and each of those certificates is held by a different person so in order to release the software it has to go through each individual person to get signed off probably very secure quite difficult to

implement the real how do you how do you get that running through a build system what happens when one of those people goes on a holiday what happens when one of them loses the Yubikey they had the keystore done or something like that it's it's one of those problems I don't really know how you fix that one but server-side attacks are very supply-side attacks are becoming more common and it's going to be a problem what's the point of hacking the device when you can hack their server and own all of the devices in one go so my summary IIT is going to get worse before it gets better there are eight point four billion devices out there an awful lot of which

we've got crap security there are developed devices in development right now that are going to be released shortly that also have crap security and frankly that's going to keep happening because developers are stupid I'm a developer I'm stupid I make these kind of problems all the time it's it's easy to do that example of the Wi-Fi connection problem I was talking about it was just an oversight on my part but it happens one product I may or may not have worked on the first time we shipped it I put the root certificate for the particular certificate we were using on the website so I could connect to the web and then a year later the web guys

replaced that certificate with a certificate from a different provider and our product stopped me able to connect my emails what are you doing we've got to get this certificate from the same provider and they said no we can't because that asset of it was 128 bit and we have to replace it with 256 we had to go back to the original provider pay them additional money to get 128 bit certificate for the next year so we could make sure the products would update in the field to be able to support the new provider you can't just rely on being able to use the same certificate authority over and over again so now we build in a bundle which

is better but have they said if your product and ends up on the shelf for five years you can't always rely on the certificate in the bundle some of our products are going to have bundles which contain a valid equifax certificate and until we upgrade them we can't really do anything about that so yeah we need help I mean some of this stuff yeah taking out 1084 credentials I mean yeah it's easy it's so obvious it's frustrating how many one managed to do this stuff but some of these more complex problems don't have solutions yet how do you deal with expiring certificate how do you deal with devices that may not be connect to the internet wanting to

communicate with each other on private networks I'm still kind of waiting for someone to to figure this one out maybe it's gonna be me it's more like it's gonna be Ryan I just don't know and the other thing is it would be really great to have some CI tools for this stuff the one I've been thinking about or trying to find a solution for actually is I want to be able to do man-in-the-middle testing on my product to make sure I haven't screwed up the TLS there but it would be really great if I could make that part of our build process so every time I do a build it then runs that check and I'm sure it's

possible to do I haven't found a tool that will do it for me yet and I suspect anyone who has done this just doesn't shared it out yet and maybe I'm gonna have to work my own or hack something together these kind of tools will be really really useful so not so much you get the product right first time but you can then make sure you're going to get right every time you do it change and finally I don't know what's gonna happen with these botnets I'd the idea of like a white hat versus black hat botnet fight it's I don't know I reckon there's a movie on there somewhere come on they made the emoji movie this is actually

you know so yeah I mean maybe how'd you make it turns out to be bad maybe it turns out to be good maybe this is an ongoing thing I don't know it's gonna be interesting to find out okay thank you very much any questions

well the EU isn't gonna collect sorry the question was who's pushing the EU regulations the EU isn't going to collapse Britain's leaving probably but the rest of the you still going quite well and I mean even if the you Durst collapse that's just a legal framework I wouldn't be surprised to find that other countries start adopting something similar but I mean even if that particular legislation collapses I think that's the general train and people are realizing there needs to be more control around privacy around data online what happens that data what happens to companies that are stupid enough to store the passwords in plain text online it we need something to drive these changes because it's not going to happen

on its own sorry said again that legislation kicks in in May next year I don't know

yeah that's the point was made about Microsoft open document formats the EU would definitely sued companies in the past they sued Microsoft years ago for Internet Explorer being included windows I think they've sued Google for something similar and weird to do with Android the microcell of document format was actually produced as a result of legal threats in that Microsoft had to open up their data so not so much with privacy but this kind of stuff has worked in the past but yeah I'm not a lawyer so okay thank you very much [Applause]

you