BG - Securing Robots at Scale - Talha Tariq

thank you everyone for coming welcome to besides very excited to talk about our journey to secure home robotics at scale I'll talk about a few things today basically you know robotics platforms what comprises of a robot different components and sensors in a home consumer robotic space you know some unique threats and attack surfaces for robotic systems then I'm going to talk about you know security and trust models for consumer robots basically giving some energies between you know your IOT devices your other hardware devices and how some of the attack surfaces are very different from you know the consumer devices that you are accustomed to then you know I'll spend some time talking about some of the security foundation

work that we've done in the robotic space and then you know basically talk about some trade-offs and challenges between security privacy and safety aspects some of the hard decisions that you know we have to make an area their company has to make when shipping a consumer device and then lastly you know I'm also going to cover some interesting privacy work that we've done as it pertains to you know robotics as well as consumer electronic devices quick introduction you know I've been doing security work for 15 years I started my carrier with the NCR it a lot of payment security work on you know terminals and ATMs and check processing machines and point-of-sale machines both hacking and

securing them then I spent some time at Microsoft doing security engineering for as your transition into a consulting role with PwC did a wide variety of security work all across us then when you want a startup called financial force and now I lead security and privacy engineering at Anki for those that you don't know what Anki is we are a start-up based in San Francisco you know we primarily focus on consumer robotics and artificial intelligence and machine learning focused on you know vision and voice and other aspects that go into robots about 200 people team you know we've shipped about 1.5 million robots as of today some of the robots are in the toys and gaming and entertainment

business they were the best-selling toy on Amazon for the last two years and then a lot of work that we have done we have also open sourced and a lot of developers and robotics professions are basically using our tech to create more interesting robotics applications before I delve into the attack surfaces I just want to get a brief overview of you know there are different kinds of robots there are industrial robots consumer robots big robots small robots some are autonomous some are semi autonomous you know some have high degree of assurance some have lower degree of assurance but the bit we think about you know robots is that space is huge and different types of robots have different security

requirements different threat vectors our focus today for this talk will be home in consumer robots to give you a sense of you know where the robotics market and it is today you know robots are everywhere they're toys they're companions for the elderly there are applications for home assistance or healthcare assistance or companionship or you know manufacturing and industrial automation you know in about you know there are some stats about how fast that market is growing so basically we want to be at the cutting edge in terms of be the thought leader on doing security and privacy work in the robotic space

what is the punky robotics platform sorry yeah they were to think about the entire stack is you know a robot doesn't do much on its own there are a lot of different components and services both on device off device on the cloud in the applications that comprise for a holistic solution so we do a lot of the robotics functionality comes from the cloud you know things like analytics voice recognition the robot said we just launched today you know you can talk to it just like you can talk to Alex and Google home so there's voice activity detection and there's some other interesting cloud functionality you know from interface standpoint this is how you interact with

the robot it has touch sensors you know it can do perception which basically means it gets cancer with the environment it is in and makes decisions on what it sees and you know who he interacts with we focus and invest a lot in the character and the emotional e.q aspect of it and the reason for that is we want people to have a relationship with the robot do you want people to have you know trust in their robots and that also intersects with a lot of privacy interest as I said I'm going to talk about then moving forward from character you know a lot of SLAM which is you know the mapping and localization and when the robot moves from point A to

point B in your home you know all the vision algorithms and all the interesting a I take that we've developed and then basically scaling that to you know more character movement and manufacturing in other aspects so we also you know abstract a lot of this functionality in our SDK not all different blocks but for us the security models are a little bit different when you think about how the robot behaves in your home versus how a robot can be programmed as a developer and I'm going to talk about that as well this is an example of one of our home robots you know it has different sensors you can see there are a bunch can hear what you

say it can interact with you know people in your home you know it has some brain of its own which is some of it is local you know a lot of AI algorithms that we've developed and how the robot behaves and interacts is local on the robot as well as some applications are you know pushed from the cloud the way to think about the complexity is or robot comprises of a B this robot has like 700 different parts there are different sensors and there different systems you know there's microphone and there's camera and it's laser sensor and there's capacitive sensors and there's other you know wireless protocols that you know it enables and uses for communications but

in terms of complexity of the hardware you know it's it's it's even that if the size is small the attack surface both inside and outside the robot is pretty complex so I just want to give you guys an idea of you know when you start be completing the attack surfaces they're pretty different from your conventional applications talking a little bit about threat models you know the way to think about this is if you are protecting your applications or cloud services you mostly worry about application security data protection confidentiality integrity availability threats as it pertains to your product and service when you have a physical product in your home and can roam around and interact with people then you have to worry about

a lot of other aspects you know safety is a good example you know if the robot fails if it's bigger in size if it's providing some utility it can cause a safety risk you know it can be it can have an environmental impact it can create hazards there costs associated with you know different applications and functionalities and how do you think about optimizing those for a mass consumer electronic device Trust is a huge aspect to us you know robotics especially in the consumer space is very very new nobody knows what it would look like five years from now or ten years from now but if people don't trust that tech you know IOT devices today are

notorious for their bad security we don't want our robots to be you know treated that way we want people to trust our robots we want people to have an association with robots we want people to have assurance of if I'm buying a device if the robot is supposed to behave in a certain way and provide a certain utility I can have some assurance and Trust on that's what it does specially if it has you know these sensors like cameras and microphones and you buy and bring it in your home environment so plenty of things that can go wrong you know I just brought in some interesting examples of what has gone wrong and just Google or

YouTube for like robot failures and you'll find like dozens of examples I mean again you know not all of them are security failures but like I said form for security safety and trust perspective there are a number of things that you know you want to be able to basically give your customers a sure and so on specifically is a funny example if you go to the Department of Labor and just search for robot like literally everything is like somebody died because of a robot understanding attack surfaces so this basically you know the way to think about the attack surfaces you know that said you have a physical device just consider all the different ways it

can capture input it's not just digital packets coming in and out through the internet you know you have physical sensors you have you know firmwares and OS and applications and the cloud and the robot interacts with all these different components you know it receives different kinds of radio frequencies and signals you know capacitive touch sensors you know the world it sees through its camera the vision aspect the voice so there are a lot of interesting attack surfaces using those sensors and I'm going to talk in detail what those are some of the other aspects are also you know if you think about what signals it receives you know even the charging ports and the diagnostic ports are you know big attack

vectors if you have ever done hardware hacking you would know you know things like JTAG and USB and all those ports that people leave open that you can use to basically either debug or hack a device and then reverse the analogy to like what signals does the robot emit that you need to worry about you know if the robot is providing some utility you know in case of this robot let's say somebody just scans through outside your home your environment of what kind of devices does this home has and somebody sees hey here's a robot you know it could be a ble advertisement it could be a Wi-Fi transmission it could be some other sensor that

beacons out that somebody can scan and basically look for interesting attack vectors so they're interesting trade-offs between usability and security of what are you enable for how long you know I'll give you an example for example when we do pairing of a smartphone with a robot there's an ble advertisement that goes out but we only enable it if you physically interact with the robot so it's not like beginning out hey I'm a robot and I'm ready to pair all the time so you know hardware threat modeling this goes back to it's a little bit different from your software threat modeling that most people are used to in terms of how do you build attack trees and how do you do

data flow diagrams it's similar analogy but you know if you start decomposing 700 parts and how does each part interact with a different part and what are your security boundaries and where do you start your rule of trust and which sensor talks to what sensor and which bus is transmitting what kind of data so we do a lot of hardware threat modeling down to the sensor level to figure out what are the failure scenarios what happens if one sensor fails and the robot can't operate how it's supposed to operate and then what kind of trigger or action we need to take this is pretty interesting but it also goes you know very low level in

terms of how you typically do threat modeling of any product so I'm going to spend a little bit of time on this the voice interface so this is the problem with with voice enabled devices you know if you look at the logical access controls and the you know the digital realm of applications people are conditioned to use you know their authentication tokens or credentials of something that is tied to them you know you go to a website you enter your user name and password that's how you're sending it you can have a two factor auth how do you authenticate over a voice interface you know how do you treat with cases of like a mischievious

neighbor who can also shout to your voice activity device the same way you can or you know you're curious child who may just you know send some commands you know this is just a symbol example of like you know somebody keeps ordering on Alexa and you know the pattern of your home learns how you order elects and now it's basically talking to her let's sort of why so it's basically a very different model of authentication and interacting with the robot so should you treat like voice interface as like an authentication signal and the problem with that is yes you could add you know some step of authentication like hey if you're performing a sensitive transactional at a pen you

could do biometric out of like hey I can do voice recognition of this person versus that person but then this goes to you know privacy and usability concerns around the more granular data you're collecting off your customers the more creepy you become as an entity the more risk you gather for us the decision we make is we don't want to collect data that is not used for any product so yes we could do biometric or if we need to but that's not a place before me at least for now your other interests similar type factors like hey what is somebody just recalls your voice and starts replaying how does a robot detect you know replay some more interesting

hardware attack vectors are you know when you have different sensors like gyroscope you know your phone has a gyroscope as well you know accelerometers and different Hardware antennas you know gyroscopes are pretty sensitive if they have a really good antenna you could basically measure acoustic signals feed it into some advanced signal processing and apply some machine learning and you could actually identify speech these are you know still theoretical academic PhD attacks but the more sophisticated the sensors become the more plausible these attacks will be as well inaudible voice command injection this is another interesting one where you know human ears hear a certain frequency of sound the microphones can hear even what you don't hear so what if you can create

some voice signals which are Oh Sonic senator the microphone the microphone treats it as a command and execute it and these are pretty prevalent now you know you can see people have started attacking Siri and Alex are devices and Google home devices where you can craft these voice waves center to your device and trigger a command in the iPhone case that's basically dial a number or Center text and the more interesting research in this area is you know when you start applying all these advanced neural nets and deep learning algorithms this came out a couple of years ago it's called Larry Bird basically it's a tech you could give it you can give it like a

minute of your voice anybody's voice just grab a video file from YouTube give it a one minute of audio it runs it's classifying and training algorithms and then it can talk like you it's not 100 percent accurate today but it will give you a sense of how creepy this tech can become especially if you're using these things for like you know you know voice-activated biometrika Thor you know like phone support and if someone can speak just like you how you speak in the same tone with the same linguistic manner things start becoming really interesting so the robot makes it decisions based on the world it sees and its primary interface for that is the camera you know we have

developed a number of training models and classification algorithms that we feed to the robot and we store them locally and they're things like if the robot needs to go from point A to point B how does it determine the best path when it's mapping the environment where the objects are where the collisions are if it's a person if it's an object the problem with that is you know the way AI today works is you know you build a model you have some algorithms you run those classifications over some data and you train it or the data and you make trade-offs the problem with making those trade-offs is you know is this a chihuahua or a

cookie or a muffin if things start looking similar you can start defeating the model classifications you could add more you know accurate sensors you could combine other algorithms you can start tweaking parameters but these failures scenarios still exist the best example I can give you is you know in the autonomous car industry this is a big problem where yes they have the same issues around mapping and look you know training their algorithms to recognize different stop signs so once you train your model to recognize a stop sign you're basically saying hey this is what a stop sign looks like this is where the text will be but if somebody puts some text above and below the line now it's

basically thinking it's a speed sign instead of a stop sign and then basically your robot whether it's a car or a physical robot or a home consumer robot you are making decisions based on the world you see and the world you know the robot thinks it sees there's a huge class of basically interesting research which is gaining huge attraction which is called you know Gans they are basically deep neural net classifications and architecture is comprised of you know different neural networks basically attacking each other this this is another example of you know researchers falling even the Google's vision tech to recognize something which technically it's not the adversity examples are prevalent in in in differing systems these algorithms are

you know a lot of them are open source you know libraries from Google training data all over the Internet and that's what a lot of people are starting to use that are recognizing all the failure scenarios that come with them yeah in Goodfellow you know he has the Google brain AI research yes a number of interesting research papers on this topic for reference so manufacturing this again is a pretty interesting attack vector you know if you think about the applications that you know that we are a two hundred people company and we don't have our own factories and we don't manufacture our own devices we partner with some of the leading companies in the world you know

who have all this tooling around scaling you know a huge complex supply chain of you know 700 parts and moving them to more factory and manufacturing those devices you know in a scalable manner the problem with that is you know if you're manufacturing and you know hostile countries or you know factories which you don't have much control over their number of things in and out that ecosystem that you know worry us you know counterfeiting is the biggest one you know your intellectual property get stolen you know people start manufacturing your clones then how do you deal with basically and the more premium your product is the bigger the problem becomes you know supply chain compromise goes from anywhere from

ransomware hitting a factory to nation-states putting you know advanced malware in flash chips that you might bundle has storage capabilities in your robots and there's no easy way to solve these problems but because your robot is comprising from all these components is taken from different supply chain sources it becomes a big risk untrusted manufacturing lines this is another interesting one you know you have to basically trust the factory to perform some privileged operations in our case think about you know they get very low-level diagnostic tools which they can burn the secrets they can test electronics they can write firmware and they can also disable or enable certain capabilities and you have to basically give those people who you know are

building those devices for you have access to these you know privileged toolkits you know end-of-life components is another interesting one you know you have to make some decisions in terms of cost of what kind of you know Wi-Fi chip or a ble shape or a processor you want to use but in terms of you know consumer devices life you know people use it anywhere from five years to tens you still like till that stuff dies but if those components are end-of-life if those chips manufacturers are out of business how do you patch or think about security of those components it's a pretty complex problem and no easier to solve it and then gray market is you

know these are some examples of people basically putting up fake stores or basically taking stolen inventory or even downright copying our exact designs and creating things that look like us you know we are small we don't go over every single body but you know as a company it's a huge interest to us of how people are you know stealing or abusing our interest property so I'm going to transition about some trust model considerations you know these are some interesting problems you know especially when you have a physical device and a robot you know I give this allergy of if you had a web application or a mobile phone you enter your credentials that's how you authenticate

the user how do you authenticate a user to a robot especially if it doesn't have a swing on the keyboard do you enter credentials do use out your password do you make it recognize you with some biometric fingerprint and if you also have some logical access control outside the physical realm of the robot let's say a web interface to control the robot you have your username and password that controls a robot but then the robot doesn't know that username and password and how do you marry basically these two worlds especially if you start thinking about the developer ecosystem and the applications and the SDKs things start becoming very interesting of like what is an optimized trust model for a robot

you know which users will you trust you know the way we think about our robots are these are family robots you know you as an adult you buy a robot you bring it to at your home or your family and friends and I everybody who's around it the robot will interact with them so how much do you trust them they can give the robot some input what kind of functionality should you enable for whom those are not easy problems it's a matter of basically usability versus security versus privacy and you have to optimize for that then you know what kind of signals and anomalies should trigger let's say an adaptive off let's say we detect

something is not right somebody is abusing the voice interface and basically sending ultrasonic waves that we don't recognize like what should the robot do should I shut it down should it become notification should it cry those are interesting decisions then this is another interesting one you know our robot is autonomous you know we wanted to be autonomous in the sense that it's basically on its own it only basically responds to commands or actions but then we also have this really more of people want to write apps and there is an SDK so how do you transition from an autonomous world to a semi autonomous world or a developer world and how does the security model change if somebody is

actually controlling it through an app versus if it's anonymous on its own and what is the net Renault's behavior it could be a signal on the robot could be a signal in the crowd could be a signal on a sensor and those are not easy problems going to talk a little bit about you know basically some aspects that I mentioned about how much do you trust and where does your trust start in the chain you know we design our own electronics we design our own hardware but we don't make our own chips we basically use commodity processors that the mobile industry has been using but you know they have certain code running on their chipsets and often times it's kind of a

black box and I'll talk a little bit about what those things are - you know manufacturing facilities primary user users basically you have to give them certain control on how they access and interact with the robot so it's talking a little bit about how we think or thought about securing our robots you know basically we had some goals about you know confidentiality integrity availability cause trust safety and those were like our high-level goals and then we basically step down and drive security requirements and a lot of controls that we designed are basically optimized for you know the security goals that we have for our products so high level you know we basically start with we want people

who have a show and so on what the robot does so there are privacy indicators for different functionalities you know if the robot is streaming to the cloud if it's in listening mode if it's taking a picture the robot shows some clear markers like that green light that you see there are different visual cues that will guarantee the user hey the robot is performing this action co-author addition this is something dear to my heart you know what we want to do is this is a huge complex stack of code that runs down from the hardware ROM to the boot loaders to the operating system and the firmware and the applications that are on the robot we sign and verify

and check for integrity the entire file system so we want to protect every code that runs on the robot and every robot that we ship you want to have a surance on if it's performing if it's running if it's operating you know we can guarantee that yes you know it's it's our robot and it's running our code and we signed it and the robot basically verifies the boot chain and the code so you know in terms of code authorization basically it starts literally from like when you power it on you know the boot from basically verifies you know the kernel and the kernel verifies the filesystem and basically the entire storage of the robot so both confidentiality and integrity

guarantees here we also encrypt and sign our every single update little bit about hardware security you know we because of problems like counterfeiting and you know we want to authenticate the robots talking to us we basically generate crypto certificates and we burned them in the robot at factory time in a hardware key storage it helps with many interesting cases like fraud and abuse and counterfeiting etc there's some temper resistance IDs that we use for fingerprinting the device and you know again if they're not cloned or you know they're not the robots that we manufacture etc there's a hardware back key store which is used for key derivation so when we think about aspects like protecting data on

the robot whether it's photos or the Wi-Fi password or other data that the robot stores locally we encrypt the hub data partition but basically the decision of keys are derived locally from the hardware key in a trust zone which is running on the robot so we try to keep our cells and our cloud and our control out of you know this q derivation mechanism gives better so privacy guarantees is you know a better security story as well and you know we have a huge you know hierarchy of keys for data protection and then we also think a lot about how to be harden the hardware itself so when we release our production robots you know we disable

all the privileged interfaces that you know commonly people used what I higher base products like you know you are in JTAG and USBs and SP is the problem with these is this is often like a one-way street you know if you disable them and you burn fuses and or SOC there's no easy way to reverse them because yes you disable all those things in production so even we lose certain capabilities as a vendor this is just a picture of like the trust zone that I talked about you know in the hardware there are capabilities around what we call trusted execution environment similar to you know how some of the Android and iOS phones undo Hardware back key storage

and key derivation and privileged operations like you know passport decryption etc so what about physical security I mean this is hard you know you'll see people like you ship product people will take it apart you'll have a blog on iFix there they'll shame you in terms of hey they don't have any physical security the problem is physical security is hard there are different physical security requirements if the robot is performing your heart surgery versus if it's a toy robot so it's a very hard trade-off to make you know what's a good enough physical security that question is not easy to answer you know you have to balance between cost of a mass consumer electronic device versus you know

customer expectations versus serviceability you know you could make a really secure hardware and add a ton of tamper resistance but then you make you know it harder for service and basically harder for you know playing in different parts they're also challenged about you know interesting property protection you know people hack your you know devices and they want to get to hear all the interesting tech that you've built and burn on the robots and then you know like I mention the different requirements for if it's a consumer device versus it's a commercial device and you know oftentimes the requirements drive the the capabilities of the robot drives what kind of physical security you want in the product so another area

which we did a lot of interesting work force around application and device pairing and you know the issue there is you know a lot of devices have these notions of catered over Wi-Fi or paid over we and you pick up a smartphone and you enter your credentials and your application discovers the robot or the device that you have and some some a lot of manufacturers rely on the Bluetooth the problem with using like Bluetooth as itself is that whole space is broken there are tons of remedies in the stack in implementations and the chipsets so we designed our own you know crypto mechanisms and protocols on how do we do key exchange how do we do indicator

encryption how do we do bonding some of those bonding mechanisms require physical interaction that I spoke about like if you have to pair a new device you have to go physically use the power tap a button only then it opens up its pairing mode and it also pairs and makes you know marriage itself if the user who is the contour of the robot and he authorizes yes I'm allowing a new device or not on the OS and former security I talked a little bit about the trusted execution environment this basically is privileged code in operations which are executed inside a secure Hardware processor not in the user space land or the kernel space you know we remove and

harden our operating system to basically remove all the debug interfaces like SSH and a DVD and fastboot basically making attacking and debugging harder we focus a lot on what are the unnecessary services that we have to remove and how do we reduce the attack surface we also do some engineering on the OSF on website to basically run these applications and process especially the ones that talk to the internet or talking to a sensor to run as least privilege and continue rising and isolating them for you know better security guarantees then you know limiting you know that for exposure again not advertised the services that are not needed and you know we encrypt the data partition on the robot so even if

somebody steals your robot you still can't get to a data without like really going and breaking trust zone and then file system integrity verification is yes you know the way this works is you basically sign and verify every block of the storage and even if somebody tempers one block basically you know it fails and then a lot of work on hardening of the actual operating system and the kernel you know around you know stack Henri's and you know basically recompiling your apps with certain flags which can harden the winery's for different kinds of exploits which are pretty prevalent there are a number of challenges and trade-offs you know for us the biggest issue is should we go and

use a mainstream OS like linux which is well supported or Ruby for a high assurance robot do you go and take a high assurance more secure OS the trade-offs you have to make is if you go for a high assurance OS there's not a ton of support in terms of the tooling and libraries and all the other applications you want to write so you know you have to basically work with some commodity off-the-shelf operating system you know I talked about the longer life cycle of consumer devices you know you ship our consumer electronic device and a robot they weren't gonna use it for 10 years your clock is ticking you know Windows XP was secure when it was launched and look at

where it is today so how long do you patch your products how do you patch things that you don't have control over especially the end-of-life chips and the drivers and you know things that are outside your control you know there are there are interesting attacks on like foundational platform security aspects you know what happens if your processors or your platform has issues which are literally things like you know specter and meltdown or you know crack whatever it is in the Wi-Fi nothing you can control or do it's basically some in the protocol or some linearity in the hardware design itself those are very hard problems and then the last one which is a very people who have spent

their time in the Linux world or Android world or embedded security world just tracking CVEs and known exploits in different linux variants and kernels it's just a huge nightmare so cloud security you know because the robot talks with the cloud you know not only the robot authenticates the cloud we also authenticate the robot back it's a real robot it's running trusted code so we do mutual TLS for all of our services we use modern you know crypto algorithms you know if you were using your laptop and a browser you got on the internet there's a key store in your browser and your browser has a way to verify the SSL cert chain the way we think about the

key store on our robot is it only trusts our cloud so we basically remove pretty much all the major CAS and all the other certificates so even if somebody hacks and even if somebody tries to man in the middle and put some certs and proxies we basically want to make attackers life harder but the intern basically is a robot only trusts our cloud it doesn't trust anybody on the Internet so authentication you know code signing and stuff you know forum availability aspect you know we think about which services are really critical secure update token management account service authorization service really thinking about how do we scale these for millions of robots all across the world in

different geographies then a lot of tooling and capabilities we have to work on basically detecting fraud and abuse DDoS is just one example but how do you detect anomalous behavior coming from robots not users you know we basically designed our of tooling for our own C and PKI there are some interesting challenges on the robotic side as well as better hardware the first one is clock you know to verify SSL and TLS certs you need a good reliable clock if your phone's clock went back two years suddenly everything will start filling so if the robot doesn't have a clock and you power it on how does it trust the variety of that search it's not an easy

problem so you can either allow in signature or you can rely on time and we rely on signature not time because the device basically powers from the genic epoch time of 1970 it has no idea of crock when it boots up the second burger talks about you know what kind of capabilities should you build you know when the home DNS is insecure when the time service is insecure they use UDP the content every networks which are not in your control for like software distribution and how do you think about securing those aspects we use some interesting protocols which are newer like gr PCA does HTTP - it can do both there should be - and I should be one in

like single call but then the tooling and capabilities of TLS libraries are not there yet if anybody has ever managed and run HSM in the cloud of how do you do key management distribution and the kind of limitations that HSM gives you from even the major cloud manufacturers that's pretty amazing even in this day and age and then the last one is a funny one when you do all this work around confidentiality and cryptography and security and guess what you know you manufacture in China and the factory can't talk over those crypto channels back to you because something failed in the Great Firewall of China so what kind of environment do you need to create in that other country to make

your product work or test during manufacturing those are pretty interesting challenges and we're talking a bit of privacy you know the hard privacy issues are if you start thinking about this notion of yes enable the user give them better privacy it also enables a ton of abuse if you can't track where the traffic is coming from which the user is what kind of things he's up to the second challenge is the way we think about management - robot is yes if this was a commercial device and you were selling a service to a customer let's say a robot projector you want to build capabilities to manage those things remotely so you know device management and your more capabilities of you as a

vendor how much control should you have directly on the device versus the privacy aspect of once you have that control then totally some three data regions you can come to your door and say hey why don't you put a back door in it why don't you tell me yet about this customer versus that customer and we don't want to be in that space so we have to make interesting decisions around you know how do we enable you know privacy for the customers the third period about privacy is you know for AI to be useful you have to collect a lot of data but the more data you collect the more creepy you become so how much

data can you keep local how much data do you stream up to the cloud and how are you anonymize that and how do you use that to basically train and make your products better those are also pretty hard decisions the fourth pillar talks about you know we have legitimate interest in providing customer support people call us and say hey my robot is not behaving this way why talk to it I gave it these commands and you know it's not working so we need to give our own people capability to like hey if talha calls you know you should go and be able to look at the failure and tell the customer this is why it failed but if

you build that capability then you also have insider threat of like somebody can abuse that thing so those are like in very hard choices you have to make of what do we collect even for our own legitimate support scenarios and then conflicting privacy requirements you know we sell in Europe he said in America you know if you've dealt with Coppa or GPR they have pretty conflicting requirements in terms of the right to be forgotten versus who can ask for deletion of radon can provide parental consent that's just a broken space and the father bird head so on you know what data do we keep local versus whatever to be keep on the cloud what we do as a company you know

we took a decision and we basically said we will not store any voice data in the cloud all the commands that the user said tells the robot says to the robot we basically take that voice stream we understand what the user wants it performed action and we think we throw away the voice we don't have any record of what anybody told or said our robots and we do this deliberately so it reduces our threat surface it's also better privacy story you know if you get hacked we don't have people's voices we don't have anything to turn on to anybody you know this is the privacy ques I was talking about you know there's a very

clear visual indicator and a privacy signal when the robot is listening and streaming up to the cloud we do collect anonymize stats which basically is how our customers are interacting with the robots which aspects are more popular and those are not tied to an individual user or robot we are mostly basically to improve our products we just curious about which functionality is more popular than the other or which things fail that people want that we haven't built yet customers can always opt out even off the anonymize users an analytics certain capabilities and you know because we use cloud providers for various things we only partner with entities that can comply with our legal and data retention requirements this is

a really interesting one which again goes to this usability versus privacy aspect of you know the keyword and the detective 'ti signals that you give to the robot in terms of you know you say hey vector do this those trainings and those classification algorithms to make them more accurate you need more data we don't use our customers that are for training we buy this data off the shelf which is completely anonymized but it does have some you know implications as well as impact on how accurate your voice capability is but that's a decision we made as a company of like we will not use our own customers data for training our voice because we throw away the

voice data and then we encrypt all data in transit as well as on the robot and the vision stomach for voice we have to stream it to the cloud because the tech of doing AI locally on the robot is not there yet but on the vision side that thing as much or a lot so all the training models and all the vision stuff that the robot does is all local on the robot we don't stream images we don't stream video to the cloud and although biometric data of even the people that it recognizes those mathematical fingerprints are basically stored and organized locally on the robot that is it you know any user is in control you can always just hard click

hard press the robot and we'll go back reset to the factory state if you want to set it off or return back to us that's all here's my contact happy do you have about seven minutes so happy to take any questions sir Oh speedy you mentioned there were challenges like I think you call that the dolphin attack worries that seems like a very simple low-tech way to get around something like that is simply to put a bandpass filter that goes to only human the human range of things so nothing can go up to your cloud for analysis that isn't a human voice at least in that range yes so we have capability group detected because we

write our own microphone formers a lot of devices they basically use off-the-shelf tack which is already developed and they don't have that level of granularity so we can totally detect if it's not our voice frequency that looks like human or sounds like human but even then you know the early voice after detection that's local so when you say a trigger word and when the robot recognizes that trigger word is only when it starts streaming to the cloud so in the Alexa world let's say Alexa in the cover where it says hey Google for us it's a vector so we only stream after we have recognized the trigger word so we don't stream anything before that

anyway hey great talk Thanks um so on your hardware security slide you mentioned you had some cost constraints you have to balance security versus cost in some cases what are some of the things that you had to out of scope because of cost consideration and rephrasing the question if there was no cost constraints what would you have done differently yeah a good good great question so I think the way to think about this is so like let's start with SOC for example your your main stream processor it's the same similar processor that you use in your mobile smartphones it's made by Qualcomm you know the 64-bit processor with the CPU and a GPU and emmc and it has some

interesting trust owns that capabilities it's a smart phone it says the processor from like three years ago right it's now way it cheaper than when it falls that long so the newer processors have more advanced capabilities but like that it drives up the price for the functionality we have today that processor is sufficient that's like on one one aspect the second aspect is you know ble sensor what you buy a five-cent ble sensor or a 50 cent ble sensor when you know nothing about what is in that sensor it's a black box firmware somebody wrote somewhere in Taiwan and China and it's commoditize which literally everybody else is using so yes if you had no constraints you could

design your own ble sensor you could write your own firmware but that basically you know raise the bar there are tons of components that not just us like everybody buys and uses where you know you kind of trust what comes from the supply chain but then on the hardware side again I think because it's a toy in entertainment toy it's it has a plastic body I mean yeah there's no like theorems of like tamper resistance and evidence there are no seals you can open it up and you know put another camera next to our camera if you want it to bug the device but yeah you know we could make it hundred dollars more expensive

and add like layers of physical security even that is not perfect but it raises the bar so you kind of make these decisions for price point based on you know what the product needs to do and how premium it is any other questions okay a big round of applause thank you very much thank you [Applause]