← All talks

I Am The Cavalry Panel: Progress on Cyber Safety

BSides Las Vegas · 201635:5060 viewsPublished 2016-08Watch on YouTube ↗
Speakers
Tags
CategoryCommunity
StylePanel
About this talk
A panel discussion featuring security leaders from automotive, enterprise software, and medical device industries on bridging the gap between security researchers and product teams. Panelists share experiences working with vulnerability researchers, emphasize the importance of demonstrating impact through technical proof-of-concept, and discuss strategies for embedding security into organizational decision-making by translating technical risks into business language.
Show original YouTube description
Glaying out the Blockchain Ledger for Fun, Profit, and Hip Hop - Andrew Morris Ground Truth BSidesLV 2016 - Tuscany Hotel - Aug 02, 2016
Show transcript [en]

gets here he'll start I don't have a mic but my voice is pretty can everybody hear me I project pretty well usually all right so I'm Frank Berman with the United States Department of Transportation national highway traffic safety administration um I'm a researcher I do Automotive Research I've been doing it now for 23 years um I'm not a hacker per se oh I don't have to scream now thanks B sure um but about uh about 10 years ago we started seeing some problems and I basically made a career out of making things do stuff they weren't supposed to do so you know we needed to do research in cars so we needed sudden acceleration to happen well I was the guy that made

that happen needed a VCR to record automatically probably don't some of you don't know what a VCR is but uh you we were able to go in and make that VCR happened when the car start to start recording so throughout this I've made a career of doing that and now at this age of computers ecus can buses networks v2v uh and all the interconnectivity it became a real priority for our agency to make sure these things do not happen in the future so that's just a little bit about my background electronics engineer all right Sasha hi I'm uh Sasha Dean I'm with Exon Mobile Corporation um I'm primarily involved in uh application security software security but in a

broader sense cyber security as well um I uh I represent a portfolio of several thousand applications and uh have a real passion for securing things like Erp systems I don't think they get enough uh attention in kind of the mainstream Community I think we spend a lot of time talking about web apps and e-commerce apps and kind of the stuff that's pretty easy and out in front but there are a lot of things where uh corporations store their most important information that makes the business run things that store you know financial information crown jewels things that run very proprietary optimization models so uh I have a big passion for securing Erp systems and finally Michael so Michael

McNeil I in charge of the product security program at Phillips or Royal Phillips now um that responsibility is all of the medical devices as well as consumer items that Phillips sells in terms of Sonic toothbrushes and Nelco Shavers and all the way up to you know Mr machines and CT scanners um my team has the responsibility of making sure that we have Secure Solutions and through our development life cycle for those products um prior to that I also was the um security officer at metronic so I have had experiences with just about all of the faces that uh that that Karen just showed um had worked directly with uh with Barnaby Jack as well as um

reporting from Jay Radcliffe um from those product areas um have dealt with Scott Irvings and you you name it um so I'll talk a little bit about some of those experiences and alignment with with um security researchers and kind of my philosophy and and sort of next steps in that Evolution all right so actually why don't we start there uh since since nickerson's not uh maybe going to join us today all right um what have been your experiences with researchers and particularly not just kind of a point in time snapshot uh but broadly throughout uh the last few years have youve been dealing with um products and researchers so it's um it it's been a real I guess

interesting ride or or a little bit of a of a roller coaster I've have been fortunate enough that when some of the the researchers have approached um the the corporations that I've been with you typically get your you know initial you know oh my God what's what's happening what are you trying to do and there's usually um not the right point of of interface and the connections and so they get the runaround which becomes a very you know frustrating type of of a of a scenario they also get you know as much as they can with the legal and the probably the communication side they probably you know have been given the old Heisman you know in terms of the the

the stiff arm and as much that that that can be done and so I I tell organizations you know that's the very first worst thing that you can do um if you want to see a recipe for disaster do those types of of activities um but it sometimes takes that that crash or burn for people to really get the appropriate understanding and for you know you to get the movements that you you need to put in place so that Evolution at least in the organizations that I've I've been involved and and currently are at you have to have an open door particular um you know policy and positioning that has to be communicated and understood at the

top of the organization um I'm at a level where I get that appropriate visibility and I get that Communications so that there is that better understanding um as part of our Evolution they know where to come you know there's one spot that comes in the door I have a team that's aligned to manage you know that level of communication any researchers that come in through that door effectively they will get a response and a Communications directly back from me it's not pwned off it's not you know um geared to someone else it's my job and my team's jobs to find out which sets of products or solutions that this is involved and then for us to do the appropriate analysis

jointly in validating that information you know from the the researcher and um from that research community so once we have that dialogue and and awareness going and the organization understands that then you can manage things a lot more effectively and that's what we've seen to you know at least for for our experiences um the floodgates have not you know opened you know over you know swamping because we have you know an open program um it's been publicly communicated at least when at Phillips when I launched it in um November of 2014 and the reactions that we've had in terms of some of the the research um worked with and and dealt directly with our baby monitors so you guys did know

Mark Stannis sloff in in Rapid 7 you know we jointly had the conversations it was a product that Phillips wasn't even manufacturing and maintaining but it had our brand and our recognition on it so I followed it through to make sure that it had appropriate patching and and it was updated just as if it was still Phillips with the the Gibson Innovations which was um care feeding for that last year we've worked with um and and the other part of my evolution is once you start working with with the researchers you then also want to build in that as a part of your overall program so majority of the people that we have worked with

or that have brought you know at least in the medical device um for our products issues to us we've been able to bring them in as a part of either my training or awareness so a number of the researchers when I bring my product security team and my development arms together they also have come in and spoken to our team needs and been able to enlight them on that handshake and how to work together you know from a Cooperative perspective so that's also been a a key you know learning um Factor we've also leveraged them in additional either verification or other testing in terms of um where some of their passions are as a part of our development

processes so again I think if it's done correctly for those organizations that do want to have have a passion and in our particular case yeah I this is about you know the very first slide or anything I ever talk about at least for the majority of of of our experience I deal with with patient and people's safety and so I can't have an organization that has an ego that becomes too big that that's not the the priority so they have to leave that at the door but it's my job to make sure that it's communicated you know effectively through our our our business as an example so now that uh we've got Nick Nickerson in the room um he will be our

moderator I would read off his bio but uh it was hilarious you should absolutely read it yeah if you haven't seen it yet you should go check it out we uh we crowdsource the bio on Twitter and if you ever do that um I highly recommend you take everything you get from Twitter and just feed it right into the bio will be perfect it was awesome it was yeah uh that might not be hot maybe you have to turn it on T oh there we go it was great technology yeah there we go technology okay so why don't you take over and carry yeah you know so so many awesome things that you were talking

about that I feel like in the winds that you guys have had separately there's a ton of questions to be asked not necessarily as much about what the win was but how you want it and to me that's what I think some of the fascinating things are and I I I feel like a lot of that is really driven by what you get passionate about I I I think that very rarely in any of our careers do we pursue something without passion and ever make huge exponential pieces of success um so one of the things I'd love to explore with you guys is really what are those passionate moments what are some of the things where you actually

had the switch turn on and you said this is not only what I'm going to do or why I'm going to do this is how it turns me on this is how it it fulfills something for myself and my role and for the company and those things join together because I I I feel like there's a huge gap often where people just go and do work um but you guys are doing work that is very very very different and you're accomplishing things that are extremely hard or even to the point where people say I I don't think it could be done in my company um so there's there's hope that exists there and I I'd love to be

able to hear some of the different stories of kind of what what are those hope moments where where did you find the the strength or the support and how did you nurture that into the successes you've had well I'll kick that one off so um this is a story I think Josh wants me to tell but uh when I first met Josh and found out about I on the Cavalry I had a kind of a realization that uh over the course of my career most of the stuff that I had done was Financial stuff so it helps make companies at least as profitable or help them not lose as much money but the first day on my job as an

infosec person uh I was at a hospital and I walked into um a situation where they were having the natal Intensive Care Unit fetal heart monitors down right so uh Physicians had to spend extra time nurses had to be around and there was patient care on the line as Mike was talking about um and they've been basically infected with a piece of malicious software that had worked its way around the hospital um so calling the manufacturer they were unable to do anything because it was a medical device and so they couldn't modify the medical device to remove the modification from the medical device I'll let you wrap your brains around that but uh the second thing that I did was went to the

executives in the organization and said look we need to fix this problem because these devices are broken simply replacing them with equivalent devices we'll get them the new devices broken as well we have to fix the underlying problem so with a justification from hospital Executives uh used Metasploit to pop the Box um kill the malware drop the patch bounce it and it came back up and the doctors were able to get back to doing doctory things I don't know I left them alone after that but I had a realization that in the last 10 years of my infoset career I'd never had as much impact as I did that first day right which was was just awful realization to

have I've essentially gone down my trajectory has gone like that um so I said I've got to flip that around and so today I regularly have moments where I feel like I'm having wins when I talk to policy maker when I talk to somebody else in industry and you just see that light bulb go on over their head not because I've beat them over the head with technical stuff but because I found a way to social engineer my way into their language and to get them to realize oh that's why this matters uh and so I have those moments a lot more often now and that's what really drives me and gets me passionate it's awesome about the rest of you

guys okay I'll um like I mentioned in my intro I'm extremely passionate about securing our Erp systems and where that's a challenge if you participate in that yourself um the industry averages that about 2% of the issued patches and advisories get addressed because those systems are designed to largely to be untouched unpatched you know un unmodified you know of the CIA Triad a is kind of the king when it comes to Erp systems um and you know the journey that I went on with my management I I said well you know first let's discuss um what makes Erp special right and it's things like well it's where by far your most important information is in your company

okay that makes it pretty special it's also supported by incredibly specialized teams because it's incredibly specialized technology um it's also written in languages so if you take one vendor example right sap it's written in languages that no one knows or very few people do in Sap's case it's abop right who knows show of hand how many people here know abop one excellent that two that's about right a right right yeah yeah so the the polling in here is about right right for the industry so very few people know it um tools it doesn't integrate with any standard security tools right it barely has any static code analysis really none it has almost no Dynamic of any kind that you can

safely do it doesn't send any Telemetry to Splunk or arite or any of those tools it it's just not designed to work with other security tools um what else makes it special organizationally it tends to be operated by different teams because of how much knowledge you have to have about the business that that Erp supports generally the people who support those systems are also mostly business folks or kind of business plus right business with some it so that's what makes Erp special but what makes them very attractive to attackers and it's basically the same list right it's highly specialized language that no one knows most your most important data is there doesn't integrate with any sec

tools I mean it's it is the you know in my opinion it's the next big area for attackers to look at and we have some information that suggests that's already starting to happen right if you look at two three years ago when USIS went down the root cause analysis was it came in through sap um and and and the vendors are starting to get wise to that right they're starting they just randomly open ports to the outside too they just like oh we sap whatever and well if you look at so that's a great example if you just look at the standard hardening if you just look at the standard installation guidelines they tell you these are the

ports we need except it's not ports it's ranges and it's like okay we need 33 655 it's a little bit tighter than that but not much right we need 3,300 33 through 3400 I don't know is there anything in that range that might be problematic if every endpoint had open right 3389 and some others so you know it's things like that where they're also trying to catch up uh with security so in that one particular vendor example they've issued 3600 security patches in the last last 3 years to systems that are designed that you spent the last 20 years engineering processes around preventing changes in those environments I have a quick question for you yeah was there an aha

moment where you were like that's my crusade like that's the like I have I have looked at all these other things but was there a moment where you were profoundly taken back and said I I have to not focus on other things so that I can focus on this yes but it might be a bit boring and that that aha moment was was when I went to a number of you know industry conferences Bim framework you know conference all these other places and I started asking people do you care about Erp and I almost never got a yes and so the reason I felt very passionate about this is because I feel like no one

else does so I actually don't like Erp systems at all I hate them uh yes you know people tell me oh you should just kind of like become the Erp guy and like everyone's got their Banner I'm like I really don't want to cuz I hate these things I hate them like they're ugly you know their interface is atrocious it's not a sexy technology to be in but I don't feel like anyone else cares so that's why I'm doing it so you love your hate yeah exactly that's my story I think that's beautiful though I mean that's that's awesome so quick follow to that before we go next um was there a moment that you felt

beyond that where you said this is my crusade and then you took that to the workplace and got the workplace to support your Crusade instead of you just waving the banner sitting at a conference by yourself going care about Erp and everybody's like we don't know what that guy is like yep yeah so you know I I would describe any company's relationship with their Erp vendor regardless who it is as you know an incredibly bad marriage with an incredibly High Cost of divorce so you just don't do it right so you stay tied in with that and with every passing year your technical debt Grows Right you add 11,000 more customizations so you what what helped is when I went to my

management and said look you know I'm not going to mention the word security or vulnerability or risk but um H how much money are you willing to pay tomorrow for a bad decision from yesterday so I basically use technical debt as a way to drive a lot of the security discussion and and it actually worked very well luckily I'm in an organization where Safety and Security are non-discretionary so we don't have to worry about prioritization and is it above or below the line so if you you know if you make the right case there's there's very good support in our organization yeah right that's awesome thank you that sure those things are really important to me does that

resonate with you guys like th those pieces that where people can get empowered by their passions and then actually turn that into work yes awesome yes we can yes we can tell me more yeah I guess apologize for earlier I'm interrupting you now I will send you an Erp Pony in FedEx okay sounds good

you know last year she crashed our panel too I love it though I feel like it's just the thing that happen audience participation I'd rather her crash than other people so I'm good that we first met chras yes ma'am it sure as hell was I told you to take my seat cuz I was done which I which you're like seconds away from that's that's why all of these the white would say participant not attendee not just dude who shows up or gal who shows up but person who gets involved that's the spirit of all of this yep so yeah I just wanted to add on to that I guess so me being a transportation researcher working with

the federal government my job is to save lives basically There 35,200 lives lost last year on our nation's roadways okay right now there hasn't been though uh as Josh I think was saying any consequences we don't know of any in the wild you know cyber attacks that has caused a fatality directly on America's roadways here um what excites me though what I want to get at is something Karen actually said so she's still here I was glad um was you know government moves kind of slow she had a slide up there and um what's exciting for me is we're actually starting to change we actually see that now um you know we've talked with Josh and had different meetings

we've talked with other people who in this room not just working with the the big three car makers or the car makers not just working with academics you know working with small research organizations we're listening we're trying to change because traditionally we are reactive we have to wait for a problem or The Burning River to happen and we understand that's not going to happen we do move too slow for that matter of fact a lot of companies move too slow for that that's why there's patches every week for stuff so you know I do appreciate that and I do think that's something that's excited me and my team now is that our Administration what we're doing now is actually

recognizing that and we are trying to change trying to make a difference and we want to keep that right now with no consequences as much as we can in the future was was there was there a point where that for you you kind of got executed into into work like was there a a thing that you did to get them to know it or did it come from aan high or was it like how did that get sticky so I should give the the truthful answer or the government answer um I mean we could turn the camera off I'd rather them know the truth let have the truth I'll say the truthful answer it's pretty

effective when you can take a high level politician uh take a policy maker uh take your boss into a car that's doing 45 miles hour down the road and from your phone push a button and have it steer to the right demos it's uh yeah demos are very very effective um it uh you know it is a dog and pony show yes but I'll tell you what you can read about it or you can actually sit in the seat be on a a test track and do this um but it really wakes people up and when you see that potential for this to happen you know you start to listen and you start to understand and um you know

these are not easy problems to deal with but we have to awesome thank you one exciting thing uh on that you mentioned 35,200 lives lost on the roads and one of the things that I know that the government is racing towards as well as the Auto industry is autonomous vehicles the simple reason being that 94% of those and these are stats from your ball correct right uh 94% of those are from human error so if you can take the humans out of the loop you could potentially save tens of thousands of lives and what equates to basically 80 to 100 deaths a day on us roadways so uh one of the things when I was talking to

an auto automaker in Germany they said Bo we're German we move slow so that we get it right so that we get the security and safety stuff right I said great while you're moving slowly there are fatalities on the roadways that could have been prevented if we had maybe abandoned some of the security controls so I'm not advocating for going the other way but either side of that spectrum is wrong right because it will cause death we have to find out what that Middle Point is and maybe abandon some of the things that we think we should do out of you know muscle muscle memory or reflex like oh put antivirus on it or long passwords right maybe we

need to rethink what that is and find a different way that could also then if we find a way to do it in cars and medical devices Etc maybe we could retrofit some of that to the it corporate Arena and have fewer failures there too last one passion to progress so it it might sound kind of corny to you but you know I've always been you know a boy scout I like fixing things or making something you know um happen the the right way and I guess my passion or my aha moments are in order for me I know that I have the ability to to fix it and and make that difference not just within the

organizations that I operate but even at the larger scale so that's why you see and and those that know I'm out there you know I I you know my word is my bond and that's how you know I'm I I'm able to execute and I hope that by bringing others that have similar responsibilities I help Elevate the entire hole so that's kind of kind of my my hit man and so in that and I'm stupid so I have to take notes for things um in that uh was there a moment there where you got to bring aha up the food chain and say I'm I'm taking this on and and and they went from uh no one's doing

this to yeah you're taking it on we got you yeah when and again back on the whole the the demo piece by bringing in you know the dark side in into your you know in into your offices and tearing down the walls so that people there is no Dark Side they have the same objectives and goals that that we do that was sort of some of that aha moment within the organization and so I needed to bring them in we needed to demonstrate you know directly on you know your babies are ugly here we got to clean some of these dogs up and so you know when you do that and you do it in a

in a you know compelling way in your organization you know it means a lot and and it's kind of and not just for us and and for you know for what we do but you know I think I remember um talking with Billy Rios about you know one of his last um pieces on the pump and until he actually uploaded and and and showed the FDA the demo of it that's when they had the the moment and they got it he could talk about you know the compromises and the vulnerabilities and what was discovered and you know until he got you know blue but you know they weren't going to get it until they could

actually you know see it and and smacked D in front of their face so keeping it smacked in front of their face is kind of how I try to and keep I think there's also a second important lesson there which is he demonstrated it to them not Dro OD day on stage right right so one of the the things that I've observed is if you go in and talk to people in a trustful clueful way with respect on both sides then they'll listen to whatever you say if you instead put them in a position where they're surprised and have to react they will probably react in a way that is opposite to what you want um so breaking down that wall

between people who are both trying to do the right thing and bringing them together is a really really critical part of doing this I think that's awesome so the again I'm very simple so I I distill those things in the way that I hear them right and to me it speaks to me everybody's story speaks to me in in a different but really congruent way where we have demonstration being one of those kind of things that helps you Break Down The Walls um but there's also some really poignant other things I think um you know when when you were talking about the Erp plan and being able to bring that to a financial means that's

that's changing and translating the conversation right but the real key of that really wasn't Financial the key to that that I heard and and I and I can see in some ways is that you reframed the context of what was going on to their language you said if you care about Safety and Security I'm going to put this into the bucket that you care about so that I can be passionate you can be passionate and we're on the same goal on the same path and I think that that's a huge thing for all of us to learn from um in the beginning Beyond demonstration what I got was this uh kind of pride of ownership right my

words my bond that's what I do I say I'm going to do it and I think that that's another really really powerful thing for all of us to to start looking at and learning from to say if you're going to do something and you do it with Integrity that Integrity is going to gain you the type of political capital and the type of flexibility that may not be present it may you may have to work 10 years to get your integrity to the level of that where it's every single time they say they're going to do it they do it and and then you'll have you'll have those pieces and and the demonstration piece is beautiful because

demonstration is actually really hard demonstration is so hard because you're trying to figure out how hard do I hit somebody with the Nerf bat to like annoy them enough to know that they could have got hit with a steel bat but not hard enough that they're mad at you for hitting them with a Nerf bat and that's a real kind of delicate line to play and so being able to do that and execute on those things I think takes the Integrity I think it takes the pride of ownership I think it takes where where with you I see this impact it takes I am driven by the amount of impact that happens and the more impact that happens the more

fueled I get about it the more passion I put into it the more impact happens after that and it's this Fireball that goes down using each one of these things that you guys kind of have some distinct views on together to break down all of the walls and to change what you love doing into what you're doing and if I can just add to your comment I think one part that you mentioned I want to emphasize even more which is you know this may not be maybe the most popular thing to say but if you want to influence your organization you have to start speaking their language right uh you you can't do a backflip every time

you find cross-site scripting I'm sorry in the grand scheme of things it just doesn't matter it really doesn't it's it you know whether to fix that or not is just another business decision just like whether it's to upgrade a valve fitting or whatever you know build a new campus it's just another business decision and um I think we will have far more respect you know in our boardroom and with our Senior Management um when we can speak more their language which is around you know business prioritization Forex adjustments you know depreciation resource Capital around the world I mean that that's how these decisions are made and we can't continue to be kind of odd stepchildren in that discussion right we

need to start being active participants in the business discussion y um lastly of my interest because I'm selfish and I get to moderate so I can ask you questions uh how do you survive the fatigue because you guys are making profound changes in macro industries that have never been able to make these kinds of changes you're breaking down huge walls that people aren't even willing to scale and breaking bricks is tiring like what do you do to stay in the game I used to say drink I don't say that uh I don't know whether it's because I'm getting old or whether my fatigue uh outlasts my ability to drink um but now for myself I'm I'm a

introvert so I just like go into a quiet room and sit and maybe listen to some music and spend my weekends like not leaving the house which is maybe not the healthiest thing but if you've been in DC lately it's like the temperature here but with also 100% humidity so it's not really fun to go outside uh but but I know other people deal with it in different ways um and I think the the uh condensed answer would be whatever recharges you go do that what about the rest of you guys because I imagine y'all have like secret tips that I need to learn from because I get tired quick well so for me I mean it's not

fatigue yet um I mean Automotive cyber security is kind of an emerging field you're starting to see companies now have you know cyber directors report to CEOs you're starting to see companies making changes in their designs you're starting to have companies come to at least us the government and say look we realize this is a problem and here's what we're doing about it and we want to tell you because we want you to get smart too um you know so right now it's not fatiguing I mean it's really an emerging area right now um you know uh back to Karen government slow move slow you know no change but you know this area right now because of automation

because of vehicle to vehicle communication because of higher-end audio systems in the cars you know this is all the new frontier 10 years ago let's go hack the car okay that meant cut a brake line pull the throttle cable maybe sever you know a steel belt on the tire and 100 miles later ha haa you know now we have the the cars are equipped with I'll say the tools to allow them to be compromised or to allow them to be vulnerable you know you can't take a 57 Chevy and steer it to the left at least from a computer um you know so for for me at least um and at least my team we're not fatigued this is a new

emerging area we've set up a new division um I have a new lab um and it's very exciting for us so so um that's we haven't hit that point you know what good I mean in that and what an opportunity for people who feel it to look at your industry and go I need I need to go somewhere where the fuel is there's also probably a lot of people in this room who can help fatigue you if you want we can get you there don't worry is this the whole like lead a horse to water can't make him drink we can make him really tired to's point I would drink and I own a brewery too so oh okay

so I'm preparing your story now yeah I I'll tell you for me uh you know when you asked the question I had kind of a little bit of a crisis I'm like wow I've never thought about that but the honest answer is I I would agree with you um it's not even close I'm not like I barely I've barely gotten started I I have so much left in the tank let's say where uh no one's even put a dent yet to slow me down I feel like I still have a tremendous Runway of passion to carry me through all this I I don't feel the slightest bit of fatigue aw um yeah I I don't feel it at all I mean and I I'll

tell you what I want to do I want to build world class software security program and you know we just got we've just gotten started not even close yet love it so I would describe kind of your fatigue from uh you know I've hit that wall and because you you you constantly as you said trying to knock the the bricks down and you hit a few of them and it's like okay this isn't getting me any any anywhere and so um I I I also don't know if I call it fatigue or more how am I going to approach that wall in a different way sure and and that for me is you know I try to to to go back into

the to the toolkit and understand you probably face this in some other fashion and another different means what did you do what did you leverage from there who can you seek out so I get re-energized by you know

[ feedback ]