BSidesSLC 2022 - Why Data Breaches Aren’t Like Plane Crashes, but Should Be - Doug Hubbard

um helping growing software companies build their security program so that's kind of what I the flavor that I'm coming from also I was recently impacted by layoffs so if you like what you hear come talk to me afterwards um so today you're talking about why data breaches aren't like plane crashes but should be so I got a little story to start off um Maybe oops maybe not all right um on October 29th 2018 a Boeing 737 Max departed from Jakarta on a routine domestic flight in Indonesia shortly after takeoff this pilot struggled to maintain control of the aircraft the nose of the plane started to continually pitch downward after a few minutes the plane the pilot lost total control and the plane crashed into the sea unfortunately it killed all 189 people on board the flight only lasted 13 minutes just a few months later on March 10 2019 a second Boeing 737 Max ran into a similar issue this one was operated by Ethiopian Airlines and it left Nairobi Kenya the flight immediately had the same issue the pilot started to lose control of the plane the nose started to pitch downward he eventually lost control and it crashed killing all 157 people on board within six minutes these two accidents made international news it was very quick people from all over the world combined from International groups civilian law enforcement Airline manufacturers Airline operators all cooperating together to figure out what happened the Boeing 737 has been flying in some versions since the 60s how do two airplanes crash on opposite sides of the world with totally different people involved so they feared mechanical problems as a result within several days of the acts of the second accident the global Fleet of 737 Max's was grounded not a single plane would fly for more than a year and a half Boeing estimated that the cost of the groundings was around 18 billion dollars so what happened as a result of all of these different agencies collaborating together well they discovered that the root cause was a combination of several things poor training software errors mechanical failures and dangerous marketing so the the 737 Max is a newer version of the 737 the biggest difference is that the engine is a little larger that means it's a little more fuel efficient and it's cheaper to operate Boeing advertised this is basically saying this is the exact same plane you're used to just cheaper you don't need to have a lot of new parts you don't need to have dramatic new training for your cruise on how to fly it because it's basically the same airplane that wasn't entirely true the larger engine caused a change in aerodynamics what they found is that during takeoff when you're operating at your fastest acceleration sometimes the nose of the plane would pitch upwards and you'd risk a dangerous stall rather than trying to solve the problem through aerodynamics or training they decided to use software to fix it so they created a software system that would measure the take a sensor on the front that would measure what they call the angle of attack or the pitch of the aircraft if it ever detected that it was pitching up too high the tail flaps of the plane would automatically activate and cause the nose to come downward now what's interesting is that the pilots did not need to take any intervention for this to happen it happened automatically and in many cases Pilots might not even be aware that this software existed they found that some pilots who were flying the 737 Max only had two hours of training on the max specifically via an iPad in both of these plane crashes the sensor on the front malfunctioned and it the plane thought that it was pitching upwards too high when in actuality it was already within a safe path so the um the tail flap started to push it downward which eventually caused it to plummet to the ground in the case of the Ethiopian Airlines the pilot actually recognized that this was happening and deactivated the software but by then it was too late um so as a result of all of this they had to make a lot of changes so they applied these fixes so first they did a update to the software so they did a hotfix a patch they improved the diagnostic lights in the cockpit or improved the UI they also increased training on the crew and provided the ability for Pilots to override this this software very very quickly so in other words they changed their Technologies processes and training when operating the aircraft and what's really significant that I want to focus on here is this wasn't one company that did a postmortem and then figured out how to make these changes this was every operator of the 737 Max in the world and that's how the airline airline industry operates right when a crash happens there's a lot of attention to it you probably remember somewhere in the back of your mind hearing about these 737 Macs in the news from a few years ago people look at it there's a tremendous amount of transparency the fact that I can tell you about all of this even though I'm not in the aircraft industry and that you can go on to the Wikipedia articles and read down to the second what happened in a plane crash shows how transparent and open they are and then the entire industry elevates themselves as a result of each plane crash so as a result of all of this the um it's a tragedy that those folks lost their lives but all the other flights that are flying the 737 Max and new Airline designers in a very real sense are now safer because of that loss and over the course of doing this for more than 100 Years of Airline flight the airline industry is incredibly safe the odds of an American getting into a plane crash in any year is one in 11 million compare that with the odds of you getting struck by lightning sometime in your lifetime is around one in three thousand so it's dramatically less and it's we we have this safety net that's paid for with the blood of the people who came before us now imagine on the other hand if the airline industry treated plane crashes like security incidents right so here's a fictional notification from a not real plane crash but this is what I imagine it would look like um we regret to inform you that earlier this year between March 10th and 23rd we suffered a safety incident between 100 and 300 people may have been impacted many of those may have suffered minor injuries oops what happened again um an unknown number may have suffered loss of life for limb we've hired expert Consultants to figure out what went wrong and to prevent future incidents in coming months those impacted will be contacted above and beyond what's required by law and receive credit monitoring and most importantly your safety is important to us um so so what why am I talking about Airline Security in a um in a security conference well even though we're not in the airline industry I think there's things that we can learn from it we have it's good to look at other Industries and take and take a good thought a good hard think about how we do things compared to them data breaches are happening at what seems to be an increasing rate and every data breach that happens is really a lesson for the rest of the industry in a perfect world every time a data breach would happen we would know exactly how it happened and then we could take corrective action to prevent all of our companies from having the same fate in that world we would never have the same data breach twice right we would all correct it and we wouldn't have to go through it again now information security is not the same as the airline industry there's a lot of different reasons why for when the risk tolerance of human lives compared to somebody's address is very very different but I think there's some things that we can take from it so what what can we do specifically um I would propose that we should learn from from security events security things that happen uh when I was first starting to learn about security and I wanted to make a career out if I was a college student and I only had like one or two classes on security so I read security news and as that went on anytime I come across a term I didn't know I would just Google it and you know what's a SQL injection what's a cve what are all these different things and I found that within about six months I was generally security literate like I could have conversations I didn't know a lot but I could understand things I knew that I really started to make it when I started to feel that warm blanket of cynicism wrap around me so what I'm proposing that we do here that you do is to learn from other people's dumb mistakes right let's take a look at the data breaches that have happened how do we draw those lessons out and apply them to ourselves and our company experience is really the best teacher but having a data breach is really hard-earned experience you don't really want to go through that and we talk a lot about the technical side of things but one of the other side effects is when you did when you pick apart how people handle the data breach you also see all the horrible PR moves that they had and you can help your companies not do the same ones because they may have never thought about it before so um what we're going to do is I'm going to go over two different stories from actual data breaches that happened in the last couple of months how they happen in some lessons that we can take from them now a lot of times you have to make some guesses based on limited information so I'm not presenting all this is factual I'm going to make some educated guesses and that's totally fine right we're not in a trial we're not trying to sue them we're just trying to learn and I think that's great so first one we have is Uber you guys all know about the Uber data breach that happened this year there was a lot to say about that um so on September 15th of this year Uber suffered a data breach it's believed to be the work of a teenager located in the United Kingdom who is associated with the lapsis hacking group lapsis has made the news a lot they are primarily teenagers in the United Kingdom they started late last year and have been very successful at breaching a large number of Brands such as Microsoft Nvidia Verizon but what's interesting about them is they're teenagers so they like attention and they talk a lot about what they do and so this guy also talked a lot about what he did so a lot of what I'm going to go through is based on comments that he made publicly to journalists um and and other people I think it tells a consistent story because you look at the statements that Uber has made and it largely lines up some internal folks at Uber also said some things that probably you know maybe they shouldn't have that corroborate this as well with some screenshots and things like that so I think that this is largely true from what happened so what what is what is the breakdown so the attacker first found a contractor who worked at Uber and sent him a phishing email just generic phishing email got username and password for the company's VPN but the VPN was protected by multi-factor authentication fantastic great uh he tried to log in and it didn't work it sent a little push notice to the contractor whose credentials that belonged to this said hey you tried to log in was this you yes or no well he didn't click yes so the attacker tried it again and again and again and again and his phone just kept getting spammed with all of these notices some journalists said that there was just some fatigue and the guy eventually just clicked yes the hacker actually claimed that the contractor never did that he followed up with the contractor on WhatsApp said I am Uber I.T um we are having some problems with our messages you may have gotten a bunch of notices if you just click once yes they'll stop which is true um so he did click yes and then the attacker was in he was Off to the Races he immediately found a network drive that was mapped that contained a Powershell script with hard-coded admin credentials to their privileged access management system right not uh not that great um so privilege access management system that's like a password manager so we had all the passwords admin passwords might I add to many of the SAS applications that Uber was working with so what did he get access to well he got source code he got AWS and gcp admin for at least some accounts and projects He got G Suite admin he got domain admin he got vsphere admin he got Financial dashboards all of which he posted screenshots of online he got access to Sentinel one so you could remote control laptops he got access to their slack admin and started making posts in their slack taunting Uber he said attention Uber's been breached and I am the attacker he then put all sorts of updates on what happened now the people at Uber were so confused by this that they started they thought it was a joke and they started posting SpongeBob SquarePants memes and all sorts of other things well security did not think it was a joke and told everybody hey nobody's allowed to log into uberslack right now not sure what that was trying to do but hey they were trying to come up with something right um my personal favorite so you got into their hacker One account and made this public post you can see it's coming from Uber so he's disclosing his own breach on hacker one Uber's been hacked and this hacker One account has been hacked also uh there's a lot of people who've had some hot takes and poked fun at um Uber this one's my personal favorite I found Chris Powell Uber's threat intelligence is amazing they had the attacker on their own slack providing minute by minute attack updates so you know we like to poke a little fun at Uber here um you know they had I feel bad for those folks they've got a lot of work to clean up um they didn't do everything wrong they did some things right but in this case Uber is our teacher so great Uber what can you teach us so here's four takeaways that I thought were interesting um they're not you could probably grab a few others but here's the ones that I like first off hard-coded credentials are dangerous no joke um they would have a totally different scenario if they didn't had hard-coded credentials um and we all here know that you shouldn't put credentials API tokens in GitHub but do your developers know that and if I did ask one of your developers or engineers and they knew not supposed to put it there could they answer where it is supposed to go if they can't give you that answer what's your company policy for where to store Secrets you've got work to do uh limit access to sensitive information nine times out of ten when there's a data breach somebody didn't follow the principle of least privilege and that happened here as well let's just say for an instant that that um the script with the Powershell script with hard coded credentials was necessary for whatever reason right how many people should have had access to that like three maybe four people in the whole company what are the odds that that was the case and this teenager just happened to fish a contractor who was one of those four people I think that's unlikely so I'm gonna go on and guess and say that that script was broadly available to way too many people inside Uber so they should have had limited access to that um ensure contractors have security training people forget about contractors a lot of time this contractor was socially engineered did he have training about how to prevent social engineering maybe he did maybe he didn't but I'm sure everybody thinks he should have a little more now um who's providing that training is that you is that the Contracting Company put it into your contract how is it going to be tracked how is it going to be managed what happens if they don't do the training all things to think about and last but I think most important today's Frontier is identity and access management it used to be you know your firewall was the edge of your network and all of your data was in this database and once you got into the network it was kind of a GUI Center yeah that's that's 20 years ago nowadays is our data in databases some of it is but most of it's in SAS applications our customers or data is in all sorts of different SAS applications our crms our data Marts uh third-party managed databases all sorts of things like that how do we get access to those usernames and passwords so our firewalls today are usernames and passwords so do you know what SAS applications have your customers data do you know how those are protected do they have multi-factor are people getting off-boarded are people getting onboarded that's just as important today as writing good firewall rules so um Uber had some great things going they did have some management of it but obviously they could do a little bit of true up with their password management especially now all right next lesson so this one is coming from the city of Detroit I like this one because it's a small one and it demonstrates that we can learn lessons from small companies as well so this last summer Detroit had a um they launched a new retirement website right and any employee of the city of Detroit could access it so it wasn't publicly on the internet but it also was um broadly accessible so on the first day that it launched they had a man came in and he started looking around at the different things and he found on the home page links to a bunch of different other pages and one of those pages contained a spreadsheet that had the names birth dates and Social Security numbers of his co-workers right he looked at that and said well that's not right he could have contacted his sis admin and just said hey can we clean this up and maybe he did but he also contacted Fox News Fox News Local affiliate ran an article about it and I read that article which I thought I know about it and basically uh city of Detroit claimed we we hired experts to come in clean it up wouldn't you know it we did have some data exposed but only one person read the data that wasn't supposed to and they read 68 people they're going to get credit monitoring and we considered the case closed so a couple of things to pick out about this I think the pr part is interesting what are the odds that one person uh actually read that when it was available to all employees and it just happened to be the one person who reported it to the newspaper you kind of have to read between the lines on these a lot of times companies will say there's no evidence that more than one person was impacted and that may be true but maybe that's because there's no evidence because they didn't log in so I think that logging is a that's security logging is an important thing that's number nine in the OAS top 10 insufficient security logging and monitoring so auditing can really save you in a data breach it can help change the conversation from one person was impacted to up to a million people were impacted and we just don't know the other thing obviously is security testing any amount of security testing would have caught this right don't put data that's out to people who shouldn't have access to so broken access controls number one in the owas top ten so when you're writing out your software you should have security tests that are specifically testing for broken access controls have a positive person who should access data or perform an API action have somebody who's authenticated or maybe even unauthenticated try to perform the action and make sure that they can't so those are a couple of lessons um people often ask me how do you find these stories there's the perception that companies are tight-lipped and they won't tell you anything which is true in a lot of cases but you'd be surprised at how much is publicly available just by reading through the internet here's a list of places to go through I put them in order the ones that I like the best uh leave it up here so you