The Unbearable Lightness of Failure

I'm glad I'm actually able to be here this year. I was supposed to be able to speak here, but unfortunately, due to circumstances well beyond my control, I had to cancel this as well as some of the other conferences. I'm really disappointed about it, because when I was in university in Canada, I'm Canadian, by the way, so don't throw anything at me, I'm not a Yankee. It was amazing. I was going through it. I was learning about ancient Greece. I was learning about ancient Roman culture and things like that. And it dawned on me, I'd never actually been here when these sites came up, and I was like, well, this is a great opportunity. I'd

absolutely love to go to Iberia. But I'm here this year and I really appreciate the opportunity to be here to talk to you today. Now, the whole premise of this talk is actually built on a talk that I was supposed to give about five years ago called Lessons Learned from Falling on Swords. Now, I've been in this racket now for about 25 years, information security or computer security in some form or another. Now, during that time, I've made a lot of mistakes as I've left along. And the best thing about it that I've learned over time is each one of these is a wonderful It doesn't feel that way at the time, but it certainly

comes to that point later on. That was the birth of this talk. The first failure out of the gate is quoting yourself, which is a terrible thing to do. I'm rather pretentious, but I had to have some fun with it. If we fail, if we don't take the time to learn from it, or at least share what we have gone through, and I see time and again, I'll see data breaches where this, that, and this, that, the other thing happens. will the organization talk about actually what transpired? And it usually leads us to guess as to what actually went down. Was it SQL injection? Was it this? There was a mobile network operator in the UK that was compromised and it turned out that none of their data was

even encrypted. Unfortunately, they thought, man, no big deal. And as we go through our lives, we see there's all kinds of signs. There's all these things that are there because somebody did something that got there that required that sign to show up. At one point, I used to live in the United States, Santa Pista, Charleston, South Carolina. While I was there, I bought a sunscreen that you put in your windshield so your car would let you out. It never really worked out that way. It said, "Do not drive with this in place." Let that sink in for a second. Somebody, before I bought that, said, "I'll drive with this in place." And this is one of those things where, you know, failure is really something that we need to

do our better job. When I was young, it was of a generation that the parents said, "Oh, kids are to be seen, not heard." That didn't work out so well. But it was really interesting growing up and now seeing the opportunities that we have to give to our children. It's really amazing. Like when I grew up, I've been paid paid. And now, for example, my daughter who just turned nine, when she was five, she came to me with her iPad. I love telling this story because she comes up and she goes, Daddy, I'm having trouble VPNing into the United States so I can watch a show that I want to see there. You're five. I'm

in deep trouble. Right? And she was able to articulate network and security things that people in the industry that I worked with who had been in the racket for 10 plus years couldn't describe in the same clarity that she did. So as kids are growing up, and sometimes we say that they don't... Charity was talking about earlier, giving these children a chance because for them, they're born into the internet. This is intuitive to them. And for a lot of us, and myself as one example, we started out, we had nothing. We had no manuals. There were no university courses. There were no training programs. We had to all figure this out. So this is how

it started for me. I remember back in the day, we had this wonderful flying toaster. That was always fun. already created a random bit of a coffee tree. One of the things that really was interesting about this is, by the time this came out, I had already been duping video games with my friend in school for quite some time and selling them to other kids in class. Not a recommended activity, but we figured out how simple it was to do it. And then it came up with, oh, there's copy protection, and then we realized we had to snip a little piece out of the disc, and oh, we're back in business again. And it was really interesting, it's just like that

mentality wasn't so much of, you know, be a criminal so much as poking at things to see how they work. And that for me is a lot of how to hack. We go forward in time, we see that things evolve and things change. And remember at one point, this was on every blasted website that I went to. Unfortunately, it's still around today. And as we go through and we learn about these things and we learn and grow, we see all kinds of different failures. Like for example, the word hack has turned into a bit of a failure unto itself and that is something that lies solely on ourselves. As security professionals, hacker to me means

somebody with an innate curiosity, somebody who wants to go through and learn. Unfortunately, the media has taken that and run with it and turned it into the person with gloves tapping on a keyboard. Now, has anybody ever successfully managed to do any work on a computer with gloves on? I had to do it once, but that was because the data center was so cold that my skin was actually starting to turn blue. It's really amazing when you're going through these sort of things and you're seeing this sort of thing. It's incumbent upon us to go forth and actually get the message across that hacker's not a bad thing. Hacker could be literally any of these people. Some of them are key. But that's one of the things is that

every time I see an image of somebody wearing a hoodie, it's like, no, this is not the image that we want to portray. The image we want to portray is anybody with any sort of identity. This is what I think of when I see a hacker. This is my friend Dave Kennedy. This was really interesting because the conference that I was supposed to give the original version of this talk, Falling on Swords, he was actually turned out, I was on the same agenda as him. We were both supposed to give this talk and I missed my flight, unfortunately. When I looked around this list to give talks at this conference, it was amazing. The who's

who of people that I've become friends and work with on a daily basis. This is a real missed opportunity for me as I went through it. When I missed that flight, this is how I felt. I was absolutely crushed. I'm sitting at the airport going, "This is not the way I wanted to start off." This would have been my first talk that I had ever given. It wasn't all that bad because the guys at Hackfest, they created what they called the B-Side Static. And I thought, "Okay, what is Inside Dave?" And they actually went to the point of creating t-shirts. The whole point here was, after the conference, I couldn't afford a hotel at the time, so I was just gonna fly home the next morning, and

the first flight was at 7:00. So they decided they were all gonna come with me and sit with me at the airport, and they sent me all the t-shirts. I have a box of these t-shirts back, where they were all gonna wear the t-shirts, where we're all just gonna sit around and hang out. So, make these sort of mistakes, you have to understand that that's not the end of the world. By and large, every time we make a mistake, it's merely a learning opportunity that we have to capitalize. Nietzsche was one of my favorite ones for this. "That which doesn't kill us makes us stronger." A bit of a cliché at this point, but by

that definition. Now, one of the things that he talked about was something called the eternal return. And the idea behind this concept was that we are destined to repeat everything in our lives over and over. And it really sounds a great deal like information in many, many ways. But as we're going through this, this concept really occurred to me, this is something that we have within our own ability to change, and I firmly believe that. If we take the eternal return and turn it on its side, we're able to actually become masters of our own destiny. It really is time for us to actually take hold of that and change it. Something as simple as the term, I literally, whenever I do a media interview or

talk to, I prefer to use the term attacker simply because it shows criminal intent. This is what happens when we make mistakes that we didn't have to make. For example, SMB version 1 is something that could have been turned off literally years ago. Granted, Microsoft came out with a patch in March of this year, and then that ultimately led to Monocry, and it was really interesting because it never had to come to this. It really is something that we didn't take the time to learn from. Especially for a certain auto manufacturer, they didn't learn from the lesson of this from several weeks ago and they had to shut down one of their factories because they

were compromised several weeks after this all transpired. The really interesting part about the virus incident that not so much a lesson learned as a sort of innate benefit was that a lot of these organizations, for them, it was the first time they'd ever actually had data encryption in their company. And you never know when you're going to need that type of data. And to be fair, you really should be taking the time to do proper security controls to ensure that all of your systems are going to be online when you need them. When Nietzsche's talking about the eternal return, we have to learn how to step off of that return if we don't meand up doing the same things over and over again.

Want to prize a perfect example of something that didn't happen happening? Well, I'm blessing you. When I read about data breach, so many times I'll see it's something that this could have been avoided. Another talk that I've been giving this year is about data breaches Basically what I did was I went through 2016 publicly available data breaches and I looked through the types of issues and things that had transpired in the back end. The vast majority of them were due to either missing patches or things that we couldn't fix. And if we don't get in front of these things and address these sort of problems, we're destined to repeat them forever and we will be

immortalized for all. Failure will happen. Being afraid of it is absolutely a waste of time. Being prepared so that you don't fail, learning from lessons from others, is absolutely a wonderful thing to do. This is how we get better. But when failure does happen, don't panic. Figure it out, learn from it, ask for help. We have a community of security practitioners, case in point, this entire room. If you have a problem that you can't navigate through, ask for help. Because mistakes are going to happen. This is a great one. In my career, I was working for the U.S. Department of Defense as a contractor. of my clients was possibly being attacked by a particular IP address at some point. It got to the point where they couldn't

get through the firewall. I just got irritated one day. I didn't sleep well or whatever it was. I hit them back. I hit them back kind of hard. I got into their systems and I left notes everywhere in every directory I could find saying, "Please stop doing that." And I left a generic email that they could send emails to that moved the track back to me. Well, a couple days later there was an email waiting for me. Someone was rather upset. I had actually hit the wrong idea. At some point in the idea, as I went through it, I had transposed a couple of the numbers. This is one of the reasons why I've never

been a fan of the idea of hackback, which a lot of governments around the world are trying to espouse at this point. Because mistakes really are going to happen. And if you're playing with nation states and that sort of level, and you make a mistake, the best part about that mistake was, I'm not alone. This is not something that was unique to me. case in point, I was at a company several years back before I came to my current job back in my technologies, where we had a vendor that was trying to sell us a product. This vulnerability scanning vendor said, "Look, we can give you a free discovery scan of the entire IP address

range and just show you what we can do." I said, "Sure." Here it is. We looked at it and went, "This is a Class B." He said, "Yes, it's all we have." We were working for a power company at the time. And they took that and they ran through it. And they said, sure, no problem. They scanned the whole thing. They gave it back to me. Nice big thick document. Flipping through it. I'm like, there's something very... As I'm going through this document, I'm seeing roughly the same thing. Shenzhen, Beijing, mine. They had done exactly the same thing that I did when I did my hackback. They had actually screwed up the IP address and

scanned an entire class B in China. So here I am with all the very vulnerable print. because they were absolutely beside themselves and they were apologizing left, right, center. I said, no, I named you to relax. This is a very clear, obvious mistake. That's something you can learn from you and you can be guaranteed they didn't make that mistake again. And this is one of those things where when these mistakes happen, you have to find your moment of zen. You have to take it because otherwise you're going to get lost in the weather of, oh my God, things went wrong. Failure isn't something to be afraid of. You have to squeak it down into a very tiny box. It's gonna happen. You have to

learn. It takes some point. This may have sounds like all the dead of the flat. This is a, um, what was the, the Tacoma Narrow Bridge, years ago. They had taken all this time to build this bridge and they took in all the data they thought they had, as it were, only discovered that there was a strange crosswind that they had not accounted for. As a result, that beautiful base that they had taken all this time to build, turned into this with a slight gust of wind. It was just a very strange twisting of the frame to the point where it eventually just buckled, snapped, collapsed. Thankfully in this particular case there was only one

fatality and it was a dog who got trapped in that particular car. This is one of those things where this was, and they learned some valuable lessons from it, this is a perfect example of where we have to make sure that we're listening to the information that's being provided to us.

This is very much how it feels like in our company where we move about 30% of the traffic. During that time, we see all kinds of entire traffic lights as a module they use on their environment called Chaos Monkey, which will go through and randomly destroy some machines. We have our own version of that called the Internet. So that was an example of something where it was potentially a one-off mistake. I don't know all of the details because it's a little bit shrouded in history, but that was something that is an example of things that can happen today. You do not take in the requisite amount of information or learn from the lessons that are

available to us. This is another example of something that I went through at a company years, years ago. This was a case where I found a web app in our environment. I was going through and testing and making sure everything was running well. This particular application, I had not seen it before because it had not gone through the proper vetting process. We had a gaming process before anything went into production. This app, obviously not the regular web page, So I had a meeting with the head of that development team, my boss, and myself. We sat down and I said, look, use source, really clear, source, that right there. So, can anybody tell me what the manager of that app dev team said to me at

this point? They let you off the hook. He said, "You hacked my application." I can't even make this stuff up. He literally thought that View Source on an application was a hack. And this is one of those problems where I went, "Oh God, this isn't his failing so much as his." Because I hadn't taken the time to work with his team to say, "This is what you shouldn't do and here's why." Otherwise, this becomes a problem. This was his reaction. And absolutely beside myself. I couldn't believe it. But that was just it. It wasn't so much I couldn't believe what he did as I had not taken it. So that it wouldn't do these sort of things. So when these sort of things happen, you have to make sure

that you don't do this. Because this, by and large, usually is our first reaction to go, are you out of your mind? But unfortunately, that doesn't help you or them. You have to really take a step back, take a deep breath, go to your place of Zen, and and realize that this is something that we have to take the time to learn from because this was a mistake that didn't have to happen yet again. Otherwise, it'll keep happening. So one company that I was working for, I had a very long talk with a project manager there. I pointed myself out to help some project managers I had stick with the wall on my office. And

this is one where the application dev team, a different team, had done what I asked them to do to protect their application, but as soon as I left, This happened. So I got a call about a week after I left, and you're not going to believe what they did. As soon as I was out the door, they reversed all the controls I had put in place. At the top of this URL, I know it's almost impossible to see, but the variable in is slash Etsy slash password. This was a publicly-facing web page where we were able to knock out all the credentials. Oh, but wait, there's more. Not only could you do it to that

file, you could do it to any file on that system. You could browse any file on this production system that was facing the Internet. Luckily enough for that particular company, they did not end up in the press for this. It's amazing. But this is one of those things where you have to understand that no matter what kind of tools you build for people, no matter what kind of things they have to use in their daily jobs, they're going to use them in new and exciting ways that you never have ever thought of. And if you don't get in front of it, they're going to think tools like this are helpful. This tells you nothing. Thankfully

this company went away, but this is just one example of many. This is not something that is going to provide you any real benefit. The real frightening thing is how many times I've seen stuff like this in security operation centers. They'll have it up on a monitor and literally the Pupium app does it. You need to become uncomfortable in your own skin. Now as security practitioners, the goal is if you are comfortable, if you're getting to work in the morning, you just sit down, you have your coffee, the most stressful thing in your morning You have to realize at that point, you have to really gotten into a comfortable rut. You want to challenge yourself.

You want to move beyond that because if you don't, you're not going to grow. You're going to be stuck in that same position. And you know what? You might be happy doing that. And that's totally fine as well. But if you want to progress in your career, you have to take challenges. You have to get out. You need to be able to sit down and reevaluate what it is that you're doing. Or my favorite thing that I always say to my kids, what do you want to do when you grow up? Apparently my daughter wants to design spaceships. Very cool. And my son wants to be Hulk. But we're still working on that. But this is one of those things where you really have to sit down

and as something as trite as it may sound like, let's sit down and write down what to do. What do you like doing? What do you absolutely hate? You may have actually answered your own question if it's time to move on to a position somewhere in the organization or in another organization. Or, oh wow, this is actually going really well. Because failures are going to happen. You also want to look at what kind of tools you have. One thing that I used to do is I used to constantly battle with project managers because in a couple of my roles in two different organizations, my job was the gate. So any product that the product or

project was coming into production had to go through my team and we had to evaluate it from a security perspective. This was painful, a painful existence, but it was one of those things where I had to go and I realized I had to start speaking their language. And this is where I went off and I got my PMP designation. And this is not trumpeting for the certain particular, but it forced me to learn their language, their lingua franca, how it was they operated, what was it for them. And this was something that was a hint for me because I was able to have fewer and fewer bumps on my wall because I was able to

articulate my concerns in ways they would understand. For example, if you run into the C-suite saying, oh my goodness, this zero data is out, my hair is on fire, we have to look at this now, the C-suite is not going to pay attention. You have to articulate it in terms of risk, something that they're going to understand. If you can't quantify it as to what is the risk to the organization, how if something goes wrong, they're not going to listen. So it's important for you to actually take the time to learn from lessons of mistakes like I did, because I've done exactly that where I ran into my something you don't have to do take a step back learn their labor frankness so you can actually help them

understand your concern now as you're building in your career you want to make sure you're building a solid foundation you don't want to you know build a house on shifting sands much like this particular place in shanghai china which actually this actually happened in 2009 that's an unfortunate thing thankfully nobody was home really built lack of proper foundation much the same way as our careers we don't have a proper foundation we don't have we are severely limiting ourselves. And when you limit yourselves, problems like this happen. Now this is the graph that has absolutely nothing to do with the story, but I just love this graph because people always say that it's meant to be a habit that goes through time. So in this particular case, I was actually

at Black Hat Las Vegas, and I was walking down the hall, and my phone starts ringing. And we had a consultant come on site before us at our angel to install the system. And this is my second, and he's like, you're not going to believe this. The consultant who we were paying $2,000 Canadian for at the time, which at that time was a significant sum of money years ago, had come in, sat down, and opened the manual page one. This was a consultant that we were paying $2,000 a day for, and he hadn't even read the manual. He had come in completely unprepared. Unfortunately for him, his job was terminated before the day was because

I had a rather serious chat with his manager. But this is one of those things where if you were going to be engaged on a project, either as a consultant or a transgeneral organization, this was a lesson that he could have learned from many, many others before. There was a complete disconnect. He was unable to actually accept that. Things can and will. Now, as you go through, you want to be able to say, I took a shot. I took the time to set up a B-side. One of the things that I get run into in a lot of conferences I go to is like, I want to get into podcast publication. How did you do it? Literally, I just started it. Didn't ask permission, I just started doing it.

My own website, LivingMajors.com, is actually coming up on 20 years online. It was born out of the case of, I was at one of my first jobs in this field, and I would bring articles to the team that worked, and they were like, "This is fantastic, where did you find this?" And I would literally find articles from Garnasman before it was called Dark Web. And I would share it, and it just had born out of it, and it grew from there. And out of that grew a podcast, grew me writing for various publications like Forbes, and formerly CSO Online. And the only advice I can give to people from that is to take your time to take a shot. Just start doing it. There's really nothing to limit you

other than yourself. And when people say they can't do it, I say, have you seen my field where I grow? Now with social media and various other types of echo chambers, a lot can all get lost in the background noise. And we have a bad habit of getting lost in We as security practitioners have to pull ourselves away from the daily drama and really pay attention to what's important. Things like building community events, building podcasts, building podcasts, building anything that can share information back and spend less time about who's more elite than who else. That sort of thing garners no positive. When we do get lost in that, we tend to spiral into very dark places. And when we do that,

we miss those fine details. things get badly things badly happen so here's an idea don't be afraid to ask for help it's one of the things that i learned early on really out of i had to take a chance you have to do that you have to be able to reach out and ask for help when you're in trouble don't go down swing it if you don't have to because if you do something like this could happen that's not the beat up on this particular company literally a day before i gave a talk in new orleans this happened It's a data security company and their entire database is on it. I had no idea how

that transpired, but it was absolutely timely. We want to actually keep all the security information to ourselves. One of the things is we get lost in the silos. We have to stop getting into that. We have to break out of our silos. We have to become a... Go talk to project managers. Go talk to your procurement team. Go talk to your HR team. If you can make security into all those aspects of the organization that you're working in, your job's actually going to get easier over time. And you need to be able to trust your team because you're not alone in this. You need to be able to cut the tether and let go of your perceptions or your preconceived notions. I got there very early on, midpoint

of my career, controlling an IDS system for an organization as well as other security things. And my mentor at the time said the most beautiful thing to me. He said, "You have to learn to let go. You have to be able to take that system and give it to somebody else and walk away from it." When you do that, it's rather a very freeing thing. And I was afraid to give it up because I was afraid it would fail. I was afraid I was horribly wrong. I didn't want the management folks to touch it because I didn't trust them. And this was my failure. And by being able to let go of these systems and

let other people take them over, I was able to get off the new and exciting things. You want to find a balance in your work life career. That's always a cliche that we love to hear about. It really is possible. You have to be able to get up and move away from the computer. At one point when I was living in the United States, I was there as a single guy living in Washington, D.C. And I started breaking into systems for fun and profit. And I started working on something at night. I figured out what was going on. I was rubbing my eyes. I was staring at the screen. I was squinting. The sun had

come up. I actually realized that I had been working all night long, and I now had the sunlight shining in my eyes. And I had about 30 minutes to get to work with zero sleep. You have to bounce. You have to learn to step away from it.

Are you planning what you want to do five years down the road? Now, whenever you're in a job interview and they say, where do you see yourself in five years? Nobody ever has a good answer. And historically, neither have I. But if you're taking the time to plan what we want to do next, likelihood of success, your rigor is rising rather significantly. And I was pointed recently to several studies, which, heaven forbid, I actually include in this, because that would have made sense. But as I'm reading these studies, it's like, yeah, people actually take the time. If you don't have a solid plan, you can take chances as long as you have a framework to work within. So a perfect example of this, when I

moved to Charleston, South Carolina, is because I was on a training course in Texas and there was a young lady who sat down at the computer in front of me, broke into her computer and said hi and took over her screen. And we started chatting back and forth. And from that point we had turned into a relationship and I moved down to the States. Unfortunately the relationship ended before the plane landed, but I stayed. And I continued on with that job and it was a great opportunity and I had taken a chance. I had a framework in place in that I had a job, I had everything up, and just because that one piece of

the puzzle didn't work out, I was able to power through and it became a very successful career move. And you have to understand that you're not always going to win. Things are going to go wrong, but the thing is, we'll survive. We'll move on and we'll improve from this point. And every time you fail, it doesn't build success. You think inventors get things right on the first time? No. They absolutely don't. When I talk about sharing lessons, when I say that I don't, this is how I built it originally. And this was actually built with stolen source code from Hacker News Network at the time, which was a spin-off of the blog. I told them

nine years later, and they thought that was absolutely funny. And I would literally share articles. This was before there was WordPress. And it was really a fun experience, especially going through the logs and seeing all the different law enforcement and militaries that were reading the site on Daily Peace. In fact, And as we go through and realize that this thing is still around, this is something that we built, this is something that I built, it was purely on an altruistic endeavor. So, things don't always work out the way you're going to expect them. There's a lot of unintended consequences. The war storm is a perfect example of this. This was code that was never meant to get out

into the wild. But it did. Things went horribly, horribly wrong. Now, This I see as one of the turning points for InfoSec people overall is that we look at things as an adversarial relationship, and I think us versus them. And I really look at March 1 as really about that time where things started to change up. And if we align ourselves against ourselves, we become our own worst enemies. This is something that we can avoid. We have to do a better job of communicating to all the other parts of the organization as well as beyond that because we can't assume that people Because when you make assumptions, things like this happen, where a commentator on CNN was talking about how all of these celebrities and their

pictures hacked by the hacker named 4chan. Let that sink in for a second. Alright, so a couple of word of praise. One thing that really got me was I was working at one organization that was a global footprint, and this had, we literally had offices around the world. And one particular office, we had a problem where we were getting letters from Rita and FKAA saying, "Your company is downloading Hack and Crack 2." We looked at all the firewalls. It's not great. Finally we realized that we had something very wrong in this one location because everything seemed to track back to this one location. We had our guy in the region go to this office and

he walked into the server room. No kidding, they had actually taken the time to build a stub network off the edge router that was bypassing our firewall entirely and then linking back into the corporate network because these guys were so intent on downloading cracked movies they built an entire stub network off of it. This abject failure, only on them from downloading these cracked movies, the failure was on the right things. We weren't watching the router loads. You have so many systems on your network that are talking to you. The question is, are you listening? Now, one organization that I worked at, we had a WordPress. So this was a WordPress install that was part of

a marketing stunt site that was a hangar. This was before I ever got to my current day job. That one had 192 user accounts on it. It was popped and ended up on the front page of all kinds of newspapers. Now the backstory for that particular one was this was a site that was identified by the security team and we said, "Look, you have to take this down." They said, "Yeah, no problem. It's a deprecated site. We're going to take it offline." Fantastic. We didn't follow up with them. They left it there. It got compromised. It had no ties into the rest of the organization, but that didn't matter because it was in the media. We didn't close that particular loop. But I learned a lesson. Another example

was one company I was working at. We had originally been a large company. It was split into five separate organizations. This particular organization, we went through and there's something wrong here. We couldn't figure out what it was, so we went out on the shop floor and started lifting tiles. The third tile I lifted up, I looked down, and there's a Cisco router blinking back at me under the floor. This was a Cisco router that was tying back organizations that had become pretty much a competitor to us. But luckily they didn't know it was there either. So we had an interconnect between the two companies and because we didn't have proper documentation or understanding as to

what was going on, that remained for years before we ever found it. It's something mind-boggling in itself. So this was a case where we had to share that lesson within our own organization. And this is something that you can take away as well, is every time you find something like a hub problem, like a red or where it shouldn't be and things like that, you have to learn from these and give lessons. Because we're going to fail. And we're going to learn from these things. And we will get through this collectively because we do it. And one of the things that really gets me is I never expected I would find a moment of zen

in a Marvel movie. But this was it. Dr. Frank, apologies if you haven't seen the movie yet. But there's one point when the ancient one says it's not about you. And that's just it. It's about all of us. This is something that we can take and learn from because as we fail, we grow. And we have to share those lessons. We can step off the eternal return. I really appreciate the opportunity.