Bsides 2018 Track 1

Okay. So we can't do the PIP because we're at technical issues. So we're just going to do this and we're streaming. That's the volume for his mic and you want to make sure you're seeing it well here. This is to the right. Front mic is that one. It's on mute right now. And down zero. Okay?

So where's the screen?

I don't know. Yeah. - How's it going? Yeah. you Yeah. Each.

I'm not there. I don't know. No. It's my idea. So. I don't believe in myself. I don't believe in you. Yeah, you are. You don't believe in your journey. Yeah, it does. I graduated high school and did a lot of times going really fast. And then I got married. I went so far and did it fast. It's been a long time. It's just like, well, I don't believe in myself. I'm 14, 12, and almost 10. - I have to leave at 9:45 to get back to Rob. I've got two more phone calls before I come back. - Thank you.

So we get into all sorts of resources, so this is a very important piece of the year for me. all right Yeah. um I'm going to start getting into it. me me um Yeah.

- That's right, that's right. Because this is supposed to be all set up now. Yeah. I know there's a slight issue with getting the information directly off of the laptop. Oh, yeah. So it's just going to be the screen. Basically like this. Yeah. Thank you. Yeah. um - So many things. - Yeah, I mean, I talked to, Thank you. I'll call you. I'll call you and watch the video. - There's a number of them, but . - That's what makes that . - Yeah, it's like . - Oh, you own that thing. - Yeah, I do. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah.

- Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah.

- Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah. - Yeah.

- Yeah. - Yeah. - Yeah. - Yeah. - You are invited to the slide. So, let's look at the bottom. I guess we're running a little behind, huh? Are we getting close to being able to do this? Oh, you guys can keep talking. I don't think we're ready yet. I mean, look, this isn't really-- that's not mine. That's Alex's, where's Alex? Paging Alex to the white courtesy phone, Alex. This is fun, Alex. Cool, all right, hang out a couple more minutes, I guess. - What's happening? okay - Okay, just kick it off. - Just kick it off right now. - Just kick it off. - So here's where I go for the hand. - Yeah. - Sorry about

that. - It's all good. - What? - Microphone. Good morning. Can you hear me back there? Can you hear me now? All the way back? I'm seeing thumbs up. I don't know what you guys are saying. All right, well, welcome to Vancouver, sorry, B-Sides Vancouver 2018. This is our fifth event. Really sorry about the delay. You know, ingestion is the biggest problem, especially when we went over capacity by about 25%. So for everyone that got in, awesome. So just to give you a little bit of a history, we started off in 2013. I was a speaker at that point. Darren and Yvonne were the original organizers of it. Yeah, that's when you were there? That's awesome. I did

not know that. So we had a single track. It was just a borrowed room in an office building. Pizza party, 100 people. But that really set something off. We formed the Mainland Advanced Research Society right after. We did it at the Chateau Granville, which is not a castle that is a Best Western. But we did one track, one workshop, and we had 200 people. And then we grew again. In 2015, we did two tracks. We did 300 people. And there may or may not have been some zombies out back. 2016, like many, B-sides do have some turmoil and we weren't able to execute one, but we came back, raised out of the ashes for 2017, which was awesome. I think most of you came, and that's

also where we started our Mars Slack channel. So we have over 150 active members on there right now. If you want to be part of the community when you're not here, join up. And what do we have today and tomorrow? So we have over 500 people registered and there was probably about 100 to 200 that did not get in, which we are streaming. Hopefully that they're now able to see that. What's really cool is like if you've noticed our logo, it's always circuits coming out of this building. That's because this building is actually a physical manifestation of the internet. This is like where a lot of the internet comes through Vancouver. So we thought it

was really cool that we'd build that logo in and for a fifth year, let's do it in that actual building. So we've got 36 talks across four tracks, cloud security panel after this talk, four workshops, women mentorship at luncheon. We have a CTF, we did a pre-con party last night and we have an after party tonight. And we have a job board with wristbands. So if you see this wristband, this is I'm hiring. There's a yellow one for I'm looking for work. So go find them, I think they're at the registration desk. And mingle, hire, get hired. What's the blue one? Oh, iron, yes. Go find one at registration. All right, so this is our attendees. And what I'm really stoked about is that there's somebody from Singapore

here. That's really cool. And the UK. But we have people from all over North America that are attending and flying out here, which I'm really stoked to see. But what I'm more humbled by is where our speakers are actually from. And humbled because most of these speakers are actually coming on their own dime to come and speak here, which is really, really humbling. The badges, if you are a leader or professional or speaker, you get one of these really cool badges. I was going to really tell a journey about that, but I don't think we have any time for that. But they do hack each other. So I'm on team green, there's team blue and red as well. And you can hack each other. I'm going to turn that

around. And we'll see which color dominates at the end of the conference. So we should actually see probably a homogenous color of badges coming through. If you want to hack them, we have some details of how to hack them at bsidesvancouver.com/badge. And again, I can't say a huge thank you enough to these fine gentlemen. Todd was the acrylic laser cutter, PCP designer, project manager, and everything in between. He really got us out of every little ism that comes from when you design, don't prototype, and then go to production in less than three months. Sandy did all our coding, and that's... Awesome, that's how we now have hackable badges. And Raj from Active Electronics, they are a local company that does prototyping, PCV manufacturer placement and whatnot. And

he was a great help in getting, within less than three months, concept to physical badge. We do have a CTF. This is hosted by Compass Securities Hack Lab. 100% less Googleable challenges this year, which I'm really excited about. I know that was a problem last year. So we've sorted that out. DC604 is running that for us. There are prizes, so if you want to sign up, that URL. If you need help, that URL. Get on Slack and talk to the team there. Alright, so some housekeeping. There's fire exits there over in that corner. Bathrooms are in the southwest corner. There are some other ones, but these are the nice ones. Let's see here. We are going to have lunch 1130 to 1,

and there are some talks over lunch. There's the ladies' session at lunch, and I think tomorrow there's also a talk. If you want to socialize, that's our hashtag. We do have wireless internet, but because we are trying to share the conference with the people on the outside, we're not going to share that code out immediately. In the afternoon, we will. But we just want to make sure that we can stream this track into all the other tracks, and that means leave our bandwidth alone. And talks are going to start up on time after lunch, unlike this morning. And we do have workshops. So workshops are limited to elite and professional. They get first dibs. There is limited seating of I think like 30 or 40 seats.

If you are a pro, get your prerequisites sorted downloading VirtualBox. If there are empty seats, GAs can definitely be in standby, but again, it is limited seating. And party details. So we've got a party afterwards. It is at a secret location. that you can then find out by grabbing a little ticket at the Trinimbas and Mirai booth. We'll have some hacker jeopardy, so if you want to join that, it's team-based. Stop by the SecureNet table and sign up to get on a team. We're going to have a chat panel about decentralized technologies. There's going to be a DJ with some MSGNU rapping, which I haven't seen, I'm very curious about. Crypto stand-up, some tasty sushi and Japanese snacks. And for transport, we actually have a party

bus. So the first round is going to be for speakers and VIPs, and then that's just going to keep on doing laps. So you can either wait for that bus, it'll be coming about every 20, 30 minutes, or you can take a cab, public transit, definitely not walking. It's not close. Unfortunately, for all you out-of-towners, we are unfortunately Uber-less. and we will not be able to have any Uber for anyone that's normally used that technology. Seriously. So, apparently it's coming in 2019 or 2020 or 2021, so we'll get it eventually. Alright, and again, a big thank you to our sponsors. This is who makes these events possible by funding it. And I please encourage you to go to their tables,

talk to them, find out about their new products, find out how their products are going to solve your problems. And I guess that's it. On with the show. I'm going to introduce our keynote right now. This is Rafael Loos. You may know him from his podcast, White Rabbit. Down the rabbit hole, right? down the rabbit hole, security rabbit hole. So he has a great podcast you should tune into. It is a bit refreshing from some of the other podcasts that are a little lengthy. Raf does a really great job. So let's give him a warm round of applause. All right, can you guys hear me okay? Can everybody remotely hear me okay? Just kidding. I'd love my clicker back. Thanks, man. So Alex, I was trying to figure out

which, because I was here once before, and I was just trying to figure out which one it was. Apparently it was the original one because I remember the pizza party. It was, yeah, it was something. Who was here the first one? Anybody remember me? We also did some sort of Japanese sushi thing afterwards. That was kind of sketchy though, I don't remember much of that. Somebody poured sake and then the rest of the night went. All right, thanks for hanging out guys. Wow, so they say this room is supposed to hold 206. We'll see. So, yeah, my name's Raf. I just joined a company called Armour. This is actually week two, so it's really kind of fun to tell my boss that, hey, second week on

a job, I'm not going to be here this week, I'm going off to Vancouver. But they actually said it was all good, so here I am. I was kind of trying to figure out what to talk about, because I got the invite and it was very nice to be invited up here again, back into Canada. Some of you laugh. But there's only so many Canucks jokes you can make before they stop letting you back as a hockey fan. But I settled finally on-- anybody know the book, How to Win Friends and Influence People? Anybody read it? So anybody seen the Simon Pegg movie that sort of parodies it? Has anybody seen this movie that goes from this title? If you haven't, you guys know who Simon

Pegg is, right? Go see the movie. It's freaking hysterical. Also, it reminds me a lot about us. So, this is gonna be a little bit introspective, a little bit about the community. I've been around, I've managed to survive close to 20 years in this industry. Before it was really kind of a clicky thing. But the title of the talk is How to Lose Friends and Influence or Alienate People. And so, I've got it down to sections. Because for me it's kind of been an interesting road. I started building white box PCs when we were installing Windows NT 3.51 on it. Do you remember that day? How many of you guys worked with NetWare 3.1? Yes, my people. All right. And then 4.1.1. I swear there's

still like 4.1.1 servers out there running somewhere that nobody's bothered rebooting in the last 20 years because they run. So look, I think from what we do, a lot of it is a matter of perspective. Depending on how you see things, there's lots of parts of our industry. Some of it's really good, some of it gets kind of bad. There's a lot of in between, but really our jobs are what we seem to make of it. And I'll talk you through it. So as you mentioned, I got a brand new logo, I thought I'd show that off for the podcast. I had a guy that does comic books and he agreed to illustrate, which is freaking awesome. We've been around since September of 2011. You guys believe that?

So I started a podcast that was kind of annoyed that I couldn't listen to any of the security podcasts at the time at the office because most of them were horribly work inappropriate. So I said, "Alright, there's got to be a better way because I'm sitting in the audience and I'm trying to get something out of it. My guess is all fun, that we're making fun of people and we're getting drunk and we're talking about things on a radio show, but what am I getting out of it?" So I encourage you guys to listen. I've got tons of interesting guests. We're coming up on 300. I had no idea what we were going to do

with it, but hopefully something good, something epic. My trip through the industry has sort of been the Val Kilmer role where, bless you, for many, many, many companies that I've worked for, I've been that guy with a shovel where you need something done, let's go do it. And currently I'm the VP of solution strategy at the company called Armor. If you've never heard of us, stop by and ask. We own the domain armor.com, which is kind of interesting because most people go there for chain mail and swords. But, you know, enough about me. Let's talk about fighting evil. So, There are, we'll start with this, there's a cornucopia of adversaries out there, right? There's lots

of bad things going on. And I know a lot of you guys feel like we're all fighting the bad guys, right? But I've asked the question is, sort of, what is it that we do here? And I feel like the guy, one of the Bobs, and so I legitimately ask audiences, and I do a lot of corporate speaking too, and it's, what do we do as a security team? And a lot of people say security. Which is great if you're a security company. If you're everybody else, your job is to protect stuff, it's not to do security. So how you perceive what you do matters. This is the only bad graphic I'll use because it's hilarious. I actually Googled horrible hacker graphic and that's

what came up. Everybody does this. So there's really no shortage of bad stuff, right? I mean look, pick up a newspaper, never mind, don't watch the news. That's a bad idea. But look at Twitter, look at LinkedIn, look at all the stuff that we talk about. There is a crap ton of bad stuff going on out there. I mean, there's bad people, there's bad actors, there's bad nation states, there's really badly intentioned good people. And by the way, bad guys existed before the internet happened. Remember that? Before the internet? No, thanks. Before all of you started getting AOL 1.44 meg floppy's, Yeah, we had a copy server address. That's right. Some of us are old. Get off my

lawn. So every organization has adversaries, right? And so, but look, adversaries aren't just the gorillas and the pandas and the, what else is there? Unicorns and lions, tigers, and bears, oh my, I guess. Right, but so everybody's got adversaries. But we don't just fight bad guys, right? There's also insider threats. Lots of bad people. I mean, there are insider threats everywhere. Submitted without comment. So, there are people that you trust, that you work with every day, that have legitimately good access, that have been vetted to whatever level your organization is comfortable with, that will do you harm. That sucks, right? And the difference between a good intentioned person and bad is literally, it's a razor thin line. I've got lots of stories of

people that worked with companies 15, 20 years that in the blink of an eye, suddenly became the ultimate agent of evil because they had, they hit a rough patch, somebody offered them a pile of cash in exchange for this one little thing, and good luck turning that down. They weren't trying to be malicious, they just needed money. And I'm telling you right now, that happens. And P.T. Barnum, what's the quote, right? Including lawyers, everybody falls for scams, everybody clicks. Because ultimately what we're doing is fighting human nature, name the show. Thank you. If you've ever wondered what it's like to be in information security, if any of your friends have ever wondered, point them to this, because this

is just, give them computers and it's the same thing. We are fundamentally fighting human nature, and we're trying to protect everything, which is just really crazy. We work for a company, and the idea is we can't get hacked. We can't have anything bad happen to us. And there's analogies to be had there, but look, if you have kids... You know, you may say to yourself, "I can't let my kid get in a car accident." Like, all right, so I'm going to give them defensive driving courses, I'm going to make them take classes, make them learn, make them take driver simulators, do all these things. You know, buy the safest car on the road, wrap them

in bubble wrap, wrap the car in bubble wrap, clear everybody else off the road, and an errant semi smacks them in the back while they're sitting at a stoplight. You can't stop everything. So the ultimate thing is, then there's this thing called the attack surface. Because the reason I asked that ridiculous Novell network question earlier is those of you guys that raised your hand, what was your network attack surface back then? Right, you're giggling because I know what you're thinking, right? Remember when everybody had, maybe you didn't, not everybody, remember when you had a T1? For like, yeah, like 16 grand, that's a telesurprise. Just kidding. That was actually cheap back then, way back then. But no, so look, we had internet connectivity back then. Through a

T1 or an ISDN circuit if you're really old. And then you had like a router that connected your network and then you had firewalls for those of us that have been around for a while. And that was your way into the company. And we all got laptops. Like, all right, well, you go home and dial up. How bad is that? Not terrible, right? Because all you're doing is getting on AOL. Okay, and then we got wireless started happening and we're like whoa whoa we don't do wireless here And then he walked around your office and you realize there's like seven of them under people's desks right this reminds you of cloud by the way where people like I know we don't do cloud here you're like That's cute But

seriously though and then and then like computers in your pocket came around that also did what that also was a Wi-Fi hotspot you could walk around and And now I ask the question of what's your attack surface and people are like, "Eh." What's the important data in your organization? You're like, "Everything. It's here, it's there, it's everywhere." So what we're really trying to do as a community, as an organization, is trying to find focus because you can't protect everything equally. Look at it this way. You've got a family of 12 kids. Three of them are going to college. The rest of them are going to have to find jobs. It's the reality of things, right? So, look, the one point I'll make is superhuman effort.

I know a lot of you guys do this. I've worked in companies. I've been that guy. I know a lot of you guys work just insane amounts of hours at the great personal expense sometimes. We leave behind families. We trash our own relationships because it's required. We do what's necessary, right? But that leads to burnout. How many of you guys have ever gotten close to that point? Where you're like, screw this all, I'm going to go 10 bar somewhere. How many of you guys have actually done it for a while? Right? If you thought about it, it might not be a bad thing. But burnout leads to this inevitable, cataclysmic, horrible failure because you're running

people at 150%. Eventually, they will require sleep, or they will make mental errors, and then Google Cloud goes down because somebody fat-fingered a configuration file. It happens. People are still at the root of the problem. So out of all of this, the desired result, the thing that we want, ultimately, we want to be the heroes. We want the big standing O at company meetings. That's what we want. We want to be welcomed. We want to be accepted. What we really get is more of this. Every day, all day, yet another breach. And by the way... If you've ever caught yourself in one of those name and shame pile-on situations where, like, oh, company got breached.

Ha, ha, ha. What a bunch of, you know, jackholes. I can't believe they did that. Give it 30 seconds. You're next. All right. All right. Next part of this, lemons. When life gives you lemons, bring vodka. Look, a lot of us are put in what we could call impossible situations, right? You walk into a job, how many of you guys have ever done an interview where you're like, "Man, this is awesome. They're going to give me all this stuff to do, all this accountability." And you get in and go, "Crap, this is not doable." Right? There's three of us in security. I've got myself and two junior associates and there's 47,000 people in the company.

Awesome. And my budget is a stick of bubble gum. And there are four very, very terrifying words that some of us get to earlier in our careers that we come to that stark realization. And sometimes you get pretty long in a tooth and it still doesn't make sense to you. And there is no secure. Last fall, I keynoted a conference in Omaha, Nebraska, where I said, and I'm glad we're finally getting to that point where we all realize and acknowledge the fact that nothing is absolutely secure. And a guy, no joke, raises his hand and goes, my group is secure.

What do you say to that? Right, right, right. I'm sorry, would you like to hold up your business cards here? I'll be right with you. But look, we get to these situations where we're being asked things that are just simply unrealistic of us. But we need the job. and it's still cool, and we do our absolute best, which goes back to the superhuman effort scenario and things go sideways. And what we're really trying to do, ultimately trying to do, anybody else, anybody here ever worked like on a biz dev side, outside of security, like a business consultant kind of role? All right, so if you haven't, I don't see a lot of hands, so if

you haven't, if you work in an enterprise, if you don't work in a security company, you work in an enterprise, I very highly, highly recommend, if you can take three to six months, if your organization will let you, go work on the non-IT side of the business for a while. Go be a business analyst or go learn risk management. It will give you a fundamentally different appreciation for what we all do here. Because I know everybody in here, myself included, has found yourself saying, "What kind of idiot would put these requirements in?" And then go work that job, you go, "Oh yeah, I would because it has to be done." But let me be a

little more clear. We keep using the word risk, right? One person clapping, so has been through this before. We keep using the word risk, and it doesn't mean what we think it means. Risk does not mean, you know, we've conflated risk with, like, vulnerability and all sorts of different things, but fundamentally it's an equation, right? Probability and impact. What's the probability of anybody getting hacked? So there's a great quote that on a sufficiently long timeline, right, the probability of everything is 100%. If you give it to infinity, everything is likely. So on a sufficiently long enough timeline, in security that generally means about 30 seconds, crap will happen. And so what you're left with is this unstoppable

force and this immovable object syndrome, where, anybody else hearing that? Should we be running? Look, unstoppable force and immovable object, I bring this up because we get into a situation where I've been in a couple of consulting in a couple of organizations where you've got a C-level executive, CSAIL, that says, "This company, I'm new, I'm going to put my foot down, I've got board mandate to make us secure." And then the business side says, "We've got a board mandate to grow the company, and we're going to do it this way." And then you've got this collision of heads. How many of you guys have been a part of a company that's happened with? Who wins? Wrong, nobody. You may think the

business side wins, but eventually they make decisions that are poorly informed on the security side and that costs them. Let me give you an example. You guys are all familiar, painfully familiar right now probably with the Spectre and all the other processor issues we're having. So what, 16, 17 or however long ago this was developed? Somebody made a business decision that said, hey, we can speed up our processors by 20%, 25% or so. If we cheat, I promise you there was somebody in the room or somebody in that chain of command that went, this is not a great idea. This could lead to, and they went, yeah, shut up. We got money on the table.

Put yourself back in that time, go to the time machine, sit back at that table. Would you have made that same decision? What would you have done? Because keep in mind, you've got 16, 17 years of revenue of kicking AMD's butt six ways from Sunday, right? Is that revenue worth the What has been the impact of this? Who's gotten hacked as a result of this? I mean, we spawned a ton of hype around it. It was the end of the world, right? How many news cycles did we spend on it? A lot. Anybody, any major events that we know? So back to that, using that word risk, right? Yes, the impact is potentially cataclysmic. The likelihood? Eh, because a lot of us are still

on an MSO 867 problem. Who's going to take the time to develop an Oday when they can scan you with Nessus from 15 years ago and realize, still have the problem. Great. So we get into this situation, and reality is when security and business butt heads, everybody loses, including us. So the number one rule of security is don't break the business. You learn that very quickly after you get fired your first time. You like how I put that? Your first time? The next big thing I want to talk about is build, buy, and partner versus ego. Look, in security, we tend to want to, and there's a lot of companies that have this trend right now, a lot of the big financials, they've been hiring

people to build all their own tool sets. We're not going to buy stuff from vendors because we're going to build our own. We know better, which is a great thing to do. If you just go into it knowing that the people who build it will be gone six months after they've built it because they're not going to maintain stuff. Because if you're the kind of person sitting in this audience that goes, I can go build something cool, And I let you, and I say sit on that for the next two years. What are you doing? Plotting your next thing, right? There's a builder, there's a fixer, and there's a maintainer. There are three different mindsets.

You are not two of those. You're not three of those. You're one of those. So build what you can. Build the things that you have to that nobody else can do. Buy some stuff and let somebody else do the rest. By the way, this checks our ego quite a bit. I know we don't have an ego problem in security, but I thought I'd just leave that out there. You laugh. So it's time to make a couple of really tough and difficult decisions because ultimately what we decide impacts how we do things, right? So some things you build, some things you'll buy, and you've got to pay somebody to do the rest. I don't cut my

own grass. You know why? So I've got two four-and-a-half-year-olds that I'd rather spend that hour, that hour and a half, two hours with, and that's worth the $35 that I pay the landscaping crew. Does that make sense? Some of you guys drop your clothes off and have them laundered. A lot of you guys go out for lunch every day. Why? Because the time it takes to make your own lunch every day, it's worth it to hang out with friends or go somewhere and chat for the ambiance rather than making. We make those decisions all the time. It has to apply to our craft. There are some things that we simply should not do on our

own. Some of you guys are working in companies that are struggling with hiring, staffing, training, and maintaining security operations center staff. Anybody have that problem? Liars. The rest of all of you do. How do I know this? Because I've worked with companies that are trying to hire, and it is impossible to find good people. Because you're hiring from somebody else, which means now they have that problem. So look, ultimately you're going to have to say, this is not something I'm going to be good at. And look, even big companies have, this is not a small company problem. I worked at GE, one of the largest companies on the planet, worked in one of the businesses. There were five security people for almost 40,000 employees. This is not limited to

small companies. This is not limited to a geographic location. This is not limited to any particular market vertical. Everybody has a problem. Even big teams have these constraints. You get overloaded. There are 10,000 things to do and only 29 hours in a day. You have to get stuff done other than work. So you have to ask yourself, what is my organization's core competency? What are we good at? And if you have engineers, go engineer. If you have analysts, go do the analyst job. Because there's a great quote, "Fish are really bad tree climbers." If you've got a group that's fundamentally analysts, people that are really good at identifying and responding to issues, asking them to go architect solutions is the

wrong approach because their brains don't work that way. They fundamentally work differently. So support critical business functions and outsource operations. That is the trend that's been happening over the last two years. And this isn't like, hey, manage my firewall and AV for me. This is like, I don't want to do endpoint security at all. This is like big companies that have been changing the way they function. Because what they say is, look, I can hire 10 people. Or I can hire 25 at a lower pay rate. I'd rather hire 10 at a better pay rate so I can get top notch business and security analysts and risk analysts. And I'll outsource those other functions to somebody that can do it better, faster, cheaper. Be careful on that cheaper

part. And that brings me back to what is it that you do here? Because that is an important thing to ascertain when we're trying to understand What do we keep? What do we move? But fundamentally, look, we have to decide who we are. And absolutely take your ego out back and leave it there. Because at Armour, there's a coat rack right when you walk in every morning. And there's a sign that says, check your ego here. I'm telling you, this industry has a problem with ego. And I get it. We're all the smartest people we've ever met. But be the dumb guy and gal in the room at some point. Understand your peers are really freaking smart. And the people that don't do security and don't do

IT aren't automatically idiots. Because I've had conversations with people. For a while, I consulted on organizational change management from a security perspective, helping companies, security teams, try to kind of get back to right with the business. And you start interviewing people. You're like, okay, what's the problem with this team? Start interviewing people. Start talking to them. You're like, oh, I get it. Their organization hates them. And you go talk to them, and they're like, oh, it's because they think they're the smartest people and they think everybody's an idiot. I'm like, great. This isn't going to last, right? So you can't have that kind of relationship with the company you work for. You just can't. And

we have that relationship with each other, so that's got to stop first. So remember the Kenny Rogers rules of poker? Anybody know those? Hold, fold, walk away, or run the hell out. So there are four situations that I will tell you you will find yourself in your career. One's where you just got, you can just, somebody's going to tell you, it's rough, but tough it out. It's going to be cool. Just give it some time. We're going to make this work. Sometimes you got to say, look, this isn't working. I'm going to try something else. Sometimes it's time to walk away because there's nothing you can do that will make the situation better. And sometimes

you just got to run for the hills because, yeah. And remember, G.I. Joe taught me. All right, hold on. At the same time. This is how. Awesome. So the lesson here is pick your battles, right? Know what battles you're willing to fight. Know what you do. Know what's important. Know how you're going to solve it. Pick a strategy. Because the desired result we want is we want to be partners with the people that run the company. We want to help enable. We want to be leaders. We want to be part of the solution. The reality is we end up part of this because we end up fighting. And like I said, when the unstoppable force

and the immovable object collide, nobody wins. It's time to update your resume. All right, let's talk about the hamster wheel. So security cycles are fairly predictable. I can tell you, I'll ask you guys where you think we are in a second, but this is what they look like. You start with need more process, and then you go, well, hold on, now we need more tech. Nobody ever remembers about the people, so I've left that out. Where are we right now in general? We're never going to remember the staff. Let's forget that part. There aren't enough of us. Anybody want to take a guess? Right. So the trend I've seen over the last three years of working with about 5,000 different

types of companies at the leadership level is your security leaders, CSOs, directors, managers, whatever, are looking at their budgets, looking at the stuff that's in the organization, saying, I've got truckloads of crap. I've got tools coming out my ears. What of this actually works? So some of the most popular things if you're a consulting company, some of those popular things your clients are asking for is go through my tool stack and tell me what I should keep, what I should throw away, and what I should improve. That should scare you if you're selling stuff. Because on the client side, we have overbought dramatically. I'm not going to ask anybody to raise their hands because it gets embarrassing, and we'll do another one later. But think about how many

dashboards or how many interfaces every person in your security organization has to look at every day. I've been to organizations with like five, six, seven, nine, ten to one. I promise you that is a losing proposition every day, right? So I think we're going over the top of that more tech part, and we're about to hit a downswing. Because you can see companies start to consolidate, right? The market's consolidating. There's a reason for that. The sales growth is slowing, right? It's not that people don't need stuff, it's that people aren't buying things rapidly at an increasing pace. They're getting rational about what they're buying. They're starting to ask really tough questions and the vendors that can't answer them don't get to sell. So what

we really want is this cycle that starts with a program strategy, a capabilities and gap analysis, process development, staffing, and then we make a tech purchase. That's what we want. That's what our bosses want. That's what our boards want. What we really get is this. Buy stuff, complain it doesn't work, fire people, throw stuff out, Get annoyed, rinse and repeat, and then shock. Why isn't this working? So my favorite is I used to work for a software vendor a number of years ago. And one of the absolute best cases that we saw repeatedly over and over is you went and sold a tool that required a lot of engineering support, right? Like the phrase out

of the box, like we threw it out of our vocabulary because there was nothing out of the box. We sold you a box of Tinker Toys that you had to then put together and make cool stuff with. So inevitably, at least a good chunk of the customers are like, no, no, I don't need your services. I got this. You come back a year later like, your crap doesn't work. Like, great. So how many engineers did you dedicate to it? What do you mean? Well, what have you built? Well, it doesn't work out of the box. I'm like, remember that conversation we had about how this is a box of parts? How do you staff this?

Well, I don't know. Well done. So we've gotten to that. And look, ultimately the three-legged stool, people, process, and technology. Technology should always be the last thing you consider. It's not that something you shouldn't consider. It should be the last thing you consider. Because by the time you get to the T of PPT, you should know exactly what the thing is you're doing is going to do, what the end result is, what it will plug into. And let's be honest. Information tends to want to liberate itself. Information wants to be free. Does anybody know, don't raise your hand, this is again another one of those questions, but anybody know where all your sensitive corporate IP is? All of it, right? Anybody have a

really good handle on what it looks like, where it is, who owns it? Most of us get the archer face. And the rest of us are lying to ourselves, which is back to this talk. But realistically, it is a tough answer because the answer typically is everywhere, right? What are the things that, when somebody's trying to sell you encryption, or data protection tools, it's data at rest, and, see what they don't, data in motion, right? What they don't tell you is the secret third one. Please, solve that one for me. I feel like a channeling Yakko Smirnoff. It's funny 'cause it's true. But this is the real world. It's not just data sitting on a server somewhere locked in a data closet

that only you have access to via badge and with your two trusted administrators who you vetted through the NSA. Because it's not. It's Janet in accounting who's like, hey, this is cool. I'm not going to be able to finish this, but I'm going on vacation. I could finish it on the airplane home. Let me Gmail it to myself so I can put it on my home laptop, which has 37 viruses and McAfee antivirus from 12 years ago. I mean Windows 98 still works, right? Security controls only work when they're implemented in environments that actually support them. So like I said, remember that the impossible task? The other side of this is you end up in an environment that tells you you can install three pieces of

software. One of them is the operating system, the other one's Outlook, and then something else that we allow and everything else is disallowed. That leads to people going, yeah, watch this, hold my beer. I've worked in those environments, trust me, I know how this goes. Because you walk around, your CISO says, you know what, we've locked down the environment, we don't have these problems. You're like, get your lunch. Three machines, two of them will have something you don't recognize on it. He's like, all right, let's do it. First one, nope, that's clean. All right, how about that one? Whoa, how'd you install that? They bribed somebody in IT with lunch. Which brings an entirely new definition to the term data roaming.

Like buffaloes, they like to roam wildly, data bits. So the question becomes, what is your actual strategy for this? And I'm very curious what we come up with. Because look, this is a complex problem that's not like, pick the tool that's going to solve that third use case. Go ahead, I dare you, name one. We'll all have a good laugh, right? It's a program strategy, you have to have a series of things. You have to be staffed for it, you have to have a culture for it. Oh, culture. The company that you work in, there's a culture that goes with it. If security doesn't fit into it well, That splat sound you hear is your career. So tool process people, rinse and repeat, right? Tools

always come first, which is wrong. You hire smart people, you let them build processes, and then you enable them with the technology they need to do their jobs. If you're ever applying for, I'm going to give you guys some free advice. If you're ever applying for another job, and the job rec has specific tools listed that you have to know, don't apply for that job. Because they have absolutely no idea what they're doing. You should be looking for companies and you should be hiring people based on capabilities rather than which version of Check Point they know. That's still around, right? Just kidding. Usually my buddy, Kelman, is in the audience to kick my butt, but he's no longer there, so I guess that's not a thing.

People are your number one asset, and I get it. We all know that. Does HR know that? Do we write job recs that way? Do we treat people that way? I mean, insecurity Forget the fact that in IT you get your own part of the building because they don't really want to deal with you. But in security, we do it to ourselves. How many of you guys regularly interact with the people that work on the other side of the company? Ask yourself. Do you know the people that work the business side? Accounting, sales, operations, risk management, finance, marketing. If you're not closely aligned with marketing as corporate security, you're doing it wrong. So, processes enable people to support

the business, right? Tools help humans automate and scale and avoid shiny object syndrome and end of story. The shiny object syndrome is extremely costly and oh, look, squirrel. So, the desired result we want is this great series of knowledge and information converted into dashboards and living things. And the actual result we get is more of that. If you feel like you're in that position, do something about it or get a new job. So, building security in, all right. We want to engineer things for safety because bolt-on security fails. We know that for a reason and I'm not going to go into those reasons. You have your own problems. But by the way, you have to show up. It's likely only obvious to

you what that means is when you're sitting in a meeting with a bunch of business people and you say, "How do they not get this?" They're thinking, "How do you not get this?" Figure a common language out, right? Guess what the word tax stands for? What it means? It's the art of making a point without making an enemy. We need to learn how to do that better. You guys know the concept of minimum viable security? What's the absolute minimum we have to get to so that we feel somewhat safe but the other side can still proceed? That is a tough thing to do. It goes back to check your ego. It goes back to know what it is that you do here. But that is a difficult concept,

that minimum viable. Because zero risk is not possible. You cannot engineer risk to zero. Back to my bubble wrap with your teenage driver analogy. Because creativity is boundless. I promise you, people get really creative in how they circumvent controls you put in place. And creativity has no bounds. And criminals become very persistent. Oh, by the way, that happens, right? Things go wrong. Our job, I will challenge you, is not to do security, but our job is to minimize the impact of failure. Compartmentalize, right? Contain. You will have a bad day. Your CEO will click the link. They'll get phished. I promise you there's been links that you've clicked. Raise your hand if you've never clicked

the phishing link. Uh-huh. Nobody's willing to lie in here. I like it. You guys are honest. I got one the other day from a friend of mine from his mailbox at the office. another company, high ranking, C level executive, said, "Hey, I'm sending this for you to review real quick, give it a shot. I put it on my OneDrive, I think it was. Just use your credentials to access it." And I'm like, "But we are actually working on something together, and it was named correctly?" So I'm like, and as I went to click it, I'm like, "Whoa, wait a second." So I text him, he's like, "No, I didn't send you anything."

Went to urlquery.net, turns out somebody was doing a botnet dropper. I was like, "Damn, this is good, it's timely. "They've got the right files, they've got the right language. "It's not obviously misspelled, thank you very kindly." They're getting good. And if you've ever been, there's an OWASP presentation that was given probably about seven, eight years ago now, where the difference between paypal.com and paypal.com, you could not tell because of Cyrillic and extended ASCII characters. Guys, we fall for this crap too. You know why? We're human. We make errors. So does your budget support or hinder that worst possible day? Because huge chunks of our budgets, and I've done enough of these, reviewed enough of these to tell you, are spent on prevention. We spend a little bit on

detection, and then for that oh shit moment, we're like, I promise you, if you were to go through and look at your line items in your budget right now for security, you're Incident handling and events, critical incident management probably doesn't have a line item. But hiring the attorneys, getting the communications out, buying credit monitoring, whatever the hell we do now for breaches, right? I promise you at least three out of four of you won't have a line item for that. And if that's the case, when you have that oh shit moment, you're going to have a bad day because that money's coming from somewhere. Maybe it's you. So looking at things like meantime detect, meantime respond, It's important because the desired result we want is happiness and everybody

giving us a thumbs up. The reality is we're bolting the engine on as we fly and hoping nobody notices when the ground gets close. So responsibility, accountability, and the paycheck. So we get to meet the Bobs again. I keep asking, what is it that we do here? Are you valued, respected, and trusted? Those words are put on there in a very explicit order. Do they value what you do? Do they respect your opinion? And do they trust you? If you can't say yes to all three of those things, you got to ask yourself, is it me or them? It's like that conversation every one of us had with our high school girlfriends or boyfriends. Is it me or is it you? Sometimes the answer is yes. Do you have

a clear definition of role? If your job title is analyst, what are the three things that you are accountable for? Only you, that you are accountable for. Architect, analyst, nebulous roles. A lot of us have these. What is your success criteria? What are your MBOs? Because if you don't have MBOs, then you've got to find the door. Because ultimately, look, your job is to get your boss a bonus, and your boss's job is to get their boss a bonus. We all work in the same world. And if you don't know right now what your boss is bonused on, what are you doing? What are you working towards? And I've been technically fired before. What that means is I was right, but I was

still wrong. Doing what we believe is right in an absolutist kind of way still gets you fired because sometimes the answer is that's okay, we'll take that risk. And you have to be able to say, okay, I get it because you're not the decision maker. If you keep talking and you've been in this situation where you're like, those morons over there, boy, I'm glad I don't work for that company. Oh, wait, what's on my paycheck? Keep in mind, where you work, everybody's in the same boat. Know the rules you're playing by. Know the tolerance, the risk tolerance of the organization. Know what they're willing to accept, what they're not willing to accept. And if people

are asking you to sign off on risk acceptance, when you make a judgment and say, this is something that should not proceed, and they go, yeah, yeah, it's fine, I'll sign off on it. If they're not a company executive, they can't sign off on risk, which means you're accepting it, and when crap goes sideways, you're fired. Advice from an attorney. So, accountability and authority, they're like love and marriage. Really? Nobody? Okay. Don't accept the accountability if you don't have the authority. That's as simple as it gets, especially if you're in a position of leadership. If they're asking you to move boulders but not giving you the tools, this is not the job you should take because you're going to get to that point where you're going to be mentally

frustrated, you're going to get nasty and angry and upset. And it's going to impact you. Because the desired result we want, we want to have a good day at the office, right? You want to go home and you want to feel refreshed. You don't want to go home and have to crack open the third bottle that week. But the unfortunate result is sometimes we get to be the cheapskate officer. I've met lots of those. And if you don't know what you're walking into, that's probably it. So the last part of this is really on to tomorrow. I think as an industry, we need to get over ourselves. Squirrel. We really do. A lot of this community still believes that we are the absolute right

and it's not going to continue that way. The result, the very simple result of security taking a week, two weeks, hell even a day to create a secure server environment is marketing using a credit card to buy an AWS instance. Good job on the win. I have met plenty of CISOs who are like, "Yeah, we've got our security locked down tight. We build every server. We've got templates. We get everything done." Like, "How long does it take?" "Oh, usually about two or three days." I'm like, "Yeah, so your employees are probably buying AMI instances right now." They're like, "No, no, we don't do that. Audit your expenses on your corporate cards. I promise you they are." Guess

what? We're security in that. Get flexible or get used to being run over. Sometimes you're right and sometimes you're wrong. Yes, sometimes we are not right. Ivory towers don't exist. We don't live in one. Go be part of the masses. Follow the golden rule. What's the golden rule? No, it's don't be an asshole, right? Yes, treat others like you want to be treated, but also when that fails, don't be that. It's not hard. Just be nice. Understand that what actually matters, right? For a lot of us, this is it. This is our lives circle around our jobs. We go home and we do security for fun. We go out with our friends and we do security for fun. Go for a walk in the park, kayak

the bay, go smell the flowers. I promise you there's more to it than what we're doing here. While this is a very noble cause, if you don't disconnect what ends up happening, and this is 100% every time, you will eventually hit a point where you're mentally in a bad place. It will burn you out. This industry eats people every single day. If you don't know what I'm talking about, you haven't been paying attention. So disconnect mentally. Force yourself. Turn off your phone. Don't read Twitter. Please, God, don't read Twitter for a while. Hashtag no politics. And understand what actually defines you. Who are you outside of, if you take InfoSec away, what are you? What

do you do? What do you like? What do you hate? Are you a sci-fi junkie? Do you like to base jump? Like, what do you get, what do you like to do? Because the sobering truth is this. The best kind of servant is the master, right? When you get really good at something, when you get truly good at something, you'll teach others, right? You're not going to, you know, the people that are really bad leaders are the ones that tell themselves they're leaders. I'm a great leader. You're like, yeah. How many people are following you? I'm forcing them to. That's not a leader, right? Most of us probably aren't, but some of us are, which is awesome. I'm not. I just work really hard. Check your ego.

Respect your peers. It doesn't matter if you agree with somebody. It doesn't matter if they work in a different organization. It doesn't matter if you agree with their responsible disclosure policy. Respect people. It's really weird that we have to do that, like actually call that out. So mental health, relationships in real life, seriously, take a moment, smell the flowers, reflect, you know, just listen to music or something. This is an amazing career. My dad worked, still works, 37 plus odd years at a machine shop. You want a crappy job, go do that. Like you guys, we're probably getting, almost everybody who's not getting paid to do this today. So most of everybody in here is

getting paid to sit here and listen and learn. to me ramble on but that's besides the point isn't that awesome how many of you guys hate what you do for a living exactly this is an amazing career we should build healthy relationships as a result there's no room for toxicity that does not need a picture you know what i'm talking about the desired result really is i'd love to work in a world like this where we all collaborate everybody works good together we take those of us that have been in the industry forever and then we apply the lessons that we've learned and the scars that we have to those that are walking in today

to give them some of the shortcuts, some of the crap that we went through that they shouldn't have to go through. They're not going to listen because did you listen to your parents as a teenager? Are you wishing you had? Probably, right? But the actual result is still to be determined. We're not there yet. So we have an opportunity, and I think we should take it. Billageville said it best. We didn't start the fire. So anyway, that's me. I'm done. Thanks for listening. Not everybody leaving. Thank you.

I'm going to be around today and tomorrow and then eventually going to deport me on Wednesday. All right. Thank you very much, guys. We are going to cut our break short to try to catch up on the delay. That's his fault, not mine. I was roughly on time. We'll get our talk started in about 10. Under the weather, okay. hey man how are you doing man good good to see you yeah i'm gonna hit me nice How's the project? Projects are projects. I think May, I will have the beta out in May. It's a project I'm working on. I've been working on it for two years. I've been releasing it in May. I'll be happy for you to check it

out. Yeah, we hope to... Good time to chat and talk and stuff. Yeah, maybe, yeah. So, maybe today? Lunch or something? I'm going to be waiting for that. Okay, okay. Maybe tomorrow, whatever. Alright, see you around. Bye. Yeah. What time is it tomorrow? Oh, tomorrow. You can find out that for you for sure. I guess I'll stop by later and I'll ask. Oh, no. I can't find out right now. No, you can. Okay. But I'll have the answer. I'll ask. Did you guys actually record? Yeah, it's being recorded too. Cool. Did you get all the audio? Yeah. It's all recorded and... I think there will be a copy on the website. I don't know when, but...

It's streaming now on YouTube. Cool. Yeah, it was streaming. That's good. Maybe. And I'm sure there'll be a recording version on the site. All right. Let's see if this works.

yeah so far so so fast so good it was big enough yeah I'm just broken I'm a hackable I'm a hackable haha that was a good joke thanks appreciate it Yeah, working on the AV stuff. The keynote just finished and then we've got another speaker coming in soon. - Like, you're good. - Was that for some kind of event or did you just rock it down? - It was kinda crazy yesterday, so I was just doing that. - I did the sides at one point. - I had that, but then the sides were getting crazy. - Yeah. - So I don't like it. - Just line up for the bear. - Yeah, okay. - I'm

sure they don't mind, but it's cool. Okay.

Just getting ready for the next talk. Here monitoring the AV stuff. I have a little special bench. Still ready? Yeah. I agree. I have two. I don't think I need it.

um um All right. is All right, we're going to get started again. Test, test, test, test, test, test, test, test, test, test, test, test, test. All right, all right, everyone can hear me. All right, we're gonna get, whoa. We're gonna get started again, and I'm gonna stand behind the mic, the speakers. Again, we cut our break short to try to catch up on time. Apologize for the delayed start. Alphabetical order badges would have been nice to have, but we can always wish for next year. So, yeah, we're going to start, try to get back on time as close as possible. If you are in need of swag that you are owed at lunch, there'll be plenty of time to drop by. If

you're a leader professional or a speaker, it's already held out for you. I think that's it for house cleaning. So I'll introduce our next speaker. Adam Szostak, he has spoken at a couple of our B-sides and a couple other events that we've done. Great speaker. I'm really excited to learn more about near and miss bias. So take it away, Adam. Thank you, Alex, and it's a pleasure to be back here. Just give you a few words on me. I helped create the CVE. I'm on the review board for Black Hat. I wrote a book on threat modeling. If you're here to hear me talk about threat modeling, I apologize. That's not my subject for today, but Bob Fruth,

who's right here, is going to be talking tomorrow. Bob and I worked together better part of a decade while at Microsoft, so I'm sure Bob's talk is going to be great on threat modeling. What I want to share with you today is an ongoing interest of mine, which is how do we do better at the things that we're doing? And this is a research agenda that goes back to the New School book that I wrote, goes back to a number of other projects. And what I hope to have convinced you of by the time I'm done speaking in about 45 minutes are three things. The first is that experiments in cybersecurity are hard to do for a whole bunch of reasons. We'll talk about what those are.

The second thing is that near misses allow us to do new and different experiments that expand our knowledge in important ways. And third, there's some complexity there. I think it's overcomable complexity, but there's still going to be some complex bits. I said I'm going to speak for about 45 minutes. There's also a little bit of complexity to this talk. And so if something I'm saying is confusing, you literally don't follow, please interrupt me, get a clarification right then and there. If you disagree with me, I'm going to ask that you wait a few minutes. I may cover the topic that you're disagreeing about in the structure of the talk. And so with that, The way I've structured this is three things. The first

is science, how we do science, learning from mistakes. The second is calls for an NTSB for cyber are not working. We've had a lot of these calls over the years. We still don't have one. I'll talk about why. And then I'll talk about near misses. I want to start off with science. I think science is cool. And I'm going to give you a very brief, limited overview of what science means. I'm using it in a particular way. I'm doing that intentionally. If you think science and cyber or science and security are a neat thing, check out this paper by Paul Van Orshaan and Cormac Hurley. It's great. The slides are online, by the way. The slides are

online, there's papers online, there's a bunch of stuff, I'll give you links at the end. But fundamentally, science is about acknowledging errors. The difference between astronomy and astrology is you never hear an astrologer say, "Oops, I was wrong. You didn't have a big opportunity this week." It doesn't happen. You'll hear scientists say, "Oops, we measured that wrong. We misinterpreted our data." Oftentimes they'll say, "You misinterpreted your data," which is a slightly different thing, but it's accepted scientific practice to critique and to analyze because this critique and analysis of our work allows us to learn what's wrong so we can set those ideas or the limits of ideas so we can set them aside and look

for new ideas. This is super important to the way we do our jobs. It may not sound like it's an important thing for the way we do our jobs, but if you keep doing the same thing over and over and it's not working, Maybe we should start thinking about ways to do things differently. And the advantage to science is that it allows us to think about, this isn't working. How often does it work? Is it worth the investment we're putting into it? If not, we need to open up new vistas for exploration. We need to find new ways of doing our jobs. When you talk to philosophers of science, they talk about hierarchies of evidence, that some forms of thinking work better, they expose the

world better than others. And so the weakest ones are expert opinions that I think this is the right thing to do. Stronger than that might be a case study where we lay out what we saw, We analyze what we take from it so someone can follow the line of reasoning. There are lab experiments where You set things up in carefully controlled circumstances and say hey I Expect to see this and I actually saw it. So like if you're gonna try and See whether or not general relativity works. What you do is you move the moon to be directly in front of the Sun so that you can observe the light of stars behind it and see whether

or not a the light moves as the sun comes close to that star. It's easy, right? You just have to move the moon, which, as it turns out, is difficult. And so there are things like randomized controlled trials in medicine. We can't get people to change the amount they exercise, to change the way they eat, to change their genetics. And so we create studies where we don't have that precision of a laboratory study, But instead what we do is we line up people so that the two different groups are approximately equal in ways that we think matter. And then we double-blind it. We don't tell people, "You're in the control group. You're in the medicated group," because it turns out that there's a placebo effect.

We also don't tell the doctors who's in which group. because it turns out that doctors will give subtle signals that change the results of a medical treatment when they know their person's in a control or getting an experimental drug. And so in medicine, they've created what they used to think of as the gold standard of randomized double-blind studies. And then they started to do these meta-studies where they'd take all of these studies, pull them together, and say, this is what we take from the combination of these studies. And then I look at what we do.

So, there are limits on experiments, right? There's time. If I'd like to understand the effects of childhood nutrition on senility, that takes 60 or 70 or 80 years to run the experiment. Most graduate students would like to graduate in that time. There are limits of money, right? Running that same study, keeping track of where people go for 60 years is going to be expensive. There's physics. You know, I joked about you move the moon. It turns out that we were lucky that right after Einstein published the theory of relativity, there happened to be a total solar eclipse that allowed these observations of stars close to the sun so that we could see their apparent position move.

Let's see, there's ethics. There's all sorts of studies that we can't do because they involve harming people. They require fooling people. This is a big problem when we try and study phishing in the lab. Turns out these things called institutional review boards, which are there to make sure that scientists don't do unethical studies, object to studies in which you lie to people. Weird. There's an issue with reward structures, right? Reproducing an experiment, right, demonstrating that Spectre works on your laptop doesn't win you the pony in the same way that discovering Spectre or Meltdown does. And so I think we have a serious knowledge problem. I think we have a problem which is that we don't know how computers are

pwned at anything approaching a statistical scale. And I say this because I had the opportunity to drive the fix for Autorun into Windows XP and Vista, which had a dramatic effect on the amount of malware out there in the world. And when I did it, what I found was that the reason we couldn't fix the problem before was because we were very focused on this family of malware does this, and this family of malware does this. And I had been doing work on how computers are compromised to try and under-- actually, I was focused on zero days, not autorun. What I learned was that if we bucketed all of the autorun malware together, it turned out to

be something like a quarter or a third of all of the infections that were being cleaned up by the Microsoft malicious software removal tool each month. And so we went, we pushed that fix to the world, and then my boss said, Adam, go do it again. And I was like, okay, that's great. So I had a chit from some very senior people inside of Microsoft that allowed me to go around to partners and say, Microsoft wants this. And if you ever get the opportunity to play with one of these chits, I recommend it. It's fun. So what I did was I went to all of the anti-malware companies and I said, I'd like to know how computers get infected. And they said, well, the malware

starts up and then it writes these reg keys and then it does the, no, wait, wait, wait, wait, wait. Before the malware runs. Oh, it's exploit kits. Well, what do they do? How do they work? Well, didn't get very clear answers from them. And so I moved on. I went to the forensics companies, the incident response companies, and I said, "How do computers get infected?" And they said, "Oh, it's phishing." And I said, "Great, can you give me statistics?" And they said, "No." And I said, "Could you give me statistics? Could you go and ask your customers?" And they're like, "No, our customers don't want to pay. They don't want to pay to figure out

what went wrong. They want to pay to fix it." And so I said, I have this chat that says Microsoft wants. So I said, if I paid you to do that work, could you do it? And they said, well, that's interesting. Let us go find out. And so they went and they talked to some of their customers and they came back and said, nope, can't do it. That was my response too. And I said, well, why not? What's going on? And they said, our customers are worried about getting sued. They don't want a report from us that might document a failing in their defenses. And so even if you were to pay for it, we couldn't give it to you. Well, that's too bad. So I

have a question for you. Since I've spoken here before, you might know that this is a trick question, but don't worry, no one's going to hold you accountable. So, between phishing and vulnerabilities, who would like to say that 90% of incidents, give or take a little bit, are due to phishing? You only get one vote, by the way. Okay. So, maybe 10, 12. How many would like to say it's due to vulnerabilities? Even fewer. Most of you are not voting because I warned you it's a trick question. I should stop doing that. So, the answer is both. So if you believe DHS, and this is a number that Veracode pulled out of some of their reports, 90% are vulnerabilities. If you

believe the Verizon DataBrute report, 90% of it, rudely, are caused by idiot errors, by which we mean people clicking on things they shouldn't have clicked on, by which we mean phishing. And if you believe IBM, 60% of all attacks are carried out by insiders. So if I combine the data here, what I know is that 54% are done by insiders using either exploits or vulnerabilities, right? And we're laughing, but if we don't know the answer to this question, how do I allocate my budget for defense? Which is more important, patching or anti-phishing tools? Without knowing the answer to this question, without knowing who's right, I know only one thing about my budget: I'm spending it badly. And that makes

me sad. I'd like to do better at the things we need to do to get our jobs done. I'd like to be able to do my job allocating my budget in accordance with the attacks that I'm facing. Now, I told you this was a trap, and it is, because we all know the simple truth, which is that The real source of attacks is sophisticated attackers. Sophisticated attackers. That was an accident. I animated it twice when I was getting going and then I made myself laugh. I'm easily amused. And so I left it in. But seriously, really seriously, if we don't know what fails, how do we go and fix it? And so I've been thinking about this question a lot.

I've been thinking about this question for a long time. And one part of the answer is people call for an NTSB for cyber. And this goes back to 1991. This is the first time Butler Lampson, Steve Lipner, and Peter Neumann put it into that report. And not only is this now old enough to vote, old enough to drink, it is old enough to rent a car without paying penalties for being young. This is a really old idea. There's a long list of people who have talked about it. That's very incomplete. There is no NTSB for cyber anywhere in the world. And so... So back in 2016, Steve Belivan and I wrote a piece for one of these commissions that's looking at what to do about the

problem and sending a report to the president. And we said, please don't call for an NTSB for cyber. Instead, call for an analysis of why it hasn't happened. And the response that we got was a little disheartening. which was not only are we not going to call for that, we're not even going to investigate the question because it turns out when you made us think and we said none of us know on this commission, none of us know what an NTSB for cyber would be. None of us know what it would do, none of us know what power it would have, and so we're not going to call for it at all. And I said well that's awfully depressing, I was

hoping to learn something. And then I said, well, you know, they don't know and I don't know and nobody seems to know. Maybe that would be a useful thing to know. And so I went and started looking into it. Yeah, absolutely. No, no, no. So your Department of Transport does something very similar in a different structure, which I only partially understand. But basically the NTSB, the National Transport Safety Board, is part of the Federal Aviation Administration and it's an independent investigative agency. So when there is an accident in transportation, mostly aviation but also maritime, public transport, commercial transport, truck and trains, A set of investigators will fly out and they follow a manual that says this is how to do an investigation. So, you know, a couple of years

ago we had that truck that whacked into a bridge pylon on I-5 North coming from Seattle to here. Some of you remember that. So what happens in that instance is the NTSB shows up. They sort of shoo the police away. They put up their own yellow tape. and they follow an accident investigation manual. It's about 70 pages long, and it says you'll have a lead investigator who does this, you'll do these sorts of, you'll interview people, you'll drug test them. It goes into the, you will find a conference room at a local hotel, that you can use for briefings because airline crashes are a big deal. And so they have in their manual, this is

what you do. And it's very prescribed. And they produce reports about what happened in the accident. There's a technical report. There's a human factors report. And they're not there to lay blame. They're there to determine the facts of the accident and explain them to the world so that the world can do better at preventing those accidents in the future. And so I want to say one thing and then go back to your question. So accidents are defined by law. in the United States in the transportation space, in the aviation space, which is that it's the death or injury of a person or damage to an aircraft. And that's important. I'm going to come back to it. With that, I want to go back to what you asked and ask

if I've answered it. So it turns out that Canada has super similar law. This was what I was able to find. You know, I hate being that person who comes to another country and talks about American law as if it was your law. So the Transportation Safety Bureau talks about occurrences rather than incidents. I don't know if that's relevant because I am not a lawyer and I am certainly not a Canadian lawyer. But the key to this, right, it is a person is killed or sustains a serious injury or the aircraft sustains failure. You have this. Being directly exposed to jet blasts, that's not part of US law. I found that sort of interesting that that was actually counted as an occurrence which you

are legally required to report to the Transportation Safety Board Bureau within 30 days. But when I compare aviation to computers, There's a single regulator. It's the FAA or the TSB. There's no single regulator who determines whether or not I can sell a computer. There are now lots of regulators. Just in the United States, there are lots of regulators, right? If I want to sell a pacemaker here in Canada, I have to go to an agency and get permission to sell a medical device. If I want to sell that same pacemaker, in Great Britain, I have to get specific permission. If I want to sell it in the European Union next year, I'd have to get additional permission. And so the

fact that there are different regulators and no overarching regulator makes a difference to our ability to create a single investigative agency. Accidents are defined by law in aviation. There's no clear law saying what an accident is in computers. Personal information breaches might be relevant, but they're not a complete definition. And I'll come back to this. An accident has a physical locus, right? A plane crashes. There's an area around which you can put yellow tape and collect the parts. When a computer has, when there's a computer problem, there's a court case going on in the US right now where Microsoft is suing the Department of Justice. It was just at the Supreme Court. Microsoft said, "We don't have access to these computers that are run in Europe."

And by the way, I left Microsoft a few years ago. Nothing I'm saying here is speaking for them. There's a really, really interesting friend of the court brief by Google. And Google says, we don't even know where a set of emails are physically located because we load balance them across different data centers. We geographically move them to be close to the people. And if they're traveling, we have no way of finding out precisely where those bits are on disk. And so we couldn't tell you where a specific email lives if we wanted to. And that's really interesting when you contrast it with the physical locus of a transportation accident. The bits might be literally anywhere in the world. They might be located at multiple

companies. They might be located in caches all around the planet. There's no locus to an accident in that same way. It's hard to hide a plane crash. It's easy to hide a breach. I think that's reasonably obvious. People die. When planes crash, the result is people are no longer with us. And I believe this will change. But today, people rarely die because of problems with their computers. And finally, there's industry support in the aviation space for objective investigative bodies. there is strong opposition for these and other reasons from the computer industry. If you talk to executives at these companies, at technology firms, they're opposed to someone coming in and looking and saying, "Oh, that looks like a problem in your code that was responsible," or, "That looks

like a problem in your procedure." And so there's a big gap between the NTSB and what we do in the computer space. And so it looks very challenging to create an NTSB, but there are other aviation safety programs, and I looked for the Canadian equivalents to these. If anyone can tell me what they are, I would love to know, because... The International Civil Aviation Organization in Montreal does a really good job at creating regulatory harmony between different countries. The laws are super similar because planes fly internationally a whole lot. And so I believe that these exist here. I'm just not sure what they are, but there's a couple. The first is there's a couple of other aviation safety programs.

ASIAS. If you fly a commercial flight that lands in the United States or traverses U.S. airspace, it is sending telemetry to Herndon, Virginia, where the ASIAS office is, and they do analysis of what happens. And so what they will find is things like, hey, you were landing your plane in Seattle, and flights that you bring in have more wing flutter than other airlines. And they'll send that message off in private so that you can figure out what's different between your flight path and other people's flight paths. ASRS, the Aviation Safety Reporting System, is run by NASA for the FAA. And it's near-miss analysis. What happens is if you have a near-miss, you fill out this two-page form, you send it to NASA, NASA stamps it, they send the header

strip back to you, And if the FAA then shows up and says, "Hey, Bob, what the heck were you doing flying that plane that day?" You say, "Hey, look, I did the right thing. I sent my form to NASA." And the FAA has agreed to treat that as what they call evidence of constructive engagement, which essentially means that they'll put their thumb on the good side of the scale a little bit and not penalize you quite so harshly for making a mistake. That's interesting. The reason that this is run out of NASA, not the FAA, is because no one wants to raise their hand and say, hey, Mr. Regulator, look at me. I made a mistake. So NASA, there's agreements that NASA won't send

this information directly to the FAA. It always goes through the person. And so I've been thinking about near misses, and near misses happen when a subset of your controls fail, right? Some fail, some don't. And that makes it perhaps a good story. It makes it less embarrassing. But most of all, it means your lawyers are probably far less worried, far less worried than they are about a time where you actually had a real serious problem. And so when I say controls, I mean everything. It's the people, process, technology, to coin a phrase. It's the human controls. It's the process controls. It's the technical stuff. And if you believe that one of those is going to stop a problem, then you only have a real

accident and incident and occurrence when they all fail, because otherwise one of them will have stopped the problem. That's for preventative controls, right? We can talk differently about detective controls. But most importantly, if some controls fail and others succeed, maybe we can learn from that. And it turns out that they do this in a bunch of other fields. So this is an example from a medical journal, and this is a model called the Swiss cheese model by a guy named James Reason. And what you see over here Our hazards start on the right and they go through the holes in the Swiss cheese where the defenses aren't working and they generate losses of various forms. This one,

which comes from the nuclear industry, the hazards come from the left and they go through and sometimes they come from the left and they bounce off a piece of cheese. This is from the nuclear industry. Why is the cheese blue? Exactly. Thank you. It's the Chernyakhov radiation. So, near misses get studied and they give us ways of thinking about what works. So if you have a URL rewriter, maybe when the message comes through your mail server, the server hasn't been taken down or flagged as malicious, but by the time a human being clicks on it, maybe it has. And so when the URL rewriter fires, maybe that teaches us something about the SMTP server. Maybe it also teaches us that there are

other ways that messages come through to the human beings inside your organization. So it's not, so we have to be careful. We have to be nuanced in our thinking about what we're going to learn. But this gives us new categories of experiments and it gives us new categories of evidence. It gives us new ways to do our jobs better. And so, Over the last year or so, I've been working with both Steve Belavin and the University of Colorado Law School, and we have a paper with the same title. That was close. Reward Reporting of Near Misses. And this is coming out in a technology law journal in a few months, but there's a draft up there.

I should mention briefly that I'm speaking on my own behalf here. I think my co-authors agree with most of what I'm saying, but really I'm speaking for me. And so to summarize the paper, legal concerns make discussing incidents hard. We can talk about near misses, and we should incentivize disclosure. And it turns out we think the argument is reasonably strong. And we think that there are some open questions. And so I would love some help with some of these questions. And there are three areas. The first is around regulatory specifics. Fundamentally, I think that better information sharing, better information analysis is a public good of the sort that we should want to have. And So there's some help we're

going to look for from regulators. I would love to talk about doing some of this here. I don't know if you've heard, but we've got some dysfunction in the U.S. government right now. And so maybe it would be easier to get things done in places that are governed a little bit more sanely. It turns out that the definition of accident is a little difficult. I'll talk about that. and maybe it's worth experimenting with reporting. So regulators in both the US and Canada are allowed to exercise judgment in lots of ways. And as an engineer, this frustrates me, right? I want clear line rules. Do this and it's okay. Do that and it's bad and everything is

covered, right? Binary, nice, easy to understand. And then you look at things like the Protection of Personal Information Act here, which in just about every paragraph refers to the reasonable person test. Would a reasonable person think this is okay or not? So as an engineer, that frustrates me. And then when I'm doing this work, it excites me because what it means is that your regulators have to exercise judgment. And if they have to exercise judgment, then maybe they are able to say that looking, that participation in a near miss program is a factor we would take into account. A lot of regulators already say things like, if you've cooperated with a cert, we will take

that into account. It will be a point in your favor. If you're cooperating with law enforcement, it's a point in your favor. So it's not far from the tree of what they're already doing. Now, I would love to see regulators agree to avert their eyes from near-miss repositories. And this one is a little trickier. But the logic goes like this: Today, there is no near-miss repository. They can't go and look inside and see what you've reported. And if they don't agree to not peak, the odds that people will report things are much lower. And so if they agree to avert their eyes from near-miss repositories, as they have done in the United States with NASA and the ASRS,

then the rest of us will learn useful things. If they don't agree in that fashion, then no one learns anything. And so I think there's a reasonable argument to be made for the averting of eyes from this, from collections of near-miss data that we might have. We need to scope out where you get those points and where you don't. There are things that might be so bad that reporting a near-miss in that way, it's not going to get you anywhere. This is not intended as a get-out-of-jail-free card. But I would love to have the conversation about who needs what. One final thing to say about working with regulators. Regulators and technologists come from two very different worlds. Regulators do not generally

talk about innovation. They don't talk about how they're doing new and different things. And so when I am talking to regulators, and if you happen to be talking to regulators, this is totally normal stuff. People already do exactly this. It's no big deal. When I'm talking to engineers, I speak about it differently. We like innovation. And both are true in a reasonable sense, right? We're doing things that have been done before in many other fields, just not in ours. And we can learn things as we do them in our field.

Now, the definition of accident is interesting, right? I showed you those definitions in US and Canadian law for what an aviation accident or an occurrence is. Turns out that in computer security, our definitions are all over the freaking map. There's never been any reason to constrain those definitions. Nobody pushes back and says, excuse me, the reason to constrain the definition is around breach notification, right? There's strong pushback, there's debate over when do you need to report a breach. But an incident, the broadest definition of an incident that I could find was the one that US cert has. And it says that the violation or imminent threat of violation of policies, acceptable use practices or standard security practices

"Gah, what doesn't that cover?" And I say this with respect for the folks at US Search. I'm not saying this to criticize them. They made an effort to figure out how to define this stuff that they were supposed to help with, and they cast a wide net to try and be helpful. But if we're going to do near-miss work in cybersecurity, We need to figure out what we mean by an accident so that we can understand what we mean by a near miss. Or we need to figure out what we mean by near miss so we can say that thing over there is not a near miss. We need a crisp definition because when we go to your lawyers and say we want you to report

this stuff, they'll say, what do we report? What are the corners of the page? How do we make that call? The more crisp our definitions are, the easier it will be to get participation. Was there a question back there? No? Okay. Sorry, I thought I heard something. That's cool. Now, there's a hack. And the hack is that if regulators agree to avert their eyes from the database, it may mean overreporting is okay because you're not going to get any benefit from the overreport, right? You report something that's really an accident, The regulator says, "Well, we're not going to give you points for that, but we appreciate that you did it as a matter of general policy." Then the near-miss analysis center learns more, because more

stuff comes in, and there's no downside. So it may be that we don't need the world's most precise definitions if we can get other ways of encouraging participation from the people who will be handing out the rewards. But as I said, crisper definitions are going to increase comfort. It might make sense to do some experiments with reporting. You know, how many reports might we get? What might be in those reports? How much time and energy would processing and anonymizing the reports take? Maybe it would help us develop confidence that we're doing these things well. And so maybe doing some experimentation would be helpful. And we're actually doing that. We've gotten the first several reports. I would love to

hear more. There's a bunch of collateral that's available. We have a sample reporting form. I'll give you a link to it. We've created sample memoranda of understanding that might exist between departments to say, "Yeah, we can concretely do this." we have a sample of a report. I did some analysis of public breach reports and one of the things that comes out is we don't know how the attacker got in. We don't know how the attacker got in. And so I said, well, where are the logs? And it turns out The most widely deployed operating systems, this is true of both Unix's and of Windows, defined their log retention policies when disk costs dollars per megabyte. Windows last updated the default retention

policies in 2003. Linux, I don't believe any of the Linuxes have actually updated their default log retention policy from what came with BSD 4.3. BSD 4.3 is even older, by the way. No, it's not quite older. Almost as old as the proposals for an NTSB for security. So we wrote up a thing that says, hey, maybe it makes sense. Operating system creators, update your default times. Operators, update the times in the images that you create and deploy. So we can learn things from this. We'll learn other things as we experiment. So some key takeaways. I told you at the beginning I hope to convince you of these three things. First, experiments are hard for a bunch of reasons. Second, near misses

give us new kinds of experiment. And third, and this is a slight variation on the opener, near miss science will be both tricky and worthwhile. I believe I'm putting my energy into this. I'm here speaking to you about this because I believe that the fundamental question we have of am I investing my security dollars well Am I putting my budget into the right things is unanswerable today and that makes me angry. I need to be able to know where my budget goes if I'm going to defend the organizations I'm helping. And I believe near miss reporting, near miss analysis, publication of these reports has been fundamental to improving safety in aviation, in medicine, in nuclear power plant operations, and it

is a shame. It's a real shame that we do not have the same thing for us doing our jobs. And so, With that, I'd love your help. Please spread the word. Talk to people about this. Talk to people about the idea. Help me figure out how to make it happen. We have regulatory questions. We have definitional questions. The better our answers are, in his talk, Rafe talked about the people. He had this great image of people staring across the desk at you, trying to decide if they could trust you. If we do, the better our answers are, the more we've done our homework, the better the response will be when we talk to policymakers about this sort

of issue. It's part of the reason I'm doing a law review paper is because that matters to policymakers. If you know people who want to help, I would love to talk to them. Lastly, at my website, adam.chostak.org/nearmiss, there's links to this presentation, there's links to a reporting form that says, "Please tell us what happened in a near miss that you've experienced." Think about it. Go fill that out. It would help us drive the project forward. It would help you understand the concrete of what we're doing and why. And so with that, I'd like to say thank you very much for your time and attention and answer any questions you have. Thank you. Alex, are we recording this? And should

we have people using the mics for questions or should I just repeat them? Just repeat them. Very good. So the question is, how would companies know if they've had a near miss, if they struggle to even know they've been breached? Oftentimes... Oftentimes someone notices it. Some of the things that people have reported using that form, one was a develop, this was a bank, the organization blocks access to GitHub and a developer filed an exception, got access to GitHub and posted a bunch of bank credentials to GitHub, which they detected. And they believe it's a near miss because they found it within a day of it going up and they went and looked at all the logins made with those credentials across the organization. And so

having talked with the person, they were like, that was painful, but there was no business impact to it. Another one someone reported was that A copy of a zero day from their security bug database was used in a tab and they thought that their bug database had been compromised and they found a bunch of configuration flaws and then it turns out that the person who had reported it to them was also using it in the wild. And so they went, they had what turned out to be a fire drill. They found old accounts that had not been disabled when people had left the organization. But it turns out there was another route by which it had happened.

So the question is, what about breaches of the near-miss database? So the way... The way NASA handles this exact question with ASRS is that the forms that come in on paper are anonymized before they're put into the main database. The forms that come in electronically are held on an isolated system and anonymized as they're transferred. So they have a very narrow window of vulnerability by design. It's possible that a state actor would bother to breach this database, but it seems likely that that's not a necessary step for breaking into most targets. So it seems like a little bit of a waste of their talent. At some level, the things we're learning are things. I want to very

carefully not say they're things that we know, but they're things that we need evidence for in order to effect change. The thing about Autorun was not that anyone was unaware of the impact of Autorun. I remember when I showed my boss some of the initial work, he said, "You know, you could fix Autorun with this." That was his first response to anything else. And I said, "No, no one's managed to do that. It's been known for a long time." And he said, "Give it a try anyway. This is new data." And so I don't think this is an attractive target relative to the value to the defenders. Does that make sense? So I think the, so the question is, could the

ISACs or the ISAUs be a part of this? Absolutely. In the United States, the computer CISA, I forget what CISA stands for, which established ISAUs, covers what we're talking about. But it's a different form of information, right? It's not MD5s, it's not IPs, it's not domains. So it's a little bit different than what's been processed before. The other piece that's different is the rewards, right? Right now, I says don't have a reward. And I think that that is one of the reasons that information doesn't flow to them as rapidly as it could or should. Yes.

- Are you guys collecting any set statement? - So the question is are we collecting any statement of risk or potential risk from the people who experienced the problem and we're not. One of the things that the folks at NASA emphasized to me was the importance of keeping the questions brief. There's a real fall off as you make the form longer. And so we've engaged in something like three rounds of beta test where we've interviewed participants about what they felt and the answer has always been, "Make it shorter, please." Your answer, and I'll answer my own question by also saying that risk is objective and objective and you guys want to focus on the objective aspect. Yeah, risk, good point. Risk is subjective

and objective and we're focused on the objective. One more in the back, you've had your hand up for a bit. Do you think that near-miss reporting will allow us to get better understanding of the threats than mandatory breach reporting? I do. And there's a couple of reasons. Optimistically, there are more near misses than full hits. If there aren't, we have a bigger problem than I think we have. I have been in the room when decisions have been made that we're not going to treat that as a data breach, that we're not going to report that as a data breach. Lawyers hate breach reporting because exhibit A in the lawsuit against you after a data breach is

you told us you screwed up and. And so the lawyers really are opposed to any more detail going into breach reports. They want to put limits on breach reports like evidence of harm. And so a lot of what I'm doing here is engineering to make the lawyers and the policymakers happy so that we can have a better conversation. And that's part of why we went to near misses is they're lower risk. All right, well Alex gave me the one last question. So again, thank you very much for your time. I appreciate it. for the a Vista thanks that's great that's here

yeah so there's not so much going on right now but if it starts i'm sure george will come and show you right now i i mute everything so you want it to go back on this is the speakers the one the mic they wear and this is the intros i want in the middle you just press this button turn this on but then it'll be on muted for the speaker it's this one yeah the same thing and then like lower it down but this is just yeah recording everything but it's all that's the camera so you can have your function what's your name by the way henry nice to meet you yeah you've been on registration oh yeah okay

yeah the budget somehow but it's good Alex said it was figured out you guys did amazing job you always put to the test when things get like that you do infosec there's lots of people here who are hiring and looking for jobs too So what kind of job? Like what exactly is security? Yes. You know Mira? Mira is security. They're doing their hiring right now. Yeah, Alex knows them, yeah. The guy is called, I forget his name, but I took it. What about penetration testing? That's good. There's lots of jobs with that. Good. Yeah. I don't have experience with it. You need to play CTFs and do like, uh, hack one. Do you know hack one? Yeah. It's basically

bug bounty stuff. Don't know how much you know about bounty, like bug hunting and stuff, but, um, if you show them things like that, they would be very interested because they basically have to It's like playing CTF but in a real good tested environment. You're allowed to say breaking to Yahoo or any other companies but under protection. You won't get in trouble. And by doing that, you get points and rewards. Most employees or people hiring love seeing the thing. You have experience. So you don't really... In the world of infosec, you don't, I mean, you could, but you don't really go get a job. There's so much online that you could do to show you

owning stuff on the island. Yeah, I do, yeah. I do, I come to Vancouver a lot. I go to lots of conferences. It's fun, yeah. I'm working on a web application that might need testing, actually. So you would like to... test it and get some experience what do you use like this app or like i will try my best like i never ever see anything oh cool so it will be my first beginning yeah yeah that's good um like you have to yeah you have to really know about this app for web apps and you really know things to play like simple mouth sql ninja sql injections A lot. It's been many years. I've been doing this for like seven years.

I love InfoSec. It's all I do. If I'm not doing that, I'm in the fall part two. So I'm writing web applications and listing them. Vancouver is really good. There's lots of people hiring. And you'll get the job because there's a huge advantage. And there are people I know in this industry that are really interested. You have a great opportunity because girls right now doing Infosec What are you saying? Girls right now doing Infosec is like so big. You have a really good idea. So keep pursuing it because you stand much more with higher chances of getting a job. But you still like, it's kind of the same. You just like, there's a huge advantage in just telling you in Infosec. It's really big. And

the need is just starting. So... Yeah, there's always new hacks, there's always new tools coming up. You really need to learn the tools. If I was hiring someone for Web App and they told me they didn't know Zapp, they're like the fundamental tools that you want to know for Web App and nutrition. But you will learn that stuff, it's really easy. get one evening and sit down and be prepared to stay up for nights and nights it's fun i love it i love it but yeah i have uh i have an app that i've been writing for the last two years so it's it has e-commerce components it's a charity application but it does a lot of e-commerce stuff

and it does a lot of like social media kind of stuff but the social media is based around people connecting to learn from each other and to make change and it's just the idea that we are all on earth and we are just moving around and we meet people and we connect and make change how do we how how do we learn about each other it's like you know linkedin yeah it's kind of because if you think about linkedin it's more like creating a professional profile and getting a job the social media we are creating is not only me and the other people involved but the main like order and security person but it's all about networking for change like

you connect to make change in the world like Facebook or Instagram whatever you get something you get a link you see an article about something maybe disaster but from there what do you do next you know like people read and share about this stuff Yeah, it's really cool. So that's what I've been working on. But then the e-commerce part of it is you'd go buy something only eco-friendly, handmade, and only things that are good for the environment. So it's like it also has the component of Etsy. It's just an e-commerce for buying things. only handmade stuff so it has to be a unique product that you've made let's say you made that scarf and it's really nice and it's only you who makes

it and you can only produce like one scarf in a month because you're making it yourself so like you're not mass producing so you would be allowed to sell something like that on that website you can come and set up a shop and stuff and and then people all over the world set up charities on that website that then your shop would support if someone buys something from you in a giving way and it's all free we're going to outsource it, open source it and it will all be free so but yeah it's such a huge project that would have a lot of vulnerabilities I'm very sure but yeah it definitely it's all based on volunteers but for you looking

for experience and stuff like that it would be really cool too check it out and because you wouldn't be limited on how much you can break into it if we know that you're testing it so the first thing I never thought about I respect so I can give it and I'm doing everything that you know I'm looking at it but yeah that's well the more you learn about the information security industry Or the more you specialize actually, but at first you just have to do a lot of stuff. But that's what you don't want to do. But like you have to be, what's it called? ethical that's the term like you have to be elite basically i like it you don't

you know there's something called rules of engagement and you learn this when you start like taking courses and certifications and you want to like certify things like cissp ceh oscp things like that but you They will teach you that there's something called rules of engagement. Never do something if you don't know your boundaries. The company doesn't tell you what are your specificities and what are your norms. So, George... I'm going to be digging a lot better. Okay. Right. You can just sit and watch the equipment. That is all that's necessary now. I'm running around trying to fix the problems that we had this morning so this afternoon is a little smoother. Also going to be doing August 20th to 25th is the SANS

class. And Steve Mathisar is your instructor. Thank you for allowing SANS to be a sponsor for this And I'm going to let Terry take over now and do her presentation. Thanks, Terry. On the home page? Yeah, it has Tanya. It's supposed to be Tanya. I just asked Alex this morning. I thought something happened. Tanya. Hold on. It's what they're trying to do. No? Okay, because it's... That's right. Yeah, but this is her and this is Cryptominer talking. Sorry. What was the question? So this is not that. This looked like it was the crypto. Mining. Oh, yeah. So it's changing. It's not. There was a confusion. Sorry. So she's in a hard place. Yeah. Okay. I'm going to go back. Okay.

Hi. Nice to see you. Nice to see you. I've been keeping it going. Oh, shoot. It's just been one of these absolutely retarded days. Oh, wow. What? Okay. So this apparently is no longer... They're just trying to figure out a speaker issue, but... Yeah, I know, because apparently this was supposed to be track one, but now I don't know what's happening with it. Yeah, yeah. Yeah. Yes.

It's good. Hi. Hi. Everything is good? Yeah, there was a little bit. Oh, I understand. So this is Push Left Like a Boss for track one. Yeah. So if you're here for that one, this is where you want to be. No, because I am volunteering. I have to make sure that everything is good. There was some minor confusion happening. Okay. Now what's going on here? Okay, hold on here. Now what's going on? Now they're switching. Okay. Okay. Alex is there so. Yeah this is where there is a... I think there's something. There was some speaker confusion. Yeah, yeah, yeah. The speaker had moved talks from one room to the other. Is this the crypto talk? The person giving the crypto talk is here,

but the schedule says it's both. Yeah. I would follow the speaker. Yeah, we just came to the track. That was good. So I would just check. Thanks for that. All right, so we did have some confusion. This is track one, but we're actually going to have the crypto mining talk here. If you want to see the pushing left like a boss, you need to go to track three. So everyone know where they're supposed to be? Yeah, all right. Sorry for the confusion. Managed chaos. All right, so before we get started, Shelly from SANS is going to let you know about some upcoming training that they're doing. So take it away, Shelly. Yeah. Thanks everyone. So I just want to say thanks for allowing

SANS to be a sponsor to this event. SANS Vancouver is coming here at the end of June, so if you're interested in some courses, they're actually doing a conference here. SEC 511, the continuous monitoring course, is also coming in August. If you're interested in being an instructor for SANS, which is something that Terry's going to be doing in the near future. She's already on the path. Please reach out to SANS and let us know or ask Terry. Steve Mathisar, who's also doing a talk later today, is actually going to be the instructor for the upcoming course in August. So if you're interested in continuous monitoring, please reach out to him. And a big thank you

to Alex and all your organization. I know this is a volunteer event, and it's a lot of work. So thank you so much for all that you do. Great attendance this year. Thank you Shelly. So just some other announcements again. If you are planning on attending our after party, please drop by the Mariah and Trina Nimbus booth to get your little invite ticket. And yeah, I will introduce Terry. I met Terry a couple weeks ago actually at a SANS summit. Her and I were taking one of the cloud security courses and I said, Would you like to come up to Vancouver and speak about cloud? And she had a really great talk that I said,

yeah, let's get you up here. So let's give a warm welcome to Terri. TERRI BUSH: Thank you, Alex. And thank you for putting this on. And as mentioned, I found out about this about two weeks ago. So if it's a little rough, I've not had a lot of time to prepare. I'm going to talk today about cryptomining in the cloud. I formerly worked at WatchGuard Technologies and I dealt with an incident related to this and I thought it was interesting. And while I was down in the cloud at the cloud summit, I was speaking down there and it's actually TA in a class down there and I heard about another incident that interested me. So

I thought I'd tell you about those two things. I have my own company now. I started about two months ago, but I've actually run another company for many years. Then I went back to corporate America. I helped Capital One move to the cloud after working in back office for many years. Got really interested in cloud, helped with a lot of the security there. And then I went, I got recruited over to WashGuard to help them move to the cloud. And then I was doing some security research. And the way I actually got into security was I myself had been breached and I didn't have anyone to help me. So I like to help other people

try to figure out, you know, we have these breaches going on, how do we stop them? So crypto miners is one of the things I looked at recently that I thought was interesting. So the agenda today is I want to tell you a little about crypto mining, crypto jacking and what they are and also some examples and how some things you might be able to do to detect and prevent these things in your environment. So how many people in here know what crypto mining is? Most people, okay good. Because I'm actually not a crypto mining expert. I actually looked into it and there's some security risks with the networking that I'm not real comfortable with.

So I haven't actually done a lot of it. But I read this book and I thought this was a really funny kind of interesting definition. This person says it's basically a bunch of computers, you know, specially built chips yelling numbers at each other and burning electricity until they find the right numbers. Is anyone here a pen tester? Anyone? So does this sound like anything to you? I mean, to me it sounds like password guessing. It's not really like, to me it wasn't super intellectual, but that's basically what it is. People are using their computers to try to validate these transactions on the network. And what cryptojacking is, is when someone uses your resources to do

their mining. Why would they want to do that? Obviously you're paying for your resources, they don't have to. An interesting one I'm not going to talk about too much is use of electricity. You may not have thought of this when you thought about crypto mining and crypto jacking, but I know a guy who went to work for a company and they have this kind of intensive compute resource that they use to demo a certain product to their customers that they offer. And when those customers are not in their office being, you know, demoed the software, they use that rig for Ethereum mining. But guess what? They're in a shared office space. And the electric bill

is split evenly between all the companies in the office. So you have to worry about your electricity too. Additionally, I'm on the SANS. There's a mailing list for people who have taken the reverse engineering malware class. And Johannes Ullrich just posted that some people are-- an issue that companies have to look out for is people hosting these things under their desks. So the company is basically helping fund their crypto mining efforts. But we're talking about cloud here, so. Whoops, I went too fast. So there's a few different types of coin and you probably know there's a lot of different kinds of coins, but I just want to mention a few things in case you're not aware. Bitcoin, you know, people are typically using GPUs for that and now

they're using specially built things sold on Amazon, you know, all the different kinds of crazy hardware for this. I do want to mention, if you were thinking of just, hey, I'm going to go up to Amazon, I'm going to throw all my stuff in the cloud and we do some crypto mining, it's going to be great. I actually was taking a SANS Netwars. Anyone here done Netwars before? It's kind of like a CTF, right? You go and you try to hack things and they give you these clues. And I thought, well, I'm just going to brute force this password in the cloud. It would be awesome. I'll do it in like minutes. And that's when

I learned. I started doing some research and I figured out that there's different types of GPUs, which I didn't really know at the time. So Amazon has the NVIDIA GPUs and those are good for floating point calculations, not integer calculations which are used in brute-forcing passwords, so that's not the right kind of hardware. And like I told you before, Bitcoin is kind of sort of the same, so it's not going to be the best type of equipment for you to use. So you may spend a lot more money than you actually get out of Then the other problem with Bitcoin is it's very high transaction fees right now. So when you try to go out

and actually do a transaction, it's very expensive and of course everyone knows a lot of criminals are using this for anonymity and what's happening is the transaction fees are so high that someone on the dark web, I don't remember the name, but the people that run the dark web said hey, let's start using Litecoin because the transaction fees are too high. And the other thing is, when WannaCry occurred, a lot of people realized, hey, it's not quite as anonymous as we thought it was. So everyone, you know, after WannaCry, I don't know if you remember, but there's this wallet, and, like, the whole world's watching this wallet to see when the coins come out, because

then they're going to catch them. So about that time, I read about another type of cryptocurrency called Monero. And Monero is supposed to be more... It's supposed to be more anonymous for people who are using these coins. The other thing that's interesting about it is that it can run on a CPU. The algorithm's different. So you don't need these big purpose built machines. It's more of a distributed architecture. So that's what we're finding a lot of in the cloud. And there's other types of coins obviously and there's some that have been noted to be more anonymous like ZDash and Dash. So my first example of crypto mining is what I like to call double crypto.

And that's because initially when I found out about it, it was a ransomware in the cloud. And I thought, oh, cool. Let me go look at this. And it wasn't something super critical. So I could just take a look at it. But what I found was when I got past the ransomware, there was actually some very interesting crypto mining software in there. And I think it's more poignant now that people know about this, but at the time I was doing security research and I hadn't heard anyone write about this. I was on my way to AWS re:Invent and I was about to give a talk on top cloud threats and I couldn't really incorporate this

into my slides because when you go there obviously you have to get everything approved. But I talked about it a little bit and I never got to finish the story. So that's what I'm doing today. So mining in the cloud in this format is basically someone's, you know, an attacker's gonna put some malware on your machine just like any other type of attack. And they're gonna try to get in there and use your resources and spin up, maybe they're gonna spin up an instance, maybe they're gonna attack an instance that you already have running. And as you can see there, we've got the little Monero sign there. So what I did when I looked at

this attack was the first thing, I hope you all have every sign of logs turned on that you can, I'm sure you do, but in this case the person didn't have VPC flow logs turned on, they didn't have CloudTrail turned on, and some of the logs got deleted on the instance, so you really want to think about your logging if you're trying to defeat these crypto miners, and any kind of malware, obviously. And then the second thing that you want to do if you're looking at a cloud incident is turn on termination protection. Now what that will do is you won't accidentally go and delete that instance. And then you probably want to do memory

capture and create an image, create EBS volume backups, any kind of backups you can get. And another interesting thing you can do with the Amazon image and the EBS volumes, which are, if you're not familiar, they're just virtual hard drives basically. You can actually make linkage to another account. So you have those in another account. So if you want to check out this incident and you don't want someone in your account to potentially delete those things, you can actually migrate those to another account or link them to another account, depending on what you want to do there. And then another thing I did right away was quarantine the instance. You can really easily change networking

in the cloud. You can change the security group that that instance is in. And then I use a bastion host to access that instance so I'm not directly connected. So let me show you a couple pictures. Turning on instance termination protection is really simple. You just go and click on the instance, right click on it, choose termination protection. And then you can't just delete it. If you try to go delete it'll warn you. You have to go back and turn this off if you want to then later go terminate the instance. And this is just a picture, I'll have these slides out later, I'm not gonna go into all the details, but you can see

here that I set up a bastion host, and it's really ideal if you can to have a VPN, because when you have a VPN, you can specify the IP addresses that are allowed to connect to that network, and you can have a completely separate network for investigating incidents if you wanna be really careful, separate from your main corporate network. And then you can make sure you don't have a lot of instances out there on the internet that can be connected to and obviously you don't want your, in this case, C2 traffic going home, you don't want that to get out, so you want to quarantine all that. I recently spun up an instance and almost

immediately I had so much traffic, I posted that out there on Twitter and just instantly, as soon as you put something out there, you're gonna see just tons of traffic hitting these instances. So by locking this down with a VPN, they can't get to your resources, at least not that way. So how did I get past the ransomware? When I went in there was a screen, ransomware, and like nothing I can click on, nothing I could get to. And I'm not gonna tell you I'm the be all end all incident responder at this point, but I just went in and went to the bottom and I could get into the task manager or something, I

can't remember exactly what it was. Just to clarify, I don't have access to all the information from this incident that I had originally, so I'm going off of my memory a little bit. Basically, if you can get to that little dot, dot, dot in the Windows machine, you can get over to the Windows Explorer and you can find, you know, you can get to the files on the system. So basically I went in there and I just made all the files on the system readable by anybody. And this was not a sophisticated attack. Immediately I could see the whole operating system. Once I can see the whole operating system, I can go into a command

window. And then of course, I can run all the standard incident response commands to go in and look at everything that's going on on that instance. So the first thing I noticed was there was a whole bunch of weird services. When I started out, I didn't have any clue about security and it took me a while to figure this out because I had no one to help me. I didn't work in a company. I had my own business. So this is probably obvious to some of you, but Back in the day it wasn't to me. So I went into the services and immediately there's a whole bunch of services with four letter names. So if

you're trying to monitor for weirdness on your network, one thing you can do is look at these instances, or the services on your instances and make sure, do you want new services added? Maybe you have some controls around that or you can look for these strange names. The other thing was there's a bunch of IP addresses and domain names in the path for the executable. So basically what it was doing is using bits to transfer files from the internet onto the machine. And a lot of these services were stopped and I believe that's because when you start up a Windows service, it has to report back that it's good to go and if it's not

programmed correctly and doesn't do that, it will just shut down. So there's just a whole bunch of services in there. And so I was able to look at that I was also able to look at the network and see there was lots of inbound traffic on port 389 and I'm pretty sure this is a red herring. These happen to be domain controller type applications for authentication service. So it's probably a red herring but just throwing this in here, if you're running anything with port 389 just make sure everything in that traffic is legitimate because there was a bunch of strange stuff there and then there was an outbound CT channel in this case And I'm

not sure if that was for the ransomware or the Bitcoin mining or just a way to update the server. I didn't get that far because I lost access to the instances after a short period of time. So the other thing that the malware did is it shut down the firewall. So there's a big debate going on. I'm just going to throw this out. There's a big debate going on about endpoint security versus network security. And I've taken a lot of SANS classes. If you have, they really emphasize the network. Endpoint security is good, but this is one of the problems. If malware gets onto the instance and it has elevated privileges, it's just gonna turn

off your endpoint security protection, in this case a firewall. So notice that. And then using this information, I was able to look at the logs and say, okay, there's a period of time where the logs are missing, and put all these factors together, get a time frame, and was able to find the suspected files. Also with the services, we're downloading files in different locations. Took a look at that, and I found some library on there amongst other things, and so I Googled it, 'cause that's what you do, right? and it's XMRig. At the time I had not heard about this. I was doing a lot of research on my work, writing a lot about security

and blogging, and I hadn't seen this before. I think F5 Networks, after this, they wrote something about this particular library. It's a C++ library. So it's running on the machine, and it's mining for these, whoever put this on there. So a couple things you can look at here. It's a little tricky to find these crypto miners, and one of the things you can look at is your CPU spiking. That's like number one, right? Because once they start running this, it's just going to spike. So there's ways you can monitor this in AWS. You can have CloudWatch or something running, and you can get your baselines for your instances, and you can run a report, and you can say, oh, well, why is the CPU spiking on this particular instance?

You can also look for this bits download. I don't have the actual commands that were run in the services, but I did find there's various ways to run bits download. So if you're not aware, attackers are using this, and I heard another speaker at AWS re:Invent talking about this type of attack. So just be aware that this is a way that people are getting onto machines in the cloud. You can also obviously monitor, you know, why is someone deleting the logs? Make sure you use log shipping. Everything in the cloud is ephemeral. It's going to vanish. So take those logs and copy them over somewhere else and lock it down. Don't depend on your instance

logs. And look for new users. There was a couple new users on this instance. And obviously if someone's changing the firewall, you've got to be asking yourself why are they doing that. This is another example of bastion host. I already talked through this. But when you're Exposing instances to the cloud. This particular instance was, I think I forgot to mention that, brute force was pretty sure the way they got in because there was a lot of activity, brute force activity, RDP brute force. And if you set up a bastion host and people can only log in to your bastion host via your network, you're pretty much gonna eliminate that problem, right? Because these, you know,

I saw it on my own machine. They were coming from all over the internet and they're just constantly trying to brute force. And I was recently at an InfraGard talk, which is, you volunteer for the FBI in the United States, and go to these talks, and an FBI agent said that I think the brute force time was like six hours to brute force some of these credentials. So use a bastion host to help you with that. And there's also a new service called GuardDuty from Amazon, and this was released at re:Invent, so it's really timely because I had just seen this incident. This particular incident service is really cool because it basically runs on the

network, not on your instances. You don't have to install anything. You don't have to open up network ports or create any new attack vector for your instances. and it's going to give you alerts for all different kind of things. I don't think it's that expensive if you think about security. I mean, if you want security, it doesn't feel like this is too expensive to me, but you'll have to look at it in the context of your organization. But you can see here, right here we have brute force attacks. They're going to alert on all kinds of things, and there's also alerts here for crypto miners. And my guess is they're looking at DNS logs or

looking at VPC flow logs and I believe that they are finding this through DNS queries. So you're gonna have this constantly updating of DNS and finding bad hosts out there, which is always challenging. But you can look at these alerts, you can take actions based on these alerts and it's a pretty, you know they've got a lot of data to work with to find these problems, so I think it's a pretty cool service. So my second example is the one I saw when I was down at the Cloud Summit in San Diego and there was an incident with an S3 bucket. And I thought that was interesting because they were using JavaScript. Now if you're a programmer you already know where this is going but when you use JavaScript

in a browser it's not running on the Cloud instance itself, it's running down on the person's computer who downloaded the file. So I have a picture of that. So what someone did is they basically got into the LA Times, S3 bucket was world writable. And I've written a lot of articles on securing S3 buckets, so I'm not going to go into that here. Just to let you know they're out there. And then basically the end user is now doing the crypto mining for the attacker. And it's just a really interesting attack. So I wanted to know how they did this. So I just did a little looking around. I only had two weeks, so this

is not extensive. But there's something called CoinHive you may have heard of. And it's really easy to use, and it's very interesting. It says right here, "Monetize your business with your customer's CPU." Hmm, is that okay? Interesting. So I looked at how to do it, I was like, okay, how hard is it? So they have these options up here, and obviously if you want to be really sneaky, you could use some API, and you could do some really, I wrote about this in different ways to hide code and things on websites in the Q3 WatchGuard security report if you want to know more about that, but I wasn't trying to be sneaky in this case,

I just wanted to see how this worked and how hard it was to do, so I just picked the JavaScript miner because that's what they used in this attack. And then I read this little, note, which is really cool. While it's possible to run the miner without informing users, we strongly advise against it, blah, blah, blah. I'm sure everyone reads this and says, oh yeah, I won't do that. Especially the people installing these crypto miners, right? So anyway, this is literally the code It took me like five minutes or less. And all you have to do is create a key for your site and put the code into your S3 bucket. If you're not familiar,

in an S3 bucket, you can host a public website. And that's how I host my site right now. It's like two pages. You want to go look at it? And so I just got my business started. So one of the pages is this. So I put this on a page and click here. It's okay. I didn't thoroughly look at the code but it seems like a lot of people are using it and I explored it a little bit. It does seem legitimate but I did not de-obfuscate the code and go look at it just to be honest. But I put it on the line and I said, hey, everyone, I'm sneaking up B-Sides. Help me

pay for my trip. You know, like, hey, go click on my website to see what would happen. And it was really interesting because immediately I start to see the numbers spinning. And I'm going to show you that. But then it just stopped and died. Anyway, I wasn't trying to be sneaky. If you use it out of the box, it doesn't work on a cell phone. It works on a computer. And it gives you this pop-up. So, you know, nothing sneaky about that. It's like, hey, help me out. And then, you know, woo, I got all these, you know, at least more now, but I just checked this a couple days ago, and you have to

get to, like, .5 to actually, like, cash in. So, yeah, I'm not doing great. But that's how it works anyway, you know. So how do we fix this? No world writable S3 buckets. Again, I've written a lot on this. It's a long topic, but you have to have processes in place to prevent that. You should have a deployment system. You should write your configurations as code. And you can monitor your configurations. I spoke about this at re:Invent. If you want to go watch that video, and there's a guy I spoke with that did a really extensive demo of monitoring S3 buckets. It was really cool. And then, you know, monitor, obviously if you can, you

can monitor with IDS, IPS, DNS logs, you can get threat lists, where these bad guys are coming from. A big one I think is monitoring CPU usage. Guard duty's not gonna help here because guard duty is looking in the cloud and this is traffic going to an end user's machine over here running. Guard duty's not gonna help you, at least not at this time. So here's what the CPU usage looks like. When I go click on my little page, it's pretty obvious. At the top you can see, you know, got very small numbers there and the percentage of CPU usage. And then down here it jumps to 577.1 and it's a Google Chrome helper and

you can see it just like jumps. So I'm sure there's other things that do this, but one way to try to find these problems, I think a really good way to do this would be to look at CPU usage. You can also tell your users if you have people in your organization, if your machine starts worrying constantly, and you have this fan going, that's another good thing to look at. You can use malwarebytes to block this, and I'm sure other companies. This particular company talks about why they block CoinHive, and I'm getting the time signal, I think. And you can also use IDS, IPS, someone on the network. Reverse engineer malware list posted some information

about that. But the problem you're gonna have if you try to do that is you can simply go to Google and you can Google crypto miner GitHub and you will find like a gajillion variations of this. So it's gonna be really hard to write signatures for all this stuff. That's why I kinda like the idea of A, block stuff from the network, all your standard network security, and B, look at that CPU usage. So in summary, attackers are borrowing your resources to do their crypto mining. And the applications you're running on your machines might slow down as a result. You'll hear your fan spinning and you're basically helping other people do their dirty work. You might have rogue

resources injected into the cloud. And if you have unintended crypto miners, you have to think, well, Cryptominer's on my machine, what else is? In the case that I looked at, it was also ransomware. And it's in the browser, I just have a question for you, is that legitimate? I mean, hey, I just want to pay for my trip to Vancouver, right? I'm only in Seattle, it's not that far. But is that a legitimate use? They have a lot of examples on that Coin.ai website of giving your users a little form and saying, hey, don't pay me money, just click this button. It's kind of interesting, actually. So that's my talk. If you have any questions, I'd be

happy to answer them. Yeah, that's kind of the question I had when I was going through this. There was actually a very funny, I thought it was funny, an article I read last night while I was just kind of reading up before this. And this guy was saying, we should stop all crypto mining. We should just make it illegal because people are doing this. And I was like, So we should make banking illegal because people hack banks, right? So it just didn't make sense to me. It's a tough challenge because how do you distinguish between what is legitimate and what the user wants and what the user doesn't want? It becomes very tricky, especially for

an end user and even for a corporation. Yeah. Yeah. Yeah. They may have been using CoinHype. because that's what they advertise for. So I don't know, what do you guys think? Should be okay or not? Yeah. Yeah, I don't know, it's very interesting because on the other hand I had an issue with one of my laptops and I just decided to delete Python off my laptop and I found in here there is a Python crypto miner. I never went back and looked at it and then I stopped using that computer but I think there's a lot of cases where people are doing this crypto mining and they don't know it. Yeah, it's definitely a tough question. And the tough question is for, and the question

if you didn't hear it was what about advertisers who are using cookies and things and tracking you? The question is the same thing like third party cookies. As a security professional, do you disallow or you allow? And if you disallow, you're disallowing potentially a lot of legitimate things. So it's a hard problem. Yeah, it's a choice that I think organizations are gonna have to make. Is anyone seeing these things in their environment? Anyone? They've seen crypto miners in the cloud? Yeah, we got a few back there. - Yeah, I love that too. - Oh, really? Legitimately though, of course, right? Yeah. Yeah, I don't know. I haven't done this a lot. One of the problems that I have with, you know, someone was like,

you should stop everything you're doing and go mine Bitcoins day and night. Well, maybe that was right during that time period we just had. But at the time, I read a book on, scanned a book. And one of the things I didn't like was I worked for a bank. And when when you do transactions, you want those transactions to be secure and you don't want to lose any transactions and you don't want to have any extra transactions. And when I read the part about, ooh, potential race condition in the network could cause you to lose your transaction, I was like, I'm out. So that's why, it has nothing to do with the mining, but that's why I haven't done a

lot of this. I just thought, that's not for me. But apparently a lot of people made a lot of money off of this, so. - I don't know the difference between light verification and mining, just to be honest. I thought mining was just verification. Oh, they don't. Oh, wouldn't that be the same difference, though? Because, I mean, you're using that-- unless the person knows you're doing it, you're still using the resources. But I could also argue-- I thought about this while I was doing this talk-- - You have a lot of bad code out there. I've seen a lot of bad code that just churns out in your CPU when you're running JavaScript, so is it any different? I guess the difference is it wasn't intentional

in that case, but. I would say that if you let the user know and they know you're doing it and it's legitimate, I would think it's okay, but it's very tricky for a security professional in a large organization trying to make these decisions about what to allow and what not to allow in that case, and they might just not allow anything from CoinHype. I just set this up, so I haven't seen that. I didn't see that on their website. I think that's kind of the purpose of CoinHive. They're giving you 70% of the mining by sending this whole contraption for you. So they're gonna give you 70%. Set the code up. Sounds like a new business model. Maybe you

wanna do it. I don't know, I haven't seen that. I was curious how the JavaScript miners work. That's why I went out and played around with this. Anyone else? Yeah? - Just on that one, I found that's how you create a . I mean, if you consider my humongous, massive payout that I got back here, you have to get to 0.5. I did this for two weeks, and I begged people to click on it. My family wouldn't even click on it, so you know, it's okay. They don't trust me. I was calling my bank to ask a question, and my dad asked me if I was hacking a bank for my work, and I was like, no,

dad, no. So yeah. Yeah. Is anyone fighting crypto miners in the cloud? I know this is happening a lot, but I'm just curious, is anyone on AWS or that you know of on Azure that you've had this happen in the cloud? No, no one. I would suggest you go take a look because I found out about this a week before re:Invent, so I didn't really talk about it there. GuardDuty came out and then another talk I went to on security, I mean I heard this multiple times, like people were having this problem. It's also on the reverse engineering malware mailing list, people are asking about it. And I went to the Cloud Summit in San Diego, I was talking down there and there was a guy from Microsoft and

don't think Microsoft has something like GuardDuty at this point. So you might want to go take a look. Go check out your instances.

- Yeah, so that's where I was getting at with kind of these different things back here. So in Amazon, I believe they're looking at the DNS records. They're like, okay, we know these particular IP addresses. If an instance is going to these particular IP addresses or these particular hosts, we know it's bad potentially, so we're gonna alert our customer. Definitely the CPU usage. Use AWS and I'm sure Azure has the same thing. There's a panel where you can go look at all your instances and see the CPU usage and you could theoretically write a report that tells you what's my baseline and if you see something jump, you could say hey, I need to go

look at that instance. Those are key ways that I would look at it right now. And I also mentioned that there's some bro modules out there. The problem is, I mean, I wrote a paper on packet capture on AWS and it's a little tricky and I kind of didn't finish my thought because I ran out of time. But it's pretty tricky to set a packet capture in the cloud. I know people are doing it. So setting this up may or may not be feasible for your organization. That's why I like the GuardDuty solution on Amazon. The other thing is, that was my next slide, you have to keep on track. I mean, you'll just find

so many of these things if you go out there. And so one change to that signature and your signatures don't work anymore. So that's why I really like being involved with a threat list that's sharing information with you. GuardDuty, I should have mentioned this, they allow you to import threat lists. So if you have a list of IPs, a particular list that you're subscribed to, you can import those threat lists. And additionally, you can, and obviously they're leveraging DNS logs to try to find that. If you're using Amazon DNS, by the way, I should mention that. Anyone else? I guess I had more time than I thought. I thought I was going to run out of time, but I'm sorry. I did

not see anything but Monero. The problem is that what I mentioned before was that the other types of cryptocurrencies require massive compute power. So these ones can run on a standard CPU. They can even run on a cell phone. I know they are occurring on IoT devices as well. So they can run on it with a much smaller compute power. It's all distributed. Different algorithms. So yeah, there's a lot of different programming languages out there that are doing this. All right, I don't have a very long presentation. I thought I was gonna go way over, sorry about that. So I just started my company about a month ago and I met Alex down at the Cloud Summit as mentioned

and I'm helping with the SANS Cloud Security and Architecture classes. So if you have any other questions, cloud related security questions, I'd be happy to answer them too. Anyone? Okay, you guys. I think cryptocurrencies are interesting and if you think about why they're used, it's sorry I was criminals, right? Because they don't want to have their transactions discovered. They don't want people to know that they're doing these money laundering or something else. The other interesting one is if you think about people who are trying to get out of certain countries. They don't want the government to know that they're moving all their money. There are certain countries that have recently blocked cryptocurrency and my belief is

that is because people were trying to get out of the country. They're trying to move their money out of the country and they don't want the government to know. Some people want to avoid taxes. Some people don't like banking fees, which I think is funny now because if you look at cryptocurrencies, one of the main challenges with them is, you know, when someone steals your cryptocurrency and it can and will happen, there's more and more articles coming out about this, who do you call? and you don't have anyone to call, where are you going to put your key? Under your mattress? That's where we have banks. Banks are a place to store your money safely.

And yes, you pay them a transaction fee, but you pay them that fee because guess what? When someone steals your credit card, you can call them up and say, hey, I didn't make those transactions, and they cover you. So when you're using cryptocurrency, you don't have any of that. So yeah, you're free and no one knows what you're doing. Maybe you don't have to pay some taxes, but there may be other repercussions. So for example, there was some very large losses due to BGP changes. BGP is like the main routing on the internet. You know, your data is supposed to go here for this IP address, whatever. And those things changed and like millions of

dollars were lost in these transactions. cryptocurrency transactions because of that network change. The other thing I mentioned was the latency issue. I'm glad you asked this because I thought I wasn't gonna have time to talk about it so I cut it out. But when I read the book on cryptocurrency I immediately said, oh, not a fan of these latency issues. When you do banking software, and I did a lot of banking software, a lot of e-commerce software, there are some very key things and that is not losing transactions and not having duplicate transactions. And one of the things that they always talk about is you have a lot of transactions going on and programmers write

multi-threaded programming. And the classic example is someone did that wrong and there's a race condition, it's a very tricky code, and someone was able to get their money twice out of an ATM because of this timing issue. You know, the bank's supposed to say, oh, you have $200 less, and someone messed up. And so by the time the bank realizes you have $200 less, the person got their money out twice. It's oversimplification. probably, but with the networking issue, I think it was an O'Reilly book on Bitcoin mining I read years ago. Basically, if someone submits their transaction at the same time as you and they get there first, they could get your money. And that's

the part I went, oh no, no, no, I don't like that. But you can say everything's a risk and you could take that risk if you wanted to, but a really big thing in software programming, especially for banks and databases, is to not have these type of transactions that have sort of latency that get there too late. They have to be really exact. So that's one of the flaws I see in it. So I'm just watching it with curiosity. I know some people have made a lot of money off of this, but I think it has some risks and I know Warren Buffett doesn't like it either. I tend to side with what he thinks

usually. So he doesn't understand the value proposition. Like who made this stuff up? Where is it coming from? It has value because you think it has value. It has value because you're willing to give someone money for it. You know, there's nothing backing it. There's no government. There's nothing. So I would be concerned about this just falling apart one day and you can't do anything about it. That's just me. Does anyone have an opposing view? So he's saying basically it's a risk proposition and it depends, you know, the value you get out of it versus the risk you take. And that sounds kind of like cybersecurity, right? We have to determine, you know, what we're going

to protect and what risk we're going to take because we can't do everything. So, yeah. Investment. Investment.

Right, so there's different kind of cryptocurrencies that have different properties and it goes back to the slide I had about the different coins and their uses. Some will have lower transaction fees. People are looking at Bitcoin as an investment. When you try to take that investment, turn it into cash, you'll pay some hefty fees. I know someone who just did that recently. So there's definitely a lot of coins for different purposes. Yeah. Any other questions?

- That's a very good question. Why do different coins have different values? And I mean that's the whole point, right? It's like why does this thing have value? You could even say that about the US dollar now or the Canadian dollar. I'm actually not sure about Canada. I shouldn't speak about that. But the US dollar used to be backed by gold. Now it's backed by a government that gives their word that they're going to pay, right? So what is backing these coins? Nothing. It's people's beliefs, right? I'm sorry, what was the last part? So the value of Bitcoin is if you run an electric company, you can make money off of it. Or if you sell those cards on Amazon, you

know? Right. And I recently heard that the cost of mining a Bitcoin is not worth it. I'm sure it depends where you're mining it. Like you mentioned, if you're mining it somewhere, electricity is cheap. But in some places, it's not cost effective because you're going to pay so much for the electricity. And that's why the model, I think, is... Yeah. Yeah. Well, except that the GPUs in the cloud, I spent $300 trying to crack a password and I realized like, oh, I wasn't looking. So just to warn you, those GPUs are really expensive and they don't work for this type of thing very well. Unless you're, I don't, I didn't, I didn't make very much

money off of my coin hive here, but perhaps if you had a, you know, a really interesting model where you could get people to mine your coins for you, you could make it work. Anyone else? Yeah. Is anyone doing, it sounds like we have a couple of people. How many people here are doing Bitcoin mining? Just out of curiosity. Oh, got a few. What's your favorite coin? Oh, okay, okay. A different, not AWS, a different cloud, like specifically for? Oh, okay, okay, okay. So it's like the old kind of like managed hosting model or something. Interesting. Does anyone use AWS and Azure to host other things? In my experience, AWS.

No, no. I don't think you should use them for mining, but is anyone else running applications and they're worried about crypto miners getting on their instances? No? Yeah? I'm familiar with that. Yeah.

Yeah, okay. That's interesting because that's kind of how I intended this talk and it sounds like people are more interested in like actually doing, maybe doing mining, I'm not sure. But that's the angle that I was coming from because I've seen a lot of this happening in the cloud and people are being affected by this. So I was trying to help people protect themselves from this. But hopefully you learned something about if you want to go out and do this, you have a few tips here as well for that. Any other questions? Okay, I'll be around tonight at the event, the partay. And if you want to reach me, I post a lot of research I'm doing on Twitter. This one was a little light because I had two

weeks of repair for this. But I post a lot of the things I'm working on on Twitter, and I've written some white papers about cloud security, S3 buckets. I did some research on malware for WatchGuard. So if any of you are researching security out there, I'd love it if you could connect and we can share notes. Anyone else? No? I think we're done. Hey everyone. We're going to get started here. So just a little heads up on our after party. We definitely want to see as many of you there as possible. It is a little bit of a trek. It's at Broadway and Arbutus area. It's a good distance. So we do have a bus that's doing

laps. The speakers are going to get the first bus and then everyone else can. So I think it's every 30, 35 minutes with traffic. So if you're at a hotel or whatever, want to get a quick pint, I'm sure you can do that and then get back for the bus. Otherwise, you can grab a taxi. and a restaurant's called The Eatery. If you want any details on that, just go to the Moriah and Trinibus booth. We have little cards to give you some details of how to get there. The bus is gonna be on the Seymour side of this building. It's gonna be parked. It is very familiar to us all. It's a big orange

school bus. So... It will be doing laps, so you'll be able to get out there and likely back downtown after. And we have a bunch of cool stuff happening out there. So hope to see you all there. Now to introduce Robert. We saw this talk at a ready room briefing last fall, and it was really awesome. I told lots of people about it. And subsequently, he was able to take this to Portland B-Sides, Seattle B-Sides, anywhere else yet? A couple other places, yeah. Wow. Yeah. So it is a really cool talk. Really interesting. I'm not going to take any more away from it. Let's give a warm welcome to Robert. Thanks, Alex. Now I have to deliver after that intro. So thanks a lot

for joining me here today. I know there's a lot of good talks at the same time, so I'm always really happy to have a lot of people come in. First time I've actually presented after lunch as well, right? So I'm usually used to seeing all the faces kind of thinking about lunch. So to have you all here after lunch is another fantastic thing. So, this talk originated after my competition at DEF CON. I did the social engineering CTF there and came in third. That was kind of my objective was top three and I just was really amazed at how much information I was able to pull from a company without really knowing what I was

doing. So, that's what this is all about. Who's seen this talk before? Okay. So, a couple of you. So, good news, new content. So I've done this talk a couple times, as we alluded to there, and I feel guilty about just giving you the same information. So what I do is I take your feedback, I try to incorporate it into this and give you some new fresh content, really kind of what you're looking for. So this talk is going to include some evil attacker info, and that's really kind of the point of view that I'm taking. I'm the bad guy. Not really that bad because it was kind of just a CTF, but throughout the

year, I've talked to a lot of people that are actually pretty bad. We've gone and had beers and had some Twitter discussions, and they've discussed with me what they do. It's not just a CTF. This is actually the real thing that they look at, and so I've incorporated that into this. Who am I? So I've got a bunch of letters behind my name. I've been doing this for a little while. You can find my LinkedIn information there. My night job is my volunteer passion is search and rescue, which is kind of weird because within that organization, I focus on tracking. So if you get lost in the woods out in the Coquitlam area, I'll probably

be the one out there looking for you. And it's very similar to when you're doing OSINT. and you're looking for sign, not necessarily the particular individual right away. So both of those activities are highly addictive. If you do search and rescue, if you're a tracker, or if you love doing OSINT, please talk to me, because I'll just talk your ear off. I love doing that. I recently started a nonprofit called tracelabs.org, and I just got an email last week saying that this is going to be an official DEF CON contest. So if you're into OSINT and you're going to DEF CON and you want to help, please let me know. I'm looking for people. It's

really focused on using OSINT to find missing people. Being from in the SAR community, I'm very excited about that because they're not really that trained on that sort of activity. So I want to kind of bring that to them. You can find me on Twitter and as well my email address is there. So if you like any of these slides, please take pictures of them, send them to me and say, "Hey Rob, I really like that." Likewise, if there's stuff in there that you think, "Yeah, it's kind of boring, you could cut that out," please let me know as well. Always looking for that feedback. So this is what you're going to get. All right.

So in a snapshot, you know, we're going to go over social engineering, what that's all about, the DEF CON experience, OSINT, vishing, some techniques and pre-techs, and then how to defend against some of this stuff. And then everybody asks for tools. I'm going to show you a few tools as well. All right. I have to say this. Everything I say are just my opinions, nothing to do with any of the companies I've worked for or am currently working for. Now let's get into social engineering. What is the social engineering stuff that I'm talking about? So the big difference between social engineering and influence is that kind of evil factor, right? So if you're getting social engineered, it's not in your best interest. They're trying to pull

information from you that you shouldn't divulge or make you do something that you actually shouldn't be doing. And it differs from influence because influence is more of a positive thing. I'm probably trying to, I'm looking out for you if I influence you, right? I might say, hey, why don't you go take a course so you can go get the get that job, that's more of the influence side. So it's all about manipulation of the targets. What are some examples of this? So I call these the golden oldies, but actually a lot of them are still in use today. Impersonation, both physical and virtual. Tailgating, right? This is still a problem. A good physical pen tester

will get into your building without too much difficulty carrying the coffees or the pizza or whatever they want to do. It's pretty straightforward. Not too many of us stop those people that are just walking in behind us. Shoulder surfing, not such a big threat right now unless you're at the airport or on a bus and there's been some pictures on Twitter of people taking pictures over your shoulder or over someone's shoulder of sensitive information. Dumpster diving, we all shred our content now but we don't actually see where that goes quite often so I could dress up as that company, come in and take your shredding possibly but it's improved a lot. Some of the current

attacks, so the email attack, phishing, right? We see that a lot these days. It's pretty good. It's going to get really good. Phishing and then smishing as well, right? So on your text messaging. So we see these quite a bit these days. They've started years ago. They're getting a lot better. And then what we're going to see next. I keep adding to this slide, I'm sorry it's so busy, but every day there's new wonderful attack vectors through social engineering. So I get really excited about this slide. We have the fake accounts, you're in the lineup to get your used car and you're waiting for an hour until you tweet them. You're like, this is crap.

And you get a tweet back from them like, we're very sorry, please send us your information and we'll give you a free credit. And they're just harvesting your creds, that's not really the company. Social engineering as a service, you know, everything's as a service now, so of course you can do this on scale as a service as well. Virtual kidnapping, which is fantastic. If I can get one of your accounts, I can use it as a trusted source to go do other evil stuff and then sell it back to you. Whaling, this is Really, one of the bigger ones, this is all monetized, right? So I want to go where the money is. So your

executives, they have authority over usually the money or to be able to make big decisions. So I'm going to go after those. There's a lot of campaigns that focus on this. Saffron Rose was one. I mean, these people have really matured quite a bit. They're utilizing social engineering a lot these days. There's a couple of these actors that have changed a little bit and are now investing much more in the social engineering. The pseudo ransomware hybrid attack, this is happening, I've seen more this year, where the ransomware or something like that will be a distraction. So all your resources get converted over to that and then they're going to actually go do the real attack.

So it works wonderful. Professional network solicitation. So this is where on LinkedIn you get that person that reaches out to you with a wonderful offer. We're seeing that more and more. And it's very dangerous because it seems very subtle and very innocent and it just kind of builds off of that and they layer on the attacks. The conference invite. So I go to a few conferences so it's very interesting now when you get the invite to a conference that's unsolicited. And that's actually growing quite a bit as well. The fake headhunters, so the thousands talent program, you can look at that on Wikipedia. It actually looks fairly legitimate. Apparently it's not. And then of course the whole false flag and fake news stuff. So all of

these areas are growing exponentially. The other thing that we see as well is large corporations and also government entities utilizing this, right? So this is not new for us, but we've seen in the news a lot over the last year. 50 Cent Army, I mean, that was really kind of one of the pioneers that I remember. So they kind of really set the standard. Russia, we've got the web brigades. But there's lots of others as well, right? And it's privatized now too. So 50 Cent Army went from very, I think, government-focused and now anybody can hire them. And there's lots of those out there. If you want to sway public opinion, it's pretty easy to

do these days. So where did this stuff come from? It's not brand new, right? I mean, we kind of think of it as being brand new, but it's been around forever. You walk onto a used car lot, and you'll get a lesson in social engineering very quickly. Those guys are amazing. I love doing that. They'll teach you so much very quickly. You might end up with a car at the end of the day as well. It's been around for a long time. The people who built the pyramids, pretty sure they were really good at social engineering. So it's not new. Why do we care? When we look at the trends, when we look at these

different attack vectors, we can look at the Verizon report for 2017, and you can see on this chart here, I always associate things with stocks, right? So if it was a stock, I would be buying it. It's a super hot stock if you look at that percentage or that inclined growth there. And you compare it to some of the other ones, you know, physical, which is kind of underrated. You know, that's pretty much a good ETF. And of course they say hacking and I don't remember how they define that, but that's the stock your friend recommended, which is never a good idea. So that's why we care about this. It's usually used, if not as

the attack itself, as a precursor to the attack. Now anybody who has their CISSP is very familiar with this model. We typically, in the InfoSec industry, we focus on the technical side of things. We haven't really focused on the human very much. So the joke here is to add the user layer on top of that, of the OSI model, because it's super important, right? Why would I spend months trying to hack your firewall when I could just phone up your users and ask them for their password and then they'll give it to me, you know, five minutes. So I think we're going to see more and more of this in the courses that are offered.

All right, so Kevin Mitnick, wonderful guy. He was here in Vancouver a month or so ago. This is his favorite expression, "Weakest link in the security chain is the human element." I agree with that. Does everybody agree with that? Does anybody not agree with that? Somebody be brave and throw up their hand, otherwise I can't do a demo. Yeah, I see a hand in the back. Thank you very much, sir. All right, so you want to do a quick demo? 30 seconds? It's stupid, ridiculous, and I hate doing it, but it's fun, so we should probably do it. And it breaks up the presentation. So it's just an example of how I'm going to manipulate

you to do something that I've told you not to do. That's basically all it is. The reason I hate it is because half of you will probably get tricked and then you hate me and the other half won't get tricked and you'll be like, "Oh, this is fraud." Right? So I lose both ways but it's kind of fun so that's why I do it. So let's do this. Let's pretend we're one big company. We've just started a startup. All of you now are working in the same company. I'm the InfoSec guy. There's just one of me, of course, for all of you. And I write these fantastic policies. Okay? I'm going to write one policy.

It's the first one I'm writing. It's the most important. And it's about an activity that you should not do. So attackers are going to try to get you to do this. And I'm writing this policy because I know if I write a policy, then you're going to read it. You're not going to do that. So the policy has got to say something that we can all see. So let's say flipping hands. So you're not allowed to flip your hands like this. Something visible. I'm going to see you if you do it. So at the party later tonight, the people who fall for this, you can get them to buy you drinks or something like that.

So let's say don't flip your hands. Now I got to see your hands in order to do this trick, okay? So let's all put your hands out in front of you like zombies. Okay, watch your neighbor. Okay, all right. So we're gonna start with our palms up. Oh, so that's amazing. You guys are the best at this that I've ever seen actually. Because some of you went like this. No, yeah, that's awesome. Yeah, well, it's a flip. So my policy has to be a little more clear, I guess, right? Okay, all right, all right. Version two of my policy. Okay, so I'm sorry for those of you, I didn't really see anybody flip, but if

you did, I'm sorry, it's a stupid trick, but it just shows you kind of the example of how we manipulate, right? So it's, you know, you click on the link, you're going to buy into it sooner or later. It's just a matter of effort. If you still don't believe me, go Google it. You can put this stuff together fairly quickly to support your, you know, when you ask for a phishing program or something like that. There's tons of information out there. There's a really good one out there where a company wire transferred $50 million, right? Stuff like that, really bad, right? But fairly easy to do user awareness training to protect against that. So I

would highly recommend it. All right, so I got into this whole thing because I went to DEF CON and I competed in the SCCTF. And just to tell you a little bit about that, it's broken up into two stages. The first is recon and the second is attack. So you're at home developing your report that you're doing based on the OSINT, open source intelligence, against the target company. Now this is a real company. There's 16 companies, 16 competitors, all in the same industry. So you have so many flags that you're trying to collect based on that company. So you've got about three weeks. That was about 100 hours for me. A, I didn't know what

I was doing. And then B, it's super addictive. So you start collecting. It's like Easter, right? You're collecting all the little Easter eggs. And a lot of fun. I highly recommend it. The second stage, you actually go to Vegas, and you're in this glass booth, and you have to perform for 20 minutes doing live fishing against that same company. And a lot of fun. You've got hundreds of your peers watching you do this, so the pressure is on. But again, I highly recommend it. If you ever want to do it, let me know. I can tell you kind of what it was like and how to prepare for it. So the flags, you can't probably

read that, but things like, you know, what's your cafeteria? Do you have VPN? Who does your janitorial service? What kind of OS do you have? How long have you worked for the company? All very benign tags, right? So legally, not a big deal. You wouldn't probably offer this up in conversation to people, but, you know, it's not super bad. These are... So this is some of my new content. I just use the Cyber Kill Chain because everybody uses that to give you some flags that I would collect as a real bad guy. So things like what kind of technologies do you use? Your response capabilities? Can you deal with a threat once I'm in? What

kind of assets do you have? I mean, that's really what I'm going after, right? Probably money. What's your patch level? Some of you advertise that. You go on forums and you talk about technologies and stuff like that. Nation states are reading all of that, right? Delivery methods. How am I going to get my exploits into your environment? Can I just email them in? Do I have to drop some USB keys in your elevator? You know, what kind of protection do you have against my exploitation, right? What's your AV look like, your endpoint protection, perimeter? So all of that pretty much I can collect. Installation, so what kind of logging do you have? What do you

use for your SIM? Are you using Splunk or the RSA solution? What does that look like? Is anybody reading those logs, right? Anybody looking at that? What are your hours of operation? What's your machine naming scheme? All of these are great information, right? So then I want to exfil my data, right? So hours of operation, I'm probably going to do it at night. How do you do your backups? If I'm going to do ransomware, right? I want to go all into your backups. So a lot of this is online, and I can usually find it. What's your incident response look like? What's your attainment? How far do you back up? How many years or months

or days even? How do you do your DRP? So if I destroy your environment, can you rebuild it? And then your policies and procedures. Sometimes those are very easy to find as well. So these are less benign flags that real bad guys would look at. All right, so on the OSINT side, once you're starting to look at a target, this is more focused on a company. If you're doing it individual, I'll get to that, but it's a little bit different. So some of the things I'll look at is physical. So right away, I'll look at your building, your locations. You know, Google Street View is a good way to start, right? I'll get your ingress,

egress points. I'll look at your roof for your HVAC. equipment, see what that looks like, loading dock, might even be a picture of like the backup, you know, Iron Mountain truck in the parking lot or something like that, right? Technical, so your websites, those are great, right? Websites, IP address, DNS, anything that you register. I'm going to probably get some prefixes or suffix for your phone numbers, email addresses for sure, names of staff. Then I'll move into corporate, and again, this is a gold mine because I'll find registration. Property management is usually really good. You'll find lots of details there, and that's a good pretext to use later on. And then you get into staff,

and this is where it gets really good. So, on staff, just go to LinkedIn, start there. It's beautiful. It's all laid out perfectly for you, right? You can see, you know, who's boss, how long they've worked for the company, what they did before that. You can really develop a sense of, you know, that you really know those individuals after you get into that. Sometimes they link to their other social media in the top right corner, which is fantastic. Of course, LinkedIn will want you to buy a membership once you start really using it. So, you can use... this X-ray which is a great tool that allows you to do it for free. If you send

me a LinkedIn request, of course I will accept it because it will allow me to go deeper within the application. So feel free to do that. That'll expand my search capabilities. So it's fantastic. Start there, you can collect so much information. And then once you've got your targets laid out, it's kind of the 80/20 rule. So very few people within the organization typically will share a ton of information. Those are your social butterflies. Those are really where you want to focus on. Don't waste time on everybody else. There's just too much information there. So not only the social butterflies, but also their friends, even if they don't work for the company, are often worth looking

at because they will take pictures and post stuff about the company as well. And it may even be even less appropriate because they don't actually work for the company. Personal websites is great as well. Now, when I first started doing this, I had no idea what I was doing. So when I was doing my OSINT, You know when you're on LinkedIn and someone clicks on your profile and you see that and you're like, "Oh, who clicked on my profile? "I might be a recruiter. "I'll go take a look." And you click on that person's profile. If you do that to a thousand people, all that work for the same company, they all turn around and

look at you, right? So you don't want to do that. There's a bunch of things you can do. Set up a VM so that you have something that you can archive. You know, you want to probably run VM or VPN or something that's going to, you know, give you some, hide you a little bit from all that. Set up some fake accounts if you want to. Buscador is a platform by Michael Bazell, kind of designed for OSINT. It's something worth looking at. You probably all have this set up already, but when I was doing this, I was a total novice. All right, some preparations that you want to do. So before you start doing anything,

you kind of want to think about, okay, how am I going to preserve my intel and how am I going to record it? If you don't have anything else, you know, OneNote you could use. It's not the best solution. There's a lot of really good solutions out there that you can buy. Your company may have something already. How do you categorize your intelligence? That's another big thing. Do you do it by social media channel or do you do it by the person you're looking at? There's a bunch of different ways to structure it. What data points are important as well is another good one. So is it location or is it some specific thing that

your company is looking for? Who's gonna read it? That's also a good thing. Who's your audience? And then how are you gonna stay undetected? And so that was a big problem that I had. All right, so my pretext development I found receptionists are amazing for two reasons. They're really good at helping you build up a pretext for social engineering, but then they're also kind of the gatekeepers of your company as well. So attackers generally are going to be hitting those people, especially if you don't have a lot of DIDs. They're also used to sales calls all day, and a lot of us don't invest in our receptionists very much. which we probably should, right? Because

they're doing, they're really the physical point of contact as well as all those phone calls as well. So great people to talk to about this sort of thing. Once I've identified some of my pretexts, then I want to focus on my marks. Who am I actually going to call and use those pretexts with? So, using LinkedIn and some other things, I would look at people that had very low connection scores in LinkedIn, maybe about 100 or just less than 100. They have to have some, not in the industry for very long, expressing a need for self-promotion. So, they need to be able to, they need to want to talk to me. So, if they don't

know a lot, but they won't talk to me, that doesn't help me out very much. And they also need to be, have an inappropriate need to self-promote or share as well, right? So if they're sharing your VPN credentials, that's perfect. Very poor judgment, right? So I'm looking for the high charisma, low wisdom. Basically that usually equals interns and contractors. And I think that's, yeah. Yeah. I think that's just because they're not fully invested in the company and they haven't been around the company for a long time. So contractors are kind of nomadic, right? They're going to move from company to company. And interns, they're brand new, right? So they don't know who to talk to.

They don't know if the question's inappropriate. And they just want to help. So those are perfect. And bad guys will target those people as well. So now some social engineering techniques. The confirmation, this of course is one of the easier ones. So if I know that you use Dell laptops from my OSINT, I'm gonna say, "Hey, how do you like those Dell laptops?" 'Cause that builds up the credibility right away. In your mind you're thinking, "Well, he knows I have those, "so it seems legitimate." And what I wanna do is have that rapport start. I want to get that conversation moving. Or the opposite of that is to say, "Oh, how do you like those

Toshibas?" Knowing that you use Dells. And then let you, just drop my ego and let you correct me, right? Let you feel good and say, "Oh, yeah. Oh, I'm so dumb. I made that mistake." Name dropping works excellent. Bad guys don't use this a lot. Well, they do for the wire transfer, the CFO and the president. So we should see that more. But saying, "Oh, Mr. Smith, your president said I should talk to you." It works very well unless the person is sitting right next to them, which I've seen before. It's very entertaining but it doesn't work very well. Blowing smoke. So everybody likes to feel good. Real attackers don't use this very much. Salespeople

do. So you were recommended to work with us. I must be special if that happens. I like to use that, that works very well. Impending doom, I use this at DefCon and I like this one a lot. This event is going to happen. There's nothing you can do to stop it. It's like a tidal wave, it is coming. But I can help you with that, right? So I was the HVAC company, we just won the RFP with them, and I had a technician coming on site tomorrow to do a site assessment, and he's going to be there tomorrow. Hey, can you use your cafeteria while he's there? How about your VPN? He might have to

up-load the firmware. Can you use your washroom? Will his car access card work there? So that worked very well. allowed to vent. If you can get people on a bad day to complain about stuff and then relate to that and just feed off of it, then you might be able to move that forward and get more information out of them. Typically when people are grumpy and want to complain, they sometimes shut down, so that can be tricky. Smarty pants. Again, sales will use this a lot. You know, how did you ever figure this out? I've been working on this for hours. You know, make them feel good. If you can make your marks feel good,

they're much more willing to usually work with you. Zero-sum, so real bad guys use this a lot, especially the bad guys. bad ones, right? This is the, oh, the first three people to sign up win the cruise. You know, we see this one coming. It's pretty obvious. But the greed factor usually works, still works actually quite well. You combine that with a time-sensitive thing and it works very well. Sympathy, you know, can you help me with this, right? You know, you drive by somebody on the road and they have a flat tire. You're kind of thinking, well, should I stop, right? right? Someone is trying to, you know, they're carrying the pizzas towards the door.

You're going to hold the door open. You know, we always want to help people. And so this will often work for an attack. All right, some pretexts. So, when I want to get in and I need a story to get into the company, these are what I use. I've got three different types of pretexts. This is my entry method. So, if I need to get past the receptionist, I need something the receptionist is actually going to buy into and let me get past. And I developed these by talking to a lot of receptionists. And so, the first one was, "How's my intern?" So, I'm calling from the University of whatever and I know who their

interns are. I know what school they went to. So, I'm asking to talk to them about developing the intern program. Sounds legitimate. Industry knowledge. So if you have any industry knowledge, use that. The best lie is very close to the truth, right? So we can remember it and so we can play into it. So if you have any industry knowledge, use that. For me, I know a lot about data centers, so I'll talk HVAC. That worked very well in Vegas. Targeted methods. So this is kind of the Mitnick special, right? You can layer this. and you can start off very small and kind of layer up. In DefCon it doesn't work because you don't have

very much time. But you can, one of the ones I used was calling in pretending you're looking to rent space there. You want to talk to them about their property management. They'll start telling you all about their building. So for physical pen tests, this can work great. Special delivery, I actually use this one too. Anything that's scripted. So for this example, it was FedEx. So you get the FedEx call, it's always the same, word for word. You just say the same words and you're FedEx. So this worked great. They gave me their FedEx number and everything. They had to beep it out because I didn't actually want that. Yeah, they'll just give it up. Can

I tell you a secret? I really like this one. I didn't have a chance to do it. But anything that involves a payout, so for example, I'm calling from a company and we're letting some people go and they actually have a lot of interest in your company and they're highly skilled people. Can you just answer a few questions about your company so that I can give that to them? And then I'll send you their resumes. Quite often you get a bonus for recruitment, and so especially for a recruiter, they're very happy to help you with that sort of thing, right? Because they're thinking, oh, I'm going to get 500 bucks for every person I bring

in. If these are high quality, talented people, that's going to work out well. So they're happy to give you that information. So if you can link to some sort of monetary benefit, that can work really well. Full dumps. So I didn't think that full dumps were actually easy to do, but I've found now through sales and stuff like that, that once you get people on a roll answering your questions, they're kind of locked in. They're on the tracks, right? And so you just kind of get them lined up to do that. You're the lucky winner, radio station contest. This one is so overused. We hear this all day from real attackers, right? Usually it's on

your phone and stuff like that. So I didn't do that. The upgrade opportunity, you got a new Dell account rep or whatever your vendors are. How often does the old one tell you that new one's coming in? Not all the time, right? So if a new guy phones you up or a new lady phones you up and says, "Hey, I'm your new account rep. "I wanna send you some demo equipment. "Just send me your information so I can update our records "and I'll send you over some demo stuff." Right, sounds legit. You're special. I use this one. This one worked really well. Employee engagement survey. So it combines a few tricks. But basically I'm saying,

hey, your VP called me up and said I should call you because you could really help us with our engagement program that we're developing for your company. Can you just answer a couple questions? And yeah, every time they were like, yeah, okay. They said I could do that. So works really well. Okay, you caught me. So quite often social engineers will say, "Never break your pretext even when they're putting handcuffs on you and you're getting pulled out into the police car." I don't really buy into that because A, I don't want to get arrested and go to jail, but two, I think you can use that to your advantage. And so for me, if they

caught me and if they're kind of catching on to my trick, I'll say, "Hey, you know what? You caught me. Congratulations. You're the only one who questioned me throughout this whole exercise. You know, I'm going to tell your president that you did this. This is fantastic. You know, because we were hired by your company to actually test the staff to see if they would pass this security audit. So can I just run through these questions with you just to set a baseline and just continue my attack, right? So I didn't get a chance to do that. I'm just dying to do that. It's so fun, right? Yeah, yeah. I would love to do that. Okay,

so I just threw a whole bunch of stuff at you. So let's take a reflective moment here just to ask ourselves some questions. You don't have to answer these out loud. But would you know if your company was attacked with social engineering and somebody gave up their credentials or something like that? Would they come tell you? So hopefully the answer is yes. If it's no, then maybe we need to do some user awareness training and let people know that, hey, you know what? Infosec is here to help you with that as well. How bad would it be if your CFO came walking in one day and said, you know what? I think I just transferred

$5 million, right? That would probably be a bad day, logistically at least, right? So lawyers, insurance, and all these other things would have to get involved. Do you have their phone numbers? Do you have their contact information? So you may want to start developing a relationship with those people before that actually occurs. Does your insurance cover that, right? So I'm sure a lot of us have insurance, but I'm not sure if all of your insurance would cover this. So that might be a good conversation to have. Do you have internal resources to take care of something like this when it happens, right? So a lot of you have probably awesome blue teams and red teams,

probably awesome forensics people, right? But are they familiar with social engineering and what you need to do to remediate that? And then finally, can we navigate the Equifax paradox, right? So this is really who's getting fired. And if you don't know the answer to that, it could be you. So nobody ever laughs at that joke. So either A, it's not funny, or B, you're kind of freaked out by that. So just, I want to, real quick, which one is it? It's not funny? Freaked out? A bit of both. Okay. I'm going to tweak that a little bit. Thank you. All right, so some recommendations. So what do I recommend from my experience here? So start

off with OSINT yourself. That's a lot of fun to do, actually. You'll find some really interesting things. I pretty much promise you that. OSINT your company. Find the social butterflies. Find your VPN information, right? Did they put it all out there or not? And then understand at risk. So what would people find on the internet about your company that would be bad, right? Take a look at that. And that's going to take a bit of time, but it's well worth doing. Fishing program. So there's some debate on this. So I've been doing this talk for a little while now. I met a couple people that said, "No, no, we're not going to do a fishing

program. That's an insult to our employees. We're not going to do that. It's treating them like victims. That's not working with them." Now, so when I say do a fishing program, I mean do it with as much respect as you would do anything else, right? So you're not trying to blame people. You're not trying to trick-- well, you sort of are trying to trick them, yeah. You know, you want to do it in a way where it's a learning opportunity for them and it's not like, oh yeah, we got you, right? That's not going to be good. So measure clicks, measure if they come and report it to you, that's important as well, right? Because

ideally that's kind of what we want. We want, you know, the people to say, hey, what do you think about this? Should I click on that or not? So that's a good idea. Don't do what I did. I sent all the emails out at once so everybody got it at the same time. It looked really cool and we put a lot of effort into it so people got it and then they're like, "Hey, did you get this?" "Yeah, click that link, it looks awesome." It's like splash screen, it's really cool. So don't do that, trickle it out, work with your people. Don't punish, reward good behavior. One of the things you can do is put

EXT on the email subject line for your incoming mail. So if it's, so at least then when the CFO gets an email from the president and it has EXT in it and says, you know, can you please wire transfer $50 million? He can look at that and go, oh, maybe that's actually not from the president, right? So that's something you can do. Some companies actually have this scripted, which is really neat. They have it so that it only shows up, what was that now? I can't remember. Oh, darn it. Anyway, they do it even better than that. You could also stop active links in email. Actually one agency does this. I don't think any of

us are gonna do that, but that's another alternative. And then just get away from email, right? So email's brutal. We get hundreds of email, right? So I don't have enough time to process every email. So I'm gonna click the link sooner or later. I get so many. Build up some defenses against phishing. Phish your executive, again, do it gently. They don't have the same sense of humor that we do. And it's also they have the ability to change your career very quickly. Be very gentle with them. Work with them. But they're really the targets, right? So if I'm going after something, you know, money, stuff like that, I'm going to target them. And they're even

more time sensitive, right? So when they're getting emails, they're processing them even faster. Create the choke points. Your receptionist probably doesn't have any user awareness training. Probably a good idea to give that person some, right? So your PBX, I love your PBXs because I can dial in at night, do dial by name, and just get everything, right? I can get your names. I can get your positions. I can get your phone numbers, usually DID. Then I'll listen to your voicemail, find out when you're on holiday, find out who's acting on your behalf. So we probably don't need dial by name anymore. So you might want to turn that off. Your DIDs. A lot of us

have direct inward dials. That means I can bypass your receptionist, just call directly to you. Everybody probably doesn't need that. If you're a developer, unless you're, say, sales or an executive, then I would say, yeah, you probably need that. Or just use your cell phone. But, yeah, I would say we don't all need that. And then stop answering the phone. This is not going to work for everybody. It works for me. So if you call my landline, it goes to voicemail, it goes to my email, I listen to it, and then I delete it. it. So that, you know, because most of them are salespeople, right? And I kind of classify them as a type

of an attacker because they will consume my time. So something to think about. And then get on the offensive, right? So don't punish people, right? But instead we want to reward people. And I write a lot of amazing policies. I'm super proud of my policies. It's got like a fancy header. I do page numbers even, but nobody reads them, right? And it's, you know, I'm not saying don't do it. We have to have them, but it's not protecting you. And the annual training, so yeah, You know, typically when I talk about annual security training, the people are like, yeah, how fast can you get it done? Right? It's a race. Done. Right? It's like, yeah,

I did it in two minutes. We didn't digest any of that. We didn't learn from any of that. Right? But instead, you know, buy a stack of Starbucks cards and start giving those out. Right? To the people that are doing good behavior. And then you can advertise that. You can say, "Hey, you know what? These people did an amazing job and this is what we're looking for." And make those into the mentors. And then people will see that and go, "Yeah, I want a Starbucks card too." And they'll start to see that behavior change. People always say, "Oh, security is everybody's job." No, actually it's not. Unless you're rewarded on something, your incentive to go

do that behavior is considerably less. So if we reward these people for good behavior or actually just even make a big deal about it and give them some attention, that's going to slowly change our culture, right? And that's what it's all about is changing our culture to this kind of proud protectionism. I can't say that word. Protectionism type thing, right? So it's where you're proud of your company and you want to make sure that nothing bad is done to it. And once you get to that point, you know, the people that are drafting and behind you, now you're going to, you're not going to be rude to that person, but you're going to say, hey,

do you work here? You know, and, you know, if you don't, I can help you go through reception and help you get set up there, right? So that's the only way that I see us really battling that. Now, Another huge chart here that I put together. This is the cheat sheet. So if you want to stop people like me or people that are actually really bad, here's just, you know, I just threw all this down here. You can take a picture of that and just do all these things and you're pretty good then. You know, your job descriptions, I love reading your job descriptions because it'll tell me all about the technologies you use and

that just narrows the scope fantastically. Nation states, they look at this, right? That's one of the fundamental things they look at. Even better is your posts on the forums where you're talking about your Aruba and how it doesn't work properly so you're at this firmware level. That's perfect. You know, do a phishing program, highly recommend it. Do it respectfully. Hatch, right? So the excuse these days or actually forever has been, well if I patch it's gonna break something. Yes it will, right? But we're still gonna patch, right? You have to. You know if you don't patch something just isolate the heck out of it. VLAN, micro segmentation. VLAN is a good starting point but I

would even move further into software defined network, that sort of thing for better micro segmentation. Your AV, you may wanna start moving away from signature based towards behavior based. block everything outbound except for 80 or 443. You know, we had a presentation in here just before me about crypto mining. So one of the things that we were looking out for last year was they'd come in and they'd encrypt everything and then ransom it back to you. Now it's even more interesting because they'll just set up infrastructure in your house that's mining at night or, you know, all the time. And so that's even more concerning because you're probably not going to notice it as quickly.

2FA, whatever you can. So your OWA, that's not 2FA'd. That's one of the, over the last six months, I've heard so much about that as the point of entry for bad guys. Firewall, whatever you can. And assume I'm already in, right? Because I probably actually am. So one of the quotes I hear quite a bit is, if you don't think you're breached, it's just because you don't know yet. And I don't know how many of us spend time threat hunting, but I would highly recommend you carve off some time for that. and look for certain things, right? Like there's been stuff out there now where the beacons are going off every six months or every

three months, right? That's a huge amount of time, right? So looking for that regular behavior, things like that. There are probably bad things on your network already that you need to go find. All right. Everybody always says Rob you gotta show tools that you actually use so these are some tools But I'm gonna warn you that it it really depends on what you're trying to do for what tools you use so some of these are very generic right so YouTube You know, for my target, I went on YouTube, found a tour of their office, slowed it down frame by frame, and I got this guy's image reflected in the window where he's holding up his

card doing this against the door. So I know they use access cards, right? So there's tons of stuff you can find like that. LoopNet for commercial properties, you know, the whole layout of the office, everything like that, where the bathrooms are, where the doors are, tons of stuff. Street View, of course, right? It's super easy. And then you can get into OIT, sorry, IoT and other stuff like that and start looking at their records with the city and stuff like that. You can go deep there on the physical. And then the technical is even more fun, right? So one of my favorites at the very bottom is Wiggle. It's not super accurate, but it's a

good starting point. You might better pick up their SSID and just tons more stuff. Their DNS. These are just some of the tools. Every day there's new tools coming out, so it really depends what you want to do. Some corporate stuff. So just going on the job descriptions is great. Set up a PaySpin account for your, and then do an alert for your account. That works out pretty well. If stuff starts showing up on there, that's a really bad thing. Yeah, I would look at, see if your parking information's on the internet. Do they talk about your CCTV, your security guards? Those are all really good things to look at. For your job descriptions, I

would just take out your technology. Don't say Cisco, just say network, right? Keep it more generic. your staff, so this is where it gets really good of course. LinkedIn, start there. You go through the typical social media stuff like Facebook, Twitter. SlideShare is amazing and it doesn't, I wouldn't think it would be, but there's tons of reference letters on there. And the reference letters usually have all the personal information of the executives. which is great. So personal phone numbers, addresses, everything like that. I didn't know, but criminal records are also publicly available in Canada, which is fantastic. So when I'm doing hunting for individuals on my SAR side of things for missing persons, you know,

I'll go find somebody and then I'll check this. I'll find their aliases and stuff like that, which is really interesting. If you rent space and you have tenants, it's also a great place to look as well. Personal websites. So then resources. These are just some of them. So for the US, Michael Buzell, great podcast, great website, some really good books. He's kind of written the de facto book for OSINT. Toddington here in Vancouver, he does courses. A lot of the agencies go to him. Again, a really good resource. Social engineer, does the SE Village at both DEF CON and Derby. Great experience. And then if you're really into this stuff, I'm going to butcher his

last name, Robert Caldini. Has some really good YouTubes and books as well on the kind of psychology behind influence and some of the principles behind it. So that's a great read as well. All right. I kind of blasted through some of that. Questions? No questions. One question? Sorry? Oh, yeah. So tracelabs.org. I'm really excited about that. So please check it out. The project's designed to look for help. first responders find missing persons. So there's tons of missing persons that happen every day, right? You open a file with the police, the police get it, and they don't have a ton of resources, so they have to prioritize. So some of these missing people will just show up, and the police know that. They'll

know that, okay, well, if you're a teenager and you're reported missing, you're probably going to show up in a couple days. But sometimes they don't, right? And then sometimes it's a body recovery, which is terrible, right? So having worked in the industry, I kind of know that the faster we start looking for people, the better off they are. Sometimes it's going to be a false alarm and they're just going to be at their friend's house, but sometimes it's not. So what can I do to help those first responders scale and then also find people faster? So teaching them how to do OSINT is kind of what I'm trying to do with that. I'm trying to

do it in a fun way so we can crowdsource InfoSec people that want to learn OSINT and not only kind of train them through this site, but then also help the authorities. So it's interesting so far. So I've been doing some work with the police, with the RCMP, talking to them about it. And they've actually been pretty open to it, which I was thankful for. Yeah, I thought they just shut me down. So I registered it as a nonprofit. I haven't had a lot of time to really drive it forward over the last few months. But now I've been accepted at DEF CON. I'm starting to ramp up for it. So pretty excited to see how that'll work out. So the idea at DEF CON is to

have an international focus. So look for missing people in different countries around the world and help those authorities. So yeah, I'm really stoked about it. So looking forward to how that'll work out. Yes, sir. Oh, right on. Yes, yes. Mm-hmm. Yeah, yeah. So we have to be very careful about not only what information we post, so it has to be all public information. Sometimes as a SAR individual, we have more information than what is public, so we can't post that as OSINT. So we have to be very careful that way. Another thing is that it has to be a person that's been reported missing by the police. So it can't be just a family member says, oh, can you help us find this

person? It has to be an actual kind of like a task, righ