BSides Las Vegas 2019 Day 2 - Ground Floor

Attackers are not looking for flags. Attackers are looking for particular access to data, particular access to systems, and you have to understand their motivation as to why they're going after something and what it is that they're going after. Also being able to take multiple approaches to something. In a CTF, there's usually one way to get the flag, at least if the challenge author has done a good job, there is one defined path to the flag. In the real world, a single service may have multiple vulnerabilities that you can attempt to exploit. A single system may have multiple services running on it, and a single data store that you're trying to get to may have multiple

systems capable of accessing it. So from the security practitioner point of view, you have to look at it and think about whether or not the path that you are currently pursuing is the right path to pursue. When you play CTF, usually there's only one path that you're pursuing at a time. Another skill that's absolutely critical to the security practitioner is communication skill. Most security practitioners, I'm sorry, we have to write reports. I know most of us don't love it. I don't love it. But it's an absolutely important part because it's really important to communicate, particularly to non-technical individuals or non-security aware individuals, what the impact of a vulnerability may be, what the impact of their business decision to not use TLS may be, what the impact of any

number of other choices that are made could be, as well as communicating to them the steps that they need to take to remediate issues, right? I've seen people file bugs And if you've ever participated in a bug bounty, you'll see some of these where they come in and they just say, there's a cross-site request forgery here. Well, it doesn't tell you anything about what the impact of that cross-site request forgery is. Like, is it a cross-site request forgery that results in the user being logged out, or is it a cross-site request forgery that results in account takeover, right? Those are dramatically different circumstances. And being able to communicate this is an absolutely critical skill, and

again, it's not something that's traditionally exercised in the capture the flag space. Also, teamwork. A lot of CTFs are team-based. But at the same time, what you'll see happening is it'll basically be just a division of labor. The teams will sit down and one person will take this challenge and another person will take the second challenge and a third person takes a third challenge. And I hate to break it to you, that's teamwork only at the most rudimentary level of the skill. True teamwork involves every step you want to take as a security practitioner. becomes an argument again, right? You're arguing with management and, you know, and with finances and that kind of thing. So picking a standard in itself has addition, and it's not just even in

vulnerability disclosure, but picking a standard has that benefit of cutting down on the number of arguments you have. I don't have to argue about each step of this thing. We've already picked the standard that we wanted to do. I think industrial control systems are kind of late to this party. A lot of other things like popular commercial operating systems have had a lot of attention in the past because of the, you know, maybe not quite accurate stereotype of a hacker in the basement who has access and some free time or curiosity or ability and explores things like Windows, looking for vulnerabilities, and notifies Microsoft, or the manufacturer, and they develop a process for dealing with those vulnerabilities. Industrial control systems tended to be rare and

expensive, and so they didn't get this scrutiny, the same amount of scrutiny over the past 20 years or so. But now, thanks to lowering prices and things like eBay, you can buy old equipment for cheap and start doing testing in your basement again. So industrial control systems are now starting to get more scrutiny than some of the other operating systems, some of the other more commercial, popular type of applications have already gotten. So they're a little late to the party. Maybe there's a little catching up to do because of that. Maybe not quite as mature. All right, so if you get this call, by the way, who in here are electricians? Anybody? Electro? Okay, well, so,

by the way, you're supposed to ground these devices. You run a green wire to ground to keep people from being electrocuted. That's ground. Okay, all right, so things not to do. If you get this call, don't hang up. Don't ignore them. Your vulnerability is not going to go away because you hang up. A matter of fact, ignore them and it will go away. Chances are good they're not the only person that knows about this vulnerability. They're just the only person who has bothered to let you know they know about a vulnerability. Other people may have, and particularly more well-funded people, probably already know about this vulnerability. So it's definitely not going to go away just because you don't listen

to them. It's still there. It still exists. Don't start out by claiming there's no problem. "Oh, our product is wonderful." You know, "What are you talking about? Our website is great. Our customer lists are safe." You know, right? Listen, pay a little attention. They're going out of their way. Don't threaten or begin legal action. We'll talk more about that in a second. Don't assume the worst of this caller without any data. Again, just because of it, you know, isn't a guarantee that they're benevolent, but it's a pretty good indicator, right? They brought you something. They brought you a gift that they potentially could have sold or used elsewhere. So... Don't necessarily assume that your system is configured optimally without any data,

right? You might have not set it up according to the best practices and so forth. Don't claim there's no problem immediately. All right, legal action. All right, so house on the coast in California. Anybody knows whose house this is? Farber Streisand's. This is her house. And how do we know this? Because back in the day, there was a photographer that had taken pictures of all of the California coast, 12,000 coastline photographs, and her house showed up in one of them. And she was concerned about her privacy, and she wanted this taken down. And so she initiated a lawsuit. Here's their lawsuit. So before she initiated the lawsuit, this picture of her house from the air had been downloaded six times, two of

them by her lawyers. One month after she began this legal battle, her house picture had been downloaded 420,000 times because of the publicity of such a... I include the picture three times here just for reference, okay? So... The action and we give it a name right the Barbra Streisand effect, right? How do we know a random house in the middle of California because of this lawsuit? All right, and that's the Streisand effect that sometimes Performing those actions to try to hide Hide some kind of vulnerability might actually bring it to life lawsuits may also have chilling effects. So It might make you, your company, your organization look like the bad guy. No, we're just going to

sue people instead of fix things. Maybe as importantly, or more importantly, researchers might not contact you in the future with vulnerabilities. And you've lost a valuable resource. They might do something else with them. They may not bother. They may sell them. They may keep them. They may hide them. Somebody else can find them and take advantage of them. So just that chilling effect of keeping people from bringing you these gifts, reporting this to you, has some cost. What day is it? All right, so we're a security crowd. What's a no day? Come on, you're awake. Y'all laughed. I heard you. Affirmability that the vendors have in cash. Yeah, you've like had zero days since a patch to fix it. What is this?

A pony? There was a speaker request for a pony. I always ask for a pony. There you go. This is your pony, sir. Wow. I don't always get the thank you. That is hilarious. Is this really my pony? Yes, really your pony. Your very own to take home and love and feed and pick up the poop. That's sweet. Look, I always ask. They're like, what else do you need? I'm like, a pony. Sweet! So now you're not just an infosec tenant, you're an infosec stable hand. Perfect, thank you very much! I like my pony. Actually, I asked my last boss every time, "What do you need?" "A pony." So many times, he brought me a

pony. I have a rocking horse about this big at work right now that I put all my conference badges on. But now I have two ponies. That's sweet. Alright, so you've had zero days. You're just jealous, you don't have a pony. You had zero days to fix this. A couple of buddies of mine, we'll touch on them a little later, ran a research project. So, DMP3 is a protocol used in industrial control systems. A guy, Adam Crane, was creating an open source version of it, and to test his open source version of this protocol, he created a fuzzer for this protocol. He wanted to make sure his protocol stack didn't have any vulnerabilities. It was project robust. And so he did. And then he got up with

another friend, a guy named Chris Sistrunk, who had a large lab of industrial control system equipment. And they got together and they said, "Huh, I wonder what would happen if we ran this protocol fuzzer on all these devices that are supposed to understand this protocol?" Any guesses what happened? It was a bloodbath, right? Tons of equipment had vulnerabilities, nobody had tested them before, they fell over, all kinds of nastiness. They went through the process of notifying vendors, and we'll touch on that a little more in a moment, but at least in one instance they came across a vendor who said, "We're not gonna fix this, ever." It's a vulnerability, but we're not going to do anything. This product's too old, you

know, whatever. We're not going to do it. So from there comes a term, I think coined by another friend of mine, Reid, the forever day. So now we have zero days and forever days. So the forever day is a vulnerability that is never going to get patched in your control system. Probably, don't get me wrong, O days are O days are a problem, but in a control system probably a bigger problem is your hundred day, right? The patch has been out there for about three months now and you still haven't done anything about it So that's probably if you're really trying to set your focus in a control system O days are fine Let's worry about the hundred day right and the 300 day and

sometimes the 3000 day right as long as we get a patching cycle Started that helps us deal with our vulnerabilities part of Doing working in vulnerability disclosure is kind of understanding perspectives on On the people who you're dealing with and there are always multiple perspectives this article came out not long ago Davey says warning Google researcher drops Windows 10 zero-day security bombs That was his headline. The story was vaguely a vulnerability that Google had discovered, they had talked to Microsoft about it, they gave them the 90 days, and at the end of 90 days, Microsoft had not had a patch yet, and so it was disclosed. I started talking to Davey on Twitter, and I

proposed another perspective on the same story, at least another headline, another summary. The summary could have said, "Windows 10 O-Day info released by Google according to responsible disclosure guidelines allows you to consider possible risk and mitigations to the security of your systems now." Well, that sounds a little different, right? I mean, it's about the same thing, but It's a different perspective. The perspective that letting you know about a vulnerability gives you the opportunity, perhaps, to mitigate some of the risk. There is an upside. That's why we do this thing. If there were no upside, you call us all jerks, we all go home. But now you have the possibility. You might shut the service down, or maybe you can change the config, or change the firewall rule, or

maybe move to another vendor, whatever it is. But if you know about the vulnerability, you have some possibilities. So I asked him about this. He's like, well, that was his headline, and that's fine. And he said I could use this with permission. Just from a kind of perspective, right? Are we, was it really an old Windows 10 zero day security bomb? Or was it advanced notification? Just different perspective. All right, so you work at a company somewhere, perhaps. What do you do? We talked about a lot of things not to do. What can we do? Make it easy for people to contact you, right? Have like security at your domain.com. it can be hard to find the right people to contact. Who usually

gets this call? Who's the most exposed people in a company? Tech support, maybe, what's some? Yeah, yeah, yeah. Yeah, customer support, sometimes sales, right? So they're the most likely ones to get the call because they're easiest to find. They're the easiest to locate. But they, unless you've talked to them, unless you've trained them, unless you've sort of set the system up, they don't know what to do with it. They're like, hackers, we don't want to talk to you, right? And there, you just took this gift and you threw it away, right? So make it easy to contact. Abuse, publish a phone number. Oh, we don't publish phone numbers. Look, right, you know the people finder apps. Everybody's phone number's

available. Perhaps include a secure way to contact you. You know, I mean, it's fine for them to send the vulnerability and plain text across SMTP, but you know, you could do it across PGP or GPG. So if you want to include some secure communication method, say thank you. They brought you a gift. At least at this point, right? They brought you a gift. So say thank you. Hey, you know, let's figure this thing out. What can we do about it? Change your contact information. Get them in touch with the right people or the right team in the organization. It's probably not tech support and it's probably not sales. It might be a QA team or security team or a bug fixing team or a product owner. Whatever the right

person is, identify that person and be able, be willing, be prepared to set up that communication. Acknowledge the bug and let that individual know that you are working on it or not, or what your timeframe might be. Because we're working toward a kind of coordinated disclosure. We want this to work out well. Supposedly they give you a little notice ahead of time and you can release a patch when they release information about this vulnerability.

Don't go dark on them. You might be working yourself day and night to fix this thing, but if you don't let the person who let you know that you're doing that, they may think that you blew them off, right? Then they get upset, they're ready to publish it, they're not hearing from you, you're not doing anything about it, I'll just let this vulnerability, I'll publish it early. So give them some notification about what your expected time frame may be. It tends to keep them in the loop and keep them a little happier. If you're nice to them, you might be able to ask them to test your fix. You reconfigured your website that was spilling

all your company data, and you say, "Hey, can you look at this now? Is it better?" Because unfortunately, sometimes we've seen patches come out that don't fix the problem, right? Then you look bad, it's like, "Oh, thank you, bye." You go and work on it for three months, it's like, "We got the fix, and it didn't work." The guy comes back, "No, that wasn't it," right? So the analyst, whoever discovered it, if you're nice to them, they might test it for you for free. Maybe not, but you know, right? So be nice, stay in contact. They might help you along the way. You're trying to work for coordinated disclosure. Again, that kind of, that trade-off,

they're going to give you some time to try to patch it, but they're going to make it known anyway because there's value in that. Other people can defend themselves if they know that. They might test your patches. You get, you end up with a win-win situation. in these cases. Look, if you're looking at a product and you're trying to decide maybe between a couple of different vendors, and you see one vendor has patches and vulnerability disclosures for their product, and another vendor has nothing, does that sway your decision any on which vendor you would purchase from? Which one do you purchase from? This guy has zero vulnerability disclosures. Zero. And you buy from this guy?

Why would you do that? Because everybody has vulnerabilities, right? That's saying something about the maturity of their program in handling vulnerabilities. They will acknowledge them, they'll patch them, they will let you know, right? We have no vulnerabilities? Right. Don't believe it. Oh, that's for my Miami talk. So, what do some of these people want? when they contact you. Some people just want to make the world a better place, frankly. They work in the space, they have customers, or they like electric power, you know, they want it to stay on. Sometimes all they want is to make the world a better place. Help them. Often they probably want some recognition, right? When you release your vulnerability disclosure, you can say, "Discovered by so and so or

such and such company," right? That's a little bit of the trade-off. They brought you a gift, they gave you something, they were working to find valuable information for you without pay. The minimum you could do is, and it's fairly cheap, is to give some recognition for their work. it might build their reputation as a security researcher and and so forth um you know just try to give them an easy time they brought this thing to you right try to make their life easier i know a security researcher is he's a really great guy he does a lot of this work and and he posted on twitter uh sometime last year you know he was just

ready to like pour gas on it and drop a match he's been fighting to find the right vendor and to let them know and he came

So how do we help the players to build these skills? Well, we have the Red Cell Pros as a sparring partner for the Blue Joes, right? So the initial start in Pros vs. Joes, the players are trying to defend their networks. They are hardening network firewalls, hardening host firewalls, changing passwords, disabling unnecessary services, patching services, all of the things that you do as a security practitioner. attributed us and asked if we'd test their fix, they were such a joy to work with, right? And I don't think that they're a bad company because they've listed a vulnerability and a fix for it, right? I think they have a mature security program because of that. And I know some of those guys, they do really care about

their security. Anybody remember the Kaminsky bug, right? DNS. So, uh, Dan has this, I was asking him, he was trying to coordinate one of the biggest sort of responsible disclosures probably in history of tons of companies and internet service providers to fix this DNS bug. You can look up the Kaminsky bug if you don't know about it. His advice was if you are a researcher or a company, just don't be a jerk, right? Try to be nice, try to be reasonable, try to be responsible, try to listen to folks. He even created this flow chart, which you all can read all of and memorize. We have a quiz after the... I will point out this one thing. His flow chart's a little older, but it says, if

you go through all the right stuff and notifying them, the company should be okay, right? This should all work out, because it's 2012 and everyone has a security team now. So I just want to remind you all, it's 2012. And all the security stuff is worked out.

Other people do bug bounties. They will pay anonymity if you want to. If you're afraid of the company or for some reason you don't want to be exposed, they will maintain that for you. They will help you contact companies. They do this on a regular basis. So they generally know the right people to talk to and they generally know the right voice to do this in. So they are your friend if you want in this place, in this space. So A little more thought about perspective, right? I'm a big believer in understanding other people's perspectives. Now, it doesn't mean I agree with their perspectives, but it's useful to know them. So, if you are a security researcher and you're discovering some vulnerability and you're trying

to talk to a vendor, some of the things that may be going through their head is, "Aren't all hackers evil?" or "I don't understand this, you have to be evil." Right? "You were talking about something wrong with my system, you must be evil." Right? If you might be approaching them with a gift of hard effort. "Are you trying to harass me?" I'll look bad if I mitigate and acknowledge a bug and work to fix it. or maybe they just have no way to deal with it. They don't have any kind of system to deal with a bug or to prioritize it. They have work to get done, right? They don't know what to do. So

they're sort of an edge case or something that really steps it up at the end. So I expect everyone to be able to get the first challenge. There's enough information there that you can definitely get it and definitely get introduced to the concept. The second challenge should be achievable by most players if they're willing to put in a little more effort. The third challenge will be a stretch goal for players that have never seen that concept before, but your more advanced players may actually still find themselves challenged but accomplish the third one. And so this gives you not only all of sample HMI's, a new patch comes out, we test it against all these machines.

Imagine all the different kinds of configurations and machines and industries that their product is in and they need to test that patch, test that change before they roll it out. That can be an immense thing for them to think about. So give them a moment to process that. We're really busy. Everybody's really busy them too right you're busy. They're busy. I like these are your kid Are you kidding our customers don't want patches? All right, who laughs why don't they want patches what it break things? It's a pain to install you're gonna shut my assembly line down this thing supposed to be running 24/7 all right some of our customers would pay us not to patch this because of they have to install them again, I don't necessarily

agree with them, but I understand that perspective and They might also have the perspective of, thank you very much. We're going to get right on this. But those are things to consider, their points of view when you come in this. Remember, too, if you are doing the reporting, you can be respectful. Realize that patching can take some time. Even if they want to and they work on it, it can be a big process. Ask for recognition. That seems a reasonable ask. See if they have a bug bounty program. Be willing to... And second of all, that's a lie. Any equipment you've bought in the last five or ten years, it's programmable. Just take my word for it. It's programmable. So don't believe that because it's from...

Another one is I was asked to evaluate a security vendor one time for my company. I went to a conference. I'm talking to them. They tell me about the features. It's an appliance. And, you know, we do a lot of patching work. And I'm like, oh, well, how do you all patch this? They're like, we're running security enhanced Linux on this appliance. We don't have to patch it. I'm like, what? I'm like, look, I like security enhanced Linux. It's a great product. It's a great tool. I'm a Linux bigot. I'm in, but you still have to patch it. So then the problem becomes, how do you say, thank you very much. I'm busy at the

moment. All right, so that's us talking about vulnerability disclosure. Hopefully you get some perspective, whether you're a researcher, whether you're going to notify companies, or if you are working in a company or business that might get this call one of these days. Hopefully you will get the call. Somebody will let you know and not keep that from you or keep it private. And so I have suggested to... Mr. Lee, a friend of mine, that this should be his next book, Hackers in Me by Rob Lee and Monte. - Attached it to like my Cali box and then just went in and got the flag. And I feel like they're kind of cheating themselves. But I

do think they're interesting, particularly if you're trying to learn a particular skill to find relevant boot to root challenges and use those as a skill builder.

Hi, my university's cybersecurity club would really like to start its own CTF team. Some challenges we face are... Give it up for Monte. Yeah, any other questions? You got another minute or it's time to shut down? We got a couple minutes. Any questions? Just raise your hand. I'll bring the mic to you.

I was thinking if there were none, that was the perfect talk because I answered them all ahead of time. Now this is, you already answered this but I missed it. What are those ISO standards that slide? I'd like to take a picture of that. Alright, no that's an easy one. I like easy questions. There you go. And that's actually the summary flow chart from one of those documents that sort of describe what they both do. I think I can quote that part without violating their copyright for educational purposes. Just like all those pictures of Barbra Streisand's house. Good morning, Monta. Monta. Monta, but I'll answer. I'm easy. Yeah, long time listener, first time caller. So I don't know if you're familiar with the story, and

I think the gentleman's name is Sam Bowne. He's a Teacher up at San Francisco City College. I don't know the story, but I do know him. He spoke like right before me my first year at DEF CON. Oh, so anyway, he basically did a disclosure because he found a hospital when it merged, acquired another hospital, and FTP server became forgotten. and records were being stolen off there and he basically got vilified in the press because the hospital didn't say "oh thank you" they said "oh this..." An off by one memory corruption that lets you overwrite some metadata... and a pony. Essentially grading on a curve, yeah. We think actually, so that way the challenge that gets solved

a hundred times and the challenge that gets solved once are automatically appropriately scored. So we're interested, it's an experiment, we're interested to see how it works out and play in that CTF. It'll be available online and you can give us feedback on whether or not that's a good scoring mechanism. I've received the stop sign here, so thanks everyone for coming and enjoy the rest of your peace sides. So is ISIS a threat? Cyber? Because that's kind of their... Test test Test test Can you hear me? It's very low Maybe I'll... I'll go to the soundboard and see if I can turn it on Mm-hmm. Hold it in. Hold it in. Can you hear me now? Not really.

Hey, hey, hey. Not so much. How many chairs do you need? Just two? What is it? Two chairs for you guys? No, no. We need the microphone. We need the mic. Okay. Yeah, we got another lot of room for you. Yeah, we're going to need another chair. You guys need water, right? test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test

test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test

test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test

test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test test

test test test test test test test test test test test test test test test test test Test one. Test one. Good. One, two. Three, four. Okay.

And we have Brett and Harlan talking today. A couple quick announcements. If you could go ahead and hold your questions till the end of the talk. We'll go ahead and pass the microphone around to you and make sure that you are able to be heard online when you ask your question. So just raise your hand and we'll come to you. Please go ahead and silence your cell phones now. Again, we are streaming online, so we really appreciate if you go ahead and silence your phones now so they're not interrupting the speaker. We'd like to thank our sponsors, especially our Inner Circle sponsors, which are Critical Stack and Valley Mail, and our Stellar sponsors. That includes

Cylance, Microsoft, and Robinhood, just to name a few. Their support, along with our other sponsors, donors, and the volunteers, they make this event possible, and we really appreciate all of them. And if you have any questions, again, please hold them until the end. And without further ado, thank you very much. - Great, good morning everyone. So I'm Brett Goldstein. I'm the Director of the Defense Digital Service. And I work at the Pentagon and now I'm in Vegas. So thanks for having me. This is my friend Harlan, he's an engineer on our team, and we're gonna tell you a whole bunch of stuff over the next hour. So it's a story that will essentially have three parts. I'm gonna give you my background. It's kinda weird, and sort

of like hopefully that'll be sort of morning fun. And then I'm gonna talk about DDS, and why we aren't what you think we are, and how it's completely-- - Alright, with that, who's got the best B-Sides joke? Who wants to give a joke today? thought that there was a better way to do that and we went up against the restaurant industry right and we wrote horrible software and all the maitre d's out there hated us like it was like we have our big reservation book and it's amazing and i'm like there's a thing called the computer and that could be really amazing as well um so i spent seven years of my life building that company and so along the way um 9 11 happened

And, you know, Open Table had been going for a couple years at that point, and I was traveling on 9/11. And so I was evacuated off a plane, I was in the terminal watching the tower go down, and it was just, it was horrific to me. And I'm sitting there, and I'm like, So I'm helping rich people make dinner reservations and I'm watching people do some really important things on the screen. So on my way home, because for those of you that remember, the flights were grounded for a while, so I just gave up. I went home to my wife and I'm like, when I'm done with this open table nonsense, I need to do

something that's more meaningful. So, okay, so what is that? So originally my plan was I was going to do volunteer work, right? I was in Chicago, and it's, if you haven't heard, Chicago's a little chilly in the winter, and I was going to go do checks on folks and do that. Prisoners are only referred to by their numbers, with our prisoner being number six. While escaping from this village, as our number 6 soon discovers, is an extremely difficult task. Nevertheless, our industrious number 6 never ceases to try. A quick question: How many of you heard of the prisoner or actually seen it? Oh wow! This is fantastic. We're going to have a lot of fun with it.

The village is going to be a Docker container that imprisons us. During this session, we attempt to escape from our village, from our Docker container with the goal of taking over the underlying host and even eventually running code on that host. Do we succeed? Well, stick around to find out. All right, so let's jump into the agenda. Now, in the meanwhile, make sure that the mic is up and running. All right, so just before I'll cover the agenda, how many of you know how the series, The Prisoner, ends? All right. So you have another reason to wait for the end of this session. And if you already know, don't spoil our spoiler. All right. So-- Sorry. All right, we'll start with the

short introductory section. In this section, we'll introduce prisoner number six, for those of you who are not familiar with it. We'll also take the opportunity to introduce ourselves, and one important thing that we're gonna do is try to understand what the hell prisoner number six has to do with containers, and Docker containers in particular. So this is gonna be the introductory section. We'll then move on to the section called Welcome to the Village. This is a container introductory section. We'll try to define what are containers, the conditions that we like, and what is a privileged container in that context. We'll see how privileged containers can be deployed, and we'll also check if and where those privileged containers are used

out there in the wild. So this is the welcome to the village. Before the caches expire. Okay. Okay. When the caches expire, hold on. Let me wait for the command. Okay, when the cache expires, you can see that the local DNS cache is empty. Now, we will send a new DNS request by sending a ping. This in turn will trigger, of course, a new DNS request, and the DNS response will resolve the new IP and map the host name to the 1337 IP. Now, let's see how it works. The victim computer sends a request to evil.com which goes to our DNS server. The DNS server will resolve our malicious server IP. Next, the browser will issue a GET

request to the resolved server IP and execute the following malicious JavaScript code. The JavaScript code will call itself until the evil.com HTML title is changed. Once the title is changed, the attacker knows that the rebinding process was successful. We're going to have to find something to do with that. Okay, can you hear me now? Okay, so this is the opening scene from the prisoner. And this is our intelligence agent who is now entering his superior's offices at the British intelligence. with the intent of submitting his resignation. And then it's very like people walk very appropriately and it's it's it's you know, it's that whole bit and I'm going there and I was catching up with my friend Chris Lynch who

I know knew from I was on the board of Code for America for a number of years and so Chris runs this defense digital service thing and I'm like, oh "What is this?" Because I'm in the Pentagon and I'm like, I'm giving people advice, but I'm like, "We need to raise the bar. "We need the best in technology. "How do we solve problems?" 'Cause at the end of the day, what do I care about in this space? That technology should never get in the way of the mission of national defense. And how do we get, like, I found it completely enlightening. I'm like, how is it that I have the very best that I'm seeing

out in Silicon Valley, but I'm not seeing the very best? when I'm at DOD. And that just irks me because at the end of the day, the criticality of that is important. So, you know, Chris comes, I meet him and then we go into this office and there's this sign that says Rebel Alliance. And I'm like, "Huh? What unit is Rebel Alliance?" And I walk in and it was like walking into Silicon Valley. And it turns out that we have this bastion of talent at the Defense Digital Service that we bring in some of the very best. And we're going to be talking about that. Which reference is? Yes. That village is Rover. We'll see more of the Rover shortly. Iron Maiden,

The Clash, and 80s influential rock band and other music bands have all directly referenced the prisoner in their songs with Iron Maiden even incorporating an entire song with dialogues of the prisoner in their "The Number of the Beast" or "Lost" and also has been featured in the movie "Shrek" which is quite interesting. We believe the prisoner is still relevant today with the rise of tech giants and their control over our everyday life. Multinational organizations track our every move both in the cyber realm and the physical one, brainwashing us with ads and feeding us with information they want us to digest. We are all living in a technological prison. Alright, so just before we're moving on to continue telling you

the story of... Can you hear me well now? Yes. Alright, so let's move on with this. Alright, so who are we? We didn't introduce ourselves properly. So this is Nimrod Stoller, a security researcher at CyberArk Labs. And my name is Lavi Lazarovitz. I'm a security research group manager at CyberArk Labs. And we are both based in Tel Aviv, Israel. And so what is it that we do? We both... part of a group called Charlie group which is focused on the security research of emerging technologies like for example things that we do is security research for containers for example and we do other hardware and software stuff as well our offensive security research is the ground of which we work to develop

new, innovative ways to mitigate the attack vectors and attacks work on a mission. And the core sort of types of projects that we do. So one, We're firefighters, okay? At the end of the day, the Defense Digital Service, if there is a technical heater which is just absolutely horrific, we are going to be the people you call. Because I will say all day long that my team has the very best in technical talent anywhere in the DoD. My folks are amazing. and they can solve problems and they will go wheels up in hours to anywhere in the world to fix a problem and that is amazing. Two, we will advise on projects. So say there is

a big enterprise initiative. We're the best in technical advisors. There are really stupid ways to do procurements and we could pick the worst in tech. So if you don't put the nerds in the room, how are we ever going to do this right? Like for those of you that sort of have some history here, you know, you go back to when did we like the idea of having security in the boardroom? There was a day long before that, I remember in the early days of Open Table having to sit down with the board and justify cybersecurity expenditures. And they're like, "Why would we spend on that?" I'd be like, "Oh, because this nonsense is going

to be hacked, yo." And the village is the fictional setting where everything is happening here. But who runs the village? Apparently, the village is run by a democratically elected council with a popularly elected executive officer known as Number Two. But everybody in the village knows that this process is rigged and the operators of the village are simply using it as a means of control, of their control over the inmates. tools and alcohol are strictly forbidden in the village but there are no walls or physical barriers to prevent escape and apparently there are no visible guards a control room monitors continuously closed circuit television cameras located inside the village and observers spy on every move of the villagers in

order to foil escape attempts with the aid of a rover, a large white balloon-like device that chases would-be escapees and carries them back to the village. Let's see now such an attempted escape with the rover. "An international swimmer at the age of 17, Olympic bronze medalist." "She'll be out of range soon." Alright! Much like container escape, you'll see in a moment, I promise. So just before we're getting into the more technical stuff and the live escape and so on, just to get us on the same page, I want to share with you a couple of definitions that we use during the process. And the first one is, of course, containers. So the definition that we use for containers is a set of software processes that

are running within an abstracted and isolated environment. This is the definition we use. And for that matter, privileged containers are any containers allowing an attacker access to the host or other containers, the data or execution flows. This is the definition of privileged containers that we use. Now it's important to mention here two things. First one is even if we run a privileged container, and if you know Docker, you know probably the --privileged flag. Even if you run a container with the --privileged flag, which assigns all capabilities to the container, eventually making it a privileged container, by deploying various security tools and defenses, we can transform this privilege container into a non-privileged container. And we'll see that in a few moments. You'll

understand what I mean. Second thing that I wanted to mention here is that we did not create or invent this concept or definition that we use here, the concept of privilege containers or the definitions that you've seen here. This concept was actually known since the inception of Linux container. Our modest contribution to it was the extension of the definition of privileged containers. So it's not reports and maybe some of them are HTTP ports. So you need this information in order to get the right response from the right web application. I did not write it, but to talk with many assets in the same web application we created iframe it's really ordered to to get the information from as many web

application as we can and to do it asynchronously so we have iframes and we connect to those iframes with web messaging so now we have the mechanism of how we can send requests, but let's see the real requests. So we send the request to the malicious web application. How we can get the information we need from the web application, the internal web application in the victim browser network from the malicious web application? So the malicious web application has a responses dictionary. and it will save the response object we're using Node.js. It is possible to create a global response object that will have a key value paired responses. And also we can just create a timeout mechanism to set a

timeout when the response is... And also the data, the body of the response that will come back to the user. And now the attacker the malicious web application finds the right response and just respond using send function and returning the response with all the headers, even where basic authentication headers and with all the information and with all the body of the response to the attacker. Even if it's binary files, the attacker will still be able to download those binary files using the same technique. So, as I said, you can find the Retanl code, source code, here in GitHub, github.com/retanl. And now I will talk about the easy part. Why is it so easy? It seems so... So hard to

set up all these things. I can agree with you, of course. But let me share with you another example that I think resembles another type of privilege containers. And this example is of Datadog. Datadog is a monitoring service. But, cool box, you know, I wonder if they imported people that I've visited in the past, right? So, put in a name, get a result, cool. name, email, and then if you click it, it pre-fills their information and it just reuses the last investigation that they had again, right, and refreshes whatever the data was in between. Perfect, makes total sense. Saves me time, saves them time, everything's great. But something weird happened when I entered names in there. I noticed that it was returning names

I didn't recognize. like just random people I'd never heard of. So I asked around the team like, "Hey, does anyone recognize Billy Bob?" And they were like, "No, no idea who that is." Huh, that's weird. So I tried a couple of things and it very quickly became apparent that what this box was actually doing was not listing all guests that I previously invited, but rather was listing all guests that anyone had previously invited. Okay, somebody forgot a where clause. Whatever. So I start looking people up. Who's visited the Pentagon that might be interesting? Well, you know, Eric Schmidt. Non-privileged ones. So what are Play with Docker and Play with Kubernetes? So Play with Docker and Play with Kubernetes are a wonderful initiative. They

are... They allow users to load and run Linux containers in a matter of seconds. It gives the experience of having a free Alpine Linux virtual machine in browser where you can load and run Docker containers and experience the Docker platform firsthand. Second thing I noticed, huh, you know, even when you're getting, you know, a handful of responses, I think there was like 20 responses in here, 9.3 kilobytes for a name and an email address. That doesn't seem right. That's really weird. What's going on here? Before I advance to the next slide, I should say for the record that the data that you are about to see is all synthetic, and I just generated it yesterday from scratch because it looks like this. You'll

see some interesting things in here, right? So first of all, you've got name. You've got a UUID. Good. UUIDs are great. You've got update time UTC, which is a string of slash date and then the Unix time. Okay, that's a little gross. Whatever. You've got their first and last name again. Middle name, name suffix. Great. Social security number. Huh. That's not good. Investigation, oh great, investigator 4979 did this investigation and this person was approved. Okay, so now, instead of scraping this to get the records of all the people who visited... Number six, along with the other prisoners in the village, all know too much. This makes them a threat to the organization behind the village, and therefore, because of their specific knowledge, And that

means that our attacker has access to secrets and credentials injected or loaded in all containers running on that host. For example, by using Docker Inspect, our attacker can inspect the inner definitions of all of the containers running on the host and extract secrets and credentials. If our attacker has privileged access to the host, it can also run code on each and every one of those containers. For example, by using docker exec and docker attach. Another important thing that our attacker can now achieve is our attacker can stop and even completely remove containers from the host, any container our attacker may want to. And our attacker can also Create new container or replace containers that were deleted

by using docker create and docker run. Well, this is another interesting point by having access privileged access to the host Our attacker will have access to the docker hub credentials if the host is a development machine now the docker hub credentials are used to to send or to push images to the Docker Hub. Now, with these credentials, our attacker can infect all the Docker images or other repositories of the organization and perhaps even demand ransom afterwards. And last but not least, the network. There's probably an inner network that our host is connected to. This may be, for example, a container orchestration network. and by leveraging lateral movement within this network, our attacker can move from machine to machine and perhaps eventually... digital

service like 18F. There are people who are trying to do the right thing, but we need help. We need help from people like you. Both to come and stand where I'm standing, to do this work, to give your time to serve, but also to help us engage the community at large. Right now the DoD did a thing that no one else in the federal government had done a couple of years ago thanks to the work of our team, which is the DoD created a vulnerability disclosure program. So anyone, anywhere in the world, if they find something in any DoD system, they can report it to us and as long as they abide by what the industry considers to be responsible disclosure practices, the

DoD will not press charges. In fact, they will thank you. And that project is incredible. We get reports from people all over the world who find vulnerabilities in DoD websites, in DoD systems. They find DoD data where it's not supposed to be, including people in countries whose own governments would pay them very dearly for that information. And instead, they come to us so that we can fix it. That is an awesome amount of responsibility on us and that is an awesome amount of power that we are putting the play with docker container or platform we discovered that we can exploit the system console of that platform now the system console is a device is a linux device which we see it tends to

be forgotten so what what we're going to do is if we have a valid user in the system, on the host, then that user may use that feature which enables users to log in and type in their username and password and after that the system console would open a shell on the host and allow that user to run code on the host. So this is our target, but before that we have another step because we need to add a valid user in the host's etc password file. So we have two steps to perform here. One would be to add a new user or to change an existing user on the host etc password file so we could log in as. And the second step would be to

exploit the system console to inject keystrokes into the system console and we do that by using a small utility we found online which is called TTY echo and we use it during the live escape. Well here it starts quite small but on top you can see that these are all the messages from the kernel. This is the Ubuntu system console on my machine and below you can see a user trying to log in so it's user one and he's typing in his username and password And after that, the system console opens a shell on the host to him So this is what we're going to exploit during the live escape And now to the live escape I'll try to use

this I don't think this is working I'll try to shout The right side And on the left side is our attack machine which we will later on run a listening netcat on which will accept the incoming reverse shell connection from the host. So we start with a brief reconnaissance phase where we try to chart the borders of the container that we are located in and see if it is privileged or not. So we're going to start with UNAME And I'm going to do this. And here UNEN shows us the version of the kernel, 440.154.generic, and the underlying host runtime libraries, which is running Ubuntu. But we may have other runtime libraries inside the container. We're going to check that by

looking into etc.release. Okay, etc.release. Now, this tells us that inside the container we are running with runtime libraries from Alpine Linux. So we're going to first use Alpine Linux's package manager to add libcap, which is a library that will help us decode the capabilities that Lavi was talking about earlier. And we're also going to load our TTY echo and other helpful apps from our attack machine. and untar it. And the capabilities inside the container, which is this long hex number. And we can use KEP and Sage decode to decode them. Let's go over them really quickly and see what we have inside the container. So we have the following capabilities. We have KEPnetAdmin, which allows us to the container

to administer the network. Capsys module which we've seen before which allows us to load and unload kernel system modules. Ptrace, Capsys Ptrace allows us to debug other processes. Capsys admin is the catch-all capability that L'Aviv was talking about that allows the container to perform administrative operations and Capsys boot is a capability that allows the container to completely replace the underlying kernel. So from an attacker perspective, it seems like we are pretty much privileged here in this container. So our next step would be to look into proc command line. So this file contains the parameters that are transferred to the kernel when the kernel is first executed. So we see the first parameter is boot image, that's the image the kernel is running from. And the

second parameter is the root device that the kernel and also the host is located on. As you can see, it is designated as a UUID. And we're going to use findFS somehow, findFS, to find out the underlying device that the host is located on. So we see the device is devsda1. Let's see if we have access from inside the container to Dev SDA 1. So it seems like that inside the container we have access to Dev SDA 1. I think as a community we all want to do better and get smarter and find the right relationships.

Thank you as well for this great presentation. I'm curious, you're competing with everybody for the shortage of skilled IT and security workers. So I'm curious if you are looking at any nontraditional type job positions. I'm thinking part-time, job sharing, very short-term, maybe 90-day contract projects, you know, and those are just some examples. If you have other things that you're doing, I'd be curious, you know, how you're dealing with the shortage of skilled workers in the competitive environment. So, I'm not sure I completely agree. So I find that when I'm like, I believe in the I'm going to go out to some hack night or meet up or something like that and go talk to people. And, you know, I did one in D.C. a

couple of weeks ago. And last night is like the best thing before you go. You go to bed. I get this LinkedIn message. And dude was like, your talk inspired me. I want to come work. And I'm like, "Okay, I can sleep tonight. That's money." And so I find that when we... So look, when I was on the outside, there's no way I could know about this amazing group. What are you gonna do? Go to USA Jobs and sort of poke around? We're going to do that. We're going to add a username by the name of number six. We omit the password because we don't need passwords here. ...that Chrome has and you don't have this problem. We tried to

bypass this problem using web workers and iframes and stuff. It didn't work, so it's still using this library we created called JSPool. You can just check it out. It's in the same project. And it's still really fast. It's something like one minute, the whole DNS rebinding process for an entire internal network. Between one to two minutes. It's the fastest you will get in the internet, believe me. I saw many tools. Okay, another question? This was difficult to see from the back. Were any of the services you exploited on the internal network SSL-enabled? No, no. Of course, it's one of the ways you can protect from DNS rebinding exploitation. So if you want to protect against

DNS rebinding, SSL, if you don't have a bot that has SSL termination off or something, you can just check that this bot will not verify, like Selenium do it by default for most of the web browsers because they don't want to check the SSL you know when you create an internal web application the SSL is just inserted a new user into the whole CTC password file so we just need to do that and by pressing enter here since we don't have a password The system console is supposed to at least open up a new shell on the host and give us access to that shell. So let's press enter here. Now on the other side, on our attack machine, it's time to

run a listening netcat on port running full fledged in a shell. So we can do whatever we want. We can run bin busybox netcat and we want the netcat to do a reverse shell to our attack machine port 1990 and run bin sh now if everything works okay i hope it does then when i press enter here we should be seeing a connection on the top left hand side of the screen on our attack machine let's try that okay yes we this is great because we just helped number 6 escape from the village and return safely back to his home. But let's see what we have here. Let's first take a look at the file system root. So

these are all, we've seen them already, and you remember the dash at the beginning. We can also look at our ID. And if you recall, we defined our user as UID0 and JID0. But we can also see who are we inside the host. So here you can see that we are logged in as user number six and we are connected through TTYS0, the serial TTY that we meant to connect through. So this all seems very good for us as attackers. But let me show you another thing. You see, we can also, if we have access to the host, As we said earlier, we have complete control over the Docker daemon. So we can, for example, run Docker PS and see all the

Docker containers running on the host. And as you can see, there are multiple containers. We are only one of those. There are multiple containers running on the host. So those may be... And now, as I promised earlier, we need to discuss what can we do when mount is not allowed because mount is a very privileged command and it is probably the first command that defense teams would block if they get the chance. So we had to find other solutions if we can't mount it. So here we show a couple of other methods or paths that we have and the first one is using DD. DD stands for data definition. And it allows a process running in

Linux, of course, to read and write blocks of information directly from a Linux device. So if, for example, we know the pattern that we are looking for, so if we want to change something in the etc password file, then we know we're probably looking for, we can look for a pattern such as root, colon, x, colon, zero, colon, zero, etc. Then we can read blocks from the device until we find that pattern. And then we change that block and rewrite it back into the device. And by doing that, we gained read/write access into the device. Another interesting solution, and this is my favorite, is called DebugFS. Very shortly, DebugFS is a Linux utility which is an

interactive file system debugger for ext3, 2 and 4 file systems. And it also allows to read and write state and information directly from the device and also change it. Now we had a lovely demo here but unfortunately our time is up and we have to finish. You're good. You can go a little longer. Yeah, I've timed it, but I'm definitely trying to respect people's time here. So you can do what you want to do. Okay? You'll have a few minutes because we're going to start in a minute or so. Awesome. Keep in mind, this is being live streamed. We do it before and after. So anything we're talking about here is live streamed. Good job. Thank you so much.

I'll keep all my incriminating remarks. Time at the end, if they want to ask. I've allowed a bit of time. Don't say anything, yeah. Yeah. Alright, let's go ahead and close the door. Thank you. Alright. Good afternoon, everyone. This is the B-Sides I'm the Cavalry track. If you are still talking, please exit the room. Unless... You are this gentleman here, Richard Manning. This is the certification and labeling of IoT. I'd like to go ahead and do a few quick announcements here. One, our sponsors. We would like to thank our sponsors, especially our Inner Circle sponsors. That's Critical Stack and Valley Mail. And a couple of our stellar sponsors, Secure Code Warrior, Paranoids, and the NSA. Thank you all very much. Without their sponsorship

and so many others, including the help of the volunteers, this would not be possible. So thank you. Another reminder, cell phones, please turn off your cell phones during this talk. It is streamed, so we do need you to go ahead and mute those right now. Thank you. Without further ado, Richard. Thank you. So, hi, I'm Richard Manning from the NCSC in the UK. So, you guys are here to see Excuse Me, Your Sword is in My Eye, Responding to Red Teams and IRL Threats in 2019 and Beyond. So you're here with myself, Jeremy Elway. I've been doing the security thing for a little while. I get to work for a pretty cool company doing threats,

incident response. And we're going to wrap up talking about how defenders can prepare. So does anybody recognize this book? Feel free, pass this around, flip through it. Go ahead, grab it, go for it. Flip through, that's a volume one copy of Hacking Exposed. It was published in 1999. There is no digital version of that book. I used to see it all the time walking around bookstores. Do you guys remember bookstores? It was published in 1999 and it really captures the hacking zeitgeist of the era. And right on the cover is a quote from Aleph1. Amongst other things, he was known for smashing the stack for fun and profit. Basically, it was the first step-by-step guide

to exploiting stack-based buffer overflows. So this was published in 1996 in Frac, and I know, 1996 sounds like eons ago. But it was actually really contemporary to this book. And around that time, early 90s, sorry, late 90s, early 2000s, there were not a ton of resources out there to learn hacking and security online. It mostly looked like this, like shady e-zines, super shady forums, like really sketchy looking places. There weren't, you know, threat reports to read or like bug bounty write-ups to read. It was basically this. and these were sort of the elite underground hacking groups at that time. Late 90s, early 2000s, I feel like really represented the end of an era. But at

that time, inside books like Hacking Exposed, this was the typical view of a corporate network. You got a firewall, a perimeter, a bastion host, the internet, and you'd always see these phases of hacking listed out. Recon, scanning, gaining access. And depending on who published it, there would be some steps added or removed, maybe vulnerability ability identification or privask, but you typically see phases like footprinting, enumeration, password cracking, buffer overflows, clearing logs and stuff like that. So 20 years later, we're literally still using the same language to describe hacking and intrusions. So again, some of the steps are added, some of the steps are removed, but you can see that they all sort of follow this

similar formula, this similar model. So fast forward up to 2011 and Lockheed Martin introduced the cyber. - So last year we published the UK Code of Practice on IT security which I'll come onto in the next slide. Now that basically had 13 points. and three that would make up what we call the baseline. These are effectively no default passwords, a vulnerability reporting methodology, and a defined end of life for a given product. Now that was developed with extensive input from the wider community. Josh and Beau were involved with that earlier on as well. And hopefully there's a general theme that matches lots of other standards around the UK will become apparent. Now, David Rogers presented

on this here last year and has been doing the same thing around the world. My colleague Peter presented on this at RSA, an event at RSA earlier this year. So we're trying to continue that engagement with the wider community to try and make sure that what we're doing isn't completely out of sorts with the rest of the world. And we know from the news stories that IoT security still hasn't been fixed, so we're still doing stuff there. It's always in the news, so... We're going to continue onwards. So I just mentioned Last year, on the left-hand side there, we produced the Code of Practice. It was delivered by a department called DCMS, Department for Digital

Culture, Media and Sport, also known as the Ministry of Fun in the UK. That's available online. Please feel free to download it. It's been translated into numerous languages as well, so you can read it in any language you really feel like it. ...over the VPN for those users. That in turn means you are opening up your network to all sorts of users' home machines, their infected Androids... and you're getting logins from unknown places, unknown devices all the time. In the last 20 years, let's take a look and see how red teams have been taking advantage of these changes. So I call these the not so new phases of hacking. The first thing an attacker might

do is phish a user's credentials, phish their second factor, and attempt to add their own two factor device. They'll then gain some form of persistence that's not gonna get wiped out with a password reset. and they abuse the trust and the access they have to gain more credentials and expand access. Now in the case of a red team, they're gonna keep doing this, more creds, more access, until they raise the noise level until they're detected. So let's zoom in on the first step, phishing. Anything that can potentially send a notification to a user is potentially a phishing vector. So there's a lot more out there other than just spoofed mail. Things like public calendars, public

Slack boards or Trello boards, public code reposts. Basically, anything that lets you at mention a user. LaCroix, you know me. You know me. Thank you, sir. And your badge actually will get you into the buffet for... Something a little more, a little broader selection of vegetarian food. Yeah, I hope you have some carrots. Thanks for coming. Thank you. Cheers. So anything like public calendars, anything that lets you mention a user and send them a hyperlink is potentially a valid phishing factor. So these services tend to let you set your own display names. So when you set your display name, you can use it to fit your social engineering pretext. Maybe that's expressing urgency or using

an emotional appeal. But the really dangerous part of this technique is that it bypasses tons of email gateway security tools. So it's super dangerous. So we know 2FA phishing has been happening in the wild for quite some time. I encourage everyone to check out this report from Amnesty International. It was co-authored by Nex, who is a really awesome Italian hacker, first of several that I'm going to shout out in this section here. So we know there's tools out there for pen testers as well. So Modlishka's reverse proxy that claims universal 2FA bypass. It can easily intercept OTP tokens and pens to steal users' credentials. There's Evil Engine X2, it's a real heavy hitter in this

space. It's another reverse proxy, but the focus, and you can see from this diagram, there's a lot more going on than just grabbing browser form data. These near transparent proxies are sending real data to and from the SAML endpoints. So this tool, Marina, was introduced at Hack in the Box Amsterdam this summer, and it was developed by two really awesome Italian hackers. The first guy's name is Ope. He's the lead author of BetterCap, which is the man in the middle framework to be using right now. The other co-author is a guy named Antisnatcher. He's the co-author of the web application Hacker's Handbook, and he's also a core developer of Beef for 10 years. By a

show of hands, raise your hand if you've ever popped a shell with Beef. Anybody? I love that tool. So when these two guys got together, they made a brutally effective phishing tool. I encourage everyone, whether you're attacking or defending networks with 2FA, check out this talk. It is super, super scary. So, they released a companion app with Marina called NecroBrowser, and it sort of handles a lot of things that you'd expect a headless browser like Selenium to do. The focus here is on automated post exploitation of hijacked accounts, things like automatically back-dooring accounts with SSH keys, automated password resets. So, I cannot talk about 2FA without mentioning this blog post from Trail of Bits. I

think it is an absolute must read. There's so many standards changing in the 2FA space. We have FIDO, U2F, WebAuthn, and there's a lot of subtleties in between them. So give this a read for sure. I probably don't need to tell this crowd this, but if you're doing 2FA over SMS, you should stop doing that as soon as possible. I think a lot of us saw the story this summer where this guy lost over $100,000 from his Coinbase account due to a SIM porting attack. So Fortnite did a really awesome campaign this year where they gave users a free emote for turning on 2FA. And I think that's brilliant. If you're a defender and you

can incentivize 2FA adoption, you should consider it. So another common tactic of red teams is hacking without exploits. This is something HD and Val Smith talked about more than a decade ago. Look, I love root shells, but ultimately hacking is a means to them. This is effectively where things could lead to. This is not defined. This isn't necessarily going to happen. But what it shows is that you've got lots of different government agencies in Europe, BSI in Germany and DIN, and different industry groups, different manufacturers, all working together in different groups to try and get a grip of IoT security and how it could affect them. Now, I was at a meeting a few weeks

ago, we were trying to merge certain standards together, and whilst, yeah, there was lots of discussion about actual specific wording or implementations, the general feeling was, yeah, let's all try and do the same things, try and make it easier for manufacturers and consumers to sort of value around one standard if possible. Otherwise, it gets too confusing, and I'll show you some confusion later on about why it needs to be fixed. But the underlying feeling is, yeah, consensus is there, it's just a matter of trying to sort of get it all into one place. So, as I just mentioned, We've done some mapping of the various standards now. It's against the UK Code of Practice, but there's lots of stuff in there which is non-UK specific. And it's

a really good example. This paper, linked to the bottom there, is a really good example of how the different IT standards exist and how they interact. David did this for his company. He's done a blog to talk about it. And it was updated this week. It incorporates various changes to the policies. It makes detection a lot harder. So we've talked about hacking, hacking without exploits, hacking without implants, but I want to talk about hacking without compromising endpoints at all. Is that even possible? So if you're an attacker, imagine your target is running a reasonably secure operating system, something like KubeOS, where every application is individually sandboxed, or more likely they're running something like a Chromebook,

which if you have exploits for Chromebook, you can easily parlay those into cache. So in this scenario, a user's running a reasonably secure operating system. We're confident we can get something done next year, depending on who's in power. We're also going to continue to work with the Europeans to, again, make that a European standard as well. However, as we get closer to certification legislation, we've still got some questions we need to answer. based on the left-hand side there, we need to understand the manufacturers are delivering what they say they're delivering. They're not just playing lip service to these sort of requirements. We need to understand how to gain confidence in such certification, how to build

that into the process so that we know what we're doing. And importantly, we want to make sure that the floor doesn't become a ceiling. This is a phrase that Peter refers to. So we know the three baseline requirements. They're quite low level. They're quite basic. We knew we couldn't go in there with a full 13 requirements of secure comms and some of that. manufacturers just wouldn't work straight away with that. But we want to keep pushing that bar upwards and try to make it better for the manufacturers to meet those needs. And to do that, we need to push these two things on the axes here. So we need to make vendors, consumers, different stakeholders,

we need to educate them more about the cybersecurity of their devices, different ways to do that. And down the bottom there, we need to make them care more about cybersecurity. And to do that, there are various methods. We can do carrot or stick, really, and different types of approaches. They can enable mail forwarding. Malicious mail filters can be deadly. They can catch those password resets. They can also catch alerts and warnings from IT and security. So when you combine all of these techniques together, you make detecting compromises extremely difficult for incident responders. So at this point, I absolutely have to clarify something. I think exploits and implants are awesome. Attackers use them because they work.

Infrastructure hacking isn't going away. But if you're a government-based body, independent labs, there's lots of ways to solve that problem. Essentially, each have their pros and cons. So yeah, that's one of the key points you need to understand how best to do that. The difference stands out there for gaining confidence in products, but are they the right thing for this kind of sector? This might be jumping into this afternoon's discussion, but how do you anticipate sort of working with Anissa as they are looking to sort of implement this new cybersecurity act? And my understanding is consumer IoT, is going to be one of the early areas that they're looking for for certifications there. So how does the NCSC plan to-- So we're working with them

already, very much so. And we know there's different time scales in play. We also, you might have heard of Brexit. We're not quite sure how that's going to impact all this. So we're working with both camps, really, to try and make sure. A lot of the common technology, the common requirements are going to be the same. A lot of the stuff in the ANISA stuff is based on what we've done before. We're feeding that up. So in theory, ANISA comes in and that's the bigger thing. We can just point to that and it will still meet our requirements. We might stop doing our requirements. I don't know. Lots of options there. But we are very

much tied into that piece of work and support. You have to be ready to switch roles. As an incident responder, you've got to be ready to ride into battle. And I feel like doing IR is a bit like being a firefighter. Not that we're doing anything heroic or dangerous, we're not. But firefighters focus a lot of time on prevention and education, but they all know that eventually, sooner or later, that alarm bell is going to ring. And when it does, you must be ready. If you're an analyst, you might be seeing dozens or maybe hundreds of alerts of a day. You have to take every alert seriously. Pulling the smallest thread can lead to something

much bigger. All the millions of dollars of security tooling, all of it comes down to this, responding to a real incident. And look, I think I'm good at catching hackers. If you think you're good at catching hackers, an incident is the time to prove it. It's a lot more than a cat and mouse game, but I think it's okay for defenders to have a little fun hunting hackers. We know they're having fun as well. It's okay for us to have fun too. And it's really important whoever's leading the incident, whoever's drafting the communications, whoever's making the high level decisions, it's important that they focus on doing that and delegating work. Ideally, whoever's leading the incident

is not doing technical work. So let's shift again. I want to spend a minute to talk about truth with an uppercase T. So this is a really weird image. If you look, you can see that there's a ship on the water, but it almost looks like there's another ship upside down floating on top of it. This is what's called a Fata Morgana. It's a type of superior mirage, and it has been confusing sailors for centuries. The exact same thing can happen when you're looking at heaps of log data. So when an incident cracks off and you're looking through all these mountains of logs, you need to be really open minded about what it is that

you're even looking for. Try to drop your assumptions. Try to drop your biases. The nature of an attack is going to be really, really, really complex. Your logs are really simple. They're just black and white. Try not to lose sight of the fact that your logs only tell you part of the story. They tell you the part that you have data for. And you know, when an incident cracks off, you know, it's everyone has their own pet theories like, oh, I think it's this, I think it's that. But you need to embrace skepticism and change. The more you investigate, your theories are going to evolve. And I know when an incident starts off, like I'm

like full of adrenaline and it's really exhilarating and exciting, but you have to pace yourself because it's likely about to turn into a marathon. So logs are the lifeblood of both analysts and incident responders. It's crucially important. You have the logs you need when things go bad. And there are countless ways logging pipelines can break down. Try to ensure you're...

Working on an incident, different people are going to know different things. So I cannot talk about standardization without talking about the MITRE ATT&CK framework. This has been one of the best resources in recent years to standardize ATT&CK terminology. Now, it's more of a knowledge base than it is a descriptive set of phases or steps, but I find it incredibly helpful. So let's talk briefly about containment. If you look through Hacking Exposed and old books from the 2000s, they describe containment as the second step of the IR process. It's really easy to get lost in the technical bits, but doing security is about working with humans. It's about working with other teams. So this is something

Werner Herzog says to all of his film students: read, read, read, read, read, read, read. This applies equally to analysts as well. Read threat reports, read bug bounty write-ups, read the New York Review of Books. It doesn't matter. Read as much as possible. It will only help you as an analyst. And look, you need to understand how your company makes money and how it works if you want to effectively protect it. I have a mic if anybody has any questions.

Hey, great talk, man. Thank you. Yeah, so I don't focus on security. I'm more of a software engineer. Of course, I should be. But what's the recommendation that you would give to software engineers, you know, security-wise? First thing I would say is, yeah, read the bug bounty write-ups. Like, whatever your software stack is that you're developing for, there's surely...

I never leave anything off the table. I'm always open to like hear people out and like sometimes there's some really cool stuff. And I will say I actually have done some freelancing. Like I live in Austin so like the startup scene there is crazy. I don't know if you know about Capital Factory but like... I like that. I like that. That's a good pitch. That's a good pitch. But I have definitely done freelance work for my friend's startups. Like they're like oh I have this photo app or like oh I have this new recipe app. And then like... I kind of step in before they do an MVP or before they...

Of course, we gotta drink our own champagne. That's right. So, is there, do you have a dashboard or anything for your business? So, for our vulnerability management specifically, so we are all of Nexpo's and I gotta say, it's tough out there for vulnerability management. Like, before, at last I worked for HP and at HP we had the biggest implementation of Nexpo's ever. It's tough. It's really hard. Like, servers go up and down in an hour, and like to scan, you know, whatever we have, like 10,000 servers, like it takes more than 24 hours. So by the time the report's generated, the thing's scored, and then it gets emailed to the service owner, It could be, hey, that AWS server's been offline

for 12 hours already. We don't have a big dashboard, but we have been working with Nexpos to get tighter integrations with Jira. For us, the actual struggle is mapping, like, we know what our IP ranges are, but it's mapping, like, some random AWS EC2 server to an actual owner that's going to respond to it and see, you know, hey, this has, you know, RCE, and it's exposed to the internet. Yes, it's only staging, but no. It takes this right now, like, put an SLA on it. But there's nothing really super special. I don't know, I don't know if my coworkers are giving talks about it, but we have spent a lot of time trying to

get Nextpost to work really nicely with our systems. We're a big smoke shop, so we use smoke a lot, but it's mostly Jira and Nextpost. We have them married together. Jira, no one is like, Jira's my favorite tool ever. But I will say, it is super flexible. And so we've tried to just do as much with the APIs as possible. If you have less than 10,000 servers, it's probably a lot easier. And if you can solve the problem of how do I map this random EC2 server to an actual human being that's going to respond to a Jira ticket, that's the hard part, I feel like. And then just routing the data around is kind

of easier. But yeah, if you could get all the responsible parties to own up to it, then you've really solved the hard problem of bold scanning. It's tough. One of my favorite phrases, and I wanted to include it in this talk, there just wasn't really a spot for it, is bold scanning is not the best tool for identifying. You got random crap on your network? Dude, I use this tool and I... From rapid7 you mean? Yeah, this isn't a rapid7 thing. I don't know what umbrella Rumble is under. I think it's just called Rumble Networks or whatever. But it's like a single compiled binary. You can run it on Windows, Mac, Linux. But if you want to find everything on your network, like, dude, I love Nmap. I

actually have one commit on the Nmap project. I'm so proud of it. It is the lamest commit ever. But I...

I'm not kidding you. I'm sorry, I said Fortran. I meant Cobra. Based on Kobol, it's probably worse. All those systems, AMEX, all of our infrastructure runs really well. Yeah, I know, it's pretty wild. So if you're not in that boat, you're probably better off. Hey, those IoT devices. I went back to you and said, there's got to be some sort of standard. When you guys installed this and set it up online, best practice, then... Yeah. Check one two three four five six seven eight. I'm not getting anything check test testing Test. Check, check. One, two. Test. One, two. Yeah, why am I not... I see myself. Check, one, two. And I sound like Poo Poo.

I'm just... Ooh, what the front door just happened. Check, one, two, three, four, five six seven eight nine ten. 11:12 check 1 2 3 4 5 6 7 8 9 10 11 12 ladies and gentlemen please welcome the event Bert Chrysler check 1 2 3 4 I think that's okay yeah well you're good on that side but I mean yeah attacks that do not use exploits and do not use implants. So that's the most fun part of hacking though. I want to use my root exploit and drop my root kit and do all that fun shit, but attackers don't need to do that. A lot of times they get that, they fish the 2FA credentials, they have access to your VPN, just get access on the

data.

It's probably, you know, and the focus is kind of more on like slightly bigger companies, maybe someone with slightly more maturity. Like I have a friend that works at Uber and I'm pretty sure he told me there's more than 2000 microservices there. So like people are really EC2 servers, all that need patching. all that have outdated libraries, all of them that can get popped with root kits and Web exploits. It's like when you take all that tax surface away, like, yeah, you can still hack microservices and maybe still get access to the data at the end of the day. But like you're not leaving RCE hanging out there. That's for sure. Like maybe you expose one or two API endpoints. It's not like you

have like, whoa, I have I've seen you're on Shodan now. Like, let's go on a Shodan Safari and find all your servers. So at least from that point of view. Hello.

Oh really? Oh my god, who do I think I am? You get the fancy one I have to hold the handheld on. I don't mind a handheld one. I don't, whatever's easier. No, this is the one that's especially for the, I don't even know if that mic works actually, but just use this one. Okay, cool. This one we know works. Great. So I snap this on. Yeah, you just put it in your pocket. Where's the on-off switch? The on-off switch is up on the top. And...

I don't think so, because I think we have a 14:30 talk. Yeah. So maybe budget in like five minutes for questions. Yeah, I can do that. We can probably stretch it for like two or three if we run really close. Sure. Yeah, that sounds good. Oh yeah, and then we have this. Oh my god, maybe I don't have a hanky in here. So off-brand for me. That should have been my unreasonable speaker request. Honestly I stamp a handkerchief. It's true. It would be very practical. Oh my god breaking news. I have one.

It should switch in a second. Cool. It just takes so long to switch, man. Nice.

Test, test. Okay, this works. Good morning, welcome to B-Sides Las Vegas Common Ground. This talk is Escape the Questionnaire: Quagmire by Katie Ledoux, who works for Rapid7. A few announcements before we begin. We'd like to thank our sponsors, especially our Inner Circle sponsors, Critical Stack and Valley Mail. As a courtesy to our speakers and audience, we ask that you check to make sure your cell phones are set to silent. And if you have a question, we will have you use the audience microphone, which I am holding, so YouTube can hear you. Please raise your hand and I'll bring it over. For this talk, we'll do... Do you want to... Oh, there you are. Do you want

to do questions at the end or questions in line? - Oh, questions during it are great too. - Okay, so questions during it are fine. If you have one, raise your hand and I'll bring the mic over. And with that, let's get started. Welcome, Katie. - Is it happening? Can you hear me? Yeah? Good? Yeah. Okay. Let me know because I can yell a lot too. So I am Katie Ledoux. I am the Senior Manager of Trust and Security Governance at Rapid7. And we are all here today because of a fun thing called information security questionnaires. We know them. We love them. And for anyone who is not familiar with what I'm talking about at all, third-party security risk management is requires collecting a ton of

information from vendors. So if, for example, you want to buy a Rapid7 solution, you very reasonably might want to know how we might be protecting any of your data that we are storing. But how do you or your risk management team collect enough information to make that call to really know, can I give this vendor the green light or are they, you know, are we taking on too much risk by working with them? Today, the way a lot of these companies go about collecting that information is they send a giant infosec questionnaire to the security team at the vendor company. And like everything we do at Rapid7, we want to address these questionnaires in the way that we think is best for our customers.

And intuitively, we might think doing what's best for our customers. But, you know, I could take these games that I wrote in basic to school and sell them for $5 and they have to pay me to get the little code and all that kind of stuff. So, you know, it was really cool to get into that and have an opportunity at that age to start actually writing software and even trying to sell it. But I want to talk a little bit about that and how that led me into security. Just some sort of screenshots. It's difficult to get screenshots off of a Tandy 1000 monitor. I started getting a bit paranoid about security with respect to software when one of the kids in school said,

"Hey, my dad cracked your game and found your code. I want my money back." How did you do that? I kind of knew, but fell into it because of that. I started building my idea of copy protection and warnings into software and you know warnings like this you will lose all files at your own cost and Then when we actually look at some of the code, I'm doing things like just plain text values obviously this is Tangential. Tangent. Analysts might think that means what are the password complexity requirements for a user who's logging into our solutions and there's just it's too subjective there's too much opportunity for misinformation to get in there so again I it's another way that I don't think this approach is really what's doing best

for our customers we don't want to be giving them misleading or confusing information. So with all those things in mind, we thought that there might be a better way to handle these inquiries. Something that was both better for us and also, most importantly, better for our on not necessarily hacking a specific system, finding an exploit in a specific process or system, but rather holistically looking at things kind of in line with the ideas of threat modeling. And we're going to be getting into that in just a couple of minutes. And of course, the hospital security part that I mentioned is really where we're going to be focusing on as far as some of the latest

projects that I've worked on and some of the interesting findings that stemmed from that. So I just wanted to throw a few key points out, primarily just now that we're getting notices of breaches and compromise pretty much every week now. The media has shaped things in a very interesting way that lead a lot of people to think that hackers are magical in some way and that maybe they sit down at a particular system and within 30 seconds, like on Swordfish, can crack into it. That doesn't really happen either. We also have the idea that a lot of tools that we utilize today, threat modeling, other processes for planning security strategy, those work for attackers too, in analyzing systems and finding weaknesses. Far-fetched is no

longer relevant or acceptable as something that I've kind of... For our product environment, which is a helpful breakdown for customers. Standardized questionnaires, if you've had to fill one of these out for a customer and you actually feel really good about the content in there, like it's been reviewed adequately and our data retention schedule, because it does make sense. Your customers want to know how long are you keeping our data? How are you destroying it when you don't need it anymore? How could I get an exception to that process if I wanted to speed it up, for example? So that's what we include. And then lastly, Obviously, third party attestations yield. Pentest letters of attestation, SOC 2 report. It's one thing for us to say, "Oh, we've got so

many great security controls in our environment. Everything looks great. Just checked it out this morning." It's another thing to have a third party come in and test that those controls are in fact working as designed. Executing on our plan. It is it's Q3 2017 at this point when we make the decision we're going to stop just filling these out and we're going to push back with all of those documents and in our first quarter of doing that 50% of these assessments are closed out with docs alone. And I am simultaneously incredibly relieved about how much work we are saving ourselves and so angry at myself for not doing this sooner because that is so much work. So even just with this first group that we're sending the documentation,

they have no follow-up questions. It's totally done. We've cut our work in half. Second group of people, and this is my favorite group, they accept the documentation, they look through it, and then they have a few targeted follow-up questions, like really thoughtful questions about how it could impact their unique environment. And actually, this group is great too because they ask questions about things we didn't include in the docs, and then we can add them and make the docs better. Obviously love this group because they're not doing like a check-the-box activity. They're like actually assessing the risk. So, yeah, but of course, five questions, five follow-up questions different than an 800-question questionnaire. So much better. So much less time. And about 15% of our

customers still required we complete their security questionnaire from scratch. and a lot of times that was because their vendor management solution. And what's interesting about it is it's meant for monitoring system resource usage by applications. So for every application that does any type of network connectivity, you'll have that logged, how much data that particular application sent and received across the network. If applications use significant resource like CPU resources, that will get logged. All of the push notifications that pop up at the bottom get logged as well as energy usage. So there is this tool on GitHub called srumdump. I have the reference at the end of the slides. You can pull that database from the machine, run it through this Python tool, and it's going to produce an

actual Excel file. So this is not like a comma-separated file. It's one of those XLSX files that you need Office 2007 or greater to view. But it breaks all of the data out for you. So these are the tabs at the bottom, the network usage, application usage like we saw on the last slide, and then this is an example of network tracking. So you get the full path of every application that made a network connection. This is the SID it was running as at the time, and then the number of bytes sent and received by that particular application. And it's very similar, I didn't want five slides of Excel sheets, but it's the same thing

like for this, it'll tie the data back, network connections and so on. Again, really telling you everything that happened on the machine, and it's always nice when we can tie it back to a particular user. There's also the AM cache. This started with Windows 8 and is still there through Windows 10. How many people have used this in investigations before? Some people look very happy that there's a slide. Questionnaire comes in through sales, goes to content strategy. They push back with the documents. 50% of the time, we never deal with it again. Love it. The other 50% of the time, if they have a few follow-up questions or needs to do the whole questionnaire, the content stride, O-M-B-U-D, but there's tons of

these knowledge bases out there. And I mean... There might be one that is in use at your company today that you just don't do anything with now. So it might not even involve bringing in a new solution. It might just involve you putting your information into a solution that's already being used by a team that already exists. So, I mean, say for the first two years I worked at Rapid7, I did not know that it a content strategy team existed. And this has turned into one of the most effective partnerships I have experienced in my four years there. So this is the same chart that we looked at before, but now instead of just 2017,

we're going out into 2019. This is that same line we looked at when we started pushing back with docs. And over here, I have a mic, I can walk around, right? Okay, so over here, We kind of started experimenting with that new workflow over here, but we didn't want to force a cutover in Q4 because Q4 is crazy. So this is when we started forcing sales to use the new workflow. And as you can see, we have gone from taking nine days to close out these assessments to getting them closed out in less than one day. So that is... delightful for everyone involved. Also, just quick pro tip, if you do track this at your

company, I would strongly recommend using the median because initially I was using the average and it was, I did not see a trend and then I used the median and I was like, ah, yes, there it is. Because you always have that one random ticket that's open for three quarters that screws everything up. - Tips for doing this effectively. Something that I could have done better. There was a lot of friction initially with sales because of that. problem we talked about before where it's very counterintuitive to do something that is different than the customer is asking. It does it certain ways. If we had that knowledge collectively, not in people's heads, but somewhere that it could actually be used, and we're achieving this, certainly.

Things are getting better. But if we don't have that overall understanding, we really can't possibly think that we could effectively secure that system if we don't really know ourselves how it works. And It sounds very simple here, but when you sort of extrapolate that to a multinational project where you have teams upon teams upon teams working on this stuff, we see a lot of that happening. The collective knowledge is strewn throughout different organizations, different document systems, learning systems, and all that stuff. And again, it gets lost. Not that I'm trying to preach to you guys, but just something that I've found along the way plays a very big role in understanding how to secure a system. Then decomposing the application or the system, classic problem solving where we're

just decomposing the problem as much as we can, identifying components and the blocks of the system, how they work together. Threats, I don't want to really go do a big talk on threats and all the different ways that we interpret what a threat is or threat agents and this stuff. But it's definitely a part of the system. It's probably one of the more difficult. That's better for everyone. You can do security work. They can get their whole vendor assessment done without having to go through sales and then security. And then that's wonderful. So we have trust.rapid7.com. Everything that is... These assets of course have different values to Batman. Maybe not all the same, but we also know that Batman has threats. the joker etc

so the part that i haven't really spoken to yet and this is really fundamental to taking the output of a threat modeling exercise and applying it to your strategy applying it to budgeting for for security spend but the protection the the controls and and other security mechanisms that make sense for this system with respect to the assets and the threats that are targeting them so Very simple, but I think quite meaningful in terms of how all of these things fit together and how we could apply it to something a lot larger and of greater consequence. And before we do that, I wanted to talk about just some things that I've kind of taught as I

work with organizations and teach classes, etc. I've always found that classic video games taught me a lot of really interesting security lessons. Probably didn't realize it back then, but looking back, I think it's pretty cool. So I wanted to show you a few of those. But if you guys remember Shinobi, Mega Man, these are awesome games. So let's talk about Shinobi. If you're not familiar with Shinobi, cutoff at which, a deal size cutoff, at which you would say, hey, if you're not spending a certain amount of money with us, we are not going to fill out your questionnaire. This is not something we have in place right now. I know it's very controversial because the amount of risk you are introducing to a company has nothing to

do with how much money they're spending on your solutions. My thought on that is, I think that it is okay. Does that make sense?

So this is the other view of credential isolation. Microsoft usually refers to it as credential guard. So again, the idea is you have this separate guest virtual machine running. That separate virtual machine has the actual credentials. And when you do something like log into the system, that password you type in gets checked in the virtual machine, the second one. The real credentials are never exposed, or at least a copy of the credentials are not exposed, or you can access it directly. So things like live credential harvesting, where Mimikatz will inject code into lsas.exe. It can still try to do that if it wants, but the credentials aren't going to be there. Even if Mimikatz gets to inject its kernel driver and it scans the memory of lsas from

kernel mode, it's still not there. So that way of directly harvesting cached credentials really doesn't work anymore. And that was a huge pain from an IR perspective, because all of those cached credentials otherwise were readily available. I saw some blog posts and stuff online. People were kind of making fun of them on Twitter too, where they said like, "Oh, key logging's dead, and this credential harvesting is dead on Windows now." That is not true. Don't believe that hype. The cache credentials are hidden in a place where the tool won't be able to access them, but you can certainly still steal live credentials as people log in. Any user land key logger is still going to

work. You can still call set Windows hook ex and get your DLL loaded into memory. And Mimikatz also has a cool module called MIM SSP. So what this is going to do is register as basically a package to... I feel like if they're being specific enough to say you didn't, that rarely happens to me. Like if they're being specific enough to say, oh, you didn't answer that question. And I can say, oh, I did. Like if they're actually reading the documentation, the follow-up questions are generally like, oh, wow, we didn't cover that. Like, yeah, I could definitely talk to you about that specific thing. I tend to get either, I want you to fill out

the whole thing from scratch or the follow-up. So if we look at just a generic example, system architecture. This is, I think, maybe the various systems involved with hotel guests checking in, the front desk people doing the cards, all that kind of thing, all the databases, all the web servers. If we just kind of generically use this as our understanding of how the system works at a very rudimentary level, we can start to kind of focus in and build paths around and this is further in the threat modeling concepts, but the idea of trust zones, trust boundaries, areas where we see communication between systems and there's effectively an inherent trust, at least the way the architecture is designed. When we start to understand those

things, that's when we can actually start to, kind of like what we did in Shinobi, see potential weaknesses in the particular system, whether it's interfaces, network interfaces, physical interfaces, just areas that look interesting and most likely to yield something interesting. And of course, once we've done that exercise and maybe we take this concept, we run tests against the system to confirm what we think we found, and we're going to come up with a list of not necessarily vulnerabilities, but... Yesterday at the B-Side CISO event, this issue came up. And what one of the CISOs recommended is that the companies charge a fee for non-standard questionnaires. Was that my CISO? It might have been. No, it wasn't your CISO, but this is a problem that's

pandemic. Yes. That actually, so that's another thing I've heard discussed a lot is will we, say we won't do it under a certain deal size or will we charge to complete them? I would say at this many engineering hours we will start charging for whether that is support beyond the regular support amount or something else. It is a framework that companies use. We don't use it yet. Okay, I think we have time for just one more question, and everybody can feel free to catch up with Katie afterwards. Hey, just curious if you considered, as you have to scale more, any of the maturity models that are kind of in the works for highly regulated industries, like DOD is kind of trying to

push one through this year. Rather than do this hand-to-hand combat, which it seems like you've sliced and diced very well, getting that cert and you can just show your merit badge and You know, it is actually, I am talking to customers all the time about if we do this cert, then what? Like, what do we unlock with this cert? It is interesting to me. You know, we have a SOC 2. I think next on our roadmap would probably be ISO 27001. It is really interesting to me that we're talking to peers now. For how many people that does not seem to reduce, especially that 15% who says, okay, great that you have that certification, but

there's only one way for me to import information into my risk management system, and it's with this questionnaire. But I do think, honestly, a company that I think does this really well is Atlassian. They will not answer your questionnaire. They do not care how much money you have. And they also have like every certification. They're just like, okay, yeah, what control hasn't been tested by a third party? Like I have certs on certs on certs. So I think they can really get away with that. If you can't accept those and you really need like a junior security analyst to write down the answer to believe it, like I think that's a you problem and Atlassian

is still thriving. I would say maturity wise for our internal security team, we aren't like at their level yet. Like we don't have certs on certs on certs. So I'm still going to keep answering these until we have those. But like when our trust page looks like Atlassian's trust page, this It's like predictive answering. So like they're like not even copying and pasting. They are thriving. They're just like auto-populate, gonna go sit in a hammock now. Everyone's living their best life. It's great. I should be sponsored by Ombud. If anyone knows anyone there, get them in touch. All right. Thank you, Katie, very much for your great talk. Oh, I should not leave with this. What? With

what? I still love you. Oh, oh yes, definitely. And actually, the international ones are, they're mad all the time. There's some angry international CISOs. Hi. Yes, yes. Oh, yes, okay. Hi. So they actually kind of screw up our workflow because I don't know if we're doing it wrong or something. I'm going to do it. Okay. Bye. Thank you so much. Awesome, thank you very much. Right now we're only on one side. So just kind of like clip it. thing as anything else does and we let them know about that too and that was something that they address pretty quickly but nevertheless now we have our persistent network access and we were able even able to to

utilize the the windows with the help me thing where you can let somebody remote access and so we got remote access inbound into that patient kiosk and that's really what led us start looking at some of the systems that we were more interested in to accomplish the primary objective without necessarily being stopped. So if you recall back in the primary objectives, we have to disable this particular device in a specific room for a specific patient. That obviously is going to require some trickery. We looked at social engineering aspects, but none of us knew enough about medical stuff to be able to talk to a doctor or a nurse and get them to give up the information. So on the flat network, we found all of

these multifunction printing devices, faxes, et cetera. The same brand, essentially the same model used all throughout the hospital, including patient intake, surgical, etc. So this particular device actually had this really nice document cache. Thank you. Real quick, we'd like to thank our sponsors, especially our inner circle sponsors, Critical Stack and Belly Mail, and our stellar sponsors, Amazon, Microsoft, and Robinhood. It is their support, along with our other sponsors, donors, and volunteers that make this event possible. These talks are being streamed live, and as a courtesy to our speakers and audience, we ask that you check to make sure your cell phones are set to silent. And if you have a question, use the audience microphone, which I am holding. Please raise your hand and I'll

bring it over. And with that, let's get started. Welcome to Luke and Matthew. - Guess it's working for, in some sense. All right, so thank you for coming. Luke and I wanna talk about CVSS scoring, how we can help you with it. Luke and I have worked together for several years doing vulnerability management, vulnerability analysis. As you can see, we have some different interests. And Luke has moved on to a new job, so he has to give a quick disclaimer. I currently work for Microsoft as a security analyst. I'm here not representing Microsoft. These are my own views and opinions and don't necessarily reflect that of Microsoft. Just want to get that out of the

way. So hopefully a lot of you have seen these acronyms. You don't have to go through them in too much detail. CVEs, those are the common vulnerabilities and exposures. MVD has a database of all the CVEs. including CVSS scores and CVSS vectors, and they have an API that anybody can use, so you can access that data. CVSS is what we're going to talk about. They're handled, managed by First, and I want to point out I work for First Information Technology Services, which is FITS, and I am not working for First. Back confusion out of the way. So what we want to help you guys do today is we want to talk about how to do

CVSS 3.1 scoring, also how to categorize your assets, look at your inventory, understand your environment, and do some scalable integration of that data, which turns out doesn't have to actually be that hard and can help you with your security and your compliance folks both. So these are some stakeholders you might recognize and these are folks that CVSS can be useful for. We're going to show you things today that internal security teams can use to drive remediation, can meet compliance requirements with auditors, can help management make informed decisions about what they want to do with the business, where they want to make investments, and can help relate your security posture to clients. There are many other

uses for this, but these are just some of the stakeholders. One thing before we get started, too, we want to make very clear, I think this happens in our industry a lot, that words get used to mean things that are not as specific as they should be used to mean. So, CVSS is specifically designed to score severity of vulnerabilities and not risk. Once you start using CVSS environmental scores, you can start getting closer to risk, but there's a lot of factors that come into play that you should be considering when you're talking about actual risk. Can I get a time check, guys? Okay, cool. So, With all of that said, now that we're kind of

really thinking about what you could actually do in a truly coordinated, planned, funded attack against a healthcare facility, if we kind of go back to just the general idea of IoT devices, I wanted to talk about something that's kind of unique to them, but bore attacks, break once, run everywhere. I'm sure you guys are familiar with these, but those and other pervasive mistakes make IoT hacking IoT really interesting because any of the exploits that we might be able to carry out, say, to find cryptographic keys, other secrets, what we see with a lot of these devices is that those cryptographic keys are the same on every single device. So if we've cracked one device, we've

cracked all of them. Now that's not the case with all IoT devices of course, but quite a few of them. I mean if you think about it from a manufacturing and just an overall production perspective, it can be challenging to automate that sort of thing so that there's an actual per device level of some sort of security. So as we kind of continued researching, if you guys are familiar with the IoT Hacking Village at DEF CON, things like that. Then you have to take that information and see how that affects 3.1 environmental scoring. These are the two pieces we're going to help talk through today. Easy, but I can also really dive in and start

targeting specific organizations. Looking just for Ricoh, a brand of printer in the US, you can see, well, maybe you guys can't see that. Telnet is the top service identified for all of these devices. The top organizations are AT&T and Cogent, Level 3, Comcast Business. And so basically these are just printers that are sitting on a network that somehow is allowing access from the outside world with really, really vulnerable services still running. Like Telnet, FTP is one that we see quite a bit. Sorry about that. So this particular device, it enumerates the ports and then we can see pretty instantly how we can exploit these vulnerabilities and get into the system. And to make it even

easier, Shodan has an exploit database so that you can actually search for specific types of devices, specific vendors, find those exploits, and then go back to your listing and just start hacking them. So what we...

what we have found to be really interesting in doing work. And it was hacked and all your data was encrypted or deleted. That would be a high impact to integrity. If it was one of the services still up, that would not be an impact to availability. Just a thing to keep in mind as we kind of go through the thought experiment. And as you can just see, there's some basic components. All you need is to be able to send traffic to this particular box and you'll be able to, if you have the exploit code, you'll be able to, or like or like the Metasploit module or the POC, you'll be able to do this exploit.

There's low attack complexity, no privileges are required, you don't have to trick the user into anything. So the base characteristics that apply to everybody. - You know, tend to not look at tools like Shodan. Maybe they don't really wanna see what is actually happening. I think that's always valid. But, you know, it's just, it's overwhelming and it brings to mind who's really responsible is, the lighting system manufacturer responsible for notifying their users of all of these different types of risks inherent in using their devices? Is it the consumer or the buyer's responsibility to understand what those risks are? We don't really know. We can put warnings on packs of cigarettes about the risks there, but

when we're actually talking about potentially exposing sensitive information to anyone, there's really not a lot of awareness on the part of vendors, etc. In summary, where I'm going with that is tools that can show us just how pervasive those types of systems are become really useful for attackers, of course, but also for us. If you have some time, definitely check that out. If you work for an IoT device manufacturer, really check it out and see if there's anything interesting that could be maybe exposing your organization. Okay, so we're wrapping up. I'm not gonna read these either, very much like the learning ability will actually alter the overall score. So there's some interplay between those two sets of components. Setting that to

not defined as we have in this example, leaves all those scores exactly as they were in the base score, but the overall score is going down lower because we set requirements to low in this particular case. And just taking the same kind of idea, but we're saying we've got some really incredible mitigations in our environment for confidentiality, integrity, and availability that bring those impacts down to low in this particular scenario, whatever they might be. And you can see the score going back down again, although not as much as if those requirements were set to low. And if you had the same scenario, the score would go down even lower. Or if those were set to

none as well for the modified. Yes. So Just to give you some idea of the components, this is a very, very small list of things that would impact certain components. If you're behind a firewall, the attack vector for this would be as network for this vulnerability. But if you're behind a firewall, say there's no port forwarding of any sort, you have to be on the internal LAN before you can route traffic to this. Then you have the modified attack vector and the environmental score would be going down to adjacent. 3.1 CSF scoring actually makes that really clear and gives some clarifications on how to change that modified attack vector. Data classification, super important. Do you

have low, medium, or high business impact data? Is this the boss's cat videos or is this like credit card information and things like that? So it's going to definitely impact the confidentiality, integrity, and availability requirements for monetary data, transactional data. For doing financial transactions, you obviously want very high availability, so that requirement will be very high. And there's some other things that we just listed up there. It's very, very important to maintain both an accurate inventory and accurate metadata about your inventory. Otherwise, you're going to start doing environmental scoring about an environment that you don't actually operate in, and that's kind of a surefire recipe for disaster. So doing this in a dynamic sort of

fashion is what I would recommend. So as new assets come online, you're constantly harvesting information about them that you can plug into these calculations. So in this particular case, what we've done is what we're saying is that this is going to be medium requirements across the board. In this example, we're taking the same RDP RCE, and what we're doing is we're saying that this is like proprietary non-public game assets. So not like high, but not like publicly available data. It doesn't contain any PII or anything like that. We'll put a medium integrity requirement. And we've decided that the server can be down for up to one to five days. So based on first.org's guidance, that

can be a medium or requirement there. Now, we haven't really changed anything on the course. So I think the answer is yes. I mean, we felt that those systems are likely exposed as much as some of the other systems that we involved in the attack. But we didn't actually look for that specifically. But given it was all right there on the printers, It's reasonable to assume that, yeah. - Did you say how long did it take you to get all that access? - Probably about a year, yeah. And that's not so much because of us taking a year to figure it out, just some of the planning and continuous approvals needed to keep moving with the exercise. - Do you have a clean version of the

report that can be published or is being published? - Yeah, there is actually one that is published. I'll get with you after. In the medical space, one of the reasons that we hear that it's so vulnerable is that systems can't be patched. You have to have FDA approval, on and on and on. And I'm wondering about what you're finding that facilities are doing for mitigation. Are they going to micro VLANs? Are they completely isolating systems so that they're not Internet accessible? Or can you speak to that at all? Sure. Yeah, that's a good question. And that makes sense. You have much more critical data, much higher integrity requirement, and fewer mitigations in place for those relevant components. Cool. So we've done this

for one asset, for one volume. How do we start getting more? How can we do this more for your full environment? So first, you run ROS. Let's talk first about one asset for multiple vulns, right? So you run your scans. You get some data. You get some vulns. They have CVEs. You can go to NVD. You learn more about the CVEs. You get physical access. Just to clarify on the comment of the previous person I was asking, medical devices, there's a myth that medical devices cannot be updated for that. The FDA has changed that. It is now a requirement of both the vendor and the hospital to update those medical devices. I wanted to make sure that was clarified and on the video before I asked my

question. My question is, is that this medical device, do you know if it was recalled or not? Because there are a lot of medical devices that have been recalled but are still in use in hospitals. And you mentioned that the firmware was in 2007, I believe. And usually... 2009, it was Gen 2 Linux, yeah. Okay. And usually it takes about five years for a medical device to go from production. And usually that means that it was frozen for five years, at which point it should be updated. Mm-hmm. So tell me that that device probably came out 2013. and it's been out for about five years or so. So I'm just curious if you'd had a chance to check to see if it had been recalled or

not, or if there were any types of FDA notices on it. - Sure. - Thank you. - Yeah, I'm not aware of a recall. We certainly had some conversations with the vendor. They were actually aware of most of the vulnerabilities that we identified in the exercise already. And yes, so to your point, the-- - Vulnerabilities, you've done the automation to integrate the data sets that you had. And, for example, you can click on an asset and see all the different vulnerabilities on it. You can sort those vulnerabilities by their scores. And those scores are now specific to your environment. So that's telling you prioritization within your environment. It's getting you closer to that knowledge of

the risk of these actual vulnerabilities. You can also select a vulnerability and see all the assets. And you can base what you're selecting based on that score if you want to. And then you can, you know, if a bunch of assets all have the same vulnerability, maybe you can go push a patch, right? So that gets things both more efficient and more... Vulnerabilities are often roll-ups of several CVEs. So CVSS doesn't actually address what to do with multiple CVEs. We tend to take the high water mark and say, "Look, this is the worst one in the set of CVEs." Vulnerability chaining, CVSS does talk about that in their guidance, but it's not built into the

calculator, of course. There are ways to take two vulnerabilities, and we can talk about that later with folks if you're interested. There's also, it'd be good to have a more robust statistical analysis of what, of CVSS and how does it actually correlate to the real world. They've given us these formulas and these numbers, but is that what we actually see when we go out there? So there needs to be more of that. And we talked about the difference between risk and severity. So you need to always be thinking of all the aspects of risk when you're talking about the vulnerabilities that you're finding in your environment, right? So yeah, hopefully that kind of outlines what

hopefully you can start doing for yourself. You can, if you categorize those assets, And if you understand how those categorizations of the assets allow you to do CVS 3.1 scoring, in particular those environmental metrics, right? That's what's key to this. Once you do that, you have a bunch of data that you guys get to automate. And hopefully that allows you to do some scans of your assets and get these compiled reports and make money, right? Yeah, and that's what we had. Those are our contacts. I can give you my contact as well if you're interested. And I'm happy to send out these slides too if people would like them. Yep. Okay, we have a few minutes. We can

take a few questions from the audience if there are any. When you work for a large organization, you need a product basically to do this for you, I guess, because there's a lot of... it's a lot of work to to do all of this manually do you know of any of the vulnerability management products that actually implement this like maybe serve like so yeah i agree i haven't seen a lot of that built by especially by those industry companies right that's where i would look i know there are some smaller companies out there that have uh implemented sort of compliance solutions but they don't necessarily address it with cbss that i've seen we've fits where i work we

have our own automation that we use we don't sell it but it's uh you know as a service and it isn't actually that complicated like i was saying you really just have these data tables you can do it easily in python and uh um with you know sql right that's all you really need and it's it's not super complicated you can mostly just build it up but yeah i haven't seen it out there yes does the aggregate um Quantify for outliers, just your systems that are out that may not necessarily conform to what the main CVSS. The example that you brought up with 0708 and how there are patches available for everything XP and above, but I was thinking more about 2000NT. I was wondering if

CVSS would handle outliers such as those, because we know those systems are still running. Start talking if you want. So I think, I mean, if I remember correctly about that particular RDPRCE, it was Windows Server 2000, the R2, through Windows Server 2003, and as far back as Windows XP. As far as outliers, I'd have to see specifically how that vulnerability impacts that specific Windows operating system to get a sense of whether or not the scoring was going to give us an accurate picture of the severity of the risk. the scoring is designed to be general so that you can you know do it for any if they if you're given a bone with a base score it should be applicable to general cases but i'm sure there are outliers

where the base score isn't necessarily perfect or your environmental scoring doesn't take into account everything because there's yeah but i mean if there's something specific about that that system that you're saying that is not going to have the same impacts of confidentiality integrity or availability should be good give it a shot just a test thank you

I want to go a little bit higher. I put a chair just behind you. Testing, testing. All right. I think it's good. Give us a sentence or two. He said we're doing four panelists sitting. Four chairs. You could put his a little bit further on the end. Right.

Okay, did we wire up the other gentleman? Yep. I mean, yeah, they're using the... No, I'm going to see how your levels are. Okay, check, check. No, you can't. You have to talk like this because you're not going to look at your chest. All right. What's going on? Look at me. Talk normally. Talking normally. Can you guys at the back hear him when he says talking normally? Talking normally. Make it louder. Be a little bit more. Okay. Is that better? Okay, cool. Are we... We're getting started. Talk for me. Can you put... Who wants me to talk for them? Sure. Would that work? Yeah. Because I'm concerned about the table. The pillar's in the way, right? Yeah, the pillar's in the way.

That's why. The whole thing? Okay, so if you move maybe a foot, everybody, did you just adjust the camera? No. Yeah, we'll shift over a little bit. Just right on top of it. Yep. We'll get like that. Almost basically where we're at. okay and we're not under a speaker we're getting close we should still be okay okay give me that under the foot this way just because we can't shoot you we can't get that so do you need him to move it yeah let's shift everybody down Point to you. And close the door. Specifically taking threat intelligence and applying it to improve your defenses, specifically through emulating adversaries and writing detections and analytics based on that

emulation. Before we go forward, we're going to do some quick introductions, hopefully to establish some credibility, why you should trust us. My name is Jamie. I'm an engineer at MITRE. I mostly work on adversary emulation, so the red team side. But I do do some detection work, mostly detecting myself, so a little bit of a purple team flavor if you're familiar with that concept. This is actually my second B-Sides, first time in Vegas, so this is pretty awesome. Previously I presented at B-Sides DC. And I'm Sarah Yoder, also an engineer at MITRE. I tend to focus a little bit more on the cyber threat intelligence side, especially within ATT&CK, and do a little bit of

red teaming as well. This is also my first time not only at B-Sides Las Vegas, but really any B-Sides, so we're really happy to be here and talk with you guys today. We both work on the core ATT&CK team as well as ATT&CK evaluations. So So the answer to your question is yes, we do have attack stickers. So by all means, come up after the talk and we can give you some of those if you want or talk about anything attack. So if you were here last year, a couple of our colleagues, Katie Nichols and John Wunder, presented Attacking the Status Quo. This is a really excellent talk, but the main focus was cyber intelligence,

basically the idea of how do you process, find, and digest intelligence and create analytics based on that. The slides are there on that link. I think it's Katie's slide share. Definitely check that out. But if I need to sell you more on that, it's littered with really awesome memes. Here's one of them. If you think that's funny, the rest of the deck will be equally. To you, the better our stories get, and hopefully the better informed the community gets. So that was sort of the impetus behind the panel. I'm going to introduce everyone real quick and ask them to sort of explain why they're here briefly, why they wanted to participate, and then give an

example of how in their reporting sort of expectations of the non-journalists differ from reality and how they went about their reporting. So from my right, all the way across the stage, we have Lily-Nate Hay-Newman from Wired. Kim Zetter, who's a freelance journalist and who wrote the book on Stuxnet. And Joe Cox, who is with Motherboard and does a lot of reporting on privacy as well. And so Lily, why don't you start us off? Yeah, hey everyone. I'm Lily. I wanted to participate in this panel because I noticed that there's a lot I want to know from the community and from sources all the time. That's my whole job. But there's also a lot that people

seem to want to know from me, which is kind of interesting. been surprising to me that people say "Oh, do you know someone at that company? That would be really helpful" or maybe we could go through the PR team to try to find out more about this thing that I am trying to disclose to the company or I would never be able to do that or "Oh, do you know the researchers at this research team at this university?" Next is the idea of distributed and very feeble randomness. So what is it exactly? What do I mean? So I will just take an example for shared randomness or distributed randomness. So if you take the Tor network, actually if you want to hide the hidden services correctly, you don't want

people to be able to do popularity attacks against them and so on. What's happening in Tor is that they generate a new random value every day and it's reliant on randomness, obviously means more to them. If you want to do sharding, so you take a lot of nodes or master nodes whatsoever and you want to select the one that will maybe mine the next block, for example, what happens in some cryptocurrencies or blockchain technologies is that you shard the network into shards. And you want to do it at random so that you are sure there is no malicious short whatsoever. And you need to do it in a distributed way on the network so that everybody agrees with what's happening. Another example could

be smart contract lotteries, but anyway. What's very variable randomness? So it's a random value that you can check has been randomly selected. So why is it important? I don't know if you know about the NIST dual EC DRGB problem. It was an elliptic curve parameter selection which was biased somehow and random parameters were not that random so it allowed them to have some nice attack. And in that case, if we had been able to verify the random values had been generated really at random, we would have been able to prevent such things. So people sometimes just take the hash of 0, the hash of 1, the hash of 2, and so on, until they find a random value, or at least a value that looks random,

so that it works for their purpose. So it's an example of how you could generate such values, but anyway, random, verifiable random function that IETF draft, so you can check it out, it's cool. And we'll be discussing a bit more about it later. What's happening? Okay. So, as we can see, there is a need-- - About the whole bit of text. So in this one we have a lot of defense evasion as well as a little discovery. So once we have the tactics, we can move on to techniques. Now since I've already done the tactics, again this narrows the field. I can look within that particular column. So in this first example, defense evasion narrows down what I'm looking for because I realize the matrix can be

a little overwhelming. Another thing I do at this step is I look for keywords in the text and try to just use the attack search bar to try to narrow down what I'm looking for. Like if you type in "offuscate" then you're gonna start to see that there's a technique called "offuscated files". I do confirm that by reading what that technique is. But we do this through the whole bit of text. That information all gets put into probably what most of you know as ATT&CK. In this example, we're using CINAC to do that. So now that we have a profile of something, in this case malware that we want to detect, we can start looking at how we build, at least at a

notional level, detections based on that. So we're transitioning from an index of all the behaviors that malware to writing actual detections for that. So getting back-- actually, before we continue, it's probably good, at least in detections, to kind of add another layer of context and describing and kind of looking at a behavior. So one way I kind of look at it, which is really helpful for detection. - I rate randomness, but then you need to trust the beacon you are asking, you are querying, but it's a detail. So, what the main goal of D-RUN was really to make it so that fetching a random value that's verifiable, has been generated in a distributed way by

multiple parties and so on is as easy as fetching time on the NTP server and that's really the goal of the... Shaming maybe and then that won't make people hesitant to name themselves but I mean everyone is in the same boat in terms of being vulnerable and I think it helps. Right. I think there's one person shaming, and another is transparency. Like in naming the victim, how are we going to change behavior and boost security without naming who was hacked? It's a delicate dance, but I think it's one that we're obviously going to be biased towards. more disclosure of that and then coming to the table with researchers who may not feel exactly that way but understand where we're coming from is helpful for everyone. Should we address

the elephant in the room? There was a story, obviously, of the one in Saudi Arabia. And when a journalist published the name of that company, he received a lot of criticism of that and a lot of flack. I would have published the name as well if I had gotten it. And I think in that case it was particularly important to name it because when the hack was first publicized there was a media outlet that published the wrong name. without vetting it. And so I thought that it was important that this researcher, or that this journalist, who when he did discover it, sets the record straight. And by the way, those stories that have the errors stay online. So when someone does a Google search of what happened, they

alight on-- - Step one is to create a transaction. So you create this lock to the file and you override it with your malicious content. The second is to load it. So you load that file into memory. So now you have this malicious content in memory under the name of whatever happened to be on disk. Third is rollback the transaction. So like I said before, you have that file lock. There's actually a part of NTFS transactions is the ability to undo a change. So you can undo that change, which in terms of looking at it on disk, the file is back to being legitimate, so AVA's gonna miss that. But now you have this residual

thing floating in memory, which is actually malicious. And as you can see before on that API talk, Final step is to animate it. So you kind of separate the file on disk from the file on memory and you execute that memory section. So now you have something malicious running in memory which actually doesn't point to anything on disk. Super interesting. So going forward, this is actually, so what Sarah talked about before, there's like tactics and techniques and procedures and ATT&CK. Within each technique, there's also metadata. So we can check out things like data sources, notional ways to detect or mitigate all these techniques. So some jerk, cough cough, wrote all this, curated this from the

internet. This is actually content from ATT&CK on how to detect doppelganger. It's split into three main parts. I try to make it a little easier to read by highlighting the crucial parts. The top section is API monitoring. So looking at transactional NTFS is done through API calls and they're very unique. So if you're API monitoring, you would actually see all these happening. You got four points, it's defining a cubic and so on. And that's a really nice feature of polynomials. The idea behind secret sharing is that you've got a secret you want to share into multiple shares between shareholders. And what you really want is that it requires at least three people members of a group of n

people. So this is kind of understanding is this something that that process is typically has should be doing or typically does so an example would be like explorer.exe if that's like you know doing a lot of file I/O and making a lot of network connections and doing like LDAP queries you're probably going to have a bad day that's not shouldn't be happening and that's probably a really good indicator of some type of injection. So we can bucket that under process monitoring. So this, kind of going forward into an actual emulation, this is kind of like, we would get this mindset of where we need to be looking for artifacts of the actual behavior, as well

as other opportunities that don't fit in this existing model. So this is actually my favorite part, emulating it. So, you know, we've talked about it, we've understood it, but let's actually do something. So funny enough, I actually started for this presentation, I started writing custom code. I think I got like half an hour in before I like obviously had to run to Google and search something. And there was, I forgot the exact string I searched, but there was one hit and it happened to be an already done proof of concept for this. So shout out to Ruben for already doing a doppelganging via PowerShell, which is exactly what I was trying to do. The source

is there, it's under his PowerShell suite project. The actual payload is start a dolon, a dialon, I think it's a Dota term, I'm not really sure what that means, but it's super cool, and in terms of, I'm an attack nerd, so every time I see something, I map it to attack immediately. So it's Windows API calls as well as PowerShell. So I decided not to trust the demo gods. So I pre-recorded a demonstration of this. So before I started, I'll talk about the command line arguments. So at this point, I've already started a PowerShell session and I've loaded the module. Standard journalistic best practices are not enough for the special situations of InfoSec. I'm just curious what you guys think because I tend to try to, in my own

reporting and making my own decisions, just try to go back to very basic tenets of what have you agreed upon with the source? What have they said are the parameters of their willingness to work with you or not? Or as Joe was saying, what is the potential for danger or harm to either a source or sort of a another party that is brought into the story. So I'm curious, I'm not saying there aren't, you know, specific things, but that's what I tend to try to use is just the very sort of most basic foundational ways of doing all journalism. That's kind of what I try to think about. There is There is sort of a different approach that you take to

someone who is a professional spokesperson, right? Who is trained to work with journalists. And there's a different approach that you take to people who actually you know don't have any experience talking to reporters. If I'm talking to a PR person, I'm not going to lay down the ground rules to them. I try to do that when I talk with someone who's not used to working with journalists. Yeah, ensures, okay? Well, what you can do then is just basically sum the points you got from everybody and through another Lagrange interpolation you will get a new polynomial and nobody will know on its own the value of the secret because the secret is always the value of the first coefficient of the polynomial because if

you take f1 of x evaluated at zero the x will cancel out all the others. I consider that part of the ethics of journalism and if someone comes back to me afterwards if I've done a I did an interview recently with a guy who he was surprised when I called him but he was surprised that it was me calling him and that sort of put him off guard And he started talking to me, and then we get off the phone, and he texts me, and he says, can we make that all off the record? And I said no, but I did do a fact check with him so he was comfortable afterwards What I was

using and then I got the facts, right? Yeah, we have questions So I want to we're gonna do questions at the end, but I will make an exception for Josh once I this kind of the moderator so I don't want to like talk too much but I had a just a comment on what Lily said about how do you approach it slightly differently with infosec it I mean simply put If it's a security researcher whose trust you're trying to gain and who maybe doesn't have a lot of experience, be a little bit gentler with the person. Whereas if we're going into a press conference with a DOJ or DHS official, it's a nice fight. Throwing

questions out, I mean there's no holds barred, I mean you're there to grill them. I'm not really necessarily all the time here to grill people who have come to me with a discovery of vulnerability or whatever, you know, something, you know, a zero day or something. So it's a little bit of a different tree. There is just a trick since we're working on a multiplicative group, we need to use exponential notation. Anyway, math. So the pairing also needs to be non-degenerated. It just means it must not map everything to one, because otherwise it's kind of useless, right? And for practical purposes, we want the pairing to be efficiently computed by a computer so that you

can actually use it in practice. So pairings-- I don't usually mean that when they say-- Yeah, they-- Yeah. But in government, when they say on background, it's usually-- It could be administration official. You don't even identify the organization often. Sometimes you do. And this is why you lay out the terms, especially when I brought up the teenage hacker example earlier. I don't really talk on background with teenage hackers, either on the record or off, typically, but using their handle. But these terms are garbage, and they mean pretty much nothing a lot of the time. So I sometimes just go and say... Do you see a background? Right. Well, usually I would just say, this means... I can quote you, but I'm only going to use your hacker handle or

something like that. I'll just get super explicit. Anyway, back to the list. Yes. Go ahead. Please. Please. Okay. So background, you can... This is my use of background. Again, it's going to vary. So that's why you want to set the terms. Background means that you can quote someone the full words of what they said, but you wouldn't name them. and you would figure out how to attribute them. Do you want to be attributed as an FBI official or you know, a government official, whatever. Deep background, can quote them, all of that, but you're going to move the attribution further away. So instead of saying an FBI official, you're going to say someone with knowledge of

the situation. You want to move them out of the agency so that the pool of people that could possibly be the source is much broader and wider. Sometimes it's about protecting the source. 2004, by Bonnell, Lim and Chacham. So BLS was part of Lynn's PhD thesis. and it's using pairing cryptography, it's using pairing-based cryptography to produce signatures. And BLS has actually a couple of really nice features because thanks to the pairing and the bilinearity of the pairing, it enables you to do signature aggregation. It may not be someone inside an exploit company, that may be someone who spoke to someone who then spoke to someone else from a rival exploit company or something like that,

which... muddies the water, doesn't make the story as good, and the reader has no idea about the veracity of the information. As long as it is not infringing on the anonymity or the protection of the source, the reader comes first. And I said that very, very carefully, because obviously you say that and people go, "Oh, you don't care about anonymous sources." No, no. If the anonymity is guaranteed and it's solid, the main obligation is in the reader and the transparency in the story itself and making it clear and fair to everyone. This panel is all on the record, by the way. Wait, wait. I agree, I agree that we are here to service the reader.

And that's primary. The sources as well, because we can't do our job without sources. And so there's a balance there that you explain to sources what you need for the reader to trust the article and to trust what the information is giving you. The off the record thing. So this gets a little touchy because there are journalists that do this and some people consider them unethical. If you tell a journalist that something is off the record, they're not supposed to be able to use that information. But you can use off the record information to then go obtain that information on the record somewhere else. So if a government official tells me something and they say,

look, this is classified, I can't tell you. It's just because, you know, I think people get into all these confusions because they're forgetting, like, what we're talking about here, you know. And, yeah, please tell us everything, you know, don't... we don't want to be on off the record we don't want you to stay away yeah i mean i i think if everyone could just keep in mind the parameters of the project here and uh... sort of collaboration uh... it would help there to be fewer surprises because nobody is trying to no one wants to know soft source right exactly and genuinely because it is a collaborative effort and i think you know journalists field you

know that positive building of something together when you work on a story with someone where everybody feels like a tough issue was well represented or well explained and the readers really benefited. So that's why I just bring that up, even though it's taboo, is because everyone has power and control in these situations and can make the choices that are right for them. So we're doing pretty well on time. There's going to be time for more questions, which are fun. I did want to ask Lily and Kim one thing, and here's my little intro anecdote to that. Preparing for Black Hat and DEF CON, I did not give my email to be distributed to PR professionals

because they know how to find me, but I got a deluge of... can't find really often in practice. So being able to sign together a message without needing everybody to reconstruct the secret first and use the secret we reconstructed is really nice because it means the secret here, the main secret, is never in memory on any machine. It's always hidden behind the computations. So why do we-- --what you're doing. OK. OK, awesome. But yeah, and feel free to shoot us questions or anything like that. We always have to talk to anybody. Yeah, I'll definitely do that. Thank you. Yeah. Very interesting. Do you guys get stickers? I have a question for you. OK, yeah. Thank you. Yes. Yeah, yeah.

So we actually, it's not publicly released right now. We're hoping that fall time it'll be ready to open source. But myself and two partners have used NLP along with some regex on ones that there wasn't enough data for models. but we have an automation tool in the works. - Oh, awesome. - So we have-- - Really, okay. Yeah, let me grab-- So I can give you both, but, so this is both my-- Okay. Yeah, so ATT&CK is a great resource because you can, so we got all the reports and all the information to build the NLP models through ATT&CK. The data is already there. So we have something in the works, but if you want to reach out, we

can chat more about what you're working on, ways maybe we can help. Yeah, of course. Did you grab a sticker or anything? Absolutely. And it will map. OK. But it would help if it was you who actually had the various-- oh, you want to do this? You should start here. And we're good. Yeah, I appreciate it. Thank you. Because it's too-- Thank you. It's too good. It does, and we do fully recognize that there's a learning curve when it comes to doing ATT&CK, whatever that means. Also, on our site, under resources, we do try to give some examples of what you're interested in defense particularly. There's links there that can lead to more in that area.

One thing that ATT&CK Good or bad. It can be used for defense. We have cyber threat intelligence platforms using it. Red team, emulation. I found the process great. On the technical side, I tried the matrix. Right. So I can, from the technical side, I can kind of script stuff and get going. Okay. Okay. Sure. Right. As you want so it's easy to achieve. It's unpredictable and biased thanks to BLS. It's publicly verifiable thanks to BLS and the fact it's chaining its random values, its signatures together. And it's fast because the process of the setup is maybe slow, but afterwards you just need the runtime trip of broadcasting the partial signature. Since we're at Black Hat and besides in

DEF CON at Vegas, of course, and that is... Actually, I can't remember the last time it's happened, but a lot of companies used to host their events at the shadow bar. And I don't know if you're all familiar with the shadow bar, but there are naked women dancing behind screens. And so you're at a bar and you're in a professional setting and you're trying to do your job and it is very sexualized environment. And so that's difficult to do. And I was also at a conference where I was, I really wanted to engage with the people who were there after the conference. And we're all gathering after the conference and they're talking about where to go and they're going to go to a strip club. And so they

didn't invite me. And I didn't really, I didn't know initially what was going on, but they were sort of kind of moving away from me to organize where they were going. And then I found out later that that's what had happened. So that kind of thing, I mean, we're trying to do our job, we're in professional settings, and I suppose a male reporter would have gone with them, or they would have invited him, or But those are the kinds of things that we run up against in trying to do our job. Yeah, and you see how the choices there are really poor, because even if they do invite you to the strip club, then you're

like, at the strip club, this has actually happened to me. So we have members going out, so you need to reassure them. And it also allows you to change the threshold value if you need to and so on. So it's really nice. So if you want to join us, you can. You can just shoot us an email at theleagueofentropy@googlegroups.com. And at some point in the future, we don't know yet when, but we'll send emails to everybody to tell, hey, we are going to do a resharing at some point. So if you've set up your node, you can join and so on. So yeah, shoot us an email if you want to join. So, live demo time, right? Nah, I'm kidding. So DRAND has two interfaces. As

I said, it has a CLI tool, but it has also a REST API. So here is a couple examples of how you can use DRAND. You can simply query any node. So here I'm querying the KudosKey security node. using the REST API. So on the top you can see the query to get the distributed key. So what's the distributed key? It's the public key of the whole group. So it's a public key of the League of Entropy which allows you to verify the signatures emitted by the group. And when do you need to verify the signatures? Well, whenever you've got a new random value, so that's on the bottom, you can fetch the latest public random value. Currently

we're at around 90 to 300 and something, because I took the screenshot this morning. And you can see it's training with the previous signature value. So basically the value of the signature on the bottom is a point on the elliptic curve and it's signing this round number together with the signature of the previous round. So you got on the screen all the information you need to verify the signature on the bottom has been emitted correctly by the group. The body took issue with that? And for us, the story was the geopolitical context around that and around this tension within the body between nation states. For us, that's the story. We weren't actually particularly... invested in the super technical details

of this cryptographic floor. So we didn't go into detail on that. Some readers on Twitter especially were like, well, where's the technical details? We link to the white paper so people can go read that if they want. For us, for a more general audience, we're emphasizing that geopolitical angle. So it really, and if that had gone to, I don't know, a journalist at The Register or something, or maybe Kim, who's a lot more technical than me, maybe the article would have been more technical in nature, but it really does depend upon the journalist and the outlet. So I would just think about who you're going to give it to rather than the first journalist you

think of in five seconds. I think you're both quite different. Or you can give it to a couple of journalists. I mean, you give it to the person who you know is going to do the technical story, and then also give it to someone who's going to do maybe the broader show. I mean, you see a story in CNN, right? And then you also see it done by an infosec reporter. And so readers who want both or who want one or the other can get them. Yeah, I think that's a good thing. Like, you know, it's positive that there are choices about how something, what gets focused on, because there's a lot of parts to

everything, you know. Of managing other people's feelings. And Google it, you'll go down a rabbit hole. But I just thought I'd throw this back out there. How big of a burden is that in your jobs? Is that something you see on a regular basis? Or is that something that's happens now and then. Just kind of curious how pervasive it is. Oh yeah, yeah, it's a huge burden. I also think of it as something called the third shift. So the first shift is your job, second shift is tasks like, you know, laundry or, you know, cleaning up trash at work or, you know, it can be in any context. And the third shift is thinking to do that or thinking to approach someone, you

know, to plan for something in advance or, yeah. And women generally are socialized to be more focused on the third shift. So I see that coming up or the emotional labor. Mike, over to you. All right, perfect. All right, thank you very much. Also, they gave me socks to hand out, so I could randomly distribute socks at people if they want. And then I was similar. I also came from government. Also more on the offensive side, but just needed a change in pace kind of thing. Yeah, Pennsburg. Someone will reference us and throw a meme on us later. Have a go and I'll be scared to see what the hell it is. Do you know who this is?

Where did that come from? Some guy was like... I'm Technic and I was like, I'm on Twitter, I don't know what that means. I didn't say that, but I give this to people that I like their presentation, so you can refer that to Defcon. I was like, I'm not going to Defcon, but that's kind of funny. Yeah. I think he was slightly offended that I didn't know what that was. But I didn't want to be like a jerk about it. Yeah, you're Technic, right? Hey! Is that a company or is that like you? Or like... I'll get my chest. No, you're good. There was like a lot of cups. Are you gonna head back? Yeah, I'm gonna go. but there is a

female protagonist who is a victim of some sort of abuse, and I'll leave it like that. And it was better that we hired a female photographer for that very particular context. And that's just something that is... more appropriate for the story, easy to handle, and it just would feel very strange if I turned up. So it's bad to delegate that work to someone more capable and able to do it. I would say also that in trying to write more about these kinds of issues as a male reporter, you want to be very thoughtful about it. So I'd ask female friends, reporters, researchers for advice on how to start talking about those things and not come off as a woke dude or an insensitive dude or whatever

the vibe you don't want to give off is and how to, it's the same reason like when I spent a little time freelancing in West Africa and me, I did a story on the LGBT community in Cote d'Ivoire where I was living and I'm not gay and I'm not from Cote d'Ivoire so those are two things that I had to take a long time to try to understand and think in a different way about before I ask questions or I open my mouth. So I guess that's why I don't have a lot of that, like I'm trying to think more about it. And I think, you know. - Obviously you have no voice. Things of that nature,

the CV scoring is an ongoing debate in the community, but I tend to come to those pages last. I kind of only look at it when I'm going to pull the link or if I want to check against other materials that I've gathered or something. I don't know. Maybe I shouldn't admit that or maybe that's just me or something. But I don't find that I really heavily rely on that stuff because... It's always coming from checking with sourcing or cross-referencing or something. So when you're talking about a CV score or CVSS, the severity of the vulnerability, I don't-- I mean, the scale is very-- Arbitrary. Hit or miss. Yeah. So what I do is just ask people

who know. How serious is this? I mean, is that score indicative of it? So it's more of a, it's not a mathematical thing in terms of describing the severity of a vulnerability in a story. Just super briefly, yeah, it highlights the importance of fact-checking. No journalist likes to screw up. One I had was just in the wake of WannaCry. Two independent researchers spoke to me. They're not affiliated with each other at all, and they both said they found a version of WannaCry without the kill switch. So I hear it from one very reputable cybersecurity company, Kaspersky, and then go to another researcher who's independently discovered it. That looks... I mean, you had only two, but

there's a point where you reach where you say, okay, I've vetted this enough and I've got to publish. And even then, you can be wrong. I'm not saying Bloomberg did vet, but I'm just saying. Josh. All right. There's a lot of interest in the room to keep going, so since we can go a little bit long because we don't have somebody immediately after you. But since this is the I Am the Cavalry track and we are focused on public safety, human life, I want to shift back maybe somewhere in that vicinity. Okay. We like to say we want to raise the alarm without being alarmist because we don't want to scare people away from life-saving

medical devices. We also don't want a blind faith and trust in things that are indefensible and insecure. But there's that old paradigm that we've all been told, if it bleeds, it leads, right, for one of the axioms of coverage. Even if it's not you, it might be your editor. And it's fair, right? If it bleeds, it leads. Yeah. So what's the right tensions for us to carry around as we pitch stories or approach journalism so that we can have one of those things that catalyzes investigation and corrective action without being hyperbolic? It's kind of building off the question in the back. But also there's a different trend where – so it's going to sound like

a departure, but I'm going to triangulate – I probably know about a hundred forever days that people tell me as their confessional during DEF CON. These are old school hackers. They'll grab me at a DEF CON. They'll say, I found this thing. I can't tell anyone. It'll never get back.

And, like, people didn't really read the story and didn't really care. And that doesn't mean, I mean, people who want to know, know, and, you know, are on it. But it's like there's a, we're dealing with a lot of weird, um, trends or sort of audience reactions to stuff that's really tough to manage where you're like, well, I don't want to overhype it, but like, this is a really big deal guys. I'm like, no one's reading this story. So like, do you not care? Or, you know, it's, it's just really hard to manage everybody, everybody's interests and where their attention is. I think we're just going to go with final words here if you have any final parting words. My only final words are thanks for coming out.

Keep talking to us. Keep opening up the dialogue. Of course, we make mistakes. We're not up here trying to preach too much. It's just about transparency and learning from each other. Who's lying at the bar? Me. Thank you.

Happy hour does start now, I believe. So I know we ran a little bit long. Thank you, everybody, and thank you to our panelists. Thank you. Thank you. And I think that where I'm coming to from is that actually hurts, like, I think that these, like, the Jewish people's words are very important, and they should be taken seriously. But when you have a technical defense being strong, it's like trying to get enemy points. Yeah. I mean, it's a frustration for journalists to watch mainstream media cover these things, and it quite often happens with mainstream media that doesn't understand or they're led by their government sources. that they have more interest in pleasing the government force than they have in pleasing the reader.

It's a constant problem. And we often on Twitter will call them out. But it's a really delicate situation for a journalist to call out another journalist publicly.

I didn't recognize you, so I'm not sure. I don't have my picture posted on, so... Yeah, so... Anyways, thanks for introducing me. We didn't have to arrange an actual time. You're East Coast, right? Well, I'm from Indianapolis so. Well... I never got that actually, that guy that was going to speak for the 20,000. So I may have priced myself out. But that's okay. I really appreciate you bolstering me up to think that I can work here. More, higher. Well, I don't know. It really helped me to have that vaccine and to know that I can ask. I'm sorry, you were from domain? I've heard of it. Domain history kind of stuff. So we're researchers now. Oh, cool. And we're, not that you've

already been to the scene, but like, you know, I think the researchers can, in a lot of places the PR team needs that relationship with the journalist and I don't think it's for the benefit of the company. And so if you develop your own relationship with journalists then you can also sort of control that bad PR. Okay, great. Hi. I do have a card. Always useful to have. It's a really old one, so I just wanted to give you the updated information. I'm sorry.

Pioneering, wow. That's nice. I just got my first royalty check. My first royalty check on a book. I just got it this week. Well, because you get an advance to write a book, and then you have to work off the advance. And I got a really good advance. So I've just now kind of worked through, and I'm getting my first book. So now I'm clear. Everything else, every other sale now, I'm a bit more aware. Oh, that's great. Yeah. I used to be a journalist myself. I was at B&A and I was in the letter. I started this business about 10 years ago. All on cybersecurity. Mostly in the federal sector, but also in the auto sector. Internationally as well. So

Josh and I just moved to the auto side of the business. I just wanted to match a face of the name. So, um, I'll start, actually don't follow you. Yeah, damn it, why not? Huh? Damn it, why not? I will now. Good to meet you. So, nice to meet you. So, I don't know where we are at, but it seems like everything's going to be going to these things every year. Right.

You're not clickbait, so that's good. I fight it sometimes. I have to fight sometimes with editors to make it not clickbait. I actually appreciate hearing that, but I'm succeeding at losing my fight. Yes, ma'am. So keep up the good work. All right, thank you.

No we were Katie and I were in the meeting together. No, no, no, that's really it.

Embrace it, you all are controlling. Well, you can turn it on. Exactly, but I don't like to turn it on, but I do, you know. I ran into, actually I ran into Blake. I think it was RSA. And I don't think I've ever met him in person before. And again, I have the same problem with the avatars. They're so tiny that I can't... But it's not just that. He comes up to me and he introduces himself and I said to him, oh, sorry, I didn't recognize you because you guys all look the same. And I realized afterwards, like if someone said that, I don't know, but it's... He said, look, I get it, and we

all do kind of look the same, but it does become more. You said we all look the same. Oh. I said I can't tell you. Yeah. Yeah. So. But there is, there is like a whole slew of male germs covering this now. I'll try the different. That's true. White dudes with ears. And sometimes glasses. When I realized afterwards, that's a very PC thing to say. And we don't think of that when we're talking about a nap. Yeah.

I know they won't give me anything about

Okay. - That was it, I just wanted to see if it was in there and my understanding is until I find it, there's nothing more to do. - Monster Swag, the great sponsor. - Hi there. Awesome.

Well, it's not so much the dongle bag, it's this specific first party adapter. It's the only one that works 100% of the time. We have our own dongle bag. So we have a system that will work with what we have. It's all shut this off before it bugs. Cool. Moving down. Right side. Right side.

Thank you.

Give me a couple sentences. Mic check one two B-Sides Las Vegas. Uno dos, trace its own. Cool. Mic check, mic check, government. I am the government. Yeah, I just got dongles on dongles because I have a Mac so...

Now I'm supposed to just make sure no one trips and falls. Lots of wires and a beer involved there. I do. There we go. I have another computer if needed. There we go. No, the gangster life chose me, okay? Damn, it feels good to be a gangster. It's gonna be really hard to stand still, because normally when I teach, I'm back and forth like I'm in a metal concert. The Human Torch was denied a bank loan. Actually, well, how's that? Cool? The asterisk somewhere would be like, from somewhere, sometime. There was a, in San Antonio, there was this tubing place on the Guadalupe Bay River that had a wooden sign that said free tubes tomorrow. So every

day you went there, it just said free tubes tomorrow. Yeah, that's fantastic. I love it. That sounds amazing. Probably not in Texas. I remember asking, "Hey, so you got free 2's tomorrow?" I was young, dumb. "You got free 2's tomorrow?" And they were like, "Yep." I was like, "Cool." It hits them off. What's the tube? It's like a raft inner tube. It's like a raft inner tube. You were tubing down a river. I did that once. It was actually pretty fun. I'm just going to grab a coffee. Anything? I think we're good. A couple of quick questions. So normally we show a sign 10 minutes before the end. Do you want that 10 minutes before it's final or 10 minutes before,

say, 5 minutes? 10 minutes before the end is fine in case we do go over. We should take roughly 45 minutes. So if we aren't to the questions by then, if we aren't taking questions by then, I'm saying if it gets to be... 550 and we're not taking questions, we absolutely need that sign. Because that means we're going over. Okay, so 50 minutes? Right. That'd be great. Perfect. That's awesome. GitLab? No, I'm going to watch your panel and I think I'm going to stick around. ... ... ... ... ... ... Hello? Yes. Yep, I think we're all set. Mic's up. So I have this super unique use case.

So that was cool. He put it up on YouTube with his talk honoring Wow, I'm gonna have to check that out. Yeah Okay, yeah, yeah, I think I was flying to New Orleans over that weekend that they they did a huge rocket launch Probably Huntsville is where the world's rockets were. So that would be Huntsville, right. Yeah. And they tried to set the Guinness Book of World Records for the most rockets shot. So they said it was going to take, Guinness told them it was going to take like two months for them to actually get back to them to see. To verify? Right. Yeah, no, that was the number of accidental discoveries that they made never would

have made the moon landing possible. They had a whole bunch of engineers and astronauts and engineers and flight controllers doing panel talks and asking questions. It's on C-SPAN. Okay. Gotcha. It was being recorded. Okay. So definitely check it out. Yeah. Yeah. I handle that. Yeah. So totally worth it. Stand at the lectern with me. Security shaman because I'm convinced that anything to deal with the cloud is all just fairy tales, pixie dust, and unicorn farts. So you got to have fun with it, right? I'm also a SANS instructor. I teach a five-day class and I'm going to try to boil down some really good key points to just kind of bring in the basics. And so the reason why I created this talk was because there's a

lot of cool talks out there. They talk about hacking the cloud, you know, DLP in the cloud, but I really looked. Towards the end, we will be having a, we will be opening it up for questions. If there is time, I'll be passing around the mic as needed. Without further ado, our speakers. Thank you, everyone. Thanks, and welcome. Hello. I'm Cameron. I'm Matthew. Today we're going to talk about providing for a common defense with coordinated vulnerability disclosure. This is a quote from the preamble. But what we're really going to talk about after we take care of some feedback, let's make it easier for the public to tell the US government about security issues in our systems and require agencies to do something about it. That's our aim here. It

shouldn't be so hard. We think we can solve for this. So it's early in the talk, but it's audience participation time. Has anybody here, by show of hands, ever tried to report something to a government agency? Those who have hands up, go ahead and characterize your experience with a thumbs up, thumbs down, thumbs sideways. I see the guvies in the back with thumbs up, and everybody else saying, like, I don't think this works very well. So... This mic's off. Spam, is it these mics here still on?

is at the domain or at the host name where you found something is to send a message to the security app. This is not a new concept, at least in May of 1997 this was in an RFC standards track. Within this doc it enumerated a set of boxes that organizations could maintain and it named their purpose. Here's security, for security bulletins and queries. You may not know this, but the US government actually publishes the full list of .gov domain names. This is something that is published and updated about once a month on average. So you can see here this list is also organized by domain type. So you can see the executive, legislative, and judicial

branches. The one here on the screen is the federal. If you could tell me the why, that would be wonderful, but it doesn't. You have VPC flow logs. Now, who here is like a, you know, IDS or IPS administrator and you're like, I love that box that I have. I get to monitor all the traffic. Your world is going to change when you move to the cloud. There is not really much east-west monitoring that we have available anymore. Unless if we architect it and thus add more technical debt and cost more overhead, east-west. We're still on furlough. We're actually working now, so there was an appropriation, but that was about 112 days past the shutdown that this was still there. Now, to be

fair, people are busy. There's no requirement for them to maintain this, but this isn't a great look. I think it demonstrates an opportunity for us to improve. I did some math, or at least Google did. And I have some metrics that say about one out of every hundred federal executive branch agencies maintain a security app. This was pretty disappointing. I didn't expect this to be very high, but this didn't meet up. So again, I'm kind of cheating here. There's no mandate, there's no requirement for people to maintain this. And within federal agencies, unless there's a requirement, people don't usually do it. They could also maintain a security a different address. They could have a web

application for folks to report that to. This was just kind of a test. But yeah, again, an opportunity to improve. Can you talk a little bit about the cyber EO? Sure. So what Cameron has started to lay out for everyone seems like a very safe and interesting story, right? It's like, oh, that seems smart. Why aren't people doing that? If they have security at addresses, why are there not people responding to them? It seems smart that agencies would have a way for you all to communicate on whatever you might want to communicate to the agency, and yet here everyone is not talking about it. I was just logging into the console, so nothing too crazy.

However, if I was standing up like a EC2 instance or a virtual machine, we would have more information returned in the request parameters. So we could see Kyle stood up, billion dollar per hour instance on this date and time. What's nice about CloudTrail logs is they include the user. It may be. So if we are doing things that do not, one, provide you benefit, and two, allow you to opine and provide us information and communicate with us in a way that allows us to resolve problems we both agree are worth resolving, we're not doing it right. So it shouldn't have been necessary that there was an executive order that tells us how to do this,

but that is the frame under which we're operating right now, which I think both Cameron and I would agree, and I hope most of the folks in the audience would agree, is a smart thing to do, a wise thing to do, and a responsible thing to do. So, show of hands, how many folks came to the DDS presentation at 1 o'clock with Harlan and everything else? Super cool, right? They gave a lot of great examples. They talked about all the superpowers they have. They talked about the really interesting stuff they're doing inside the Pentagon. Well, Cameron and I, after we just showed you that really nice story of telling you how we operate IT on

behalf of the American people, we're now going to tell you how that doesn't happen anywhere outside of the Pentagon. So I know some folks brought some beers in here, which I think is great, because you're going to need them to see how things are totally different in the space in which we operate. So we're going to paint that picture for you and then tell you what we're trying to do to fix some of those inherent problems and how the executive branch, the sort of non-defense branch of government operates, right? So these are the... I mean, it's the funnest thing ever, right? So say we set up the plumbing for our cloud trail logs. Do we

want to look at flat files or JSON all day? No, we want to look at pretty dashboards. What's great about visualizing this type of data, especially if you have it all centralized, is you could see where it's all happening. If you know that most of your developers are in a particular region, and then all of a sudden thousands of API calls are firing off outside of that they're denied. If all of a sudden, you know, Mark the intern is trying to stand up, you know, 20 EC2 instances and it's failing. Do you want to probably have a conversation with Mark? I would say, hey, yeah. Or do we need to get you the appropriate permissions?

Always blame it on Mark. Another nice thing about CloudTrail logs, and this is kind of a big topic at the moment, S3 logging. Who's doing what to what files? And what's nice about this is it, well, it's not nice that it's not enabled by default, but if you enable it on the S3 bucket, and then enable it in your CloudTrail, you then get these access logs. Then you could see what data is being written to, what data is being deleted, what is actually happening with those objects that reside in that S3 bucket. If you don't do a good job at data classification on-premise, guess what? You could suck at it in the cloud too. I knew there was going to

be one heckler. And we talked about flow logs. You know if you've ever looked at like t-shark That's really what the flow log looks like if anyone got t-shark do do do do do do stuck in your head I apologize, but visualizing the traffic coming in and out again This is great for developers and also security use cases And you'll notice that a lot of these different types of log sources are dual purposed You could argue that cloud trail could be used for developers to understand what is changing in their environment They could use that for configuration management You could also use it for security auditing and investigations. With flow logs, again, not so much east-west information here, but what is coming in and out of

our VPC. The VPC flow logs-- Position of authority hierarchically over these organizations. We are sister agencies, which is different. At OMB, they have kind of a similar authority where they can do more things than just information security, but they can issue memorandums. So here's an example from 2015. that was issued that required secure connections, that required the federal executive branch agencies to use HTTPS for all of their web, for all of their internet-facing infrastructure. This is a good thing. So we've built this story because there could be a federal mandate to do coordinated vulnerability disclosure to the federal government better. And that's, we'd like to talk a little bit about that. So what might that look like? Well, There's some prior art here within the General Services

Administration, which is the most excitingly named organization within the government. There is a sub-organization called the Technology Transformation Services. Some folks may know 18F, which is top notch folks who work there. They created the first civilian branch, Vulnerability Disclosure Policy. You can search for it. It's a great doc. The intent of a vulnerability disclosure policy is one, to make it really clear where folks can be able to deliver their findings. But it's still not logging to what we call a trail, and that's how we get those logs out of your account and into your SIEM or log at analytics tool. But when you turn that trail off, it sends one more API call saying, "Hey, I've been turned off," kind of like

the last shot in the sky. So looking out for that, because it could be disruption for evidence tracking is a good idea. Now you can argue, yes, you can prevent this from happening through organizations and service control policies, but saying CISA and the systems of the, so my organization, and the systems of the civilian executive branch agencies is a difference of kind and not degree. So here's my cute kids. CISA is not responsible for the operation and maintenance of individual agency systems. Again, we administer the implementation of information security practices, but we don't run those systems. And our relationship is ultimately one of sibling to these agencies. So each, again, each agency has their own unique mission and the maintenance of their systems is tied to

the delivery of the services that they manage. Ultimately, vulnerability remediation is not a different thing from the maintenance of your system. And effectively, as a third party, CIS is not in the best position for us to evaluate the severity of a report, nor to rank it against the range of priorities that the agency has. They are closest to the care and feeding of that system and they are in a role where they can provide immediate or much faster than we can without playing a game of telephone and sending it to us and for us to try and triage and navigate it. So there's a couple of approaches that could take care. There are clear benefits

of having things be centralized or kind of the DoD model where they are the front door. But there's also the standing way of doing things and that is the law of FISMA where there is... But verify that. Because if you don't have mailbox logging enabled, you're not going to know anything about those mailboxes, especially if you have high-level executives that need to have sensitive actions performed. Another thing about it is, is you use an Office 365 directive or a policy to say, "Okay, everything needs to be online." I think having a single federal VDP is a good goal, and I think that that's something that we can work towards. I think that it would be very difficult and painful immediately. So some of the benefits of having

this is that there's a single point of entry for everybody, for people who want to report vulnerabilities. There's also a single point of management for DoD in this instance. It provides them a sense of visibility of what's coming in. It allows them to be able to check that the fix has actually remediated the flaw that was found. And hey, DoD is actually presently resourced in their environment to do this. The organization that was known as US Cert, even though the domain still exists, doesn't actually exist anymore. Do they have a security yet? No, I didn't get a response from that. Got a 550 there. So the... I think that there is power in trying to figure out a way, how can we do something that gets us

the benefits of a centralized model but still operate in the way that things are going. I would love to be able to say like, well, let's fix everything. If you don't know what you got, how can you protect your neck? That's a shout out to Wu Tang. However, now we have a new unique problem, shadow accounts. There's been times where, you know, I'll be talking to someone and, yeah, so I just stood up an account and I'm using my P-card for that. Well, we can't protect that, right? Now, you may have to get clever with your management team on... "All agencies thou shalt do X by Y date." Right? And most of the time that

happens because we on the management side have won a battle, a bureaucratic battle inside the agency to do that, but that in no way means that the agency has resources available to actually do what we've told them to do. And then we spend months and years and budget cycles trying to say, "No, no, this is important because M1617 or M1814 says that you should do this." but most of the agencies do not have resources or it's not really a priority for them. So the other way to go about doing this, which I think is where Cameron and I and our agencies, which have a history, even though FISMA says OMB does X, DHS does X,

we have a history of not working very well together, right? At the macro level, at the political level, OMB wants to do one thing, DHS often wants to do another thing, and it's a more combative relationship than a productive one. So what we are saying is like why don't we sort of roll this out through the normal processes and try to get a try to get agencies accustomed to all this right so in our 2020 budget I'm getting really bureaucratic and wonky here in the 2020 budget we put language in there that basically said agency some agencies already moving out on bug bounties some have already seen successes in implementing their own vulnerability disclosure policies

this is something we should move towards. Then every spring, OMB puts out budget guidance that says agencies. So now if you just have a rule sitting there. Publicize it. And we get you all in the information security community like super hyped up about it. And then they all just sit on their hands. Or they all put up a web page that gives you another email address that goes into an inbox to nowhere. Right? Like that is the way we shut down all of the goodwill we've tried to build with this community. whether it's the very specific stuff that the Defense General Services is doing, whether it's the broader governmental efforts that we have when we

go to RSA or we come here to B-Sides or Black Hat or DEFCON and try to open ourselves up to all the great work that you're doing. But we've made the choice already as a government that we don't want to spend a lot more money, although Congress has something else to say about that with the new caps deal. So how do we get better, smarter information at less cost where we're not hiring more federal employees? We allow this community who cares, who wants to report responsibly and wants to help us fix issues that could have deleterious impacts on the public or on government operations or anything else. We can do that in an effective, thoughtful

manner. Now, it may not be as much as what folks would want if you had total control, if you had that parent-child relationship, but this is still a pretty broad stretch within the constructs we operate in within FSMA to actually do this in a thoughtful, responsible way where we force agencies to build that floor. How do we go about scaling this, though? Another really cool tool is by Netflix, you know, small media company called Security Monkey. Easily one of my favorite names for a tool, Security Monkey. It's part of the Simeon army. What's nice about this tool is it uses what we'll call cross-account role authentication. And we'll drill into that in a second. But long story short, there's TLDR on there. So I think

that we can achieve some of the benefits that come from a central approach. but do so under the norms that the current law creates. Most of the reasons that I think that people want for CISA to be a front door for the federal executive agencies, I think can be addressed in a more meaningful way than CISA immediately being in the middle of vulnerability reports. I think that there's value in minimizing the number of layers between the initial reporter and the team that's actually performing remediation. I think that's most likely to result in a speedy and secure outcome. So when people say, well, this is the thing that I think that will come out of a central source, a central place that we've reported. Well, they want to be

able to ensure that all agencies have the same vulnerability disclosure policy, particularly with respect to prosecution. I want that too. I think that what you're really saying there is that you want equivalent protection for researchers across the executive branch. It doesn't necessarily mean that all agencies have the same policy. It means that you can have, you know, words are words and we can accomplish the same goal in slightly different ways. I hear folks say, "Well, let's create a single point of intake for vulnerability reports." That's like having a single 911 number. That's actually not how 911 works. There is one single number, but it works geographically. When you call 911, you don't call Washington DC,

and we don't route your phone call to- What does that actually mean? Well, most of us probably use Active Directory and some sort of federated identity like Ping or Okta or OneLogin. What this allows us to do is manage access at a global group level and then use federated identity to... That we would make it very plain that that would not be a thing, that the things that are reported to us would be for remediation as well as for things that come out of R&D and the research community. So what might the potential requirements of a potential directive or an MMO or requirement, a mandate for federal agencies to do some of this better? We

want to enable them to be able to receive unsolicited reports. So we could do this a couple ways. One that we may do is to say, you need to set a security contact for each .gov domain. It's interesting, the whole world is running away from Whois. But in 2018, the .gov registry made it possible that all .gov domains, not just federal, in fact, plug 80% of the .gov, the number of .gov domains are state and local, so they're non-federal. But we made it possible so that the individuals could be able to set a public security contact. And hey, it'll show up in Whois. So this is pretty powerful. So if you find something on a

.gov and there's a security contact, you should consider reaching out there first. Check port 43. I don't actually know who the security contact is here, Sissa. But here, going back to all my emails, I did kind of cheat. Again, like, I didn't check this. I really just wanted to email the security ads. I could have... Without a doubt that he no longer has access to any of our accounts.

That's a scary thought. And that's why we want to look into how do we do lifecycle management with local users as well. You know, if you fail to plan here, you could really plan to fail. It's really hard to herd all the cats back into the barn after they've been let out and running around. Who's ever tried to pick up a cat from a field and try to bring them back in the house? I know I haven't. I see some hands and they're just like, oh my god. This was pretty energizing to me and was part of the reason that I tried to advocate to get .gov to stick domain points of contact back in

Whois. Great paper. So more potential requirements of this could be we would instruct agencies to develop and publish a vulnerability disclosure policy. PDFs, it needs to be on a webpage, it needs to be in text or in HTML. We would instruct them what the policy would look like. We would say you need to define at least upon outset the set of initial systems that are in scope. You would define the types of tests that would be authorized. You would say this is where we're going to receive vulnerability reports. That could be at your security contact. It could be at a platform that you have procured or that you might manage. You are going to commit

to a remediation timeline and say it's our goal to be able to remediate these in 90 days. Of course, it doesn't require people to abide by that, but a pledge to adhere to that timeline. As well as clearly a commitment by the agency to, you know, a promise not to sue if you abide by, in good faith, the policy that's been outlined. Let me come back real fast to security.txt. But the other thing that could be done is an instruction to tell agencies, hey, upon issuance, you need to have one system, host name, and asset within scope of your VDP. And it needs to increase within... ...property and probably have me arrested. So how do we adapt our incident response plan to a service

provider? Do our imaging standards change? How do we do our incident response? Do we test our incident response? Have we verified that we can actually perform incident response? ...or at least... was pushed along by the Federal Source Code Policy, which was another MMO that came out of OMB. There was an instruction here down at the bottom that tell agencies you need to have a code.json, a code inventory that you will manage and maintain, and it's going to go on your agency website. The code.gov folks then scan that out. It's in a defined schema that they instruct agencies to pull things in. And then it makes it so that people can be able to find code underneath. top level agencies. So you can say, show me all

the code that the Department of Homeland Security publishes. This is a project that our team works on to be able to scan out HTTPS within the federal government. It's a requirement. So we were able to track that. Similarly, Maybe there could be, kind of like a code.gov, there could be a vulnerability.gov site. To make it really easy, you can go to that one place to be able to see, and we'll scan out the information and then represent it on a site that you could punch in a host name and say, is this in scope? What's the point of contact that I send this to? And that itself could either be the point of reporting, or

it could just point you to other places out of band of the site. The out-of-band mechanism may be more beneficial just because it's challenging to stand up a new system. Particularly in this instance, you'd have to manage the authentication. Instead of using a local user, it'll vary. There is no silver bullet.

Yeah, so in terms of doing, say, detection response, automated response side, seems like AWS, SNS, and Azure Event Hub have minute level granularity, not sub-second level granularity. Do you have any solutions for sub-second level or sub-second level event generation? Not really because we're limited. So the question was, How can we get results or events faster than what the cloud service provider will do? They can try and characterize that. In my mind, this is the greatest value of having a platform because then the metrics gathering just becomes a function of people using the system. And I think, again, that's a thing that we could get to. Other things. I do think, and this is similar to

what the UK's NCSC, the National Cyber Security Center does, they point people similarly to say, if you find something on a gov.uk system, go find the system owner and tell them. But if they're not responsive or you don't know who to tell, tell us. I think that's a role that we could play. Again, we can try and do a base level of evaluation and triaging and saying, "Is this real?" I think that's powerful. But there's also something you said like to allow agencies to flex those muscles and be able to experience the process of like, "Is this a real thing?" Some of those base level evaluation of like, "Is this a vulnerability?" are things that

agencies, that I am loath to remove from agencies. We want them to have those skills. because it's just part and parcel of working in security. I mentioned, I think, that any kind of a direction, we would try and be as clear as possible and offer them. There's great prior art here. Not only are there great vulnerability disclosure policies within the government and external, but there are great guides talking about how to manage programs like this. There are ISOs. Federated model, do you have any recommendations on the best way to accomplish that? Sounds like a million-dollar question. We can hire you as a So... So it really depends on how you want to leverage accounts too, because I've seen businesses leverage separate accounts to limit the blast

radius. The US government trying to approach information security practitioners and trying to tell folks how to do their job when our house is clearly not in order. So we lack credibility in trying to direct or instruct or say you should do it this way when there's a lot more that we can do on our side. And I think if we can demonstrate some heightened confidence, I think that then we'll be an invitee not just because we're the government, but because there's some demonstrated competence there. So the aim here is to try and improve things. I have a sense that we're pretty close on some of these things and there will be opportunities to provide more

formal comment and direction and feedback. But we're here and we're happy to take any questions and to hear feedback and concerns from you all. We're trying to act in good faith and recognize the different incentives that individual agencies labor under while trying to push them along to do things that are like They're not necessarily complicated, but they are hard for these agencies. Any other thoughts you want to offer? You said it perfectly. Cool. That's all that we have. Thanks for hearing us out. Our email addresses are on the screen. Internet, send us an email. We're happy to take comments and questions. Thanks. And it's been available now for, I want to say, a couple months. I know they announced the service publicly at Reinforce in

June. But yeah, that's a cool capability that could have some cool functions. Hey. So just curious, have you ever had a situation where you've had to aggregate multiple sims into a singular sim for... um like sort of information security council that works across these issues and provides recommendations on sort of how to address their particular issues not i can't imagine that we would force the smallest agencies who have one person that is both their cio their ciso their chief data officer the other chiefs that either congress or omb has told them to make they're basically just one person or like two people right so we can't do that um but another thing that that camera and I

have discussed and our agencies have discussed is what if what if you make DHS or what if you make them the entity to receive that information on behalf of the smalls and try to work across them right there should be somewhere to report that or or the smalls can say like hey here's our part of this closure policy report it to these people like we like we are not going to be able to deal with that based on our sort of budget and everything else but we understand that this is important and we hear you we just want to make sure that whatever agency is doing this that they provide a good sort of citizen

experience to the community that's trying to report right there's one of the things that this administration cares an awful lot about and it goes back to eo 13 800 right we operate sort of information technology on behalf of the American people, right? So we want to make sure that no matter what experience you have, if you find information, if you want to report it to us, we accept it and we try to work with it, right? And put it in our workflow and make sure that we mitigate those issues or address any vulnerability with respect to the tons of other things that agencies sort of... Your classic SIM, your classic IT security shop, and where

incidents are happening in the cloud and where developers are living in a honestly more cloud-centric world. Yeah, so closing the knowledge gap. But at the same time, having each of them individually create a VDP might get really messy, especially for some of the, say, local governments that have less resources. And given all the attacks we're seeing targeting these smaller localities, what can we do to, say, provide resources or make it easier for them to work with this community to receive reports? So in our brand of fede