BsidesLV 2025 - Ground Truth - Wednesday

Woo! [Music]

Wow! [Music]

Heat. Heat. [Music] [Applause] [Music] Heat. Heat.

Heat. Heat. [Music] Heat. [Music] Heat. Heat. Heat.

[Music] Heat. [Music] Heat.

[Music] Heat. Heat. [Music] Yeah, [Music]

down. [Music] Black. [Music] Yeah. [Music] Yeah, [Music] down. [Music]

Black.

[Music] you got. [Music]

[Music] by pretty [Music] Baby, [Music] baby. [Music] Hey, hey, hey. [Music]

[Music] Heat. Heat.

[Music] Heat. Hey, heat. Hey, heat. Heat. [Music] Hey. Hey. Hey. Heat. Heat. N.

Heat. Heat. [Music] Heat. Heat.

Heat. Heat. N. [Music] Heat. Heat. Heat. [Music] Heat. Hey, heat. Hey, heat. Heat. Heat. N. [Music]

[Music] Heat. Heat. [Music] Heat. Heat. [Music] Heat. Heat. [Music]

Wow. [Music] Yeah. [Music] Heat. Hey. Hey. Hey. [Music]

Heat. Heat.

[Music] Heat. Heat.

Heat. Heat. [Music] Heat. Heat. Heat.

[Music] Heat.

Heat. [Music] Heat.

[Music] Heat. Heat. [Music] Yeah, [Music]

down. [Music] Hey, hey hey. [Music] Yeah,

[Music] down.

Black

[Music]

Hey. Hey. Hey. [Music] Heat. Heat. [Music]

[Music] [Music] Heat. Heat. N. [Music] Hey, hey hey. Heat. Heat. [Music] Heat. Heat. Heat. Heat.

[Music]

Heat. Heat. [Music] Heat. Heat. [Music] Heat. Heat. Heat.

[Music] Heat. Heat.

Heat. Heat. N. [Music] Heat. Heat. N. [Music] Heat. Heat. N.

Heat. Heat. Heat. [Music]

Heat. Heat. N. [Music] Heat. Heat.

[Music]

[Music] Heat. Heat. [Music] Heat. Heat. [Music] Heat. [Music] Heat. [Music]

Wow. [Music] Heat. [Music]

Heat. Heat. [Music] Heat. Heat.

[Music] Heat. Heat.

Heat. Heat. [Music] Heat. [Music] Hey, heat. Hey, heat. Heat.

[Music] Heat.

Heat. [Music] Heat.

[Music] Heat. Heat. [Music] Yeah, [Music]

down. [Music] Hey, hey hey. [Music] Down

down down down [Music] down down down down down down down down down down down down down down down down down down down down [Music]

Black

[Music] Heat. Heat. [Music] By far. [Music] Baby, [Music] baby. [Music] Hey, [Music] hey, hey. [Music]

Heat. Heat. [Music] Heat. Hey, Heat.

[Music]

Heat. Heat. Heat. [Music] Heat. Heat. [Music] Heat. [Music] Heat. Heat.

Heat. Heat. [Music] Heat. Heat. N. [Music] Heat. Heat. [Music] Heat. Heat. N. [Music]

[Music]

Wow. [Music] Heat. Heat. [Music] Heat. Heat. [Music]

Wow. [Music] Yeah.

Heat. Heat. [Music]

[Music] Heat. Heat. [Music] Hey hey hey. Heat.

Heat.

Heat. Heat.

[Music]

Heat. [Music] Heat. Heat. Heat.

[Music] Heat. Heat. N. [Music] Heat. Heat.

Yeah, [Music]

[Music]

yeah yeah yeah yeah. [Music] black

hey black cheese. Yeah, [Music] down down. [Music] Down

down down down.

[Music]

[Music] [Music] be [Music] down. B down. [Music] There you go. [Music] Doo [Music] doo doo doo doo doo doo doo doo doo doo. [Music] Heat.

Heat. [Music]

Heat.

[Music] Heat.

[Music]

Heat. Heat.

[Music] Heat. Heat. [Music] Heat. Heat. [Music] Heat. Heat.

Heat. Heat.

Heat. Heat. N.

[Music] Heat. Heat. [Music] Heat. Heat. N. [Music]

[Music]

Wow. [Music]

[Music] Heat. Heat. [Music] Heat. [Music] Heat. [Music]

Wow. [Music] Yeah. Heat. Heat. [Music]

[Music] Heat. Heat.

[Music] [Applause] [Music] Heat. Heat.

Heat. Heat. [Music] Heat. Hey, heat. Hey, heat. Heat. Heat.

[Music] Heat. [Music] Heat.

[Music] Heat. Heat. [Music] Yeah, [Music]

down. [Music] Black [Music] Yeah, [Music] down. [Music]

Black

[Music] Heat. Heat. [Music]

[Music] [Music] There you [Music] go. [Music] Heat. Heat. [Music] Hey hey hey. [Music] Heat. Heat. Heat.

[Music] Heat.

Heat. Heat. [Music]

Heat. Hey. Hey. Hey.

[Music]

Heat. Heat. [Music] Heat. Heat. Heat. [Music]

Heat. Heat.

[Music]

Heat. Heat. Heat. Heat. Heat. [Music] Heat. Heat. N. [Music] Heat. Heat. [Music]

[Music]

[Music]

[Music] Heat. Heat. Heat. Heat. [Music] Heat. Heat. [Music]

[Music] Heat. Heat.

Heat. Heat. [Music]

Heat. Heat. N.

[Music] Heat. Hey, heat. Hey, heat. Heat. Heat.

[Music] Heat. Heat. [Music] Heat. Heat.

Yeah, [Music]

[Music]

down. [Music] Black beats. Yeah, [Music] down down. [Music] Down

down down down down down.

[Music] Las Vegas. Uh this is the the track name for this is Round Truth. Um this talk is once again a winning competition and it's going to be given by my dear friend Wasabi and uh so I just have a few announcements that I just need to go over really quickly. Uh number one, shout out to our sponsors. We'd like to thank especially our diamond sponsors Adobe and Aikido and gold sponsors um Run Zero Drop Zone. Um it's their support along with other sponsors, donors, volunteers like yours truly that make this event possible. Um so this is a talk about cell phones. These talks are being streamed live. Um and as a courtesy to our speakers and audience,

we ask that you that you check to make sure your cell phones are on silent. I don't want to give anybody a dirty look in the meantime. So I'm not trying to be your mom. I promise. And uh just to let you know as well, if you have a question, use the audience. Um we're going to do this at the end, just a question and answer. So um if you have any questions, you know, feel free to ask Wasabi. I'm sure you you'll be cool with that. But at the end, I'm going to go around and with a microphone, so you don't have to get up. So, um, so, uh, far as last thing as a

reminder, uh, besides, um, Las Vegas photo policy prohibits taking pictures without explicit permission of everyone in frame. Um, these talks are all being recorded and will be available on YouTube in the future. And I just wanted to let you guys know that as well. So, just, um, without further ado, uh, let's go ahead and get started. Please welcome Wasabi. >> Thank you. Thank you. [Applause] I'm going to turn that off because I have a attached microphone today which may or may not be working. Okay, there we go. So, thank you for coming to a winning competition. I um did this uh talk as an effort to just challenge myself. I normally talk too much. So, instead of getting too much

talking this time, you're going to be getting too much text. So, please ask questions as you get it. Um this talk is mostly about how to improve competitions. I myself and a lot of the audience here are uh volunteers for different competitions and the this talk is a speedrun on how you take that competition from something that people just participate in to something that people really like and uh there's a lot of different components of this and I'm only covering one area. So please afterward if you have questions please actually talk to the audience because a lot of fellow peers and co-volunteers are are here today. So with that I am Wasabi. I am a security

researcher, cloud engineer and competition creator. Uh I decided to do this as a challenge to myself. So I'm sorry in advance as there's too much text. Um people may be wondering what types of competitions are you going to be covering? Well, there's a lot and we're only going to be doing roughly three. But when people think of competitions, they probably are thinking of CTFs, capture the flag competitions. Those are great. Those are really cool and there's a bunch going on right now, but they're a very specific type of content. And that content makes things very limiting to a larger scope. For example, if you have a capture the fly competition, you're not going to see

people trying to actively defend a system. That's just not the way CTFs work. And so then you also have attack and defense competitions like CCDC and things like that where you are doing attack or sorry you're doing defense and sometimes attacking as well where it becomes a purple team competition. And then there's of course red team and forensics competitions like CPTC and circus. These are a lot of fun because they are a very different type of competition. In the case of circus it took us almost four years to get the formula. I don't know that we got it 100% right yet, but we've turned it into something where students actually perform the work, make a report, and

they're graded on their work, their report, what they turn in, and instead of being live and dynamic, they can do research. They can adjust what they found instead of the heat of the moment. But overall, that's not the point of this talk. What we are talking about today is we want I want to share how competitions can get made improved and what goes well and terribly and as you grow what those things do to the competition. So what's the big deal about competitions? Well um interestingly it got cut off. That's interesting. Um anyway uh the uh so I'll just adlib that part but um why do competitions? Well simply put the long story short of it is

that a competition is meant to simulate a large variety of real world in something that is easily digestible and performable. So for example um you can build new themes as things change. For example, social being a social media network, being election systems, being a bank with ATMs, industrial infrastructure for uh you know baking and cooking. Um and then simulating the whole business infrastructure to that simulated business so that everything functions. So is there HR? Why is HR doing what they're doing? Is HR making things harder for um your people? Is there corporate espionage going on? These are things that you can put into scenarios when you build competitions that are just not available if someone just read

a book or I mean I guess if they ask chat GPT there's you know there's there's a ton of stuff that just doesn't get seen. It also gives students a chance to see the emerging technologies that they don't normally see in their courses. Uh academic competitions are force students to deal with new things in the past couple years. One of the big ones is LLMs. we've put them in, we've used them in variety of different ways. And then also the other thing that students don't ever get to see real uh is industrial control systems. And I can't say competitions are real because they're not. But it gives them a simulation that actually lets them use

and challenge themselves. Are the protocols dynamic and uh proprietary? Yes. So, do they have to figure out how this thing works live? Yes. And that's where competitions come in. You don't get to see that. I can talk to you for hours about Modbus and it really won't matter because you're never going to see that until you actually have to get your hands on it. So, uh, it also builds teamwork and teamwork is something that competition teams are really, really good at. There are teams playing right now in CTFs and pros versus Joe's that may have never met before, but now they're a team. And when they finish that competition, they're going to still be a team

afterward. Well, unless there's some dramatic drama that goes on, but overall they're going to be a team and they're going to talk to each other after the event. This networking builds and shares the technical skills as well as all of that communication where people become a network. And it also forces students finally to just work in a high stress environment, whether it's any type of competition. It's really stressful. And uh when you have something high stress, it lets students feel that stress in a real way. And the that's what I love about competitions the most is that real stress. I mean, nobody else likes it. When I when I was a competitor, I got super stressed myself, but then when I

got to working, I wasn't as stressed about those scary situations because I was prepared. And students come back and tell me, "Oh yeah, that situation where we had to do all these incident reports, well, we did it." You know, it wasn't bad when we we worked in it. It was super calm. I did an interview and they said, "How would you deal with this?" the center was oh yeah just I repeated the stuff we did for CCDC and so those things are where competitions really win and also competition alumni are everywhere I think over half this room is um former or current competitors so it's it's really good to see I mean it may be a bit of a niche topic since

there's so many of our lap there but it's a really cool environment and so you may be wondering what about that AI thing and I it's it's cut off, but I can summarize. AI is really helpful. AI is extremely powerful. Students are able to utilize it in in most of the competitions I help organize. But they it cannot solve for things that you do not know how to ask for. So as students work on things and they try to do things, we've seen them write reports where AI wrote the report and it did not understand the context that the report was being written. So it failed and they of course failed the task. or they're

trying to fix systems or solve challenges that we've given them and they don't understand the context. So AI helps them and solves that but they uh they just didn't get it. So it it it it helps but you still have to have that hands-on experience to ask the right questions. So um I've already been given the notice that I'm over halfway through and uh that's exciting. My clock says seven minutes in. So but uh >> sorry >> it's okay. Okay, I I I did this as a challenge myself, so I'm sorry for all of you. Um, but about the CCDC thing, um, I I really wanted to just do a a background of what CCDC is. Basically,

it started way back in 2005. The region I'm a part of, Western region, started in 2008. It is congressionally recognized as a framework for building out cyber security talent to meet demand. Um, and I know I say that almost ironically because there's not a lot of job demand right now for cyber security in some cases, but um, it is a pipeline into the industry. Um, students are in teams of 8 to 12 and they are attacked while having to deal with all the stuff that we throw at them. And so it's a lot of fun. Last season we were the Federal Bureau of Control. It was a um SCP like secure, contain, protect aliens um scenario. And

so we had all these signal generators, jammers and things that they were having to maintain. It was all simulated, but they had to understand these systems and how they were intercorrected, interconnected. They had classified systems and they had to do data classification and these were things that students had never seen before. So it was a lot of fun and teams learned a lot. So then there's the circus thing. This is new. It's one of the newest cyber competitions that I'm aware of. Um, this was mostly put together by um, a trio of us. We've been wanting to put this on forever. Um, this was made out of Coastline Community College and it originally started for community colleges, but

we're expanding it. And basically we give students a company that's been hacked and the red team scenario is the red teamer breaks in or the nation state breaks in and we actually have either a real redteamer going through the network and then we capture the artifacts. Sometimes good where we give them full disk images, sometimes really bad where we just copy files from their desktop and say here's your artifacts and they have to build a forensics case and find out what happened. Um, and then they have to present it and they have to prove their uh chain of evidence and the handling. So, this was really cool and I have to give a um shout out to uh Dr. Brown who's in the

audience and he and really gave this the um academic spin that was needed to allow students to actually have this and make it into something that can be like courses almost. It made the competition a lot of fun. And again, we're expanding it because there was so much demand. So, how do you design a competition? Um, well, there's a lot of things. Uh, we'll go back to a little story mode, but originally CCDC was the same schools. You had eight schools. They always were the same schools. You pretty much knew which school was going to win, one or two schools, and it was very repetitive. It wasn't exciting. Um, but that gets boring because you you're doing the same

type of stuff. But even with that, you had all these things. You had to bring laptops in. You had to bring equipment. But that that make keeps things very static because where do you live offseason? And that's the biggest uh point when you're building a competition is where do you live? There's a lot of logistics that go into running a company. They're not as many for running a competition, but you're still an organization. How do people log in? How do people see information about the events? How do people do anything to log in? Do they need a VPN? Do they need credentials? How do those credentials work? What does it give them access to? These are just things that you're

probably like, "Oh, yeah, that makes sense." But you have to think about those. And if you're constantly tearing down your environment and bringing them back up, you don't get very good resources because you're constantly just trying to keep the things working that are working. And that's also a cost. If you want to keep stuff up 24/7, you have a couple of options. You either have to have stuff hosted in a data center or you have to have a VPF VPS or some combination of all of that. And then when you get into in person, what about logistics? Where do you host your volunteers? Where do you host your competitors? What do you feed them? Do

you feed them anything? Maybe not. Do you have internet? How much does internet cost the room? I know probably here in Bides, they spent a long time on that question because that question is never simple when you go to venues. Um, and then how long is the competition? Because that increased costs. How do you store stuff? What does the additional gain days gain versus lose? So, if you have a five-day competition, what does that benefit? Or if it's a one-day competition, what do you lose by having it all packed into one day? Um, and what most of all asking all these questions, what do students get out of it? If they have to deal with the all the logistics

and all of the processes that you're trying to get working and they can't actually use it, it's not great. However, if they have a lot of opportunity to learn and gain and have workshops, is it memorable? Do they learn something? A lot of what competitions come out to after the fact is the memories. It's, you know, you come back and you're like, "Oh, you remember that time when I was in and we were competing and we did this thing and we found that sol that solution." That's what people will come back to. So the final thing is when you're making a competition, you're trying to build one out, you have to ask the question for

everyone, what will bring me back for next year, both as volunteers and students. So competitors are sometimes organizers, but organizers are not competitors. And I I know that's literally written there and I'm not trying to read from my slides directly, but this is a very true exact statement because you don't have overlap or at least you shouldn't because a competitor can play in your competition but and then they can become volunteers but you really shouldn't have a a volunteer also be a competitor because you get this weird state where do I know too much about the competition to be able to play fairly? Um, and then also you get ideas because as a competitor you start thinking

like how would I play this? But as an organizer you're disconnected from that and you are asking sometimes the wrong questions because what's funny to you and what's fun and makes it enjoyable for you that keeps you motivated may not be the same thing. So suddenly you're making things too hard and too niche essentially for the main audience. Um, we also one thing that we learned that was very popular was sharing resources. We were the first uh group uh competition group to share all the resources out at this point. Um I'm pretty sure we are the largest uh we have something like 10 to 12 terabytes of previous competitions, PECAPS. They've been used by researchers um PhD

students and all sorts of things. They've been turned into workshops and training material and that training material we bring back and share with um the students again. The final thing on this is though um if the word gets out that your competition is too hard, you get really dedicated players. But really dedicated players are not the majority of what keeps the competition going because remember that cost balance thing. What we've learned is if you make something really hard, you get good players and you can't keep it. There's no pipeline. So you have to balance it. You have to create tiers. Something that we are doing this uh past year is um trying to make it so that um

we have something called invitationals which are like pickup games. We've always had them but we have more now. So the pipeline is in to get students interested and then they can go into the competition. Any competition can have this and I really encourage it because otherwise if you make things too difficult you just lose all those things. And then um one other thing about the realism and asking the right questions. Sometimes making the competition, this is especially true for CTFs. You make a challenge that exploits something really interesting, but you miss how it relates to the real world. And that's fine for CTFs. It's really good for CTFs because you're going to have those niche skills, but as a

broader scope, not as it's not approachable and it doesn't always apply directly to finding a new job. Um I have only a few minutes left but um this is something to think about with resources when I first start uh and that's yes that is smoke um uh but um uh when I first started taking over for WRCCDC um the we had an we had to move our resources from a data center hosted at uh the university it was hosted out of into um a dedicated hosting provider that worked. But the first competition we ran, we got emergency maintenance on our systems. So I was sitting there my first time running an event and I got an

email saying we are shutting down your servers immediately and then everything went down. There was no that was all of our resources. That was all of our ops. Everything was gone. So the question that comes in is what type of equipment do you do? And I'm not going to repeat all of this, but the the summary of this is how do you handle those logistics? What happens when systems go down? Do you have backups? You don't have to have mission critical level of stability, but you do need some plan, especially when you do loaners or rentals like that. You have to have that plan in place for what happens if 50 of your 100 systems are

broken. What are the resources? Are they too old? What do they run on them? A fun thing with loaners is they'll sometimes have operating systems still installed and so you can't actually get like they'll be locked down in like uh MDM mode and managed devices and then you can't actually unlock them. So you have a just basically a paper weight and you've just paid for that paper weight. So one thing about for competitors and students uh students generally uh winning teams generally specialize whether it's CTFs, CCDC, CPTC, they develop their own process for making the competition work. Um, one really cool example was Blue Spawn where they built their own EDR as students and it worked

really well and it caused the red team attacking team a lot of challenges and teams will practice more that are good but it doesn't mean perfect. So they will constantly rotate out because it's four years usually, right? So they will constantly have new teams and then the skills change and so they're constantly having to redevelop. If you are a a competition team, the one thing I would suggest for you is to prepare notes and take notes. Even if it's 10-year-old notes, you want a tree of all the effort that you put in so that the new people coming in can work off where you went. Um, final thing, um, uh, the some of these changes that I've

been talking about. When we first started this, CCDC never had a winning team at nationals. Um the highest we got was in 2011. We had third place. After all of these changes, and this took many years, we started making it to um nationals. And not only did we make it to nationals, we've had two winning teams, two winning teams at once um four times in the top three in the last 5 years. And so we went from being the school the team that was, you know, the never actually won to winning. And it isn't to say that we as organizers did all that. That is most of the students. But we made the changes to make it

easier for students to succeed. And as organizers hopefully watching this talk, make those changes so your students can succeed too because that will really bring the make the difference between teams. Um, finally, here's our uh winning team for this year. I just wanted to show them off because that was pretty exciting. Um but with that I am at time. >> Yeah. So uh with that I hope do I have a couple minutes to get questions if there okay?

>> I want the microphone so that the video >> thanks I appreciate the talk. Um you mentioned discouragement kills competitions. You had the vi picture of students in hazmat suits. What have you learned about discouragement in western region CCDC over the years? >> Uh those are not students. That was the scenario. Uh there was a containment breach in the scenario. So we all went in in full hazmat suits and we removed equipment as a but for discouragement. Some of the things that really discouraged them uh were things like that we didn't even think were big deal like putting like mines in like bash shell like when they did ls it makes the slow locomotive go

across the screen just these little frustrating things that we didn't think were a big deal that really discouraged teams or not getting feedback that's a big one as well if teams don't get feedback they don't feel like they're doing anything because they don't know what they don't know and so they just give up and so that is the biggest one I is if I I were to think about it, it's really getting good feedback was the biggest discouragement if well not getting feedback is the biggest discouragement. Sorry. >> Any other questions? >> Or is there another >> I don't see any other hands, so feel free. >> Feel free. >> So you didn't talk much about scoring.

Um what makes for a good scoring system, good scoring architecture? How do you balance it? How do you keep the game competitive but not so close that you don't have a winner? >> I honestly probably would hand the microphone behind you and answer because this is this is Dr. Brown and >> hi, I'm Pro aka Dr. Brown. >> He has helped so much with scoring. I just make the changes technical. >> I I have to give a shout out to my chief judge in WRCC DC Alchemy. uh because between he and I and Joe and um our red team leads um C uh they they have helped try to level the playing field so to

speak. Um students will try stupid stuff and we will catch them and we will tell them don't do that. And it's kind of like um you know college teams that cheat when we catch them then we have to penalize them. I've penalized teams and said hey okay great you did stupid stuff see you in three years. Um on the scoring engine side for the services. I wish basic was here because he's well qualified to answer that question. >> But we've gone through three revisions. >> Totally give you a bit. >> Yeah. I I would boil it down to honesty, transparency, and taking accountability, right? You will >> you will have issues during your competition. You will have a time that a

um sponsor goes in and trips over the power cord and takes down a team. That happened to me and Joe in nationals in what 2012? >> Yeah. >> Um >> unforgettable. >> You have to figure out how to make that fair. You have to figure out how you're being honest with the teams on what you're doing. The most frustrating thing for me as a competitor was how is how is my score calculated? What's going on here? And how can I learn more? And you have to figure out how to be transparent. >> Y thank you. Well said. And I I handed it over because it is a team effort. There's literally no way to make all of

these scoring things one person. So I know I'm at time. I will be wandering over there for a few minutes afterward. Thank you all and I hope you got something out of this um and learned some cool new things. So with that, thank you >> and I will

>> thank you 20 minutes. That was [Music]

to [Music] heat. [Music] Beauty. [Music] Oh. [Music] Heat. Heat. [Music] Hey, [Music] hey hey.

Hey, [Music] hey hey. [Music]

[Music] Good morning. >> Good morning. >> I'm glad that you're here. This is going to be a different talk. I'm just going to tell you it's going to be a different talk. I hope you leave here feeling a little bit confused and then like mainly confused but also curious. >> Okay, just do full screen. That's all good.

>> Yeah, you can minimize. Good. So I hope you leave here a little bit confused and a little bit curious. What I want to share with you are some is a methodology that I came across some time ago that I've used at various points in my career. My goal is not to show you all the ways that I used it, but for you to understand the basics of the methodology and then I'll talk a little bit about how I've used it. The idea is and thank you. Hi, my name is Manish. I'm going to tell you about me, but I don't know. It just doesn't feel like the important thing. Here's what I want to cover with you.

How to innovate. Period. How to innovate. There is a methodology. There are lots of methodologies, but there's a methodology I would like to present to you on how to innovate. We're going to talk about how to innovate in security, but broadly how to innovate. This could be used in a lot of different areas. It's been used in a lot of different areas. So, I'm going to give you a little bit about the history and development of this methodology, how it actually works, some use cases. So, we're going to spend 80% of the time talking about that because it's a little bit complicated. And then a bit of time on security. Okay. Who the hell am I?

Why am I here? What have I done? These are all existential questions. I don't know. That's my answer. But the short version is I am here because Bides has been an important community for me in a lot of different ways. Uh I found probably the greatest job I've had at a bides in my city and I went to it and I learned about it and then years later I met somebody there and I ended up working there and it was an incredible opportunity. I've met some amazing folks. I've learned so much. You know, we talk about paying it forward. This is my little bit of an effort to share what I've learned some hard lessons and in ways that I hope are

apply. The most important thing to know about my background is that it's weird. It's all over the place. And I mean that both in cool ways and old ways where I spent a lot of time looking for work and unemployed and doing odd jobs. So whatever professional you see standing before you is an amalgamation of all those things and that's important because this has helped me understand okay I understand that I think differently and weird and that's okay and the last thing is the what have I done I've applied this methodology in areas of security ones that I'm going to talk about data privacy dark web economies software supply chain critical infrastructure ICSOT and a little bit insider thread.

That's all really sexy M. What a great promise. I'm going to spend like five minutes on those, but we're going to earn our way there at the end. Okay. So, here is something that you're already hearing about. AI, AI is everywhere. All right. Well, how do we think about it in this moment? My frame is AI is tomorrow's critical infrastructure. However, we think about critical infrastructure, it's import, national security, cyber security, resilience. That's what AI is going to be. It arguably already is, but it's accelerating digital transformation. It's exacerbating the strain on defenders and it has really complicated supply chain and governance around it. You've probably already been thinking and hearing about aspects of those this week

and you will continue to. So, we can establish that. Second, without thinking differently, we get stuck on complexity. We erode trust and our systems are brittle. And sort of in that order, complexity slows us down. And by us, I mean broadly, security, risk, community, those of us thinking about defending broadly. And so if we're too slow to change, those gaps increase, downtime rises. We know all of that. There's no FUD here. I'm just trying to state it as an oystism. So the third is, and this is hopefully the new idea that breakthroughs come not from balancing trade-off and how do I make the compromise, but by breaking them. How many of you have um picked a lock?

Do you know that dopamine hit when it pops open, right? You're like, I got it. That's in my mind what hackers do. We don't break things. We break things open. Now, sometimes we break things open that we probably shouldn't, like locks, but other times it's rules and methodologies and limits. And so, this is my effort to present you a methodology for hacking innovation. So, first the history and development. This methodology is called TRIS. That is an acronym in Russian. not going to try and pronounce it. In short, it's called the theory of inventive problem solving. Just so you know, I'm not making this up. You can look this up. I want to tell you a little bit about the person that

founded it. I'm not a huge fan, is you saw my my intro of focusing on the individual, although people are important to me, but in this case, I think this person's story and the context in which they created it is formative and is key to the methodology. And I think you'll see why. So fundamentally it's born from adversity. Can I walk or you guys are gonna Is that okay if I move a little bit? All right. So this is Genrich Alcher. He was born in Cash Kent in the 20s, the 1920s, 100 years ago. When he was 20 years old, I don't know what you were doing when you were 20 years old. me. I was doing the

finding out part of Fafo when I was 20. This guy was working in a patent office in a Soviet patent office. That's crucial. Soviet patent office and he was researching all the patents. That's pretty much all he was doing. But as he did that, he started to think about a systematic way to understand what are the patterns of invention. Let me ask you a question and I really want you to think about this. I've struggled with this. What is the difference between invention and innovation? What's actually new? New, new, new, new. Have not seen it before. New versus improved. There's an old thing new and improved. What's the difference? How do you know? How do you know it's

just not new to you and that it's genuinely new? So, that's what he was looking for. All right. Uh about four years after that um he was arrested for inventor sabotage because he pointed out the inefficiencies in Soviet invention and innovation and technology. And so they threw him in a goolog, a Siberian labor camp. And for the years that he was there, he continued to work on this in his head. Then about 10 years after that when there is the if you know sort of Soviet Russian history there was the thaw with Kruev after Stalin passed away he published the paper 10 years later so don't feel bad if it takes you a little while to get to your idea out there and

then 40 years after that not even during the manufacturing boom or anything in the 1990s a little bit in the 80s but in the 1990s big companies like Boeing and HP and Samsung started to pick up on this idea and we saw it in the 2000s. I'm going to give you some examples. So, number one, what an incredible story this person creating. I mean, we talk about necessity being the mother of invention. I mean, he really sat with it and tried to cultivate it. The other thing I want to present here and in this current world and environment, there's a meta story here and I hope you'll get it is to learn from our adversaries. At the

time we were early days of cold war and when he came out of the goolog they were like oh yeah come back in we'd like to understand how do we do innovation and invention all of that much better but I I know some people who would bristle at the idea of studying what a Soviet anyone does but for me there's some very important lessons here and some very powerful ones and I hope you'll agree. So he came up with this structured innovation. It's basically a systematic set of proven strategies to figure out how you resolve a conflict. And he distilled it from thousands of innovations. And what he did is go, okay, here's the stack of things that

are actually new. Here are things that are applied from a different industry, right, that just come over that are new here. And what is underneath here? And so he was able to parse that out. So the way this methodology works, I'm going to go into great detail about it, but the most important thing for you to understand is to seek conflict. What are the things that are in conflict? We are trained both personally and in a business sense to avoid it, minimize it, compromise. And I am telling you the first thing you need to do is identify the conflict. Really understand it. This requires empathy compassion insight analysis. So broadly he came up with these 40

principles and I'm going to show you a few eye charts and they are intentionally that way. This is not something that I could I if we were to do this as a workshop I would have you get laptops. I would get you a spreadsheet and we would go through it. So I'm going to try and illustrate for you. So here are examples of some of the 40. It's not all of them. Length of an object, speed, force, stress, weight of an object, temperature, power, weight of time reliability measurement accuracy. You can see these are kind of meta ideas. And then across the top, same thing. Length of an object, speed, force, stress, weight. So here are the

parameters you want to improve. These are the parameters that get degraded when you improve. There's a tradeoff here. And so you can imagine who remembers the game Battleship? >> It's a little bit like battleship. That's what I thought here. A1 hit. So you're looking for the intersection. That's why I'm saying you have to find what the conflict is. I'm going to go through the methodology in detail, but essentially you're going to look at this and in here there's going to be a set of numbers that are going to indicate which principles you should look at and go all right here are the principles I need to consider and there are specific solutions or ideas to those principles.

Let me see if those apply in this situation. I know this is all abstract right now. I will make it practical for you. It's summer. It's 104 degrees outside. Would you rather be in a room that is a little bit hot but quiet or cool but really loud? Who would rather be in kind of hot but quiet? Okay. Who would rather be in cool but kind of loud? All right. So, conventionally, we would go, "All right, well, here's your choice. You know, you can go over there or over here. If you want that, you go over here. If you want that, you go over here." This is what we're going to do. We might try and make adjustments. We

might bring in a fan or something else, but it won't quite get there. So, let's talk about that. The primary function of an air conditioner is to cool the interior space. But the cooling fan and I spent way too much time on air conditioner because our AC broke down. I'm not a mechanical or engineer person at all. So I was asking the repair guy. I was like, "What is that?" Okay. Is that important? Uhhuh. What's that? All right. Is that important? Like I didn't know anything about it. And cooling the whole space requires a strong, you know, compressor and that produces a lot of noise. How many of you have installed an air conditioner in a window?

Yes. Okay. It's really annoying. Um I I live in New York and you can measure someone's wealth by how many AC's they have. That's how we measure wealth in New York. And um or whether or not you need them. So the trade-offs here is you need to cool, but it creates a lot of noise. That's a kind of harm. Kind of harm. So, Tris aims to reach an ideal state where the noise of the compressor is not heard inside the room. Compressor is outside. Now, this isn't always perfect. If you've seen some of these ACs, it's still loud, but it's not nearly as loud as if the whole thing was inside. Simple example. So, oh, I should have made this bigger.

Sorry. Here are the steps. We're gonna go through this a little bit, but at a very high level, it's going to be relatively straightforward except for two steps. I'm saying that to you because I know you're smart folks. So, first define the problem. And this is involves that root cause analysis, right? You got to This is hard. This part's hard. Then analyzing the problem is understanding what are the contradictions you're looking at. We're going to talk through this a little bit. This is this part's challenging identifying the ideal solution, what it looks like together. And then this other part you're familiar with generate potential solutions evaluate and refine results in evaluation testing. I know that you've

done this in various aspects of deploying a tech trying to manage a patch fix something build something. So I'm not going to spend too much time on this. I'm going to spend more time on this. How do you identify and analyze the problem and identify the ideal solution? So I'm going to give you three examples of Tris in action. First it's got cut off. Sorry. Is this wrench. If you can't see it up here, it's almost like a regular wrench except for two things. One, the metal turns a little bit here. And then there's this ratchet box on the end. All of these were designed with Tris. So that one um there is a principle called separation

in space. We just talked about it with the air conditioner. Um and that ch that change in the design of metal forging. So it increases the area that the user's palm can grip and also um the leverage available, right? Think about a flat thing versus like you're gripped all the way over. It's going to seem like a small thing, right? The second is this keyboard. Has anyone seen a keyboard like this before? If you can't see it, it's a keyboard. You seen one without letters? Think about that for a sec. Why do you think that is? How would they remove letters and numbers? So there's a principle, another principle called trimming. They remove the numbering and the letter

on the computer keyboards. If you are a proficient typist, you're not looking down at the keyboard. See, you don't need them anyway. They found it increased the the effectiveness, the proficiency of the typus by 17% because they weren't distracted by what they were looking at. Last one, Samsung Galaxy. Straightforward for us. I think we all work in tech. We think about this. These ones maybe not as much. Trading off the battery life and weight of the device. Think about that. We know that in a phone, right? You're like, weight and battery like though, you understand that that trade off, but really they're looking for a conflict in ways to put that together. And this isn't just about

a neat way to find a compromise. That is not what this is. So, if that's where your head is going now, that's fine. That's a good first step. But no, that's not it. What I'm going to show you next is going to be a bit of an eye chart. I'm going to leave it up for just a second. I don't expect you to read it, but to comprehend the idea. These are all the 40 principles this way and this way. This is what that 20-year-old came up with in his head. This is what he was mapping while he was in a Siberian labor camp. So, remember I talked about the principles. See, did I bring that one up over here?

Yeah. Okay. So, here is you can't see it. This is number 36, device complexity. And here is number 31 object generated harmful factors. This is our air conditioner. We go to that one. It shows us 19 and one which is um 19 is use of energy by moving object. Number one is weight of moving object. So they took those two together applied it to the AC. So this is going to seem a little nuts right at first like okay you have this crazy chart. This is weird. This seems really static. I'm going to give you some broader examples now of how this has been implied in manufacturing, design, IT, and even security. Let's simplify the methodology. This is

how I thought about it and then I found someone that illustrated much better than I could have. This is how I think about it. Your problem, the TRIS concept of the problems, TRIS's suggested solution, your customized solution. Think about that. Specific problem, generalized problem, generalized solution, specific solution. Let me pause there and see if that lands. I'm actually asking you, so you can nod your head if that's yes or you don't have to nod your head if the answer is no. Okay, this is starting to land a little bit. We're not going to spend a lot of time going through the 40 principles. There are consultants I've actually thought about becoming one. I don't know about

like who just do this. This is all they do. They come in and go, "All right, I know the principles. Let's talk through your conversation. All right, you're thinking about these four. These ones go over here. All right, here are the things. Now, they work with the specialists. Let's come up with a solution. Work through the whole thing." I'm going to give you some examples of where that's happened and then how I've tried to do this on my own a little bit. All right. So, first is this doesn't work with all problems. That might be apparent, but it doesn't work with all problems. What I have found is and and this is backed by the research around

Tris, but specifically in the way I've tried to apply it, there's three things that I think about what makes a good candidate. So the first is, and tell me if these sound familiar folks, a resolved a recurring unsolved contradiction where, for example, security demands and business needs continuously clash. There's one. It's almost like so obvious, right? Second, persistent pain points. So, our incremental fixes are not really resolving the real tension. I hope you're thinking about something that you're working on now, whether it's like fishing or apps or vaugh or pentesting, whatever it is. It's ideal for, wait for it, chronic problems that resist conventional solutions. You've tried the things. either they're not working, they're not working fast

enough, or something has changed where they stop working. So think access control or access versus control, privacy versus monitoring, resilience versus agility. These are things that are always there. They're always there, these conflicts. And if every year we have the same unsolvable debate crops up in security meetings and conversations and conferences, that's an opportunity to use Tris. It's less effective for one-off incidents or best practice work. If you have ever been in a situation, and if you haven't, you probably will be sometime in the next year, where nobody can tell you what right looks like, but everybody can tell you what wrong looks like. That's where this comes in. Let me talk about some industrial and

business applications. Um, actually some simpler ones that that came to mind. Um, like tuna cans, you know, you can stack tuna cans or soda cans on top of each other. That was designed using Tris. You know, upside down ketchup bottle, Hines, that was designed using Tris. And you might think, what? How? They looked at, you know, cement mixers and how they keep things turning. They were looking at how do you keep a liquid and store it upside down and dispense it because that's not intuitive to store a liquid upside down. So, they use the Tris methodology to then design that. I'll come up with a few other examples. Automotive safety. I didn't know this one until I started this research.

Airbags. So, how do you provide comfort? Comfort in the softness of the bag and high impact resistance, which it has to be in a hard state and then expand to a super soft state. Um, ziplockc bags, so realable packages that balance ease of access with freshness. Uh, and then there's a a bunch in manufacturing that I'm I'm going to talk about some of those, but here are some like very basic consumer goods, right? very basic things. So I'm trying to give you this idea that there are a lot of things that you see around you that have used the TRIS methodology. One of my favorite papers about this was written 15 years ago. This is the paper.

It's applications of tra to it and it's HP fellow and basically looked at a bunch of use cases. This 15 years ago uh here's URL aitis.org. You can look it up applications of trace to it. That's it. relatively easy to find. So um this researcher took this class of problems in IT security and split them in two categories. What is the next big thing? At that time the next big thing was future of IT services, future business intelligence. You can just change that to artificial intelligence and like software as a service and it's totally relevant. And then a specific problem secure but open. If that's not the bane of our existence, I don't know what is.

high CPU cycle costs, change that to high token costs. Private cloud conflict, still kind of there, hybrid cloud, least asset management, we can talk about BYOD or devices. So, uh, I'm going to pull some of the cases that came from here and there are three. The first is so secure but open IT and there's a pun in there. Open it. You're welcome. uh a global beverage company. So they they had this challenge where kind of traditional thing security leaders were t caught between stricter controls and keeping business fast and creative and they didn't have budget for it. So um Tris helped reframe the issue not from how much security but how to increase security without harming openness. Think

about that for a sec. how to increase security without harming openness. Instead of the principles of weight or power or noise, it's other things about what a human experiences. This is where it's a little bit challenging to get there, but it's there. So, they talked about using again, how users, how secure behavior benefits their team. We know this now from a lot of the fishing security awareness training, all of that. But they got here pretty quickly, not after years and years of doing the training. So they made security features part of the business projects and gave users transparency and involvement. So better adoption, more trust systems that fit real world use cases while remaining resilient. This

was 15 years ago. They did this in I think six months. Like this wasn't a year-long thing. And think about all the journeys we've had to go through to get to this place. Here's a very specific one, but it has to do with security as well. It's around hardware. So, Seammens was told, "Make our key switch," and I couldn't find it in English. It doesn't matter that it's in German. Made our make our key switch smaller without making it less safe. So, their contradiction pretty classic compact and affordable versus uncompromised security. I want you to think about it literally here and then we're going to think about figuratively how this applies in security. The kind

of security we think about. So, what they came up with, and you're going to be like, "Oh, that's cool." But they came up with it during the TRIS methodology. If if we talked about how we would solve this problem, I don't know that we necessarily would get here. What they designed was an auto eject key. So, when the switch is off, the key pops out. No one forgets the key. The risk is dropped. And because of the way they designed it, the size shrink, the size of the whole thing shrinks from here to here. And it was cheaper to manufacture. Now, I could keep giving you these examples. I'm going to give you one more,

but the point is not look at all these cool examples, but I was really curious. All right, I get in technology. I get in design, I get in process, but can it happen in securities, DevOps? So, they needed to test, they needed authentic test data, but real customer info is too risky. Now, you might be like, okay, yeah, today synthetic data, got it. They didn't have synthetic data. They couldn't create it. So they wanted accurate testing but compliance and privacy demanded you know protection. So there's one principle called segmentation where they split the data into high-risisk and lowrisk groups and then there's there's a principle called taking out literally removing different from trimming where you remove the

sensitive elements. So they anonymize the sensitive records they simulated realistic data for the rest and they built pattern into the test data safely. And so now they can stress test and innovate confidently without data that reflects a real world that carries no real world risk. By the way, a company that did this synthetic data for AI. So this example is over 10 years old. A company that did this for training data for AI just got bought by Nvidia for like nine figures. So let me pause here before I go into security cases. I've given you a lot of information. We're gonna have a bit of a Q&A at the end, but before I get into the specific

ones that I've talked about, let me pause here and see if you have any questions up until this point, either about these cases or the methodology. I know it's a lot. Okay, I'm going to go in one by one the examples. So data privacy, software supply chain, ICSOT, dark web economies and insider threats. Okay. So the challenge around data privacy I think most people are aware of data utility versus anonymity. So here I was working on um how many of you are familiar with the notion of differential privacy? Okay. So that in a sense is resolving a conflict that's here between data utility and anonymity. Uh, I was working at a startup where we were

trying to find data that had been leaked on the dark web without having possessing that data. So, if I want to find if your social security number has been compromised, you have to give me your social and then you're kind of compromising it. So, how do we find that conflict? And I didn't come up with it, but the company came up with and then it made sense to me. So, I'm not saying it's like, oh, we invented this or I use this to invent, but it helped me understand it and then explain to people why that technology is unique. It was a particular kind of fingerprinting where they they took a series of fingerprints. So if there's

like nine digits, you know, they did six the first six digits, then digits two through seven, then three through eight, then four through nine, and they would look for an overlap of all of those and say, "Hey, we found your thing. I don't know what your thing is, but we found it. There's a really good match between these." Second, software supply chain. This is issue that I work on right now a fair amount. How do you reconcile rapid deployment and security checks? And so there's two principles in there. One is called preemptive actions and the other is like an aspect of automation. And here I've applied it to software bill of materials and identifying transitive

dependencies. If you're familiar with software supply chain and how there are certain libraries that are uh dependent on others and that library is then dependent on another one. There's a bunch of libraries that are dependent on that one that might not be immediately apparent. And so you have to find this conflict of how do we move quickly and do it securely. ICS and OT. The biggest thing for me when I started working on industrial control systems and operational technology was this real shift in focus away from any of the things we conventionally talk about in security in cyber security it cyber security confidentiality integrity availability really to high availability operational continuity and and so there there's a notion of

segmentation. How many of you are familiar with that? You've heard this before. Segment your IT and your IT networks. Okay. When I heard that, I thought about Tris. I was like, "Oh, okay. That's one thing that they've already instantiated." And then another. So, the reason I'm bringing this to you is because it's helped me move into different domains of cyber security because I understand what is when I hear about what the solutions are or how they approach it that's different. I can go back to this and go, all right, that's that's what the core problem is. Probably the the two for me that I used in advance. So at these these three that I gave you

later, I kind of already had Tris in my mind. So I was like, oh, okay, I see where that applies. Let me use it to explain or understand. But the two where I used it before to go, I need to understand this. again working on the dark web and thinking about the resiliency of that underground economy. If you're not familiar, broadly speaking, there's darknet markets. It's a very very resilient environment. Think about I I think people think about it like a dark alley or something like that. It's more like a flea market. It's a really big weird flea market. That's what the dark web is. And if you ever think about a flea market, you ever been

to a flea market? Yeah, you can shut that one down, but that group's pretty dynamic. They'll find another spot. But the question was, how do you There were two conflicts. And that's where I was like, I'm going to there's a conflict here. It became clear. Anonymity versus trust. I don't want to reveal that I'm selling these credentials, but I need someone to be able to trust that I'm going to do that, that I'm going to deliver on what I promise, that the goods that I commit to, whether they be credentials, cards, opioids, scripts, that it's I'm good for it. So, how do you maintain that conflict? How do you navigate that conflict? And the other is

decentralizing versus coordination. So you have to decentralize across the markets and then you have to have coordination between initial access brokers and as we saw their evolution of ransomware as a service. The TRIS methodology not only helped me understand I used it to understand all right there's this conflict how do they resolve it but number one what about this environment was particularly resilient I was working in threat intelligence at the time and two where if at all were law enforcement interventions going to be effective. It got me into conversations with criminologists and other folks like that where it was like okay this is where we should put our attention. this is what we should do with the data once we find

it or this is what it actually means. So we shifted away from like you're exposed on the dark web, you should be worried to like what exactly kind of data is it and is it new or not and what is the likelihood that it'll be sold by a group that has this already established and is part of an initial access broker group or can leverage it for recon or anything else. The last that I'll put is around insider threat. I still kind of do some work on um you know you and entity behavioral analytics and there it's really how do you manage autonomy of employees versus monitoring like very simply it's a s it's again a simple

conflict but I want you to think about it in this way that when you're in these conversations instead of being like how do we find the compromise to come back to Tris because for me it helped at least me identify where can you have privacy aware surveillance where do you separate requirements over time and space Um this particularly came up over like the pandemic when people were working remotely and the whole environment had changed and people were working from home. How do we how do we make that shift? Where do we put our resources? Last thing I'll say um is there are a couple limitations here. There are a lot of limitations and they really have to do with how

complicated this is. You saw that grid. It's um it's not easy to get to, but I've tried using Tris with genai and it's been pretty good. Like before you had a spreadsheet and now I could say here are the 40 principles. I want you to take this thing and figure out and justify to me which of these 40 principles are in conflict with each other. And then I verify it and validate it and everything else. But it gets me to the solution much faster. Um so the learning curve is less steep. Still there's some still some steepness there. And then you need to have the context. You need to know the domain to a to to a certain extent

or be working with domain experts. So in some of those cases, I'm not an expert on ICOT, but I knew the people who were and I could ask them about the separation and the Purdue model and is it because of this and and so same thing with software supply chain. I'm not a dev or an engineer, but I could validate those. So, the last thing I want to leave you with and encourage you to think about is what's next. What's coming next? We're already in it. We're already in the transformative moment around AI. I don't know if you're feeling it. I'm feeling it from a lot of people. Some fear, some uncertainty. They don't know how to think about

what's coming next, how it's going to affect what's happening. This to me is a bit of a map. This is the muscle that at least I've built and I think you can build you can build in being resilient and minimizing surprise around huge huge developments. One of the ways that Tris is used is on forecasting trends and changes in technology. I didn't even talk about that here. That's a whole separate conversation. This is just how do we prepare for this moment. AI is going to accelerate conflicts and people are going to try and use conventional ways to do it. This is a moment when this is needed. So what I invite you to do is to find

the conflict and to really understand it. Use empathy and compassion to understand it. Try out this methodology. It's not as hard as it seems. Air conditioner, wrench, keyboard, your phone, ketchup. These are all things that have been innovated, improved through this methodology. Thank you so much for being here.

Welcome any questions. But if you're timid, come on up after I'll be here. But yeah, questions. >> Oh, we're on time. >> Oh my gosh, we're over time. >> Yeah, but >> wait, I thought I said 12:20. >> No, no, you're my fault. Don't worry. >> It's not my fault. Haha. >> But if we could do Q&A in the hallway. >> Sure, we can do Q&A in the hallway. No problem. >> Q&A in the hallway. >> I was trying to be that person. >> No, no, it's [Music]