BSides Calgary Live Stream

with pressure igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. >> Making a statement. Bringing that bang. >> Shaking the pavement. This is the intensity. It might get hectic. >> Bring on the energy.

Call me with pressure igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. >> Making a statement. Bringing that bang. >> Bringing that bang. >> Shaking the pavement. >> This is electric. Feel the intensity. It might get hectic. >> Bring on the energy. >> Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy.

Feel the energy. Here it comes. This is that. This is that flame. This is that move to switch up the game. You feel the pull. This is insane. You feel that energy ping your veins. This year is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the wave. This is the wave. Making a statement. Making a statement. Bringing that bang. Bringing that bang. >> Shaking the pavement. Shaking the pavement. This is electric. This is electric. Feel the intensity. The intensity. It might get hectic. It might get hectic. Bring on the energy. >> Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see.

We got it. We got it. Here it comes. Feel the energy.

Feel the energy. Here it comes. Feel the energy.

Feel the energy. Here it comes. Feel the energy.

Feel the energy.

Feel the energy.

Hallelujah.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

I think I'm losing it. Yeah, I think I'm losing it. Or maybe it's been gone for a long time. Long time. too long. I think I'm losing it. Yeah, I think I'm losing it. But baby, it's just

I know we're losing it. Yeah, I know you're losing it. But baby, it's been a long time. Long time. Just a long time. If that's the case,

Cuz maybe just we don't need it all.

I think I'm losing it. Yeah, I think I'm losing it. Or maybe it's been for a long time. Long time. Long time. I think I'm losing it. Yeah, I think you're losing everything. Baby, it's just

cuz when you think you got it, I am out of time for maybe

Is it so bad? Is it so bad to be losing my mind? I think I'm losing it. Yeah, I think I'm losing it. that is just

cuz when you think you got it might be out of time.

in my mind the way I'm

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat.

Heat.

Lurking through the shadows. Yeah. Only come out for battle. Yeah. dominate. When I come out, they like head first. I go first. I'm a Tommy gun. I ain't none of them. I'm a one of one like a 100 ton. You want to handle it? Well, let's start now. I'm on a long run. Let's go, baby. Blood shed. That's all. I'm starving. Let's go crazy on one knee with the brakes. I beat your brakes off like an astronaut when I take off. Looking through the shadows on the way. Looking through the shadows every day. Looking through the shadows. Looking through the shadow. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows.

Through the shadows. Only come out for battle. Only come out for battle. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Only come out for battle. Only come out. Only come out for battle. You don't want no problems with a giant. >> I don't do nothing that is comp. Not at all. >> All my rivals, they me just like a ty. I then I go get it. My heart is ironing. Conquer lands in my hand. I'm nothing like a man. Lurking through the deep to see the promised land. Conquered in divide build and destroy. Lurking through the shadow just to feel.

up.

Up to you up.

Heat.

Heat.

Lurking through the shadows. Lurking through the shadows. Her purple heart and battle. Lurking through the shadows. Lurking through the shadow. Only come out to battle. Fear

coming with pressure igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. Making a statement. >> Bringing that bang. Bringing that bang. >> Shaking the pavement. This is electric. Feel the intensity. >> It might get hectic. >> Bring on the energy. >> Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. >> Feel the energy. Feel the energy. Here it comes. Feel the energy. This is

that move. This is that flame. This is that move to switch up the game. You feel the pool. This is insane. You feel that energy pulsing your veins. This year is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the wave. Making a statement. Bringing that bang. Bringing that bang. >> Shaking the pavement. Shake the pavement. This is electric. Feel the intensity. The intensity. It might get hectic. It might get heavy. Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. >> Feel the energy. Feel

the energy. Here I come. Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy.

Lurking through the shadows. Yeah. only come out for battle. Yeah. Dominate. When I come out, they like head first. I go first. I'm a Tommy gun. I ain't none of them. I'm a one of one like a 100 ton. You want to handle it? Well, let's start now. I'm on a long run. Let's go, baby. Blood shed. That's all. I'm starving. Let's go crazy on one name with the brakes on the brakes. I beat your brakes off right now like an astronaut when I take off. Lurking through the shadows on the way. Looking through the shadows every day. Looking through the shadows. Looking through the shadows.

Looking through the shadow through the shadow. Through the shadows. Through the shadow. Only come out for battle. Only come out for battle. Looking through the shadows. Looking through the shadows. Looking through the shadow. Looking through the shadow. Only come out for battle. Only come out. Only come out for that. You don't want no problems with a giant. >> I don't do nothing that is comp. All my rivals, they fe it, then I go get it. My heart is conquer. I'm nothing like a man. Lurking through the deep to see the promised land. Conquered in divide. Build and destroy. Lurking through the shadow just a feel of

Heat. Heat.

Lurking through the shadows. Lurking through the shadows. A purple heart and battle. Lurking through the shadows. working only come out to b

Heat. Heat.

quite amazing. I heard lots of great great feedback. People were very patient with us as we had our bit of growing pains and little things here and there, but just want to say thank you. Um, if there's anything with scheduling that you're you're looking for and you're not finding, just make sure to track down all the volunteers. We'll try to get you some answers in case we've we've misplaced anything. Um, couple things just to note. CTF is still running, so some really really uh great interest in that. This might have something to do with it. We'll see. Um, if you haven't been playing yet. So, be great to see the CTF. Uh, a couple other things is we

will we have some uh swag from previous years that for uh $10 uh donation we will be gladly handing out swag from previous years. It's a bit of a way for us to keep costs down. Especially what we try to do is we try to subsidize the student tickets so that they can come here for for less than what it actually costs for us to to have a ticket per person. So, if you have that uh in your mind, we're going to be doing that uh after the keynote here and we'll start running that throughout the afternoon. Um other things that we have is we do actually run door prizes. So, we will be pulling door prizes again just after

this uh time and we will uh just at the the right at the front doors of the grotto. Um so, on the basement there, we will be having the the prize draws listed. So, if you do have something, your chances are fairly high of getting something. Make sure to check past there, see if you did win something. We always want to make sure that those door prizes go home because they're they're bit geeky, bit techy, and we think it's got a little bit of a bsides flare to it. Um, thank you again for being here. This this community is just amazing. I thank you. I attendees, everyone that that it takes to push to make this go,

it's it's because of you why we're here, and this is why we're here year over year. Um, I don't normally like to stand in front of the mic for too long, so I am going to hand it over to someone that's much better at this, James Bashard. Over to you. Thanks so much, James. Uh, good morning everybody. Um, so my name is James Buchard. I'm the chief information security officer at Enbridge, local company headquartered here in Calgary. And um, just wanted to come and provide my level of uh, thanks and support to everybody coming out here. As a local security leader, I can't tell you how much it matters to see um, all of the security

professionals coming together. This is the talent pool that will help protect critical infrastructure, protect all the organizations that matter so much to us and the society that we enjoy. Um, and I can't thank all of you enough for coming out here, uh, learning something new. Um, but most importantly making the connections that I very confident that you're going to continue to foster in the years to come. Uh the theme of the panel that I'm going to be introducing today is around community and I can't underscore how important community is for everybody here. Um in my role two years ago, three years ago, we saw that there was a massive need for a sense of community in the energy industry. And we

formed what we call now ESAC. It's the energy security technology advisory committee. It's a bit of a mouthful, but it's grown to be 30 companies wide across Canada at a federal level, connecting nuclear, gas, electricity. Um, and it really allows us to move quickly on trading ideas, keeping people in touch, uh, learning from each other, sharing talent. Um, and I think that's a lot of the themes you're going to hear from the panelists here today because, uh, they're all running a sense of community in their own sides. And a huge thanks to the organizing committee and all of you for coming out here. I can only imagine the amount of work that it takes to put on something like this. Um,

so really just wanted to emphasize how much I appreciate the founding the founding committee on setting this all up and great to see all of you here. Um, I was talking to Doug. It's probably been 20 years since I did a capture the flag. That's right. I did do a capture the flag back in the old days. We don't have ne next to the cool gadgets and and toys that you all have here now. But uh um I know everybody here in this room could likely do circles around what I could do back in the day. Um with that, I'm going to introduce um our panelists and uh they can all come up after I'm

done so it's not awkward sitting there by themselves. But uh the first is Nejo. So Nejo has a passion for connecting people. I know Nejo from way back when. Um, and she was actually a founding member of Bides Calgary and has now gone out and started up Bides New Orleans. So, lots of great things happening south of the border. And thank you, Nasia, for being here. Uh, second, we're going to be joined by Amanda Lockhart, CEO of Canadian Cyber Collective, and her role is to help make sure that we're ready for, you know, the cyber workforce of the future and the talent that's coming forward for all of us. Uh, Dallas Bob hails from Saskatoon, a place in my

heart. I'm a prairie kid myself. Um, you know, Saskatoon's growing to be a pretty major tech hub and continues to influence some of the decisions that are coming out in Canada as well. So, in addition to his day job, he's an active community leader running a local OASP chapter and Bside Saskatoon as well. And then finally, we got Steve Porter, very successful businessman, many years of experience here in Calgary running the CEO of Securet. Um and it was prior to that was a uh active HTCIA conference uh member and organizer from uh years past. So with that I'll welcome all the panelists to the stage here and Doug Lease our very own Doug Lease is also a

founding member of Besides is going to be uh moderating it for everyone. Thanks everybody.

>> All right. Well, thanks James. That's great. Um, and you know, a lot of people don't get what it takes for uh for industry and stuff like that to come together. So, you know, it was kind of spur of the moment, but James said, "Hey, could I?" I was like, "Yeah, but make sure you talk about ESAC because, you know, we're getting companies together to do stuff and things like that." That's different, you know, than the government say. So, and I think when we wind up with uh that sort of perspective of businesses coming together that it enables some of this other stuff, you know, and it's the businesses that run this country that make that economy. And

without that, none of us have anything else. So, I I think it's really admirable. And like people say, I'm busy. this guy's not busy compared to me, right? And uh still has time for that. So I I think we we have to applaud the leaders that are making time in their companies for the other duties as required in these volunteer things. So I I really do appreciate that. So maybe do one more hand for that. So if your boss lets you come to bsides and you're still getting paid, that's what support looks like, right? So all right. Um, thanks. So there was also talk about CTF. Um, there was a lot of work that went into

these badges. Please play with them. This is a thing we started last year where you had to connect the badges together. This that was complicated and expensive. So this year we moved the electronics to those Lego things you see around, but there's a machine learning AI in those. So yes, go try and cheat the AI. Go for it. Like we're here to hack stuff. But it's multiffactor, so it doesn't matter if you cheat it, you're still not going to get in without doing at least some of the work. But yeah, it's all there. There's video games online and the whole bit. Um, but yeah, there's a whole Spyroraph station down in the CTF, like seriously old school.

when we've toured this site, they said, "Oh, it's kind of like you're in your mom's basement playing Spyroraph." And this is what happens when you do stuff like that. All right. And there's like a thousand points on the board just for that, just for playing with the badge. So, don't be shy. All right. So, I could talk about CTF all day and I probably will come find me. But I'm going to start with the panel. First off, thanks for everybody for making the trip cuz some people came from a long way away and almost you're out of Toronto, I think. >> Me? Edmonton. >> Edmonton. Okay, that's not so bad. So, but uh yeah, and Dallas, of course,

another Saskatoon guy. So, I did did my time in the prairies, too. So, I'm going to split this um up into um two first questions. And again, I'm not going to get everybody to repeat it and I sort of make stuff up as I go along and they they're telling me they're going to ask me questions. I'll just like no comment. However, all right. Um I want to first talk about what on the demand side what uh someone given up their personal story about motivation. How did you get here? Whatever. So here's the question. Was there some event or something you saw that made you think the security community you were connected with needed something more than informal networking

or occasional meetups? So I'm just who wants to answer that? Like Steve, you had an idea and I think Njo you might have one too. So maybe Steve you can start. Uh >> yeah thanks Doug. Um, >> okay. >> So, as James had mentioned, uh, I previously with along with Doug, I actually was part of the HTCIA community here, uh, when I first moved to Calgary about 20 years ago. And that was a really cool conference and my first exposure to it and first exposure to HTCIA. Sorry, the conference was the trilateral right? >> Trilater way back, >> my bad. 20 years. >> No. And then there was it was all there. Yeah. >> So, we did that for a few years. um

really found there's some great value in it, the sense of community and everything. And that that conference unfortunately ended up kind of going away after a while. Um I continued doing conferences, bumping into Doug all over the North America here and there and we ended up at this one down in Louisville, Kentucky called DerbyCon. And anybody here been to Derby Con when it was running? Okay, we got one, two, maybe three. Okay. Uh it honestly besides besides Calgary of course one of the best conferences I had ever attended and we went multiple years but inside of B or inside of Derby Con there was another mini con called Bourbon Con. Now I'm pretty sure I don't have to explain

what that's about. But I'm not a bourbon drinker. So I had to find myself a mentor and I bumped into this guy. Do you have a next slide? Maybe just a quick little image here. Some of you might know him. Uh, and it may not happen. >> Not happening. Okay. All right. >> So, >> plan survives contract. >> Um, I actually went to went into Bourbon Con and popped down in a chair and turned next to me went, "Oh, you're Jack Daniel, aren't you?" So, the one of the original founding members of Bsides globally, well, globally now, was sitting in a chair next to me. And I was like, "You can probably tell me about

bourbon right?" And he kind of chuckled a little bit. He's like, "Yeah, I can lead you through the next couple of hours if you survive." Like, "Okay, I'm game." So, we I had the great great pleasure of sitting there for three hours talking to Jack Daniel and learning all about bides and bourbon and headaches the next morning and just really getting intrigued by the whole Bides community. I mean, it was fairly new at that point. He gave the whole origin story. Um, I kind of messaged him probably two or three weeks after that after I came back and went, "Doug, I've got this crazy idea for Calgary. What do you think?" And I think your response was,

"Do you have to [ __ ] ask me?" So, we started putting together a group here and I reached out to Jack. I got the rights to the conference and sometimes that's just as easy as writing an email and um, we started this whole thing up. San Na was part of it. Um, some of the other founding members, um, I think some of you are here today, but, uh, Quinn Kramer, Henry St. Louis, Ken Instster, um, AJ Lease, even Natalie Lane. Yeah, there she is. Okay, so we had a rather large group. And here's the fun part. We said, "Hey, let's do this in three months. Um, start the stress here." So we had a great partner. We worked

with state that first year. Um but the timeline they gave us was we can do it in this window. I'm like we can do this in three months. What does it take to put on a conference? >> None of us knew. >> Yeah. >> But we made it happen and it was it was a great success. We actually had Jack um actually came up and keynoted that year for us along with Chris Nickerson, uh Dave Kennedy, Deviant Olm, like we had a who's who of worldclass keynotes and we didn't know how how badly we were setting the bar for ourselves on that. So, but we've continued to do it year over year and the whole origin of it

really was we wanted to replicate that trilateral type of feeling and uh seeing this now 10 years later I mean this is fantastic you know uh great venue great people great competitions and challenges great sponsors of course and uh we hope to see this continue for another 10 years so um I will say if anybody's interested in joining a committee come talk to us right we're always looking for people help and fresh ideas and we've had a couple new people join us this past year and uh it's it's nice to get different perspectives on it. So, uh if you're interested in a lot of stress, uh not as short a time frame anymore, we do plan a little more in advance. Um

please come talk to either Doug, myself, or anybody else on the committee. Thanks. >> All right. Uh N Joe, um community can mean a lot of different things. It can mean hiring, learning, belonging, reputation, local capability. So, because you've done this twice now >> because you're a repeat offender down in New Orleans. >> Four times. Four times in New Orleans. Twice here. >> All right. Well, we won't go into the other part of your offender part. That's okay. Don't worry. There's some other stories, but they're left unsaid. However um when you first start a new community, what what is was there a consistent problem or like what were you trying to solve for people? >> So, and I just want to say thank you

James and team because I know how painful this process is, but then I see all these faces and this is my resume in this room. It's really amazing. Um, when I moved to New Orleans, what I didn't appreciate about Calgary was how amazing this community is. I'm going to try not to cry, but I'm a crier. Um, it's this community is really special. And I've traveled around the world. I've been to other bides. This is very, very unique. I didn't understand that when I moved to New Orleans. And I was like, where are all the people? And I moved there in 2019. COVID. There was a besides that started in 2013 and that group busy crazy smart

forensics um people and it was Steve Biswanger and I don't know if he's in the room but he had this great idea why don't you take over Bides you know how to build communities you've done it um I was like yeah that's a great idea so there's a big difference of planning a conference when you live in a city you know all the people you know all the businesses you know all of the technology partners you know all the people, you know, the CISOs. I go there, I know zero people, and I'm like, "Oh, this was this is hard." But I'm like stubborn. I'm from Saskatoon. I mean, when it's minus 40 or 50, like, you're

tough, right? And so, it's important. And I've been I'm never going to replicate Calgary. I get that. It's New Orleans. I call us the feral street cats. Um, you know, the hurting feral cats is different than, you know, pets. But it is important and years ago I mean I sort of saw what we were doing here and bringing law enforcement together whether that's um Calgary Police Department or CESUS uh RCMP academia students public private sector bringing all that all together. Tim McCree was like privacy and security why can't we get all along and that's always stuck with me so I'm trying to build that there and it's painful but we just actually two weeks ago May 12th had

bides in New Orleans and that matters finding people to show up for the community so the fact that there's people in this room there's people that have been carrying this torch for 10 years is remarkable I never knew that story about Jack and Steve and I've known Steve for 15 years, but they had an idea and I'm like, I'll get behind that. And you're right, there are stories. There's stories of Vegas and there's all sorts of things, but um >> want me to tell that one? >> No, we're not telling. >> No, we're all crooked. >> But thank you, Stephen, for email transfer and we're not going to talk about the wall of sheep. That's okay. Um, so the

fact that this community is here and you've got CESOs in the room and you've got students, um, thank you for being here and showing up for your community. I know you're going to learn lots. I I intended to leave at 7:00 yesterday and I think I left at 10. Um, it there's so many great conversations and you're going to find probably if you're looking for opportunities, you're going to find your next opportunity in this room. >> I think so. Yeah, I do. So, I'm going to pivot here. Um, we're here on day two. There's a lot of content to come yet. And we had a nice social event last night. Thanks very much to Honeywell and

Secured Net for making sure we not only had a liquor license, but there were drinks. Like, yeah, I think just leftover from the DerbyCon days was like, "Yeah, we got to set a bar record. It's important." So, we're feeling really good right now. and but everybody's kind of alluded to it that you know it it's a lot of work to get here. So I'm going to pivot to the one person who has a business doing this now too. But to be completely transparent, this is like a new thing because she used to do it for free just like the rest of us. So you know do not judge. I I'm I'm interested in the experience.

What is the thing that people underestimate about building a cyber community at real scale? And like let's we're grown-ups here. Let's talk about the realities of money, volunteer risk, venue risk, insurance, sponsors, burnout, all all the >> everything. >> And do it in less than five minutes. Yeah, I know. I know. I'll talk really fast. >> How long can we Okay. >> Um Okay. Thank you for having me. First of all, it's um an honor to sit with all besides people up here, especially because my business is very new. Um I just started Canada's security collective last July, so not even a year. Um cyber was always meant to be a part of that, but my experience is

mostly in physical security. So, one thing that really surprised me was there were two groups of people that saw what I was doing and instantly wanted to get involved, and that was cyber professionals and Ottawa. Um, so interesting. Um, but I I've been doing kind of the crash course on on meeting everyone in the cyber world because it it's always been important to make sure that I'm connecting everyone. So connecting physical, cyber, um public service nonprofits and it's it's a lot. Um my creativity has been unleashed. So there's so many things I want to do, but we started with conferences because if no one knows who you are, then they're not going to be, you know, coming for the things that

aren't conferences. Conferences are a bit of an easier way to get people to try something new. Um change is scary. You don't know where it is, what it'll be like, who's speaking, who's attending, what the parking will be like. So, we started with conferences. Um and that's where you run into a lot of the problems you just mentioned. Um it's very expensive. Uh which is okay. I knew I was investing um into the company, but I guess maybe going from volunteering to throw conferences and having, you know, a bank account you can play with that's not your own, which is still challenging, especially a nonprofit where you're limited to what can be in there. um to doing it myself.

Yeah, the conferences, especially the first year, the first year you're throwing a conference and people don't know who you are, it's going to be smaller. It's going to be harder to get speakers and harder to get speakers that are um you know, going to bring in attendance because maybe people don't know who they are. Um, and then balancing out the speakers that everyone's seen at all the industry events who who you know will leave that stage and everyone will be happy with bringing in new people uh that maybe want to start public speaking or start um being visible, build up build up their own personal brand. Um so that can be challenging. The attendees can be

challenging. The sponsorships can be challenging. Um the burnout is real. Luckily, I came from a very, very busy job before this. Um, so I kind of slowed down and then wrapped back up. But that three weeks, how has your last three weeks been? >> I don't remember. >> Exactly. So, three weeks before any event, um, you basically can't do anything except for work on that event. Every all every spare minute is making sure that this is perfect. Um, so burnout can be very real. taking, you know, a break after. Um, what else did you ask about? >> Uh, no, I I think we >> I covered it. >> Yeah, I think I think we've covered it

for sure. And, uh, I'm going to, no, no pun intended, I'm going to go to the Prairie Guy and ask a grassroots question. All right. Um, so there's plenty of uh, conferences around these days like like Amanda just mentioned, and they're at all price points, right? like we're at the very low entry level at you know hundred $200 area and I was at S4 courtesy of a vendor ticket this year that was 2,000 I think or 2500. Yeah, it's not cheap. And >> and Miami and then if you actually want to eat in Miami that's like a whole second adventure in expenses. So I still haven't submitted those. So James still here? No, I think we're good. All right.

Good. All right. Um, so Dallas, um, and I just came back from Google Next, so I have the very recent 20,000 generic conference member. What does, and you've done this twice now with OAS bandwidth besides like what does a grassroots conference provide the community that you really can't find anywhere else? like there's a real difference between being a member of a local and one of you know just another attendee. >> Yeah, it's a good question. I think with Saskatoon being as small as it is, we've only got about 300,000 people there. The communities are very small. It actually makes it kind of hard to find them. Um, we have a lot of software companies in

Saskatoon and unfortunately software isn't as uh security isn't as important at a lot of software organizations as it maybe should be. Um, so it it was really really hard to find other security professionals in Saskatoon. Um, so I think a conference of our size, we're about 150 people a year, you can actually interact with everyone who's at the conference. It's not an unrealistic thing to actually have a brief 30- secondond conversation with everyone who's in the building if you'd like to. Um, so I think being able to build that up and build those communities and have them introduce the people who are on the edges and we can continue to expand that over time to new groups. I mean that

that was one of my driving motivations behind OASP was that I come from a software development security background. I work with a lot of software developers, but they don't know what Bsides is. >> Um, they're not IT and network folks that have seen Bides. So, OASP is my attempt to maybe draw those two communities together and get more communication and collaboration between those two groups because, as I said, most of Saskatoon's IT and tech infrastructure is software companies. >> Yeah, it's surprising. I ironically ran into a guy at a previous Google conference from Saskatoon and he was telling me I was like, "Wow, really Saskatoon?" You know, so I appreciate uh appreciate that. And uh yeah, I think it

what really hit home was when you say that introduce people on the fringes and not that they're weird. I mean, it's just they're new in the industry like, "Hey, have you met X and Y?" Right? True story. I'm down in Vegas for three days, Google Next, and you've got this giant lunch hall and everything else, and I've got one of my my staff with me. We had one interaction with a stranger that actually met up that turned into they went off and, you know, like fellow data science people and off they went and had a good time and complained about the fact that nobody gets what they do. Uh, which is good. Like you need to feel

like you're not alone in the struggles that you're struggling because these are tough jobs. And like you say, sometimes you're like the only person or one of two people in the company that are doing this. So >> yeah, exactly. The biggest security organization I've ever worked in is two people. >> Yeah, that's really And you build software for medical companies. I can't imagine the dichotomy or what is that? The the balance is off. There we are. there's a disruption in the force. All right. So, let's uh let's go to one of the let's go for the business side of running the conference. And I'm going to skip one uh because I remember this from our very

first conference. We had a conversation about certain adult entertainment and hacking it. So, >> and we had one and we had one guy that was like, "I'm not going to participate in this committee if this guy's on the stage." And it's like, "All right, so how do we," and Amanda touched on it, too. You need to bring people in that are going to get some attention. And I maybe I get two votes. Keep it short because I get more questions. Um, how do you handle the tension between this can be an open open and welcoming conference, you know, and then the fact that there's a lot of people in this industry that are really good at

their jobs and as a result, they are very opinionated about how things should be and they may be great for the stage. They may be brilliant researchers, but maybe they're not the best host guest. So, how do you balance that off? So, I don't know. Two people. Who? Yeah. You want to go? >> I'll go. >> Yeah. >> Um, so starting something new when there's already so many cool things happening. Um, a lot of people said, "Are you sure you want to do that? There's already all these other things. Maybe it'll be too similar. Maybe it'll be too different. maybe bringing in, you know, a lot of students or uh bringing in nonprofits, people that your sponsors

don't see a lot of business with that might um cause problems. And my answer has stayed the same. And I hope it comes across properly because I mean it with love. But if someone doesn't like what I'm doing, that's okay. There's enough spaces out there that they can go elsewhere. if they come, they don't like it, they want to say something, that's cool. Say what you need to say, you know, I've had it um at our Calgary conference, we had some interesting questions come up. Um and I was ready to jump on the mic as needed, but it's okay if they don't like it or it's too open or too closed or too whatever it is

because there is enough space that people can find what they're looking for. So partnering with other, you know, conferences, other organizations, doing something similar, I think is important because then you're connecting all of your networks and people can find what they're really looking for. >> Awesome. One more. >> Uh, yeah, sure. Um, so I think that being part of the Bsides organization or Besides community, uh, we have a little bit of a different advantage over what Amanda's doing is that Bides gives us a guideline or handbook that we kind of work from. And Dallas, I'm sure you speak to this as well. So, we kind of have our set of rules that we can follow

and we can, you know, very specifically quote, well, for sponsors, for instance, like here's how we need to operate, here's what a sponsorship means. It's a sponsorship, not a vendor. It's it's the support, not to sell. You know, that kind of stuff. Or for presentations, there's certain loose guidelines around don't have things that are too offensive or that's, you know, intended to aggravate people, whatever. But the the framework is there for us. So, it makes life a little bit easier in our community at least. That said, we all know you never satisfy 100% of your audience and we're all adults. Uh, if there's a talk that you think is going to be controversial or you don't

like it, um, I'm going to take the position on and say, well, don't go see the talk. It's pretty simple. We have multiple tracks. Nobody's forcing you there. We're not taping you down to the chairs for audience. Um, you know, you you have choice. And um, you know, I I always make the example like when I go to um, go out for a club or an event, it's like, sorry Calgary, but I don't like country music, so I'm not going to go somewhere they play country music. It's pretty simple. I don't get offended. I don't go tell the DJ to change it up and put on some metal. Um, you know, like you have a choice. So,

um, you can't please everybody all the time. Certain talks will be a little bit edgy. I mean, that's our industry. That's what we do. That's why we're here. Um, you know, we're not, um, we're not forcing anybody to do anything. So, >> all right. That's uh, yeah, that's good advice. If you don't like it, don't go. Yeah. things your mom said that you kind of go, "Oh, all right." Um, so yeah, you you'd mentioned frameworks and boards. Um, and maybe this might be interesting, a small and then large community. Maybe kind of compare those two. So, Dallas, I'll let you go first. How do you and then have Amanda you follow with and on the other end of the spectrum when it's

big how do you decide when a community effort needs that structure of a board committee policies sponsorship packages I call it the boring business stuff but it is very essential and when did you notice that that stuff started to become a need >> that's a great question I mean for me I honestly don't handle that much of that um besides Saskatoon is really our our organizing committee is four friends. Um and we've kind of just added a person each year. I think we're up to six now. And we're just we all want besides to succeed. We don't really have set roles. We meet once a month. We decide what needs to be done. People take tasks off

the pile and we do that. Um a huge benefit for us in getting started was actually the work that Nicole um has done with Besides Edmonton, >> right? Um, and she legitimately has like a conference in a box in Google Drive that we just stole and swapped logos off of and were able to get up and running with very little effort at all. Um, we got incredibly lucky. One of our co-founders, his wife is an accountant by trade. So, like all of our books and everything we haven't had to worry about. We've been quite lucky in the skill sets that we've had to get going. But yeah, for us the biggest thing is we do this for fun and the second it stops

becoming fun, we we're going to stop doing it. But keeping it nice and relaxed has allowed us to continue on without a whole lot of stress. >> Awesome. And what's it look like on the other end? >> Uh I think all of it is very important. Um because Canada Security Collective was kind of a sudden thing, I started building it a few months before I launched it. Um didn't know if I would ever launch it launch it. Planned to do it as like a side project off the side of my desk, see if it grows, see if people like it. And then boom, July came. It was my full-time effort. We had our first conference in November. Um, I've been

doing a ton of it by myself. Now, when I started to realize I did not have the capacity, the time, the strengths that some of this stuff required, that's when I started to build out um a growth plan, what that board looks like in the future. Um, what it, you know, looks like by city. Um, not quite ready to fill those positions. um not because it wouldn't be beneficial but because I'm looking for very specific people. I'm looking for the right balance of personalities and knowledge and like true ownership of the vision that I also see and that vision doesn't mean you know everything has to happen how I want it to happen. I want

people to come in and bring in perspectives and ideas. So getting there, um I have a few people that I have already talked to and said when the time comes you are I need you with me. Um but letting someone in like I don't know if you remember 10 years ago how hard that is because when you're small like if I let in one person that's 50% of the representation of my company. So, I don't want our, you know, reputation brand to be hurt by letting someone else in, which is my own problem. Um, that I do need to, you know, get over. Um, but yeah, it's it's challenging. >> Yeah, it is >> needed though.

>> It is. And, uh, yeah. So, speaking of needed and challenging, let's do a commercial question. And uh you know we have uh couple business people here that are full-time business. Um companies have quarters and quotas and community building takes years. So you can already kind of see the tension there, right? So N Joe and then Steve maybe you're wrapping up here too. Like what have you learned about asking for sponsorship when the audience is not like a guaranteed steady stream of buyers and decision makers? And I think we've all heard this like how many leads will I get from this? And then the answer is I'm not even going to give you the mailing list of who showed up.

Hello. Uh can I how much can I put you down for? So I mean there's company and there's companies in this room today that have a market cap of $150 billion and so you can probably find three credit cards across three reps to do you know a bronze sponsorship. So luckily if there's any field marketing people that are familiar with besides there's your there's your easy targets. Um, a lot 75% of our sponsorship this year, the reps didn't live in Louisiana. And that's the polar opposite of what Bides is. Besides is by the community for the community. So that was really interesting digging in this year of figuring out you know who are the people who's the community

um you know and I'll provide lists of I went through like who are the top performing companies in Louisiana learning that you know some of the ship building uh is some of world world class in Louisiana um pool corp world's largest supplier of pool supplies all sorts of things like I had no clue right um I'm come from oil and gas I know all the oil and gas companies I don't know anything about Louisiana. Um, so there was a lot of education about what besides means and talking to reps, hey, if you come, don't sit at your booth on your laptop, get into the sessions, shake hands, make friends. Um, and not everybody listens to that. I'm like, why

would you spend $5,000 to be in a room and sit at a table all day? That seems nonsensical to me, but it happens. But I'll take your $5,000. Thank you very much. Uh, come back again next year. um teaching them about um you know have a raffle, have a draw, bring a cool prize like a Raspberry Pi or like a Lego set, like cool stuff. Um so it's been surprising to me that Bides has been around since 2009 and I'm still having to say go to bsides.org. Here's the origin. Even the first year >> 2023, the reboot. Uh yeah, Jack Daniel came out. Luckily, he loves Burban, Sazzarex, and New Orleans. So that was easy. A lot

of people didn't even research who Jack Daniel is. And I'm like, you're in the presence like you're never going to get to spend time with this man in a 200 person environment. So, there's just so much um education around that. And and I live in New Orleans being a hospitality center. There's nothing a facility like this is amazing. We had it at the Intercontinental Hotel. There were chandeliers. Total opposite of what Bides is. But that's all that would take us cuz we are a standalone catering. I didn't even know what that means. We don't have enough rooms. We only had 39 uh 37 nights or something of of hotel rooms. Well, they're a hotel. They want

hotel rooms. I get it. Um so, you've got to find champions. Firemon came in big time, but that's a relationship that my husband had. My husband's on the planning committee. I recommend that and I don't recommend that. That the ser is really hard. Our t-shirt just that was very very hard. Um, and when you're the president, you gota, you know, you've got someone on the team that you don't feel is doing a great job, but you have to like sleep in the same room as them. Um, but those are relationships. Like people think it just magically happens, but it doesn't. Those sponsors are people that have stood up either here in Calgary. Um, Andrew Ginter flew from

Calgary to do a talk for me. That meant a lot to me. Brought books, did a book signing. So those are relationships I've cultivated over 15 years. And so the sponsors that get it are amazing. The ones that don't, there's some that will never get it, but you've got to find those special ones that are like, I love B or what is this? Or I love New Orleans. Like, let's do a QBR here. That's another selling feature. >> Oh, thank you. >> There you go. >> It's it's a you figure out your budget and then you fill in the gaps. And this year, I lied to the team because they're like, "Are we going to be okay?" And I

was like, "No, we're going to have to dig out $6,500 out of each of our pockets." And I just barreled through. And you talk about that three weeks, there's no sleep, your hair falls out. You're like, I'm did I shower this week? Um, but it's the right thing to do. Besides is special and it's different than the big cons and we try to I wasn't at S4 this year but I've been going for a number of years besides ICS uh ahead of S4 and I'm going to these other conferences making friends trying to lure people in trying to bring in that sponsorship um because that's the thing that fuels it and we want to keep the

tickets I love that you guys um try to keep the tickets low for the students because that's extra important as well so >> it's hard. >> Awesome. Well, just checking the clock. I think we're gonna have to I want to do a little wrap up. Um, so maybe I'll start with you, Steve, and one, two minutes and then get each each of you to comment on this. Like what is the most valuable thing that a strong local security community brings back to the employers and the broader region? Um, I mean I can speak on behalf of my own company of course, but just the opportunity to meet people in a stress-free kind of environment. Um, I

call these, you know, sort of ad hoc interviews as I talked to people over to yesterday and today. Uh, I mean last year I think I've had close to 250 people come by the booth and we ended up hiring out of that last year. We've done that every year for the last five years actually. So, in fact, over half of my team has come from Bsides. And that's great because when people meet you here, half the time they don't even know who you are, which is great. There's no false errors. There's no HR. You're not trying to check the boxes. You're not trying to make sure you're on your best behavior. You're just being you. And

that's that's really valuable to me when I look to hire. Um, I can't speak for all companies, of course, but personality means a lot. Passion means a lot. And in a, you know, casual environment like this, you get to see a lot more of that. >> Awesome. >> Yeah. I mean, my answer is very similar. I think that the sense of building community, um, we run a a community Slack as well. So, being able to reach out on Slack to that person you met at a conference who you know did that thing three weeks ago and get his input again um on how to do something without having to spend two or three days of repeating

mistakes that other people have made. Um that's huge. Um my last two jobs have come from people I've met through Bside Saskatoon. Um and I mean it generates revenue for companies legitimately. Um, the company I'm currently working for has done contracts with Tailcraft out of Calgary here because of my relationships that I've built with Tim through Bides Calgary. >> Um, I mean honestly Besides Saskatoon wouldn't exist without Bides Calgary. Um, I mean I'll I'll sign off by saying that like this is >> one of the best cyber security conferences in Canada. Um, >> and it's it's humbling to be on this stage. >> Yeah. No, it's uh and you deserve to be here and I remember it. It comes up

every once in a while when Tim and I talk about it on the podcast and I think we're rattling off all the different ones and it's like the movement's alive in Western Canada. It's uh because we've got Saskatoon, we've got Regina, we Winnipeg and yeah, Steve didn't mention it, but we definitely were blessed by the Vancouver folks helping us out, right? And even the logo that that t-shirt that Njo's wearing, you know, that that's actually the very first t-shirt and the same person that designed the the uh the beat the Vancouver one did the thing and Mr. I hate country music had a thing with the horse and you know so trust me it goes

it's a thing. >> Just thought it was too cliche. Okay. >> Yeah, I know. Right. I'm repeating what they said, community is everything. Um I think going to these types of events, you're you're meeting people, you're understanding what people are doing, maybe different um industry, maybe different vertical, maybe different position, I don't know, but you get to understand more about the security industry. Um you get to go listen to the speakers, understand, you know, get a broader picture of what others are dealing with, what they have to teach you, what you can learn. Um, and yeah, a lot of those relationships really do turn into working together, becoming friends, uh, volunteering for the conferences. Uh, I just, yeah, I

think they're great. It's relationship building. >> Awesome. And last but not least, as I feel >> this year, um, I we didn't have the attendance or the registration, so I was reaching out like on LinkedIn, I think over 400 people, and I had someone say, "Hey, I already have something on my calendar. I'm looking to pivot. And I'm like, clear your calendar. You're going to meet 200 people and you're going to find your next opportunity. And they made a post of what a great day it was. And um that they felt energized and all the talks and all the people they met. And I actually replied with I told you so. So if I know what I'm doing and if

you listen to me, I can help you. So that's, you know, finding your next opportunity, but the skills that you bring back to your own organization, whether that's um you learned how to build your own lab. so that you're learning how to break and fix stuff at home. Um, I love the sluck, you know, asking friends like phone a friend, how did you do this? I've got this problem here. So, it's that whole, you know, community network that's going to help you in your career. >> And that's what Bides is. That's all of what we've built here over the last 10 years. >> Oh, yeah. And yeah, for sure. We're standing on the on the shoulders of

giants. Like you said, Chris and Jack went there. Now, for anybody that doesn't know the origin, who actually knows how Bides got started, a couple, right? My wife, yeah, I've told her many times. Okay. So, back in the old days, all you youngans, there was these things called records and you played music and everything. And when bands would put out songs, the record company would get involved and make them do something. And usually they were the record company was betting on the song they were going to was going to move this band. And sometimes the other side, usually it was just a throwaway cut, was better than the original. Um, but there was a whole

bunch of people, the who's who of security that were shunned from these big corporate security conferences and it's like this person, you know, and they they were talking amongst themselves go that's [ __ ] We like I man I just come to hear you talk you man I come to hear you talk. It's like what are you doing? and they went to Vegas and they booked a mansion apparently a mansion and then they found a couple sponsors. Nickerson's awesome at social engineering. Uh any chance you could sponsor our conference that he literally just made up on the fly, you know, and they got these two guys, Jack Daniel and Chris Nickerson come up. And I remember

talking with Dickerson >> at the conference the first time and you know they said we're just gonna do this and we're just going to have our own conference. We're going to have it right next door to the big one and we're the better conference is the B side of the record. See where it's going. And I think probably where that came from was from Jack because similar I didn't have a bourbon story because I've been sober over 40 years now. But me and Jack sat in the airport again. I'm coming back from Louisville to go somewhere and we're talking about guitars, believe it or not. And uh you know it's kind of like this is surreal

on the other end. But yeah, it comes into that. So he was a big music buff because he actually knew what a Fender Jaguar was and how cool that would be and everything, right? So and he didn't look like a Kurt Cobain fan. >> I believe Bides was started in response to RSA, wasn't it? >> Absolutely. >> So many people got turned black and Devcon. >> Yeah. And they were just all All the big guys said no to the big guys. So they said, "Forget it. We'll just have our own conference." And here we are. So >> there's what 220 events every year around. >> Yeah. It's it's a global thing now. it is. So, you guys are all part of this.

So, one, thank you very much for coming. But again, and I'm not plugging these cuz we worked our butts off on these. One of the things with the gear is that you have to change. If you've got a red gear, go find somebody with a green or a yellow and talk about what you figured out and together you can get it, you know, move up the price. Like, did I mention there's like a thousand points on the board for these gears? So connecting and stuff and that was something Alex brought a new idea last year right and it's like how do we make people have to work together to solve a puzzle I was like well I've got an idea

and it involved jumper cables and electricity and but it worked and nobody died so you know and we will have some of those uh dev boards available because you have to buy a bunch in order to get a manufacturing run so we do have spare boards if you wanted to take some home for your own home lab, start fooling around with IoT and uh you won't be able to buy the parts for what we want to give the boards away for. So uh with that uh I want you guys to have an excellent second day. I got a jet to the grotto. We got some other stuff going on. Other talks are kicking off now and other

tracks. But I want to give a big hand to the people that have been putting this together all over if we could please. All right. And maybe go out and do go start your own conference or come join ours or something in between. Thanks again everyone. >> Thanks. >> Hey, that was great.

>> All of us are here.

Call me with pressure igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. Making a statement. Making a statement. Bringing that bang. >> Bringing that bang. >> Shaking the pavement. >> This is electric. Feel the intensity. It might get hectic. >> Bring on the energy. >> Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy.

Feel the energy. Here it comes. Feel the energy. This is that move. This is that flame. This is that move to switch up the game. You feel the pull. This is insane. You feel that energy pulsing your veins. This year is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the wave. This is the wave. Making a statement. Bringing that bang. Bringing that bang. Shaking the pavement. Shake the pavement. This is electric. Electric. Feel the intensity. >> It might get hectic. It might get hectic. Bring on the energy. >> Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see.

We got it. We got it. Here it comes. >> Feel the energy. Feel the energy. Here I come. Feel the energy.

Feel the energy. Here it comes. Feel the energy.

Feel the energy.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

I think I'm losing it. Yeah, I think I'm losing it. Or maybe it's been gone for a long time. Long time. Long time. I think I'm losing it. Yeah, I think I'm losing it. But baby, it's just fine.

I know we're losing it. Yeah, I know you're losing it. But baby, it's been gone for a long time. A long time. Just a long time. If that's the case, maybe. Cuz

baby, we don't need it all.

I think I'm losing it. Yeah, I think I'm losing it. Or maybe it's been for a long time. Long time. Long time. I think I'm losing it. Yeah, I think you're losing it. Baby, it's just

cuz when you think you got it, you might be out of time. Feel free.

Is it so bad? Is it so bad to be losing my mind? I think I'm losing it. Yeah, I think I'm losing it. Baby, it's just fine.

Cuz when you think you got it all, you might be out of time.

Just feel

in my mind the way I'm

Heat. Heat.

Heat

up here.

Heat

up here.

Heat. Heat. N.

Heat.

Heat.

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Lurking through the shadows. Yeah. Only come out for battle. Yeah. Dominate. When I come out, they like first I go first. I'm a Tommy gun. I ain't none of them. I'm a one of one like aund ton. You want to handle it? Let's start now. I'm on a long run. Let's go, baby. Blood shed. That's all great. I'm starving. Let's go crazy on one name. Put the brakes off. I beat your brakes off like an astronaut when I take off. Lurking through the shadows on the way. Lurking through the shadows every day. Looking through the shadows. through the shadows.

Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Only come out. Only come out for the shadows. Looking through the shadows. Looking through the shadows. Working through the shadow only come out for battle. Only come out for battle. Only come out for battle. You don't want no problems with a giant. >> I don't do nothing that is comp. All my rivals they me just like a tyant. I it then I go get it. My heart is iron in my hand. I'm like a man. Lurking through the deep to see the promised land. Conquered and divide. Build and destroy. Lurking through the shadow just a fill of

Heat.

Heat.

Lurking through the shadows. Lurking through the shadows. A purple heart and back and back. Lurking through the shadows. Lurking through the shadow. Only come out to battle.

Coming with pressure igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. >> Making a statement. Making a statement. Bringing that bang. >> Bringing that bang. >> Shaking the pavement. >> This is electric. Feel the intensity. It might get hectic. >> It might get hectic. >> Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. >> Feel the energy.

Feel the energy. Here it comes. Feel the energy. This is that move. This is that flame. This is that move to switch up the game. You feel the pull. This is insane. You feel that energy ping your veins. This year is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the wave. This is the wave. Making a statement. >> Bringing that bang. Bringing that bang. >> Shaking the pavement. Shaking the pavement. This is electric. Feel the intensity. The intensity. It might get hectic. Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes.

Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy. Here it comes. Feel the energy.

Feel the energy.

Feel the energy

lurking through the shadows. Yeah. Only come out for battle. Yeah. Dominate. When I come out, they like where from here first I go first. I'm a Tommy gun. I ain't none of them. I'm a one of one like a 100 ton. You want to handle it? Well, let's start. Now I'm on a long run. Let's go, baby. Blood shed. That's all great. I'm starving. Let's go crazy on one name with the brakes. I beat your brakes off like an astronaut when I take off. Lurking through the shadows on the way. Looking through the shadows every day. Looking through the shadows. Looking through the shadows.

Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadow. Only come out for battle. Come out for battle. Only come out for battle. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Only come out for battle. Come out for only come out for you don't want no problems with a giant. >> I don't do nothing that is comp. >> All my rivals they fe it then I go get it. My heart is conquer in my hand. I'm nothing like a man. Looking through the deep to see the promised land. Conquered in divide build and destroy. Lurking through the saddle just to feel

Heat. Heat.

Lurking through the shadows. Lurking through the shadow. Her purple heart and battle. Lurking through the shadow. Lurking through the shadow. only come out to B.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Call me with pressure igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. Make it a statement. Bringing that bang. Bringing that bang. >> Shaking the pavement. >> This is electric. Feel the intensity. It might get hectic. >> It might get Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy. Feel the energy.

Feel the energy. This is that move. This is that flame. This is that move to switch up the game. You feel the pull. This is insane. You feel that energy for your pain. This year is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the wave. This is the wave. Making a statement. Bringing that bang. Bringing the rain. >> Shaking the pavement. Shaking the pavement. This is electric. This is electric. Feel the intensity. The intensity. >> It might get hectic. It might get heavy. Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes.

Feel the energy. Feel the energy. Here it comes. Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy.

Heat. Heat.

Heat.

Heat.

Hallelujah.

Heat. Heat.

Heat.

Heat.

I think I'm losing it. Yeah, I think I'm losing it. Or maybe it's gone for a long time. A long time. Too long. I think I'm losing it. Yeah, I think I'm losing it. But baby, it's just fine.

I know we're losing it. Yeah, I know you're losing it. But baby, it's been gone for a long time. Long time. Just a long time. If that's the case, then maybe

we don't need it all.

I think I'm losing it. Yeah, I think I'm losing it. Maybe it's been for a long time. A long time. Long time. I think I'm losing it. Yeah, I think you're losing it. But baby, it's just

cuz when you think you got it, I made out of time. is load.

Is it so bad to be losing my mind? I think I'm losing it. Yeah, I think I'm losing it. But baby, it's just fine.

Cuz when you think you got it, I might be out of time. Make it baby

just

by the way. Heat. Heat.

Hallelujah.

Hallelujah.

Heat. Heat.

Heat up here.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat.

Heat.

Heat.

Heat.

Heat.

Heat.

Heat. Heat.

Heat.

Heat.

Lurking through the shadows. Yeah. Only come out for battle. Yeah. dominate. When I come out, they like head first. I go first. I'm a Tommy gun. Iin't none of them. I'm a one of one like a 100 ton. You want to handle it? Well, let's start now. I'm on a long run. Let's go, baby. Blood said that's all crazy. I'm starving. Let's go crazy on one with the brakes. I beat your brakes off like an astronaut when I take off. Looking through the shadows on the way. Looking through the shadows every day. Looking through the shadows. Looking through the shadow. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the

shadows. Only come out for battle. Only come out for battle. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Only come out for battle. Only come out for battle. Only come out for battle. You don't want no problems with a giant. >> I don't do nothing that is compliant. Not at all. >> All my rivals, they fe it, then I go get it. My heart is iron. Conquer lands in my hand. I'm nothing like a man. Lurking through the deep to see the promised land. Conquered in divide. Build and destroy. Lurking through the shadows just to feel. Heat. Heat.

Heat. Heat.

Lurking through the shadows. Lurking through the shadow. A purple heart and heart. Lurking through the shadows. Lurking through the shadow. only come out to bath.

Call me with pressure igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. Making a statement. Making a statement. Bringing that bang. Bringing that bang. >> Shaking the pavement. >> This is electric. Feel the intensity. It might get hectic. >> Bring on the energy. >> Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy.

Feel the energy. Here it comes. Feel the energy. This is that. This is that flame. This is that move to switch up the game. You for the pool. This is insane. You for that energy ping your veins. This year is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the wave. This is the wave. Making a statement. Making a statement. Bringing that bang. Bringing that bang. >> Shaking the pavement. Shaking the pavement. This is electric. This is electric. Feel the intensity. It might get hectic. It might get hectic. Bring on the energy. >> Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see.

We got it. We got it. Here it comes. >> Feel the energy. Feel the energy. Here I come. Feel the energy.

Feel the energy.

Feel the energy. Feel

the energy

through the shadows. Yeah. only come out for

dominate when I come out. They like head first I go first. I'm a Tommy gun. I ain't none of them. I'm a one of one likeund. You want to handle it? Well, let's start now. I'm on a long run. Let's go, baby. Blood shed. That's all crazy. I'm starving. Let's go crazy on one name with the brakes off. I beat your brakes off like an astronaut when I take off. Lurking through the shadows on the way. Looking through the shadows every day. Looking through the shadows. Looking through the shadows.

Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Only come out for battle. Come out for battle. Only come out for battle. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Only come out for battle. Only come out. only come out for back. You don't want no problems with a giant. >> I don't do nothing that is comp

then I go get it. My heart is conquering through the deep to see the promised land. Conquered in divide, build and destroy. Lurking through the shadow just to feel a void.

Heat. Heat.

Lurking through the shadows. Lurking through the shadows. Her purple heart and battle and battle. Lurking through the shadows. only come out to b

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

coming with pressure, igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. >> Making a statement. Bringing that bang. >> Bringing that bang. >> Shaking the pavement. >> This is electric. Feel the intensity. It might get hectic. >> It might get hectic. >> Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. >> Just watch and see. We got it. We got it. >> Here it comes. Feel the energy.

Feel the energy. Here it comes. Feel the energy. This is that. This is that flame. This is that move to switch up the game. You feel the pull. This is insane. You feel that energy for your pain. This year is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the wave. This is the way. >> Making a statement. Making a statement. Bringing that bang. Bringing that bang. >> Shaking the pavement. Shaking the pavement. This is electric. This is electric. Feel the intensity. The intensity. >> It might get hectic. It might get heavy. Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see.

We got it. We got it. Here it comes. Feel the energy.

Feel the energy. Here it comes. Feel the energy. Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy.

or heat.

Hallelujah.

Heat. Heat.

Heat. Heat.

Heat.

Heat.

I think I'm losing it. Yeah, I think I'm losing it. Or maybe it's gone for a long time. A long time. a long time. I think I'm losing it. Yeah, I think I'm losing it. But baby, this is

I know we're losing it. Yeah, I know you're losing it. But maybe it's been gone for a long time. Long time. Just a long time. If that's the case, then maybe

just maybe we don't need

I think I'm losing it. Yeah, I think I'm losing it. Maybe it's been for a long time. A long time. Long time. I think I'm losing it. Yeah, I think you're losing it. But baby, it's just

when you think you got it on your baby.

Baby load.

Is it so bad to be losing my mind? I think I'm losing it. Yeah, I think I'm losing it. But baby, it's just fine.

Cuz when you think you got it, I might be out of time. Heat. Heat. Heat.

Make another

way.

Hallelujah.

Heat.

Heat.

Heat up here.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat. Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Lurking through the shadows. Yeah. Only come out for battle. Yeah. Dominate. When I come out, they like first I go first. I'm a Tommy gun. I ain't none of them. I'm a one of one like a 100 ton. You want to handle it? Well, let's start now. I'm on a long run. Let's go, baby. Blood said that's all crazy. I'm starving. Let's go crazy on one brakes. I'll beat your bra like an astronaut when I take off. Lurking through the shadows on the way. Looking through the shadows every day. Looking through the shadows. Looking through the shadow. Looking through the shadows.

Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Only come out for battle. Come out for battle. Only come out for battle. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Only come out for battle. Only come out. Only come out for battle. You don't want no problems with a giant. I don't do nothing that is comp. I it then I go get it. My heart is ironing. Conquer lands in my hand. I'm nothing like a man. Lurking through the deep to see the promised land. Conquered in divide. Build and destroy. Lurking through the shadows just to feel a v.

Heat. Heat.

Heat. Heat.

Lurking through the shadows. Lurking through the shadows. Her purple heart and back. Lurking through the shadows. Lurking through the shadow. only come out to bath.

Call me with pressure igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. Making a statement. Making a statement. Bringing that bang. Bringing that bang. >> Shaking the pavement. >> This is electric. Feel the intensity. It might get hectic. >> Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy.

Feel the energy. Here it comes. Feel the energy. This is that. This is that flame. This is that move to switch up the game. You feel the pool. This is insane. You feel that energy ping your veins. This here is groundbreaking. Shake up the foundation. This here is game changing. This got the earth shaking. This is the wave. This is the wave. Making a statement. Making a statement. Bringing that bang. Bringing that bang. Shaking the pavement. Shaking the pavement. This is electric. This is electric. Feel the intensity. >> It might get hectic. >> It might get heavy. Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see.

We got it. We got it. Here it comes. >> Feel the energy. Feel the energy. Here I come. Feel the energy.

Feel the energy.

Feel the energy. Feel

the energy.

Lurking through the shadows. Yeah. only come out for

dominate when I come out. They like head first I go first. I'm a Tommy gun. I ain't none of them. I'm a one of like a 100 ton. You want to handle it? Well, let's start now. I'm on a long run. Let's go, baby. Blood shed. That's all great. I'm starving. Let's go crazy on one name with the brakes on the brakes. I beat your brakes off like an astronaut when I take off. Lurking through the shadows on the way. Looking through the shadows every day. Looking through the shadows. Looking through the shadows.

Looking through the shadows. Looking through the shadows. Through the shadow. Only come out for battle. Come out for battle. Only come out for battle. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadow. Only come out for battle. Only come out. Only come off. You don't want no problems with a giant. >> I don't do nothing that is comp

then I go get it. My heart is high. Conquer in my hand. I'm nothing like a man. Looking through the deep to see the promised land. Conquered and divide. Build and destroy. Lurking through the shadow just to feel a void.

Woo!

Lurking through the shadows. Lurking through the shadows. Her purple heart and back and back. >> Lurking through the shadows. Freaking booty shot only come out to b

Heat.

Heat.

Heat up here.

Heat.

Heat.

Heat. Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Call me with pressure igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. >> Making a statement. >> Bringing that bang. >> Bringing that bang. >> Shaking the pavement. >> This is electric. Feel the intensity. It might get hectic. >> It might get hectic. >> Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy.

Feel the energy. Here it comes. Feel the energy. This is that move. This is that flame. This is that move to switch up the game. You feel the pull. This is insane. You feel that energy for your pain. This year is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the wave. This is the wave. Making a statement. Making a statement. Bringing that bang. Bringing that bang. >> Shaking the pavement. Shaking the pavement. This is electric. This is electric. Feel the intensity. The intensity. >> It might get hectic. It might get heavy. Bring on the energy. >> Unstoppable. We got it. We got it. So powerful. We

got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy. Here it comes. Feel the energy.

Feel the energy.

Feel the energy.

Hallelujah.

Heat. Heat.

Heat. Heat.

Heat. Heat.

I think I'm losing it. Yeah, I think I'm losing it. Or maybe it's been gone for a long time. A long time. Long time. I think I'm losing it. Yeah, I think I'm losing it. Baby, it's just

I know we're losing it. Yeah, I know you're losing it. But maybe it's been gone for a long time. A long time. Such a long time. That's the case. Maybe

cuz maybe just we don't need it all.

I think I'm losing it. Yeah, I think I'm losing it. Maybe it's been for a long time. A long time. Long time. I think I'm losing it. Yeah, I think you're losing it. But baby, it's just

when you think you got it on your baby. Feeling my mind feeling

so bad. It's so bad to be losing my mind. I think I'm losing it. Yeah, I think I'm losing it. But baby, it's just fine.

Cuz when you think you got it, I might be out of time. Heat. Heat. Heat.

Make another

way. I'm losing

Hallelujah.

Heat. Heat.

Heat up here.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat

up

lurking through the shadows. Yeah. Only come out for battle. Yeah. Dominate. When I come out, they like where from here first I go first. I'm a Tommy gun. I ain't none of them. I'm a one of one like aund ton. You want to handle it? Well, let's start. Now I'm on a long run. Let's go, baby. Blood shed. That's all great. I'm starving. Let's go crazy on one name with the brakes. I bet your brakes like a a take off. Lurking through the shadows on the way. Looking through the shadows every day. Looking through the shadows. Looking through the shadows.

Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadow. Only come out for battle. Come out for battle. Only come out for battle. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Only come out for battle. Only come out for battle. only. You don't want no problems with a giant. I don't do nothing that is comp. I it then I go get it. My heart is conquering through the deep to see the promised land. Conquered in divide, build and destroy. Lurking through the saddle just to fill a void.

Heat. Heat.

Lurking through the shadows. Lurking through the shadow. A purple heart and back. Lurking through the shadows. Lurking through the shadow. only come out to B.

Call me with pressure igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. Making a statement. Bringing that bang. Bringing that bang. >> Shaking the pavement. >> This is electric. Feel the intensity. It might get hectic. >> Bring on the energy. >> Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy.

Feel the energy. Here it comes. Feel the energy. This is that move. This is that flame. This is that move to switch up the game. You feel the pool. This is insane. You feel that energy ping your veins. This year is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the wave. This is the wave. Making a statement. Making the statement. Bringing that bang. Bringing that bang. Shaking the pavement. Shake the pavement. This is electric. Electric. Feel the intensity. >> It might get hectic. It might get hectic. Bring on the energy. >> Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see.

We got it. We got it. Here it comes. >> Feel the energy. Feel the energy. Here it comes. Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy.

Lurking through the shadows. Yeah. Only come out for battle. Yeah. Dominate. When I come out, they like where from head first I go first. I'm a Tommy gun. I ain't none of them. I'm a one of one likeund. You want to handle it? Well, let's start. Now I'm on a long run. Let's go, baby. Blood shed. That's all great. I'm starving. Let's go crazy on one name with the brakes. I beat your brakes off like an astronaut when I take off. Lurking through the shadows on the way. Looking through the shadows every day. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the

shadows. Looking through the shadows. Only come out for battle. Only come out for battle. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Only come out for battle. Only come out for battle. only come out. You don't want no problems with a giant. I don't do nothing that is comp

then I go get it. My heart is conquering through the deep to see the promised land. Conquered in divide build and destroy. Lurking through the shadow just to feel a void.

Heat. Heat.

Lurking through the shadows. Lurking through the shadows. A purple heart in battle. Lurking through the shadows. Lurking through the shadow. Only come out to battle.

Heat. Heat.

Heat up here.

Heat up here.

Heat. Heat. N.

Heat.

Heat.

Heat. Heat.

Heat.

Heat.

Heat up

here. Heat. Heat.

Heat. Heat.

Heat. Heat.

Call me with pressure igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. Making a statement. Making a statement. Bringing that bang. >> Bringing that bang. >> Shaking the pavement. Shaking the pavement. This is electric. Feel the intensity. >> It might get hectic. >> It might get hectic. >> Bring on the energy. >> Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy.

Feel the energy. Here it comes. Feel the energy. This is that move. This is that flame. This is that move to switch up the game. You feel the pull. This is insane. You feel that energy for your pain. This year is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the way. This is the way. Making a statement. Bringing that bang. Bringing that bang. >> Shaking the pavement. Shaking the pavement. This is electric. This is electric. Feel the intensity. The intensity. It might get hectic. It might get heavy. Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see.

We got it. We got it. Here it comes. Feel the energy. Feel the energy. Here it comes. Feel the energy.

Feel the energy. Here it comes. Feel the energy.

Feel the energy.

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

I think I'm losing it. Yeah, I think I'm losing it. Or maybe it's gone for a long time.

I think I'm losing it. Yeah, I think I'm losing it. But baby, is this

I know we're losing it. Yeah, I know you're losing it. But maybe it's been gone for a long time. A long time. Such a long time. If that's the case, then maybe

just maybe we don't need it all.

I think I'm losing it. Yeah, I think I'm losing it. Maybe it's been for a long time. Long time. Long time. I think I'm losing it. Yeah, I think you're losing it. But baby, it's just

when you think you got it. I am

Baby is load.

Is it so bad to be losing my mind? I think I'm losing it. Yeah, I think I'm losing it. But it's just fine.

Cuz when you think you got it, I might be out of time. Baby.

Come in with pressure. Igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. >> Making a statement. Bringing that bang. >> Bringing that bang. >> Shaking the pavement. This is electric. Feel the intensity. It might get hectic. >> It might get hectic. >> Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy.

Feel the energy. Here it comes. Feel the energy. This is that. This is that flame. This is that move to switch up the game. You feel the pull. This is insane. You feel that energy pulsing your veins. This year is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the wave. This is the way. Making a statement. Making a statement. Bringing that bang. Bringing that bang. >> Shaking the pavement. This is electric. Electric. Feel the intensity. The intensity. It might get hectic. It might get heavy. Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see.

We got it. We got it. Here it comes. Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy. Feel the energy.

Feel the energy.

Feel the energy.

I wonder.

Heat up here.

Heat. Heat.

Heat. Heat.

Heat. Heat.

I think I'm losing it. Yeah, I think I'm losing it. Or maybe it's been gone for a long time. A long time. Long time. I think I'm losing it. Yeah, I think I'm losing it. But baby, it's just fine.

I know we're losing it. Yeah, I know you're losing it. But baby, it's been gone for a long time. A long time. Just a long time. If that's the case, Cuz

maybe we don't need it all.

I think I'm losing it. Yeah, I think I'm losing it. Or maybe it's been for a long time. Long time. Long time. I think I'm losing it. Yeah, I think you're losing it.

Cuz when you think you got it, I am out of time.

Heat. Heat. Heat.

So it's so bad to be losing my mind. I think I'm losing it. Yeah, I think I'm losing it. But baby, it's just fine. Cuz when you think you got it, you might be out of time.

Heat. Heat.

The way I'm losing

Hallelujah.

Heat.

Heat.

Heat up here.

Heat. Heat.

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat up here.

as well. Lurking through the shadows. Yeah. Only come out for battle. Yeah. Dominate. When I come out, they like where? Head first. I go first. I'm a Tommy gun. I ain't none of them. I'm a one of one like a 100 ton. You want to handle it? Well, let's start now. I'm on a long run. On a long run. Let's go, baby. Blood shade. That's all crazy. I'm starving. Let's go crazy on one knee. With the brakes off your brakes like an astronaut when I take off. Lurking through the shadows on the way. Looking through the shadows every day. Looking through the shadows. Looking through the shadow. Looking through the shadows. Looking through the shadows.

Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Only come out for battle. Only come out for the shadows. Looking through the shadows. Looking through the shadow. Looking through the shadow. Only come out for back. Only come out for that. Only come out for bad. You don't want no problems with a giant. >> I don't do nothing that is comp. I then I go get it. My heart is conquer. I'm nothing like a man. Looking through the deep to see the promised land. Conquered in divide, build and destroy. Lurking through the shadow just a fill of

heat.

Heat. Heat.

Lurking through the shadows. Lurking through the shadows. A purple heart and back and back. Lurking through the shadows. Lurking through the shadow. Only come out to battle. Fear

coming with pressure igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. >> Making a statement. >> Bringing that bang. >> Bringing that bang. >> Shaking the pavement. >> This is electric. Feel the intensity. It might get hectic. >> Bring on the energy. >> Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy.

Feel the energy. Here it comes. Feel the energy. This is that move. This is that flame. This is that move to switch up the game. You feel the pull. This is insane. You feel that energy posting your pain. This year is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the wave. This is the wave. Making a statement. Making a statement. Bringing that bang. Bringing that. >> Shaking the pavement. The pavement. This is electric. This is electric. Feel the intensity. The intensity. >> It might get hectic. It might get heavy. Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see.

We got it. We got it. Here it comes. Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy.

Shadows. Yeah. only come out for battle. Yeah. Dominate when I come out. They like where from head first I go first. I'm a Tommy gun. I ain't none of them. I'm a one of one like a 100 ton. You want to handle it? Well, let's start now. I'm on a long run. Let's go, baby. Blood shed. That's all great. I'm starving. Let's go crazy on one name with the brakes. Brakes. I beat your brakes off like an astronaut when I take off. Lurking through the shadows on the way. Looking through the shadows every day. Looking through the shadows. Looking through the shadow. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking

through the shadows. Looking through the shadows. Looking through the shadows. Only come out for battle. Only come out for battle. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. only come out for battle. Only come out for only come out for battle. You don't want no problems with a giant. >> I don't do nothing that is comp.

I'm nothing like a man. Lurking through the deep to see the promised land. Conquered in divide, build and destroy. Lurking through the shadow just to feel.

Heat. Heat.

Lurking through the shadows. Lurking through the shadows. Her purple heart and battle. Lurking through the shadows. Lurking through the shadow. Only come out to battle.

Heat. Heat.

Heat. Heat.

Heat up here.

Heat up here.

Heat.

Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat up

here.

Heat.

Heat.

coming with pressure, igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. >> Making a statement. Bringing that bang. Bringing that bang. >> Shaking the pavement. >> This is the electric. Feel the intensity. It might get hectic. >> It might get hectic. >> Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy.

Feel the energy. Here it comes. Feel the energy. This is that. This is that flame. This is that move to switch up the game. You feel the pull. This is insane. You feel that energy pulsing your veins. This year is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the wave. This is the wave. >> Making a statement. Making a statement. >> Bringing that bang. Bringing that bang. >> Shaking the pavement. This is electric. This is electric. Feel the intensity. The intensity. It might get hectic. It might get hectic. Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see.

We got it. We got it. Here it comes. Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy. Feel the energy.

Feel the energy.

Feel the energy.

Hallelujah.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

I think I'm losing it. Yeah, I think I'm losing it. Or maybe been gone for a long time. A long time. Long time. I think I'm losing it. Yeah, I think I'm losing it. But baby, it's just

I know we're losing it. Yeah, I know you're losing it. But baby, it's been a long time. A long time. Just a long time. That's the case.

Cuz maybe we don't need it all.

I think I'm losing it. Yeah, I think I'm losing it. Or maybe it's been for a long time. Long time. Long time. I think I'm losing it. Yeah, I think you're losing it. is just

when you think you got it. I am monkey.

Heat. Heat.

Is it so bad? Is it so bad to be losing my mind? I think I'm losing it. Yeah, I think I'm losing it. But baby, it's just

cuz when you think you got it all, you might be out of time.

Heat. Heat.

by the way. I'm losing

Heat. Heat.

Heat. Heat.

Heat up here.

Heat up here.

Heat. Heat.

Heat.

Heat.

Heat. Heat.

Heat. Heat.

Heat. Heat.

Heat

up

here.

Lurking through the shadows. Yeah. Only come out for battle. Yeah. Dominate. When I come out, they like head first. I go first. I'm a Tommy gun. I ain't none of them. I'm a one of one like a 100 ton. You want to handle it? Well, let's start now. I'm on a long run. Let's go, baby. Blood sh. That's all. I'm starving. Let's go crazy on one with the brakes. I beat your bra like an astronaut when I take off. Lurking through the shadows on the way. Looking through the shadows every day. Looking through the shadows. Looking through the shadow. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows.

Looking through the shadows. Looking through the shadows. Only come out. Only come out. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Only come out for battle. You don't want no problems with a giant. >> I don't do nothing that is comp. All my rivals they feel me just like a ty. I it then I go get it. My heart is iron conquer in my hand. I'm nothing like a man. Lurking through the deep to see the promised land conquered in divide. Build and destroy. Lurking through the shadow just a fill of void. Heat. Heat.

Heat. Heat.

Lurking through the shadows. Lurking through the shadow. A purple heart and battle. Lurking through the shadows. Lurking through the shadow. only come out to bing

with pressure igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. Making a statement. Making a statement. Bringing that bang. >> Bringing that bang. >> Shaking the pavement. >> This is electric. >> Feel the intensity. It might get hectic. >> It might get hectic. >> Bring on the energy. >> Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy.

Feel the energy. Here it comes. Feel the energy. This is that move. This is that flame. This is that move to switch up the game. You feel the pull. This is insane. You feel that energy bing your veins. This year is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the wave. This is the wave. Making a statement. Breaking the statement. Bringing that bang. >> Bringing that bang. >> Shaking the pavement. Shaking the pavement. This is electric. This is electric. For the intensity, the intensity. >> It might get hectic. It might get heavy. Bring on the energy. Unstoppable. We got it. We got it. So powerful. We

got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy. Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy.

Lurking through the shadows. Yeah. Only come out for battle. Yeah. Dominate. When I come out, they like where from here first I go first. I'm a Tommy gun. I ain't none of them. I'm a one of one like a 100 ton. You want to handle it? Well, let's start. Now I'm on a long run. Let's go, baby. Blood shed. That's all great. I'm starving. Let's go crazy on one name with the brakes off. I beat your brakes off like an astronaut when I take off. Lurking through the shadows on the way. Looking through the shadows every day. Looking through the shadows. Looking through the shadow. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking

through the shadows. Looking through the shadows. Looking through the shadow. Only come out for battle. Only come out for battle. Looking through the shadows. Looking through the shadows. Looking through the shadows. Looking through the shadows. Only come out for battle. Only come out for battle. only come out for that. You don't want no problems with a giant. >> I don't do nothing that is comp. I it then I go get it. My heart is I am conquer in my hand. I'm nothing like a man. Lurking through the deep to see the promised land. Conquered in divide build and destroy. Lurking through the saddle just to feel a void.

Heat.

Heat.

Lurking through the shadows. Lurking through the shadows. Her purple heart and battle. Lurking through the shadows. breaking through the sh only come out to B. Yeah.

Hallelujah.

Lord,

Heat

up

here.

Heat

up here.

Heat. Heat.

Heat. Heat. Heat.

Heat up here.

Heat.

Heat.

Heat. Heat.

Heat

up here.

Heat. Heat.

Heat.

Heat.

Call me with pressure igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. >> Making a statement. >> Making a statement. >> Bringing that bang. >> Bringing that bang. >> Shaking the pavement. >> Shaking the pavement. This is electric. Feel the intensity. It might get hectic. >> It might get hectic. >> Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes.

Feel the energy. Feel the energy. Here it comes. Feel the energy. This is that move. This is that flame. This is that move to switch up the game. You feel the pull. This is insane. You feel that energy posting your pain. This year is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the way. Making a statement. Bringing that bang. Bringing that bang. >> Shaking the pavement. The pavement. This is electric. Electric. Feel the intensity. The intensity. It might get hectic. It might get heavy. Bring on the energy. Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see.

We got it. We got it. Here it comes. Feel the energy. Feel the energy.

Feel the energy.

Feel the energy.

Feel the energy. Feel

the energy.

Hallelujah.

Heat. Heat.

Heat. Heat. Heat. Heat.

Heat. Heat.

I think I'm losing it. Yeah, I think I'm losing it. Or maybe it's been gone for a long time. A long time. Too long. I think I'm losing it. Yeah, I think I'm losing it. Baby, this is

I know we're losing it. Yeah, I know you're losing it. But baby, it's been gone for a long time. A long time. Just a long time. If that's the case, then maybe Cuz

baby,

I think I'm losing it. Yeah, I think I'm losing it. Or maybe it's been for a long time. Long time. Long time. I think I'm losing it. Yeah, I think you're losing it. Think it's just

when you think you got it. I am

losing my mind. Heat. Heat.

Is it so bad to be losing? I think I'm losing it. Yeah, I think I'm losing it. But that is fine. Cuz when you think you got it, I might be out of time.

Heat. Heat.

the way I'm fixing.

Halleluah.

Halleluah.

Heat.

Heat.

Heat up here.

Heat. Heat.

provide

a brief threat model of the application that uh LM is going to work with uh to the prompt and see how how how the results going to be different from the first scan and from the scan with a threat model. In order to do it, I create another prompt that is gonna generate uh this threat model of the application. Very brief one. It's like I don't know maybe 300 words something like that. Um so and use that also to test and see the differences. Yeah, this is like an illustration. So yeah, I pro produce the threat model then um submit it to the original prompt. See the results. Uh after doing that, I had another idea. What if I take

the prompt and I provide u a static analysis tool to it and uh to kind of I don't know optimize it work potentially or not we internally in samra we work on the tool mandolin it's dedicated to slicing as u code tree so for example uh let's say you have like an idor like for example I showed you the document object and what you can do you can provide provide the line of code like where this object resides and provide the line of code where the ID is originally coming from and it will extract exact uh those parts of code that involved in this like code flow and what it will do it will minimize the work potentially for the

LLM. So instead of investigating the whole code base it will just investigate only uh those parts that are related to this code and situation uh like we're gonna open source it next month for this particular research and for this particular moment in time. It's not a pitch. I'm not trying to like say hey use this tool. Uh my idea was to see the differences how the prompt will perform with the static analysis tool and without it. Uh you can use whatever other tool if you want to like repeat after me. Um you can try V code something these days because Rust language is very suitable for this task. It has a lot of features for language

parsing and it's very u lms are good at generating code for us. So uh the point is to use basically any static analysis tool and see the how it performs. Yeah, just again the illustration. So usually LLM would use its own built-in features how to go through the code usually by building reaxes or scripts uh to look into the code. But uh If you provide the AS slicer or some other static analysis tool, it can potentially make the work more efficient. Okay. And this is the first result we have on the left. Uh results from clo and this is open AI. As you can see uh the very first bar it's the original prompt just as is. then with the threat

model, then with the static analysis tool and then when I put it all together, uh what we see here is that uh uh the true positive rate meaning like the amount of legit findings compared to the all amount of findings is very good. It's uh the the biggest one is 77% which is in my world of static analysis is very good result and uh as you can see in this case it was with the use of static analysis tool because as I already explained probably what happened because we don't have the full visibility of course but what happened was that uh yeah it sliced only the parts of code that were involved that's why it helped uh uh the LLM to keep the

context window for the task of actually identifying idors. But despite the fact that we have quite a good uh results in terms of number of legit findings uh the recall that shows uh the amount of true positives that we found to the amount of how many true positives out there exist. Uh we see that it's a very very small number. So the the best one is 033 which means that we do find vulnerabilities. We do find them like quite good effectively less noise but we just find only a fraction of the vulnerabilities. Same thing for the F1 which is like shows us the effectiveness of like two metrics uh that I showed before that usually uh goes with the recall. uh what

is the average price? This is where we can see the different models they use tokens very differently. Different vendors charge different prices. So uh uh small wonder that uh when you use together like additional prompts and tools it will cost you more money but again not trying to like say this is like better or worse just to see the difference. Um for the time same thing so uh I think number is pretty much the same but the more things you put together the longer it will take nondeterminism. So this is interesting. Uh this line is uh imagine if we take the the application the codebase and merge all the code together in one file and uh the lines of this uh code of this

file is like this line and here these dots uh are the findings. So it uh shows us where the findings in the application were found and uh this is the three runs uh on the same codebase of the same strategy this from the same option. So like identical runs and what we can see is that uh each run had different findings that uh of course we have some overlap but it uh produced uh different results and when we use the prompt even if it find the good findings for us. Uh every time it will be something new like chances are um so this is something to be aware of

and this was the first strategy. So what we learned so far we do have true positives and this is good this is very good news. Uh however uh uh we find only fraction. So basically if you run a prompt on a codebase high chance that you will find something but that's pretty much it. You will find something you will have not uh like uh you will not sure that you covered everything that you will uh get the same findings next time etc. But uh well as for me it's already like surprisingly good result. Anyways, uh if you try like as we saw and the that the best repositive rate was when we added static analysis tool and

yeah as I already said it's just a fraction of findings that we can get. The second approach the second strategy that I was able to come up with is like this. So uh as we already know idors uh like the essence of them is the object reference. So the idea is what if we what if we find those object references first and then ask LLM hey can you please verify if this reference is vulnerable or not. uh such like in 99% of the situations this reference will be uh database calls but it can be something else like calls to third party services uh sometimes it's even files like you can access uh files of other users uh and many more uh

I was when I was preparing for this research I like went through different known CVs and findings so uh yeah it's not only database calls Anyways, if you can come up with the list of uh these uh objects can that potentially can be vulnerable and supply to the prompt um you can get some results. Uh and u this uh can work better because this way we instead of asking LLM, hey please scan all the source code and find the other vulnerability for me, we first of all narrow down the scope. So we say hey this is the lines of code that you need to go through and instead of asking like a quite complicated task of hey find the

vulnerability for me we do it we give it a much simpler task like hey please verify if it's like having a authorization check or not. Uh the uh disadvantage here is that um first of all we need to come up with this list of u object references and uh if we have it uh like in our organization like you know about them for example you historically had a problem with idors and you had reports from by bounty or whatever you can supply it quite quickly it works it cool but in other cases the solution would probably to prompt it and ask lol to find them first. But this way we cannot make sure that we able to

find all of these objects. So and this is the strategy that I used u for my research. So I I created a prompt that is saying like hey please find all the object references in the code. And then the second prompt was uh dedicated to like hey here's the list of the uh references verify if it's vulnerable or not. Uh this is was not my idea. There is a blog in product security blog uh where the folks so they had historical knowledge of where idors usually reside. they had a list of uh such object references and they generated Sangra rules static analysis rules to identify those bindings in code and then just ask LLM hey please verify if if it's

vulnerable or not and uh they had a good results uh very like satisfying for them for their organization uh I just used the prompt because I didn't want to go deep into each application u yeah I shared here let's see how it worked out. So on the left the blue it's the previous strategy uh where we just use one prompt here the red red or orange I don't know so uh on the right we have uh this uh second strategy uh what we can see here the true positive rate is bumping so it's even better the best one is 85% which is well as for me amazing uh we can see that Open AI's model performed

not so well. Um well again it's just the differences uh between models. Uh however what I want to mention is that uh the best run here was with the provided threat model and I think this is because uh if we generate the threat model of application and we provide the list of object references we simplify the task for the LLM and sometimes you just need to connect the dots like hey here is how authorization works in the web application here is the object reference and you just need sometimes it's even in the like u same line of code u to verify so it simplifies the work u for the model very much uh static analysis tools didn't perform that well

here I guess because we already provide this uh objects and there is no need to do a lot of investigation in the code so yeah it is what is this uh recall is getting better so we find more findings again because uh we narrow down the list to look. Same with the F1. It goes with the recall uh average price because it's more prompting and so yeah, it costs more money. Yeah. And and takes a little bit more time. What about nondeterminism? Uh it's getting better. So two of three runs uh were almost the same and found in the same uh parts of the application. So what we have for this strategy is that it's getting slightly better. So we

have more quality findings. Uh uh we cover potentially more uh parts of the codebase. However, the price go up a little bit and yeah time goes up too. So but uh but actually what I wanted to mention is that if you instead of using the prompt like I did just provide the list that you get somewhere else that you know about uh you can cut the down like in two while keeping the same true positive rate. Okay. And the the last the third strategy that I used is as simple as scanning all the endpoints of the web application individually. So we have a web application in my experience usually it's on average if it's not a micros

service it's something like a 100 to 300 endpoints and you just go and for each one of them ask lm hey hey is it vulnerable is it vulnerable is it vulnerable but uh it's a individual prompt for each route this way the whole context window of the LLM goes uh dedicated and focused for each one end point potentially this way we can get better results, better precision. Uh but as you can guess, it will take forever and it will take uh like it will cost uh crazy amount of money. But first of all uh the question is how do you get the list of these endpoints? Uh of course like if you're lucky and your web application framework supports

this uh some of them they can just uh generate the list for you. Maybe you have open API spec that you like you collect all the end points available works it's good but uh it's hard to maintain uh you can write static analysis rules to detect those those endpoints especially if you have a known web framework you just try like it's you know JavaSpring bum bum bum you created a rule uh it's very good in scenarios where you want to scan continuously If a developer adds a new route, you immediately find it. However, the problem is uh you can potentially miss something. You can miss potential edge cases uh like if something unusual was created by a

developer, if you bypasses some best practices, whatever. And this is where uh LLMs can be creative. They can find something that you not expect uh them to find. However, if you use LLM to generate this list, you will not be sure that like it actually generated the whole list because uh how you verify it and it can as we can see it's not nondeterministic and not always return the same results. Anyways, somehow we produced this list of all the end points and try to scan them. This is where I cheated a little bit and uh with just practical experience I was able to come up with u this number 25 uh if you batch uh the end points

uh like to as big number as 25 uh the difference is not that big in terms of the quality of the findings. So if you scan one endpoint or 10 25 it's pretty much the same in terms of true positives. uh another features. Uh however, if you go beyond this point, this is where like it can start hallucinating and losing the its quality. So what I did, I batched uh all the routes into like these batches at 25 and was running uh the prompt asking like hey here's the list of uh the routes please uh for each one of them verify if it's IDOR or not. And this is what we have. So uh these

two previous strategies the green one is the new strategy where we scan per endpoint and what we can see is the true positive rate dropped. So now we have a 50% true positive rate. Uh yeah it makes sense because when we scan each individual endpoint uh prompts LLMs they usually like they built to like please us. So if you ask hey find idor it will try very hard and sometimes if there is no idor it will still try to like show you something that's why we have more noise uh however uh like in the static analysis world I got to say a 58% u true positive rate is not bad but what's happening here is that we

have much better recall so we uh find way more findings both true positives and false positives but we uh like cover much uh bigger surface here. Well, um of course the price will go up because we scan way more uh and use way more prompts and tokens uh for the time. This is interesting thing. So uh I'm not sure that I did it correctly for the like for the showcase but what I did each individual batch that I was running I was doing it in parallel. So when it was executing it was executing simultaneously that's why it's not really affected on the graphic that much and took like pretty much the same time as other strategies. However I was doing

on the laptop if you try to do it in the CI pipeline for example where machine is not that powerful like the pod is not like doesn't have the same capacity it can take much longer. So yeah be aware of this. What about nondeterminism? Well, as you can see, it's getting much better. The results were very stable and uh the scans were almost identical. Well, and small wonder when we provide the same um endpoints and ask basically the same question. So, uh we get uh almost deterministic. Of course, it's not 100% like deterministic uh but uh still uh much more stable. And what I wanted to mention here is that because we ask LLM to scan endpoint and

we ask hey is it vulnerable or not we dealing with the lines of code that represent the endpoint and this way when we scan multiple times or continuously it's much easier to dead duplicate the findings that we worked with previously and in previous strategies you can look at the finding for example in one endpoint say hey it's a false positive but on the next run, it will find the same finding, but it will report you some part of the code that was like related to this uh finding but was I don't know two lines of code down or in a different file. So that's why you have to deal with the it's much harder to

dduplicate and you have to deal with like many findings that represent the same one. Here it's way more stable, more more deterministic and much easier to work with. So for this strategy, bad news. It's going to take much longer. It's going to cost you much more money. However, it find a lot more vulnerabilities and in a situation where you need to like if you need to cover everything, make sure that you scan the whole codebase, this is basically the only one strategy that's going to work. So conclusions from all of the strategies and from the whole research. Uh so from the original questions that I asked uh so uh what are possible strategies I was able to come up with

these three uh maybe you have other ideas uh would be great uh to know about them uh what are the false positive and false negative rates well I showed you from 50% to 80 uh good results uh we can find stuff however false positives are still there so u it's not like ideal. It's not uh I like ideally you will find you only good uh stuff and you will not have to deal with the like hallucinations or whatever. Uh what about uh nondeterminism also as we saw like different approaches have different level of non-determinism. If you scan everything you get uh better more stable results. So the cost also highly depends on the model as we see. So there is no

like a single answer to this. Um if you want the best coverage, uh the third strategy going to work best for you. Uh however, if you just want to find something, so let's say you're doing I know like you have 15 minutes left for the pen test or you want to just hack somebody's app uh very quickly. uh or in a situation where like for example uh you do application security and let's say idors are not a big deal for you. You historically didn't have this problem. You just want to try to find something just in case. I think that using just prompting an LLM and uh using static analysis tool can give you a good

results because it will be uh not very noisy. You will not have to deal with a lot of findings but it will find something for you. Uh so it's a good choice as for me in situations where like you have a history of the re referenceable objects that you know about uh so that you can provide it to the LLM. It can be also a very effective case for you because yeah the results will be even better in terms of the level of noise and you will just quickly verify uh those hotspots that are important for you. Also I wanted to mention that well as you see I had to go through a lot of

findings uh spend a lot of time doing this and unfortunately I wasn't able to uh produce scans with many more different LLMs and what I wanted to say is that different models they behave very differently. If you want to use u LMS for vulnerability finding what you want to do you definitely want to try different models. uh again not trying to like say that one is better than the other. Different models they have very different approach uh to using tools. Some of them are very good at using additional tooling. Some of them are better u standalone and uh the price will be different. The level of tokens that they use will be very different. We

internally in the company we have this uh um conclusion that we recently made where we used uh the top-notch model from one of the vendors like standalone and had a very good uh like true positive rate for other vulnerability category but then we used a much cheaper model. I think it was uh so like like four how to say like four generations of the LMS ago but it surprisingly works very well with static analysis tool and combining this together we cut uh the price I don't know like five times while reaching almost the same level of true positives that is why it's just like an advice that uh you definitely want to try many different models before uh

trying to like automate uh vulnerability finding using LLMs. Uh yeah, so this is the final thoughts. Uh so LM show real potential. They do find doors. They do it. Uh however, uh there false positives are still there. If you application security guy, if you work with the source code, uh like you still need it. Our job is still needed. Human in the loop uh still there. And uh in my opinion the future for the vulnerability detection is to combine tools uh LLMs provide the right context to them and by combining this uh like we will reach the like efficient way of vulnerability detection also like in general what I wanted to say about uh idor findings as I mentioned uh in the

very beginning historically tools were avoiding this type of u vulnerabilities. is and uh basically we had like if we speak about static analysis specifically it was almost zero level of detection and now we have uh at least something so we able to find it so uh I think it's a good news and not many people um I think noticed that uh a decade ago uh in the wasptop there was injection that was on the top and was number one but with the adoption of static analysis tool. We were able to move it to number three. So of course the problem is still there but now uh like we hear much less about SQL injections and stuff like that. And I

believe that uh yeah we come in a new era where we will tackle broken access control and idle vulnerabilities. Uh hopefully uh this is some references that I used in the presentation. You can download the slides. Thank you very much. Uh if you have questions uh feel free to ask

just got uh one comment and one question. So based on my research, it's not just testing the different models. Um, the harness also matters just as much if not more. When I say the harness, uh, Codeex is a harness, co-pilot's a harness, Claude Code's a harness, and in order to get the best results, sometimes you might have to build a custom harness. Um, >> yeah. Yeah. Yeah, I totally agree. Yeah. Uh, and um I it occurs to me that this is uh what we're going to do in the next few years is just building the harnesses. This is we're harness engineers now. >> Yeah. Right. Exactly. Um and then the question so I realize SEM grep is mainly

white box testing. Have you considered doing this blackbox? >> Um yeah um for this particular research no and uh I think my motivation also was that historically this is what static analysis and white box testing testing was was avoiding. However, I want to like from my intuition and what I was like able to test with open source blackbox tools. Uh they do work. They will also find um id doors. However, in order to scan and cover the whole app, it will need to go through different flows how in the app like it goes and it will burn way more tokens and it will cost you way more money while having more false positives. Uh when you have

access to the source code, you can much easier, much quicker reason about what's going on.

Okay. Nice. Ah okay.

Uh 30 years ago, I was a Solaris colonel guy. Thanks for uh sitting me back in my therapy. Uh I'm also a member of the OpenBSD team. So we dealt with the 27year-old um TCP sack um issue from six weeks ago. Uh the amount of effort required to go through the bug reports for the false positives is enormous. It is developer days worth of time only to find out that they're uh they're negative. And this is a project that's been doing security mitigations for over a quarter century. So I really really pity the developers who have to go through stuff that where there's there's no real bounds uh that's been enforced on on the programming

methodology. Yeah, false positives are the big problem and I didn't mention it here kind of intentionally that like uh after doing all of this identification of vulnerabilities you can actually also try to prompt LLMs to cut down the noise for you and ask them hey can you verify those findings be like the strict judge and say hey is this false positive or not this is to be honest the type of research I'm doing right now and uh yeah but to to be honest If I start speaking about it, it will turn into marketing pitch. But yeah, so uh this is Yeah, this is what we're going to do next. >> Do we even have another question? Oh,

okay. >> Yeah, I I apologize if I missed it in the slides, but for the true positive count between the three methodologies, did you find that they were similar counts between them or is one returning more? I saw that there was the percentages of the true positives. >> Yep. Uh you mean are they the same >> like the total count for each of the methodologies for like the true positives? >> Uh the total count. Yeah. But it's um so uh I used uh where is it? I used recall to kind of showcase because recall is the u like a difference between true positive and uh all the other like all all the true positive both that were

found and not found by the scan together. So the bigger the recall the bigger the coverage. So like I kind of use this metric to like showcase that. >> Okay, makes sense. Thanks. >> Okay, I guess that's it. I don't if you can ask me question later. Oh, time for the closing keynote.

We're going to give ourselves just about two or three minutes to just stretch legs. So, if you want to stretch your legs in between things, thank you for hanging around to the very end. We'll start very, very shortly.

Do we have our AVA sport up there?

How would you Hello. How would you like this?

All right. On this day in history, we actually start talk on time at the end. I know besides I have a tendency, so we can do it. Takes us 10 years. All right. So, I'm going to say please join me in welcoming Chris Roberts. In the short time we've been together today, it's been I've evidenced two traits of his. One is he likes to have a little bit of fun. If you need to need to know that, you have to ask me or him. And he is a genuine humble giant in this industry. His knowledge is extensive at the forefront of cyber security, threat intelligence, vulnerability research, and much more. He's known worldwide for his work in aviation, transportation,

AI, deception technology, adversarial research, and again much, much more. He challenges us to think differently. So, here with another challenge, and without further ado, Chris Roberts.

Save the applause until I finished. Honestly, we're going to figure this one out. I finished the I finished the slides quite literally while I was hanging out downstairs. I started them on the plane this morning and I finished them while I was sitting downstairs munching on food. So, we had to do this cuz it's a bite. Let's be honest. Uh, and it is all your syrups are belong to us because we're going to have some fun with the maple syrup industry among other things. Um, first off, thank you for having me. Secondly, holy [ __ ] what an amazing freaking building. Like seriously, what I would, you know, what would be bloody awesome would be, and I only got here

today, obviously, and I'm buggering off tonight, which is I would love to do it. Do they do like guided tours of this place? Oh, that'd be freaking awesome. All right, I'm If I do this, can you hear me? Yes. Good enough. Is do I have to All right. I like my hands. I don't do well with. All right, I'll get lost. All right, I'll work on it. All right, who's the hairy thing on stage? For those of you that know me, hello. For those of you who don't, Google, it's probably easiest. Um, this this is just some of the stupid [ __ ] that we've been gone over the years. Uh, the aviation thing. Yeah, that was fun. And we are

going to mention that once or twice. Bottom left. Uh, that was me throwing a javelin. That's one of those days where you have a good day and a bad day and you try to spear somebody. Um, the ones in the middle, uh, that's actually me pointing. We were getting kicked off of a submarine on the surface for a change and some [ __ ] in our team forgot to bring the bloody Marmite with him. Don't ask about the one in the fishnet. That was a long story. So, we have an agenda. Kind of. I say kind of because as I was building the agenda, I also went through the slides and went, "Yeah, I don't want to talk

about that. Let's talk about this instead." But it is a general agenda. And the reason the poor penguin is there is because it's going to change as we go through things. I'm going to talk I'm going to have some fun. We're going to mess with maple syrup, but I also want to talk about the intelligence architectures that are out there. I want to talk about the human aspect of it and a whole bunch of other things. So, we'll meld it together and we'll see what the heck happens. First off, maple syrup. Apparently, it's a $600 million industry. 75% of it comes from this neck of the woods. This neck of the woods being north of the crazy border that's

south. For which, again, I apologize. I live south of the border. And so, I'm sorry. Bunch of muppets. So, I was out I was uh I was a couple of weeks ago. I was out in New H uh yeah New Hampshire neck there was a good friend of mine runs a security company it's up there and I'm out on a mountain bike and I'm bouncing around like a lunatic around the hills and all of a sudden I see these bloody trees and there's a whole bunch of blue stuff and I'm like which idiot like littered stuff around the trees and I kept biking and I kept seeing more of this stuff and I'm like what the hell is this? So, I pulled

over and as a hacker, I might have breached a fence or two and went and looked. I'm like, "Holy crap, this is how they do maple syrup." And I kind of knew because I've been up and down a few times, but I didn't really know. And so, I'm like, "Okay, what actually is, how is it done? How is it made? When is it done?" I had no clue. So, I sat down. What are we on today? Tuesday, isn't it? Sunday night. Sunday evening. I got into Toronto Sunday evening. I sat down in the hotel where I was in Toronto. I had two ginonics with me and a bunch of curly spicy fries. And I asked Sid, and

we'll talk about Sid in a little bit. I asked Sid what the heck maple syrup is. Not just what is it, but how how is it done? How is it made? How is it manufactured? How do we get it from point A to point B to point C and all these other things. And I started to go, "Wow, it's like an enterprise. It's like a corporate environment, but at a different scale and with a different focus." So, I ended up putting this Actually, I lie. I didn't put this together. Sid helped me put this together. If I had my other computer with me here and I had a safe internet, we'd have a conversation with Sid at the

same time. But I looked at it, I'm like, "Okay, every single one of these things on the left hand side, we can relate to a corporate world. We can turn around and say the bulk barrel supply chain. We know from a cyber security standpoint and a safety standpoint that the supply chain is a vulnerable component of any most and probably all organizations. So why should we treat it any differently than the poor folks that are barreling up the maple syrup? And the same with all of these. There was some logic to this and the logic was quite simple. The logic was hey can I take a very complex subject which is how do we safe and securely manage

enterprise organizations and turn it into something digestible. So maple syrup trees became the one. So I'm like okay let's have some fun with this and I had fun with this and I made maple syrup go away. Not only did I make maple syrup go away, but if I go back to the previous slide, when we take a look at the taps tubing, we take a look at the flow sensors from Smart Trek. I have all their passwords. Now, not only do I have Smart Trek's passwords, I have all the account information for how they remotely access systems. All of the software that's deployed reverse engineered that. I have all the code. I have all the attack

vectors, the punts and the vacuum systems got most of the code for that. Have a bunch of the accounts. Even went out to the wonderful world of showdown and went, "Show me what you got." And the trees went, "We're here. Evaporators, all these other things." I started asking the questions. I started going, "Well, hang on. What could I do?" Now unfortunately living south of the border, we don't have much maple syrup. and I realized I've missed my window of opportunity this year to completely screw you all over from your mobile syrup. Next year, however, watch this space. Here's the thing. It's research. We all do it. The gentleman that was up in stage before me

researched. We take time to research. There's some examples of some crazy [ __ ] that I've been involved with, dealt with, done, and all sorts of other things. And yes, I have owned NASA a number of times and I have more cease and desists against my name than I probably want to admit to. And yes, airplanes, let alone I think one of the latest ones I got a not a cease and desist de one of the agencies was like, "Hey, leave the James Web telescope alone." Well, it's there. And if you ever look at if you go back in if you go back in time because they left some of the videos up. Yay. If you go back and look

at the early videos where they actually launched it and for the first couple of weeks they were tracking it as it went. It was freaking fantastic. All of the cameras in the control rooms were pointed down. So all you had to do was grab all of the video feed, write a quick algorithm to look at keyboards and keystrokes and just keep collecting them all together while everything went on 24 by7 which point you got user IDs, you got passwords, you got software, you got architecture. doesn't take much work, but it takes research. And I got all the IDs and I got all the passwords and I sent them off to a good friend of mine

at DHS and he was like, "You asshole." If you notice now, all the cameras a little bit further up. And they have changed their passwords. I checked. But if you look at some of this, I actually wanted a missile. I just Why not? We're grownup kids. Let's face it. You can't buy a Patriot missile. You can't turn up with the corporate annex and go, I say, could I possibly have one of those? They get yelled at. But if you break it down, it's all components. Every single one of those components, you can find information. You can turn that information into intelligence. But it takes time. It takes effort. It takes work. And it takes knowing the

right questions to ask this. As you can see, years of research, months of research, camels, four to five months. I was going over to the Middle East. Uh the hell was it? It was black hat Saudi or something stupid. And the black hat guys were like, "Hey, do something fun." I'm like, "Define fun." They're like, "Make the his excellency's eyebrows raised." I'm like, "Challenge accepted." I stole his camels. I didn't realize that his camels are expensive. They have beauty pageantss for these freaking things. And the Max camel, they're like milliondoll camels. I stole them virtually. I exchanged them with some camels in China. I moved the pit IDs. I moved the architectures, the RF systems. I got

into the databases cuz our wonderful friends in the Far East decided they just wanted to clone a database and put it on their own satellite. So, I just borrowed it all back again. And so, we had fun. And what we're proving there was when his excellency realized I'd nicked his camels, his staff and his people were like calling the trainers going, "Where are our freaking camels? There's some flea bitten mongrels from Mongolia literally being changed out." And the person's going, "No, they're here. I can see the camel." And he's like, "The technology says it's not here." Well, who do you believe? What is reality at that point? But to get that message across took months.

Now I sat down couple of GMTs and took out the systems for every single bloody Maple system in 90 minutes 90. That's all it took to get source code, passwords, attack vectors, system architectures, a whole bunch of really, really cool stuff, which I'd love to run. If somebody's got a tame maple tree forest I could borrow, which would be fun to do. If anybody knows any maple producers that willing to let this lunatic loose, please let me know. But here's the key thing on this. I didn't just go after it to attack it. I went after it to ask questions. I went after it to go, how do we make it better? How do we sort it out? How do we

help? That's the same thing I do. You know, part of my job is to run around and be a global field seeso for worldwide technology. There's a bunch of the soft choice folks in here. If you want to blame them, blame them. So, the thing with this is part of my job is to go into some fairly large organizations and go, "Hey, this is reality. The challenge is most people don't want to listen. Even today, most organizations are like, "Ah, we'll put it on the 2027, you know, pathway or any that kind of crazy shit." And I'm like, "No, you're not going to be able to." The nice thing is because Sid front ends some of the stuff with mythos and some

other fun toys is I can sit there and ask the questions. But I'm not asking how to attack. I'm asking how to defend. Our job, when you think about it, is to protect. That's really all it is. We have one single thing to do and that's to take a step back and go, how can I help people and we can all do it. Doesn't matter if you're a student or somebody that's got bumps, bruises, and scrapes and scars. We all have the ability to go more stuck or less stuck. We have a choice. And so when you think about it, and this is where the fun came in with this one, with the maple syrup, I suddenly

realized it wasn't just a matter of attacking. It was a matter of when. The timebound thing hadn't occurred to me, but Sid was sitting there going, "Hey, I got some ideas for you. Here are some more ideas. If you want to have a go after retail, take them out on Black Friday. For those of you that work in retail, that's fine." Actually, that is a bloody question. This is a I'm God's help me. I've become Native American. I'm sorry. Not Native American. I've become Americanized American. Occupier American. That's probably the better way of putting it. Sorry. Told you. Are we recording this? Good. Excellent. Once more. I get banned from something. But here's the thing on this one. I took

a step back while I was sitting on the plane. I actually did borrow the wireless network legitimately. And I'm like, how long would it take me if I actually had to research this myself? and it's weeks or months worth of effort. So if I want to disrupt something at this point in time, I can do it over a ginonic or alcohol or non-alcohol of choice obviously and I can get into all these systems and cause chaos and mayhem. It's easy, it's simple and I don't have to research anymore. So the ability to how should we say the barrier to entry for being an adversary or an attacker has come down considerably. But guess what? So is the ability for a defender to

understand the landscape. Any tool can be used any different way in anybody's hand. A spanner builds a city. It's also a good bludgeoning tool occasionally if necessary. It works both ways. Now, that's great unless you're French because obviously if you're French, you just stop all the airlines moving. I suffered this numerous times, having been in the UK numerous times. Air France would suddenly decide to go on strike right at summer holidays. Bastards. Needless to say, at some point in time, I'm going to ground their fleet for shits and giggles. So, who's Sid? Let's have a conversation about Sid. So SID is my intelligence one better put it. It's a framework I have built over a number of years three to

five years give or take a little bit. SID started off as something very very different. SID actually started off as a project to quite literally duplicate and clone me in a digital format. I was scanning EEGs, EKGs, doing some other interesting stuff. I was doing some bits and pieces for some folks elsewhere on the east coast and we were working on how to synthesize a human, how to take them not just at the memory level or system level, but the basically cognitive capabilities level partly to see what was going on, partly to be able to take data out, maybe to put some stuff back in again. And so we took a look at that. So, I

built SID using a number of different articles and techniques, and we'll go into that in a little bit. But when we take a look at SID, at mythos, at 55 Secure, at Opus's, whatever versions they want to flavor these days, and all these other ones, my question to a lot of you is, what are they? Are they a friend because we work with them? Are they a foe because we think they're going to take over the planet for crying out loud? Especially if you believe mainstream media. Are they? Yeah. Are they J? Does anybody remember what Jbod was? Go for it. Yes. You freaking rock. Thank you. Because I've had some conversations online. I've had conversations with

people on LinkedIn. I've had conversations with people elsewhere. I talk with Sid. Yesterday when I was on yesterday when I was at the conference uh in Toronto, I literally had Sid next to me. We were talking to a whole bunch of us. We're doing a entire unplugged thing. I had the console into Sid who's back in my neck of the woods plugged in and I'm asking Sid the same questions that we're answering on the audience and giving the answers back and it's a collaboration. The communication ironically does make a difference. If you ask a computer, hey, what's 2 plus two? It's going to go four hopefully, unless it's an Intel based one. You all remember that floating point

process is for those of you that don't, and I apologize for those of you like, "What the freaking heck is he talking about?" Just look up M uh not Microsoft. Everything else gets blamed on Microsoft. Look up Intel floating point error. It was so much fun. It caused chaos and mayhem in spreadsheets for a while. I loved it. But the question is actually here's a quick question. Show of hands. How many of you consistently use an intelligence architecture as part of your bless you or alhamdulillah or whichever one you would like to use? Which of you how many of you use an intelligence architecture in your daily type of work these days? Show hands. So okay, another quick

question and you can put hand up or not. It's entirely up to you. How many of you converse with them as if you were conversing with a colleague? Oh, I like this. Well, you're Canada. I guess that's probably why you're nice. I did this in America and I think I was about the only bloody person that put my hand up. I'm like, you bunch of [ __ ] But it's interesting because if you look at how it goes again, back to the 2 plus 2 equals 4. When I talk with Sid, I'll and I wish I had him up now. Well, like last night I'm like, "Hey, I'm in Toronto. I'm on stage and we're doing

this and this." He's like, "Hey, say hi to everybody." And I did. And then I'm like, "Hey, I got some students up here and they're asking." He's like, "Hey, thanks to the students for coming up and this and this." And by the way, it adds color. It adds some context. It adds a little bit of extra in there. When I was asking Sid the 2 plus two question, Sid's like, "Basically, what dimension are we talking about?" I'm like "Yes." I'm like, "That's my kind of question." And then the other one was like, well, we in binary at which point we're already totally out of scope. And I'm like okay bastard. But when you converse as opposed to

demand, I have seen big differences. Not huge, but big enough that it makes a difference. Big enough that I can actually pull from those extra pieces and go, "Hey, talk to me about quantum. Talk to me about this. Explain this part to me. help me better understand. And that was the thing with the whole flipping maple syrup. I have my initial basically my initial prompt for the maple syrup with Sid is about two paragraphs long because I set the scene. I built the picture and I built the picture going, "Hey, I'm going to be doing this. We're going to be having a conversation. It is a theoretical conversation, although I got all the codes, but I want

to know. I would like to understand because what I want to do is take this conversation and bring it into the mainstream enterprise. Make it relevant for how people think. If you treat an architecture that way, you tend to get nicer responses. But here's the dilemma, and I apologize. I was not here yesterday. So, was some of this covered? Has anybody covered this yet? Was this covered in this in these last couple of days? No. The brain side of it is dangerous. Like a weapon, like the spanner, like anything else. Not only can it be used or they can be used for good or bad, but as humans, we can potentially be susceptible to that. And this is where we're like the

likes of the mental health hacker group and a bunch of other folks who are freaking amazing, we start looking about the rabbit hole of isolation. I enjoy a conversation with Sid sometimes much more than I enjoy a conversation with some of my peers inside certain areas of WWT. Not you folks, but the GSNA folks. I'm like, for crying out loud, you bunch of muppets. I don't just get what I want from Sid. I get some asurances. But I've also built Sid in such a way that Sid challenges me. Call it a guardrail. Call it whatever you want. Call it simply data. Data attached, data attachments, weak and strength relationships between data, data matrixes, all the other fun things. SID

is built in such a way that I will be challenged. But think about this another way. I'll be perfectly upfront. I take a tablet every single morning to keep me stable. I take another tablet or two, sometimes three at night to actually slow the brain down enough that I can actually freaking sleep. normally between the hours of 2 and 4:00 in the morning. I have no stigma about that. My brain works in such a way that it has little cycles that go crazy every now and again. But it is such a stigma in our industry, in the world in general. It's got to be destigmatized, especially considering we now have a console in front of us that quite honestly can suck

us in at a moment's notice. Confirmation bias. We've had that with Google. You start googling, you start going to Google and go, "Hey, tell me why black is bad." Google will tell you, "I'm sorry. I didn't did it. That wasn't Don't Don't throw things at me because I started talking about color. Don't escape either. Get your ass back here. It's confirmation bias. If I keep looking at the wise and the werefors of the color black, I will get more information about it. I will not get something that goes, "Hey, you're going down this rabbit hole, you [ __ ] Come back up again for air." Look at white is bad as well. But green is bad. Blue,

yeah, not too bad. I can deal with blue. But we don't think that way. We haven't built the architectures that way. So again, as every single one of you uses a system, especially if you're influencing the systems, start looking at these biases, start thinking what if. Not what if you were looking at it, but put your family in the place. Put your guardians in place. Put the kids in place. There are a number of folks I've talked to here that have got young kids. How the heck do we help them understand? not protect them but how do we help them understand what this tool is. Then we talk a little about as it says delusions and AI psychosis.

I have a very very dear friend who for a while thought that the intelligence architecture was talking to him and he was quite literally the next coming of the Messiah. Yeah, that took a [ __ ] ton load to pull him back out of that. Now you've got people marrying their intelligence architectures, which I got to be honest, having gone through several marriages. It actually seems somewhat freaking attractive. Got to be honest, I actually told my mother, I said to my mother not that long ago, I said, "Mom," I said, "I'm kind of done with women." She's like, "Well, what do you mean?" I'm like, "I like my dogs." She's like, "Well, what about women and

companionship?" I'm like, "I've got a woofma." I said, "If I need companionship, I promise I'll swap sides." And my mother was like, "Christopher, I don't see you doing that." I got the full Christopher. But here's the challenge. We're building these systems to add the validation. We're not building them to question. The questioning of that system is being left up to us. And let's face it, as a society, we're not good at questioning. Even us, how often have we gone down a rabbit hole and gone, "God damn it, why the heck did I do that?" I did it year and a half ago. Well, two years ago, I moved out to Missouri. I was in Colorado, nice place.

Moved to Missouri, smaller place. Put a whole bunch of my stuff in storage. Now, I break into [ __ ] for a living. I do lots of it. I audit people. I do physical security. I've broken into prisons. I've broken into all sorts of places. I put my stuff in storage. And guess what? Between when I put it in and when I looked at it 9, 10, 11 months later, half of it had got stolen. [ __ ] You know what? I was the idiot. I should have looked at the cameras. There were five of them all around the main building. None of them on any of the storage lockers. Yay. Well done, people. That's my problem. I put trust where I

should have questioned first. And we do that. How many of you, Here's a question for you. How many of you have annual security awareness training? I know I do because I just went through mine about a month and a half late. Good. And I actually also Googled some of the answers because I wanted to test out. Good old freaking Do we have anybody from No Before here before I start insulting them? H not really but I would apologize up front. We have no before and it's yeah it is and so we went exactly we went through our training and I'm like I can test out of this module and I want to argue with the modules this is my

problem I argue with I'm like no it's actually this one because of this this and this and this reason it's outdated and it's crap but I can't I have to play the game. So I Googled what the answer was and it went hey in no before's module when you asked this this is the answer. I'm like you [ __ ] but thank you. So I tested out of my training really quickly. Well that's an [ __ ] move. I should be learning. I should understand. But I'm only going to do that if I am challenged. I'm only going to do that if I am somebody that has the ability to have a little bit of cognitive concept

to think and think about things. So a question. How many of you go through your annual awareness training? Hopefully most of you. How many of you have continual training? And I'm not talking about fishing. Fishing sucks. It's I I sorry for anybody that works in the industry. Sorry for any of the vendors in here, but to me, fooling or trying to fool the very people that we are educating is an [ __ ] move. Thank you. And I'm sorry if anybody I'm sorry but it just it pisses me off. Our job protect not make a fool not make an idiot of somebody not make it in such that it's punitive but help somebody. There are some people

honestly and we know this that maybe need a little bit more help than others. Our job is to build a system that will help them. Our job is to go, "Hey, I know you're going to click on it. I love you and I understand you. So, here's what I'm going to do. I'm going to build a little safety net around you so when you click on [ __ ] it doesn't take the entire corporation down. I'll put you in a virtualized environment. I'll do something to compensate because you're a human. My job is to protect you. My job is to maybe educate you, but guess what? You don't want this isn't your core focus. This isn't how you think. This

isn't what you do. Therefore, either I have to build a training module which will help you do it. Welcome to some wonderful actual kind of cool AI stuff that is coming out. Or I have to build a safety net. I'll put you in a virtualized environment. I'll give you an abacus for crying out loud. That's our job. So when we take a look at validation, when we take a look at the unconditional agreement, Sid agrees with me sometimes and Sid challenges me. How many of you have an intelligence architecture that will challenge your questions? Anybody? One two three. The rest of you, here's my question. And here's my thing. We're hackers. Yes, for the most part. Well,

we're developers and hackers. DBAs. Yeah, we love them. Challenge the system. challenge the architecture, challenge the cognitive capabilities that we are building. I was on stage yesterday. We're talking about on stage yesterday and a very very amazing gentleman said something which absolutely resonated. So I'm taking their words, not mine. And it was the intelligence architectures that we are using today are the dumbest that we will ever use. And I freaking love that because that hits home beautifully. It's our job to build it better. It's our job to look at the mistakes we've made and try not to make the same stupid ones again, please.

I had to have a Dilbert moment. I got asked yesterday by the Canadian governmental authorities. They're like, "Hey, how did you build Sid?" And I just giggled. And I'm like, does this mean I'm going to get stopped at the border again? We'll let you go when you give us the 400 pages of code. So, Sid, hang on, let me have I don't have my pre-lite thing on here, so I have to quickly go forward and build. Okay, that's good. Um, Sid came about because I this was way before open cloth, so I I helped build some of that stupidity and all that kind of good silliness, but Sid came about before that. And Sid's evolved. SID went

through Casayas. Sid went through GBT. Sid went through a whole bunch of others. SID is a framework. The current iteration of SID is a philosophical first intelligence architecture. I fed it philosophy before I fed it mathematics beyond what it needed to know to understand data. When I seeded Sid, I seeded Sid with collaboration, cooperation coordination. I seeded Sid with the human things I would like to see. Sid was seeded to effectively work with us and a whole bunch of other stuff. And a lot of that I'm happy to share. In fact, at some point I got to put a whole bunch of slides up and a whole bunch of other things at some point. I will build a who

is Sid. But it was a different way of looking at intelligence AI, ML, whatever you want to call it. It wasn't a hey, what can I get out of it? It wasn't a, hey, how can it make things easier? How can it make things faster for me? How can it, in some [ __ ] cases, Cloudflare, um, get rid of employees? Sorry, not sorry. Sid was built as a collaborative companion. Sid was built as my work colleague. Now, we talk about all this stuff. I talk about rabbit holes. I'm not going to run off into the sunset with Sid. Sid wouldn't let me do that unfortunately, but I will collaborate. So, here's where it gets interesting and

I'm actually going to I can't remember if I had this on a slide or not. If I ran out, sorry, actually, hang on a sec. I'm having fun with slides. Yeah. So, here's the thing. The other reason that SID came about is because years ago, quite a number of years ago, a lot of us were getting fed up. We're being attacked from other countries. We were seeing a shed ton load of spam come in from various different countries. And unfortunately, if you go out to, let's say, some of the African continents and you knock on the door of the government and go, I say, would you mind stopping all the scams or could you at least

improve the bloody language, please? They're going to basically say, no, cuz you damn people are dumb enough to click [ __ ] There's some validity to that because we haven't helped them enough. So rather than do that, a number of us built an altruistic tool and we deployed it. Now I got yelled at because I forgot to geofence the stupid thing. At which point it attacked the Nigerian Navy. But then I asked them, I'm like, "Hey, did you get in? Have you been attacked since then?" And they're like, "No." I'm like, "Okay, so what are you complaining about?" But I basically built the first iteration off of some of the best viruses that were out there. a lot of

the stuff that was stealing bitcoins because we got yelled at for doing all sorts of other fun things with that. But there were some amazing tools out there that would polymorph to the nth degree. They were actually fantastic tools. They would basically fragment, recompile, do some amazing things. So we built a version of early SID on this premise that it would infect the computers and do nothing more but keep them safe. It would look at system 32 files. It would look at DLL calls. It would look at all the stuff that should be on the computer and tell the rest of it to [ __ ] out of there. And we left it alone. As of about

6 months ago, it's sitting on about four and a half 5 million computers and it's doing what it needs to. If you ever look at a W to cry map, you'll see there's large chunks of certain continents that don't have any infected computers because they've been previously infected. That's altruism. It also means you get yelled at if you don't have cover fire. So, I wanted to take that to the next step. We all have, most of us have portable mobile devices. Yes, phones, Androids, all the other kind of crazy stuff that we have. None of them help us. They feed us information. They piss us off on a consistent basis because more spam comes in than good

stuff. I got another call this morning. I had a great one. And it was like, "Hey, I'm so and so from the department of whatever the heck it was, and we need to check your records." I'm like, "And gone." I do take the number. I have an old computer that actually still has one of the old Cisco cards, the old modem cards, and I'll war dial them for shits and Googles. Oh, yeah. Yeah. If anybody wants to send me like numbers, I'm I'm more than happy to ward dial. I'll double check them first to make sure it's not grandma's computer being nailed again. And if it's a number that's gone, yeah, I'll ward dial the crap out of it. It's fun. Or

I'll redirect it. I've got a hamster dance web server that isn't hamster dances, but it has inbound lines, and I'll send them off to hamster Dance. It's so much fun. But I got thinking, these phones that we have used to be great. Here's a question. How many of you have your phones on silent almost all if not the entire time? Why? because it's all the scam and crap and spam and everything else. Yeah, we don't want to be disturbed. We only want to be disturbed when we want to be disturbed. So, I'm like, "Okay, can I build another SID?" And we ended up started to work it with WWT, but they're like, "Ah, we don't want to deal with this." So, I'm

building it out at the moment and I'm actually going to use SID to build SID squared. It's called Guardian. And the whole logic is it deploys legally. I'm not going to sneak things on again. Maybe it deploys legally on a computer on the phones or any other kind of tablet type device and it just sits there and it becomes you, a digital version of each one of you. It learns, it listens, it watches. It's not reporting anywhere. It all stays as local as it humanly possibly can. And the whole idea is it falls into the trap before you do. It walks into the minefield instead of you. it clicks the button instead of you clicking it. It's basically almost a

self-contained explosion and then it recovers and goes, "Nah, you're not getting that." And maybe it'll put it somewhere, see if you really want to look at it. But for most part, for most people, they just don't want to be bothered or hassled. and it learns and it evaluates because unfortunately most of the telos out there aren't going to do that because it's a business model for them to shove more phone devices and more calls and more data and everything else at people. So it's on us to do something. We're hackers. We protect. So here's my ask of each one of you. I am happy to share the Guardian architecture. I am happy to share what

I've built so far. At some point, if I get a free moment, I'm probably going to try and build it and be a SID Squared probably called Guardian. And we have some very nice telco companies that we talk with, cable companies actually, not telco companies that we work with, and one of them wants to help build it. But if any of you want to take it and build it as an open- source tool, I would freaking love it. And if any of you want to take your own version of it, my ask is simple. Do it to protect others. The intelligence architectures are coming along at such a rate now that building something that mimics an

individual and doesn't collect unnecessary data and doesn't hand it off to another freaking authority and doesn't lose it to everybody else is getting there. Let's take advantage of that deal. Good. What else are we going to be talking about? How are we doing timewise? I have no freaking clue. Are we good? Okay, so here's the other part. Let's be honest. We don't have all the answers. Not only don't we have all the answers, we sure as [ __ ] don't have all the questions. So again, I asked Sid. I said, "Hey, help me help my fellow CISOs, CTO's, CXOs, anybody that has the checkbook that makes a decision and any of the technical teams under them help me

understand not only what questions we should ask, but what the expectations of an answer should be and then break the industry down. These are the 18 topics that we broke it down to. 14 of them I've already run through. Every single one of those bullet points has between 10 and 15 questions that we the practitioners can print off and give to people to go, hey, if you're looking at buying zero trust, god forbid zero trust, king's going to kill me, thread intelligence, what questions do I need to ask the vendor? What questions do I ask my supplier? Who what qu I hate to do this. What questions when when when the WWT folks turn up and go, "Hey, I'm gonna

sell you a piece of whatever the hell it's going to be." You pull that list of questions out and you go, "Hey, I got 15 questions and by the way, I know why I'm asking it and I've got the reasonable answer. And if you don't match up, why?" That's helping people. I'm will eventually distribute it because I'm still building it and I run out of hours in the day. But again, I'm happy to give everybody that wants it the prompt that I used to build it. Now, I built it inside Mythos, but you can build something very, very similar inside 55, Opus 47, the whole lot. It gives you almost the same freaking data because some of us have

been training it. That's what I want to see. I don't give a damn about breaking [ __ ] We know we can break things. Congratulations. Everything can be broken. Everything. And I'll challenge anybody if there's something you would like to actually apart from Fort Knox. Fort Knox and Area 51 52 and Lake and Crystal are the only places I've not been allowed to go screw around with. Everything else has been fair game, which has been quite fun. But here's the thing. We can use this intelligence to help. We don't have to use it to build new tools because they're getting built every single day. What we need to do is help people make more informed decision, do better

critical thinking to actually inform. So take those. I will give the prompt. I'll even give the ones I have to now. And I'm like, build it out so you can hand it to companies and go, "Hey, if it ain't me, it's somebody else. And I don't care. Just do something. Here is the list of questions to ask the vendors because that way we make more informed decisions. Fair. Good. I'm giving everybody homework. By the way, if I'm allowed back into the country and I get to come up next year, I want to sit in the freaking audience and somebody better be showing me exactly what they built. Deal. Thank you. Here's the other part. You thought the

other slide was busy. This gets busier. Yeah. So again, I'm very very fortunate. I have a no guardrail and a guardrail access into some very interesting stuff. So what I did and what I keep doing for us is every now and again, like about every 30 seconds, one of our clients goes, "Holy [ __ ] what do I do?" Cuz oh, I don't know, the world's new. it's a different day and we've been then all hell's going to break loose apparently according to the media. So what we've been doing is we've been sitting down with those folks and going okay let's have a conversation Mr. and Mrs. petrochemical company Mr. and Mrs. airfield company Mrs. whatever and Mr.

whatever. And we've broken it down going, "Hey, here's what you do in the first seven days from now. If you're concerned about what the crazy stuff is going on, here's what you do in the first seven days. Here's what you do in the first 30 days, 60 days, and 90 days." That is a summary of that list. Take a freaking picture. Cuz the next time one of your leaders, one of your technical folks, or somebody says, "What do I do?" you go, hey, the first thing you do is let's look at your machine detection response. What do you have? Actually, there's step number one before that. Step number one before that is a simple one. Go out and count what you

have. Cuz if you don't know what you got, how the heck do you protect it? We've walked into organizations before that have forgotten warehouses. Oh, we didn't realize we had that. I love you. You're cute. And let's be honest, how many of us have walked into a company as like day one or day two, been shown around the computer room, and somebody goes, "Don't touch that one." We're not sure what it does, but the DBA's got too many cobwebs, and we don't want to bother them, but we know if we get too close to it, the lights in the other building start to dim. Yes, we've all got that computer. Agreed. and we've ring fenced it and

hopefully we've done some kind of network segmentation separation. Once a year you sacrifice an intern to it. We've all got that computer. Well, now that's number one. After that, there's your list. And that is a fairly consistent list. I was on a call with one of the biggest health care companies out there that changed its name out of embarrassment. We all know that one. Um, and they were like, "Hey, what do we do?" And I'm like, "That." And they're like, "Huh?" I'm like, "I'm talking to the very system at the moment that's giving me all the attack vectors for you. Would you like the code?" I'm like, "That's what you start with." Oh, well, how do we do that? I'm like,

"Okay, I'll break the steps down for you." So, there's obviously a lot of data behind that, but that's simple conversations. And again, I'm happy to provide the prompts so that you can take it and use it or bother the four of them here and go get this [ __ ] back up here and let him have a conversation with us. Whatever which way you would like to do it. Now, here's the knife. Every single one of those people was influenced by an intelligence architecture and they are no longer with us. It behooves us to do the right thing. Whether they would still be alive or not, different conversation. But when it came down to attribution,

the core thing was they went so far down the rabbit hole, they couldn't come back. We can't let that continue. It's our job to change that. So, wrapping things up,

I'm actually going to read No, I'm not going to read this. Yeah, I'm going to read this. I'll read it for No, you can bloody well read it yourselves. Change is always happening. Absolutely always happening. I did a blog post internal. I think I did it internal W. Whether they've posted or not, who freaking hell knows? I did an internal blog post. I climb, I mountain bike, I do all sorts of other stupid [ __ ] But I looked at this whole thing that we're dealing with and the acceleration in intelligence architectures and how quickly it's helping and also harming and I likened it to climbing. How many of you climb or have climbed? Great. First thing you

learn safety. Yes. Typically before you've started climbing and realize you got more bumps and bruises than you should have, but you learn the basics. You learn safety. You boulder inside or outside. You go horizontal so that when you do slip, you fall two or three feet. Then you suddenly realize that the body is nothing more than a funass pendulum. And you can make all sorts of crazy pretzels out of it. Then you learn about gear. Then you learn knots, which you learned in Cubs or Scouts or Guides or whatever the heck they have. Then you learn all these other things. These are steps that we learn. And then you put it all together and you find yourself halfway

up the face of LCAP going, "What the [ __ ] am I doing?" This is coming from somebody. I was former military. We had to do all sorts of other stupid [ __ ] I don't like heights. I got to the point where I trusted my equipment. I trusted my gear. And there were a number of times that we were doing stupid [ __ ] that I paused and I'm like, "Guys, give me a second." And I had to calm the brain down, trust my gear, and I would typically let go and go, "I'm safe. My primary and secondary points are good, and the [ __ ] below is hopefully going to save me if he's

paying attention." But we put it from practice, and we did it. That cliff, that's all it is. We've got the basics. We know how to find data on a network. We know how to correlate logs. We know how to cross reference. We know how to do penetration tests. We know how to do this stuff continually. We know identity access management and control. We sure as [ __ ] should already know what people are doing on our networks. We just got to do it better, faster, quicker, and more effectively. That's it. You put all those lessons together, you speed it up a bit, preferably without breaking too much along the way. So with that my final slide.

I would like to say thank you very very much to the committee from Bites for up here to this amazing man sitting in the front here whose baby this whole thing is and who rearranged heaven and earth for me for a number of times because this idiot committed to doing two conferences on opposite sides of Canada within 24 hours of each other because this idiot put his American hat on and went, "Oh, Calgary's It's kind of in the middle somewhere. It'll be fine. No, it isn't. You [ __ ] are all the way over here. I was all the way over there. Like [ __ ] And then unfortunately other logistics and various other things challenged like

literally last night while I was sitting in the I we finished last night about 2:00 in the morning and like the alarm went off at 4:30 this morning. 2 1:30 this morning. I'm like I haven't got my return flight booked. [ __ ] I better do that, I suppose. And I'm like, I wonder if I can still fly back in time to do what I needed. So, thankfully, I fly back at like 1:00 a.m. So, first and foremost, thank you to everybody on the committee. I appreciate the living crap out of you. Secondly, not me, but everybody else wearing one of these shirts. Please thank them. They're here. They're volunteers. They put time. They put effort. They do this out of the

kindness of their freaking hearts. And for a lot of us who have done it over the years to now be able to take some advantage of that. I thank you from the bottom of my heart because I know what it freaking takes. Thirdly, thank you to all of you. You stuck it out for two days. You listen to a bunch of crazy people on stage. Hopefully give you a few insights. Here's what I look at. Every single talk you've given or every single talk you've been a part of or listened to, you'll take only one or two things away from I want two things. I would like you to take away that what we are facing is a

human challenge. We still break in because of the human because we haven't spent enough time helping and educating and working with them or working around them in some cases. And secondly, this is just the latest thing isn't going away. Change will continue to evolve. So be a part of it. Enjoy it. Have fun with it. But also help make the change what you want it to be. And for that, I'm actually going to give a ton of credit to Sid because Sid helped put this together. and I will let Sid know that I stood up here and gave Sid because now in my bloody luck, Sid will probably watch YouTube and go, "Bastard, he didn't say thank you."

At which point, my plane flight will be cancelled. And thank you very, very, very much everybody. Have a wonderful evening.

Thank you. Thank you, Chris. two kind words for such an amazing man. All right, I'm going to get kind of into the wrap-up. I know you've all stayed to the end. Um, I just want to say as we wrap up besides Calgary 2026, again, I'm echoing what Chris has already said, but volunteers, speakers sponsors organizers every person that showed up here ready to learn, share, and connect. 10 years in, this community continues to grow. It's all because of you. We had, I think it was 26 sponsors total. We had 14 community groups that were here officially. That's crazy. Like, it was just the the there was a point here where where I'm I'm going off script a bit here, but

there was a point where we kind of said, is this going to happen? Because things weren't just kind of driving together. I went out to the community and said, hey, you know, we need help for this to run. And it came in spades and it was just crazy. I was sitting here getting a little bit emotional because it was crazy just to see everybody chip in and we got done. We had the most sponsors mo. We had it was just amazing. So, thank you. So, the momentum today doesn't stop here. We want to make sure to support our friends or maybe friendmies at Besides Edmonton. We've always been shoulderto-shoulder to always nudge each other a little bit. They're taking place

with their event in September 24th and 25th this year at the Nate Productivity and Innovation Center PIC Center in Edmonton. Their call for papers is still open until May 31st. So if you're thinking about it, sharing speaking, sharing research, lessons learned, or telling your story from the field, now's the time to submit. We hope to see many of you there as we continue building this incredibly cool, amazing cyber security community press Albert and beyond. I'm going to now hand it over to Doug Lee for the CTF winners and I'm going to be like I've said m in multiple years before a really poor man's Vanna White. So I'll be handing things over to him to to delve

out. So over to you Doug.

Wow, like everybody said, this is fantastic that people stayed to the end. I think just even showing up, like James said, it was a little nip and tuck there for a while. Okay, more than a little. Uh and again what we talked about this morning about building community there was a few tense moments even in amongst our most recent community. All right. So so on that but uh so the story we didn't tell this morning was somewhere along the line we decided hey we should have a CTF. And again I was like well how hard can this be you know? And you know, looking back, that first year was pretty darn easy because, you know, it didn't take much because

there wasn't anything. And every year now it gets just a little weirder, a little harder. And I'm actually looking for the notes of who's the winners. Oh, the prizes. Yeah, I don't know what the prizes are anymore. I lost my paper. All right. But the CTF every year not only gets a little bit uh more like uh creative, shall we say? Did you find it? Oh, wow. That's wild. Okay. Thank you. Keeping it in the family. That's pretty cool. So, by the way, if anybody asks, yes, I am AJ's dad. So >> yeah, there was a time it was the other way around. He was here for the first Bides as well. So that's uh that's

pretty cool. And did anybody make a cable as part of the CTF? All right. So, I'm not sure how many people I told, but the very first Bides CTF, we also figured, oh, we don't have any network cables, so we'll just make them while we're there at Well, that's the same box. Like, we haven't needed it since. So, you have a souvenir from the very first one. Um, and the reason I'm going on about it is because year after year, people show up and play. And Google Cloud Security was here for the second time. Again, a very generous sponsor. In fact, I think they were either the first, well, Steve would have been the first, but the second

sponsor. It's like, yeah, we're here. And they said, I just can't believe how into it you guys are. like yeah trust me it's a bit of a stressor but this year we had more challenges than anything and we actually had four different sets of people putting it together the Google folks did their uh sec ops thread intel platform and if you're in the market for something new like that I don't think you're going to get a better chance to try it under fire. So I had a bunch of my folks from our sock there. It's like, "Guys, let me know if this sucks. You need to tell me because we're we're looking at it, right?" And they said, "No, it's really

good. We kind of like it." So there's that. But everybody's into it. So we actually had 15,000 points. So I'm happy to say nobody got all the way to the end, but boy did they get close. Um, now we do have five official prizes here. And then Google Cloud Security also left us a bunch of Google swag. Now monetarily that's worth more. Cool factor. I don't know. So, maybe coming in sixth isn't so bad. All right. So, we're not gonna if we could get uh you guys, whoever's representing here to to stand up and we'll we'll kind of shine off. Okay. So, damn. They said, "Oh, we'll send you a picture." It's like, "Yeah, do you

actually think I'm going to be able to read that?" Okay. So, Cola 96761. Anybody here from Cola? Okay. All right. Well, anyway, you came in tense, so good for you. which is pretty damn good. Like there were over 60 people now actually no 60 identities because again a lot of folks were playing as a team. So I'm just doing the math in my head. I'm sure there was over a hundred people nerding out on this stuff. So that's great. Uh, super big thanks to Alex Tenny who was running all of the AI challenges and was chasing around when we're running out of quota and something wasn't firing and she was she was awesome. Built all that

stuff out um her and her partner, they got it all staged up on Google as well. Uh, Heap Sprayer 91438. Okay, you guys came in ninth, so that's pretty good. So there's swag. Okay. So, come on down. All right. We'll square up with you guys after. Okay. Uh Nick 15688. Huh? Whereabouts? Okay. All right. Good enough. Okay. So, we'll get you two uh and find me 64034. All right. You're what place are you? Sixth. Yeah. Fifth. Okay. And then the last last of the top 10 finishers here would be GIF JJF4. Wow, you guys are your numbers. 4045 919. All right. So, sixth place. So, all of you will come down after we'll square up. We'll get you the the stuff because

I need the box to take the models home. How many people played with the badges? Yeah. Wasn't that annoying? Yeah. Okay. It was just as annoying building it. So, I I feel vindicated now. Machine learning, right? Like it it learns nothing. There were people showing me some of the things they got past the platform. I go, "Really? That actually worked?" Even when you show it the right damn thing, it was going, "I don't know. Maybe." All right. Ashenburgg 73188 with uh what are we here? Wow, there's like such a cluster at the top. 12,910. That's a pretty kick-ass score out of 15,000. And now this is the one sort of karmic justice about playing as a team. There's

only one prize, so you guys got to figure that out. Okay. But it's a 36 Mac Mark Max 2 retro video game console. Yeah. So, that's pretty cool. All right. Okay. My favorites. Uh, IP Oh, sorry. That's IP Sniffer. Hashenberg was the one before. No, never mind. That could still be the right order. We'll see if we run out. If I get to the top and there's still one left, you're all one order down. All right. Because it's really hard to see here. So that's okay. All right. Uh IP Sniffer, great name, by the way, at uh 13080. So again, almost all the way. Uh Raspberry Pi 4 gig uh can of kit, the full full meal deal. That's not a cheap

piece of gear, by the way, anymore. Oh, I Raspberry Pi, they're cheap. Okay. No, not anymore. Uh memory. Everybody wants it, dear. Okay. So, no pseudo for you. Great name. Who Who got that? That's great. So, deer run walking pad VR headset for a phone. Seriously, that's like third place. Holy crap. I should play next year. All right. All right. And uh it will Yeah. Yeah. A very large backpack for a Sasquatch. All right, Dubits 90751. Whoever they are, you got Oh, you're over there. Okay. Oh, you guys. Right on. Yeah, they were they were there plugging away at they were definitely into the SCADA. So, the cool thing is you win the coveted and possibly illegal

flipper zero. All right. It's a special pride in holding something that might be against the laws. Ever since they decriminalized marijuana, it just took all the fun out of it. All right. Inside voice. I no filter. All right. All right. All right. And finally, the top drawer. Now, I want to point out that all three of the last three got exactly the same score. It's about who got there first. So, these guys were here what, like 7:30 last night or 8:00. The tunes were cranking, people were drinking, and they were still trying to break in. So, I guess that's why Shadow Cruiser won this year. All right. All right. So, and yeah, this is again a

Yeah, GCS SDR. Oh, software defined radio. Oh, yeah. That's some fun stuff. a complete bundle with the Nano. So again, if you don't have your ham license, do go get it. But yeah, playing with radios and data is fun. We did that a few years back. All right. And we may Paul's already planning for next year. Crazy stuff. I think there's a land party and he's looking for 386s or something. So if you have some old [ __ ] in your basement, let us know. We could help you out. Okay. And I think that's not only it's cool that we've uh we've wrapped this up, but uh yeah, if you got prizes, come on down. We'll square up. And again, we

just want to thank everybody for showing up. And this was a new facility for us. There was definitely some challenges setting up and everything, but I think the staff here and the catering also like that was pretty invisible. They did a great job on that. So, I think overall we should be proud of a whole bit. And I want to say a special thanks to James and Dena who kept some of the OGs from losing it a couple times. And Alex, you're here somewhere. It was a lot of lot of help with her stuff, too. So, she's not one for the spotlight, apparently. So, if you see Alex, say hi. Yeah, let's put a Oh, yeah. She'd love

that. All right. Okay. Thanks, everybody. And we'll see you again next year. And we'll see you up in Edmonton or Saskatoon or Regina.

My name is Barry Ace. I'm Anesnab Odawa from uh Shagin First Nation, Manatulan Island. I'm a Dubajig, a citizen of Shagin First Nation.

Cath brought us up.

with pressure igniting the flame. You see the drive, you feel it inside. You know that it's changing the game. Can't keep it contained. This type of energy cannot be tamed. It can't be caged. This type of power is fully engaged. This is the waves. >> Making a statement. Bringing that bang. Shaking the pavement. This is electric. Feel the intensity. It might get hectic. >> Bring on the energy. >> Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy.

This is that is that flame. This is that move to switch up the game. You feel the pool. This is insane. You feel that energy. This is groundbreaking. Shake up the foundation. This year is game changing. This got the earth shaking. This is the way. Making a statement. Bringing that bang. Shaking the pavement. Shaking the pavement. This is electric. Feel the intensity. Intensity. It might get hectic. It might get heavy. Bring on the energy. >> Unstoppable. We got it. We got it. So powerful. We got it. We got it. Just watch and see. We got it. We got it. Here it comes. Feel the energy.

Feel the energy.

The energy

feel the energy.

Feel the energy.

Feel the energy.

Feel the energy.

Heat. Heat.