BSidesSF 2026 - CISO Series Live Podcast Recording (Panel)

So, a as we're getting set up, I'm just going to kind of explain what's going to happen here. Um, so we are going to do a live podcast recording of the CESO series podcast. Hopefully, many of you are listeners of the CESO series podcast. Yes, that's awesome to hear. I know a bunch of you were in line. I handed out stickers. If you did not get stickers and you would love to have stickers to put on your laptop to tell everyone you listen to the podcast, that would be great. Thank you. If your phone is not off, please turn it off now. Uh, so we don't have that. This microphone right here is pointed at you. So that

means any applause, any laughter or anything like that is going to be picked up here and it will be recorded. By the way, this episode that we're recording right now is actually going to air on May 5th. So just so you know, we will edit it and so there might be a stop down and something like that, but we're going to try to run just 45 minutes straight and record the whole darn show right in front of you. And we're going to do it as a live fun show just like you hear it in general. Um, also there's a bonus for those you are having fun with this immediately after this. So we're running this from here. What time is it now?

Well, from 2:45 to 3:45 or I'm sorry, 2:45 to 3:30 immediat. Ah, thank you very much for putting up the house slides. So 3:30 at um, and by the way, any of you in the back want to come down even closer? They're great to fill it out to make it cool for the photographs. that would be wonderful if you want to come down. Um uh at um immediately after this at 3:45 in theater 1, we're going to be doing a game show and there is going to be a chance to win a prize. It'll be a lot of fun. We're going to play, god willing, nine different games and it'll be a lot of fun. Does I'm

sorry. Was there anything here I needed? No. Okay, cool. Um All right. So, uh are we all ready? Sarah, are you ready? >> Yeah. >> I will introduce them all in just a second. All right. Let's I'm going to hit the record button. I'm going to ask for 10 seconds of silence. Uh, and then we're going to begin the show.

>> Sarah, you ready? >> Biggest mistake I ever made in security. Go. >> Saying I told you so. Um, there's nothing wrong with mistakes as long as it doesn't have a material impact on the business. I love the saying uh there's no losses, there's only lessons. But when I think back across my career and I think about the mistakes that haunt me at night, it's the times that I have bent to the pressure of business and I went against my better judgment and I put myself in a position of seeing things you can see around corners and you know it's going to happen. And when that does and you say I told you so, it's never a good feeling because it's

not our jobs to say I told you so. It's our jobs to manage risk and never have that happen. It's time to begin the CISO series podcast recorded in front of a live audience in San Francisco.

All right, welcome everybody. Welcome to the CESO series podcast. I am David Spark. I am the producer of the CESO series. Oh, look this guy to my left. You may know him. It is the CISO for Rivian and my co-host since day one, Mike Johnson. Let's hear it for him. >> Hi everyone. What an amazing audience. >> We have a huge crowd here today. Great. This is great. Now, by the way, uh we are at Bside San Francisco 2026. Let's hear it for Bides. >> And this is our four, I believe, our fourth time doing a live show here. I I think that's right. Yeah, this is >> not for you personally because I know

you missed one. >> This is my third. >> Number three. And we've done four live shows here. Thrilled to be back. This is pretty spectacular. For those of you listening, we are in a gorgeous movie theater with our giant logo and all our sponsors logos, which I do want to mention right now. Our three sponsors, and I want to hear a huge round of applause for them. Our sponsors, Nudge Security, Quiller AI, and Zenity. Let's hear for all three of them. We are going to hear a lot more about them in just a minute. Now, I want to bring uh our guest in who is our far left, a brand new uh actually we we had

her on once before. So thrilled with how well she did before I said you got to come join us live for our show at Bsides. Big round of applause for the CISO of Convera, Sarah Madden. Let's hear it for Sarah. >> Hey everybody. >> All right, so here's my question to everybody and this is my challenge to the audience. Now, if you listen to the show regularly, you know I often talk about this thing where I love and I don't get to see it often and I'm hoping you will where people are in an event or trying to get into a party and there's going to be a lot of events and parties this week during RSA

>> one or two >> in one or two and they're trying to get in and the line that is used or a variation thereof is do you know who I am? I so want to witness this. I want to witness it badly and I want to throw this out to you. So if anyone witnesses it, by the way, bonus if you get it on video and can send it to me as well. But let me ask both of you, have you actually witnessed someone pull this off? Try the do you know who I am? Do you know how my influence like why am I not in this event? Sarah Mike, >> many times in my life. Yes.

>> But I don't have trouble getting into parties. >> Oh, there you go. >> Well, I I I pull that card all the time. >> Do you? Yeah, absolutely. >> And and what do you drop series? Do you drop Rivian? What is it you drop? >> Well, I I dropped that. I know David. >> That And does that not get you kicked out? >> It usually does. Yeah, >> it does get you kicked. It's never been successful. >> By the way, it doesn't count if they say it because they've had a concussion. Uh do you know who I am? Or ask, you know, what year is it? >> I know who I am. Yes, >> you're a very important person.

>> There you go. >> I believe I am. Is this the best use of my money?

>> Quote, "When you only see 5% of the options, you can't make the best choice. You can only make the best choice from what you saw." End quote. Now, here's a pattern noted Richard Stennon of IT Harvest. Evaluate three or four vendors. Pick one with the best demo. sign a three-year contract. Now, 18 months later, rip it out because the vendor got acquired and product development stopped or a better solution emerged or the market leader turned out to be the marketing leader. Now, the implementation time is wasted. And don't forget the squandered political capital spent justifying the original decision. Due diligence isn't just about evaluating vendors, but knowing which vendors to evaluate in the first place.

So, Mike, I'm going to start with you. What does your vendor selection process look like to avoid this trap? And second, how do you balance the fear, if you have it, of making the wrong choice against the need to make a decision and just move forward? >> Well, the first step is don't sign a three-year deal the first time that you're working with a vendor. Period. >> Period. Full stop. >> And have you, let me ask you, have you maybe in your earlier days ever signed a three-year deal at the beginning? >> Oh, I think everybody makes that mistake. Okay. like at some point you you go this was really bad and then you spend the rest of your career thinking

I'm never going to do that again. So that's step one. >> By the way that I got to what you just said that's going to be the life of a CISO and I'm never going to do that again. >> Oh daily like oh well that was bad. Let's not do that again. >> All right. >> But the the reality is this really is where our networks come in handy. like if I'm looking for a solution to a problem I have, the first thing I do is I turn to my peer CISOs like how are you solving this problem? And that helps build the short list because you then have context. You're not just going to

Google and saying, "Hey, I need to solve this." Uh or worse, going to Gartner and saying, "Hey, I need to solve this." Uh it really comes down to talking to your peers. And then it's like construct your list of questions. What are your requirements? Make sure send that out to all of your short list. Get those answers back. Pick a couple that make great answers. Then you do your proof of concept and that makes you understand whether or not it works in your environment. And then you pick one from there. And at at the end of the day, you're still going to make bad decisions where things are going to happen. One of mine is we worked with a very early

vendor. They were great. We actually we had a one-year deal because we signed one-year deals first and then we signed a three-year deal and they got acquired and product development stopped. >> Ah, just like I said, >> so it can still happen, but at the end of the day, >> can't predict the future. >> At the end of the day, we felt confident in the decision. We felt that we had all the information that we could possibly get. And sometimes you just have to recognize that you're going to make some bad decisions here and there. >> All right. I know Sarah, you've had struggles with some vendors, wonders, vendors that have been implemented. What Well, what do you do when things

are going ary and you're trying to figure out how how to get out of this? >> That's a great question. Um, I'm in a situation where we're stuck in multi-year licenses. Yeah. Hold on. Were these three-year deals you made or somebody else made? >> They were threeear deals. So, I got to build a green field security program at the company that I'm at now. and four years into it. And so, well, we did three years initially because we didn't have time to displace vendors. Like, we still had to >> hire and deploy all the controls and you're not going to mess with that in the middle of, you know, one or two year license. And so, came up for um renewal

last year with most of our three-year vendors. And I put them in either one or two-year um licenses intentionally for that that reason. Um, but you run into a situation where um, you have to continually air your grievances with your vendors to force them to do the right things and you leverage the stick of I won't renew, right? Um, we're in challenging times where the majority of the tool sets that we have are SAS products and they're going up and down. They're unavailable a lot. I think product development has taken a shift where we used to do N minus one because you don't trust the latest release and we've got tons of vendors now that are

saying don't trust until I have like a preferred release and that could be months if not quarters if not nine months out. So I've been pressing a lot of vendors lately of just sticking to fundamentals of software development and being confident about the software that you ship to us. And if you get into a situation with your vendors where they're shipping product that isn't stable, isn't good, is causing issues, I'm all over them all the time now because I think we're taking a a shift into lowquality releases. And this is across most of the vendors we have in our tool set right now. It it's it's happening across the board. So I'm pushing on that a lot as a CISO because

I don't want that to be like a new normal. We can't get ourselves into a situation where we used to be an N minus one and we thought that was okay and then we're dragging further beyond that. like that's not a good position for us to be in. Right. The other thing I'll say in terms of um finding good tool sets is it's it's not it's also the pier. I agree with everything that you said, but some of the best tool sets we have, I've learned about them from new engineers that are entry level people that got to test out cool tools in their prior job and they're bringing in really cool new tool sets. And it's as much

about finding the right tool as it is making sure that you have the skill sets on your team to manage that. Oftentimes we pick tools because the engineers we have on our team have expertise in being able to manage that product because you're never going to get a perfect product, right? There's always going to be issues, but if you don't have the talent to manage it, you could have the greatest tool set and you're you could buy and it and it's not going to work because you don't have the right engineering to manage it. So it's about people too. It's about their skill sets. >> Is AI going to help us or hurt us? quote, "Jen AI is deceptively complex.

It gives you the false impression it's simple to use and delivers excellent quality, all good or better than your own work." This is the trap. Now, Howard Holton of Gigome recently shared a striking Gartner stat. 74% of organizations are seeing AI productivity gains, but only 11% see clear ROI. We've convinced ourselves AI output is quote good enough right out of the box. But the reality is that AI presents information with such confidence that we believe it. You know, the polished formatting, authoritative tone, and coherent structure. Holton argues that if you accept lowest common denominator AI output without investing time to develop content about what defines quote good, you're telling your employer you're as replaceable as a

public AI model. Sarah, I'm starting with you here. How do you help your teams recognize when they're falling into AI's confidence trap? >> We don't trust it and we use AI every day. Um, >> hold on. You don't trust it, but you use it every day. >> Yeah. So, how do you >> I mean, you question your output every time right? >> So, you question and so how do you and does your team know to do that? >> Yeah. I So, we use AI every day, especially in sec ops. Like, it's so efficient for us to run a particular IDE tool in a very big cloud, you know, fork some code, throw it in there, have it

analyze it, we get results back in five minutes that's contextualized and it's actually super helpful. And then that usually starts like a little bit of a panic and a spin cycle and we like, "Oh my god, look at these results." And then we don't trust it initially. We go and we look at it and you end up finding that there's pieces of it that's super helpful and there's areas that you could tweak and it's usually not as bad as what the tool says because it likes to be flashy and dramatic, right? So you just don't trust it and you look at the results and to me it's not too much different than our regular vulnerability

testing that we do with the various different tools we have or the pentesters that we use. There's false positives in almost everything we work on. So just simply don't trust it. Um, and then when it comes to just use cases that we're building out, the human in the loop control is just super important until we get more confident with the accuracy of the AI output. >> All right, Mike. Uh, same philosophy with your team. >> And I think that really is the new normal. Like I I think the six months ago it was, hey, these things are really confident. I'm just going to blindly believe them. But nowadays, everybody understands that you need to doublech

check. And that is the evolution that we've seen. I you know this airs in May. I imagine things have even changed by then. And so we we will continue to have that moving forward and we will learn along the way. And you Sarah mentioned >> false positives is something that we actually work with a lot in security. That is not a new normal. That is not new for us. That is normal. And we should make sure that we're reminding our teams like, yeah, you need to check these things work. And you're seeing this in the industry where AWS had an outage recently that was AI induced, however you want to to call it, and they're making sure that they have

senior engineers checking AI generated code going forward. And these are things that we're going to continue to learn. And yeah, nobody trusts these, but that's okay. Who's our sponsor this week? You know, AI is spread to every corner of your tech stack. We were just talking about which is great for innovation, yes, but not so great for security and governance. And that is where one of our wonderful sponsors, Nudge Security, comes in. Nudge discovers shadow AI across your organization. Here's what's good. also including chat bots, agents, MCP server connections, AI in the supply chain of other SAS tools, and even more. And Nudge gives you workflows and automation to scale AI governance without slowing down productivity. The

best part, you will have a full inventory of AI assets on day one of your free trial. Take advantage of this, by the way, even those introduced before you started using Nudge. By the way, they will scan your environment to tell you well how how SAS loose it is, which it is. So, no time machine is required here. Gain visibility and control of Shadow AI risk. Get started by going to their website nudgecurity.com/shadowai. And it is spelled just the way it sounds, nudge sec nudgecurity.com/shadowai.

It's time to play What's Worse. >> All right, everybody. For those of you most of you are familiar with the What's Worse game if you've heard our show before, um, our our fans, they send in great scenarios of just horrible things that happen. Uh, all usually fictional. every now and then we get a real world scenario and those are actually kind of fun because >> uh it was like a an you know kind of a Sophie's choice decision that they had to make and uh we find out the real story and we see if our the panel had actually matched that. So if you've got those please send them in. We always like to hear great what's worse

scenarios fictional or real. All right this comes in from Craig George of Guidepoint Security. Mike will answer first then Sarah. You can agree or disagree and then we're throwing it to the audience to find out what your answer is. So here we go. Scenario number one. For years you've been running a security program held together by a few very exhausted security heroes. You know it will collapse if two people leave. All right. Okay. Your security department is gone if two people go. >> Okay. >> Now for years you've had that's scenario number one. Second scenario. For years, you've had unmanaged service accounts, API tokens, and non-human identities that no one fully owns. Kind of partially, but in both cases here, the

two scenarios, you're stunned that nothing has exploded because they're both powder kegs, just ready to explode. Mike, which one is worse? >> Yeah. So, in the first scenario, what's interesting is these aren't opposite sides of the same coin, which is what we usually get is like this or that. You got >> which is like Yeah. It's you got all of this and none of that or all of that and none of this. >> Yeah. So, you've got a a fragile team or a fragile environment. Maybe that's >> a poorly managed environment in general. >> Sure. Sure. >> Uh realistically, and I'm, you know, I'm not trying to change the >> By the way, this is a rule of what's

worse. You can't change it. >> But the the second one that those are technical problems that those are far more solvable than people problems, >> but they're not going to get solved. That's the thing. it stays like this, right? And like the reality is if you have some people who really know the environment and if they leave everything comes crashing down, you're really going to be in bad shape. And that's that is where you're going to end up with a a big issue versus uh it's kind of like the known knowns versus the known unknowns and the unknown unknown. >> Well, but the thing is first scenario, you're kind of your your security program is running reasonably well.

Sounds like second scenario, it's kind of crappy to start with. >> Well, the the the first scenario is something is about to happen. The second one is something has >> has happened or something is very visibly weak. >> But both of them are something will happen. It's a matter of time. >> It's a matter of time and then one is the the issue. And frankly, I think the the problem that you've built a team around two people that actually is the worst. Well, no, no. It isn't just two. There's a bunch. But if two go, >> but you've essentially built it such that >> built it that two go. It's going to be >> So, I I think first scenario is the

worst is the worst. >> First is the worst. All right, we're throwing this to you, Sarah. What do you think? >> I agree with you. I The reason you've got unmanaged service accounts is because you only have three people. >> Well, who knows? >> So, if you lose any of those, you're you're probably >> Well, it's a difference in the second one. It may not be that you have exhausted heroes in the second term. It's just >> well if you lose your if you lose your resources then you can't fix the technical problems which is the the principle that I agree with. I think the the worst scenario is losing your good people when you already have a small

team for sure. I mean none of us have been in perfect environments where we don't have the second scenario. Let's just be honest with ourselves. >> That's true. >> But if you lose your people, you can't fix it. >> All right. You're saying so first is worse cuz it'll be unfixable. All right. We're going to throw this to the audience by applause. Remember applause. Don't raise your hand because we can't pick it. That's always happens. I say by applause and someone raises their hand. I'm like I can't pick up your hand from the microphone. Uh but if you put it together with your other hand and make a noise. I can pick that up. All right. So

by applause, how many people think that the first scenario that if you lose two people, you're screwed. That is the worst scenario. Oh, it's looking like a lot of people here. A lot of people here. A lot of All right. Second scenario where h it's kind of weak but you know if two people go you're still running at the same speed. How many people think that's a worst scenario? All right. A few brave souls. I appreciate that. >> Dension. >> All right. >> What is Dave's mom talking about? >> All right. We've played this game here before. It's a hit. Uh, my mom has become a hit with uh with this crowd. >> Your mom is awesome.

>> My mom is awesome. I' I've known her all my life. >> Uh, so here's how this game works. Uh, I know this is going to come a shock to you, but my elderly mother is not a cyber security expert, >> but when her son comes up with a stupid game idea, she plays along and she plays along with this game. So, uh, my mother is going to define some terms in cyber security. Everyone in this room knows what these terms are. My mother does not or she gets varying degrees of correct and wrong and you know you'll see in in general. So you have to kind of use reverse logic. If you had heard this

term before did not understand cyber security how would you best explain it? I will ask the panel first to answer. If they can't get it I will throw to you to the audience. All right. Are you ready to play and I can play I can repeat them if you don't get them because they're some of them are very quick. All right. Here is the first one. >> I think that's just carrying on in some fashion. >> All right. This is very wrong. I will just say and it's a this is a tough one. The first one's a pretty tough one. >> An audit report. >> That's a good one. You want to take a stab? I'll play it again.

>> I think that's just carrying on in some fashion. >> Connectionist protocols. >> No, no. I'm gonna This is a I'm going to be honest. This is really tough one. Anyone want to take a stab at this? Cross >> talk. That's a good guess and wrong. Anyone else? >> Carrying on. >> No, let me let me give you kind of a hint. The word itself kind of sounds like carrying on. It is one word >> and it kind of sounds like carrying on. >> AI. >> No, not AI. >> It's good answer that. >> I'm only going to give a couple I'm gonna go give a couple more guess and we're going to go to the next one. I'll

explain. What do you think? Anyone else? >> I'll play one more time. Fail. I think that's just carrying on in some fashion. >> No, that would be hashing. Sounds like a little carrying on. A little hashing. All right. >> Okay. >> All right. Over as well. >> All right. We're going to play another one. Here we go. >> Taking care of personnel issues. >> Okay. There's a hint of being correct here. >> There is definitely a hint of being correct. >> Taking care of personnel issues. Oh, wait. Let them answer first. Hold on. >> Uh, I don't know. Human resources. No, it's a it's a it's a cyber term. >> Uh, security awareness. >> No,

>> I give up. >> Take a stab. >> This is stump the seeso. This is not what >> taking care of personnel issues >> and and just a hint >> account termination. It's like that's where my head. >> Well, that is that is a good one. But again, it's a cyber term. It's a cyber term. >> Hold on. Identity. What? >> Identity. >> Hold on. It's your >> identity. >> Yes, it's identity access management. Very good. Very good. Oops. Good job, everybody. All right. The audience got it, not you. >> Very good for the audience. >> All right, we got another one. Here we go. We got two more. You need a plan to figure out when

someone is trying to invade your organization. >> Incident response plan. >> Yeah, it sounds like an incident response plan. >> Yeah, >> that again that's if she had the correct answer, but as I told you, these are variations of being wrong. >> So, >> cyber insurance. >> Okay, you're giving my mother way too much credit here. Um, let me play it again for you. You need a plan to figure out when someone is trying to invade your organization. >> Okay. >> Thor. >> So there's parts of that that are correct. There are parts of that. Correct. >> Of Thor. >> No, not disaster recovery plan. Like >> Oh, God. >> No. Hold on. Wait. I'm sorry. What would

somebody say here? >> Cyber. >> Hold on. Someone say intrusion detection. Good job. Well done. >> Good job. >> See, this is why we all need to work together. >> Yes. All right. Very good. Last one. >> So, we're 0 for three. >> You Yeah. Yeah. We're The audience is way smarter than you. >> The audience is doing much better than >> Yeah. Way better. >> Make available the best information possible. >> Make available. >> Okay. There is again a hint of something correct here. Make available. I'll let them answer first. I'll play it again. >> Make available the best information possible. >> Wow. >> I got nothing. >> Okay, this one. This one you should be

able to get. Come on, make them. You got this. >> No. I'm I'm I'm thinking this is why you need expertise in this industry to do your job. Well, all right. >> This is why our industry is so difficult. >> It's a hard job. >> It is. All right. Who did everyone Did you hear that? Here, I'll play it again for you. This thing is blocking up on me. Whoops. Here we go. Play one more time. >> Make available the best information possible. >> There you go. They got it. >> Good job. Once again, the audience is way smarter than you. >> We We suck at this game. >> Yeah. Yes. Yes, we do. >> Who's our sponsor this week?

>> You've got DLP, you've got CASBY, you've got alerts, but here's the question no one's asking. Can you tell if an action in your environment was performed by a human or an AI agent? Because today, agents don't just generate content. They take action. They move data, trigger workflows, and change systems without asking. And your existing tools, they'll tell you what happened after the fact. Now, that's the gap Quiller AI was built to close. Quiller AI's decision engine sits inside every interaction. Browser, endpoint, SAS, LLM, and agent workflows evaluating the content, context, and intent before an action completes. Not more alerts, better decisions made in real time. Now, if you're serious about securing your AI transformation, I know we're all going

there, and not just monitoring it, check out Quiller AI at Quiller.ai. Let me spell that out for you. Quilr.ai. Security can't live after the decision anymore. Quiller AI makes sure it doesn't have to. >> What's the best way to handle this?

How can a CISO or really any security professional get the most out of a security conference like besides SF and RSA? So, we've actually discussed this many times actually on the show, but this is an interesting take that we saw over on Reddit because, you know, whenever we talk about the go, oh, you just got to network. But one one Redditor on the CISO, one Redditor on the cyber security subreddit advised a more strategic approach. And I love this. Ask yourself, quote, is this a person who I can seek advice from if I was in a situation? Now, the benefit of networking is to have connections who can fill in gaps that you aren't soon

planning on filling. So, for example, I don't know much about AI IMCP, but this guy at this booth seems to know a lot. So, another commenter focuses on technical talks, community events, and hallway con. All great advice. So, I'll start with you, Mike. How do you set expectations for your team when they attend conferences? And how do you measure whether the conference attendant delivered value? I >> I think these two examples are really the right way to think about the conference itself. Some conferences it's around the talks, the content. Uh Bides is great for that. You're getting these communitydriven conversations that the attendees have the opportunity to learn from versus the vendor driven events. Like maybe your goal is to go speedrun

the vendors and understand what all is in that space. The important part is to have the plan going in like what is it that you want to accomplish? Why is it that you're going to invest your time in whatever you're going to and then lean in on that? Maybe it is hallway con, maybe it is the networking events. And so that's really what you should focus on. So that's what I tell my team, have the plan going in. But I also tell them, I want you to bring something back to the rest of the team. If we're paying for your airfare or your ticket or your hotels or something like that, >> you only do one of the three

>> or andor. I would assume you pay for the hotel and >> or does it have to be exclusive? >> Okay. It could be all of them. >> You would think andor is what you'd say. >> Go andor. >> I'm giving you crap. >> Thank you. Thank you. Never happened before. This is the first mark date and time. >> Yes. Uh, but it really does come down to there's an investment the company is making in that person. That's great. They're advancing, but we'd like to see them bring that back to the rest of the team. So, have a debrief. Have a document that you write up. Here's what I saw. Here's what I learned. Here are

the talks that I sat through that you should go watch the recording of. Bring that back to the rest of the team along with the plan going in of what you want to accomplish. And that's how you get the most out of these events. >> Sarah, what do you do with your team? >> I I mean I send my team to the technical conferences so they can sit and learn and then come back and do a brown bag. So like to your point, it's like purpose-built, right? Besides, Black Hat, AWS conference, like those are good technical conferences that our teams go to learn stuff and then come back and share what they learned. Um and and I I

we frankly send the people on the team that like to do conferences. There's people that don't like to do conferences, right? And so you can incentivize them with different kinds of training. And so you send the people that actually are going to get out there, go to the talks, meet people, learn things, and then come back and share it. So I think that's another important part of it too is like send the people that are going to get the most out of it. They didn't think that through all the way, did they? Your incident response tabletop, your IR tabletop, is lying to you. Not because the scenario is wrong, because the incentives are. Now, this is what Joshua

Copelan of Cresendo argues that in tabletop exercises, everyone talks fast, decisions are clean, ownership is clear, and nobody protects themselves. No real incidents ever unfold. In a real breach, the first control to fail isn't a firewall, it's authority. People don't hesitate because they lack training. They hesitate because escalation is political. Detection becomes quote let me validate one more thing or containment becomes let's wait for leadership and disclosure becomes legal is reviewing the language. If nobody in the tabletop worries about consequences, you didn't simulate incident response. The breach won't expose your controls. It will expose who is allowed to act without permission. It's like playing poker without real money. You play the game very differently when there are actual

stakes. So, I'm going to ask you, Sarah, how do we actually inject stakes into tabletops to reveal those fault lines? Because that's what you you want to see fault lines. If a tabletop goes without fault lines, you're like, we did something wrong. >> We do tabletops quarterly, and my feedback 100% of the time is, you didn't bring me in early enough. You should have brought me in here. You should have brought me in here. Um, and it's so hard to sit there because I haven't been called yet. So, I have to be quiet. So, I'm just sitting there on the edge of my seat like, right? Um, but but that that that's classic, right? That happens every

single time. But the reason you we do tabletops on a repeated basis is because we get better and better at it every single time, right? And two years ago, my team was bringing me in way too late. And and in some cases recently, they're bringing me in too early. And so, it's just a it's it's trial and error. It's education. It's it's keep repeating the tabletop exercises and you get better at it. It's that simple, I think. >> All right, Mike. >> Yeah. I think the key is there's no such thing as a perfect tabletop. Like simulating the actual stress is not something that you're going to be able to do, but you still learn. Like you

still get better. There's still advantages to doing that. And so you keep doing them recognizing that they're not perfect simulations and there's still something that you can learn and keep going forward. One of the things that we've done uh started doing this past year was we've actually brought in an outside firm to do a tabletop exercise for us. And that then does change the stakes because they have less familiarity with the environment. they aren't making assumptions in a tabletop exercise of well of course we would just do that. And so I think that really is one of the one of the opportunities is continue to uplevel your tabletop game and make that better recognizing that

it's never going to be perfect. >> Yeah. And I I love that suggestion and also just resist the urge to manage them. They don't learn if you tell them how to do it right every single time. So sit there quiet suck it up. Let them fail and fail forward. Right. Like we talked in the beginning about mistakes. Like there's mistakes are good. It's how people learn, right? >> And let me ask you that getting to the, you know, the line of, you know, playing poker without money is not playing poker. And I think if you bring in a third party, correct me if I'm wrong here because I don't do tabletops, but I would envision they know how to put on a

performance, you know what I mean? And so they'll even though it's not quote real, they'll create sort of some false sense of urgency that will make your team act a little bit more intense. Do you see that to be the case? >> Yes, I I do think they are more adept at creating scenarios that still can inject some concern, can inject some urgency, again in a way that your internal team will have a hard time simulating. And there's also the fact that you know you spent money on this and that does create some additional pressure of well as well to get value out of it because you could have spent that money on something else.

So it really that outside party does help raise the stakes. I have a question for you Sarah. You say you do it quarterly which is fantastic. I don't think I've heard anybody do it that often. um what are what are some of the big leaps that you see that like that they're making from a tabletop you know from like the earliest to the latest one >> so I have three different functional teams and we do the tabletops with everybody and so I think usually an incident response is primarily your security engineers that are running it but there's a lot of cross functional knowledge that is been really valuable for like the identity management team to

be a part of it right like they see how to figure out things quicker and they are contributing ing more and more useful insights as as we go through the tabletops. And so I think the cross functional knowledge is super helpful. And then another part of my team does all of our audit and compliance and like security programmatic stuff. And so as we run through tabletop scenarios, they'll realize, oh, I could tune this policy this way to make sure that this kind of scenario doesn't happen again. And so I think the consistency of it and the growth that we've had as a team has been crossunctional more so than it's been lateral with the SECOPS team

because they're good at this and they do it all the time. But I think if you broaden it out and have a lot more people in the room, um, you know, rising tides with all boats kind of scenario. >> Who's our sponsor this week? >> Join Zenity and contribute to the future of Agentic Security. On May 27th, 2026, the AI agentic the AI agent security summit hosted by Zenity Labs returns to San Francisco. Local speakers from leading platforms and industries will provide discussions, panels, and keynotes surrounding the most pertinent findings around agent behavior, access, and risk. In addition, security pioneers will unveil the most effective practices that security professionals can take to scale agents across the enterprise securely right now

today. Now, this event is vendor neutral, free to join and community focused. Join to network with brilliant minds across the industry and get ahead of the curve in securing AI agents. I know this is of a concern. So, you know what you want to do? You want to register. Go to zenedity.io. io/cisoseries. And let me spell that for you. Zenit ty.io/coseries. You better already know how to spell that. Go to that website and help shape the future of Agentic Security. >> It's time for the audience question speed round. All right, I have some questions in my hand from audience members and these are some really good questions. >> Great. >> And we actually have a good amount of

time to get through a a few of these questions. So, they have not seen these questions at all. It's fake news. But here is the first one, and someone overheard this on the All-In podcast just two days ago from Jensen Wang, uh, who's the CEO of Nvidia, and this is his quote. And essentially, I'm going to tell you what the question is right now. I want to know your thoughts on this quote. Okay? Either one of you can jump in. And this is what Jensen Wang says. If that $500,000 engineer, that's how much he's paying for engineers. If that 5,000 $500,000 engineer did not use at least $250,000 in AI code tokens, I'm going to be

deeply alarmed." End quote. Your thoughts on that, Mike? I mean this is coming from the CEO of the company who makes hardware that burns tokens. >> Yeah. So he wants everyone to be spending that kind of money financial interest on that. Yes. So that is pretty aggressive thinking and if everyone spent that kind of money they their stock would go up a lot more. Yes. >> Yeah. They they'd be what the first 10 trillion dollar company. No. I I think there like if you look at the the gist of it, it really is uh engineers really should be leaning into augmenting themselves, getting more done with AI. And I I think that really genuinely is true. That is

where we're at today. that you can be that much more effective by partnering with AI rather than assuming that it's just going to do all the work for you or just that somebody else is going to do the work. So the the gist of it I agree with. I I think at the same time the amount of money raises my eyebrows >> a little. And by the way I I forgot to give the appropriate credit. This was uh from RE Joshi of Elementary who gave this question. Anything to add to this Sarah? I I mean we're we're thinking about how to measure um effectiveness of of AI and how much our engineers are using AI and we're looking at what

metrics make sense um and it can't be like a conflict of interest in a kind of scenario. So you have to look at like the goals for AI that you have as a business and what those outputs are and then measure it that way. If it's engagement with the particular ID tool that you're using in engineering and you want to make sure people are using tokens, sure, but obviously it has to just be proportionate to what you want to do for your business. But I I mean it's important like there are a lot of engineers that don't want to adopt AI because they think they're going to displace themselves. So what are the kinds of AI use cases do you have going

on in your environment that you can incentivize engineers to do and then roll with that. >> By the way, I hate to break it to you. I just realized all these questions have to do with AI. So get ready for >> I know it's shocking. Shocking. All right. So this comes from Colin Dup Prey of Run Zero. Um so Agentic AI was introduced last year um in a significant way significant way. It's still the big story this year. What had do you believe has changed significantly with agentic AI from last year to this year? >> Adoption, speed, accuracy, reinforcement learning models are getting better. What I'd say is you've seen more uh the concept of agents controlling other

agents like it used to be this is an agent that it's standalone. It can do many things. I think what we've seen is smaller but more agents along with some sort of master control agent on top of that and that's what I think we'll continue to see more of. >> All right, this comes from Jordan Ka Marado. Again, another AI question and you know we there was a little bit of scare in the uh the market with uh with Claude's announcement with security. So Jordan's question is do you see yourself phasing vendors out because of AI? You're smiling Sarah. What >> I mean every time we renew a software license we ask ourselves can we replace

this with AI? And I already made that call in a vendor a couple weeks ago. So it's it's evolving fast. So, yes, for sure. >> Yeah. And we've already done it. Uh, and we will continue to do it. >> You did phase someone out already. >> Absolutely. And I I think that is that is the world that we're in today where a vendor needs to set themselves apart from a generic LLM because if a generic LLM can replace them, they're not differentiated. And if they're not differentiated, we can just do it ourselves. >> All right. Good point. And I know you've been very engineer focused. Absolutely. With your All right. So I have two questions that are very similar and I'm

going to go to you first Mike because this had to do with a a post you put out that got a lot of response. So >> uh from first from Peter let me get through both of these. Peter Zir is a Marado who said who wants to know from your AI post cuz you and you can sort of summarize a little bit more but you put out a post saying give your team some tokens and let them have at it to try to fix their own problems. Um, and also from Caleb Anafrey, a paradigm, what's the most realistic output for cloud code? So, I'm interested to know what have you seen that's pretty cool that

either your team or somebody else has done and and I I want to also talk to you Sarah on that. >> So, part of the inspiration for that post was where a TPM on my team came to me and said, "Can I can I have a cursor license?" This is someone who has never written a line of code in their life. And that really stopped to to have me think of we can really empower the whole team. Everybody can augment themselves regardless of their role, regardless of how technical it might be. You know, engineers, it's a more obvious use case, but where you've got folks who aren't used to writing code who are adopting, I'm going to go build a thing to make my

life easier, that's really where we are today. And you know the cloud code example is is a good one where you can go from idea to solution to your problem in a scalable way very quickly whereas in the past you have to you used to have to go talk to vendors or try and get time from an engineering team. So it's very empowering is what I'm seeing. the coolest thing you've seen. >> I I mean, I have my compliance team writing AI little bots and automating responses to audits and customers and displacing vendor solutions we have with that. And so, it's not just the engineers that are using AI now. I think it's just the the broader creativity

that I'm seeing across all of the different roles we have to figure out different things to automate and then figure out what else are we going to do with our time. That's the fun part about where we're at now. It's everybody's playing with it. It's not just the traditional engineers. That brings us to the very end of the show. Let's hear it FOR OUR TWO GUESTS, Mike Johnson, CEO of Rivian and Sarah Madden, CEO of Candera. Let's also hear from Bside San Francisco. And most importantly, let's hear it for our sponsors, Nudge Security. Remember, go to nudgecurity.com/shadowai, Quillriller AAI. Go to quillriller.ai and also zened.io/ciso io/eso series to go to their event. I we love coming to

this. We are thrilled that you came to see us do our show. Thank you very much to our guests. Thank you very much to our sponsors and thank you to Bside. We really really appreciate you contributing and listening to the CESO series podcast.

We are out. Thank you everybody. >> Theater number one will be there at 3:45 doing a game show. We're going to have a ton of fun. If you had fun here, you're going to have even more fun in theater number one. And if you would like stickers, come see me. >> Also, if you have additional questions, um Mr. Mr. Mike is going to be up on the third floor by the escalators and you're more than welcome to connect with him personally and or please be sure to add these individuals on your LinkedIn. Um we also want to thank our sponsors uh that are actually giving gifts to our wonderful three guests here by Aikido, ArcJet, Clover, Data Dog, Socket, and

Sublime Security. You were given two drink tickets by the way. Please use them whether it's alcoholic or non-alcoholic at the bar. Also, um The coffee bar will be closing at 4 pm today. >> All right, thank you so much. Big round of applause. Come on, give it up for them. That was awesome. >> Thank you everybody. By the way, if you want stickers, come see me.