Pushing Left - How We're All Doing It Wrong by Glenn Pegden & Stephan Steglich

Hey everyone. Um my name's Glenn. Uh I'll start with the obligatory introductions and a bit of an apology. Um there's meant to be two of us on stage. Um this is really Stephen's talk. It's a talk of Stephen's journey. Um unfortunately, there was a bit of a medical incident earlier. And Stephen's now um recovering in his room. Um but as this I I didn't want to be the sort of manager that came on stage and claimed credit for one of my team's work. I also didn't we want to be the manager that came on stage and presented somebody else's slide deck. But unfortunately, that's what you've got. So, if it feels a little disjointed, keep in mind that um

that yeah, that this is a lot of this is new to me as well. So, um Stephen and I work for a company called Flutter who you probably haven't heard of. We work for a division of Flutter known in the industry as Sky Bet who you may have heard of. Um between the two of us, we're responsible for application security, vulnerability management, that sort of thing. Um jokingly got described on a slide that accidentally went far wide and it should have been the day that our responsibility is to stop things. That was what we used. Getting popped. Um yeah, it's all preventive security. So, a little bit about the company. Um this isn't relevant this isn't this

isn't a pitch about what we do. It it we this helps understand some of the things later on. Um whilst we're a gambling company, we are a tech-first company, a very very tech-heavy company. Um exceptionally agile, we're designed to move at speed. We're designed to move quickly. Uh very big on innovation. We are very very keen to let people experiment, learn from experiments, move on quickly. The whole idea um we we we sorry, we we we have an approach to mistakes of always ask how not who. We encourage people to go and make mistakes, learn from it, do it better the next time. Um which from a security point of view is really interesting cuz trying to be

you know, trying to do security and keep ahead of your development teams when your development teams are given that much freedom and that much scope is really quite hard. So, a little bit of um history of where we came from just because it again this does become relevant as we go through the story. When Stephen and I first sort of got um came together probably four five years ago, um our security team in the company was very small. Um We were a gambling company, but it was compliance first. One of our former CTOs said something that haunts me to this day because um he's not wrong. He always said that compliance is more important than

security because we might survive a breach. We won't survive the ability to take credit cards. As a security person, that really upset me. But on the other hand, he's probably right. So, at this point, we were very much compliant first. That that that was the whole aim of security was to make sure we got through compliance stuff. Um we were very isolated. We were a little subdivision of the infra team. We were off in our corner of the office. We um we we wrote what we what were jokingly called sacred texts. We put out things on you must use this encryption. You must do build this thing this way. You must you know, management vulnerability management must be done

this way. We wrote these big tomes of data that we'd give out to the rest of the building and go, "There you go. Do that. We're security. Go get Go get that done." Um we'd just throw it out to them. And as you can imagine, that didn't particularly fly very well. It went about as well as you'd expected. Development teams they yeah, they're empowered to go as fast as they can. Security go, "Here's a bunch of things that'll slow you down." They just go, "Yeah, no." And just carry on with their um with with their own thing. Um however, our ace in our back pocket was you know, if we really needed them to do it, we'd go, "Ah, but compliance."

We'd find some [ __ ] reason to wrap it up as a compliance requirement. And we'd get stuff done. Um It led to some really odd and interesting things. We'd get new software engineers in the company that go, "Don't really seem to do very much security." It's like, "Seriously? And we spend our entire lives doing it." On the other hand, we were very ops-focused. So, the ops team felt that all they were ever doing was responding to things from security. It was very sort of incident-driven. It was very um operationally focused. Um So, we started you know, as the company grew and the security grew, we started to try and look to do things properly.

Um we got a new role in the company, a role that Stephen Speak a bit louder. A role that Stephen and I did ourselves for a while where we sat between security and the development and testing teams. Worked with them to help them understand security better. We um we got a new CTO who in turn brought in a CISO. And suddenly, we had board representation. So, the company as a whole started to get security a lot more. We started to a lot more work with risk management because ultimately, uh at the end of the day, all all security is really just risk management in different terms. Ultimately, all security is just managing risk one way or another.

Um and we started to work with the operations communities to actually help them understand what we do a lot more. So, it it's it's started to improve somewhat. But we became victims of our own success where all these different teams all sort of coming in so our team of these frontline people started to help. And they were doing all kinds of things. They were doing security doing security testing. They were doing risk management. They were doing you know, they were working in these communities doing third-party management. And they just couldn't keep up. And suddenly, it became a case of, "Hey, we're doing security better." And this just doesn't scale. We can't hire hundreds of people to

you know, to work with with this. So, we did what everybody what you know, what everybody in this situation does. Like, "Actually, how can we palm this work over there? Well, how how can we make this their problem?" Um the traditional way to do that is go, "Ooh, we're an agile company. DevSecOps, right. Dev teams, it's your problem. We'll run away from it. You're empowered to do security. We'll get the hell out of here. If you get it wrong, that's your fault." Going back to our um ask how not who culture when it comes to to problem retros, that didn't really work. We couldn't just throw it over the wall. So, um this is where Stephen who should now

take over. This is where the it's all going to go horribly wrong cuz this is his bit of the talk. Um this is where he started to um to lead our appsec team and set off down a very long journey to actually build a an appsec program. His background we he started at the company as a full-stack developer. We we hooked him out of his lovely cushy dev and dev environment and put him into security because we he thought like a security person. He um he he kind of got it. It's like, "Ooh, developer that gets it. Well, we'll have him." So, we kind of pushed him backwards and go, "Right. You you've been on that side that over there, that

side that doesn't like us, that side that doesn't want to play nice with us. You've been one of them. Go go go talk to them." So, he took an entirely new attitude on it. Said, "We're a tech company." And this was like a big thing in our head. Now, you know, ultimately, we are a company of engineers. Um you know, we're not a gambling company. We really are a tech company. We're going to push best practices. We're not just going to write these massive tomes of you should do this, you should do you know, TLS 1.2 all the things. We're we're not actually sitting and explaining how some of this stuff. We're going to actually

embed security Excuse me. Security as a support service. So, we're actually going to help you understand this. Um we're going to help you want to work with the teams. We're going to take our texts that we used to throw out about what you should do. And we're going to um work with you you work with the actual developers to mold what should be in them. And more importantly, we're going to take one of the big lessons from Agile. We're not just going to go, "Oh, appsec is a six-month project. We're going to go. We're going to buy some tools. We're going to stick them all in your build pipelines. We're going to produce some reports and go, 'Either

That's great. You can really see how much crap it is.'" No. We're going to actually build it as a continuous improvement thing that's going indefinitely. And again, we we grew to a department in our own right with we both grew. This I was just joking with Jenny outside. This is where it all goes horribly wrong cuz this really is all Stephen. Apparently, this is some famous appsec model thing. It's got a name and everything that I didn't bother looking up and it isn't on my notes here. Um But ultimately, what we worked out is that we we had the strategy piece. We we got our new LT representation. We had the team doing the design reviews and

the implementation reviews and all that. Security testing, that was where we should start. That's actually what we needed to do. We'd already been in charge of pen testing for a while. Appsec's just another another level of that. So, we kind of worked out it's like, "Okay, if we're going to shift like the all our security testing, we're going to give these guys this, what do we do? We we do what me and Stephen always do at the start of these projects. We started with a whiteboard." Um you know, that's the obvious meme, but we actually did genuinely have a whiteboard session. Um and we were very much ahead of our time. We we we we

found this. We just chucked it in because we we we came across it while we were trying to put the talk together. You'll notice whoops, come back. Our causes there. That's kind of just a condensed list of the OWASP top 10. And bearing in mind this is three years ago, software supply chain. Long before SolarWinds. We were ahead of the curve on that one. We saw that one coming. We kind of worked out we went through all the different types of software testing from simple vulnerability scanning to pen tests to reconnaissance tests to um build reviews and that sort of thing. And worked out what what type of problems they actually really addressed. Um with a view to seeing how much of

this we could shift left. Um Trying to remember what this was all about. So, we we ended up with our two teams. We got our tech team over there that weren't very security-focused. And our infosec team there that with the exception of Stephen weren't very development-focused. We wanted to do appsec. They wanted to DevSecOps. So, we had to come up with this center ground of something that fit both of us. Um one of the first things we did was actually go and learn how they worked and realized that in our heads we just have to work with the developers. Developers write code. Makes sense, doesn't it? We We teach developers how not to write crap code. Maybe work with

the testers, tell them how to test for crap code. But we soon realized actually there's a lot more in this. The product team that actually work out what goes into the products, the op team op teams that support it afterwards, the delivery teams that actually work out how I get stuff to live. Support teams look after We have to actually build a program that fits all of these people. That actually works for all of them. It was a bit of an eye-opener for us cuz until this time, you know, our attitude really was just write better code. Um so Steph did his bit of um we spent a lot of time embedded with them. We tried

to work out where the common grounds was. You know, what we cared about, what they cared about. Um few really interesting findings. Um their engineering managers are like Our main aim here is to write quality software. We don't give a stuff about security. Security is a speed bump. Security is something that gets in the way of delivery. What we actually care about is code quality. We care about writing quality code. So we care about quality and if it's insecure, it's not it's it's poor quality. So um we kind of had canvas on on what they actually cared about and it's like oh we need to build things that support in business needs. We you know we need to

ensure the highest level of quality. We need to ensure business continuity. And an industry like ours where any second of downtime is huge amounts of lost money. Continuity is everything. So um we looked at that and thought, you know what? That isn't too different to what we actually care about. Quality, security, that's the same thing. It's only we realized we we we can repackage this. So on our journey on this um I can't actually read this slide. Um Yeah, so sorry. We We We took that um that idea and said, "Okay. We're not going to teach you how to do security. We're going to help you write write better quality code." Suddenly developers on board is like, "Hang on,

you want to help us? You want to buy tooling? You want to buy stuff to let us write better quality code to improve the quality of our stuff?" All up for that. So they ate, jumped on that. Then started working with the testers and the testers were saying things like "Yeah, you know, testing is great, but we we do feel that we we get ignored when we bring up issues." And security is saying, "You know what? So do we sometimes. You know, we feel that we're not very connected." Uh so we we kind of got talking and it's like, "You know, why why is nobody really working with each other? You know, what what why do you guys not care

about security? Why can't we learn security with you?" And it turns out the answer is it's scary. And there have been some really really good B-Sides talks on this recently and it's something I feel really passionate about. What we as an industry are really bad or really effective, well it's a terrible thing we do, at [ __ ] gatekeeping, at making our industry sound really hard, really complicated. Pen testing is this amazing mythical skill that only a small percentage of people can do. Infosec is really hard that you know you need if if you haven't gone through all the right levels to get there, you know, you you can't be deemed to be part of our

industry. It's absolute bull. What we do isn't really that complicated and we've proved this because we've gone down this road. We've tried to as much tried to remove as much as we can all the scary elements and gone to the testers, "You want to learn how to security test? You want to learn how to do basic pen testing? Great. We'll teach you that." And it's amazing how quickly that all came together. So um we started out with a little bit of a mission statement. Security testing is gaining the confidence required that the impact and likelihood of an unwelcome outcome um happening is within our risk appetite. And you know, the testers said same that's literally that that's that's

what we're doing in security that's what we're doing in software testing. It's literally giving giving that level of compliance. It's not black or white. It's not getting everything perfect. It's about fitting with that risk appetite. So we're on we're on to a a good thing here. And they realized it's like, "Ooh, actually we're already doing bits of security testing." It's like, "Yeah, you are." And "Can we do more?" It's like, "You absolutely can." So we went down this whole path of changing language. We stopped talking about pen testing. Pen testing when talking to devs and uh product teams and develop and testing like that became a taboo word. We don't talk about pen testing unless we're

actually engaging a third party on site. I can go, "He's a pen tester." I can't talk about, "Oh, we're going to do that project and do a pen test at the end of it." Nah. It's all it's all now security testing or software testing. Similarly, we stopped talking about secure coding. Secure coding was it was a compliance requirement. It was some [ __ ] learning module that developers had to do that um you know, taught me about buffer overflows in when they're using languages where that's you know, with managed memory. It was just nonsense. So I said, "Okay, no more secure coding. We talk about code quality." And suddenly we got people on board. Suddenly developers and testers start to

get it. It's like, "Ooh, this is really useful and really interesting and really good and not all that hard." And ultimately you're improving our quality. We love improving our quality. It's like oddly enough when it gives us more secure software, so do we. Um we also stopped took a big thing of being a naysayer. Another thing that we do really badly in this industry from is we put ourselves front and center of security policy. It's like, "You did a thing, that's bad. Stop it there." You know, it's like, "Your code isn't good enough." Get it there. Our from the business can scan it said that you're not patching that. Go patch it. We're we're dreadful

at again really efficient in a bad way at stopping at blocking at slowing people down. So we said, "Right, we're not doing that. We're going to think more the way that the rest of our company works. We're going to be a support engine. We're going to be an enabler." So it's like, "Okay, what do you guys want to guys girls want to write to what you want to write more secure um you know, higher quality code?" You know, "We want tooling that you've got loads of great tooling." So I said, "Great. We'll find some tooling." So even that we took a very different approach. In the past we've gone off security and looked at a whole bunch of tools. We'd

even talked about budget for some of the big name um SAST and DAST tools out there, the you know, um source control things like that. We've got our own minds. We've got our own favorites. We went and sat down with them and went, "Okay. We'll work together on this. Security will do all the bits you don't like. We'll First of all, we'll pay for it, which is the bit they really don't like. But we'll do all the dealing with vendors. We'll do all the you know, the paperwork, the recording of decisions. We'll help you with the research. We'll give you SME support. When you're saying, "Oh Glenn, you know, these two things both claim to

do this bit of security." And we'll go, "Well, that does it this way and we like that and that does it that way." We'll we'll [ __ ] Trying to keep my hands in my pockets. Um we'll provide that level of SME support, but ultimately this is your tool. You're going to choose it. You're You're going to be the leaders of what to get for that tooling. And then as a team we we kind of all worked mixed with those and the developers and testers and product managers. Between us we set the requirement. We test those scenarios. We discuss the results and we went through and we we did a whole shoot out of a whole

different um bunch of um tools trying to meet you know some base things you've got on the screen there. But tools there there were much and much as we felt they were good ones, we felt they were bad ones. The results were really really interesting. We'd got a firm favorite. It was one of the more expensive ones. The dev team comes back and goes, "You great? Yeah. Feature-wise, that that is by far the best thing. But we want this one." It's like "Well, that's much cheaper. That we they make doesn't do half the things. Why?" One of their requirements that they'd put to try and judge a good product was could a first-year junior who had never

heard of concepts like SCA figure out the tool and give a tech demo to tech leadership the next day. They wanted something they could pick up and play with. They hadn't got the time or the resource to learn a complicated tool and they wanted something that they could just plug in and go, "Yeah this that's it. Go and look at that bad code smell over there." Or that you know, this stuff over there. They ended up coming up with two of the sort of better value formula that I wouldn't say cheap cuz they're both amazing products. But they ended up going for Snyk and SonarQube. Not because they were what security wanted. They They were I mean don't get me

wrong, I've become a huge fan of Snyk of sorry, Sneak. Not Snyk. Since then um SonarQube you know, it it does what it does. It's already quite common amongst developers. But it it was really interesting the fact that they didn't choose the same tools as security. And we looked at it and went, "If that's what you want, great. You know, ultimately we still get out of it. This is your thing. We're teaching you to write more secure code. If this gives us that, we're not going to force a tool on you don't want." So we rolled it out. We made the mistake of letting security have the first cut at that. Security, because we're not a

development tribe, because we're not a product tribe like that, we kind of did it in a very waterfall way. We just pushed it out to them and they went, "Yeah, we might get chance to look at this at some point in the future." And it kind of went nowhere fast. Then again Steph and his infant wisdom went, "Hang on, we're doing this all wrong. We're do These are tribes designed to actually um build things like this. Let's go talk to them." And they they took it on and went, "Oh, actually we'll just do it as you know, part of our normal agile thing. We'll We'll break it down. We'll do it in little chunks. We'll build it into our

sprints." They took the entire lead. We bought this tool. They pretty much two tools. They split it all up and then went, "There you go. You can have it now. We've done that for And because they could do that all piecemeal themselves, they were so much more involved. It became their thing. It wasn't a security thing anymore. It was theirs. We just happened to be the ones that were paying for it and running it afterwards. Um And it just works. So That was great. We've now got some tooling. We've now got developers actually caring about um um writing more secure software. They don't They They think they're writing um higher quality software. We think it's

more secure. But every everybody's winning. We then start working with the testers and they they they're really engaged with it. They're loving this cuz you know, the first things that testers look at are the code quality, the results. And it's like we need to understand more. You know, we spend our life writing tests. Can't Can't we write tests for this security stuff? It's like hell yeah, of course you can. You know, it's So it's like well, you teach us all your magical ways. Teach us all you know, teach us all the the the complex ways of writing tests. It's like it ain't that complex. You can test for most of this. You know, especially with source access, you can

test for a lot of this stuff really easily. Um Yeah, obviously there was the initial thing that we tried it. You will learn security and they went nah, bro, too hard. Um when it turned into can you help us with this? And we're like sure. And it became this constant feedback cycle of they wanted to learn so we helped. They learned a bit more. We helped. And bit by bit it got untangled. Um I would do it for time. I was like I think I'm doing it okay. Um We started to feed back a lot of our data. I do personally do quite a lot of more offensive stuff. So things like um the scripts I was writing for perimeter

testing in nuclear, I was feeding that back in my testing channel going like I'm testing this at the perimeter. You know that you could test this at build time so it doesn't become an issue further up. Similar with the bug bounty reports. The bug bounty program started like most places as a big triage thing that um reports would come in. We go back to the dev or infra team looking at fixing and going hey guys, you screwed up. Go fix that. And that would be the end of it. Whereas now we get those bug bounty reports and go oh, really interesting. Hey testers, somehow this made it past you. You might want to learn from this cuz

you might be able to test for this further up the pipeline. So we were actually using those bug bounty reports not just as a triage thing to fix the problems. We were using them with the testers so that the the the testers were testing for them well before you know, before they'd even made it out of dev environments. Um we also did a lot of fun stuff. Um we the the our I enjoy putting together internal training. We've done a few CTFs. I love doing CTFs. I wish I had more time for it. A few people out community even came along for the ride and now play hack the box with them and stuff like that. So we

we've got a whole bunch of people now that are actually that have gone from the security's too hard and it's security's job to spending their evenings playing CTFs. It's like haha, yes. Suckers. So um My god. It really is weird presenting somebody else's slides. So oh yeah, is pen testing dead? So with this you could think hey Glenn, didn't you just save a million pounds on pen testing here cuz you you're now doing all your you've got your own internal staff. No, our pen test spend didn't actually change a penny. However, what we pen tested changed massively. In the early days before we did all this, some little team little three-person team would write some

microservice or some API or some little tiny cog in thing. And we'd throw it over to a pen testing team and give them sort of a two or three-day engagement. Little tiny thing that every pen tester hates cuz it you know, by the time there's really nothing there to get their teeth into. And we'd do that over and over and over again. We don't do any of that anymore or do very little of that anymore because it's not really that important. We know that's getting tested part of the pipeline. What we can now do is take that same spend and go right, this collection of 50 things over here that make up this mega service or say

you know, our our Sky Vegas service. Go test all that. Big open scope. Have some free credit on our websites. Go and lose. Try and whatever you can. Here's all the docs. Here's all the source. So we can now give like huge month-long really in-depth engagements to pen testers that pen testers seem to actually really love when you give them a wide scope and huge amounts of time to play at it. So it didn't change how much pen testing we did, but it did change how we spent our time. And again, we got so much more value out of that. Putting it all together. Um All right, yeah. Okay. Um The final bit we did there was was more

with the project and um delivery teams. We sat down with them and tried to explain where we'd gone. And we said sort of said right, you can help here cuz you you understand products. You understand how we get this stuff live. To help your testers help your devs, what you need to be asking at the beginning of a project is what could possibly go wrong? And you know, we we do that. That's part of what product managers do. That's part of what we do. And then you want to think about you know, what ways could possibly go wrong? So yeah, we do that. It's like and then you what controls can we put over that? And it's like if you can't

stop that thing going wrong, if it's an inevitable consequence, what controls can we put over it? Be them sort of um uh I've got it. Detective or or preventive or or whatever on that. So I'm okay, we get that. Um some people will be more will be probably already half a step half a step ahead of me. Whilst we didn't teach them stride or anything formal like that, that's just threat modeling. So essentially we we've managed to get security testing moved down. We've got secure development moved down. We've got product managers doing threat modeling on the way down. Um and it suddenly it's like we've all you know, all this stuff is moving moving left and very rapidly.

Um and suddenly we we've got entire teams engaged. Back when I used to do that that BISO role, that front-facing role, I may have you know, a tribe of say 200 people. I may have one security consultant tacked in there. Suddenly now a squad of like 10 or 15 people, all 15 people are coming into my meeting because it's like well, that bit's relevant to me. That bit's relevant. Suddenly they get that security is all their job, not just that one guy that isn't scared of talking about security. Um Compliance is something I've not really talked about. And again, this all came part of it cuz um there's another talk that a colleague of mine Lee Hall did recently and there was

a great blog piece on it on how we use the um how how as a company we we put out lots of repeatable patterns. So we say to dev teams, you can build stuff anyway you want. You We don't care how you build it. We don't care what you do with it. These are the requirements that we we've got to come out of it. However, we've already been through all this this this thing over here. This team's already done. They they've they've done this. You want to follow their pattern, great. Cuz they've been through all these questions. They've built this thing that way. That means you get all the compliance stuff for free because we know that it's

handled the auth properly. We know that it's handled the um logging and monitoring properly. So all these compliance checklists, you can ignore them because you're inheriting their solution. Um and we kind of baked that into this in so much that it's if it's been gone through a team that's properly properly adopted this, it's like that's like fast tracks it to become a a pattern very much. Um and we've talked about the fact that yeah, the testers were building their own testing scenarios. But they to them they were just functional testing unit tests. To us they were security tests. They were actually designing them off the back of this threat modeling. The sort of stuff that

we could only have dreamed that we had time to do as security people. They they'd already nailed it before we'd even got there. Um you know, they they started doing simulation rigs. They started you know, they they they've built these really complicated load testing things cuz I guess they said to us up time is is everything. Our um traffic is very very spiky. You know, 3:00 on a Saturday is huge. 3:00 in the morning is is virtually nothing. So they they built all these great tools for pushing huge amounts of data. And it's like oh, can we start pushing this data through that through that you know, can we can we start pushing potential SQL injection

data through that? Can we start pushing XSS through that? For the boots, you know, you built all this testing rig. You can it's your service. You're whatever you want. But it was more they were asking the questions. They were using all this tech they'd already built for up time testing and starting to do security testing it cuz it it suddenly dawned on them, why don't we do this? Um and the various aspects of testing came together. Um And again, as I mentioned last slide, pen pen testing contributed but in a different way because we were testing bigger and more. And again, we were sharing the pen test results not just with the dev team and going hey, you

screwed up over here. Fix that. Or the infra team and it's much like oh, you know, you forgot to configure that server, right? We were sharing it with the full team and going these are our findings. These are the things that you had the chance to learn you know, to to spot on the way up and didn't. So do you want to actually um Yeah, do you want do you want to learn from this? And then we can build some of these findings into your stuff. And they did. And it was great. Um last minute addition to the slide. Um when I first pitched this talk on sort of Twitter and LinkedIn and stuff, it

was designed to be entirely tool agnostic. You're not meant to not really mention the tools at all. But people kept pestering me and say oh, you know, what tools are you using? What have you got? How does it stand up to this? How does it stand up to that? Well, there you go. If you want to see, I'll make the deck available later. That is the the tool set that we're using. But realistically, this talk is not about those tools. I've only put that in there cuz I promised people I would. The tools aren't important. You could swap any of those tools out for things that work better for your team. And there will be things that work better for your

teams. There will be things that are actually better for you than these tools in there. Use them. They It's all about finding the right tools for you. That just so happened for the shape of company that we are. That's what we went with. Um 30 minutes we're doing fine on time as well. Full circle and that um some unexpected wins. Treating it with the same We We treat it the same way they do. It was never meant to be a delivery project this year. It was a journey. It was a can we take this from doing this over here to doing it better by the time we get to there. The analogy I used in a management meeting the other

day was it's like we we've been that parent. We've been spending ages We've We've got our kid on the bike. They've been pedaling. We've been holding onto the seat and we're now just at that point where we've let them go. It's not that we're never going to go anywhere near the bike and consider bike a tick list that our kid can now do, but they're there. They're pedaling on their own. They're going at their their They They've got it um as their thing now. And it has had some really unexpected things. Things like the um like um Sonar Qube and Snake that we use in the pipeline now. We don't ever meant these as passive

tools. We We've meant these as um reporting tools so that the um developers and testers could get an idea of sort of what we call bad code smells of of areas to look in. We know that the way these tools work, they're not It's It's not like um front-facing vulnerability management. It's not a case of oh, you've got a severity or high severity thing, go fix it because it might be oh, you've got a high severity thing that's only an issue if you're allowing arbitrary user data in there and we never do that in this company. It's a lot more situational. So, we never ever intended to gatekeep based on that data. However, the tribe has started to do

that now. They're coming to me and going, "You know, in Sonar Qube, if something comes out with like severity or high severity stuff on there, can you actually block it so somebody senior has to check it over before they push it?" It's like we can. We never asked for that. We We We as security have never said that we should do that. However, if you think for that project back then, hopefully suddenly they've got to the point where they want to actually block themselves because they've got their their so much faith in this. Um the whole collaborative thing has been has been great cuz we now work so much closer together in so much that they'll

come to us with a problem and go, "We've got this problem. We think we've got four-fifths of this sorted. What are your thoughts on the rest of it?" And we go, "All right. Your Yeah, our thoughts are on that bit over there." And then it will it's a much more collaborative thing now where we're solving each other's problems. Um the fact that we adopted their ways of working, part of that we started thinking like our product delivery team means that um things now just work. It's like they um things happen by default because we fit into their way of working rather than being the security policeman and going, "No, stop doing it our way." cuz we know

what we're doing. We're smart guys in the room. Um And it it it's having knock-on effects all over the business. Um we're seeing sort of teams that weren't even initially involved with this going, "Hey, we we saw what you did over there. We're going to introduce this in this thing over here. Are you down with that?" It's like you're saying that this thing over here you you want you're asking our permission to do better security on it. Well, fill your boots. Um so, it it kind of it brought in right across the business. Um So, on 35 minutes, which is a little bit faster than I'd intended, the closing line for this the the actual

real thing for this is we had Stefan on board. What Stefan did because of his previous life in development, listened to his colleagues and rather than being like the security person sat in their little dark security office going, "We are the smart people. We will tell you how all to do security." He actually went out and went, "No, you're developers. What can security do to help you? How can we make you better at your job?" It was them that taught us how to do this. We just fed back in some requirements and how it worked and how we do it and a little bit in there. That was a little bit faster than I expect I I expected. I probably missed

loads and loads and loads of good points that Stefan wanted to make in his stuff. Um I will encourage him to write a blog or something like this on the back of this cuz say this was his talk. I I'm I am that horrible manager that coasted along on his staff's great work. Um and yeah, it's just really really unfortunately he can't be with us to um today to present this himself. Um I'm around on Twitter and most social platforms. Feel free to nudge me and ping me with any questions. But now for the room, any questions? Yes. And I will try and remember to repeat Oh, you've got a mic. I don't need to

repeat the questions. Brilliant. Okay. Um you briefly mentioned threat modeling. Are you Are you using that for like design and architecture review or for test planning or Yes, we For bigger, wider projects, we do have a a specific security architecture team that go and engage outside of that loop. So, we do actually for if if somebody was building a big thing, then yeah, security architecture get involved. They do do proper threat modeling. Same with the that B cell team. They still do a level of that. It was more that we'd accidentally taught the um product managers in the team how to do that. We hadn't even realized we were doing it. It wasn't a you know, we didn't have a secret list

of things we needed to teach them. It was like, "All right, taught them taught them threat modeling." It was a pure accident when they they showed us how they were looking at these problems and that was like that's literally threat modeling. Okay, it's not STRIDE, but it it it's all those same steps. But yes, we we do use that for bigger stuff. I just just outside mine and Stefan's little world. Yeah, I think that's back to your gatekeeping [ __ ] thing. It's like I'm read about threat modeling when I was a developer, but thought it was all It's just like some process, not a skill. Yeah. Absolutely that. And don't get me wrong. I I am a big fan of threat modeling. I'm

even a big fan of STRIDE itself. I It's useful, but if you just go and throw it at a developer and go, "Right, we're going to go through this." It looks like 16 layers of [ __ ] It really does. You have to say Bringing it in through stealth like that or just What I What I was in that B cell or what I used to do for that was I'd sit in a room with them and ask them lots of questions. You know, what's the worst thing you can imagine happening? What about this control over here? What keeps you up at night? And then I'd go away and sort of on my own without the

developer in the room try and retrofit that into a threat model framework. Anyone else? Oh. Sorry. Like the guy with the mic decide who's speaking next. I think it's the lady in front the row in front. I mean, just pick random people in the audience. That'd be quite fun. Just yeah. Um really good talk. Enjoyed that. Thank you. Um I was just wondering from like a security testing um code QA tooling perspective, did you find that that was more effective in teaching engineers kind of secure development practices than typical secure dev training and OWASP top 10? Was it more efficient or would you recommend doing both for a good app sec program? I I think it depend It literally was down

at the individual what worked for them. We We We've tried different ways. For us, our sort of CBT-based computer training really isn't very good. We do it It wasn't very good. We've talked to lots of organizations. One of which is actually out there today about using that tooling to make it good. And we've mixed and matched it. You know, I I I've done one-to-one sessions. Like we I say we've We've done the whole gamified thing. Some of it works for some people, some of it doesn't. Some people just immediately when they said, "Oh, we can now do security testing." went out and got the um What's the web app? The big tome, the web app hacker's handbook. And literally

devoured that cuz it's like, "I've got a new thing I can learn." It really changed from person to person. So, I I don't think there is any one right way. And I think in an organization our size, that that you you you literally have to go, "There you go. There's a There's a destination we need to get to. Here's a few different ways we're going to try. Whatever Whatever works." And yeah, basically anyone improving themselves I'll take. As he's put his hand up, you're going to have You're going to have to go back to the person you nearly bullied into um speaking when she didn't have a question. Um so, for your um automated testing pipeline,

um do all the devs get involved with that? Like if one of them wants to try out a new type of testing, would you let them set it up themselves or are there two different teams to do that? It It varies, but Our entire company ethos is based on sort of autonomy all the way down. So, at a as a Each team can basically do their own thing. That goes right through the company. So, at at a philosophical level, absolutely our our company will go and go away and experiment. Go Go um go play. Go learn yourself. One of the reasons that we have like 160 separate VPCs in AWS is that we encourage you to

just go play. Individual teams may have their own processes, their own things. I would hope that our C tap team, the the people that deal with the money and the finance and the really important um not the not the finance, so the actual transactional taking money off people's I wouldn't I would really hope they weren't just going and I want to try something different with life credit in production. I'm really hoping that they that that there will be a little bit more gateway with that. But no, as a company, we're very big on try it. If it doesn't work, go try something else. Thank you. There was other hands there. I don't know why I'm pointing. You can't

see me. That's a good talk, man. Thank you for that. Cheers. Uh so, you talked about a little bit how you brought your teams is with the process, right? Yeah. Uh but everyone has an opinion, yeah? So, how do you align your C-level, your CTO, your head of software to understand the security you're trying to drive quality, not only security requirements, yeah? The reason I'm giggling at that is that in an early preparation for this, Stefan tried to explain this cuz this is literally in his last 2 years of life. Tried to explain it to our CISO a few weeks ago and I kind of went to the CISO after this was feedback. Yeah, he blew my mind. He's a monstrously

clever guy, but I don't get any of it. It's like, great. Wait until I've done the B-sides talk cuz hopefully that'll like we can watch the video version. Hopefully that makes a bit more sense. So, Shawn, if you're watching, hopefully that now explains what Stefan's been doing for the last 2 years. Um but no, it's it's very hard and that's partly why me and Stefan work so well together in so much that we've both got an engineering engineering background. We're both um you know, sort of techies at heart. However, he's very good at strategy and doing and and and the big picture. I'm very good at saying the right words to the right people to land

ideas. So, uh in in answer to how do we sort of run that message from everyone down to from engineers through to CISO and the CTO and stuff like that is me. I don't mean that as a big mouth. It's literally that as part of my job is to try and interpret his galaxy brain stuff into dilutable parts for people. But it it it is hard. I mean, literally we you have to sort of explain it in a different way at every level. I couldn't There there's no magic where we explain to you like this and everyone got it. It's like, no, I've I've done equivalents of this talk in about 10 different ways focused on

different things. And ultimately, it still boils down to about three sentences in the end in so much that we let devs and testers do their own thing to get what we wanted cuz we're people hackers at the end of the day. Anybody else? Sure. Can you Oh, sorry. Can you comment on the onboarding experience for So, let's say an imaginary scenario. I'm a new development team. I want to write some microservices on your AWS accounts. The thing I noticed is that you have an infra team, a security team, different teams on a different bunch of tooling. How is the onboarding experience? Are the teams just like ordering through some self-service portal everything or is it like a lot of governance checks?

How do I learn to use Snyk correctly? How does like my experience as a dev on day one look like? Again, that that varies massively from team to team and if we were trying to if if we'd have just got to this point today and it's like, there is, go. We'd probably do it very differently. This has actually been a building experience where we can go where where we've gone out to key influences in the company and gone, we're doing this thing. Can you help this land you know, land this with your product managers, with your test leads? We we actually use quite a network of people to try and work out how to um

sort of onboard them through that thing. Um the So, if we were doing it today, actually, it probably would be very different. We we would go for something much more I won't say prescriptive, but um where where it could be a click and go on board. As a company, we're very big on click and board, you know, click and go, here's access to your tool thing. Um for the for the two particular if if we're talking specifically about the the technical tools we've used, we have automated that quite a lot. The um literally, we we built them both into our internal um I am an identity system, so a developer can just go, "Hey, I want you know, I'm working on

this repo over here. I want to use Snyk on it." And you sort of a dozen different people get a request and we'll go, "Yeah, why would we ever say no to that request?" Click the button and away they go. So, there there is a little bit of gating around that process, but it's ultimately, we it's more about making sure that people have access to their data within the tooling rather than whether they can use the tool. You know, basically, it it stops some sort of junior in marketing going, "Oh, I want access to all your credit card payment um stuff the um vulnerability of So, in the case of Snyk, all the dependency problems or all the code

reviews and stuff like that. It's like, well, there's a bit of gating in there. Oh, no. Um in terms of that though, it's mostly it's mostly self-serve. Most of the developers can just go and click in that they want access and it just magically Again, Stefan's you know, galaxy brain stuff. I have absolutely no idea how any of that actually works. I just know that they go click a thing in one identity and suddenly they have access and it goes away. And literally, the only real support we get off the back of that is I get a lot of managers moaning at me. It's like, you know, um you know, we we requested access on

Thursday. It's Now Monday and we haven't got it. It's like, you're a development team and you're complaining that I you can't have access to security tools after 3 days? It's like, normally, you know, a year ago, you wouldn't come and talk to us. Now you're complaining that we can't give you tools to do this quick. Yeah. Sorry, I I did that answer your question? That was a lot of skirting around the edges without actually answering any of it. Okay, I think that's all the time we've got. I've got one more question. I forgot. Oh, no, just Tom. He he he he he's got to speak cuz this this is going to be a hard one.

Um in terms of aligning security into code quality, Yeah. did you find that existing metrics, the metrics that the dev teams were already using to measure code quality, did they need modifying to include security? And and a secondary question, did you see a measured improvement in code quality? Yes and yes, but only because they chose the tooling. If we'd have gone with the tooling we would have got, they wouldn't have got that same comparative um quality score. You know, they'd have got like, new tool, new score and it would have it would have been much harder to measure. The fact that they'd already built existing tooling to spot what they considered poor coding practices, and now they've just used that to

include security as what can be covered. It made it very, very easy. But again, from security point of view, we didn't want to actually impact on that. We collect that data, but we don't publish it. We With vulnerability scores and stuff like that, we actually publish that data across the company people see. With the code quality stuff, we absolutely don't because we feel that that is the tribe's own data. It's like, they're you're working with this to improve your stuff. We don't care. It's like, if you're getting happier and we're getting less problems further up the pipeline, we don't really care how much of a mess you're making down here as long as by the time it gets to live,

it's all good. So, um can I see a marked improvement? I absolutely don't know at the code quality stage, but I can see a marked improvement at the vulnerability management stage. So, we know it's working. Well, we know something between these two is working and I'm guessing it's all this stuff cuz it makes sense. And I do apologize for running over 47 No problem. Well, I would like to say that's a great talk, Glenn. If you'd like to put your hands together for Glenn, that was great.