The Halcyon project: applying real world DevSevOps to a vulnerable app - James Clapperton

[applause] Thank you. Thank you everyone for coming. Um, so hopefully you can see it. Yep. Perfect. Um, I should caveat this by saying that someone this morning told me that the little image I've created looks a bit like a new Angry Birds thing and now I can't unsee it. Um, there we are. Welcome to the Hian Project. I did need to include a slide on exactly why I called the Houseion project. So the title is inspired by the develop the DevOps novel the Phoenix Project. I don't know if anyone's read it. It is a good read. Um, but I had some concerns with it. I feel like they raised the security team within the book as being sort of

sensationalist and being a blocker to all their projects. Um, and I thought this cannot stand how how can they do this to us. So I came up with this idea of sort of giving a talk about how um we can incorporate security within DevOps practices obviously within dev sec ops um and and what exactly that would look like. So like other fine Belfast establishments I have opened up a new shop but it is riddled with vermin as you can see from my totally AI generated image here. Um how am I going to get rid of these bugs? Well, hopefully I will engage some dev sec ops tactics in order to exterminate them. So, I thought it

best off to start with what exactly are the principles of dev seccops um what does it encompass um before we actually introduce some tools and things in order to get rid of those bugs. So, dev sec ops and we love a buzz word in security but is mostly about shifting left. What does that mean? So shifting left is taking security and rather than engaging it at the end of a product where it's been through the development, the testing, everything, it comes out at the end and then you bolt on some sort of web app firewall and call it job done. We want to engage security in every aspect of that and shift it left into

your green field space. You're just starting out a project. So let's try and inject some security before we even build anything. Now the second pillar I would say of dev sec psychops is automating that as well. Uh we want to make sure that we're getting timely results. We're making sure that we are getting these faster and more efficiently and enabling the dev teams to do their job in you know introducing new functionality um into the product and also giving operations their space as well. Third thing is a culture of security. I'm sure most of you here will have been through your compliance training at work where they say every member of our team is responsible for security and it is it is

true of course. Um so we want to foster a culture of security within the team within the company the wider space. Um I'm I'm all big for having a security champion within the development sector. to someone within the development team who's maybe a bit more familiar with security and can bolster those resources up and and just sort of like be the security hype man as such. So next I'm introducing um the secure software development life cycle. Um so you've got your traditional life cycle at the top. You sort of your planning requirements then your design phase then you go into your development into testing and then onto maintenance when you've got your built product at the

end. So, how does the secure software development life cycle differ? We're just going to use those same steps. We're just going to insert some of the security things within it as well. So, when you're doing your planning and your requirements, let's do a risk assessment. Is it a product that's going to be open to the public? Um, are we now going to be taking in personally identifiable information and things like that? In the design phase, uh, let's conduct threat modeling and design reviews. When you get to your development state, we want to incorporate some um things like static analysis of your code. So SAS scans, secure code reviews as well, making sure we've got someone on the

security team who's looking at those uh merges and pull requests and making sure that we're not introducing new vulnerabilities. When we get into the testing phase, things get a bit more fun. We can introduce some DAS, um some fuzzing techniques and things like that to simulate a penetration test against the the application itself. And then when it comes to maintenance, we're generally looking at things like vulnerability management, instant response. So if a hack does occur, how are we going to respond to it? How the operations team is going to deal with it? Uh things like that as well. [snorts] So now that I've defined sort of what dev sec ops is and I've got my juice

shop app, I need to come up with my extermination plan to get rid of these bugs. So I've listed some tools here. Other tools are available. Um just find out what works best for your stack. I think there's actually a talk later on today about the various scanners and things and their efficacy. So, well worth checking that out. But what am I going to use for my SAS which is static application security testing? I'm using SEM GRP. Um for my software composition analysis scans this is going to be my third party dependencies open. And for my dynamic application security testing I'm going to use OAPS. [snorts] So I have a little video at the top. You

probably can't see it. It's not really anything you need to pay attention to, but it's just me running the scan just so there's something going on in the background as well. So, as I mentioned, static application securityities testing. That's what SAS stands for. It's going to take your code base at rest. Um, and just run through, scan it, analyze it for any vulnerabilities, um, things like maybe a developer has not used parameterized input strings for their SQL. So that could be um vulnerable to a SQL injection attack or something like that. Ideally, we want to do this within the CI/CD um pipeline and preferably on a per commit basis. There's no point in waiting until you

know you've got a monolith of thousands of thousands of lines of code and then running one big scan and it's pumping out huge number of results that you then have to throw back at the developers to fix and they're going to be unhappy about it as well. So what will we want to give to our developers in order to help them to analyze and fix these? Uh we want good information about what the findings are um where they are and any potential fixes um as well. So I was mentioning earlier about how you run one big scan and it could pump out hundreds of results. We could use something like stage gating so we can reduce the number

of findings that we're passing on to the developer team and making sure that it gets in their backlog and that they are working on it efficiently and in a timely manner as well. So we could for example um run a scan and just check for anything that's critical um and then pass those critical test critical findings on to the development team to put into their backlog. Once they've fixed those, we can reduce that down. So the threshold is now high and again reduce that down to medium, reduce that down to low and just enables the development team to get through things faster. Um reduces their work in progress, their whip limits, which actually means they can move at a faster

rate than if they've got 100 tickets in a backlog and they're just never going to touch that. So software composition analysis, um this is going to be your third party dependencies that I talked about before. Um during the scan I obviously did find a critical which was to do with SQLize. Um no surprise with OASP juice shop that there's a critical vulnerability against it. So what we could do to fix this one is just simply bumper the versions up. So I think we're using an outdated version here. If I look it's 4.28.20 and on sneak here on the website it's saying you need to bump it up to above 6.19.1. Um so software composition analysis and

sbombs go hand in hand. Essentially the sbomb is just a recipe of what goes into your um your code, your product, whatever you're handing off to the customer. And I think it was a Joe Biden executive order that mandated that if you're selling software to a another company that you must provide sbombs with that as well. And I think it came from off the back of the log for 4J log for shell attacks. Um, which was a great Christmas for anybody that was working on that one. Um, yeah. So getting these third party dependencies uh I think we've heard about them in other talks as well this morning. So a third party dependency is obviously code that um if you want to

perform a certain function, you don't want to have to write the code yourself. you can go to Pi or other packages and pull down code that will do it for you. Um, and has already been written and trusted and used. Um, so that's why a lot of it is imported in a Border to make sure that it's safe and secure and is usable. Um, so you could tie these uh software composition analysis into your vulnerability management and you could have something like uh I guess dependency track the OASP one where they go around and they will check periodically for your vault your packages and say oh hang on there's a later version now or oh there's a uh

vulnerability that's been found against that version you need to bump it up to a later version etc. So really important to to get that into uh your your work. Um and again for my dast I have used uh zap which I think I mentioned before and the dast is dynamic application security testing. So this is going to go around perform a web crawl of my application just find out anything there is about it. Um, ideally you would package all of these tools up that I'm talking about into your CI/CD pipeline. So when you're integrating it, you could use your SAS tool then on a per commit basis. Um, when it's merged and then being built,

you could use your software composition analysis and then uh again after that you could use your DAS tool to find any vulnerabilities that are actually happening on the application itself. So what DAT is going to be doing is like a fuzz testing. If it sees input input parameters and things like that, it's going to spam it with weird characters with extra long characters with special characters and see how the application performs. If it handles it or if it breaks something, etc. Um really what we're looking at with the dash scans is focusing on your OAS top 10. If you can focus on fixing those and getting those sorted, you are 90% there because that's just going to be your low hanging fruit.

>> [snorts] >> So, I read a LinkedIn post um a couple weeks back saying Dev Sarcops is dead and hopefully not because otherwise I'm out of a job. Um but why they said that when I read into what they were saying about it is that security was becoming a blocker. Security was putting uh things in place that were hindering the development team were causing frustration for the operations team. So really dev sec ops wasn't dead. It was just that they weren't introducing it in a strategic manner, in a good manner, in a way that was going to benefit the company as a whole. What they didn't do was balance those parts out. So security needs to be in balance with the de

development and the operations. Um I think I've mentioned before about how to do that. So um putting stage gating in in place could be a good introduction into adding dev sec ops in automating your testing um to make sure that it's happening in a timely manner so it's not taking out too many manh hours as well um and introducing short feedback loops so if something does crop up developers know about it straight away maybe we could use a IDE plugin so as the developers are even writing the code it's going to flag up and say hey by the way you've introduced a buffer overflow here or something. Let's fix that before it's even put into a commit.

Most of the issues that have cropped up with um juice because it is a known vulnerable app and it is designed to be vulnerable from the get-go. Uh really could have been mitigated by a proper threat model. I'm not going to tell you how to threat model. You could use something like the stride u model to sort of go through all of those things. Um but fixing issues in development is 30 times less expensive than production. So had they considered you know when they're introducing these technologies from the start to use different approaches or different methodologies then this could have been avoided from the start. Um if you're low on manpower or if you are like a small

team you could even use AI to help you with that. There's a tool called stride GPT uh which literally helps you to threat model um using AI which is a good good place to go. Um yeah I think I've been through that quite quickly. I know it is a lightning talk but I think that was pretty lightning. So uh I am happy to open the floor to any questions if anybody wants. >> Yeah, go ahead.

Yeah. So the question for anybody that didn't hear was how long does it take to run zap and and yeah you are quite right in that it does depend on the size of the application the the you know the number of uh pages and the intricacies of the application itself. I think the one that I ran on OS produce shop took about fiveish minutes. um just for a point of reference, but yeah, if you've got a big sort of web app, then it's going to take a lot longer. And obviously, you can fine-tune it as well. So, if you are doing a deeper crawl or a deeper scan, it's going to take longer, but it's

going to produce better results than if you do a quick scan as such. >> Yeah, go ahead.

>> Yeah. Uh I have introduced these uh so I work as a dev sec ops engineer um and yeah working with trying to get these put into um a DevOps environment. The number one thing that I found that I thought was a hurdle is getting developers to actually use the tools. Things like ID plugins um getting them to actually use them and then fix their findings is is a hurdle. Um, I also think that trying to get them to um, you know, work on tech debt is another hurdle. Tech debt is is sort of where I would put the bucket for, um, finding these vulnerabilities and fixing them. Um, so yeah, getting to actually work on that

can can be tricky. That's why you need a good relationship with them and why I suggest having like a security champion within the de within the development team to sort of foster that. Go ahead. >> I think there's one thing we've also works well for motivating on that sort of stuff is centralizing the scanning a little bit more and just dashboarding the results. Yeah. >> And that way because a lot of times the developers are find that they don't have the time allocated to fix them. So if you're dashboarding it and showing it to the managers and here's how your department is doing then they'll make the timely pressure to give the developers the bandwidth to work on it.

>> 100%. Yeah. If you can get the management buy in to spend the time on fixing these things 100%. Maybe you could take it to the risk committee or risk management team or something like that and highlight these are the findings we've got. This is the risks if you don't um yeah once you got >> we actually just titled like vulnerability management processes. >> Yeah 100%. >> Yeah. Go ahead. So

>> um I don't know if you have to make them public. Do you have to make them public or just share them with the customers? I'm fairly sure you just have to share them with the customers. Although >> maybe that's in a public domain. I'm not sure if it's under NDA or whatever. Um it could potentially put a target on your back if people are out there and know that hey um I don't know company A have incorporated log 4j into their thing then they'll say like oh let's use that and we'll see if we can get a nice good bug bounty out of them or something like that could paint a tag on your back

here. Um but I think it's better to be aware. I think bringing the awareness of what is actually under your systems or your systems are built on is is better awareness. I guess painting that target on your back also means that management etc are going to pay a lot more attention to getting these issues fixed. >> Yeah, go ahead.

Yeah. So blocking applications from being deployed if they have vulnerabilities. I mean that's the goal, right? We don't want to deploy any applications that have vulnerabilities in them to my knowledge. um if you're getting pressure from above and things like that because management want this application out then I guess it comes down to just sort of managing those risks managing those expectations are these findings actually exploitable that's another thing because you could be bringing in these dependencies um and part of the dependency might have a vulnerability against it but is it actually reachable and exploitable that's another situation as well so yeah putting context behind them uh is is definitely going to help in scenario.

Cool. Thank you very much, guys. Really appreciate it. [applause]

>> Thank you, James. Um, is Harriet here?

>> Okay. If Um >> Barry's not here. We'll take just take couple of minutes of think break. >> Yeah. By the way, um we've had a um just so everyone knows, there has been a minor fire in the Grand Ballroom. Uh battery for one of the battlebots shortcircuited and a cloud of smoke came out and set the fire alarm off and that set the sprinklers off. Everything is fine up there, but people might be looking a bit shell shocked. So, uh, apparently the there was a little bit of damage to the carpet from the sprinklers, and our event insurance is going to get a workout. You hear anyone talking about