Too Many Cooks In The Code: The Security Cost Of Collaboration

Hi everyone. Thank you for joining this session. How are we doing? Kicking great. Great weather. I love the weather. It's just right. Okay. So, I'm going to start by painting a scenario. So, please imagine along with me. So, it's 3:00 a.m. You're still in your PJs and your phone will stop ringing. your uncle in here finally gets through to you and you see the message. Sorry, I didn't know what's up with my voice. So, you get a message from your uncle and you say we are down the payment processing library right now. That library was probably written by someone. It's not managed by someone in some far away country who you've also never met and just last

month you had um half of your code half of your code was generated by AI and a AI has made hallucinations right about dependency that doesn't even exist. Now Antaka has noticed this this package and React actually helped create it and now injected malicious code inside. It may sound like science fiction but this is a reality of so many organizations nowadays. So um my name is Adora. I'm the giraffe lead at TJ Enginey based in Manchester and my goal basically is to pull back the curtain on the unintended consequences of collaborative coding as we discuss this title so many co cooks in the code. the security cost of collaboration and why the biggest threat to our systems might already be sitting

inside our code base. So here we have a few statistics. Um 75% of software supply chains have experienced cyber attacks in the last 12 months. Now this is a search of about 180% in compared to 2023 according to Harrison's data bridge investigation report. Now what this means is that if we take a look around the room it means that um 3/4ers of us of our organizations not saying it is but like 3/4 of our organizations represented here based on that statistics have probably been hit um we might be aware of it we might not be aware of it but based on that statistics that's what it means putting it in context and now we've added AI to

the mess because um 97% nearly all enterprise developers are using AI coding tools. How many of us have used GitHub copilot or are using G co-pilot perspective? Exactly. Now there's another interesting statistics here. It says that um researchers found that 19.7% of package dependencies suggested by AI models were completely hallucinated. So that's packages that don't exist but clever attackers are creating and are filling up with nasty surprises for us. So it it it's it's something serious. Now Gartner predicts that by 2025 45% of organizations will experience supply chain attacks which will be three-fold increase for 2021. So I can't wait to be in 2026 to see if they will be proven right or wrong. I

mean, we've seen the recent attacks on co-op, M&S, and co. So, now in as much as collaborative coding, you know, has been really um helpful in our modern workplace, the way we, you know, build our systems and all of that. But we've also created the largest attack surface in history as a result of that. Now we're dealing it with AI generated code and we are just flying blinds. Well, let's, you know, let's just, you know, go back in time a bit, you know, go through how we got here back in the '9s. And then if you wanted to build software, you probably wrote it yourself. Every line of code, every algorithm, you knew everything that was

in the library. It was slow. It was painful. It was expensive. But again, you knew everything that was in the library. You knew everything that was in the code. Then the 2000s came with the web revolution and we had dynamic websites in version um writing code from scratch using you know JavaScript PHP um and the likes. The mid 2000s and reals came in Django and we started building on the shoulders but we still need everything in our applications. Yay or nay. Yeah, kind of. And then microservices in the 2010s come came into mix and we had modern applications became distributed systems spanning hundreds of services and then in the 2020s in the rise of AI

assisted um AI is the AI assisted era and AI to now write a significant portion of our codes. So there's another statistics here that say that 27%. There's been an increase in the usage of AI tools, regulatory usage of AI tools just by 27% just between 2023 and 2024. So this is something that is on rise and it is also in our tax office that I mentioned earlier on. So this is where we've ended up and it's quite interesting because um now we use a lot of third party code 80 to 90% of that third party code is other people's work. Um it's like you know having is like selling sandwich you don't you don't

make you're just applying the napkins we're using other people's work um a large percentage of that AI generated code we don't exactly read I can see the spells yeah so um we also have dependency seems to have dependencies. They have dependencies and you know it's really complicated and tedious right now and all of this work is done by people we've never met and we probably will never meet. That is the you know current situation on ground. interesting but also causes room for concern and caution even as we deploy these tools. So you take a look at this diagram here. Okay, this is what a typical So this is what you know installing one simple NodeJS in your library looks

like. It's like an explosion. So that's you trusting someone with your with your business. Every single one of you use it is trusting somebody with your business. Now the thing is you're not just trusting someone your business. It's like giving someone the key to your house and the person gives it to another person gives it to another person who gives it to another person. Now in as much as it helps us you know skill build faster you know faster deliveries skill you know bigger than ever before but it is very uncomfortable because every code you did not write whether from a human being or from our new AI overlords is a line of code that

you don't control and that's where story gets interesting. We're just going to briefly I'm just going to touch the traditional um threat models and the modern threat models. So traditional threat models, the attackers thinking of the perimeter, the firewalls, intrusion detection or trying to bypass all of that, the network security. But with the way things are now, it's different. Why? because it's already based on the code that you deploy. So think back to the fall of Troy which is quite similar to what this is. AI is like a gift you know that giant gift that is very very fresh never been seen before. But inside just like when it was offered to the king of Troy inside that Trojan

horse where you know the the attackers waiting to take down the city it's similar to what we have now we've got this beautiful beautiful tool that can do so much for us but inside the code base right these pe these attackers have baked you know malicious code inside. So, look at some case studies. In 2020, December 2020, um, Fire Art announced the discovery of a highly sophisticated cyber intrusion that leveraged a commercial software application by Solar Wings wings. So um it was discovered that the attackers gained entry in 2019 September 2019. This is only discovered in December 2020. So they had 14 months they had 14 months of unfettered access in the system. So how did it work? They compromised the

way build system and by March 2020 they plunch remote access to um remote access to malware into their updates thereby troll jing them and 18,000 customers downloaded this compromise code between March and June 2020 and the funny thing is that these malicious codes had legit digital signatures. It looked completely trustworthy and it affected they had access to US Department of Homeland Security. They gain access to department of energy, Department of Commerce, they able to target such type of um you know organizations. So think of it like poisoning the instead of trying to break into individual homes, poison the water supply to my home to those homes. I mean one compromised thousands of millions of victims that's what we are facing now

another example is um one from 2016 a programmer took down the left pad package here published to npm so there were thousands of software projects including Facebookify PayPal Netflix I used left pad as a dependency so Um what he did was um he he um was able to they were unable to build or install after that attack. So this caus widespread disruption as technology corporations and large used left part in their software.