Shift Left, Secure Right, Building An Open Source SBOM Driven Vulnerability Management System

good afternoon everyone thanks for joining me uh my name is Cosmo and I'm going to talk to you about how you can use an es bomb uh which stands for software Bill materials to drive your vulnerability Management program but before I get into that let me just introduce myself I don't know why that up there we go so who am I so like I said I'm Cosmo I'm a Staff devops developer for a company called owners box it's a small startup based out of Kitchener uh I'm a family man I have my wife Rita my two daughters help algra that I love hanging out with I'm a SE devops and open source practitioner I've been in this field for

over 10 years now in various roles and I worked in uh various Industries um including telecommunications Automotive uh embedded iot industrial control systems and paid fantasy sports and and gaming Industries I'm also an open source Advocate I actively contribute to open source uh projects and Community initiatives what I'm going to talk to you about for the next 20 minutes or so is uh what is an bomb so introduce it um and why are they becoming more and more popular uh how do you generate them what do you do with one once you have one uh how can you automate some of this and then we should have time at the end for uh conclusions and some

questions so first off what is an es bomb so it stands for software building materials and the easiest way to think of an sbom is similar to what you would see on an ingredient list of the back of a cereal box uh it's very rare in today's society that your software project is a pure library with no dependencies for the most part you're pulling in dependencies and those are pulling in transitive dependencies so a software billing material is essentially a list of ingredients that make up your product whether your product is a library or an application or a mobile app or firmware and it goes on and on uh there's two uh commonly used formats one

is Cyclone DX it's from the oos Foundation uh they run that one another one is spdx that's run by the Linux Foundation and the output of an esom is machine readable so the two most common formats are either Json or XML so you could read one by hand if you want to but really it's easier just to parse your way through it to pull out the pieces of information you need a quick timeline of how we got here and why they're becoming uh popular so if we go back to 2019 2020 there was the solar winds attack uh solar winds is a fairly big company they have a product called Orion which is a network management system that product

was compromised and shipped to thousands of their customers and their customers are many Fortune 500 companies and US Government entities and what this allowed is attackers to gain access to their network uh and this attack went on undetected uh for quite some time uh there was also the colonial pipeline uh ransomware attack that occurred in May 2021 uh where all of their data was encrypted and it was a ransomware attack they had to pay to get their data back this caused uh fuel shortages on the Eastern uh United States and panic Bine and it also raised awareness of our uh critical infrastructure is very important and has to be uh better protected so we fast forward a little

bit uh the first big Catalyst here that's why I have kind of a star on it was the White House Executive Order 14028 improving the nation cyber security um jits last week uh an even bigger one I think was the European cyber resilience Act was signed into law and both of those acts I'm going to go into them shortly I'm going to skim through them very quickly um I said enally mean if you want to sell to the US government or you want to sell any digital product in Europe uh you need to provide an sbom uh you need to know what's in your uh product and make that available and I have a couple other uh

examples here the FDA the US Army um if you want to sell to them you're going to have to provide an es bomb and moving forward in March 2025 PCI version uh 4.0 so payment card infin structure they also have their own sbom requirements these are just a few smaller examples but it is becoming more and more popular and there's more and more uh Industries uh both private and publicly funded that are requiring es bombs so if I just quickly go forward so I did mentioned executive order 1428 there is a section in that executive order where uh you need to the purchaser you need to provide the purchaser in this case the purchaser is the US government a

software bill of materials either giving it to them directly or publishing it on a website with this executive order uh they also um uh asked the uh ntia the national Telecommunications and Information Administration that they need to publish what are the minimum elements that make an es bomb and as I mentioned in uh just last week I think it was on October 10th the European cyber resilience Act was signed into law and this basically says if you're selling a uh Hardware software product in the European Union that has a digital element which is pretty much anything mobile apps smartphones laptops uh firmware smart bulbs anything like that you need to be able to provide a uh

uh an bomb in a machine readable well supported format which is most likely either spdx or or Cyclone DX um so I did mention then Tia was tasked with coming out what are their minimum Fields required so there shouldn't be too many surprises here uh especially those top six supplier name component name version a unique identifier dependency relationship and an author uh with those six pieces of information you can pretty much figure out what something is and then a time stamp so you know when this uh sbom was was generated but there's so much More Than This here's just a snapshot I couldn't really fit them all on this slide of the latest spec of

cyclone DX which is version 1.6 as you can see there's dozens upon dozens of fields that could be added and and and are added to an es bomb so how do you actually generate one so I'm going to show a static demo here uh using a project from oops called The Juice Shop The Juice Shop is a very sophisticated and insecure uh web server that's great for demos like this um there's many tools both commercial and open source that you can use to generate an espon this is a very small list of some of the more well-known open source ones I'm going to use this first one on the list the Cyclone DX npm gen Tool uh because

the Juice shot project is a node npm based project so I have this installed on my laptop here so it's a pretty simple command uh my Cyclone DX npm is installed I want to tell it what output file to write to I want to give it the spec version in this case 1.6 and I need to point it to my source code uh in this case I want to point to my package.json file you could also point it to your build artifacts this is going to be a source s bomb but you could also point it to your final build artifacts and it could pull out information from there or anything in between so after this

command runs and it only takes a few seconds to run I do have my s bomb generated there uh take note that it's 4 megabytes in size so it's fairly large there's quite a bit of information that goes into these and we could just parse through one now fairly quickly so starting these are the first 10 15 lines of the ESP bomb um what I have circled is just one of those minimum elements uh a time stamp is required but there's a lot of other key pieces of information here uh for example the bomb format in this case it's Cyclone DX the spec version 1.6 uh what tool was used to generate this s bomb in this case it's Cyclone DX npm

version 1.9.3 um what component did I jit scan so there's the component section so I scann jice Shop version 17.11 it has an MIT license and here's a unique package identifier I can use to identify this package and then the component list this is a list is the very long list there's dozens upon dozens if not more dependencies here I'm just using this uh one example for now so I know one of my dependencies for the jeice shop is a library called uh vm2 version 3.97 it has the MIT license um and I have a unique identifier that I can use to identify this package so so my s bomb contains many many uh components I could

kind of parse through them or read through them to kind of see what I'm dealing with here um so what does this mean right off the bat without doing anything else without just just by generating s bombs what does this give you well you have an upto-date list of uh software components including versions and relationships uh not only do you know your top level dependencies those are pretty easy to get from a package or Json file but you also have a list of all your transitive dependencies and those dependencies and so on and so on so the next time a big uh vulnerability comes out take for example the log for J1 that happened in December

2021 or heart bleed way back from 2014 instead of looking through source code or build files or build artifacts you could just reference your spons to see how are you impacted by this or any versions impacted by this you can write your own custom parsing tools as well but you can do more so the big use case with um s bombs is for vulnerability management so let's see how you could do uh some quick vulnerability scans so so I'm going to use the vm2 example so I know I have a package called vm2 version 3917 let's pass this to the osv API the osv is an open- Source online vulnerability database I believe it's hosted by Google uh so I could just make

a post request to this API and what do I get back well this library has vulnerabilities I should probably look into that maybe I'm impacted by them maybe I'm not maybe I just need to update some versions maybe I could ignore it maybe it's a you know drop everything and and deal with this um obviously we don't want to do that V ability um package by package as there's you know hundreds upon hundreds of dependencies in your es bomb so we could just pass our sbom into this tool called osv scanner so I'm just giving it my whole s bomb and what it's going to Output does excuse the blurriness on that um but it's just here's a table of

all of the vulnerabilities uh in my juice shop so it has a reference their osv URL so I could click that to see more information about it I have my CVSs score the ecosystem the package and the version um and the source so instead of doing this dependency by dependency I could just pass it a tool that gives it everything for me um the other big use case with s bombs is what about license compliance so I like to say just because it's open source doesn't mean it's free uh depending on what you're doing with your product how you're selling it how customers are installing it there might be obligations for you to kind of give

back to the community or publish uh your software as open source as well um so what if I want to just see what licenses am I using so uh same tool the osv scanner I could scan my spom I could enable the license summary check and then I get a table of um all of the different licenses in my product um what if I have a list of approved licenses so in this case I want to allow BSD version two version 3 Apache 2.0 and a couple other ones uh tell me if I have any software that falls outside of this list of licenses and the tool can give you that so with your esom you kind of find

out if you're in or out of compliance with in terms of your uh licensing requirements like I said this doesn't necessarily mean it's a red flag but it's just something to you should probably look at it and kind of see what's going on and one of the last things I want to quickly show here um is what about verifying the quality of V bom so I did mention um earlier that the NTI did come out with minimum elements what if I want to test for those es bombs after all are meant to be shared um there's many tools that generate ESP bombs some do really good job some not so good so I could use

this tool here called s bomb us to score my sbom I want to give it the category of ntia minimum elements and I'm just going to pass in my uh s bomb and it's going to give me an output like this so for my ntie elements everything scores a 10 out of 10 that's great except uh component with suppliers my s bomb doesn't contain any of that so this is where a topic like sbom enhancement comes in that's a talk entirely on its own uh but there's a lot of tools out there where once you generate an esom you could enhance it with additional uh data to increase the quality of it so this is great I could do this on a

regular basis um but what I really want to do is just automate this um I want to generate s bombs either on every release or on every commit to the mainline Branch or maybe even on every poll request depending on what use cases are uh I don't want to manually have to generate all of these all the time uh so I'm going to have a couple static demos here I'm going to use uh uh bitbucket which is a product from alassian so bitbucket is very similar to uh GitHub G bar or devops just an online uh git-based solution uh a bitbucket pipeline you could think of it kind of like a um GitHub action and a bitbucket

pipe is kind of a preconfigured uh uh pipeline that you could just drop into your uh CI and it'll run it for you um and to write a bitbucket pipe you're essentially just defining a Docker file uh and the entry point runs uh and that's your bit bucket pipe so I have a couple quick static demos I'm going to go through the first one is just how can you write something that just generates an s bomb uh as part of your CI Pipeline and the other one is now that you have an s bomb let's make it more Advan let's get some vulnerability information and quality information out of it so like I said the first example uh

bit bucket pipe to generate uh an sbom um so this is the docker file uh for that and at the end of this presentation I will have links to GitHub uh where you could find all these examples and other links as well uh but this is the entire Docker file so it's pretty straightforward to just generate an spalm all I'm really doing uh to Summit uh summarize it is I'm installing that Cyclone DX npm tool that I demoed earlier onto a Docker image I have a simple bat script that acts as a wrapper uh that's going to be my entry point like I said I will make the source code the source code to this is available I

will put the links after this presentation so very simple um Docker file to actually add this to my uh pipeline uh bitbucket pipelines just use yaml files similar to a lot of other CI tools so I have a couple different steps here one is build and test the other one is uh generate Cyclone DXs bomb uh I want to pass in my pipe so all that's really doing here is just downloading this image from dockerhub version 1.5.0 of it and then I'm just setting some envirment variables that my rer script will grab and then make some decisions of so I want my npm spec version to be 1.6 I want Json output format and there's a few other things so what does

this look like when it actually runs so I have uh in my bit bucket pipelines I have my two stages that run my build and test takes about five minutes uh generating the Cyclone DXs bomb takes only about 35 seconds I have some output here I can see what switches were used and what this actually gives me at the end is I have artifacts and I have build artifacts I can download so now I could download you know Juice Shop version 17.11 CDX Json and for every however I want to configure it like I said every release is usually how I do it but you could also do on every commit uh to the mainline Branch you're generating an

upto-date sbom now you could do other things with it you could call it a day here and at least you have it you could archive these and and keep them secure uh you could share them publicly uh if you want but at least your CI is generating it every build um what about if we want to take it one step further so what if we want to pull out some vulnerability information and we want to make sure rbon meets a certain level of quality uh before we we share it or do anything else with it so I have another example of a Docker file here this one's a bit longer so it's a few different slides

but it's basically I have a goang uh wrapper uh I have a Docker file that just installs a couple different tools so in this case osv scanner and sbom Qs my run commands actually do the installs and I have my uh uh go wrapper here that's called esom UTS that this will just run on the entry point and based off what environment variables you give it it'll make certain decisions so to pass this into your pipeline looks very similar to the one before I have a step uh scan sbom with sbom utilities pipe I need to uh tell it where the pipe is and in this case it's in Docker uh Hub I want to tell it I need the path to the

es bomb so here's the path to the es bomb and I need to tell it what to do so I do want to scan with osv and I want my output format to be a table I do want to scan it with esom Qs and I want my output format to be detailed which in that tool it's a table and just put all the output in my build directory and uh archive my build bu directory so I could download it after and take a closer look so what this looks like when it runs similar to before I have my build and test step I have my gen Cyclone DX s bomb step and I have my scan s bomb with

sbom utilities pipe uh so this runs I still get my build file uh my output so I have my juice shop. Json file I have my esom Qs results um and I have my osv scan so I could take a look at this every release and there's much more advanced things you can do um so this is just the bare bones kind of scratch the surface basic I just want my es bomb and I want my vulnerability reporting and Asom Qs reporting as part of my CI but most of these tools give you more advanced capabilities like what if I want to fail my pipeline if there's a high priority CVSs score so if it's anything CVSs 9.0

or of above fail the pipeline or what if these vulnerabilities don't impact me I did reachability analysis and none of them really impact me or there's just no fixes available or I just don't care about them you could give these tools ignore files so it'll only show vulnerabilities that you might actually want to take action on so where to send all of these s bombs that again is another topic on its own actually interesting enough today there's a bsides conference in New York and there's this is one of the topics what to do with all these es bombs um couple things you could do with it just there's different tools out there to kind of send your es bombs to so instead

of writing everything from scratch you could just host a server or send it to a SAS service so so OS dependency track is a free and open source Tool uh you could set stand that up you could send your s bombs to it and it will give you um more prettier graphs and more detailed information uh another one is Interlink SAS Interlink is a it's a commercial provider but they do have a free tier and once again you could send your sbom to them via API so your pipeline could do it and you could get much more insight into kind of what's going on so just conclusion and key takeaways here so due to executive order 4028 and

due to the European cyber resilience act um there is widespread spread adoption of esoms uh Industries and governments are mandating their usage and even private companies as well uh there's plenty of open- source tools available to kind of build your own really cool Solutions this is just one uh example uh so you don't necessarily have to spend thousands on uh commercial uh Solutions that's really how much they cost especially if you're a smaller company so ex said the tools are out there it's becoming more and more uh prevalent and some neat stuff going on in that uh in that area so I think that's everything I had right 20 minutes so yeah thanks everyone and ex I have

the the easiest way to contact me is just through email it's up there it's uh or just LinkedIn uh connect with me reach out to me I have some links to some sample projects I will post the slides there as well and I think the like bsides also post the slides the slides so perfect whatever you want us to do with them yeah cool and then the very last slide here is just all the tools that I showed here's some links to them so once you get the slides uh feel free to take a look at these projects great thank you thank you Cosmo sorry I was calling you commo and then that's your last name yeah uh so we have

time for questions and uh I think the Checker shirt we've been using triy to scan containers successfully but I think you talked about uh management of issues um have you found any tools that help

um manag problem the management part is tricky uh we use one Tool uh sneak but it's not like it does its own thing right you can't really give it an es bomb uh I know with the Interlink tool I believe that has a hook to jir if you're using jira I think it has hooks to other uh ticket management system where you could do things like that yeah so I got a question for what happens if one of the dependencies falls outside you want detect the vulnerability or it falls out change the LIC then what happens you can only DET it it differs use case by use case um so if there's a varability that's

detected it's you know you should look at it to see does it impact you do reachability analysis or maybe it just Falls outside of your range at the CVSs score is too low um you can see if there's a fix available and update that I different languages have different kind of rule sets so for node npm projects one thing we do pretty often at least once a quarter just run npm audit fix and it'll go in and just update a whole bunch of dependencies for us in a non breing fashion so we're kind of kept up to date um it's hard to have zero vulnerabilities uh especially with ecosystems like node npm or python or go

uh but really it just it it differs based off of kind of your company and your rule set and how you want to manage that thanks you're welcome so if you're building supporting this product this would be the S you would provide to customers or downam what's your esom going to look like if you're building and supporting a close Source product like let's say you're inidia what's your esom going to look like when you sell drivers you could it's it's a great question I don't know if I have a answer to it I focus more on the open source side of things uh close Source companies I believe they would I I don't really know what they do leg

they have to so it's yeah there's been a lot of talk about we sold you this we made it yeah so I know there's been in a lot of groups so there's been a lot of information about what do we red doct from as bombs like what do we just not want people to know uh that's kind of a a a topic out there now um yeah on the close source side it's it's interesting because yes Nvidia is definitely using open source just what do they tell their customers and maybe their Risk Managers are working diligently on answering those questions speak RIS management the vulnerability tools they need a a machine readable vulnerability feed are

they clear about where they're getting those yeah so the example I gave is the osv API uh they have a lot of just documentation on their website of kind of where they get their sources from uh there's the Sona type OSS index there's lot of other ones out there as well man yeah yeah I don't have a question but I have answer to that guy's question like you asked about the tool we and flux had started doing that part and you can explore those tools I just use it we and flux CD CD they are two different tools CD has like I used it and it's more comp yeah uh you talked about briefly about reachability analysis there um can

you talk about how you approach that and um like what happens after you have this bones like what what's the process to to remediate do you tri on the security side you give them to developers we definitely triage them um in a perfect world there's just a fix available update to this version uh a lot of times that's not the case and then it just it's a caseby case basis how urgent is it how vulnerable it is it typically the big ones like the log for JS the Heartbleed there's a fix available right away um for the ones that aren't like that it's just you know risk that you need to be willing to to

deal with yeah but who does that security developer it will differ company to company uh I work at a small startup we were multiple hats so a lot of times it's us uh bigger companies that have the red team blue team mix it might you know it it it differs and theability Analysis how do you do that uh there's tools out there there's threat modeling I think there's a couple presentation today about threat modeling um there's a lot of different tools and and literature on it uh there's you know many different approaches I think it depends on your runtime and and and how your product is configured I think we have one more question here and then we'll W so uh

what are your thoughts on Source based distribtion matter of basically sased distrib okay

comp what is my opinion on that I haven't I don't have a strong opinion on it it's we all use Linux I use Linux I never compiled my own version of it but it's yeah I I don't I don't I don't have a strong opinion there IDE yeah it would be good to know what's in there especially if you're you know compiling Linux for an embedded system something that goes into Healthcare yeah you definitely kind of want to know what's in there um so yeah it's I I don't it's definitely a good