Securing the Digital Supply Chain: What we should do for Supply Chain Security in DevSecOps?: Panel

all right so the uh next talk panel actually uh securing the digital supply chain what we should do for uh supply chain Security in devops uh we have uh harvinder doni uh Tanya Janka am I saying this right like just okay it's one of those you read your name constantly but you never say it out loud so like you're never sure uh uh Dwayne buzak and uh Michael Schwab so take it away I'll let them introduce themselves I'll start so my name is harinda dami and I'm a principal security consultant with epic security um so I did uh incident response all my life but uh nowadays uh like you know kind of uh I got to call into incidents

where like you know Dela cops related so I thought like you know it's a good to talk about like you know how we are doing like you know this Security in supply chain for devop so yeah um I'm Tanya Jenka and I work at semrep and I'm head of their community and education so I run a a big Academy I speak at conferences and I try to share information and then I give secure coding training and I also love software and helping to make it more secure Dwayne Dwayne budzak um I've been working in it as software developer architect and pointy her boss for something over 30 years now in fact someone out in the audience recognized

me from a job I had back in 1993 if that's not depressing I don't know what is um so my interest in security is well uh try not to victimize people with software I've got strong opinions about that but I'll pass it off to Michael because he's got opinions too I think anybody who has uh been in this industry for quite a while has opinions some of them well formed some of them not um so Michael Schwab director application and product security for the government of Alberta um trying to develop a Dev psyops program for the government to ensure that we can instill um security practices in the full life cycle of apps and uh also looking at

some policies with regard to that and security architecture for the entire GAA

so so so as you see like you know there is one government guy there is one like you know uh private consultant sorry Dwayne and one service provider which is tan so uh like you know whenever you like you know kind of uh start implementing devops you always need a technology something like Sam grap and you always need like you know someone to implement uh like you know Dwayne and um there's always definitely a customer right and uh so this is the theme that we are running with um so my question uh to all of you um what are the current exposures uh and recent supply chain attacks uh we are seeing in the wild and

um we heard a lot of shift left approach like you know kind of wherever like there's a lot of talks on it um is it working uh with devops uh with supply chain so let's start with T I feel like you asked two questions so a recent big exposure with a supply chain is the polyfill attack so basically there's this really popular open source library and there was one human being supporting it and he got tired and had a life and other things happened and so he told everyone he's going to stop supporting it and then he let his domain expire and whoever lets their domain expire that's like saying my dreams are over and so he didn't have

122 more dollars to renew it he let it go A malicious actor got that they changed the thing people downloaded it bad stuff happened so that's an example of of bad ones I don't think that's a that's a problem with the ecosystem of the way we don't support open source in my opinion that's the problem there if we're going to all use open source especially at like giant companies yeah maybe we should pay for it yes and get something right so yeah I want to hear your opinions this is a tough one um everyone can bust me too if I say it depends that's illegal up here right it's illegal yeah it's not allowed so

some of the recent attacks um this one you probably wouldn't think of as an attack but to me it's probably one of the most massive denials of service that we've seen recently and it's probably going to result in some really interesting things crowd strike yeah yeah yeah how much stuff did that take out do you think there's going to be multi-billion dollar lawsuits out of that one yeah and of course there's our classic ones like move it everyone was using this piece of infrastructure it happily got compromised and guess what your Solutions are toast and then of course as Tanya is alluding to open source is Big Lots of camping going on around common package names and the like

and I'll let Michael go because I think he probably has a comment or two about stuff like that so um there's a couple of things um so with the development of devops one of the things we want to do is have a carrot for developers to come in and do things in an automated fashion a lot of those tools npm for example will go through and check to see if you have vulnerabilities and there has been this new thing called Ai and uh it's been being used to generate cve some of them rightfully correct some of them not so correct and it's what somebody on YouTube has called uh deos or distributed harassment of service so

think of a single open- Source developer being inundated with a bunch of reports that are questionable and having to go through and sift through all of these in addition to actually doing the development of the open source library or package that they're supposed to be doing it's becoming a bit of a problem because those reports may have a high uh impact assessment and it may go through and kill those uh those cicd pipelines continuous integration continuous uh deployment another aspect I think that's coming up um is things like um Bad actors coming in XZ that Library coming in using this similar type of uh approach um the individual like Tanya said may be a single developer um they

may have other things going on in their lives and they may find that they're inundated with this stuff and can't handle it White Knight comes in tries to save the project and all of a sudden you get these compromises injected in and you find that you have somebody at Microsoft who has probably a little OCD and determines that you know half a millisecond or half a second of something is too long and starts to again that's not the typical user experience so yeah okay you want to add something oh yeah um you had two questions the second one was yeah the second one is shift shift is shift left approach is working when it comes to

supply chain okay so I really like the idea of Shifting left which means starting security earlier so we're going to build software and instead of showing up at the end and saying you did everything wrong you know how I gave you no guidance you didn't follow what's in my brain that I never told you about so that doesn't work and so shifting left is like what if we showed up at the kickoff meeting what if we gave you guidance on every single part of the system development life cycle and it does work if you you do it that way but unfortunately marketing people and I work with lots of marketing people and we have friction sometimes over things

like this they're like oh shift left means buy our product I'm like that's not what it means and so we have had a lot of marketing people who are better at marketing than us security folks telling the world shift left means buy a product and that's not what it means and they've really blurred the meaning unfortunately and um yeah we had a marketing consultant who was telling me shift love kind of means whatever you want I'm like no it doesn't it means this and she's like no no it does like what made you the expert and then I showed her the top page of Google results for pushing left and I'm seven of them and I'm like I'm liter like no

um and she doesn't work for us anymore anyway what do you think D unfortunately I'm in violent agreement with you um shift left is kind of an interesting concept because people are taking it and running with it in ways that resonate for them I was chatting with a gent who's a software developer fairly recently and I won't mention any names and he noted that his instructors at school were telling him that shift left is cicd pipelines to me that's way too late if you've already got code you haven't shifted left yes security needs to be a first class consideration like Tanya was saying if you're not baking into your requirement set you're missing out if you think you can test it in like

you can test in quality into your system that's just crazy talk and again this gets back to as Tanya also noted and I appreciate her perspective on the products and vendors no disrespect intended um it's too easy to put slap that label on something and for us as professionals it's too easy to go for retail therapy where we think if we actually buy a product it will solve our problem rather than having to retrain people to consider security up front and bake it in through the life cycle you folks all know this stuff so it's too easy to talk to and I'll let Michael off for his perspective culture eats tools um so you know you can have a nice

tool it might have a nice dashboard um really easy to ignore if you don't go through and incentivize the individuals to do the right thing think the right way change the way that they're delivering and operating the tool isn't going to do anything for you you'll have an invoice and you'll have a capability that no one is using yeah so har just add a little color to that too I don't know if if you folks have been keeping up with some of the research and stuff that's going on in this field it's about 5 years ago in 2019 meta did a massive study of what works in security and security pipelines they found out that if security items

come up outside of the developer workflow it will not get addressed if it's after check-in the developer has moved to their next task they will not go back doesn't matter High critical whatever they will not address it and this was the security team at meta raising critical vulnerabilities to management y didn't happen well and it cost so much more yes like so who here's been a developer before lot lots and lots of us right so imagine you're actually coding and it tells you oh hey you know we don't use inner HTML here instead we use this you're like oh yeah I forgot and then you do the thing but if six weeks later after you've closed that code and

you're working on something else maybe in a different language and then you get an email from the security team you're like I'll do that later and then later maybe never oh yeah it happens all the time so next question like you know we talked about like let's go to your first point uh where like you know kind of you said uh like you know uh there is one person supporting uh the package so if you see like you know on GitHub repositories like you know GitHub repos there is so many packages like for node for python for go like you know kind of and uh Developers like downloading them and executing as system on their computers

and then they got popped right and um uh like you know kind of as a service provider um like you know what are the primary security challenges that you are facing in ensuring the Integrity of third party components like you know the using in the tools uh or and also in open source libraries uh as we saw numbers of op Source packages are like you know increasing every day like you know in thousand and like you know Millions actually so okay so I have some bias because I work at a company that works in that so take everything I say with a grain of salt so we have a lot of products right now that you can get and

they'll be like you have 100 libraries cool 50 of them have vulnerabilities in them of some sort awesome we're not going to upgrade and change literally 50% of all we don't have time we have things to do so now all of the tools are starting to change so that what they'll do is they'll say you have 100 Li 50 of them have vulnerabilities but only two are actually reachable from within your code so if you have a library the library doesn't do one thing if you include the math Library it's not like oh I do is addition it does everything right so it might have 500 or thousand functions the the vulnerability is in one function or two functions or three

functions usually unless it's log 4J solar winds Etc but most of the time it's in like one function if you don't call that function you're cool you're fine and so I feel like we need to look at context more or for instance it's like yes we have this vulnerability but it's on a system that's like air like it's air gapped so how is someone going to attack that so yes we should upgrade it at some point but it's literally air gaap so maybe that could come after something else so I feel like we need to look at context more because we can't fix 4,000 vulnerabilities in one app we have work to do so I I think a lot of

absec teams are working harder to prioritize things so that they're giving a reasonable amount of of work to the developers and doing the shift love that Dwayne and Michael were talking about so that in the first place we're avoiding some of those vulnerabilities if we can so in the idea it's like hey so like image tragic was pretty bad what if you just didn't use that library at all ah anyway yeah and so this is again where I'm going to riff off of what Tanya said to me a lot of what you said is absolutely bang on and for the developer sandbox that's one place where we're underutilizing the tools not everybody again this is contexual lots of teams

and products and places are mature but the same tools that exist in that cicd pipeline also exist in your developer sandbox why don't you catch the stuff up front the other thing that you can do for me as a developer and I confess to having all pretend I was a developer at one time and wrote a few lines of code um some of them were even good some of them were bad um but write something in there to tell people which paths are not meant to be used if you're going to go forward with the vulnerable Library do something to make it obvious that you've managed that vulnerability as Tanya said if it's a path thing where some things

are not going to get used what's the next guy going to do if he sees you're using that math Library there's another function there the math library is already there what's he gonna do yeah use it so wrap it put a comment in do something to tell people not to do it I mean to me one of the big defensive things you can do is actually use the tools and I apologies I know I'm going a little bit of a rant here most development organizations are still at a level zero or one in terms of maturity yes we're not doing the know good practices I've seen very few places doing actual honest to God code reviews I've seen very few

places doing effective code reviews your human brain is probably actually your best tool and it's really interesting because really smart people don't necessarily apply it well because again the context is King there was another thing that was done recently which was kind of interesting they tried to penetrate the Linux kernel so some re security researchers submitted some changes to the kernel all of which were poisoned in some way they were all benign poisonings but they were all poisoned some way you think those things got through the review process they did the study was retracted so you won't find it but it was retracted on ethical grounds because they didn't ask for permission to do this I hope the team took away the fact

that their code reviews from a security perspective weren working and they need to add that to the Cod reviews but again Tanya's bang on GIF left Works adding the tools and part of that tool is the wetware don't forget that okay and the one thing I'll add um so I recently had a conversation a couple months ago and it was about a vulnerable package and it was a discussion about I don't really want to fix this but it's a critical and High um and so the conversation was okay you've gone through and you've checked your code but what about what you're actually relying on so your Django server your your other things right did you did you

assess those you get things out of the box for those and those have been published to the internet have you gone through and do done the due diligence on that I know what the answer was it was pretty much silenced look down mute can I add one more thing um and a thing that I've been trying to do with a lot of clients recently assuming they're on board the security train so that's a big jump is trying to do ERS of abstraction so if we are you know shifting left we're doing all these things that's great but what if we add other layers of mitigation so just like Dwayne said adding a comment that says

this part's not okay right so we're using this Library we know there's a big risk in here don't use these functions you could also put like a security wrapper Library so when you tab off of it it's like it renames it to please don't call derivative function or come see Tanya if you do um right like it could be something like that could be adding a WAFF and adding a rule to the WAFF that if anything looks like an attack on that thing it blocks it like there's layers of things we can do if we want to take things seriously and sometimes it's faster to write a rule for your wa than it is to actually

upgrade a package if the thing's really old and depends on it and yeah for was just tip up the iceberg so oh yeah um so my next question to the implementor um like you know kind of from an Implement um perspective how can organizations effectively integrate supply chain security measures into cicd pipelines without causing any significant disruptions to the development processes uh is there any quick win that organization can get like like you know kind of with this uh approach so we've talked about a bunch of these already um the without disrupting developer process is a funny one because again in theory good practice is good practices it should just get used in practice good

practice is not used which is why we exist um so to me the things that you can do are take a look at your stlc and see where you're falling down from a security perspective if you're not considering security requirements upfront if you think you're doing Dev SEC Ops and you're not thinking about security upfront you're not doing Dev SEC Ops have security requirements have them articulated have people have to opt out of them that's one of the first things do design reviews do security design reviews if you've got an asset that it's highly critical if it's at your highest protection level bring in Professional Security folks to take a look at your design too not saying the

developers can't do it not saying The Architects can't do it but other folks can help um same thing with the tooling again we talked about the developer sandbox the tools exist in your sandbox use them I mean lots of folks like turning the warning levels way down on their C compiler because it makes the job easy but it also makes the software more vulnerable put the tools in there and then on deployment too look at what you're deploying validate that what you think you're deploying is what you're supposed to deploy actually sign your sign your binaries and get things out there and then as Michael said at the end of the day a lot of this is about

culture the first thing you're going to have to do if you want to do this effectively is change the culture and to me one of the key elements in there is the accountability so you see a lot in the we'll call it the industry press these days about developer flow and not interrupting developer processes to me that's cool you need to get work done the developer productivity is about getting functionality done but it's not about writing code it's about delivering good function that works and part of the work thing is the security of it so if you're going to download a library and you're going to use it I want to give you the the decision rights to do that

but I also want to give you the accountability where if you've downloaded something that's really naughty guess who's going to get called out if that thing gets busted so give you the decision rights you also get the accountability and I also have to support you by letting you know what you can do about these kinds of things getting back to where Tanya was talking defense in depth right yeah and you guys are easy you're easy to sell on that developers are harder but we also have to change their jobs to make it so it's not just about cranking out code it's not just about lines of code it's about functions that work and if you'll pardon me for a little bit

more of a ramble before I hand it off to this guy um and this is again a minor rant that's where actually AI today is falling down but maybe AI tomorrow will help because the AI of today is about cranking out code yes it's directed at Junior developers yeah the thing that differentiates a junior developer and a senior developer is the senior developer is thinking about the whole context in the system and try not to write code the junior Dev is trying to solve today's problem the senior trying to solve tomorrow's so that's another problem to okay Michael do you have any comments on U I guess what I would also say is the development life cycle isn't

just about code it's also about the context that that code runs in and it's also the context of what that that code is running to automate so understanding where your code is going to run what the intended purpose is what the context of the information that that system um will operate on very critical um it also um will require a bit of a change in the way that the business approaches this as well because I think least what I have seen the business usually takes a hands-off approach to the technology that's not us that's Tech right I think there has to be a stronger relationship um between the business and the technology um side of things because if

you go through and you have a a crap business process and it has holes software isn't going to solve it it'll make it faster and it'll automate it but if you've got a process where for example you don't have segre segregation of duties or other types of things technology is not going to solve that for you yeah Donna do you want to comment I thought they were awesome I don't need to say any they said everything oh but Dwayne has more if I if I can if I can cheat one thing that Michael touched on I think is really critical is the business side of the equation I actually chatted with someone recently about some issues that I saw in

a project they were working on and asked why they were there because I know these folks knew better and they said we talk to our business folks and they wouldn't let us the business is the ultimate decision maker right it's their function they're paying for it I'm currently working with CGI and the governor of Alberta and governor of Alberta has an awesome risk management process put in place so the question was then was that risk formally accepted by the business and given the risk that's there is it isolated to just this product or do we have to escalate it because it hits more so again just wanted to note that Michael hit it the business can also be

an impediment to security because is the accountability managed can I add one time I'm sorry sure go so I who here's done threat modeling before threat modeling is a super great way to get the business on board when you explain I remember working with the Canadian government and explaining then this could happen to our citizens and all their faces changed and all of them are like that we can never allow that and I'm like so please let me fix this and then all of a sudden I had Authority and so if you can ex talk to them in their words which is a lot of what threat modeling is and show them potential results and I I don't mean use fear

uncertainty and doubt all the time I mean literally this could happen let me show you how it could happen um I find you can get really far so threat modeling for the win and I I like that too because to me that's the ultimate shift left if you can do threat modeling and do it early double thumbs that's how you start right um as a customer like you know kind of um um my question to you like you know kind of uh what are your main concerns uh like you know um because as a part of the government you like acquire many tools like you know many third party tools uh but as a like you know kind of a

customer what is your main concerns um when it comes to the software supply chain uh like you know kind of um can you share any experiences uh like vulnerabilities or any incident that caus like in oh um so I think with respect to what we have to hold the vendors to account for again if we've hired them for something we are not we we can't go through and externalize the accountability at the end of the day the business or technology someone in your organization even if it's contracted out still owns the risk yes right um there are certain mechanisms you can go through as a purchaser of services or products to ensure that the proper

processes are put in place so these would be things like the requirements for the contract what types of um behaviors or or programs do they have in place how have they been assessed right by a third party so that they're not going through and gaming their report not that they would do that um so things like sock reports um vulnerability reports if you can get them um and in some cases I've there's one vendor in particular I'm thinking of where things had been fixed and then two months later they pop back up and it's like Whiskey Tango Fox Trot um why um and then you have a conversation about that right about where you know we've contracted

you to do this we expect you to perform in a certain way if you're not being held or sorry if you're not performing to that level you have to hold them accountable the contract is the only way to go through and do that that you have legally Y and so you have to go through and ensure that you have pretty damn good contract management to make sure that those terms in the contract are upheld like really good procurement strategy Bic okay do you want to add any comment T I agree please government hold them accountable yes my tax dollars ask you one thing I'll add to that is it's extremely difficult uh especially if you're Contracting for agile or an in

agile way um because in theory you don't know what's going to be delivered but again to me you can still qual you can still contract for Quality believe it or not um things that one of the things that development industry has forgotten over about the last 30 or 40 years is there's actually predictive models around delivered defect densities those things aren't it's not magic you can actually predict how big a piece of software is reasonably You can predict deliver defects You can predict all this kind of stuff pop it in the contract and again allow room for negotiation but set your expectations that yeah this is what I think the world should look like when you're done I

accept the fact that you're not perfect you'll deliver some defects you'll deliver some security issues but after this it's on your nickel and I'm going to reject your project if you don't do that you're accepting whatever you get yes if it's just Co if if AI equals Cod and fix you're doing it wrong something you can add to your contract so I um I work at sura full-time but then I also give secure coding training because that's what my last company did and people still kept offering me money and I like it so I say yes um and over the past like year year and a half so before that I would always train the in-house software developers

and then they would record it and then this the contractors could watch it if they wanted to and it was this big divide if we don't train contractors blah blah blah so recently a bunch of companies have started putting it in their contracts if you want to write code for us you must take one secure coding whatever per year and so I'm starting to get hired by the Consultants themselves and they're all taking an unpaid day to come and train like one or two or three or whatever we're doing right and so you can add that to your thing if your devs haven't had any secure coding in the past year I'm sorry you're not coming like no security

training I don't want you writing stuff because this has been a point of conflict especially with governmental organizations where why like if I'm hiring someone they should be qualified to do the job so I'm not allowed giving them the training so you can force them to have the training in another way this is really interesting because one of the things that comes along with what Tanya speaking to is craft versus profession and that could be a very long and very beer discussion for all of us so we won't get into it but I think it's absolutely fair to have an expectation that people have been doing development that they have an understanding of how to actually write good code not just how

to code they have to understand what good is a gentleman I know has been calling himself a professional software developer for years and I keep getting surprised by the fact he doesn't know anything about software metrics he doesn't know what actually drives risk in software development to me that's shocking and he's a professional software developers he's been cutting code for 30 years yep yep yep um so this is the last question to all of you after that we will take questions from the audience um uh looking ahead what emerging uh Trends or Technologies do you guys see or believe that will have the most significant impact on securing the digital supply chain in the next few

years and how should organizations prepare for them like you know do you have any recommendations for that we can start with you Michael I didn't want to say AI but I I'll start that off so I think that's probably going to be one of the big things um and I know in in reviewing some of the opinions online about um like co-pilot GitHub co-pilot the question of you know what is it outputting is it of high quality um what are you accepting in terms of uh blindly accepting things and turning this off um because again software is developed by wetware on Hardware um so you know ensuring that you have the capability of understanding what good is and uh what you have to do

to meet the mark and one of the things I was going to say is um in the previous question is um incentivizing not just for the delivery date at the business level but also making sure that you're protecting your information and how that that software is being developed as well okay great now for me this be slightly different um the big deal that I see coming is more legislation regulation and litigation around software and software security I mean you can correlate that to the AI thing regulation of AI is already real but we're going to see the same thing with the software G crowd strike cost billions do you think there's going to be class action lawsuit out of that you

think there'll be legal repercussions absolutely for sure not an intentional security breach but effectively a massive denial of service and effect ly Microsoft unfortunately probably ends up with some liability for giving access to its kernel to third party providers that's going to hit the fan and those folks have Deep Pockets so this also goes back to where Michael was going accountability is going to land it really is the uh end user license agreement that says you bought the software but you don't own it oh and we're not making any declaration about its Fitness for use so if it actually happens to do something useful it's purely a coincidence to me that's got to go away

and open source is going to get hit by this as well at the end of the day most open source contributions today are by big companies Google IBM Microsoft are the biggest contributors I can see some liability from that even if the source is open I've seen some things like I won't call out GitHub but uh the advanced security but I just did the advanced security features that they put out is open source except the only thing you can do with it is extend it for their use how opens that they'll be change yeah no that's okay I I have three things and one is a thing that's already here which is autofix so when we started

security tools would be like you suck this is where you suck but now they're like there's a problem this is the risk of the problem which is why we think you should fix it and now they're actually doing like pull requests with fixes and I'm very excited about this and I remember in 2022 being excited about this and having the CEO of the place I work tell me that's impossible that will never happen and now I work at one of the many places where that is happening and so I I want all of you to demand autofix from whoever's reporting bugs to you yes and then two dreams so things I want to see one is I want to see

universities and colleges teaching software developers secure coding no it's not just for the cyber security Masters program to learn security if you if you are an electrician you think they don't teach you safety they're like oh if houses burned down that's you know the safety person's problem no computer science must teach security from now on starting today and the other dream that I have because that's a dream the other dream that I have is that companies that use open source will start contributing back and they could contribute fixes they could contribute a pentest they could contribute 5,000 bucks a year big companies you have 5,000 bucks right like I see companies that use zap my

friends at zap are like yeah this company runs 4,000 scans a week they haven't given us a dollar in 5 years like come on you have some money like you don't have to give them hundreds give them a thousand Buck give them something um so those are my dreams that I hope happen Michael yeah I was just going to add um to Dwayne's point about the regulation so we're starting to see this happen so bill c26 here in Canada the US has got an executive order that's probably a couple years before for that um and it's starting to go through and creep into the security realm if you have an instant you're required to report right and if you have an incident

there could be some monetary damages to that and you know as far as Bill c26 I won't get into detail but for some of those Industries they have long-term financial goals they have hard assets that are priced out to 30 years and there's not a lot of wiggle room for them to go through and include that stuff M so I'm hoping that this is going to be a bit of a um a wakeup call to try to do the right thing as early as possible to ensure that over the long term of that physical asset dams generators Etc that they don't have those types of things crop up and then all of a sudden the customer is

paying for it because at the end of the day the revenue is tied to this one thing like there is there's many regulation bodies like for like you know like you know there is a regulation body for oil and gas there is regulation body like for even like you know government like privacy um is there any like you know do you think bringing a regulation body for software security kind of makes sense not just like you know kind of uh um just like you know kind of like you know government is coming and passing a bill uh maybe someone going to like you know kind of follow it or not like you know maybe people like when I go to

incidents there's like we're not reporting it to like you know privacy because because uh we have these loopholes to like you know kind of um and I said like no no you have to kind of notify at least the customers and uh I'm not a lawyer but legal has to like you know so don't you think that something is missing from the regulatory part on the software Security in North America like you know kind of I don't know about the us but like Canada I don't I'm not aware of anything like that so wait this has been talked about again for 30 or 40 years about professionalizing the discipline of software development there's various kicks at that cat um to me I wouldn't do

it for software security solely embedded in software development like Tanya was saying it should be part of your professional job in theory um the i e has taken a look at this you got the association for computing Machinery here in Canada you've got Kips but nobody's backed it and some of that's been because professionalizing has also been looked at to some degree as a way to exclude practitioners which means people fight it like my background my degree is in Commerce been in it for 30 years okay I could be excluded yeah yeah uh and uh now I'll open floor to the questions like to the audience kind of if anyone has a questions I actually G

to Dive Right In uh since I have a microphone uh a lot of the problems I see is where someone will will Design uh have a bunch of specifications for a piece of software or sometimes Hardware saying oh it needs this it's like designing a car it's like okay it should look vaguely like a box okay four wheels you know oh you actually have to specify the four wheels in the corners okay you know things like that but we're seem to be forgetting a step in a lot of these design documents that the these uh software development places overseas and stuff like are building exactly the specification oh by the way you should specify that car should not explode

after 500 Miles putting a simple line of and oh by the way this should pass you know basic muster for security and and best practices would actually go a long way to to stopping this because if you're a a small little startup you don't know uh what to what questions to ask or or things in this design um if you just put by the way this should meet this you know standard I think that would go a long way to solving things because at least then you're you're offloading it and they might say oh well we can't actually do that because we don't know how because we're you know the cheapest bidder what are your thoughts on having like a a

boilerplate thing like that in contracts it should be part of the requirements document for every single new project and that's a thing I do and I give away one for free on my website um because I just kept telling people so I was like I'll rate it for you just copy and paste it and start just delete anything you don't like add whatever else you want but just like there must be a threat model done or there must be a a scan with this and you must fix these things Etc like the more specific you are the more you'll get what you want and I find a lot of Security Professionals expect the devs to just know and that's a

mistake expecting the car not to explode after five months we're expecting them to know that we don't want the car to explode and we're expecting them to know that they should do this or that there's no well they should know that doesn't work you don't get what you want as a person with little ones at home like you don't you need to tell them exactly or there's got toothpaste in their hair and stuff in their mouth and you're just like oh God you I'd say generically specify it and the other thing I like is the idea of templates especially templates for requirements but keeping in mind that those have to get adjusted because everything is contextual right I may not

care if the car explodes after 500 miles if I'm only going 490 so if I've got a purpose purpose specific system that's insecure and is only going to live for a week that's a business risk I might be willing to accept that risk that's okay and I'll put a little bit of a plug in um cuz my big boss is here um but one of the suggestions with respect to that um would be maybe go talk to cyber Alberta maybe suggest that as a community activity right various different vendors and the government will have different experiences um collaborate put out a communitybased um recommendation just just thinking through it Martin did you have a question and then there's questions at

the back get his I know big fan of you actually Tanya and those guys are okay but my question actually um you talked a little bit about that Michael in terms of context understanding the context rer you were going exactly where I was thinking I like shift left shift Le shift left a second time uh for me data you need to understand the data that you're managing before you even start coding um so I'm going to go with something a little bit more controversial before I go there Alisa is doing a really good Presentation tomorrow on threat modeling if you guys don't know so if you want to know go see her tomorrow morning yeah good plug

right but but what I was thinking is uh to be a little bit controversial being government we get hit sometimes with we need this app out within the next couple of weeks as opposed to having a normal cycle to do things do you believe in various levels of thoroughness from a security perspective as you're developing an application depending on maybe the dat you're dealing with it's sensitivity the fact that there might be privacy issues versus no privacy issues do you believe it's something like that where you can adjust so the effort is not necessarily as involved depending on the context and the data you're dealing with that would be part of the requirements thing so

when you're Gathering requirements it's like is this going to contain super sensitive data is this going to be a mission critical app is this going to be available to the public or just internal employees another thing you could do is do templating like when I worked at treasury board we created a template because we created services for all the other Federal departments and so we had a template that had all the security built into it so we had this huge Kickstart whenever we started anything and then I don't know if you know but you can call the other governments and they'll give you a copy of whatever you want if you ask nicely and so I feel

like maybe the government like all levels of the government Municipal is hard but definitely provincial and federal maybe they could work together more often when I worked at CRA they had they were very obsessed with risk um and yeah we would just raate everything like you know risk is medium this and that or whatever and then we had like this Matrix of okay so then we got to do this that should be fine we got to do more of that Etc and I feel like if you have this check box that's very clear like a checklist you check all the things and you're like oh it turns out I have ADHD I mean oh it

turns out I'm super sensitive whatever the thing is that you have to be right um then it makes it really easy and obvious of the things to do and then have those things further cycle down to now my requirements are these also you saying with requirements not just web apps apis serverless every single technology you're dealing with like from the beginning there's an API that's awesome here's your gateway here's this here's that here's all these things that I expect every time you use an API and so if we had that from the beginning the devs can work faster especially if you have code samples for them but add on yeah the one thing I would say is it's

great to have policy or policy instruments but if you don't have a compliance and enforcement um side of the relationship as they say as um uh Jeff said in the keynote it it's not going to get you much compliance and culture correct you need both right because if they're only doing it because you're going to whack them with a stick that's a lot harder If instead you can get 80% of the people to do it voluntarily because you've built a culture around that and the expectation is of course you would do that but you only have to hit the stick once in a while I see her vender he's the boss of us we should listen yeah yeah um so very

recently we had a situation where toward your mou crowd strike was um you know they rolled out a patch and it affected systems and so and so I asked this question to uh my EVP in the office that if we have a situation like this say for instance my Microsoft rolls out a patch and people are not able to work and I have different kinds of answers so question I would like to asks is what would you recommend as the business uh bcpd um program in this kind of incident or risks as it were and also I I beyond you know because I I listened to you know one of the speaker and he was

saying something about um um penalization in contracts giving you know you know giving U maybe putting it in the SLA that oh if there's a failure or something beyond that the immediate the immediate recovery to the business what would you recommend because I mean take for instance my organization we we are like 10,000 users and we are scattered across Canada we in Montreal we you know have users in Vancouver Toronto and these are laptop users you also have um infrastructure critical infrastructure servers too so how would you if that scenario plays out how what would you recommend you know for Recovery quick recovery of the users and services to come on board online if I'm going to start um so I'm a

software person not Network person so I don't do the people in the office with their host machines but generally if you have software a bunch of it like custom software a bunch of it should be Mission critical and that should be on your business continuity planning your D disaster recovery and you should have a plan to be able to roll that out when everything's screwed you should and I've worked at places where they can and like a bunch of things went down and they're like don't worry we just rolled out into the other cloud or we did this or we did that for the mission critical only um and for the rest of it it's like it's

not much and critical you got to wait right for networking stuff of like my colleagues needing to have access to they're that's not me that's not the work I do but I feel like you should be identifying your mission critical software and that includes software as a service as well right like maybe Office 365 is Mission critical for you I know I can't get a lot done if I don't have slack so finding um all of the types of software that you can't do business without and making that part of your plan and having a plan and also having a backup plan I've seen a lot of companies where they have a cicd and it's Jenkins

and it's local and they lose it and their lives are very bad for a very long time so if you are doing stuff completely locally and not cloud-based that's bad and if you are doing everything cloud-based think about having all those eggs in only one Amazon basket or only one GitHub basket that's this concerting too and having like a second backup that happens once a month somewhere else I think is a good idea and by once a month I mean maybe more often I'll just put offer one more thought about that one um again like where Tanya was going but obviously one thing that that hit with Crow very interesting incident that Tanya mentioned was we don't seem to consider

our operating system for our clients as critical software so maybe we need to be everyone patch tests but obviously not for that one but maybe one of the backups would be even something like having avds available that could be provisioned quickly with your critical software on it if the local boxes are down because in that particular case your local box was out and you couldn't roll out anything to it but if someone's got a home box that they can get into an avd an Azure virtual desktop or what have you for it it's a way to have business continuity we couldn't do a patch test that's why this was so catastrophic because of the type of you had to roll

it out everywhere or nothing that's why it was so bad sorry and and just one more thing so the the other thing too is if you have a Dr process or BCP ensure that you're testing it make sure that it actually works um you know we we do this annually for most apps um but understand what that is and then also look at the dependencies because I I I was on a a program lead architect for something that the government of Alberta did I had to argue that you know Skype at the time was a mission critical app how else are you going to communicate with your other team members and they didn't get it yeah

last hi everyone my question is on the shift left approach um from a development standpoint uh do the developers feel like it's an overhead on them like they don't need to do the security work whereas this is This falls from the on the security team and what is what are the steps that can be taken in order to um fill that Gap moving forward thank you that's a good question you can you can start um so yes um this is where I was saying like you can have a tool and if it's if it is ignored that dashboard looks awful pretty in red um the other thing too that could happen is if your developers don't understand

what the system is telling them the context why it's actually important that they fix this they're more likely to just go through and say addressed and there's no pull request associated with it so education is key and that's one of the things you talk about dreams one of my dreams is having a tool um hint hint hint that will go through and look at a vulnerability that's discovered and then also go through and not necessarily do an autofix it but maybe give context as to here are some samples given the context of something that you may want to consider to fix it Mozilla Observatory one of my favorite tools one of the reasons for that is if it

discovers something it has if you're running Apache HTTP consider this if you're running engine X consider this if you're running something else consider this right context it's not just something bad is happen big red critical maybe high it's actually giving them some steps as to what to consider and where to look because maybe one of the first things that that developer is thinking is where do I start I really like that question and I've seen a huge variance among the development staff some folks are absolutely spot-on Rock Solid regarding security others no clue and again that's the lack of professionalization of the discipline but one thing that's interesting there that I think could be a

compensation um people treated as damage if they haven't planned for it so if you're doing a waterfall project and it still happens there's no reason not to have your security activities planned in if you're doing an agile project make sure the security requirements are in the backlog if there's nothing in the backlog it ain't going to get done and one thing that this gentleman has done um and with that gentleman's support um is trying to make a role called security champion and make that effective so every product team has a security champion and that person is accountable not I won't say accountable for the security because that's again a business decision but accountable to make sure the right activities are

planned and executed and right and right means context specific so the business doesn't want to do it cool we get the decision recorded and he said you said what say awareness

so the current Incarnation is I have Dwayne on my team I have one other staff member Sean excellent guy there's three of us we have H well we have close to 200 product teams for just new builds I'm not up for being split I as much as I like being a whole self I I'm not being like even going through in quartering me it's not enough um so I wrote a book about this called Alis Bob learn application security and then I gave free lessons on it all throughout 2021 which are available on my YouTube for free so you could start there I also have a free course on how to build an application security program that helps you shift

your whole program left so if you already have one it helps you improve it and if you don't have one it helps you build from scratch and it's free available on srap Academy so that's just for starters I start with a training the training helps change the culture they start learning the things they need to know they start learning why there's a value they start talking about it then a security Champions program was a great way to continue you centralize as many things as you can and standardize in your software development process so you have a system development life cycle that's the same for everyone instead of everyone doing their own thing and then you start building security activities

in that happen every time that are supported by awesome humans like this and also update your tools so that they're not for software developers you know when you pet a cat backwards it doesn't technically hurt it but it wants to murder you right get tools that don't get tools that don't feel like that right so sometimes that means you have to throw things out and start again that's okay but if the devs hate it they won't use it and I have more thoughts and I we all would love to talk to you outside after um when you eat lunch thank you so much everyone

oh