Zero To AppSec Hero: Software Security From Scratch - Mark Goodwin

where the weird things grow I just want to say um given that my Steel Con finished with riding a hobby horse while being chased by uh Scott with a coconut making horse noises there's some competition so why am I talking twice yesterday was Mark the um enthusiastic Amer I was building robot despite no experience or professional qualification in doing that thing today is Mark as a reluctant professional I'm wearing a shirt and everything and I'm talking about application security and my notes aren't displaying for some reason even though I told it to do that never mind we'll wing it so I want to start out by telling two stories there's a reason for this the

first story is part of the story of how I got into information security I got into security by complete accident um I was looking after some people's web servers as part of uh some consultancy work I was doing one of them got broken into and um I ended up ringing a bank who were um the receivers of some of the malicious traffic that was going via a compromised machine and said did you know this was happening and they said no do on a job and um on hearing that they had jobs available in application security I spent a couple of months learning everything I could about it and it was a really difficult experience for me

because I'd worked as a software engineer for all of my professional life up to that point I had built a lot of systems that frankly I was quite proud of you know obviously they were brilliant right um and one by one I I lost respect for the things that I'd done as I realized that every one of them had a different sort of set of security vulnerabilities in them and I think all of us had a moment when we were learning about security where we got that sort of cold n knot in the pit of our stomach once a penny dropped and we realized that um not everything was the way that it seemed before now that was a long time ago I

know I don't look old enough um getting on for 20 odd years anyway so I ended up getting a job at this bank it was my first full-time job in infosec who remembers egg yeah yeah it was egg at the time though the world's largest internet bank and it was a fun challenge for lots of different reasons we used to get loads of fishing when fishing wasn't really a thing we had to build a lot of Technologies to to um provide counter measures for that sort of thing before the industry had really caught up so it was really really useful for me in lots of ways but I remember on my first day the first bit of work that I was given

there's a chat there called garant and I've been given the corporate issue laptop and I had no tools and he said um oh we've got this thing I'd like you to take a look at it and it was a Visual Basic application that talk to a database and I had no tools but I had word pad and I'm thinking hm what can I do so I had a look at the the Visual Basic application I thought I wonder if I can open that in wordpad and I could and I scrolled down and I saw what looked like ucs2 encoded SQL strings in the binary and I thought so I made some changes and and that's

how I ha A Bank using wordpad but um my point there is that first of all there's a realization there's that that cold knot in the pit of your stomach when you realize that the world isn't the place you thought it was and the second is that there's this sort of massive enthusiasm for a technical thing that gets you into ABC in the first place right absc is what people who would otherwise be pentesters would do if they want reasonable hours and not have to write reports all the time right anyway um as I said that was a long time ago and this talk really is about what I've learned since then because i' I've spent most of my time

since then working for three different companies one of which was that bank and the company it subsequently got bought by um I then spent most of 10 years working at Milla where I did um security engineering and security Assurance on Firefox um and I now work for metian where I look after upsc there and and um by chance all three of those places I joined um at or around the time that software security first started being a separate thing considered within that organization and it means that I've had a perspective on how that works in different environments I wanted to share that and the first lesson is this I excitedly on my first day attacked the

task that garant had given me um thinking that I was going to be a technical job but what I've learned since is that security is mostly cultural that's the first thing that I wanted to talk about so let's say you're entering a new organization they've got no absc program what's the first thing that you're going to do chickens people would be disappointed if they weren't chickens uh the first thing to do is to be curious okay this is important for a bunch of reasons it's important because you need to know how the company that you're joining ticks what makes the company work what what is the the thing that they do what's their product is the the software

which you're dealing with the security 4 actually one of the companies products or is it enabling something that they do what is the relationship between that that software and the company's business model because you know it's very very different for um a company like a bank where software enables the business than it is for a company where the product is is software um you need to work about work out how the the company is structured you know what are the different teams what are the different departments how do they work together and this is really important because it tells you a lot about the power dyamics within the company it tells you a lot about where

the levers are that you'll need to pull to get influence within that organization and also it tells you the parameters within which your day-to-day work is going to operate so for example software security is a very different game if you're just doing software security Assurance than if you also have engineering responsibilities where you need to build part of the security Machinery that the company uses so the first question is do you have a security engineering department uh or am I responsible for bits of that as well if I'm responsible for bits of that as well well am I part of engineering or on my separate from engineering and that's really useful because you need to know

whether you've got um actual control over or something or whether you're going to need to exercise soft power when you're doing things but there's another really really important reason why being curious is one of the first things you need to do and that is because it's all about the people right it's all about the people and if when you join an organization you sort of sit down behind your desk and you do the thing that you're doing and you spend no time going out and finding about finding out about other people you're never going to get anywhere um I learned something from someone that I work with recently um I work with a guy called

Aaron um if you're watching hi if you're not um don't worry uh he's a really nice guy but I noticed a thing which is that um whenever we go out for lunch or something when we're working together he'll always have a conversation with the person that serves him it's not just transactional you know handing over the money and perhaps saying thank you if he's bought something he always say how are you how's your day going you know there always a little bit of small talk and the effect it has on people is incredible you know what started out as a transactional interaction actually turns into something that's a lot more pleasant for everybody and here's the

really weird thing and neither us actually work in in the office we're fully remote but occasionally we'll catch up an altering room and we'll go out for lunch and the people that served him before remember him again and they'll say oh hello how's it going and immediately the relationship has improved and it's the same in the workplace right and so if you've gone out and you've taken the time to understand what people are doing and why and how the company works well do you know what you just put a face on your department you've told people who you are as well as finding out who they are and what they do and that's really really

important now remember the cold knot in the pit of the stomach story there's a learning moment there right and what you do when you join an organization and start their security Journey kind of mirrors that in a way because you're giving them lots of moments where maybe they're not feeling very comfortable and there's a bit of empathy here as well because as someone who thought they were a pretty good developer I had stuff that I could learn and the people that you're working with are in the same place as you were now I find it quite useful to think in terms of perspectives here because nobody who you're working with in application security wants to build

insecure software and so I think it's helpful to think about why it was that I needed that that realization the sort of uh horror of of having that realization now when you're using a computer system you've got a goal in mind um if you're a software engineer you're thinking about how the system you building is going to work perhaps you've got a ticket and you're trying to work out how you can organize the code restructure things to to make that the software do this new thing if you're working in quality assurance you're looking at the spec and trying to find out ways in which the code might not do that thing if you're a user you're

probably not actually thinking about the software at all it's just a means for doing something you know you're trying to log into your bank or um put a picture of a chicken on a slide or something and as security people we have a different perspective right and the one of the most important things we can do is teach people how to see the world from our point of view and my first question for you really is to think about how reasonable it is for us to do that if we're not going to take the time to do the reverse to understand what it is that they want to do so they go that's the first thing be

curious second one this is much shorter um find a Direction that's from Alice in wonder land you may recognize it um not every organization will have the same direction when it comes to software security um if you are a bank and the software serves a business purpose then your needs are going to be very different from if you're Milla and you want to ship Firefox or something um but also um depending on the type of organization and the security environment it lives in then your requirements are going to be very different so what is applicable for for company X is different for company y uh One S is does not fit all it's really really important to to try and figure

out what's appropriate sounds a silly thing but you need to work out what's appropriate for the organization you also need to find out what people at that organization think is appropriate for them it's actually really important to ask this question when you're interviewing for a role because if the answers for those things are different you're going to have a really bad time um how can you do this sort of thing well there's things like cmmi where you can figure out um what a company's uh um appeti is for for various things um what level of maturity they want you can map that to various things things like the the various maturity models for for software

assurance and that sort of stuff and you can you can get an idea of where it is you want to go but finding a direction is a really really important thing you've got to be able to tell people where it is that you're going and to do that you've got to uh determine that in the first basement and I had a weird moment a few weeks ago where I was looking at something on YouTube and A video popped up about someone I knew in it and uh I've never looked for them on YouTube or anything like that but I watched the talk and it was a talk on leadership a chat called Peter anderton uh he did this talk about

leadership where he's got um two rules and they're really simple the first is it's not about you and the second is it's only about you um that's really clear right um it's not about you um is around the fact that when you join an organization and you want to affect change because that's what leaders do it's really really important to understand that you can't do that thing on your own yeah um very often an organization will succeed um in its own way regardless of whether or not you're doing a thing your job is to help them do that quicker and um sometimes when you're doing Security in an organization other people do your best work right I don't know if that makes

any sense sometimes other people do your best work um so so going back to this idea of finding out about an organization and figuring out what makes it tick finding out who people who who the people are that might be allies in a particular area is really really useful looking for aligned incentives ways to make things happen that aren't necessarily labeled security when they're being driven really really useful give you an example um all of the the big browser vendors these days are hastily um finding ways of supporting safe languages in their um development efforts so um chromium are looking at including rust within their code base and Milla have been doing so in Firefox

for several years now um that happened in Firefox because of a project called Stylo where they took the the style rendering engine from the servo browser and Port it across to Firefox in I think Firefox 57 um and it's led to massive security improvements in the browser because loads of the memory safety issues temporal and um uh spatial are fixed by having a a language that's guaranteed safe at compile time right um but this wasn't a security project the the work that was done in in Stylo for Firefox 57 was actually a performance thing they wanted a way of having concurrent rendering of a stylesheet and to do that they needed a way of allowing concurrency and to do

that the easiest way of doing it was to use a language that supported that stuff better and so the biggest Improvement in browser Security in probably a decade actually came from a performance thing which I think is quite interesting um so other people will do your best work for you it's not about you and it's only about you is about a different thing it's only about you is from the observation that leadership is almost entirely in how you show up yeah if you're the person that nobody wants to talk to you're never going to get anything done and so that's a really really important thing to learn um it's only about you and you know what it's um only

about you applies not only to you it also applies to the people in your function as well because everyone in a function like appsc is a technical leader in their own right yeah if they're talking to developers well they are mentoring other people in technical issues which they are experts in and other people aren't and so a lot of the leadership tips you'll hear are important for people even if they're not people leaders within your organization so I think that's a useful thing as well so yay that random Youtube finding and thank you Peter anderton so you're going to need to measure some stuff we all like metrics right um first observation on measuring stuff your problem is almost always

going to be that you have too much information not that you don't have enough um and sometimes that the the noise to Signal ratio is a bit too high um I think it's worth thinking about metrics in two ways firstly internal stuff you need to measure what you need need to drive a feedback loop um it's all very well known where you want to go um you need to figure out whether or not you've got there yet and measuring is a key part of that so you've got this feedback loop driven by the stuff you collect um but then there's the stuff that you feed to other parts of the business and again it comes back to

security being cultural right you're telling a story and what you measure can help tell a story to other people in the business so think about not only what you measure but also how you can use that to help other people make the right choices know your tools so I was talking to uh Andrew last night and I said I'm tempted to throw my talk away and just spend 30 minutes ranting about how bad abset tools are um and that might be a talk from another time so if you'd like to hear that talk if you want to hear me rant for you know half an hour or more on why ABC tools are all terrible um you know

give me a shout I'll do that sometime um and you know there the usual stuff you might know about you know might know about software composition analysis tooling um that will give you comprehensive and accurate information but not all of it's pertinent um the way I like to think about this um let's say you're using a platform like Java um Java has really dangerous things in it like um system exac right you know that's there it's always there the question for you as an absc engineer is does that ever get hit and so when an SCA tool tells you that you've got such and such a dependency and it has this vulnerability actually that's no more

informative than there is dangerous stuff in the API you're using sure it's better to upgrade to the non vulnerable ver version of that thing but unless you know you're exposed to the vulnerability the finding itself doesn't actually tell you much um this again is a a case of you've got too much information and the the noise to Signal ratio is sort of probably quite High um you've got tools like Dynamic application security tooling um generally the quality of this tooling isn't brilliant um the open source stuff seems to be best um I know Simon Bennett but that's not necessarily a plug plug for for his stuff um how well this tooling Works usually depends very much on how your

application is built and the weirder your application is the harder time you're going to have getting good results from it then you've got things like static analysis tooling that sort of stuff um and you'll get a lot of false posits from that sort of thing um and again it's an issue with the amount of data and and how much noise you've got there you shouldn't just think about Tooling in terms of the stuff that you can buy there's also the developer tooling there's also the question of um what platforms do people choose to use what toolkits um what utilities have you got internally that stop people from making mistakes and my main bit of advice on

tooling is that um systemic is always way way more effective than superficial if you can find ways of making a change to an application or an application architecture such that it becomes really really inconvenient for people to do the wrong thing that's going to help you a lot more than bolting on some kind of tool to uh retrospectively look for a hole um so think about that sort of stuff as well um I would like to rant a lot more about tools but I'm not going to verify um How do you verify your efforts within an organization um this internal stuff right you can have security things as part of your um internal testing activity so there can be automations

there can be um security specific stuff within your um unit and integration test automation definitely a good idea you can have internal manual testing as well if you've got um quality insurance Engineers looking at stuff you can give them a list of things that you can look for you know I've got this theory that um injection flaws will never get past a QA team that knows what to look for right you don't need to be a security expert to find a lot of this stuff and and there can also be internal security specific testing from your um absec folks as well so that's useful and then there's the external stuff right um pentests um really really blunt tool for

actually verifying how well you've done your job um they have their place mostly in sort of sales enablement um or things like bug prags as well um they're a good thing but it's good to know um what options you've got there for verifying your efforts and working out how effective those various things are um I seem to sort of run out of time and that's kind of all I wanted to talk about anyway but to to finish off I just want to sort of um spend a few more moments thinking about where I started which is that um there's a lot of people stuff involved in doing application security work you know it's a technical

discipline obviously but also it isn't your success is more likely to be down to how well you do the non-technical things um than it is um how well you do the technical things um and uh empathy and um leadership skills are an important part of that um yeah so that's all I had to say thank you for your attention I love the fact you've got the bread board today when you didn't have it yesterday yeah I I did actually have a breadboard yesterday I just didn't use it fair enough any questions for Mark please start down that end of the room just like watching you run that's dist I want to hear the other talk as well yeah yeah you're bypassing

cfp for next year we're having that questions oh hi Mark um in terms of a career in absc do you think a development background is essential desirable is ABC um an entry-level job in cyber security or do you think people need to have a certain technical background before doing it um I don't think um experience as a developer is essential I think that it's much easier to go from a role in development to appsec than it is to get into any security field any other way so um current developers are a really really good talent pool for potential upsc people um I think it's useful to understand software development how it works I think it's useful to understand um the

basics of the technologies that an organization you're working with uses um I don't think you need to be a particularly brilliant programmer um interesting aside on on this actually I I discovered when I went to Milla and spent some time going back to actually building stuff there that my experience of working in security made me a much different programmer and not necessarily a better one and I think what I learned from that is that you know maybe it's just the way my mind worked but I was constantly agonizing about security stuff when I was trying to work on a ticket and it made me a lot slower and I think that perhaps tells us a little bit about how appc second Dev

can work together I think if you you know Foster this environment where you're um approachable and helpful then you can free up developers to do their thing well and quickly um and know that they'll come and talk to you and there's something that looks interesting if that helps at all any more questions for

Mark go wife come on my this is a fun game I'm sorry I'm so sorry well a little bit sorry we were having fun watching you run back and forth um so what is the best way that you have retained your sanity uh through your journey there's an implication there he has yeah I was going to say did I well he didn't get dress today by himself you presume I did get address myself today to clarify um uh it's kind of fun I've got a theory about absc if you if you draw a chart where um on one access on one axis you've got um the amount of hassle you have to endure on the other you've got the the

rewards the curve isn't flat one um and and so it means that you know there a there a a Maxima you can find and my personal opinion is that um having done Dev stuff having done security stuff um probably what I'm doing at the moment is the optimum for me at the moment so I do actually genuinely really enjoy it um I I find surprising things draining I spend a lot of time in meetings um meetings are a lot harder um for me now than they used to be because I'm spending so much time on Zoom I'm so disconnecting doing a lot of stuff that doesn't involve computers chickens and chickens are great um I like to work a lot with my

hands you know mending stuff making stuff um yeah hobbies and things like that so my anote to the stresses of work is lots of not work that's an excellent point yes I approve of that message 100% yeah but your hobby is drinking still not work maybe it is work sometimes any more questions for more oh that's all right you stay there mate save your legs if you find an issue and you pass it back to the devs to look at and fix what do they appreciate that or are they annoyed that you found stuff and just want to ship it out as fast as I can um again I think it is entirely defined by

how you build a relationship with that Dev I think if the only interaction you have with them is to give them bad news first of all they're not going to be delighted to talk to you in the first place you know you know granted there's probably still of a a still bit of a pause for breath when I say hi to someone um but you know if most of the conversations I have with the dev is on how to do something rather than what's wrong then the what's wrong conversation is going to be a very different one um I've never found a Dev that isn't interested in fixing a problem sometimes they won't understand but I think folks

generally aren't resistant if they if they understand what's going on one more question how we for time actually would they be there last one talk to us about Supply chains because obviously the thing is that you've got some control over your staff and your code base and all that the majority of organizations have zero control over the stuff that they use so you're you're in a very privileged position to be able to talk to the dev how do we talk to three devs away or however many to get that same message across so that the stuff we use that's old and out of dat or needs to be repaired in one way or another can

actually get looked at fixed than sorted um excellent question actually um it it's not easy um yes you can influence the stuff that your organization maintains itself um but no one is only using their own stuff there's always libraries even if it's the platform standard library right and therefore there will always be Upstream problems um I think there are two interesting things here one is the the question around open source I'm a massive fan of open Source I think most of the problems that we have in using it are down to the fact that we want to be able to use it and not help with the cost of maintaining it um I think that there are efforts

underway to improve that situation but it's going to take a while right um so so I think that um a good starting thought on that one is to um factor in the cost of ensuring that you have enough maintenance of that thing when you make the decision to use it and it was more common at Milla than it was at other places that work in but you know it's not um entirely unexpected for for open source projects to get sort of random commits from contributors for security reasons right so so that's one interesting thing another one is that um the nature of the way that we aggregate stuff on the web really changes software supply chain you know when you load uh

a website um you've also got a whole load of third party analytics and services and things that you're using and I think a lot of people haven't fully internalized the fact that anything that you're allowing to um execute um in your page has the the full privileges the user has right um and so that's an interesting thing that's harder because very often security isn't front of of mind for any of the people producing those tools um and there's obviously a lot less transparency around the way they're developed or even what's in there than there is for a lot of the open source stuff does that answer your question is is there another aspect I miss that's a question I think for the

break because we're going to get into it's tea and coffee time St yep and I I love your questions I just think the two of you should sit down and stare at each other out a cup of yorra tea nice not NES Cafe why did Ben buy NES Cafe I don't know coffee's dummy okay everybody I'll round of applause for Mark