BSidesSF 2024 - Effective security on a tight budget (Felix Matenaar)

well thank you so much for having me here today really appreciate it yeah I'm going to be talking about effective security on a tight budget I'm Felix masar the head of product security Asana there's a little QR code here or actually a huge QR code on the slides for uh my LinkedIn if you miss to pull your phone out uh on the last slide it will be there again okay before we start uh quick disclaimer uh the views and opinions expressed in this presentation are those of mine and they do not necessarily reflect the official policy or position of my current or former or future employers all right let's start with a problem statement here um you know if

you've been around security conferences or you know ceso dinners or general expert conversations like there's all this um talk about perceived tight budget in the industry like we almost like never have enough budget to get things done there's so many things we need to get done and it's just always been been difficult for us in the industry uh more recently unless you have been living under a rock you probably seen there's a tech downturn and it amplifies this problem throughout the last two years um if that's not already enough we're also seeing regulation pushing for more security so recent examples are like privacy regulations now there's you know AI coming has some security implications but also you know if you heard about the

solar winds uh story for example there's more push towards you know transparency and one can argue it helps us as security practitioners to be more empowered to do security but it also adds to the liability and the pressure you know so put all that together and we you know kind of having a resource problem and I want to talk about that today and the major questions I want to go after today is number one um and this is a little little bit spicy but I want to say some of this I believe is self-inflected pain and I'm going to talk about that part which is how to assess security organizations's Effectiveness and prioritization the other part however is like how do we

effectively influence our funding and get more budget or like make more out of the budget that we have that's going to be the second part so number one is talking about how we can rearrange our resources potentially to be more effective at our work and number two is how can we maybe offload more of the work that we're doing today to our organizations or how can we get organizations that help us get more funding for our security organization and so for the first question the uh mental model that I want to present here is what I call the Pyramid of security needs and it's you know very similar to the massof model of human needs um basically if you you know

probably most of you know the mof model right it's like human needs it's like on the basis you have like um health and shelter and all the food and all these needs and then go all the way up to like self-actualization and one of the core ideas of the model is you cannot achieve self-actualization at the top unless you have achieved everything below and I think there's a similarity here to security is that if you if you don't achieve some basic security and security coverage like even if you mature your more uh if you try to mature further into more uh you know specialized programs they actually don't give you a lot of value and we'll we'll be talking

about that so I'm going to be presenting an abstract security maturity model here that you can use to assess where is your current security program at and where does it need to be and based on that you can make those resourcing decisions at the bottom um there's the the idea of the the first level of maturity is adhere to applicable law compliance in regulation um the objective here is like basically don't get suited right or um based on Law and regulation it's a pretty trivial case for uh your budget isn't enough right if it's not funded so we're not going to talk about that a lot but obviously those are must have areas to stay on top of and cover um obviously

they change basic on the type of business and you're operating in customer location um once you achieve that you can move on to the next level that's where you establish security fundamentals and hygiene uh the objective here is I call don't get hacked in a stupid way um and it still sounds like very very basic but if you think about most of the breaches that happen in the industry today most of the stories that we hear we can root cause them to some sort of inconsistency or lack of coverage on security fundamental right whether it's a um you know like account that didn't have tofa or whether it was an unpatched system or an employee that wasn't trained properly

you know all of a lot of these like simple fundamentals and hygien that can get us hacked so the objective Fe don't get hacked in a stupid way and yeah it sounds easy but I believe that in order to achieve this you truly need to be consistent and investing in your coverage right so like it's not enough just to say we have a program that patches your system the question is are you patching all your systems and only if you do that have the coverage across all the organization need to support you can achieve this type of um maturity once you achieve that level of maturity you can start to deeply understand your security posture so the

objective here is you beyond your initial hygiene and fundamentals which is mostly like a checkbox style you know kind of program building up now you actually understand your individual weeks spots and what you can defend against with the controls that you have today you know some examples here is like threat modeling having a risk register you know maybe doing fuzzing back Bounty all of this gives you like some good data the question is like how to approach getting to that level of maturity um I think it's very important here to start thinking about declaring what you are and aren't protecting against so like for establishing security fundamentals and hygiene I personally believe this is sort of a

no-brainer like almost every serious organization like should try to achieve that level of maturity here it starts becoming a little more of a question of like what your individual needs are and I would uh encourage you to for your own organization to think about what do you really need and how far do you need to go to to protect whatever you're protecting um and also like if you if you go after this go bread first search I'm seeing a lot of Security Programs that start threat modeling for example and they go down a rabbit hole and they miss significant alternative opportunities now if you actually go all the way down and you you achieve this

level of maturity you're going to um start working on establishing security resilience so here the previous maturity level is really important because how are you going to identify those critical opportunities if you don't have a good mental model of what your weak spots are right and so the objective here is to keep motivated the attackers at Bay um so those examples are zero trust assum breach Security in depth Etc how can you approach this I have the belief that the first question here really is like can do you have the funding in your organization to truly achieve this and are you realistic about your assessment I believe in a lot of cases you can argue this may be covered by Insurance

actually and I know as security pract practitioners this may not be a very popular belief but if the tradeoff is that you are going to uh work on security resilience without having proper coverage on your hygiene and fundamentals one can argue that the ROI of your program is at this point almost zero because what can get you hacked is that lack of coverage with that you're potentially not investing in enough right so it's always a question of um you know opportunity cost investing like do you have the means to invest into security resilience without actually sacrificing on those lower maturity levels few organizations will ever get there but you know if you do it's very

exciting um testing novel attacks against your systems kind of the Pinnacle of what you can achieve in your security organization the objective here is to keep sophisticated attackers at Bay so there's obviously a strong um dependency on security resilience otherwise sophisticated attackers are going to easily breach you um but examples here you know zero days assumption in third party which leads you to you know stronger isolation and compartmentalization needs um maybe you want to assume some sort of broken crypto in your in your most critical uh systems and see how you mitigate that you may assume like chain of exploits and techniques are going to be run against you if you actually go for this

again ask yourself the question do you need funding I would say for most organizations you the answer is you probably don't need this level of security again can you cover this by insurance I think is a very important question that every security practitioner should ask themselves if you do try to execute you require unique talent you may want to contract out again very difficult to actually achieve let's go through an example of like what this actually means in practice right and so like I I work in product security so appsc was kind of the closest example here so what I'm showing here is like different programs that are actually categorized by the level of maturity

that they intend to achieve um and you know I'm not arguing like you need to like go these go through these one by one but before you invest in a program what I'm arguing is like ask yourself the question whether you truly achieved the maturity or you're go about to achieve the maturity you need on those programs on a lower maturity level um right so like for example you know how much is the best buk bony program good if you don't have effective vulnerability management and you actually can't get those vulnerabilities remediated you can still argue there's importance here and the visibility but you need those foundational programs in order to be to be effective all right so summary so far

here in terms of the resourcing um we went through the permit of security needs so there's this idea that you have multiple layers of maturity that you can achieve and it's really important for you as a security practitioner to ask yourself how far you want to go and how far your funding reaches and be a realistic make a realistic assessment about that um especially problematic is like when the ground is Shifting like um Merchant acquisition for example maybe you had really good patching 3 years ago but there's been two new acquisition and now you're not covering that area anymore your funding should consistently like cover for these cases and I would recommend you adjust your funding if you

have events like this um assume you're on blind spots and yeah Beyond hygiene and fundamentals my belief is that security need truly is a decision it's not an automatic yes of course we need to go beyond because if you don't get the funding would you effectively doing is you're trading you're trading the excitement for a program that is um that you're actually not funded for and you're trading against potential fundamentals and hygiene that ultimately will get you hacked um also I want to say don't forget to have fun um I mean as SEC practitioners we want to learn we want to experiment want to build exploit sometimes but we need to remain realistic about what makes an impact and

what is play and often times play does lead to learning and learning does lead to more effective programs but we just need to be aware in like what's buckets we're spending our time all right so this is the first part I was talking about opportunities for security practitioners to think about how to rearrange their resource in investments in order to be more effective at the level of security that they decide to go at now we're going to be talking a little bit about how to fund programs smarter and I'm going to provide four examples here one is sort of like a a cultural thing that I like to push for and it's definitely not easy to execute but if

you can get there it's actually a huge um a huge time saer and uh um enables focus of your security organization so the core idea here is basically whoever owns maintains builds operates or otherwise is responsible for a system is implicitly responsible for all security unless the security organization provides explicit support so the opposite of this is like you know if you hear the CEO saying hey we have our ceso because that's great because our ceso is going to make sure we are like secure right that's like all the responsibilities on the CSO and their organization and as a result you know whatever that scope is which you know is really difficult to assess is on the

existing funding and um with that I think that that's a very problematic stance and if you if you're able to turn this around in your organizational culture you're able to build a charter for your security organization and say this is exactly what we're protecting and nobody else should assume that we're taking responsibility for something that's outside of that Charter and you can you know emphasize that message and interestingly what what tends to happen then over time if you're successful at building this this culturist the teams will come to you to ask whether you can take more responsibility and that's the decision point where you can ask for funding and say okay I'm going to I can

take responsibility for those systems I can make patching easier for you but I need funding for this I'm not going to just go straight into my management chain I'm going to talk to my partner and say you know if I can give you this but you need to you need to at least Advocate or maybe even move over funding to my organization right like some examples here include fixing vulnerabilities uh you know patching third party software um you know incidents uh if you if you have a good incident response plan you may be able to you know shift some of the work to to the developers um all right the second one is um and

vendors are going to love me for this um I call this buil if and only if you can buy um there's a you know a lot of build versus buy conversation um my stance here is basically I've observed security organization build way too much stuff themselves uh and there's a lot I don't believe there's a lot of opportunity here um was it a vendor um security yeah I mean it's kind of inherent right like security Engineers when you know lot we creative people we want to invent we want to build new stuff it's very understandable but it just is a conflict with our organizational goals in many in many cases right like even even me as a

manager like I want to hire right so yeah let's build stuff let's hire more people great that's an incentive for me but it is is is just it's wrong in many ways because it creates issues right and once you went down the path um you have this sun cost FY right so um you know I'm sure many of you know these cases where organizations have built Big systems and then then later on they find out there's a vendor that can do this you know and then try to you know try try to try to get the vendor in it's often very difficult because there's a lot of you know kind of personal personal opinions about this so how to

avoid this right like there's different things you can do to facilitate the culture of build only if if and only if you can buy um ask Finance to make headcount and vendor budget fungible I think is the the most effective thing you can do here to enable managers to think about um how to you know do I want to hire or do I want the budget to go with a vendor like have that flexibility because often times that's only said at annual planning time and then later down the road you actually discover a really helpful vendor but then maybe your buckets onone the line um the other aspect here is like individ set individual ex uh examples and incentives

like vendor evaluation and integration should be the standard path to become a senior security engineer or even a staff security engineer depending on what is the scope of the problem that you're solving right and I think that's the other important aspect here that translates into incentives for managers is that man managers in a security organization should be promoted and compensated by the impact and the scope that they're having not by their size of the organization um I think that's another important reason because managers are often the decision makers here in these cases right and if they don't have the right incentives um they're uh you know they may be making decisions that are going against the

interest of the the larger organization all right another one that I see very often is sort of like U part of the discussion around security and velocity um a lot of the security Investments once they have matured to a certain um degree like let's take static analysis for example there's this notion like okay let's start shifting left right like you you got your basic rule set setups you have processes to evolve these rules now why wouldn't you expose findings to your developers earlier and I think we all agree like it's a great idea it makes things smoother and also it contributes to security like for example if you know if you find issues earlier I mean most of the argument is

cost but like you can argue these issues get fixed more likely and if they get fixed more likely they're less likely to hit production so there is a security argument here but there is a lot of velocity value a lot of cost saving here and the cost saving is actually not on the security side right so in these cases where you're shifting left what you're really doing is you're investing a lot in your security organization to provide these services to the developers that you supporting and they benefit from the velocity like you just at at the beginning you have more work yeah you're doing the the right work you're doing the right thing but you have more

work overall and that is work that can potentially invested in your security um so now what I what I recommend in this in these cases talk to the developers and use this as a check-in point to ask for funding like make it explicit before you start larger shift left initiatives and explain to them what is the value uh you know maybe you have enough data about vulnerabilities and how much time developers spend that you can actually put a number on this and ask them to contribute funding maybe they they can fund 50% of the initiative but you know sometimes as as as good as we want to be as SEC security practitioners and provide a great service to our

organizations I believe that often times we just fall into the fallacy of doing this work under the hood and trading it for potential like Direct Security impact work and you know never like losing the opportunity to have this conversation and get more funding with the develop ERS that we

support last but not least um this is more like a general idea of like how to uh fund your organization um is the the idea is like scale your security automatically so we're not in that economic mode right now but generally like we've seen a lot of aggressive hiring and even sometimes today if you're support if you're soft in a software company for example chances are they're still building new organizations let's say you know in thei space is like very popular today so sometimes you do have aggressive growth in like certain areas and the question is like how do you support that um what I'm um what I'm recommending is like have a conversation with your um

management chain and get to a certain like key ratio like for example for midsize and large companies what I've seen go really well is like 2% security to engine ering for example if we're talking about the the product security or application security space um for smaller companies my recommendation is like at least one engineer per program and then multiple Engineers sort of like they can share share programs so you have coverage for out of office and all this stuff but later on you know as you grow does like 2% rule usually I've seen it work out really well um a really like counter example to this something that I've seen as an anti- pattern is um

situations in which you individually go to your management chain for funding and this often results in a um in a situation where you only get net new work funded um and so like for example like if you only get net nework funded then the problem is first of all you're providing incentives for you know shiny new projects to be proposed Without Really investing into the scale of your security fundamentals and hygiene um but also like as your organization grows so so grow the number of programs based on the funding and then you actually end up wasting a lot of resources um in the case where these each individual programs can't give you the coverage you

need and so you may end up getting hacked in a stupid way which is what I believe happens a lot in the industry and it's the reason why um you know why we haven't seen a lot of a lot of progress um overall with protecting organizations well those really EX examples that I had um I would like to move forward here with questions if there are any um we've today we've talked about how to build effective security program on a tight budget we talked about security maturity models um that we can use to be smarter about allocating our resources and we talked about specific strategies on how to obtain more funding and have these explicit checking points thank you so

much [Applause] that was amazing we have some questions for you sir uh the first question on the topic of build versus bu what's your experience with open source versus paid tools with implicit requirement for more Ops effort or OSS I mean personally like I mean the the advantage in open source right like often times is like basically it's you know comes for free uh if you don't have a company supporting it right if you do have a company supporting it then you probably it doesn't count as open source for that for that matter um I personally believe there is a lot of value in many cases on like getting the explicit support and having uh some sort of an

Aur path of product and service development going forward I I can see cases where that may be less valuable like let's say you're you're trying to buy a specific Security Control in you know you're trying to get to security resilience and you're just looking at that technology um open source like can save you some some money here for sure but I would first look at like you know am I effectively solving the problem and that's solving the problem require getting a service right what's the value of the service that I'm getting versus the money that I'm spending um so I would not generally you know say one is better than the other I would say it's

really like depends on uh doing a a proper problem evaluation which you know is is one of the reasons why I believe this should be part of becoming a senior security engineer or even staff security engineer because those are some of the important questions that I expect um security practitioners to ask themselves when making these decisions very well said okay how do you assess a migration from a custom build solution due to lack of a product to a vendor provided solution can you repeat the first

part oh I see yeah um so I would I would honestly treat it as a vendor evaluation like you know whatever you have today like what is the service what is the technology like you can look at this similar way as you look at the new vendor uh right and like important is to not um to not go with the sun cost FY right like a lot of times part of the equation will be oh we've already invested so much time into this and this is the pure definition of sun cost FY so my my recommendation is like make the Assumption you do not have anything today and what you know option A is you

you you're building out or you're you know you're using what you already have today as a vendor and you basically say you know you look at what is the support for this custom build solution today what does it cost what does it look like and how does it evolve in the future um and you you kind of compartmentalize that as vendor one and then you look at you know vendor B CD whatever you want to do how many how many vendors you want to evaluate you know I I often go by the rule of three but like um yeah I think basically try to abstract the problem of already having a solution as if if if as

if this is a vendor you're going with and then if that vendor wins you're going to stay stick with the custom solution if the vendor Lo losers you're going to get rid of your custom solution what kind of roles do you compare in the 2% role what kind of roles I'm comparing so yeah that was like mostly for like security um security uh supporting engineering um I've Al I've seen this like apply to larger security organizations as well like I think if i' I've seen the two like 2% of security practitioners in an organization they support like for example you know if you're looking at the entire uh security organization then you're looking at the

entire rest of the company if that's what you're supporting if you're looking at product security for example you're looking at you know like the product folks that they're supporting um I've seen the 2% rule relatively you know like work in almost like any scope I can you know I've also seen this work uh relatively well for corporate security for example you know usually what you do is like you look at it of obvious ly there are exceptions right like sometimes what I mean what what what matters here is like how the how the responsibilities are distributed right like for example if you have a very small it um it team then the 2% rule may

actually not match up so you may want to look at like who truly has these it responsibilities Beyond it highly likely that you have such a case if it's a very small it team um but yeah like basically look at like where where do the responsibilities lie and and and what are what are your team supporting and then and then look at look at if the 2% rule matches up to that final question what criteria would a company need to meet to consider Staffing an internal red team instead of using

Consultants no it's a really good question there's a lot of really good questions here yeah yeah yeah I love it I love it um I mean my personal experience is like buak Bounty programs do a pretty good job finding a lot of the you know like lwh hanging fruit product vulnerabilities um I personally believe that funding a red team um you really do that if if bu Bounty programs you know even like assuming you get really good engagement on The Bu Bounty program of course right because otherwise you can drive maturity there once you have done that and you you believe you need to test out your security resilience like that's what I would that that's what I

would expect from an internal red team essentially right it's like if they're only giving me as good of an output as a back Bounty program then I would not like the investment doesn't make sense um because because back Bounty programs like if you manage them well they can you know you can build community and you you can you can get very very good engagement so my my my bar for building a red team here is basically a lot higher like what I would want to see is like uh probably going into the area of like testing testing novel attacks um and if if there's a need for that in my security model like if I if I make an

explicit decision that the security maturity should should go into this area and I want to test my security resilience like that's when I would start looking at a uh a security red team and you know before that um you know probably consider Contracting it out if there's if there's individual needs um you know I mean sometimes you even need it for compliance for example right but those are those are different pentests like I consider pentest here as like truly you know tell me tell me something that I truly didn't know about the security of my organization Mr matar that was truly insightful everybody give him a a round of applause he deserved that thank you

so much for coming to bides and presenting for us that's a little gift for you from one of our sponsors um actually socket security is the one who is giving all the gifts this year so we greatly appreciate them and we couldn't do this without all of you our volunteers and of course our sponsors um again there was some questions that we did were not able to get to Mr matar is going to be up in City View to answer any additional questions that you may have so uh feel free to reach out to him there and also link get linked up with him and connect um we have a couple quick announcements we have head shots

all day today right outside of the talk tracks go get a new head shop for your LinkedIn um and also did you know we have a prayer and mother's room please stop by the info desk and we can help you find that if you would like so thank you all for coming to bides 2024 San Francisco and we look forward to seeing you all next year