Hardik Parekh - Navigating DevOps Security Journey with OWASP SAMM v2.0

hello everyone this is Hardy Couric first a quick check can you see my screen and can you hear me out we're all good okay all right so good after everyone and thanks for having me here I'm really excited to be here presenting on this topic which is near and dear to my heart which is navigating their web security Ernie with OS per Sam software assurance maturity model and one of the core members of Austin project first of all I want to thank besides Knoxville for scheduling this atlas for me this is the first online event that I've been attending I'm sure the same thing for many of you hopefully I'll get what you come to Knoxville next

year before we dive deep into the topic I want to share my professional journey line so just let's spend two minutes on Who am I so I signed my career as a software engineer and I got introduced to security when I was doing my master's in complete science at unity of Florida my very first professional exposure to security was when I was Adele and I was asked to kind of build a hickory and pen testing team from there I got an opportunity to bootstrap EMC spark security program after that I led various product information security teams at companies like Intuit Amazon and Splunk I'm also on an advisory board for a few security and data privacy

startups as well as the nonprofit trade organization called CompTIA which many of you might be familiar with pompeii shoes IT security certifications such as security plus and size Atlas but throughout my career I also contributed to industry through safe code publications science top 25 programming errors CBS s 3.0 and besom since version 1.1 1.0 so for those of you who might not be familiar with besom besom is another so physically maturity model which is descriptive in nature it was developed originally by acetyl by interviewing six independent software vendors from safe code founding member companies and EMP was one of them that is how I contributed to be sim a besom is more of a descriptive model versus

prescriptive model like Sam and by that what I mean is this him helps you compare with other secure organizations and which organization is performing which security activity at which level that is what it helps to describe I started contributing to OS Sam some around 2016 and have been one of the core members of La Stampa logic since then you so before we start the talk just wanted to get some legal stuff out of the way I'm not speaking on behalf of my current or previous employers nor am I here as a representative of my current or previous employers my opinions are solely my own and do not reflect those of my current or previous employers

you so quick agenda was Sam I'm sure like some of you might have our used or asked them in the past some of you might be very new to watch them so I wanted to spend some time for those of the folks who are new to us Sam giving them introduction on who is awesome what is awesome why do we need a model like was them an overall project history and background Sam version 2 has been launched recently in about February this year so also wanted to quickly go over the highlights of what changes we made in version 2 or version 1.5 and last we will discuss how to apply Sam in your particular organization my goal is to make sure

that you walk away with enough knowledge to start applying Sam in your organization starting Monday and in the end I'll open up for a few questions or feedback if you already are using Sam and if you have some feedback we can open up for that or if you are new and if you want to start using it if you're any questions we'll leave some room for that you so if you are in one of these 15 plus different roles then this particular talk is very useful to you and as I mentioned you will be able to walk away with knowledge that you can apply to your organization and this is our like just representation of our target

audience but mainly ba stamp is geared towards everybody who is involved in building and securing products so first of all without them Sam is the result of hard work of the contributors around the globe over the past three years this group represents different geographies companies of various sizes consulting and product development as well as academia as a result wast time is very versatile and can be applied in companies of various sizes across the globe there are many times during the design and construction of osm when we were having debates and it always was very helpful to get diverse perspective from this group of new Jews because many times when we were discussing about certain language to use

certain metrics certain guidelines it was always helpful to understand the perspective from somebody from South America somebody from New Zealand somebody from different parts of United States as well as somebody from Europe so this has helped us neutralized the language for them and also oral model where we wanted to make it so useful and so versatile so that anybody and everybody across the globe as well as all different types of companies can use it be it small a little startup or big corporation so what is awesome what Sam is one of the flagship projects and flagship status is given to projects with strategic importance to both laws and application security in general om is one of those projects I believe

there are about fifteen or sixteen of all the us projects they have received the status and what then is one of those projects om is the framework of Software Assurance that provides effective and measurable way for all types of organizations to analyze and improve their software security posture that is tailored to their specific risk that particular organization is facing and the key thing is one size doesn't fit all so you really need to look at your organization's risk and your own maturity level when you start applying osm fam is full of useful resources that does help in evaluating your organization's current security practices providing recommendations or suggestions for growing and maturing those practices providing a way to

demonstrate concrete improvements over a period of time and defining and measuring security activities throughout the software development lifecycle one of the big benefits of Sam is that it is vendor agnostic Sam can be done in-house or you could have one of the several application security consulting firms help you with the assessment and creation of plants and watermarks you so one of the key thing about osm is also it's very versatile and it can be used across different types of organizations who are using different types of methodology beat agile waterfall or DevOps so why do we need a model like watch them lately if you see there is a quest to increase speed across all different

types of organizations and many times security might be perceived as more of a hindrance in accomplishing this goal of increasing speed at the same time due to this increased pressure of increased speed many organizations are growing in complexity with increased number of texts texts various organizational structures growth of open source software and various deployment models beat cloud based model on frame model multi platform model as well as responsive design the result almost 75% of the monopolies these days our application related to standardize security activities in such complex software environment we need a model like what SEM so why was them the most that can be expected from any model is that it can supply useful

approximation to reality all models are wrong some models are useful change or box a George box is one of the great statistical minds of 20th century and I think one of the key thing over here is as he says all models are wrong in some models are useful why is that because we all understand that there is no you know each organization is has its own challenges it has its own risk profile and its own maturity level as a result there are no two organizations which are same if they are same our life would have been much more easier it would have been more like a cookiecutter approach if you go from organization to organization but reality

is different the point is that you cannot find a model that will exactly describe the reality there are too many variables and most of the models are built mainly in the academic world and we live in the real world but you can have a model that is close enough to be useful and that is what Sam Sann is that model which is close enough to be useful but not exactly representing your particular organization Sam was defined with that type of flexibility in versatility in mind so that it can be utilized by polarizations of various sizes with different types of complexities and organizations employing different types of development models be it a gel water form or DevOps

in addition this model can be applied throughout your organization or a single line of business or individual project within the organization now let's look at some of the core principles of spam when Sam was designed we had at least this four core principles in mind first an organization's behavior changes slowly over time changes need to be smaller and iterative to really take hold and make a difference second there is no single recipe that works for all different organizations Sam is built with that in mind and supports organization building a program that is tailored to the risk profile culture and maturity they have in that particular organization part the guidance related to security activities must be prescriptive solution

must provide enough details for non security people because there are many security initiatives which fail due to poor details lack of communication or invalid assumptions or all the subsets of the program will be based on being this model being simple well defined and miserable now for those of you who might be new to Sam let's look at the project history so Sam is not a brand new project was project or at least even before there's a history before even it came to us the first version of Sam was developed by independent software secret consultant named Kovac Chandra under the name of open Sam and it was the first job was made possible through funding from 45

software after a number of years it was kind of being stagnant from like somewhere on 2009 till 2015 a small group go out together at was and work together to breathe some life into this dead project or not dead body but it was connected project and San became awesome project so it is no longer open Sam it is now considered awesome but this is the history like it started as an open Sam project back in 2009 the ocean first version after it came under was umbrella was version 1.1 it expanded and restructure its predecessor into four complementary resources core document that describes the course and model how to guide that explains how to apply this model which Start Guide to help

accelerate learning and eruption and the most important one is the toolbox which is kind of like a spreadsheet that voids simple automation for detect election matrix and graphs back in 2000 1700s them came up with a new version 1.5 version 1.5 incorporates refinement of the scoring model to provide more granularity to the scoring in assessment the applying more model has been designed to have Sam Assessors and organizations avoid the awkward discussion on whether to mark answer yes or no when it is honestly something in between and show incremental improvements we just launched Sam put ATO in February 2020 where we have changed the measurement model one more time and this time with aim to as qualitative

measurement to represent how well an organization is performing the security practice because in 1.5 we definitely added a lot more granularity but it was still missing the key element of a qualitative measurement and that was one of the key feedback that we received across the globe from Sam practitioners so that is why one of the key things that we wanted to change was this particular aspect of the assessment and measurement now let's look at the maturity levels and the assessment score at a very high level so at very high level therefore maturity levels zero one two and three at level zero particular practices completely unfulfilled at level one security practices perform but in an ad hoc

fashion at level 2 security practices perform with increased efficiency and effectiveness and at the highest level which is level three security practices perform with mastery at scale across the organization not everyone needs to make level 3 in all the areas this is one of the mistakes that I've seen many of the new organizations make they try clean for level 3 in all different areas and that is actually honestly not the boost good use of your limited resources the goal is not to max out on each practice security practice in each area what the target machua should be for your organization is largely up to you depending on the business drivers and the risk your particular organization is

facing

so in version 1.5 we had modified the scoring model to provide multiple choice answers to allow for more accurate assessment previously in Sam and most of the models the questions we're like yes and no which is great from like more of academic perspective but in real world as we all know the answer many times lies in between the two let's take one example so looking at Sam 1.5 if you are doing the assessment if you are trying to answer the question on education and guidance practice are those involved in software development process given role specific security training and guidance that's the question that you as an Assessor is trying to assess you know you have trained some of the

developers and would like to train some project managers and cue engineers but given the situation how do you answer yes or no now this is the dilemma if you answer no you get no credit it looks like you aren't working on it but actually you are and if you answer yes you may get full credit and may have issues down the road when you are trying to ask for training budget for other roles such as project managers and QA engineers because the - forces you already did this particular activity and that was the challenge in the previous model 1.5 with the new model you can answer something like some or at least half so I meant in version 1.0 and 1.1 in 1.5 we

added this new granularity levels where you can add some or at least half and get the partial credit but also have the ability to show improvements in your score when you finish the initiative to train the other rules with the new model we also change the wording to many and most and not all and these rationale behind that as well in practice there's always some activity which will not be performed for small group of applications for the right reasons for example if you know that some service is retiring in near future well you may not want to spend your precious resources in getting add a particular application in compliance and you can still get the full credit

because that work application is going to retire anyway very soon so that is why we also change to artists we propose menu our most and not all because in real world accomplishing all is kind of a tall task and it's almost like that long tail of like you know last few applications takes you forever so that is where we want it to be more pragmatic and wanted to provide this flexibility now before we look at Sam Porto wanted to spend some time on introducing to the folks who are new to Sam in version 1.5 so at the highest level Sam version 1.5 is defining four critical business functions in each business function the category of activities related to nuts

and bolts bolts of software development practice that your organization might be following for each particular business function Sam defined three security practices each security practice is an area of security activities that build assurance for that particular business function for each secret practice and define three different levels of maturity as objectives and each level within a secured practice is characterized by successively more sophisticated objective activities as well as more stringent success metrics than its previous level so this is SEM 1.5 at high level now one would ask what are the motivations behind a new more new version for this model so there are five main motivations first is aligning with the most recent development methodologies such as agile and DevOps

and when we wanted to do that our goal was to make it software development methodology agnostic because version 1.5 and its predecessors look more suitable for waterfall direct mythology even though it was not meant to be so we kind of collected feedback from sand practitioners across the globe and we realized that it's missing key aspects of guidance in terms of how to securely build and deploy software specially since eh series part of agile and eros mythology second motivation is to improve the measurement even though Sam 1.5 address the feedback about different granularity levels it still did not address equation how well an activity is being performed thus needing some qualitative measurement the third motivation is to avoid orphan

and unrelated activities in different maturity levels there are quite a few security activities in version 1.5 and its predecessors were defined such that it lacked consistent theme across different levels of maturity within a security package which also resulted in few orphan activities such as scores I mean if you look at version 1.5 their security practice called course code signing and there's literally no consistent theme in the same security practice and across different maturity levels and also there is no similar activity yet mature level one and mature level three so it was kinda like created more like orphan activity the fourth motivation was arranging maturity levels in order of increasing difficulties so this was another shortcoming of previous Sam

versions it was sometimes possible that some of the security activities at high level very in fact a little bit easier to implement compared to security activities at the lower level now it was one-way intuitive like you know if you look at material level it's one would assume that as you go higher in the level of maturity the implementation cost should be higher but that was not the case so we had to fix that one as well the last but not least was Sam production process itself it was slow and waterfalls resulting in major overhaul of work every time we needed to release new version just recently in fact Sam itself Sam project itself is on

th any itself that's a great progress and that was also one of the motivation behind the second version of Sam now let's look at the same framework for version 2.0 the sample dotto at very high level and the area highlighted are changes from version 1.5 which we'll call in few more minutes sampled Auto is defined in three levels again at highest level Sam defines five critical business functions versus four in version 1.5 and each business function is the category of activities related to nuts and bolts of software development in your organization for each business function Sam defines three security practices each degree practice is an aerial security related activities that build assurance for that particular business function and for

each security package again Sam defines three maturity level as objectives with insecurity practice is characterized by successively more sophisticated objective and more stringent success metrics than the previous levels and overall as you increase the level of maturity you should expect higher cost of implementation as I mentioned earlier this is one of the area we fix in version 2.0 if you look at the framework you can see that governance is more focused on program itself looking at more strategic elements such as strategy and metrics policy and compliance education and guidance we have renamed construction business function to design business function into dot o and introduce the new business function called implementation design implementation verification and operations together cover the core of a

software development lifecycle high level design is focused on three things threat assessment security requirements and security architecture or secure architecture intention is focused on secure build secure deployment and defect management and we are going to spend a little more time on this particular business function a little bit later verification is more focused on architecture analysis requirements driven testing and security testing so these are more like testing and verification of this model last but not least is operations which is focused on incident detection and management and environment management where the apps live on so at high level this is Sam Boudreau framework now let's take a closer look at security practices in this new framework if security practice is divided into two

streams stream a and stream beach the purpose of the streeams is to align and link the activities within the practice or different maturity levels each stream has an objective to be reach and this objective can be reached in increasing levels of maturity this way we ensure that there are no orphan activities that seem only relevant on a single maturity level like for instance I mentioned code signing in version 1.5 let's take a closer look at one of the security practices which is the requirement student testing under verification business function to understand this screen concept a little bit more so requirements testing security practice is divided into two streams the stream a control verification and stream B misuse or

abuse testing this dreams aligned and linked activities in the practice or different maturity levels as you can see eastream has an objective to be reached and this objective can be reaching increasing levels of maturity anystream also provides that consistent theme across three different maturity levels

as I mentioned earlier the key changes in this particular model are imitation business function under which secure build and secure deployment are the key practices which are more years towards DevOps and agile practices so let's take a look a little bit closer look at this particular security practices so secure bill this practice focuses on creating consistently repeated repeatable secure build process and accounting for security of application dependencies as I mentioned earlier in today's complex world there are a lot of organizations a lot of developers they use third-party open source libraries and some other dependencies so it was important to include that particular aspect in coming up with the new model the secure bill practice emphasises the

importance of building software in standardized and repeatable manner and of doing so using secure components including third-party software dependencies as you can see the first stream which is Bill crosses focusing on removing any subjectivity from the build process by striving for full automation at level 3 and automated bill pipeline can include additional automated security checks such as fast and as to gain further assurance and flag security regressions early by failing the bill for example now I'm not recommending that tomorrow everybody should start failing to build based on SAS and as result because there's a lot of work need to happen before back such as you need to remove false positives you need to remove noise from some of these tools

because most of these tools they have some level of noise that they introduce once you remove that noise and then you have full confidence that the issues identified by these tools they are high confidence high fidelity issues then you could reach to that level and that is the reason that that particular aspect is as metric level 3 the second stream which is software dependencies how is the prevalence of software dependencies in modern applications it aims to identify them and track the security status in order to contain the impact of the insecurity on other ways secure application the most advanced form it applies similar to each extra software dependencies as to the application itself and there are few

pools available and there are a couple of very good towards projects in this particular aspect so for stream B this was dependency check and was dependency crack are the two main tool which are available from us to help with that particular stream now let's look at the next stream which is secure deployment on stream so next if you practice which is secure deployment this particular practice focuses on automatically securing deployments to a production environment and all required secrets one of the finest ease in believing secure software is ensuring that the security and integrity of develop applications are not compromised during the deployment - this add this practices first stream which is deployment process focuses on

removing manual error by automating the deployment process as much as possible and making its success contingent upon the outcomes of integrated security verification checks it also forces separation of duties by making adequately trained non developers responsible for deployment some organizations they have only certain developers who are more in DevOps roles they are responsible for deployment the second stream which is secrets management goes beyond the mechanisms of deployment and focuses on protecting the privacy and integrity of sensitive data such as passwords tokens and other secrets required for applications to operate in production environments in simplest form suitable production secrets are moved from repositories and configuration files into adequately manage digital roles in more advanced forms secrets are dynamically generated

and deployment time and routine processes detect and mitigate the presence of any unprotected secrets in the environment and I am NOT here to recommend any particular tool but there are some tools available some of them are open-source tools some of them are build by various companies so you can look at those tools to accomplish this particular objective now this slide I'm not going to dwell on this slide too much because I already have covered some of these suspects during the motivations behind the new version but this is a quick summary of key changes that we introduced in Sam version 2.0 I want to focus on the last bullet which is on the measurement so the difference between SEM 1.5 and to

auto is also on measurement 1.5 focus mainly on coverage based measurement the 2.0 version focuses on qualitative aspects in addition to coverage business measurement so after looking at all the changes that we introduced in Porto as well as introducing this overall model to folks who are new let's spend some time on how do you apply this model to your particular ization as I mentioned each organization is very unique in its own aspects especially their risk profiles and their existing level of maturity so it is very important to start with preparation and preparation goes beyond just doing the assessment so preparation I'll talk about preparation a little bit more in detail in the next slide but at high

level typical rollout approach for Sam includes six different phases the first phase is prepare phase then assessment during the target defining the plant implementation and rollout and this curve again goes back to assessment thus creating more of a cyclic process or cyclic and continuous change improvement process so let's look at the first phase which is the prepare phase preface in my mind is the most critical phase called success of Sam application your particular organization it consists of four activities the first one is to define the scope so before even you start to implementing Sam you have to determine you want to apply Sam to the whole organization a particular business unit or some particular applications or projects

the reason for that is the scope really determines the amount of work it needs to be done as well as what kind of buying you need to receive so that is what leads to the second one where once you identify the scope of your implementation you can identify the key stakeholders and they vary from sport to score because if you want to roll out some Sam across your whole organization you need to get buying from much higher levels of leadership courses if you are confining your scope to just particular business unit then you need to take buy-in from that particular business units leadership team and if you are doing it on that worker one particular

application or project then you just have to get buying from that project lead but the key thing is once you define the scope you need to get critical buying from the leadership team once you get the buy-in you need to start spreading the world and do the evangelization because that way you will get lot more broader support as part of preparation please review the resources that we provided as part of same version protocol guide for maturity levels and defines objectives benefits activities assessment questions quality criteria success metrics and the cost I was shown one example here so if you look at this guide it's pretty self-explanatory and as I mentioned earlier that this guide has been created

with group of volunteers across the globe who work at companies of different sizes as well as academia so the language is very neutral and it's very self explanatory spend some time in starting this guide so you understand how to use them after that you start doing the assessment in order to do the assessment you need to start conducting interviews with key stakeholders to evaluate the current security practices here we recommend the in person approach versus emails this way you can explain the key intent behind any activity and clarify any potential doubts they may have there are three ways in which you can perform an assessment the lightweight assessment there is a detail assessment and there is in a

hybrid assessment lightweight assessment is simply interviewing key stakeholders and recording their response during detailed assessment you asked for evidence for performance and quality of each activity being performed in the hybrid assessment model you asked for evidence only on a need basis for some of the activities and not all of the activities and at least from the most of the practitioners and practitioners the assessment takes a good amount of time like it depending on your organization or the scope that you choose it may take somewhere between oh one hour to two to three hours it really depends on what scope you are trying to cover so the most particular approach is the hybrid assessment now once you record the responses from

the key stakeholders and look at the evidence you can assign maturity levels using the Sam spreadsheet that we poured it you one of the key thing about this Sam spirit will curl a bit more in detail in the next slide is how we have come up with combined coverage based measurements as well as quality based measurement as I mentioned each degree package needs to be assessed on those to access coverage as well as quality so let's look at the example of spreadsheet or the worksheet that we have built for version 2 Auto as you can see in the previous version we just had the question for example if this was an assessment for 1.5 it would

have been the question new test applications for correct functioning of standard security controls now in version 2.0 the way we have added this quality criteria are clubbed together with the question so if you say no for any of this quality criteria or if you do not meet any of the quality criteria the answer for the question should simply be no you cannot have yes if you do not meet the quality criteria that is how we added the second dimension of measurement in this particular assessment and the key goal here was we a lot of debate when we are deciding on this one and we went back to this some of these sandport principles and one of

the core principle is simplicity so as a result we decided to add quality criteria for each question this way time to complete an assessment did not significantly increase with sample dotto and this was the key thing which helped us guide in making this decision because we did not want to create this very long and laborious activity that is why we added simplicity and we added this quality criteria to the existing question which was aim to cover the coverage all mature score for the security practice is calculated by taking the average of maturity level one between stream a and screen B and adding that the level of maturity at level two and level three and once you add all of this

you come up with the rating for that basically practice once you finish the assessment you need to define the target as per business drivers and the risk profile for your organization now the most important thing over here is to estimate the cost so on average it takes about five to ten percent higher cost for every level of maturity when you try to increase every level of maturity it takes about five to ten percent higher costs we do not have concrete data but this is based on most of the Sam petitioners feedback this is the rough estimate so you need to keep that cost in mind as well and this costs again varies from organization to organization

there is profile as well as the existing level of maturity so keep that in mind once you set the target you can define the plant and in order to do that first you need to determine the chain schedule as per the upcoming release and develop and update roadmap plan for the next four or five phases now this phase wise approach is also good for from the change management perspective and what we recommend is no less than three phases and no more than five phases to roll out Sam we also recommend that you start with the most impactful security practices such as training in awareness and site assessment in the first phase once plan is defined start the

implementation implement activities using sample auto guide that we have worded as part of the resources leverage other Sam projects Sam aspires to be an umbrella project of all samples so always projects and what it means is what Sam project can map back to one of these Sam business functions and security practice so we want to have this mapping done as part of future directions and here are some of the example or voice projects and how they map back to various same business functions and security practices so as you can see some of the worst projects edges was top ten that can map to multiples and business functions and not just one business function

after implementation we need to create and update scorecards on regular interval by capturing scores from before and after an iteration of assurance program build out and communicate this progress to the management the key thing over here is communication and again this phases they have lost somewhere between three months to 12 months 1215 months so it again depends on the scope of roll-off the phase can be can last about 3 months or it can last up to 12 months as well but the key thing is you need to communicate the progress to the senior management on regular intervals now here are some of the available resources which have to get you started using Sampo all these resources are

linked to our Sam to our G website so if you forget everything just remember one website over time dot o-r-g all the resources are linked to that particular website so loo visit that website and one more thing sand benchmark initiative so what is sem benchmark initiative it helps answer the question how do I compare to other organizations if you remember we discussed on others operations maturity framework earlier during this talk called vision same benchmark initiative is inspired by vision be seen does really good job at providing comparison with other secure auditions the goal of this project is to collect most comprehensive data set related to all organization maturity of application also physical programs this data should come from both

self-assessing organizations and consultancies that perform third-party assessments we understand privacy is key here because many organization may not want to reveal their level of maturity or immaturity to rest of the world keeping that in mind or data collection process will be anonymous and we will make sure that privacy is always at the forefront when we collect this data you so before finishing this talk just quick highlight on what is the future for Sam so Sam will continue like the previous version we are able to have smaller and faster iterative versions version 2.1 2.3 so on so forth in fact we just embarked on ch CD ourself the second one as I mentioned earlier is the references to otherwise projects

because we envision Sam being more of an umbrella project under us and all of us project can map back to one of the Sam business functions so we will definitely provide more references to other wise projects and last but not least is it's a very aspirational goal of making this assessment and roadmap creation more of an online process that would really help design this to match the moon more modern software applications itself currently if you can see you have to download some of these resources such as the spreadsheet which would call it worksheet to do the assessment and create road maps eventually we aspire to make all of these things through online assessment and online road map creation

templates having said that I the key call for action here is to start using Sam Stemper Auto and somebody says proof of the pudding is in trying it so I really asked everybody over here to start using Sam 2.2 and if you have any question you can always reach out to me and others Sam volunteers and also you can reach out to us for any feedback you might have thank you Thank You Hardy we will go into a short break and then we will be right back with Ryan tick thank you