O'Shea Bowens - Pushing the SOC Left To Achieve Nash Equilibrium

cool thanks uh awesome uh very happy to be here actually i i moved to boston almost four years ago so besides hasn't happened since i've actually been here so this is uh this is great uh well done on i'm patrick levy's part and the whole crew for b-sides and thanks well most definitely thanks for uh thanks for having me uh so welcome to my session uh this is called uh pushing the sock pushing the sock left to achieve nash equilibrium uh but quick run down excuse me uh a quick rundown uh i'll do a quick introduction for myself and walk through the three stages of the talk uh and then we'll take questions at the end

so if you have any questions just yet please save those towards the end uh i tend to talk kind of fast so especially if you know me i talk very fast actually that was being modest but uh i i purposely you know put in a lot of effort to ensure we at least have 10 minutes of questions uh cool so who am i uh yeah like i said my name is o'shea bowens uh i've spent the majority of my career uh operating in security operations or uh if you have the government uh background c d computer network defense so most of what i do is security analytics architecture incident response threat honey intel all the fun stuff pretty much almost

anything that isn't pen test and actually that's a good way of putting it uh if you're in i'm located in boston uh if you're in the boston area you can typically find me around uh the boston security meetup uh which is every third thursday of the month issa i'm also the director of training for blacks and cyber and also you can find me around the devcon uh the boston defcon group dc617 sweet so why this title why did this weirdo go with that title um when i think about you know i've been i'm 37 so i've been in security for almost like 12 years but when i think about uh kind of the direction in which

security operations uh overall the defensive side of security is fake posing i mean uh focusing or what they're beginning to phase in and phase out uh as i was writing this the idea of like you know john nash kind of came into mind and you know you can look john ash but essentially the the premise for for this talk was really structured around game theory right so uh if you imagine a situation where you know you have two players and a chosen strategy but no player benefits from like changing strategies you know uh no player would actually benefit from changing strategies you know you'd have to be on the same page in order for both players

to essentially find that level of reward right so it's essentially you know you all you stick to the same strategy on both sides or if one side changes it pretty much everyone loses and that's essentially kind of the way that i look at uh what's happening from a uh you know from a sock perspective uh security operations center so when i say sock that's essentially what i mean just to just to shorten it i spent a decent amount of my time living in uh the security operations lands i've probably spent 80 percent upwards of 80 of like my career working amongst you know a huge team you know so what is a sock uh i'll let you read through this

yourself but essentially you know if you're looking for the official definition it's you know this the it's it's a team that is primarily put together to focus up security efforts for against uh people processes and uh uh technologies right so how do you bring together a group of individuals with various skill sets and apply them or tar or focus them towards specific challenges for your for your organization um so that's more like the official term in reality it's a nut house uh you know i've had loads of fun uh kind of working in multiple socks across different uh vectors uh and in different businesses you know and honestly that's what i attribute to a lot of my

a lot of my knowledge uh as you know working with people that were great with intel led me to uh you know pick up an interesting and uh cyber that intelligence and put kind of put that tool in the tool belt right working with people that were great with uh into the uh digital forensics that led me down the ir path right so you start to like surround yourself with these individuals that can do some really awesome uh things really awesome talent and you slowly begin to pick that up right i think you know as for myself i know that that's the best way i learned is kind of by doing it when i'm in a

conducive environment where everyone's kind of pushing and everyone's you know essentially trying to level up themselves and they want to tackle really really difficult challenges that's you know for me that's kind of where i thrive um and when i speak about you know you know most in in most organizations you know you will typically find a charter for any ita eit operation right so if you think about you know devops or if you think of us as admin or networking there's simply some type of charter which essentially consists of their mission statement of how they how they reach their goals and how they make their goals uh i you know i must i try to be a simple person so it's i

kind of summarize you know the socks overall uh focus around uh monitoring detection and response right you know those are the three pillars at you know a very at a minimum that you should uh you should be focusing upon especially if you are building out of soccer you've just been awarded a budget to actually build out you know the resources and bring in uh people and the technology you know focusing on three these three areas if nothing else will will get you it will get you pretty far right so you know focusing for monitoring side what what what is available to me and what am i protecting right so how can i what am i what can i use for logging

what can i use for identification how can i tag data resources if i have some type of log correlation tool splunk devo uh elastic you know whatever it may be from a detection perspective you know it sounds just like what it is right ability your ability to detect um i know how how fast are you what's your lag time which was what's well twined for attackers you know how are you baselining your environments and then response you know responses essentially you know what steps are you taking to either kick an attack or out of out of the environment or maybe it's not necessarily related to you know what an attacker's zombie response ever is maybe it's related to a

pin test and you moving through the remediation stages for that uh you know it can it can vary but those are the areas that i say you know kind of make up this fun nut house called the sock um so where did the stock go wrong uh well i don't i don't think the stock actually went wrong anywhere i think we just kind of took our eyes off the ball as security became a much more focal point for companies you know you go back you know 10 12 years there really wasn't a huge budget attributed to individuals within security operations or the security program uh over the last you know five six seven eight years you're seeing these

these uh budgets you know just balloon up i have buddies that work at pretty large financial institutions and it's not uncommon for you know a chase or uh uh or a wells fargo to have you know one or two million dollar budget you know for security operations right that's a lot of that's a lot of people there's a lot of toys that you can essentially buy but what i think has kind of happened is you know as different areas of i.t have changed and shift shifted you know we aren't as tuned in we're yarners tuned in to devops and often at times uh when i can when i consult you see that you know application

security teams sit outside of the actual sock right so there's not that there's not that delineation there's not that line of communication between you know what the sock is working on and what security operations working on when you think about you know moving towards really staying current with automation staying current with moving towards the ability to rapidly scale up you know you have to think about devops and you know uh if you're not you know one of the things i want to want to take away from this if you're if you're in security and you're not you and you're a bit unfamiliar with you know devops practices i always recommend to anyone people i mentor

younger people i run across you know take some time to really understand that and devops it it may seem complicated on the surface but really you break it down into a few different areas right you're thinking about you know you're thinking about coding you know the code development and review process you're thinking about building so that's you know continuous like integration uh and build status think about testing how do you put the first two together uh to reach a you know a tangible goal uh packaging refreshing configuration like monitoring these are all areas of like how you how you begin to like package together a specific application when you release it how you configure it

how the settings are actually configured within uh within that particular application within that particular instance and then you know how do you deploy it how do you monitor it right that's about five or six different areas but those are really kind of the core errors of what makes up uh devops you know and like you know in plain english it's just you know a process to remove a barrier to rapidly release programs um in order for you know devops to be successful there is a few areas that you know that are uh kind of heavy hitters in this heavy hitters for in its entirety so if you you take the devops uh you take the devops

uh goal of you know being able to rapidly release scale up scale down you know in a much faster time period well how does that get done right uh this is essentially where you know devops pipelines kind of come into play uh again keeping it super super simple you know think how do you break down you know a devops pipeline you know and that's you know cicd continuous integration and continuous development right if you want to make that really really simple think of it as more like the universal control uh that you can make with and within the within releasing or within packaging within devops of sem or uh our source content management it's really just you know of trying to

avoid like merge conflicts when you when you're releasing code or when you're making new polls or when you're adding adding deleting whatever you're actually doing just ensuring that whatever you change from a modification perspective you can always recover from it uh think of it as uh dr i guess within devops um build automation tools uh web application service uh and code testing coverages these are all just other areas that are kind of akin to really solidifying uh solidifying the devops process and ensuring that whatever you make can be recovered you have a you have a regularly deployable uh schedule you know what your packages would look like you understand the interfaces that the interfaces can actually render and you

know this is all manageable and trackable right skips one um this other area that i mentioned that you know uh there is that kind of that separation essentially is application security so uh so what is application security i mean in simple simple terms you know it's it's really your ability to statically or dynamically review code uh from a security perspective that's that's a part of an application or really a part of a system um you know from a securities i mean from uh from a defenders or from the cnd side you know i say your focus is really on uh you know why we care and how can we win um you know as a defender's job this

isn't too different across really any areas uh security right i look at and i kind of take the approach as i'm a defender i'm here i'm meant to protect you know i can protect moderate to respond going back to those three pillars but that's a win at the end of the day for me right so one of the bigger things that i've seen from a lot of the past couple years of like consulting and working uh on different projects is when this sits outside the sock and you come to you know the people in security operations like hey what's going on your application security program what type of findings are they coming back with from scanning

which we're going to go into in a bit and you know the answer is usually i don't know right and that's not the fault of anyone it's just that it we haven't been at a position where it's uh it's totally integrated into security so the stock typically won't have those answers right off the top of their right off the top of their head right like you know if you think about a traditional vulnerability management program how is there really any different from incorporating application security and understanding where those vulnerabilities sit right that's something that you would want to be locked in and focused upon right that's something that you'd want to have an idea of how can i

build out the counter measures and detections that gets it um an application security is you know and you know i didn't i don't come from an abstract background like i said i come from a sock background the last two or three years i begin to think more around after i think it's honestly like since i moved to boston you know i became friends with like people like you know ori who's uh uh running a workshop client who's really big in application security getting to know a lot of the people at oh wow starting to go to those things you really start to like listen to what they're what they're working on from a day-to-day and at least from my perspective i think

well how can i incorporate that into security operations from a duties perspective and from a monitoring detection perspective but you know just some quick little a quick little statistics you know around from every year verizon releases a report uh the dbir report data breach uh investigations report and this is just chock full of useful information essentially if you boil it down what it really is it's just an annual report uh the top vectors that are targeted by attackers and the top techniques that attackers leverage uh to reach their goals um and some of the findings from last year was you know were you know kind of astounding in some ways right 70 data breaches caused by

external actors that's not new but the new one is like you have organized crime in the mob getting involved with it 45 of uh breaches or calls uh by application are caused by web applications uh vulnerabilities and web applications right like these things are on the rise you know and it's kind of it's kind of creepy and innocent um but and also if you're going to be positive it also allows us to really take to take a step back and look at reports like that and as an individual working within the stock thing how can i tackle some of these areas right um and when you begin to think about okay how can i get involved and

how can i tackle that you really need to understand what's like the crux or or what essentially is you know the makeup of abset and we already went i've already mentioned that you know part of this is really understanding static and dynamic analysis but this doesn't mean you you know yourself have to have to have to do this uh have to you know sit there and go line by line uh to analyze code there's a lot of tools out there which we'll speak about in a bit but kind of those primary responsibilities around a dynamic and static analysis also incorporating threat modeling but modern lean is something that's not totally new to an individual from a

security operations perspective you run this across your networks and systems or you should be doing this you know at least once a year at a very very minimum but ideally like you know once a quarter or biannually you know throughout modeling your environment understand where your ingress and egress points are what os's are there what type of uh networking equipment is available there uh where do those vulnerabilities sit between those uh against those two areas right uh you know existency taking pin test results and uh really taking effort and and moving towards being proactive and creating countermeasures from penetration tests and results right these things aren't totally different uh uh on the abstract side it's just different types of data

and it's structured differently so you know begin to take on the mindset of well how could i do this as a defender a lot of the times the things that i hear from individuals uh actually probably the top one is well i don't you know i don't code right so how the heck do i do that and you know and and that i i totally understand that totally ballot uh you know it's a valid question to ask but you know living in 2020 right now there are some really really awesome tools from a dance from sas perspective that kind of help you get where you need to be right you know there's you know you have uh burp suite which

has a free version you know uh there's breaker man there's uh there's actually sneak you could use snk that you could actually use to really determine where those vulnerabilities sit from from an aztec perspective and then taking that output and attempting to incorporate that within uh security operations so you it helps the code drastically it was it's uh you know it definitely helps in your day-to-day in your day-to-day aspects but you can you can get by without it for a decent amount of time especially if you have individuals kind of dedicated to apps you can open that line of communication and begin to think hey how what what more can i do as uh as a defender uh so how do we push

left uh so we'll just walk through a quick example of kind of what this will look like right so here we have an app nothing really special uh just uh app and beta for a restaurant that uh that's been created when you think about the threat modeling perspective i'm sorry when you think about the threat modeling stages you know there's a couple questions that you want to ask yourself of understanding hey well what type of code runs where are uh when we scan what type of problems we worried about who leverages this is there an api hook into it you know what type of data are we storing you asking these type of questions from

you know a threat modeling perspective and then you prioritize you know those concerns again taking the same steps you would you uh taking the same steps that are applicable during uh system and and networking uh that bottling station uh stages right uh category i mean uh categorizing your concerns uh you know and you keep moving down okay great now that we understand what we're worried about how do we begin to act upon it right so how do we move into vulnerability uh detection and actually test it again not too different from what you would actually think about from the attack life cycle it's not working system there's a vulnerability which leads to the exploit which likely is a payload that a

malicious actor would create which leads to infection and attack so you're moving down that chain of trying to reproduce this within the sock this goes to like you being mr and mrs proactive you know defender thinking okay well we've had output of my dash and sas people you know how do we actually recreate this uh you know and is this realistic i mean yeah what we're going to show right now is something i just simulated in a lab environment but it's not it's not totally different from what you would want to try in your actual workplace you know you go through the stages you just you scan initially to identify you know your attack vector diff

identify uh the functionality of the application here we know you know the big thing to walk away with for this for this particular uh simulations that we know that you know uh the app accepts uploads it has to take some type of upload for uh restaurant reviews you know you walk through again same things you you speak to give out from vulnerability and penetration testing perspectives you know you scan i mean you understand functionality you understand your scope you begin moving into scanning uh from this one what we did we just ran a quick static analysis against of against against the application so a couple of vulnerabilities came back of interest in the directory traversal is a

big one that you know but you know when you look at cross-site scripting credential management you know these are things that as a defender if you're unfamiliar with these terms owasp does a ridiculously amazing job of breaking these type of attacks down into super super simple terminology which will allow you to take that output and start moving towards incorporating different monitoring detection tools to build out some type of countermeasure from the data analysis dynamic dynamic analysis perspective uh here's the output uh from the code against uh against the lab environment right so nothing too out of the ordinary there just some exploits we're going to attempt to vulnerabilities we're going to attempt to run against

uh and once you actually have this data i think this is another area that uh i think there's a couple of people speaking about this today also but i think this is another area of like really making that uh the jump right so essentially okay we know where the vulnerabilities ex uh live we got the scope of the whole application down okay so now what do we do uh and this is where that gap kind of kicks in within security operations typically what i've what you'll see a lot is like you don't have a group of individuals that are really uh trained or that have have knowledge on moving towards attacking right um you know so

what do we do you know when you have a security program that can attack you higher pin testers right but something i've become acutely aware of over the last like three or four years really is that you don't need pen testers for everything right pen testing uh pen testers cost money and often there's not too much too many differences between the reports that they provide you from uh from uh consultancy firm to consultancy firm uh so what i recommend is that you know you begin to get familiar with you know optic from an attacker's perspective and begin to incorporate and learn about some of the tools that are out there right so there's that i don't know the number but i'm

guessing there are hundreds of hours of video for uh madison within youtube right like i feel fairly fairly certain of that uh take some time upon yourself to begin to really understand how you leverage those type of tools right uh going back to our the previous example we understand that you know one of the vulnerabilities that was identified uh from the static and dynamic analysis perspective was you know local bible and local file inclusion vulnerability so what this would essentially allow us to do is find a way to not only upload a file of uh upload a file to the app and but also have the back-end essentially executed uh which gives us a hook in and all

we're really looking to determine is like hey can we browse across file paths from within the url right if we know if we can navigate follow paths or bring back directory against i mean uh directory information from the system then we're a winner okay boom great we have that now so again i'm in a sock i'm thinking okay what more can i do um and you and you truly take on the attacker mindset right you're thinking okay well if i can get in uh you know maybe not our alarms haven't went off from our ids or sim whatever you know security product you may have in place let's start pushing a bit deeper to determine

what type of logs are available to us after we complete this that we can leverage to create counter measures right if someone begins to move laterally what are those 400 and 530 windows logs that we want to take a look at uh with if it's cnc communication or some type of encoding uh why do we have these weird you know much larger dns uh dns requests that essentially are pointing to urls that no one in this environment has hit like ever right so you start to kind of move through the stages of this you know uh you you've attacked it you have your hook in now you want to dump some hashes you grab those hashtags

uh uh grab those hashes uh uh uh zip those up incorporate i mean uh zip those up encrypt it and then send it out the environment right super super easy um and what you should be thinking about while you're doing these type of things and learning how to use the offensive side of uh offensive tools is you know where does that connect to you know where does that connect to on maybe the pyramid of pain you don't have to reference a pyramid pain but i recommend you have some type of framework or some reference that you're working from so when it's time to sit down with management you can kind of walk them through the

stages of what you reproduce and really show value right you know you need to find a way that you can make this measurable uh you know because if you can't measure it you know does it really exist we you may have your own opinion about that but management definitely has a different opinion considering that they have to report all these things up um and then you know again going back to you know thinking about pushing left incorporating all these different areas of devops and application security uh into security operations so what does that look like uh from a tooling perspective you know again you have like tools like oauth sas and burp suite that can help you get there

and if you need to break this down into like really really simple areas of like phasing this uh from from uh from from a staging perspective again when i say that i mean where am i starting and where am i finishing begin to always think about identification the thing that matters you know in my mind one of the the biggest matrix in the areas of importance within uh within the sock is the area of identification so if you can't see it you know it doesn't exist right so when you complete these type of assessments you know going back from leveraging whatever offensive tool it may be uh beginning to think about or you should be continuously thinking

where can i identify this within our tech stack right what logs are produced that i could actually gather and hunt for if they're not actually produced via my current searches or our current dashboard you know uh where else should we be looking right and this moves into more of thinking about thread hunting you know we know we have logs there we're just not sure what artifacts are present that uh that lead us to the conclusion of like whether the attack was successful or not um so this goes back to what i said earlier about logging practices right ensuring that monitoring detection are there and that you have solid lot of some solid logging practices backing

you up uh because logs will save your life um and again think about pushing left again what other tools can you use to kind of to really incorporate this uh and begin to think about your detection capabilities so you've moved from you know your identification phase or your identification uh mission to really logging okay now that we have logging kind of squared away what uh what other tools can we use to recreate this and actually push those artifacts into more of a detection capa of the detection uh the detection side of the house and also remediation side of the house um you know and where we understand what to do from a traditional standpoint i shouldn't say we we we may have an

understanding of how this looks from a traditional standpoint of c d because we've been at this for a long time but the gap essentially comes in when you have to think about you know do we understand this from a devops perspective right do we understand our bills do we understand the logs that are output from leveraging jenkins with aw uh aws cli and moving towards like ansible or or maybe not even angela but using like vagrant with azure or docker or packer like you think about these other areas of like app uh devops that are still fairly new to us that we haven't really begun to think about how to actually pull logs out of there and there's you know a

lot of information that's come out over the last two years around leveraging different cloud platforms and and they're logging right it's a newer subject and it's still fairly difficult but things like uh of you know aws with guard duty uh you can know you can still have a decent amount of detection that comes right out the box and maybe you don't need to go to directly you know uh tag or define the logs you want you know if you're unfamiliar with it because a lot of that information is readily available to you right um i don't think you can do a security talk lately without mentioning attack miter uh you know i won't go into it

on this talk but something that i recommend is taking a vast amount of the output that you're moving from your purple teaming uh incorporating purple team and after devops and begin to identify specific techniques that you believe are relevant to your environment um this is a bit more easier said than done like what i what i tend to see is organizations will grab attack miter and essentially staple it to the wall and say hey we want to hit everything every technique within within uh within the framework right it's a noble goal but it's going to take a low a large amount of resources uh and it still doesn't really give you the focused information that

where you where your worries are right before you even leverage attack meyer this isn't a minor talk but i think before you even leverage that you need to have an understanding of what your risks are right what are your kind of pris or primary requirements for like intelligence if you're leveraging cti for this type of stuff but really defining your own risk before you move into that uh once you define those risks begin to understand which techniques are applicable against your risk and against your environment and your tech stack um that's just a side note uh but i do you know and maybe this is another talk but you know i do believe that when you

take this information and you apply it to uh uh attack minor you're in a much stronger position to really uh request and request budget and request people for something like threat hunting right so it's not just hey i think something's wrong i want to i want to start thread hunting 40-60 percent of my time uh within the sock it's really it's really taking the time to say hey we can link this to this specific technique for this particular thread acting group that's relevant to our specific vector this is why i want to start thread hunting you know taking more of a an adult approach to asking for what you want um and from a remediation perspective you

know so what does this look like when you're putting you know like all these things uh all these things together um you know when you i'm when you put back on the sock hat and the current uh the current duties are what exists within the charter uh code fix and patching typically kind of falls outside of the sock right user privilege i could i think you can argue that that that's within the stock is understanding access rights and implementing access controls across your environment kind of you know that's something that individuals within the sock would be aware of but a couple of these areas you know outside i mean outside the sock they're totally separated and

and not as useful from a feedback loop creating that feedback loop but when you bring those together uh and you sit all these stakeholders down in the room uh you as you know an individual that's in charge of security operations or an individual that works within the security operations sphere you begin to have a better understanding of what that truly uh what that truly looks like right from the devops and from the abstract perspective uh and from the devops perspective of something that you know i've been kind of throwing myself into over the last uh year and a half is really thinking about you know how do you create you know golden uh golden images and

golden uh pipelines right this is something that uh uh there's a gentleman named casey laxton he's over at uh at toes he kind of had me thinking about this he comes from he comes from like more of a devops background but also has a pretty good uh a really really good amount of security knowledge but something that you know that some in our in our conversations in our interactions you know something that i begin to become very very curious about is like from a security perspective how can i understand when uh there's a new image that is that's a a new image that will be deployed across the environment that's been tagged you know how do we really begin to

understand you know uh levels of like patchment uh levels of patches there uh what vulnerabilities may exist within the within that specific image like how do we begin to think about you know hardening uh hiding against that specific image right like so how do you begin to move towards creating this you know this uh this golden image right so if anything not anything but if certain uh uh certain if specific uh items are out of place whether that's a patch whether that's a specific level of a vulnerability a vulnerability management then you essentially you know you allow that bill to break right you go back to zero so instead of having individuals essentially uploading or deploying vulnerable code uh or

dependencies that have vulnerabilities that exist with them with exists within them you you you uh instead of pulling you go back to zero and then they have to remediate that before they can actually push out the environment and make those prs um it's something that it's just a really really cool mindset i'm super happy that uh that actually made casey because it has me thinking like totally different around you know what i should be worried about as a defender and then you i know when you still in the mindset of you know you know pushing left right do you understand how the microcontainers work within your environment and how that incorporates or how you incorporate vulnerability

management into that uh are you defining you know correct vpcs and iam policies with and uh within aws or you know within other uh cloud providers and there's tools out there that can you know definitely help you uh accomplish this uh anchor and claire are pretty good tools for vulnerability the vulnerability assessment side of things just for specifically within of your automation tools uh i believe interest still free uh up until a certain point um so what does this look like right so again me guy or person in the sock i'm thinking about you know how can i incorporate security practices into uh the devops pipeline and really devops life cycle uh and what i how i kind of see this

working you know i welcome you want to challenge me on it but you begin to incorporate security here incorporate security into uh the devops side of things really this goes back to almost like the the kind of the basics of sclc right uh you're looking to determine if something is out of place or if something opens up a specific risk or a specific vulnerability then you need to like essentially break that bill right so if you have microcontainers and you're trying to take on the golden pipeline or golden image uh perspective of security or enforcement i guess enforcement's a better role a better word enforcement of security uh you know you then have to really

understand from a repo management perspective from a micro containers perspective you know what all those people uh what are the bpcs that's actually going to call out to um what uh are there have we appropriately scanned this specific repo to understand are there like any hard-coded uh tokens available right like a huge portion of like going back to the dvi report if you look at that i think it was 43 or 45 number uh attacks that uh began from a web app i mean breaches that led that breaches that originated from a web application security perspective there is a decent percentage of those where literally all the all the attackers had to do was scan across kit

and look for credentials within code and then turn around and leverage those right like this isn't the most you know this isn't the most sophisticated you know attack in the world it's just really taking some time to scrape github right and you'd be surprised how often this actually works i mean luckily there's tools out there that can kind of help developers avoid avoid this mistake but it sounds simple but you know it's it's a surprising it's surprising how often that that type of attack is successful of just scanning across repo to actually look for credentials um but you know moving down the development pipe up about the devops pipeline you know you should be in this you want to ideally find

yourself in a position where you can really think about you know what is in place right before that pr right or right before that deployed to the deployment into a cloud environment how can i create those checks and balances to understand from a security perspective speaking solely from security but to understand where you know areas within our risk register or areas that we're concerned about from a vulnerability management perspective from a data access perspective from a right perspective anything along those lines how can we bake that into the device pipes uh the devops uh life cycle and really create those blockers so we can avoid you know essentially ending up in the news or essentially ending up on some

blog right and then if this is not to say that um it's easy it definitely isn't you know it takes a huge amount of time to kind of get these things under your belt but really it's starting to think about you know okay from dev to qa to pre-productive production you know what happens if you know an issue is identified in people you know whether that's like exposed credit or something along those lines it should be an automatic blocker that breaks the bill it forces them to go back to zero and i mean that's in the nicest way because we're all trying to win here but unfortunately the force is gonna go back to zero

uh before they uh before the you know any internal and external data or essentially put at risk um you know and you know again kind of closing out around how we actually do this uh the one thing that i think is the i won't say it's the easiest path board it's the best path forward it's really taking the time to sit down with development teams uh uh appsec and also devops i mean uh and beginning to kind of build out more of a security champions program um it's imp it's it's really difficult to try to incorporate all these things that we just spoke about on your own as as a as a member of the soccer as a

member of the security team you're going to need you know loads of communication uh loads of resources to us to essentially help you uh build out this type of program and and you know it does take a lot more of the human effort this isn't really hands-on keyboard work this is just you know good old-fashioned interpersonal communications and taking the time to actually sit down with individuals within your corporate uh within your organization and really ask them questions around how uh how you can actually uh improve how they can help you improve security but also how you can help them uh avoid any potential problems uh at the end of the line right uh yeah so in conclusion uh you know a

couple of things that you know i recommend is really one you know as a as a member of the sock begin to adjust your uh the the the charter uh to incorporate abstract practices within the org uh definitely go out get involved uh with devops you know some of the tools that i covered here today are all open source um maybe just trying those out in a lab environment if you don't have if you don't have the capability to to try this out you know at home i mean i'm sorry at work uh give it a shot at home you know figure out a way to kind of loop yourself into a continuous learning uh cycle so you're always you know not

always but you have a pretty good idea of how to leverage these and it's not you're not in the situation where you have to try these tools out in a bind or in a pinch or doing uh uh during the incident response engagement like you want to kind of have an idea of how to leverage some of the attacking tools uh you know well before uh something actually goes wrong you want to have an idea of like how do i leverage uh some of the development tools right i i recommend shadowing individuals within devops um it's something that's honestly worked for me sitting down with individuals and understanding what our deployment looks like how we

actually leverage whether they're tangible whether it's shelf whatever like what do those recipes look like within chef right what the heck is a recipe in chef that's just you know the code that actually makes up the deployment you know so really understanding what that looks like and asking questions you know something uh i tend to kind of live by is like learn to be okay with temporary ignorance you know it's not it's not as if you were born the best example i could think of it's not as if you were born walking right out the room right you had to you were born you had to crawl around for a bit on your stomach your back

whatever kick your legs and eventually you know you stood upright and you begin to walk right but as that infant and as that child you were essentially ignorant to walking you didn't understand that you know over time you begin to learn that uh so kind of take that team approach to i mean not only security but different areas in life you know learn to be okay with temporary ignorance temporary is a key word here it's going to take you some time if you don't understand these areas to learn so begin to sit down with individuals within your org of shadowing them take some time on your own to study up on different areas of devops and apsec

and then when you come to work don't be afraid to over communicate uh it's you know it's always better to over to go over than under right you want to understand as much as possible because you as a an individual on the within the security program or within the sock it kind of falls on you when things go wrong right the first thought a lot of times within the organization if there's any type of outage it isn't you know i think we're past the days of like oh wait is this something with our code uh maybe i don't know but the first thought is typically crap we're being hat we're being hacked we'll be an attack it's something with

security get those guys on the line you know so you know begin to like understand who you're working with and who your counterparts are well before uh that stage uh that that activity occurs um you're not at the stage of trying to hunt down different individuals that are responsible for different aspects of it development or administration um over to communicate is always a better path to take and that's it for me uh if you have any questions uh you can email me uh you can find me on twitter uh at certain web blood uh and we'll take some questions now awesome thanks o'shea let's take a look at discord i think uh there was only one question on what

pushing left meant and it was answered um basically just detecting issues earlier in the sdlc process phases um do you have anything else you want to add to that um yeah i mean so uh about a year and a half ago actually uh when i began to kind of ask that question myself something that became apparent to me and it was kind of a premise of like uh not this talk but just some work i did uh last year was thinking how do i reflect that within security operations and within our tools right so a good example is like you know whether you use splunk or you're using you know uh elastic devo whatever whatever uh

log correlating tool that you're leveraging um you know within from how can you show how can you actually prove that your your seo of c your seoc practices are being fed into security operations right and they're they're norm there's no uh there's not really a plug-in for that within a lot of the the correlation tools right you kind of have to build this out from scratch you kind of have to understand okay well let's make a specific search around uh these type of api calls within our environment to this particular part of the application and understand what that what is normal there right uh who's who who are the top requesters there uh what region of the world is

that coming from uh and but maybe let's add that with uh http status code right understanding you know okay well what's the what is the average number like four or fours with see or maybe 300 against uh results from these api calls um you have to like take what's essentially given or really the the genesis of what a clc is and apply that going back to the secured operation side to those three pillars of monitoring detection and response and really thinking how do i how do i prove it right like if someone were to ask me uh me as an analyst you know how secure is the application or do you understand normal activity against it

and the answer is no then i would say maybe slc isn't like as effective as it potentially could be from a security operations perspective because you're not totally looped into that it's not your day-to-day uh to sit there and shift through code you know your day-to-day is really understanding the login and the output and leveraging different tools to correlate that so how do you build out you know really solid output from sdlc into you know your logging practices right

awesome um thanks for that answer another question from i think it's australian b is how do you think the sock absence dev feedback loop should be sustained are you suggesting that the sock and others use a common framework or common issue tracking system to keep everyone on the same page or path that's a good question so one uh i'd say that first step of that is really creating that uh creating that security champions uh program uh you know sitting down with those individuals at a minimum once a month uh to walk through uh events that have occurred something i did in the past was like we would sit down with our application team and we would bring

some of the top cases from an ir perspective whether that was the previous month and maybe the previous 60 days uh when we would essentially kind of look at the top uh the top of events that they had anything that led to like an outage or anything that kicked off uh the security i mean the uh the the development team's ir cycle um and we begin to kind of compare nodes and then really begin to understand like okay what is the form of escalation or what is our our procedure of shifting things from application team into security operations right so what it's like a checklist that up the application team runs through to understand that hey this isn't

necessarily an issue with uh an issue with the code or the configuration this seems to be something of an external factor and how do we you know uh open this ticket and shift it over to security operations so they can further investigate i mean you kind of have to walk you kind of have to walk through these but from that feedback loop i think there has to be like a level of of essentially uh requirements uh before you actually start you know tossing tickets like over the fence right so application team does xyz within their checklist of when they have an outage or they have uh uh some type of some type of error occurs

uh and then you know from that from that checklist being completed they are able to either escalate or transition that over to security operations security operations has essentially the same thing right like we reviewed xyz logs we have compared that against the output from all these different tools that we may have that are within uh the security uh program up here here's our output right and you know you're continuously uh it's continuously rediscovering this or we uh not rediscovering but uh sharing this and revisiting this to really understand where those gaps kind of exist from you know from devops side to an application security side to a soft side right like you need to first

set kind of expectations and goals and then find a way to track it you know jira is great for that uh and then from the tracking perspective being sure that you ensure that you at the end of the day there's some variant of lessons learned that's available so you can go back and revisit everything that's taking place and really look for you know spots where you can clean this up or get better

awesome great uh another question came in this was from thomas k sec uh with i believe they're with times one of our sponsors and they said i like your suggestion for post mortems it gets a lot of the visibility plus chat and gets folks on the same page but it feels it still feels like a lot of socks are overwhelmed with the alerts and are struggling to stay afloat so it's really hard to persuade management so i see so to get more resources are there any other recommendations to persuade management that we need to shift left and get more resources yeah i think you have to you really have to prove it first like the i'll give you an example like when i

because i faced a similar problem around uh budgeting uh so the concern was like how do we know that this is like a legitimate problem uh so what i had to do it's not too dissimilar from the answer i gave a second ago i had to sit down with uh our application team and really begin to understand from uh from let's just say the standard logs within guard duty right like what are some of those standard logs uh from against the sql injection to like some of the oauth top 10 and really understanding hey did any of this activity actually see through right any of this make it to uh any internal systems uh you have you have to find yourself a

really you have to force this in reality but you kind of have to find yourself in a position where you can uh take output from uh from development teams whether that's you know creating some type of uh of search or report within data dogs specifically for security or within splunk for security uh and then begin to leverage that against activity for the last 30 60 90 days whatever it may be from security perspective again again begin to find those holes right you know you want to understand that like hey we see the start point from this specific ip at this specific time with these specific satisfied with this specific api call on this specific result

and some of this activity is linked or is similar to information we may have from you know a threat intelligence vendor right like this this type of activity is similar here so maybe it's something where we need to dive a bit deeper because it could be an indicator of a malicious actor attempting to poke around or find weaknesses within our uh weaknesses within the app and with our within our systems you know this you know i i wish there's an easier way to do this but a lot of it really is sitting down and talking to them you know bringing your guns to the fight you know bringing your detection capabilities in your logging and

comparing against their what they have and looking for those areas of

correlation right that's a great answer um let's see there's no additional questions right now but there might be some more coming in so um let's see so ask20 said um there's still a little bit of confusion on what shift left is is it um left side of a chart or process a quadrant i i guess just a little bit more clarification on what so really shift and lab is something that came out of uh uh secure development life cycle right something that came out of seoc and it's incorporating security into uh the earlier stages of development um in its simplest term it's really you know incorporating security into the early stages of development but the

the gap essentially is at least that i found is you don't typically have security operations uh a part of that right like that usually lies either with appsec that's that may sit outside of the security program or let's say there isn't a security application security application program though of those responsibilities and those uh those duties uh that that make up of shifting left from code requirements to secure code practices all those different things are kind of handled internally with the application team and with the dev team so what happens is maybe they are following best practices put for put forth by olafs or any or any other or any of the other bodies but as a member of security you're not

totally included into that so when things go wrong you're bought in at the at the end at the end of the stick right you're bought in after there's been an attack or after there's been a breach and you're kind of and then you're forced to sit down with these individuals to understand like hey how does this app work okay cool well have you guys had any vulnerabilities before oh that would have been great to know like a month ago right so you find yourself at the latter end of it versus being kind of looped in at the very beginning hopefully that clarifies it