We are Handling Security the Wrong Way

everybody my name's Brett and I call it so for those of you that don't know me I may seem like kind of quiet but don't worry I'll get work done on that but does it good no me neither so this is basically security we're doing it wrong and there's some funny pictures here my favorites the lower left-hand corner that's antivirus and then this anyway okay so cool so um yeah I'm basically gonna go and those lights over there right I'm gonna go on a rant basically and start talking about why we're doing security the wrong way give a bunch of examples a lot of data to support what I'm saying and then it's kind of more of

a conversational thing so if you disagree with me so my name is Brett I'm the CEO of source project we do open source management so we make sure that you aren't running code that's outdated the guy that talked before me rant guys who talks about handling that stuff and in the cloud and that's what we do so it's previously penetration tester I've been a developer I authored a book called hacking the next generation the next one writing is hacking enterprise and then deep space 9 is following that and then I'm also looking so what are we doing wrong if you start thinking about security and you start thinking about all of the problems that we have today

they're not new they've been around for a long periods of time the majority of these problems come down to in the validation and buffer closure and pop validation across a survey that's argument but sequel injection etc etc so the first question asks yourself is is InfoSec a science or an art right I've been on phone calls with PCI counsel maybe I shouldn't the flavor I'm gonna feet phone calls a PCI counsel when we went through an ast test just basically all the people there PCI certified Sanders you gather throughout the New Testament and at the very end of the test while we're on the phone this kind of guy goes well this isn't exact science

I'm pretty sure it is a science that's okay so I'm gonna make the assumption that it is a science science or should you be using things like scientific method we should also be measuring which we love to use the term securitymetrics but most people don't understand what that is it's like yeah we understand we're going to measure something well you either need to improve it or decrease something etc etc and then also hypothesizing and when you hypothesize you should actually state what I'm about to say is my complete opinion that there's some data that supports it it could be wrong okay and then the one thing that really irritates me is whenever I read any doc document that is

data supported and they use soft language so they'll say oh often most of the time you know most most vulnerabilities out there it's like dude stop saying that like that you're just throwing in added gibberish that doesn't need to be there alright so rule one don't offer opinion if you can't draw conclusions don't guess get more data that's pretty standard science thing right it's like well crap what my hypothesis was the data doesn't support my hypothesis on the rehypothecation uday two sets imma go with it the problem that we really have in the security community is understanding group cause we don't care about root cause and I'll get into some principles that you can actually get into to

understand what root causes that Toyota develop like 100 years ago but you know on the surface whenever you say something like oh yeah there's a security vulnerability we'll have the security vulnerability get in there oh well you know this guy did this or some sorts like okay that's good it's like you're not understanding your cost so the first thing I'm going to pick on is the Verizon data breach report how many people read that or have read it awesome love you is there any worker Verizon so just checking so my favorite part and so I have the 27 I have the 2011 the 2012 is at least yet but in the 2010 there's a nice comment where they

actually say there wasn't a single confirmed intrusion that exploited a patchable vulnerability I want you to think about that for a second and then I want you to think about the security programs that you guys have and you basically can completely get ready for vulnerability management tools you don't even need to use them because there's clearly no data that supports that that's how they're compromising systems so I know that everybody in the room just kind of got shipped where they're like now if that doesn't feel right at all right and that's basically the that's your security guy coming out claiming something is wrong with that data so they have a I know there's somebody that we're surprised in the

room that just one didn't want to miss oh there's also a second part where they actually say based on evidence collected over the last six years verizon wonders if we're going about security programs in the most efficient effective manner which is basically it's all my for me whatever the basically whatever the overarching concept is right they totally made me it was nice train of thought there goes yeah yeah so basically they even state exactly what I said well if that's the case then maybe we're going about security programs the wrong way now here's a here's a screenshot from the actual section so for those of you not familiar with the report it basically breaks down to kind

of core concepts it breaks on data breaches and it breaks down how many how much important stuff was actually pulled how many records were pulled from certain attacks so that's what the red number is and the left is is the the ISIL data breach so out of 38 percent of data breaches which is 86 percent the information that was pulled in in total so this is in the section that's entitled hacking there's also a section entitled malware and so in the hacking that you can see here at the top these are the top three its use of stolen login credentials exploitation of a backdoor or command full channel and sequel injection so the first two things there I would actually

go into an argument of saying that that's actually exploiting patchable things right use of stolen login credentials well how did they get those login credentials and horizon states they got through malware okay well let's ask the next question have the now working on the system and oh well it disappeared and so that's what I mean by we're not taking enough steps to actually understand what the root causes so excuse me so here's the next section of the report where it actually breaks down some terms so they actually say there's a soft language I was talking about using the word often so the malware infection vector isn't is installation or injection by a remote attacker this is

often accomplished through sequel injection or after the attacker has root access on the system okay that's fifty one percent of cases so let's just say it's even let's say that some of that sequel injection which is obviously going to be almost like latest hussam code which can't be patched and then they also have the other section that's basically we know they already have read access on the systems like well how did they get that read axis well that's explained through malware and then they break that down they say drive-by downloads auto executed drive-by downloads was nineteen percent of the cases so those are all infecting zero day vulnerabilities every one of those they obviously don't go into that because you

know their their logs are hard to find etcetera and then they also have nine percent that are user executed so you know you send a message and they're like okay click which they actually do separate from phishing attacks so there's you know somebody goes to a website and 4chan and basically it pops up and says do you want to run this code and they go yes I do and then you know popped it so not only do we have soft language here but we also have these core confident you know core concepts that says although they go back here and they actually say there wasn't a single confirmation of intrusion of an exploitable vulnerability that's kind of

not the case when you start actually breaking down and looking at the data so that's real one well to find the root cause so this is the second point of it so if you can't draw conclusions don't guess get more data understand that route costs it's kind of the same concept right so let's look at the 2011 data breach so in there they actually state that there's five CDs that have been exploited to gain and and deal with breach and pilfer information and so in the report they actually say hackers prefer other vectors or organizations are patching well most likely it's a little of both again soft language most likely okay so you're taking the concept

that basically hackers prefer other vectors the other organizations are patching well well some of those see Dec 2007 I don't know if I would draw the conclusion that organizations are patching well if certain organizations are getting compromised by that but basically because there's only these five CDs that's what it boils down to for those of you that are familiar with the attacker space you know that attackers are just like everybody else they're going to use path of least resistance right and so if they have stolen credentials or they have a way to get drive-by malware on their yeah of course they're not going to go out and start hacking systems that just doesn't make any sense there's easier ways to go

about it they do you know point that out but if you just look at the dates you know it's it's kind of concerning to say the least they also state I have a big blob of data here they also state that we continue to maintain that patching strategies should focus on coverage and consistency rather than raw speed bla bla bla bla bla okay all right so it's been 10 minutes and so now you guys all need to relax your brains for a second so is this anybody's first besides heyy there's a lot of people holy crap there's like 80% of the room awesome you guys do Pilates okay two second prints it don't come to

a conclusion where data supports the opposite so one of the things about the Verizon data they give all numbers it's awesome like they do give a lot of data there it's just a problem of the conclusions that are that are drawn the other thing is is it's only the this will be the fourth year that the fries and data report came out if you guys know anything about statistical data for pieces of data who's not good to support any type of conclusion so anyway that's that's that so let's let's look at some breaches that happened and let's see if the data that we have supports the Verizon data here's the CEO of Heartland he said 13 pieces of malware capitalized

on weaknesses and Microsoft software infiltrated one or more of our network servers okay that wasn't a you know guessing password thing they actually exploited some piece of known code how about this one this is gene Spafford who's very spiffy looking in his tie makes some authoritative so he spoke in front of the congressional congressional testimony and he said Sony was running an outdated and obsolete software on the PlayStation online entertainment networks leaving the systems extremely vulnerable to the kind of attack that subsequently led to breach of over a hundred million customer records okay so we have to now cases which are fairly large I'm pretty sure they're actually the two largest of how the data was how

the data was compromised I couldn't find a picture of Lee Morgan is a he's actually a journalist I actually sent him an email and said hey could I get a picture he's like I don't have one online whatever so basically he stated they simply logged onto part of a group site reserved for credit card customers and substituted their account numbers which appeared in a browser address bar with other numbers so basically when you go to your website right it says account number one two three four five they do one two three four six and push enter and be like oh hey there's more customer data so that was the root cause of the city hack again extremely complicated

attacks these are using right okay um by the way I kind of do this thing so that's my Twitter thing bigger and if you have a question just tweet it and as I talk all all can do it if you want to interrupt me that's fine too okay vulnerability management so first off and this is not vm's this is vulnerability manager's vulnerability managers are not to be used to identify assets okay I talk to people all the time where they're like oh I found a bunch of servers I don't even know I had okay you clearly have bigger problems than just hire you know getting a VM so who who uses vulnerability managers on a

daily basis or weekly basis six people okay I would think that there would be more but okay anytime you ever read a comparative analysis on VMs they always break down into two things it's false positive rates and false negative rates and if you work with VMs all day long you understand this is the biggest pain that you suffer all day long because the whole cycle is I'm gonna scan a bunch of stuff I'm going to get a ton of false positives back when I go through those and see if any of those are actually either exploitable or if they actually even exist and the biggest problem is with backporting which I'll get into in a second the third thing with an

application if you do application VMs the big thing there is is that the spider nobility actually dictates how good the scanner is there's been work by Mike Shima that talks about this and other players the fourth thing that's interesting about IBM is is aidan remediation I'm gonna go to a definition of what a vulnerability manager actually does and it's kind of interesting to read what things they actually are supposed to do versus what they do when you think the word management you would think all of the management associate with the vulnerability so that would mean mean things like assignment to a responsible party aided mitigation etc etc so how do we reduce false positives from a

vulnerability manager well when you off indicated stance which I'm sure all of you that have vm's want to do that because it just makes your life a little bit easier and then backported fixes which is a huge issue so for those of you that don't know Red Hat Debian named any Linux distro that you use they actually do what's called backported fixing so when there is a vulnerability that's reported in SSL and they say okay an SSL actually fixes it the way that you would fix this in old-school ways is you would actually download the new you know binary it download the new source you recompile and you're good to go the problem with doing that is is that they sneak

security patches in with other things and so when you do that you may be using an API that they deprecated and so therefore it breaks your system so to get around that what Red Hat and W and all these other players do is they back port the fix so they'll actually take the snippet of code that's the security line and they'll copy and paste that into what they have and then they'll recompile that and they push that out as binaries the reason this is interesting is because the way that VM is actually do scanning is that they an authenticated scanning and non authenticated scanning they ask the server what version are you running when the server comes back and says I'm

running one three three seven of Apache and then it goes and looks in the lookup table and goes huh 133 seven is five years old and it goes oh you're clearly out of date and so then you know you get that on your desk that some dude ran opium stand you no need to explain why you're running a three-year-old or five-year-old an outdated version of Apache what it actually boils down to is that they don't change the version number they just do a revision number because that's essentially when they took that when they forked the codebase and started back porting all the fixes in well that doesn't help you if you're trying to use a VM to actually identify

vulnerabilities it just kind of I don't know if that's a clear definition what it was identified vulnerabilities but it seems like it so what makes the get me up so I'm Wikipedia there's a huge article about an actually looked at the history was I wanted to see who contributed to if I know it know any of them but there's essentially four things that be a good VM supposed to do so there's supposed to be you vulnerability to severing I would pretty much argue that all of them do that they have highest by false positives false negatives but they do do being you know discovering the next is classification so some of that is kind of interesting because a lot of

the VMS had to start doing classification because of regulatory compliances so with PCI specifically there's some rules in there that basically say if you you know have a vulnerability that's a CDSs a 4.0 or greater than your you can't you're not PCI compliant which is actually kind of yes because if you read the substitutes and it says this excluding things that that our availability bugs so you can basically be completely possible and you're still PCI compliant because you're not pilfering data right so if you start thinking about other things complicate constipation that was like things like unique prioritization do you actually have the ability in your vulnerability manager to say this is actually a more severe bug for us

personally then it would be for anyone else so we want to update this right we don't think this high attend or major backwards this is a little one right so most of the things that I seen they don't have the ability to do that third thing remediation so dou viens actually help with the remediation process typically they'll of you like hey you're running Apache 1 3 3 7 update to the newest version of Apache if you're running Red Hat that clearly is not gonna help you at all right you're like oh okay I go download and make source I'm gonna pop that in so remediation is kind of a you know I would say that they

do give some some tips but not not for the whole before business mitigation so mitigation with the way that it's defined is is finding the root cause of the problem and addressing it right and if anybody wants to works for a vm and they want to come about how they actually do that I would love to hear it but just that that doesn't support it so clearly vulnerability Mannesmann isn't helping that's a huge step that I just made so hopefully you guys caught my little joke there how do you find root causes of problems so this is a methodology that the guy that used to run Toyota back in the 30s of what he said so when they did

things on the line they would actually ask five lies so the first thing would be like the vehicle won't start that's the problem so of course it's a good manager you go why and then they would say the battery's dead now in security land that's where we stop there's still great replace the battery lids in the bunk right oh that's an issue like that in everybody in this room should really understand that that is an issue so then you go okay well you asked again so the battery said why well the alternator is not functioning okay that's that's still good we're getting a little bit more data so that's the second one third white so why is the alternator not

functioning well the alternator belts broken okay well we're doing better because replacing the battery obviously wouldn't address the root cause and therefore we could just keep replacing batteries now insecurely and we love doing that right we're like oh he patched it cool hey there's no vulnerability let's patch it let's just keep patching okay I love passion so say then you know okay the alternator but alternator belt is broken why is that well the alternator belt was well beyond its useful service life and not replaced okay Clark they're getting clearly getting a little bit more information so why is that well the vehicle is not maintained according to recommended service schedule perfect so now we actually have the root reason

right the root reason isn't the battery just happened to fail the reason is that the person didn't service the car right and so we can kind of use the same mentality with with with vulnerabilities and security in general is that when you actually get in a room you start trying to understand these your causes just keep getting what you sooner or later it's going to be the developers are stupid that's the reason why now we love saying that because it makes us feel like we're God's gift so we love saying oh it's the developers problem and it's their responsibility to fix it and blah blah blah blah blah and so there's been a lot of things dealing

with security programs of let's educate developers so in my last gig that I had that was actually my task so I can we have a long conversation about that if you want yeah audience participation so who has a security program actually let's ask it the other way who doesn't have a program booyah that's funny I actually saw some people to raise their hands that work for security companies I'm not pointing you out so does it consist of running so those those of you that do have a security program is there anybody that their security program doesn't actually they don't they're not responsible for running a VM anyone okay so if that's the case we have this concept right now

where then it's like okay let's run at the end so who also hires pen testers okay it's quite a bit okay so I'll come back around at the pen testing in a second so rule 3 stop trying to Solomon possible problems there was the last talk that was in here there was a couple of the times and they said this is a problem looking to solve right I want you guys to all understand that if you're in security you will never solve a security problem like you will you'll figure something out you'll patch it but you're not going to solve the problem of like security is gonna go away ok it's just not gonna be there so

if you take that in a classic developer land and you start thinking about okay well what's that you know what's a good metaphor for that I would think it like P versus NP right we're basically trying to solve P versus NP as security people and there's people are like yeah I'm sure that some time you may figure it out but we again going back to God's gift right we this mentality of we can solve it we're smart we'll figure it out so one issue is is that security product programs are too complex my best example is who actually uses the Microsoft dusty dusty LC or is there program based on SDLC Wow ok oh there you go ok

if you read the SDLC documentation my favorite parts they get like you know there's like nine different things and working into the development cycle and blah blah blah blah blah and then when you get to the transient they go every single one of these steps is optional I hate to tell you but if you give that to somebody and they go in there they look at it they're you know let's implement this and they just give it to you none of that is gonna actually sit down thank you for training flights none of that's actually gonna sit down and they're not going to say this is a piece that actually do we can directly this will

help us right we can use this let's do that and then if you do implement a security program are you actually measuring things right that's one of the hardest pieces in security is an in general is when you want to when you want to do something what you actually measure right it's pretty simple if I cut a board let's measure length yeah measure twice cut once right and at security land we kind of go okay how many vulnerabilities do we have open 15 cool let's close those hey how many do we have now we have one suite we did good right it's like well again the root cause ality all that stuff you don't even solve anything

also remember vulnerabilities will always exist I'm sick of hearing about like technologies and there's a huge conference going on like over there where they sell products that solves security issues and you basically just plug one of these things then you don't have any security problems ever again right now to you and I we kind of laugh about that but I'm not kidding you there's people over there right now and making purchasing decisions that that's what they think that's going to happen they think they're gonna buy a box they're gonna go to the security team and then with that money they do it and the way you can tell this is is because after you do it and nothing has changed

or you've only made a small reduction and stuff they'll come to you and say well I bought that box for you it's a quarter million dollars what are we spend the money for right and it's it looks that if you actually say well that was your purchasing decision not mine I don't like protip so here's a true story I'm old enough to remember whenever the argument and it still used pretty much the term hacker right if you guys remember getting arguments does either older like me just kind of everybody in the room I think so you guys remember having conversations about with what a hacker actually is vo it's getting like I it's about any like no

it's not hamper is about like mocked by technology to do something that's not originally intended supposed to do right and then we'd still read like media journals but it's like hackers broke into bubble um like I'm so angry like people I'm gonna keep ranting about this so as security people we thought that that was kind of almost a solvable things like we thought hey if I tell enough people about this they will adopt that as a practice and they will go ahead and go ahead and start doing it right now if you write straight and start think about this you know who actually saw this problem as developers and specifically Facebook when they started saying things like oh well we

use the hacker mentality let's as break any computer systems no here's what the heck we're in Saudi is in the number is all rock on the top that's what a hacker is you know and all security folks even know that we're trying to solve that for years that clearly doesn't work alright quick check dear developers code securely did you guys teach me anything I mean come on fine even where your company a while all right like teach them to code securely it's not hard right that's like huge programs dedicated this stuff so there's great there's a great developer conference that happened and I got some stage and start starts talking about developers and security and starts talking about do

developers actually write good code right and sorry he said how many people think that they write good code bunch of people raise their hands right he does okay great you guys I'm gonna ask you a question saying that you were gonna get on an airplane and you're gonna fly from Los Angeles to New York if your development team wrote the software that that plane ran on do you feel comfortable flying that plane everybody's hand window now what scares me the most is I'm sure Boeing was in that room Dave don't you work for Boeing don't worry about that so you know that's kind of somebody think about and when you get on a plane next time

realize that the guy that was the product manager for that plane he probably thought the same thing right all right so what this boils down to is is incentives okay developers are not incentive incentive to code securely that's not what they're doing and it's our responsibility to actually make sure that their code is secure right if you do not have something like partner programming or code review just even from another developer a security person has not looked at but think about it from a security standpoint when they do it you're losing the battle like that's definitely something that's needed um if you think about what developers are a Centon for incentive for their incentive to push out new features so that the

sales guys can sell more product that's it and in fact if you really start thinking about it think about your individual company how are you incentive you're typically incentive by what the sales cycles of like how much money is the company bringing ok I'm gonna get back to that in a second you guys start thinking about how can how could how do i incent them to do that ok so security programs are not mature we all love to say oh I have a security program really how long about four years and we great metrics everything something I've got school dashboard like I bought it and it shows me behind go down so something is clearly working

so first up realized security is always an afterthought and the way you can really start thinking about this is is think of how many people have become wealthy by doing a start-up they were the first security engineer at that started Facebook just made a lot of people rich or will make a lot of people rich I guess I can tell you that there may be one security guy and that's a fluke okay get used to it like security is always an afterthought you will never go into a company and into some extent that's actually good for us because it's job security right here's the Metasploit example um if I can compromise your systems of Metasploit you do not have a security

program I don't care like what you do your your Metasploit using vulnerabilities that are known about you're not patching that stuff okay I [Music] heard it the last guy that presented Rand said I was required to put a cat photo in there and I laughed I was like [ __ ] so a PT's so I'm sure that I don't have to go on a huge rant about a PT's because everybody in this room I'm sure has a feeling about what it is so first off the term advanced okay does it mean originally know where a bt came from that's right well you kind of it's a military term that is specifically reserved for state-sponsored attacks right so it's

like China got mentioned anytime somebody says a PT by the way RSA that was an apt attack again to you huh that's around what apt apt I don't see any lifting drink so it is past noon so cloud the cloud stuff but the last talk would have made me really messed up here's the thing that really frustrates me about apt if you start thinking about what a PT is okay great a PT is in the classic sense of the word it's a state sponsor attack let's throw that out and just say it's it's a supremely advanced attack that I would probably wager maybe one person this room could probably pull off like that's an actual apt okay the problem is when

you start reading the data of what people are saying goes maybe to attack like really so what do they do oh we had a phishing email and there was like a link to that to the zero day long that's an apt attack like no no no that's not an apt attack okay so the best for out the term apt and the reason you see it all the time is because it's the best excuse ever let's think about it right if the real reason you got breached is because your vulnerability program you're like well we ran stabbing stuff against it and it didn't detect anything you know like okay well what about that server all that was really vulnerable don't worry

about that apt that's a constant right it's a very very easy get out and you can see more and more and more and more this up the other interesting thing about apt attacks if you look at the cycle of who actually detects them they're typically security companies they're not incentivized to like make you fear anything right all right so let's let's talk a little bit about threats and then what they're addressed by so this is kind of a very security program 101 I'm trying to get things down to like the route dogma I think was like Calvin or somebody and like which I take all the religions and then well all down the dogma right so I'm trying to do

the same thing so there's basically three threats right you have known threats you have unknown threats you have custom threats so a known threat is is that there's a vulnerability out there that exists all of us can go to nvd and type in the CBE and they'll give us some data maybe we go over to pack a storm security we download a you know the exploit and you know keep hitting it right okay great those are known things security program should definitely address those that's something that your secret program should just address second things unknowns attacks right so these are like zero days etc etc so mitigating technology should kind of prevent the majority of those now

realize again you're never going to solve everything so there's the one gets in I get four there's a bunch vendors out there to go look at all the there's one here that sponsored if you go look at their data sheet it actually says prevent zero days leave you guys to figure out how they do that it's magic the third thing is is custom so this is actually stuff where when you think of classic penetration testing so I'm an X pen tester I do not classify myself as a pen tester the way that I look at people that that are pen testers right if you can you know sit there and you start looking at servers and let's say that

there's nothing that's available to you that's known and there's only unknown stuff can you actually sit there and break into that computer and if you can then you are actually a pen tester like that's in my opinion what an actual penetration tester does so the first thing is we need to identify vulnerabilities etc etc so I've kind of gone through that already make sure nobody's paints me yet hey these are just quotes just make sure nobody oh wait there's a question that big dead comma somebody's gone like screw you Brett I think that kant's someone asked if your dev team wrote code for her oh sorry that's just code okay let's get a new one I'm never gonna be able to get

you guys through that now okay so if you want to oh I totally skipped it yeah yeah penetration testing okay so penetration testing first off is not a security process okay if you hire a pen tester and they only work a week they're not doing much okay they're gonna find some new threads they're gonna write up but are pretty reporting they're gonna give you deliverable and I'm gonna check you $37,000 for okay penetration testing should only be used after having a security process if you are not running like a vulnerability manager or some type of vulnerability discovery tool and if you do that and installs finding things then you shouldn't hire a pen tester like you need to get back down to that

thing and you need to figure out how do we tighten this up how do we prevent this stuff before we go hyah pen tester to do it and the other thing too isn't and you do that the penetration tester you're gonna get the most bang for your buck because they're not going to find a little hanging fruit and they're gonna be like all on the chakra system go get crunk right they're gonna actually sit there in there and have to work at it there's nothing worse than being on an engagement where you can't find anything good you start looking at little things like oh you don't set that cookie set secure let's really high vulnerability let me

explain why I used to actually joke that the you can always tell a good pen tester by how many low findings they actually have because the low finding is this my stuff they know and they just have in their back pocket there's like yeah yeah I didn't find maybe they actually did you know you said secure cookies SSL bubble bubble so if you if you disagree with me and you start say you know a pen it taste testing no it's part of our security process I want you to think how much of the threat surface does a penetration tester actually cover let's say the pen tester works for an entire month right is it 50% are they

covering 25% of the surface it's I think it's probably below 1% right they're gonna identify sup they're gonna go after it they're gonna you know do some clever google searching to try to find something they're not gonna write custom code for you and if they do it see again it's still at the 1% level okay all right so [Music] interesting so there is so I just I'm gonna start up now right so company down there so there's this interesting concept called the lean principle that this guy named Eric rice wrote and I think security is one of those sciences where you can actually learn a lot from other different thank you so it's a vertical we learned from these other

verticals so yeah other disciplines thanks so basically the lean principle is this concept of build something very very small and it's called a minimal Viable Product right what's the smallest thing that you can actually sell to somebody that they are willing to pay for and identify that and they just sell a crap out of it right and the way that you do this is is that you do something you measure it and then you kind of get rate and you keep doing that the reason this is so interesting is is that a friend of mine has a very successful startup and what he did is is to actually validate the idea he had a page

just a splash page that said Bobo will block this is what the product does right didn't like one sentence said sign up or a pricing right see pricing when you clicked it brought you to a pages at all you can will early enter your email address right and the reason why is Eric price actually has this concept that says the minimal Viable Product is much more minimal than you actually think it is so he his concept was I'm not gonna build anything I'm just gonna test this theory right so you got like 300 signups from that he's like okay he goes okay well there's kind of a demand now let me see if the people really in pay for it so what he

did is he actually stuck a page in the middle and he said that's miss Pawnee stuck the page in the middle and said when you clicked pricing and signup the next phase of popped up said what plan do you want free plan $15 a month $30 a month $99 a month right when you click that there we go Oh too early visitor email address right but if you think about what he's doing right he's taking these small littering steps he's figuring out exactly what people need and so what he figured out is is that just from that little test he figured out the 5% of people that actually came to be willing to pay for it right so he can

easily figure this out he's like okay 5% if I get a thousand signups I you know have 50 people and 50 people paying $20 but he just breaks it out right and so he goes okay well now it's worth building a little product so he builds a little product which is out there and then just keeps going right so security we can learn from this okay the Saleen principle of Minimum Viable products like what's your Minimum Viable security offering right so the first thing you have to understand is you have to understand what to measure and this is probably the hardest thing so in start-up land it's kind of easy because it's like okay signups right then you

figured out science he goes okay I'll change my measurement now to people who are willing to pay right security when you do the same thing we need to identify something to measure maybe it's maybe the first step is just how many how many vulnerabilities does the vulnerability Center find and how can we reduce that number okay so then you figure out what you wanna measure you measure it and then you do some stuff and then you check your metric you say okay and then it did it improve right did it increase or decrease whatever your hypothesis was what would happen if the answer is yes great continue doing stuff right what you're doing is clearly

working so let's keep doing stuff right if it doesn't work then you have the wrong metric you need to do something else you need to figure out some other metric that you can actually measure so yeah so basically fail fast iterate fast these are these are concepts that the startup community has and I think that we should adopt them as security people you know and this is not like oh I'm gonna take six months to do this it's like in a week see if you can actually reduce something right if it's by five or six bones whatever so when coming up with a security process take very very small steps right maybe the first step

is just code review right see if just by doing that does sins according scary identify stuff right can you use your VMs security is cultural and unfortunately that is completely it's very different from what I was saying about you're never going to get a security guy barrier company right if you think about that well if you don't have a security guy then you can't have cold fresh security right because there's nobody from the beginning saying we need to make sure we have unit tests we need to make sure we have you know integration tests we need to make sure we have security in place there is none of that so the time the security guy gets there it's very hard

to read to find the culture around you right around security 3/8 don't teach developers security ok we've been saying for a long time we need to teach developers how Haute's thoroughly that does not work it go measure it I'm telling you your developers are not coding any more secure than they were before you got there ok and if they are they're doing it to check the stats those who that watch the wire know what I'm talking about don't you the stats ok so yeah ok here's a very simple thing blank developers make blank products security secure you know security dollars make secure products I don't know arrogance secure developer ace ok but you know kickass developers make you guess

products right this is very very simple so just kind of take that and go with them all right rule 4 don't pretend you're something you're not that goes back to God's gift right it's God's gift over there so don't pretend you're something you're not so we love to basically say like I'm solving these problems and doing all this stuff right so don't get frustrated so 62% of financial services industry thanks that time to market and the need to release products with shorter development cycles was their number one issue that came from Coverity Coverity the entire job is to basically like make sure that there's no problems in code okay if that's what financial services company who basically

it breached like all day long just go like data loss DB like right understand security is a call center it's very very hard for us to identify what metric explains why we need to be there right the only way that they can do hits go well we had a four billion dollar data breach how much do we spend on security we spend 130 million okay so now security disgusts me point one three billion dollars so the simple exercise you can use is that if you have these people on your team who would it be the first one you get fired that's the fastest way to figure out whether you know you're a call center or

whether you actually create revenue so security increases expenses developers increase profits executives increase profits sales increase profits business development increase profits I if someone want to argue with me why securities increase profits I'd love to hear it you're not oh there's one guy that raised his hand they do costs money but they they have it's very easy to correlate what they cost versus what the impact they can do the fastest thing if you actually want to figure this out just start a startup and you'll be like I need to spend $150,000 in the developer am i able to my going to be able to get more than $150,000 in sales from higher than this person answers is pretty much yes so

that's it I have nine minutes left for questions so are there any other debating on whether I should is not present and we can all take naps after lunch when they stop we'll be in that beat I'm sorry yeah I have a hard time with that kind of stuff is that no one to stop yeah one of the action items so there's basically three takeaways so the first is is that hopefully you'll go back and actually look at your security process and understand what parts that you can cut out like if there's something that actually is process and that's really all it is you need to remove it right okay the second thing what are you measuring like if you have

a security process the first thing in that security process should be these are the three things for measuring right we're measuring this business and also should not be 50 items it needs to be something very simple in start-up land there's one measurement right Facebook for instance the only thing that they measure is daily users how many people come back on a daily basis that's the only thing they care about so anything that will drive people to go back more on a daily basis they will do right so you need to kind of find your key men that you need to use and that's unfortunately I can't give you what those art because it's very unique to

each organization so that's one two three is vulnerability scanners suck [Music]

as a vulnerability scanner so the question is is there anything that I like as a vulnerability scanner I think the vulnerability scanners the initial thought of them whenever people developed them I think that they had very good reasons as to why they wanted to do it right I don't know if I have ulnar abilities that sounds great right at some point though what ends up happening is they lose that that startup mentality right which is kind of pleasing the customer they don't care anymore they just once they get once they start getting a certain amount of profits in the door and then it's basically just how do we scale this out right they don't need to do anything and

the reason that that's justify that's whenever I asked how many of you guys use vulnerability scanners or vulnerability management tools everybody uses wonderfully management tool right so they clearly have a huge amount of money that they can make for you guys and you guys will still buy it even that they suck it doesn't matter you guys will still buy it yeah

[Music] Oh you should mention names to God

the product is at least as pie [Music] I the problem is is that even if you implement the framework that any of them suggest its back-end we're going back to the process it's almost too much overhead right because if the majority of my day and those that actually do this as a daily basis how much of your time is actually sitting there figuring out what's false positive yeah it's a huge it's a huge portion of your time is wasted to that so you know there is a very very good framework the majority of your time is just sitting there doing essentially what the VM should be handling a lot for you now the argument from the VM standpoint is is that well

we'd rather have false positives and false negatives I think that's an easy cop-out

[Music]

but so yeah so the question is is what the state home was is that most people would actually run the ball if they could what do you think penetration testers do

[Music] right so

what counts are still extremely low hanging fruit on the other thing about a penetration tester if you guys are hot if you guys are having penetration testers it depends like what you said about well hiring the penetration tester depends on what the fence Tyson tester you're hiring is if they're not in a trading then they're not doing what they're supposed to be doing right I'm sure there's some clever sexual metaphor that I can use here but I'm not going to

they of course yeah oh no no I think that I think the penetration tester should definitely be used but I think that a lot of a lot of companies actually use penetration testers way too early they don't have a good a very basic security policy in place before they actually hire a penetration tester yeah that's right see you oh yeah seatbelt like a seatbelt could kill you like their their reported accidents were like if he wasn't wearing a seatbelt he would serve watch the floor I'm not wearing my seatbelt yes you're totally right but in 80% ne+ percent cases if you're not wearing your seatbelt you're gonna go through the windshield yeah I don't know I'm just

here to point things out and cause problems I don't come up with solutions no which is basically what it boils down to that question may be nervous um what it basically boils down to is is that's your responsibility like as security guys it's our responsibility to identify problems in their codebase and do things things like parallel programming I should think that Yahoo I don't know if I can talk about how whose process but I think it's actually very very smart because that you know in the groups in different developer groups one of the developers is actually considered like a paranoid like a local person that does the security for them and they'll look at everybody's code from the security

mindset I think just having something like that very basic in place actually helps out a lot I had no idea cuz they're nursing Yahoo hose metrics there's people I could talk to afterwards they tell me that but I think that's actually an extremely interesting concept of taking a developer and saying it's your responsibility to look at the code base and then having somebody else feel like the performance engineer but it's their job to look at the code from a performance standpoint right I think that that's a very smart way to go about doing it [Music] doesn't require what of training developers you'd be surprised how many developers actually understand very very basic security things and pretty much again going back

to my point there's pretty much one thing that you need to do as a security person and that's to make sure your data is validated this is valid like that's all some 95% your problems

yeah

that's right ya know it could be

it's it may be a little above so the first thing is is whenever you choose to pick a metric don't shoot like I'm sure that Facebook did not start off going like we need to see how many users are going to use us on a daily basis that is not their first metric they use their first met request let's hit tons of customers every single day let's just do that and then as overtime they kind of changed let the metric is that's important right which makes sense you know you're in unfortunately our metrics are never revenue increase in revenue so yeah you're right you need to evaluate am i measuring the right thing or is

what I actually did it wrong let me try something else typically the the problem with doing that is whenever you identify a metric there is a very very clear thing of what you need to do in order to get that metric lower or increase or whatever and so you typically all may only have one idea right again I'm using soft language I don't know if you caught that typically most of the time there's much other things yeah

[Music] yeah that's true yeah at some point all right so seven seconds thanks guys actually it looks like the clock I have ten minutes is that right oh stop now [Applause]