BSides Delaware 2016 Day 2

for

because it's going to be an extremely exciting talk uh W thanks everyone for showing up on a Saturday morning uh oh see I was right there there was the coffee there was a coffee line uh thanks everyone for showing up on Saturday morning Bri and bushy tailed as always and I hope everyone is having a great conference uh the talks yesterday were great I find it interesting that uh when I did this last year uh every talk seemed to be technical technical Technical and this year but it's about 50/50 half technical half uh management which is obviously the side of the house that I'm falling on um so today we're talk about being a being a ciso what does it take to get

that get that ranking get into that stature and basically be the head person in charge of security for an entire company organization it's a little bit about myself been doing infc for about 16 years now uh started off remember Y2K yeah that's that's when I that's when I started uh funny I just saw that referen in a movie and I thought geez how old is this movie and I realized it was something I'd seen in the theaters uh currently ciso for an auto finance company so if anyone sees the used car dealership say we Finance everyone that's us in the background giving that giving that loan to to people uh I also teach here in the infos

SEC uh graduate program so if anyone wants to get their masters in information security uh come by come and talk to me about it and I can tell you about a little about the course um if you want actually get in then you got to talk to admissions obviously for I work for Comcast the most Lov beloved company in the world I always told people um they actually do a really good job uh it you only notice it when it doesn't work it's kind of like breathing you know you only notice it when it's when it's not working uh before I worked for HSBC which is a very large International Bank and again so I've uh worked in some Fair large

organizations um in case that's not enough for you I got a bunch of letters after my name I'm a certification junkie only because I I love doing it and uh it keeps me up to date forces me to keep learning new new stuff uh to keep up with those certifications all right I love these mems so information security this is what you know people think about when they think about information security this is what people think of themselves when think about doing their jobs in information security um and to me it it wraps up a fairly uh you know in in one slide what the perception for fa security is um so they got update a

little bit that show their reference in what people think I do what my friends think I do has been canceled for like four years now um so we have to update that with something newer um I'd probably put Mr Robot in there for what my friends think I do because they they should see that and they go do you do that kind of stuff and I go yeah I because it's a lot easier to say that than trying to explain what I actually do um so the other Ms there and again what I actually do is mostly what we'll talk about today it's hurting cats so going back to 2000 when I started information security this was literally

the entire information security world uh you had your network you had your host you had your application you had your data that was it that was the the only only four things you had to worry about obviously the it's grown a lot since then but just to show you how far it has come um literally this this was the entire infc world uh just not not too long ago then the capability maturity model came along has anyone come across this before in their in their uh their dealings with it or dealings with uh regulatory bodies often references a lot as well so they put together some structure around what does a mature organization look like uh so you start off with the

kado at hoc so this is your if you think of like a startup company and You' got five guys doing everything someone has to be doing everything that that company needs to do so that's uh it HR payroll everything else and they're doing it in ad hoc manner because they're just flying by the seat of their pants get to level two and it's a repeatable process okay Dave is in charge of it now uh Joe is in charge of HR that they assign people specific jobs they're they come up with some strategy around hey every time a new person comes on we're going to do this this this and this every time we build a new a new

widget we're going to follow this process so it becomes a a repeatable process and now you can start saying well where do we go wrong in the process and why is or do we need to fix the process and do something different uh defined so it's uh defined and documented so it's not just in Dave's head every time someone comes on it's actually written down somewhere so if Dave leaves and we and someone else takes over for them they've got a manual they can follow to say hey let's do it this way uh now we made make improvements to the process we've got some sort of manual we can follow then we get to managed so not only are

you following the manual you've got a manual but you're checking to see if people are following the manual uh you're checking to see that every widget that goes out the door is is is a quality widget uh anyone watch uh the profit on CNBC uh this guy named Marcus lonus goes in buys pieces of companies and fixes their process and so that that's what he's doing is bringing them up to a managed security or sorry a managed level of maturity so these businesses can run properly it security is no different we we all we have widgets that we create everyone's got users they have to create users they have to manage everyone's got you know

uh new applications they have to bring on board everyone's got security requirements they have to fulfill everyone's got government regulations and requirements they have to fill you can PL you can plot all those on your maturity model and figure out how do you move them up up the maturity model chain do you have to move everything to a five no there's be certain proces you're gonna say you know what we're at a defined State and that's as far as we want to go with it we don't need to go any further but other other processes you want to and procedures you want to put in you want to move up that move up that chain is optimized so we've got a manual

we're following the manual we're quality checking against the manual we're making sure the right things go out the door and then we're looking how to how to improve that process on a continuous basis uh if anyone's done any Six Sigma training that's all this is about is how do we get to five it's six Sig that's 5 9 so 99.9 99.999% good so 99.99% of your products go out the door in a good in a good way you said to achieve that that Six Sigma uh level of maturity all right now we're saying how's this apply to infosec well we've adapt that same model for information security so at the the bottom left here we're reactive and manual

so again if if you've uh working in a shop that again every time that someone new comes on or new system gets brought on you bring it up in a different way you're you're probably in the reactive and manual mode uh tools based okay now we're not just there we're not just informace secure people standing there at at standing there at the front desk going hey yep yep come on in yep you're allowed to come in you got should got some tools in place that you can use to monitor the monitor your information monitor your networks monitor your people uh and you're you're you've got some something to to work on to work with uh integrated picture means now I'm

taking all those tools and all the data I'm getting from that bringing that together and saying well overall what's my risk posture what's my security posture so I may be very good at the perimeter I've got my firewalls I've got my fil web filtering but at the lower end in the uh user space I've got nothing on the the end points so I have no idea what's going on with my end points um so I I've got may have a a good tool and a good maturity model on one end and and not so great on on another sector uh next is dynamic defense so again if you're using Tools in a productive and uh less reactive more uh

proactive manner you're you're in the the four maturity model uh next is resilient so if you come under attack you can quickly switch over to something else you can uh spin up alternate sites you can use your cloudbased provider and switch over from the London to the Hong Kong Data Center and it's as if nothing happened it's a it's a blip on the on the radar you know it's also in this chart uh on the right hand side here we've got our conventional threats that'll be taken care of if you're at level three all up to nation states at level five um I don't know about your organizations but you're rarely going to come across you

know the nation state actors trying to break into your systems uh it's kind of like worrying about are you going to encounter a Ninja on the street I'm sure there are ninjas out there and that they're walking around but how whether your chance of actually encountering one just as you're walking around and have to actually fight them pretty low now if you're in government agency or or something like that you might that risk might go up but a lot of people prepare for that nation state and it just never never happens uh so take your risks at your at face value if you are risk for nation state you should be able to counter nation state attack but if

you're you know got a 1% chance of having a nation state come after you um that I would say you probably don't need to necessarily go to that level of maturity so when you're when you're done assessing yourself you can figure out where you fall in the Spectrum uh and then as as the ciso you kind can uh figure out you know which kind of Captain you want to be um for those who are Tre treky fans you know he always old versus the Next Generation Um I like them both so that's why I use this slide but if you if you think about the the two captains the uh Kirk was the the startup he was the one that sort of flew

by a seat of his pants didn't follow the rules all the time when it wasn't convenient for him he have a card in the outs side he was by the book uh always followed the rules um but he had a more if you look at the you know the the um worlds they lived in uh Kirk was the startup card had the the mature organization they that's the kind of Captain that they needed at the time so as you uh mature as a ciso you may start off as as one and mature into into the into the second one and your company as you grow with the company as the company grows you may mature uh in in into one

or the other as well all right so what you actually came here to talk about so about five years ago uh um and he's got a really long name it's always it's down at the bottom here uh freak and I'll I'll blow up his name put together a mind map of if you took the entire ceso position and broke down into domains what would that look like this this is what they came up with I know it's a little small on the on the screen here um I if anyone wants the the slides obviously we'll have them available to you um or if you just do a Google for ciso mind map you'll see something like you'll see

this uh this model come up and this is the the fifth version of it so when I saw this this is when the light bulb went on for me to say hey this is what the ceso job actually is uh I was a uh up until that point I was a you know pure techie um I was great with application security I was great with response I was great with forensics and Pen testing but I didn't have all everything on this on this picture and it's basically I was basically a one-sided one-dimensional um W wannabe manager ciso once I saw this and started broadening out my my horizons and my my skills and my uh my focus that's when I

moved from being a being techie to jumping over the ceso side um then go to the the seeso talk yesterday I was at the the end of the day I didn't I didn't make it uh how how was how was their presentation was it good stuff did someone even go to it uh what what kind of things they emphasize on on

theirs

okay right okay okay all right good I wish I had gone I could have probably stolen half his presentation and and made mine better that's all right um so I'll I'll take it from the the if he that they address mostly on the the education side I'll say yes it's a big part of it um but I'll we'll take a look at it from the different domains that you can that you need to be aware of at least um and that way you can again transition yourself from the technical side to the more on the management side all right so management is probably about 35% of of my day uh and again this this is the part where

in people think well I'm the best uh firewall person out there I'm the best INF security technician out there transitioning over to the to the to a management side uh should be easy right well that's that's not the case uh as the person who talked about being a So Yesterday U management is going to be a large portion of your day um I've got labeled heals here is 35% um that's probably a low estimate for a given given week uh so for the most part this could be anywhere from 50 to 60% of my time if I'm trying if you know depending upon the what we're actually working on so our first thing we have to always worry about inface

secur is budget it uh sorry information security as as part of it is always going to be a cost center what I mean by cost center is I'm never going to say well give me $100,000 for my uh whizbang tool and I'm going to make $200,000 on that on that on that investment no give me 100,000 in for my tool and I'm GNA be able to give you a good compliance report if you're a business person do I care about the compliance report more than my $100,000 I've invested in in in the in security tool they're going to say what do you nuts if it was on a pure return investment basis uh in some cases I've actually

seen where they said well I can either buy the $100,000 tool or I can pay $50,000 in fines I'd rather pay 50,000 in fines and just skip the tool so you have to you're constantly fighting for Budget uh because again there's there's no way or very few times when I can say yes give me this money I'm going to make more money for you um I can approve efficiencies so I can say well if I buy this $100,000 tool will get people on boarded in two days instead of two weeks so if I can then take the uh extra 12 days that person is productive and say well we pay them uh you know each person

makes $100 an hour well now I've just made you $12,000 oh okay that's great the business people like that so budget is something that's going to be uh as a as a ceso something that's on your mind constantly and you're constantly fighting for all right business enablement is inface secury the Department of no or the department of no but you want to be the department of no but no you can't do it the way you want to do it but here's the way you can do it in a secure manner oh just open up the FTP to everyone who cares if they can who cares if other people download it it's not worth anything to them no

but let's do a a secure FTP and only let certain people download stuff um that again you run into that all the time where business just wants it now now now fast fast fast and you have to pump the brakes a little bit bit and say you can do it but here's the secure way to do it uh so merg and Acquisitions again if you take over another company what's their security status uh What uh bugs are hiding under the rocks that when you pick that rock up and you say oh my gosh they're doing some some things very wrong in a in a in a bad way uh Cloud everyone's mooved to the cloud right a lot a lot of places are

how do you secure that how do you make sure that the uh servers I'm putting up in Rackspace or in AWS or in Azure are secure mobile we have more problems with mobile than than anything else because people are are using their own devices what do you mean you want me to put airw on my personal device it's my device yeah but you're accessing my my M meaning the company's data so if you can access the data it now is not longer your your device it is a hybrid company/ your device so how how do you manage the the mobile uh business it process development yeah so the if business come to my business comes to me all the time

say well I want to do X I want to uh bring everyone on to this new application and you don't have to do anything because it's all hosted somewhere else they take care of the admin they take care of the logins all you have to do is provide them a access through a web browser okay well how do I make sure the right people are going into it how I make sure the right people are getting taken out of it what data they going to have access to what controls around that data how do I know where that data is going is being is that website being hosted uh somewhere outside the US I have contracts with companies that say

you will not host our data outside the US okay so now I have to make sure that those those applications stay within the US borders again all these things that should be easy and and quick to do aren't because I need to make sure that people don't shoot themselves in the foot it's okay if I shoot you in the foot to stop you from doing dumb stuff but if you shoot yourself in the foot it's going to be my problem later on down the road uh project delivery life cycle everyone loves projects right projects are fun projects is where everything gets done yeah got to make sure those projects get delivered on time on budget

and get deliver the uh requirements that were put in the project uh security architecture well throw it all together and it just works right nope you got to design it from the from the ground up you got to make sure it's moving the direction that you want it to go uh there's a lot things that go into again man heading the ship that you have to make sure that you're make you you are the person in charge making sure that's happening uh compliance audits always fun any ever been audited before by yeah I see most of the hands going up and I see the rest of you sweating when I mention the word auditor uh so

everyone's gone through these again I'm I'm in charge of making sure those happen and are done on time and that they meet the requirements that the uh auditor has put in place uh legal and Human Resources we never interact with them right nope interact with them all the time why oh I had a breach uh of someone's data it was even if it was five people well I I have to send my legal team to go figure out where those five people live and what the breach notification laws are are for each one of those States because there's 40 different uh breach notification laws in different states around the around the US so they live in uh Utah I don't think

has one I don't have to tell them anything I would probably would tell them encourage them to do tell them as a courtesy and as a good corporate citizen but by law I'm not required to uh in some states there's a there's a time limit I have to notify them within 30 days um in some cases it's as expeditiously as possible I have to be at least be aware of these things so when I go and tell talk to my legal team I can say look I know there's a law here you have to tell me what what we have to do about it but I know there's there's there's some parameters here we need to to

fulfill uh Resource Management people and technology so it just runs on robots right we don't have to worry about people no anyone have any robots in their it teams no me neither so it's people processing technology uh the reason why you doing that order is because the people who come first you have to have a good process the technology supports the people in the process so I have to deal with uh vacation sick time Joe's not showing up on time uh Dave is working at two o'clock in the morning all the time uh somebody screwed up and shut something off they shouldn't have um somebody else somebody the uh CEO's laptop is infected with malware for the

third time this month um how do how do I deal with all that and making sure that the everything is is working in harmony uh selling info SEC internally all right who's ever gone to a meeting with business people and they say well I don't believe what you're telling me about information security and you look at them and say well I don't believe what you're telling me about your sales numbers and they go well I've got data and I say so do I it's and you know you can see where this is going um now are the the meetings ever actually like that no but we we're constantly reinforcing with our business partners and our and the business folks

that we support to tell them hey this really is a problem this is not something that happens to everyone else but us this happens to us we need to make sure that we're in compliance we're make sure that our our systems are secure that there our people are secure uh so that everything again works works in harmony so that you can go out and make money to support my job always remember that I don't make any money for my company I like like we said before I'm a cost center they make the money so I got to make sure they can make the money securely so that later on there's not some government uh agency lawyer uh

whoever suing us saying give me that money back plus some more um we want to make sure that we're that we're doing things in a secure manner all right govern's risk and compliance boy that's a long title I've got uh someone on my staff uh she's a senior risk analyst for governance risk and compliance her name tag on her on her door is like that long because she's got a lot of a lot of letters and a lot a really long title so physical security most uh organizations are merging their physical and it security because for the most part physical security folks are protecting the it assets so most people have gotten rid of the guard at the front desk for everyone

uh for all buildings and now they only have a guard at the front Des desk for the data center so at that point and all the all the cameras all the doors they're all it assets now because they're all over IP so all the cameras um I don't have any in here they're all they're all over IP cameras so that falls under that of course falls under the it shop now uh the doors are all electronically locked controlled by computer so that falls under it now and again mostly the protecting it assets uh so again now it's all under it uh risk assessment methodology so if you just say well I evaluated that risk I rated a nine out

of 10 and you an auditor comes around says well how'd you get that number and you say I don't know audit Point does not have a established risk methodology you'd be able to explain not only get to a risk number but explain how you got there you got to show your work it's like it's like uh math in elementary school you can't just write down the right number you have to show all your work how you got there uh awareness education training another fun one businesses love it when you say I'm going to pull your people out for two hours to give them a a class in it security because that's two hours they're not

working uh data classification so uh it's required in for under some regulations that all your data is classified so it can be usually it's a uh one to four scale uh public meaning if we get lost it nobody would care because it's our um brochures it's our business cards it's other things like that we give away anyway next would be internal mean it needs to stay within the company second would be departmental mean it need to stay within that department and third would be personal uh Eyes Only basically when only certain people are allowed to see it so think about your the file shares you've got going on in your in your organizations how many people know they're in

terabytes of data and you have imagine have to go through each one of those documents and classify it some and in some places you do uh governance policies procedures again another fun part of my day where we all sit in the room and say well how do you do that then we write it down and we say well actually go do that and when they try to go do it they say Noe we had to change this we say okay go do it again follow the procedure and see if it works that time nope fix it again and then some regulation will come in and say well you're not allowed to do it that way anymore you have to do it this way

while we got to change our procedures again um it's almost like programming in people again people process technology you you have program that the people the same way you program uh program your computer uh reporting metrics maturity models so I have to tell people not only have to do these things I have to tell people how I'm doing it and if I'm doing it well or not again always fun to have to justify to uh Auditors and and my board of how I can with these metrics because again if I can't explain it doesn't doesn't matter I can say I'm 99 I'm 59 99.999 okay how'd you get that number well and I start uh having things to do

to figure out how I got to that number all right so operations again probably about 40% of the day uh this is the stuff that you think that you're doing when you take the ceso job on this is all right how do I uh build my firewalls how do I do my architecture how do I what tools we need to bring in to make things work um how can I make the uh operations that we're doing more secure can I bring in single sign on so people don't have 50,000 passwords and usernames to remember that's that's a a big um item going on now with especially privilege management that's another thing we have to operationalize how do

you manage people that are domain admins or have root access on on their servers how do we make sure they're doing that using those in a secure Manner and if they lose those passwords if they're breached or their laptops compromised how we make sure that that doesn't become a companywide uh issue because you know we've got good password management techniques we've got uh isolation we've got our uh our system setup so that if you reach one you can't automatically get into another um how do how do we do all that uh role-based Access Control another big thing in companies as they grow up and mature so uh the company I'm currently at everyone has their own uh

role the new people that come in they don't get assigned uh as a funer or as a collections person or as a um administrator they just get a role and they get assigned assigned applications as they need them so my rights are all over the place someone came to me and said hey how does how do you know who has X to what I said I don't know because I don't have role based Access Control yet we're working on it we're going to try to put something in for next year um but that's a big uh management nightmare as well because now I have to convert all these single people and say you are you are this now

you're this you're this and drop them into those buckets so I can manage th manage the buckets uh HR integration so when someone new comes on today they get loaded in the HR System and then HR sends us an email and then we load them up and with stuff anyone see the problem with this the there's no connection between between the two someone gets fired leaves a company in HR unless they send me an email telling me to delete them I don't know about it uh if someone changes roles in the company I don't know unless they tell me so one of the things that you look at if you're doing a Six Sigma analysis is how do I reduce

the handoffs how do I reduce the time that takes from someone happening something happening in HR System to something happening in the it system well to do that you got to merge those two together and have a direct link between the two so instant someone gets uh hired they get their it it uh credentials the instant someone gets let go their it credentials are suspended so we're we're trying to move it from uh days and weeks to minutes um so we're we're working on it it's going to be a a Jan a January launch um but we we're using two uh outsourced vendors both for our HR System and for our identity manager system they've worked together in the

past so we actually seen it work uh on another company they they show us a demo so we're we're hopeful but it we'll see how it works in in practice no it's it yeah oh yeah yeah uh especially dealing with uh Legacy systems meaning anything you know that's not that's more than 5 years old at this point it's pretty much Legacy system so if uh you're working at a bank and they've been using the same HR System for the last you know 15 years um it's an old Mainframe app um that thing may never communicate to anything that's uh you know modern up toate identity management system um so you'll have to deal with it and create your

efficiencies where you can all right so thing we have to work on in operations is threat threat management um and threat prevention so again this is typical it security stuff that you uh think about working on um but when you're working out from the management side I don't think actually one that's going out and doing the pen testing I have to go find a pentest company that'll do it for us or I have to find someone on the inside that's gonna that's G to do it for us and I have to understand it well enough to be able to call BS on contractors and people putting bids in but I'm not going to know the keyst stroke commands to

necessarily type in to make it happen uh so when you get into the the management ranks into the C Level ranks again there's an inverse prop prop inverse proportion of what your techn knowledge is and how much influence you have over the over the company the more uh influence you have the less technical you've got it's always going to be that that seesaw so you have to be aware when you want to move from the techie side to the management side that I'm going to lose some of my technical prowess uh again it's not that I'm an idiot it's not that I don't know what I'm doing but if someone comes to me and says well go

make that happen uh it's probably take me twice as long and be you know 80% as good as someone who does it all the time uh vulnerability management another fun topic so I have to constantly scan uh scan my network find systems that are not the par and get the people to fix them that last part is always the tricky one of getting people to fix them because they don't want to touch it because it's running the way it's running fine for them that may be great but but if it's got uh holes that can be exploited I want them to patch patch those holes inion prevention systems so this is a a big one now on the uh endpoint

side so attackers have not stopped but start lessening the amount of taxs going against the servers uh and network-based uh systems because we've hardened them as Security Professionals we've gotten them to a point that they're relatively hard to break into so they've said well I'm instead of attacking the servers I'm going to attack the people that are at the end points of the all the laptops that you guys are you know typing away on now um the one person who's taking notes on pen and paper they're secure as far as they're as far as I'm concerned for it security um everyone else with a laptop you know you're taking your chances so if you click on something that's not

you're not supposed to downloads the malware now I've got control of your laptop all the systems that you've got control over now I have control over so it's a it's a huge issue in uh in security now I'm sure everyone is aware of it but again I have to manage that and make sure that that endpoint protection is pushed out to all my all my users uh Ingress filtering uh your users love that when you do that right when they make that pop up that says you've been denied they love that can't go to Facebook anymore why not why do you need to go to it um to check on my cousins or check on my

brother or sister okay that's not a business re I'm sorry go talk to your manager and have them if they want them to want you to access Facebook during the day then I'll put an exception in otherwise forget it so it's a technical problem it's a technical solution but it's all based on the whole reason why it's in place is a business-based reason and if they want an exception they got to go to the make the business case for it all right threat detection I read a statistic uh not too long ago that 80% of tools that are purchased today are in the uh prevention space and only 20% in the detection space that needs to be closer to 50/50

in my opinion because what's happening is people throw up the wall they say well I got a wall there they can't get over that wall can they and then they turn around and say I'm not going to look at that wall anymore the wall the Wall's there no one can get through and then they don't see the person with the the grappling hook throwing the throwing it on the wall and scaling the wall they don't they don't look for it so detection tools need to get on a equal or better equal footing with prevention tools uh so that we've got complete protection and complete knowledge about our environment so if uh anyone's put in a

Sim before uh security event night and Incident Management um again that's type of tool that you want you want out there to correlate all your events see what's going on um but then when you get the report I can't just take the report out of my sim and hand it to my my CIO or my board and say well there you go we're protected they don't go that and say yeah this doesn't make any sense to me this is like if you hand it to me in Chinese so I have to interpret that report give the give my CIO and give my board something that they can digest and be able to prove that hey yeah the 10

grand a month that we're spending on that Sim solution is worth that money so again budget tools technology all stuff you have to take into account if you're goingon to try and be the top dog for uh security all right Incident Management again another fun topic that you have to deal with so as the ciso I'm expected to know exactly what's happening in my company at all times and Sur far as Incident Management so if uh it's a little easier in my current position uh with a smaller company at Comcast however you can as you can imagine there's always an incident happen happening all the time so at any one point I need to be able to say well this

incident we at this St stage this incident we at this stage uh We've wrapped up that one we're we're closing that that investigation out um here's the report for that for that last one that happened two weeks ago uh you have any questions about it you're you're constant in a larger organizations you're constantly fighting fires uh with the incident incident response inent management again you're expected to know that all the time so again this is the uh management versus technology skills graph so if you want to see in a graphic format then just me making my funny little hand gestures I'm not having a seizure up here I promise I don't think so anyway uh this is how it's repres represented

graphically so as you can see you know your hands on so and again that's the other key Point uh as you move up the management ranks Hands-On goes goes down attas those task oriented skills go uh go down but your your management and Leadership skills go up um I would say this is this is probably if you're looking if you're looking for positions in management this is the the progression so if you look for something that's a team lead that's G to be a step up from from your regular staff supervisor and you go manager director and you can get into the vice president ranks and above um so even at the even though on this

graph staff is down at the the low end here in mature organizations there are actually staff people that are would be considered staff that are higher ranked as it were than like a than even a supervisor or manager it's not that you can't become a a super techie and make that a great and very long and fulfilling career uh and you'd actually and in some organizations the higher up tech people are actually uh ranked or bandwise higher than than the managers they might be working for um but again it's two different tracks so uh again at Comcast they had six levels of engineer go all the way from Junior up to the considered a fellow um I didn't know

anyone that was a fellow they it's basically like a doctorate in information security uh you but once you if you got to that that level and that rank I mean you were basically considered you know the guru for the for the company um the so the the top ranked engineer was three levels above the lowest ranked manager so if you looked at him on a a on a band level on pay scale level the the lowest ranked manager could actually be theoretically be managing someone those three levels above them on the engineering side but again it's different different tracks so just be aware if you're trying to move up uh in the engineering engineering or the management ranks of

where you are relative to the people that you're working for you or that you want to work for you and if one's below or above that's fine just again it's something to be aware of all right so what kind of searchs do you need uh certifications in my mind again I'm like I said before I'm a sech junkie uh one of the reasons though is when you do put them on your resume they will open up doors for you if you come across a job opening it says CP require or cism required or equivalent required if you don't have those letters on your resume the computer that's scanning that thing will just throw right in the bit

bucket so the the reason to get Sears for me is not the job you currently have it's for the job you want or the job you're applying for next um they're that's going to show that you know what you're doing and open up those doors for you uh the two big ones that I see on the management side is cism suring security manager that's from my isaka and the cisp from ISC squared again either one of those um cisp is a little older and has a little more uh I I see a little more often than cism a lot of times it says the job descriptions will say cisp cism or equivalent so if you can uh find a

certification that uh is equivalent to to those um again I I would still you know apply for those jobs all right so one one of the other things you're going to have to do when you're on the management level is organize your team uh there's basically two activities that you can do for any in any it organization either run the run the business or you can improve the business so run the business is the ba work of um you know setting up servers uh you know making sure people are are on board and off boarded all the day-to-day tasks the improve the business is those projects so that'll be install installing a new system that'll be uh

replacing a uh existing system um all all those types of activities fall you're all everything you do is gonna fall under one of those two activities um again the the you can do both and have a long career but I'll just tell you when you're on the management side it's really the projects that get recognized the baau stuff I I'll tell you it just is just that it's baau so I process uh you know 10,000 uh user requests every every year I'm expected to process 10,000 requests every year if I process 11,000 do I get a bonus not really but if I only process 9,000 and there's a th left over then I get dinged

so it's kind of a a lose it's a you know neutral or lose situation for me to for the ba work so the projects is are the ones that get the the most attention next you're have a hierarchy for your team so again if you're shooting for the top spot that's going to be gener be a director ciso level uh then you then you might have some managers or team leads uh and then on the on the the staff side um I've I've always advocated having the three layers of architect uh engineer and analyst and again you can have any number of tiers within those the those three uh job descriptions so you might have analyst

123 engineer 123 uh an architect 123 um and for the most part the analysts going to be doing the Day-Day ba work Engineers going to be off half and half and Architects they're looking mostly at the the improve the

organization uh so security providers the layest trend just like with everything else it is to Outsource uh so for instance my current organization I would never have the enough enough people to run a 24x7 sock or security Operation Center so I've outsourced that I've hired a company that every time an alert comes in they they look at it they do a triage on it and if it's bad they send it over to me um and they're doing that 247 they're also doing that not only for me but for the 10 other companies that they've got uh on that shift um for that stock so I only have to pay for on tenth of a person 247 as opposed to entire team uh

to be around 247 uh so awareness education training again good thing to Outsource metrics uh application security again there not if you're in a smaller shop you're not going necessarily have enough applications going out the door enough to have someone on staff full time so you can Outsource that uh threat detection again I just talked about my my my sock uh DLP and forensics again in a in a smaller shop uh forensics are going to be something you do on a on a ad on a oneoff basis it's not going to be something you got need to do all the time so that's a great thing to Outsource so what's the end result and again I know up on here it's a little

small but again I'll I'll email everyone the the slides is you need to have a strategy road map for your organization whether that's on the infosec side the it side just in general for where the company is going and you need to uh at the top here it's how do you add value to the organization again I'm at security I'm a cost center but I still need to add value to the the organization in some way shape or form I need to manage the perception of key stakeholders so my my boss is the CIO his boss is uh a member of the is on the is head of operations and also on the board I need to manage their

expectations and their perceptions of what I'm doing in order to make myself uh at least seem successful and actually have success and move the organization forward critical capabilities I need to again good at governance I need to be good at uh communicating I need to be have a strategy uh I need to be a a builder so if I went to a new job tomorrow and they said well we just want you to maintain the status quo uh we don't want to change anything I'd say no I'm not interested it's to me it it uh the changing and improving the organization is is where the fun is if you guys if you just want me to come on and steer

the ship in the same direction it's always been going well you know that's not the way I work um again not going to make changes for change sake but if there's a better way of doing it I'd ra I'd like to investigate that so the other thing we always need to manage is our third parties security controls again being able to prove that your controls are actually working having a framework either ISO or 271 or uh the uh critical infrastructure framework um idle some kind of framework that you're working with or working in some version of uh it's going to be critical again risk management I'm not going to put a $100 lock on a $10 bike so we we manage our

risks appropriately um hiring and growth again I want my my team members to come on have a career path or and be able to grow their career or or we can be the we can be the minor leagues and then they learn what they into here and jump into the major leagues uh you got to be able to especially in security have that growth path and be able to let people go otherwise people AR going W work for you if there people see anyone that comes in your organization is stuck there for five years and they never get promoted and never get uh never get a raise never get to work anything different people aren't going to work for

you uh and again last is security culture so how do we take the security knowledge that I've built up over 16 years and assonate that out to the rest of my organization I get to know everything no but if I can do a two-hour education and then a year down the road someone raised their hand in a meeting and says wait a minute I think it says a security impact we need to call them that's all they want them to do if I can get you know one person each business line to do that then I've gotten the security culture pushed out to my organization to the point of it's going to help me in the long

run all right here's my contact info uh so this wraps up my presentation um I also run a security networking group it's called paste Billy area security technology enthusiasts we're actually doing a joint uh meeting with Leo's Group Security shell um so we're partner sister organizations whatever you want to call it uh and in two weeks on the 20th we're going to we're going to go to Dave and Busters um play some play some games Thursday Night Football uh and there's actually um a company that's sponsoring us so it's free and you get to talk infosec uh drink beer and hang out with other infos folks just like we're doing today uh so if you're

interested just let me know and I'll get you on the list and we'll see you in a in a few weeks Leo do you have something else

yeah so it's so uh so I try to do this every couple months uh once a or either once a quarter every couple months um so again it's free uh and it's just to get it's it's almost like a uh you know bsides but just with you know on a more more frequent basis um so if you like the like bsides format where again you can just come hang out and learn some new stuff this I encourage you to sign up with me and and join again no pressure no uh you know people AR going to try to sell you stuff if if you don't want to listen but every once but if you you know do like what they say can get

in contact with them and they'll uh you know they will tell about the technology they've got all right thanks everyone for coming out uh if anyone wants my info I've got my my cards up here and thanks everyone for a great uh great lecture and hope they have the rest uh have a good time with the rest of besides thank

you

good to go all right so uh sorry for the delay everyone we had some technical difficulties apparently technology is inherently difficult um so this is preventing a hostile Matrix to game in virtual reality security called arms uh I'm Pi kco uh handle Ali ghost so I want to open with a quote what I thought was 50 years away was only 10 years away and what I thought was 10 years away it was already here I just wasn't aware of it yet Bruce Sterling there's a reason I bring this quote up ultimately in 2012 I was working with a group called real extend and in IRC we were talking about the future of virtual reality everyone was

thinking virtual reality was 15 years away in 2013 the Oculus Rift debuted on Kickstarter that was the beginning of commercial virtual reality of the cycle what we thought was 15 years away was one year away so let's get started little bit of a parental advisory at first some content may not be appropriate for children or people who cannot realize that we are currently in and walking further into a cyber Park dystopia so who am I I'm a security consultant with foresight a VR developer and entrepreneur a former collaborator on real extent currently diving into Unity 3D I'm currently working on the side on VR applications to make tools for use in intelligence and and cyber

security and I'm a VR cyber pathogen and of course I'm also a cyber Punk because you that's why so alphabet sup let's get started with some acronyms and definitions because otherwise everyone's going to be lost VR is virtual reality it's full immersion sensory deprivation think the Oculus R after the HTC VI AR augmented reality it's basically the hollow lens it's essentially overlaying a digital HUD onto the real world using basically goggles or glasses presence a sensation tied to VR which causes your high Lev brain to realize you're still in VR but your lowlevel brain thinks you're actually there so because we all have basically primitive monkey brains for lack of a better term our brain starts to think

wait I'm actually in in this world and because of that after about 10 15 minutes you forget you're in virtual reality ultimately this can cause some very interesting phenomenons and has been known to allow people to forget their in Virtual right and do stupid stuff like trying to lean on a virtual object or have deeper emotional connection it's also why it's useful in medical Technologies at times and FPS frames per second or how many frames a render on the screen within a second 90 is the minimum required for virtual reality otherwise you're going to get sick blame your inner ear uh your inner ear if you have that Distortion between what you see in the centy

deprivation combined with what's expected physiologically you're going to get sick nauseous and might throw up not a good thing and MMI man machine interface you'll understand why I'm bringing this up in a little bit so this is not a technical talk this is a call to action about VR and AR and game security I'm not here to sell you on VR it's something you have to experience for yourself and you have to use a real headset not one of those junk gear vrs or Google cardboards or whatever they push at you know insert cell phone store here you need to use something like an actual HTC Vive or an actual Oculus RI to truly experience it and this is a

warning that the present let alone the future is going to be a cyber Funk dystopia and we need to step in now so why are we here virtual reality and augmented reality is the next generation of user interfaces and ultimately this is going to be when all those problems we overlooked with game security because it's just a game come back to bite Us in the ass this is where everything starts to fold into game security becoming industrial applications and medical technology IES that's right this is used in medicine and I will explain a few and the hardware and software is coming closer to wet wear what this means is within a decade or two virtual reality and

augmented reality technologies will serve as the basis for cybernetic interfaces that's right if technology goes the way it has gone the code will be reused in cybernetic eyes and things like that and congratulations that's possible root access on your actual monkey brain so yeah so another quote the future is UNR there are best case scenarios worst case scenarios both of them are fun to write about if you're a science fiction novelist but neither of them ever happens in the real world what happens in the real world is always the sideways case scenario world changing Marvels to us are just wallpaper to our children I had a dream you see years ago I had a dream

that as Humanity became connected and more technologically advanced we develop new tools not just virtual reality and augmented reality but cybernetics and even other tools like that eventually the virtual reality and augmented reality Technologies we develop now would start to interface directly with our own brains unfortunately this became a nightmare because this was unsecure and poorly designed people were able to be influenced spied on or even controlled by malicious attackers it's caus terrorism crime and worse ultimately what we really don't want is our insecure Technologies now leading to someone being able to cryptool Locker your brain 20 years from now that's ultimately the biggest threat we have in this specific area if you know your if you know the

enemy and yourself you need not fear the result of 100 BS cliche but appropriate the current AR and VR Hardware set looks pretty much like this you got your Oculus in your Vive which is are your two mainstream high-end headsets you've got your phones gear VR Daydream that's all your mobile VR that you would see in a cell phone store or you know also cardboard but you've got the fave which is a experimental fated rendering headset it's currently not available and I think I need to explain this fiated rendering uses eye tracking to detect where your pupil is focused so that way it can select where it's going to focus rendering power and thereby reduce detail in other

areas so that way only the areas that aren't affect that are actually in your vision and not in your prary are highly detailed this improves quality a frames per second compared to current methods this is currently experimental but it's speculated it's going to be in the second generation of consumer headsets so it's coming soon I should also note something very important here it tracking is used for things like marketing purposes and analytics there is a group of people including some people that worked on the original polygraph that's working on using eye tracking to do lie detection anyone who's familiar with the polygraph knows it only has a 50% effective ratio they're claiming with ey tracking it's

up to 80% so have fun with that thought you also got the meta and The Meta 2 and the hollow lens which The Meta and The Meta 2 is a tethered meaning HDMI Al reality headset and that plugs directly in your laptop or computer and then you've got the hollow lens which is an Intel atom based basically mini computer on your head with two gigs of RAM the problem with the hollow lens is unfortunately it's very restrictive and underpowered and unfortunately Microsoft does not like you creting custom gestures and as a developer that aggravates me that aggravates me a lot because that restricts what I can develop and what I can experiment with and then you've got

the bt200 and 300 by Epson which are Android based based augmented reality goggles as well the bt300 was like just released like within the last month um then you've got Google cardboard which is well a cardboard box which you stick your phone in and has some lenses and then you've got other devices as well you know just to lay land for input on the other hand you've got Oculus touch and the Vive WS the Oculus touch I do not have much detail on because those are not publicly available yet those will be publicly available in December and I hope to have more information as the technology grows and as I can get a hold of it the Vive wands come with the

HTC Vive the Vive ones I'll explain a little more later uh you've got clickers and remotes stuff like Daydream and gear uh the new gear VR those are based off clickers and remotes not exactly very Advanced but it gets the job done you've also got gesture motion tracking there are tools like the leap motion which hopefully at least some people in the room have heard of you've also got the manage VR which is essentially a glove that goes on your a glove that goes on your arm and your hand and it uses sensors to actually detect how your hand is moving and it uses the lighthouse sensor from The Vibes wand to track the movement of

the hand itself or rather the arm itself and it can do full inverse chics and all that stuff uh the problem with the leap motion is that because it's an infrared camera based system and has a limited field of view if there anything including tracking it sends it all off the Manis VR does not apparently have that problem you've also got ey tracking um like I said fob rendering have fun with the idea of using that for authentication or for marketing data or things like that hello 1984 and you've got wearables Android Wear iatch uh Fitbit all these things have been on some level integrated to Unity 3D which is the most common virtual reality engine you can pull data from

those devices into Unity 3D to do things like Health tracking and more you also got EEG and other Solutions there are actually devices out there specifically the ones I can think of off top of my head are the emotive and the Muse which actually read your brain waves that you put on your head and reads your brain waves so you can actually control the brain or you can control a computer with your brain there's also versions which can be used for EEG research and stuff like that so you know if you were to pop a box and someone was wearing one of those you could literally read the person's mind have fun with that thought

and you've got others as well and this is only the beginning because there's a lot more coming down the line and there's also stuff from R ranging from treadmills to basically bicycles that are designed for VR and it's going even further than that this is only the beginning so couple points of interest on the Oculus itself tracking is the only INF foring cameras constellation system it uses infrared detectors in one camera by default but with a touch they including a second camera and they're suggesting buying a third camera for room scale tracking um I should note that room scale tracking is essentially you set up multiple cameras in a room and as you walk around the room it tracks you and

your movements and the movements of the devices in question so you can actually physically walk around and interact with things in the virtual world um you've got HDMI and USB for made connections to the system uh I'm still waiting on data on their Guardian system which is their movement boundaries and the Oculus touch which is their motion Control Systems because they haven't been released yet I should also note one more thing the Occulus Rift last I checked uses one HDMI and two USB cables I think usb3 cables for main connections to the system I'll explain why I'm saying this in a minute and I should also note that Facebook's privacy policy on the Oculus allows them to slurp all your computer's

data by putting the Oculus drivers and software on your system according to their privacy policy you're basically giving them access to everything on your computer and this is Facebook we're talking about so we all know how that's going to end hi Zuck have total root kit access on my system I don't give a crap so a couple points of interest on the Vive for Hardware tracking is done with lasers from base stations the lighthouse system the Vive comes with two Lighthouse trackers which are designed to be put on two opposite corners of the room but you can also do them right in front of you or whatever the maximum tracking is supposed to be 5x 5 meters so that's about 15 by 15 ft

uh front camera for pass dur and shoper own system the shoper own system let's say you're walking up to this table right here it'll actually put a little uh wireframe of the table or any objects in front of you so you don't say walk into a wall because head injuries suck uh you've got HDMI and USB for main connections to the system it goes from a headset to a breakout box to the main system and you plug the headset cables into the breakout box and then the breakout box has other cables which go into the back of the system now I should note in the actual headset itself you've got a little panel on the top of it

which you can pull out you have one power cable you have one HDMI cable and you have one USB cable by default but you also have a second USB port that you can plug whatever you want into so you know if you say have a rubber ducky or something like that have appropriate form factor just say a nice little attack vector and you've got a link box between the headset and the computer computer it also acts as a Bluetooth receiver so this is important because Bluetooth is used for various components including headset sync with phone and tech phone for calls and texts in VR so you are literally using the HTC Vive Android app to take calls and do texting in your VR

headset so you are linking your phone to your VR headset which adds another attack Vector there but on top of that Bluetooth is also used for control between the controllers and the base stations and this is all using Bluetooth 4 or Le and while I think it's encrypted I have to double check because it looked encrypted but I was not completely sure if I just didn't manage to properly tocode it but it looks like it's encrypted thank god um just imagine the kind of man the Mills you could do with that so common virtual re and augmented reality engines you got Unity 3D which is your most common one unfortunately because it's heavily based on third

party assets and unfortunately a lot of game developers don't properly learn how to code and just hey say here's a tutorial let's go jump in and sell a couple things for say $60 on the asset store and not bother to name space our code or you know not doing integer checking which is rampant heck there's a valve I think it was okay so valve on their steam VR asset as in Steam valve they have not named space their code it gets worse in that I have detected multiple instances where Unity will pop up a little warning saying hey there's a no pointer reference and people ship their plugins and code without making sure they fix the no pointer references

and they charge for this um source code requires extra money so you can't really audit if you're an amateur they want full Enterprise access uh the most common virtual R and augmented reality engine it's C and mono based but it's using an old version of mono so yeah they're in the process of upgrading it but for now it's limited to net 3.5 feature set so yeah bit out dat it's also using JavaScript and Buu Buu is a bastard version of python which they decide hey I know let's say you know python isn't exactly what we want so let's just create a new implementation that's similar to python has similar syntax but isn't actually python great great job creating your own

programming language guys we really needed that and you can also plug in additional language support as well there's plugins for things like Lua for example uh Unreal Engine assets tend to be more expensive but I do not have data on those assets yet because money and sources given when you sign up on GitHub but not actually open source so luckily you just have to sign up for the engine for free and then link your GI Hub account and they'll give you access to their GitHub repositories and they give you access to Unreal Engine and Unreal Tournament source code which is always nice because you can actually audit it and there have been audits done but it's not actually open source you

can't redistribute it outside that Eula um it's the second most common virtual re and augmented reality engine uh doesn't quite have the support that Unity does though it's C++ based unfortunately there's a lot of Legacy code in there up until last year there was let's see what was it up until like late up until last year or early this year there was AES code in there which they thought they could actually modify they thought it was a good idea to try and modify the cipher so that way they could try and make it so if the stream was interrupted it wouldn't break things we're talking about a stream site for people congratulations so blueprints visual

scripting is a bakedin option essentially it's node based scripting which you just connect the nodes and expose classes and C++ to that so you can use those but there's also a third party JavaScript plugin for it from NC soft um the state of game and virtual reality development today uh game asset and tutorials especially tied to Unity 3D are horrible from a security standpoint and horrible from a code quality standpoint a lot of these don't bother to do integer checking or for that matter namespace their code and that is why I want to scream sometimes because unfortunately when you're working with unity 3D you're going to have a lot of assets where you're going to have to just overhaul

things for things you paid a good amount of money for just because people can't be bothered to fix their stuff because they didn't know any better uh game development or game developers usually don't have any security mindset at all or they think it's or they think security is DRM and anti-che rather an anti- shell luckily MMOs seem to be the exception but but that is the exception and not the rule luckily um this seems to be changing I recently saw news that Bohemian interactive who made the Arma Series has recently added security exploits and vulnerabilities to their bug tracker for private reporting luckily these guys seem to be taking it seriously but I suspect that might be tied to the fact

that they had oday dropped at Defcon on their game um a lot of engines don't encrypt their net code in chat luckily this seems to be changing and people are using op SSL more for encrypted net code but people still don't bother to encrypt their chat and unfortunately the primary chat protocol I can think of that would be useful which will go unnamed is GPL implementation based and doesn't have properly documented information on it so the only way to make any sort of implementation is to derive from GPL code and unfortunately the developer who will go unnamed thought it would be a good idea to try to charge an open source project that tried to reimplement that protocol in

another programming language $2.5 million for licensing because they didn't want to use the GPL there's now a court case over this apparently really this isn't helping anyone so technical considerations and game development mentality virtual reality requires 90 frames per second other wise you're going to get physically sick um this is a physiological response from the in a this is not a technical limitation um this is a wetwear limitation most game developers have no security background and most CS programs do not bother to teach secure programming or security mindset this extends to game development programs and other programs as well as school and Tut schools and tutorials uh bad habits form early and are hard to break so this is

ultimately an application security problem this really ultimately comes down to application security and a lot of bad third party code comes into play as well uh virtual reality developer mentality in particular needs to be having some specific notes on Virtual re virtual re virtual reality requires us to throw the entire user interface and user experience book out the window the objective now is to focus on how the interf interface feels first and build it from there so we do a lot of rap prototyping user tests are common and are trying to make the interface feel more natural not artificial the idea of oh well I'm going to hit a button or I'm going to check a check box or whatever

if it's natural feeling maybe but if it's like oh I'm going to hold a tab and do yada yada yada yada hitting buttons or moving sliders that's not what people are trying to do because ultimately the objective is to make things feel more natural and make it feel more visceral uh programmers are also user are experienced designers and the Gap is closing every day luckily this is a good thing uh virtual reality and augmented reality are still very experim still very experimental though and there are very few best practices in place and most of them were along the lines of stay seated and develop for seated environments or make sure the user doesn't get sick there's nothing really

there about code quality or anything like that just like basic you know how to make people not get sick and how to do user experience over code quality uh rapid prototyping is key in a lot of designs and code get thrown out because it fails usability testing uh Google suggested turnaround time for prototyping is two days so you're actually developing these tests for user experience and user interface design in two days and determining whether you're going to keep it or throw it out not much time for code quality there and code quality as I have been saying is often the last thing on developer mind I'm unfortunately because the speed of development most people don't really have time for

that so another interesting quote the most interesting thing about virtual reality that you can't find out anywhere is the feeling of act or the actual feeling of presence and the feeling of being in virtual reality it's not something can be communicated by talking about you very quickly accept the fact that you're in a different place the feeling is something incredibly novel it's a visceral experience to be able to trick yourself into believing you're somewhere else that a quote by Aaron coblin who is a virtual reality developer and he's right ultimately as much as I could stand up here blowing air saying oh well we need to develop for this hour next thing and it's a

totally different thing until you say go to Christiana Mall and go to the GameStop there and try the vibe which I suggest everyone do you're not going to understand what I'm talking about because ultimately it is a completely different experience to what we've experienced up till now um so so very specific note on presence but first I want everyone to watch this gift because it's

perfect so what we just saw here is the kid was so immersed in virtual reality that he forgot that the object in front of him was not physical leaned over and oh so there's a reason for this gift um as much much as I talk about presents and stuff the reality is because you have that process of not realizing you're in virtual reality at one point there's a possibility where say if the chaon system did not detect say the stairs in front of you you might take a fall um just saying if you don't see your cat that could end badly um yeah so ultimately we're on New Horizons and virtual reality and augmented reality require new design methodologies

and old design methodologies were Obsolete and must and there must be a more organic and natural feeling and how they work everything is still experimental and no one really knows what works best yet and there's a lot of research and development going on to figure out what works best from everywhere from Google to Facebook to startups ultimately everyone is working on this in the big fields and a lot of people including Mark Zuckerberg himself have been pushing this pretty heavily so ultimately we need to jump on this now before it gets real bad uh virtual reality and augmented reality web 1.0 stage but that's a good thing because it gives us a chance to step in before

things go horribly wrong ultimately it's better to bake Security in from the early days than to have to bolt it on later um so there's one very specific reason bringing this up this is the Gardner hype curve which shows from left to right the process of building hype then coming down to earth and then actually building practicality to the plateau of productivity over here so ultimately uh human augmentation is right here augmented reality is right here and virtual real is right here so augmented reality is just finishing bottoming out and then virtual reality is just starting to come up to being practical virtual reality augmented reality are 5 to 10 years out from being at the

plateau and human augmentation is more than 10 years out there's a very specific reason bring up human augmentation also I should note that machine learning is two to five years from plateau and just a approaching Peak hype so I expect to crash on that soon because this has been a very accurate model uh this is basically based on mobile and all that stuff and we are ultimately coming into a new mobile phase um so another thing I need to point out ultimately this is going to be very big financially and business-wise as well Augmented Reality by 2020 is going to be a 90 billion business according to dig capital and according to dig Capital virtual reality will be

30 billion these are going to be extremely big and these are going to be extremely vulnerable unless we act now so the reason I'm bringing this up is because we're looking at the next mobile and ultimately if we don't fix this now we're going to be hurting later just like how we're hurting now because Android was a mess security wise at the beginning and no one was bothering so another quote does virtual reality provide us with new ways to augment exper enhance and experience reality or does it undermine and threaten that reality virtual reality is equally prone to portrayal is either the bearer of bright utopian possibilities or dark dystopian nightmares and both these views have some basis to recommend

them and that's from Derek stenosi or however you pronounce his name sorry so there are problems on the horizon unfortunately we're going to have shells some of with ghosts in them uh privacy and other nasties so we're going to obviously have OD day all over the place unfortunately as much as we want to push software development life cycle changes people don't care they really don't not until they get bit then you've got privacy and I'll bring that up in a moment but you've also got other nasties think about malware for example think about iot botn Nets crypto lockers things like that because that's going to be hitting this as well so this is where privacy comes in I'm

sure everyone knows about Pokemon go and how hey let's request access to everything on the phone and everything on the Google account to play Pokemon what I mean sometimes it's not malicious sometimes it's just complete in total fail or QA fail which this apparently was but yeah what uh ultimately what it comes down to is this is ultimately going to be used for buying intelligence and everything else I should note explicitly that Sony is working on cont Sony has patented contact lenses which act as both a camera and an augmented reality display device contact lenses and you thought Google Glass was bad um this is going to be a mess privacy wise and we need to act soon

we're going to have to establish some practice to fix this um current uses of virtual augmented reality including entertainment everything from movies to games to you know music videos uh Commerce uh there's shopping apps there's people using it in uh let's say uh real estate and all those markets and you've got industrial applications such as Cad and you've got vehicle interfaces I think it was Mercedes was working on an augmented reality headset for using their cars um medical devices and not just medical devices it's being used to train surgery and perform surgery and it's also being used for treatment of things like PTSD and Phantom limb syndrome so Phantom limb syndrome in case anyone doesn't know if your arm if your arm

gets blown off you have this really nasty sensation because your brain still thinks the arm is there but it's not actually there so it it's really painful virtual reality is being used to help treat this among other things and it's also being used by military intelligence law enforcement Emergency Services apparently law enforcement agencies are even using virtual reality to practice for hostage rescue situations a lot faster than they were because previously they had to build up kill houses to actually practice clearing rooms now they can just scan the room and put in VR and they're done practice that way uh it's also being used for court cases and there was recently a news story about

how they're using virtual reality to hunt down the last Nazi war criminals by recre aitz to try and determine who is actually telling the truth about what they could actually see so they've actually gone after at least one Nazi war criminal that's still alive well was still alive he died right before being extra died but they've been going after the last remaining Nazi war criminals to chase him down with this technology uh it's not exactly uncommon at this point it's going to be gain more common but what's the worst that could happen I mean really what could possibly go wrong here I mean this is actually based off a mockup of an actual device that's being developed by a defense

contractor I should mentioned this is actually something that they're working on this could not possibly go wrong I mean oh no I'm just going to say that the UAV Reaper video was here and I'm going to say there's an enemy right here even though there's none project that enemy right there and I'm going to make it look like he's about to Fire and then all of a sudden you have other guys thinking oh my God I'm being attacked I'm being attacked and then you wind up having people reveal their positions or shooting themselves it could end badly but luckily it's not all bad uh here is actually a picture of virtual reality being used to treat PTSD

veterans ultimately this is being used to treat medical problems this is also being used to treat people in therapy and things like that there's a lot of research going on right now as far as psychology and therapy and how virtual reality can be used to treat those furthermore augmented reality could be used for things like teaching people social skills um you've got this which is an actual product uh I will redact the name but it actually displays windows and can actually display Windows Windows as in like excel in virtual reality and a virtual reality desktop so you can actually move your windows like this and this and this and just look around all over the place and while unfortunately

the technology right now is kind of limited because the screens were ultimately 1080p in each ey so text is really blocky it's projected within five years actually was projected like two days ago by Michael Brash over at Oculus that in five years it's going to be at 4K and each eye and then soon after that I'll probably be at 8K and then you've got tools like Google tiltbrush which is ultimately used for creative stuff such as painting in 3D space and can ultimately possibly be used for other purposes such as making movies music videos whatever um ultimately there are other potentials that actually allow this to be completely awesome this is Elite motion demo right here and they're actually

able to detect hand movement and I'm pretty sure that's a Vive wand where they can actually manipulate a map and do things to look at places this is only the beginning on top of that this is a meta 2 demo from The Meta 2 augmented reality headset where they are able to project 3D interfaces in the real world so ultimately while it sounds bad there are a lot of very interesting use cases we're coming up on and ultimately this is only the beginning and no one really knows where this is going to go yet but all we know is that if this continues the way it is it's either going to be really good or really bad but it has the

potential to be really good just imagine that for cyber security so who controls the past or who controls the past controls the future he who controls the present controls the past that's a 1984 quote and it had to be done so a bit Legacy VR does anybody remember this thing the reason why this sucked is because they skimped on the hardware and they were shining lasers into people's eyes yes how could that possibly go wrong uh virtual reality was a mess ultimately the hardware was not powerful enough back in the 90s when the last boom was even into the early 2000s it was not powerful enough we're only just now getting Hardware powerful enough to

truly power these things and virtual reality ready laptops are literally only just now hitting the market up until now there have been technology limitations such as Nvidia Primus and graphics card power switching and not actually directly interfacing with the graphics card on HDMI which have been causing issues up till now now they're finally starting to fix it on like the laptops that have been releasing in the last month uh historically VR failed because it wasn't the hardware wasn't powerful enough now it is and the hardware is getting more powerful every day luckily the current generation is a lot more powerful and doesn't have anywhere near as many issues and with gesture controls more powerful hardware and more

experience developing means it's going to be a truly amazing time to be in VR and AR ultimately the big reason why displays specifically for the Oculus and the Vive have come into play the way they have is because they're they initially start using the mobile displays from phones in the headsets so these headsets you're getting now for virtual reality are actually using phone displays uh so technology leapfrogged with mobile and is now going into virtual reality based on mobile technologies um ultimately you've got the Vive WS the lighthouse base stations and the Vive itself with the pass through camera right there and you've got the Oculus cv1 with the headset and the Oculus touch controllers and you know timately

future's going to look bright so in the next 5 to 10 years virtual augmented reality are likely going to become the next user interface of choice for computing ultimately these two technologies will likely merge in a one headset and will ultimately likely be two mod of the same headset in the near future they're guessing in certain circles it's likely to be two or three Hardware Generations out before this happens so guess about 3 to four years probably right now we have controllers and gesture right now we have controllers and gesture controls coming soon and by gesture control I mean gesture control without having something in your hand uh we can already do stuff with gesture control with the wands but soon

it's going to be hand tracking based Google recently filed a patent for actual hand tracking in 3D space from mobile VR so it looks like they're going to be using Google Tango to track hands in VR headsets using Daydream within the next few Hardware Generations right now the reason they don't do that is because the hardware is too big to be in the same system um 10 to 20 years out though is where we really need to be worried although we really need to be working on this now the virtual reality and augmented reality systems of today and the code of today is going to be rolled into the interfaces in cybernetics and man machine interfaces later on

and this is what truly scares me because having looked at this code for so long I am truly terrified that someone will get root on my brain if I have cybernetics and it's using this code I am truly terrified of that and ultimately privacy and malware is the near future and cyber cyber next is a decade or two away but the cards are all falling to place as we speak and ultimately may seem like it's science fiction but historically technology has shown people reuse code all the time and ultimately un is currently the most common tool for virtual reality development and because people like to reuse code I have a really bad feeling about this so those who fail to learn from

history are doomed to repeat it wise words from Winston Churchill so historically security has been afterthought thought money time and ignorance have killed us so far and if this continues this might physically kill us as in like kind of dead uh we have a chance to do it right this time though and ultimately we have to learn from the past historically we've had bolt-on security measures https OTR things like that um and historically security hasn't been usable but now VR that's focused on usability more than anything else anything and everything we do has to be non-invasive we cannot have little check boxes pop up saying oh update your antivirus or stuff like that people aren't going to want that at all and

ultimately usability is going to be bigger more than ever now it has to be completely and totally transparent uh and we have to decentralize distribute secure and encrypt everything ultimately if this technology starts going the way it looks like it's going to go go and I'm going to just jump into La La Land for a second but if we start getting to the point where this starts going into cybernetics do we really want our brains connected to the internet at all times hi Google have fun drawing everything from my brain uh virtual R and augmented reality are Next Generation interfaces for military and medical Tech Lev Al and everything else so ultimately we do have

to pay attention to this right now the Navy is actually experimenting with an augmented reality visor for divers in combat situations where they can overlay things like instructions on how to diffuse a bomb so hi let's hack this and change it so you cut the blue wire or the red wire and boom so what can we do now Now's the Time to demonstrate and fix these problems before they become widespread ultimately time is ticking every second we talk time is ticking every second we sit around going oh let's just hunt OD day and whatever and security bugs and gaming and relay code those need to be fixed it's any way we can get access to them we need to fix it

and we need to start establishing a security and application security mindset in not just virtual R and augmented reality communities but game developer communities as well proper secure programming and design must be hammered into education and it's not optional anymore people we can't just say it's an elective class no this has to be Baseline programming classes and we can't just say oh well here's how you teach here's how you code now here's how you secure no it has to be all in the same class and all at the same time and teach how to do it properly from the start otherwise this is going to end horribly and we have to design the future to be distributed and

decentralized and secure we really do not want to centralize this stuff and we are going to need new specifications and standards and I have a history working on that stuff I've got something coming so what must we not do and this is important the future is bright but if we bring it down with Doom and Gloom we're done and it's going to end horribly ultimately VR and AR has amazing potential and when we're in the security Echo chamber we start going Doom and Gloom Doom and Gloom nobody wants to hear that nobody wants to hear oh well we need to be careful otherwise we're going to get popped why do they care oh we're going to terrify them well

that'll either stifle Innovation or make them ignore as completely as chicken littles uh we must not alate alienate the game in virtual reality developer communities I'm going to say this right now the virtual reality community in particular is a lot more diverse than impos SEC is and I hate bringing diversity into this because this is not that talk but we must not be the way we have been because ultimately by being we should shut people out uh when we alienate people by acting like Pricks we kind of can't get our point across and we must not push security at any cost because performance is King and we performance is king and the King will not be dethroned ultimately that 90

frames per second number it's not optional it has to be at least that fast at all times it cannot dip below that or people will get physically sick and then it's game over for that so so if we try that welcome to ignore Township population infos so ultimately we're going to need a plan of action the best option we have now is to reach out to game augmented reality and virtual reality developers and those who teach them ultimately we have a chance to fix this now and do it right from the start or at least the start of virtual reality and we can't do this alone as much as we want to say oh look at me I found OD day ultimately

that OD day is probably going to have several others like it so just finding one OD day is not going to help but if we can get people to properly secure their code ahead of time we can reduce OD days overall and then you know the OD day might be more special and also we have less OD days to worry about which could cause less harm always a good thing unless you you know like to pimp vulnerabilities all day long may be a bad thing then maybe a good thing because then it drives the price of OD days up on the market teach people how to securely program and a security mindset this isn't just buffer overflows this is for

example teaching them that just because something does just because SQL injection is a Hot Topic and I have actually seen this personal personally I had a student from an unnamed School approach me with a project they worked on and they want me to check if for SQL injection they were sending the SQL statement from the client over the wire to the middle server and then running that same SQL statement that they crafted on the client as in the full SQL statement on the back end you don't need to inject that uh all you really need to do is just say hey I'm going to send an SQL statement and you're just going to execute It Whatever remote code

execution done so ultimately you know just because you hear SQL injection is a Hot Topic does not mean you actually have to inject if someone does something like that and unfortunately this was apparently done out of laziness and because they didn't know any better I wound up having to spend 30 minutes trying to explain this to them and we have to root out bugs and mitigate them so they are harder to have happen again and we have to create an enforced standards and specifications for not just secure but also ethical virtual reality augmented reality and game development because ultimately we do not want someone going oh look at me I'm going to be the Cyber God and

everyone using this is going to be forced to worship me or I will kill them in real life or oh I'm going to remove the log out button and if you die in game you die in the real world Sword Art Online so one final quote the way technological re revolutions actually happen involve smart people working hard on the right problems at the right time look to your left look to your right everyone in here is working on something and ultimately it may or may not be someone in this room but we are on the Forefront of fixing this we're on the Forefront of developing tools Technologies security everything and if it's not us it might

not get done period because we can't rely on someone else to take action we can't rely on someone else to actually fix this because ultimately someone else might not be thinking about the same things or they might be thinking about the same things but they might think someone else will take care of it ultimately we need to actually get our butts and gear in order to get this done so any questions

yeah haptics yeah yeah

haptics tactile and haptics um that's currently being researched but there isn't much Beyond pressure controls and maybe a little bit of I'm guessing electronical stimulation to cause Sensations that's really a very emerging area of the technology and that's only going to become more over time but it's not there yet yeah uh

yeah the sub pack that's already on Amazon you can already buy those and yeah those are nice but that doesn't cover things like wetness and stuff like that it only covers impact uh anyone else uh

yeah um the Hollens in particular they restrict you to a certain set of gestures and the certain set of gestures are very much inspired by mobile but the problem is that when you have a restricted set of gestures you're not able to experiment with with real natural based

interfaces the hardware is extremely weak on the hollow lens and it's a limitation of the hollow lens itself ultimately it's only got two gigabytes of RAM which means you can't even run Unreal Engine on it and it's extremely limited and using an Intel adom processor its limitations are not just API but also hardware and the hardware is what makes it go from being potentially really useful if they were to just fix those gesture controls to being extremely limited ultimately a lot of the things we're going to want to do in the virtual real and augmented reality communities are going to require a lot more power than that and the power itself is the problem um tethered is going to be a lot

more powerful there's also the bt300 which is Android based but I don't have the technical specs on that aat yeah also Oculus is working on a mid-range solution which is going to be VR without a tether without a computer so it looks like they're going to be possibly doing that Oculus as well in the VR front uh they only just barely tease that Oculus connect a couple days ago though so no one really has any details on that outside Facebook uh hang on yeah no that's 90 frames per second in both eyes yeah so you're literally running 90 frames per second on two displays simultaneously yeah uh yes I have and it's extremely underpowered and frankly the hardware is

not up to Snuff it seems so when play Sony announced the PlayStation Pro they said that their intent was to try and compete with PC gaming in order to try and do 4K but it doesn't actually do 4K it just upscales um the problem with PlayStation VR is that the hardware is not powerful enough and it's too low resolution and it looks very smear um the graphics themselves are actually nauseating so that I suspect is going to do more harm than good good uh anyone else

yeah um kill it with fire I don't want that why do you think I have a five anyone

yeah theun it's going to have to be that M further education on secure programming and application security it's also going to require more tooling that we don't have yet possibly stack analysis and other tools just to make sure things are secure as well as getting people to stop being lazy which we know will never happen but you know unfortunately people like to be lazy and that causes a lot more issues than you would think yeah

ultimately we're going to have to drop some OD days that's my suspicion ultimately it might require a few Defcon talks with a few OD days in order to get them to wake up that's my guess and I mean OD days as in like publicly revealing for the first time not even responsibly disclosing o things like that we might need to do some irresponsible disclosure very publicly to get the message across anyone else uh buer buer buer thank

you

e

e

e

e

e

e

e

e

e

e

e

e

e

e

e

e

e

e

e

e

we can use distinguish the B the normal okay and collect point you know uh at collect point two we can know the parameter uh comes to the recursive that we can control we can handle right but there are so many open recursive so many open res on the whole internet while what we can control is only part of them right so if we to look at some attacks like uh DN reflection attack they usually use a bunch of open uh resolvers at the same time so we need a perspective that is cleal first and then that's the collecting point three at this point each client uh no matter how many open res it visited we can see

it at CL uh at cl3 any problem also at uh collecting point three we can you know get more Rich info like the queries without response or the response without par you know lots of anomaly data okay now we got data what's the next we currently have a huge amount of data to be processed in real time you know like uh for this flow we have about 30 B every and the peak speed is over 3 milon per second and DS data is even bigger it's about 300 Bing A and P speed is over 5 million per second many EX existing tools can be processed the data you know stored on the disk while the data we have is even

hard to write on this in real time so we have to develop the process service on our own and here are the tips for my work the first one is uh how data be processed in memory to avoid io2 dis the memory is fun right and considering the performance I use C++ to develop most of it and US Z mq for transferring you know Z mq is a very good message mle can handle over one minute messages per second in one process easily and communication moves is very easy to use and the author of The mq Peter you know develop a new message mware nanom it's also worth the try uh I know that Peter

got canc several years ago and uh just several days ago he chose to use n it's a very sad news uh here I you know may him be PC also I I I link one of his blog article you know on the right bottom is protocol of ding is one of his AR so it's worth a look getting back to the topic uh the second one another one is horizontal partitioning you know horizontal paring means we have to split the data to the data unit that's independent with other data unit so in this way we can process each unit separately and processing data unit separately means we can extend the processing noes right so easily we can

easily improve the processing performance the last but not the least is tired assembling tired assembling refers to that we do not simpling on the whole data set you know for nlow is already kind of sample data right s again more like to increase assing rate it's meaningless and also simply on the whole data set we you know Miss some list but important data that's not not the result we want and we only simply on the you know big like 48 IP 48 Google's IP and like domain name go. we can set a progressive uh filter filter rate like a currency count below 100 we do not filter and a Curr count over 100 and Below 1,000 we F

half and Curr count over 1000 we will 90% of that you know in this way the less data will kept while the big head data can be filtered to a small amount and uh can still keep it statistical characteristics we can still see uh what we want to say why not missing and the data amount uh you know needs to be processed in real time can be significantly the following is uh how to deal with you know is our architecture diagram and it's too specific I I keep this the modeling and the feature selection here but I don't want to dive into the two details so I keep it here if you are interested in it we can talk

it in private besides a very big here you know uh pict we discussed the data collect point and how to De is speculated in real time and the slides about data modeling and data feature selection those covers most of the uh key points in the whole process but there are some uh issu some INF issues requires our attention so here I list I listed a large the first one is the manage object you know for some people do security guard at the corporation boundary they have a very good perspective of M they can get like a group of host is a clust of myle uh myle my SQL you know the database pass right they can get uh IP list is a

proess server so it mean something so they can develop develop specific monitoring and the defense strategy based on the specific Amo right but on backb network monitoring there's no clear M for me the only one available for me may be the domain name of theend from the PDN system we can mapping an IP to a specific domain name then I can you know according to the dominum to do the further speculation the second one is data partial data this why is very important you know uh it refers to the situation that we can what we can get perhaps only be parts of the data because we collecting collecting data in the background R and lots of the Tex such as

simp traffic you know uh their traffic flow is hard then not real so there are two possibilities we might see the traffic flows to the victim why but we can now see the uh the reply flow comes from the victim and the second onse uh the attack traffic comes from the target IP uh comes from the target IP is internal so we cannot see the attack flow while but we can see the reply flow to the proof for it's a little tricky but these are the real of par data we can say in our net flow data me I'm not English native speaker so might be problem to express my detail it's okay so when we do detection one single

rule for a completely idealized situation is far from enough for one attack type we may need many rules each room for a different situations like the par data we only can the traffic in or only can say the traffic out and each rle for different comies the third is icmp as side indicator you know this one is interesting we know that icmp itself can be used for Tech like a pink BL right moreover according to our statistic most ofmp blow Spike are normally related to other other attacks you know uh for example scanner will scan to the IP may not exist or the IP with the part are not active right scanner they are blind

they don't know the target what they want to do is to find the active Target right and also dios reflection flows to the victim's random and open part and also dios you know the Checker might use some reflector the reflector is dead one so all these situations will bring some icmp and reachable traffic right so therefore if you uh cannot make sure whether an IP is normal or not you can check ismp data for more details icmp data will tell tell you you know some specific uh about the situation we have been and if you are wondering uh whether your detection covers the majority of the anomaly it might be a good ide idea you know to

check whether all icmp Spike related IP had been detected or not the four is uh integrate combined with the third part data now third part data will not uh will not only complement the data Dimensions which we cannot get from the net law uh but also you know can help us to refine our detection rules and uh to refine our uh detection policy which I will discuss later finally the backround network monitoring although is very effective but it's in ability to some situations like you know HTTP slow HTTP attack no since N Flow is a simple slow HTT but will not be showed on the net flow uh this there's no it so with spe

on data something we can do but something or something we can't hav't done so much we this is a screenshot on our dashb you know we can monitor the vast majority of the real time scanning events the vast majority of D attack events and attack changing tendency and so on more important we got the profile you know the profile is uh a data uh is a is a IP IP status data you know in a specific time and uh with the we can we have the back tracking ability and C with formation data uh we can continuously check out the history profile you know to extract or to refine our detection

rules you know here uh my talk I talk to security viability and how I it hope hope to be helpful for you but at the same time we are not only share our method we all also share our data all our system are free to access this is one of our system donet so you can carry real time attacks on this system and you can register to add your consent IP and site so if there's uh if if there is a t theem the system will notify you in real time here is our Global scanning monitor system we will static statistic the latest real time scanning events and uh you can also carry a specific scanner

you know to sa this is uh to say is history history behavior and to find out more interesting things uh so that's any questions

did was most difficult to collect the data I yeah uh because uh we have different collecting point you know for net flow DS data or the sample you know the the virus sample or Charing SLE this the most important part for a okay

please you mean the open

SCE

you know open right but the most part I I wrote it by

myself no

just

if you have time could you go over some of the slides that you f forward through you mean the model right it's okay yeah I'm

notl let me see

what's your part you know dealing with the big data modeling or the feature selection you go all okay okay this one this is the architecture diagram of our system you know data flows from the left to the right and database finally at the left each dispatcher uh connects to one data collecting point and split data to proper unit and pass the data unit to write ETL model uh splitting config say the horizontal policy can loaded in

the the next the next you know uh dispatcher only do data receiving and sending so according to our conf so if the original passing policy is not good enough then I can you know change the config and reload it it's very fast the ETL is the most heavy part of all steps here we do a this format unification cumulation calculation statistics and POS data to uh add the proper time you know the proper time means such as uh such as we already cumulate the enough time enough data for the next step such as the processing time window is already is already over and so on then is the engine you know engine here means the r

based machine engine and the data here will be judged to black white or green and the engine here has two key uh key points the first is we have to implement and efficient algorithm for maing process and second is we have to define specific effective and sufficient data features for mat rules you know we have flow data uh anybody here familiar with N Flow data yeah okay so uh I just you know uh simply describe it each line means one N Flow you know it has start time end time here I show duration you know equals to end time minus the start time and the source IP Source part destination IP destination part different uh TCP Flags uh to means

type of service it's a TCP flag and how many packages in this flow and how many bites this flow transfer also we have btp numbers but uh I don't show it here you know just in man we have it see in one flow there are two IP here right uh either on soft position or destination position so if we want to see uh whether something security events happening something security events here we mean it's uh related to one specific IP right so we need to change the angle of our observation so let's put the IP in the middle here we use for it as an example we can see that some IP sent data uh two

for it with smaller packages while it replies to with slightly larger packages just looks just like a normal yes open right so here I uh I introduced my dat model you m layer net P model I don't want to abbreviate to mlmp you know it's not a good name so I just call it multi layer P mod uh it has two aspects the first is net pavment you know it means we have to say at least in or out directions and the second is multier is the uh common data analysis operation like the J and R up you know it and so we we we have to say not the IP that IP situation but also to every IP to

specific protocol or IP to protocol with every specific part now here's the Feature Feature metrics we use here's including all features I use left in right out by passing through all feature Matrix we can get one feature first we have to choose the direction in or out and then we choose uh the data terms you know let's omit the spike type of Spike ratio at the bottom so each Direction has eight related terms and every data terms can be organized in dat finally choose one Statics statistical method you know the m l m dispersion Val so on so our uh oh I'm sorry let's take the uh the second one in IP cont and as an

example so we can calculate how many unique IPS comes in and what's the IP dision you know the in IP is concentr or disperse right and Al uh what's the average count of all in IP and which IP is the biggest the top one and what's what percentage H IP accounted for and so on it should also be noticed that there are some you know some data TS some columns are on GRE background right uh what I means the GRE method can only be combined with the GRE data typs the meod from the key average and the bottom right key average you know this method can uh can be only combined with duration map the package

cont map and package size cont map you know uh respectively representing the average duration that the average package countes or the average package size for only these three DS are continuous variables while the others is dis variables right so we cannot say that the average IP of ip1 and ip2 is IP 1.5 right it's meaningless and the last one is uh flow Spike right on the bottom flow spike is a good feature you know for like do attack many kinds of D attack are Bas consumption attacks so uh using the huge amount traffic to De online services to this kinds of attack always comes with a blow SP and so here we have to introduce

it to our feature matri for IP on IP is a specific protocol on IP with protocol and with every specific part if there a spike if there is what kind of spike is you know uh flow comes back is one comes back right or package comes back you in one flow uh there might be many packages and uh it's also l you know maybe one flow the package is the same but the flow uh package size is uh shows a very huge spike is also anomal so different attacks may have different spike types so right now our data modeling is looks more like this you know for H IP or IP with protocal or IP prot part so we can uh to see the in

out in two directions and actually we have over 100 Fe quite while here I just list some of it you know this is our completely dat model while in specific detection cases uh we don't need them all we just need some specific cases specific features now so let's see how to choose the feature in specific cases the first one is post Scanner with this post scanner post scanner is the most popular scanner on the internet because the H number has a definite you know service Mees like 80 means web server and 22 means SS server right so uh also mean meanwhile the S scan is the most popular one among all SCS so how to how can we use the feature

described before to detect the s for scanner the basic feature easy right is it's a scanner so it's out talk IP should be scatter right it's a scanner out talk IP should be scatter while as a it's a port scanner so it's output Port should be concentrated to one or more you know few specific P right and it's a s scanner so it's out TCP flag dispersion should be concentrated to Sun Flex right so is it enough you know with this uh three basic features we definitely can you know get the same part scanner but far from in you know some some FL SS will be mistakenly Capt mistakenly captured by these rules by this three basic rules so

we need to eliminate it in we can see some simp simp attack is against a whole class C suet or in bigger so uh we must check whether the out talk IP SL 22 24 scattered or not also samon FL will carry an extra pillow to consume more benefits so we can use the out talk package average size to uh as as a filter and also you know scanner normally detect each IP only once po scanner only detects every IP only once so it's out duration average would be definite one definite zero you know and uh only in one second and and it's out talk IP count average should be definitely one can you get

me so with this four deise features so we can eliminate the th flood Source noise and get the pure simp scanner let's this one and I have a question here know uh we can see that although we have more than 100 features but some we can we are earning enough for when it comes to a specific case right here I have a question suppose we can see I is outflow appears very huge and out TCP flag is concentrated to S and act so the question is how many possibilities this IDE could be you have the answer

section is no yeah so the question is the first question is how many possibilities you know the outow spike rate is high with is out TP flag to say and act you know this the only thing we can say so how many possibilities the first p and the second p is you know we can add more features to distinguish the variety of possibilities so what feature can we used I don't see the answer right now if you in in it you know we can talk in private if your answer is right I'll you know send some present for you from China the answer is three you can think uh what different three possibility this could be you know outlow Spike it's very

huge and it's out TCP flag is concentrated to S and EG there are three

possibilities okay already done with this right yeah that's all okay

please uh Sol the

same yeah actually uh here's the real that uh I can show you the example

minutes

okay

that

you mean the most difficult one right most difficult one is uh is the one that I don't know you know lots of scanners on the internet some scanners is for specific protocol you know from the N FL data we can say it we can see it might be scan but we can't to you know give out the detail all the evidence to say actually is a scan I can do

that

please

sorry

I

so we can talk in detail so

okay that's all thank you thank you so

much

real

I'll explain it you'll see it'll be fine you're good all

right Bob you good all right cool all right welcome everybody so yeah yeah yeah they seems fine uh so uh going to talk today a little bit about uh the sock World security operations center is somewhere I've worked good part of my career involved in as a vendor as a analyst as an engineer I kind of built one uh or two um so kind of had some problems along the way and fun lessons to share with you uh so got a lot of choices of IDs and IPS type Technologies and kind of causes some issues so uh we just start off with fun details of why the sock is so important because even when you have security folks and

you put out a tweet like this people still click the link so this is about as sketchy as I could make it uh including ending in the uh Shockwave extension I still had 47 people click this I I'm not quite sure what to say it's a completely madeup thing ex eyy yxo is something that you know pretty much made up that the URL is completely made up although kind of made me want to buy the domain just for fun but uh yeah so people will click on links no matter what no matter how sketchy you make them even when you're intentionally trying to make them sketchy still 47 people clicked this link I do not understand that but that

is why the sock will always be necessary until we can completely remove the user's ability to cause an issue realistically probably not anytime soon uh so little bit history uh kind of started here so slightly jaded on the IDS side uh Source fire makes snort one the primary IDs is available uh so slight Jade towards that realm of things uh so do a lot of stuff right now uh R another bides Charm City or Baltimore still trying to figure out exactly what the Charming part is but you know that that's not here nor there uh an allocated space nice little hacker space down near BW airport and we did a CTF and made a funny logo um how

many people remember what that's from so anyway um so one of the fun uh exercises we did was building one of the rare things to combine these two does anyone know what the left is or what it's supposed to represent is anyone beer yeah industrial Control Systems this is actually a couple plc's connected together uh so the idea was we were actually taking an industrial control Network and trying to apply security to it and monitoring um surprisingly this is not very often done by people who have a lot of security experience um it's kind of a weird situation when you're like wait no one does this how does our critical infrastructure get protected and there's a lot of praying

involved which scares the crap out of me but that that's kind of the world that is um it's growing quickly so the number of people that have gotten interested in indust control system monitoring is increased dramatically but trying to put snort on it a little bit tougher it's a little complex uh so now I'm actually doing a lot of sock and snort training and some free stuff with an allocated because that's what we're all about uh so network security monitoring because I'm definitely stealing Rob's awesome kids comics uh thank you rob uh so as we're looking at stuff on the network we're trying to identify what's bad what's not and deal with it so typically that involves do we need to

send this over to instant response to handle a problem or not pretty simple yes or no but it never is it's always questionable because we never have enough information or the time to do the full research to guarantee oh this totally isn't malware or this you can generally figure out if it definitely is malware cuz you know the bad things that happen callbacks but not always so probably the most important component of the sock is its ability to do this and we uh so tuning uh this is actually one of the most common problems with a security Operation Center is you get a ton of tickets more than that team can ever hope to handle in any given day

throughout a given day so you're forever falling behind well that doesn't work you're going to miss stuff it's a guarantee by Design so tuning is actually a lot of people consider turning off noisy rules no that's removing visibility from the environment that while it definitely reduces the workload you also might miss stuff so the hard part is actually making that judgment call how to adjust the tools that you're using for monitoring to remove the noise without losing visibility of threats ideally threats legitimate to your environment so I justest but if you have no Linux in your environment is a Linux exploit going across the wire really of huge importance well maybe but usually not unless you're trying for more thread

Intel given the current world of the security Operation Center where you're struggling to keep up with tickets you don't have time for you really don't have time to mess around with things that you are not vulnerable to unless you're trying to identify specific attackers we're going to ignore the whole discussion of the government side of things where they do have to be concerned about nation states because the average business that gets breached not necessarily by a nation state uh so one of the uh fun things Everyone likes a model and it levels and fun things like that so sock maturity is one of the things that tried to put a model to so reason being

current it's kind of that flail scenario um kind of sounds bad but when we have more tickets coming in than the amount we can process on any given day and we're guaranteed to miss stuff it's kind of bad and kind of like we're accepting that we're flailing but that's the reality today uh a lot of Security operation centers really don't have visibility into their environment if you say okay we have SQL injection that would work on mssql and you have to ask do we have any mssql no one knows that really sucks makes the job incredibly hard so visibility into what's going on in your environment is really important and this is not just an asset list that

you make once a year because let's face it how often do people stand up things without letting you know in security if it's not daily you're probably disillusioned in some cases unless you're in a small or um so level one and level two arguably are interchangeable visibility into what's going on in your environment what should be there what is there ideally the difference uh and detection of known threats so what I mean by that is there are certain threats for example certain exploit kits certain exploits certain indicators that every time a security Operation Center sees it they know this is bad there's no nothing that we know of that's a false positive against this we have almost 100% accuracy on these

specific rules things that are have matured over years and all the false positives have been mostly removed or identified at least then we can say okay that's almost a guaranteed win the combination of these two is kind of the basic level that most socks should be targeting right now unfortunately a lot of them are stuck on number four um it's kind of a little early for that uh so once you have things that you know you can detect accurately blocking them in certain orgs especially industrial Control Systems moving to block things can be really hard in industrial Control Systems if you block the wrong thing people might die so there's a a huge risk associated with that so most of

those orgs in those control networks you will struggle to ever get that agreement that whole idea of getting business involved and working with the security processes pretty much getting that prevention inside the control system Network probably never going to happen um it's just a reality of the earth of the culture right now so kind of hard in that scenario in other corporate scenarios the first time you block a uh sea levels email or website they want to visit um questionable decision or not right that always becomes a problem and a discussion that falls downhill and generally turns out badly sometimes ending up with a separate internet connection for the sea levels iPad um that has happened and a whole security

stack to go with it so once you've moved in that prevention mode then we can start okay once we're preventing our easil easy wins then we can start to look at threat indicators this is not a popular opinion among companies that sell threat intelligence for obvious reasons basically saying hey if your sock can't prevent things you know about do you really need to have more information than you can handle when you already can't handle the almost guaranteed threats uh for some reason this is not that common today of an idea I haven't heard that from practically anyone everyone scringy at thread Intel feeds um it's definitely not wasted money but the value you're getting when

you can't handle the known threats is secur is seriously diminished um so once you're able to look at things that might be bad such as threat Intel indicators hunting and actively searching for threats or things that might be wrong in your network is a great thing typically in a lot of so organizations executives are pushing for this because they heard about it on insert podcast here see whatever RSA or some other buzzz bingo game you will hear you know they hear these things and they demand we must have this thing and it's really hard to push back and say here's proof that we need to do X Y and Z before we're here stating we

can't do this and we can do that is very dangerous at times with certain Executives and explaining the reality of where things are currently at where capabilities exist and even evaluating that in a lot of cases is hard the difference between one analyst and another may be dramatic so that kind of puts it in a very difficult scenario to have this discussion uh so we come to some stories uh so we've had a lot of breaches lately this is why I say that the socker world is struggling and or failing depending on who you ask um so funny thing uh the example on the left actually had uh some members of Target in the audience the last time I did the

half version of his presentation that was really interesting uh so apparently there's a lot of misinformation out there not surprising but there is a lot of information about that particular breach and OPM obviously had a good bit of information um which is a little more disturbing even uh but there are some similarities and some recommendations that they made that are ironically pretty globally appropriate um so recommendations this is the one that scared the crap out of me the most establish a sock apparently that was not something OPM had from uh reading the recommendation report that's kind of disturbing for that size organization um again with the possible misinformation on that but that was the interpretation from the report granted

240 something pages gets a little hard to read but um segmenting between trust zones so there's a lot of it's kind of buzzwordy to me but actually having proper segmentation so that well your high your high protected Zone such as your control system or finances or other critical thing that makes you special so the coke formula that's in the vault that's something you want to have behind additional protections right seems reasonable ironically sometimes those are just lumped in with every other server sometimes in the same subnet as workstations this seems like an easy separation hey servers are here maybe even the more secure servers are in one spot other servers are in another segmentation is huge and today it's

really easy for small networks you can literally set a simple cheap pizza boxes a firewall if you need to or a very inexpensive firewall to separate these things and you lose a lot of your issues where one data separation for compliance and other issues but also actually protecting it so that if the web server is infected or the email server having that not be right next to your critical components it's kind of important but a lot of times this segmentation doesn't happen additionally with the workstation same idea Finance should probably be in a slightly more protected area than sales CU sales is going to click on everything Finance they're just going to click on everything from financy

things um instant response team so uh a lot of organizations have this idea of a great document that's written that shows who's going to be involved when a breach happens or uh any type of emergency scenario but actually walking through it it's kind of a you know thing that doesn't happen often and tabletop is good but obviously when you have a you know fake piece of malware that's calling out to a server you own such as from a red team or threat emulation team that's even better but not many organizations are testing that um database encryption for OPM kind of out there the system was ancient and they said they couldn't do encryption on it um encrypting your critical data

seems very reasonable it can be very expensive and hard but if it's that critical you should probably figure out a way to make it happen so before we condemn them too badly right uh how many organizations are really that different how many organizations have proper tuning of their tool set so that they're not getting more tickets than they can handle I'm still looking um great segmentation where you know different workstations are in separate groups with firewalls in between them and detection of threats in between them usually not two factor in all remote and high priority on a remote this has become a lot more common on high priority not as much but in some ideas

um ticket time so from one of the the fun details that I had heard that I not sure if the accuracy of and doubt at this point based on things were said was in one of the breaches it was three days from where the initial entry point to when they actually exfilled data so I've heard all kinds of numbers a day five days depending on the complexity environment the attacker how lucky they are how much uh design they find all kinds of things but the reality is right now their guesstimate based on their research was something like 140 days the attackers don't need that long if we can't get the amount of time it takes to determine if something is an

actual issue dramatically reduced we still can't win because if we can identify things a 100 Days Later well that we're way screwed anyway they've already gotten all the data they need yes certainly it it was a year and a half at one point may have been higher at different points but the reality is it needs to be in days not months or years so it is getting better but it still is not near what it needs to be um application wh listing becoming much more common still has its issues but it defeats a lot of the um simpler attackers most of the things that people are actually being breached by today aren't necessarily incredibly sophisticated except for the fact they

did good research and planned um customization of tools isn't typically happening in a lot of scenarios for regular commercial breaches it's commodity tools it's commodity equipment so can't really condemn these organizations too too much OPM maybe but some other ones we really aren't that much different we're just really lucky we haven't been that guy yet or we don't know we're that guy yet a lot of organizations are experiencing the reality of finding breaches and I question that 10 years ago how many more were being breached and didn't know it and never found out that they got owned because they reimaged or replaced Hardware before they ever were able to detect it do you think if the same level of

sophistication as some of these attacks do is your organization really prepared to deal with these attacks if there were an someone on keyboard and they were able to get let's say a common exploit kit and then use Metasploit to move around your network would you be able to detect it within couple days most organizations I've talked to lately the answer is no so again kind of a dark turn but that's kind of the way things are so IDs wise the four most common that I've found um snort circot bro fire eye there are others um these are probably the four most popular that I've run into snort obviously you know uh seems the most popular open source uh free if you know

how to implement it um that can be kind of tough although it's gotten easier over the years um corporate support is through Source fire again where I got started so obviously some one of my favorite uh signature based started off in 1998 so has a good amount of History um a lot of implementations today all the way up to they have a 40 gig a second box so at least from the corporate perspective if you're talking huge networks it can handle it if you have the money to support it or the resources there's a um next we have uh cirata so cirata was actually snort project forked back in 2010 but the primary group seems to be

uh oisf um also signature base took a slightly different approach on things I'll get back to that later um so kind of diverging paths a lot of people have questioned and said they're pretty much the same thing and 90% they are but you know really when it comes down to the engineering of these things they're kind of complex so when configuring snort a lot of people look at the roughly 30,000 rules or so that are publicly available and that's what they see as far as configuration and that's a lot of stuff to configure completely understand and there's categories and things to make that easier but it's still kind of painful a lot of work depending on how much time

you expect to spend it can be a significant undertaking what most people don't look at is the actual configuration of snort which is another 2,000 lines of configuration that directly affects how it interprets those rules um this is an incredible amount of data and the documentation does an excellent job of explaining what a component or what the options are in components but it's kind of hard when you don't really have diagrams of like what's going on and how it works and how it affects the whole deal um so kind of complex uh so just for firewall wise uh at dinner the other night uh zero chaos from the wireless Village was talking about a problem he he was playing with

where he was noticing differences between uh what he was getting from I uh wire shark with what some firewall rules he was implementing so this is actually the diagram sorry it's a little small um of at what layers different components of Ip tables and EB tables are actually working at um he was using this to try to figure out where his problems were happening because the path of how the windows or sorry the uh Linux libraries are actually interpreting traffic kind of complicated as you can see and trying to figure out oh wait where does lib peap grab stuff fun question no one really has put out that information um so apparently the tool that actually grabs out rules and was

showing uh the actual where rules are catching with some of this uh was um net filter kind of an interesting tool to play with uh pulling out where rules they're triggering so that was kind of fun playing around troubleshooting IP tables and EB tables rules so anyway sorry for the tangent uh so snort inside this is actually the components that are going on and the breakdown of them this document surprisingly did not come from anywhere at source fire and nowhere official for that matter it actually comes from my buddy's blog um they have no document similar internally at least when I worked there and from the engineers and programmers who I talked to were still there they still

don't have anything that explains how traffic flows within snort kind of worries me um blind seeker.com is the uh website for this particular document you know just to break it down how traffic is acquired within snort and at what happens when certain issues take place if the interface is busy is never inspected by snort the acquisition libraries Brickley packet filter again there's a lot of things if you don't know what they are Google them if you're interested or come talk to me after um pre-processors this is kind of the bread and butter of how snort uh does its initial analysis and protocol uh breakdown where it will actually normalize a lot of things so that the

rules can be more uh have better coverage with a single rule so for example the httpp processor differentiates between if you have Apache or if you have uh IIs it will apply the same rule but with slightly different analysis based on the actual uh web server you're running this comes back to why it's important to configure stuff because that's definable by you but most people don't Define it kind of a makes it complex for it to do its job there's also pre-processor rules that identify uh protocol issues so this can sometimes be helpful a lot of times this is primary source of noise um because I think there's even an icmp detected in other words a ping detected rule

somewhere in there so if you turn it on you get a few uh you get a few alerts uh in some environments Millions a day or an hour uh so then we actually come to where it processes rules this is probably the most simple portion of it but not really um there's a thing called Fast pattern matcher that makes the rules more efficient and how they're analyzed based on the exact traffic each packet or stream only gets the rules that are applicable to it it gets kind of complex from there so I'm going to leave that one alone but it only applies the rules that are applicable based on a couple things it's kind of a little bit more

than just oh it looks at traffic and identifies patterns not quite there's a little more to it that's why there's a little config Ro IDs which another awesome tool I'm a big fan of um kind of a different design than snorer bro or snor shirata that look for explicit things explicit signatures bro is more designed to analyze traffic and give you an idea of what's going on so uh started actually a lot earlier than some of the other projects uh over 20 years old at this point which is kind of funny considering how much network security monitoring was happening at that point initially uh was started at one of the National Labs and kind of became a doe project um because

they wanted the ability to analyze odd track traffic that was not standard such as industrial control systems that don't like to follow any standards even the ones that are specific to that industry they just don't follow a lot of them sometimes they do sometimes not so it outputs by default a whole bunch of different types of logs this is what I consider the go-to ones um there is uh 60 or 70 different log types that bro can output depending on what it sees um some of the data can actually be converted into flow data directly so kind of that Network metadata of what connects to what and how much of it doesn't seem that important to the sock

until you start to think about wait is that server supposed to be communicating over SM or I'm sorry is that workstation supposed to be connecting over SMB to this other workstation why is that happening you start to look at flow you start to see interesting things that should or shouldn't happen in your organization or hey why are we suddenly seeing 50 gigs of traffic leaving over an encrypted channel that we don't normally use is that intentional should that be happening maybe flow can be a really good indicator of major traffic changes or unusual protocols that you don't see in your network bro similarly does very similar types of things you can also identify um things using non-st Ed ports

uh handshakes certificates all kinds of interesting metadata about your network traffic this can be really helpful when you're looking for large Trends um or just understanding what's going on in your network how things are communicating sometimes it's different than what you expect um there's sorry it was over 50 log 50 log types or cell um 10 of which are self- Diagnostics literally what's going on Within row uh this is a understanding how your tools are working kind of affects how you interpret the data kind of an important bit um I'll share out the slides later that actually has the link to the breakdown of all the different log types because there's a lot of them uh

history-wise for bro um I said originally started in 95 about 2004 it got doe funding and really took off and grew dramatically and has become hugely beneficial especially in industrial Control Systems where a lot of standard tools don't really do as much because the protocols are pretty specific sometimes you're asking questions that really you don't want to write a signature for if this happens why does it happen when does it happen that kind of thing so kind of becomes important but it also is huge for just general understanding of what's going on in your network and ability to ask a question do we have anything that's using ntlmv1 write a signature or sorry it's a

under Bros terminology a script to look for that specific traffic behavior and identify it and then associate it with a particular packet which can be really helpful when you look back if you have full packet capture not always available but if you have that the ability to look back and and say oh here's the source destination and get all the information to figure out how to make that not happen anymore um kind of an important bit when you're researching or asking questions and trying to solve problems fire eye um from everything I could tell very customized snort and cuckoo combin in a cool way it's really nice pretty interface very scalable close Source um about a little over 10

years old my own only problem with it is that you can't see the secret sauce as an NSM guy I want to know when you alert on something why how how do I prevent that from happening is that a false positive well I can't really look if I can't tell why it alerted on that traffic unless I reverse engineer something of theirs and try and do that and that's not encouraged and probably violates some Rules and Things agreements probably bad don't recommend it um realistically though you can talk to them and they can help you but having to send every questionable thing from fire eye to fireye say Hey why' you alert on this it's pain the ass and

realistically it's not going to happen again we have that time verse value battle really hard to win um for our signature based tools snort and cirata we have two rule sets um competitors there's a few others other than these two um um I have found kind of a difference in their focuses um this is actually what I got some argument on in the past after I did the first iteration of this talk uh so far Source fire seems to be a little more focused on these are things I know are bad seems reasonable uh emerging threats has those and but they focus a little more on the threat Intel side so things they see IP addresses user

agents things that aren't necessarily um they might be bad that threat Intel side of things proof Point emerging threats kind of focuses a little more on that seire definitely has some of that um so slight variation and there's actually a bit of preference with this Source fire as you amazingly might guess Works a little bit better with snort than it does with circot and vice versa ing threats Works a little bit better with ccot than with snort especially when it comes to IP based rules now based on the engineering alone I have not done the actual timing testing of this but emerging TH or uh cirata actually analyzes IP address based roles uh the IP address portion is

analyzed almost first within a rule whereas snort says well I know how to make a firewall rule I don't need an IDs to make a firewall Rule and basically if you have an IP address based rule it's a firewall rule not an IDs rule again some people non-definite indicators it's a point of contention I have not proven either because it's a lot of work to do that testing but according to the circot developer who responded on Twitter they analyze IP address B our IP addresses first first snort it's pretty much last um other sources digital bond has some nice stuff for IC FBI throws out some stuff especially through the ISAC community so specific uh sector

information so if there's a threat against energy or financial or whatever they tend to share that information through these organizations there's a few other paid sharing groups or opt-in sharing groups especially government stuff there's a whole bunch of threat groups that share data that way for signatures some of them are good some of them are bad or I should say well written and not well written um and then there's the threat intelligence groups that you can buy the feed to and see what fire or what IP addresses do you think are bad today so deployment is always a complex issue with this um a lot of people struggle with actually like how do I set

up these tools so one of the more popular versions is the security onion distribution very nice package comes with pretty much all these tools already set up and about six or seven goys for it um if you use the iso it's great for a lab environment but you have a lot of stuff you're not going to use so please turn it off or the I found the more efficient way would be to actually take the uh C or security onion package and deploy that method there's a package deployment grouping that you can use for security onion um other options Rock NSM is kind of a neat little other tool similar idea it's a distribution except

there's one tool for visibility and then um they have in that same box everything as well and they actually added a full pack capture tool which is kind of neat but takes a crap ton of resources to run uh Rock NSM box more than security onion because you're doing full pack capture um Autos snort is one of my buddies projects da um he actually put together a tool that is a bash script it will install and configure snort and Au gooey and even does some of the painful things with like SE Linux with of those goys that don't play nicely with SE Linux um very painful so if you ever looked at 1500 lines of bash I call that a programmer

but you know he says not anyway other fun discussions kind of a neat way because you can apply your corporate security policy to a image that you create and then add snort and a configured on top without having the struggle of like trying to implement XYZ requirements onto an already existing ISO or package so uh kind of the other one of the other painful things is training um within the sock world how do you take a say high school student with no experience make them a sock analyst who is beneficial and prepared so not very solid way right now um a lot of it is like networking background and get a degree and maybe couple sand courses that are pretty

new um I know it's kind of an expensive option but realistically have vendor trainings as well here here's how to push the buttons in this tool how do you learn how like uh what was the question the lady asked me uh when I did a variation of this in Augusta lady asked uh well how do I teach someone how to identify the bad things on a network work well crap that sucks that's a hard problem what do we do today well we kind of hire people that might have an idea and throw them in the river and watch them drown um kind of people pull them up when they have time and overtime and the pain of oh I screwed this up and

this was malware and I thought it wasn't or vice versa false positives false negatives you start to improve realistically we need a better method or at least a way to simulate that so you're not doing that with like live malware on the network because when you say live malare in the network is not a problem that's kind of bad or vice versa you're creating more work for an organization that has a struggle with time management and not having enough time to do its job this is a really really complicated problem there isn't really a lot of solutions out there um on the good side there's more company popping up that are trying to work on

this Sans is trying to work on this they made a new class specifically to Target continuous monitoring uh about a year ago so still out of beta but not quite as mature as I'd like to see but hey we're all dreamers right it's developing a course for this is kind of hard there's a lot of problems and a lot of understanding a lot of tools that you need to know how to take a b c and get them to work together and that's really complicated because you need to learn all the tools and understand how to put them together what's the PRX for that class and then how do you start right like that's not

easy so kind of a hard problem so some steps forward um kind of that tuning thing of like hey what's our biggest thing that's alerting us right let's start to look at that and start to identify how can we not lose visibility in the actual threats and start to filter that out so if we don't have Linux do we really need the thousands of Linux rules and alerts that we're getting every day maybe not similar with database architectures or other things you can start to turn off things or filter it down so that SQL injection for example you have matched the correct database type with correct database server again details things lots of complex in-depth time

researching your network and asking questions hence why for I typically recommend a signature-based IDs and something like bro to learn about your network the two combined actually work really well and are powerful furthermore you can pump snort into bro to actually get like connection IDs for alerts that's really Nifty and helpful um password vaults and uh that's everywhere um instead of focusing on closing tickets and trying to close the most number you can at a day reducing the volume of tickets if I can T close a 100 tickets or I can take out a category of tickets from happening again I will win much faster by actually removing the category and reducing noise for the future that that needs to be the

way we start to adjust tuning and for some organizations tuning needs to start happening not just turning off rules or not just ignoring snort alerts I actually had an organization that was getting um it was 2,000 emails a day of about 50 alerts a pop for two guys and we didn't get the option to actually adjust the tools filtering out the things that we didn't care about was probably the mo the only way we could ever hope to catch up it's only 8 it was only 800 machines so there's no way that number of attacks was going on or bad things it was all false positives pretty much mostly from the same things there were certain top rules if you look up uh

there's Performance Tools for all these component or performance components to all these tools where you can actually look at what's taking up the most resources that's generally what's the highest number of alerts you can also pull that in your sim and start to look some of the statistical data and remove your most alerting rules not remove them but actually solve the problem that's causing them as well because again turning off rules versus solving a problem it may be something in the network is misconfigured and you tell the network admin hey go fix your crap you need to turn off this thing that you didn't know is on that you don't know what it does and is causing

noise or figure out how to configure it correctly again complicated problems um as a uh sock improves have a pentest focus on testing the sock emulate a new threat and figure out how long it takes to deal with that know how good or bad you are that way you can gauge Improvement it may not be perfect but if you can simulate a threat once a month and have your senior guy say okay we're to customize whatever and one of the Nifty things about Cobalt strike yeah Rafael mud just plug anyway um he built in a way so that the implant that you use you can actually customize to look like a specific piece of malware

so if I want to uh send out something and make the call back look like a particular piece of malware that I know my sock has detected that is a hugely beneficial way to test how do they respond we're not pentesting the sock today for the most part almost no pentesters I talked to have ever done that they've never tested with the intention of getting caught that is a reality that we need to figure out how are we doing if we know we can detect something how long is before sent to remediation this is a really tough issue pentesting the sock is not common but it needs to be so lots of information lots of stuff I'm trying to work on this

myself with some lovely classes starting a training business called forgotten security hey when you know I already have the handle for the last few years anyway um yeah I'll be doing a beta in the next couple months locally down in Maryland hacker space fun times are there any questions a lot of information but hopefully not too many confused stairs

yeah so again it depends on the maturity of the team right I've got known definite indicators that are I can fix this I can solve this problem it is a nearly guaranteed win or I can submit a false positive and have them actually adjust the rule to fix that or make the ad adjustment myself that's solvable when I have something that might be bad it's really beneficial for mature sock for a sock that's struggling to deal with known threats they're not ready for it it's a progression and while we need to protect our organization as best we can focusing on things that aren't cost effective in time is not the best use of our time

again not a popular view but the best chance of stopping today's commodity attacks is to focus on the commodity identification that we have as the tools get better so all sock as we solve the easy wins that aren't so easy right now we can start to look towards that more complex threat or just something different new variants if I can't stop five years ago does the attacker need to make a new thing this is the joke right now with ap right well what's a well if they can use commodity Tools in win they're not going to burn something that's custom I can win with Metasploit why am I going to burn a custom thing and make the world

aware of its existence I'm going to use the cheapest tool that I don't expect to get caught unless somebody deems it necessary to use a customized thing right now the crime groups and the crypto group or the uh all the fun stuff that we're having right we're not really seeing that much custom work we're seeing mostly commodity stuff working and it's causing our problems when we can stop commodity let's start to focus on thread Intel we're not ready that's kind of a bad unhappy view but that's reality other questions okay thank you I'd love to talk more if you come up with

any how's it

going

that you get permission and I I think this is probably the the for many people this is the one that usually drives the point home right um going to affect your job security and I see a couple Chuckles right oh yeah if I do a pent test if I do a fishing attempt on somebody what's going to happen I'm gonna have that awkward phone call the email of course if you really screwed up you get the email the phone call the text message another email you know it sort of goes on right and one of two you I don't want to say you're going to get fired right you probably won't get fired that day you know that's

actually the worst thing because what have you just done your career opportunities your upward Mobility that we all want say bu-bye right because you did if you do a big enough boo boo without permission you're just you're not going to go there you you're GNA have to take yourself out of that firm right it's going to be that career suicide type of move right and of course why don't you well the reason they don't want you to do this right you affect the business business of the customer you know what is it your I don't know your interest Industries here right Financial how many anybody no nobody Financial okay so is everything bad about Banks today how

about Healthcare okay got a couple uh education couple utilities nerp scada manufacturing anybody transportation we all pentest for fun that's what we do the rest of us well so this talk is is going to be one of those TR try to understand the help you understand the points of how what a pentester does affects the customer right the business of the customer right because you're going to directly affect that end user or the customer or the business of helping that customer right because you do this attack and you take down a system what was that system used for most businesses don't just power up silicon they don't power up machines just because that system has a purpose

right was it the ATM downstairs serves a purpose right and you know that that bank I don't know I didn't look at the fee on that MNT Bank what is it three bucks I see a knot of ahe somebody actually used it right if you if you had to use the ATM they made three bucks for using that ATM plus all sorts of other backend charges right that maybe I don't know about that's how businesses make money so if you take that ATM out they can't make their money so we want to get some Buy in and I'm going to talk a little bit it's you know it's actually not just a yes or no

decision I'm actually there's actually some nuances here that I wanted to make sure I conveyed out because these are things that I've seen in various Contracting engagements you know now that I'm internal working and trying to get some buying on some of these efforts right uh who can authorize it that's a right and what is their power level that they have right you know you may report to a manager right but if you're in that like Silo do they have power to do everything to anywhere I saw a couple people say Healthcare right your CEO your CIO may say okay but the CMO the chief medical officer in my experience they tend to frown upon that my God don't no no no no

I'm saving lives here right you're paying the these surgeons like what a like $500 an hour crazy numbers right don't affect my surgeons don't affect my nurses don't affect right don't take down Blackboard the educational system God help us you know especially at like end of semester right so when you're getting this authorization do they have the power to do so right you know and is there communication you know you want to do things covert and Silent right you want to be sneaky but actually a certain degree what do I mean well say I'll work and I'll report into a CIO but I make sure that the CIO has talked with at least one person in a couple other

segments of the business that I might be touching hey by the way I'm going to send this email to your tellers I'm going to let you know okay I can't count them as an attack right but I'm going to let them know or I'm going to be working with a floor or a department anybody do a pentest against workstations and semantic endpoint agent on that machine the semantic uh their little IDs solution the host-based intrusion detection system what does it do when you attack it pops up doesn't it you get those little popups from the agents it goes hey someone's trying to attack me right so usually I have to check in with at least one person hey when you're people

on the force when they see this pop up it's okay we're doing a scheduled test we're doing a penetration test and you say well why well I get to the sometimes why is they'll tell me what hours I have to do it in or they're 24-hour shops so I let somebody on that shift know at least one person knows what I'm doing and this is not red teaming okay of course red teaming you're going in and you're supposed to be very stealthy right but a lot of the penetration test assignments you're doing you're just seeing if you can exploit systems you know and you have to get by and and you have to address what their

needs are uh and what what do I mean by that you know I've done a couple projects where I could pretty obviously tell that they just wanted to meet a compliance checkbox was pretty obvious it was like one of those checkboxes on a form somewhere have you run a penetration test it was almost you know it was like what are their needs they were very minimal I could have walked in did I pen test and said okay I'm done and then they could check the box right they their needs were very minimal other customers though what are they asking for hey I want you to test my people right I want you to see what the

response time is you know I want you to other times it's very guarded what you can do though so like your your internal customers in your organizations you know if you're working inside industry those folks who are doing a pen test in your organization or you're hiring somebody right who's the customer who wants this report you know is it management right is it Department management or is it Senior Management is coming up from up high you know if you see the Senior Management you need to we need to as pentesters we need to dumb that down quite a bit though don't we we can't talk about packet traces and peaps if you walk up and say peaps Senior

Management has no idea what it is right you know you know you can't say I pwned the Box they'd be like well what's pwy uh you pwned a box um yeah but uh I don't know what it is is it compliance and do they just want to meet a checkbox you know Healthcare we had meaningful use requirements and you know in PCI you have these requirements you know I love that you know PCI I have to require an external third party to do my penetration test to prove that the boundaries are sufficiently defined it doesn't really say how good they have to be that's up to PCI the the security standards Council to judge later right you know so you know are

they just meeting a check box the the risk Department maybe these folks actually want to see some more detail right because they want to see the different levels um you know and you know or are you just proving your own work I mean some of you are you doing pen testing to prove that you're securing your systems not bringing in a third party sometimes that's what you're doing and are you proving your own work you know so what's you know what are your customers needs you know how how in depth do I need to make this test and when when you're working with these clients you know your internal customers or your external customers you know we

need to make and I kind of I start to talk about external clients here but really our internal customers we have to be mindful of this too right if you're working for a consulting firm you know I said what is it peaps uh you say peaps yeah watch your those external customers that you're trying to get on the hook if you want to do a pen test for them and you say peaps and you see their eyes glaze over right they go you know what's the terminology what do they care about they're concerned right it could be a compliance need it could be a risk need it could be proving their team's needs but you want to make it pable you want

to make it understandable I think it's the other word that I really want to convey to you right here in that it's something that they can relate to right what are their needs that they have right and you do want to stay a little bit within their Comfort Range even though I know all the this cool stuff and I could break into the Fort Knox and everything they have maybe they only maybe they're starting to walk right and maybe they're not ready for the full-blown the the full guns right so we know what's their level of pain right do you know sometimes you get a customer who says hey take me down go ahead I

dare you right how often has anybody actually gotten that usually when I get what was that so you know yeah that's about the percentage it's it's a very low percentage go ahead take me down um you know why you've probably been through a few comfortable penetration tests you kind of tested the waters and that's what I mean it's sort of like people will test the waters and they get a comfort level up and then you can come at them with everything you've got and their level of pain is going to change you know I've been on projects where it was like well if you find something that Met esit has a module for come see me

first okay oh okay so I have to wait and if I think I'm going to break something down I have to bring you in first I have to make sure your people are here first right what's your you know can I even take the system down or did they tell me to stay away from a whole lot of systems right they take all the fun out of it right you get that a I found this great vulnerability on your on your EHR your electronic health record system or your core banking system right well you as a pentester should know up front if your customer is okay with you taking that system down I've been on those projects

where I I could just go it's an obsolete version of AIX what's your comfort level don't take anything down going to stop right here I've already hey need to upgrade that system right and I'm not going to take it down because it's going to cost you too much hurt too much harm it could also cause a system malfunction right nerp SK of systems we have a little we we have some real loss of life this malfunction could be catastrophic to us right um anybody done a pen test on a printer yeah oh yeah you can actually and what are the one of the things that happened to the printer spits out paper right you know systems not available

remember a second ago I mentioned the semantic agent when you go and probe it it starts yelling and screaming hey someone's attacking me right that's a system malfunction people would call the service desk I thought it was kind of funny though actually and this is just real life example I'm sitting in the room and I'm sitting over in the chair with my laptop and I'm using a machine that is not named anything remotely like their environment and the semantic agent comes up and says hey this machine is attacking me with my machine name I really wasn't being that silent right I wasn't doing any silent pen testing it was like yeah this is to meet a

compliance check box we're just going to hit the button and go but you know it's funny listening to the service desk and I'm hearing like the third call and I'm like sitting there going okay I'm I'm inadvertently testing your incident response system here because hi I'm attacking your network and you're on the third phone call okay now did I call some system malfunction thankfully not really we knew that semantic was going to just scream and yell and and and they said that the system would time out on some queries and actually for this system I it was like go get a cup of coffee the machine will be done with you in 5 minutes so go

get a cup of coffee and come back because it was the individual workstation and they had a whole bunch of them and hey we have to check the box yes we've we've we've checked and we can't attack we can't get into your machine okay I can't so just get a cup of coffee but if this is your core system you can't do that right and how long does it go down right with your external customers and getting that Comfort level with them like how good are your backups do you have fault tolerant systems can you switch over to somewhere else you know is this peak ordering time like you know uh you know your e-commerce sites on

Black Friday or Cyber Monday or if they're activating all their devices the day of Christmas or the day after right peak times for some of these folks right and when you're working with external clients why do I why do I care about keeping their comfort level or stay within their comfort level it's this repeat customers hey I'd like to keep coming back again right so you know you look at some of these these customers oh I used XYZ company and I'll never use them again has anybody worked for a firm that you used a pen testing company and you said I'm never going to use that guy again that guy was an idiot the guy was a buffoon he was ridiculous

he didn't know he was doing nobody nobody no wow CU I don't know I've seen some bad reports in Consulting I've seen reports and you go to read the report and you see it disagree with itself in the same paragraph like oh now I see a nodding of the head there yeah yeah that was from a major accounting firm and I was like excuse me they you know or you'll see these other reports you're like what did you do right did you scan everything what did so you know you have to address the customer needs whether it's internal or external right and you know if you read a report that shows everything it's what do you we do we

tune it out don't we oh geez stop telling me that I've got SNMP Community string on every single system tell me once and shut up right you have to address their reporting needs you know drinking from the fire hose right we have to give the them something they can use and this is these are the ones that sometimes with pentesters what are their needs do they want to see everything that's wrong or do they want to see just the do they want it like distilled that was one of our customers they wanted me to distill it down into a project plan for them actually so not only and that was actually part of getting permission

because it was well we know we have to do a pent test to me to to meet the compliance check box but they said but our team can't keep up with this this this all this drinking from the fire hose you know we got too many things to fix here right so actually what we had to do we actually had to take our efforts in our pent and part of that buyin we had to agree to give them a prioritized remediation list so that was something so when we went into the test we had to think about that right and so it it really tailored what we did because our goal was actually not to bring systems down in

that project because if I find enough things that are very lowle fruit I don't need to break a system down I can give them those high and so we're staying within their needs right um you know and these people here you know the the clients they really don't want to see nothing of course right um and when you're getting permission and buying make sure you ask these kind of questions because if you ask them do you have an IPS do you have an IDs if if this is assuming of course it's a white box test right and some of these tests are going to be white box and you can ask at a high level do you have an IPS I

didn't ask you what model it was I didn't ask what firmware it was but do you have one so I can as a pentester right I'm getting I'm asking you permission but I can also dial back my efforts a little bit and give you a useful report and the reason I say this is you know if you you if you don't get enough information on one test you come back for the second test well they may not actually ask you back right if you don't get enough information so as you're doing your test make sure that you're doing this test so that you're going to be asked back because if you give me a report and

I I saw a major company a service organization I I jumped in I jumped in I looked at the report I would really did you bring my IPS tea and cookies too and I'm thinking it's like because there was nothing on the report they didn't ask any information so they went and part of this Buy in if you're going to do this you got to ask me enough about my environment so you can give me a useful test now that I'm an internal I'm like hey you got to give you a useful test so I expect people to ask me a couple questions right I expect people to ask me what what's your environment like at

a very high level because that way you can tune your efforts against my defenses I mean now as a pent Tester the other side of the coin of course if you're working with a team there are tricks I've used them you take rapid seven Suites of tools and you do your vulner scans and then you give it to your colleague who shows up on another IP address but they know all about the environment or at least they know pieces of it but you know how many IP ranges can you jump from right to SPO to get around the IPS there are ways to do it but you know if you do it without permission and this happens a lot of

times internally and also if you cross over a department line or a division line that you don't have permission to touch right well it becomes ethical it's just another attack it doesn't win friends because the system administrators hate your guts now I know we don't like system administrators do we wait a minute weren't we system administrators at one point before we became pentesters isn't that usually the way we go you know so hey you know you know it doesn't win friends if you don't have permission right it makes enemies and it doesn't yield positive results because people start shutting down on you they won't talk to you they won't resp respond to you you ask them things about their

environment if you're trying to tailor the report as a pentester I want to have that dialogue with my customer because I want to learn more about that system because you know sometimes your customers can tell you well the reason it's not upgraded isn't because I didn't I I chose not to do it maybe the system's not upgraded because I didn't have the budget I need that as a pentester because I need to put that back into my report because that colors the risk right it's going to change my writing and remember this is all about getting management to pay attention they'll read I've heard people say the first four pages of a report and then

they're out to lunch you see them flipping and looking at all the pretty colors no they're they're just G what what do I do with this right first four pages on these reports right so you want to get positive results if you don't have permission how do you know when you can end what what is the goal to win the game wasn't that actually all of our I'm not wearing the shirt right war games right what's the goal of this game to win the game but he didn't know how he won right well that's what if you have permission when you get permission I'm going to talk about what that means in some detail but you you start setting

boundaries and scope and limits and what you can do and what you can't do what's allowed what's not allowed that also tells you when the game is over okay I know this attack is over because I got to what my project plan said I had to and of course if I don't have permission unemployed I don't like that you know I always joke I you know I don't like paying for my mortgage but I like being able to pay my mortgage okay I like having a roof over my head so you know let let's try you know as pentesters we know all these Cool Tools and yes businesses should listen they should but you know sometimes you have

to say things in a way that the business will understand that the customer will understand how is it going to affect them what's important and how can we help them down that path so with permission oh okay I'm going to start talking with my stakeholders right what's in scope what's allowed what kind of activities are allowed right what hours what hours can I what hours can I what kind of tactics can I use how brute force can I be right our certain systems go away setting these ground rules in place this is part of getting permission it's not just can I do a pent test but we have to start asking what's allowed and what's a success what's a

failure what's like my safe word to get out of here right what's the safe word go uh quit done you know we're test over right and we have that negotiated and that's important because when I was a consultant yes I wanted to know okay you as a customer if you want out of this this is a word this is a word this is a phrase this is a term this is a condition where you could tell me to stop now of course it's also valuable to me because if you tell me to stop as a consultant we're still paying according to the original terms okay so that's okay you know that this is all part of that negotiation you

tell me to stop okay I will stop but we've had our you know we have these terms arranged beforehand what is a failure of a test and what are the legal issues that we need to be aware of and I think that's actually my next slide I wanted to talk about legal issues and this is a big part of why companies no don't come here don't pentest my system why they're worried about confidentiality and the non-disclosure agreements and you know are you really going to honor my non-disclosure agreement do they have sensitive information they may tell you no so when you're working with a customer make sure that they they can trust trust you right I'm going to break

into your systems I am going to test the security of your systems but all of your information will stay secure it will be retained XYZ you know rule set right and this could also affect what is what systems are in scope and which systems are not if they are that concerned right don't just say hey it's not just can I do a pen test it's how far can we go with this you know do they have security issues like PCI right uh are there privacy issues I put a few down here just to make you aware you know healthc care has got hipa issues to deal with you know medical centers insurance companies and so forth Wilmington un

University and schools like this right Burpa all right glba you run into in the banking space you've got this privacy regulation you've got Federal rules State roles and other governments uh other governments do they do business with Canada do they do business with entities in other countries how far is this right now how do you get permission to to take the system you know if you're going to penetrate a system in another country are there different rules because you know here in the US we're we're very L fair but if you look at uh like Europe European Union they talked about that whole Safe Harbor the gdpr you know they've got these you know I was working at one time for an

organization managing a console and if a device from North America showed up anywhere in the console because the console was physically over in Europe I wasn't allowed to see it until they specifically put it into a dedicated place that I was allowed to see it after they verified that it was a North American system because Europe said I couldn't even see the name of that machine because I would know your first and last name and I would know your IP address and I would know what office you were sitting in I really didn't care but I had to be aware of that legal issue and so pen testing we have to pay attention to that

as well especially now that we're looking at Cross boundary issues right are they using a service provider is it a system that they can authorize you to pentest right that'll be a very quick shutdown right being aware of those service providers that are out there especially if it's a shared infrastructure right you may not even be able to do it it may be one of those like failed projects now we say what's in soap what's in scope networks right what sections of the network are in scope you set a limit you don't attack everything what systems is it servers is it desktops right statistical or judgmental scope so you could do like I want to do

10% of systems right you don't pen test everything or is it you know I think these systems over here are riskier and well you know why I you might use a judgmental scope I don't have time and I don't have money to scan everything but if you only scan certain systems I have enough money for that right so there here's like that that if you're doing like a internal right and your Contracting with somebody how do you control your costs I've been on this side how do you control your costs you tell me to only scan certain systems how do I control my effort I ask you how many systems I have to scan or

do I really have to scan everything do I really learn a lot by scanning every single system they're all pretty much cookie cutter especially workstations if I and even if I do a statistical sample and I see variances then I just know you have variances all over the place so you could do like 10% of systems that's a very high sample really at that point I'm really seeing some variances and I Know How likely I am to get into these right so you can ALS so what are we doing here we're limiting the damage we're limiting the impact to usability cuz people can't use these systems right I might do a denial of service on them

all right well I'm only going to take down 10% of systems remember I said to that one system go get a cup of coffee if you have another workstation I could say hey go use another workstation because my scanning software will just come to that one and a half hour and we'll just rotate right uh departments or groups certain groups are going to say nope I N I don't have time for you it's critical business time stay away right and make sure you work with your business units and also Define what devices and I I gave you the example of a printer but you know there are other devices and pitfalls that we need to be

aware of and these are some real world things you know nowadays you go into a financial institution they'll print checks for you right check printers check scanners all these other devices that they use now the ATM is that in scope does it create a malfunction on that device right instant issue devices so you know manufacturing controls uh biomedical devices sometimes there's not allow of value in testing these why the uh biomedical devices they can't patch them it's this whole song and dance with the FDA I can't patch the system because the FDA says it's no longer FDA certified so don't bother scanning it because they can't do a darn bloody thing about it you could scan one

of them that's all you that that's all you're going to get in value from right and so what are the like some of these biomedical devices I'm just going to scan that on a test Network I don't need to test that on production Network because if whatever I scan on one device every other device should be just like it right and at that point I could do a vulnerability scan not a full penetration test far less intrusive in fact I probably wouldn't even vul scan it I would probably just do a manual check to the ROM ver you know the firmware version you're like all right you're done so multi function devices you get some strange

printouts core systems you know we get into some systems that can cause a catastrophic failure I say catastrophic because you know I've now seen these where these are weird systems that you'll like saturate the buffers and the system looks okay like you know we used to always just say I'm GNA take the system down right and you go I'm done because I didn't take it down today you know that's what we expect right we're going to pone the Box we're going to take it down today what I've seen now with some of these systems it takes it down sometime later during the week cuz actually you set you took up all of its buffers and

it didn't tell anybody didn't scream and yell but then like two or three days later there's a malfunction oops nessus will do that and I'm like really yep all right so we need to be aware of this right um of course in the healthcare space it's when the patient is on the table you know that that is you know I don't know when they're going to be on the table so I need to be very cognizant of that right um I I put this one down here because we all probably have run into these guys when that VP or that important person is affected you hit that person especially if you oops right okay it happens right they

told me certain subnets and he decided to go to the office that day or a different office that day right so sometimes huh you know it depends I mean usually I if if I see a failure with the box I go it's okay there's another one over here I I do laugh when VPS get infected with different systems oh no never never never never and then you send them a fishing email and they bite oh crap I didn't want that to happen now of course how do I get out of that situation remember I said communication people knew what know what's going on all right people know what's going on enough people know not everybody but enough

people so you know of course a catastrophic failure if you take the systems down and the staff can't serve the customers or clients that's a problem so what is allowed you know obviously uh you know test systems often a good candidate you check on each system or class of system right I don't need to check all of them uh sometimes I have to check to see if I have tech support on staff text support handy on call or sitting there right now right is it or sometimes after hours I've actually seen organizations do both of these that's why I can't give you a golden rule because I've seen some that say only do it after hours and I've seen

some some that say only do it when my application developer is here because if you take the system system down I need them here so you need to make this is part of that buying and making sure it's a true buying that they know what they're in for or what their comfort level is Right does the client have to supervise it I had one project it went on and on and on the scope was large right and I had to do it with supervision and then after we R we ran through it then we had to redo parts of it I said can we please just rerun this and then you don't have to really watch

me so I can just let it run overnight success or failure what does that mean to a client do they want a clean report that shows nothing is wrong how many people work for organizations that love to see clean reports there are a lot out there especially if they're meeting a checkbox um some organizations right the staff wants to see a clean report because they're using it for the evaluation of their people and and as pentesters we need to be aware of those hidden motives right because they're not going to like us if we don't you know we not that I'm going to change my report of my efforts but I need I may tailor my

efforts you know what I'm saying because I don't want to single people out I have to be careful when I write that report how the test results and how I write that report because I want to get out their needs and I want it to be a useful test it's not about punishing people but it's training moments and that's a phrase that I think is very good when we do pen tests and we find failures they're really teaching moments you know definite failures if you fail to meet the C client's needs or if you deviate from the scope you're going to be in trouble here right you cross over the line and definitely don't think this

is cool in front of the client hey I pwned your box like um in front of the client in front of the customer we're evaluating our security you know we may think it's cool right absolutely I I've been there I've been like sitting there in the room and thinking wow this is awesome I pwn that by wow I've got a an image of the Radiology system and I'm like excited like that too right do I look that excited when I'm in the meeting with the client no okay they're not going to want me to come back again and they automatically yeah I told them something was wrong but they're going to say the the test was a

failure so we need to make sure as pent testers part of that job also is going to be reporting that out and that gets us the permission to come back again all right so our call to action trying to keep us on time here I know I'm in between you and the Raffles you know we learn a lot of cool stuff a lot of cool penetration testing techniques tools techniques organizations need to start somewhere and sometimes we need to uncover their pain points we need to find out what they want what do they want to learn how much pain do they want to feel right how realistic do they want this to be right color Within in the

lines they give you lines they give you a scope don't add to it right don't I think it would be really cool if I go ahead and do this right because a you just deviated from your permission and you're not going to get to come back again uh make sure you have the Buy in um use your get out of jail card some people say oh I could just get out of jail yeah yeah I have that right I have that notice from legal that says yes or that notice from the CIO that says yes could do this pen test I should only be using that as a last resort it's not a hey I got this I'm only using that if I

really have to okay because what happens if we need to be responsible in what we do and how we report what we do because you know we want to see some progress in our organizations in our customers that we work with you know that was one of our customers that you know I had to actually change the whole project plan and I had to give them a whole prioritized approach to fix fixing their systems CU they had done it for like four years they weren't happy so we changed our approach to get them to buy in and to do the and to do the test yet again and so what we did was we we set

the pace to what they were willing to achieve and we we we were happy to see that they made some progress so that they slowly move along that maturity model and ironically this is I think this is Sans this is like control number 20 this is one of the last ones you're supposed to do this is actually for mature organizations and you know why a lot of people are drugg into this compliance requirements you have to do a pen test what's a pen test I don't know you have to do a pen test I don't know what that means there are many shades of pen test and different organizations we just as pentesters we need to be aware of

what the rules are get the permission stay within the color with the in the lines if you will and that'll hopefully help us continue to do more of these so questions was that no yes go ahead what was my best client's experience I think when they finally after the four years and they understood uh the prioritized approach of the remediation and then they actually bought in and they had been kicking and screaming to do it and then they said well we're going to do this and they actually they actually ended up taking possession of it and it's funny they're so they were no longer a customer of that organization but you know what but they're doing the right thing and of

course this is a small industry they're going to tell their friends other questions Al no questions I'll stick around for a couple minutes thank

you I didn't do

it