← All talks

AdamDoherty

BSides Calgary49:136 viewsPublished 2022-12Watch on YouTube ↗
Show transcript [en]

foreign

[Music]

best practices I think on a Friday afternoon so uh way to be Troopers um this is this is going to be me rambling for I want to stay close to an hour I'm hoping I'll get close to an hour but if I ramble a little faster maybe we can have some questions at the end so uh this is uh Thanks for opening the lights on and uh what I want to talk about are the Forgotten blind spots so uh with Donna I'm gonna go ahead and start uh able to close this door

all right so who am I why would you want to listen to me for an hour uh my name is Adam Doherty and I am somebody who is a professional Problem Solver because that's how I label myself I'm a senior consultant with the ground strike and what I do is strategic advisor effectively what that means is I am in your organization I am looking through security boxer and I tell you how to do it better I really enjoy that part of my job because at the end of the day I believe finding your problems before their problems and fixing them is a much cheaper solution um you know what kind of what kind of credit do I have uh what I did cyber

security the oil and gas industry for almost a decades but I have been interested in this for far longer so I'm going to start with uh with some statistics because I mean who doesn't want those on Friday afternoon um this one is absolutely classic 81 percent this is uh the number or rather the percentage of incidents that organizations faced where threat actors had valid credentials and I've met valid credential was kind of molded there but you can really read that as people passwords or password hygiene um really it doesn't actually take a lot of effort for threat actors to get into an environment and um your users bad password Etc are a good place to look

277 it's the average number of days that spread our kids in your environment before you discovered I mean a long time it's like what is that a almost almost nine months maybe 10 months I don't know it's it's Friday Matt um but that is a really long time and a lot of people when you talk to them you think oh you know we would we would recognize right away that we have a threat requirements and we would deal with it right away while um the statistics prove out that that is actually otherwise 7.3 million dollars and that is in Canadian dollars that is as of this year that is the average cost of a database

in this country so you think about the smaller the size businesses that are out there um you know the costs that are involved with dealing with the reach itself and dealing with the after effects um organizations you know they they either can or they can't afford this in a lot of cases they can't uh and what a lot of people don't consider are they run artifacts it's not just the bridge itself it's not replacing equipment it's not securing your infrastructure better it's the situation when you look at the employees that work for a company if you've worked in a small mom-and-pop type Organization for the last 10 years it gets hit something like this is going to shut

them down and now you've got everybody that's out of a job so this 7.3 I mean it's a figure but uh the figures are a lot worse than that if you were to add it up so I mean with that weak introduction I want to say that I think what we have nowadays is fantastic modern infrastructure is amazing I mean we've got the cloud we've got hyper-converged technology and that is it's wonderful it's changed the way we do businesses change the way that we interact um you know it's uh this is actually sharing uh this is sharing these slides of my presentation so you're welcome um yeah this is uh this is what modern infrastructure looks like and what I

want to talk about

okay this is where we are we've got serverless functions we've got containers we've got virtual machines it's all really fantastic a unit to compute keep getting smaller right which is great for reducing the attack service but you know um we'll get into that it's also good for the bottom line organizations can do more with less and at the end of the day they end up making more money is to not having to spend money on maintaining server farms and resources and whatever but have you ever been uh thought about things at the other end of the scale uh pasty containers virtual machines and hypervisors all of that out of the clouds and down at Ground

levels uh what do we have here we've got things like our Banner Remote Management storage arrays backup appliances iot devices other things right these are the low level assets that make our modern infrastructure possible and what I want to talk about is the remote out of band management because if we go back right at the top the remote out of band management is on all of these other devices and it's something that's been present in Enterprise servers and moving on down into the medium Enterprise and small business uh since 1998 uh when Intel first introduced space and uh you know it's it's great it's it's made system administration that much easier but you have to remember that it's an embedded

computer with complete control so you've got your big expensive server but then you've got this little systematic chip that has complete control so you can power recycle the system you can get into the biosk it changes but the interesting thing about this remote interface which is always not is that it's accessible via weather phases SSH if you're lucky um right it goes by a lot of different names that will calls it eye graph which the closest ILO all right IBM calls it immi 2 now um but really the protocol person introduced by intel was ibmi intelligent platform for management of infrastructure um and this this is the Crux of the problem how to ban Remote Management is great

but what's very persistent with things is also very good threat actors ipmi allows the threat actor to try and cut the system all right it's uh I'll find Goods to have the modern security stack but you know looking at the image we have on the screen you can just drive around the gates uh can you really call it security and It suffers from a number of weaknesses right at this level there's absolutely no safety net right you've got your modern stack your operating system all of your EDR your other security tools all stacked at a level where you can use them if you can see them down here there are no stupid tools and the

implementations for authentication are weak and the default passwords just Google them right there and we'll find them um you know the default passwords I stroke there rarely uh there but let's be honest it'll never change people people don't change the default passwords on their own Brothers and this is an interface where people don't change the default passwords so you know access at this level happens over an unencrypted channels right it's not using TLS it's using uh old school HTTP protocols for the webinar place if it is using uh SSL it's using self-centered certificates and uh you've got that ubiquitous telnet interface which is a fantastic protocol that sends things in plain effects and you know the firmware

for these devices is rarely and maybe I should have insert it never complicated I mean when was the last time what being in the firmware on your other band management anybody I would love to see somebody today

what could possibly go wrong well let's let's talk about that uh at abandoned milk management abuse uh these are just a small list of things that I can think of um at this level with complete control um you know be it a a VM host a shared storage array um some kind of sand uh backup points right you can re-initialize the rate uh it's your uh there's no controls there the system will happen we go oh yeah you want me to reset great uh give me give me a couple hours

drop that malware right inside a uh not a file system uh or uh encrypt the file systems at this level um you don't even you don't even need access to the operating system itself you can just go ahead and mount the data that exists there um my next line who runs are fantastic but uh in this scenario it's a very scary thing plus this thing has got network access so why not just exfiltrate the data at a real low level when uh be done with it store malicious payloads down here you can store information here that survives uh rehabilities of the system and you can just keep me infected surviving things even when you provide

replacement right these things have their own onboard storage uh I'm sure you can come up with other creative things to do in this environment and this is this is just what I came up with off the top of my head but I think that for any organization in any of these were to happen uh this would be a Medicare scenario especially if you don't maintain multiple properties I give my it's got home meets Visa it's everywhere you want it so that's her turn

how about a thermostat it's to stop someone from using that kind of status you'll notice here this particular thermostat happens to be riding a game at kennel what can't do or a printer or your security cameras right you've got all our IP security cameras are are hooked up I mean how many times have we heard about IP cameras used in blackheads and even the light bulbs I don't know what the lights in this room but in my house 80 I think I think I've got my 80 light bulbs down every single one of them is smart right every single one of them is on my iot network and when you think about that treatment think about this all of these are

network connected computers right it's a computer that just happens to be brightening up the room or capturing surveillance or printing a document for you and uh these these systems are all running software and what is software but somebody's interpretation of an idea and all software is vulnerable right and in the case of these devices these pivot Points classically they had weak security default passwords if they have passwords at all sporadic updates although really it was ever updated the firmware on their smartphone don't bother is anybody monitoring the traffic from their printers or their phones to their IP cameras to see if it's we'll talk thanks but I'm going to put the asterisks in I'm going to say that

lightly most people aren't doing that and uh that's that's a scary thing are we sensing to think these devices exist within the boundary of our networks right lingling with phones and computers storage systems and file servers and whatever else we have in our organizations our Enterprises our homes our schools doesn't uh doesn't matter okay and they're only here with implicit cost and that's the problem so what can we do sir let's see what we can do continue quite a bit segment your network this is something that in my job I spend a lot of time talking to people segmentation is possible and segmentation is absolutely something you should do there's no reason for your smartphones to ever need to talk to your

I don't know your Exchange Server right exchangster is going through some difficulties right now um I I would hate to think that I could access email from uh from a compromise light bulb or an optimized thermostat or some other mechanism right segmentary networks make sure that traffic is only moving between authorized devices for authorized reasons segment your services is there any particular reason that a database server needs to be talking to oh I don't know the service that manages your sprinkler system or your Access Control right maybe you need those databases for those things but think through what those needs are and make sure that your data flows between those Services uh are documented and absolutely Justified you know limits

the amount of communication between your services as uh as another type of segment paper segment your applications right we've got web applications we've got database applications HR emails Finance you know group these things together logically for the areas where the information needs to be shared right there's no need for information in your HR departments to be accessible to the people in maintenance you know and if that's the case you need to look at how you've segmented your network your services throughout

I'm just gonna hide this one but I'll talk about it anyway identity aware of proxies are your friends and it goes by a bunch of different names some people call it zero trust right but it's a risk reduction strategy that allows you to integrate the technologies that already exist in your environment for authentication and optimization to Grant access to these applications and services and systems right it's just basically a traffic cop that's saying not only there is this system have access to this service but the user who is requesting that access do they have permission to access this service on this server from this system right it's about applying those policies that add the user into the mix

right for that

information when we're down at this level right use dedicated hosts dedicated systems for administrative activities there is absolutely no reason you should be conducting any sort of administrative system activity from the laptop you check your Corporate email or a personal device okay I have a dedicated laptop or a dedicated voice station that is time to a dedicated Management Center and that system is only able to access the systems in the title that management you can't get into it from the Executive Suite or the coffee shop

change default passwords and I'll just say change change the forecast please I I wish that devices and device manufacturers would take the concept of default passwords and make it such that the device is actually in function until you change and wouldn't let it function if you choose a bad password right so please look in your environments what kind of systems that you're responsible for you know or or even your home networks at home if if you're still using please guarantee you a two-second Google search is going to be able to find that password for your device or your system or your application okay this thing will get 12 accounts right there's no need for them

create accountability create user accounts that have appropriate privilege for the tasks that they need to complete right there's no need for anybody to have a god mode account or a super user account or a new level account for day-to-day database those should be the exceptions and those should be the things that restoring and privileged access management where that gluten password for that device is very and when somebody needs it it sets off an alarm with your security and somebody says hey did you just check out your password what are you doing okay create that visibility create that accountability update your firmware okay we talked about updating operating systems updating our phones updating anything and everything that's that's

running a piece of software you've got bios firmware lyos firmware on our servers we've got firmware on iot devices that stuff has Flawless too and vendors do fix them they're not exactly louder than telling you hey we've got updates but make an effort to go find them and update the firmware for these devices in your life because the threat actor you can compromise don't because they already know about it update the song for it too more even even low level devices running software will have tickets but especially with these low-level devices right we don't we don't think about these things because it just works but uh for example run unify devices at home when they won 4J uh you know compromise

came out we're a lot of patches going back and forth I updated old my Wi-Fi infrastructure they called me you know because I did not want to become in some way you know because you can't think that so update yourself this thing will unnecessary services I don't want to exactly know why a multi-function copier needs a mail server or FTP or any other insecure service running on it exactly uh but if you're not actively using your environment turn it off it's just another Avenue for a bad accurate to cause a really bad day so you know go back take a look look at your configuration what's on by default go turn it off remove the Gateway on devices right

especially in a segmented Network right if there's no need for devices in your network or at that level or in your management to get out then turn off the vehicle you can't exfiltrate data out of a network if the data's got nowhere to go such a simple one that I I don't think anybody ever talks about that one but just removed it doesn't stop devices within the segment but it sure is having prevents you know some Infected malware that got onto a device somehow maybe a USB stick don't use those uh it's it's not reaching out to seat you right because it's got nowhere to go

stop using outdated protocols now officially helmet became recognized standardized protocol in 1983. uh but the first RFC that it actually talks about it was RFC 50. it's an interesting read from 1969. that's a protocol that's only 50 years old why why are we still using it right I know why it was it's easy to implement but security is is a balancing that just just because it's easy doesn't mean you should use it SNMP for uh device monitoring and something that's been around since 1988 there are more secure versions please use them use good passwords um HTTP it's been around since 1990. if you look into RFC 1945 I written in 1994. um you know that is

a whole discussion around that protocol the interesting thing about low-level outer band management like I came on um it it already ends you know weak mechanisms for interacting with with the environment um they recently added uh the ability to perform API calls to it and address their place but it's still insecure so it's just that just another mechanism for maintaining security at the environment so please stop using outdated protocols show you something like SSH I'm sure everybody here is everybody in here's a analytics user or a BSD user and nobody actually uses windows um but uh yeah no always use the secure option if it's available and if a secure option isn't available find your router find a way to protect

that device with additional hardening additional firewalls jump boxes you know the uh the bad day that you can ask if these plain touch protocols uh compromising an environment doesn't matter how long and the security that that password is if a threat actor can see the traffic across the work they've got a classroom turn on remote access altogether I mean at the end of the day if your Hardware is sitting in an extra moment get up from your desk and go and interact with the server you don't actually need to administrate from your desk right and and really you should be interacting with it if you lower at your desk from that dedicated host that doesn't have any other connection

um this one uh I I am really particularly keen on the idea of what I call a get cloud understand the assets that you have in your environment right you can't defend what you don't know about and if you don't know about a device in your environment go find out right locate those devices and organize them so that they're no longer unmanaged if they're supposed to be there document if they're not supposed to be there document them and get rid of it and once you've gotten around make lots of noise notify your operations team mode uh infrastructure supporting the animals I'm realizing that that got cut off this idea is that you've got your devices in your

environment you know what they are but it's the guys that are supporting logging or security or anything else help desk right it's they don't know that these devices are there they can't do anything when somebody calls and says hey I've got this situation where you know my connectivity is broken or I can't access this file and mowing the whole you know it was a file share on Bob's Raspberry Pi we plugged in six months ago because it was convenient for him to share a large USB drive your help desk can't support that they truly support that uh but if your infrastructure team doesn't know about it they can't support it they can't support it they can't be

offended right and a long all the things I can't stress enough how important it was one of my favorite things long it's somewhere else it's a really great that you've got lost uh but if those volunteer is sat on the compromise device first of all that device is no longer yours and neither are the ones the ones that are not going to be helpful to you so while it's somewhere else well I'll get to somewhere immediately right that actors nowadays to delete their tracks and delete any other ones associated with any of the activities they already done right if you send your logs somewhere they don't have access to and you send those logs to a device

where it's an append on what you want even your assistant administrators or your higher-ups or whatever you can't delete them then you effectively have an audit Trail that allows you to go back in time and say hey at this date and time you know this happened no listings okay those lungs don't live on the device but they will give you that all important information Institute for instant response of the word um legal actions there's a whole bunch of reasons that you want I mean even if an application breaks and all of a sudden it's not generating laws send the application because you don't have that record this is an interesting one and I get some some questions from folks on this

now Monitor and justify all your data forms you know understand how traffic is moving especially at the low level and between your applications and services is this service supposed to be talking to this survey is it supposed to be talking to this server on this port using this protocol right monitor these things and ensure that it's document ensure that somebody knows that has it been approved right this all helps when you need to figure out why something happens or you know in the inverse it helps you to find situations where something should have been happening almost so take the time I know it's a lot of work but understand how data is moving across your network between your devices

between your users between your services and and have a reason for all of it create network situational awareness that's that's really what it's all about you can't defend your network if you don't know what's going on and at the end of the day that is the end of your business organization or you know whatever you have to be into that's uh it's a very bad day be your chromium with promotes right the guy uses a plan oh security is making me do this or security is who cares security rules exist for amazing right to protect the organization to protect the asset to protect the user okay work with your users work with your teams to try to make things easier to

come up with solutions that allow a user to do their job but don't ever skate around the rules right because your users are going to try to figure out how to get around those rules and be straight up with them I mean we're in you know on cyber security awareness really we should be working hand in hand to say you know this is why we do this and this is why this security controls because you know threat actors will exploit the chinks in your armor or the weak links that you can insert uh I don't even know what you call it attics or whatever here but that's exactly what the bad guys are doing they will find that weakness they

will export it same goes for any other controls you have in your environment remember at the end of the day whether you're a school or a business you know be that smaller or Enterprise Fortune 500 it doesn't really matter at the end of the day these are your assets this is your organization the controls that you have in place are there to protect you and the people uh that work for you so again if you've got users that are saying hey these controls are are causing issues with what I'm having to do for my job we're quickly to come up with a solution yeah and and try to find that solution before they try to work around it

but I'm sure some most all I don't know I've probably heard all of this before

automate everything I'm not a fan of doing manual tasks and processes and if I had to do anything more than once it deserves to be audited there are such good tools out there for security automation everything included on the slide that ansible is one of them you know there there are tons of vendor tools there's vendors out here that have got great security use them put them to use in your organization right resources are finite people are finding money is fine you can't have all the points by the continuously we do have automatically to make your Journeys automated to a lot of things automate them to alert you when people are logging into things that they should

automation is fantastic and use it to your advantage that helps secure your environment make your attack pass really really expensive okay and what that means is that concept of segmentation of your services applications networks users right if a threat actor comes in or looks at an environment and says you know what this is going to take me a whole bunch of time just to get down to this one thing that I'm looking for like a security system or white it canvas or whatever right it's a deterrent where a threat actor is going to say you know what I'm going to move on to an easier Target because this is going to be a while right it's a compounding thing and if

you remove that implicit trust that exists with a singular boundary in your life you do make that attack as really expensive and and remember that security isn't a some cost I come up against this a lot and I have come up against this in my career with managers who said you know what this this tool and this solutions you can be expensive but you have to remember what what exactly are we trying to defend here what exactly does the day look like if the business is and I'll give you an example I used to work at an oil and gas and one single day of downtime for that asset is in excess of 30 billion dollars

one day of downtown now I will say that the organization I work for does a lot of incident response works with very big customers all around the world we've got an engagement going on right now with a customer that we've been doing instant response for them and all of the things that are associated with that since January and so you can think if you've got an asset that it costs 25 30 million dollars a day and it can't operate because now we're conducting an investigation right somebody at the end of the day is going to say you know what we're going to just drop that from the books because we can't afford to be down

and all of a sudden 1502 34 5 000 people around me Joe and what does that mean what does that look like for your organization if a tool costs a hundred thousand dollars what is that compared to all of the expense of all of that okay security is not the same cost and you saturated if somebody says hey we're not going to get this tool because it's too expensive asset what is one day adaptment purpose what is one weekend always a good counter argument proactive measures are always cheaper to have me and my team come in to do an assessment for you assess your environment tell you where do you have the option make recommendations on how to fix them

that is a technical life cheaper been dealing with an instant response team who comes in for this one particular Point they've been there for 10 and a half months now right and they pay for all that Consultants time they pay for all the infrastructure that we need to run to prevent investigation right this super expensive proactive measures are always cheap paper is not going to protect you right I don't know can I get a show of hands in here who has had a manager say oh you know what we'll just we'll just accept that we'll write an exception for it there's a here's a document it's accept that oh yeah yeah you know what at the end of the day

paper doesn't save me a threat actor doesn't go oh oh you've got an exception here great I'll I'll stay away that doesn't protect your business it is not a security control that is you saying you know what I know it's a problem I'm not gonna fix it it's always cheaper to fix it and and you know going back to the idea of what's capable of either management it's been down this foreign

remember that firefighters are called after a fire sticks your incident response means like the company I'm working for who's got some of the best buyers in the world but in term responders are the Artisan investigators they try to explain how it happened they're not there to hold your hand and try to save your organization they're there to explain to you

remember incidents are not accidents I uh I have a wallpaper that I created for this I really should share it online because this is this is the personal adage I share with people okay security incidents are never absence there's always a cause for whether that's bad configuration default we have four practices you know lack of security Awards these are all things that all organizations everywhere should be practicing a lot of cases apart remember to turn the lights off don't make it easy for them

[Applause] so uh get a nice small space here we get time for questions sure anyone anyone yeah on a slightly different topic what is your take on honeypots or honey nuts to try to lower hackers and understand their results I mean realistically there there are another two um you know they're they're a great way to you know try to ensnare people and deal with the situation but I like to think that that's just extra infrastructure that you now need to manage really you should be focused on plugging the gaps in your own environments um security tools are great but you have to remember that tools on their own won't save you you can put in the most expensive tools the

best tools on the market um if you don't have the base best practices those tools are not going to save you any okay it's getting around your controls because you allow correct doesn't really matter

questions um but I always encounter devices that require multitasking and this thing wants to talk on the same you know behind that your cell phone is on for example and I've never really had much success marketing multicast or getting it to go close directions [Music] uh well I mean in those particular instances uh I I have actually set up throwaway networks uh that can communicate um I do device isolation on my Wi-Fi so that um cell phone clients for example can talk to other devices but I mean you can get you get really granular I I can't really speak to Consumer grade devices I will say that if you go get something like unified which is

I want to call it cheap Enterprise here I love it because it's got that level of control right um you know I think you have to look at what your um threat model is right and what's realistic um in in our home environments keeping the perimeters secure and keeping your you know non-chromecast devices for example on a separate Network segment like I keep all of my lights and things on on a different setting right right it's it's uh abundance finding that value it's it's really hard at home because we've got all of these smart devices and you know kids their own needs to get on the Wi-Fi to work on whatever it's where it's uh I I

don't have a good answer there when it comes to stuff at the Enterprise you know I'll say throw a firewall at it I'll say why are you using a Chromecast device um there are better Solutions being Enterprise but I think in the home space

well thank you foreign