Modern IT disaster recovery: Ensuring continuity in the age of cyber attacks - Tarek Habib

welcome everybody we got the first talk for the basement today it's from T from abmg and it's called it Disaster Recovery I want to tell you his resume to his own slides I don't want to delate any further but I do have one thing I want to tell about him that is he doesn't like to tell people but took a pandemic to get him to learn how to ride a fight so and that at the of 30 so it was it was just that thank you let's so

little bit better

yeah okay I can y okay um so welcome everyone uh today we're going to be talking about modern approaches to Li disaster um so we're not talking about the natural disaster version um I'm not really sure decided to allu disaster the term that so we'll be talking abouts my name is my senior manager cyber security and res practice in kmg um my my focus is on business resilience cyber security

and Str especially the C that's part of big part of the focus that we um I

know page I'll bring it back up at the very end so we um the the very first thing that I wanted to put out there the disclaimer is is is the I recovery is is a key effort um so I'm not an expert on every single piece of here especially like architecture backup Solutions it really takes team eff you got and their systems systems work you have the B ofs if there's a cloud component you need arit and you need someone to bring it all together and do the planning the communication so it is a key eff you won't find a single person um that knows how to do all those things that can stand up with youram on their

own so that's just a disclaimer that I wanted to put out there um and other person that Bally mes all these things together all the different professional and skill sets together the VR program and say program capability recovery capil same I wanted to start off by talking about the current challenge the problem with v and the very first thing that I wanted to get out of the way is the difference between business and V like I said before V iology necessarily you know floods and rescuing people with floods they actually call that mer mer response so go fig um but I wanted to call it the difference between business continuity which is the side that the business has to take care of um

where you know like business fol have to figure out had a manually survive without system while the disaster recovery efforts take place and we have the concept of two safet n where the very first saf is is that manual mode where the business has manually systems and then that in parallel the ID folks are back systems so that the business operation folks don't just sit around wait for them you need multiple sa especially as conru especially as issues with one of those um there are a few different scenarios so a lot of the times disaster recovery programs been planting with one specific scenario in single data center is is is impacted data Cent are imped

but there are more different scenarios more types of disruption that you have the common ones that people talk about are a single site so you know my main dat Center my back Center they fire everything's gone or both sites are impacted whether probably like Cyber attack or they're close close together it's a natural weather event

those are the most common to but there are more there's Communications only outages so you know my data running fine especially inal places if you don't have your network connectivity you don't have access to those those systems how do you recover that there need to be consideration for that um there's also the what we call the user workstations desktops all the techologies that users have um Horizon in cyber attacks rest need a rest get focused on those on those end users laptops um then you shi it to organizational assets and now we're seeing it have more um where some of the target part of the Target isation and there has to be protocol on how cover those laptops

once they um then there's the third party struction so there's a scenario where gets ignor get Contract cover where a third party is actually Bic disaster not you you're to is fine but the third party was with dis and that's where you I'll talk a little bit about this at the end but be consideration for especially and together together or your operations inte with there has to be consideration for get get a t result from that company understand how can recover and I'll that and then the last one is is a bit of a a add on where IP personel may not be available so if a disaster happens and the key folks are not available that's

when the you to have PL in place place people and thats into that cross over into because you Personnel not being available is more but when we're talking about V those people know how to do very specifical things we have to account for that in our plan not a single person can be a single um whenever we have I disasters um the response the recy is not the first thing that takes place so whether it's a fire whether it's a Cyber attack the very first thing that you're do is not the disaster recovery part is the damage control I trying to figure out what happen trying to assess the damage supp try to put it out to make

sure they assets side attack you trying to figure out how to eat those um and then there's also step that takes a bit of time which is digital foric you're talking about Cyber attack there a bit of delay where are happen before you get touch those assets to try to so right before these things these things happen and we're not going to dig into that we're focus on what happens after you decl start trying to recover those system but I just wanted to to mention this because it's one thing thats timel and timelines is everybody the biggest thing that

got a lot of questions from people on okay telling me it may take some time me

longer and decided explain this to a diagram so I'll verbally walk through what's happening here um you imagine between where the instruction happened and where the systems are recovered users are good to go there are a few things that happen depending on the scenario we have a couple scenarios that use for this example one is just like a fire that data center and the second scenario The Ransom attack which causes more delays uh it's a more complex situation to try to recovery the common goal at the end is to recover the systems so there's a journey that happen and the steps at the end of that Journey are are similar they overlap but the damage control and investigation is

different depending on the scenario if we talk about the the physical disruption like let's say the fire scenario the very first thing that happens is you got to evacuate people you got put up the fire you got to assess the extent of the damage um and then if you don't have uh you know if you just lost those those access that infrastructure you may not have enough infrastructure ready to go at a site and so these things cause theay time to put up the fire figure out what was impacted uh go buy extra Hardware or spin that up in the cloud environment and that takes time and once you have that those assets ready you have to go grab the backtop

and put it on those assets and and depending on where you have those backops the time down also take time when you think about these things you know these three three steps take some time we're talking about a traditional scenario once those backups are download the backups to install them um users have to to check that the systems implemented correctly and then if there's any data to be reentered especially you have to go back to a older backup that takes time as well users are even clear to keep using the system as as as they would have normally done and if you think all these things these can take depending on what you have how much

preparation you have this could take out that much time in the Cyber attack scenario the very first thing is again try to do a damage assess so you know Cyber attack happen um sometimes business folks say you know all system will be recovered in hour and that may be true in if you have a lot of redundancy like a fire scenario but in a Cyber attack scenario that's probably the time that takes just to figure what happened get ear indication of you what kind of Cyber attack was it how how how white does it go it takes time to isolate the assets that were impacted the forensic investigation you got to wait for the forensic analyst to get

outside come in to be able to their analysis that takes that takes out I've seen it take more than a day especially if you're your dat Center you don't have any go people and you have to fly someone in that take a couple days before there before they take what they need and they say sure now you can wipe and use this this environment um and then trying to figure out how far back can I go to get the good backups that also takes time and you may have to do to do aoup runs in these recovery as the investigation PS and this can take days and weeks depending on again where you're recovering to how how far back

the goes the M message then they meet together to you know once you have the The Backs how back do I go then you know the back it goes back into those simar steps the reason of showing you all of this is just to show the the the reasons the the different things that could C when when recovering from the plas so that's why we typically see differences in terms of um you know the business things you can do it PR fast and it ends up taking weaker to especially some of high that we've seen uh in the news the last thing that I wanted to mention as all of this is happening the business folks the the

people operations service they're not just sitting there if this takes two days they need to have their manual workn they need to be able to work in parallel while you're doing all these steps to try to bring back the systems they need to have a protocol on how we work manually without some of those systems so so I wanted to share a bit the spotlight on on what need by manual work the example that I'll use here Isn bu of different factories and they all depended on it systems to tell them what to produced to tell them you know that this product is now in this stage of manufacturing process and if the it system that they office is disrupted

they would just stop they don't know kind of track the prodct have a label of ship figure out where to need to go and manage their inventory and as part of the business of things that's where we develop manual Works they say no you keep going you don't you there's a way where you can manually record you using raw materials shipping out a product doing manual and so we ended up developing a protocol for them templates that areed in a binder Factory on how to keep the products moving if the it systems are destructed same thing for if the what we call M systems are disrupted the systems that run the machines for some of them there

are ways where you know um you don't have to wait for for a central control system to tell the machine what to do and oper and the so there are ways and you can document so that those business operations can survive while you're waiting for system to be recovered and it's sometimes cheaper to use that manual option than it is spend Millions on duplicating of youru so question um by your show hands how many people are really comfortable explaining the the the difference of a service agreement in SLA and and an RTO recovery time objective as you see in like a vendor agreement for

example so so from from what I've seen over the last 10 years um there there's a huge difference like specifically when we talk about third party in in the commitment that you get for Recovery you get something that's like an SLA that says here's the time for our systems it will be up 99.99% of the time you'll have this many minutes of downtime per month ke thing there is that that apply during this it doesn't apply just goes out the window anytime there's a disaster for for measur as it would Beed in those in those contracts and that's when a second measure R the recover time objective that's when that kicks in and a lot of

the some some of the contracts I see just don't even have any mention of it they have an exclusion that says uh this you know this for for sure Cyber attack natural disasters and other stuff like a board off service level that happens and you need to negotiate that because stuff like shouldn't necessar let you off from the recovery time frame just say it's so we can so this is something like the difference here is the business that metric versus the disaster disruption metric um so just to wrap up the like the like where we are today um I asked any to show me IM of what it thought I think recy look like uh justly it's

messy I do agree with this representation of uh you know people running all over the place trying to figure out what's connected to what and and what to do they're stress they're breaking C data center so this is only the second attemp generating last one because it is you know if you don't have a solid game plan if you don't have a tested Dr plan that you're confident in this is what it kind of what it would look like um some of it from behind screen but this is what it look like the TR probably St so I thought that would um so I wanted to spend a bit of time talk to you about the solution to

the problem so the problem is there delays are there the very first thing that's the most popular question in the Dr topic is is immutable backups and I wanted to just start off by addressing this and talk about what is in the very simplest definition is it's a back that just can't be changed or deleted so you can't go in and change the retention to zero can't go in and delete all the backups which is what Cyber attack try to anytime they try to reach an environment they try to reach the backup solution first delete they encrypt everything and your St um again I asked a to say to show me what an IM backup what an IM backup look like and

it showed me this like futuristic a of driv like as security X like Alien X guard or something um it's not quite as as as as secure as that um but the whole concept of ual backups is splitting the control of those backups so usual you have to get a third party involved whether it's a self managed backup that you implement with like aure BL storage for example it has an option for inability Microsoft can delete the backups you can't and the idea there is there's always someone at some point that can that can mess with your backups but when you split that basically use an organization let's say your accounts get compromised they can't that the attack p

with those accounts to mess with your backups now if the vendor accounts get compromised and the vendor has that level of access into your environment then that's a different story and we'll talk about other solutions for that in but the idea is you hear vendors talking about backups that's the whole point that you they'll take the backup they'll keep it and if you want to delete it you got a call and go through a whole process where it's more difficult for factors to implement to hit your backup and your primary property at the same time there's also the debate of back backups which are typically online connected versus offline backups and the idea of offline backups is like you know

putting data ontick putting it in your pocket it's very hard to P into that uh kind of back up but it's slower and there's a case to be M where having V stick in place um they are different and I'll talk a little bit or the P um so there's the the golden rule of backup in recovery is called the 321 rule or it used to be called the 321 Rule now there's another one and a zero in the rule I'll explain why um but it started up with you know there these three things that you have to have in place and then you know they looked at and said we need more uh to

this Golden Rule and they added the one zero the original version is you have to have three copies of data so you have copy you have another copy that's like on site is that makes Hardware uh fails and then you have a third copy that you send off sidw the two is for two Med types so let's say there's a particular vulnerability and whatever backup solution you're use it it's harder to get the compromise the backups if it's on two different two different types and that's where you see people like thinking a backup to like some sort of device and then putting that on the T the traditional sense and then one of those copies has to be off

site in case there's a fire that burns that whole building the one and the zero are really talking about having one of those copies completely offline and that's not your main backup that's your new kind of backup uh where you know if your beau backup vendor gets compromised this is the one that you back it's going to be older be slower to to recover from going to be as good um but it's the last three Bo so that you don't have to pay the attackers theive or if you do pay them and they don't you know they mess up the the decryption you don't have a backup to resort to and the last one the zero is

about zero errors to make sure that you test your backups not just take them and keep them somewhere but test the test the individual system recovery Test the full recovery and test everything you're backing up so not just the systems but uh license license configuration the test test to make sure that you don't have any errors when it's actually com those backups there are a lot of backup options and decisions to make um out there the very first point is do we put file or do we keep it on on our site and these are decisions that you have to make so I'm not recommend one vers the other because there's always a situation where one is better than the other U if

everything your hosting is in environment it easier to have your back re but if you need to if you're in a business where you have to be able to operate locally I said for patient out let's say you're hospital um a cloud backup might not help you as much you need to have something that's closer that you can use communication background Communications out the other decision is well do we want to manage it or do we out and these are combinations so you can have self-managed Cloud backups that you can know there's a beam solution that that you can put your back storage and make that unable and you have to manage it yourself or you can Outsource

uh to like v as a service to vendors same thing with on premise you can self manage that you have a company that manages that backup Appliance and you don't have access to it it really depends on the use case that you have and these are decisions that you have to think about and then there's online versus offline can you get more both there are different advs the offline one is much slower but it's it's it's much harder to mess with the online one is faster you have to keep in mind the speed of recovery so if you're doing a cloud backup but you have a really slow internet connection from where you're trying to download the backups days and

weeks depending on how CL try down and the other part that I wanted to mention is is there's decisions to be made on where to recover to so having the backup is one thing by having those servers and capacity to be able to recover those assets on those backups on to is is also critical and then there are some cases where you can't have spare infrastructure especially if we're talking about devices that control machines in a factory you have to recover onto that same machine get just recover in a cloud environment or somewhere else in some cases so there are all these decisions you have to make all these scenarios you have to plan for to to have a let's say

a complete or comprehensive backr program and then that can take a while um for you to implement Dr some of the Dr maps that we do are sometimes three years it's a very complex environment um but the cyber attacks the struction they can Happ tomor and that's where the concept of FID fandid Solutions come in um just things that you know there are a lot of examples but I be here for you today these are things that you do in the meantime while you're waiting for the new sh to implemented um there's one that have an example of T in your backup environment where you go backup or offline backup for example we had a

client that invested in in a backup tool that they running in in a separate site that they had and they didn't have any you know support financial support to be able to change the backup solution because they just invested in it so the solution there was to do extra paranoid monitoring extra heart extra on that back Appliance so that yes it's connected there Expos um but to basically Extra Protection for that one week critical backup system because that's all they had access to at that time there's the more low cost approach where can be backing things up hard drives this works for small businesses uh you know I know a lot of like clins and restaurants you know they do hard

drive backups and they swap hard drives every couple days it works for a small business don't have much traffic going on or as many transa actually that they have to back up and work work with them as a low cost and then there's there's just manual mode if we're talking about assets manual we that's a lower way you still have to use which is the most expensive piece to prepare for manual to actually operate equipment manually but it's a it's a it's a Band-Aid measure you can put in place just in thetime and in terms of the backup Solutions so I don't want to put any logos or give any preferences here there are lot of solutions out there and even

different that there are solutions that are focused specifically on Office 365 backup there are solutions specifically for be brains and and different types of operating systems um when you're selecting backup solution and the very first decision is do we want to manage it or do we need a third party to help us with it and there are a lot of we call Dr as a service kind of vendors um you know some of the cloud providers have it like Microsoft recovery um as a surface U but it doesn't do all operating systems that are out there so it really depends on what you're trying to back up and what feature that vendor has um I know there

are other vendors that will give you both like an onsite Appliance for quick recovery and their own like backup in their own data center that's moreable and Che can for the same fire so they give you solution same op and in terms of what you should be backing up so a lot of the times I see people focusing on well I got back my server so you it's all virtualized I take virtual Mach stots but there's a lot more and sorry for the here but there's a lot more that should be backed up on the administrative side we we support contracts we some of that documentation we are plan numbers for for our people our emergency recuitment protocol

that kind of stuff um in terms of like more documentation it's not enough to just have the G plan you need the a lot of lot of documentation whether it's Network diagrams uh whether it's like license keys for operating system depending on how the company works and then there's the actual technical PES where you know yes you are taking backups of servers and databases but that's not the you have to have a way to recover laptops laptop you have to have configuration files for you know if I have to rebuild my part I have to rebuild some networking devices and so that's not the only thing that have to be backed up even s applications

sometimes lot oftimes they don't come with capability the two examples that have here is um one is Salesforce so I know um recently discovered a couple years ago that um if if there disruption you made with your user data so you have to either pay Salesforce for an extra piece for back up your own user data and then recover it if something happens it's a cloud application but there's a back there's a thing in there that you have to back up the other example us off 365 there are backup Solutions up there Microsoft will help you like some tolerance you those small errors but if there's something where someone goes in Messing your environment you have to

have backups with that um it's nice to take a lot of backups but there there's the concept of cost and and and ACC the bonus um this is where the recovery Point objective comes in so we talk Ty before uh the RPO recovery point is how far back do I go in the data how I back it up and the normal question here we trying to get that requirement is how much data l tolerate and it could be in hour be zero if you're like an online banking system and the two things that you have to decide here frequency I want be taking backups every minute every hour every day or week dep on how fast the

system and the and then how many of those backups do I want to keep and that's the restore points do I want to keep every backup for every minute do I want to just keep you know every daily backup for a week every weekly backup for a month how far back do I go you know maybe three years backups before that it's really as you sp to recover 20y old data unless you have to arch keep it full storage and the key thing here is it's nice to have more restor points but the cost sometimes it's a big factor so keeping more those more copy with that data and the more frequent snapshots can get very expensive very

quickly so you have to decide what what are the best how far back I need to go some cyber attacks begin they work in your environment six months they St messing around with it and they execute the the big that you may have to extreme circumstance go back six months a year or Le have access to some of that data toally be able to check something so you have to keep a year two that and determine how many how many copies how manyen want to keep at the cost um there's also the concept of automation um the easy answer here is aut back it's not always possible there are some devices that just have to man do the

backtop especially if it's like configuration firmware that you keep a copy of um the VR plan itself whenever you update it offl copy put on USB stick for example or some PL storage some things still have to be backed up manually aut the other the other part where automation comes in is isn't backups so you'll see in Long of them like the backup tools is and and the service operates is they will do a test where let's say once a week once a day they take a backup and they run it in their own environment to say and we'll give you a screenshot of that login page to say look the system turned off it's

not a full test because you still have to make sure that users can go in and the application still works and you know all the dependencies work but it's just a quick test to say can

run um this sounds pretty expensive um so right up front negotiation with the business fol on what our appetite are we going to things or toward and very Mar and strong backup and Recovery Solutions and it's helpful to have this discussion at the very beginning because you can go down the path and P aome the strategy architecture and they don't want to be any of those things you just wasted some time if you're better off doing spending that effort on manual works because that's a preference you have to know that but they also have to understand what your is a lot of the times I see um in some departs they have conversation with the management team and they say

yeah yeah you 24 hours like that but they don't have that conversation on what it's going to cost to to achieve that or the test results say three days um they going have to invest some more in different things whether it's training most the capacity to be able to to um to achieve those time frames you can give them some some time frames like you know tier one slice of your occations 24 hours tier two this many hours it doesn't matter what the ti about it as long as it's agreed with the business and you understand how much they have to invest to get there so if they if they say I don't have for this

this year have to tell them okay based on the current capability you know don't want invest anymore it's you can be down for week if it's a Cyber attack you can be down for five minutes if it's simple but if it's the worst case scenario Cyber attack me your fire you can down for a week and that way you for them of the risk of that decision they can decide the way to formally analyze this and put some numbers towards it is through a a business analysis and this is the way grab every business process you say okay you know going back to the manufacturing example starting from the end of the process if you can't share

products how bad is that for one day two days seven hours whatever it is and how bad is it over time then you go back to inventory man production production planning all those different steps and you say okay if the systems supporting that process are dead are gone or that process just can't run at all how bad is that for your business how expensive is it for your business and that helps you just find the investment either angle GRS or mod V solution or combination of both um I mentioned we'll talk a little bit about third parties um so we talk about the contract Goa moment some of theed systems um the solution there is is

first of all treat it like partnership so you know nobody wants to be slap with contract definition it starts off with you should have these commed contracts but a lot the contracts only exist and you can only make these changes when you're contract or getting contract so you have to go through governance channels to get at least like a soft commitment to say you know we you know we'll do a test using like a Cyber attack scenario give you the test results you know we give you at least a handshake or will just help you what you is so if you get a test result and you thought it was recovery four hours but test said it took us a day of that helps

you understand how much you can expect from that vendor if if C that we have seen vendors just check the news we've seen vendors that got we um and um whether it's healthare or industry whether it's a building system or any Ty operational system it could be down for a while and so you have to understand what those third parties can can and then eventually get in the contract and then the last part is there there there should be joint exercises where there's a huge codependency I'll give you an example or two examples there's one where let's say the third part isting a key system for you but it's very they integrated with your systems and if

they experience a disaster and they Implement you know in a different different site location you still have something to do on your side to hook it up to that that new recovered environment they have depending on you if there's changes in networking changes anything like that that way users access the system so at some point you need to have a joint exercise where you know they run their Disaster Recovery test and then you try to connect to that to see if your side works the other example that I'll use is is more on the on the manual workaround side is there are some vendors that like right your platform like the one that I that I had most

recent experience with is uh like a logistics management platform and the the work around was that that vendor would St up a call center to manually uh coordinate those those those Trucking shipments um and then we had to have a way like on the client side we had to have a way to work with that call center to coordinate our logistic and the joint exercise to to to to basically test that process test the people where would have to be emailing to that vendor instead of using the logistics platform that everyone's comfortable with without any of automation that had that it had in there and so they have to manually you know talk about the roots talk about the

rates and all of that and so those kinds of joint exercises are very practical especially if a third party is the one uh that's hit with the disruption um so a couple of uh couple of things I just wanted to go through just as like the remaining thoughts that I wanted to to lead you with um like I said before it's it's it it's not an it only responsibility it's not just cyber people that have to worry about this it's a whole business issue and that's why it need needs to be a team effort needs to be it's a complex issue U Dr is is is messy uh it's it's never clean especially when you're under attack um

and so it's it's all about being survival mod and we're not trying to replicate the entire the entire it environment the entire business we're trying to basically do enough to make sure that the business survives and eventually over time can recover their full it capability um understanding the appside for investment very Cru so you don't waste any time design nobody has any intention of investing in implementing temporary measures these are the Bandaids that you can Implement tomorrow um there are a lot more than the three that I just mentioned but think of different things that you can do that would save you if a Cyber attack happens more if a major fire happens in the data center tomorrow and then

looking outside the box so not just looking at your own uh systems in VR but looking at how it works in ecosystem where there are so many third parties that are integrated usually it's easier to start with the smaller vendors rather than starting with hey my soft is down start with the smaller players that that may have a small not as strong of a of a recovery capability and then work your way up to where's the bigger vendor as well unit Microsoft is down you have our Communications are very critical communicate out of Bend and so these are the thoughts that I'll leave you with um I'll bring up that innocent QR code again I promise it goes to LinkedIn you

want to want to connect afterwards um and I'm open to your questions and I got a five minute warning in the back there you have five a question I'd have a question couple maybe um so going back to the 3211 strategy that we're talking about like how would that apply in in a in a cloud-based environment because typically in a cloud there's regions M multi- regions and then there's availability zones right then there's also media types so how would this typically apply for workloads or anything that's hosted on a cloud environment yeah so if you take it with the concept of three copies that's why they have those those like clusters of theity zones um but then you also have

regions so let's say you're only using uh like one region and there's a natural disaster in that area that impacted them they don't guarante they're not going to bring you up in another region unless you that that's an extra subscription so it really depends on what you're trying to plan for so if you're let's say a local business and you operate you know I operate in Nova Scotia and or let's where the data center is and there's a major national disaster and then nobody in my community expects you know my restaurant to be open and during a disaster then you're kind of off the hook but if you're operating in multiple jurisdiction there's an expectation that

you know you're running a hospital Network there's an expectation that you're more resilient um you're running an Ecommerce platform you have to you have to look at those scenarios and say okay am I okay having my business taken down by by a single like major weather event if not then I have to pay Microsoft or the to to to to put my stuff in multiple regions because they they're getting more copies that helps you with the copies you they have fall tolerance in one availability Zone you have a a backup and that helps you with like you know a single data center burnt out but then you have to have the other region or a separate kind of backup uh

that's sufficiently far away so that you know a single major you know hurricane or snow Stone can take them up um the data centers are pretty resilient but you know nothing is is invincible so we'll see where climate change takes us um the the other part of your question is is is is having two media effects right the idea behind this is um let's say there is an electrom magnetic so let's say there's a type of probability in the backup software those are different things that can PR both copy if they're on the same type of solu Shar media and so if you think about the more you know like this is like the more advanced uh let's say that

Dr needs um let's say you're you're a hospital you're the military it's not enough to say sorry you know both my backups were like a single piece software I don't have my backups anymore you have to have separate typ gen um whether it's it's a it's a let's say media might be a maybe it's like a separate a separate vendor separate vendor solution it could be tapes uh just for like long storage um it depends on how far back you have to keep your data and the protection requirements there so do you get an excuse for losing you know this type of data versus that data it really comes into play um in Cloud environments you still have to

think about because they're just you know cloud is just a bunch of dat that one company is running you still have V consideration there they just look different because the way they Market it theability zones and so on is not as clear okay perfect one more question um if you don't mind um on the third party this disaster recovery options that you were talking about so again let's say if if my organization is focused on one cloud provider um like for let's assume it's AWS so where every cloud provider has their fancy tools U and natively available options for disaster recovery so how would a third party solution really help out or what are the

advantages and said he just choosing between an in-house cloud-based assed recovery versus a third party solution here um some of the challenges like like just just to talk about a specific CL provider for example if you build your system natively in in in in their environment it's very hard to recover it anywhere else right um the big players are you know they have enough footprint enough uh like enough investment in security where you know it's less likely that the capability would be wiped out um so like I I have a ton of clients that just want want to use you know it's more seamless to use their own tools but it really depends on like I'll give you

a concrete example um I have a client that's in Professional Services they have a document management system not SharePoint and they basically have a a backup set up where they have a copy of their documents so they don't get all the features that come with the document management tool but they have a they have a backup copy where they needed to access documents while waiting for that that vendor to to recover their stuff they could access individual documents if they had a team of people that had access to them that would hand out documents Bas on permissions because if you only have a just a bunch of the documents and not a not the system that

manages the permissions you need a team to to filter through those requests but it's kind of like a last resort I need access to the documents to you know to go to Port and that kind of stuff that they needed to have that in place um but at the end of the day you have to trust some vendors and I'd rather trust you know ones that have a good like a better track record so you know if it's Amazon if it's a smaller provider they have a good track record you can go with them it's all about the risk cap right um some risk appetites are just not there yet for for for cloud people want to you know manage

and control their own their own backups and you can do that as long as you have you know like enough skills on their team to to do that better than the providers that are up there yeah thank you

questions a right welcome everyone I appreciate your time and listening to me run through Disaster Recovery disaster mode thank care everyone welome [Applause]