Cyber Security Risk Assessments and Their Impact on Data Breaches in Small-to-Medium Businesses

[Music] before I get started I just want to thank all the volunteers for bsides uh James car and his team and Bo Valley college for for hosting I really appreciate it I'm quite excited uh to be back uh connecting with colleagues in the cyber security Community here in Calgary so uh so my name is Stan ronek I've been working in Cal for about 22 years securing and supporting it environments in in various Industries I'm currently an information security and Technology manager at the Real Estate Council of Alberta managing a great team of individuals a really exciting portfolio of technical programs so about two years ago I had the idea to further my knowledge and skills in information security as one

does two years ago post pandemic so I decided to enroll in my a master's program at the uh University of Leicester in security and risk management it's actually in their their criminology faculty so it was a little bit of a different take on uh cyber crime and cyber security it seemed like a good idea at the at the beginning but uh you know partly I I I thought okay this is going to further my career aspirations and this will be great um you know a in I kind of realize this is an interesting life that I have so working full-time managing you know a family uh reading a bunch of different articles late at night waking up 6:00 a.m. because all

the classes are in UK time attending class writing papers on the weekend uh you know going to soccer games and swimming working in the car so uh I'm here it's done recently graduated this summer so woohoo got it done so this presentation is really about my academic research for that program and I want to uh explain kind of what we did for the last six months so uh next slide

please we still having technical

issues

unfortunately this presentation is very visual ah there we go great here's the agenda I'm going to just go quickly through this so um I'm going to explain why uh cyber security risk assessments I I selected that as a research topic you'll see CRA a little bit in my slide deck that means cyber security risk assessment I may even say CRA uh research methods and research aims and objectives I'll go a little bit through that uh the sample the data the findings that's really the bulk of the presentation and then the conclusions next slide so what is a cybercity risk assessment so one of the definitions I like is a cyber security risk assessment is an assessment of an organization's

ability to protect its information and its information systems from cyber threats the purpose of a cyber security risk assessment is to identify assess prioritize risks to information and information systems that's from it governance uh USA now you notice the scorecard on the right hand side there uh that's based on the National Institute standards of tech and technology so that's a nist scorecard if you will based on their cyber security framework uh very common to see a scorecard like this after you complete a cyber security risk assessment based on a specific type of risk uh framework and a lot of it is an aggregation of some of the technical vulnerabilities that have been detected whether it's you know medium low but

this is kind of that aggregate score now nist in particular has several uh special papers or special publications related to cyber security risk assessment one of the interesting things that I read U from researching nist is in the 80030 risk management process should be should not be treated primarily as a technical function carried out by it experts who operate and manage it systems but as an Essen business management function of the organization and so this as I started asking questions I thought this quote was actually uh pretty interesting so are Business Leaders and managers truly involved and invested in the CRA process do they understand the Cyber risks that are identified to their organization and

do Business Leaders actually act or own those risks um after the cyber security risk assessment uh next slide please so when you take uh a master's program or a post-secondary you end up reading a lot of academic articles uh during during that time Le lession or Le lession I think is how you pronounce it I probably act that um is an academic author and expert in cyber security practices he wrote an article about the applicability of various risk management Frameworks so ISO nist Fair uh CIS and he breaks down the CRA process into two phases so there's really on the right hand side of your slide here you see cyber security risk assessments various methods that are performed during a

cyber security risk assessment and those are uh inputs to the actual risk assessment so these are two independent uh exercises if you will um All That All Leads down to to risks at the very bottom there and so that would be kind of the risk catalog or the risk uh identification process and then finally that that like little dotted Arrow there that's where the risk framing um I would imagine happens in terms of developing a final cyber security risk po posture for the organization and so the O overall cyber secur strength of an organization is really what a cybercity risk assessment does how well the organization can predict prevent respond to everchanging cyber secur landscape and it's really a

final score to measure the organization's uh cyber security resilience next Slide the research Gap so you read a bunch of different academic articles and you try to figure out what the heck am I going to you know base my research on uh you you kind of talk to different people and really kind of understand what the Cy cyber security Community could use as value for your research um most of the articles I came across were qualitative which is you know people interviewing people and getting that information back but there really wasn't a lot of quantitative research in this area so I decided to develop a research project based on measuring the effectiveness of cyber security risk

assessments based on how well they mitigate risks and the effects of risk for that organization and it's in part about the after effects of the cyber security risk assessment what happens next so I rushed to my academic advisor filled with excitement told him I have this research topic I'm going to start telling him about the research design and the methods and the techniques I'm I'm going to uh employ I have all this cool stuff I'm going to be doing in the research and his response was no you have too much stuff you have five months to conduct an academic research narrow your scope and figure out exactly objectively what you're going to achieve so uh I made some adjustments

and uh in my research application I really focused on small to medium businesses those are businesses with less than 500 employees um and then the university accepted my application next slide please so these were the three research aims and objectives in my application so objective one was really to understand the common methods for conducting cyber secur risk assessments uh objective two uh a little bit more ambitious so I wanted to assess the relationship between the frequency of cyber security risk assessments uh and the occurrence of data breaches in small to medium organizations I'll talk a little bit more about that objective three was to understand the perceptions and the attitudes of Security Professionals and Business Leaders so I wanted to create

two two groups Security leaders or Security Professionals and Business Leaders and get their perspective and compare the two next slide in terms of the research design and methods it is quantitative research I was developing an online survey tool that took you know probably about a month of uh of effort to to test and and make sure the instrument works well uh then we did some recruitment on LinkedIn we call that snowball sampling which is just Word of Mouth primarily and then because it's an academic uh project we did data analysis using IBM SPSS and the Excel functions so just using Excel next slide please I just spilt a bunch of water on me sorry team so there's these two sample groups

um that I really want to focus on so in terms of taking the survey we have something called an inclusion criteria so that's the idea of of the things that we require to to take the survey it was fairly open it was very very simple and that is uh the person had to be at least 18 years of age that's a requirement of the University they need to be proficient in English because the survey was done in English and they need to have least participated in a cyber SEC risk assessment in the past five years so at least one and so with that inclusion criteria I started thinking about sample groups so I wanted to

understand the business and the organizational involvement especially around Business Leaders and decision makers and so we came up with these two sample groups Business Leaders and Security Professionals the one discussion that we had was where do we put risk assessors uh should we have a separate sample group because they really are a little bit different from uh uh Security Professionals that's more the client or the customer risk assessor is the one that's actually conducting the CRA so we group them into the security profession for pragmatic reasons keep it simple and we went with one questionnaire and any you know notable differences between these two sample groups is um is something we'll we'll put in our dissertation and and

and kind of further explore so next slide please so here's the sample um and the survey was open for about uh four weeks in April this year um you know I was it's such an effort to recruit I was posting the survey information every week on LinkedIn trying to recruit folks and the goal is 100 participants um and that's a considered a minimum number in Academia to generalize your findings based on static sampling methods so um and in fact one of my colleagues offered me hey I'll sponsor our company will sponsor your research and we'll offer um folks to win a free iPad if they take and complete your survey and unfortunately you know the University's research uh

code of ethics doesn't permit uh winning a iPad um or sponsorship so unfortunately I had to pass that up but it would have been nice to have something to offer during the recruitment phase and so we uh we only ended up getting uh 58 participants that completed the survey uh we were really truly not able to generalize the findings from an academic perspective however we were able to conclude and interpret some of the DAT data and it's fairly interesting so 58 participants 80% of uh small to medium businesses headquartered in Canada not surprising there 60% are employees and about half are Business Leaders now one of the things to note here is I miscoded one of

the survey questions that creates the sample groups the reason why I say that is where it says 48.3% business Executives Security Professionals 43% and then other that's a miscoding because those five participants um weren't included in objective three to really understand the perspectives of the two sample groups so live and you learn okay next slide please so here's a breakdown of all the uh the industries that participated in the survey there was various Industries um that participated um there was a bit of bias here uh in in the participation there was a lot of uh real estate uh industry organizations that participated that's just because of my um uh just because of my colleagues and in my Professional

Network and also some uh a lot of academic and education participants so I again that's just a reflect of the colleagues uh in my Professional Network next slide please so one of the things we asked participants was to uh provide uh you know a little indicate which department departments participated in the cyber security risk assessment I don't think there was any really shocking uh this the the values that came back were were expected so primarily it accounting executive HR and information security department obviously some small to medium businesses don't have an is Department um but you know what what we saw here is what we were expecting next [Applause] slide so uh going back to the original

academic the uh le le chenna I think is how you spell it Les chenna um provided uh you know that that basis for the cyber security security uh assessment methods that we wanted to know and so we're growing our research based off of of his article and so the top one there would be it process and system audit so typically a firewall audit or you know um understanding change controls in an it department is the most common and then compliance uh risk assessment so PCI DSS nist ISO those type of Assessments and then the third one was vulnerability identification so like an internal n scan to an organization would be um that type of uh method next

slide so these were the type of risks that were identified based on the participation so again not uh not surprising here I think this is really expected so 55% of participants identify technological risk um that's you know that's expected uh then we have this grouping of information risks so confidentiality and data integrity and you would did a secondary analysis on that thinking that that was mostly Security Professionals but uh it was actually 60% of those folks were actually uh were Business Leaders so the majority was actually Business Leaders on on that count and then you have organizational risks which accounts for about 40% so operational finance reputational and and human uh we we see those as organizational risks uh service

and supply chain risk about a third of the folks uh noted that and then legal risks uh at 19% now at the very bottom there we have a not sure and that's 14% of the population uh or participants that indicated that and so uh we thought we might have a self selection bias which is some folks that really uh aren't qualified to answer the survey answered the survey uh we broke that down into five Miss Business Leaders and three in that other category uh none of the Security Professionals uh answered not sure which is a good good news story next slide please so here are the res risk Frameworks um that that folks noted here

so um so ISO 27,00 000 you know the iso 2701 information security management system is probably the most popular uh within the group and then you have uh nist 800 and uh sock type um uh uh risk management Frameworks that are being applied but the the big thing that came back and and you know my academic adviser and I talked a long time about this but about 40% of the participants responded not sure um and so uh we thought we might have a problem with our survey survey data especially for this question and so again a secondary analysis about 80% of that those uh folks that selected not sure are Business Leaders so uh 19 out of the 23

that answered not sure so my subjective interpretation of this is uh that Business Leaders don't really care about uh risk management Frameworks that are applied or the technical details of a risk analysis uh um you know the methods don't really um are the things that they don't recall what's important to them are risks that have the greatest impact to their operations and to their organization uh just notably the one that was another uh was a glba so gra leech uh built um um gra leech blimy act uh which was one of our financial institutes that participated next slide please so objective two was to see if cyber security risk assessments had a measurable effect on the number of data

breaches for small to medium businesses participants were asked how often or how frequently they conducted cyber security risk assessments uh in the survey and they were also asked to qualify the number of data breaches the small to medium business um experienced in the past years and then we did a correlation um to take those two responses and and chart chart it so we provided participants as well um a definition of a data breach so data breach is an instant wherein information is stolen or taken from a system without the knowledge or authorization of an information owner or organization so that was the definition that was on the survey and what you're seeing here is the sum of data breaches qualified by

all participants in the survey uh for the past 5 years 40 1% of the total sample population which is 24 out of 58 qualified a data breach we were expecting 60% so we thought it was a bit low um but you know that's based on academic research that happened prior and may be in part the limitation to our sample size but if you notice the purple line there uh that line represents the participants that um you know qualified all all the sums of the the data breach reaches and that's based on a yearly value so on the fifth year there for folks that uh qualified annually as their frequency uh we had 13 data breaches uh summed up

there now there was an interesting um you know again a lot of discussion about this there seems to be a trend from the past to uh you know today in terms of just a generally a reduction in cyber security breaches for the participants uh so that that was noted but if you look at like quarterly there's a little bit of variation to that year three looked good but you know year 2 and one did still had uh data breaches uh next slide please so we tried to answer the question what is the ideal frequency for conducting cyber security risk assessments and so we calculated the average of uh the data breaches grouped by the fre quency and so what you see

here on the x axis is you know the different frequencies and then on the Y is the the the average or the the value and again that's just adding up all the data breaches uh qualified in that frequency or in in that category and then dividing it by the number of participants again we see sort of a downward Trend um so the the more you do uh cyber security risk assessments the data breaches did go down however in the quarterly category we had two participants in this category that qualified several data breaches in the past uh five years in fact there were bigger companies and so we have this ski jump effect uh for the quarterly um

unfortunately notably continuously would be like monthly and almost like you know running cyber security risk assessments on um just a continual basis um and we had I think three participants in that category and again this probably is a reflection of the sample size uh only being you know 58 participants next slide please so objective two we had some challenges it went a little left field and luckily objective three we're we now look at uh the perceptions between these two groups uh Security Professionals and um Business Leaders and so we had a light cart scale so you know your typical strongly agree agree neutral disagree strongly disagree that type of scale um and we we have a

histogram analysis where we separate the two groups and then we uh we show kind of uh the attitudes and perceptions and so uh 58% of the population strongly agrees that CRA are easy to read and understand 177% are neutral 25 disagree and so one of the things that we concluded in the dissertation is some of the cras potentially have lot of technical jargon and it's not necessarily tailored for every target audience including you know sea level type audiences um and that may be an area for improvement next [Applause] slide cras are valuable for risk management practices um this was very positive so 73% uh suggests that cras are valuable for uh risk management practices 25% are neutral 2% disagree

next slide and so we just asked a question or put a statement up uh do cras help mitigate data breaches um and get perceptions on that 55% agree or strongly agree 40% were neutral and 5% disagree next slide please cras influence change and Investments and this one uh really stood out for Us in the analysis 83% of the population agree or strongly agree 13% were neutral and and 4% uh disagree and so we we even asked this question in a in a different way uh next next slide please uh did the uh small to medium business organization act based on the guidance they were given of a cyber security risk assessment and then to me

this was one of the bigger indicators uh in the survey that indicat Effectiveness and influencing decision makers to remediate and act and overwhelmingly we had 84% uh that said that they made an investment or a change based on a cyber security risk assessment next slide so towards the end of the survey we asked folks to give their comments and their recommendations for cybercity risk assessments uh one of the comments that came back was to make cyber security risk assessments more a say accessible for Lay people again going potentially to the language and some of the recommendations in in technical jargon cras should focus more on business specific risks and prevention risk management standards should be endorsed

but one size does not fit all incorporate automation AI uh robotic um processing automation for certain assessment processes so I think some of the folks that were in the sample were actual risk assessors and they provided uh this feedback and just further knowledge sharing among cyber security Community to improve cras and the perspectives of of what cras do next slide please so this this is our conclusion so the final the findings indicate that cras do help to reduce cyber risks for small to medium businesses over 80% of the surveyed participants indicate that the CRA guidance provided them enough information to act or invest um overall pers participants perspectives were very positive about cyber security risk assessments however

uh the sample population wasn't perfect ideally we'd like to see over 100 participants in the next survey and more academic research needs to be done on this uh particular topic great and that's the presentation any questions please go ahead

yeah

yeah yeah so uh the question was um that one question that had over 80% of the participants um ask about uh you know did they make an investment did they make an action right uh so you know I think that probably doesn't relate to the data breach as much because we didn't really ask that we asked more specifically like did the cyber security risk assessment provide you value in able to invest or create an action out of it um but it is a good question there are there are certain aspects of this topic that we wish we could ask more questions but um you know it was about 15 minutes of people's time to complete the survey so we we had to

limit some of the things that we we initially put in there go ahead you might have to speak up so I can hear you you're welcome yeah

yeah y y

right yeah right okay yeah I'm going to repeat the question just for the folks that are online so question is is did we consider other risk management Frameworks that are more Conan based outcome based um in in that particular question in in examining that so um so we we just look at to be honest a lot of the research was just based on us standards ISO I would argue is is more International because that's that's from you know the organizations in Switzerland I think isn't it um but I I think uh that's something that could be done is is we we initially thought that we were going to get us and Canadian organizations and these were the common

risk uh management Frameworks that we went with with um there was a category of other but even that uh participant qualified a us-based um risk management framework go

ahead

yeah

beginning event they get in line I'm just kind of like to me yeah I mean some of the discussions I had with my academic advisor on on this particular um chart was um you know I had anticipated uh so there's only three participants for two to three years right and I think in part sample size kind of um is isn't ideal because it takes one participant to have um a few data breaches and kind of exuse excuse the data but in terms of awareness um 70% that qualified a data breach was they were in the security professional sample group 30% to Business Leaders and it is a bit difficult to do the research the way we did it asking Business

Leaders to qualify their data breaches in the past five years uh without their general counsel beside them and and an NDA I think in part this this uh test wasn't 100% um I think we did get some good data from Security Professionals I don't know if we got great data from Business Leaders and it could be awareness as well because that is one of the things that that uh you know is in the literature is that small to medium businesses especially on the smaller scale of that you know 0 to 500 employees tend to not have the the security awareness because they think they're small and you know they're not a Target but really in in reality they are

a Target they just don't realize it another question sorry she was first go

ahead

yeah so the question was is um what frequency would I recommend to small to medium businesses based on the research I looking at the data and just experience for me it has to be at least annually um you know the the we typically do by anual to quarter it kind of matters what the cyber security insurance provider wants matters you know PCI DSS has to happen um every year um and there may be other assessments there one thing that I I would point out is when we talk about cyber security risk assessment there's sort of a formal cyber security risk assessment and that's those two phases so a lot of the times we're doing you

know security tests uh and we're we're figuring out the vulnerabilities all the time we're running you know if if you're a customer of like you know tenable IO or qualis you're running vulnerabil assessments all the time but an actual formalized report that goes to your CEO I would suggest that would be at least annually go ahead um did you notice

that

say I'm not I'm not an expert ER in this field but what I learned in the literature review is it feels like fair is one of the risk Frameworks that really uses you know Financial value you you really can easily communicate that to the sea level because this is this is our Financial Risk based on this information system having these vulnerabilities having having that is is you know really easy to qualify to the sea level I find that you know just based on my experience that a lot of there's a lot of literature that says that the nist is like you know for US government agencies and it's can't be applied to other organizations I would

say that's inaccurate because I think when you really go deep into nist 800 um especially the 53 that's applicable to a variety of different organizations and I like nist that's just who I am but you know if you're uh my colleague Mike primo and you like the iso 27,000 the isms that's also a great great standard as well but it also sort of matters what you're trying to achieve is is this a sock audit because it's a financial audit or is this um high-tech because you're a healthcare so yeah hopefully that answered it yeah yeah it did okay sorry it was a bit of roundabout answer I apologize go

ahead yeah I I wanted to make that uh so sorry I'm just going to repeat the question just for the online so was there a correlation of the risk management Frameworks in relation to the data breaches and so that was one of um we'll call it a subobjective that we were looking at doing we didn't make that uh we just felt like our sample size wasn't big enough to even um make that correlation we were just mostly focused on frequency but I encourage folks to look at this and say you know there should be a a correlation of data breaches to risk management Frameworks to understand which risk management Frameworks perform better than others that's what I suggest great question uh

oh sorry in the back first

sorry number of pages it depend on amount of work the team is putting behind the scene so if you are not going to fix your vulnerabilities yeah second risk assessment is going to give you the same result if not more so like is there a relationship behind the scene how much work each company has done and my second question is with respect to data PES like one company have five data Brees but what was the cost of that data bre one company has one data breach which can cost like yeah yeah yeah the research didn't go down that road but that that's an interesting question so so two questions first question was what's the effort in the remediation

like the after effects like um it's hard to say that again that that's something that we didn't really go into um we were mostly focused on understanding that decision makers and and Business Leaders in the organization actually do something with this document this report that they get back it doesn't just get consumed by the it manager the security manager and they you know do some changes with configuration for example so I we really wanted to know if organizations own these risks um and so that kind of research around how much remediation effort is something that I'd like to see on the second um was the cost of the data breaches that were qualified we didn't ask that as well but

I think you know having a business leader qualify the the data breaches you know a laptop gets lost into the wild um even though it's encrypted still a risk you know uh considered a data breach uh so I I think you know we did a quick definition of what a data breach is but this kind of research is probably done better when you're actually doing in-depth interviews with the right uh folks from that organization rather than doing a survey instrument like we did so go ahead thank you

yeah

right

right size that more diverse but it's interesting because I think right now there's a lot of problems with kindah doesn't have a legislation around there's no there's in the US they're coming towards tra smaller so I'm hoping that we Evol over time that there is more have become more conscious right you know back Micha president is I think it's not like they will be compromised in the future I think it comes down to that uh they already probably have someone on their Network and they don't know yeah exactly yeah I you know again U you're you're given 15 minutes of a person's time a survey instrument what do you want to achieve with your objectives and I would say

objective two would have been much better if we had that sample size um especially if it was over 100 so can't offer a free iPad though that was the challenge and four weeks get it done in four weeks good luck see you later that's the academic advisor advice so any other questions yeah one more question

do I I wanted um we had I think two participants from energy and you know in Calgary I guess I got to get out there a bit more and get more colleagues in energy I would have liked to seen more more participants in oil and gas I anticipate yeah the evidence would look different there would be potentially more data breaches the thing is though is the small to medium businesses stats can Says Z to 500 employees there's a huge range there so it it might make sense to break break it up by industry sector or or or something to that effect so yeah good question thank you what did you define

[Music] [Applause]

as yeah so the definition we put on the survey is um a data breach is an incident wherein information is stolen or taken from a system or organization without the knowledge of the authorized information owner which again might be too much technical jargon um or organization so um we threw that definition up there I just I don't know if explains it well enough um but really it's it's the loss of sensitive information like that's it's simple as that and if you lost control again like if you have a laptop that leaves the leaves the organizations lost in the wild even if that thing's encrypted data

breach oh go ahead sorry okay thanks for all these questions appreciate

it yeah RIS assessments might not be exactly the same risk assessments that the business owner you understand they're doing an organizational level risk assessment right understand the mitigations from the C question management system or did you see how business owners are understanding it and what stage man understand dat yeah yeah and I think so can I just summarize that when when are when are like sea level or Business Leaders able to understand the data um Can can we go back to the slide for uh L I'm going to ask his name again Lenna L cha side it's the one with the two phases right at the beginning I think it's slide three okay so I think just based on the

research and and talking to people risk framing is an art and there's there's there's multiple factors to getting Business Leaders to really understand and own a cyber security risk assessment one is their participation that includes in-depth interviewing and you know and and so all this stuff right here for security testing Business Leaders aren't necessarily interested in the details or you know the minutia of what's happening in here it's more around uh risk evaluation and then firing it over to the cyber security risk posture because it has to be framed in in a way that business understands it so it's almost like cyber security risk Assessments in a lot of ways have to be tailored for

two targeted audiences Security Professionals and Business Leaders and it's really the executive summary to the Business Leaders that you know if it's framed in regards to what their operations how it impacts the operations or their organization then then that's when a CRA is very effective and and I don't think that always happens yeah I I hope that answered the question I I wish you were on our research team you seem to have the right [Laughter] questions okay we might want to talk afterwards okay sounds good any other questions I I hope it answered everyone's questions okay good okay well thank you for everyone's time here [Music]