BSides Buffalo 2026: What Is IT Audit: What IT Auditors Do and Why It Matters

Okay. So, this is Jared Bartzak? >> Yep, Jared Bartzak. >> And he's an audit IT audit manager. He's got 5 years of compliance experience, multiple certifications, uh multiple compliance regimes. Um he's a subject matter expert on a couple of different areas, including AI. And he's helping with AI governance and compliance initiatives. And he's been He had something for the uh world government organization which I'm sure he'll talk about. >> Yeah, yeah. Awesome, yeah. Thanks for the intro. Yeah, so for this presentation what I wanted to do is keep it pretty light since it's, you know, towards the end of the day. Thank you all for staying, first of all. And just really wanted to hit the high level

of what is an IT auditor, what do we do, what's our day-to-day, cuz there's lots of misconceptions into what we do. A lot of people think, "Auditor, you're the bad guy. You're the one yelling at me. You're the one making me do all these things, tons of emails, right?" There's lots of things that people think when they hear the word. So, really just wanted to hit the high level, what we do, why you shouldn't be scared of us, and take it from there. And I like to keep these pretty informal, so please any questions throughout cut me off and ask me anything. We'd rather you learn than me just lab on all the time. All right, so yeah, so I'm an IT audit

manager for AI and special projects at A-LIGN Partners. Um but have been in the space my whole career. I've been in external IT audit. Um and I have conducted audits across many frameworks, you know, SOC 1, SOC 2, the ISOs, HITRUST, CMMC, and most recently AI compliance with all the new frameworks coming out. You guys have the UAA Act, SOC 2, AI basic act, all these governance laws forcing organizations to start enacting some frameworks to prove they're meeting these standards. Um so yeah, as an IT auditor, you know, I have a lot of the same credentials you'll see in the cybersecurity space, right? So I have a CISSP, a very common cybersecurity credential. The CISA, which is a

certified information system auditor, an audit specific credential. A I A I SOC 2 AI audit credential. A couple high-trust specific ones in there that you could probably hear for the first and last time today unless you're really interested in high-trust. And uh yeah, outside of work, um love soccer. Happy that the World Cup is coming up. I've been a player, coach, fan throughout my whole life. I'm beginning my entrepreneurial journey. So outside of cybersecurity, trying to start my own little side project here, build something out of it, and using a lot of cloud code to try to build web applications and stuff. So really early on in it, having a lot of fun with it, though. So

All right. So what does an IT auditor actually do? So there are a lot of these misconceptions, right? Number one, you're a hacker. No, I am not that smart. I can't hack into your systems. I can't do all that stuff. I'm not I don't pen test. I can't red team, you know. I do not like coding. That is the first misconception, right? Oh, okay, then oh well, you're a firm. You work for [clears throat] a CPA firm. You know, in order to conduct IT audits, for SOC 2 and the AICPA, you have to have CPAs as part of your board. So you're an accountant. You know numbers. You know how to do all this financial stuff. Nope, don't know how to

do any of that at all. So Okay, then you must fix computers. You must be IT support, you know, you must know everything about computers, everything why they're going wrong. Uh no, once again, I have done that in my career, but not at this point. I'm not fixing computers. I'm not help desk anymore. Uh we evaluate, we don't operate. Right? Um so, as an IT auditor, the reality um we independently check whether controls an organization claims to have are really well designed and actually working. Um so, I agree like that definition. That seems to, you know, at a high level cover what we do, you know? So, an organization says they're doing something, we're going in checking that

they're actually doing that is well designed and actually, once again, working. Um and then issue a report at the end to uh verify things.

So, what an IT audit actually is. Uh it's an independent, evidence-based evaluation of whether an organization's controls over its technology and data are designed properly and operating effectively. So, those are the two sort of hats there. One, that's designed properly. You have policies in place, you're you have all these governance into what you're doing. And then it is actually operating. Once again, you can provide that evidence, you can show us configurations, you can show us policies, you can show us all these things that it's actually operating as they the the way you wrote it out, right? Um so, number one, um independence is the big thing in IT auditing, right? We can't run the systems, we can't you know, implement

anything into the systems we're checking. We have to be completely independent from the systems we're auditing because, you know, you can't be building the stuff that you're going in and checking and signing off and saying, "Yeah, this is all okay cuz I did it all." No, we have to be objective and independent from uh what we are auditing. Um it is evidence-based, right? So, opinions don't count. We're checking configurations, we're checking logs, tickets, records, policies, things like that. Sure, we might, you know, conduct walk-throughs, talk with engineers, talk with HR individuals, talk with security individuals, as well. Just to get an overall view of what the systems are doing, but in the end, we need to

actually see what it is. You don't have to just take your word for it, right? Um so, that's another big pillar. And lastly, it's risk-focused. So, we evaluate risk across all these frameworks. We take deeper dives into stuff that is more risky, but stuff that is less risky, right? Um so, that all all comes into play within the environment of frameworks, as well. So, once again, the auditor's question question, show me the evidence that the the control works, not just that you say it works.

All right. Now, break down of what it is and what it isn't. So, some of this we already covered, but we'll run through this again, right? So, IT audit is not penetration testing or hacking. Uh the security team that builds and runs controls. It is not IT support fixing laptops. It is not a gotcha exercise designed to catch people out, although a lot of my clients would argue otherwise that we're just trying to pick apart everything they built. Um and it is not a guarantee that nothing will ever go wrong. That's a big one. You know, a lot of clients will get an audit saying, "Okay, now we're perfect. We have this, you know, piece

of paper that says all these controls are working perfectly." No. You know, there are still chances that you know, things will go wrong. You know, we take samples of stuff. We're not looking at the whole big picture, right? Um so, there are still chances that things will go wrong. So, what IT audit is, an independent second set of eyes. So, once again, someone else saying you're doing things right, not just your own organization saying you're doing things right. >> [snorts] >> A structured test of whether controls work. A translator between tech, risk, and business. So, that's a big one, right? So, I mentioned before, right? That I'm not a hacker. I don't have that deep of a technical knowledge, right?

Um, but once again, I'm not an accountant, so I don't have this big business numbers knowledge, but I'm somewhere in the middle, right? So, I went to business school, I have a computer science degree, I'm right in the middle of all of that, right, where I know a lot of tech stuff, sure not as deep as all the experts, but I can hold my own in that room, but also hold my own in a room with board members, CSOs, right? Um, CEOs, CTOs, people who have just a business side of things, they could care less about how deep it goes into the technical side, they just want the big picture business side. Um, so, IT audit

really falls right in the middle of that, you know, it's a good mix of business [clears throat] and technical. Uh, so, as a a source of credible third-party assurance, we've already talked about that a little bit, but once again, a driver of accountability and continuous improvement. Yes. >> Would you kind of explain what the word assurance means in this context? >> Yeah, yeah, so, the big thing with assurance is that I don't know what this is. So, third-party assurance, right? It's so, you're buying um products from third parties, right? So, you're working with third parties, you're bringing in their solutions, you know, they might have contracts with them, you might have things of that nature with these third parties.

Assurance is an evaluation that, okay, these are the controls, it's you know, they'll be okay, right? What once you evaluate it, right? Um, and accountability is the other big, you know, A-word there, you know, someone has to be accountable for all of that and making sure that it is all operating, right? Um, there has to be somebody who is in control, right? So, when we start talking about AI, a lot of the talks today were probably, I don't know, AI or answered. Um, we are running into clients that are starting to use it more and more in their systems, right? So, change control, big thing in IT audit. A A of the code changes now are written

by Claude, are written by um Codex, written on by all these tools. Um they're like, "Okay, so how many of these do people actually have to go in and review?" My argument is most of them, right? You know, you don't want AI reviewing AI and being accountable for AI. It doesn't really make sense, right? Um you know, having a person, someone you can tie back to, someone you can, you know, slap on the wrist and say, "Hey, you know, you shouldn't have done that, but you approved it, right?" So, it's holding someone accountable um that is not a technology because, you know, that doesn't really hold up with in in argument in the end when you just say,

"AI did it. Sorry, right?" So, accountability is a is an is another big one, I think.

>> So, once again, so where does IT audit really sit sort of in the whole big picture? >> Um So, you're asked to trust an organization with your data. So, you as a client, right? You work for an organization. You want to use their products, their SaaS products, their tool. They say, "Okay, we have this brilliant tool that could take all your patient data, reformat it in a way that makes sense to you, makes it all pretty, right?" Um just trust us, you know, we'll take all your data, we'll keep it all secure. It'll all be fine. Um your PII, patient information, your defense secrets if you're in the government. Um but you can't walk in and inspect our

systems yourself. Often times that's a contract, right? You can't come in here and audit yourself. So, how do you actually trust them? And that's where IT audit, third-party assurance comes into place. So, it's right in between the organization saying, "Trust us." We are there independently testing what they're saying in order for stakeholders, customers, regulators partners boards um things of that nature um to actually trust their organization. Um once again, you know, going back to some keywords there, right? The audit various independence is what is uh stronger. It makes the claim, builds the trust, and all that. Um so, it's not the organization themselves saying we're perfect, it's actually an outside party that has no association with that organization

saying, "We checked these controls and we agree uh that they're perfect." Or we They're mostly there, here's some gaps on their report, right? Um so that the stakeholder can have good feelings about it and know what they're getting into in order to sign that contract right? Um So, the deterrence, once again, is is that perfect?

All right. So, where does it fit in GRC? So, you might be familiar with the term of GRC, governance, risk, and compliance. And really, it sort of sits across all of them. Um you know, they tie in at different portions throughout. Um so, we have uh governance. It's who sets the rules. So, going back to the policies, who's writing the policies, who's saying what organization can do, what directions you want to go in, what directions you don't want to go in. Okay, we want to use AI for these use cases, really stay away from AI with these ones because we don't want to use them there, right? So, that's governance. Risk, identifying what can go wrong and

what you do about it, right? Determining if you want to go in certain directions based on risk. Okay, that's really risky venture. We might not want to use AI in this use case, right? We don't want to use it with our patient data, but we could use it in the day-to-day, you know, reading documents or writing documents, things like that. And then, compliance, just meeting the obligations laws regulations contracts, standards, um things of that nature.

All right. So, what is the main unit of everything in IT audit? And it really comes down to controls. So, going back to sort of the overall flow of it all, right? So, if we're talking EUAI Act with the new laws that are in Europe, right? That is a regulation. That is a law. So, how do you prove you're meeting that law? Frameworks. So, it goes down to frameworks. ISO 42001, you know, other some of the new AI frameworks that are out there. Okay, so if I meet this framework, there's a good chance I'm meeting that regulation and that law. Um you know, speaking on that note as well, a lot of the state-based laws are actually calling

out ISO 42001 and saying, "Okay, if you have this sort of framework, you're all right by law. You're meeting our laws." Everything sort of lined up there. So, a lot of the new laws that are coming out are trying to attach to these frameworks give organizations sort of a clear, "If you have this, you're right." But, you know, all the states are moving in different directions, especially here in the in the US. Everyone has a different opinion, regulate, don't regulate, be creative, you know, what should be number one, right? So, that's still sort of hashing out here in the in the states. Um but after those frameworks, it comes down to controls to meet that framework.

So, in order to meet this overall framework, here's a list of a bunch of controls you have to meet, which are basically the rules, right? Um and then how you prove you meet those controls, evidence. So, those are the pictures, uh screenshots, logs, things that we're evaluating on a day-to-day. So, that's really the overall high-level flow of how this goes. Um and then so, the controls break down into design effectiveness, once again, and operating effectiveness. Design, you know, the set correctly, are you calling out everything? Uh the example here, does a policy require access to be removed when someone leaves the organization? I sure hope so. Right? You don't want people still having access when they left the company.

So, that's the design effectiveness, saying, "Okay, here's this policy, remove access after 12 hours of person leaving, you know, being terminated. Um Um or something like that, right? Operating effectiveness would be, is that actually working? Did your team actually go in there and disable the user accounts after 12 hours or sooner than that person being terminated? Um so, we would sample, okay, 12 people left this year. Um let's just look at all 12 of them. Did they all get access removed within 12 hours? Yes, good. Okay, was one of them maybe a little later? Okay, we'll write that as a gap, right? But, um overall, it was operating effective. There was just one small, you

know, gap of, okay, they missed it by an hour for one of the 12 people. Um so, that's the difference between design and operating effective.

All right, so, a little bit uh into this presentation, we'll start getting into what are some of the different frameworks based on, you know, where your organization might sit, um what industries you're in, things of that nature. But, there are four domains that sort of spread across all of them. You're going to run into them no matter what, right? So, there is access management. Um who can get into what? Should they be able to get into what uh they have access to? Who provisions that? Least privilege, right? So, people in HR should not have access to production. Why would they need that, right? Um MFA, you know, having that added layers of security. And once again, timely removal once

people start uh leaving companies that you're taking away that access, you know, if someone moves switches roles within an organization, that you're changing what uh software they have access to um based on their roles. Uh change management, like I mentioned, uh you know, across the board, you know, that's every organization is sort of going to have that change management for their products. How changes to systems are requested, tested, approved, and tracked, so nothing slips in unreviewed. So, that's the big thing, you know, that one, you're pushing these changes, but two, you have all that documentation to back it up and say, "Okay, this is exactly what we changed. This is who did it. This is who reviewed that person's

work. And this is the high-level person who said, 'Okay, here's the stamp. I agree with everything that happened.'" Once again, accountability on that person who stamped it. So, if something fails down the road, they can tie back to those people who changed it, right? Um and once again, not AI-able. Endpoint security. So, across the organization, everybody has laptops, servers um you know, uh databases, everything of that nature. Um you got to patch all those. You got to have encryption on all of them. You got to have anti-malware to make sure no viruses are getting on and all that stuff. And just overall configurations of how you're hardening laptops, things of that nature. And then lastly, uh data protection.

Almost everything has, you know, data running into it nowadays. So, you know, safeguarding that data both in resting transit. Once again, going back to encryption, backing up the data if something were to fail, you know, organizations have to be able to fall back to a certain, you know, point in time, like an hour ago, 6 hours ago, you know, to keep that data as close to real time as possible, so they don't lose everything and all their customers are mad. Uh retention. How long are you keeping it for? Cuz once again, if you're keeping it for too long, then your customers might get mad if a leak occurs. Like, why do you still have that

data from 5 years ago that you shouldn't have anymore? So, keeping it at for a, you know, a a time frame that is makes sense to what industry you're in. And you know, uh yeah, go ahead. >> A question for you about change management. >> Yeah. >> Um clearly, if you have a change management process that expects certain things to happen, you could have the did the 10 expected changes happen this year in the form that we expect. >> Right. >> Obviously, I think that the concern with change management is the changes that you don't know about. >> Right. >> How do you attempt to gather evidence to see if there's changes that have fallen

through that crack or ones that you haven't created the documents you'd expect to see or the sign-offs you'd expect to see for a valid change. >> Yeah, yeah, for sure. So, a lot of organizations, what we see is they will use like automated logging of some nature to track, you know, everything that's going on in that system. And then from there, you know, as auditors, once we go in, that that's sort of our source of truth. Like, okay, here's all this automated list the system that's itself recorded this of these changes that went through. Um GitHub maybe would GitHub that's being used in an organization or down to, you know, the system itself where we

have access to actual system logs, right? Um so, that's sort of our source of truth. From there, as auditors, right, we can't look at a thousand different changes, you know, in any time and fashion, so we got to take a a reasonable sample to make sure everything is uh once again operating as uh effective. Um so, then we would take sample of that and then that's when we would sort of get into that documentation of okay, did you document all this? Um so, right, it's best practice that organizations know what they're pushing out, know what they're changing and all this and following those procedures of okay, here's the the paper to back it up, right? Um

obviously, some slips through the cracks, right? You know, no organization's probably going to be perfect on that on that front um unless they really nail down that process somehow. Um but to answer your question, right, they they do fall through the cracks sometimes. You know, sometimes things slip up. Someone didn't review it when they should have. Uh someone didn't sign off. Uh there's going to be gaps, right? But the overall goal as long as it's operating effectively, most of what you're doing is following that procedure, then, you know, that's sort of that assurance of okay, you're doing it for the most part, okay? >> Uh yeah, I think so. >> Um I I I have a couple things. One of

the things I'd like to add is you know, well So, so some change managements are not planned. Right. This could be emergency change due possibly due do some urgent patching or something. So, they need to that that actually bypass the uh standard procedure. >> Right. >> So, >> Yeah. >> That's one thing and the other that's a follow-up question that uh about the change management is not all companies are software like IT-related companies. Some company has totally different functionalities. >> Right. Yeah. >> For example, one of the companies I work is is more automotive engineering. >> Yeah. >> So, they don't do much IT stuff. They have more laborers. But, if they want to get like ISO 27001

for certification, for example, do they need to have a change management? >> Yeah, so in in in that aspect, you know, so once again, not every organization is building their own product, building their own platforms, writing thousands of different, you know, [clears throat] code changes, right? They're just using, you know, vendor products, vendor software, things of that nature. And a lot of times that'll just fall into common patches from those vendors themselves, right? That'll be their change management. That, "Okay, there's a patch. We, you know, sort of reviewed it before we pushed it out. We had someone approve it before we pushed it out to the overall organization." You know, maybe we tested it on a few laptops before we

pushed it out company-wide, right? Um so, that would still fall under change management even if it's not you yourself, right, doing all those changes. It's more of a patch that's pushed out by a vendor. Um and yeah, to go back to your example of emergency changes, right? Often times we will see that specifically called out in policies, right? So, okay, here's our normal procedure if everything's going normal, right? This is what we follow. And here is what we do if there is an emergency, right? Okay, maybe we don't have everyone's signing off, everyone reviewing, all this stuff happening. Maybe we'll push it out and have a review afterwards because, you know, it's after hours. We have to fix this

right now or else we're losing everything, right? So, it really comes back to these frameworks of what is defined back in your organizational policies. So, if you're calling out like, "Okay, we'll push out a change in emergency and review it within 24 hours after that maybe or or you know, as long as you're following your own stated policy and the design of that is sort of what what is investigated as well when it comes to emergency situations like that. >> Thank you. >> Yeah. >> On what you talked about, you said once you go in, like are you actually being given access to the CICD environment or the cloud environment or something when you're auditing or is it just like you give it

to a senior person who signs off that we did this thing? >> Yeah. Yeah, so when I Yeah, so when we go in there I got to use that word again, right? So we're not actually going into systems. We're giving organizations a list of okay, this is what we would want to see from your cloud environment, the configurations that are set up to protect it and things of that nature. And often times they're giving us back screenshots of configurations, you know, or if they don't want to give us screenshots because they don't want us holding on to that type of information, we'll go in with them. We'll view them doing it. We don't really take over and

do it ourselves. We have someone walking us through it. And you know, we're we're reading the policies and then we're looking at screenshots and configurations and things like that. So I should change my language and not say we're diving into these systems cuz we aren't really. That That's like the the hackers in movies with the hoodies and stuff. Things that isn't actually going on. We're not actually doing that yet. >> Yeah, just to add to that, usually an auditor will have a data request where it's usually some immutable form of we send something like text files and then like someone could change those along the way over time. So they started asking for like PDFs or screenshots.

Um when I was at IBM I also saw a move to live testing where the auditor will sit there and they share out your screen and they show me this. Um we had to do that in some cases where we had a database that contained some data that the auditors were not allowed to see. And they're like, we like access to the database and we can't give you access to that database. We'd be more than happy to pull what you want to see. So, it depends on the audit and level of access that they need and things like that. So. >> Yeah. Yeah, and as common practice, right, as auditors, like if an organization is in

these sort of environments where, you know, they shouldn't be sharing everything else, right? We tell them, you know, censor what you need to, right? You know, block out what you need to. We don't need to know everything. Like, if you're giving us a population of, you know, stuff that we want to look at, sure, block out what's not needed for that. We'll pick our sample of 10 and then we'll want to see the details on those 10. Like, we don't need to see everything, right? Um so, there are, you know, ways to protect organizations that are working with more sensitive data. Um like you mentioned, actually screen sharing or, you know, if you know, a few years ago, before often

pre-COVID, when we were going on site to actually conduct audits, now it's all virtual. Um it's very rare that an auditor goes on site. Um more so maybe in the ISOs that you're actually jumping around on site there. Um but if you're on site, you know, it's a lot easier to sort of be like, "Okay, you can't take any of this here because, you know, we're right over your shoulder watching you look at that screen, right?" We're making sure you're not taking anything. Um where now that it's more virtual, it's a little harder, but there are still the the the protections that uh that can take place. >> All right. So, bunch of different audits

for a bunch of different things. So, there is no single IT audit, right? So, there's a bunch of different frameworks, depending on what industry you're in. Um so, SOC 2 is going to be your most basic what most organizations have. Those are going to be your SaaS products. Um you know, that is going to be how they prove to customers that they are meeting certain controls. Um so, that's SOC 2 is probably going to be the most common one you'll see. Um HITRUST holds a special place in my heart. I did that for 4 years. Um so, that's the health care oriented uh certification. So, it's a very robust. We'll get into it. It goes

It's a beast to entail, very expensive for organizations to take on HITRUST as well. But, it is becoming more and more standard in the aspect of if you're in the healthcare industry, a lot of times now it's being written into contracts. If you want to work with us, you have to have HITRUST. So, that's a lot of its growth more in the in the last few years. So, there's large organizations like, you know, UPMC and Mayo Clinic and a lot of these bigger organizations are saying, "Okay, if you want to partner with us and work with us, now you got to prove your, you know, HITRUST compliant and that you're matching our our security there."

HIPAA, an interesting one, right? So, it's a law, not a certification, an attestation. So, SOC 2 and HIPAA are attestation. I don't think I have a slide on this, so I'll cover it now. HITRUST, CMMC, and the ISOs are certifications. So, what's the difference? Attestation, us as auditors, we're going in saying we're reviewing it against this framework, and we think it's good. We attest to everything here that they're meeting all these things. So, it's just a third party signed off on it, that's an attestation. Certification, there's going to be another body involved. So, HITRUST, there's an organization called HITRUST. So, they they are the body. They accredit us. So, at Advanced Partners, we're accredited by HITRUST to perform

HITRUST audits. So, we audit them, still do all the work, still review everything, here are all of our findings, here are all of our gaps, but then we have to pass all that up to HITRUST. They sample some of our work. They say, "Okay, yep, these, you know, this group we accredited is still doing it the way we expect them to." And then they sign off on it, issue the report, it's HITRUST certified. Same with ISO. There's a big ISO, you know, governs everything from information security and AI to cars to food and how it's food is handled and yeah. >> I don't see PCI up there. >> PCI [clears throat] is yeah, another one. Credit card

standards yeah. Yep. >> Purple was another one. >> Yep. >> Yeah. Uh just going to hold off. Just throw it cuz I do federal government so >> Yeah. >> Yeah. This 853 risk management framework how it works, so that's >> If that's on there, then yeah. >> CMMC is a little bit spun off of that. >> Yes, exactly. Yep. So, it's uh it's that on steroids. It's built up into uh the the supply chain of the Department of Defense. Now, CMMC is if that you haven't seen it already and you're in the the defense space, um by the end of the year it's going to really take off cuz there's a lot of deadlines in November of organizations needing to

be CMMC um certified now in order to work in the supply chain for the Department of uh now Department of War. So, yeah, these are just a few. Like we mentioned, PCI does exist. That's the credit card you know, so every organization that is you're tapping your credit card on, they have to be PCI um to make sure that that information isn't going anywhere. It's stored and correct with Um so, that's another good example there. >> Not even just taps. Any place you enter your credit card like uh collections agencies, anybody that deals with financial and PII. >> Yep, exactly. >> Just a quick thing. >> Yeah, I don't see GDPR on here. >> That's another one.

There are a lot of regulations out there. That's another one. >> You guys do not do them as a US focused board and they will bring in someone from Europe to do it? >> No, we also do that as well. Um it's that's another one that sort of falls in that attestation category of okay, we can review against uh GDPR. There's not a set certification, but we can, you know, take that overall GDPR guidance, uh take a look at organizations, you know, mostly abroad and you know, make sure they're meeting that or US based organizations that are also, you know, working to take this from going everywhere now. So, um they have customers over over abroad that need to

be GDPR now. So, we are seeing that as well. So, uh last slide there will be a link to our website. You can see a lot of the other frameworks as well. These are just a few that few of the high-level ones I wanted to touch on. So, >> One other question if that one. There's another eight word too that I've heard in the past, assessment. Now, maybe assessment doesn't mean anything. Maybe it's just people conflating attestation and audit. How would you characterize assessments? Would they fit into here or >> Yeah. Yeah, so assessments are Yeah. Yeah, assessments are I consider it sort of interchangeable with audit, right? So, audit and assessment, both of them it's

it's sort of the aspect of us actually going in and conducting all the work is what I would call the assessment portion right? So, whe- whether we're attesting or certifying, I would say, you know, all of that falls into the sort of assessment audit things as well. But, yeah, that's another word that is You'd probably hear me if I was to sit here and talk for a few hours, I'd probably interchange those two of assessment and audit, right? It all sort of falls back into into the same group. Yeah. >> Yeah, and again, from the federal government perspective, for many years, they had certification and accreditation and mid-2000s, it was clear and I I really support this

to change the CNA to ANA, meaning assessment and authorization. So, the government official high up has to like a CIO I authorize but the reason was certifying is almost like a hey, it's it's secure. You just put that stamp. So, you know, there was a big shift in in terminology. The assessment was the work that and it's kind of equivalent to audit. So, just terminology history for you. >> Yeah. >> Okay. >> Awesome. So, we'll dive into these a little bit. Once again, I don't want to go too deep into them all, but just give you a good idea of why different organizations are going for these different ones. Why are So why are some

going for SOC 2 and others are going for ISO right? So SOC 2, once again, attestation. So we're just signing off that we're attesting that they're meeting these things. It's not certifiable. Once again, from a CPA firm. Um so the AICPA is going to be that body that accredits us to conduct these SOC audits. So once again, AICPA is sort of that body. We are a bad partners or a CPA firms. We have two sitting CPAs in our organization. Um and then based on their, you know, training, them having a CPA, they're able to sign off on these. A CPA has to sign off on the final report for a SOC 2. Um so the AICPA

AICPA developed the trust service criteria, which you know, falls under security, availability, processing integrity, confidentiality, privacy. That's all these, you know, sort of aspects. The bare minimum [clears throat] of a SOC 2 is security. You're always going to find security in a SOC 2. The availability, processing integrity, confidentiality, privacy are all add-ons, depending what industry you're in, what your customers are asking you for, what they want to make sure is, you know, secure, what you're actually giving. Um but bare minimum, security is being tested across every SOC 2. So who needs it? Often, you know, once I like I said, SaaS providers, um cloud platforms, AWS, all of them, they're going to have a SOC report. Um most

major vendors you work with, they're going to have a SOC report. Um and any vendor whose customers ask, uh prove you can be trusted with our data. So going back to the AWS example, right? Anybody working with AWS, they have these giant data centers across the US and the world. Our data's being sent there. Now we can prove to us that we can trust that you're storing it securely. So they will have a SOC report with the audit that they went through to say, okay, we were audited by a third party. Here's this piece of paper, you can trust us. They said we are, you know, storing it securely. >> So, one of your third-party engineers

puts up an AWS database that's not secure. >> I have a question. How did you and I guess your team feel about companies like Vanta that are like speaking out this process for startups and things like that? They have a whole I hear like an ad for it like all the time. >> Yeah, so it is becoming a very saturated market. We are actually partnered with Vanta, funnily enough. So, we do work with Vanta a lot. Um, but there are more and more every day being spun up of these GRC platforms to automate, you know, you plug in all these tools you're using in your organization. It'll pull all the logs for you. It'll do all these things for

you and automate. Um, so, you know, with those tools, you know, it's sort of a pros and cons list, but you know, oftentimes it is helpful because, you know, it speeds up the audit, right? It is automating everything. There's no real back and forth with the client in that nature of, you know, these emails you're getting all the time, right? It's a lot of Oh, we can dive into your AWS instance ourselves because you hooked it up for us and now we can see all those logs that it pulled for us. Um, so, it minimizes a lot of that. Um, but you know, depending on who you ask here, right? In the field, there's going

to be those pros and cons of, yeah, we don't like it as much because, you know, how can we really trust these tools, right? Things like that. Um, but then once again, you know, okay, maybe, you know, it is pulling in a lot more data than a person was going to provide you. Maybe it is giving you better, um, evidence than a person was going to give you. Less back and forth, things of that nature. Um, but yeah, there's Vanta, there's Drata, there's, uh, a bunch of different tools now that are, um, speeding up the the clients and, uh, working with organizations. >> So, that's the main value proposition is that there is no Vanta. You have to

manually You go back and forth. >> Manually have to know your environment, which you should know anyway. But then also you would have to go in and pull all the evidence manually from all those systems. Okay, now I got to go log in to AWS, go to these different things I have spun up. Download a report, get it back, T package up, here you go. And then what's my next system? Go over here, sign in, pull it down. So then Vanta and all those different tools are just like, "Okay, sign in to this once, tell us everything that's going on in your organization so that we can try to pull all that down ourselves and package it

for the audit there." Um so so yeah, that's the the main thing that's going on in the background there. >> And then so for SOC 2 you'll see two different type one, type two. Really what that's saying is type one is at point in time. So at this exact moment this audit is happening. And type two is controls tested over a period. So at minimum that'll be three months. A lot of big ones are going uh uh every year as well. Um or that's backwards. At minimum it'll be every year. A lot of organizations are starting to do it more frequently um to prove that, you know, uh that it's happening more often and they

are compliant throughout the whole time, not just once a year. They are retaining data as compliant. So that's But in most cases customers want a type two. Um so type ones are going to be organizations that have never done a SOC before. They're a little scared to get into a type two. They want to get their feet wet, right? They'll go for a type one. At this point in time, you know, check all these things. Okay, we're good there. Now let's move on to the type two. Um So just a little bit of a um which check is there.

All right, so HITRUST. Once again, certifiable framework. Um harmonizes many standards. So HIPAA, NIST, ISO, PCI. A lot of these standards I like to call it like a Frankenstein standard HITRUST puts together. They took all the best things from all these different things, put it in together into one, tailored in towards the high healthcare space and made the high trust seats. Um So once again, it's a prescriptive control set assessed by an authorized external auditor. So once again, managed partners, we have been accredited by high trust to conduct these audits. So they review our process of auditing all these things and you know, trust us to conduct these. So who needs it? Healthcare and any

organization that wants to rig a rigorous certifiable badge of assurance, right? So HIPAA once again is an attestation. If you want a certification of we're meeting HIPAA plus a lot more, they would go for high trust rather than HIPAA cuz HIPAA does fall into the high trust layer and a lot of times organizations just prefer high trust over HIPAA because once again, it's covering a lot more aspects of the general security stuff. Um And yeah, unlike SOC 2, SOC 2 is a little bit more flexible. High trust has very defined scores, defined requirements, much higher bar as well depending on what level you're going for. So E1 foundational is going to be more and more basic

effort where if you go up to an R2, that is going to be here's 300 different controls you got to meet and an evidence list of 500 plus things and show us all this stuff, right? I do that at every single level E1, I1, R2. R2 is a real beast so what it sounds like. E1 is good for a HIPAA organization getting their feet wet of okay, here's 44 of those controls that you would find in R2. These are what we call the essential ones. Start here, you know, make sure that your organizational line has the essential ones down. So it sort of builds from there. E1, I1, R2. The numbers do matter in those. So an E1

is a one-year certification, so you re-up every year, you get it re-conducted. I1, same thing, more controls, but once again, every year you're getting it conducted. R2, a million controls, right? But, it's evaluated every 2 years. At the one-year mark, we take a sample of those bunch of controls just to make sure you're still You didn't change anything, you're still meeting them, but a much smaller sample size at the one-year mark. All right, I'll move fast through HIPAA since I sort of covered it. Um once again, it's a law that turned into an attestation that we can conduct saying you're meeting with the law states, right? Um once again, for health care providers, uh clearing houses, uh

business associates, and health plans. Um this one is interesting though because non-compliance with HIPAA carries, you know, penalties, right? So, if you do not If you are not meeting meeting HIPAA, I had to watch my words there cuz I almost said if you are not HIPAA compliant, which doesn't exist, you can't be HIPAA compliant cuz it's not a certifiable framework, right? So, that's sort of where we run into that issue of what word means what then, right? But, um non-compliance equals penalties when it comes to HIPAA, right? Cuz it is an actual law. >> Just a question about that. >> Yeah. >> I know that HIPAA has business associates and a business associate agreement. But, you're saying that the

way that HIPAA and HITRUST work together is that say maybe you have signed a BAA with a hospital system or I'm sure >> Yeah. >> they still will not, from a risk standpoint, do business with you even if they could under federal law because you are not HITRUST certified. They expect you to to hit a higher hurdle basically through the certification. >> Exactly. Yeah. Sort of prove it a step further of how mature your organization's compliance frameworks are with HITRUST. All right, so like I mentioned, CMMC. So, that's going to be Department of uh Department of War's program uh requiring contractors to prove they uh protect federal contracts and controlled unclassified information and CUI. Um, so that's going to be any sort of

organization that wants to get government contracts, you now have to be CMMC compliant. You have to go through an audit if you want to get contracts. You can't even be in the portal to win contracts if you don't have the the CMMC right. Um, or so, what the deadline is November. Um, so that is going to be a more federal, uh, government type uh, control. Different levels, once again, similar to our high trust level one, more basic self-assessment. Level two aligns back up with NIST standards and third-party assessment would be conducted by an organization like mine there. Um, and level three, which is, uh, uh, the big one, the highest level that you can be certified against.

And once again, this is, uh, working in the contracts. Um, so a lot of organizations won't work with you if you don't have this type thing. And like I mentioned, you can't even bid on different, um, contracts if you don't have this certification.

>> All right, so big thing to take away from the ISO 27001 is going to be your main information security management system standard. 42001 is your new AI management system standard. Um, the good thing is, so a lot of organizations will start with the information security one cuz that's been around for a lot longer. Um, it has a lot of the same bare bones, right? So, um, your ISO 27001 will tell you, "Okay, do a risk assessment, you know, have management sign off. Management needs to be meeting at these things, right? Right?" When it goes to the 42001, the AI one, it's like, "Okay, conduct an AI risk assessment, you know, make sure all these things are happening with AI." So

it's the same sort of baseline, just now you do it for AI. Um, and there are some additional things in there like impact assessments for AI that you're actually conducting impact assessments and things of that nature. Um, but once again, ISO is, uh, right there in the name, international, so it travels across borders, so a lot of other organizations across the world recognize ISO more so than they would recognize some of the other frameworks. Um, and uh, yeah, so that's the sort of main aspects there of ISO, but lots of ISO standards, um, 27701 is going to be a privacy one. Um, and yeah, a lot more are coming out, but um, those are a lot of big ones. Um,

there is um, some more environmental ones as well that are coming out that we can see organizations get through. So, at a high level, what do you need? So, if you want to sell software, so I have to, and this is not an exhaustive list, this is just a high-level list of what you would be going through based on where you are. Operating in healthcare, okay, HIPAA. If you want more, I trust. Um, rigorous, certifiable assurance badge, I trust. Uh, uh, you want to contract with Department of Defense, CMMC. Uh, globally recognized ISOs, you're in AI or you guys want something. So, what does an audit actually look like? Um, so this is sort of the flow it

goes into. Number one, you guys scope it off as a whole. What is in scope? What is the architecture that we're evaluating? What exact system are we looking at? What employees? Is it the whole organization? Is it a portion of that organization that we're looking at? All falls under scope. Plan and request. Okay, so now we plan the audit and do that list of all the uh, evidence requests we would want from a client, uh, often called an IRL, um, that we would ask the client for. Okay, here's a list of everything we want to see from from uh, that they need these controls. Then we move into testing. Okay, we're going to take all the

evidence you gave us, we're going to look at all the evidence, mark it up, once again, prove how it's meeting these controls, or if there are gaps, prove how it's not meeting controls. Then, report in the end any findings, expectations and um, communicate that back up to the stakeholders of the organization. So, getting even more detailed, what my life look like in um a weekly basis, right? So, you know, when we mention walkthroughs, like so I would be talking to clients saying, "Okay, show me how you do these controls." "Oh, you don't understand what I'm asking for." "In this evidence request, here is exactly what I'm asking for, right?" So, having these walkthroughs, walking the client through

the evidence requests and controls. Reviewing evidence, so opening up all those PDFs, opening up all the Excel documents, opening up all the screenshots, looking at them, marking them up, circling things, you know, marking down gaps of, "Okay, um there wasn't a performance review conducted for two of the 15 employees I looked at." Um marking that down. Um writing up the exceptions and gaps like I just mentioned. And then ultimately closing calls, so reporting that back up to um the stakeholders at the organization, and then issuing that final report once you communicate that and all those um those findings that you have.

And then what Why matters, right? So, going back to what audit terms claims and the trust, right? It's not just an organization saying they're doing it, someone else outside your organization came in, tested it, said, "Yes, they are actually meeting it." Five things to walk out with. We are going to learn perfectly on time. So, if you don't remember anything today, remember some of these things, right? IT audit is independent verification. Everything reduces to controls. Uh are they designed well and do they actually operate well? The framework depends on who's asking and why, so what your customers are asking for, what your partners are asking for, and what industry you're in. Auditors don't build controls. Um they

give other um entities the trust them. Um so, we're not in the systems like actually telling organizations, "These are all the controls you should be meeting." And actually designing them, we're going in and just evaluating a policy set or a framework set. Um and then the real product is trust, accountability, and resilience. Um that is in that piece of paper that we did. Yes. >> So, I have a question. >> Yep. >> How do you become an auditor? >> Yeah. Yeah, so I right after college decided I don't want to be into coding and all this stuff. I had this amazing uh computer science degree and I did not like coding. So, I decided, you know

what? What else can I do? Because cybersecurity is a very big field, right? It's not all coding. So, what else can I do? Oh, there's this whole business side of things. I didn't even realize. Let me go get my MBA. Let me go learn business and meet in the middle there and get into IT audit, right? So, I will say right in my undergrad, I liked the information security policy type courses, how you write policies, how you, you know, do governance across an organization. Once again, didn't really like the coding side. I wish someone had this presentation when I was in college because I didn't know this existed in my senior year, probably last semester of

undergrad. I was like, oh, didn't even know IT audit existed. Like, maybe I should look into that. And then I went to masters, right? Um If Yeah, so that's that's how I personally came into, you know, like people come from different backgrounds, you know, accountants often come in and then learn the tech side of things. Tech people often come in and learn the business accounting side of things, right? And how to audit. Um so, really there is a it's really is sort of that middle ground. And then from there you start building up knowledge from the CISA or gaining your certifications that are audit specific and uh things of that nature. Yeah. >> Two words I often see in these reports

are the phrase compensating controls. Are those words that you'd care to give any color about or context about how an IT auditor might use an expression like that to justify or explain something? >> Yeah. Yeah, so compensating controls are okay, this framework says this thing. As an organization, we evaluated that, decided we didn't want to do that, right? But, we are implementing these other controls to make up for it. So, that's like the high level of what a compensating control is, right? So, okay, it doesn't make sense for our organization to do exactly to the letter what this is saying, you know, maybe your organization's too big, you can't be looking at all these things, you

can't, you know, that all falls into sort of the risk assessment of an organization. Okay, we're willing to take this risk, we're willing to take a gap on this assessment, and say, yep, we know we messed up there, we're aware of it. We're not going to improve it, or maybe we will, you know, we're just not there yet. But, yeah, compensating controls flow into okay we don't do this, but we do this to make up for it, um ultimately, and you know, >> So, is that usually signed off on by management? Like, if you've got a company that's an automotive one, might not have a large need for change management. Maybe they would describe how they have a system that works for

them sufficiently that you'd be writing the narrative on, they'd be signing off on, and you'd still be providing that third-party insurance. >> Yeah, and often it ties back into the policies and how they write their policies. So, it's SOC 2s and things like that, you can sort of tailor it a little bit more to, okay, we evaluated it against their policy and how they do it within their organization, and we're attesting to that more so than SOC 2s and other frameworks to do things exactly like that, of, okay, let's drop this, but add these three in there to make up for it to make the whole picture there again. Yeah. Yeah. >> So, what's the cut point for passing and

failing? So, when you go for an audit, you find some are like they met the eligibility, and some they didn't fully provide you the screenshots of the evidences. >> Yeah, yeah. >> So, at what point you go, like you, for example, driving test, there's a 10-point cut or something, like, do I have something like that? >> Yeah, yes. So, for the individual control level, like someone's going to be testing, okay, if someone was terminated, you got to remove access, whatever. We'll take 10 employees that were terminated within the last year. Um it sort of falls into where the majority is. So, if eight of them didn't get access removed, we'll say, "Okay, that's failing. We can't trust that's operating

effectively." Or we'll take a larger sample, review, make sure we didn't just get a weird batch, and you know, if it is still failing very often, then we'll say, "Okay, now that fails. It's obviously not operating effectively if the majority of it is is failing that control level." So, each framework also goes up to a higher domain level, where, okay, here's the big domain with all these different controls in it. So, a control in here could fail, but if you're still meeting the overall domain, okay, you're passing, you know, the framework as a whole, but here's a gap at a control level of uh you know, this very specific thing you're failing, but the overall picture, you're

basically there, right? You're doing all the other stuff. You're You're meeting all these other things, and you know, maybe it's a less risky thing, so we can accept that gap, but you're meeting that domain. When you get into the high trust and some of the other frameworks, they have a very spelt-out for each one. Assessment in high trust, each domain has to hit an 83 score, and then you get that score by doing all this math. They have a whole rubric I could show probably pull up for you with charts that you go and determine each score on each control. Um so, different frameworks will issue that. >> So, you're doing your taxes. >> Yeah, exactly. [laughter]

You got FedRAMP, CMMC is another scale as well that is determined if you're passing or failing, so it depends by the framework, but um you know, some frameworks have it very spelt out, and SOC 2 some it's it's a little bit more, you know, are you meeting the words that are the AICPA um spells out in the framework. Um and if you're not, you know, it's kind of saying controls, how you're making up for it, things like that. Okay, based on all that, we determine you're meeting it or not meeting it. So, it's a little bit more opinionated in in a SOC 2 than it would be high trust where it's spelled out. You know you failed that cuz you

got a score of a of a threat. >> Yeah, so I think that's right. >> Yeah, yeah. Any more questions? Awesome. >> Thank you very much. >> [applause]