A Practical Guide to Governing AI Integration Before It Governs You - Susanne Bitter

Morning.

How well do you know all the AI that is touching anything in your stock? How do you know it? All 100% 99% we like to say just to be sure. Who would confidently feel that you know everything that is AI into all your technology that you have? No. no one. That's quite weird, isn't it? Cuz that's all what the world is all about right now. And even we want it or not. AI is everywhere already. And when we think about that, it was um probably an experiment at the beginning, but nowadays, can you imagine life without AI, without your ch without the cloud, without all the LLMs? who can go back a year or two. Hello my

friend. Me too. Good old days. >> I quite happily do without it completely. >> So the craziest thing is when we going to think about that is the board. Maybe not everyone is part of the corporate or the board but most of the CIOS, CTOs, CEOs or the seuite they all having one expectation right and that's to make the AI work right has anyone in your careers or day-to-day basis experience that the seabboard would say no we don't want AI no don't put it there You are a very big anomaly. >> I know I have a board that are terrified of it because I educated. >> Okay, that that is a clear consequences of that then.

So sadly enough or maybe for the generations that aspire from the from the uh cyber perspective the AI is a good thing. We may think about that how long is actually AI as a term available. When do you think that the artificial intelligence have been spoken first? Is it 2 years, 5 years, 10 years, 200? >> Absolutely. You know what? We go with you cuz that was the Roman letters which probably put AI next to each other. Right. So the thing is it's not anymore about the hype. It has been here. I just think it's suddenly within the past few years has emerged [clears throat] significantly. And I have a one question. I don't have the time. That

means I can speak unlimited. Oh, we go old-fashioned. Okay. So, the crazy number which you see on on the presentation, can anyone imagine what 155 billion doesn't matter if it's dollars, pounds, or whatever. um how that that number actually look like like can you comprehend how big this number is and that is expected by 2030 being taken by the AI market. By the way take any pictures if you want of any of the data if you want the presentation after we may work together and you may get the presentation as much as you want but take the pictures whatever you need on that. So the crazy thing is that it's not about the money which is the story is

the exposure of it. How much the AI actually is expected to grow and that is why what I'm about to tell you or try to tell you is about the exposure of the AI. So when we think is the biggest and the worst thing on all the AI isn't the AI on its own, right? We like it. We like it Hazel. We like it. >> Yeah. You love it. Um when we think about that, it makes our life easier, right? We kind of feel like we don't have to do all the admin things. We don't have to do all the type. That the biggest problem which I see is not the AI on its own, but is the integration.

how we can actually place it that it fits its purpose but also it brings a certain expectation being met if that makes sense. So when you think about that that you have your ERP, you have your CRM, you have all the informations around then you have a new tool that is coming into your stack and all of the sudden it has the AI elements or there is an update from the current technology. Oh, we just including one of the feature which is based on the AI and we like yay finally one more. Right? So the biggest problem isn't the trying um the AI to embed in you. The problem which more of the things we are looking

in is from the measurable and from the board point of view is how much P&L is actually making it and the problem is what has been actually developed and I think this has been the MIT project is that up to now the rough estimate is only 5% of the AI integration or AI things actually bring or are reflected in PNL in the different numbers 95% is just play around and doesn't contribute to anything what the businesses are built upon that is quite scary but what we are expecting is most of the companies having something like this you have some employees you have some internal stuffs you have some of the data and we know in the regulatory point

of view you have GDPR or any of the privacy settings. Then we have some chatbot. We have some CRM which all of the sudden gets the AI in place. We are having some marketing things. Does it look familiar? Does it look like you know this architecture a little bit on top level? I'm not going to go too technical here. We have experts for that. But does it look familiar? No. >> Yeah. >> Thank you. Thank you for collaboration. I hoped because this is my day-to-day bread. So the problem is we don't have any I call it like the choke points or the things that would stop stuff. We don't have any gateways. We have no

governance over it. So when we think about that we are ending up with the things that data are trained on. We are having we are having u retention we are transferring out of the region. Why do we care that our data are transferred out of the region? Because of the legal things, the consequences of the legal and the fees and everything connected with it. But the biggest problem which I see currently from my day-to-day experience where I'm responsible for the third parties uh assessments and integration of the AI is that we are introducing to our estate or I could say that into our infrastructure for parties we never ever vet it nothing that we actually sees. So

the biggest concern for me is do you review your vendors vendors? Does anyone go to the fourth fifth? Any hands up? You know we have a beautiful morning. We can do the arm stretch if you intend and inclined to collaborate with me. A few people do, right? But [snorts] do we really know? I had the one example where I was doing a supply assessment and I said, "Okay, tell me your infrastructure. tell me how does your tool that we are getting from you works and it was a component after component database here server here you know functionality here lm here over there over there over there I was like what's yours nothing I was like damn so

I had to go and actually verify the key components but how much time you going to spend on it and the biggest problem isn't only just to know which I love to know but is we do have a situations why where AI or we could say artificial intelligence hides and I'm going to talk you through a few of the stories that has happened in the world and hopefully I have enough time. Can I check how much time I have left? >> 25 >> 25 minutes. Excellent. Thank you. So where we see that the [clears throat] risk except of having any AI into our inside our technology is that LLMs are var to the products. There are

autonomous agents which have credentials. We do have a lot of shadow AI. Do you remember a year ago we were talking about shadow uh it now we talking about shadow AI? Does anyone come across the term already shadow AI? Yeah. So, it is a real thing. And the problem is that some people can mean it well. They can mean it like they want it and make it better for whatever reason. But sadly enough, it's unsanctioned. We do not know what that tool can do. And that's a big problem. And then we're getting to the number four where third party and AI supply chain good old supply chain will happen, why it matters, and what stops it. We're going

to just cover on a few of the examples. So for the matter of the time, I'm going to run through this, but I just want to put you into the picture where you remember back in the time, I think it was at the beginning when the AI and the LLMs started to be a big thing. I think it was like 2022. Um, Air Canada has actually had a chatbot that uh told the customer about some reference about some money being happened and when it came that uh airline was supposed to pay, they said no and the customer say yes. And how [clears throat] it happened just in the just in in the in the in the fraction of

the of the story is they had to pay because they made the selection of the AI. They make the selection of the technology. They program what it has to say. The fact that they didn't have any ground of the answers. They didn't have any human review and they actually didn't even own the story is a problem. They didn't own their decisions. How reputation and from the risk point of the view how can it end when companies select a technology and do not stand for it in face of the customers that's the air Canada so we call it that the chatbot invented a reference policy because it didn't have enough data it doesn't it didn't have enough

guide roles or guidance or something like that the next thing is we can go about the M365 co-pilot our good old co-pilot vulnerability. Anybody familiar with the echolik aka CDE 2025 371 32711 anybody face that do you actually know that it has happened in the co-pilot it's crazy right so what has happened was that um due to the vulnerabilities there has been an email that was crafted and send emails uh sorry send send the files out. zero click no user action the fields real that is the first real prompt that actually utilize injection as a weaponization for the data to be found and that was all in the life of AI product we talk about Microsoft just

saying the thing is what could have happened what would change it it's quite simple just treat every model that it's untrusted you know zero trust you know do we trust trust until we do not trust. It's more like we don't trust until we trust. Something like that, right? So, it was simple. Or we could just strip it and scan it, right? We could just have a kind of like the techy geeky things around it. Or we could just go to the good old locks and alerts. But who knew? It's a new technology back in the time even though we talking about last year only. But it's a long long time ago. Um, another thing is imagine nowadays we

moving to the agentic style, right? Agents do things. We have like a virtual terminators and robots and everything like that that do the things for us. Do we actually know that what that agent works on and what is their capabilities and what kind of credentials, authority and abilities we actually weaponize them with if we're going to use the word. So there were two examples on the replet where the AI agent deleted all the production database. Yay, no more work. We don't like that, right? um Amazon fume Amazon good old Amazon I can't live without Amazon by the way but it was a poison pool of the information and about 1 million developers actually downloaded the database and

used it because they trusted it Imagine that. Imagine how crazy it is. Why does that matter is if you have the agents with a live credentials and they generally use the credentials usually what was given to you in your scope of the work. So it's like your duplication right like I use the uh agent Smith because he can duplicate and everything like that. It's the same thing. So when you're using the agents you're literally duplicating the alterator alterator. The higher you are, the more privileges you have, the access you have, the problem becomes more and more. And that without no guard rails, no restrictions or even let's say a change checks or controls can go quite wild. So that being said,

we can go to my old employer, my company who has been giving me a breath for 10 years. I left before this has happened 2023. I think everybody has to remember this making it the news. Samsung engineers in semiconductors which they they do the chipset in electric vehicles in mobile phones and TV nothing works without chipsets. I do talk here to the geeks. You know what I'm talking about what the chipsets do. So imagine that five people have taken the code and put it into um into I think it was um CH GPT. Yeah, it was CH GPT. They probably meant it well. But the problem is it was the source code. It was the data that

has been really really like the knowhow the IP of all the stuff. So giving the people the tool um that has been verified and has for example a zero retention or no training on the data would solve it or a simple procedures around DLP and about reduction of the data before it leaves your text stack or we can just go to the good old policy everybody loves that not just ban don't ban things because people and I think everybody has used some workar around whenever I'm prohibited something. Here we go. Work around. And we don't want that. It's a risk to the organization. [cough and clears throat] Null. I can't pronounce it. Nullifi is actually a great great great example

where the malicious model on the public has broken the files and literally reverse the shell pass scanners. So technical. So technical. So technical which actually happened that there has been uh a reverse shel inside on on on the files and if you imagine it was packed with the zip instead of the zip it was 7 SP or 7 Z's or something like in the compression the model itself has included the malware but because it was downloaded and it passed all the detections what has happened it materialized. The second case what has happened was that AI invented a package name that has been put to the you know documentations and everything like that. What the malicious actor did they use

the name because it was already verified and used across the people. A easy way how to maliciously intervene with the technology and again what could have happened we could just do the scan. We could just have a little bit of the governance. So I wanted to give you a blast over a few of the examples and these are let's say past 3 years four years only there are thousands of the cases and which we know about but there is so many we do not know about. So how actually we can fix it is having a governance around that without sounding that everything has to fit to your company or to your needs. I just think

have a gateway have a set of the controls on the inputs of the policies on the outputs and audit them good old basics right so when we think about application and imagine that you are producing the application you will have and I will now try oh oh and here as well just in case you would feel like left over so you have the client application you have the first check is through the upgate way which is already a one thing that you can kind of put in place then everything when it comes to any of the AI is actually working within the subnet where you are having for example this is on the Azure vinet you have the internal

mode you have all the private endpoints and if you can see everything is happening inside of the model I'm not going to talk to you through the through the technicality of the models but If somebody would like to know the technical behind it and all the controls that can be placed, I'm going to share that with you just for the sake of the time. I want you to just show how the good let's say technology architecture could look like. But somebody can ask like oh we don't work on the Azure so this is not applicable. Guess what? AWS same principles, same thing, just different models, different RS that you can actually work on, but the same

principles. And guess what? GPS cloud Google the same thing. We just talked that now instead of the of um Azure Open AI, we have the Vert.x AI or aka Gemini and stuff like that. The thing is the pattern is uh around clouds is agnostic. That means you can make it work anywhere. Right? So if you want to take a picture here is the overview of the six cases which we just covered through that and I also put on the side what actually the easy fix is. So you don't have to take the picture of every single slide but I put make it easier and you can see how crazy it is that in in order for uh for example the echoing

not happen is just don't trust them scan and block them you know for the nullifi nullify uh scan [clears throat] scan know what you're having. So if you want to start and it's something that you want to somehow put in in the work do just the basics. So I already show you the picture around the gateway and having things in place. What I also would somehow um recommend you to do start with those four basics which when we going to look into them who does AI inventory anybody has a tracker one two started three right do you uh any of you work with within let's say European Union and you are familiar with the A

I AK AI act you are regulatory and legally required to have inventory of SAS. When I start into the introducing it into British quite a big company they were looking at me like a crazy person. Why do you want to do inventory of the AI when we do not have inventory on our devices? Leave alone of all our other tech. You do not know why you want all of the sudden this. Well, I told them because we cannot defend what we cannot see. Simple as that. You can't do that. So, it does the inventory will help to kill you the shadow AI. It's also going to help you uh remove the biggest risk of not knowing and also

maybe some of the money because when you know what you're doing you stop reducing uh sorry introducing and you therefore reducing amount of the technology because people want I want this I want this I want this function I want this function and so on and so on. So you can start with that. Second thing is the second basic is all the sensitive data that should never leave your com your your your environment or your uh I would say company which is quite hard to imagine right now uh make sure that it doesn't leave you know it should then help you reduce any potential breaches and the exposure all world drops if you are not introducing it so redact the PII or

slash any business any confidential data. The number three who knows me or at least have an awareness around me. I do love a good contract. However, we know [snorts] not not every single negoti negotiating power is always available. So I would at least say make sure that within the TNC's if you can't influence the contract you at least know that the data are trained or or not and they are not retained. You would be surprised how many companies actually are okay to tell you yes zero retention yes zero training they just want to sell you the product. maybe they would be interested in it but at the same time if it's your requirement they're happy to switch it

on. So um therefore that way you literally are cutting off the four pass four parties that actually send the data at the source already. And the last bit, evidence and early warning. Set up some logs, set up some uh some audits, make sure that you are ever what's happening and you can catch a rogue agent like agents mate is going to be looking at you. Um, that's going to also help you save the time before your company and your breach is going to make a topline media post or maybe just local because it's so common that it's becoming kind of a media fatigue and they don't want to report that. But if media is not enough for you, maybe this

value is and that is 7% is and 35 million euro for breaking and the possible fees or penalties I could say from the AI act. If you are somehow working or utilizing AI and you are not going to do it alongside of the regulatory requirements. So the problem is you have a deployers and you have providers under the AI act. Doesn't matter where you are as long as you AI you are falling under that in either one of the regimes. know how you're doing that because if you use AI for any of your process you are using AI who develops it they giving you the AI the rec um the responsibility sits with both and you can imagine who does in the

privacy GDPR introduce 4% and 20 million AI act is pushing it up so we pay if you think you will somehow fulfill will uh fulfill um the requirements. You have another thing to actually worry about. Another a huge number 365 billion of data in the data broker market. You know your data that you're submitting you find on the Facebook or Instagram when they want you to see what kind of potato you are. That kind of data. I always wanted to see and know if I'm a green potato or yellow potato and in order for that to know which clearly there's no logic behind I give them name I give them surname I give them my

address sometimes I give them the credit card because I so want to know what percentage of the potato is actually okay and which is not so all those things but it doesn't have to go to the silly situations like this the biggest problem is that whenever you touch your data touch the information is there and whenever it is it's building up the picture and I can tell you that there is some person hopefully it's a person knows more about you than your family ever did which is quite scary because I'm sharer and who knows me I love to talk and I love to share everything I'm open book but I think there are some stuff that people know over there more

than my mom or my brother or somebody around my closer nose. I don't know if I want that to be to be um so much available. So whenever something leaks out of the your s solution that is where is happening and it's a big big big big big market. Um, if that isn't enough, Attek has got AI too. Surprise. Who would ever think? [clears throat] So, we are not anymore at the billions. We are at a T. Is it trillions tetraillions or whatever? Like I can't even compare never appear on my account yet. So, I do not know that number. Once it does, imagine what I going to do with that. So, this is a projected annual

cost of cyber crime. It's rising and guess what? Increasingly AIdriven. So when you think about that 10.5 is actually a possibility of a budget of a country and it would be third in the world by the GDP. So imagine we wouldn't have a borders that on its own as a cyber crime would be third biggest country at the world based on the GDP gross domestic product. Can you imagine how huge it is like third in the countries who would be the first one? I don't I don't dare to ask if it's China or US but probably alongside so we will have US China and then cyber crime. That's it. If you think about that, that's a good outlook

for us, right? So imagine Oh, sorry, sorry, mistake, mistake, mistake. Um what all actually also happened is that based on the knowledge based on the availability the sharing and everything like that AI made attack so cheap so cheap that even case we don't have them here anymore that I would lure them on the dark side could pull it the problem is the controls against that are kind of delayed so the information already leaked it's available and nowadays 4.4 4 million cost around one of the breaches um in the United States. Sorry, the average is between sorry I think I put it wrongly there. The cost sorry the bridge cost 4.4 four between it should be

between sorry about that. Um but when you think about that from governance point of view we we we say the basics know what you have put some controls in place put some policies that can actually save and compare to these doesn't matter if it's dollars or billions I just use the uh IBM cost of data bridge so I'm just using the native uh currency then translating it to the pounds but you get the sense of it the government is cheap next to us right nobody no co company will ever ever use 4.4 four or even 1 million just for the governance right so you have the argument to make it cost efficient and actually save the headline so if you

cannot govern uh sorry you can't govern what you do not see so there is a three questions I would like you to ask on the dawn of my retrief and going back to the cyber world is where does our data actually go when someone uses AI do you No, if not do your homework try to do it. Second question is what can our agents do and with those credentials? With whose credentials? What it is actually? Do we know? Because if my agent is replicating the same authority like I have and I'm the admin with the highest privileges, damn that agent is powerful, right? So maybe think about that. And the third question is who are our vendors vendors

and when do they sit not only physically but within our infrastructure and how they do that. The good old diagrams the good old things that you can actually somehow somehow um request or at least paint yourself if you want to or you can ask AI to do it for you. That's also what has happened in my life. Um make sure you do that. On that note, I have had a situation where I was reviewing one of the supplier and asked, "Oh, can you give me the policies? Can you give me like all your procedures? Can you tell me how you do it?" So, I sent him a questions like on top of my standard questioner. Do you know what I got back?

Here is a governance pack of what you were asking clearly from Claude. And actually, I got an answer. No policies, nothing. just an explanation, a very technical, very detailed 25 pages of a cloud BS. I look at that, I'm like, you can't be serious. So what I did, I weaponized my age and I said, have a talk. On that note, thank you very much. I think I made it on a minute. >> And if you want any of the sources or any of the frameworks that I have utilized, you have them here. Presentation available. And I hope you have got some insights and some provoking thoughts how you govern AI. Thank you so much for your attention.

Have been a pleasure. [applause] >> Any questions? >> Okay, I've got time for a couple of questions. Look around the mic so we can pick up on the tape. So, anybody got any questions? >> Oh, okay. Woke up yet? Oh, come on. I'll show it back. Right. Okay. Give me a sec.

>> Um, thank you for the talk. I just have a quick question. How can people like test their governance and how can they just make sure it actually works? >> That's a very very good question. I would say there are two elements to what level you you do the AI governance. I would say the maturity will progress how you are doing it. If you're going to start with the basics, it's actually going to be just a sim I would say not symbolic but you would have the artifacts for the physical like the inventory stuff like that. going into the future it will be literally um having a checks where the data goes what the flows do through the locks and

everything around that how and how can you position it in your environment and how it's going to look will depends on the maturity of the company I can't tell you like a good governance and how you going to monitor that you can maybe create an agent who going to do that for you and there will be like analytics going how many data goes where it goes but I think Before you go too technical, start with the basic that you at least know which AI models you have, what technology actually have AI and what it does and how does that work. So for example again from my experience we do have a technical uh technical uh

subsolution that are working with personal data very very prominent top level people in this country. Do you think I want to use an open AI for proceeding with this? I would prefer to have internal self let's say hosted model only that will be in the context and how it's going to fit to you. If I do have a marketing I don't care if it's open source and it sits behind the wall or something. It's how your company actually analyze and knows what actually goes through the AI and why they do that. Similarly what happened with Samsung back in the time everybody prohibited CH GPT Amazon prohibited cha GPT there was a big wave of prohibiting

that technology until you can deploy it safely and securely that you have at least a confidence not 100% but at least some confidence that you know what's happening and you can we have a in in check we have a saying that you can put the hand in the fire for it once you feel like that you know the governance works otherwise you're going to burn your And and you don't want it. It looks ugly and it smells. >> It does. It does. [laughter] >> Any more questions? I've got a question. Ask a question. >> Of course. Should it? >> In terms of what you talked about earlier, the different diagrams within. >> Yeah. Yeah. >> What's most important to do first? DLP

and protection of the data for identity because there's a lot of tax on both. >> Which one? The first one. actually know what you want to do. I would actually start like what what is the critical I would say submit netting it putting it in the condens environment would be my first point and then definitely the the authentication and the assets that would be my choice but it depends cuz I'm more coming from the governing than technology people would probably put the technical controls first before they do but I would be segregating the RS and making sure that I have it condense around first. >> Does that make sense? >> Yeah. >> Any more questions? Quick one. Um,

>> um, so where do I work? We have got a slightly annoying problem that we chosen not to use anthropic servers because they're in the US. We don't data to them. >> Oh, sorry. Which services is it? >> Anthropic. >> Antropic log. Yeah. Yeah. >> Yeah. Um so tools like mirror and co-pilot they're they're starting to force you to use >> claw service to do stuff. So it is stop us from getting the most out of these tools. What sort of ideas you have around that but is this a real problem using >> uh so it depends and I will just be very careful what I answer here. It depends on the context of of your of your

organization. If you work in the secret services, if you work in the government, I would say big no no. If you are in the private sector and you're just naturally working because of the um I don't want to say fear of GDPR and the US and you know kind of depends what kind of data you would be proceeding without. You could have a model and policy where you're going to have nonsensitive data can be pro processed by entropic by cloth or anything like that and something will be using just selected model in the co-pilot. You do not have to use cloth in copilot. You can select the model that will be available right. So you can

do um let's say a hybrid of the things where you use and what you do not use and what is okay what is not okay and put the controls in place. I personally would trust much more anthropic than anything else and this is my pri private opinion because I have done a lot of due diligence and a lot of of their let's say verification and vetting and I do actually like the company methodology and what they stand for and how they have been created. So if you know the owners on the entropic they come from the open AI because they couldn't do everything in open AI they selected themsel and they develop it to be

similarly what was up to signal in the communication so I do trust it the model can change which is a risk so when it comes to US base and the data I will start with classifying your data if you are okay with it or something like legal or some restrictions or regulatory things prohibit you from it I know and this is current status can change tomorrow that the models have a zero retention and when you use them correctly and you set up and defaultly do not put uh into cloud you do the basic that no retention no copies and stuff like that you can do a lot of personal and company setups [snorts] that the data will be quite okay

but it depends in the context I would always operate in context of the organization and make sure that when you do something like that you understand what data you will allow to go through that and if you're okay with it but I would say because the collaboration they see the market we spoke about the number for the AI the biggest problem is like it's inevitable it's the same if you imagine 20 30 years ago when everybody start putting everything on the cloud and everybody was like no no no we losing we losing the control out of the infrastructure what we going to do now everybody's like cloud we don't want going to worry about that. That's going

to happen in the future as well. It's just knowing and understanding and doing the assessments and the I would nearly say like the risk analysis. Are you okay with that? If things going to go the worse way and if you all semi okay or okay I think the capabilities of anthropic and what it can do is marvelous when you prompt it correctly and you use it correctly otherwise it's going to split and cost a lot of money which a lot of people do not think about is the amount of the transactions it cost to produce something through cloth is very very high compared to others and the outputs are not always the But that's my personal opinion and

hopefully I answer your question even though it was more rounded around the whole wire nets. >> I have one last question >> if we can. Yes. >> Which is uh with 155 billion as the projected revenue in 2030. >> Yeah. >> And open AI burning through cash at about $300 billion a year at the moment. Will any of this stuff exist in 18 months? >> Well, I can't tell you because I don't sit on the board. [laughter] If I did, I probably would be under the NDA and very disclosure, but it's a rhetorical question because if one of these goes bust, how do we deal with it? I think that's >> yeah I think if it's more like if I'm

understanding correctly what be going to happen if anthropic for example gon opened a gun because they have underestimated the business operating model that on its own is your business continuity you could use ISO standard for it or just a normal I would say we call it um common sense and be ready for it because it is possibility and you would be surprised how many times I have talk and ask the board why are we only Azure why are we only on the Microsoft what we going to do if Microsoft going to have an issue all our business going to stop because we do not have anything and for example government for their contractors are requiring a landing zone

and being able to go across the across the different different uh platforms. So you do have a possibility to migrate even though it takes time from the Azure to GCP to AWS and so on and so on as their backup because the contract with the government is so lucrative they don't want to lose it and they're going to invest into that. The same way I would say for the models yeah don't think just one have a dependencies clarified and make sure that you review all what is coming through that but thank you very rhetorical good question in the end and I think we are up to the time now. Sorry about that. [applause]