Combating Generative AI's Privacy Abuses

So hi everyone, welcome to our upcoming panel. I hope all of you had a good lunch and now ready to listen to our wonderful panelists, among which — among them is Raji. Hi everyone. Trti is our panel moderator. Hi everyone. Nandita. Hello. Muhamed. Hello. And Aura. So please give them a round of applause and a welcome. Thank you very much Alexandra.

Before we begin, I would like to express gratitude towards all the amazing volunteers, sponsors, organizers of San Francisco BSides, and most importantly all of you, the attendees. Because of you, this is so special. All of us feel like movie stars here right now. This hall is fuller than a typical movie show. I feel so honored and privileged. I hope all of you got your popcorns and coffee. I don't want to see result of food coma here. I know we just had delicious lunch, so you guys are ready. Let's get rolling.

It's no secret that the world is smitten by GenAI usage everywhere. Whether you're booking airline ticket, you see the ChatGPT, or you're writing user story to help your product manager or engineering manager, we are using ChatGPT. It's everywhere. Let me tell you some of the amazing stats that I recently found out. By 2026, 80% of the enterprises are projected to adopt LLMs, and we are seeing that almost every day in our organization as well as our friends' organization, and all the common tools that we are using, there is influence of GenAI. In fact, EU law enforcement has predicted 90% of the online content will be AI made. OpenAI platform is fueling a booming app market with 2 million developers. How amazing is that? By 2030, we are going to see GenAI to be set for $180 billion market.

Now, with such astronomical growth, let's focus on what kind of privacy aspect we should be aware proactively, so that we can make sure there are no privacy violations. Before I introduce our esteemed panel, I would like to just mention one thing: that all of these are subject matter experts, and their opinions or their remarks do not necessarily represent their employer. With that, Aura, would you like to introduce yourself?

Hello everyone, so glad to be here. And this is a first for me, talking about privacy in a movie theater, so I'm excited for that. My name is Aura, Aura Deshpande. I'm currently a senior privacy engineer at Google, Google Cloud. I work on cloud privacy and data governance. Prior to that, I was at Snapchat. I was a privacy engineer there for a few years, and learned my fundamentals there, privacy by design. And did a lot of work around designing and building PETs, privacy enhancing solutions. And prior to that, I did my PhD in cryptography from Brown University. And when I'm not doing privacy, I'm also a professional musician, so you can check out my music as well if you are interested.

How exciting. Thank you Aura. Over to you Muhammad.

Thanks. Do you hear me okay? Yes. So my name is Muhammad Tay. I'm a responsible research lead at eBay. Our team works on safety and trustworthy, so we ensure our products for our sellers and buyers are safe and trustworthy, specifically in generative AI applications. Before joining at eBay, I've worked at Bell Labs, which is a R&D team for Nokia, doing similar topics looking at responsible AI. My background though is privacy and security, ensuring that we are including human values in technology. How do we include ethical values, how do we include human values in technology, and making sure that technologies that we're building are not just because of technological advancements, but also making sure those are accessible and usable for people. I transitioned from academia to industry. I'm enjoying that very much at the moment. I'm also very new to the US. I moved here about eight months ago, so enjoying a lot of the hikes in California. Yeah, excited to be here.

Thank you Muhammad. We absolutely needed you to add the diversity factor, don't we? Nandita?

Hi everyone, I'm Nandita Rao Narla. I lead the technical privacy and governance team at DoorDash, which focuses on privacy engineering, privacy operations, product privacy, and privacy assurance. Before DoorDash, I had my own privacy tech company, was the founding team member. And prior to that, was an EY and consulting focused on privacy, data protection, and data governance. I've transitioned from cyber security to privacy over the last 12 years, so it was not a sudden pivot. My background is in computer science. I got a master's degree in cyber security from Carnegie Mellon. Currently, I'm also pursuing a master's degree in landscape architecture, so that I'm not always in front of a computer for the rest of my life.

Much needed. And I am so grateful to Nandita because she's also part of many standard bodies that define and support the next gen privacy laws and regulations. So thank you Nandita. Raji, our final panelist.

Hi everyone, my name is Raji Vanan. I'm an engineer by education, currently working at MSRC at Microsoft, and we run the end-to-end vulnerability life cycle for when vulnerabilities are reported to us all the way up to release. So if you've heard of patch days, that is us. Throughout my career, I have worn many hats: security engineer, security architect, compliance. When GDPR was a thing, that's when I did a little bit of privacy, privacy by design, data protection. So those are kind of what I have built a lot of programs and leveraging both security by design and privacy by design overall.

In the world of GenAI, as we've all seen, GenAI and ChatGPT and co-pilots around, the line between security, privacy, safety is all coming together. It's all blurring. There's a lot of connection between technology and humanity, and that is why it's really important for us to think outside the box. And it's an area I'm very passionate about, because in general I want to make the world a better place, and I think it's now we need to do this before things change. Yeah. And when I'm not doing security, I love to hike. So last year I did the Half Dome hike at Yosemite. If anybody's interested, you should try that.

Thank you. Thank you Raji. Hi everyone, some of you know me already. I'm Tr, founder of Trill. Trill is an open-source project dedicated to protecting very sensitive data like genomic data. If that gets compromised, it can compromise medical privacy and confidentiality of not only the patients but as well as all the blood relatives. And this type of data, when it gets hacked at mass, it has potential of introducing bioweapon type of attacks. So I'm really passionate about solving this problem. And prior to this, I did product security for almost 16 years at numerous different organization. So software is my thing. Software security, I can't imagine world without that. And when I'm not doing security, I love to practice, promote, and teach mindfulness. So I'm also a certified mindfulness instructor, so if anybody's burned out here and want to chat with me afterwards, more than happy to help with that.

Let's get started. Today we are going to talk about different privacy attacks, and we're not going to stop there. We are

Also going to talk about AI safety and ethical aspects. I'm especially intrigued — how do you really define ethics in this world of responsible AI? And then we're going to discuss results and insights from some of the case studies, as well as some of the incident response practices. Can we use the same incident response that we have been using since the last 20-30 years, ever since software is eating the world? Or now, with this introduction of AI, we have to change, and what needs to be done? So this is going to be our agenda, and I will shoot my first question to our very own Apurva.

So Apurva, we have seen agentic, enormous, astronomical growth in AI usage. Please tell the audience, what are some of the privacy attacks or violations that you have experienced that worry you, keep you up at night?

Yeah, as you mentioned, GenAI and LLMs are a truly exciting technology, and we are glad to be experiencing this. But we need to slow down to understand the potential privacy issues, abuses, and their potential impact, and then we can hope to address it accurately. So I'll try to lay out the land in terms of the major privacy issues here. And by privacy here, I want to focus on personal data, leakage of personal data, sensitive data. So that's privacy in the context of what I'm going to speak today.

So these are general attacks that I'm going to talk about, known in industry and sometimes spearheaded by academia. So the first setting I want to talk about is this black-box access to any LLM. That's the most common way of interacting, through prompts, and that's the way most people use these tools — ChatGPT, Gemini, Anthropic, and so on — and even all the applications that are being built on top of that. So one of the main threats that comes up is that of memorization, which means that the large language model would have trained on certain sensitive data, and then it has exactly memorized the sensitive data that it has trained on. And then, even worse, it kind of outputs that data. So that's the major threat.

And we have seen that just last year, we've seen a paper where researchers tried to put in a word — poem was the word — and you just repeat that, you just ask ChatGPT to repeat that word forever in a loop, and eventually it spits out some PII information.

Which was, so, kind of like buffer overflow?

Yeah, kind of. And who would have thought something like this would expose PII? So that's the other thing: we are very nascent in our understanding here, what can be revealed. So one of the biggest things is a reveal of sensitive data that a model is trained on. The other thing is also hallucinations — the LLM responding in an inaccurate fashion. So it can be a privacy threat because it can just say inaccurate things about people. And again, we are just seeing a lawsuit around this where, in context of GDPR, people do want the ability to correct their erroneous data, which right now is not fully supported.

So this is really interesting. I always thought hallucinations are an accuracy problem; they need to first solve the accuracy problem before even thinking about security, privacy. And Apurva brought a really good point — hey, with this inaccurate information, you can actually have a privacy lawsuit, right? Because people don't like to see inaccurate private data being leaked. Is that the case, Apurva? Can you shed a little bit more light?

Yeah, so that's the high-level context. Because if something is associated with your name in a digital forum, then that would be personal data. Whatever digital data is associated with you, you want it to be accurate. So that kind of falls into your personal data, right?

Mhm.

So that's the general thing. With a more motivated attacker, there's what is known as membership inference attacks, where you can infer if a specific person's data was in the training samples. So this is especially harmful for medical health data, because imagine a training data that trains on people who have cancer or an illness. So that's particularly sensitive.

She brought a really good point — like, how the inference attack works in the medical world is: let's say your genetic information is completely de-anonymized, somebody has removed the patient name, identifier, zip code, whatnot, but just by looking at a certain DNA marker, one can trace it back to not only that patient but the entire family. And this has happened in the FBI world. So thank you, Apurva, for sharing that example.

Yeah. And then I'll just wrap up with: if an attacker has access to actually influencing the training data, then they can do further harm, poison the training data, and so on. And as I briefly mentioned, in context of some of the regulations, what is top of mind in terms of threats is not knowing where your training data is sourced from, and not having appropriate consent and deletion mechanisms. So these are some high-level threats in the space, I would say.

Thank you, Apurva, for talking about hallucination, prompt injections, some of the privacy violations related to regulations. I would like to know, Raji, is there anything you would like to add that Apurva didn't cover?

Yeah, I would like to know, how many people remember the not-so-pleasant Taylor Swift video which came up in like Jan 2024, right? I mean, what would you call it? Is it a privacy incident? Is it a security —

Oh, maybe that's a good question. A lot of not-so-pleasant videos — do you say it's a security incident, it's a privacy incident, maybe safety?

This is what I'm talking about. That world is actually combining; it's becoming really, really blurry. And so basically, end of the day, it's an ethical violation. And these kinds of things happen because of deepfake technologies. What happens with deepfake technology, similar to hallucination right here, is it's about where you start to create bias and exasperate bias through these different technologies. And these are either present in the training data itself, or sometimes it's the creator adding it to it, to basically create those kinds of technologies.

Nand, do you have anything to add?

Sure. I think Apurva covered most of the attacks that I was thinking about. But just from a taxonomy perspective, one easy way to remember — there could be a lot of different privacy attacks possible, but to think of it in terms of either, in which phase are these attacks happening. It can happen in training phase, so all types of poisoning attacks, where training data is poisoned. And I read some recent research that even if 0.001% training data is poisoned, it would lead to a privacy harm. So this could be something very easy to do. You understand what the data sources are, you find some expired web —

Domains and poison the data, it could lead to significant privacy harms. And then at the same time, when you think about deployment phase, any type of inference attacks could be understood as an inference type attack where you are jailbreaking the existing controls that have been built in. For example, if a control has been built in that, as a helpful medical assistant who is responsible for gracefully and safely disclosing information — that's the control that's been built in. And then you jailbreak and you remove the word graceful, you remove the word helpful, and then you're pretty much having an adverse output.

Similarly, you could think about all the attacks possible from the point of what does the attacker have access to. If it has access to training data, then all sorts of poisoning attacks. If the attacker has access to the query, then all sorts of prompt injection, prompt extraction, model stealing. If the attacker has access to resources, then indirect prompt injection attacks. NIST did a very good paper on defining this taxonomy, as well as drafting what the specific strategies to address each of these adversarial privacy attacks and mitigations.

I do want to talk about: these attacks may be novel, but the harms are what we have always known about. They are the traditional privacy harms. A lot of the privacy folks in the room might be familiar with the Solove taxonomy. And this is where I get to ask the audience questions. So, in terms of GenAI, and feel free to be creative, what could be some of the reputational harms? Any answers? We data? Yeah, somebody like you, you probably have inaccurate information about some celebrity and that could cause reputational harm.

What about autonomy harms or manipulation? It could write something horrible on someone's social media. All these people realize they have — yeah, all these Instagram AI models, you don't really know. This also leads to this place of you don't really know what is AI generated anymore. So people have this sense of loss of control. You have a feeling of loss of, you feel helpless. Similarly, chilling effects: individuals know that their data is being used to train models, so it can lead to people not expressing their views, not writing blog posts anymore. We see a lot of people on LinkedIn not contributing to those collective articles.

Ever since I found out 10, 15 years back that our personal data is being used on social media for specifically targeted attacks or targeted advertising, it shut me down. So I agree with you. I'm being very careful to just express my professional opinion about security, privacy, but rest all the controversial stuff I keep it to myself, because I don't want to have a bad online social image, right? She's too left wing or right wing, or don't want to get into any controversy. So thank you so much.

One point here: interaction between privacy and also decision making. So I think privacy on its own, all these violations also adds in some decision making eventually for AI, and it can be very discriminatory eventually. So someone may not want to share that data, but that data is being shared and that data is being used to make some decisions against them. So I think that discrimination eventually and the interaction between different principles of responsible AI is also so important at the moment, when we're talking about generative AI and AI being more and more used to make decisions for people.

Yeah, so Mohammad, you have said a very good point, and I want to move towards the second question. We just heard from our esteemed panelist about prompt injection, data leakage, data protection issues, as well as the ethical and safety issues. And Nandita covered a lot of good points about: we don't have to look at the whole GenAI as a whole and come up with just black-box attack, but we have to take a look at the entire LLMOps life cycle and see throughout the life cycle what kind of different attacks are possible. And then Muhammad brought up really good points on bias and unconscious bias.

So I want to ask, maybe Nandita, you — now that we know all the privacy attacks, please enlighten us about some of the mitigation strategies, probably throughout the life cycle. When we talk about solutions, unfortunately I think some of the solutions need to come from regulations, because I feel, unfortunately, the world we are in, a lot of companies won't do anything unless there is a regulation that says so.

Taking the example of — taking a look at the audience here, how many of y'all have gone engagement ring shopping, have bought diamonds before? Okay, some here. Wow. There is no way to tell the difference between lab-grown diamonds and natural diamonds. And lab-grown diamonds came into existence since the 1950s, and the certification to identify something as a lab-grown diamond, the regulation or the certification body, didn't come in till early 2000s. So for 50 years, there was the wild wild west. You could sell lab-grown diamonds as natural diamonds at marked up prices.

I think that's what's happening for GenAI. Unless there is regulation that has very clear guidelines that it needs to be labeled, there has to be enough transparency, it needs to be explainable, and there is clear deadlines for when these regulations come about. And we have seen across the pond, as for EU AI Act, there are some requirements for transparency and accountability. There is an AI Liability Directive that's coming up. Things are happening, it's being set in motion, but it's still not clearly defined, it's not in effect yet with clear deadlines. So unless that happens, I don't think in reality a lot of change will happen. That's just my pessimistic view of things.

No, I think that's very realistic view. I don't think there is any pessimism here. So Nandita, another follow-up question, and then I want to move to Muhammad. So the question is: we saw that there was GDPR in 2017, 2018, the whole rise of GDPR, and then companies started getting GDPR fines, and then we saw a lot of different standards and regulatory frameworks all over the country, different countries, as well as here in United States. Each state started having their own privacy framework, which was pretty much inspired from GDPR — we can clearly draw the parallel. Is that same thing happening in the GenAI world as well, with the EU AI Act?

So the way — and I'm not a lawyer, but I work very closely with lawyers, so if I have not accurately represented something, that's on me — the way EU AI Act defines AI system includes generative AI very specifically, so it applies to generative AI. In general, actually GDPR also has basic principles of accountability, privacy by design, transparency, that should apply to this. It's technology agnostic, so it correctly applies to every type of technology. Baseline training and awareness, baseline privacy assessments and best practices, building a culture of privacy — those need to be in place irrespective of what specific technology we're talking about.

Yeah, makes sense. So thank you, Nandita. Muhammad, Nandita talked about the importance of following the regulations, the government standard body, learning

From them and then enforcing as we work in industry, you have an interesting mix of academia as well as industry. So what do you think — should we rely on these frameworks, or can we get a little bit more creative? Just as a point, NIST just recently released a generative AI initial draft just a couple of days ago, so if you're interested you can have a look. I think these frameworks are great. Most of them are built, I feel, for companies that are building these foundational models. So very specifically, for example, for OpenAI, Microsoft — not as specifically to industries or specific companies.

So I think they're really good to get inspirations from, but eventually how you want to adopt them and use them in your company or in your specific use case, I think that's where it gets very difficult, because you need to think about all the nuances. And generative AI is non-deterministic, so it gets very difficult to understand and realize how these frameworks and regulations can be applied in specific use cases. Which is totally understandable, because these are built for general purposes and not necessarily for specific use cases. But it needs a lot of research and development to get to the end point and eventually implement them in the company.

Cool. So the way I interpret this is, we need to be at the front line, work with actual consumers and operators, and see how we can translate these industry standard guidelines dictated by frameworks and regulation to bringing theory into practice, basically. So thank you, Muhammad. Apura, what about you? What does your experience say about this?

Yeah, I'll talk a little bit about what technical solutions exist as of now in addressing some of the threats we mentioned. And while I agree with Nandita partially on having the right regulations to keep us on track, I also feel today a lot of companies are trying to do the right thing. And it's not necessarily just out of a big heart or something, but it's also something users and customers are demanding more and more. So that's the other aspect. Privacy is top of mind for customers, for users, and companies do understand that. And I see that taken seriously as part of engineering designs and implementing products.

So for example, as we were talking about the different life cycles, the different steps in the AI life cycle and what solutions can exist in each phase. So first is your training data, or data hygiene. To avoid some of the memorization or sensitive data leakage, make sure that your training data doesn't have any sensitive data. Of course, there's some nuance here, but at least there are ways to filter out basic PII, sensitive data. So that's a low hanging fruit that you should absolutely do. On top of that, you can use differential privacy for your training, or use synthetic data. And even on top of that, you can still have test suites, because none of these techniques are 100% guaranteed. So you still need to have some testing on top of that to see where the gaps are.

Then on the inference side, there are new technologies coming up. The problem there is keeping the inference cycle isolated, whether it's for a specific customer or if users request that, for example. Then there are actually cryptographic solutions there. Using fully homomorphic encryption, you can have the entire inference flow in an encrypted way. So while that is not practical, it's maybe a north star for us right now. There is confidential compute which offers some solution in that regime.

Let me ask the audience here, how many of you are aware of fully homomorphic encryption? Wow, that's like half of the crowd. I'm impressed, guys. So tell us a little bit more, Apura, why it is not ready for industry-wide adoption. GenAI is already very resource intense, right? So tell us a little bit more.

Yeah, and just so that we are all on the same page, fully homomorphic encryption basically is a kind of encryption where you can operate on the cipher text, and you can compute on the cipher text. Basically you start with encryption of X and you output encryption of f of X. So in this case you would give your prompt and then your output is your f of X. And that's what we want to be in the fully homomorphic encryption regime. Basically, as we know, LLMs are full of billions of parameters, and imagine having to do that entire inference in an encrypted fashion. So that already is a big blow-up. Right now, I think the latest — there's a startup I know, Mithril, which works on this; there are a couple more. Right now the inference, they have been able to get it down to hours, so it's not yet in minutes. But they're working on the hardware level, so we could see some great things in the next few years.

Well, let's keep our fingers crossed and let's hope this technology becomes functional and it can honor the latency-sensitive applications and whatnot.

Now I want to move the audience's attention to a totally different segment. Just the way 12 years or 15 years back, we did not have data privacy officer or privacy engineer type of job, but we have them now, right? Similarly, I'm hoping to see Chief Ethical AI Officer. We are already seeing ML Security Officer. So I really want to know from Muhammad and later on Raji here — Muhammad, what exactly is responsible AI, and how do you even define ethics here? Because ethics is such a subjective term. Something that is ethical for this country may not be ethical for that country, right? So when you join an organization, how do you really set up something like this, a responsible AI program?

Yeah, that's a very good question. It's very much like privacy; it's very contextual, and you can't really define very objectively and say what it means to have ethical AI in California, or even a different state, or a different country. It's a very cross-functional kind of team, and it's very important to have diversity in these teams, bringing different perspectives to the table, making sure that different people are bringing their perspectives, both from different ages and different axes — let's say different ages, different ethnicities, different functional areas. Making sure these are all included when we're building that responsible AI program.

And I can see in the future having Chief AI Officers more and more to enable that cross-functional team within the companies, and also having Chief Ethics Officer, or Chief Responsible AI Officer, or Office of Responsible AI — there are different names for this — to enable that functionality within companies. And those teams are going to bring different teams from privacy, from UX, from design, trying to have all the different perspectives. And eventually they can decide together what it means to do ethics for that specific

Product or for that specific company, I don't think there is a silver bullet solution that we can just propose to everyone and everyone can follow.

So you're saying there is no government or regulatory framework that defines ethics for us?

I'm not sure, but yeah, I can see having these really general policies that are coming in the future, what Nandita was saying. But I don't think they will be very specific. As I said before too, I don't think there are going to be very specific to industries or specific to products. I think at the end of the day, there are so many trade-offs and balances that need to be made by discussion and talking to different people.

So maybe, audience, if you're bored of doing security privacy engineering and need newer challenges, maybe you can consider these newer roles, right? Variety is the spice of life, right?

Right, we have to keep things spicy and interesting. We discussed quite a bit about the proactive approaches, what we learned from the privacy attacks and violations. And Nandita was nice enough to explain to us, during different life cycles, how we have to worry about different attacks. And Apurva and Muhammad gave us a really nice perspective on how these can be avoided.

I want to ask Raji: this is a proactive world, but anything can be a weakest link in security, just anything. And we live and breathe in the world of security events, security incidents, and security breaches. So if something like that happens in the world of Gen AI, what are your thoughts on compacting it?

It's a great question. I think everybody should always have a plan for not if, but when a breach or security incident will happen, right? And then again, we talked about how all of these are coming together, so it's very difficult to say this is a specific security incident or a specific privacy incident. So we need to think differently here.

The way I think about Gen AI — I'll just throw this out here that credit goes to Mustafa Suleyman, who is the Microsoft AI CEO, who basically called Gen AI to be a digital species. That is what we are bringing into this world. He said AI is a digital species. So with that in mind, right, and it's non-deterministic, so you need to approach Gen AI in two aspects of it. One is as a software, and I think we all know here about software and how do we attack, how do we manage attacks in software. But there's another aspect of this Gen AI as a person, right? You give it a question, they give you an answer.

So you really have to think about all these things we did when we had that security education and privacy education and try to do that with the AI model, right? Basically saying, hey AI model, now you are a security expert. So then the kind of conversations you have with that model is going to be a lot more different. So it's bringing that conversational aspect of human nature to how you plan your incidents. Obviously keeping in mind that you have detection strategies in place, you're also thinking about managing the incident and responding, but also really taking those lessons learned and trying to bring it back.

Because the thing with generative AI is that it's not about prevention anymore, it is truly mitigation. Because you never know, and it's defense in depth. I feel like security and privacy, we kept saying defense in depth all our lives, right? But now it's truly defense in depth, because it's about trying to avoid this attack and this attack and this attack, because you don't know how the model is going to respond. And throughout the panel, we talked about all these different types of attacks and mitigation, but all of you should remember, it is not stop — we are not preventing yet, we are only mitigating at this point.

Makes sense. So we have about five, six minutes left, and I want to open the floor for the audience to ask us questions. Yes sir?

Okay, I know a lot of companies are trying to use AI, Gen AI, to do things better. But there's a risk that there's always something new. Simply stabilize and say, using this LLM, these agents, or whatever, and not chase the new thing, because when you do that, you incur risk. So I was curious about the panel's thoughts on, not if that's important, how important it is, thoughts on how to resolve it?

I can take that. So the question really is, companies are adopting AI for making themselves better in certain things, right? But should they? Because a lot of new things are happening in the Gen AI space, do we really need to wait? What's the risk introduced when you jump into the next new technology? That's what is asked.

Well, the way I'm going to answer this is, I think it's really important to think about this complexity of humanity and the data, right? So what we need to do is really think about the context of how this application can be used, and also think about the way the application can be misused. So keeping that framework in mind when you're trying to go and adopt a new AI technology is really important. And also really understanding, right, a lot of times when you think about, oh, the risk prior to Gen AI being in the picture, it used to be like, oh, somebody's life is not at risk. But right now, somebody's life can be at risk. So that is the huge difference. The risk is higher.

Anybody else want to comment? Or maybe we can take the second question. Are you happy with — did that answer your question? And folks, if we are not able to answer your question because of the time limits, we will be available to all of you just outside this, and feel free to connect with us on LinkedIn, and we are more than happy to share our insights and learnings.

So the one question I'm going to take from online is, maybe this is something Nandita and Muhammad can tell us: given that the majority of organizations are not building their own LLM-based models, how do we manage privacy violations within a base model? Great question. Tarka, take it.

Oh yeah, I knew this was going to come in general. And I can speak from experience. Without going very deep into the rubric of how to evaluate risk, and what the measures are, how do you evaluate a certain harm or a consequence, I would say starting with a very broad framework like the NIST AI Risk Management Framework is probably a good idea, because they are general enough and it's not specific to privacy, which is one myth which a lot of people think about. It includes a lot of security, transparency, explainability, safety, everything else that you can think of. So using a general framework is probably a good idea because there are so many unknowns at this point. So when you're evaluating Gen AI models, using a high-level framework like that may be good enough. And this is always going to be an iterative process. You don't know what you don't know.

Maybe you come back and then your thresholds change next year.

All right, so folks, we have less than 2 minutes remaining, and I want to create a little bit of a thrilling moment for our panelists. So in less than 20 seconds, please tell — each one of you, and we'll start with Apura — what's the next thing we should do, must do, in the direction of responsible AI?

Yeah, I would say one thing to focus on would be your data hygiene. That would be my call to action for everyone. Plan — in fact, I would go ahead and say plan for not using raw data at all for your training. And that might come up in regulations or not. Irrespective, that's something to invest in. That would be my call to action. Think of how you want to sanitize your data, how you want to use privacy preserving methods, and how you want to have your testing, red teaming around it.

Perfect. So Apura is all about data hygiene, PETs, and red teaming. What about you, Muhammad?

I'm all into diversity and having teams with diverse perspectives, having people — yeah, just bringing that in and making sure we are not working in silos.

Perfect. Nandita?

I think the first step is to get executive buy-in to have a responsible AI program. Nothing can work without getting the leadership involved. So the next step is to start there.

Well said. Raji?

I want to leave you all with a word: data is radioactive gold. So really think about it as — and credit goes to OWASP for that — but really think about that. So it's gold because that's going to solve a lot of our problems, right? Marketing, security, privacy, everything. Data is going to solve it for us. But it is radioactive. It's going to come and bite you. So make sure that you really understand how you use it.

Thank you, Raji. Thank you very much everybody on this esteemed panel: Apura, Muhammad, Nandita, Raji. And thank you, my dear audience. You really made this session super special. So let's have a big round of applause for everybody. Thank you. Thank you all.