← All talks

BSidesSF 2026 - From Auditions to Opening Night: Selecting Security Tools that... (Saurabh Sharma)

BSidesSF39:2410 viewsPublished 2026-05Watch on YouTube ↗
About this talk
From Auditions to Opening Night: Selecting Security Tools that hits the high notes Saurabh Sharma Vendors put on dazzling demos promising to solve your problems, while your engineering team gives the performance a thumbs down before the first act is over. But what if you could orchestrate security tool selection like a Broadway production from casting calls through opening night to rave reviews? https://bsidessf2026.sched.com/event/1bc443065174ec0f8b3b257bed9cfc6b
Show transcript [en]

So, uh the next talk, I think is leaning heavily into the theme of this edition of B-Sides. I think you can't uh you can't miss it. So, as they say, the great Broadway show starts even before, much before the opening night with the right cast. And I think this talk is going to be a perfect example of how to select the right cast or well, the security tools to ensure perfect uh performance and perfect opening night. So, please give a round of applause for our next speaker, Saurabh Sharma. You can get started. Hello. Hello, everyone. Thanks for being here. First of all, I would like to express my gratitude towards all the B-Sides volunteers and organizers

because I've been part of these volunteering groups and security conferences, and it's hard job. And it's kind of thankless as well. So, thanks a lot for making this happen. So, I'm I'm Saurabh Sharma. I'm a founding security engineer at Lime. And just like a lot of you guys here, I spent most of my time doing things to make my company more secure, but with not enough people, not enough time, and a lot of vendor emails in my inbox. So, we all all like to talk about the cool technical stuff at conferences, like the new attack techniques, zero-day attacks, fancy detection logics. Uh and that stuff that definitely matters, but honestly, a huge chunk of my job isn't finding zero-day exploits

or writing detections. It's about convincing people that whatever I'm doing to secure my company uh is is something that will work. And I I want to convince the other people to use it, as well as to show the value that I've whatever I've created. So, here's the thing. I would like to talk about something that doesn't get a lot of space-time on security conferences uh about how to pick and deploy security tools successfully without losing your mind or your credibility or both. So, on a on a personal side, I've been working in the security industry for 12 years now. And in my free time outside security, I like to do outdoorsy things. Anything that can get me on top of a

mountain and like finding different ways to come down of it without dying. I like those things. So, I'm not saying that the outdoorsy people are better security engineers, but you get to develop tenacity, which is very valuable security skill in an engineer. Um So, moving on to my talk. So, I think we have all been in this position where we see like a very flashy vendor demo and we get excited about it. And then we think that okay, let's get back to our team, start the pilot. And within 2 weeks, we realize that the engineers are complaining about the false positives. Your security team is is drowning in like changing making changes to configuration of the tool

without and not doing actual security work. And the leadership is behind you asking you questions as to why did we approve a six-digit contract and the people are not using it also. So, I call this the security tools tragedy. And today I would like to walk through some of the things that I have we have done and how can you guys avoid it in in future. So, I'll talk about two of my experiences. These are not polished case studies that you find on vendor websites. These are actual problems that we have faced while deploying tools and what are the things that we learned while while doing these things. The first example is from a runtime

security tool and the second one is a supply chain security tool that we that we got on board. So, I'm going to tell you what what worked, what didn't, and if I'm doing it again, how will I do it differently? So, by the end of this, you should be having a framework that you can use in in your in your own company. And not like a document which stays in your Google Docs. So, let's get into it. So, just like you that you can think of in terms of starting a movie, you always kind of write the script before you decide the casting. So, similar to that, before you talk to a single vendor or seeing a single demo, you should be

aware of what exactly is the problem that you're trying to solve. So, you do see some of the controls listed down in in the examples, these actual controls are not important. The thing that you should take from here is to actually align the business outcomes to the problem that you're trying to solve. And this can be anything. For example, if your company is planning to go in Europe, then you need to find out what are the gaps that you that you have in your organization in GDPR. And basically use those framework gaps to align on the problem with all your stakeholders. And after you get a good handle on what exactly is missing and the problem that

you're trying to solve, that's when you that's when you try to think about, "Okay, this is the kind of tools that I want or the vendors that I would like to interact to solve this problem." So, basically this is kind of a past performance. I know it's not very high, but if you see from lens of founding security engineer, there was like no security engineers working in the company. So, you can see like there are around 6,000 odd vulnerabilities and they're climbing. And we saw that the total number critical and high vulnerabilities that were mitigated from a supply chain point of view was only 13. So, we kind of effectively had um 13 out of 1,000 vulnerabilities that

were fixed in in the whole calendar year. So, it it it is kind of a little embarrassing, but you have to also think in terms of a a fast-growing startup where these kind of things get overlooked very frequently. Um so, the point that we want to make or the problem that we want to show is moving from a reactive state of security uh to a proactive one where we're already aware of the problems and we're stopping them before they reach production. So, this is one of the examples uh of our repositories that existed. So, the thing to note here is none of the repositories had zero alerts or zero critical and high alerts. Everyone of of

them have had a few. So, it's not a problem that exists in a certain tech stack. It was a problem that existed in the whole company. So, the problem that you can see and screaming is the ever-growing number of vulnerabilities that exist in your software supply chain. And as as a defender, how exactly would you get ahead of it? So uh in part of like composing a pitch uh you have already done your homework in identifying the framework uh the framework gaps that that are aligned to the business outcome. You have good set of baseline numbers to show that the problem exists in your organization. Uh now, the hard part is to get other people to care about it, uh which I have

found uh very interesting because we all here uh in the security community, we have great understanding of risk, but we are terrible at explaining uh risk uh to the people who don't think about it the whole day. So, for example, your CISO might think about risk totally differently than your VP of engineering. Um and VP of engineering will think it totally differently from a developer who just wants to ship their own feature and go home. So, over time, I've landed on a structure that works for me, and I think of it as writing a song. Uh you need a hook, a chorus, a bridge, and a finale. So, the hook is nothing but uh it's

condensing the problem in a single sentence, and we never talk about the solution here. Uh we are always talking about the problem. The goal that we have here is to get the people lean forward uh and understand the problem. So, for example, uh for our supply chain security vendor pitch, we the the hook was we are resolving 13 critical vulnerabilities in a calendar year out of thousands, and our mean time to remediate is close to a calendar year. So, that's it. One sentence. Everybody in the room understood the problem. And the chorus basically is the data that backs up your hook. So, this is where you pull out the numbers from your current state and baseline.

And the open alerts over time will always show you that it's trending up and towards the right. So, the goal here for from a security engineer point of view is to repeat the pain until it sticks. Um and the hook and the Sorry, the chorus should be able to do that. The bridge is basically the thing that connects problem to the solution. So, if you note that the solution part is referenced first in the third kind of phase, and we're only talking about the problem initially. Uh so, this is where you connect how exactly the tool is actually solving the problem. Uh and you have to be mindful that you do not explain features. You are

actually explaining the outcomes that people care about. For example, nobody cares about reachability analysis and engineering. All they care about is if I have 200 100 BTs and basically this tool can help me find out the five which are exploitable. So, that is the value that that this outcome offers and more or less you need the bridge has to cater to this. Uh the finale is mostly your success criteria. It's almost binding you to a pseudo contract where you as a team hold yourself accountable where you try to prove whatever hypothesis you had while buying or thinking about buying a particular product and making sure that the success success criteria is trying to manage that and you you almost like feel

like accountable towards it because you've defined it even before looking into any vendor to solve the problem.

Okay, so two examples that I'll share is the runtime security and the supply chain uh security tools. So, I'll I'll dive into the actual pitch that was made for both of them. So, in for the runtime security, the problem that we wanted to fix is the environmental drift that existed. So, as a security engineer, you can propose solutions or have guardrails or have mitigations and you can make it work in staging, but if the environment is not same as production, then you cannot say with confidence that it will be working in production or it won't break production. So, that was the first problem that we pointed out. Uh the chorus was all the different alerts that existed in our

organization and the lack of runtime context we had into identifying which ones of them are actually exploitable in our current setting. Uh so, the chorus basically is uh blind spot data. So, we had we had no way to tell whether the known vulnerability is exploitable, and we were prioritizing based on CVSS scores, which anybody who's sitting in the room and has worked in security for more than a year uh can can say that it's like basically like horoscopes for vulnerabilities. So, the critical doesn't mean exploitable, and exploitable doesn't means exploited. So, without the signal of knowing whether it's exploited or not, uh we were actually flying blind. So, this is what the the chorus tells

you that we are trying to fix this specific problem and the data behind it. The bridge was basically the reachability analysis. So, out of our number of critical and high vulnerabilities, the number of actual exploited vulnerabilities are less than 5%. In some cases, it might be even like 1 or 2%. But, uh five is like the buffer that you can take. So, this is like the kind of bridge that shows that, "Okay, with the tools and the features that exist, the outcome that we will we would require is the re- reduction of the manual workload of triaging individual vulnerabilities." Uh second one was the cloud security best practices. So, we did not did not have any way to feature

gate security related uh best practices. So, this tool also provided us that signal, that something which works in in staging will also work in production. Uh the finale is the actual real-time workload monitoring uh for zero days. And uh this is what we wanted to prove after the pilot. So, the pitch got approved. It's not like I'm some incredible sales person. I'm almost as equivalent as you guys. Uh but, I let the data do the hard work for me. And the second second what pitch was for supply chain security tool. So, the problem here was totally different. Uh we wanted to make a culture shift where we move from a reactive state to a productive state.

And the data that we want that we showed was uh 325-day mean time to resolution for a vulnerability. And the bridge was basically mapping the SLSA threats uh and blocking some type of squatting on malicious packages even before they reached uh production. Uh and the developer workflow was the part that actually sold engineering on it. Instead of asking the developers to do uh and go to a different tool to triage and see a wiki page before they think about adding a new open source library to their to their application. We basically reached where they were. So, we provided feedback on the PR level. There was a browser extension which which was provided so that that the

engineers can see the packages and know before even putting the code in production in the PR that this is going to fail uh once it is uh it's added to the PR. And the finale was basically the one-day less than one-day detection time that we wanted and almost less than one week remediation time that we wanted. Um so, pointing this out in the beginning helps us to have a pilot or or look into tools that help us with this instead of the other way around where we are looking into a solution that's looking for a problem. So, handling the tough crowd. So, I think every pitch gets uh a lot of hard questions and honestly, I feel very

happy about it. I think uh if nobody pushes back, that means that either they are not paying any attention or they have already decided to say no and just being polite about it. So, in in general, you get a lot of questions, but the three questions I always get is uh will this break production? So, they're not looking for a a definite answer from you, but they are actually looking for how much thought did you put in to identify if you can determine if it's going to break production or not. So, they are looking for the ways that you will experiment and get data to prove that this won't break production. And this is something that

keeps the platform team under um check. Then, second question is what's the burden of engineering? So, everybody who works in engineering wants one thing is that it should not add more work to whatever they're doing. It should basically be just providing value where they are and make the decision-making easier for them to make security conscious decisions. Third one is how different is it from the what we already have. So, that this is one of the questions that I've always seen where there is some form of the product already existing in the company, but you have to paint a picture of what exactly is that current tool lacking. And for our example, we did have a tool

for supply chain, but the way it detected the vulnerabilities was checking a static vulnerability database and presenting the vulnerabilities after the fact it has already explo- already been exploited. So, we were trying to solve a totally different problem by making it proactive and kind of things that the new tool that we were proposing almost offered the value that did not even exist in a company. So, it was a very easy decision for us to explain. Yep. So, this is the product which is I think the most important where majority of the security deployments fail, which is the pilot. So, my approach is always to time box it. I have found 30 to 60 days as the sweet

spot for a pilot or a POC. So, the number is not important. The number might change depending on your organizations flexibility and how they work. It might be more or it might be less, but the point you have to note is it should not be too less to have any meaningful data in your pilot. And it should not be too long so that people just lose momentum and forget what they were trying to test in the first place. So, initially I would always recommend have an opt-in pilot where the teams are opting in to be part of the pilot. The The reason why I say this is because whenever we have a mandatory pilot situation, it's always been a case where

the team does not even want to be a part of the pilot and they do the bare minimum and do not offer you the honest feedback that you need to make the product successful once it is in production and also impacts our decision-making if you don't get the right amount of data. So, I would recommend pick two or three teams. The team selection is not very important, but the thing that you should keep in mind is they should have different tech stacks, different deployments. Uh and you're you should write the success criteria from day one even before the pilot starts. So, I've been part of so many pilots where after 2 months people ask the

question, "So, did it work?" And nobody has an answer to it because we never defined what success meant. So, there's everybody's looking at each other to figure out who's going to answer the question, but nobody has the answer. So, what exactly are you measuring? I always bucket this measurement into two separate buckets. The first one is the technical part and the metrics that show that your solution is actually solving the problem that you set out to solve. Uh these are very obvious ones. For examples like how What is the false positive rate? How long did the detection take? How much is the remediation time shortened after the after this tool? Uh the next one, which is the most

important, is the adoption metrics, uh which people forget a lot. So, it can be both qualitative and quantitative. Um so, some of the examples are are developers actually looking at the alerts? Are they acting on them? Did anyone turn turn them off? How often do they reach out for help in using the tool? Uh this last one is my favorite because it tells me how intuitive the tool is and how much of the hand-holding is required uh once this gets in production.

So, this is in the pilot stage, this is uh the command center, I would say this is something which the security team uh gets a lot of value out of. So, this is a centralized UI that you could see all the repositories that were protected, how many PRs were analyzed, how many anomalies were detected. Uh developers don't live here, this is for only for security. And we track this sort of metrics in both of our pilots. Uh so, on the technical side, this is an example from our supply chain pilot, where we enabled it on two repositories, and it shows the number of PRs analyzed and the commits analyzed. And the thing to see here is there's

only Sorry. Yep. So, the thing the thing to see here is there's only one only one commit that was alerted. So, I would say this is actually very positive, since it shows that uh it's not creating any noise, and the alerts or the blocks are actually things that make sense. Uh on the adoption side, we watched whether developers were clicking on the reports of the dependencies, they were how deeply were they checking it. And they also told us that integration itself was working. Second second part is the actual feedback loop that you need in front of the developer. So, this is a huge one. So, any developer whenever they make a change in a dependency,

uh a bot commented on this PR uh stating what changed. What are the new capabilities introduced? What is the security score now of the new dependency that was added? Uh and now we are banking on two people. First is the PR author and second is a PR reviewer. So, ideally we would like two people to make a judicious decision uh that helps the helps them to identify risk and get the signals whether they should be accepting the risk or they should actually fix it even before this uh moves to production. Third one is like the thing that is the extension where we are providing this sort of knowledge as a browser extension because before before AI people used to

just go and check the internet to see what kind of open source libraries help the the problem that they're trying to solve. And if you get to know this feedback right when you're searching for it, you should ideally not be putting it in in the code or into us because you know that it's this is a dependency that will fail the PR. So, this basically moves a little more left uh the security that we are trying to do. And the form factor of this browser extension offers a lot of value uh to us in terms of moving it even more left for the developers. So, even after doing all of these things, it's not going to be a flawless

performance. There's a lot of things that will go off script. So, the reason why we want to share these with you because so that you make some new kind of mistakes and not the same ones. Uh so, the first one I would like to point out is the configuration mistake where the security team uh had such stringent policies which are so aggressive out of the gate. So, it created so much false alarms and the developers started ignoring the alerts. And once they start ignoring your tool, it's incredibly hard to get that trust back. So, always start with permissive policies and tighten it over time once you realize how the tool works and you have a sufficient amount

of confidence that if if if it's if it's blocking a PR, it means that there's something valuable in it. Second is the integration challenges which every tool has. So, in in the vendor documentation, this they make it look so easy. Uh but you should always budget extra time for integrations because every single team uses their own kind of tech stack. So, it doesn't matter if it's working in a single team, you might have to fiddle a lot with the integration to make it work for other teams. Third third mistake was the set it and forget it trap where we launched a pilot and we checked in after a week. Then we forgot about it for 3 weeks because of

other priorities. So, just don't do that. You'll need someone, specifically somebody from the security team to be basically babysit this pilot, collect feedback weekly, adjust the settings, fine-tune the configurations that block certain things in in from the tool, and get get those feedback directly from the pilot teams. Fourth one is basically the part where you show that okay, this all the measurements and the success metrics that you mentioned is actually doing the job it should be doing. Uh so, now the pilot is done, everybody is looking at you and asking whether it's a go or a no-go decision. So, I would like to challenge everyone here to basically think about success metrics for security differently.

So, the old way of measuring security tool success is like counting the CV tickets and seeing how many vulnerabilities it it found. Uh but that number itself is is sort of meaningless uh if it's if it's not enabling people to fix the problems that you're finding. So, the metrics that we actually cared about is how long did it does it take to detect a a vulnerability that we want to see and how quickly does it enable us to remediate it uh after we detect the problem. Uh this is the the third metric is basically the most important of developer adoption because I've been part of multiple uh tools and security products uh which no matter how fancy they were, they were

worthless if nobody's using it. So, coming coming to the results basically it's it was pretty easy in these two cases. It might not be that easy because uh the current state that we had was almost like zero. So, we did not have a detection of environment drift uh that existed in our environment before. Uh so, we were already starting at zero and after the deployment our detection time dropped by in within a day. Uh and that's an infinite improvement if you if you're into math jokes. Uh but the real win was confidence. Basically, the fidelity that we we see in those in these alerts is is what helps us and we were comfortably enforcing security uh best practices

without being in the fear that okay, we're going to be paged in the middle of the night at all. So, I would also mention some of the alternate benefits that we have had. So b- I call them alternate because we did did not set them out initially in the pitch. But the other benefits we got is enabling certain security best practices in our environment uh more confidently and also tracking some PII data uh which we were being asked a lot to do, but we were not able to do it, so this tool allowed us. So, it's definitely good to point out the added benefits that we didn't think about, but we were somehow given those benefits as well by

using the tool. For the supply chain security one, again the baseline was so bad. Uh plus, we did not detect any any zero days in our supply chain. So, we would never have an idea of any malicious packages or type of squatting kind of supply chain attacks that existed. But after the tool, we were able to do this in a day of detection time plus a week in within a week of remediation time. Uh the actual show stopper for us was this detection of unauthorized code and third-party anomalies that were detected before merging it into production because before this week could not even do it after uh even after we went into production. So, the other set of metrics that are

really important is the adoption rates because adoption rates uh actually matter more than any kind of feature list that you have. So, I have used or seen multiple incredible tools with feature lists that nobody in the team could even figure out. And I've also used tools that only did one thing, but very consistently and properly. So, the supply chain tool uh won because it reduced the manual security triaging workload on both developers and security team. So, instead of asking them to do the right thing and going checking a documentation on on the wiki and evaluate what kind of libraries they should be using from a security point of view, we just added that feedback on

where they were, which is on the PR level in the browser and as a CI output. And we also did not fail to measure qualitative feedback. Uh so, we checked whether people complained about the tool. Did anybody ask to turn it off? Were there any Slack messages about it being annoying? Uh and these are kind of real signals. Uh you will never find data about it, but it's a qualitative feedback that your team sees day in day out. So, I think this is very valuable and you should not leave it out of your analysis. Uh and one more thing is uh after 60 days of pilot data, if you think that the tool isn't working or the

engineers are totally hating it, and the false positive rate are so high that it just takes out all the value that you thought it it would provide. So, you should just be willing to close the pilot. Uh I don't think this is like a failure, but it's a success in your evaluation. Uh the failure would be to actually go ahead and like waste another another year on the tool which you have already determined which it does not work. So, how how exactly would you make this go no-go decision? So, what I look at three things. First is the technical metrics, uh which shows me whether it's working and solving the problem that was initially mentioned in the pitch.

Uh did the adoption numbers held steady or improved during the pilot? And do I have at least one team uh which says that we cannot be able to live without this tool. We we need it so badly. Uh the third one has always been uh my favorite because that gives me the litmus test of whether it's working or not and somebody likes it to a level where they cannot see a life without this. Then coming to the integration bloopers, again I promised in the beginning that we would like to show like the things that failed in in our deployments and in in in thinking about like basically you guys should not be making the same mistakes.

So, the first one is basically we underestimated the integration effort that took to uh integrate this tool to be in a state to provide value consistently. This is not because of there was any problem with the tool, but because every team had their own slightly different development pipeline configurations, their own quirks. They will have their own Oh yeah, we have a custom step that does X for us. So what what worked for one team does not necessarily work for the other teams. So if I did it again, I would actually keep around double or triple time for integration at least. Then also we did not assign a clear owner early enough. So for the first few

weeks the deployment was everybody's responsibility. And when that happens it basically means that it's nobody's responsibility. So the moment we had like a shared responsibility model, things started improving. And probably the most embarrassing one would be that we spent too much time configuring the tool to work in staging environment. And the staging environment is usually clean. There is no production traffic production level traffic in that staging environment. So I would just recommend testing it in your own production environment which would have years of accumulated weirdness. So ideally the tool should be able to work in your in your own mess instead of you trying to optimize it in a in a demo environment or a lower environment for

you to understand that it's going to fail in production. Now, how do we sustain this performance after the pilot is finished? So assuming we are moving forward, I would recommend against doing a big bang kind of roll out. So it's very enticing for anybody particularly in our leadership to show the numbers. And as soon as you do the big bang roll out, you see the coverage number going from like zero to 100 very quickly. But I have I have watched enough big bang rollouts fail and create so much noise and disruption in teams which causes a revolt and actually you end up in a worse position than you were in in from in the beginning.

So, I would always recommend go team by team. Go with the teams that have a lot of credibility for you and the kind of relationship that you formed. Start with the teams in your pilot. They're already familiar with it. Then expand it to the teams which have sort of the similar tech stack as the teams in the pilot that you had. And then you go to the rest of the org. And each phase should be roughly around 2 weeks. And you should have a a tight feedback loop after every phase so that you have some learnings that improves every next phase that you have. Then every every company will have some skeptics. Every org will have some skeptics. So, the

best way I found to deal with them is I make the early adopters the sales people. So, when a team A tells team B that this works brilliantly, they're much more willing to listen to them instead of security telling them that this is amazing. Then to keep the show running, I would also say that it's security tools are not deployed and done. You need constant monitoring, constant tuning of policies, your environment, the changes that happen in your production. New teams will get on boarded. You will have to train them. And tune your tools to work in the in the environment as it evolves. Always you should be maintaining a very tight feedback loop with the vendor.

Tell them what's working and what's not working. The the vendor that's that we have interacted with, they were very responsive to our feedback. And some of the suggestions that we made are also part of their roadmaps. So, this this kind of partnership is worth way more than just writing checks and forgetting what what and hoping for the best. So, and also I would ask everybody to keep an eye on the markets based on the current trajectory that tech is in. I think everything that we use is like kind of old in 12 to 18 months. So, instead of doing like a full-blown assessment, just do a smaller version of this every 12 to 18 months to identify

whether the problems have evolved or the solutions have evolved and whether you should be looking into something new to solve those problems. So, basically I would like to wrap up with certain things that I hope it sticks with all of you. So, first thing is always start with the business outcomes and align them to the problems that you that you want to fix instead of going vendor first or product first in your in this journey. So, map your gaps before you talk to a vendor or before you have a meeting. Otherwise, shopping without a list always ends up miserably. So, build proposals that connect the pain to the outcomes. The baseline data that you have to show

the current state is your best weapon. So, show the problems with numbers. Show the solution in terms of the business outcomes it helps you try to achieve. And define the success even before you think about spending a single dollar. Uh then measure what actually matters. Uh the number of vulnerabilities don't really count. The thing that counts is how effectively are you reducing the risk that exist. So, always go and go hard on the actual measurable impact that helps you reduce the risk in your organization. So, I've got some templates that that follows evaluation framework and structure of proposal. So, I can share that if if anybody needs it. Other than that, thanks a lot everybody

for attending the talk. Thank you so much. It was a great presentation. We have a lot of questions. So, the first one to get started as do you use the same tool selection process for big and small purchases? I would do yes, for big and small both. The trigger might change for big or small. Some of the things like the success criteria or the or the size of the pilot might change, but usually the process should stay the same. Of focusing on the data and let the data do the talking for you. Okay. And how many pilots do you do per year? I would say around four to five. Both for small and big or do the size

>> big ones and small ones might be around around five or six. Okay. And how do you get leadership buying to shield from your from changing priorities so you can stay focused on the pilot? >> thing that you want on board is your own leadership. Before you get alignment from your own leadership, I don't think you should be even looking into any product or anything. Like first thing is aligning on the problem with your leadership. And once you're done with that, then you expand your scope and try to get alignment from the other stakeholders that are going to be impacted. And how do you decide whether to implement a tool yourself versus buy a or get professional services to

implement it? So, this is a very good question. Like this is a struggle that everybody faces to decide a build versus buy kind of solution. So, in in current current day and age if I think that something that is very easily achievable by building our own custom solution, then definitely the needle is always more inclined towards the build part. Uh especially with the growth that we have seen with AI. Um but usually we would like to differentiate between how much time is the team willing to spend on a particular problem. And what are the kind of benefits we have of developing something custom in-house versus buying something from a vendor. And the last question, so you recommend

writing failure criteria as well as in if this happens, the product is automatically dropped from consideration. Yes, I would say it's it's like opposite of success criteria, but I would also this is a good way to add a failure criteria as well where we are completely like, "Okay, if this happens, then we are just stopping the pilot ASAP." So, that's another angle that is is pretty beneficial to have. Okay, great. Well, thank you so much, Saurabh. It was a great presentation. Please give him a round of applause. Another round of applause.

[ feedback ]