BSides Denver 2024

from International regulators and stop your entire investigation so two weeks ago and I was just in tears with this exact situation hot tell going I saw things that I don't totly have confirmation yet we Elevate this to somebody outside inter Outside Agency could completely [ __ ] up the investigation and then more lives are that [ __ ] sucks the no like be right it's it's not something you want and you think okay these are here to protect people when they're not and you're the one absorbing that it's not um so there's there is a personal element to that that I take because I do want the regulations to be there to help with guidance and allow

people to do the right things I I want them to be a carrot not a stick but to often we have Regulators that are also using enforcement actions that don't use it as a can so things like the SEC rules that are purely focus on the financial regulations they have nothing to do with data privacy they have nothing to do with really protecting the consumers it's about protecting the stock owners now there's different ways to use that and unfortunately they have targeted even postmortem of that certain cesos and individuals help right the weest brief from that say this is [ __ ] but you know it's it's something that could be used for good or bad just all the

dose makes poison additionally United States we're going to have a lot of Regulation by litigation which can be good and bad and it's all about how these things are going to be effective effective to the populations are protecting fortunate from the situation now are folks that they're never going to have a credit card they're never going to have a credit score they're never going to own a goddamn thing because they're areas of the world that their identity is their currency their personal well-being is their currency if there can be traffic put into slave labor or anything like that and that type of information to be used against if you go back to why we have things

like gdpr was because of things like um you know Bing post postal warer the Geneva conventions it wasn't tracking people because we're were worried about financial effects because it's tracking people because bad people do bad things when they can aggregate data about them so it has to be thought out in a much more expansive way than we do it now so we look at data privacy data breach from all these context is what data do you have who has access to it how long are you storing it has a much bigger consequence and when the regulations come in that are disjointed with that that it's all [ __ ] so it can be used but there needs to be better

participation with the community to help drive those conversations with The Regulators to help do things enforcement wise that doesn't [ __ ] us all the time thanks for bringing all the Happy Feelings oh I forgot to tell you by the way I just found out so when they do now the new manware threat guys it's uh it's it's pop prop and SWAT so they're actually placing uh a cand material so child protected data on networks um so you can shut down entire networks in people's political careers short stocks um destroy people's lives I'll be at the bar after this so I have one of those cases now is that somewhere uh not [Music] yet knock knock open up FBI any cheer

thoughts I know I just yeah you're a big ray of sunshine over here let's switch subjects I I War people if I'm in your office there's nothing good it's like R down M shap the wolf like [ __ ] up so we're gonna switch to something a little more hopefully optimistic answer this no I don't future Technologies okay so are you currently using any tooling that incorporates artificial intelligence or machine learning in your environment and I think it's important to also talk about your industries respectively how you're using them within your industry yes uh I if anyone says no they are lying right I think um AI is a horrible term because it's not actually artificial

intelligence um but you know people have been doing machine learning and other similar techniques for many many years and it's been built into all of the products that we use for many many years so it it's nothing new um if if we're talking about specifically about generative AI tools um yeah I think we have we've done some interesting work at at my particular organization um trying to use some of those tools uh on the security team to our advantage doing things like uh building tabletop exercises helping us do uh third party vendor reviews some other things like that so uh I think that there's there's great promise in leveraging it for for good things I think Rob was saying it's

going to eliminate a lot of sort of that base level uh work that people hate the monotonous tedious uh things that that entry level jobs have that that are probably the worst part about it um and so yeah I think it's here to stay I think it's uh overall going to be a good thing I personally like the machine learning thought as well I mean anyone that's lean manufacturing my company manufactures variety of different products uh that's been around forever but I think we're seeing it probably more prominent to do work like things of contract reviews I'm not talking to specifically security but how can we get speed to Market quicker on things that have traditionally taken a long time

it can be automated they can also help the small security team do more with less lesl are struggling for resources um and then there's just the things that the average uh you want to create a quick customer satisfaction survey if you're on a sales team why spend hours thinking about that when you can just do one quickly that someone's already helped you with I think there's a lot of great uses for generating that type uh there could be some misuses for it as well we got to solve for what we do with our the data critical data and where do we put that but I think overall everyone is using it somewhere in their life they may not even know

it yeah I I found it's you know with anything it's like it's it's how you use it and use it responsibly and for me I found it to the point of like even like some just quick lifts on documentation one thing I've been working lately with your organization working with that works in this particular space and it has to do a lot of U data governance and distribution services and the co and other folks like well I don't really understand the difference between CIO Cals I time up oh so do these things and with some ideas I had on templates that I so just build in the blanks I was like good enough right good enough for government and they got

out oh okay I kind of see it I was like this many words these bullet piss points and I just ran it a couple times cleaned it up and it was like out the door just knocking things out instead of going okay now I got to find somebody team that can stop what they're doing to write up this or me without a million other things so it became really great to allow me to get things done quier and faster um yeah there's just a lot of and when we look at some of the things for the security tooling there there's just a lot of false positives imagine God forbid that we actually had a long

standard we didn't have to have today morning figuring out okay well because in PA Al it's going to be like this and over here it's and like normalizing ons things like that can be incredibly valuable because that has been human intervention now everybody's oh it's going to take jobs away it's like no it's going to take menial tasks away people are going to do better at their jobs and they're going to be able to do more of what they love as opposed to just spinning tires on you know runting rexes to clean up spreadsheets it's going to take jobs away it's going to take millions of jobs away that's not Happ I think there's a lot of but I

think it opens up the opportunity for B development on the top that then drives it more not fast so I can take all the jobs away it's going to take a lot of jobs it'll make a sign sad than embracing AI in like three different ways every vendor who sells something to your security teams has said that they do AI for years maybe 18 ever since Chad GPT you came on by fire um I would say year and a half ago it was [ __ ] they didn't do much of anything useful like they it might have been using a but it wasn't doing budet today there are real vendors really solving problems using more than just

data machine learning they're actually using AI to do things not everywhere but they're pushing that way and this is a way for you to take get the advantage of efficiencies that AI can provide without having to become fantastic at AI figure out which tool doing email security or identity security or whatever other thing is important to you is actually leveraging the value for that um I think you can figure it out pretty quickly when you ask questions around all right well how is it I making you better I don't want to hear what algorithm you're using I want to hear what value that's giv through product if you're not currently having that conversation with your vendors I

encourage you to Second um a lot of conversation around things like generative Ai and as a security team yes we can get better and we can create board presentations using that you know for scenarios for tabletops but we we should be thinking about what risks Associated it takes mostly a DLP risk like youve seen for every other Sasa that your company puts to unsensitive info into and that be think thinking about that you know at at my company at Pax AES where I work we we appointed a chief AI officer just a couple months ago and his job is to is to implement AI throughout our company internally and externally and make sure we doing it the

right way and I I've never heard of a chief AI officer before that and i' I've come to really think it's it's great where does that individual report is it like the CEO does it like an ESP kind of function like ethics or um he's technologist okay he's a technologist so the way so the way I way I think of it I don't know if Eric who's who's this person he likes my my analogy or not but I'm going to use it here because he's not here I think it like if you were if you were in a factory in 1890 whatever year electricity became widely available and you hired a person whose job is to

say you're the electrican person go figure out how to make all these things go better right you're going to go one stage at a time and say this would be more efficient with electricity than it would would be without it and you know I don't know 10 years later everything's got electricity right everything in the factory is now electrified that is exactly what Eric's job is with impact a figure out how do we do all of these things more efficiently more effectively effectively faster more consistently whatever that value is um I think that that is the key and I'm so excited to be there as he's appointed with that to go alongside and say and how do we do it in

a trustworthy manner in a secure manner as we're building this thing out I don't have all the answers I'll say that it's going to be a big Journey we're all going to learn along the way but um yeah gra 3 need more regulation Rob I do agree that uh that AI will take jobs but do you think it will end up with a net loss in jobs massive massive net loss of jobs many millions tens of millions of jobs so what are the jobs that won't be lost let's let's open up yes we all AG we're nearing the end of this talk I want to open up the floor for any questions out of guys we're going to time

lock four five minutes to do that I have a question um mentioned earlier about entry level jobs um and the reason why I'm asking is I get I get this question a lot is when folks are getting a degree or certification what is the gap between a hiring manager what you're looking for and what these folks are coming at of school with is it the hands-on experience I mean what can the education side do to help with that I say short of recycle I mean some of the stuff is like by the time they come out it's like too broad too long the question was what can education do to help enhance folks coming out of

those programs to be more employable and what I've seen is a lot of folks come with almost too generalist in areas that are too theoretical rather than what our hands on and what we're seeing is more practical and it's almost like they're too well-rounded in certain areas and to many esoteric things and what's happening now versus what's happened in the past first time ever I agree with [Applause] d the problem is there is no such thing as a security person security G security guy I I would never hire a security person that is not a position I need you to be great at something and that requires like do you know the language of security which you which is I feel

like what most of these degrees teach you is like how to what a vulnerability is versus a risk whatever P TCP connects on it's it's going to give me um I need I need someone who can go much deeper into one area than that so I don't want a security person I want a cloud security person and how do how do you help fill that Gap well you go take all of the AWS free training and you go SP up some instances and you go go secure it you go attack it you go like beat the [ __ ] out of this thing and you can get free versions of all the software you need to be good get your any of these

things to go play with and when you walk in you said yeah when I configured swun on my in my cloud my ec2 instance here I did these things look what happened right that's what I need I need someone who's done the thing but that's if they want Cloud security if they want to be an absc person they go they go you know get their their GitHub and and start contributing to different open source projects and show off what they're doing and go getting bug boundaries like like actually do the job as a hobby and then then I can go hire you to do the job because I see you do it my additional thing on that is never

underestimate the power of the internship so many internships have gone by i' struggle to get interns this year and that baffled me and the interns can do that in a college setting in a real world setting and put that on a resume and speak to that this is what I did as an intern for XYZ company for nine months I say like that before I was about four the insurance I had ke seven years ago we're all now in security management positions I me going around everybody else because they just had that first year of just it yeah are your internships paid mine mine have good one I do I I have a former intern uh back there um who who

I'll say successful you know Daniel can can talk about it or not but um I think internships are great I don't have much else to add you know besides that I think it's so we have a couple question you have something to say I I was an unpaid intern I'm I'm gen xer so I was back hardcore we had to get coffee and jackets for people back in our day but they are paid mostly yeah I've never done an unpaid internship um and I think they actually pretty well some laws won't some states won't even lie you yes so kind of piggy back on her question how do you feel about programs like ha thex or try me as as

how do you feel like uh programs like hack in the Box are for giving the experience I think it's great um that's the you're not going to be able to to do well in those if you don't understand the the basics behind what it is you're trying to attack and so you're going to have to learn some of those Basics to learn those attacks and I think it's going to show that you can uh you can do that work I think it's great so someone can speak to that you consider yeah definitely one of the things that I look for an interview is you know what do you do in your spare time what you know what is your interest

in this not just oh I went to school for this because I think it could make me some money or something like that and I it goes to show if I you spent 100 hours doing at the box and you come and talk about that it shows me that you really mean it right you're actually committed and that you know past performance is an indication of how much you're going to work in the new job awesome you're really going to get after it not just hey I know what I know and I don't want to learn anything else so I'm add kind of what Alex said and along these lines one of my favorite interview questions to ask security

practitioners is tell me about your home network and that tells you a lot like are they home labing or they just running some ly router from 1996 backing off with the neighbor

Wi-Fi back to AI I think AI is a great tool in isolated environment but are you this a huge risk I think to incorporate AI so what happens if someone misconfigured it or bad act or is that a risk you're willing to take I actually this is a topic came question oh sorry um putting boundaries and safeties around AI essentially right because it could go a little look sorry that was a question and this question came up I remember one of the panels with a bunch of crypto people at um last year and I thought I I POS a question I what about it having it more to something almost like FDA regulation around drugs like we

don't know the ficacy we don't know the safety there's kind of Curves on that and to me it's like don't want to say the more regulations is the answer but in some kind of framework of that before we just release things into the while yes I'm sure I can do a lot of work on math but there's some downsides chemically similar slightly more is going to be something like a buy bance or maybe HD it's there's going to be some testing and safety I think that we want to incorporate into the process before we just let the stuff go although it was the anniversary of August 29th ma she couple and to be a a food not a

nutritional supplement yeah I mean there need to be safeguards just like with anything else um if if we think we can just just deploy it and not do anything with it you're going to end up with in a bad spot but there's lots of ways that you can do it more securely run your own interal models um yeah you know do lots of things like that things are moving so fast I think a lot of people just dismissed AI generative AI out of the box when it came out um and I think that that's a mistake guys I believe we are out of time I would like to thank our panelists [Applause] [Music] relas don't get any music to play us

off I thought you wereb

okay yeah you're good I'll

[Music]

[Music]

[Music] it now

[Music] why [Music]

I saved some of them I didn't quite say that Reg that

[Music]

[Music]

we can slide it over a little bit more can you hear me in the back [Music] you hear me back there does it Buzz or M or S Tes I can move this back and then you can just sit on this side okay I see what you're saying now all right there we go [Music]

yeah wonderful I think that'll be plenty of Slack [Music]

yeah um this is

disconnected it was working it might just take a second to pick it up all right

sep are you sitting here huh this you to make sure I got this set up the right way [Music]

[Music] it's really weird to have all right guys looks like our next speaker is uh ready to go and before we get to you her thrilling talk on spifi and thrilling world of AI surveillance um I'd like to thank our sponsors uh we have blue fashion uh Bank of America and God point and uh here at you side we're all fall to run and we run off of donations so if you would like to throw money at us we would greatly appreciate it because that we can do this again next year and uh with that we will return to our regularly scheduled programming with Zoe samp and her wonderful talk about AI

awesome all right so I'm going to go ahead and get started here so as they just said we're going to be doing a quick talk on basically AI Vision surveillance and how you can set up your own at home and I know that sounds kind of funny cuz you're probably thinking I've got a security camera WS camera all that good stuff well you can also hook it up with AI models and track even more great things and we can also see how this is used at scale and I know what everyone's immediately jumping to is some dystopian future where you know the government watches you and all that good stuff but we're also going to talk about

the actual more friendly use cases where it gets used so really quick my name is Zoe Stein Camp I'm a developer Advocate at a company called impl Data we're a Time series database so we monitor things like iot devices hard devices uh we are used for things like observability so in general we're used for observing the hardware that gets used for powering uh AI surveillance I'm not here to uh sell you an AI model or AI Tech and I don't represent Nvidia that we're going to be talking about them and their technology quite a bit so the agenda is first we're going to do an introduction to Nvidia deepstream and Jetson Nano I actually have a jeton nano here with me

originally I thought I might be able to do this demo live but it's kind of a whole thing to hook this thing up so so you can kind of see it and I'll go over what this is in a second we'll also talk about influx DV and telegram which is an open source metrics agent we're going to really quick briefly discuss some of the vision a technology use cases that are good for for our lives and how you would actually set up this project at home so setting up all of the hardware all of the um mqtt sensors Etc and then originally it was going to be a live demo at the project I have a video

because I'm smart enough to think on the a backup project for this so we're going to see a Qui video of how this would look when it's visualized so first things first so for this solution we normally use a I'm going to kind of hold these up if the cords will allow me sorry guys very very Tangled over here everything likes to hold itself so we use a USB webcam you can obviously use something a bit nicer but for this project which by the way at the very end of this there will be like a ton of QR codes with links GitHub projects uh links to buy materials Etc but basically USB web cam it gets hooked

up to the Nvidia Jetson I guess I've got a collection here Nvidia Jetson basically this is a Nvidia device that can run uh actually it's kind of like a Raspberry Pi just imagine it like a more powerful Raspberry Pi is how I would describe it but basically it can be used to run Vision AI pipelines and ingest things like the raw data frames that come off that USB webcam from there we feed them through what we call an object detection model this one specifically will detect four things which in this room there's only a one thing really to detect which is people but it can also detect cars and bikes it can even detect plastic

cars because we've occasionally used it in a similar space like this and we put a plastic car in front of it and it it was it was tricked it thought it was a car so at least it somewhat Works uh and road signs as well though I've never tried to see if it would work for that basically when it does it will give us two parallel output streams we have something called an rtsp which is a realtime streaming protocol which basically is just the video frames it's basically when I I get to the next couple slides you'll see an image of what this looks like but it's just a box around the cars or people or things that

is detecting sometimes for example it also detects things like a chair it will get confused and be like there is an item in my scope but I'm not quite sure what it is and then our mqt client allows us to basically get our detection results and that's more of a result on a statistic so for example with the chair it would say something like 40% accuracy it believes it's 40% close to a human why I don't know because the chair looks just enough Like Us Versus if you're standing like right in front of the USB camera it's pretty much 100% it's like very certain that you are a person that it's it's recognizing you and it gives

these percentages because it's important to keep in mind that like again used at scale there would be like you know this could be an environment where there's lots of people kind of moving around we kind of look like fuzzy ants to the camera at times and so it's very good that it knows a relative um confidence level of how confident is that's getting this right so one thing that's powering underneath this is a model which again is basically just the pre-trained neuron Network ours is pre-trained specifically like I said before for environments with cars and people basically a street a street corner is what you can imagine other models are trained to look for things like animals different types of

wildlife for example will be hooked up with model models that are specifically looking for birds or for example the wildlife cameras that you might see things that are looking for wolves or coyotes etc those are powering up with a different model and with all of this basically let me go on to the next slide so this is all being powered up by Nvidia deep Stam in its simplest form it's essentially a series of gstreamer elements that have been optimized to make use of the Nvidia GPU acceleration it's provided an ecosystem of visual analytics tensor RT which basically is the modeling iot plugins which in include things like mqtt and Kafka that abstract you from learning those based

libraries and systems basically it makes it so you don't have to know everything to get this running it's abstracting it away and just making it easier to package as you can see from this screenshot that I grabbed from nvidia's website you can see that they're expecting you to send in sensor data webcam data and then from there they especially like to talk about it being used within Factory and Warehouse locations which is also where we see our customers use this as well uh as a Time series database we tend to work a lot in industrial iot but you can imagine like an Amazon warehouse with there's lots of boxes lots of items lots of things the

camera will have models trained on being able to recognize size and such and being able to also do uh get to in a little bit but also do like safety measures as well around those machines so this is what this looks like as I said before in a real like this is from Nidia website this is how they expect you to use most of these kind of projects this is how our project for example would be used more appropriately instead of being used within a conference setting it would be on a street corner where yes you can see those bound boxes that I was talking about before where it's saying here is the like item that I'm recognizing it's

a little hard to tell in the screenshot but there's actually a little number above them that says something like car and then the confidence level so again as you can see when the cars are clustered up next to each other and again you can imagine it like humans clustered up next to each other the confidence tends to drop because it's seeing a boxy item is taking a good guess but again the confidence of just the single car is probably the highest the car that you know is the most recognizable as a vehicle so the Jetson Nano is basically like I said before it's like a com compact um Raspberry Pi it is specific though for things like robotics iot

projects uh from their website their newo one offers 128 coola cores a quad core it's all this great stuff mine's actually a little bit on the older side I want to say this is about 2 years old and this thing gets so hot I swear you could like cook an egg on it and actually I've almost burned myself and I have to be really careful setting this demo down because yeah it has a little warning on it but yes these things get hot I don't know that anybody actually uses them in like an industrial iot setting I feel like on the factory floor you have something a little more powerful than this but people definitely

use them for at home use for Robotics and Ai and just in general tinkering kind of similar to the Rasberry Pi you could use it in scale no doubt about it but normally you would see it in a more home environment so when it comes to influx DB basically what we are is we're a Time series database which I've kind of already talked about and that means that we store data based off of time and our sources are very similar to the Nvidia Jetson sources we take in things from networks infrastructure applications again sensors we don't store like raw video files for example and even if I had set this up I can assure you we're

not saving the footage because I don't need it I don't need to store video files that would be too much uh and I have no interest in it anyways so when it comes to things like where you store this kind of data you might think that you could just throw it into any database or that it doesn't even really matter which is kind of true but you also want to keep track of um when I actually show the video of this demo you kind of want to keep track of the confidence level you want to keep track of how many cars and people it is detecting especially for the use cases that we'll be getting into soon where

you actually need to go back and reference that data you can't just always throw it into the wind or be real time monitoring it sometimes you need to be able to go back and reference it so we are an ideal solution for this because we already deal with that time stamp metric data and we are specifically made for it because we also query on time so you can say something like in the past 5 minutes 3 days ago uh you know I'm getting a bunch of like sensor hits what was happening basically like what is the confidence level here and then if you actually were keeping your video files you'll be able to look back on that one as

well and then this is just basically talking about some more of those use cases things like ingesting a high stream of data like what this device tends to give us compression which is basically the ability to store that large data set without breaking the bank we also have things called retention policies which allows you to delete your data after a certain amount of days or time so a lot of our customers that to be like in 7 days just delete it if I haven't looked at it by then it doesn't matter kind of thing basically it's somewhat real-time monitoring with like a little bit of leeway room you could say a little bit of lag uh cardinality

which is just the ability to store a lot of metadata that comes with these kind of data types and as I already mentioned quering on time so let's go ahead and overview of telegraph as well so Telegraph is a really useful tool for this one we're using two input we're actually using the mqt input plugin I'm going to mention another one that's specific for Nidia Hardware it just doesn't work for the Jets and Nano but it works for uh other customers who use Nidia Hardware at scale but basically this is a really cool uh open- Source collection agent for ingesting metrics it's driven by the community although we're the caretakers we uh we make sure nothing buggy or

security Hazard gets through when people put up PRS and these are just a few of the examples of telegraph plugins so although we're going to be talking about the mqtt one we obviously have quite a lot of different options and these can all come from different data sources most of them are the more I like to call Professional they're things like open Telemetry or uh looking at your mongod DB but we also have some unprofessional as I like to call them ones like Minecraft and csgo yes people do use us for monitoring their Minecraft data I have no idea what they're getting out of it but I hope it's valuable but yeah there's 400 of them so

this is just a very small list I have I do have slides where I go over all of them but it's it's too much and the big thing is that we technically normally use two so we have the telegraph Nvidia system management interface plugin which by the way this plugin is supported by Nvidia Engineers that's why it also has you know actually have a few different Nvidia plugins not just this one but it tends to work uh within embedded systems such as the Jetson family but for this one unfortunately it does not work with our old Jetson device basically unfortunately our device is too old I'd have to go spend a lot more money to get

the new one they cost like 500 bucks and this one was significantly cheaper and it was in my home so we just decided to run with that instead but in general Nidia system management interface can help you get things like your GPU utilization power consumption all kinds of awesome data that is important when you're monitoring Nidia Hardware well we're going to end up using though is the mqt input plugin which is basically just going to allow us to uh it normally is used for things like iot devices and sensors but it can also be uh manipulated or I guess you could say customized it's probably a nicer word it can be customized to insert other types

of data as well so for this project we take advantage of that mqtt plugin as well as the exact D plugin which is just even more General use something that I also wanted to mention is the telegraph starlock processor plugin which allows you to do like custom metric processing which we're also going to be taking advantage of basically it's going to allow us to do a little bit of down sampling and logical operations on our data set this is good so that way then we don't have to do it like let's face it the database isn't really made to do data manipulation as much there's many other tools to do that and this processor plugin is one of them so Telegraph has

input plugins which is how you get the data in it has output plugins which is where that data goes on to it can be something like influx but it could also be other databases or data syns as most people tend to call them and then processor plugins which allow you to run aggregations down sample just in general manipulate your data before it even hits the data center which can obviously save money if you're dealing with things like a high data ingest or just in general make your your life a little bit easier so let's talk about some technology use cases where this project was originally actually inspired by now for security purposes I cannot name

anybody by name that's just kind of how this goes but Nvidia actually does name some of them on their website but for this one this is a company out of France that's all I can say about them and basically they use this basically they use a much more advanced version of the webcamming to monitor how many people are within the train stations at a time and the reason that they're doing that is to prevent basically a crush as many of you might know that is a thing that happens you have things like crowd crushes just in general too much crowding there there is a limit to how many people can safely be within a space

in a station in a an environment that you know there are walls around you to an extent and so what they do is they use the camera to basically keep track when they might need to uh basically have security guards go and stop people from keeping going into the train station you know you have a big ta with concert it gets out you got like 60,000 people all trying to get to public transit but there is you know there's safety involved in that and these cameras can immediately alert because again in this image especially imagine this with you know a nicer camera basically you can obviously tell that there's just like hundreds of humans and

again it could be done by a person it could be done by a human watching the camera to be sure but again imagine that you're the security person who needs to go down there and just start dealing with stuff you don't have time to always be staring at a camera sometimes times you need to have extra staff come in and you need to make that call and it's just easier if you just have something else doing the automatic monitoring for you to an extent and although I mentioned something that you would expect a lot of people to come from like a Taylor Swip concert it can just come out of nowhere in general these kind of things just

happen who knows there was a giant 5,000 person wedding down the street and it just like got out you were like I never expected all these darn people where'd you come from and so this is a real life use case that gets used for safety around mass transit and that's just one of them this is kind of something I mentioned before which is Geo fencing for safer work environments So for anybody who hasn't gotten to see these kind of robotic arms in real life because I go to manufacturing trade shows so I get to see them they're kind of terrifying like like they're cool they're very very cool they're great in what they do they you

know they save people's backs they're here to save your back so you don't have to like pick up heavy items they're great for that they sort things that we don't want to sort but in general they are very large and they don't they don't recognize that we are not like them I guess you can say they don't recognize us at all half the time they don't have cameras on them all the time and so a lot of these arms are just operating they're just living their daily lives doing their daily task and if you as a human get too close Unfortunately they might just hit you it's not their fault they didn't see you coming they don't

know why you were there they're you're in their space you're invading their space and so what some people do is they have ai smart cameras that basically means you no longer have to have this like normally what you would do is you would have a barrier a fence around the robot that it's kind of like the obvious step is like okay if you're going to hit me I'm just going to put a fence around you like a bad dog um but what you can instead do is have an AI smart camera that would recognize when the human or and and that means any human body part by the way not just like your head for

example or your whole body like maybe you like you know sorry guys maybe like do this I don't know you're waving at a coworker or something and the robot is right here with these AI smart cameras it would actually immediately stop moving it would just stop in place and it would realize that you're within its like its personal bubble and it would stop for the safety of you because again even if it's just your arm next thing you know your arm's being you know attacked by a robot because again it's just continuing its life it doesn't recognize that you are there and so these are really important in you know Factory floors Amazon warehouses anywhere where people are working next

to these robotic uh robots that you know are becoming more and more common in these environments and again as humans we're not really used to per se working with something that doesn't stop when we hit it to be sure the other thing that it gets used for is forward-thinking Street design so it's really nice to be able to say like this street is used more commonly by pedestrians and bikers than it is by cars for example like there are streets out there for sure that it would be more valuable to have a wider sidewalk more bike Lanes like I don't know just people aren driving on the street maybe it's not the ideal Street to drive on maybe

parking sucks there who knows there's many reasons that like we designed a street originally to have six lanes and it doesn't need it and one thing that street designer like to do is they like to use this kind of technology and again they're not always saving the video footage they're actually saving the counts that's all they want they just want the count so they can come back and prove look in a single Saturday a thousand people walked by in an hour there was 50 bikers and there was five cars and you can be like look you don't need all this infrastructure for the cars when the pedestrians and oh and of course like 10 scooters of course and

you you know you're like I need we need wider sidewalks so the scooters stop hitting the pedestrians the pedestrians are crowded and crowding still good for the pedestrians and like the street shops they want more space for their pedestrians etc etc and again sometimes you need the data to back up these decisions it's like I said it's nice to imagine that you could just take some photos you know prove it that way but a lot of times if anybody here has ever sat in like a a council meeting for your uh for your area I've actually sat in a few in Aurora where I live you need data because and even then people will argue

with you and be like I don't care if it was it was a random Saturday they'll be like how dare you assume that people like this tree um you need a lot of data you normally need months of data to back up your obvious at times assumption your obvious things that you could see with your own eyes and this actually does help quite a bit in being able to prove especially with the model that we're using like hey no I'm dead serious like there's more dogs on the street than cars I swear and then another big one is being able to help in remote locations so basically what they do here is they hook up Vision AI software onto the drones

and this is being used especially in remote areas of Northern California that are known to set on fire and basically what the drones do is they're going and they're monitoring the infrastructure and this is already in use there's a few different drone companies that do this across the US and they've become very popular because to be fully candid it's not a fun job to get up on the top of electrical stuff it's just not safe it's not easy there's many issues with it the Drone has less issues and if the Drone gets electrocuted who cares uh you just replace it and you move on or you know the Drone it gets in the wind and it

gets a little angry and then goes back but basically the Drone is just a much easier thing to work with and half the time it's being controlled by a human who's sitting comfortably at their home on their couch with the dog where they would rather be and rather let the Drone go deal with the you know Less Fun environment or very very remote locations where these wires end up being cuz again if anyone here has done like the Mana 2 incline that was basically how people like built stuff back in the day and you know you still don't always want to just go have to hike up a mountain to get to the electrical lines

to see if they're looking good so that's where this Vision AI gets hooked up onto something again like a drone or some other technology that can basically travel along with it and again this isn't like a menacing drone it's just there to make sure that the wires look good so hopefully nothing uh collapses or catches on fire as it has done in the past and will'll probably do in the future for being honest so setting up our project so again I'm going to leave links for the GitHub for all of this but basically the gist of it is that you have your Jetson Nano hooked up to your camera you could use like I said a nicer

one but I just have a really crappy $10 one for from Amazon that does the job uh it freaks people out when it's in use that's all all its purposes uh and then from there we have our mqtt broker which is hooked up to Telegraph then we have our right endpoint which basically writes to an influx CB OSS instance running on the Jetson device that's something I guess I forgot to mention we technically have an open source uh database I always forget this I it's just common knowledge for me but we have an open source database that allows people to run it on devices like this we run on Raspberry pies we run on home

assistant it's a pretty common use case for the open source database from there they can go ahead and send it up to inflex CB cloud and they could run it through something like a detection mlops for example like you could take this further which we're not going to show here and it's not in the code it's more of like a it's like something you can imagine into the future in real life but basically from there with the little Jets and Nano we're doing a uh we're doing down sampling we're basically saying instead of the amount of humans every second just give it to me every like minute or something like that so let me just go ahead and go through some

of this code so setting up the project is pretty straightforward once you have the hardware the project itself is living in GitHub pretty straightforward there you pretty much just install the PIP requirements and you have that going then we have as I mentioned before we have our mqtt client which is publishing samples based on that custom data model to our mqtt broker so this is for example what a telegraph file for mqtt would look like um or sorry my bad this is actually the uh po the pyo mqtt client which is basically a library that's open source and super easy to use here we're specifically getting the interface results that are sent during our what we call if frame Loop you can

see here that we conditionally only send the results every 30 frames since our webcam produces about 30 frames per second this is basically a value per second it doesn't always make sense at least for our use case to report each frame since most objects will not have changed within one second that would have to be a really fast dog just has to be like instant so so for the mo and a really fast human too for the most part we don't move that fast we kind of especially when we're staring at ourselves in a camera that tends to slow us down even more tend to stand right there our metadata is then saved based on the interface data structure that we

created which we'll go into next and basically with that this is this for those of you guys who aren't aware mqtt also takes advantage of something called mosquito this is basically just setting it up again all of these commands are within the GitHub file so this is really just a a quick overview of how this would look so as discussed previously when we write our own like mqtt client it gives us a little bit of flexibility over the data model that we're going to be delivering so with this one we can see our detection frame number which tells us basically the frame number for which the detection was taken this number basically continues to go up as we keep

getting more frame rates you could honestly throw this one away it's not necessarily the most useful because it doesn't contain any pertinent information just says when something happened C up being if you're actually keeping your video recordings this would be a helpful thing thing to keep on the total number of objects being detected by the model so for example with this one it's saying that at the frame of 2,280 we see one thing now obviously with this it's it then from there it says within our interface results actually what it's seeing so it's got a class ID for example uh the one is a car the two is a person so with this one it's got person we also have a human

readable version of this class ID that we can see as person so it's very easy for us to understand what we're looking at we do not have to remember that class id2 equals people because that's a little hard for us to always remember especially if we were tracking more than just two things for example then we have the confidence level which I was mentioning before that's the value between zero and one you can imagine zero as 0% confidence and one as pretty much 100% confidence so anything in between there just basically add a zero or add like three zeros so if it's like 6 it's basically 60% so yeah just add like two zeros to it and you're good

from the coordinate left that is the distance between the left edge of the frame and the top corner of the bounding box most of these next uh the the core top is basically the same except for it's the top edge of the frame to the top left corner of the bounding box basically it's saying where the box is within the frame it's a little bit weird to to actually talk about it imagine it like latitude and longitudes within the image basically then we have the width which is the width of the bounding box so that's like how big you or the item within the camera is so again if you're super far away from the camera your

bound box is pretty small because you're pretty small but if you're like standing right in front of it it'll be like a pretty big box because it's like look there's a person right there I can I can see them it gets very excited makes a really big box and that's just basically telling us how big it is the object ID is technically a unique tracker to be clear our model is not smart enough to recognize if you come back it it does not know you that well it's not using facial recognition so this number for us is okay to have but it's not like and at least for our model it's not great because it's not actually like I can't

tell that Mark came back 3 times I really cannot tell it doesn't know it that well this number is irrelevant for that uh example and then the unique Comm ID we're also not really using but it's normally again a tracker for um all the detections so you may be wondering in the current data structure how we basically would dist distinguish the different parallel detections given the time interval with influx B we simply just overwrite the field and we can read on how to do that within our Telegraph section so this is how this looks like when we're actually sending this in uh so this is a telegraph yaml file this is the ingestor in particular so you can

kind of see up here we have an alias we have an interval of every 5 Seconds go ahead and send this to our Local Host value because uh sorry our Local Host instance because again we're running this locally we have a topic here called interface we have a connection timeout just in case we need to have that and here we're using the Json V2 parser as well so this basically is just saying go ahead and send this data in this is how often we want it sent in and then you can actually start to see part of of this is the mqtt uh input consumer as well and then this is pretty much very straightforward this is just the output

plugin again as I mentioned before you have your input plugins for telegraph then you have your output plugins this one is pretty straightforward because it's just set up for influx but basically it's saying uh for this one for example it's saying send it to the cloud but we actually end up sending it to Local Host we give it a token an organization and a bucket which is just a databased destination basically to write to and this is the starle processor so this is back within the telegraph file so all of these live within the telegraph file you got your input your output and your processors so for those of you guys who aren't aware Stark is a

python like scripting language that basically just allows you to manipulate your metrics in real time this can be anything from performing more advanced custom calculations to converting time stamps for example into something like UTC for this one we're adding a tag to our metric which will act as our unique

identifier so for example here you can see our before results and our after results where we have now added our unique identifier into here so this is the Json sample produced in our basically our line protocol entries which is basically just the telegraph entries that we're creating one for the first object which doesn't have the Starlet processor and one for the second basically this is our metric being passed through our script as you can see here now we have a new tag applied to each metric just making them B just basically making them more unique entry so we can reference them a little bit easier when we go to visualize them and finally this is just the

outputs again just for like with everything basically added a little bit more information so just to clarify this is basically how this goes in you have our Jetson stats going to the exec you have our interface stats going into the Mt consumer they both go through starock so they can be connected via that tag value and then we're sending them with two different output plugins one to cloud and one to OSS and that's the other thing too is you can have multiple output plugins with Telegraph so you can choose to send it to many many many different places there's plenty of people who use like 10 so the data can go to all the different uh syncs that

they required in now one thing to note though is that we don't always want our Jets and stats actually to go through starlar so we can all versus the actual exact ones so you could also do this as well where you're basically just saying you're basically saying a name pass which just saying like pass this not through the processor just go ahead and send it onward to the cloud versus a name uh versus just sending them both through this is a we we actually have a version where we do both it just depends on which one uh basically it's more useful for your use case so now that we're actually adjusting our metrics into inflex with

this Telegraph file the goal now is that we basically wanted to have a simple dashboard for our raw interface data and we wanted to down sample in transmission to our Cloud instance so that basically means that we wanted to down sample our data within our local instance before we send it up to Cloud people do this for multiple reasons the big one is just honestly cost savings and also just doing some pre cleanup before it heads into cloud and this is any kind of cloud although it's a mentioning influx in general when you send your data onward into a cloud environment you kind of want to down sample it first this is just a quick down sample file what this

one is basically doing here is for every 10 minutes it's down sampling some of the measurements that we've been receiving and it's just basically putting them into it's getting the averages of them if I remember L this has been slightly cut off so I have to I have to actually go back and look at this file again but basically from what I remember it's getting an average down to every 10 minutes of how many uh counts it has so this was supposed to be our visualization and live coding I'm going to be pulling up a video here so give me one sec you have to look behind me yeah you can see all my stuff up here

for I was hoping to show all right let me see what my video

n that VC is not what we want that is actually where you run the uh you actually run the I wonder if it actually still yeah so you can kind of see this shitty image down here at the bottom that was the camera running earlier in the hallway where it was catching random people and it's also crashing go away go away so let me make sure this is also muted I don't think there's any noise on it but it would suck if there was I guess you can't mute this from here oh that's right it would be on my laptop all right we're good so basically this is how actually I'm going to start this from the

beginning I almost forgot so this is how it would look within the inflex CB interface so as you can see this is it running locally so we've got our Local Host running here we have our Edge interface and our VW interface results sorry I'm trying to get this camera thing to go away there we go with this we're specifically looking at the confidence level but you can also see things like I just realized I wasn't talking to the mic I'm sorry guys let me grab this okay cool Sorry Sorry everyone in the back so basically here we have uh this is how you would see it on the local instance of implex TV so this is just basically a very simple

it's not meant to be like a dashboarding visualization it's just basically to make sure that you can see your data coming in so here you can see we have our Edge interface bucket we have our BW interface results and here we're filtering on our competence level so that means that you can kind of see it's it's a little fuzzy here because it's uh just starting let me playing the video yeah so you can kind of see our confidence it immediately shoots up then shoots down depending on what was happening during this it would and those are like the full results all rode out you can see like our time stamp value the host the object I actually don't remember where

we were and then this is how this would look within our graphon visualization so you can see our average confidence level is at roughly 45% or so you can see our frame number the last label which was a person so that the last thing that we saw I think I was doing this at home so I think this is just me waving at the camera you can also see our bound box and our height basically where it is within the you can also see some of the detections here I'm going to pause this sorry like I said I was planning to do this live so I wasn't quite ready for going over the whole video um so

basically with here you can see like the bounding box height and width and the detection status as I said before this one's a little more boring because it was just me waving at the camera but you can see down here some of our detection St St and that would come with a much longer list if we just had a few more objects basically in in our camera's range but yeah this graphon dashboard is available as well within the GitHub project so that makes it pretty simple to just go ahead and download if if you've never worked with grafana you basically just download a Json object which then you port into this to build the dashboard it's very straightforward

and we also have a graphon connector as [Music] well I'm a really quick see in my oh my goodness this thing is still go away youc is the worst literally the worst I wanted to see oh my good yeah of course I have I wanted to see if this is just a recording of this I think it is unfortunately darn I was kind of hoping I had a recording where we used it with more people basically roaming around but I don't think I got to no I don't think this is no this is like a different one sorry guys I wish I had more videos but basically that does kind of go over pull back up my slides yeah

basically that does kind of go over how this looks when it's actually in place and you can imagine there we go you can imagine at scale you be able to see things like multiple people we tend to get a confidence rate that goes up and down depending on like I said if we were in this room and the camera was up there it would probably be pretty high probably at like 70 or 80% once we were in a room where it was basically just the camera staring at a bunch of chairs and the confidence was at like 20% it was like I don't know what these are but they're something it detected that they were

something but it was not very confident it's detection so here are some of those learning resources that I mentioned so I like to put everything in one slide for the QR codes because honestly there's just a lot of stuff going on in this project the Jetson series is pretty much where you can find information about the devices the sorry the GitHub project is obviously where this entire project lives where the code is it has much better uh instructions I guess you could say more more simple instructions to just get this up and running once you have the hardware set up there's a YouTube video here for the Jets Nano plus deepstream basically how you get

them to work together and get everything installed on the device some deepstream examples so this is nvidia's own example category of you know how people tend to use it what they're using it to be powered on like I said you're going to see a lot of industrial use cases there you're not going to see so much about the ones that get used in uh in um I'm trying to think of the word in the outer world like in like outside space because again they don't want to be uh you don't they don't want to be associated with you being spied on by like your neighbor uh or your government or anything like that then you have the RT sensor blog

that kind of talks about some of the modeling options that come with rt sensor a walkthrough blog which is where one of my co-workers walked through this entire project so that way then he could basically explain some of the reasoning on some of the technology that we picked how it was built out Etc and then docs about the two plugins that I mentioned earlier we also have a try it yourself if you uh this also by the way this lives within the influx Community that's why this is up here on the screen that's actually the like that's where multiple other projects live as well if you're interested in uh trying out other types of iot projects we have things like

plant buddy where you can monitor your own out plants and such so when they die you can be blamed completely for it or you can try out inflex data.com as well we have like a free Cloud free open source all the good stuff I'm sorry I didn't mean for this to I I meant to do this a little bit more live so I have a little bit of extra time here for questions uh if anybody has

any um so I know for in the kind of like home security realm uh blue iris is one of the main yeah this guy now he knows what's up uh is one of the main softwares where it takes rtsp protocol camera feeds and then people have adapted uh like Google Tesla video cards to run AI uh uh object recognition on it so that you know when a person comes up in like a box it'll tag it it's like delivery and then you know message you uh this seems to go in in a different more analytic route is that the my understanding correct is that you're not trying to figure out like specifically like who is a person that's not end goal

of of this but rather like how many people and you're using that data for whatever yeah that's so the question here is uh if this is being used as a more analytical purpose versus sending you a text that might let you know that a person with a package has arrived or dog is on your porch or something like that and that is correct this is definitely like I said this was inspired by a customer who uses it for monitoring mass transit and again they're not again like I mentioned at the beginning this is assuming that you're not really keeping the video feed you might be looking at it as a human like you might be like staring at the camera or

something but for the most part it's counting without your intervention and it's just counting it's not necessarily always storing like for example I use ways at home it's a you know an app that again it will tell me like a person came by with a package from Amazon or it tells me A Pet's on the porch and it's always some random cat that's not mine but regardless that's that's sending me video footage it normally saves me like a minute or 30 second clip this isn't meant for that this is more for like yeah a longer term an analytical look at this and being able to count up like I said for Street design or for safety

concerns but yeah a little bit less one uh notification and a little bit less even tracking the actual camera footage as well because like I said we don't we don't even store this camera footage a lot of the customers who we work with they don't sore the camera footage either that's not really the goal the goal is to analyze what's happening either right then and there like I said for the robots who work with people or analyze it over the longer term and be like yes we have 50 people on the street today we have 500 tomorrow it's etc etc they're just basically two different ways of getting to the same relative place of yeah using Vision AI to power

useful things you could say just in different

ways yesal invb up uh the upgraded Jetson runs somewhere between $200 to like $1,000 depending on the hardware running on it mine was about I I want to say it was either 200 to 250 and it's an older model so the whole home lab should run you under 500 though because everything else here on my little table is like I said $10 from Amazon very crappy you could get better for like 20 bucks uh ethernet cord from the project for this because I need to actually hook it to my laptop and share internet with it because that's just how it goes so again basically the Jets and Nano is your most expensive piece here deep stream has okay one thing to know

deepstream has paid packages and then has free packages is we're using an older one so it's free so that's cool with something a little more free but probably still does the job you can go old school it doesn't matter and it will work for you so that is one other thing to note I suppose but all in it should be under 500 depending on if you can get an older Nano it might even be cheaper these used to be cheaper Nvidia has gotten very uh uh you know popular recently I don't know [Laughter] why yes have you looked at doing this with um there are make cameras that have have an actual embedded processor on

board the camera so that it can actually do some of that down sampling some of the analysis on the camera I had to work with one back in like 2018 have you looked at kind of porting some of these kind of porting this project to a camera like that where it's the camera itself is doing the processing for it sends the video feedback yeah actually we looked at that originally for this project and I was not opposed to doing that kind of Hardware instead I swear my cooworker just really wanted to get Nvidia Jetson like I I swear there was ulterior motives that play cuz he was like no I want to do the hardware for this and

when I had to we both we basically did the project together but he slightly let it at the time and I was like when this thing showed up to my house I was like why did we not get the camera why did we have to I wanted the simpler route like you're describing something that already had everything uh hooked up and he was like nah we're going for the gold I'm like all right that's also how I have like three raspberry pies in my house as well which again his I think have gone on to do something more fun in his own home I swear he has like a plant jungle that he's monitoring with them or something

but yeah there was ulterior motives to use the Nvidia Jetson you can do this with lots of other Hardware by the way definitely you can do it with a piie you can do it with other basically anything that's got like you said a camera with the smart Tech on board you have many many many options this just is specific to Nvidia that's where the GitHub file will have I'm going to be honest yes when did you start developing this uh roughly I want to say about two years ago about two years ago that's why this one is an older model as well yeah I'm just slightly curious I was time for architecture some of

yeah now the original project has been done I want to say two years ago and it's gotten updates over the past year or two not like major updates really just small uh bug fixes and stuff like that the Deep stream and the sorry yeah the Nvidia deepstream version that we're using I think is from like I want to say it's from 2019 or 2020 because like I said we're using an older version because it's free uh if you get some of the newer ones you have to pay a little bit of money for them but they tend to the older versions tend to become free at some point point I don't quite know how Nvidia does it but that's basically

what my co-workers said when we pick the old version uh in the library to install and I think Nidia stream itself I want to say it's been around since 2015 2016 but I think at the beginning it was definitely a project that was in its infancy to be sure cuz you still needed to get all the modeling done and obviously as anybody who's worked with AI models nowadays knows it takes a while and a lot of iterations to get things to where you really like them because like I said this one's actually pretty decent at recognizing uh at least people because we use it mainly with people it recognizes a plastic car but who knows how long it took the model to

get that good I didn't build that model I didn't make it you know as good as it is today somebody else had to do all that work and that probably took a couple years to be sure BF and nobody else has any questions currently I have a really cool use case for you actually if you want to talk to me after this sounds good thank you guys so much for coming to the talk [Applause] I know none of you are tired of listening to me talk remember we have four more presentations so don't go home these are even better I [Laughter] [Music] hope difficult and very recently I had mildly scary I had cars there was a car that I

thought I see but I can't recognize myself I can think of a few different ways to use something like this to help navigate the world around them especially in terms of things like you know who coming to my house [Music] how many you know my neighbor in my yard I you know are there random C in my neighborhood all this stuff basically all the things that a person who would not be disabled could see yeah you should talk to like the um oh I'm orig from newey so I know the group in New Jersey um but there's a lot of groups you know a lot of groups for the fly that make use of all these different

types of that's why I got into it in the first place to help with everything from things like the menues to recognizing you know how much actually an interesting use case would be if you can do it in something like a smartphone for me to be able to tell how many bills oh yeah to recognize different money right now what I right up here or trust the cashier theh so things like being able to dump my wallet on the you know table what is the good thing is that as the models improve you're also going to be able to get what you said at the beginning which is the ability to re-recognize things like cars

and people like if basically you would have on board facial recognition like eventually maybe it could be hooked up to something that you know is in like ke or something for example and it says to you hey this is Thomas Thomas was here yesterday basically like even if he said nothing to you you can see it with a person but it can recognize them how many people showed up anytime iology like for [Music] accessibility around you know and that's important because I feel like we build a lot of Technology but we don't even though it has disability use cases it's not the first thing that jumps into everybody's mind it's basically extensions or come from that background

or who are in that Community to kind of be like hey this Tech is useful to me like if we could leverage it a little bit more the only Power I know is connected to it already as a whole tracking things like for monitoring things like what's the one of our um co-workers uses it to monitor his his mom's heart monitor she has a you know I don't know what it really iser yes basically yeah pacemaker that monitors her and uh he uses it to basically monitor her from afar cuz she lives in India cuz you can use it you can use it kind of similarly to things like um you know like the epileptic seizure detection dogs like

diabetics detection dogs you could use it similarly especially in the diabetes if you put it up to the out from start learning things about your normal habital the real time time series data I can see a lot of use cases for being able to store that data for a short period of time run some analytics over it and then forward it off to the doctor yeah like we I always point out like Health trackers like fitment and stuff are great examples of Time series data like although they when they grab it to you on the phone and stuff they extrapolate it out to be like by the hour or maybe by the minute in reality

they're tracking you every second they're not tracking you by the minute I can assure you they're definitely tracking you every second they just extrapolated down for human readen also if you ever need somebody to come with you to argue with the Aurora Council about uh making our streets more pedestrian friendly I can I will mobilize my Army of people because I have been fighting this fight well luckily that's why like I said that's why this technology is great for the people who do because it it kind of kills arguments in the water and also another uh group that might be interested in it is RTV because of the way that they built the stations for the

trains they have no way of actually measuring how many people are are actually using trains which is why we do issues with the trains becoming too crowded and the station's becoming too

[Music] [Music] cred I do have an extra spot over here I cannot yeah it's definitely very important that's why I by hold the mic I can move it back there there one over there there we go yeah he can probably use one of the animal model to thank youate guys

okay we got juice Step One is he going to sit

herei

are it'll work I don't know if you have

one wild CER and

stuff yeah

[Music]

come [Music] onug it again try unplug [Music] Ito takes a [Music] second is that you the projector I hope [Music] [Music] there we go let me get Spit hot fire I you Super Hot [Music] Fire where's your game on that M uh I don't know on this one is it not why you worried about the game [Music] bro see if you're understand JY

is it working are you happy with [Music] that try again 1 2 3 4 five microphone cranked you up [Music] twice next speaker two minutes [Music]

of course thanks for speaking you as well

[Music]

[Music]

[Music] where' you get the beer thank you for speaking Yeah got thank I microphone guess it's [Music] almost I'm not going to be in the way over

[Music] here it's cold I need a drink leaves never comes back need's a confidence Pier confidence scotch social Scotch social scotch [Music]

[Music]

she went to go get soci you just dipped off yeah I know like what am I giving the stock now [ __ ] I'll give the do you know Z I know oh thereor [Music] [Music]

[Music] get out the way get out the way [Music]

all right so uh next up is JY square and uh he's going to teach you uh who the people in your neighborhood are you have Bluetooth thank thank you turn the extra microphones off one up here hi I'm Jimmy Ali or Jimmy two times or Jimmy Square um thank you for the opportunity here to speak at bsides today um we're going to be talking about who are the people in your neighborhood and yes it's around the Bluetooth stack and the signal of Bluetooth um little bit about me 30 years of in puac uh blue purple red and all it apparently the CEO lost Labs um former member of the US national video game team and I like to [ __ ] around and

find know this presentation is going to cover an inspiration or why I wanted to visualize um it's kind of dark night we couldn't see it so we wanted to bring it to the light um correlating some of the ois new uids and devices and the people who carry those devices um enriching the data with ENT classifications some of the practical uses for red team and blue team and uh hooking it up to a Sim so we know Bluetooth is everywhere um it's all the graphs just point upward and as we again throw in AI we think we're going to create some really smart Bluetooth out there and it's just being shipped out left and right um we'll say there's not really

any current any security being applied there we don't have any firewalling we don't do any controls we just kind of open it up and hope that our G Services don't get spoed they Stillen um there was a little Twitter post here about somebody uh at the MF who had a BL attack spam mess with their insulin P so that's where we are with uh Bluetooth it's affecting medical devices um and all kinds of Industry so this is from the hatri perspective and you know we do it for the I guess F in the blank we're all different sometimes we do it for the lws um some of us are here at the fend I a Defender so I do all of

this offensive security to help defend the base um we like to say have to live not live to ha and a quick shout out to the DC goons Franny 303 and the 414 family 303 represent so the original idea for this Bluetooth device that we're going to be talking about that's made out of a red3 p came from the DC darket contest where I did all the elements I was busy volunteering I wanted to get sleep and they told us you have one more element to do to get to the party and I thought if I just monitor these guys Bluetooth signals I'll find them at the party later and so I didn't do that that year

did the uh The Dark Knight competition in a normal ethical way but I did go home and started thinking about how I could do it next time if I needed it and so Bluetooth control was built into the Chromebook from the very beginning so Bluetooth control and blues is basically the Bluetooth stack you'll find on many Aro devices uh Android um Linux it's everywhere so we have all the binaries on the system to do everything we want to do to talk to HCI um gas sdp all the different protocols so originally I had a kind of a hack where I followed uh the blues program through expect and delt the laog file out as we

saw traffic and so we're able to see Mac addresses IDs um and things like that so in this log it might be hard to see but we can see a a name of death time Furs on some of the devices and there are several Mac addresses so we're getting somewhere we're able to identify objects and then spent a long week learning map plot with which was horrible um I had to learn way too much about that program but it helps me get these signals plotted over time and the g charge so they make sense um wrapped a little chrom UI around it um to be a little more functional but it was completely passive we're only watching uh signal come in

and we can configure whether it's uh classic Bluetooth or ble and so looking at Stats overall um scores what devices we've seen the most What U IDs topn addresses of the last 10 minut minutes where this is all the time we're able to see patterns now and see when things come and go or whether they stay so after playing around the r pie boo I learned a lot about where I could catch the signals choke pointing I started learning patterns about my neighbor when they woke up I'd see their lights come on I'd see the lights go out I would also see things like my neighbor was cheating on their spouse because I would see their car

leave every morning and a new Mac address come in and I I just went out one day to kind of investigate and it was real so you can see real world events by watching bluetoo Sigma um also if you use more than one raspberry pie you can get some analytics around directional um you know even uh X Y and Z coordinates so I had for in my house to see I could figure out which direction and stream and you can there's algorithms out there with it so my next idea was to be maybe annoying and create a little bit of a l device around you know the TV be on and be open SES me and

it was all persistent bearing on BL St and so I started actually trying to think about connecting not just looking at the signal uh doing info commands or connect commands or even trying to repir so it started off as a fun project to bring to our happier conferences so with some friends we wh together a handful of raspberry pies over powered uh with a bit of a uh a game on there we'll get to that in a second but it's a Raspberry Pi with a unicorn uh hat just LED Shield SD card Battery Source and a little Pi z h look like a little stick of dynamite with some LS on um I wanted to test uh to make sure

it was going affect anything negatively and so I used to go into an empty parking lot in a mall where there's no Bluetooth at all and do my testy and that worked it was great to isolate the signals and make sure it wasn't affecting anything when I felt like it wasn't really affecting anything negatively I took it to Red Rocks one night at a concert and I think my pie almost melted in my pocket I Tak like 10,000 ma dresses in like 10 seconds and it overheated the stack and it burned me actually it melted a bit of the case from the 8 gr High Cas also so then we came up with this all

year Bluetooth are Belong To Us game that we started bringing to the conferences and so I grec back in 2017 we made some Punch Out booo badges where they fought each other um it's kind of a little haer game with Bluetooth um it was a lot of fun thank you to the devil's voice we made that a great experience for me um Herby as well in fact the next year Herby had posted on Twitter somebody ask what's your favorite memory from last gerod and Herby says my highlight from last year was when the Bluetooth Punch Out badge uh trying to pair with Dave Kennedy's laptop during his talk so we we know that it was working and it was

definitely causing some sh uh so to make up for Dave I took one in Deron but it was passive only um it did ble he used the HCI tool it was more about again enumerating looking for things and I was able to take part in the uh the car heck CTF using this badge that was pretty cool um we went to noon we called that invasion of the unicorns and we had a competition to see who could collect the most M addresses and we had evil M who didn't even hang out with us at the conference he left he went all over New Orleans trying to collect map addresses and came back with like gigs of data and

so he was King of the con there for that contest but it was again about collecting information for us to just you know in order to find a needle in a Hy stack you have to build a Hy stack and so we're building Hy stacks and Bluetooth signals uh went back to giran made an 8 in floppy disc shoved a raspberry pie in the back of it and made an Eddie Stone CT of um that was a lot of fun as well and so I just wanted to kind of explain how sometimes it starts off as a game but it ends up turning being something real down the line but we went out there to have fun with

and I'll say that on each of the badges there was different code um some of was for pen testing Bluetooth l2p commands correlating with again Gap services and using rcom and things like that so the point was to build a Bluetooth penetration speed eventually so the next step here was to put another updated UI on our tool here see if you can see that on the left you'll see a bunch of devices it's really hard to see but they're being enriched by uh what device oi they are the vendor ID ssit and some of the weak data come out we also had the concept of um doing persistent pairing but in a d Dar dish type away and so you could

actually send how many times and the the delay on each pair packet and once you start doing floing Point pen starts spamming the stack and we noticed that uh you know we were affecting some of the security elements in Bluetooth um the other interesting thing was what are all these uh uids what do they mean um we can use in fingerprint devices if you see a heartbeat service you probably can read that and get somebody heartbe um a lot of the time we don't know what these are so we can uh search Google or we can actually grab uh mobile applications that work with some of these services and rip them open and find the uids to correlate with the

function um so again looking at some of the data that was being pulled in through the Bluetooth badge I started seeing weird things in my neighborhood um you know and doing oing around some of the devices I didn't know Prov to be fruitful so we have this ampo device and it's connected to uh an offshore leaks website where there's some type of weird stuff going on with that company so I want them to be tax suspicious going forward in in our op um also saw this this one prob of a w Electronics polish military and some crazy looking plain drums and I'm not sure what was there but we saw that in the neighborhood here's another good one um

CET I don't think anybody's heard of them but they're they make devices that overlay your Outlets to back whole data and so started seeing all this weird stuff so again why to start trying to CL ify some of it making sense of it so we updated our Bluetooth software as a team to make Bluetooth toif fire from Myer and we spent time doing oent and categorizing different devices we saw and so on our new software it's going to plot with colors if there's microphones they're yellow if there's cameras uh versus devices they might be red we want to know these things around us that might be harmful in some and so this is what the

the newer interface looks like and that is a 15minute view of Bluetooth map addresses or sorry this is the last 15 minutes but it's going to show this say PL all at the top there's a lot of functionality here anything that's been seen before 15 minutes it shows everything over all time that it has but these are things that if You' seen them in the last 15 minutes and they're new you see one do and of course if they been around you'll see this big long line which tell you again they're around you a lot or you can figure out what times they come and go some of the other neat patterns that uh Bluetooth creates um depending on you

know the length of time and uh certain things you're looking for and of course whether you're looking at Classic Bluetooth or blle they'll appear a little bit different on the gra so you can start fingerprinting certain scenarios around the radio signals from this data um this again was an example of doing top 50 math addresses seen all time and see I can see how long this one this might just be like a a couple of weeks or so but you can see patterns here especially with the the three dots correlating across the whole screen there and um you can see the red there we got yellow so that's again a microphone or some sort the red could be potentially something

dangerous again based on what we classified uh with our over um that's a one we sampling of 32,000 nap addresses that basically is just a raspberry pie hanging in my kitchen window in front of my house um all of the patterns you see that like a double slid experiment are things that we have seen repeatedly so every day we see these columns filled up those are things that drive past go each Direction every day um it's funny cuz actually the White and the reference whether it's day or night because traffic kind of stops at night and you don't see as much overnight so it's white that's in the middle of the night and where you see

the blue colors is when it was busy during you know hours we'll call it but it's a lot of data kind of get into Big Data aspect around it it um and really find you know get more if you want um this is a war walking this was me walking uh basically a mile and then a mile back and it makes an x pattern because you're seeing the same things again way back and so it's again it's a way for us to identify certain types of signals um spatial awareness just things of that nature reles whole patterns so again back to um the devices that we SE um this right here are a bunch of blue dots that are all Hue

balls and so in our software we can tag them all and group them together we can tag these items and I call them neighbors lights and I use a little Emoji like and now when I look at that device P the graph it tells me oh these are all the neighbor lights and you can see they're all correlated these are at the same time you're couple of them like to do a little bit different maybe that's a different room we can get this information from again the Bluetooth dark color softw search for emoji so we have Apple phones and there the Apple emoji so now you can to search for an Apple icon own emoji and it will pull up any

device that is a Apple for sometimes apples don't just get AR Bas off ID or you will see apple services and other devices so here is an a group device that apple and so now being able to correlate certain devices and vendors and how they uh work together in some way we can start building more tables and how things are connected even across from the vendor phas or again maybe somebody did some kind of custom app but we'd be able to figure that don't sign dat this is like a month worth of hearing that drove past the house um again you can just search for an ear in our tool to find these things or you can

search for again we have filter I guess I didn't really mention that but up in the upper right you see see it says the word here h e a r and that was used to pull up uh you know use this query to pull up all the and you can see a lot of them come in twos and some come in threes I didn't know why they came in somebody mentioned to me there's a battery p with your it will also broadcast so that may indicate that there's a power source along with the automobile same thing you use a car Emoji or we can search for different things that we see in the cars so that's

again um might be we worth um but again you can see different cars that come and go and start classifying as as maybe you need to expressive devices lots of them out there a lot of them are buggy this is about security and so right now we're doing info Gathering but this really is to help penetrate tests or you know get the blue team monitoring or try to get some kind of fireing around your device if it is insecured and so we know we have L of Express devices out there some of them are not activated they broadcast that through their uh through their name and so we can actually see when things are in a default configuration state by

also looking ATS to but some of the Nam away um we have sometimes where we have data we could just happen again for it's broadcasting so in this case we have an IP address being linked on the left 1921 16816 and then I feel like this is more of a tree advertisement than doc St Austin here but I did want to provide some real time data around you know if you have a device with your name on it it's easy enough to take that name and go search on Google and find a lot more about that person so in our case here Austin apologies but got pre- advertisement real estate region in Winter Park um if

you need anything maybe he's a good guy to hit out not sure but he pass the house several times um wanted to make sure that we um we do see all these weird devices with AI on them and there's no security including toothbrushes so like you know I didn't mess with my neighbor's toothbrush I'm not out here trying to do any illegal activity but I'm pretty sure that I can brush his teeth for whatever way I play or tell the AI to do that for him randomly uh some devices like Audi's have a a blube broadcast coming from uh their engine will call it and sometimes it's in a default state where you can connect

to it without a PIN and it literally tells you that by the name it says Audi SOS no 911 do not delete as a name and so we just have a lot of things that have been overlooked again in the Bluetooth World Part security um and we been a lot of pigs all over the place interesting um logo when I was doing my o so they just deserve it call Holy Stone in National um you need some kind of drones and whatnot they have the coolest logo I've seen in a really long time but seen several of these devices uh around our neighborhood and again maybe that is benign May something I looking for there I don't know that logo

kind of it's cool but it kind of scares me again back to some devices that um we categorize as being suspicious again based on homeland Security's definitions our own definitions things we've seen out the field and so I was looking through some of the sus devices which you can just put in the word sus as a filter we see back door 1B on one of these devices here and whether or not it was a default name cuz I've looked into it some actually have Bluetooth on all their doors so I was like is this really a back door from a car or is it a back door to some car and so those are easy

things to find when you P um here again we had more evidence that the Polish military is in the neighborhood uh picked up some more of those beacons and I used do the Wolverine slided from Red do for fun but this is probably the way that we would see you know maybe somebody come and take us over again R Stu but yeah interesting things out there so we get to the Amon devices now and tasers and fun things and so shout out to keys for finding the first taser with our Andi and so we didn't have it in this database at the time and so this one liner at the bottom shows you how to add

one line into the public Express database iated map address with um again our type of U set up our framework our o and everything else but you can get a oneliner just put a new device in and now we call them tasers got a little police car Emoji next to it and now we're able to track that and so we can track the police and their devices and they spit out all the U IDs and stuff that we can see you can actually match all that up online uh to see what those mean and so again you can either do po to with those things up or you can maybe go find some type of application that may

be involved in some of the axon software and use that to uh further and what those Services might be and this was an interesting one this was where one night outside my house foring like FL hours there was that many taser devices and at the end somebody geted like we have this at the end and so we can validate that 3 ID devices tell you when they are in different states what col it and so you are able to identify what's happening with these tasers um so again it's both sides of the fence you know our police need to be able to to do their job but we also need to identify them if they're not doing

their job so this type of stuff can be used for Brady list and making sure the police are doing their their correct job as well all Sid as the defense for security um apis are really powerful um and being able to integrate them in other systems and so our software does have API functionality um so that you can use it in multigo or some other type of ingestion if you will um into your own routine defenses things like that

so we'll talk about uh some of the red team usage real quick um I've used it for Red Team before just monitoring where we wanted to know what the activity was inside of a building a week before we went there and so we can leave one of these there and watch everybody for a week and now we can see when they comeon go we can maybe identify the security guards when they're coming to go so now we know we can go in or if there's a Target we want to be in there we can wait for them to show up um many devices in the corporations have their personal names on them and so it's easy

to identify whose assets are whose or what devices are what um and so again this is red teaming for helping people become safer this isn't Criminal Behavior but we are able to monitor people's activity kind of see when we should go in um the Bluetooth sta does allow you to do popups on other uh systems F computers and sometimes you can use that for fishing people will fall for popup that happens that directs them to some action we all know that and so that's actually a really minor use of of spaming or pairing with somebody but you can change the host name on your own device pair and it will use that as a message that's popped up and so if you

can change your whole same on the Fly and send another pay request you can have a conversation with people over these it's about Bluetooth pay requests um denal service is a big one so we did talk about floting point for the ched pairing or the Von V and you know what it really does is it overloads the stack a little bit it causes you to keep getting pop ups and maybe you don't anymore but that doesn't mean it's not talking to the stack um we notice that sometimes it will put our device as a pair device already and that you didn't accept that and we found out that's actually used in a lot of um

exploits in the past couple of years um like blue ducky was a tool that was around spamming the Bluetooth stack and then connecting over hid and sending keyboard commands and so you can do Rubber Ducky ATT meaning spam these interfaces um and I mentioned the pentesting speed the whole kind of the the evolution of this was yeah it was to find the party and it was to you know stalk somebody and that kind of thing but it let us into a place where there was real value around looking at all that data and then giving us ideas and how to defend against it as well um so we'll keep going here a little bit of course you can find signals with it

mention that if you're targeting somebody and then we called it the BL Canon which is our thing on ditching but more rapidly Al theing point and that is actually I'd say on three times it altered the stack of Bluetooth to where some device that was broadcasting after we hit it with the cannon I'll call it it no longer broadcasts and it no longer broadcasts after hard uh rebooting and things like that and so a cold reboot doesn't do anything um it confirm don't know why yet again there's some you know we're looking at some of the research there maybe they're really crummy Stacks they were resilient but we have seen slightly permanent damage on some sta

call with that tool so it's a little bit dangerous to use but we use it for again security and QA and looking for exploit uh some of the blue teames so running a honey pod to detect anous activity around Bluetooth is key just set up this phoning device or a series of them and see what happened and of course we can tie them in a gry log and learning and things of that nature so um honey Hots are really good um as I mentioned sending the logs to gry log for continu monitoring around your environment scanning for changes around your uid um open shars that type of thing um being followed it was something I didn't

really mentioned earlier that was put in so I kind of considered if I was moving and I had a Bluetooth signal that was hanging around me for 5 minutes that was almost too long for me and that meant I was being followed or it meant I was following somebody else and I need to stop stalking but it was really cool to have this follow monitoring because it actually worked for me I would testing was like there's a sign around me and it's oh hey what's up it's somebody I knew they were trying to get a hold of in here didn't see any NE perious uh type of you know info there but it's good now

follow um and I talk about the uid thing and everything that way the way we would use that too is you know hashing we're going to Hash values of things that we see and again WEA it's through uh pairing uh shars that are there the Gap staff whatever the case may be we're able to monitor for any kind of changes around the staff so that's going to be probably our next level for security next kind of proactive thing we can do um Wii so I started using this a couple years ago to see if I could identify a MAC address of a person while a do off Happ and see if I could figure out with

di our net and you can like if there's a pattern that happens over and over again you'll be able to correlate these other signal so getting Wi-Fi involved in this too and doing more analysis is key and we are doing that too um but again this whole project was really based around Bluetooth only and so let's see here and automation around uh the gpio triggers as well so with the Raspberry Pi you can use the gpio if you see certain signals it's like home aimation you can have a trigger events to happen um we use that to open our doors now open the garage and things like that so again for the blue team there's some use there uh

potentially around that functionality and also defending yourself we're not suggesting use the BL Cannon to fire it up at you know all your enemies but there might be a time and a place to do that spamming and we'll kind of talk about that in a minute here too um for me like security is very DST it is you have this coin and each side of the coin are almost the same you say blue team red team and none of that really either know how to do system aemy network engineering all these things are kind of dumb and so for for some of the realizations here that we again got to from all this research was device food

being like for doing man in the middle can also be used to protect ourselves so like we're going to spoop our own devices for firewall purposes and so we can take the same techniques that are being used from by the red team and turn them all right back around on R team um spamming so same thing I just mentioned that in and I think it was in the last talk I saw a geoping but if you want to protect your Bluetooth you're going to have to spam everybody else away from it and so we're going to take these same red team techniques and use them for blueooth um Geo fencing Bluetooth is going to be

huge I believe probably that's already happening in appropriate places um but as commoners out here we don't get that luxury so are we going to really be able to set up our own spammers to do G fencing maybe not but through our companies and trying to apply real security we may be able to leverage some of these things and maybe we need to um you know some things like maybe the real device has to be on low power and you have your own proxy then pick it up again in the same way a gap that ta work and you do man in the Middle with the end user so that the end device is protected those are things that probably

are easily deployed but it's not taking the next steps um again I mentioned there's no fire La when they r so it's pretty wide open um then I wanted to show this real quick too I think I have it with me so shout out to OD Simpson um I did a version of his talk at hushon Cater and he introduced me to uh This brilliant labs monacle and he he just put it on the glasses and it's kind of screen inside and so now we're able to take our Bluetooth code and put it into this and see BL so we're able to like look and see Bluetooth signals devices we can see a card we can see this um one button can

take a picture one button will record your voice and so you can use voice commands to Spook Gap or this or that and so it's a really neat tool to visualize uh signals we've seen some of that before out there already like in the Wi-Fi world but just to be able to provide advice now you know we're getting out the eaing add AI this also ties into uh GPT or local models for you to do additional tabs so I've been working with doing our o on the Fly we don't need to do it in a file anymore we're able to look at a device and maybe get some real info back from the internet and it Scrolls like Star Wars in your

high it's really crazy and so these are really neat um and other you know again new trying to think of new ways to either visualize Bluetooth again for security sfe or curiosity sake um it's just a neat a neat thing uh the other thing I wanted to mention again with the doas part was you want to check on the undersides of all of these protocols and services Bluetooth is kind of a mess right now a lot of bandaids are on the standards and there's like a whole world in Bluth St there's a there's dialog networking in there you can literally connect over archon and do atdt commands and things like that like we're in the

'90s again um you are just so much in there service Discovery protocol will show you that you have FTP shares you can download contact address books and things like that and so I think for like the last 10 years that's been a wild west maybe in the last five we tighten some of those things up um it's harder to make phone calls through other people's phones it used to be a lot easier so some of the security of the stack is getting better but we know again through talking through lower level protocols like C command or using scappy to create your own frame even around an advertisement um that's where the spamming comes from when we just sending these frames to a

Bluetooth device with a certain data set on it or configuration and it responds accordingly so in order to talk to Bluetooth we don't have to have a Bluetooth stack that's really Advanced we just need to know the flow of the conversation in order to get real quick to where we want to be and the reason I bring it up again is because the whole stack is really vulnerable um spamming is opening some things up it allows you to connect devices that you shouldn't you know be able to connect to um and you're also able to access things like hid or hiid Draw uh HID RAW and and talk to those devices directly through Bluetooth whether you're authenticated

or not and the other thing I'll mention is again a lot of what's old is do again we keep saying that so we had to think called the nav attack where you can negotiate your pre-shared key with Bluetooth um and there was a vulnerability to where you could create or ask for a key of one white adventure and the rest of it would be Pat so they put a control in the uh ACI stack and a couple other areas where you can't do that anymore you only can ask for a 7 to 16 bytes key however if you recompile a Raspberry Pi and you change S.H file and you put that back down to one you can connect to Windows 10 boxes

cuz in the registry they have a setting for one bite of entropy that you have to manually go fix and so we do see things like that happening where we can take a again recompile version of the Bluetooth STA and connect it to vulnerable devices and nobody's really fixing that so these are all the things again we wanted to make sure that we're you know bringing up things that we found out there things that we we found through the game of vacation of Bluetooth if you will um and again it's the full cycle you know you can find these red team techniques but then you can turn right back into blue team um and so I really enjoy that cycle

and turning you know threats into defense um just a couple more logs here around the gry log and you know if you par the uh Bluetooth correctly this case it was a little messy but this was Rick's cavet from M uh you know again we just walk around all kind some of the type of things that we saw some PayPal and some commercial equipment Apple devices that had the word infected for their uh for their name and so those might be things to look into just like Square readers are everywhere and pain devices again like I mentioned medical deves so we have to remember to keep these things secure um some of the uh scenarios we have out here aren't

really a game they're pretty damn serious and so we started trying to find the party we did actually but we found a different party out here um shout out to Rex went to and Z con but we are able to find you know again anything we want to move to into preservable and we do have these tricks to bring things out from the Shadows if you will by creating the right frames so um really going forward we're going to have to try a little bit harder with the Bluetooth staff firewalling the spoing for Blu team defenses and things of that nature and that's kind of what we found out through our research so we just wanted to share

a little bit here today on bluo stack um if you want more info uh we are lost RIT Labs you can reach out to us or email we have a GitHub and I'm certainly happy to take any questions on anything I presented here today so thank you I kind of went through things quickly but I wanted to uh see if there was any time for questions and things like

thater I think so yeah um because this spamming has been going on for years it's nothing new I'm not the only one doing it either um we all have similar thoughts as hackers how can we break something they do something funny or but I do believe yeah the Apple spamming and the The Flipper and what was it around last Decon everybody was getting spamed not this year but last year and so I think like from then we had a lot more people who were really aware around that um people who hung around with with our group were getting popups a lot in their lives and they were used to it but not everybody was and so I think it made

a big impact that they became more of a national or even inter toe that is there any kind of like wiggle not right now we on doing that though we wanted to release this so I can like R party XYZ and things like that they do that and we have wanted to play with them we've just been it's been private research I guess also you know this is what we do for a living also we do these type of penetration tests but we are releasing a penetration testing to be around Bluetooth soon we're working on that to finally be we just didn't want to put out too many tools that dangerous yet right we place

now where we probably have to play C J up and so we would like to share some of that info but we have

I don't know if I should answer

that the answer would be yes I have been in hospital for the last 10 years and there are times where I may have my stuff turned on obviously in a pass away only um like all devices are right it's the pairing that so the pairing makes yeah the passive yes done that and we already know like there's a lot of out here and so that's why again we're giving this talk um I'm not trying to expose things to make CU dangerous uh the last time I gave this talk I somebody from the military come B I don't know for sure but he saw something and so like it's about bringing awareness to whoever needs to have it

here um cuz we're not looking at it all the so yeah medical is really really affected right now when you look through the modle what does what so it's back to the Emojis and back to the map address and a couple the pieces of Linko and the menus will cycle through things where you kind of see some of the uids or some of the data and it's kind of basic we'll say but it's NE that again like you have this thing on your eye with a screen in it and it's not like it's directional I can see oh there's the car over there but I know it's around me and because it's RSSI based there is a way to get directional

a little bit there is there's some tricks we could do on the board to do that and that that would be nice Lev so um but it's really about seeing the device um if it's suspicious maybe even just to have awareness again if I see 17 taser show off I'm running but yeah um it's just so got a little bit of a visual just so you see it in your eye versus having to have a screen in a computer so very similar have you look a try ulation getting multiple of these devices together yeah yeah and I had a slide in there where I had four in my house right so we were working on our badges so this is for red

te we do have a version of this badge for red te that isn't built all the way but I've used the code on Raspberry pies but yes we use it so three of us can wear them and we have like things like PS with radio on them that we can drop down in the building so we can see when somebody's coming and it actually shares and we can triangulate so yeah you can triangulate with these the algorithm is pretty basic as long as you throw that algorithm in your data set it's pretty quick on the fire to tell you yeah we think it's you know keep going this way so yeah it's a really cool concept and

that's I mentioned too it's XY and Z so now we have full spatial awareness whenever we would want that right it's not like we always need that but if if we are prot in a Zone we we would want that and I would argue again you want Geo fencing in a lot of these places which we always say jamming is illegal but there are legal and appropriate ways of using them I guess I'll call it in order to defend yourself in a real way any other questions well thank you so what is service the Nordic Ur it could be many devices but you could try to connect to that and talk to the UR like a Serial

board if you have NRF connect on your phone you could go connect to that and maybe see what services uu IDs it has um you might be able to go change those values sometimes there's common ones like you know the alert um and you can change that to it's binary you know zero or one kind of thing and if you turn one you'll start hearing a noise in the room like sometimes we have that's how the tiles work in The Trackers they have no word on them there's actually part of the Bluetooth stack I won't call it hidden but there's a certain listener that is involved in find your device and so that is something that you can access

on Blu device and actually cause the alarm to go off yeah you use NF connect on your phone or again Bluetooth control or Gap tool or some of the other to Comic BL uh [Music] staff thank you sir all right thank you everybody appreciate [Applause] it next talk 10 15 minutes we not going to want to miss it so I'm I'm pretty sure I I was I worked in law enforcement for 6 years I'm pretty sure that what we seeing are the body cams actually so taser makes taser axon is the is like one of the largest drips body cans with BL totally so yeah I I did some work with them before and so again like some

of the uu IDs will actually give you an idea as to whether it's a taser or a body cam or another device in the car some of the tasers have a camera on them too they do yeah there were some of the old ones did it maybe some of the newer ones do I'm not really the loop with their new tag this is like 6 years ago that I was kind of more I'm not involved in like trying to take on it was more about like it was fun to kind of show like how anybody could go from zero little Google searching a little bit of whatever it's it's that's incredibly fascin I didn't even think about that

and you know the way that you can then determine you know especially talking about placing down a be terning if somebody's coming if you know it's a security guard and you doing a red team engagement and you know that you you know you fingerprinted their phone and you know that where they travel and that's fantastic I had a I had an undercover officer I lived by at one point in time and I told them dude I can see you [ __ ] coming I got you ha you need to fix that I'm about helping people this be this be I was a parole officer so I was undercover car all the time so this type of information like

knowing about that and you know because there were times where we were caught on CCTV cameras from the bad guys and that was an issue the radio signals worse right now this [Music] ised he's going to do

that this should be the clicker part

[Music] oh you got a clicker for me too yeah they don't F oh you try that one I have USB there they give they give you one usually takes second awesome you Al that was e hey that was easier than I thought it would be perfect um and uh Mike how's sensitive like do I have to like yell into the damn thing it's pretty sensitive yeah you're going to be like mouth against it but close I think is

good the I think we're good [Music]

[Music] yeah maybe [Music] yeah you like that I I uh spend minutes on Mid [Music] journey I want to talk to and see if I can run audio next year like the signal coming out of these lights is so

need yeah if you want next want all right cool assuming this [Music]

works everything works appreciate it nice good [Music] DT said yes yeah DT said that the speakers were supposed to be directional but they weren't for some

reason instruction people shis Wireless syst they're on their own like joint and you can adjust for for Stuff just having like a couple of better mics that uh are more directional so like front facing or even like La those would have to be Wireless they have to [Music] go a lot of don't [Music] have I I would love to cuz every talk I go to and I'm like some people can't hear people around like let me just mic them up just want to go up so yes next year roughly the same time frame we do a call for volunteers 6 months in advance something like that um yeah I'll rent yes we're interested I'll rent all

the additional equipment that I need cuz some of the stuff is expensive yeah like I've got some stuff and not like I don't have a production company um but I thought about like starting my own production company just CU I've been in enough places where I'm like the audio here is terrible let me let me fix it please it just amazes me rooms that are designed for presentations have AIO video usually PA systems I have no idea what you said yeah you defin need a person in the rear that can like live mix you know so that like whenever four guys are talking you can be like oh one guy's talking one guy's talking you don't want to have to unplug

and replug stuff all the time mess with these dials cuz the the signal coming out of these xlrs is so hot I was like oh I need I know Jimmy talks real quiet so I need to crank the game but then like I crank it too much and they're like oh can you turn off their mic that's my bad that's my fault and there's some really cool stuff where I got the idea from DC uh devcom is that you take the audio and you run it through a transcription engine like running on a laptop or something and it'll spit out Clos captioning and then you can project the Clos captioning on the screen for the hearing impair yeah

I'd like that idea some of that stuff is wild I had to record something for teams day for work and it just like transcribing while hocky and I was like I don't like this it it works better with some voices than others like I remember Cory Doo's talk like the close captioning cuz I asked the media guys at DC I was like hey how are you doing this close captioning cuz it looked like for just his talk someone was like stening cuz everything was capitalized nothing was misspelled the periods were there I'm like yo who is typing this I'm like looking around and apparently they're just running it through twitch and rebroadcasting the twitch stream oh wow

uh on the with the captions so I was like yo whatever twitch their close capturing Services good I feel like that would be a good use of actual AI wear for sure people be like oh yeah this was a good close caption there are some good uses for AI there stupid uses for not a lot St and there's some bad but there are some good yeah I would love to you know run audio run a camera yeah just reach out and let us know I want I mean and and honestly if you tell me now I'm likely to forgive yeah I'll reach out to you after death next year that's about the right time frame definitely okay make sure people

can hear and have a good experience minutes better you yell than me so what's your last name hon okay [Music] cool to the papers I'm sure you do that every

[Music]

year I like I'm [Music] intimidation download some firmware and maybe buy [Music] device cuz you almost have the answer right and then you can kind of see how you would get there and kind of check against [Music] [Music] oh [Music]

nice couple down south there [Music]

get

[Music] [Laughter]

[Music] that's the hardest thing is like structured unstructured [Music]

[Music]

tiet

[Music] [Music]

[Music] so yeah we're even just setting up

[Music] I [Music]

[Music]

[Music]

c a lot something lasts [Music]

[Music] service microft [Music]

[Music] it off got itroduction and then you should be good so yeah a couple minutes [Music]

[Music]

[Music]

when I came here day oh yeah [Music]

[Music] doing very well particular we sold [Music]

[Music] let's see which one Jimmy that was the last one you are Wesley I'm W Wesley I actually have a decent shot not one West that doesn't make it any all right ladies ladies and gentlemen Welcome to our 400 PM talk API security nope ai aii ai security API you said API okay my job is definitely stuck in my brain AI security everything old is New Again by

Wesley so if I just shout and project my voice can everybody hear me no oh is this a sarcastic no or a uh okay all right so can everyone hear me now fantastic all right I'm going to talk about AI security because we haven't been talking about AI security since like you know early 2023 at this point but I think I have a new spin on it or a different spin um than a lot of the AI security top that you guys have probably seen at ducon and black hats and whatnot um and at the end of the day common use cases we bought a new AI tool fantastic uh I hear that at least daily

in my day to HUB right now um I just hired a VP of AI development I don't know what the guy's going to do uh nobody knows what he's going to do but the CEO the CIO the CTO hired a new VP to inject AI into all look our products fantastic okay AI is being added to all of our products we don't know what the AI is going to do but we're going to add it to the products because it might bump the share price up or because it might give us a Competitive Edge or because we don't we might get left behind if our competitors figure out a good AI use in the uh the in our sector in the

space we just signed an agreement to with an AI company this one's even scarier than some of the other ones um so you know hey we made a multi-million dollar commitment $10 million1 million commitment with a AI company we don't know what we're going to do but we're going to trust this AI company that's probably 15 guys that live in the valley or something to go ahead and Infuse AI in our products work gave me the day off to attend that that's also a valid reason of why we're here all right so uh why I know what I'm talking about generally um I have been working with AI products since 2016 2017 is um I am not

new to this space uh I do work for Microsoft I'm a senior manager I'm actually the senior manager for security services in the Americas um so this is something my team has been dealing with with a uh for a long time uh the little timeline in the back which you guys probably can't see because the projections really really small um is kind of how long we've been involved with AI in my organization um going back to 2016 I know chat GPT blew up and it suddenly became this big thing that llms and machine learning and whatnot is going to take over the world we've all been experiencing llms we've all been experiencing AI anduse produ uh products

since like 2010 2011 2012 the really bad chatbots and stuff that you got at you know your bank or your credit union or your uh insurance provider or whatever ever some of those products were using early early versions of what we were now referring to as llms in the industry so a lot of this stuff is not new um and it's kind of funny because I woke up like everybody else uh sometime I don't know March or something like that of 2023 uh maybe even a little earlier than that and it turned out to be like oh llms are taking over the world it's the hottest thing AI is going to you know solve all our problems and do all this

great things and I kind of like raise my eyebrows as I'm in the middle of a llm project at a major bank that was been going on for like two years previous to that so it was kind of an interesting uh interesting Evolution um that we've had so that's a little tiny bit about me it's a little bit about my background uh but let's actually dive into it um also for the purposes of this presentation I'm focusing on generative AI um that's what most people uh in the room and probably most people in the industry are dealing with at the moment um so I'm talking about conference call transcription chatbots insanely powerful search engines if you get good with chat GPT

you never have to actually use a uh use Bing or use Google or use whatever uh ever again digital assistance text documents a lot more stuff because it's still early people are still figuring out creative use cases like this um many of the principles I'm going to talk about this presentation could be applied to other AI Services other AI platforms other AI capabilities but it's not a one for one like anything else if we find something that works really well with generative AI can I apply it to Optical stuff can I apply it to visual stuff I don't know maybe maybe not um something to think about so let's talk about common common scenario we bought a new AI tool um this

is not anyone tool this was 2 seconds of mid journey to generate a symbol right there so I'm trying to be very agnostic about this um so I've got new AI service let's connect it to a whole bunch of stuff let's connect it to our GitHub yeah let's give it all the source code information let's connect it to Dropbox let's connect it to a whole bunch of Office Products and Google products and databases and AWS um buckets and whatnot so sure great fantastic um I see this on a daily basis um in Corporate America right now we bought an AI tool we're going to connect it to a whole bunch of arbitrary things out there some of this

stuff might be super sensitive some of those databases might contain public health information some of the databases might contain PCI protected information um your source code at a company that writes code do you really want to give it some random AI vender access to your source code it's something to think about there's risk um involved in any of this we also got to think about we have an AI service it probably lives in a cloud somewhere the the AI service is hosted in a AI service data center it's hosted in a cloud it's hosted who knows we don't really know um unless you're really start to dig into this and we're going to talk about what digging into

this looks like what is that AI service connect to is it connecting to open AI or llama or pick your pick your third party AI service something else to keep in mind just because some app vendor has an AI service that doesn't mean the AI service isn't connected to some other AI service to actually provide the large language model to actually provide the services um other thing what happens when the data leaves the AI vendors data center is it going out to something we don't know I'm going to talk about a use case where we found data going out to some random place um super super fun super scary very sensitive data too uh so let's talk about three data

flows actually four on this one um I'm big on threat modeling and I'm not going to sit here and give everyone like a uh class on how to do like Enterprise architecture threat modeling that's really boring and probably not a good use of anyone's time here when I say threat modeling in this context at least I mean I want to identify some threats and I want to identify some mitigation so I'm keeping it simple I'm not using so if anyone in here is an EA or anyone's a thre modeling expert I'm not using like stride or any of the like a stride or toga or any of that stuff I'm just using like some simple explanations

for threat modeling so here's our four data flows uh which we're going to cover each of these four data flows for this case Okay so data flow one we're going from the AI to the data sources the data sources being your SAS platforms your code repositories don't really have um uh uh to define the exact data sources for the purposes of this presentation sure as heck have to define the exact data sources when you're vetting a new AI service what does this AI service actually connect to should be the first thing out of any security officer security practitioner's mouth when you're talking about potentially evaluating a new AI service um so let's talk about a couple prints um internal

exposure of data so this sharing this is probably one of the most prevalent um uh threats that that we've seen um my team have seen uh at Microsoft I've seen in the industry I've seen other organizations struggle with we all do a really bad job at Access Control around data and data protection and DLP and labeling and a lot of these things have been super hard for organizations of any size you think about like a 50% organization and it's really easy easy to maintain Access Control help one person can manually do that for a 50 person organization gets harder with a 500 5,000 50,000 100,000 person organization this task becomes impossible at scale so when we think

about controlling access at the um uh the the threat hasn't changed so if I have too many permissions to SharePoint sites and drive and uh uh S3 buckets and shares and whatever else I got in my environment as an individual user I can manually crawl through file shares I can go search on the internet if you guys have some sort of Search tool hooked up I can do all those things that's always been an issue it was an issue a decade ago it's an issue 15 years ago with AI Services it makes it a lot easier to go ahead and one person can suddenly um suddenly go ahead and dump a whole lot of data really really

quick uh issues we've seen um HR databases having poor no access control suddenly the uh AI Service uh can start dumping everyone's HR profiles things like pay things like addresses personal information um health information all that fun stuff that HR might have been collecting on the along the way um the mitigation for this potential oversharing 100% Access Control Data protection capabilities DLP capab cap abilities um notice I'm not really saying a specific vendor or specific name there's lots of different ways to do this a lot of times this is very um data uh source so your Access Control capabilities in your Amazon S3 buckets are going to be different than your Access Control capabilities in Dropbox

and you're different than your Access Control capabilities in get GitHub or something like that so it really is understanding the access model on a service by service spaces which is no fun it's tedious and there's not really a silver bullet for this type of stuff um let's talk about external exposure of data I have an AI service that's internet facing it does some cool chatot stuff um if I have a customer able to freely crawl around file shares and and databases and access to information that they should never have um that's a bad day that's a very bad day when you're dealing with health information card um sensitive data Etc so external oversharing um definitely a threat uh

mitigations look the same by the way um again down to access control down to whatever sort of data protection capabilities whether it's some third party tool some sort of built-in native thing in the platform it's a cloud platform um there's different ways you can go ahead and Control Data um data loss prevention labeling depending on the platform we're talking about um intention external exposure of data uh this is something that we've been concerned about for a little while so uh we think about an attacker getting living off the land we think about an attacker um doing things to go ahead and abuse your local resources what happens if that attacker goes ahead and starts uh undoing access

controls what happens if they start being able to use your AI service to query databases that that AI service should not have access to what happens if they undo data protection or DP capabilities what SIM in the world is probably going to catch an attacker that is is dumping prompts of sensitive data that you think is actually secure and control so something to think about um when it comes to an intentional um exfiltration of data sources this is another creative one that instructions and uh add a billing person a clerk um is going ahead and taking those wiring instructions and getting them from a GPT service getting them from an AI service hey give me Navy federals wiring

instruction bam it just saved that person 5 10 15 minutes to go look it up fantastic it made their life easier what happens when an attacker changes that Navy Federal wiring instruction to point to some bank and uh check's lucky or something like that that might be interesting that could be a bad day for a bad day for someone in the organization again when we think about the the actual mitigations to these threats uh very real threats unfortunately they're all the same and it all boils down to access control and data protection capabilities um unfortunately nobody has some silver bullet or magic product that makes Access Control super easy um this is like a real engineering and architecture

challenge this is understanding the platforms that your AI service is connecting to and understanding um how they go ahead and uh interact um in your environment so it it is very much a problem that large companies are facing on a daily basis let's take a look at another dataform all right so the provider is training their models with my data um this data flow is really about the AI service to the provider so data center Cloud whatever the back end for the AI service um this is really really scary and we are seeing this pop up all over the place um you really have to dig into and we're most of us in this room

probably osick people they probably we're probably security people we're probably fairly technical most of us I would Hazard a guess probably do not routinely participate in the vendor vedding process that tends to run through procurement Department that might run through legal compliance privacy there's usually all sorts of people at large organizations that really care about vetting software sometimes there's an info set check box where someone in the uh it security or security organization has to check a box and say yeah we think this is secure some yeah sometimes there is sometimes there's a actual betting process where people are requesting source code people are reviewing the uh any of the slas licensing agreements stuff like that

from a security perspective I'm not talking about contracts or a legal perspective um this is something that I highly recommend um infosec folks start leaning into when it comes to a lot of these AI tools again though this isn't new this is the S the proliferation of SAS in the last 10 15 years you run into a lot of these same problems in in traditional SV products hey I'm storing my data someplace let's take a of the equation I'm storing my data on a cloud or I'm storing my data on some you know cool startup thing that does some awesome widget for us someone should really look into what we're actually agreeing to um and in this case training

uh with your data Maybe it's acceptable maybe your data is a bunch of public stuff you don't care about um definitely comes down to risk comes down to what the AI tooling is actually connected to and it comes down to what your organization will agree to um so training is something I highly recommend everyone lean into as you look at new AI tools vendor AI vendors should specifically say if they use your data to train their model the safer answer is no we do not use our your data to train your model or sorry our model if there's a different answer if they want to put some Nuance on it if they want to like

spin it around you really got to answer that core question okay second threat fails to secure my bre uh my data at the infrastructure layer layer and results and a breach um this is again Common Sense problem nothing new uh something that we've all struggled with um mitigations Access Control what data are you actually giving them access to if there is a breach at the vendor is this something that is going to negatively impact my company is it going to cost the company millions or billions of dollars is are going to put us on headlines that some AI tool was breach and suddenly my company is now in the news because we had a big public agreement um and we're

sending out those lovely disclosures that say e one of our partners uh might have lost all your data we're sorry um not fun um something that needs to be carefully vetted but a lot of these AI tools tend to be black boxes so it comes down to slas it comes down to Legal agreements it comes down to um business relationship how many people actually pick up the phone and call the AI vendor and say Hey I want to talk to your security team I want to talk to your sist I want to talk to some you know director of engineering um sometimes it happens sometimes it doesn't so the provider exposes my data to other users um this one's

particularly damning because a lot of the AI vendors are specializing you can buy some cool AI tool that does something neat in financial service guess what all the big Banks insurance companies whatevers are going to use that AI tool do I if I uh and I'm just tossing out generic names please do not read into this if I am JP Morgan Chase do I want them to access Bank of America data through the AI tool and vice versa it's probably a uh probably a bad day for both Banks re regulatory issues competitive issues um it's not something fun so how is the AI vendor actually men in and separating what controls are in place that will keep my data um from

ever falling in my uh competitor's hands about the next data flow so AI data center to third party AI service and this is not to pick on open AI that is the open AI symbol but you can replace any any AI service there uh they just happen to be popular right now um the AI service is training models so again this is very very similar except instead of the AI app vendor it's the AI service something that needs you need to get to the bottom of fails to secure my data again very very similar uh bre and then exposes my data so not only do we have to worry about the app itself we have to worry about any AI Services it

might be using so a lot of these AI Services actually are not they're not processing your data in their data centers or their Cloud environment they're sending it over a we're assuming secure connection to open AI or llama or whatever X calls their damn AI service um and they are so your data is not just existing at the app service it's existing at the AI plane as well um and somebody else's cloud or data center or platform that you have to kind of trust to make sure that um they're doing the right thing again slas licensing Contracting all right this one's a fun funner one um there be dragons so this comes down to reading the

agreements I've got a fun story I think on the next slide about this uh actually happened so AI service is selling my data to third parties um this 100% happens I have found AI Services which I will not name because I'm not here to like throw anyone under the bus um that 100% in their agreement it says yeah we sell your data to third parties if your organization's okay with that fantastic if it's your organization source code for a major project probably less fantastic um AI service leaks my data to someone else so this is just unintentional exfiltration um very similar to the other threats uh a breach causes regulatory impact actually let's talk about this one because I actually have

this but uh ciso of a Fortune 100 brought this one to me not too long ago so regulations um regulatory bodies have been an overdrive in the last two years if anyone does not deal with GRC type stuff um I'm envious and fantastic um for those in the room that do have to deal with regulators and deal with uh compliance to regulatory Frameworks they have been cracking down on for L practices has been very widely publicized that several cisos have been targeted with personal legal charges because they failed to do the right thing they failed to r a s organization recently um if my AI service is breached and it has a bunch of sensitive data is

my company uh impacted from Regulatory Agencies there are multiple incidents at the moment where people are actually exploring this and I don't have the answer because these are very much ongoing things um if your company has an agreement with an AI service and you have a whole bunch of public health information or banking information or financial records um and that AI service is breached it the AI service is in a all bunch of trouble your company's probably in a whole bunch of trouble as well um that is not an official statement yet like I said these are rapidly developing at the moment because this is all real time as we're living in it um but I would highly highly highly

let's talk another kind of recommendation figure out if the risk of using any random AI service arbitrary AI service with regulated data is worth it um some companies might say yes maybe it does something fantastic maybe it makes you a billion dollars a year maybe it's not worth it um something everyone has to consider so this is fun little story so actually happened slightly paraphrase just to fit it all on the slide it was actually much more panicky um hey need need help with something fantastic sure what's up we bought a new AI transcriptions tool and I have a concern it says they will use our data and this is the uh exact purpose uh wording for training

and Commercial purposes and how do we stop that good luck don't use the tool yeah good good luck was probably my initial thought actually so think about um freely sending your source code sensitive data by work documents doesn't even matter what it is to some third party SAS vender AI tool doesn't matter um we I don't have't answer like I can't Shem something in there if you're sending them the data you agree to send them the data if you don't want to send them the data fantastic in the case of AI transcription I can't this was like Zoom routines or something it was a regulated company that was basically sending all of their potentially senstive regulated

conversations to some third party startup and wherever I don't even remember where they were based out quite frankly so since data regulated industry yeah the some CTO bought some tool and we're now mandating transcription and all conversations kind of bypass the S bypass security probably not a great place to be in yeah this also by the way I think that tool cost like seven Biggers on off a million 2 million 3 million for whatever the subscription was for the tool yeah they still had to pay the dam subscription even though they turned the tool off for the uh entire organization so talk about uh waste of money so moral of the story and and I kind of hit this before V these

things so legal compliance procurement set privacy uh whatever facilities and capabilities you have in your organization to actually review um some of these tools especially things that could potentially have broad blast radiuses um yeah being told like hey all of our all of our conversations are being reported via AI doesn't matter what it is probably not a great idea um so that kind of wraps up the first scenario I want to walk through and I am actually pretty decent on time this one's a shorter scenario um and I do have a couple salum points and then I'll try and save the last like 10 to 20 minutes for questions oh not used to this SP all

right I got an app it's an AWS app um it runs some Lambda functions there's some S3 buckets there's some containers or something in the app doesn't matter just arbitrary hat fantastic let's add AI to it all right so we're going to go inner agreement because some CIO or CTO said yeah let's use this aure AI thing the Microsoft sales people were super great uh all right let's connect the app fantastic all right oh got to have users so there's users somewhere in the environment they need access to the AI service they may need access to the app doesn't matter what the access looks like users need access this little triangle as I think about it is kind of

the typical thing we see right now app hosted somewhere on on Prim AWS don't care Azure Google I don't care AI service somewhere doesn't matter and you're on Prim Network whether that's remote users whether that's uh um you know traditional offices if anyone actually still to an office nowadays doesn't matter for data flow purposes one we're going app to AI service two we're talking about the AI service that now instead of a vendor being responsible for the AI service we're all responsible for the AI service three on Prim to AI service so app to AI service let's talk about a couple quick threads these are these are actually really should be straightforward app to AI

service data is unintentionally exposed because chances are your app and your AI service is going to have to communicate over the Internet it's 2024 um I am going out on 11 saying everybody in the room should know that we should use htps and whatever other secure protocols you need to use once in a blue moon we find people not doing this and it's pretty basic and it's 2024 so unintended exposure over the Internet could be an issue um secure protocols obviously the mitigation there app to AI Service uh traffic may lead to regulatory impact so you you fire up an AI service and haher or AWS or I don't do gcp that often but I'm going to assume gcp if

anyone's a gcp expert you can correct me a lot of those AI Services aren't necessarily natively secure it just it's an API inlo it does some things there's not necessarily controls around it this AI service is exposed over the public internet even if your data flow is encrypted let's say the AI Services natively exposed over 443 fantastic we all know that communication over 443 for all intents and purposes should be pretty darar secure Regulatory Agencies get little little fussy about that if we're dealing with highly regulated data they have things that say oh everything should be over private networks oh nothing should be exposed over the public internet etc etc etc depends on the regulation we're

talking about so private networking API security role based access control all of those things will potentially go ahead and make that threat of your auditor saying who why are you communicating with this highly sensitive app up to some AI service at the moment um definitely something to keep in mind natively these AI Services may not be as secure or as regulatory compliant as you would like them to be depends on the industry let's talk about the AI service itself so AI service fantastic I got a little little icon it represents the service what about all the other stuff um this happens like weekly for me right now customer does this hey I did Azure AI I'm going to go integrated it into

all my apps and stuff fantastic what about all the other stuff what other stuff oh wait we're back to ah API Management Services how are we managing it ah how are we storing stuff if there's any sort of persistence there ah how's private networking done H what's Key Management look like who's provisioning you know what's Jenkins look like or pick your Dev off's tool of choice who's provisioning all this stuff you guys didn't use anything in Azure or Google or AWS or wherever spending up your AI service six months ago so who's actually managing all this stuff uh that this conversation happens on a very scary regular basis at the moment so thre uh in security AI infrastructure

leads to public exposure again secure configuration arback private networking those are common mitigations for that public exposure insecure devops practices lead to bres uh at to reach fantastic we're going to spin up a brand new Jenkins instance or GitHub actions or whatever um again pick your poison doesn't really matter who's securing it who's managing it is anyone monitoring it what's going on with that um lots of this stuff gets uh forgotten about because every company wants to move quick with AI and it turn turns into quick quick quick but we're not doing basic hygiene we're not doing basic security controls we're not doing basic management type stuff um infrastructure scw creep over time this one's actually a big one I do deal with

also in a daily basis so we start off with this Ali service and everyone's super happy and may let's maybe everything's actually locked down and this all looks good um and it is it's a healthy lockdown controlled AI implementation what happens when the next five divisions of the company or 15 app teams decide to do this are they stamping out standard patterns to secure all this or are they just wild Westing it and doing it as quick as possible because they don't want to be left behind lots of app teams right now are not standardizing stuff they're not sharing they're not playing nicely even within the same organization they're cowboying and cowgirling it up um as

quickly as humanly possible to get an AI service spun up and they're not writing this down they're going back to the best of 2000s practice of uh send it and forget about it um same thing with uh thinking about management somebody actually has to manage these Services long term it is not set it and forget it when it comes to most of these AI Services you're literally building an application uh it isn't Magic um a service abuse goes undetected monitoring is forgotten about it SC scary amount of the time for some reason um both infrastructure level monitoring and then that prompt level monitoring and I know I I've set in on one of the conversations this morning

and some of the gentlemen actually brought up some good points around um prompt monitoring I'm not really digging into prompt monitoring here but most of the AI Services there's ways to go ahead and capture prompt and response information it's not native though usually you got to go turn stuff on you got to configure stuff you got to specify some log targets like there's things that have to be done um the these remember these are all building blocks to you know doing the right thing you can make a really really lousy stack of Legos or you can make a really cool stack of Legos um so let's talk about AI service to onr um this is again similar similar

issue here you're on Prim to whatever your AI service is are you using a secure protocol or not um are things being passed in Click text again pretty basic still running tissues with that flat environments may give attackers a way to Lally move into your AI services this one's a fun one um because lots of big organizations especially Legacy organizations the fortune 2 300 that have been around for 20 or 30 years they have big flat nasty networks with very little segmentation inside them think about crown jewels like I'm going out live and I hope there's segmentation and there's controls around Crown Jewel type application sensitive data Etc should your AI service be treated as a crown

tools type thing maybe maybe not if the AI service is I don't know making pretty pictures or something probably not your poor business if the AI service is supporting one of those crown jewels applications or one of those important business critical things you better start putting some segmentation on PR to keep attackers from moving into that AI service they shouldn't be able to arbitrarily compromise some random Deb's account or sis admin's account and suddenly they have keys to all your AI service just like they shouldn't be able to compromise some random de's account and they have domain admin or Global admin or gcp whatever um so something to keep in mind ah attackers may start to use your AI tools

against you I have seen this is interesting um so uh attackers that maybe get into an organization that is embracing an AI tool and maybe they're actually doing the right thing with the AI tools and it's pretty locked down but the identities aren't locked down someone gets fished or smashed or whatever I'm going to start crawling through your organization using uh M365 co-pilot CRS that's a bad day for uh for an organization better make sure you've got data protection better make sure you've got guard rolls around your sensitive data because now you know how we can all move through our share points and drives and databases really quick using tools ters can do the same thing now now so we

have seen them all right so I hit this actually exactly in 40 minutes which is nice um I'm just summarizing basically and and clippy that was 10 minutes of my life on Mid journey to come up with that is thaty live yes so is apocalyptic flippy um so in summary um AI services are here we need to do a better job at vetting them we need to do a better job at Access Control labeling data protection when your organization starts to say to develop their own AI tools we we need to all do a better job at ensuring the good development practices infrastructure security practices monitoring practices all the things we all know about um are applied

to those super critical break neck AI projects that are bypassing all the normal security control and you know good good software development practices because somebody high up the company says we need AI now um happy to take as many questions as you guys want in the next 15 20 minutes so sir one of the biggest problems I've seen is you know talk about working I invol process they involve security the problem is getting the vendor to have that transparency as what's there including Microsoft digging into co-pilot how is it big terms says it's not going to my data out understanding what controls are in place and actually that people like visibility have no visibility they say

no yeah controls you guarante you you choose to do business with the vendor or not do business with the vendor um including Microsoft itself I know who cuts my paycheck at the end of the day um but there's very clear document I can only speak to my own personal experience but there's very clear documentation around what they're saying you won't do it's a risk determination either yeah I trust Microsoft's going to you know play Within those guard roles um or no I don't and I'm not going to use the tool so no very few times is have I seen a SAS vender an AI vendor just uh completely open up and say this is exactly what the controls look like this

is exactly what the guard RS look like here's what all the internal data fls look like because they do keep things pretty black boxy at the end of the day so so sure yeah um so I mean in terms of a lot of the AI you mentioned a couple points I'll touch on that specifically people kind of circumventing security controls in an effort to Fast Track AI because AI is here and it's here to St what has a security individual or even leadership because that's usually that's going be what what kind of conversations can be had to not necessarily push back on a but encourage people to think from a security perspective instead of just their own

individual so um we I have successfully seen um a couple things work paper tra the heck out of it um emails meeting minutes whatever whatever works for your organization um and direct executive sponsorship um understand there's generally several layers of management um typically we've been bubbling that up as high as it can go if there is an concern um in Microsoft for example that is a siso level concern we go I track down the damn siso and be like hey man this is what's going on you Gore and uh he will he will address it obviously everyone in the room different different levels and different organizations um but it is a risk determination that should be at a like

SBP ciso type level in most organizations and if they're signing off on it they're saying I mean that's their job hey this is this is within risk tolerance let's F let's FastTrack that um I definitely have seen um you know cisos get overridden and whatnot because the CIO or CE CEO wants the AI thing or once the capability but as security practitioners at the end of the day we can eliminate the risks of the business but we can't always keep them from doing the wrong thing and also flat networks the legal yeah like one that they would have maybe figured out that they could be viable but it's it's I am surprised when I walk into an organiz ganization and

the network is not flat like oh yeah we got vs we subnet and segment and the like oh that's awesome you guys are better than like 99% of the industry uh was there any was that it thank you sir in the back [Music]

oh I prob no I I good good question um I'm speaking as a so what's documented today basically as long those lines I highly recommend even your Microsoft agreements go go read them and make sure that you you you know concur with what the Microsoft agreement says um I probably am not qualified to answer that question either unofficially or officially just for the fact that somebody high enough up in the company says no we're going to go train with customer data or something I think there would be massive backlash as an individual um if people Shi in Midstream but I can't can't predict the future quite frankly so I I don't have a good answer I'm going to Ft on

that anybody else all right thanks all I appreciate the oh what yes yes yeah oh wait uh I am familiar with that one yes I actually don't what is I don't remember if that was Microsoft or not was it Microsoft yeah it happens all right y thanks all w [Applause] and now for a spot of good news this is the last time you have to hear me we have one more speaker we have saved the best for last so be sure and show up in I don't know 5es awesome [Music]

wait who was I'm sorry I misspoke there's two [Music] more us so you're GNA have to hear me speak one more time so no what you speaking

[Music] never and also now I'm some [Music] which who lost your

rays this was it all right if you set up m& Mur and acquisition Mur acquisition number 48 you're going to smile sorry for the

night so I have conter oh wa I

thaty there's a blank one I love there's a blank one there's a blank one was it me oh nice you guys have it set don't don't ever believe her when she says it is I'm just saying I don't know this guy he knows me that's right that is right thank [Music] there it is there it is oh we the presentation how did our camera get how did we get this I don't know I'm just makings you got hit it you got hard to do that one well I thought there was supposed to be one after this there is one after this she

[Music] CLC hackers burgers

[Music]

yeah there we go now it's [Music] right where she these aren mine those are lost and found for now okay name Michael sh

sad worked from home too long now I it's got benefits for sure like for me three days at home with a day or two to you know have a lunch you want to use one of the other ones

[Music]

[Music] no right here y I think that's good [Music] awesome I'm happy to start early if we want to get faster to happy hour well I think there's one after you yeah there's one top of the hour after you

[Music]

thanks for yeah you're welcome co-workers and previous co-workers they said hey you're here video us okay perfect so I hope there wasn't with anybody's view but I got video I could oh no app I think most of I'm sure it's recorded some usually put you put them up on YouTube or where do you yeah so it's live on YouTube right

now that's how they did them in Vegas they were like 8 Hour videos so as long as you have the schedule you could go back to you know the time and [Music] watch a couple of those I want to watch again yeah just to take some notes and for [Music] sure she fast she knew her stuff one of my buddies Jus set some of that stuff up oh yeah and yeah he's like it's kind of cool it can detect the dog versus the person Prett all that makes you think for sure yeah lot of there's a lot that I guess as you walk by it'll detect your Bluetooth and they're trying to do like Po ads so like

if you're walking through the mall and there's a store with SE 32,000 you're to do some sort of data probably figure out all the vendors and exactly who's where military that was funny cool all the odd devices that come by your house lot people don't realize that you know they probably have multiple devices on them you don't know too yeah mic Bluetooth connect [Music] I yeah I don't need my anything on the interet I don't think need things that connected the scary thing is when the uh you know the SIM cards and the sell chips are cheap enough they're just going to send it to you automatically it won't be like you don't you know now you have to it up to have

like activate it yeah something like that and pretty soon it's just going to be like hey I turned on and made toast today yeah said worked at a smart grid company and one of the things they said was based on power usage you could actually tell you know say oh it's allous you could tell like toaster or coffee maker had a very specific fingerprint for us so like like the coffee maker would shoot up and then kind of you know come down and you know certain things would draw a lot of electricity and then you have like a refrigerator that would go up and down you know throughout the day so it's cooling and so you could almost tell

that deves in in a house just by P yeah just just based on what they were using like that's pretty cool when you hook that up to a zigg or you know whatever it is you got all kinds of data [Music] app

[Music] crazy 17 of these coming [Music]

[Music]

got picture looking [Music] good I only wanted two but I'll do really generous also it's

[Music] smile so you said you've been involved the bides here for a while or this just like your first being but I've come to [Music] bid it was a kid I think last year the year before they had two water talks the kid was talking about picking back I remember his parents were in the [Music] hallway yeah and they were so proud of them like really was really young left high school that day to come that's

[Music]

awesome camera off may get well there's one guy [Music]

see I think over to Christian

[Music]

[Music] [Music]

[Music]

know

was was

long yeah

[Music]

[Music] all I didn't go this sure we jam and Cesar [Music]

[Music]

[Music] it's theyes have to the other Casino try [Music]

[Music] is everyone paying attention no no no this one's on hey everybody I uh I hope you enjoining your day at Denver and I hope that you're really interested in giving us money so that we can come back next year there's a guy back there his name is ton if you have cash for us you should go give it to him Make It Rain yeah make it rain all over tons he's raising his hand right now well 3 seconds ago anyway back to your regular Le schedule programming I I'm ay and one of my favorite rules of acquisition is uh the uh rule 48 yeah rule 48 the bigger the smile the sharper the night

the bigger the smile the sharper the night some was listening to me but I we're going to introduce right now actually faing uh with the Cyber tactics of murders and [Applause] executions yeah so actually my talk is about m&a AKA murders and executions but I feel like executions is with an E so anyway um so I was told I needed to rename my talk uh for that reason um by the way I am a I am a treky in next generation is the best let's just clear that up right now next Generation right Voyer a erer and I won't stand for it all right I'm not going to answer your questions then so a little about me why am I here why am I

talking to you today and what am I in front of you I thought of the craziest cyber journey and I have loved and hated every second of it so um I started out in the Air Force and I graduated as a commissioned officer my college next door freshman neighbor is sitting right up here in the front Rob okay so we go back 20 some years yeah I know right and he's here to support me I appreciate it so um I spent some time I would say I would call it DOD because the freaking Air Force kept deploying me with the Army and I couldn't stand it so after my second tour that was Afghanistan I came

back and I was like hey I signed up with me in the Air Force um why do I keep getting deployed with the Army I'm kind of done with of this so I did a little bit more time as a GS at the defense Logistics agency and learned about Acquisitions and I was like what a [ __ ] show have you ever seen the map of DOD Acquisitions it is the biggest freaking chart I've ever seen in my life worse than the Google Network we'll talk about that later so um after uh I was like okay working in the government's going to kill me I need to do something different I looked at commercial sector

and um I've been in big tech for the past 10 years I've had the pleasure of working at companies doing security incident response I built incident response teams globally I've responded to incident crisis of billions of dollars and billions of users and now I was like I'm done I'm done right no I want to go after the little guy so I spent my time in Tech you know and Collective my beautiful stock vesting and all of us know about that right that's why we work in Tech and um now I'm starting a company of my own and pursuing my PhD so that I can secure critical infrastructure so when people think about cyber security right this is this

is the meme you most often see so the government they're spying on us right they think that we're all Anonymous when my friends think they like how does she know that stuff how does that stuff make sense and then what I think I do is I think that I just run the team I think of myself as like a fire chief uh when [ __ ] hits the fan I come in collect the team clean up the mess make sure everything's safe and secure and then walk away um but honestly back to Star Trek what I really do is just hit my head on the table when I come across problems um okay that's not how I think of myself is I am a

cyber Crusader and I'm here today to talk to all of you about becoming a cyber Crusader with me one of the things that keeps me up at night is the fact that in the United States we have threat actors lurking in our critical infrastructure and it is terrifying they can take out our electric grid they can take out our power L our utilities you want to trust the water coming out of your tack that it hasn't been messed with the chemical content how much chlorine lead going into your water right now at this very moment our adversaries can mess that up and poison our citizens so I am now a cyber warrior in a different

way I wore a uniform for seven years took that uniform off and now my life goal is to secure critical infrastructure so back to the point what our real agenda is about is m&a and mergers Acquisitions and divestures so what I'm going to talk about today is what are our risks during an m&a who here has been an m&a who's been acquired or required a company okay keep your hand up if that went smoothly okay we don't have any hands up anymore so what I want to talk to you about is some case studies and lessons learned from those case studies during my time at m&a and what happened to our stock options um which company did you

see uh no smart sheet was my last tech company and they just went up 10% because they're talking about being Acquired and I was like Hey I was on that team can I do the due diligence I know where all the dead bodies are hidden please can you hire me for that so I'm hitting up a Blackstone who's doing that acquisition um so then we're going to talk about the real world success stories of um some of the m& I've worked on and then the action for you guys to take home because this is going to all you almost everyone here raised their hand that they've been in an position so I'm hoping that you can

take away some key items here uh back to your team and make it a little smoother or higher me so m&a it starts out with the selection process so a selection process is when the sponsor company or sponsor organization says Hey I want to buy this company I'm looking at buying something in this market sector they typically evaluate multiple companies at this point and look at which one is the most valuable to that sponsor so this is when they bring in what we call our security privacy and compliance team they bring us in the deal and while I was at Google um when this happens when you're in the selection process you have to sign your

life away that you won't do any insider trading based upon the acquisition information so the the number of people that are involved in the selection process is Sundar the CEO of Google and then the five of us on the m& 18 that's it the entire company is not allowed to load this needs to be confidential and it absolutely is important in the process that you do not expose any deal information pre-close am I too loud sorry so free CL due diligence so free close due diligence um that's a systematic process where we bring in what we call a clean team so this is the security PRI Y and compliance clean tea you're not associated with any part of the company

that you're acquiring or acquired so we have a sponsor at Google and then we have our m&a alphabet team that's separate and there's we can't have any stake in the deal essentially as a clean team so that way we're providing unbiased information uh to our project sponsor and we can be on and up front and um tell all the stories so in that pre-close due diligence we call it the the tell me phase at this point of an acquisition you're not allowed to dive deep into the tool or architecture because then you could run into the problems of if a deal falls apart lawsuits happen because they're like oh you stole my own intellectual property

what about this if you keep a clean team that doesn't end up being a problem so I highly recommend the clean team for due diligence and during that we come up with closing conditions so does anybody know what a closing condition is on a contract kind of like a house when you get your house inspected and you're buying a house the inspector comes in and they give you all the list of items like hey your roof has a leak in it this electrical box needs to be so we do the exact same thing from the cyber security perspective we evaluate that company we do reputational analysis do external scanning uh we do do um code analysis at

this point with a third party so Google does not touch the code and um and we look at at this this point we start preparing for what we call integration but I'll get to that in just a second so pre-close the most important part clean team and closing conditions closing conditions are what you're asking for fixed prior to the acquisition so it's your cut sheet of items that you say okay you need to fix this [ __ ] before I buy you because I don't want to be ped so close close is a day closing day is the day that you officially become responsible for that other company so day of close midnight hits in whatever time zone the

deal is you're going to see all kinds of crazy things happen you're going to see massive amounts of attacks you're going to see um like malware trying to trying to embed malware we've seen um like crypto lockers and all kinds of crazy stuff happen on closes because once the Public Announcement goes out they're like oh let's see if I can break into the acquisition so that way when Google brings me on board I got my foothold so that's what we're watching out for we put on a big security hug on Day Zero we deploy our tools we monitor immediately and we um go in a war room essentially and work with the engineers from the

acquisition the engineers from the clean team and we do that security hug so that the acquirer is responsible and has visibility into the new company that they you just purchased so that's cloes but integration you know I talked about pre-owned uh we got some poned environments a lot of startup companies do not think about security at all they are solely focused on building this product and selling it and security is an afterthought I honestly I don't think I've ever been no never been on an acquisition where there was a security contact that I had at that team it was always the excuse me it was always the senior security Engineers uh most of the time it was the

actual owner who created this product um so we during integration we evaluate where they're at current day and this is um before it was show me so this is tell me it was tell me first and now it's show me so if you told me that you're implementing this level of encryption I want to see it show me a screenshot if you tell me that your program is doing this now you have to show it to me and then we we come up with that assessment we come up with a road map and then we integrate them into our TCH sta that that sounds great but we all know that integrating two companies is extremely

complex so managing this from a security privacy gosh I'm sorry I don't want to give anybody a headache fine yeah it's good okay okay um so integrating into the tech stack is um extremely important because why else are you buying this company if you're not going to use what you're purchasing them for so there could be a deal that's you're just buying the tech you're buying the talent so sometimes s we would acquire um some developers just because we wanted their brains so we would acquire the company for the talent so there's many many different types of deals and how you work on it and then finally it ends up with handoff when the acquired company

has met all of our security requirements and then we hand them off to the PA in the product area and we walk away as a Cen team after that's held done so it's a it's a great process and it's always fun being a part of the team but it's sad when you pass it on all right so let's let's get into the real need of it what is this risk why do we even need to do cyber security due diligence um who here is anyone has anyone done it heard of it yeah it's super rare right like it's not very common so um the most important thing is the financial impact on the deal when you're collecting your

evidence and assessing the acquisition Target it is very important to understand what the financial uplift is going to take to get them into your into your company so um that could affect the price of the deal the reputational concerns so there was one deal that I was on um and I was brought in Pre uh at the interest phase right so the sponsor was like oh I really want to buy this product and it's a security product and it'll go great with you know this other Google thing I need to be very unspecific I apologize because I can't talk about it but I came in and I said no I am recommending that we cancel this deal because of the

reputation of that company and I don't think it's something we want to get mixed up in it is not something that our company and it doesn't follow our company ethics and values and so I recommended against that deal and that deal di so it's great to have a team and listen to them to get that information and being enabled to provide your opinion because this is your this is your likelihood online if you acquired and we'll get to those like the Starwood murgers and Acquisitions but um un undisclosed data so we looked the dark web to see if their intellectual property has already been stolen because that that happens so we cancel deals if we can find their

product already being built somewhere else because their IP was stolen um r W gosh darn it ransomware when is that going to end I don't think ever right but as soon as if someone hears about the deal in Day Zero we have seen a lot of ransomware attacks that's why the pre-close conditions are so important that you lock up that acquisition before you own it on Day Day Zero um Insider threat uh we've seen people who aren't happy with the acquisition and are saying [ __ ] you I don't want to go to that company so I'm in to seller [ __ ] on the side that's also the reputational analysis and seeing if there's any DLP

from current employees super important because not everybody's happy in a merderer or an acquisition so you need to think about that um so our first case study is going to be about yaho so 2017 is when I started on the m&a team and Google and this was new to me I've been an incident responder for 15 years so I literally have cleaned up dead bodies and every freaking Network you can imagine um so and by dead bodies I don't be physical right you know um but uh I started in 2017 and during the deal process um so Verizon was acquiring Yahoo who has a Yahoo account oh my gosh oh my gosh hey guess what I still have a Hotmail

account and I use it so um so what happened during this acquisition process is that yaku is not forthcoming nor did they know about the fact that they had two previous breaches so when the analysis the due diligence was happening in 2017 it was disclosed and it was identified that in August of 2013 do you guys remember that reach 1 billion users so pretty much everybody who has a yahooo account was compromised and then in September of 2014 an additional 5 million user accounts were exfiltrated but the problem on this acquisition is that yahooo wasn't upfront and honest and didn't know their own environment so what would you do if you're looking at buying a company and then just found out

that five billion of their users were already gone would you still want to buy them that is yes wait I said I wasn't going to respond to yoush darn it yes because it really depends on the motivation of the sponsor how bad do they want that product and what how much money are they willing to spend on it and remediate those issues right so the sponsors really wanted this deal and what ended up happening was the it was sold for $4.48 billion and because of the cyber security due diligence the price was dropped $350 million and there were Federal fines afterwards as well so Yahoo and Verizon split the fines um between uh the liabilities

that they had to pay and and professions so that ultimately affected the deal about half a billion dollars because of all of the fines and information so my lessons learned on this one in particular is transparency being transparent throughout the entire deal being upfront with your issues like no security is perfect just be honest about it and we'll figure out way to make it work right so case study 2 um so in 2018 I was still on the m& team and I was traveling around the world to these m&a deals and I was staying in marot and I was like [ __ ] this is horrible they stole my information what am I I hope

that I'm not making these mistakes in my Acquisitions that I'm working on so we as a team would take these cases after they came out and we would analyze them and take that feedback to make sure that we were building it into our process so 2018 Marriott Starwood Burger this one's a hot mess look at this picture you guys see the picture on the right I guess I have a pointer um so uh Starwood had an outdated infrastructure and they were poorly managing it and it was a hot mess they had a rat remote access terminal okay rat um they had rat go embedded on a server for Starwood and it had been actively exfiltrating for four

years four years oh my goodness like what is happening in that Network that Starwood couldn't figure out exfiltration and a rrap for four years so that um ended up oh and they had RDP open I mean come on that's just like a just unlock the front door with RDP open right so they had already open um so they did no due diligence in this deal so marott had no idea they were buying a pre-owned environment and yeah um so when marott finally discovered it in November of 2018 because they have a better security team than Starwood did and they detected it they had a call in law enforcement the whole thing Secret Service FBI gosh yeah

that's fun in that incident um so what really happened here was Marriott because they didn't do the due diligence and didn't do what we all expected a global company to do to protect our data uh it was it was frustrating that they didn't spend the time to do the due diligence to find out about this so that's the biggest lesson learned here is this was a huge number of credit card data personal data passports all that stuff that was stolen and could be turned into fake identities going for like it's terrifying so that was the biggest lesson to learned on this one is do your due diligence all right this one gets a star I like this one because it's my favorite

because who also was a part of the Equifax breach I know I was I mean gosh how many breaches OPM like my data is gone I already know it so um in 2020 this time um this was was it yeah okay so in 2020 why I put this one as a star is because Equifax did everything correct in this acquisition they did their due diligence they acquired um Estonia credit institution and they were rebuilding their reputation after the 2017 hack so 2017 F effects had the major breach and then in 2020 when they were doing this acquisition they said you know what we're going to get it together we're going to focus on on data security is

our number one thing we're going to make it happen number two we're going to set up continuous monitoring super important to understand the environment you're requiring and then we're going to spend the time and effort to rebuild trust with our users okay everybody's falling asleep um even though I'm loud like I'm saying we need it that's why I wanted cocktails what happened I wanted cocktails I wanted shears um so so I'm going to talk about best practices and and then I'm going to talk about a deal that I did while Google and then we'll wrap it up so that we oh there's a talk of this I keep thinking I'm like yes let's go to beer no okay so

clean team I talked about the clean team very important to hire unbiased uninvolved party so that you do not get sued if the de falls apart very critical due diligence think about it security privacy compliance due diligence it's a package it needs to be done when you're requiring a company even if it's just an IP snapshot that you're buying if it's a code stack that you're buying or if you're buying Talent it's still important to do your due diligence day Zero think about that ahead of time be prepared for it work with your engineering teams train your engineering teams and make sure everybody is ready to put that security hug on Day Zero so that you can see what

you're buying monitoring know what you're buying Central logging send it to the stock get that visibility you are legally responsible for that company now get your visibility and training it's like it's like when there's an acquisition it's like people just assume you smush two things together and it's going to work out uh training is so important to get people comfortable with the new product you're bringing on the new team you're bringing on training that new team what it's like to be at your company integrating them so that they feel welcome in your new company and start being profitable and helping your business day one um and then integration so with the integration the key point there is

estimating how much it's going to take what is the engineering lift of this acquisition I'm buying people are like oh I'm going to buy it for 10 million but how much are you going to spend on integrating it so uh we like to do what we call t-shirt sizing we would um once we came up with that the list that we were saying okay this is where they're at this is where they need to be to be in our environment and then we would estimate how many engineering hours it would take to fix each of the issues and we would give that to our sponsor to say hey this is how many engineering hours at this rate

it's going to cost this much to do the engineering uplift you always always always have to plan for that and allocate funds so that you can correctly and try to smoothly integrate them into a new company so real world this is me um this was 2020 this was right um I this was Co because I was up in Seattle and covid had just broke out I think this was February of 2020 and Co was happening in Seattle and this deal was up and um we acquired appet out of Seattle so we were on site for the day Zero deployment when we were hearing about the outbreaks of Co so what um I started I got involved in this deal from the

very beginning we were looking at multiple companies uh to acquire uh we ended up choosing not we the sponsor chose uh appet and so then we started meeting with abet started meeting with their Engineers doing our due diligence um and this happened to be the due diligence piece was Christmas right deals seem to happen at the end of the year because that's when funding is available so uh holiday break doesn't really happen for us in m&a also those of us in incident response we don't really get holidays sales sales yeah yeah there you go um yeah so um this one we were able to do the T-shirt sizing um to say this is what the uplift

was it was enormous I'm not going to lie so when I did the analysis and I came up with closing conditions I said this this deal is too risky because of their lack of security however I cannot find any evidence of a breach we just need to be aware and I want all of these controls implemented before we purchase them and so what I thought would take three months the engineering team had actually fixed and remediated all of those security issues in 4 days wow four days so that they were able to they were motivated they were motivated and they got that [ __ ] done they were like oh got it so I was happy um pre at at the

closing I had to ask for verification of those controls that were put in place get the evidence then the deal was able to go through and then we go to Seattle and we do the integration and from the T-shirt sizing we were it was estimated to be a year and a half of engineering time to get them up to security standards uh so what we did was we worked with a partner protiv in Seattle and hired them as a contractor to come in and implement the security uplift that we needed prior to integrating them into the Google Cloud stack and if you look at the Google Cloud stack today ad sheets on there and it's fantastic and

it was a very very successful project um after I had been there for a while right this was a good one so uh actionable guidance for you guys this take pictures take it away I want you to take this to your leaders I want to say SE sweet you guys need to think about these things when you're buying these companies stop shoving it down our throats let's plan this let's look at it as a program let's do it so prioritize cyber security cyber security encompasses privacy and compliance you always have to think about what that means to your company what it means to your reputation and what are you willing to to do to keep that so allocate

sufficient resources make sure you have the engineering power or money to pay a contractor to do that uplift every acquisition nobody's perfect everybody's going to need to fix something and open communication so when I was working on a different deal this one was in Australia it so I found that this particular app that we were looking at purchasing um Capa was created by Google are you familiar with Capa you got to you know pick the thingies this company had built into their code a way to bypass capture and sold were trying to sell it to us and I was like uh you literally are trying to bypass Google security to do your thing this is literally illegal but you know

we were working with companies that were being built in different areas of the world so their regulations are different so that's why the uplift of meeting our security and compliance is so important so those are the things that you catch and you're like what the heck were they thinking right but they needed to get their job done so due diligence you find the craziest things um technology teams this is the core these are the people that make it happen right make sure you're talking to each other talk about requirements talk about integration talk about access controls what are the appropriate levels of controls that you have and these new people have you know and then security by Design I say that

but I've never been in an acquisition with security so um legal and compliance honestly lawyers are my best friends the legal team that I have worked with um at the tech companies are just phenomenal all three of the tech companies I've worked at I really enjoyed working with the legal teams and um always get advice from your legal team regardless of how much we can read compliance they always want a lawyer to sign it whatever it's always good to get their perspective and get them involved um and also make sure that the lawyer includes cyber security in the actual contract because a lot of times it's not considered not written in the contract make sure there are

articles about the cyber security posture of your acquisition in the contract so I'm going to stop here for a second any question yeah so with the marot and Starwood Starwood they found a rat yeah was that true during due diligence or when did they find no after poose close they didn't do due diligence what they were asking about when um when did they discover the rat in the Starwood Marriott murger it was postacquisition because Marriott did not complete a due diligence so they they did not find it until they were actively monitoring the environment so if you don't mind on that your suggestion for uh due diligence was risk assessment and check posture that doesn't seem like it's that

those would necessarily catch a rat I feel like you need to do an threat C catch so is that something that beig yes and no so pre-close with the clean team you can go in and check the environments and you can do an internal you can ask for an internal scan we we would complete the um we would complete the external scan we would ask for an internal scan um but a lot of the Acquisitions didn't even have a scanning device um so we would sometimes we would look through the logs and see what we actually could find so um a lot of times these companies and Technology are built in clouds that we don't trust at all and so we just kind

of assume that they're pre they're pwned and uh we burn it to the ground we take a snapshot of their IP and the code and we literally burn it to the ground and rebuild it into Google so um what we did at app sheet is you build in parallel so you build you build the new product in parallel and you have both up and running the old and the new and then you mirror the sites and cross traffic to test the loads and all that stuff during the integration phase so most of the time we assume that the environment is PED and we take the IP we burn it to the ground and rebuild it that's most of the

Acquisitions I worked on we would not bring in um we did not bring in any Tech from outside but Google also has a lot of money which is very different than the normal customer right so it's all about comes down to the dollar yeah anyone else yeah it's actually kind of similar you probably kind of answered it but is is it normal then to have a third party come in do some type of assessment or do you have like a full like a CR some big company like that going to do that or is it it's not usually going to be your this sponsor so we um we as a clean team would hire a third party code review so

and then we would also have um a snapshot of the code evaluated for maturity and how well that code was written because it also had to do with how much the engineers were going to be paid and offered a job at the new company well not even like the code with like infrastructure and you know just like that type of yeah so that's when we put our monitoring Tools around it on Day Zero and that's when we know if it's poned or not as we find out on Day Zero but then if you're having some like conditions with that pre stuff have something what you yeah and so that's why I'm talking about adding it in the

contract that the cyber security language so that you can come back and say like in the Marriott Starwood one they could have came back to Starwood you know even though it had been dissolved but they could have came back and got reparations for the that particular area of C security if it's in the contract so then you do have a legal standing to say hey you didn't disclose this to us now we want more money back so it it's all part of the negotiation yeah all the way the back

yeah yeah yeah so um so I've created my own special sauce actually so throughout my career I started with the iil life cycle and change management and then it turned into incident crisis management and then I started doing uh program management so I learned the PMP style and then I was like you know what cmmc would be great to add on top of this so that's kind of how I look at it I look at it as a program and then each Lan I compare to the cmmc model basically and then um create a heat map that says okay based upon um I also use the CIS 18 controls and grade you on how you're doing and

then showing that track that progress over time so and I associate it with risk so like number one on your is knowing your assets right and knowing what's on your network um so I start with the Heavy Hitters first risk based and then go from there go gole you know Google really doesn't like following anyone else's ideas so literally like when I came in I I worked on the nist uh critical controls framework and they were like don't use that language at Google you were not allowed to use that here and they came up with their own words for everything and uh so Google is really it's own world uh the rest of us can talk the

same language but talking to a googler is like a foreign Lang langage not going to lie y we've been building a service for this effort for the company that just

did I would love to

AC

situ right yeah I think it's happened to like 140 sub controls now um yeah back when I worked on that I was on the board for about 5 years when it used to be the 20 critical controls under Sands back in the day um it was a lot less complicated but I really like the detail that we've added now especially with the implementation groups I find that is like if anyone's asking me where do I start with a security program I recommend they follow the S8 and start with implementation group one like got to start with the basics cyber hygiene and really in an m&a deal we're not looking for a perfect situation by by

any means we just want it secure enough that we can buy it without a reputational damage or risk so it's all about risk is your company willing to risk it how much risk do you have and how much money do you have to pay to reduce that risk so yeah you mentioned information GA do tools that have been sour like

so I actually use all of those things yeah I'm glad you asked that question the question was how do I do the reputational analysis and I do the reputational analysis um through dark web because that's typically where we'll find the IP you won't find that using ENT tools um so using like Mandan uh their Intel and Google also has the best freaking threat Intel team in the world right like so we would work with them specifically our tags so thread Action Group at Google and work with them if we had questions about um a company we were looking at and they would see like what APS are going after them um what activity they've seen so we were able to

do a do a deep dive now that I don't have access to tag um I do I use ENT tools recorded future and I'm demoing the new M tool yeah um one more second we will I want to just wrap with my final thoughts and then we'll do some more questions okay uh so my key takeaways is be humble in all of your interactions like meet people where they're at be humble security is scary to most people it's overwhelming it's this unknown that can ruin their life by emptying their bank accounts so people are scared of security so just be humble in your interactions and be easy to talk to you won't believe the stuff you're going to

hear if you're just nice to the team you're talking to they will tell you all their dark secrets and you will find out so much more about the deal if you just are a little nice about it right so that's being you know what they say you collect more with honey absolutely honey be sweet be nice be approachable don't be intimidating um and just be open assume best intentions assume best intentions is one of the greatest lessons that I've learned in my career and coming from the military and having like a Marine Corps boss and then having an army General and like being treated not that great right like as a lower person so at Google we

are trained from day one to assume best intentions and meet people where they're at that day we we all have [ __ ] going on in our lives we all can be distracted but treat everybody with respect and kindness in your interactions because you don't know what they're going through so that is one of my biggest career Lessons Learned uh do your research in an m&a do a research when you're looking at acquiring company um think about it when you're interviewing with a company when you're interviewing with a company remember you're the asset that they want so you should be interviewing them not the other way around you need to interview them to see if that's a place that you want to work

do you like that culture do you like that opportunity do they provide training so think about that and then motivations so motivations will change everything right so the motivation of the sponsor he'll throw money they'll throw money at it to fix a problem so if you understand the motivations of who you're working with um you're able to support them better throughout your deal I mean that's the same for being you know on any of our teams like if you understand the motivation of your director or leader you're able to support them better so always be willing to ask the hard questions in security sometimes it can be intimidating to ask well can you show me that you're

actually using AES 256 because I thought you were on 128 you need to show me like don't be scared to ask the hard questions don't be like show me your offboarding process show me the fact that your engineers don't have access to your system anymore because you forgot about some platform that didn't have single sign on right so you're letting Engineers go out the door that can still log into your monitoring console so trust the verify sure everybody's going to put on a nice face in the deal and they're going to tell you what they think that you want to hear trust but verify ask ask for the evidence show me the screenshot I want

to see it um and make risk based decisions every interaction that we do in security is based on risk that's how much effort we put in each day that's how much money we put into things it's all based upon risk and our ability to reduce that risk so I I want to thank you guys for listening to me and being my Friday night date in Denver I appreciate it my husband is appreciating his computer gaining time now that I'm away so I appreciate all of you and um you can always book a consultation with me for free on my site 30 minutes and we can talk about your situation I can you know talk you

through whatever you're going through um with your program I worked from one person single startups all the way to billions of users of Google so I'd love to have a conversation with you talk to you about securing your organization or if you need help on an m&a deal I'd be more than happy to help do we have any other questions we did have a question right here yeah you you mentioned internal SC external SC and do

youever so we do the application security testing uh post acquisition so we have our application security team that will fully analyze help them so that's during the integration piece so our security team works with the engineers to Mentor them and help them understand this so that they're empowered to do it again next time right yeah what it doesn't matter like yeah even if there's really bad bugs like you bought the product already like you're not going to see that and you shouldn't see it because that's intellectual property owned by so