GCP BigQuery as Security Detection Platform

hello i'm urban martin director of the ground truth track at b-sides las vegas uh the next talk is from diana kramer and norberto garcia marin it's called bigquery as security detection platform diana kramer leads the security operations team for king she's an engineer at heart who is passionate about video games and she views security challenges as a state-based system that can be improved norberto garcia-marin is a junior security engineer at king working with incident response detection and automation processes if you have questions the live q a will immediately follow the talk and you can submit your questions via discord in the ground truth channel alright so i hope you enjoy this talk hello and thank you for coming to our

talk this we will present our lessons learned building a security detection framework in bigquery google cloud's platforms big data managed service but first let us present ourselves hi again i am diana i am a security engineer currently working at king part of activision blizzard and i'm leading the security operations team here king is the creator of candy crush if you haven't known and as somebody just mentioned we will take a lot of advantage of the sweet ip that we have in all those presentations for the past three years i have been part of our cloud leadership team i helped understand and implement security controls and also i've led the initiatives for detection and response overall cloud infrastructure and data

by origin at heart and by trade i'm a network engineer so the cloud kinda has been a challenge for me i was used with cables hardware physical infrastructures you know things that you can actually touch so this made it difficult for me to change the way i was being infrastructure basically adapting to the software as everything philosophy on the positive side because there is always a positive side the cloud gave a new sometimes magical world of opportunities through tools integrations and freedom that in the end kind of enriched me and the way i do approach security so in the end i loved exploring cloud tools and services and especially gcp and what i've been doing during the last

three years is to come up with ways of leveraging them for security and of course i will always be a little biased and my projects mostly focus on detection and response here you can find me on twitter with roruza and the bat is a quick reference to my origin as i am a romanian norberto hello everybody i'm roberto i'm currently a junior security engineer working at king in which i've been here for a year now i come from a computer science background so i have experience in different fields including security software development and appropriate intelligence actually in my previous job i researched and implemented cyber security solutions that leverage machine learning i really enjoy this new

cloud computing paradigm and the possibilities it offers and at the moment i'm holding a few workload certifications i think we can use cloud native tools to build amazing things and so i hope you enjoy this presentation unfortunately i don't have twitter but you can connect with me on linkedin perfect so now let's see the content of what we're going to be talking today so today we'll be start by stating the problem that we have explored and that is using high volume data for security monitoring then we will introduce google cloud's platform's bigquery and of course its main functionalities and what basically made this project possible norberto then will introduce us to the design and implementation of the

detection platform and he will also present the next steps and where we actually want to take this project in the future and that is bigquery's machine learning capabilities now let me say from the beginning that our story is not only about the cloud platform it's not about tools it is about how engineering is creativity and innovation how we can embrace every new idea and let us be inspired even if the final product ends up in production or just in the drawers of anecdotes so we not without further ado let us dig into the problem and this amazing little drawing here that represents a problem also in one of our games so in this session we will try to see

how we can use bitquery together with other gcp managed services to solve a problem that has been already solved and redesigned several times and that is high volume data logging storage and querying and if it's been already solved why are we here you might ask well the existing solutions have shown themselves to be expensive high resource consuming difficult to maintain and more or less to have issues when we need the most and that is when we are exploring the source data and when we are accessing it how many times were we relaxed for having logs and telemetry only to discuss discover when we needed them that the platform failed or the queries took too long

i personally have anecdotes from working in a stock where analysts queued at the end of the month to perform reports as there are platforms that basically don't support more than a couple of concurrent queries also maintaining those platforms is extremely difficult you need constant engineering support and it can be a very monotonous tasks and challenging for the operational teams so what we used for the requirements of our platform it was an elastic guide on high volume data sources we are searching for a solution that scales fast offers speed when exploring data focus on what we are searching which rich filtering options and the possibilities for visualizing data high scalability is important because we can have births of data or

new log sources being added and we need a platform to be robust everybody has a story with an ml project forgot over the weekend or new log sources during an incident that need to be added asap so going for a cloud managed solution we don't worry about anymore about scaling the infrastructure for hardware engineering time or any expansion of the platform when this happens also we need speed when exploring data and this includes queries over a long period of time as we know that dwell time of attacks can be very long there is one report gives us the insight that this time can be up to two years of course it has improved lately but

still this is happening bigquery can query terabytes of data without worrying about memories in minutes of a time and with reliability and as new malware families and techniques appear every year mandiant 2020 threat reports estimate that more than 40 percent of them were previously unknown we need to use several data sources and have strong baselines for detecting this unknown the sql language together with many aggregation and parsing function offer us these flexibilities and allow for querying many tables and data sources visualization is also a very important part of detection and security investigations having native flexible tools makes the platform more powerful and bigquery have several integration from its native data studio to grafana and looker

looker in particular already presented several dashboards directly focused on gcp's data access and admin activity logs and have been presented in the last couple of days okay so bigquery we talked so much so how did we arrive to bigquery exactly so bigquery is a gcp managed service that provides a highly scalable data warehouse basically it's their managed data bit manage big data platform it allows to query streaming data in real time and have access to historical data it uses sql language for query which actually saves us from learning a new syntax allows for injection of any type of data and with some help from a data engineering can do actually some magic with it being

able to store the data in ways that facilitate analysis also and alberto will show this as well its machine learning feature allows to build and operationalize ml models directly inside bigquery so we arrived to bigquery as our games produced huge amount of events so basically it's a big numbers game king has more than 250 million monthly players more than 50 million lollipops that lollipop there the pink one more than 50 million lollipop hammers are being used every day in candy crush we used to store all these data in a 50 pentabyte 500 nodes on-premise hadoop cluster and we were reaching its limits so what our data team did was a benchmark for bigquery and decided

to migrate oral data oral data lake to bigquery and that was with amazing results and you can see all that benchmark the migration and so many interesting things that they did in our tech blog and with a little example here we have including a machine learning example that is uh actually simulating true players playing candy crush so virtual players and it's amazing to see how they model that it's it's very fun and there's also a talk from um our teammate that did this in one of the next so um what we did here was to build on this experience from king and use bigquery also for our security data and we actually started in a very

obvious place gcp is on cloud audit logs gcp has several type of audit logs including admin activity and basically you can see the or activity performed over gcp resources like the creation of a vm or the change of permissions to a budget then it has data access logs which are kind of crude access logs uh you can they can be very noisy and they log access to any resource so if you think about it in a case of a bucket in admin activity logs you see its creation or im permission change and uh in the case of data access logs you can see read write access and actions over that bucket so this platform in the end proved to

solve or are issues and those are the main free views of big data volume can store huge amounts of data velocity it can query the data in seconds or minutes and then variety many different data tabs and can query with xql and with many other options for data analysis and then as we said it goes even a step further giving us the possibilities to explore its machine learning features so as it solved are all our requirements for restoring an investigation platform we have arrived to the need of creating detections and not only atomic detections but also multi-event or anomaly detections which bigquery opens a big real-time door too this means that we explored how to apply to our data and to

create a framework that leverages several cloud products actually a lot of cloud products to create a consistent experience native cloud first very important norberto will show us now how he implemented this idea taking advantage of bigquery's conversation bigquery's characteristics and gcp's rich tool environment norberto thank you vienna so cargo deluxe iniquity and the ability to leverage sql queries to perform security detection is there a way to manage orchestrate and exterior arm detection and the answer is yes we would like to use cloud navy tools in the gcp ecosystem to build a solution gcp offers different tools that can be leveraged with minimal operation of regional cost because most of them are managed or serverless solution

tools such as storage packets monitoring fafsa service cloud functions and cable schedulers can be integrated to build a solution i'm very pleased to present to you our bigquery detection framework our serverless cabinet solution with minimal operational costs next slide please we were designing the framework uh we thought about our needs and our possibilities in the cloud so in the end what are we trying to achieve we would like to leverage bigquery as a local analytics platform bigquery is the mind pillar of the framework this fully managed fed up by the scale analytics warehouse lets us run analytics over vast amounts of data in real time with non-operational cost logs can be stored in bigquery and later be retrieved using sql

language allowing us to use powerpoint analytics queries to perform any kind of security detection having those direction in place we will need a mechanism to to schedule and orchestrate the detection depending on some metadata that we define and lastly since we are working in the cloud specifically in gcp the best approach to build this solution will be using gcp cloud native tools that provide a managed solution and easily integrate with every service with these requirements set we will need to define protections tailored for for bigquery this is using some templates for method is easy to maintain and deploy also use cabinet tools for the logic such as functions and cable scalers that we are going to talk about later

but these tools allow us to implement functionalities in serverless way that we have a minimal operational cost we will need to build kind of a size cd pipeline in which we can perform something basic testing of the detection and automatically deploy them to be up and running we only want to fill out our template and let the magic happen optionally we may send the final results to of the detections to acm to perform for the analytics or notifications and lastly there should be an automatic monitoring of our solution to make sure that every company component is working as expected and in case there is something wrong we can be notified in time next slide please

as i mentioned before uh we will need an impa temporalized way to create protection so we can automate their deployment for this purpose we have defined two different jump templates one for the detections and another one for the schedulers that will run the detections ways the injection template has several metadata it has a unique identifier to properly distinguish its detection and we also make use of the mathematics in which we classify by technique and tactic and also a category such as production or development the severity of eudexia on your url that contains a description and information about it and lastly the bigquery query that will be executed when running detection um why do we need this metadata because

we would like to scale detections depending on the on these fields imagine that i'd like to run only specific detections that have a meteor tactic or even a specific category and we can leverage this metadata to discriminate elections detections under schedule for instance as you can see this detection in the stream was built to identify a simple sql injection in which the url contains the word select and different fields are returned these detections as i will explain the next slide will be stored in a big created asset and therefore the schedulers will perform the escalation queries to get detections to run the scaler template is more simple it has a name a description a current job that

indicates the frequency in which the detections we run and lastly a discrimination query that will retrieve the detections id that meet the condition for instance a specific meteor tactic category or stability as you can see in this example we would like to run detections that belong to the production category every 15 minutes next slide um what about the logic which tools are we going to use to implement this framework we are going to focus on cloud-native tools that gcp is offering we prioritize manage services that allow us to focus just on the development of the detection and everything else is automated we are going to use cloud destroyer packets to restore the detection definitions this packet is the entry point of the

framework in which security directions are pushed as objects the packet will cause the definitions to be ingested by another cab tool we are going to leverage bigquery creating a data set and a table for storing the detections definitions these definitions are retrieved from the packet that we previously have created using bigquery to external detections allow us to perform discrimination queries as mentioned before to link bigquery and have destroyed packets we are going to use a several solution called functions cad functions are functions as a service solution to run code in a fully managed way with serial server management these functions can be triggered by several events in our case these functions are going to to to

update the bigquery data set accordingly including new detections or even deleting some of them we have leverage python 3 for the implementation that uses the bigquery api to perform updates queries these queries use the data manipulation language to run transactions we are going to have another cloud function in terms of performing the detection for itself against the equated asset that has the actual developed this function receives a detection id as input and we will retrieve the detection query from our bigquery detection dataset and finally running detection for itself the result will be published to a cloud cloud pub sub topic in which the results are gathered this topic may be used by a cm to collect the results and prepare

analytic research or notification another function will be in terms of orchestration when the detection run it will run an estimation query to get the ideas of the detection that are going to be run and then called the cloud function that performs the actual detection this orchestrating function is called by the cab schedulers catastrophes define when a group of detections based on the metadata are going to be run the definition is done by the scalar template that i have presented before and they trigger the orchestration cloud function finally we are going to use the gcp cloud monitoring solution to report errors whenever an error occurs behind the cloud functions that implement the logic another will be rise and we will not be

notified next slide please in order to deploy everything in an automated way we are going to use a github repository to store the logic the detections and a scheduler's definition we are going to to leverage infrastructure as code specifically terraform to deploy everything terraform is going to watch changes to detections and schedules that are defined in medical and so be able to deploy changes automatically there are four leverages who were provided to make changes to the gcp infrastructure it also reads the young templates for security detections and excluders and performs a basic validation in this way we can add and delete detection or schedules under a form will handle everything it will either parse the detection

definition to the productive storage bucket as the entry point or create a new cloud security exchanger that orchestrates groups of detections to enable an automatic run of terror form we are going to use atlantis atlantis is a software that integrates with github to run terraform when there is a full request in this way whenever we introduce a change in the github repository via a pull request atlantis will run terraform and test and deploy the changes automatically next slide please here you have an overview of the different components needed to for foraging detections let's imagine that a user wants to add a new detection to the framework the user will create a new branch in the

github repository come to commit the changes and then generate approval request when the request is approved atlantis will trigger the draft form workflow and the uploaded changes if everything is validated the detection will be uploaded to a gcp packet and this packet will trigger the update cloud function also known known as storage watchers this gap function will update the equated asset that contains the security detections and later it will be used when an ejection is scheduled next is like this on the other hand if the user performs upward request with account security definition terraform will deploy a new cloud scheduler with that definition discover scheduler using a chrome frequency will just trigger the orchestration cap function

that we run a discrimination query to get the group of detections id to run then this orchestration function will trigger the detection function one is transparency a good thing to know about functions is that they can massively escape and we will have independent instances per detection this detection function will retrieve the ejection query from the bitcoin detection data set and perform the injection query against a bigquery dataset that contains the databox finally the result is published to a cloud pub sub topic which an external cma can make use of it for ieds or even notifications nexus live please finally here you have a complete overview of the components that integrate the architecture in a single image

the user can either push detections or preschoolers and terraform will trigger the corresponding path to update the framework also the monitoring is running behind the scenes ensuring that everything is properly working and notifying us otherwise next slide is when i check when a detection is run and as a result the result is sent to a top sub topic using a json format it generates a single message for the detection result here you can see messages released to the gct pub subtopic in this example the injection with id1 has 5 results its results can be ingested in real time by centralized lock tool or cm to perform part analysis or notifications next slide a result contains some metadata about the

detection such as ide major technique and metadata in tactic and lastly the result itself and a time stamp the result is just the data retrieved from the bigquery way and its fields vary depending on the security detection that is being run for instance this detection has a date a society b an hdb method the unit in which we found the sql injection notice the select word at the end of the string also an http response to packet size and finally the user date next slide is imagine that a detection smart form or some error occurred while retrieving data from the query how would you be aware of this we leveraged cloud monitoring to create policy allies when an error occurs

during action function execution an incident is created when a function prints a message a message with errors ability then a notification is sent by a male or even to a cm thus we can tightly investigate the error next slide here you can see an example of an error triggering to do to a table that does not exist when the when the cloud function is executed and runs the query if some error happens it will trigger another facing all the error information through this seller we can check the there were alerts inside intensity and see all the errors that are being triggered in real time here we can get more context about what actually happened and the python exception of trace in our

example using this trace we can properly identify the problem and apply appropriate fixes let's select this finally let's let's talk about our future plans on this topic it's good to define detections and automatically deploy them but wouldn't be awesome to have detections in place without even defining them gcp offers different tools that we can leverage to cope with this situation we can create an artificial intelligence-based anomaly detection model directly built from the data logs this model will be capable of detection of detecting outliers now in the docks finding an error when something is not expected or classified as a possible threat for this purpose we can use gcp data flow to perform extreme data processing data

dataflow is a serverless patch and streaming data model for parallel producing pipelines the relation of a civilian open source project to implement the processing workflows using dataflow we can perform initial data normalization and feature extraction from streaming logs ingested from bigquery where the data logs are stored then we can perform outdated detection delivery in a machine learning model that we have trained beforehand the model can be built using bigquery itself bigquery provides a machine learning model in which you can create models using standard sql queries in our case we can train a k-means clustering model to perform unsupervised learning and identify anomalies or even use a encoder to be a to build a deep learning anomaly detection model

besides the big query basic learning model there are specific tcp machine learning products to build a model we can leverage vertex ai to train a model based on the tensorflow library therefore it's highly customizable and enabled to sum up independently of what product we choose to build the anomaly detection model we are going to achieve a real-time layer detector that works in streaming allowing us to tell you detect threats the result of this real time pipeline can be exported to a visualization dashboard and enables organizations to create alerts and action for automatic motivation and this is our future plan for our framework that we expect to implement it very soon diana is now going to present our

conclusions well norberto it was amazing to see how many tours cloud gave us and how many you have used for this project it's basically like a lego pieces that allow our imagination to run wild on so norberto used here around 10 different cloud services and yes i will use that word again it's almost magical it helps us get from a basically hackathon idea to a full working framework solution just with imagination engineering and a little sweat here and there this is a great example how creative security engineering actually is we start with an idea or a need and go looking around how to combine all we have available to create systems that not only solve problems

but also fast efficient and of course fun and this is the important takeaway engineering is fun and it's great to discover and play with new tech and cloud brought new challenges to security of course but also new toys and opportunities even if sometimes it's difficult to adapt to the new philosophy the result of applying the same engineering principle to the new world helps security and its engineers basically keep up with the same pace with that's what the tech evolves and one of the main engineering principles is that every design and system can be always improved and there is a next step always for our project we really really hope that our detection framework is an inspiration

for the possibilities we have and gets you as excited as we are about security detection and cloud computing so thank you very much for letting us share it with you and please if you have any questions hello campers hopefully you're all having fun and none of you have gotten lost here in the random forest uh we have deanna and norberto here to answer some of your questions so if you haven't already please put them into the discord uh in the ground truth channel um so i think we have uh let's see one here right now um from a username i'm not sure how to pronounce uh what was the most difficult part to orchestrate to use gcp bigquery as a detection

source okay hello i think that can be answered from two points of view so i'm gonna give maybe the first answer how the idea and how to to actually think about it and what was difficult to start approaching it and then maybe an alberto from the actual implementation part so for us initially was because it was a new environment like we didn't we had the data there but we never thought you know we were just using of using about it as investigation and threat hunting but you didn't think for a long time to use it as a detection source maybe as we just talked uh as we're used to you with security platforms we dedicated

security platforms for that so this was a new environment and it was a little weird to you know use a general data platform for this and it took us a while but we actually got inspiration for projects like anadot which also there are some presentation that king made around using the that data for seeing deviations from baseline and the fact that more and more projects appeared around the castration around big data in the last two three years so we were just like why don't we start doing it as well why we have the data there let's do something and we i think it also helped having more knowledge about how to use those about what a skill orchestration

means about what detection has the code mean about other gcp products and then the more you knew you know those lego parts you start putting the more more lego parts we had we started putting them together but i think the difficult part was the learning curve you know getting the lego pieces together uh yeah i actually really liked what you said in the presentation about engineering being kind of a creative process or a creative outlet and uh i guess you know in that sort of vein do you have any advice for people who might be more used to working with security focused tools kind of transitioning to a more general data tool again i'll talk from my point of view

and then if no letter has said i was always so this is a story that will start when i was seven years old and my mother was not allowing me to make my drawers for the drawing class because they were so ugly and she was doing them for me so i was raised in an environment where everybody was telling me i'm not creative i just know math and physics and i had that idea that i'm not creative for all my life and that engineering is a very you know that the engineering ideas are not creative ideas are structured ideas and this change actually uh with a team of tech writers which are the most creative people i know doing

the security awareness program and they taught me that actually those things that we're doing they are creative putting those puzzles together getting ideas of how to transmit an information on how to build a product how to create a function you know where that's creativity and i think what's most important here to change the chip is to understand this and start exploring and yes we're used to security tools we're used with some syntaxes we're using a framework in a structure in a stack just go out there find the first platform the first tool your team uses a lot to have a tool in your company i don't know whatever it does go and see how people use it and go and

see how you can leverage that for security understand why they're using it and any data source is interesting and no matter where it is now we know from norberto that even if it's something on prem or something in aws or in gcp can create somewhere a function to query it with the data that is important so i think the secret is a little this open the mind and see see pieces get lego pieces from wherever yes actually from my point of view i think that gcp and cloud computing in general is fun and that's the important thing that if you want to build something and have fun that's the right path so i think maybe leverage tools that

you can build from scratch and code that tools are awesome to to to have everything in place cool thanks um there's another question here from jeff so what would you not use this pipeline for for example what would you say this is bada doing um well this pipeline um was built for security detection so i think for security detection work work well it works well but i don't see any other things that it would be bad at this i mean this project is for security detections i don't i don't really we haven't thought about other use cases for this so as far as they are detections it would it will work well yeah that makes sense would you say that's

like one of the benefits of using a general platform is that you can kind of do whatever with it or yeah of course you can you can build whatever you want that's a good point imagine you want to integrate another thing and build i don't know other pipeline you can create several pipe signs for different different things so it's up to you i mean you can build everything you want and it's it's pretty general actually yeah this is a creative detection but imagine you have a web app and you need to make some some traffic lights for seeing if that fails and you have logging for that application and you know that some error codes that

are happening so it can be used with anything that queries logs basically that queries data sources and that you have a structured way of adding them so it doesn't matter what the data is or what you are in what the result is and it as we said it can do it atomic it can do it over several events it can even do it over uh baselines and searching for anomalies so you have anomalies in normal web application you know you you can soon have pickups of traffic you can see when you have cells or traffic or of anything and that makes you see that there are questions around there so the idea and this is one of the cool

things about the general platform go to the experts so go to a bigquery expert and this is how we did when we try to understand how this platform works and what type of sql queries we can do and what kind of detection we went to to somebody in our team in that is expert in bigquery and they worked with it for free for years and they stood in the hackathon with us and we were explaining what we tried to do and they started shifting our data around and said no no how you're storing is this bad you need to do it like this and this also goes with the general platform you have more people that know

it if you go to just security you'll always have this little pool of us that did the trainings and everything if you go to a general one you have so many more people that can help you and you can ask and i'm a horrible human i'm always going and bothering people with questions and taking advantage of them so i think this is also very positive and it helps our industry a lot to be this open you know um so um yeah actually tagging on to one thing you mentioned there um with the detection rules that you're using um have you had to create most of them yourself or with other sort of collaborators that you're

working with have you been able to leverage vendors or you know repositories of detection rules or anything like that so the ones that we're having right now as we said we started this poc over gcp's on audit logs so some of them were created by us because uh in the last three years we were trying to understand behaviors around it and you know doing a normal uh detection engineering life cycle of a detection from several points of your database the tag base and all that so some of us we created others already are public like splunk enterprise security has a lot of detections already prepared for cloud behavioral for their platform but they can be adapted

so they're already a lot that can happen basically you can even go to a sigma repository and get everything and just convert it to sql so which can also be done with a function but the idea is that those things can be happened and we used everything around it and as you have other logs you can use you can even have a thread intel feed and just do ioc direct matching so there's there are a lot of options and google actually two weeks ago they had their security summit one moment to find my mouse and they even proposed to take something similar and even to take it to a dif to the next level and create this as a

stack so you can you might at some point even integrate with other products and imagine we have five detections just five detections done by uh by norberto that are querying bigquery and the next five in the functions instead of querying bigquery they query your coralite products or the quarterly or surycata or zeek repository of detections and get the detection feeds from from et pro or whatever you're ingesting them and you have those data and then you just need to change the filter inside the query so that is completely possible and i'm gonna share that link that google was talking about and how um they were going in the same direction so basically it's this idea of

using any product and with those little function you can inter connect anything interesting so do you have some plans for additional things that you'd like to integrate in the future going forward um i don't know if you're using sort of you know thread intel feeds from different sources or things like that

roberto maybe talk with people about the the machine learning plan yes first of all i think we can integrate some intel thread uh from well from from the internet to a big query table so we can have those iocs or thread thread information in the table and query that table and perform the detections and join all the tables that that we have and so using that we have like more information about everything and we can build consistent queries and also talking about machine learning our next steps for this project is creating a machine learning model to detect an anomaly detection model to detect other lawyers so we expect to to work on this in the next months and

we will create a blog with this information and make it probably available so it will be awesome i think yeah it sounds cool uh exciting what about data visualization are you doing anything interesting with that well actually we mentioned that there are several ways when where you can extract this data for visualization and that was one of the other reasons that we chose bigquery or we saw bigquery helping you know the famous dashboards and everything and exactly also in the past to three weeks uh looker is a data visualization platform that works directly integrated with the carry and actually is from google now so gcp brought it i don't know how it is and they

um you can do yourself the dashboards there and they can query huge amounts of data but uh they actually put the project up where and and i think it was after we recorded that okay we're not mistaking where they're already making pre-built uh dashboards over the gcp audit logs that we talked about it's mapping them to mitre and i think it even has some some abilities to to extract more data so that is something that we can you can test at any moment and they directly go you just connect it to bigquery to the logs that you have and then those kind of kind of platforms do offer you the possibility to start uh creating your own dashboards with

this huge amount of data so you can actually see baseline of terabytes of logs from one day we to another without you know without any loss and in real time and this is pretty amazing and we also see those see we've seen this in other teams also in the blog is what they are monitoring our own infrastructure with that and we were seeing the dashboard then we were like wow you can see that much data so smoothly and so in real time and to get notification we want that how do you do that tell us teach us so we just went knocking on doors and uh and basically this is how we discovered it because the experts

in in the end in those products is not us and we will never be experts in even in our security tools i think you know every they have their experts and something new appears and you're just using a little fraction so in the end let us adapt what we already have and use our experts around it and we it's amazing it's super fun right now we're suppressed by them yeah it's that's very cool it does it is a lot of fun i love working this space as well um but uh so we're getting pretty close to time here i don't see other questions okay four minutes yeah i don't i don't see other questions coming

through in the chat um so this was like a wonderful talk thank you both for being here and thank you for getting up in the middle of the night to come do this q a session i hope the weather is not too bad and uh is there anything else that um you want to you know talk about before um we go to the next uh content i think we kind of covered everything i i just want to reiterate on around this idea to just go and try them you know this is one of the amazing things that engineering has you can actually be a squirrel and go to the next shiny thing and that little attention span that we have

when we go to another shiny thing it's actually positive it doesn't matter that we all have one million labs started and not finished and one million drawers full of you know those projects we just did one weekend those are the lego pieces we that's what creativity means starting though we don't need to make every time a masterpiece but at some point those lego pieces will come together and they're going to make an amazing talk or we're going to have so much fun creating it so just keep being curious is that that's what it's all about that's what it's all about awesome thank you both so much again and uh i hope you have a good night

thanks everybody thank you right