Scaling a Bug Bounty Program

time to move on to catalin next speaker is catalin that works at vismas as product security engineer and he specializes on infrastructure and product security along with a strong focus on security operations cataline is also the oauth team mizora chapter leader and accepted our challenge to speak it besides portugal about how to scale the big bounty program catalin yeah thank you so let me share my screen

uh suck

okay uh are you able to see my screen yes okay perfectly moving perfect so uh first of all uh uh i'm really glad to to be here at the first uh b-side sport edition uh and hope that maybe yeah uh in the future we will meet in person after the pandemic ends of course but who knows how this will go so today i'll i'll speak about scaling a bug bounty program from scratch and uh first of all uh who i am uh i am um catalin colorado uh i'm not having a developer background i am having a system engineering and networking background and in the last couple of years i've been specialized in doing a lot of product security stuff

in bisma where we have lots of developers over 2 000 developers in many countries and also we have lots of salaries delivery teams which is quite uh interesting based on the approach that we have because we are a company of companies small companies we are a big fish in a small lake uh besides that i'm uh the chapter leader of vast mishara where i can say that i aim to create a strong local security community because we all want more uh products that are secure and we all want the environment that could be developed into something uh something else besides that uh a couple of passions that i have uh like cycling reading and sometimes breaking a couple of stuff

doesn't matter what stuff but a couple of stuff [Music] so the story begins with once upon a time in a far far away galaxy between some some forces but okay just kidding it's not that kind of story and definitely is not that kind of battle of forces uh sometimes there is definitely a battle of forces but there is another story between those those battles so bug bounty what what it is and um what what can can we achieve with it most of you probably know about what bug bounty it is but the story is about bug monty and the tips and tricks that you can [Music] on how you can scale it from the start

and the security posture that you you will need from your applications to have usually when talking about bug bounties uh we here talks about um hackers uh especially from the hacker's perspective not only from um their size but i think that we should also concentrate from the other side from from the company's perspective the challenges that they have to to scale it and to to gave those awards to the hackers so uh bhagavantis uh what is it most probably you know you you heard about hacker one [Music] but of course there are other platforms like background integrity scenic uh very big platforms but today i'll focus more on hacker one because in our journey in visma we've chosen hackerone as a bug

bounty platform so uh bug bounty what it is um it's basically crowdsource community where you can gather lots of hackers around the world and those are very specialized hackers with specialized skills depending on the sectors for example you can have experts on java or other other languages also for example one of the strengths of the bug bodies is that the testing it appears that if you you will pay uh for uh those bounties those holes that you have into your applications um and you will pay only for what you what you'll have in in those applications uh basically in the bug bounty you have two options you have public bug bounty and private bug bounty

in the private one you invite only the hackers that you want only a limited amount of hackers you can invite you know 10 20 100 1 000 or more and of course there is two options for managed or not managed what means managed managed it means that it's managed by the platform if you choose for example integrity they will manage your uh your uh reports in the first triaging part if you choose for example uh hacker one they will manage the first reports and they will see the if those reports are valid or not and comply with the policy uh if you choose to not go with the managed by the platform then you'll need to do all the work to try

and to do the initial assessment uh based on the policy that you have and after that to go deeper uh with all the uh work for validation um replying to the hackers replying to the service delivery team and also uh to reproduce the poc that the hacker is providing so uh why you want to do this uh bug bounty basically it's a real great and proven way of uh of battle of testing let's say like that uh of a service with ethical hackers around the world who will pay to report security vulnerabilities to us the strength of the bug bounty program is with the number of eyes and expertise in the techniques and technologies used

here in the black bundy program that the hackers will provide so if you will have more hackers more findings you'll be a very good posture of security so you will have a better security if for example your service is an internet is on the internet most probably it will get hacked anyway so by utilizing this uh you will receive reports of vulnerabilities by ethical hackers in order to improve the security in uh in your services so a couple of things that needs to be mentioned is that we actually live in um in a crowd service security area these days things are a little bit different and on the security scene as well so no longer

we rely on only on the tools and uh on the methodologies and penetration tests we we rely also on the community of security researchers uh and for example we also rely on uh cyber truck intelligence which is also one of one important aspect into into security uh besides that uh i want to mention that the world is changing and what you can see here is that uh most of you probably uh recognized a couple of uh people from this picture uh hopefully uh because uh these are uh well-known hackers and we can see now hackers of the live event uh where they were hacking specific targets of a company that then paid out a bunch of bounties

for the findings they found this is a real uh powerful because perhaps one day your company or you can show a picture similar uh to this and tell your customers that here are some of the security researchers you use to secure your product but [Music] these are not all of them this is really important since all of them couldn't fit into the picture of course besides your regular security team which is mandatory then maybe you are able to share values and get an emotional investment from them from your customers you might even want a few hearts to increase the the funding uh the world is changing actually so uh like you like like you see in this

picture uh this is the biggest uh the biggest uh step into the bug bounty because only the big players into the market are are doing this so um i hope that we share values and this can be your dream or maybe even your reality in [Music] recently there was a uh an event like this for a real big payment company so this went really good but it wasn't due to the pandemic it wasn't uh live hacking on person but it was was real cool because they found lots of bugs okay uh sounds really cool now uh the next question maybe for you is that can i just start can i just you know jump into it of course yes you can jump into

it if you have uh lots of money to burn if you have lots of money from your investors or from your board otherwise no it's not indicated the first recommendation is to build an exec program so uh to start our journey uh i will tell you about how we started in in this month we started really small in october 2016 uh to a small company because we are a company of companies so we started with a company in finland with a private program through hacker one so we choose at that moment we have chosen only uh hacker one uh there were multiple assessments but the latest assignment was that hacker one is the best uh okay uh

went a couple of years that we figured out that we need to scale it to go because the company got bigger and bigger and we figure out that we need to scale it to include lots of services to include uh lots of products that we deliver to the customers and how can this be achieved to be with all the services uh public uh to hacker one uh or to other uh platforms uh we've chosen and in the beginning only uh four assets uh and four service delivery teams so it was pretty small all in finland uh but there was an uh there was a dream like someone said there was a dream so uh why we want to do this uh are are we

crazy um maybe maybe yes but actually no because uh all the big companies all the big companies that are in the market are already doing this so companies like facebook apple microsoft are doing the bug bounty and they are rewarding the hackers for doing this with high amounts of money so uh we wanted to to start small but to go um uh to the next level so we are on the internet uh we uh are getting hacked anyway so why not to pay for those uh findings if of course the hackers will find the bugs before the actual black hats will do so uh i like to tend that we will get minimized and the the risk of being uh

pawned but i like to think about the bug bounty as a uninsuring that we we play a preemptive insurance uh program so uh a couple of years ago we developed inside our company we developed uh an application security program uh which we think at this moment that it's one of the most advanced and interesting security programs in europe based on the methodology and approach that we have and what was the approach we targeted of course security privacy and complete transparency that was our main target so to this we developed uh the wasp like we like to call it the this application security program based on the software uh security maturity model uh from the bottom up approach the main services

that we offer to the companies that we have because like i said we are a company of companies small companies we offered services like security trainings we offered like ssa so here we you can see lots of acronyms most of them maybe you you are thinking okay what's with psa what is array what's with mava for example so for example we delivered services and solutions to the themes like security self-assessment like um privacy service assessment like risk assessment array like security and privacy incident response we've built and product security operation center for incident response we've delivered cybertech intelligence to all the services or we also delivered to the themes if they want to board for the source code analyze for the sas

we delivered the dynamic application security tests uh if they want on board also we have uh mava which is manual application assessment also automated third-party vulnerability service and the last one uh which is more interesting and which gives you the the posture of your application is the security maturity index if you are on board to many things like i've mentioned uh and if you remediated all the issues that you have or let's say are into your backlog you'll get you'll get an index you'll get a level of maturity into security so uh in this short animated game gif our internal maturity index tool is displayed and we are looking at one of our products we can see that they have over 1 000

penalty points penalty points are given uh basically when the team fails to do some tests at current they they need to go into details of the product and then we'll check on details to drill into the details of specific vulnerabilities uh in this case there are just recommendations but the same process can be repeated for any product uh overview to any vulnerability this is what we are able to do for example in visma for full transparency from top level uh to the way down and also from the bottom to to up because in this way we will see if the application for example it's uh platinum if it's golden here and what is the required tier

and if they remediated all the issues that they have then yeah they are way good to go to the bug bounty so uh scaling uh and the story goes on uh are you ready now to go for bug bounty uh so how how can our application be ready for bug bounty if the application have all the prerequisites uh for from the application program uh if it's enrolled or not uh to all the services that your security team can offer and have zero vulnerabilities uh zero no issues then of course you can say with confidence that is ready for the private program another important topic in the success it will be the engagement that the you

will take for time to period time to respond time to fix and the more important for for the from the hacker perspective time to bounty if you have four days for example that will be really cool from uh from the hacker perspective because they'll be more engaged this creates engagement from the hacker's perspective and they will be motivated to go deeper uh if you have many assets they will dig and find other abilities so uh the story begins in uh january 2019 we've managed to get the approval for uh funding the colleague of ours managed to get all the fundings uh we had 10 of the best hackers in the world invited many hackers were

invited to the private program and started with uh one team one service delivery team with three assets in scope in the first day we received seven reports um i think that the main thing for success is transparency so transparency with the hackers uh disclosure of vulnerabilities uh also transparent with the service delivery teams so that they they will learn from it if they will learn from it then they will not create other issues after they will release other features so this is uh really really important so what are the responsibilities of the product security operations teams here uh basically our approach was to to have a program managed to have a managed program by ourselves not by

hacker one so we wanted to do this because we were thinking that it's really important that we will keep the communication and uh also the engagement with the hackers so we were the line between service delivery teams and hackers also we we managed the the pay the payment for the hackers which was really important or at least it seems that it's it's it's important so what happened next we added more hackers month after month we added multiple assets we increase the support and engagement through social media to twitter to slack because we've seen that constant communication with the hackers it provides engagement and it provides actual uh digging into the findings and into the application

that we have that are public so i think that we can say that we have very good dialogue sometimes uh i know that there are different opinions about managed by the platform or managed by yourself one of the challenges is that you receive lots of not really good reports in in the first phase but if you are constant with your replies and with your manner of replies then finally you will get what you'll need really cool reports with really cool findings so uh after january we received quite an interesting report for example uh in um in the same day we received uh this message uh and what's really interesting with this with this message is that the same guy

told us that uh he tried to get uh back to hacking on uh of our teams but got discouraged because there is absolutely nothing to be found and after a couple of more minutes hours that was it uh one 1000 bounties so what's maybe the most important thing for for the hackers from the hackers perspective it's the bounty amounts so we were thinking to have a reasonable body amounts based on the budget that we had for the critical ones 3000 high for uh 1500 for medium that's not really high when you compare with other programs that are paying much in comparison with us but we've had lots of reports uh and lots of issues is covered like you can see for

idlers you can see crosstalk scripting stored cross-side requests forgeries so we had quite interesting uh findings uh these years one important stat that i want to show you is the journey that we had from beginning of 2019 till this quarter so this is really important because we had quite a good engagement so if you can see in the beginning we had only 12 submissions okay of course our team was not so big at that moment we are glad that we didn't receive 300 reports in q1 but you can see also the number of reports you can see the valid numbers and you can see also uh how many uh what was the percentage of the valid ones of course

the percentage decreased for example this year but also the number of reports uh increased a lot so this is really important for us that we are targeted by the hackers and we received quality uh and maybe you are asking okay what changed in our landscape why we received this year so many reports or why we decrease the valid percentage there is a there is an answer a good answer this year we get mature with uh with our assets we went from private bounty program to the uh public one so uh in the public everyone can hit you everyone can target you so there will be more noise and you need to be prepared for it of course

but after you you are getting prepared and everything it's it's scheduled you'll know what to do of course if an asset is getting few reports if it's mature enough then it's a good approach to go to the public to be targeted by everyone because in the private it's more up to you if you want to invite 10 hackers if you want to invite 100 hackers if you want to invite 1 000 or 10 000 hackers so this is more up to you so the next maybe you're thinking what okay what is the what is the benefit of this or what is the cost of this there is a real cost of this because we paid

quite a lot of money so you need to to have good amount of money prepared because uh like you can see here uh in uh from last year to now we had uh over uh 1000 reports 1200 reports but it's the total amount of boundaries it's what matters so we paid quite a good amount of for bounties based on the conversation that we have with the hackers what's really important is how fast we paid bondings and we are paying in uh two days maximum three days uh the boundaries so this is real cool because we have a very fast striking and we are paying the hackers extremely extremely fast so there is a very good engagement between

us and hackers so this is um this is cool and this is a definitely a recommendation for for you if you want to to to run a bug bounty so uh if you can see here it cost lots of money so what's the purpose of what's the benefit so a of course there are benefits because you have lots of cool cool bugs for example we received a really cool bug based on server request forgery and that was chained with an html injection uh was really cool because based on this uh we received quite an interesting uh interesting response so this was the report that we received with the endpoint with all the steps with all the poc

and this will flick so um this quite hurts because we licked the abs key but we are glad that we find it before and not we but the hackers found it before the actual uh black hat hackers would find it so there are benefits into doing this definitely i will highly recommend it because you'll find all all the cool stuff before other malicious entities will target you the second one it was an rc the knot yeah rc you you know what's uh north c is it uh remote code execution and um yeah we we had it unfortunately of course so uh we had this uh we received the report we were quite concerned when we tried to reproduce it

reproduce the poc and to see if it's valid or not and when we've seen that then that's real real valid we've seen the request we've seen what we can what we can have to the code execution and really worked unfortunately of course it was a critical vulnerability and we paid the the amount of money for the critical one so major takeaways here from uh from our story the first one or maybe the first important one it will be just do it just do it it will be free maybe you'll fail in the beginning but it will be a good journey for you just do it but of course just only if you have money only if you

have money if you are making the assessment because there are some key points that needs to be considered before that so you'll need to have a very strong application security program store small with uh with baby steps but uh start let's say the that's one of the most important things uh and uh uh what what can uh i only say is that you need to be confident to not have too many issues only after that you will need to go uh for example to to the private program and only after that to the public so another important aspect is to add multiple assets if you add all your public assets then it will be good

because you'll have a better uh better posture also uh knowledge is uh is the key treasures for example should have at least a basic understanding of every vulnerability type otherwise it will be very hard to determine the correct severity of the report we can't know everything for sure that's that's right but the great attention of for detail uh for researchers uh spent writing their reports uh helps to understand and also assess the correct impact uh if a new vulnerability for example type is released or found a new cde uh is the responsibility of the treasurer to understand uh this new vulnerability type and if you want to go really big then you can host a live hacking event like

like i've showed you uh but that will cost you lots of money uh i can tell you that for sure um of course after a proper assessment maybe it's worth to to pay uh the bug bounty uh as an insurance uh just to not be not be packed stronger success factors it's uh transparency and communication also engagement be communicative and transparent with the researchers and the service delivery teams it's important for us to uh to feel that researchers uh feel like they are talking with the person not uh someone sort of a robot for every decision we make as a treasure a detailed explanation must be given to the researcher that's highly important and it's really important also

the service delivery teams um for the for them it's uh important to assess it properly if for example they need attention to fully understand uh the impact of the vulnerability uh i highly encourage you to schedule a call with them to just make sure that they fully understand it and they can uh they can reproduce it and it's unders it's understood so live life goes on uh you'll have more secure applications uh you'll be more confident based on uh your findings that you'll have into your bug bounty program if you have all of course all your public assets to the bug bounty public program developers will get into the hacker's mindset and what's more important

all the features that the developers will design it will be secure by design also it will be security or top level priority so everyone will be scared but that's good so uh thank you for your attention and please wake up thank you catalin thank you for sharing your experiences on how to improve application security on through bounty programs very very nice to hear that experience then time for question and answers at least looks like any question appeared yet by public okay now from here i have here two questions from the team um how are companies and the county hunters protected by by these platforms initiatives from the legal point of view because an attacker can be

in different countries the company can be in different countries the the admin or the platform could be also on different countries and on your experience all those all those lego questions legal questions are being dealt with yeah so uh there is a very good question because uh for example i have a uh both experiences as a a guy that protects the companies but also guy that it's targeting the companies you will adhere to the platform policies so for example if hacker one it's in america uh you will adhere to their policies if integrity is in belgium then you will hatch their policies and you will acknowledge that okay uh i agree with your policies and we i will comply with your policies

more than that it's uh highly important for the security researchers the hackers it's really important to read the policies and if they comply with the policies that uh they will find uh into the platforms uh from the programs uh from the uh this program from facebook program from other other programs then they will comply with those if there are some out of scope vulnerabilities then they must not target those vulnerabilities also of course there could be some cases when uh some of the issues are well known and are in developing but it takes a lot of sprints to to be remediated thank you for your presentation

you