Beyond Quick Cash: Rethinking Bug Bounties for Greater Impact

please give a round of applause for our next speakers Jason Grace and parova who are going to present you their talk on Beyond quick cash reinking back bounties for greater impact just a quick reminder please ask your questions it's on the slider so you either had a QR code at the entrance or go at the besides uh website on the Q&A page and please and we'll take questions in the remaining time at the end of the session thank you and please a round of applause for our speakers check check hey bide San Francisco how are we feeling on this fine Saturday morning Make Some Noise beautiful all right we're going to introduce ourselves and get this thing

going yeah so my name is Jason very nice to meet all of you uh I'm the founder and a technical lead on meta purle team uh I've also got a good bit of experience in web application penetration testing and Bug bounty Hunning dating back to 2017 um I've earned about you know over 20 cves to date I've established responsible disclosure programs and have both triaged and submitted bugs across various platforms uh and before I got into Tech I was a touring death metal vocalist uh I don't tour anymore but uh this was from last weekend so we still out here having fun all right thank you so much for taking the time out and attending our

talk my name is Farah and I work on meta's bug Bounty program program before I joined meta I was working at buckout as a Trier validating bug Bounty reports and an occasional bug Bounty Hunter myself I'm super passionate about all things cyber security bug bounties and sharing my knowledge with these communities I do this by creating cyber security content on my social media channels by speaking at conferences and by collaborating with Hackers from all around the world all right so let's get into the talk uh the overarching goal of today's talk is to advocate for an evolution of bug bound programs so we'll start by reviewing the bug Bounty industry's current state uh we'll go on

to discuss Shifting the focus in bug hunting and how programs and Hunters can both adopt strategies for more significant discoveries uh then we look at meta's bug Bounty program discussing what we do differently and how it can serve as a model for elevating the bug Bounty industry um we'll go through some bugs that showcase the real world effectiveness of our discussions uh and taking a strategic approach to bug hunting and finally we'll conclude with some takeaways offering some key insights and some action items for both programs and Hunters to adopt um all right so looking at the current state we all know that there's no question that the market is growing but I want to give

some numbers just to provide some more tangible specifics so public programs have projected a market size increase from $200 million in 2020 to an estimated 5.5 billion by 2027 this is only for public programs um hackers that submitted bugs grew by 63% in 2020 alone and the average spend on bounties also increased 50% from $2,000 in 2021 to $3,000 in 2022 um in 2023 hakan's Community surpassed $300 million in total alltime rewards and I do want to call out that AI is still new it's still relatively top of mind for everyone and the first one of the first inclinations that we had as an industry was also to go ahead and start a bug boundary

program to see what the difficulties are what the nuances that come with these uh that come with the place um I want to talk about what the a bug looks like a life cycle of a bug from the time it's submitted and then triaged uh the first thing is speed I think you have to make rapid submission speed is of the essence in bug bounties Hunters are motivated to submit their findings quickly so that they don't get duped if their submission is labeled as a duplicate uh they don't get any reward uh and once a submission is received the triage team then assesses the impact of the vulnerability and they might use the common vulnerability scoring system for it this

is a standardized way that allows you to assess the impact of a bug and at this point in the submission process the hunter needs to wait because the triage team will complete their assessment and sometimes even resolution before they get their Bounty and this can be really like a test of patience you don't know how long you're going to wait for it depends on the organization's processes and on the complexity of the bug uh but there are some things that are going well for the industry let's explore those first before we go on to what the potential problems might be uh bug Bounty helps with strategic value because when organizations collaborate with a global community of hackers they

get access to Unique perspectives and unique talent that they don't have available inh house uh the bug hunting Community has also grown into a very supportive and unique ecosystem where Hunters are no longer gatekeeping from each other they're coming together fostering a culture of knowledge sharing of collaboration and continuous Improvement and I want to say that Safe Harbor policies are a total Game Changer in bug bounties because when you provide hackers that legal protection they are motivated to uh go further and go into bug hunting because they don't have to worry about going to jail so what is the problem what is the problem in bug Bounty right now because bug Bounty programs aren't cheap they're

not easy to run but good bug Bounty programs don't just happen sometimes a company might start a bug Bounty program look back at it after a few months and decide it's not worth the AR why and why is that because you often companies often prioritize quantity of reports over quality of the findings this is why they end up getting a lot of informational reports that are just like bordering on like the validity and then it feels like it's not worth the effort U they also have maybe insufficient triage support because the triage team is drowning in false positives and duplicates which makes it feel like it's too much effort for too little benefit um and okay let's look at meta's bug

boundary program because I want to talk about how we at meta see our bug boundy program it's the last point of catching a vulnerability and this is probably similar for a lot of other programs it's going to be bugs that are missed by your internal tooling that are missed by your automation because companies are shifting left product security is uh becoming a lot stronger so the landscape is stronger that's why newer folks that enter the industry might feel like it's harder to find a bug these days but I feel like the duty of bug Bounty is that you don't need to be a person of X years of experience x amount of degrees and like certifications on your resume as

long as you find a valid bug you're able to participate in it um on meta's program in fact some of the top Hunters have been students they're just doing this alongside their University uh it helps them pay off their student loans and also make some extra cash along the way um and at meta the bug Bounty program also influences security efforts across the organization for example if we see a trend of reports coming in maybe it's a report that is reporting similar issues on different apps that we have like messenger or Whatsapp then we get in touch with the product the security teams of those products we try to fix the issue more holistically so we

don't have to go around squashing bugs one by one and okay now I'll pass it off to Jason to talk about Shifting the focus hey friends all right cool uh so let's start by talking about some things that bug Benny programs today can do to start uh the Shi have to uh facilitate more impactful bugs coming in that are top of mine and ultimately make the whole program feel worthwhile first off we need a means to capture cumulative risk uh while the CVSs scoring system as uh I'm sure most of us know is widely used for bug triage it often falls short in assessing the combined impact of multiple vulnerabilities so consider a scenario where you have two bugs uh and when you

assess them individual you have a low and a medium as far as the CVSs scores are concerned now if you go ahead and just bring those two together you may have a critical all of a sudden uh now the problem is from the vanilla CVSs perspective we've got a low and a medium often times that's reflected in payout bug bny Hunters are quite displeased rightfully so uh so the good news is enhancing CVSs with additional information tailored to your company and its specific needs uh it's quite doable so you want to start by incorporating contextual elements such as asset value existing security controls and environmental conditions into your risk determination process for ref finding and building on this

consider how chained vulnerabilities might interact and affect your entire system uh and really is could be as simple as adding a checklist item for uh your assessors or the triers to consider the risk of all of the findings that come in together okay now uh this uh ultimately is going to require a little bit of a shift when it comes to triaging bugs so let's talk about that next to begin you want to develop a prioritization strategy that enables your triage teams to dedicate sufficient time to complex vulnerabilities the objective is to ensure that comprehensive investigations of intricate issues don't lead to your entire team uh falling behind on everything the last thing you want is a

single Deep dive to cost your entire program a bunch of uh stress and time and you want to go ahead and promote a culture that encourages triage teams to collaborate with Hunters when you're dealing with complex bugs or attack chains at the end of the day the hunter is the one that found it right so when you're triaging something you don't just magically know exactly how to do it like you got to take some time to learn it and understand it isn't it a lot faster if you just talk to the person that found it in the first place and work with them to fully understand it I think so um and lastly you want to evaluate

your definition for a significant bug by focusing on the actual impact of the findings rather than solely relying on quantity or perceived risk derived from a scoring system like CVSs that has not been enhanced uh and so this could involve developing new metrics for assessing bug sity it could be incorporating context into existing scoring systems like we just mentioned it could also be working out a joint triage process with the software engineering teams that are in charge of a Target um ultimately they're going to have a lot of great insights as well if you're not able to collaborate with the hunter now obviously what we're talking about here needs to be motivated in some way shape or form at the end of the day

bug Hunters are getting paid for their findings right so bug bny program should consider slightly revamping components of their pay structures to incentivize depth by offering substantially Pardon Me Higher rewards for vulnerabilities that possess a more profound impact programs can Inspire Hunters to actually take the additional time and energy that it require that's required to you know better understand the Target and it's also important to recognize and reward Hunters who provide unique insights and uncover uh really unique vulnerabilities uh these folks are thinking outside the box and they're thinking about things that maybe you aren't even thinking about so these findings if you reward them and show that you really appreciate them it can really help to provide you with a

lot more information about areas that you may not have thought about um also exceptional Hunters uh should be provided with any opportunities that you set up to do exclusive events this could be uh a bug bash with uh some of your network INF structure opened up to your Elite hunters or it could be a specific um app that you stand up for an event like bug bounty hunters love being able to come out and hack together it's awesome you should definitely do that and as I'm sure any triager in the room knows the quality of submissions uh they can vary pretty significantly and uh also the write up sometimes can be pretty tricky to understand there are

some bug Hunters out there that actually take the time to write awesome reports and you know what we should pay them for that uh if you think about it they're saving you a ton of time if they provide you with the nature of the vulnerability the impact of the vulnerability reproducibility in terms of the steps and propos mitigations I mean that right there you've got your triage started just off of that report all right now we talked about the program side let's talk to uh Hunters if uh any hunters in the room where you at oh all right got a few of them up there hey friends all right so the first order of business

is you're going to need to pick a program that not only interests you but also offers a variety of targets this diversity ensures that there's enough to explore and learn and really dig into which will make your time that you invest in the Target worthwhile and also when you're digging into the target you want to delve into the business and technological Frameworks of your target understand the key aspects of their operations their industry role Revenue mechanisms that one's important teex stack internally and externally and compliance requirements this information arms you with everything that you need to know in order to identify some of the worst case possible scenarios for your Target and suffice it to say these scenarios can be

quite profitable all right so let's say that you let's say that you discover a vulnerability with unexplored potential the first thing that you need to do no matter what you want to talk to the triager immediately before doing anything uh because they are going to provide you with authorization and permission the step is pretty darn key uh it ensures that your continued investigation is sanctioned that you're not tying up the company's blue team uh chasing after you thinking you're an adversary and ultimately it'll keep you out of uh prison which is pretty dope right now upon gaining approval your next step is going to be providing your plan of Investigation believe Believe It or Not companies aren't going to give

you permission to dig around wherever you want for however long you want to uh that's quite a dubious proposition so when you come with your investigation plan this needs to include a problem statement this will outline what you aim to achieve uh specific assets that you're targeting and a proposed timeline for the investigation to take place and as you're doing this work once you've gotten these permissions and given them your statement of work uh you need to record everything you do uh there's a few reasons for this one it aids in reproducibility which is great uh two it also provides additional information that can be kind of difficult to capture in a PC um and it also provides a clear

record of what you've done and that serves some nice cya in the event that your research activities coincide with a security incident and so uh to make all these uh Concepts a bit less AB ract uh let's cover a couple of examples uh which are based on some bugs that I found in my past first I'm going to drink some

water so hydrated all right so uh at a previous job uh a new internal search engine rolled out um and naturally I I couldn't help myself I threw some uh xss payloads some blind accss payloads and uh that resulted in a really annoying popup for one of the admins that uh deployed that thing um so that manifested in the ad in an admin endpoint that uh represented uh with cphs a lot of the different queries that were going on um so once we kind of socialized that with the owners uh we got additional accesses and uh I also discovered a XML configuration upload endpoint that was vulnerable to xxe cool so we've got some pretty good uh pretty

good things going on right now um and what we did basically is we brought these individual issues to the product owner and they said well I don't I I'd like to see like the realism of this I just don't I don't know if this is actually a thing it's like okay and so uh we basically used the stored cross-site scripting as an internal employee to compromise an admin of that particular system upon doing that uh we had a little bit of uh looping logic that would basically uh use the xxe to enumerate files on disc in turn we eventually found credentials that allowed us to get into the system and then boom there you go proof is in

the pudding and uh ultimately was requested so we were very happy to do it next story I was uh I was a Defcon 30 wandering around a few different Villages um one of them had handson demos and so I sat down in front of a laptop running miter Caldera and uh if you're not familiar with Caldera it is a open source framework that is designed to automate breach and attack simulations and uh one of the features of Caldera is the means to enroll systems that can be used as part of these simulations uh whether it's as an attacker or as a vulnerable system so uh after sitting down and messing around for a couple minutes I found a stored

xss in the operations page which is the area of the application where you configure these simulations uh I made sure to alert the Caldera Engineers that were all standing uh in that general area I let them know that I was going to dig into into it a bit more once hacker summer camp concluded so there was the timeline and uh you know uh fast forward uh a week later I discovered a couple more stored crossy scripting bones and reported all of them uh shortly after my last report uh one of the Caldera devs reached out and they asked for more realistic pocc uh they needed something that would help them to prioritize that work over the amount of other work that

they have totally reasonable ask so uh within two days I had figured out a way to deliver a weaponized exploit that leveraged one of these stored xss phones to facilitate getting reverse shells to all of the enrolled systems uh which definitely was a good um good indication of the severity of that risk um and so once they had this POC Caldera team was able to swiftly prioritize the issue and they developed a super robust fix in four days huge shout out to that program that is some awesome awesome work um and also if you want to learn more about the particular vul um there's a link there uh we were able to actually fully disclose it including the weaponized

exploit so check that out and uh back over to my friend far okay so in this section I'm going to try to go over what we do differently at meta in order to address the risks that Jason mentioned in the previous slides um in meta's bugb program we have taken efforts to understand the shortcomings of the industry uh so we invented firstly unique payout systems which don't consider CVSs they used an impact driven payout method uh payout decision method which prioritizes issues according to what meta as a business cares about uh we also prioritize the researchers experience when they hunt on our program and we do this by collaborating with them on reports where extra work might be needed to come up

with a full exploit or to maximize the impact from a vulnerability we even have case studies coming up where a researcher reported a bug that initially it might only have qualified for a minimum Bounty of $500 as per our payout guidelines but it ended up with almost $100,000 Bounty because of our investigation and how we ended up communicating that with our researcher uh even though our payouts might be among the highest in the industry uh you will notice in our case studies that transparency and consistency played a much bigger role they were also more valued by our researchers and they were also completely free to do uh cuz at meta's program if you submit a report

that triggers an internal investigation and we were able to discover additional issues based on your report then we pay you for the Maximum Impact that we found internally and not just for the issue that you reported um so let's go over some case studies I will go over two standout bugs the first one is going to be an example of how much of a difference it can make when you as a program communicate with your researchers and end up with a successful report and the other example is for researchers how you can put in the work chain multiple bugs for impact and give the program maximum value from one report uh so case study one is on AES

which is a JavaScript engine created by meta and an example use case for this engine is to render AR filters uh using JavaScript using user provided untrusted code and running that within the app but also this engine is open source so anyone can use it um we received a report from a researcher who demonstrated a memory corruption on this engine this was simply by running a malicious job Javascript file with MMS uh but at this stage it was only a crash this may or may not have been exploitable but without further technical details in the report it would only qualify for a $500 Bounty uh but whenever something like this is reported we raised this with our a security teams

and see if there is a possibility of rce here uh the team investigated they said that yeah there could be rce but without like more time investment uh without more uh low low hanging fruits we don't we can't say for sure so so we did go back with our investigation to the researcher told them that hey we'll give you some time come up with a full exploit full POC there could be rce here there's no sandboxing between the JavaScript engine and the product using it uh the researcher came back to us uh after some time with their rce exploit we confirmed this with the MS security teams uh and just to emphasize the impact of this rce when you have this

you have the same permissions as the that Facebook app uh so for example you can read uh access tokens you can read chat history files that Facebook might have stored on the user's phone you could uh potentially read smss if the user has given Facebook that permission um and yeah anything the app has access to on the on the device uh so this is how an issue that was initially reported as just a crash turned into a full-fledged rce uh on an open source JavaScript engine so if we as a as a program didn't communicate our investigation with our researcher didn't keep like that open Channel and transparency we would not have had this successful bug Bounty report here um

let's talk about the payouts here because we had just increased our rce payout guidelines just a few months before this report from $455,000 to $300,000 I do think that this probably what in this is probably what incentivized this research uh so they got a $90,000 base payout because it was a oneclick rce uh they also got some bonuses $6,800 for hacker plus hacker plus is like our loyalty program for our B Hunters because our researchers are placed in leagues starting from bronze all the way to Diamond depending on number of your reports the impact and your signal to noise ratio I think in this case the hacker was in the silver league they got a

1.75x multiplier which was $6,800 um you the higher you go you can earn up to 30% of the original Bounty with hacker plus and you also get benefits like paid travel and accommodation to Defcon to our annual live hacking event you get invites to private bounty for our unreleased features and also exclusive swag we also gave a $9,000 delay bonus because our reward uh took 110 days uh from the original submission date uh and finally there was a cve issued for this so feel free to check it out and okay the next case study this one was um one of our top researchers Yousef they submitted in bug an account takeover bug this could have allowed an

attacker to take over a victim's Facebook or Oculus account by stealing their first party access token uh the way this bug worked is Yousef chained a bunch of uh open redirects starting from facebook.com or Oculus Oculus and then finally facebook.com/ URL um the first one here facebook.com was an O endpoint the response type was set to token so the token passes through the whole chain until it reaches the attacker's website because the last one here facebook.com/ shorturl had an open redirect vulnerability the other redirect were all intended nothing bad happening there um an important thing I want to call out here is that uh this open redirect depended on the victim being logged in and logged out of some of these apps uh

but we didn't make any Bounty deductions for this weird requirement because we assume that login and log out csrf exist and those are not things that we're going to reward by themselves but they can be used for these sort of chains so no Bounty deduction for that uh let's see how much we paid for this uh they got a $25,000 base payout for two clicks uh two click account take over $5,000 hacker plus bonus and a bunch of other bonuses they were participating in our live hacking event so an event bonus special scope bonus they also won the award for the highest impact report uh and they also ended up publicly disclosing this so once again feel free

to check it out I want to give it back to Jason to give leave us with some takeaways here yes indeed all right let's talk about the things that we'd like you to home with you throughout our discussion we've underscored the importance of a strategic in-depth approach for bug hunting both hunters and programs this approach goes merely Beyond identifying vulnerabilities at a surface level and delves into understanding their interconnectedness and potential for complex attack chains that pose significant threats we've also explored the concept of combined risk which highlights the importance of a holistic assessment of a vulnerability that comes in this understanding allows you to understand that was redundant um this understanding allows you to comprehend

the true potential of a finding or a collection of findings uh which in turn leads to more impactful discoveries a lot more uh benefit from having your program uh and so in conclusion uh this is going to require both programs and Hunters to kind of advocate and push for this so uh Hunters I saw y'all are back there we obviously got a lot of program people think about it and if you have any questions we're around hit us up uh also if you need more motivation to come give us a nice crisp high five uh we have coupons available for folks that want to bug hunt meta you don't need to be a active Bug Hunter to redeem

this if you're aspiring here's your reason to jump in uh the coupon offers a additional reward of 15% for any valid finding with a maximum of 2500 bucks so let's say that you have a valid finding is worth 10K uh with this coupon you would receive uh an additional 1,500 bucks 15% of $10,000 making your total reward 11500 and uh these are all the resources that we employed while putting together this talk um that's it thank you all so much for coming to check it out and we really appreciate each and every one of you thank

you well thank you so much Jason faret was very interesting talk so we have some time for Q&A so if you still have if you still have questions please submit them where slido and the first question is do you think the recent open sourcing of back crowds VDP will help other back boundy programs with collaborations with Hunters yeah definitely I mean the more that we put out when we as a you know as a mature component of the industry bug crowd and hacker one as they put out more information about how they actually do things and how they function um that takes a lot less guess work uh when it comes to building a new program and

subsequently uh that uh really makes it a lot easier to start thinking about these more nuanced problems uh because you have the initial part of building the program out of the equation hopefully that helps you have any thoughts on that no I think you cool yeah good and the next question is from Camila so she's trying to create a national bug boundy program in Sweden that can help the find critical V hell yeah vant sectors like energy sector have you seen on your own National initiatives you want to take this one uh I think I've seen couple of national ones back when I was in India there were a few of them but yeah I

haven't I haven't ever seen like an industry specific I think you mentioned energy sector uh that's pretty cool I I do think that there are big players now uh that are kind of like controlling everything and uh you see someone like Integrity that used to be a smaller player come now and uh become big in like Europe so I think region specific ones when you understand the region you understand the pain points of that region uh then it can work but if you apply something like a global level business model to an region specific industry specific bug Bounty company maybe yeah that that that might not work okay thank you so much I'm sorry uh

we're at the end of our talk so give another round of applause for Jason Farah and I