Human Fingerprints: Building Better Tabletop Exercises with Realistic Adversary Tradecraft

top exercises and I think they can help um highlight gaps in budget or expertise and capabilities as well. They can also be used for um very intense security theater. So, um, please keep in mind that while my talk is, um, about kind of how to make these better with realistic adversary tradecraft, the goal is to include your players, to help them learn, and to, um, build relationships, not to show off how how much you know about adversary tradecraft. So, [Music] yeah, you can click it. Okay, awesome. So, this is a little bit about me really quick. I know we're starting a little late. Um, I live in Portland, Oregon. Um, I like doing artsy things. I like

the outdoors. I like lifting heavy things. Um, those are some of my security certifications. And I've planned and facilitated about 25 tabletop exercises in the last two or three years. My players have been utility companies, incident incident command team staff, security teams, um, high school students, federal agencies, uh, state and federal regulators, operational technology engineers, industrial facility operators. It's been a broad spectrum, but what I've learned is that audiences respond to realism in the scenario. And one of the best ways to get realism is through incorporating threat intelligence and adversary tradecraft into the tabletop exercise and really tell that story of an adversary in your environment and the choices they make or their fingerprints. That's a big part of making it

realistic. Next slide, please. Thank you. So, quick questions. How many of you have um are familiar with tabletop exercises as a concept? Wonderful. I love that. Okay. How many of you have participated in one? Great. Little few a few a little fewer but still quite a few of you. How about planned one? Awesome. Facilitated one. Awesome. So, the numbers kind of got smaller and smaller. um wherever you are at in your journey of um in being involved in these things, I'm hoping that this talk gives you something that will help you the next time you are you are planning one of these things or participating. Next slide please. So this is our flow of show for today. Um since exercises are

fundamentally stories, um this talk is in the format of the hero's journey which is a template of a story um one of seven elemental story types. Um so we will start with um at the beginning with your call to adventure. you've been asked to do a tabletop or you want to do one, what's next? Gathering your tools, kind of crossing the threshold and helping your players kind of ease into their um bad day that you will make for them. And then um how to put them through the ordeal, which is the bulk of the exercise. That's where I'll get into the human fingerprints um element, and then how to bring them back home with the elixir, which is the learnings that

you get from the from the tabletop. All right, next slide, please. Okay, so the call to adventure. So you want to plan a tabletop. Maybe you've been asked to do one. Maybe um you know, maybe it's an executive request, maybe it's a compliance thing. I've heard from somebody at the speaker's reception like, "Well, I have a policy that tells me I need to do one of these things once a year." Um but, uh regardless of where it's coming from, I highly recommend having an exercise planning team or gathering your fellowship. It doesn't have to be eight, nine people um or creatures in this case. Um but it can be um really people from inside IT or

security. They can help you map a viable attack path and understand your environment um underneath the scenario. Or if they're from outside IT and security, they can help you understand what kinds of business impacts a realized threat would have and they have a chance to learn about security. So it's really a win-win. So I I recommend having a diversity of perspectives in your planning team. This also helps you avoid assumptions that are built into your scenario that you will for sure the players will find them during the exercise play. Um next slide please. So gathering your tools. So what is tradecraft? Um it's a pretty broad term. So it is an adversar's tactics, techniques and procedures as well as the

tools that they use, how they use them and the infrastructure that they that they use to deliver their capabilities. So, it's pretty broad. One of the best sources of information that you can use for designing a tabletop is incidents that your customers or your clients or your company have already experienced. You know, kind of going back kind of recasting it in a slightly different light, but trying to extract more learnings. Um, this can is especially true because by the time you're through a real world incident and in the recovery stage, you just kind of sometimes companies just want to get that over with, you know. So there still sometimes can be learnings from past incidents to extract. Um but regardless

of what tradecraftraft you decide to focus on for your scenario, um knowing your audience is totally crucial. So um you know including if you include specific tradecraftraft, if you know your audience, you'll know how much detail to go into about that tradecraftraft because you know maybe your executives don't really care about uh malware compilation, you know, environment markers. maybe they care care more and will respond better to adversary working hours and patterns in those choices. Um so knowing your audience using that those exercise those empathy muscles there was a great talk on empathy uh yesterday about learning to see things from another person's perspective. I recommend doing that with your players as well so you can

understand what do they care about? what are their priorities and then how can you weave those into the scenario in ways that um will ensure that during gameplay they have some skin in the game and then other tools to use the MITER attack navigator is a free public tool. It's just another way to view the MITER attack matrix. Um you can sort by a thread actor to see what um what TTPs they use and it's nice and visual. I have a screenshot in a second. Um but again that's great tool. And then an honorable mention I don't have time to go into right now um is the Homeland Security Exercise and Evaluation Program or HEP. It is an exercise planning

framework developed by the the Department of Homeland Security and it is again free public to use. Next slide please. Quick screenshot of the MITER attack navigator filtering by threat groups, selection controls, search threat groups um into layer controls. you can export um as an Excel document and have like all of their all of the threat groups TTPs available for you. Um this is great inspiration for tabletops and you can totally base your you know the tabletop adversary off of a real threat group. You can also make up your own threat group. That's something I've done before. Um so making your own your own threat group you could take a TT ttps from a known threat group and then

kind of rebrand it. I once created my own kind of fictitious ransomware group that was built on cat puns and just had cat puns everywhere and in the ransomware note which someone ended up actually reading aloud. I didn't expect that and the players had fun with that. Um and they got some laughs out of it and that ultimately is is part of the experience as well. Next slide please. So you have your fellowship, your planning team, you have um your tools, some of your tools. Now it's time to start drafting your scenario. And I usually start with the known environment. So your first your first event in the scenario called your inject um should be something that is a

somewhat common operational event. So I like thread intelligence bulletins or common detections or even just a news article. you know that some maybe one of your players if you're if you're playing with a more strategic level audience maybe one of your players says hey um I found a news article and the group talks about it. The reason why I like starting with the known environment is it gives people a chance to settle in to the operating model of the exercise and to align on language because especially if you're involving non-technical non-security people in your exercises which I highly recommend doing um these are a lot to walk into. There's a lot of specialized language in security that

can be very daunting for new people in the room. So, giving them something to ease in to the exercise really I I think helps set people at ease. And but all along, you know, however you're establishing the known and then crossing into the unknown, um, which I'll talk about in a second, considering the adversary and their objective, um, what is their goal in your environment or the company's environment and how are they what are they choosing to do um, in that environment? So once you've established the known, um, crossing into the unknown by giving your players the first little inclination or indication that they're about to have a a bad day. That bad day should be fictitious but plausible. So

maybe again it's a detection firing. Maybe it's a user reporting saying like, hey, this server's down. Um, it could be, you know, anything, but crossing into the unknown over that threshold um is is an important piece of your scenario. Okay, keep going. Thank you. So this is really kind of where the the scenario starts to feel more incidenty. Um and this is where we get back to the concept of the adversar's choices or their human fingerprints. So this is just a few examples. Um but there should be a clear scaffolding in this phase of the the scenario of suspiciousness. So maybe your first inject is a news media article. Your second inject is somebody calling the help desk to report kind of

a sluggish behavior on a device. Maybe your third inject is hey we found um some suspicious network traffic on this server or something. Um but scaffolding up having at least three injects that's what I found has worked well um in this phase of the scenario. So, so human human fingerprints, some of the adversary choices you can highlight at this phase, mistakes, offsec failures. Um, these happen. Um, patterns in working hours and days or days off. You know, different different places in the world have different holidays and you can play with that. Um, IP addresses, kind of going back to the um the OBSC um idea of that choice. If you if your IP address, say you have a fishing email in

the beginning of your scenario and that fishing email has a sender IP, maybe that sender IP is the same as their C2 server. Kind of sloppy, but it does happen. Um, same thing for the email address. Um, maybe the email address that sent the um sent a fishing email or something was the same had the same domain as the billing contact on who is, which they forgot to make private. Um, so these are just a few examples. Also, naming conventions for malicious files and folders. Um, has anybody read the the book Sandworm by Andy Greenberg? Okay, a few of you. Awesome. Um, highly recommend it. There's a lot of, um, one of the threads that he pulls on that

helps with attribution is a series of references to Dune, the science fiction series. So, um, if you're playing with naming conventions during your, um, during your exercise, I recommend checking out checking out, uh, that book, um, because that's a great great theme and great example of adversary choices kind of fueling attribution as well. Hashes, this is kind of a a gimme, but um, if you find a malicious file and there are, you know, it's it matches the hash of another known known malicious file file, you can include that in your scenario. Also, there's a great SANS white paper about um human fingerprints in malware. So, like code reuse, header metadata, that sort of thing. Um we

don't have time to go into that level of detail right now, but again, great resource. I'll have it uh linked at the end as well. All right, next slide. So, uh you've thrown the ring into Mount Doom. You've put your players through their um their paces and put them through their the ordeal. um you've incorporated human fingerprints into kind of the the scenario and um what the adversary's done in the environment. Recovery is still a very important part of the incident response life cycle. It doesn't get as much um airtime in my experience as other phases like detection and in the initial response, but um an effective way to incorporate it in a scenario um and to further expose the

adversar's human fingerprints is by doing a time jump. So you can say kind of towards the end of your scenario two months into the future these things have happened. This is what we found. Um and then uh you can discuss as well. So this is also a great opportunity since that is complete guesswork for the most part um to validate your understanding of what would happen during the recovery phase and to talk about with your with your planners and with your players um what they think might be a priority after an incident. Next slide, please. Returning with the elixir, the learning from these exercises um should be immediate. So um there are some brains who like to immediately verbally process

when they go through one of these things. Um and they like to share what they think should be done and after action items and all of that. Um other people need time to process. So, I do recommend having maybe a short hot wash or retrospective for more verbal processors in the moment and then something longer, probably no more than like a week later um to accommodate other brains and types of processing. That's also another opportunity to ask your players about um the adversar's choices that you think they, you know, might have picked up on in the scenario or maybe they didn't pick up on them like why would the adversary have gone after this set of assets? Why do you

think they did that? those sorts of questions which can lead to really interesting conversations about asset criticality and what those players care about and what they think is a priority versus what you think is a priority. And that's part of just what I think makes exercises valuable is they're just an excuse to have a conversation in a safe no fault environment. Um, and then whatever your afteraction items are, whatever action items your group, um, your players, identify, make sure you document those in after action report, um, as part of part of the process. Okay, thank you. So, some of the key takeaways um yeah consider incorporating adversary choices and highlighting those in your scenario design. Um, what are they choosing to do

and why? There's a lot of great conversations that can come from that. Um, you can use intelligence about real incidents, either ones that your company or your customers have experienced or um or ones that you read about like in in Sandworm. Um, also leverage MITER attack um and the navigator. Those are free wonderful tools. And then just getting creative and and if you can have fun with it, like it can be an enjoyable process, especially with a good group of planners. Um, but most of all, meet your players where they're at. I already mentioned the language barrier that can come up but with uh with you know folks who don't live in breathe security all

day which is most people um and tailoring the experience and the language that you use to describe adversary behavior to things that they will understand and to a language that makes sense to them is is what this is all about. Um and then be ready for a lot of questions. you know, there's um in my experience for for scenario design, you'll usually throw out more than you keep. And you'll also have it's kind of like the tip of the iceberg in um the things that you surface to players through the scenario is just the impacts that the adversary has and the observable artifacts. Um otherwise, there's a whole lot of other stuff that the players might never

see, but they might ask about. So, just be ready to expose more of that iceberg. All right, next next slide. Thank you. Um, these are a few of the resources that inspired me and this talk. So, AP1 is a pretty historic um, cyber threat intel paper um, by Mandant which dealt in attribution of one of China's cyber espionage units. There's that SANS white paper um, also Sandworm. Thank you, Andy Greenberg. And Cuckoo's Egg by Clint Stole. Somebody else yesterday, one of the speakers, uh, Cat mentioned Clint, uh, mentioned Cuckoo's egg. And I love that because it's a very accessible narrative of somebody who is observing adversary behavior in real time. And to me, when I read that book, that

adversary felt very real. It felt like they were, you know, Clint was like, not that I know Clint in person, but dear Clint. Um, it felt like he was, um, you know, really interacting in some sort of way with that adversary and making it feel real is it was really great. And then of course the SANS cyber threat intelligence class is wonderful. All right, next slide. Thank you. So that's that's everything that I had for you today. I think I'm good on time. I have a I have time for a few questions. Um that's my email address if you want to argue or just discuss anything that I shared today. Please reach out to me. I

would love to have a conversation. And that's my LinkedIn as well. That is my link to the conference talk from today, the the PDF. Um and it's in a folder called conferences. There's just one talk in it because this is actually my first security conference talk. So, thank you all for being my first audience. Really appreciate it. Yeah. All right. Do any of you have questions? Yes. Um, the last time I did one of these talks was for a small company. The tech company was 20 employees. The technology group was about four. That was outsourced. So by the time I had finished reconnaissance trying to figure out what the staff was and where an attack might occur and

identified an opportunity players all knew what I was thinking. Any advice on how to deal with a situation like that? I would say just include them in the planning process. Then you know in in an ideal world you would have uh some people in your planning team and then some people as just players and so they wouldn't necessarily have spoilers but um in a small group like what you're describing you know just involving them from the get-go. I don't see the harm in that. Um because then you can still talk through and maybe that changes your learning objectives too for the exercise of um you know what the goals are for your if if you already know what the exercise is

going to look like then you can just start to think about in advance what is the response process look like and maybe you can focus even on recovery too because that's a more of an unknown. Awesome. Any other questions? Yes. Just wanted to ask how many hours do you plan for planning and how many hours plan for the actual exercise? That's a great question. How many for the write up? How many for the write up? Oh my gosh. Well, co-pilot's been helping with that, but um so so hours for planning. Um so the number of players as that grows, the amount of planning time expands as well. So um I like to keep and I recommend

keeping your exercises small if you're just starting with this, like you know, 10 to 15 players max. Um smaller than 10 can be really enjoyable and fruitful as well. Um, I find that at least three one-hour planning meetings and each of those probably takes an hour or two outside of that um to to work work on the scenario and draft um kind of preparation materials. So at least 10 hours for planning I'd say. Um and then I generally unless it's like back doors or and breaches or something I I don't like to spend less than two hours in exercise play. Um, so a two-hour exercise and I've done four hour exercises as well. And then I've helped

plan but haven't facilitated an eight hour one as well, which is much longer. Um, so yeah. Does that answer your question? It does. Okay, awesome. And then who else? I thought I saw more hands. Yes, I think you were next. Yeah, you stole my question about about time. Yes, it is time intensive. I had one scenario where I had a PR gal. She was really really good at her job. And I banned her from future future prep future tabletop exercises because she was she would just take over and answer all the questions. Oh, so if you get, you know, maybe tell somebody to shut up or something like that. I didn't at the moment, but looking back on it, I kind

Yeah. Yeah, it's a it's a facilitation I think is a a bit of a an art and a science. Um, so balancing input, you can also what I've done before for very senior participants who will know all the answers. I've said this person's on a plane and they're unavailable and I put that in the scenario and um maybe I I have them going to Iceland or something and then the next inject I free them up um and I give them a heads up in advance of course that I'm making them unavailable. But that can be a a good way to um to help other people learn as well as asking that person to send a delegate too. I've done that as

well. Any other questions? Yes. How do you adjust the difficulty of your exercises as groups get better at their incident management process? Yeah, that's a great question. So, um, exercises in general, like if there's a if there's no process, they can help figure out what the process is, but if there is a process, it can help mature it and validate it. Um, so you can always, uh, mature and validate that as much as possible. And then if they're doing really well on discussion exercises, you can make it functional, too. So, you can actually emulate some um, some threat actor behavior in your environment. So, they can validate the detection part of that in addition to

the response part of that. Um, yeah, that's one idea. Yeah. Any other questions? Yes. Do you have any recommendations for uh prop generation for like a smaller uh tabletop just to generate the screenshots or to generate you know some of the fake emails and aside from just use co-pilot or chat? It's a great question. Um besides generative AI and just hand like hand producing them um I've created like fake alerts from like state agencies um and had like this is an exercise like in as a watermark. Um but yeah putting putting an official looking logo there's a lot that can be communicated with just like here put put a put a put the state or federal seal on

it. Um so I'd say keep it simple. You know, you could spend a lot of time on props, but I actually haven't incorporated a ton of those um in my exercises. Yeah. Yes. Oh, yes. Yeah. You haven't asked a question. So, kind of back to the small mediumsiz business scenario, though. Yeah. One of the things I've struggled with is just convincing stakeholders that this is worth it to do a desktop exercise. Usually, they're we've got great retros and we've had some incidents, so we're talking about those. Why do we need to spend this extra time engineering time doing? So how what we get out of this how can I take that? Yeah, it's a great question. Um so I

think there's always the argument of this helps with muscle memory so we can handle incidents better um when we when we practice together. Um there's also compliance requirements if you have those um as well. And there's um a lot of sometimes you can f it depends on the risks that you're trying to address with the tabletop and sometimes these risks seem like um very high impact but low probability. So I think having the question or having the conversation about the probability of something happening um and then exercising your response to it that probability conversation is probably what's holding up your stakeholders if I had to guess with my very very limited information. And that's that's a hard conversation to

have, but um pulling pulling things from um IBM's um state of the the data cost of a data breach report is great. The Verizon annual report is also great. Um and assigning a dollar value to like hey and also a dollar value to dwell time like if we don't catch the adversary in our environment um or our response is is inadequate or ineffective this is what it will cost you in dollars and cents. Not a perfect answer, but that's what I got for you. I think maybe one more question. Yes, kind of that similar line. What do you measure to show people make progress through the tabletop exercise? Yeah. So, um that goes back to

learning objectives and I may have skipped talking about that actually now I'm thinking about it, but all your exercises should have learning objectives ideally no more than three that should be specific, measurable, actionable, time bound, all of that. um and whether or not the exercise achieves the objectives is one success metric and then collecting feedback from folks um who go through incidents and asking them if they think the tabletop helped them respond to the incident better. So that more of that qualitative um feedback as well. I think that's all we have time for. I want to get out of here for the next speaker. Thank thank you all again. All right.