PG - Interrogation Techniques for Fun and Profit: Designing Better Tools for Your SOC Team - Karolyn

I'm gonna turn the presentation over to alright girl thank you very much thank you thank you all for coming I know there's a whole bunch of other exciting talks going on so thank you very much for showing up to mine my name is Carolyn bachelor I'm a user researcher and user interface designer primarily work in the field of enterprise software and security software I'm an independent contractor or consultant at this point with Brazil research and design and I will move on from there so let me give you a quick overview of what this talk is going to cover I'm gonna talk a little bit about what UI designer is doing giving a talk of 'besides talk a little bit about

building tools and then get into the real meat of research and interrogation techniques so what led me here okay story time let me tell you what design is in an enterprise software environment so we start with research we'll go out we do some research we try to understand what our users are doing what their goals are you know a lot of important things you need to know when you're building a product for them so great we've got a good base there go back create some you know wonderful you I work really hard on you know putting together a design that takes a you know usability into consideration workflow user needs accessibility all those great things that help make things really

usable we prototype it we test it we tweak it we test it some more we tweak it we test it we have users going yeah this is awesome when this product releases its gonna be so cool it's gonna solve all my problems great so we go back and we get to development they start working on it everybody's happy everybody's cool right up until we go oh hey look the deadlines coming up and we're six months behind which is pretty much always and then it's a case of okay do we push the deadline do we drop stuff well pushing the deadline is never an option you know you've got VP's on your ass you've got you know VC's on your ass

you've got board directors on your ass and they're all going yeah we've got investment in here we've got customers waiting there's no way you're missing that deadline so start dropping [ __ ] okay so you got your feature list those features have to be at least compatible with your current your competitors features so you can't really drop any of those you can kind of like kind of degrade them a little bit but you can't drop them then you've got the features that have to be a little bit more than your competitors so your sales people say hey we're better than those guys so you can't drop those but then you've got usability and you've got all those

cool workflow designs and all the you know accessibility and little things that have make life better well they actually take a fair bit of work for the developers to do so what do you think it's dropped all that cool stuff that you spent all the time designing and we're so excited about doing and it just gets dropped oh but they promise you they say oh you know what don't worry we'll put it in the next release yeah [ __ ] never happens never happens because the next release of course is another set of features so you know instead of instead of when the product gets released you know instead of being able to go yay I'm part of this amazing

product and people are gonna love it and you know I'm gonna be a star and on the wall of fame and it's actually god I hope nobody knows I actually was part of this product and you know well shame the whole nine yards so it happens it happens over and over and over again it gets a little discouraging sometimes so what I really want to do is find a way to get that cool stuff out to the people that need to use it like the analysts the end users the ones that are getting ripped off so I thought well [ __ ] why don't I just take it to them so and I don't mean

go out and do the design for them but go out and teach some of the methods some of the techniques some of the tricks that I've learned over lots of years doing this and teach it to those people you know find a way to incorporate it into those in-house tools that everybody builds because in-house tools are there to compensate for all the crap that the product people didn't do you know bringing all that those data all that data together doing the integrations that the products won't do because they don't play well together so basically just helping people make those in-house tools take them to the next level so what I'm going to talk about today is

one of those tools which is upfront research now that sounds like it could be like really time-consuming and like extensive but the fact is it's not I mean it can be but you can get the biggest bang for a buck in actually a fairly short set of questions working with the people that want the tools or even working with yourself because you want the tool so to give you some background let's think about when you're building a tool for yourself there's a number of questions you can ask yourself and if you answered yes to any of these then these are cases where doing some upfront research is actually going to help you answer these questions and

you'll get a lot more out of what you create do you want to share it I mean you can build a tool for yourself but if you want to share it with somebody there's a little more going on it's a little more overhead required to actually share a tool should I maintain it okay so I've got a tool things change this might be a tool that I mean it's going to be good for a month and then maybe something changes that makes tool not useful anymore or maybe I don't know there's all kinds of reasons to maintain something and if you want to maintain it that's something again to think about is it going to be useful to

others this is different than sharing it the question is would other people actually want to use this tool that you're making and if they do are they going to use it exactly the same way you would use it or can you maybe modify or add a few things that would make it more useful to a broader number of people how many versions of this tool are there out there I mean how many times did you sit down and go hey I need to quickly you know put a script together to do X Y or Z and you do it and that's great and then you find out this person did something similar and that person did

something cylinder and that person did something similar but it was like totally killer in it five other things too when you build tools for other people it's the same set of questions but it gets even more complicated because you have to understand what they need it's not just about what you need it's about what they need which means you know either you're going to guess based on a you know casual conversation or a few sentences or you know you actually might want to do a little bit of research in the background to find out what exactly they're looking for this is a classic old cartoon for the design industry and it it actually is true right across the board

you know you get a solution from your your user they say they don't tell you what their problem is they say we want X okay fine we'll build you X ish well turns out X ish isn't quite what they're looking for and in fact X really wasn't what they were looking for they just didn't think it through what they wanted was something else actually what they really wanted was something totally different but they thought it was so far out of their budget they didn't bother but you know what it's actually worth finding out what like the way out one is because who knows it's entirely possible to send the whole team to Hawaii so okay there is a little

overhead in doing research upfront so what's it gonna buy you well you're gonna learn how to determine the value of your tool before you actually build it you've got maybe drawbacks and restrictions to that tool that are actually going to limit its use some of those things will tell you whether you actually truly want to take the time to build an and this is probably more in scope of something bigger like not it you know not a 15-minute script or something like that but if you've got something that you're going to put a little bit of effort in understanding what value you're gonna get out of it is gonna go a long way to deciding whether

it's actually worth time you know worth wasting the time who to build it who you're gonna build it for you know what are their needs and you know last but not least toll validation and ROI because the first thing that you probably know when you want to build a tool and it's something that's going to take more than you know a day you probably have to get approval to be able to take the time to do that well how do you do that you've got to prove that there's a bloody good reason to spend the time doing it so you know is it gonna save us time is it gonna save us money is building it can be

offset sufficiently it actually weren't that if you do this you can just walk up with the answers just go hey look it's good let me do it it's gonna be cool okay the research interview this is how we get there so just in general there's a number of things you have to think about when you're doing a research interview by the way I can hear this sort of going in and out can you guys all hear me okay okay good okay so an interview it's not a conversation although you do want to have a comfortable rapport with the person you're talking to so it's it's structured it's a set of question you want to keep the comfort level there

so they're not just giving you like yes/no answers which are completely useless you want to kind of engage them now the biggest thing you're gonna want to do is listen it means shut up you ask the question you wait for the answer you don't prompt them you don't like put words in their mouth you don't like make faces like oh god you know it's like you don't want to you don't want to skew your results that way but by basically contaminating your information and then then there's the uncomfortable pause yeah that one so you know the one that everybody wants to fill so don't fill it don't feel it if there's the uncomfortable pause like

bite your tongue you know put your hands down don't say anything because the person you're talking to if you don't feel it they will fill it and they will fill it with something that they actually had a few minutes to think about and it will be better information it will be richer information so I'm gonna like whip through the list of questions in the questionnaire super fast cuz there's no way in hell I can get through all of these in you know the time I've got allotted and then what we're gonna do is we're gonna walk through a few and do kind of a little mock interrogation interview okay what are you trying to do so this one gets to

what the goal of the person is trying to do with the task that they're you're dealing with right now what is the problem or pain point with your current method this is about where the blood is okay where are you bleeding walk through the steps of the task for me describe in detail where the problems are it's like okay you're bleeding where are the blood drops show me how do you manage this problem bleach I used what's your idea our solution okay see me or not they may not have an idea or solution but that's okay because there's some brainstorming that could come in here it's you know that's all good can you think of other ways to solve this

problem this is actually a really fun one because you can bring people in from other domains to kind of brainstorm it because you know two people may not have the answer but you bring it people that know the area you may come up with the trip to Hawaii who knows what would be the impact of your solution to you and to your other two others okay this is the this is the money one this is the ROI this is this is the one where you figure out okay we it took us this long and now it's only gonna take us this long it costed this much money and now it's what I'm gonna cost this much money

and it only cost there's much money to build it and like in four months you know we're actually all in the clear are there any internal or external constraints you need to keep in mind and this is actually an interesting one this is more to do with being able to use it across multiple people location issues permission issues licensing issues you know all kinds of stuff that can impact on how people can use a tool and how would you measure the success of the solution okay this is the one its metrics but it's the way you prove to management that you've got the ROI thing nailed so the next time you try to run this thing and get permission if you've

actually come through and your metrics actually are bang on and they reflect your initial guess and it's gonna be so much easier you'll have cred okay so what I'm gonna do here is I'm gonna start going through these questions and until he like waves like about the 15 minute one and then we'll we'll break over to the next part so what I'm going to do is start through the questions I'm going to give you an example of this first one so I'm basically going to give you an example goal and some example background and then I'm gonna ask you guys to do some roleplay because I'm gonna get feedback from you for the rest of the

questions and we're gonna done kind of a mock interview sort of thing so our goal we've got somebody who's regularly sending packages and letters through the mail for work and by the way I picked a generic kind of topic here rather than you know going into something technical or InfoSec just so that you know we could actually get a get to a point where everybody can play along so you're sending these packages and lettuce to the mail forward great but you've got a bunch of restrictions you have to use you know USPS your packages are all kinds of sizes and shapes and weights you have to track on with a return receipt you can't use in-house media

metering and weighing systems because they are billed to a different department and you don't have access to it and you don't have your own metering machine so somebody has to truck to the post office every day and pay out of pocket because you don't have a corporate card and they have to do this about four times a week well every day four times a week and it takes up over an hour of their time each time so this is this is kind of our ground truth here so my next question is and this is for for you guys if you are this person you know put yourself in this person's shoes it's poor you know sod who else to do

the mail what is their biggest problem or pain point with the current method okay anybody I'm gonna go back to show you guys the screen travel time okay that's a good one and buy travel time do you mean between work and the post office or do you mean the time that it takes to actually perform the whole task okay great thank you and the point there was is I don't take his first his first response is kind of like the bear thing that I write down it's like I want to find out exactly what he means by that and as you can see it it was actually not just the travel time it was the

whole thing okay anybody else yes yes exactly so I'm sure when you get to the post office and you do this every single day you're gonna get some really cranky people and they're gonna make your life miserable absolutely anything else out-of-pocket and what do you mean by out-of-pocket what are the pains around that right right absolutely absolutely and what do you what what part of that is actual pain okay so you don't know what you're gonna need to be able to deliver this with absolutely yeah yeah and then the parking at the other end if you have to take something awesome so these are a few of the ones that I came up with it's inefficient and you know

lots of duplicates for what you guys just said and yeah that's great so the next question we've got is walk through the steps of the tasks oh okay so you're this person and you've got to perform this whole you've got to actually do all these things and I'm going to go back to our very first one you have to perform a task that meets all those restrictions and requirements does anybody want to have a go at doing a walkthrough go for it [Music]

okay okay okay okay

beautiful that was awesomely detailed okay that's the kind of detail that is brilliant for this thank you thank you for that see I didn't even have to like prod him for anything anything out so it was good so yeah so I actually I think I had less detail than you did in my walkthrough here you know a couple of other little little things but I was pretty darn close so yes so our next question is how do you manage the problem today so basically is there a way that you can mitigate the pain and agony of this whole process you know what could you do what could you do today given those constraints to mitigate this this pain yes it would and

that is one of the things we said right at the beginning that we don't have one but that being said one of the things you'll find in an interview is people will say things in response to different questions that are actually answers to questions down the road and I would take that as a response to a later question which is a you know what are the solutions there okay I'm gonna quickly show you the results of this slide and I've been poked at for my two minutes and so we that way we'll get some time for questions so this is this is the easiest so it's like get somebody else to do it find an intern you may get them to do it

so I mean there's all kinds of ways to deal with that but you know absolutely and the whole getting a metering machine that is a solution you know if you go through and you make a point of showing that this is actually worth it yeah you're probably push that through okay so let me quickly get through to the summary here what I want you guys to take away from this is that and I'm gonna read this you know like read along with Caroline take the time to understand the goals pains and potential users for a tool it leads to better tool validating your tool before you build it tells you its value and ROI and you can

use those to help get this thing to happen so any questions do I have time for questions excellent design this is more of a comment rather than a question as a former interrogator myself spent many months learning how to properly question in my case detainees one of the most important things I learned was never ever ever ever ask yes or no question so you get asked for in this example how did you like to talk today and you get a lot more information well the slides were a little too white the chairs are uncomfortable you're getting more information rather than just yes some of the other things that we learned were of course the five we call the five W's who

what when where why how and other ones we learned with especially for detainees where we'd ask questions like what kind of weapons do you have mace a ak-47 what other kind of weapons you have RPG R what other kind of weapon you just keep going until you're exhausted exactly yet the the why why why or what what what is like little kids you go back to the why why why and you know you kind of keep going there because you're absolutely right you just you keep getting more and more information then you make that person think about what they're actually saying but that's that's really interesting in fact it'd probably be worth my while to go through some of

those those rules and see if I can incorporate the nicer stuff okay great thanks thank you very much [Applause]