Video Games, Quality Assurance, and Business Logic - Paul Hardin

please welcome Paul Harden to the stage Paul is a cyber security professional and out of gamer with extensive experience in quality assurance and it pulling from his previous background in QA and video game production Paul brings a unique perspective to the field of cyber security lovering methodologies from diverse domains to tackle complex security challenges please welcome Paul

[Applause]

thank you everybody for having me um my talk will be

sorry all right hey even better uh so today I want to talk about video games quality assurance and business logic um a little bit about me uh my name is Paul Harden I am a senior application security consultant uh currently with direct defense um all opinions on the slider man not have to do that thing uh so a little bit about my background uh so I started in quality assurance um and you guessed it in video games so I was the stereotypical uh quality assurance game tester um it was kind of my first job out of college it was honestly fantastic I did that five or so years before moving into quality insurance for uh web

applications um and then shifting uh into security I've been doing that for about five years now um with the focus on the applications also as well um so yeah what are we going to be talking about here um I had this idea quite a while ago uh I play a lot of video games in case anybody didn't guess that and I was thinking okay well what about video games I'm playing these games I've been doing this for decades how is that going to influence what I do for quality assurance and then when I switched over into application Theory um kind of where that worked as well so what I kind of I want to talk

about is uh how you can take your own experiences just the things that you're doing uh unconsciously subconsciously while you're playing video games uh talk a little bit about quality assurance processes and practices and how you can hopefully build some Better Business logic test cases out of that I do have to do the what is everything uh in case you don't know what a video game is just in case really really basic they are interactive electronic games that is the lowest I could make the explanation um you're probably familiar with playing them probably seeing them everything from console games computer games mobile games um web games if you were on Facebook a decade ago and you

were playing Farmville with you know everybody else on Facebook um I might have had a my C I work to you guys for a little bit so sorry about that but that is a video game this is a huge industry right um the last statistic that I saw I was actually reading this yesterday the day before is that their budget like blows Hollywood out of the water so this is is a huge industry quality assurance as yeah if you haven't seen the movie uh it's about the testers uh quality assurance so if you're not familiar with quality assurance it's a little different in software there's quality insurance everywhere there's quality insurance in your for food production there's quality

insurance in manufacturing um if you have ever seen uh the little marks on your you know printer there's a little um RGB that's for quality assurance you can make sure things are printing software quality assurance making sure that the product is meing specifications and preventing the def from reaching the end users and then business on test bases um set of rules that determine how the application operates so uh as an application security tester what I do uh for any application that I'm testing um we are working on you know between the dynamic testing we have our manual tests we are working on um you know how is the application doing what it's supposed to be

doing um so business logic test cases are a good part of

that okay have to find things what are we doing so my thought process for this is a perer example what do I do when I play a video game one of my favorite types of video games are action adventure usually third person usually single player that's what I prefer to play the first time I load into a level I turn around how many people when you turn around on a level there's a chest behind you there's another route there's any other number of things behind you it's not what you're supposed to do the camera's pointing forward there's obviously a path forward you're supposed to go forward the first thing you're doing is turning around that's just kind of I never even

thought about that you know I just automatically do it you spin the stick you turn around you see what's behind you um so what that ends up being in as we go on to the second bit for testing um we also want to go backboards right so if I'm doing quality assurance when I was doing quality insurance you have what is called a happy path and the unhappy path the happy path is the developer said okay so you're on my e-commerce website I would like you to uh go to the product page and then select a quantity of items and then you go to the checkout and you put in your payment information and you put in your

address and you check out and you watch your p and that's great well so what's the unhappy path there uh the unhappy path is I maybe I forgot something in my heart maybe I changed my mind maybe I forgot to look for coupons and I delete the item I need to go add another item um I've moved since the last time I bought something from this website so now I have to change my address all of these are edge cases and not necessarily just the happy path so that's what I would do for QA if I'm writing a business logic test case while I'm testing a web application it's going to be very similar to that it's going to be

okay I'm doing what you told me to do what you're expecting me to do this happy path now what else can I do so I'm playing a video game I've turned around I've hey you know what that brick wall is a different color pixel than the rest of the brick wall so I'm going to try to do something destructive uh so if I'm writing the business logic test case it is going to be I have followed the happy path and I have followed an unhappy path with a couple of examples um some of the things off the top of my head uh you know removing all the items seeing what happens uh if your url is going from the

cart to payment to shipping what happens if I skip to shipping can I skip to shipping can I force the URL to do that and honestly this is what started the whole talk uh was turning around uh another fun example I had um so if you do pay attention to video games you've probably heard of a speedrun a speedrun is how fast can we finish the game uh there's different rules you know are you doing AED uned are you using tools different things like that um I have a little brother he also likes video games and he sent me a speed rug for a video game called Doom Eternal where somebody had mapped the jump button which normally is say the

space key by default and you can press it and you can press it pretty fast they mapped it to the scroll wheel of their mape so they had near infinite junk what that allowed them to do was break through the boundaries of the game fly over the whole whole map just like a skip straight to the end you know you just jump on various artifacts and you just skip the whole thing that's how a video use what happens when we do that for um our application testing if anybody here uses burp if you're using burp professional uh if you're familiar with the repeater tool one of the things that they added uh is called turbo repeater

if you haven't played with that go they that they have a lab forer has a lab uh for Turbo repeater and basically what it is doing uh burp is a proxy so you can capture in their lab a um discount code for your cart and what happens if you send that same discount code over and over and over you can group your requests and send them all at the same time I think the spoilers for the lab you get it down to like a $100 item down to like two bucks um because it's exploiting race conditions so when you're doing QA um I did this I I worked on a couple multiplayer games um you know we checked for things like

boundaries we're checking for Collision errors uh you know you run around the map you walk around the map you walk into everything you see if they forgot something uh of the things that a lot of testers do is you can Crouch that changes the size of your hit box you jump and you jump you crouch and jump backwards and the amount of things where I've seen people exit the map that way is is wild so when you're writing your business logic um thinking about what happens if we are just really playing with those boundaries uh we're using be there we're trying to invoke a race condition we're trying to break

through um another lesson out a video game side quests I don't know who plays RPGs if you played a role playing game there are a lot of side quests you you might have a main story and then you run into the tavern and they're like hey I know you're trying to save the world and do all this stuff but like I need three chickens so uh if you can grab me three chickens I'd really appreciate it and I'll give you my grandfather's knife and then you get the grandfather's knife and it's a map and you go do something else and you do something else well okay so what are we doing for quality assurance what are we doing for

our business ler to spes do you know everything that's in soap are you a completionist are you making sure that everything has been clicked on um if you're writing your business test case your business logic are you making sure that you've checked every page that you've checked the contact page um I ran across an issue that had a contact form it was for like contact our lawyer you think that there's been some sort of breach and it had unrestricted file uploads I could send anything to their B te um I don't know what kind of protections they have on the back end I never got to see that I never got to see the full impact but

it was embedded in the about in the contact it wasn't necessarily I never saw it on the site map specifically but it was on there about P like hey you need to contact our lawyers and I brought you to a whole separate content page so making sure that you've done all the side quest that you've checked all the

functionality uh I think one of my biggest things that I would like actually be able to take away from this um is play with your application and I did put play in quotations um what we do is serious business sometimes uh I've touched a lot of things that I I really hope have no issues I use a lot of applications on the daily that I have hope have no issues but you need to explore your application you need to play with it you can't just trust that the last guy got everything that uh the application hasn't changed you need to play with it you need to understand application um remember your happy paths remember your unhappy paths those like I

gave the example of checkout and cart so what else is there in your application is there login is there sign in any other number of things that you want to make sure that you're checking how people expect you to go and how to break that um thinking of the edge cases I you know when I play a video game um I was trying to think of examples the other day and I was thinking of Donkey Kong Country for the Super Nintendo Entertainment System it's an older game but there were multiple places in that game where you could destroy seemly innocuous walls and get bonus area it was great so what are your edge cases what are your business logic edge cases

if you are uh adding things to C uh do they have a little slider do they have uh something to increment the number can you add half of an item can you add a symbol what happens when you add these things how does the application deal with that um edge cases are really important in quality assurance and they're where you're going to find most your issues uh when you're doing your business logic and application testing if you can read the patch notes uh they're obviously not always available for the application that you're testing um but one of the fun things if you are looking at uh video game patch notes if you have ever seen uh you know removed the random

coconut out of level four um who who knows what they could have replaced them with I've seen some some great patch notes um I would definitely recommend uh looking up some they're they can be funny they can be very serious as well um but every time somebody changed something they could have very well introduced a new issue um I I've worked in quality insurance long enough uh that every time anything changes there's a very real possib that something else went wrong um look behind go backwards uh there are many instances in quality assurance in video games where you know you go backwards you have got a new item and now you're going to go backwards and

check something else uh for your business logic test cases you want to make sure that if you add something your cart you're can add it from check out that you can add it from the uh product page

itself uh I did touch on clearing the map and just clicking all those links um use your team speak if you guys have ever played any MMOs uh any uh first person shoers you've probably seen team speak it's talking to your team it's it's making sure that if you are doing something that you're communicating it to the rest of your team right if I am going to in a video game I'm going to go take the base you know you you tell your team that if you're doing application security testing you might want to tell the analysts that you're about to you know run a scan that is going to send a whole bunch of noise

to the law do you have a way to communicate that with your team uh button mashing uh I did mention I have a little brother if you have ever tried to play a fighting game where you know what the combos are and you're playing against somebody who just is hitting all the buttons it's actually wild how often they'll wa so how are we doing that in quality insurance how are we doing that in business Logics um you were throwing everything out there we have our fuzzers we have uh repeaters we have all of these things send everything out there send your uh alpha numeric character send your symbols send your M lights send everything that you

can with inscope obviously um which kind of goes into just cuz it says you can't try it anyway uh if you've ever did a video game that You' played before and they haven't told you that you can jump yet but you know what the jump button is you might jump over something um I had an instance uh with a business logic test case when I was doing uh that I used multiple times now in business logic where this was a product that is sold in eaches it could only be sold in eaches you could only press one two you could only incremented in number however if you copied 0.5 well you could add that to card

luckily we caught it before men production so I have no idea what they would have done when it got to shipping and they were like um so do we just cut this in half we what uh but it only charged half the price for as far as I went with it so try just cuz somebody says you can't do it anyway um and then there is no platinum trophy uh if you're a completionist you just have to realize also we are doing our best effort this is for quality assurance this is for penetration testing this is for we out of testing this is for your business logic test cases you might miss something you want it documented so that

you can say what you did do if you did miss something you might want to have a reason if they like hey how did you miss this you say well I tested this and this and this around it um in video games I have done testing where I literally had to test the platinum trophy which was like you know do 1,000 multiplayer matches um and we were getting a new build every other day uh there was not physically enough time I got it down to Having Eight controllers laid out in front of me entering a making everybody else quit I won the match with the One controller I didn't quit um I never got that trophy we did without you know

with the best effort um because it was literally physically impossible for me to do

that you have to know your application um I I spend if I have a week or two weeks if I have the time I devote a a good half a day to going through the application so that I know exactly what this application is doing I may not know all of the edge cases and everything but I know what the application is trying to do before I start writing my business of test cases there's some general things if you have log in you there's login you can write your you know can I do sequel injection and get through login can I bypass your login and anyway you know you know that you're going to have to

test that but every application is different um you know I'm a consultant so I see a new application every every engagement um when I was QA I was on the project from really really early pre is to the Shi and post support I knew that application I have gotten hate mail uh through chat um because we were testing the production environment in a multiplayer game and I had been playing the game for 60 80 hours a week for months before it was released obviously I was winning I knew the game I knew the game inside and out um but if you don't know it play with it make sure that you can get that that

experience uh I did want to give some extra resources uh so the international software testing qualifications board does certification if you are not familiar with quality assurance uh you are a tester especially if you're doing application testing uh I've always called application testing security Plus+ or q++ sorry CU when I moved over I was like well these are a lot of the same things I'm doing I'm I'm checking your inputs I'm making sure that everything works the way it's supposed to work but now I'm getting an error instead of saying hey there's an error I'm saying okay there's an error and it says this and now I can and then potentially exploit it further uh if you have not checked out

Port switter web Security Academy I am a huge fan it's free um they require an email assignment if you want to keep track of your stuff but there's so much information and they do have a um session on logic flaws uh as well as OAS if you're not familiar with OAS and your security web application Security check it out um but they do have also uh some sessions on business logic testing and data validation I am very close to time but that is my game over slide so feel free to reach out if you see me around come say hi I like talking about this stuff I like talking about security I like talking about I like talking

about Q I like talking about my daughter no she is not a password and she has never been a password so you don't have to try that this is Moji yes this is a real billboard I had a billboard for my dog and and it was great you can ask me about it it was super fun but I wanted to leave space for questions uh if we have any so um thank you

[Applause]

as you on two specific topics that generally aren't covered by security scanners like Earth or go or you know whatever Pi your flavor can you talk to me about how you see the industry today really leaning on those scanners as a result often times with um new or in experienc pin testers we'll just send in B report sure uh so if I understood the question correctly you're you're saying you've seen you know bur or O app or any other of these scanners and scanner runs and somebody hands you a report and you're like hey this is trash this is isn't super awesome yeah so I was very lucky when I was transitioning I was already in a Consulting role doing QA uh

we were starting up with practice we brought somebody in who was senior they told us not to do that so for those of you who haven't played with b or Os it will generate an automatic report if you said it that way there's always settings for software there's so many positives um I use bir pretty much daily in my professional life and um I'd say maybe two or three times so if you see something in your resulted go check it it it's not that it's always the false positive but you do have to verify everything um and I hope that answered your question U yeah just verify it don't do raw report that's not a report

that's a scan output um if you have never watched the there's a YouTube hack for show report for no go watch that it talks about uh hacking is secondary to the report the report is what is being paid for that is how you are making your money any other questions

all right well nobody had questions by my dog so I'm a little I'm a little hurt but thank you everybody for listening and it's still not my P and thank you thank you