The Cognitive Stairways of Analysis

hi my name is nicole hoffman and i'm giving a presentation titled the cognitive stairways of analysis this presentation is going to be a story about how i unintentionally created my own analytic framework it's all started with a blog post um and it's come a long way and i'm i'm really proud of how far it's come so if you're interested in analysis or you just like good stories or cognitive science for that matter stick around because it's going to be a good presentation so a little bit more about me i'm currently an intelligence analyst at group sense don't know why it's doing that sorry about that so i'm currently an intelligence analyst at group sense i have a

bachelor's in information technology with a minor in cyber security i have a blog currently at uh you can get to it at threehuntergirl.com my passions lie in threat research and analysis my family just recently moved to texas and it's been a lot of fun on weekends and sometimes in the evenings exploring and trying new food when i sign off from work and get away from screens i really love comic books um whether that be physical comic books digital comic books um cinematic universes i i love it all really anything in the sci-fi sphere uh sphere i'm really into also medical dramas is also a notable mention and if you want to reach me on twitter

my handle is threat hunter girl without the eye and girl so going over the agenda i'm going to begin my presentation by discussing the topic of analysis and what it is and some of the challenges that i faced as an analyst that led to me to want to pursue more information next i'm going to go over six analytic models that i researched during my deep dive into the analytic tray craft i did go over or i did look into more models which you can read in my blog post but for this presentation i'm just going to focus on six then i'm going to go over my framework which is the cognitive stairways of analysis right now there are four

um it started out with three so i'm going to go over all four and then i'm going to apply the fourth stairway to an intelligence investigation and then conclude with some helpful resources so what is analysis you might hear this term all the time but what does it really mean how do you analyze data unfortunately this is something i had to learn um on my own when i started out in infosec as a my first role was a cyber security analyst intern and i've learned a lot since that day but i still feel like there's a huge gap in training when it comes to analysts and data analysis so i wanted to take a deeper dive

into the tradecraft trade craft of analysis last year and as i researched the topic i felt like there was a lot of great information but there wasn't a lot of information specifically for cyber threat analysis at least not how to perform the analysis you know i felt like analysis itself was a step in a lot of frameworks or or a step in a lot of processes but no one was really going over what's happening in that step how do you analyze the data what are you looking for what's going through your mind what is your brain doing how do you know you're analyzing it the same way as someone else and if you aren't analyzing

it the same way is that okay you know um so i decided to expand my search and look at how are other industries performing analysis and what can i learn from them you know if they expand upon this step of analysis um and when i first gave this presentation um i did get a lot of feedback regarding well there's so much written about analysis why didn't you read this and why didn't you read this well when you're doing your own research and it's just for fun you're not going to you're not going to identify every possible source and that's one of the great things about the internet and communication and really a collaborative project like the

cognitive stairways of analysis if you know of a great analytic source please send it my way i am [Music] happy to read it i really implore others to not only contribute sources but also their own take on the stairways it's not a you know one person or i don't know what i'm trying to say but i guess what i'm trying to say is i understand that there's a lot of sources out there but there wasn't enough for me when i was starting out and i did the research last summer and i just felt like i could add value and so that is what i did the first model that i want to go over is titled a cognitive interpretation of

data analysis and this is a white paper um by garrett grolman and hadley wickham the authors of this white paper compare the process of sense making to the process of data analysis but when i first read this i thought well what's sense making well sense making is the process of how our brains make sense of the world around us more specifically our mind or the human mind creates and manages internal cognitive structures that represent certain aspects of reality and the authors define these models or schemas as mental models that contain like a wide range of information about a specific object or concept and the schemas are organized in something called a semantic network inside of our brain

so this was really interesting to me and it made a lot of sense because analysts are regularly creating hypotheses and either you know before or after uh the analysis and trying to determine you know is this information valid you know dude have i do i already have a schema to match this data you know is there any discrepancies about the data do i need to throw this out and one of the best examples that i can think of when it comes to understanding the process of sense making is picture yourself as a child experiencing a thunderstorm for the first time your brain may not know what's going on but it will start collecting information about this event

and then it will store this information in a schema and it will title it you know like let's say rainstorm the next time you experience a rainstorm let's say it also has thunder and lightning your brain will try to match the data to the rainstorm schema and determine do i need to create a new schema or do i need to update this schema or is this information not valid and do i need to throw it out and those are the three things that your brain does when you experience a new event it either creates a new schema updates an existing schema or it just determines that this observation is untrustworthy and it just throws it out

um kind of like sometimes when you feel like you see something in the corner of your eye and then you look and then you're like maybe i didn't see something and you can't explain it so your brain just decides it didn't happen that's kind of what sense making is and that's really interesting so i felt like the authors or not the authors but just the process of sense making i felt like it it really just defines analysis and what it is at its most simplest term um so i get kind of really nerded out over the process of sense making as a lot of others analysts in the field i think have at some point or another

found sense making so then um the authors of this white paper compared the process of sense making to the process of data analysis so this more specifically they broke it down into exploratory and confirmatory analysis exploratory analysis is one that starts with no hypothesis or any preconceived notions about the data you just start exploring the data and then you try to find like a relevant schema after you explore the data whereas confirmatory analysis it starts with a hypothesis and then you try to validate the data to the hypothesis so for example when my computer is super slow and laggy um even though nine out of ten times it's probably going to be a windows update i'll still

explore around and see what do i have running what's taking up the cpu what what what's going on is it my internet and like i said nine times out of ten it's a windows update you would think the next time it would happen i would start with confirmatory analysis and think oh it's probably a windows update i know this from past experiences and then try to validate that by looking and seeing oh do i have any updates let me validate my hypothesis and that's kind of what exploratory and confirmatory analysis are so my key takeaway from this model was confirmatory and exploratory analysis the next model i want to go over is called the statistical investigation process

and this was uh written by or it was created by dr christopher chatfield and it was published in his book titled problem solving aesthetician's guide within this process i was immediately drawn to step three which is assess the structure and quality of the data or clean the data and i personally break this down into two steps so i broke it down into two key takeaways which is um i broke it down into quality of information check or qoi check and this is where you're checking the completeness of the data you're checking your confidence level how confident are you with the source if it's not a really confident source you might have to collect more data so

that you can boost that confidence level whereas cleaning the data is you know omitting any useless data making sure if you're working with like a large data set and let's say you're like querying a database you might do some data normalization to make sure that like let's say you have a field within the data set where it's like city and some people who who put together the data set put sd for san diego instead of writing out san diego where then other people put the full word san diego and let's say you don't know that there's different versions and so you go to query it you're going to miss out on a lot of information so it's going to affect the

completeness of your analysis um so that's why you want to normalize it so that you make sure you get all the results that you need so you can do the full analysis um the next part of this process that i thought was really interesting was the select step which is where i think most of the analysis is taking place and i felt like this process that the author put together is really taking an exploratory approach to analysis because it starts with a data set and it's like a group of variables um in the text and um a model is then created after exploring the data and the author refers to this as regression analysis and it's really where

you have a group of variables sometimes it could just be two variables and you're trying to find a relationship or an underlying structure between the variables and i thought this was really interesting because i felt like it's kind of like and i felt like it expanded upon the process of exploratory analysis so i took that as a key takeaway for this process and to go over regression analysis a little more so you kind of understand if you think back to grade school if you remember those math problems where you're provided a series of numbers and they say um find the pattern if you're given a number set like this one two four six eight ten you

might immediately know oh well it's adding two each time and you may know this because you you just know it by memory at this point but let's say you have this pattern which is 28 20 13 2. you may not immediately generate a hypothesis like you did with the previous data set so you have to explore the data and then generate a hypothesis so you might immediately or you might first say okay well 28 and 20. there's eight difference maybe it's subtracting 8 each time okay well 20 minus 8 is not 13 so that can't be right so that hypothesis has now been proven wrong so then you subtract you do 20 minus 13 oh it's 7.

perhaps you might come to the conclusion that the number being subtracted is being is subtracted by one each time it's it's going down by one each time so it's eight seven six five and so on and so forth um and then you could validate that hypothesis with the data set and that's kind of um what you're doing with regression analysis is you're trying to you know explore that data and and try to find that underlying structure pattern

so the next process i want to go over um is actually one of my uh one of the most favorite ones that i that i uh researched and it's the model of police operational intelligence analysis and it's from a white paper titled how analysts think think steps as a tool for structuring sense making in criminal intelligence analysis and i don't want to totally butcher the authors names but i do have them listed and i do have um each of these resources um in a slide at the end of the presentation and i'll be making my slides available so this model is broken down into three stages which is prepare analyze and report slash advice and i

like i said i absolutely love this model for a number of reasons but specifically it breaks down the analysis step into different stages it doesn't go too far but it does more than just say analyze so i just felt like you know we as an analyst community cannot just assume that everyone has the same definition of analysis um and instead of just writing analyze the data you know perhaps take a lesson out of this model's book um and break things down into further steps so that less experienced analysts are not just pondering if they're doing it right because that's kind of at least for me where imposter syndrome starts is you know knowing all the stuff

you don't know and start starting to freak out so this process starts with a briefing from an investigator and then the um intelligence or the criminal intelligence analyst would then establish think steps think steps provide a template that enables the analyst to approach the case decompose it into separate elements and classify associated data accordingly so in other words the criminal analysts are attempting to choose a schema or multiple schemas to match the data in the case to um and for criminal analysts the schemas would be like crimes like um like murder burglary human trafficking things like that and each crime has its own set of think steps or excuse me things for the analysts to think about when they're

looking at the evidence of the case and it's kind of like how when um in information security we were analyzing an incident like let's say if it's a malware we might have different things or different think steps that we want to think about when we're investigating that you know if we're investigating a domain we want we might have certain things that we ask ourselves that we want to think about things to consider and things like that so the idea of of think steps is really one of the best pieces of analytic advice i think i've really ever received and it really captures like that um i can't think of the right word the that i think it's like tacit versus

i can't think of it i wrote a blog about it but there's certain types of knowledge that you have specifically um in like a work environment there's some of it's it's easy to write down some of it you just kind of know from experience and capturing that information from like senior analysts it could be so helpful so useful to expand upon like standard operating procedures and things like that so think steps i think are really important um and if you can take anything away from this presentation i hope that it can be think steps because i literally use them on a daily basis and they help me out a lot it's kind of like when they when you

hear like take notes after you experience um an investigation so you know what to think about next time think steps right so i'm so glad that you know i i kind of stepped out of my comfort zone and found this model of police operational intelligence analysis so that i could find this key takeaway of think steps but um moving moving on i could talk about think steps all day um so then after he uh he or she or they have the think steps they can then go forward and request more information more data you know if there's certain things that they need to consider they get more background research based on that data structure the data

and then you know query the case database if they need to schematize the data if we think back to uh exploratory analysis and said the process of sense making and then recreate the path which is putting the pieces together and then obviously the dissemination whether it be oral written and so on and so forth so this is a really interesting model and i'm definitely one of my favorites so the next model that i want to talk to you about is the business analytic model life cycle and this was from an article written by michael coveny hopefully i'm pronouncing that right that he wrote and there was a lot of processes and models in business analytics but this one i

really liked because of the sixth step which is monitor the model performance i thought that was really interesting and i thought you know that might be super helpful maybe in a future stairway i didn't add it as a key takeaway but i thought that might be really interesting for something like you know if you had a process of like creating um a policy or something like that where you want to monitor the performance um and then maybe update it as needed if um if the policy or something isn't working so i just thought that was really interesting and something i hadn't seen before um and then the key takeaway from this one was actually the

determine the scope realized i was covering up part of the words so my key takeaway from this model um is determine the scope and i got that from the first step which is to find what is being investigated and this wasn't the first model or process or framework that i found where they're kind of determining or defining what their investigation is going to contain but it's the first time when i was going through this model it was the first time i thought about it and i and maybe it was just the use of the word define and i just thought you know how important it is specifically in cyber security and really an intelligence analysis you want

to make sure you're defining what's being investigated what's the scope what's allowed what's not allowed how in depth does it need to be what's what's the final product is there a deadline all these things you need to know up front because it's going to um impact your investigation and how speedy it is and and things like that so i really enjoyed um this model and and that key takeaway and again i might add number six the model monitor the model performance in the future um if you have any ideas of ways that you could use this definitely let me know because i think it could be really interesting

i'm sorry my camera is moving around i'm still getting used to pre-recording when you have like huge um graphics in your slides but just bear with me so this next model that i'm going over is um the diagnostic process model and it's from a book titled improving diagnostics in healthcare and it has a lot of authors the committee on diagnostic error and healthcare the board on healthcare services the institute of medicine and national academies of sciences engineering and medicine and like i said earlier in the presentation i actually came i think i mentioned it maybe i didn't i came from the medical field before i got into um uh infosec i actually came from the

medical field and then i got into financial fraud and then i moved into infosec so that would be really interesting to include a process from the medical field and trying to think of like the analytic process of like a nurse or a doctor and so i really wanted to include this model this one was really interesting um mostly because um of like the cyclical process in in the the middle that you see um and they kind of you know took information they collected it and like as they're doing the physical examination and collecting data from the patient they determine or you know they got a working diagnosis and they continue to collect information whether it be from a clinical history

physical exam um the you know the the testing that they do and and things like that but they're basically um taking a cyclical approach to the collection and analysis phase um and they're interpreting that into a working diagnosis um one of the main things i thought was really interesting with this um framework or this process was the treatment step um and it just made me think immediately about like an infosec like if you're in a client vendor relationship um how like i know like personally i am on the vendor side so i have a list of clients and things like that that i help out and there are times where i only get part of a story like they say

hey nicole can you help me out with this and they give me like part of the story and then i have to ask questions and then sometimes when i give them information they have to ask questions and we both kind of sometimes forget that i can't do like a physical examination of things that are going on in their environment kind of like a doctor does because you know if i was a doctor there's so many things that you can get from their clinical history and their medical file but doing a in-person physical examination is completely different and it could change everything it could change everything that you see in the file and this is kind of what we need to

think about in infosec you know when we're giving information to a vendor we have to realize they can't do a physical examination so let's try to answer every question that they might have um because they can't see what i see and it can be hard to kind of step back and get like you know look at your problem from like a holistic approach and same like if you're a vendor and you're talking to a client you know that you can't do a physical examination and that you're going to have follow-up questions so try to get those up front so that you don't have to go back later but my key takeaway from this was actually the treatment

plan and because and the treatment plan to me is the dissemination it's the so what what so you know this is the diagnosis this is the treatment plan these are my recommendations this is my full report and so i thought that was really interesting and definitely a great key takeaway

so um i don't know i don't think i talk about it that much on twitter um but something interesting about me is i'm kind of a huge meteorology nerd um i discuss it sometimes but probably not as much as i should nerd out about it actually when i was growing up i really wanted to be a meteorologist but then i found out um how much math was involved and i'm awful at math so it i knew it i knew immediately it wouldn't work out but i've always just been really fascinated by weather and and storms and really love uh watching i could watch the weather channel all day um so i just really wanted to like you know

really dive into that analytic process um i dived into the scientific process which you can read in my blog but i really wanted to squeeze in this process specifically in the field of meteorology so this model that i found is called the simple weather forecasting workflow and is actually from a white paper not specifically about meteorology but it was it's titled optimization of a heterogeneous simulations workflow and again i don't want to mess up their names but i do have it included and i do have a link to the white paper in the resources and this model it's kind of cut off but the first step is understanding the environment and i think it says like air and water

for this particular process and i thought that was really really interesting um it really just kind of establishes kind of like just determining the scope but a little bit further it's really understanding the environment that you're collecting information from and this was the first process that i found that defined the specific environment and not so much like i guess it could be an involved and determine the scope i just thought it was really important specifically you know if you're investigating something like internal to the firewall let's say you're in a sock and something happens to the ceo's computer you're not gonna go then look at like karen and hr's computer because it didn't happen on our computer

so understanding the environment and where the information or the incident took place is really important um so then they in this process uh they look over or they have like a time period that they're measuring data and then they're collecting and processing that data that they've collected over that specific time period and then they're analyzing it doing some mathematics and then the model can actually takes two paths it can either go to a statistical interpretation subjective meteorology subjective interpretation by a meteorologist or i can do the direct model output and or dmo and this is an objective weather forecasting technique which consists of determining a statistical relationship between predicted and predicted variables by a numerical model at some projection

times and it's in effect the determination of weather-related statistics of a numerical model again lots of math inside of uh meteorology but still very fascinating and then it can go down to statistical post-processing and such as like the kelman filtering is what you could see as one of the examples and it's also written in the the white paper and this is a complex statistical algorithm and it's kind of like a process of confirmatory analysis where they're validating that the data fits the schema and then again either way either path you go is going to do that subjective interpretation by the meteorologist and i really liked that step because i felt like it's kind of like an additional layer of

confirmatory analysis kind of like peer review i mean i guess for this particular one it would be like an expert review and i thought that would it was really interesting and maybe a step that i can add in the future um for like more complex analysis investigations where you kind of want additional team members to kind of like review it to make sure it kind of makes sense and then of course the the weather forecast for users so my key takeaway from this process was the understanding the environment and really like defining the environment which would kind of get rolled up into the determine the scope

so i listed all my key takeaways you know from the first one um exploratory and confirmatory analysis and that excuse me it was from the process of sense making and how they compared it to exploratory and confirmatory analysis um from christopher chatfield's statistical investigation process we have three key takeaways we have clean the data quality of information check regression analysis and cleaning the data if you remember is getting rid of any useless data data normalization whereas quality of information check is really where you're checking the completeness of the data how confident you are with the sources do you need more information and then regression analysis if you remember back is trying to find that underlying

relationship between different variables and or like a pattern and then from the model of police operational intelligence analysis we have my favorite think steps which are very important and those are kind of things to consider from previous investigations and they kind of help you as you're investigating something sixth key takeaway is from the business analytic model life cycle and that was determining the scope of the investigation and my seventh key takeaway is the treatment plan or the i say confirmatory analysis but it's really if you think about it part of the dissemination it is confirming your diagnosis or your hypothesis but it's really your recommendations um at the end of the investigation and that was from the medical diagnostic process

model and the final key takeaway is from the weather forecasting process and this is understanding your environment that you're investigating so moving right into my framework the cognitive stairways of analysis um i have four stairways right now this is the first one each stairway starts um and finishes instead of being like a cycle there are some optional cycles but overall there is a beginning and an end because there's always going to be a dissemination and that's kind of why i picked the name stairway because it's always going to start and finish with dissemination and i feel like the dissemination is one of the most important parts not the most important part but is one of

the most important parts obviously the experience that you have throughout the investigation is very useful and educational but i feel like it mostly determines um the dissemination and how you can how you can respond and provide that feedback after your investigation and explain it um to either a technical audience or a non-technical audience super important so stepping right in um this particular stairway was the first stairway and it started with an alert

next you're determining the scope and remember this is where you're setting your understanding the environment if you need to pull logs from the ceo's device because something happened on their computer you're not going to go look at a device in hr because that wouldn't make sense so determine the scope determine the deadline and determine the environment that you're investigating step three is compiling the data and quality of information check so as you're compiling data look at the sources are you confident with the source are you not confident you may need to compile additional data step four is cleaning the data omitting any useless data step 5 is eda or exploratory data analysis and this can really depend on you as an

analyst uh you can use a visualization tool like a graph you can use uh like workflows you can use excel which i personally think is great um you could you know draw out mind maps however you explore data best and how whatever works for you is what i recommend i personally use um excel the most and then regression analysis it's really just trying to find the underlying structure or pattern between different variables sometimes there's not going to be a pattern sometimes there's not going to be relationships between variables and you have to understand that and it's okay and next step number six is generating a hypothesis what do you think is going on what is the story that you think

and generating those think steps and or if you already have think steps just going and getting the think steps i mean remember the think steps are um the specific things that you want to consider from like previous investigations or or like templates that help you think about the case and questions that you want to ask and so you can start using those when you go to confirmatory analysis in the next step and remember exploratory analysis is where you don't have any preconceived notion or hypotheses about the data confirmatory analysis you have a hypothesis that you've already created you're now validating that the data meets the schema that you the schema being the hypothesis after you've confirmed the data you can

move on to disseminate whether that's an oral report written report and so on and so forth if your hypothesis if you cannot validate it you can go back and explore the data to be able to create a new hypothesis sometimes you just need to collect a little bit more information and then you can confirm the data that is okay as well so um the second stairway begins with a brainstorm session so it's a little bit different than the first stairway so this is a great stairway for like if your ciso or manager ever asks you like hey i saw this thing on the news are we susceptible to this and you can just start thinking right

away hmm well i i'm already thinking about it i can already generate well these are these are the ways that i think that it could happen these are some things to consider so you're already thinking about think steps you can get all that information ready after you've come up you've finished your brainstorm you can go straight into determining the scope which is determining the environment that you're investigating determining the end product and and things like that and then you can go into what's known as the key assumptions check or kac for short and key assumptions check is an analytic technique where you write down or you can just think about it all your assumptions about any given

topic and determine the likelihood if that assumption is true it seems kind of silly but when you're doing it specifically when you're in a group it can be really helpful about fleshing out certain biases that you may not realize that you have and devil's advocate is another great one for fleshing out those biases is really just trying to think of every like if you put yourself in not like necessarily an attacker but you're you're basically just trying to think of like every possible alternative to the hypothesis to prove it wrong and it's just a great way both of them are a great way to flush out pisces and then just put everything out on the table

the next step is you can start compiling data and as you're compiling the data check the quality of the data are you confident that the data and then the data sources does everything look right um and then you can start cleaning the data get rid of anything that's not pertinent to your investigation omit it if you need to organize the data you need to clean it to make it easier to analyze go ahead and do that now since you already have a hypothesis you don't need to visualize the data you don't need to try to find an underlying you know structure you can just go right onto confirmatory analysis to confirm that the data meets your

hypothesis or validate it but i usually just like to explore the data anyway um just to make sure i'm not missing anything but i am not everyone so i just put it as optional otherwise you can just move straight on to confirmatory and and then dissemination and again obviously if you cannot confirm that the data meets the hypothesis or the schema you can just go back to either brainstorm or you can um go back to visualizing the data and kind of try to determine what you missed

so the third stairway um this one can start um one of two ways the first way is if you're doing like a an audit on like a specific asset if you're like trying to determine how safe that is or um anything like that this is the stairway for that because you already kind of know a specific thing that you're trying to determine from a malicious standpoint how it can be exploited the other way that this one can start is if you're ever just like doing not even an investigation you're just kind of like going about your daily business and you just you either read an article or you're or you're doing something at work and you and you think

about something and you're like i wonder if i can exploit that you know you're already thinking about it from an attacker's point of view you can immediately start this framework and that's really the red team analysis it's where you're putting yourself in the attacker's shoes you can immediately generate hypotheses whether it be one or multiple think about those think steps if you know if you already have them go collect them and if you don't have them you know write them down because they'll be useful not only for you during your investigation but they could also be used for others in your organization for training purposes you can start compiling the data check the quality of the information of

the data make sure everything looks right make sure that it's complete make sure you're confident in the findings clean the data if you need to normalize anything get rid of useless stuff that's not current into your investigation specifically when i made the omit useless data i was thinking about like threat hunting and certain fields that you want to get rid of when you're you know like querying a large database but it's also useful in other forms of analysis like intelligence analysis and things like that um because sometimes you do collect a lot of data but there's one particular use case i wanted to mention and then once again since you already have a hypothesis you

don't need to explore the data or do any type of regression analysis but you can if you want to i sometimes still do or you can just move on to confirmatory analysis to confirm your hypothesis and then obviously disseminate the results this stairway i feel like at least for me personally goes a lot faster than the other the first two stairways because i'm kind of already thinking about something and trying to find if it's exploitable or not so the the final stairway this one is a little bit different than the others um special thanks to juan's bino hopefully i said her name correctly um he helps me out a lot during the creation of this stairway

that focuses on intelligence investigations excuse me so this one starts with a trigger which is really um anything that starts the investigation whether it be an alert an idea a request anything that starts the investigation that would be the trigger and then you determine the scope similar to the other framework or to the other stairways you're understanding the scope you're understanding what is the end product who is the audience what's the deadline that's juan really helped me figure out that the scope really affects the end product gather relevant think steps previously i had this with generate hypothesis but i moved it down because i feel like um as you're creating think steps it could be more useful for you to have those

thing steps at the beginning of the investigation versus the end of the investigation because then you kind of have to backtrack and then you can go right into a cyclical process which is the data enumeration process that's basically collecting data or compiling data exploring the data checking the quality of information omitting useless data and we kind of came me and juan kind of came to the conclusion like we're kind of doing this on a kind of all of it at once as you're collecting data i'm i'm exploring it i'm i'm looking for anything that's useful getting rid of stuff i don't need and then as i'm researching the information i'm starting to create the story i'm

creating information from data so i'm starting my report i'm starting to put key findings in my report i'm pivoting off information within my um like things that i find like if i find the domain is communicating with malware well then i'm gonna pivot and start investigating that malware so that i can include that in my report so pivoting is very important um and then once i have a story put together like the so what as i'm creating information i can create a hypothesis or not necessarily you're always going to have like a specific hypothesis like this happened but you might have um this is kind of what i think is happening and these are my recommendations

and then you can go into that confirmatory analysis and validate what you believe is true look for anything that doesn't look right and review your think steps and ensure there's no like data sources you missed or any things that you need to consider before you can move on to dissemination and then obviously if you prove your hypothesis wrong you can go back and collect more information so i'm going to apply this fourth stairway to an intelligence investigation and i don't have too much time so i'm going to kind of go a little bit fast but i will answer any questions that you may have after the presentation so let's say the trigger is a request

from a client to investigate a phishing email with a suspicious link as we determine the scope we're thinking about who's the audience who's the expected or what is the expected product is it a full report is it a technical report is it for an executive audience is it indeed just an executive summary and a technical portion when's the deadline ask questions such as you know can i have a screenshot of the email so i know who the sender is and who the receiver is can i have the email headers what is the title of the individual being targeted because i could help you in your investigation um relevant think steps remember this is things to consider from previous

investigations just from my experience i know i would start collecting sources for investigating domains as well as ip addresses some helpful resources that i have listed such as fire's total talos reputation center um that one's really good for like looking at email volume um as well as like email reputation and then obviously good old google you'd be surprised how much information you can find just by googling a domain not so much a domain but um an ip address sometimes you can find it in like malware reports and stuff like that that could be super helpful in your investigation i really love alienvault otx they have a great site that i do recommend for ioc investigations and then notes

from previous investigations with things to consider and some questions you know when was the domain registered did they did the registering use who is privacy is there any malware communicating is the ip mentioned on twitter all great questions and things to consider that i think about based on my previous investigations and remember the cyclical process you know start collecting data research all the things explore the data if you need to create mind maps don't lose the connections as you're um you know like deep in analysis modes and then you know constantly check the quality of information make sure that um you don't have any like random information on there that came from like untrustworthy sites and get rid of all

the stuff you don't need and then start creating information from data start putting the story together as you're doing that other cycle and as you start identifying key terms or other things that you can pivot off of like you know if the domain is malicious because it's you know dishing out malware what is the malware pivot start investigating that malware because you're going to want to include that in your report let's say the ip is shared with five other domains well that's really important and let's say there's malware communicating with them well grab that malware pivot off that or even make a note to come back later this is all super helpful and so that's

kind of what i do is i just start outlining my report so i don't lose any information and then i generate a hypothesis it's really just you know the so what so what happened um put it all together um explain it kind of like the devs do explain it to the duck make sure it makes sense and if it doesn't you might need to go back and collect more information confirmatory analysis and review think steps can you validate the data matches the schema that you chose review your think steps make sure there's nothing that you missed anything any sources you missed and then dissemination you know go write the report which is one of the most

important parts um so i am going to share this slide but here is a glossary of some of the terms that i use throughout the report in case you forget i will make this available and obvious um some of the sources that i said that would be in here and thank you i know it was a very long presentation and thank you so much for joining me and if you have any questions at all please let me know and the cognitive stairways of analysis do have their own website it's cognitive stairwaysofanalysis.com um and you can read more about the framework i do implore others to contribute i'm trying to grow it to expand to other forms of analysis that

um not just intelligence analysis but maybe something devs do or you know people that do audits really just anything that can help um like guide young analysts or analysts that are just starting out in the field and that's kind of the goal that i have with this framework so i really implore others to contribute whether it's a source or a stairway of their own so thank you so much