Streamlining Report Writing with AutoRpt

thank you mark i'm really glad to be here hey everybody welcome to besides greenville 2021 super happy to be presenting to you today can everyone see this slide that i'm sharing and presenting sharing my screen out can i get a thumbs up or a hey yes we can see it thanks hilary good to see you hey you as well i'm glad you're here it's it's exciting to be here for besides greenville even if it's virtual uh gathering together with the infosec community is always so much fun today we're going to be talking about in this session specifically about report writing in the context of doing penetration tests or even training engagements that we might do including like certification

exams so the title of my presentation is streamlining the report writing process with auto report an auto report is a tool that i wrote to fill in some of the gaps that i've experienced both within my own workflow when it comes to doing security reviews and assessments and also some of the confusing information that i've gathered over time i don't know if it's going to speak to your use case as well but i'm thinking that it's going to so i'd like to present that to you today uh real quick who i am uh for those of you who haven't attended an official besides in person enjoy this nice slide of what happened in 2019 i

get a feel for that community aspect i think we're all missing that community aspect of information security at these conferences and it's been a pretty amazing run for the last few months i mean august we had defcon 29 and then recently b-side charlotte augusta and then you got the capstone of b-sides greenville so it's always good to have our local community here my name is ben acord but i go by overcast on the defcon discords and typically anything inside of the defcon uh umbrella for the defcon group i go by overcast so i answer to both it's not my the name my parents gave me but i make them call me that anyway i work at floor uh primarily with mark

funderberg we work in mission solutions where we deliver successful solutions to our government clients but i'm also one of the co-founders of the local defcon 864 group here in the upstate which is just a collection of anybody who wants to come and share in that amazing ability to to communicate what what i'm learning and then learn from each other so that's that's one of my passions is to actually be able to give back to those who are in that same community and recently i've also started teaching at greenville technical college i noticed some of my students have shown up in the our former students have shown up in the in the session so it's good to see you guys

there and i teach the penetration testing one class over at greenville tech and that's the main focus for this this talk is around penetration testing in fact the actual code for auto report came from a failure a personal failure of my own in may i sat for the offensive security certified professional exam and the next morning as i sat with a coffee cup in hand and a notepad and doing my lessons learned i realized that there were some pretty big gaps in my workflow um and some things that it cost me a lot of time and actually i'm not blaming the tools i am blaming my own workflow but i realized that that those gaps led to the

smoldering rune that was the failure email that came into my inbox that day so what i'd like to do is walk through how i realized that those gaps exist so let's take a look quickly briefly at what the penetration testing methodology is it's this layered approach and we love layers or diagrams or triangles or venn diagrams to explain things in information security i mean we can't talk about confidentiality integrity or availability without some kind of a triangle diagram of sorts but in this case we're looking here at the very bottom layer we're doing that planning and scoping of the engagement getting the rules of the road so to speak and then we're moving into that open source intelligence and

information gathering phase collecting vulnerabilities and doing that active reconnaissance attacking and exploitation and then at the very top i'm combining two tiers into one here we have reporting reporting is the last thing that's done but out of everything in this cake this layered approach it's the most important if a company engages us to do a penetration test or security review they don't want us just to go and do it they actually want to be able to receive results that they can tangibly hold and act upon and that's where the report comes in unfortunately during this process it's not sheets and ladders or a board game where you're only ever in a single state at any given time you're often required

to multi-thread task an action where you're doing some work on a vulnerability while thinking forward into either a chained exploit process to gain the shell or whatever else you're looking at there so if i could take this image and maybe just swirl the cake a little bit that's typically what it may look like during the process whereas reporting at the very end is you're taking all the results you've collected all those notes you've accumulated over that penetration test and you're boiling them into an official report a slide deck something that the customer can hold and act on but when it comes to guidance for how we write that report it's kind of a hot mess we have some

official bodies who have released some guidance on this topic and depending upon the industry vertical you're in for example if you're in fintech pci is your go-to if you're in the government space like i am nist is the body that you look almost every day and so when it comes to for example nist special publication 800-115 their reporting section in that document is literally two paragraphs long and most of that is bullet points i can summarize it for you very simply you should write a report it should contain results and you need to provide it to someone who asked for it that's the bulk of what 800 115 says about reporting and then there's other

optional bodies that have produced some pretty helpful information on report writing a few of them have gone so far for example like the penetration testing execution standard have given headings and subheadings and content descriptions for what should be contained in that report that that will aid you in the report writing process and as you can see from the vulnerability assessment dot co dot uk which just flows off the tongue they've done the same thing where they're brought blocking out what that report template should look like but as someone who's either new to the industry or working for a company that now wants to start doing security assessments or penetration tests which one of these do i pick

do i have to review all of them and merge them into something that i can act on and that's an open question here so we look to further guidance out into the industry and we do have some amazing blog posts or companies who are tried and true pen test companies they're doing this every single day and so we can glean a lot of wisdom from what they've put together and offered online but we're hoping that they've you know included some of the secret sauce to make it all come together and uh and help us in that report writing effort in the example of like a a testing body like offensive security they've given us a a template to use for the report

writing um and you can see that in the top right there from mega corp one but there's a really interesting statement when you review the penetration testing or the pen 200 course it used to be called the pwk now it's pen 200 in that section that first paragraph it says that every penetration tester will have their own style and preference of workflow and documentation and that's pretty much the standard approach you hear across the industry is it's personal it's up to you you can take notes you can document any way you want to just you know do it well enough that you can produce a report at the end and if we look to a broader scope

today if i want to find something quickly and know how to do something i'm going to look on youtube and see somebody show me how it's done and we have some amazing you know thought leaders now that are putting out onto youtube examples of how they conduct their workflow how they document it if you've ever watched an ipsec video he's monologuing it's almost like a stream of consciousness for his entire workflow through an engagement it's very impressive john hammond's been really upfront with how he takes notes and resources the cyber mentor and in fact tcm security in their course for the pnpt exam is one of the first cases along with ine where there's a dedicated training module for

how to take notes during the exam and where note taking and report writing aren't left just at the very end because if you wait to the end it's too late you may have forgotten something and you may not have access back into the company to do it so if we take all of those options and we gather them together we're left with you know ipsec's tool set we're left with john hammond's tool set trusted set gave some examples well if i've gathered all of that guidance together all that wisdom and i keep testing and trying it out which is the case that i've got here i've got a file system that's littered with examples where i've tried one

person's way of doing documentation and report writing and then i move on to try someone else's because i'm trying to find efficiencies i'm trying to find something that's really going to help me get through that finish line well so i've got this huge collection of tools i've got a huge collection of scattered documentation around on my file system and even within kali linux we have tools that are included but you either quickly hit a paywall or you have limited functionality so what i wound up doing when i sat for the offensive security certified professional exam is if the penetration testing methodology was a layered cake all of my little tips and tricks that i gathered for

note-taking were kind of like a bunch of little candies jammed into this cake baking process and really didn't fit what i was trying to accomplish just trying to if i'm trying to make a cake and honestly this analogy is breaking down really quickly i don't know how far i can run with it and this is a an image heavy slide deck apologies in advance halfway through but what i really needed my notes to be is that icing the icing of the report around the overall cake really comes from the locking in of the icing in between the layers as each layer is formed and established with the one above it my notes solidify that bridge between the two and

ultimately i'm able to produce a solid report at the end because i have a consistent flow between them i don't have like a gummy bear and a skittle and other brands between them so what if i built upon what works and this is what i had i had a notepad with me while i was drinking my coffee and the smoldering embers of my oscp and i was making notes about lessons learned what could i prove on and i thought i really love auto recon it's a great tool to do that initial uh overview of a large list of systems that you have their ip addresses and it has a directory structure okay i can work with that right you know

that's a good starting point and then at the very end i came across a product by someone named no raj on github and they take notes in markdown with predefined templates and then you run this ruby script and it produces the pdf output of your report that was really helpful for the oscp exam because you have to submit a pdf included in a 7-zip archive okay so i've got something during my engagement and i've got something at the end of my engagement but you'll notice there's white space before between and after that process and so i started writing and documenting what would go around those two what would go around that how could i enhance that options yet

build upon what they have and that's what led to auto report and that is a really bold orange so what i'm going to do now is i'm going to stop this presentation and slide deck and we're going to go straight to a live demo because if there's one thing i love to do at b-sides i did it in 2019 it's running live code all right so there's two tools that i've added in here and only two tools the first is obsidian it's a markdown editor that lets you switch between the edit view and the actual display view of markdown you know that rich html like view and display and then the other pieces i have

incorporated no raj's offensive security markdown templates those are all built into this tool so let's see here i'm going to go over and there are five use cases that we're going to walk through in this example and the first one is going to be training auto report and first let me show you an example of why i'm going to be using a shortcut auto report on my system right now

is a symbolic link over to opt auto report which is where i've done the github clone to a local instance so now whenever i run auto report just know it's really calling that python script in opt auto report has a wonderful classic menu system you don't have to use it for everything but it's here to guide you along the way if you'd like to so i'm going to use startup i'm selecting a training effort because today i woke up and i decided i wanted to do a little bit of training to get ready for my next exam attempt and in this case we're going to do a vuln hub machine we're going to do metasploitable 2.

and i do know the ip address and the output for this this is on for my own efforts in fact my instructor wants me to submit it through the greenville tech system so i'm going to submit it as a document file a libreoffice document or a not so libreoffice document and behind the scenes what auto report does is it creates a directory structure for us in a consistent fashion and it preloads a bunch of templates for us that we can then go and use now i you notice i'm i'm not sitting in any real directory to use that right now but i can navigate into that directory and take a look at the files that are there

but i can also take a look in my obsidian and notice that it's already loaded everything that's on the file system because these are just markdown files i already have a targets file that i can use and my next step here is and i'm going to introduce you to another feature called the situation report

where i'm just going to say what are the what are my thoughts and the situation report is a log of your thought processes as you're moving through a pen test and what's happening behind the scenes is auto report is time stamping that into a file that you can then review at a later time so you'll notice that it logs both its activities as well as your own and the value for this is if i get down a rabbit trail and i realize i may be on a rabbit trail maybe i can't really exploit that mysql database well i want to look back and say what was i doing 45 minutes ago before i got on this rabbit trail i just

look at the situation report and if i've been good about putting in my entries i'll know immediately where i got sidetracked i can also look at the situation report on the command line because if i don't have to leave the keyboard if i don't have to leave the terminal then i'm being more efficient so i'm able to update my note taking through sitrep while actively maintaining my engagement against the target so at this point now obviously we would run auto recon minus t and the target stop markdown file that will take 16 minutes to run against this target so what i'm going to do is i'm going to preload what i have already done before i've

already run auto recon before and i've loaded these results here pretty common we're all used to auto recon in the directory structure that just directory structure that it loads it has its own report directory but we're not going to use that because we have our own report directory structure up here now as i'm going through my engagement i have a template already defined for me in markdown that has some latex header information if you don't know what latex is don't worry that's just a way of helping to convert to other file formats and it's pre-stubbed with information about basically it's an easy way to can your write-up for your blog post your linkedin post however you want to put

this out after the fact remember we never just want to retain information that we're learning and training on we want to be able to give that back to others part of the reason why i'm speaking now you'll notice that there are boilerplate variables included in the document when you go to create the final report which is automated auto report replaces these out with specific variables for your engagement so i'm going to come down here now let's take a look at what we've got next step that i want to do

i'm going to take a look at the ports so auto report has a feature that will actually look for common nmap output file names and show you the results for each target that it finds what it does behind the scenes and i need to open the folder here

is it creates an excel file one of the things that one of the penetration testers in defcon 864 recommended when i was explaining what what tool i was building he said i like providing my clients with an excel document of all the port services and versions for every ip address and so you have here a worksheet tab for each ip as well as an all ports tab that lists every host and all the ports that were collected there and that's separate now because we know metasploitable 2 we also know that we have a bind shell that's already existing for us so we're going to test this vulnerability to see if we can confirm it and sure

enough we do get a root shell i'm going to use flame shot which is a tool that i highly recommend for screen shotting and i'm going to take a screenshot to save as evidence documents auto report training vulnerable metasploitable to report images and we're going to call this 1524 and that's all i'm going to call it and now i can come over into the report and say this is a critical vulnerability and i can start writing my documentation for this right here and we're going to use the standard markdown formatting for or putting in our images

before png and depending upon the type of format that you write in you can even do some css to increase the size of the the image i'm just going to leave it as it is for right now so there's the image and we don't have to remember to click save or anything like that it's automatically being done for us but because i've now validated a vulnerability and i've logged its evidence both saved it to disk and written it into my report now i want to go back and actually log that vulnerability with auto report

and if i do listing there's zero vulnerabilities currently listed so i'm going to add one pick my target the port number is 15 24 and for this one it's going to just be buy and shell the business impact here i can put a likelihood let's say 90 i could even if i if it reveals company sensitive data i can list that here as the indicator there remediation um pick up where i left off

after collecting the metadata about the vulnerability we're prompted to add a cvss level three um score and if you know it right here like the number i'm just going to say nine i can just enter it here and it it collects it for me if i said no to do if i know the cvss score it will walk me through every prompt to calculate and build that string for the vector and that's logged with your vulnerability as well and then it moves on to prompting us for the miter attack framework we're going to pick the the tactic and then we're going to pick the technique in this case i'm simply saying it's in initial access

of a exploit of a publicly facing application it prompts me to confirm that my values that i entered and when i say yes we now have a listing of a vulnerability again that's written out on disk as its own spreadsheet and that's going to get merged later on into your final report so let's let's pretend we finish this full engagement we are done we are ready to move on to the next thing and so what i'm going to do is i'm going to run auto report finalize and that's it your report is done now if i had completed everything in my markdown file here obviously this report would be much more thorough and complete but as you can see there's now a docx

file for a not so libreoffice uh file format and it's created the document for us for whatever reason i haven't figured this out yet it doesn't automatically update the table of contents and you can see as we scroll through some of the tables get messed up here a little bit but it has the screenshot and the information that we had keyed in for this vulnerability and exploit

all right so that's the first example is training now what if we decide we're going to move on we're going to do some more training but this time we're going to go for something big we've we've got a training budget we're going to go for an offensive security pen 200 course we're going to sign up for and start the lab they require a pdf and a 7-zip so i'm picking that and you'll notice the directory structure is the exact same i don't have to worry about wait did i did i log all my documentation in documents or was i working out of a different file system to use this it's consistent it's consistently always in your home directory documents auto

report and the markdown files are if you if you leave obsidian open it automatically recognizes that you've put new files on the file system ready to go and i can just jump right in and start working either on machines or here we have the exam report this is the exact template that offensive security released in a odt and docx format that we used to work in but now it's marked down thanks to nora no raj i've just broken these out into individual machine names that you need to record for logging one other item that i want to point out here is there's also a scratch pad marked down this is where you can just throw random notes and

collection pieces along the way without mucking up the markdown for a specific machine and again the sitrep report only applies to the engagement that we're doing at hand and if i get confused and let's say for example i'm over in doing something else and i decided you know what i want to start back on my pen 200 class again what was what even was the the last engagement i was working on i just run auto report active it queries what it knows about all of our engagements there could be thousands of them and it's going to say this is the last one you were working on and here's the directory you were working out of

so i'm going to cd back over into that and the targets file for this is empty there's some training where you're going to have to update that manually and it's true of your exam it's true of an actual pen test because we're not going to be able to preload that for you from a programmatic standpoint easily go into report and you'll notice report just contains basically markdown files at this point for for the sake of time and argument let's say we completed pen 200 we've successfully poned every system in the lab all the domains and our report is complete and we're ready to finalize that we just run one command the pdf is created for us by merging all

of those markdown files with a number and then the 7-zip archive is created and on the file system if i go back to my training i now have an offensive security folder and i can look into that reports directory here's the 7-zip archive with the pdf included and here's the actual pdf i'm going to scroll out a little bit now you notice i forgot to put my offensive security student id i am working on a bug to fix that it prompts for it for the exam but it doesn't do it for the training yet we also have a breakdown on the side for the pdf as well as in the document itself so if you want to

jump to a specific section you just click the link and it'll take you right there it's already pre-built just by running auto report finalize you don't have to do anything special within your markdown all right that's training the second uh use case that we have is actually taking an exam i mean after all one of the capstones that we want to do is we want to show that we actually have performed some training and gain some knowledge and one of the ways we do that is a we can write stuff out on the interwebs we can post videos about them but ideally we're going to want to get an security certification so i'm just going

to run auto report startup up it's hard to talk and type at the same time and this time instead of picking training i'm going to pick exam and there's a pre-canned list of exams included within auto report where it knows the report structure it knows the layout and it has those pre-canned for you thanks to no raj i want to point that out no raj's work with markdown was phenomenal love it one of the ones that i'm working on personally is the practical network penetration tester exam and converting heath adams document file into markdown so it will be baked in from scratch that one's got a little bit of rough edges right now but for right now we're going to take

the oscp what is my os id it is going to be one two three four five apologies if that is a legit os id number for somebody out in the wild um and there you go i mean we can now change directory into that into that location and wait for the the countdown and the email to come into our inbox we can kick off that vpn access and get started uh on the lab and my workflow for documentation on an exam is really no different than training i'm gonna edit my

i'm going to edit my targets file to include the 4 ip addresses that aren't the buffer overflow box pop those in there i'm going to run auto recon on those and while those are going you got it you guessed it right i mean the same workflow that we do with everything else i'm going to get started on the buffer overflow box i've got a scratch pad here for ideas that come to my mind while i'm doing it and the sip rip sit rep is still growing and and running with us and just like with the pen 200 course whenever we're complete with all of these items we just run auto report finalize and i'm not trying to drag out the

meeting here i'm really not i'm just cutting through it as fast as i can to show you from a from a workflow perspective from pre-engagement during the engagement and after the engagement everything is seamless with with auto report hopefully i mean it so we've got once again i'm gonna go back over here to auto report exams directory offensive security oscp and let me pause here for just a second you'll notice that each of these directories contains the engagement that you're going after as well as a date stamp that means i failed the exam in may if i take the oscp again in in three months i'm going to get a new working directory so i'm not going to gunk up my

previous attempt that now gets to stay pristine as an archive that i can reference later on and this new one is something that i can push through with success because i'm going to pass it this next time and here's our report directory it's created the 7-zip archive file for us if we wanted to we could include or add the file from our pen 200 pdf to this so we can submit both the pen 200 lab report as well as the exam report for the oscp and here's rocp exam report you'll notice that it does include our student id which is required and we have everything we need to showcase what we've done on the systems

complete done obviously we would need to write the actual report for each of these boxes as we as we own them so that is the exam let's take a look at doing a ctf what if and we're kind of hoping we can do this we're going to do a ctf uh the ctf event name is b-sides greenville 21 and the team name uh the team that i'm on is defcon 864 and we publish a bunch of our stuff into martin for some of the things that we do if you want to write it to a blog report or something like that afterwards because we're all concerned about that that write up afterwards because the lessons learned you'll notice i now have

a whole new area here called ctf for the engagement of b-sides greenville 21 and my team name so if this was on a network share uh as we engage in what we find in the network let's say we find a new um hang on let me get to that directory so i can show that to you so right now in this directory there's only one file for the actual work the actual engagement and for the most part anything that is a template related to training or ctf right now includes a section at the top for lessons learned this was put in there to help you with the after action review where not only do you complete the learning and the

training but it includes those wonderful areas where you look at it not navel gazing but you think through what did i do great what did i struggle with that's now included in your report by default if anything the the training and the ctf templates make it a one-stop shop you do the work and then i wanted to make it easy to be able to share that with the rest of the world post it on medium post it on your blog whatever you wanted to do use it as a script for a youtube video there it is but let's say in the ctf uh we discovered that there is a new machine something that you know one of the one

of our teammates says hey there's a new machine on the on the wire i can just run auto report add it says do you know the target ip address i'm just going to say yes i do here it is and it creates a new file for us and it updates the target.md file so if i cat

it's now added that ip address into the mix and i now have a new markdown file specific to that ip address ready to go so if we're on a network share the team's working together we're working and adding those files on the fly and auto report is just automatically adding and picking those up in the workflow no problem i do want by the way those of you in defcon 864 when we do get back into our ctf lab again i really want to test this with a full-blown all of us hammering away on it i want to see if we can break that so there we go throwing that out there nothing like senior code

for one die um auto report ctf is done we won so let's go ahead and can that report and wrap it up for us we've got the final report is in markdown and you can see that it changed it from the obsidian markdown that we were working with before and it's changed into the common markdown file format that i can just click the button at the top here and actually see what it looks like in the finished product the last two we have from example perspective our bug bounties number three for a bug bounty we're gonna do bug crowd and the program that we're going after in bug crowd is called defcon 864 and this time we're going to do it in

the github formatted markdown language which is number six again exact same workflow i can cd into that directory and the the cool thing is i can be anywhere on this file system um i mean i'm not in that working directory right now but if i run

what i want to do next it knows where that sit rep file is located for the active engagement and it writes it there so i can come back up to my bug bounty for this engagement and i can say where did i leave off oh yeah i kicked off auto recon when i run the ports command it logs that it ran the board if i log a vulnerability it logs the vulnerability uh here in the system in fact if i go back into metasploitable 2 for just a brief sec because i forgot to mention this and look at the sitrep you'll notice that it creates a backup of the vulnerability i logged i'm really bad

about fat fingering something and screwing up a file but here in this case if i needed to i could rebuild that file and then a pen tr pen test is the exact same thing it's the exact same thing um

1964 my client name is going to be besides greenville i don't have authorization to do this so technically i probably shouldn't be doing that and they have a jira instance that they want to allow all of their their findings to so i'm just going to pick jira as the output and there you go no surprises exact same workflow everything looks the same i'm now no longer worried about um my work my workflow in that regard there is something that i want to point out that you may have noticed as i was working through this and that's the all ttps folder that gets created to have that in your same workflow area that's if you wanted to create like your

punch list for how you pop active directory what do your common sql injection workflow that you do that's where you collect all of those notes and that's the bulk of it i do have one super secret thing that i want to show you guys that's just recently been released but first i want to open it up for questions before i fly through any further are there any questions how long have you been working on this since about may and it started out super simple but it's definitely gone way off track

all right so the super secret thing that's just been released there may come a point when you've done a ton of engagements and at the end of the year you want to look back and say you know man i've done a lot of things this year but it'd be really nice to know kind of maybe some stats and statistics on that so you might sit there in a very dramatic fashion say what have i done well i just ask auto report that and see what it comes back with and it tells me that i've started a few things and never finished them i've started some other stuff and actually finished them and if i had started an engagement and

run ports or volumes or something like that it would show it as being in progress and this is a great way for me to look back and just take stock and inventory about where my strengths and weaknesses are at a macro level

so it was a really quick introduction but my hope is that this is a tool that all of you can use and i hope everybody uh grabs it and kicks the tires on it if you have any problems or run into any issues with it open an issue on the github page and i'll address it as quickly as i can with all that spare time that i have and uh but yeah it's for you it's released gpl3 so it's open source uh enjoy