Crafting Compelling Pentest Reports by Piyush Verma

hello hello hi everyone welcome to the talk my name is p Verma my pronouns are he him today we're going to talk about the most exciting topic when it comes to pent testing pentest report writing this is where you as a pentester gets to flaunt your findings right you get to communicate whatever critical or high severity bugs you found in a document to the customer right before we get started how many pentesters are there in the crowd could you raise your hands I see a few nice to see you as well how many of you enjoy pentest report writing is that your favorite part oo I'm glad to see at least one hand brilliant well I was in the same

boat where pentest report writing was not one of the favorite parts or one of the favorite phases when it came to pentest in general I would rather spend my time finding critical bugs than reporting them but after spending a lot of time finding the bugs if I'm not able to communicate them properly in a fashion that the technical and non-technical audiences can receive it then what's the point right that's where pains report writing comes into picture and there are lot of elements that we can look at to enhance the overall quality of the report itself okay so starting off I am currently working at hacker 1 as a staff technical engagement manager managing the pentests and

challenges within the hacker one assessments team over the course of my career I've written reviewed and seen several pentest reports and the experience that I have had that's what I aim to share with you today okay quick disclaimer this is a personal opinion and not not that of my current or former employers the writing style May differ so your writing style May differ from mine and that's totally cool as long as the report is professional and consistent overall right so this is the agenda for today divided into four different sections starting with the elephant in the room that is the pent test report why do we even need a report in the first place can't we just find bugs and

call it a call it today unfortunately not because that's required by the customer it's the only tangible product that they get at the end of the engagement itself moving on we'll talk about impactful writing now this is not going to be an English gramar grammar course okay but there are some nuances that need we as pentest report writers need to be mindful of and careful of and avoiding some cardinal sins to say the least in a pentest report itself then we'll move on to the artificial intelligence piece see what is the scope of AI when it comes to pentest report writing and finally wrap it up with the next steps if the topic is interested to you then you can delve

into deeper uh with the different courses modules so on and so forth pentest report writing why do we need a pentest in the first place what is the importance of the report itself again as I said it's a tangible product that the customer gets at the end of the engagement it's some something that they can use to demonstrate compliance to the different regulations and standards something that can be used to show and demonstrate due diligence and finally address broader areas of improvements within the company so as an example if appentice reported sensitive information and sensitive files available in network shares then that could translate into security awareness trading curriculum for the employees within the organization

let's take a look at some of the key sections and elements of a pentest report now there are okay starting off with a caveat here that this is just one style of writing there are several thousands of pentest reports out there with different formats and they are totally cool okay now this these are the key sections for a pentest report starting with the executive summary the executive summary is seen as the most important section in the report because this is what the executives within the organization receiving the pentest will actually be looking at right so keep the audience in mind this the target audience here are the non- technical people the executives the key decision makers within the

organization right so don't uh add lengthy statements keep it short and sweet when it comes to executive summary as a title suggests it is only a summary that needs to be no more than two pages and that's also a lot of text or area for you to cover the idea here with the executive summary the content that you typically see is the different parties that were engaged in the pentest engagement when was the pentest conducted what was the scope of the engagement at a very high level if there were some logistical requirements that need to be mentioned Within the executive summary that will also be highlighted here if a VPN connection was required to perform a pen test so on and

so forth right the idea here is you do not also you do not call out just what the issues were you call out the strengths as well where the customer succeeded what were the good things that you as a pentester observed in the report so both strengths and weaknesses at a very high level needs to be reported here right and finally add add some key matrics just to show some victorial depiction of the vulnerabilities the key matrics could be in a form of a chart or a bar a bar chart or a pie chart which need which can be actually um you can actually add vulnerabilities by severity or vulnerabilities by asset depending on how you actually want to

depict uh the key idea there but again keep it very high level the audience is non-technical right so that's the theme behind a report there would be SE that are technical there would be sections that are non-technical right so it's like a blank canvas and you are the artist you need to paint it in such a manner that whoever sees can actually appreciate the art in a different fashion talking about technical summary this is where you can actually dive a little more technical into the technical uh vulnerabilities themselves you can include the attacks performed the results achieved typically you can divide this into most severe vol vulnerability which could be based off of the CVSs scoring system or you can

also mention the most common vulnerability that could be based off of the most uh C the most common cwe category like broken Access Control idors sensitive information available on public shares so on and so forth and finally you can it's always good to add a table of vulnerabilities starting and a typical table would have the ID of the vulnerability the vulnerab title and the severity it's also common to see another column called the status column that reflects the status of the vulnerability whether it is open or closed right moving on to the meat of the report itself this is where you actually dive into each individual vulnerability identified during the assessment starting with the title you don't have

to write an entire paragraph for a title let's try to answer two important questions with the title what was identified and where was it identified right so what was the vulnerability and where did you find it command injection in the HR portal it's a very simple example let's keep it short and sweet and then try to capitalize each word again there's a debate about it where people want to capitalize every single word even the prepositions and adjectives whatever your choice is keep it consistent throughout the report right the affected asset this could be the IP address the URL or any parameter that was identified as vulnerable and finally add a CVSs string that's totally optional but it's always good to have

that justification in there and once you know the CVSs string you can actually translate that into the description itself right so if you know what the attack Vector is the access control is whether the Privileges are required or not you can expand it into a statement and that becomes the description of the vulnerability itself right so in the vulnerability again you're trying to answer the same questions what was the finding when uh where was it identified and if you follow the CVSs it becomes easy to describe the vulnerability itself coming to the impact mention the worst possible case but keep it real don't add over dramatic or sensationalizing language because it does not serve anybody's purpose right

keep it real keep it short keep it concise and with the description one important thing to note given that the pentest is a point in time assessment it happened in the past right so write it in past tense again no English grammar coaching but then some logical things that make more sense right because if somebody picks up a pentest report 2 years later and see that the vulnerability is written in present tense then they would think the vulnerability still exists right whether it has been fixed or not that's a different question altoe finally with steps to reproduce you're trying to walk the reader on how they can reproduce the vulnerability on their own systems this is where you can

actually add screenshots you can add code Snippets that are required to add more depth to the uh steps of reproduction here talking about screenshots some of the best practices when it comes to screenshots is highlight sensitive in highlight the key details redact sensitive information nobody wants to see a password or an API key in the report itself right so redact any sensitive information obfuscate there are several tools out there like green shot flame shot so on and so forth right make them clear and readable don't add an entire browser as a screenshot when you only want to show the top right corner of the browser itself right add multiple screenshots there is no limit but don't let the reader zoom in and see

a pixelated screenshot where they cannot really figure out what's happening so highlight the key details that need to be shown in a clear and readable fashion adding a caption is a best practice it basically helps the reader understand what you're trying to depict from the screenshot itself and finally add a border that's an optional thing because a white screenshot and a white report doesn't really show where the screenshot ends so adding a border is an optional thing that just makes it uh look better in a report here's another screenshot of BB Suite so recommendation okay this is another brilliant topic where we have seen recommendations that are generic we have seen single liner recommendations as well such as update

to to the latest version how is that actionable or Implement proper Access Control it's not actionable right so keep the recommendations effective and actionable avoid using single liners and specify what was what can actually be actioned on the technical team to fix the vulnerability itself so instead of writing update to the latest version expand upon it do the research find the latest version and add the link from the vendor itself to the references section at the bottom so references section can be simply used to uh for for the reader himself to actually go on and explore the vulnerability themselves sometimes these are vendor links such as Microsoft Patch so on and so forth another quality

of life suggestion here is change the url to a name so that the reader understands what the what's the purpose of the U of the URL itself right and do not over reference do not add 75 URLs right keep it short and sweet two to three legit references are a good medium talking about appendix this is where all the supplemental data that could not fit anywhere in the report or made the report look ugly could be added so if you're doing an internal Network assessment that had 75 different cidrs you don't need to mention them in the report just added to the appendix under the scope heading right then you can actually add the methodology that was

used to perform the assessment add the approach that was taken in different stages starting from reconnaissance all the way to the reporting stage and through the retesting as well Define the risk rating what was used so if you use CVSs scoring system what does critical mean what does high mean what does medium mean Define those under the risk rating sections identify the key people involved for the assessment the man manager from the Consulting Department the lead pentester the pentesting team add their email address or contact information so in case the customer has any questions they can always reach out to them and finally the tools that we used to perform the assessment you can actually add the name

of the tool itself and add a oneliner as to what was the purpose of the tool right you can always add attack path diagrams they look beautiful they are great they are my favorite so you can actually see in a visual fashion as to what was done from stage one till stage five like how did the attacker approach what systems were compromised how they exploited and how did they reach the final keys to the kingdom right you can use tools like draw.io for the purposes of this finally looking at restrictions so these are the things that impeded appen testing efforts common items listed here are the testing window whether the testing window was during business hours

outside of business hours weekdays weekends so on and so forth highlight those things right it just adds more weight to the point in time assessment whether the environment that was used was stable or not whether the credentials that were received did you receive the credentials to perform the pentest 7 days into the assessment that's almost halfway so mention those restrictions under this particular section couple of samples with restrictions and without restrictions and finally the disclaimer this is the cya basically right you are trying to limit the liability protect the firm from any potential legal actions and also add in weight to the fact that this again is a point in time assessment an attacker or a Threat

Vector with unlimited time and unlimited resources would have more time to attack the same assets and also the fact that the product evolves over time right so this there was something that might have changed a week after you performed the pen test so again all of those things are important in a disclaimer so that's about the pent test report let's move on to the impactful writing again the key here is avoid using casual language okay it's a professional report keep it professional writing styles may vary your writing style may look different than what I will be presenting today and that's totally cool as long as the message is conveyed to the customer so in no particular order of

priority some foundational topics to consider are font styles again avoid using funky fonts comic SS right stay consistent with the font Styles so it doesn't mean that you start with a heading uh with a Times New Roman and you end another heading with a different font altoe consistency is the key so if there's one thing that you can take from this talk is keep the report professional and consistent okay font sizes are different for headings paragraphs and code blocks I feel very silly talking about these things here but then these are important capitalize the acronyms and personal nouns please please do that it's a cardinal sin not to capitalize all the acronyms and whenever you use an

acronym in the the first time in a report Define it Define what an acronym actually means date format again this is very debatable whichever date format you want to use use it but please use it consistently through the report right so this one that you see on the screen is defined by ISO feel free to use that as well T we basically covered with the vulnerability description where a vulnerability is a point in time assess uh pest is a point in time assessment so write the vulnerability details in a past tense and present tense is where you're actually Walking The Reader through how to reproduce the steps so steps to reproduce could be in present

tense another exception there would be executive summary that could be in present tense as well or parts of it writing in third person comes across as professional if you write something in first person it rather reads as a journal entry or an entry in your diary of a person right I identified this I did this today that's not really the purpose of a report right so writing in third person is again another good practice and finally avoid using contractions there's there's there several topics around this as well but these are the key things that I think would improve the overall quality and standards of a report so right to the point avoid using fluff avoid bloated statements I know we love

writing we want to show the impact of a vulnerability but keep the statements concise avoid using redundant words as you can see in this example the same message can be conveyed in a simple statement another example under the bloated statement the same message again can be conveyed in a smaller statement right to avoid the fluff AI let's see what's the possibility with AI today today so start with caution again avoid sharing sensitive information sensitive customer information with any random llm model you can use AI to draft vulnerability writeups based off of the cwe or the name of the vulnerability category itself right but don't enter any sensitive customer information in the AI if they're not aware about it and

you don't know how the information will be translated further in their systems another use case is tailor the content for specific audience you can enter a technical jargon there and ask the AI to present that message to a ceso how would you translate that for a ceso or a CEO how would you actually translate a SQL injection vulnerability for operational reputational or financial damage to the ceso right because that's what they care about at the end of the day so suggest concise strategic recommendations and validate the CVSs score so validate the CVSs you see a asterisk next to it cuz I haven't seen any accurate results with the CVSs validation talking about ai ai is built

into multiple different platforms it's not uncommon to see that a lot of pen testing platforms do it hacker one has their own AI system built into the platform called high that helps the pentesters hackers bug bounty hunters so on and so forth to assess with their report writing skills you can help you can use high to summarize the vulnerabilities to find a fix for the same vulnerability that was reported right you can actually help use high to help you with the um how to fix the vulnerability itself how to um address the different audiences based on that and then finally you can use high to enhance the grammar of the written content that you had

right it's very similar to grammarly in that sense but then it's available within the platform itself so you don't have to go outside the platform that's the idea if everything is within a certain platform within the same ecosystem then that's perfect right think of AI as an assistant whatever AI spits out is not the gospel right try to use that as the template create your own automated templates for different vulnerabilities and customize them that that's how you start to stop hating report writing right and finally in terms of next steps if you want to dive in deeper into the topic there are multiple courses such as the zero point security um this is the same company that has the CR RTO

certification certified red team operator they have a course on specifically dedicated to report writing called The Art of report writing offsec and hack the box has have specific modules within their pent testing curriculum that speak about report writing right looking at the reporting tools themselves Microsoft Word Google Docs have been the traditional tools but there are multiple platforms that can be used to generate and draft vulnerability reports such as CIS reptor here Plex TR and Riders CIS reptor is open source it's relatively new but Plex TR is a very commercial Enterprise product that can be used to create vulnerability writeups and then at the end of the day you use that database of vulnerabilities start a new project for a different

customer and generate the beautiful PDF that is customized to your work right that's it thank you so much everybody for Patiently listening and besides thank you PRI so we may have a time for one question before we hear from our lunch sponsor so does anyone have any questions

okay I can't do them when you're taking the

screenshots keeping being the background white is better cuz it uh the screenshots pick up black black

blobs okay so that is a little piece of advice actually thank you for making a white background on your screenshots okay um gentlemen in the sunglasses one last question sorry I'm not going to get to um the the other one but go ahead gentlemen in this glasses thank

you MB how much is pentest supporting cyber security masturbation or well that's a very interesting was that what you said sorry I I might have misheard you that's what you said okay I heard that right yeah wow nobody prepared me for that question are you are you basically saying is it is it cyber security theater not theater necessarily it's more like one cyber SEC professional saying to another hey Che thisit out look at the stuff that discovered look at how elaborately I've done at the end of the day question we need to be answering is so

yes's say the presentation needs to be elaborate this is necessarily disagreeing on elements of the report but the exan is often the only part that gets presented to the business additional interpretation by RIS right are we focusing too much on that PDF which is problematic of itself risk management systems and so on and so forth but is that PDF that we produced really just to impress ourselves and to justify the spend we just put into a pentest or is there how do we make it a better product to incorporate into our processes vulnerability management because it is a point in time artifact it's been produced and the vulnerabilities have been hopefully corrected that's it value

diminishes rap okay so I can answer that you can answer that I have so many thoughts but go ahead lost my chain of thoughts there but then I think the pentest report document helps justify a lot of things and then it can fade away over the period of time as you said because it loses its value but then it helps paint a picture over a period of Time how the product or whatever was tested has evolved right now I get it like you know painterest report writing is not the greatest thing out there that as pentesters we enjoy doing and there are a lot of different ways to automate things make life much easier but then at

the end of the day that's the only tangible product a customer gets I don't

knowe yeah quity can range from this is activing me as I read it to I want to B this and put it on my sh at the end of it if I'm ingesting this and I need to do something with it I want something I can Jon

to fund the remediation that clearly and very articul says this is why we need to fix the problem understanding that some of that responsility

absolutely so again append report is just one of the deliverables so let's say for example again I'm not going to talk about hacker one here because that's what we use our platform itself if you can import the vulnerabilities and reports into jira for example with your tool that makes more sense to you in a way that you can create a dashboard of vulnerabilities yourself and you can transfer those to your developer team that makes more sense for you because at the end of the day you're not going to use the PDF and send the PDF to different departments and alongside that alongside the PDF F there are different other deliverables that a customer perhaps ask such as a CSV file of the

vulnerability reports right that's one oldfashioned way but nowadays with platform Integrations you can use software like J and others to integrate within your own ecosystem so you can actually use it the way you want to use

it I think yeah I think maybe we need to sort of wrap this up so I want to say thank you maybe the two of you can uh meet afterwards and and continue the conversation uh but for now thank you so much for PR for your talk