Reviewing a Pentest Report: Six Red Flags

The pentest happens out of sight. The report is where you find out what the testers actually did. It may go to your engineers, your board, an auditor or a regulator, so it needs to be more than a list of possible vulnerabilities in a polished template.

Start with one finding

Pick an important finding and ignore its severity label for a moment. Does it name the affected system? Can your team reproduce it? Is there evidence of what the tester achieved, and does the remediation fit your environment? If the answer is no, the red or orange badge beside it does not tell you much.

A useful finding report should connect a weakness to your host, show you a route to exploitation, affected role or business process, and then show what happened when the tester tried to exploit it. That is the standard to follow for the rest of the report.

Six red flags you should look out for

One weak passage does not prove the whole test was poor. Repeated signs are different. If several of these appear, ask the provider to explain the work.

• The findings could describe any company. The report should identify where the issue exists in your environment and what the tester observed there.
• There is no reproducible evidence. Phrases such as "may allow" and "could potentially permit" need support, unless verification would be destructive or you explicitly listed it as out of scope. Depending on the issue, that might be the request and response, a screenshot, a command, or carefully redacted data that should not have been accessible. Unsafe exploitation can be excluded, but the report should say what was and was not verified.
• Severity has no business context. A scanner can assign a technical score. However, it cannot know that one host is an isolated test box while another handles customer payments. Business context matters when assigning severity at ClarenSec.
• Another client's details appear. A foreign company name, hostname or IP range in your report is a red flag. It also raises a confidentiality question: if their details are in your report, where might yours appear?
• Scope and method are unclear. The report should state what was tested, what was excluded, when the work happened and which methods were used. That includes whether the test covered internal or external attacker positions.
• The engagement ends with the PDF. Findings still need to be fixed and checked. Many reputable firms include a retest, but you should confirm the terms in writing before the engagement begins.

A clean report is not automatically suspicious

A well-defended system may produce few findings, and a responsible tester should not inflate the count to make the engagement look busy. A short report can still show serious work. It should explain the coverage, the accounts or roles used, the important tests performed, and any limits the team encountered.

It should also distinguish between "we tested this and found no issue" and "we could not safely test this in production." Those conclusions are not interchangeable. No penetration test proves that a system is secure; the report should give you enough detail to judge what was examined and how much confidence to place in the result.

Length is not depth

A long report can be thorough. It can also be unfiltered scanner output presented as a pentest, and finding counts alone cannot reliably distinguish the two. Look at how much of the document is specific to your systems, how many results were manually investigated and whether related weaknesses were combined into an attack path.

Scanners are useful during a penetration test. They cover ground quickly and help a tester decide where to investigate. The problem is not that a tool appears in the process; it is that untouched tool output is presented as the finished finding. 

Poor triage pushes low-value work onto your engineers. They spend time clearing duplicate findings and informational noise while a smaller, exploitable path remains open. The tester's job includes separating that signal from the noise before the report reaches you.

What the report should let you do

Leadership should be able to understand the plausible business impact and decide what needs attention first. Engineers should be able to reproduce each verified finding and apply remediation that fits the affected system. 

The executive summary should therefore do more than count critical, high and medium findings. It should say what an attacker could reach, which business process is exposed, and whether the immediate concern is an exploitable path or a weakness that increases future risk. 

After remediation, the report and retest record should show which findings were verified as closed, and which remain open. That record is what makes the test useful months later, when a board member, customer, auditor or regulator asks what changed because of the engagement.