It has your logo in the corner and the words "Penetration Test Report" written across the cover page. Forty pages, a severity chart, a severity summary. What it fails to tell you is the exact thing you paid to find out: did a person actually try to break into your systems, or did a tool run for a few minutes, and generate a 'report' for your environment. An vulnerability scan and a penetration test are different products, but are routinely sold under the same name, and most buyers cannot tell one from the other.
Two Products, One Name
An automated vulnerability scan and a manual penetration test answer different questions. A scanner asks "what known weaknesses are present here?" It checks versions, banners, and configurations against a database of signatures, it then reports everything that matches, but never tries to break in. A penetration test answers a more difficult question: "what can an attacker actually do to this organization?" Hrer, a person chains little weaknesses together, gets a foothold, moves further, and reaches something that matters, a database, an admin console, customer records. The scan finds the doors that might be unlocked. The test walks through them and tells you what is in the room.
Both have a place in your organization security. A scan earns its place as routine hygiene between tests. The tool is not the problem, the problem starts when the scan output is dressed up, renamed, and sold to you as the penetration test. The frameworks that the market respects keep the two separate for a reason: standards such as PCI DSS distinguish a manual penetration test from automated scanning, and methodologies like PTES, the OWASP Testing Guide, and NIST SP 800-115 describe testing as work a human performs.
If you need the two-part definition first, our guide explains what VAPT means and how a full engagement runs.
The Signs To Look For In The Report
You can usually settle the question without any technical background, just by reading how the document is built. A scan dressed as a pentest leaves some visible marks. Here are the things to look out for in the report.
- Raw scanner formatting: Findings are written as a table with columns like ID, Host, Port, and CVSS, etc. A real test report is written to be easily understood by your management, and IT teams, not lifted straight out of a tool.
- Long CVSS lists with no business context: Page after page of severities and scores, with no explanation of what any of it means for your organization. The tester is meant to tell you which of these actually threaten your money or your data, and which are noise on a server nobody can reach.
- Generic copy-paste remediation: "Apply the latest vendor patch" and "disable unused services" repeated under every finding is the tool's default text. A human who looked at your environment gives advice that fits your environment.
- No narrative of how the tester moved: This is the clearest tell. A pentest report tells a story: started here, found this, used it to reach that, then escalated. If there is no account of movement, nobody moved. A tool produced the findings, and a person changed the cover.
Then check the dates against the effort. A serious test of a reasonably sized environment is days of human work.
The One Question That Settles It
If you remember nothing else, remember this question to ask the vendor: can you show me, step by step, how you got in? Proof of concept is the line that seperates the two products. A scanner can tell you that a login page accepts weak passwords. A pentester would actually login, and show you the session, and tell you what that access provides. A scanner can flag that a service is running an old version with a known flaw. A pentester exploits the flaw, captures the result, and puts the evidence in the report.
What Exploitation And Post-Exploitation Look Like
Exploitation is getting in: turning a weakness into actual access. Post-exploitation is what happens next, once a foothold exists. This is the part a scanner cannot replicate, and it is where the value of a pentest lives.
A genuine pentester is able to chain little different findings into one impactful exploit. A single low-severity finding on its own might look harmless, and a scanner would list it that way. A pentester sees that the harmless finding gives a username, the username plus a weak password policy allows password spraying and authenticated access, the login gives access to an internal share, and the share holds a configuration file with a database credential. Four small findings a scanner would likely list as low, combined to gain access to your data. That attack chain is the work. It is the type of reasoning a tool cannot accomplish, because the tool grades each finding in isolation and is unable to ask what they add up to.
The question is not "how many highs did we get?" It is "what did the pentester actually reach, and what would it have cost us if a real attacker had reached it first?" The Sterling Bank data breach and the FCMB heist of about N677 million attributed to the ByteToBreach actor were not theoretical findings on a list. They were impact. A pentest is meant to find that path before someone outside does.
What To Ask Before You Sign
- Ask for the methodology in writing.
- Ask for a sample report.
- Ask who actually runs the test.
