Vulnerability scanning
vs. penetration testing
One tells you what is wrong. The other tells you what someone can do about it. You need both, and one does not substitute for the other.
— THE DIFFERENCE IN ONE SENTENCE
A scanner finds weaknesses one at a time. An attacker uses them three at a time.
That is the whole gap. A scan is an inventory of what is known to be broken, item by item, scored item by item. A penetration test is a person putting those items together until they reach something that matters — and what they reach is rarely the sum of the parts.
— SIDE BY SIDE
Two tools, two questions
The first five rows follow the structure PCI SSC itself uses to separate the two, in Penetration Testing Guidance v1.1, §2.1. That document predates PCI DSS v4 and uses the older requirement numbers; in v4.x, scanning sits at Requirement 11.3 and penetration testing at Requirement 11.4. The frequencies did not change.
Neither one tells you whether your team would notice an attack in progress. That is a covert exercise, scoped and priced separately.
— GETTING THE WORDS RIGHT
Four terms used as if they were one
Most of the confusion in a security proposal starts here. It is worth asking a provider which of these they are quoting you.
Vulnerability Scan
The automated run itself. A tool compares what you have against a list of what is known to be broken, and reports the matches.
Vulnerability Assessment
A human-validated scan to filter false positives and rank risks. Beware raw scans sold under this name.
Penetration Test
Authorized, time-boxed and goal-driven. Someone attempts to reach something of value, and proves whether they can.
Vulnerability Management
The ongoing program that fixes what the other three find. Not a deliverable — a cycle, with an owner and a clock.
— THE SHORT ANSWER
Do you need both?
Yes — and not because we sell both. They answer different questions on different clocks, and the standards treat them as two separate obligations.
Scanning is coverage
New vulnerabilities are published continuously and your estate changes every week. Scanning is how you keep up across everything you own. It is fast and cheap enough to run on a schedule, and that is exactly its value.
Testing is depth
Testing is how you find out whether your controls actually hold, and what a stack of “low” findings adds up to in your environment. It costs more because a person has to do it, and it answers a question no tool answers.
Because almost everything worth protecting sits behind the login. The most common serious flaw in modern applications is broken access control — one signed-in user reaching another user's data, or an ordinary account reaching an administrative function — and it only surfaces when a tester can sign in as each role and try to cross the lines between them.
OWASP's method needs at least two accounts to prove that one user cannot reach another's records; PCI's guidance expects application-layer testing to exercise every role and access type. Without credentials, you have bought a test of your login page.
OWASP's method needs at least two accounts to prove that one user cannot reach another's records; PCI's guidance expects application-layer testing to exercise every role and access type. Without credentials, you have bought a test of your login page.
It means no tool matched a known signature on the systems it could see, on the day it ran. That is a useful floor, and it is not a verdict.
A scanner rates each weakness on its own. Real compromise usually comes from a chain: a piece of information here, a permission that is slightly too broad there, a service that trusts the first two. Each link can score “low” and the chain can still end at your data — which is precisely the case NIST warns about.
A scanner rates each weakness on its own. Real compromise usually comes from a chain: a piece of information here, a permission that is slightly too broad there, a service that trusts the first two. Each link can score “low” and the chain can still end at your data — which is precisely the case NIST warns about.
Scanning more often is a good idea on its own merits. It will not answer the test's question.
A scanner checks whether a known weakness is present. It does not check whether one signed-in user can reach another user's data, whether a business process can be run out of order, whether a permission model holds under pressure, or whether three harmless findings form one path. Those are judgment problems. They need a person, and they are where most real incidents live.
A scanner checks whether a known weakness is present. It does not check whether one signed-in user can reach another user's data, whether a business process can be run out of order, whether a permission model holds under pressure, or whether three harmless findings form one path. Those are judgment problems. They need a person, and they are where most real incidents live.
Not at the top of the list. A scanner's severity score describes the vulnerability in the abstract; risk is a property of your environment. The same score means something different on an internet-facing server holding payment data than on an isolated test machine.
Two things make that list usable: removing what is not real — false positives are a documented weakness of every scanner, and NIST says results need an experienced person to interpret them — and re-ranking what is left by what can actually be reached and exploited where you have it.
Two things make that list usable: removing what is not real — false positives are a documented weakness of every scanner, and NIST says results need an experienced person to interpret them — and re-ranking what is left by what can actually be reached and exploited where you have it.
NIST SP 800-115, Technical Guide to Information Security Testing and Assessment, §4.2 (vulnerability scanning, false positives, combinations) and §5.2 (penetration testing) — nvlpubs.nist.gov
PCI SSC, Information Supplement: Penetration Testing Guidance v1.1, §2.1 — pcisecuritystandards.org. Guidance, not a standard, and written against PCI DSS v3.2; in v4.x the normative text is Requirement 11.3 for scanning and Requirement 11.4 for penetration testing.
NIST CSRC Glossary, vulnerability scanning: “a technique used to identify hosts/host attributes and associated vulnerabilities.”
— NEXT STEP
Not sure which one you are being sold?
Tell us what you need to protect. We will put in writing which of the two we are proposing, what it covers, what it does not, and what you get back.