Black-box, grey-box, white-box
Three names for a single variable: how much the tester knows when the clock starts. What each one buys you, where each one is blind, and how to choose in about two minutes.
— THE WHOLE PAGE IN TWO SENTENCES
You are not buying three grades of rigor. You are buying a fixed number of expert days — and the only thing that changes between black-box, grey-box and white-box is how much the tester knows when the clock starts.
That single variable decides where those days go: into finding the way in, into measuring how far someone gets once they are already inside, or into understanding how the system was built.
This is the normative definition, not a marketing one. NIST classifies the three by the tester's level of knowledge of the internal structure of the target — not by how realistic the exercise is.
Not a vulnerability scan
A scanner lists what might be wrong. A test proves what is, by exploiting it under agreed rules. Unvalidated scanner output is not a finding and does not go in the report.
Not a detection exercise
None of the three approaches measures whether your own team would notice. That is a covert engagement, and it is scoped and priced separately.
Not a guarantee
A test describes what was true, inside the agreed scope, on the days it ran. It reduces risk by showing you what to fix. It does not remove risk.
— FIRST, THE BASICS
What a penetration test actually is
An authorised, time-boxed attempt to reach something of value in your environment the way an attacker would: find weaknesses, chain them together, and prove where they lead. What comes back is evidence — this is what someone could do, this is how far they get, and this is the order to fix it in.
Everything that follows turns on one question: how much does the tester know on day one?
— The comparison
Three names for one variable
How much the tester knows (black, grey or white box) and how much privilege they start with (unauthenticated, standard user, administrative) are two separate scoping decisions. We state both in writing.
None of the three measures whether your team would detect the attack. That is a covert engagement, scoped and priced separately.
Terminology: NIST records these as black box, gray box and white box, defined by how much knowledge the tester has of the internal structure of the system. PCI SSC uses the grey-box spelling and notes that PCI DSS tests are typically performed as white-box or grey-box assessments. The spelling varies; the level of access it describes should not.
— SCOPING
Three decisions get confused with one
01
How much the tester knows
Black-box, grey-box or white-box. This is the only thing those three words describe.
02
Where the tester starts
Outside your perimeter, or already inside it. A white-box test can start from the internet; a black-box test can start on your internal network.
03
Whether your team is told
Overt or covert. Only a covert engagement tells you anything about detection and response.
White-box does not mean “we hand over an administrator account.” How much the tester knows and how much privilege they begin with are two separate lines in the scope. A serious white-box engagement tests from several privilege levels on purpose, precisely to measure escalation and the limits of each role. NIST SP 800-115 treats these as independent choices, and so does a scope of work worth signing — both of ours are written down before anyone starts.
— HOW TO CHOOSE
Which one should you buy?
You only have to recognize yourself in one line.
“What is exposed to the internet right now, without any help from me?”
Black-box
External
“What happens if a credential is stolen, or a supplier is compromised?”
Grey-box
Authenticated
“We built it ourselves, and it holds money or regulated data.”
White-box
On that application
“Would my team notice, and how fast would they respond?”
Not a box at all
Covert exercise, priced separately
Buying your first test and none of those lines settles it? Grey-box, overt, external and internal, authenticated.
It is the approach that turns a fixed budget into the largest number of findings you can actually act on.
PCI DSS
Requirement 11.4 asks for internal and external testing at least every twelve months and after significant change, a documented methodology, remediation of what is exploitable with a retest, and segmentation validation. PCI does not mandate a color of box; the Council's own guidance notes that PCI tests are typically performed as white-box or grey-box assessments. Service providers revalidate segmentation every six months, not every twelve.
SOC 2
The Trust Services Criteria never name penetration testing as a requirement. CC4.1 asks for ongoing and separate evaluations, and a point of focus lists penetration testing among them — in practice, the auditor expects one. Grey-box and authenticated, against the product your customers actually use.
HIPAA
The HIPAA regulation (§164.308(a)(8)) currently requires periodic security evaluations. The proposed rule for 2025 seeks to establish a strict requirement that would include annual penetration testing and semi-annual vulnerability scanning. Specifically, the proposal indicates that these must be "grey-box" tests applied to all systems that touch Electronic Protected Health Information (ePHI).
— THE FOUR QUESTIONS WE ARE ACTUALLY ASKED
Answered, not just raised
Because almost everything worth protecting sits behind the login. The most common serious flaw in modern applications is broken access control — one signed-in user reaching another user's data, or an ordinary account reaching an administrative function — and it only surfaces when a tester can sign in as each role and try to cross the lines between them.
OWASP's method needs at least two accounts to prove that one user cannot reach another's records; PCI's guidance expects application-layer testing to exercise every role and access type. Without credentials, you have bought a test of your login page.
OWASP's method needs at least two accounts to prove that one user cannot reach another's records; PCI's guidance expects application-layer testing to exercise every role and access type. Without credentials, you have bought a test of your login page.
It depends on which question you are paying to answer. If the question is “are these systems sound?”, then yes — allow-list our source addresses, because a WAF or IPS that blocks us also hides the weakness that will still be there on the day that control fails or is bypassed. PCI's guidance is explicit that the test is meant to measure the exposed service, not the filter sitting in front of it.
If the question is “would we notice?”, that is a detection-and-response exercise and should be scoped and priced as one. The middle path is the one PCI's own sample rules of engagement describe: leave logging and alerting fully on, remove only the blocking, for the test window alone, agreed in writing before anyone starts.
If the question is “would we notice?”, that is a detection-and-response exercise and should be scoped and priced as one. The middle path is the one PCI's own sample rules of engagement describe: leave logging and alerting fully on, remove only the blocking, for the test window alone, agreed in writing before anyone starts.
That is a legitimate exercise, and it answers a different question. NIST is direct about covert, adversarial testing: it exists to show the damage an attacker could cause. It does not test every control, does not identify every vulnerability, does not assess every system, and typically takes the most basic way in that works.
It also measures your team's response as much as your technology — and it measures our testers against a calendar a real attacker never has. If you want to know how bad it could get and whether you would see it, commission that deliberately. If you want to know where you are weak so it can be fixed, an informed test covers far more ground in the same hours.
It also measures your team's response as much as your technology — and it measures our testers against a calendar a real attacker never has. If you want to know how bad it could get and whether you would see it, commission that deliberately. If you want to know where you are weak so it can be fixed, an informed test covers far more ground in the same hours.
Only if the objective were to win a game rather than to reduce risk. Information handed to a tester creates no weakness; it removes the guesswork and leaves more of the engagement for proving what is already there. The security-engineering principle of open design has held since 1975: protection “should not depend on the ignorance of potential attackers.”
Real attackers reach the same starting line by other means. In Verizon's 2026 report, exploitation of a known vulnerability is now the most common way in at 31%, and credential abuse appears somewhere in 39% of breaches. OWASP is blunter still, calling testing with no documentation and no source code an assurance activity that “should be actively discouraged” — much as a financial audit with no access to the books would be.
Real attackers reach the same starting line by other means. In Verizon's 2026 report, exploitation of a known vulnerability is now the most common way in at 31%, and credential abuse appears somewhere in 39% of breaches. OWASP is blunter still, calling testing with no documentation and no source code an assurance activity that “should be actively discouraged” — much as a financial audit with no access to the books would be.
— BEFORE ANYTHING STARTS
What we put in writing
The scope
— and what is explicitly excluded.
The approach
— which of the three, and why it fits your question.
What we need from you
— roles, credentials, contacts.
The testing window
and the rules of engagement.
The escalation path
if we find something that cannot wait.
The deliverable
— what the report contains before we write it.
The honest limit
A penetration test establishes what an attacker could do, with a defined level of access, inside a defined scope, over a defined period. It is evidence, not insurance — as the UK's NCSC puts it, a test can only confirm that your systems were not vulnerable to known issues on the day they were tested.
What we commit to is the part that can be committed to: scope agreed in writing before work begins, a documented methodology aligned to NIST SP 800-115 and OWASP, and findings ranked by what can actually be exploited rather than by a scanner's severity label.
A question worth asking every provider. Before you sign, ask them to put in writing: which of the three approaches they are proposing, where the tester starts, whether your team will be told, what is in scope and what is explicitly out — and whether a retest is included.
Nothing starts on a verbal scope.
NIST SP 800-115, Technical Guide to Information Security Testing and Assessment — nvlpubs.nist.gov
NIST CSRC Glossary — black box, gray box, white box testing
PCI SSC, Information Supplement: Penetration Testing Guidance v1.1 — pcisecuritystandards.org. Guidance, not a standard; PCI DSS v4.0.1 Requirement 11.4 is the normative text.
OWASP ASVS 4.0.3, Using the ASVS · OWASP WSTG v4.2, WSTG-ATHZ-04 · OWASP Top 10:2025, A01 Broken Access Control
CREST, A Guide to Penetration Testing (2022) · NCSC UK, Penetration testing and Terminology: it's not black and white
Verizon, 2026 Data Breach Investigations Report · Saltzer & Schroeder (1975), open design · AICPA Trust Services Criteria CC4.1 · HHS OCR, HIPAA Security Rule NPRM, 90 FR (6 January 2025)— NEXT STEP
Not sure which of the three fits?
Tell us what you are protecting and what is driving the timeline. We will put the recommended approach and the scope in writing before you commit to anything.