A secret scanner reported an entire repository history clean. The password was in twenty-six commits, reaching back to the initial one. It was a real password, in use, protecting real infrastructure, and it had been sitting in that history for the life of the repo. The scan was not misconfigured and it did not error. It ran correctly, applied every rule it had, and found nothing, because nothing it knew how to look for was there.
That finding surfaced the way these findings usually surface: somebody searched the history for the literal value, for an unrelated reason, and got hits. The gate had already said clean. This is the most dangerous result a security tool can produce, and it is worth being precise about why — a tool that crashes gets noticed and rerun, while a tool that returns a false clean bill of health ends the investigation, and then gets quoted into a gate, a handoff document, or a client-facing audit report where it hardens into fact.
Key Takeaways
- A secret scanner's zero means no known pattern matched, which is a strictly weaker claim than no secret is present — the correct status for a zero is UNKNOWN, not PASS
- Pattern-and-entropy scanning is structurally blind to secrets used as default arguments (
os.getenv("VAR", "the-literal-value")) because such a value has no prefix, no length signature, and no entropy marker - It is equally blind to human-chosen passwords, for the same reason: a password a person invented looks like an ordinary short string, so no rule fires
- It is blind to credentials for any service it does not ship a rule for, and the rule set is always behind the vendor landscape
- The productive fix for the first class is to match the call shape rather than the value — a getter for a secret-named variable with a non-empty string literal default
- An over-broad check is worse than no check, because it gets suppressed and then catches nothing; ours failed eight correct lines before it was scoped
- Close a zero by searching history for the specific value at risk, and sanity-check every "found nothing" against one input known to be positive
What a Zero Actually Means
Secret scanners work two ways, usually at once. They match regular expressions against known credential shapes — a provider prefix, a fixed length, a checksum-bearing format — and they flag high-entropy strings that look statistically like generated tokens rather than English.
Both techniques are good at what they target. A cloud provider's access key has a distinctive prefix and a fixed length; a random 40-character token has an entropy profile that ordinary source code does not. Scanners catch those reliably, which is exactly why they became a standard pre-commit and CI control, and why we run them as part of a repository audit.
But notice what both techniques require: the secret must look like a secret. The regexes only match shapes somebody wrote a rule for. The entropy heuristic only fires above a threshold, and that threshold has to sit high enough to avoid drowning the output in hashes, base64 blobs, minified assets, and test fixtures. Everything below the threshold and outside the rule set is invisible — not deprioritized, not low-confidence, but absent from the output entirely.
So the honest reading of a zero is narrow: no string in the scanned range matched a shape this tool knows about. That is genuinely useful. It is not the same sentence as "there are no credentials in this history", and the gap between those two sentences is where every miss below lives.
Miss One: Secrets That Live as Default Arguments
The clearest structural blind spot is the fallback default. The pattern looks like this, and it is everywhere:
CRED_PASSWORD = os.getenv("BOSS_CRED_PASSWORD", "the-actual-password")
The intent is reasonable and the author usually knows it is a shortcut. The environment variable is the real mechanism; the literal is there so the thing runs on a laptop without a full environment. Then the shortcut ships, and on the deployed host there is no environment file at all, so the literal is not a fallback — it is the credential the service runs on.
We found exactly this on our own estate: a password protecting an encrypted credential store, present as a plaintext default, and confirmed by hash comparison to be the same password configured across five separate deployments. The scanner was green on it in both modes, history and working tree, the entire time.
It could not have been otherwise. The value was a short string chosen by a person. It has no provider prefix, because it is not a provider token. It has no length signature, because it is not machine-generated. Its entropy sits in the same range as any other short string in the file. To a pattern-and-entropy engine it is indistinguishable from a default timeout, a filename, or a log prefix, and there is no threshold adjustment that separates them without flagging thousands of innocuous literals.
Miss Two: Passwords a Human Chose
The twenty-six-commit case in the opening was the same blindness in a different costume, and it is worth separating because the remediation differs.
That secret was not a default argument. It was a reused human-chosen password, sitting in ordinary configuration and documentation across the life of a repository. Machine-generated credentials have shape. Human-chosen ones deliberately do not — they are memorable, which is the opposite of high-entropy, and they carry no vendor prefix because no vendor issued them.
This class is worse than the first in one specific way: reuse. A generated API key is typically scoped to one service, so its exposure is bounded by that service. A password a person chose is a password that person likely chose elsewhere, and the blast radius of finding it in a git history is not the repository — it is every account that shares it. The audit question after this kind of find is never "which repo", it is "where else did this value go".
Miss Three: Services the Scanner Has No Rule For
The third class needs no theory. A real, working API key for a self-hosted service sat in a file that the scanner called clean, because no rule existed for that service's key format.
Rule sets are curated, and curation follows market share. Major cloud providers, large payment and communication platforms, and popular developer services get rules early. The long tail — self-hosted tools, smaller SaaS vendors, internal services that mint their own tokens — mostly does not, and never will, because the maintainers cannot write a rule for a format they have never seen.
This is not a criticism of the tools. It is a statement about coverage that any honest use of them has to account for: the scanner's rule set is a floor on what it can find, and that floor is always behind the vendor landscape you actually run.
Match the Call Shape, Not the Value
The first class is the one you can actually mechanize, and the trick is to stop looking at the secret.
You cannot write a rule that recognizes an arbitrary password. You can write one that recognizes the syntax that puts a password somewhere it should not be: an environment getter — os.getenv, os.environ.get, and the equivalents in your languages — called with a non-empty string literal as its default, for a variable whose name contains PASSWORD, SECRET, TOKEN, API_KEY, CRED, or PASSPHRASE.
That check does not care what the value is. It fires on the shape of the mistake, which is the part that stays constant. It is a handful of lines, it runs in a repository's own quality gate rather than needing a scanner at all, and in our case it reproduces the original bug as a falsification probe — the check is verified by confirming it still fails against the real historical defect, not merely by confirming it passes today.
One refinement is not optional, and we learned it the expensive way. The first version of that check flagged eight correct lines, because legitimate code defaults secret-adjacent variable names to filenames — a credentials-store path variable whose name contains CRED, defaulting to the store's filename, is entirely correct and must not fire. Exempting path-shaped defaults fixed it.
The general lesson is worth more than the specific exemption. An over-broad check is worse than no check, because it gets suppressed, and a suppressed check catches nothing while still appearing in the gate. A control everyone has learned to ignore is indistinguishable from a control that is not there, except that it is still generating the paperwork that says it is.
Close a Zero With a Content Search
When a scanner returns nothing and the question actually matters, the way to close it is to stop asking the scanner and start asking the history about a specific value.
If you know what you are worried about — a password you are rotating, a key you just discovered on a host, a credential a departing contractor had — search for it directly rather than for its shape. git log -S'<literal>' --all --oneline finds commits that changed the number of occurrences of a string anywhere in history, across every branch. Searching the trees themselves with git grep over git rev-list --all is slower and catches cases the first misses.
Two cautions, both learned by getting them wrong. Quote and brace aggressively in your shell — we have had a shell history-modifier bug silently empty an input list and turn a scan into a confident "zero tainted commits", which is output-identical to a genuinely clean result and cannot be distinguished after the fact. And sanity-check every "found nothing" against one input known to be positive before you believe it. A grep that matches nothing looks exactly like a grep that is broken. Running the same command against a value you know is in the history costs seconds and converts an absence of output into evidence.
Finally, on remediation: finding it in history does not remove it from history. Rewriting history helps, but there are always other clones, forks, backups, and archives. Only rotation actually closes this. Treat the rewrite as tidiness and the rotation as the fix.
The Sentence a Buyer's Counsel Can Hold You To
There is a language rule underneath all of this, and for anyone producing audit output for a client it may be the most valuable part.
Write "the scanner found no known-pattern secrets in this history." Do not write "the history is clean."
The first sentence is true, bounded, and describes what was actually done. The second is a claim about the world that your evidence does not support, and it is the one that gets extracted into an executive summary, quoted in a due-diligence response, and read back to you later by someone with a copy of the history and a search box.
We enforce this mechanically now. The evaluator in our audit tooling used to print a pass with the words "no secrets in git history" — in a client-facing product. It now returns a distinct UNKNOWN status on a scanner zero and says so in the detail line, and our gate contract holds that UNKNOWN is not a pass. A build that cannot determine an answer must not be allowed to report a good one.
That is the same failure shape we wrote about when a DNS blocklist silently broke vendor APIs — a control working exactly as configured, producing a downstream silence that reads as a legitimate result. The check passed; the check was measuring the wrong thing. It is worth naming that pattern explicitly in any environment where automated evidence is trusted, which is precisely what logging and audit controls are supposed to establish.
Honest Limits
None of this argues against running secret scanners. They catch the largest and most dangerous category — machine-generated provider credentials — cheaply and automatically, and a pre-commit hook that stops one live cloud key from ever landing has paid for itself. Run them. The argument is about what their output licenses you to say.
The call-shape check described above addresses one class, in the languages you write it for. It does not generalize to secrets embedded in compiled artifacts, committed binaries, notebook outputs, lockfiles, or the many configuration formats that have no call syntax to match at all.
The content-search remediation only works when you already know the value you are hunting. For the genuinely unknown — a credential nobody remembers creating, for a service nobody remembers wiring up — there is no complete method. Scanners plus shape checks plus targeted searches narrow the space; they do not close it. Any audit that claims otherwise is overstating its instrument.
And the classes above are the ones we have confirmed on our own estate. They are not an exhaustive taxonomy of what pattern-and-entropy scanning misses. They are three we can prove, which is a different and more useful claim than a complete one.
Where This Fits
Secret scanning is a control, and controls need the same treatment as any other production dependency: a known scope, a recorded reason for every suppression, and language in the output that matches what the tool actually established. Most of the value in a repository credential audit is not in running the scanner — it is in knowing which of its zeros mean something.
The build-and-verify side of this, where a gate has to return an honest verdict and an unknown must never become a pass, is infrastructure work. The evidence side, where a silently failing check stops being an inconvenience and becomes an audit finding, is cybersecurity.
If your repositories have been scanned clean and nobody has ever tested that result against a value known to be in the history, book a call and we will run the audit properly — including the part where a zero gets closed rather than believed.