Oregami
Repositories/oxedyne/fe2o3

oxedyne/fe2o3/fe2o3_text/src/secret.rs

27.5 KiB, 71 runs

created by r1870400018:35193, which is this file's identity for as long as the history lasts, whatever it is later renamed to

download · who wrote it · its history

1//! Credential shapes in text, so that a machine can refuse one before it is written down.
2//!
3//! The shapes, the placeholder excuses, the skipped paths and the `allowlist secret` marker are
4//! those of the global git `pre-commit` hook at `~/usr/code/bash/githooks/pre-commit`, written on
5//! 2026-07-10 after a live API key was put in an example as a fallback default, pushed to a public
6//! repository, and used by a stranger nine days later. Three people read that file inside those
7//! nine days and each scrubbed a different copy of it. What was recorded there is that a
8//! credential has to be stopped by a machine, and what this module adds is that git is not the
9//! only thing a person writes history with: one marker and one set of shapes have to serve every
10//! tool, or a fixture marked for one is refused by the other.
11//!
12//! # Bytes, not text
13//!
14//! A file is scanned as the bytes it holds and never decoded. A source file carrying one invalid
15//! UTF-8 byte in a comment is exactly where an unnoticed key would sit, and a scanner that decoded
16//! first would pass over the file entirely.
17//!
18//! # Key material with no text shape
19//!
20//! A private key written as raw DER is a credential that no run of characters describes: it has no
21//! armour, no vendor prefix and no field name beside it, and it holds NULs, so the binary skip
22//! below was passing over precisely the thing this module exists to stop. It is caught instead by
23//! the fixed bytes the encoding itself puts in front of one -- an algorithm's object identifier,
24//! which is as literal as the PEM header above it and is a heuristic in no sense at all. Written
25//! on 2026-08-23, after a live DKIM signing key spent four months at mode 644 in a replicated
26//! folder and nothing here could have seen it.
27//!
28//! The bytes were read off keys generated for the purpose and are stated in [`DER_ALGOS`]; none of
29//! them came from a file in anybody's tree, and nothing in this module was tuned against one. That
30//! matters to the next reader, who will otherwise assume the opposite and be right to distrust the
31//! result.
32//!
33//! # Why a key is looked for at every offset, and only in a small file
34//!
35//! Ring 0.17.8 holds the head of a PKCS#8 key as a `const` template -- the outer `SEQUENCE`, the
36//! version, the algorithm's object identifier and the tag that opens the private bytes -- and a
37//! compiler puts that template in the read-only data of whatever links it. Those bytes are the
38//! structure of a private key because that is what a template of one is, so [`der_key`] says so,
39//! and it is right: there is no test that separates a template from a key which is not a guess
40//! about the bytes standing after it. A sweep of every file under this tree on 2026-08-23 --
41//! 623,722 files, 202 GB, nothing skipped -- found the structure at 1,729 offsets in 303 files,
42//! and every one of the 303 was a compiled artefact carrying that template: executables, `.rlib`,
43//! `.rmeta`, `.o`, WebAssembly modules, and one capture of a tree holding an executable. Not one
44//! was a key.
45//!
46//! So a scan of every offset in a compiled artefact refuses it, and a guard that refuses an
47//! ordinary build output is a guard somebody turns off. [`DER_SPAN`] is what stops that, and it is
48//! a size and nothing cleverer: the whole of a file is read at every offset while the file is
49//! small enough to be a key and the things a key is bundled with, and above that only its front is
50//! read, which is where the rule stood until this was written. The smallest artefact in that sweep
51//! was 101,960 bytes, three times the span.
52//!
53//! # Why the shapes are matched by hand
54//!
55//! [`crate::regex`] would say these patterns in one line each, and is not used for two reasons: it
56//! matches over `str` where this works over bytes, and every shape here is a literal opening
57//! followed by a run of one character class, which one pass along the line decides. The
58//! [`interesting`] prefilter is what makes that pass cheap -- a byte that opens no shape is
59//! rejected on a handful of comparisons -- and [`leads_are_covered`] is the test that keeps the
60//! prefilter honest as shapes are added.
61
62// Fewest bytes an assigned literal must hold before it is worth suspecting, how far into a file
63// the scan looks for a NUL before calling it a binary, and the marker that excuses a line, spelled
64// as a caller should tell a person to spell it.
65pub const MIN_LITERAL: usize = 20;
66const BINARY_HEAD: usize = 8000;
67pub const MARKER: &str = "allowlist secret";
68
69// Lockfiles, which carry long hashes that read like keys.
70const LOCKFILES: &[&str] = &[
71 "Cargo.lock",
72 "package-lock.json",
73 "yarn.lock",
74 "pnpm-lock.yaml",
75 "go.sum",
76];
77
78// Directories holding somebody else's code, or a build's output, and the one directory name that
79// says a person wrote whatever is under it. A name on the vendored list skips only while no `src`
80// stands above it: every convention that put a name there -- a bundler's `dist`, a package
81// manager's `node_modules`, cargo's `target` -- writes its directory beside a source tree and never
82// inside one, so a `dist` below a `src` is hand written by construction. Matching the bare name at
83// any depth read fourteen hand-written Rust files under one `src/dist/` as build output and
84// exempted them from this guard and from the git hook, which is a hole rather than a saving. It was
85// found on 2026-08-21, when the tree holding them was put under a version control system that
86// cannot forget what it captures.
87const VENDORED: &[&str] = &[
88 "node_modules",
89 "target",
90 "vendor",
91 ".venv",
92 "dist",
93 "build",
94];
95const SOURCE: &str = "src";
96
97// The two halves of a PEM private key header, which names its algorithm in the middle. Held apart
98// so that this file does not itself carry the header a scanner looks for, its own included.
99const PEM_ALGOS: &[&str] = &["", "RSA ", "EC ", "DSA ", "OPENSSH ", "PGP "];
100const PEM_KEY: &str = "PRIVATE KEY";
101
102// Widest a DER key may declare itself and still be looked at, and narrowest a file can be to hold
103// one at all. The ceiling is on the length the SEQUENCE declares rather than on the file, because
104// what stands after a key is not the key, and gating on the file was what let a key with one byte
105// appended through. No private key comes near 8000 bytes: an RSA-8192 key in PKCS#8 is about
106// 4.7 kB and everything else on this list is under 2.4 kB. The floor is below the smallest key, an
107// ed25519 at 48 bytes.
108const DER_MAX: usize = 8000;
109const DER_MIN: usize = 32;
110
111/// Widest a file can be and still be read at every offset rather than only at its front.
112///
113/// Four times the widest a key may declare itself, so that a key and the certificate chain it is
114/// bundled with are inside it several times over, and far below any compiled artefact: see this
115/// module's header for the sweep that says so and for why a size is what stands here.
116pub const DER_SPAN: usize = 4 * DER_MAX;
117
118/// The `AlgorithmIdentifier` that stands after the version in a PKCS#8 private key, one per
119/// algorithm, each an object identifier the encoding fixes and nobody chooses.
120///
121/// Every sequence here was read off the front of a key generated for the purpose -- `openssl
122/// genpkey -outform DER` piped through `openssl pkcs8 -topk8`, on 2026-08-23 -- and off no file in
123/// any tree. There is nothing to tune and nothing that was tuned: a file opening with one of these
124/// is a private key of that algorithm, and the question has no second answer.
125pub const DER_ALGOS: &[&[u8]] = &[
126 &[0x30, 0x05, 0x06, 0x03, 0x2b, 0x65, 0x70], // ed25519
127 &[0x30, 0x05, 0x06, 0x03, 0x2b, 0x65, 0x6e], // X25519
128 &[0x30, 0x0d, 0x06, 0x09, 0x2a, 0x86, 0x48, 0x86, 0xf7, 0x0d, 0x01,
129 0x01, 0x01, 0x05, 0x00], // RSA
130 &[0x30, 0x13, 0x06, 0x07, 0x2a, 0x86, 0x48, 0xce, 0x3d, 0x02, 0x01,
131 0x06, 0x08, 0x2a, 0x86, 0x48, 0xce, 0x3d, 0x03, 0x01, 0x07], // ECDSA, P-256
132 &[0x30, 0x10, 0x06, 0x07, 0x2a, 0x86, 0x48, 0xce, 0x3d, 0x02, 0x01,
133 0x06, 0x05, 0x2b, 0x81, 0x04, 0x00, 0x22], // ECDSA, P-384
134 &[0x30, 0x10, 0x06, 0x07, 0x2a, 0x86, 0x48, 0xce, 0x3d, 0x02, 0x01,
135 0x06, 0x05, 0x2b, 0x81, 0x04, 0x00, 0x23], // ECDSA, P-521
136];
137
138// Widths of the private scalar of the curves whose keys `openssl ecparam -genkey` writes in the
139// older SEC1 form, which names no algorithm and is what this machine's openssl produces by
140// default. Read off one key per curve, the same day and the same way. P-256, P-384, P-521.
141const DER_SCALARS: &[u8] = &[0x20, 0x30, 0x42];
142
143// Field names that say outright what the value beside them is.
144const FIELDS: &[&str] = &[
145 "api_key",
146 "api-key",
147 "apikey",
148 "secret",
149 "passwd",
150 "password",
151 "auth_token",
152 "auth-token",
153 "authtoken",
154 "access_token",
155 "access-token",
156 "accesstoken",
157];
158
159// Openings of a value nobody has filled in yet. Matched at the start of the literal, with anything
160// after them, so `your-key-here` and `example_token_1` are both excused.
161const PLACEHOLDERS: &[&str] = &[
162 "your", "my", "the", "some", "a", "an", "test", "dummy", "fake", "example", "sample",
163 "placeholder", "changeme", "redacted", "insert", "replace", "todo", "fixme", "none", "null",
164 "empty", "abc", "foo", "bar", "baz", "secret", "password", "token", "key",
165];
166
167
168/// What was found, which is what a refusal names.
169///
170/// Every variant bar [`Kind::Assigned`] is a shape that is a credential and essentially nothing
171/// else; `Assigned` is a named field holding a long literal, which is noisier and is why
172/// placeholders are excused from it.
173#[derive(Clone, Copy, Debug, Eq, PartialEq)]
174pub enum Kind {
175 Fireworks, // fw_
176 Anthropic, // sk-ant-
177 OpenAi, // sk-proj-, sk-or-v1-
178 OpenAiOld, // sk- and a long run, the shape before the prefixes
179 Aws, // AKIA
180 GitHub, // ghp_, gho_, ghu_, ghs_, ghr_
181 GitHubPat, // github_pat_
182 Slack, // xoxb-, xoxa-, xoxp-, xoxr-, xoxs-
183 Stripe, // sk_live_, rk_live_
184 Google, // AIza
185 PrivateKey, // a PEM private key block
186 DerKey, // a private key written as DER, at any offset in a small file
187 Assigned, // a named secret field holding a long literal
188}
189
190impl Kind {
191 /// What to call it in a message to a person.
192 pub fn label(&self) -> &'static str {
193 match self {
194 Self::Fireworks => "Fireworks key",
195 Self::Anthropic => "Anthropic key",
196 Self::OpenAi => "OpenAI or OpenRouter key",
197 Self::OpenAiOld => "OpenAI key, older shape",
198 Self::Aws => "AWS access key",
199 Self::GitHub => "GitHub token",
200 Self::GitHubPat => "GitHub personal access token",
201 Self::Slack => "Slack token",
202 Self::Stripe => "Stripe live secret key",
203 Self::Google => "Google API key",
204 Self::PrivateKey => "private key block",
205 Self::DerKey => "private key in DER form",
206 Self::Assigned => "assigned secret literal",
207 }
208 }
209}
210
211/// One credential, at the line of the scanned bytes that holds it.
212///
213/// The value itself is deliberately absent: a caller reports the position and the shape, and
214/// whoever reads the report opens the file. Putting the value in a message copies it into a
215/// terminal's scrollback, a log and a bug report.
216#[derive(Clone, Copy, Debug, Eq, PartialEq)]
217pub struct Find {
218 pub line: usize, // 1-based, counting line feeds
219 pub kind: Kind,
220}
221
222/// What a run of bytes after a literal opening may hold.
223#[derive(Clone, Copy, Debug)]
224enum Set {
225 Alnum, // [A-Za-z0-9]
226 Token, // [A-Za-z0-9_-]
227 Upper, // [0-9A-Z]
228 Word, // [A-Za-z0-9_]
229 Pem, // an algorithm name, then the words that matter
230}
231
232impl Set {
233 fn admits(&self, b: u8) -> bool {
234 let alnum = b.is_ascii_alphanumeric();
235 match self {
236 Self::Alnum => alnum,
237 Self::Token => alnum || b == b'_' || b == b'-',
238 Self::Upper => b.is_ascii_digit() || b.is_ascii_uppercase(),
239 Self::Word => alnum || b == b'_',
240 Self::Pem => false,
241 }
242 }
243}
244
245/// A literal opening and the run that must follow it.
246struct Shape {
247 kind: Kind,
248 lead: &'static [u8], // matched exactly, and case sensitively
249 set: Set, // what the run after the opening admits
250 min: usize, // fewest bytes that run must hold
251}
252
253impl Shape {
254 /// Does the shape stand at the start of these bytes?
255 fn at(&self, from: &[u8]) -> bool {
256 if !from.starts_with(self.lead) {
257 return false;
258 }
259 let tail = &from[self.lead.len()..];
260 if let Set::Pem = self.set {
261 return PEM_ALGOS.iter().any(|algo|
262 tail.starts_with(algo.as_bytes())
263 && tail[algo.len()..].starts_with(PEM_KEY.as_bytes()));
264 }
265 let mut n = 0;
266 while n < tail.len() && self.set.admits(tail[n]) {
267 n += 1;
268 }
269 n >= self.min
270 }
271}
272
273// Every shape a hit on which is a refusal, ordered by opening byte so that [`shapes_for`] can
274// answer with one contiguous run. The ranges there are what makes the order load bearing.
275const SHAPES: &[Shape] = &[
276 Shape { kind: Kind::PrivateKey, lead: b"-----BEGIN ", set: Set::Pem, min: 0 },
277 Shape { kind: Kind::Aws, lead: b"AKIA", set: Set::Upper, min: 16 },
278 Shape { kind: Kind::Google, lead: b"AIza", set: Set::Token, min: 35 },
279 Shape { kind: Kind::Fireworks, lead: b"fw_", set: Set::Alnum, min: 20 },
280 Shape { kind: Kind::GitHub, lead: b"ghp_", set: Set::Alnum, min: 36 },
281 Shape { kind: Kind::GitHub, lead: b"gho_", set: Set::Alnum, min: 36 },
282 Shape { kind: Kind::GitHub, lead: b"ghu_", set: Set::Alnum, min: 36 },
283 Shape { kind: Kind::GitHub, lead: b"ghs_", set: Set::Alnum, min: 36 },
284 Shape { kind: Kind::GitHub, lead: b"ghr_", set: Set::Alnum, min: 36 },
285 Shape { kind: Kind::GitHubPat, lead: b"github_pat_", set: Set::Word, min: 40 },
286 Shape { kind: Kind::Stripe, lead: b"rk_live_", set: Set::Alnum, min: 20 },
287 Shape { kind: Kind::Anthropic, lead: b"sk-ant-", set: Set::Token, min: 20 },
288 Shape { kind: Kind::OpenAi, lead: b"sk-proj-", set: Set::Token, min: 20 },
289 Shape { kind: Kind::OpenAi, lead: b"sk-or-v1-", set: Set::Token, min: 20 },
290 Shape { kind: Kind::OpenAiOld, lead: b"sk-", set: Set::Alnum, min: 32 },
291 Shape { kind: Kind::Stripe, lead: b"sk_live_", set: Set::Alnum, min: 20 },
292 Shape { kind: Kind::Slack, lead: b"xoxb-", set: Set::Token, min: 10 },
293 Shape { kind: Kind::Slack, lead: b"xoxa-", set: Set::Token, min: 10 },
294 Shape { kind: Kind::Slack, lead: b"xoxp-", set: Set::Token, min: 10 },
295 Shape { kind: Kind::Slack, lead: b"xoxr-", set: Set::Token, min: 10 },
296 Shape { kind: Kind::Slack, lead: b"xoxs-", set: Set::Token, min: 10 },
297];
298
299/// The shapes that can open with a byte.
300///
301/// Every position of every line asks this, and most of them are answered with nothing, which is
302/// what keeps a scan to about a comparison a byte. [`leads_are_covered`] is what stops a shape
303/// added to the table above from falling outside the ranges and reading as live while matching
304/// nothing.
305fn shapes_for(b: u8) -> &'static [Shape] {
306 match b {
307 b'-' => &SHAPES[0..1],
308 b'A' => &SHAPES[1..3],
309 b'f' => &SHAPES[3..4],
310 b'g' => &SHAPES[4..10],
311 b'r' => &SHAPES[10..11],
312 b's' => &SHAPES[11..16],
313 b'x' => &SHAPES[16..21],
314 _ => &[],
315 }
316}
317
318/// Could a byte open a named field, in either case?
319fn field_lead(b: u8) -> bool {
320 matches!(b, b'a' | b'A' | b's' | b'S' | b'p' | b'P')
321}
322
323
324/// Every credential in these bytes, in the order the lines hold them.
325///
326/// Bytes holding a NUL near their start are taken for a binary and scanned no further: a
327/// compiled artefact matches these shapes by chance often enough to make a scanner nobody
328/// believes, and a credential compiled into a binary was in a source file first.
329pub fn scan(data: &[u8]) -> Vec<Find> {
330 let mut out = Vec::new();
331 // Asked before the binary skip, because a key in DER form is exactly what that skip passes
332 // over: NULs at the front, no text anywhere, and nothing the line walk below can see. It is a
333 // property of the bytes rather than of a line, so it answers on its own and stops here,
334 // whatever else stands around the key.
335 if let Some(at) = der_key(data) {
336 out.push(Find { line: line_at(data, at), kind: Kind::DerKey });
337 return out;
338 }
339 let head = data.len().min(BINARY_HEAD);
340 if data[..head].contains(&0) {
341 return out;
342 }
343 let mut kinds = Vec::new();
344 let mut prev: &[u8] = b"";
345 for (i, line) in data.split(|b| *b == b'\n').enumerate() {
346 // The line above excuses this one, so that a marker can sit in a comment over the line it
347 // speaks for rather than trailing off the end of it.
348 if !excused(line) && !excused(prev) {
349 kinds.clear();
350 kinds_at(line, &mut kinds);
351 for kind in &kinds {
352 out.push(Find { line: i + 1, kind: *kind });
353 }
354 }
355 prev = line;
356 }
357 out
358}
359
360/// Paths that are a secret by name, as ignore rules in git's glob syntax, one per line.
361///
362/// The other half of this module. [`scan`] reads bytes and refuses a credential it can recognise;
363/// this names the files a credential conventionally lives in, whatever their bytes say, so that a
364/// tool writing a history can keep them out before it has read a byte of them. A `.env` holding
365/// `DB_PASSWORD=hunter2` has no shape [`scan`] answers to, and a PEM certificate is not a secret at
366/// all but stands beside the key that is, so a rule by name is what stops both.
367///
368/// The syntax is a `.gitignore`'s, so that a tool which already compiles one can compile this by
369/// prepending it: a repository's own rules then come last and win, and a `!` line in them
370/// re-includes anything here by name. The one re-inclusion this list makes itself is the example
371/// file every `.env` convention ships beside the real one, which holds placeholders by definition
372/// and is the file a reader needs most.
373///
374/// This is not compiled here, because the glob machinery lives downstream of this crate; the test
375/// that every line is a rule the matcher accepts, and that it decides what this comment says it
376/// does, is beside that machinery.
377pub const SECRET_PATHS: &[&str] = &[
378 // Environment files, which hold credentials by convention and nothing by shape.
379 ".env",
380 ".env.*",
381 "!.env.example",
382 "!.env.sample",
383 "!.env.template",
384 // Key material by extension, and the certificate that conventionally stands beside it.
385 "*.pem",
386 "*.key",
387 "*.p12",
388 "*.pfx",
389 "*.jks",
390 "*.keystore",
391 // The names every SSH client writes a private key under.
392 "id_rsa",
393 "id_dsa",
394 "id_ecdsa",
395 "id_ed25519",
396 // Machine credentials for other services, kept in the home directory by convention and
397 // copied into a project by mistake.
398 ".netrc",
399 ".pgpass",
400 ".htpasswd",
401 // Directories whose name says what they hold.
402 "keys/",
403 "tls/",
404];
405
406/// Is the path one whose long hashes read like keys, and which is therefore not scanned?
407///
408/// A lockfile by name, or anything under a vendored or built directory that no `src` stands above.
409/// The path is relative to the root of whatever is being scanned, with `/` between its components.
410pub fn skip_path(path: &[u8]) -> bool {
411 let mut last: &[u8] = b"";
412 let mut dirs = 0;
413 let mut sourced = false;
414 for comp in path.split(|b| *b == b'/') {
415 // Something follows `last`, so `last` is a directory rather than the file at the end.
416 if dirs > 0 {
417 if last == SOURCE.as_bytes() {
418 sourced = true;
419 }
420 // A source tree inside a vendored one is still somebody else's, so the first of the two
421 // names to appear is the one that decides.
422 if !sourced && VENDORED.iter().any(|v| v.as_bytes() == last) {
423 return true;
424 }
425 }
426 last = comp;
427 dirs += 1;
428 }
429 LOCKFILES.iter().any(|f| f.as_bytes() == last)
430}
431
432/// Where a private key, written as DER and left unarmoured, stands in these bytes, if one does.
433///
434/// The front of the input is asked whatever its size, and every offset in it as well while it is
435/// no wider than [`DER_SPAN`]. Until 2026-08-23 only the front was asked, and `cat cert.der
436/// key.der` -- a bundle nobody has to tamper with to produce, and one `openssl pkey -inform DER`
437/// reads the private key straight out of and signs with -- went free because the certificate stood
438/// first.
439///
440/// There is no marker that excuses a finding here, and there cannot be: this reads the key's own
441/// structure and nothing around it, so there is nowhere to write one that it would look at. A test
442/// that needs a key should generate one, which is what this crate's own suite does.
443fn der_key(data: &[u8]) -> Option<usize> {
444 if der_key_at(data, 0) {
445 return Some(0);
446 }
447 if data.len() > DER_SPAN {
448 return None;
449 }
450 // The version INTEGER is the one thing every form below has in common, and it stands at a fixed
451 // distance into the SEQUENCE, whose header is one, two or three bytes wide. So a candidate
452 // opening is at one of three known distances back from a version, and a walk looking for the
453 // version rather than for the SEQUENCE tag asks the full test 210 times less often: three bytes
454 // of a compiled artefact answer where one does not. What the two find is the same set.
455 let last = data.len().saturating_sub(2);
456 for v in 0..last {
457 if data[v] != 0x02 || data[v + 1] != 0x01 || (data[v + 2] != 0x00 && data[v + 2] != 0x01) {
458 continue;
459 }
460 for hdr in 1..=3 {
461 if v >= 1 + hdr && der_key_at(data, v - 1 - hdr) {
462 return Some(v - 1 - hdr);
463 }
464 }
465 }
466 None
467}
468
469/// Does one stand at this offset?
470///
471/// The three questions are the encoding's own, and each of them has one answer. The outer
472/// `SEQUENCE` declares its own length and that is where the key ends: everything asked below is
473/// asked of those bytes, and whatever stands after them is no part of the key and is not looked
474/// at. The version `INTEGER` is 0 for PKCS#8 and PKCS#1 and 1 for the `OneAsymmetricKey` form that
475/// carries the public key too, which is the 83-byte shape `ring` writes and the shape the DKIM key
476/// was in. What follows the version is then an algorithm's object identifier from [`DER_ALGOS`],
477/// or the modulus of a PKCS#1 RSA key, or the private scalar of a SEC1 elliptic curve key.
478fn der_key_at(data: &[u8], at: usize) -> bool {
479 if data.len() - at < DER_MIN || data[at] != 0x30 {
480 return false;
481 }
482 let (len, hdr) = match der_len(&data[at + 1..]) {
483 Some(v) => v,
484 None => return false,
485 };
486 let whole = 1 + hdr + len;
487 // The SEQUENCE has to be all there, and the ceiling is asked of the length it declares rather
488 // than of what is left of the file.
489 if whole > data.len() - at || whole > DER_MAX {
490 return false;
491 }
492 // Bounded by the declared length, so that a SEQUENCE too short to hold one of the shapes below
493 // cannot borrow the bytes standing after it to finish the match.
494 let body = &data[at + 1 + hdr..at + whole];
495 let after = if body.starts_with(&[0x02, 0x01, 0x00]) {
496 &body[3..]
497 } else if body.starts_with(&[0x02, 0x01, 0x01]) {
498 // The version says the public key follows the private one, so a SEC1 scalar can stand here
499 // as well as a PKCS#8 algorithm.
500 let after = &body[3..];
501 if DER_SCALARS.iter().any(|w| after.starts_with(&[0x04, *w])) {
502 return true;
503 }
504 after
505 } else {
506 return false;
507 };
508 if DER_ALGOS.iter().any(|a| after.starts_with(a)) {
509 return true;
510 }
511 // PKCS#1, which names no algorithm: what follows the version is the modulus, an INTEGER whose
512 // length is written long form because no key worth having has one under 128 bytes.
513 after.starts_with(&[0x02, 0x81]) || after.starts_with(&[0x02, 0x82])
514}
515
516/// The line an offset falls on, counting line feeds, so that a key written into a text file is
517/// reported where a person will find it. A key at the front of a file is line 1, which is where
518/// every one of them was reported before offsets were looked at.
519fn line_at(data: &[u8], at: usize) -> usize {
520 1 + data[..at].iter().filter(|b| **b == b'\n').count()
521}
522
523/// The length a DER header declares, and the bytes that header took, or nothing where the form is
524/// one no private key is written in.
525fn der_len(from: &[u8]) -> Option<(usize, usize)> {
526 match from.first() {
527 Some(n) if *n < 0x80 => Some((*n as usize, 1)),
528 Some(&0x81) => from.get(1).map(|n| (*n as usize, 2)),
529 Some(&0x82) => match (from.get(1), from.get(2)) {
530 (Some(hi), Some(lo)) => Some((((*hi as usize) << 8) | *lo as usize, 3)),
531 _ => None,
532 },
533 _ => None,
534 }
535}
536
537/// Does the line carry the marker that excuses it?
538///
539/// Two spellings are taken, `allowlist secret` in any case and with a space, an underscore or a
540/// hyphen between the words, and `pragma: allowlist` as the detect-secrets convention spells it.
541pub fn excused(line: &[u8]) -> bool {
542 for at in 0..line.len() {
543 let from = &line[at..];
544 if starts_ci(from, b"allowlist") {
545 let rest = &from["allowlist".len()..];
546 match rest.first() {
547 Some(b' ') | Some(b'_') | Some(b'-')
548 if starts_ci(&rest[1..], b"secret") => return true,
549 _ => (),
550 }
551 }
552 if starts_ci(from, b"pragma:") && starts_ci(blank(&from["pragma:".len()..]), b"allowlist") {
553 return true;
554 }
555 }
556 false
557}
558
559/// Could a byte open any shape, or any named field?
560///
561/// The prefilter the line walk turns on, derived from the table rather than written out beside
562/// it, so that the two cannot drift apart.
563pub fn interesting(b: u8) -> bool {
564 !shapes_for(b).is_empty() || field_lead(b)
565}
566
567/// Does the prefilter admit the opening byte of every shape and every field name?
568///
569/// A shape added with an opening the prefilter rejects would be dead code that reads as live, and
570/// nothing else in the module would notice. Exposed so that a caller's test suite can hold the
571/// same line as this crate's.
572pub fn leads_are_covered() -> bool {
573 for shape in SHAPES {
574 let lead = match shape.lead.first() {
575 Some(b) => *b,
576 None => return false,
577 };
578 if !shapes_for(lead).iter().any(|s| s.lead == shape.lead && s.kind == shape.kind) {
579 return false;
580 }
581 }
582 for name in FIELDS {
583 let lead = match name.as_bytes().first() {
584 Some(b) => *b,
585 None => return false,
586 };
587 if !field_lead(lead) || !field_lead(lead.to_ascii_uppercase()) {
588 return false;
589 }
590 }
591 true
592}
593
594/// Puts every kind standing anywhere in the line into `out`, once each.
595fn kinds_at(line: &[u8], out: &mut Vec<Kind>) {
596 for at in 0..line.len() {
597 let from = &line[at..];
598 for shape in shapes_for(line[at]) {
599 if shape.at(from) && !out.contains(&shape.kind) {
600 out.push(shape.kind);
601 }
602 }
603 // Only a field name's own opening is worth the walk along the twelve of them.
604 if field_lead(line[at]) && !out.contains(&Kind::Assigned) && assigned(from) {
605 out.push(Kind::Assigned);
606 }
607 }
608}
609
610/// Does a named secret field stand here, holding a quoted literal that is long and is not a
611/// placeholder?
612fn assigned(from: &[u8]) -> bool {
613 for name in FIELDS {
614 if starts_ci(from, name.as_bytes()) && literal(&from[name.len()..]) {
615 return true;
616 }
617 }
618 false
619}
620
621/// Does what follows a field name amount to it being given a long literal?
622fn literal(after: &[u8]) -> bool {
623 let rest = blank(after);
624 match rest.first() {
625 Some(b':') | Some(b'=') => (),
626 _ => return false,
627 }
628 let rest = blank(&rest[1..]);
629 match rest.first() {
630 Some(b'"') | Some(b'\'') => (),
631 _ => return false,
632 }
633 let rest = &rest[1..];
634 let mut n = 0;
635 while n < rest.len() && Set::Token.admits(rest[n]) {
636 n += 1;
637 }
638 if n < MIN_LITERAL {
639 return false;
640 }
641 // The run has to end where the quote does, or what stands there is not one literal.
642 match rest.get(n) {
643 Some(b'"') | Some(b'\'') => (),
644 _ => return false,
645 }
646 !placeholder(&rest[..n])
647}
648
649/// Is the literal one nobody has filled in?
650fn placeholder(value: &[u8]) -> bool {
651 if value.len() >= 4 && value[..4].iter().all(|b| b.eq_ignore_ascii_case(&b'x')) {
652 return true;
653 }
654 // `a` is one of the words, so any literal opening with an `a` is excused. That is the git
655 // hook's behaviour and is kept deliberately: one convention across the two tools is worth
656 // more than a marginally tighter rule on the noisier of the two classes, and a real key of
657 // any issued shape is caught above regardless of what it opens with.
658 PLACEHOLDERS.iter().any(|w| starts_ci(value, w.as_bytes()))
659}
660
661/// Does the haystack open with the needle, ignoring ASCII case?
662fn starts_ci(hay: &[u8], needle: &[u8]) -> bool {
663 hay.len() >= needle.len()
664 && hay[..needle.len()].eq_ignore_ascii_case(needle)
665}
666
667/// Drops leading spaces and tabs.
668fn blank(from: &[u8]) -> &[u8] {
669 let mut n = 0;
670 while n < from.len() && (from[n] == b' ' || from[n] == b'\t') {
671 n += 1;
672 }
673 &from[n..]
674}