Oregami
Repositories/oxedyne/daimond

oxedyne/daimond/hand/src/verify.rs

102 KiB, 1 run

created by r2519314175:927, which is this file's identity for as long as the history lasts, whatever it is later renamed to

download · who wrote it · its history

1//! Running a named verifier from the tracked tree, and refusing to report a bare pass.
2//!
3//! Written on 2026-08-21. A daimon cannot produce browser evidence: `listen()`
4//! is refused by [`crate::seccomp`], a browser needs the display server's unix
5//! socket and [`crate::seccomp::Unix::Refuse`] takes that away, so every
6//! `dev/verify_*.mjs` that drives a real page dies under the fence a command
7//! gets. Half the proof of a release session is those scripts, so the machine
8//! that could write the code could not check it.
9//!
10//! **The reframe this module is built on is about provenance, not sandboxing.**
11//! The fence exists to contain a command a MODEL wrote. A verifier is the
12//! repository's own code, and the model supplies no part of it: it supplies a
13//! *name* that is looked up in the directory, and at most a *break* that is
14//! looked up in the file's own declarations. So the verb is in the same trust
15//! class as `cargo test`, and it runs the script outside the command fence
16//! deliberately -- which is stated in the report, stated in the tool's
17//! description, and recorded in the journal as `fence:none` so that nobody has
18//! to take this paragraph's word for it.
19//!
20//! # And the reframe has to be enforced, not reported
21//!
22//! Until 2026-08-25 it was not. `REVIEW.md` §1.21: the trustworthiness of a
23//! verifier was measured -- `git ls-files --error-unmatch` -- and its only use
24//! was a phrase in the report's first line. A daimon writes `dev/verify_x.mjs`
25//! with `file_write`, which every turn may do because the granted root is
26//! writable, and the verb runs it with no fence on it at all. Proved with a
27//! script whose whole body wrote a file outside the granted root, which appeared.
28//!
29//! **The question is content, not membership.** Appending one comment to a
30//! tracked verifier leaves `git ls-files --error-unmatch` exiting 0 while
31//! `git diff --quiet HEAD` on the same file exits 1, so enforcing what was
32//! already measured would have narrowed the hole without closing it.
33//! [`provenance`] therefore asks whether the working file IS the commit, and
34//! [`unfenced_refusal`] is the gate; [`conduct`] passes it before anything runs
35//! and [`once`] before every spawn.
36//!
37//! **The refusal is a capability difference and not a dead end**, which matters
38//! because improving a verifier is exactly the self-development this app is for.
39//! A verifier the daimon has just written still runs -- through `run`, inside
40//! the same fence every command gets, which 253 of this tree's 282 verifiers
41//! need nothing more than. What it cannot do fenced is drive a browser or open
42//! a port, and the refusal says so, so a fenced failure is not misread as the
43//! script being broken.
44//!
45//! **What this does not close, and cannot from here.** `.git` is inside the
46//! granted root and every fence leaves it writable -- `src/tools.rs` says so in
47//! its own words where it explains why a push injects `core.hooksPath` -- so a
48//! daimon can commit its own verifier and then be told, truthfully, that the
49//! bytes are the commit's. That is one `run` call away and no check on this
50//! side reaches it. What changes is that the act is now a commit in the
51//! repository's history rather than a `file_write` nobody sees, and that the
52//! report's first line stops being false. Closing it needs either `.git` out
53//! of the fence or the owner's consent bound to the bytes; both are the owner's
54//! to rule on, and the reasoning is in `REVIEW.md` §1.21.
55//!
56//! # The one thing this module exists to make impossible
57//!
58//! Running a verifier clean and reporting "27 checks passed" is exactly the
59//! evidence that has been lying. On 2026-08-20 three separate instruments
60//! passed while aimed at nothing: a `--break` whose output was byte-identical to
61//! a clean run, a leg count blind to a 1,856-request runaway, and a check
62//! asserting a destination a re-route also reaches. All three were caught
63//! because a person demanded a break and the agents chose to report honestly.
64//!
65//! Honesty must not be load-bearing. So the verb runs the clean pass AND each
66//! declared break, and [`Verdict`] is an enum whose every arm carries the count
67//! of breaks that reddened nothing. There is no accessor for the passed count
68//! on its own: a caller who wants it must match, and both arms hand back the
69//! bad news in the same breath.
70
71use crate::journal::{
72 Event,
73 Journal,
74};
75use crate::wire::{
76 Capture,
77 FenceSpec,
78 Req,
79 Resp,
80 Stream,
81 CHUNK_MAX,
82};
83
84use oxedyne_fe2o3_core::prelude::*;
85
86use std::{
87 collections::BTreeMap,
88 path::{
89 Path,
90 PathBuf,
91 },
92 process::Stdio,
93 sync::{
94 atomic::{
95 AtomicBool,
96 Ordering,
97 },
98 Arc,
99 Mutex,
100 },
101 time::Duration,
102};
103
104use tokio::{
105 io::AsyncReadExt,
106 process::Command,
107 sync::mpsc::Sender,
108};
109
110// ┌───────────────────────────────────────────────────────────────┐
111// │ Where a verifier lives, and what it may be called │
112// └───────────────────────────────────────────────────────────────┘
113
114/// The one directory under the granted root a verifier may be read from.
115pub const DEV_DIR: &str = "dev";
116
117/// The one file-name shape a verifier may have.
118pub const PREFIX: &str = "verify_";
119
120/// The one extension a verifier may have.
121pub const SUFFIX: &str = ".mjs";
122
123/// The longest a verifier's short name, or a break's, may be.
124pub const NAME_MAX: usize = 64;
125
126/// How much of one run's output is kept in memory.
127///
128/// Only the check lines and the clean run's text survive past the comparison,
129/// so this bounds a working buffer rather than the report.
130pub const OUT_MAX: usize = 8 * 1024 * 1024;
131
132/// The default whole-sequence budget, when the caller names none.
133pub const BUDGET_DEFAULT_MS: u64 = 20 * 60 * 1_000;
134
135/// The largest whole-sequence budget a caller may ask for.
136pub const BUDGET_MAX_MS: u64 = 4 * 60 * 60 * 1_000;
137
138/// The least a single run is given before the budget is called spent.
139///
140/// A break handed four seconds is a break that will time out and be reported as
141/// unrun, which is worse than saying plainly that the budget ran out.
142pub const RUN_FLOOR_MS: u64 = 15 * 1_000;
143
144/// Where a verifier leaves pictures, relative to the granted root.
145pub const SHOTS_DIR: &str = "dev/shots";
146
147/// The most shot paths one report will name.
148pub const SHOTS_MAX: usize = 40;
149
150/// The largest report this module will send, in bytes.
151///
152/// Under [`CHUNK_MAX`] so the whole report is one chunk and a reader never has
153/// to reassemble it.
154pub const REPORT_MAX: usize = CHUNK_MAX - 4_096;
155
156/// The marker every report ends with, which the app checks for.
157///
158/// A second line of defence at the far end: `Tool::Verify` refuses to hand a
159/// model a result with no trailer, because a result with no trailer is a result
160/// whose three numbers were never computed.
161pub const TRAILER: &str = "[verify:";
162
163/// The exit status of a sequence that proved itself.
164pub const EXIT_PROVED: i32 = 0;
165
166/// The exit status of a sequence whose clean run had failures.
167pub const EXIT_FAILED: i32 = 1;
168
169/// The exit status of a clean run nothing has proved.
170pub const EXIT_UNPROVEN: i32 = 2;
171
172// ┌───────────────────────────────────────────────────────────────┐
173// │ Nobody is at the keyboard for a verifier this verb runs │
174// └───────────────────────────────────────────────────────────────┘
175//
176// A verifier is spawned with the hand's own environment, and the hand's own
177// environment is whatever started it. Started by the browser as a native
178// messaging host, that is the browser's -- `DISPLAY=:0` and `WAYLAND_DISPLAY`
179// among it, because the browser is on the owner's screen. Half the verifiers in
180// `dev/` drive a HEADED browser, and `dev/display.mjs` allowed `:0` on the stated
181// grounds that watching a headed run on one's own seat is a thing people do.
182//
183// It is. Nobody is watching this one. A model asked for it, and the window
184// would land in front of whoever happens to be at the machine, mid-sentence,
185// exactly as it did on 2026-08-24 -- the incident that file's header is written
186// about. That fault was a check reading the display STRING and not the
187// environment; this is the same shape one layer up, a check reading the
188// environment and not knowing who asked.
189//
190// The hand knows who asked, and it is the only party that does. So a verifier is
191// handed no display at all and is told that none is coming: it must start one of
192// its own, which `dev/verify_reflux.mjs` does and every other headed verifier is
193// free to. Refusing loudly is the point. A headed verifier reached this way
194// used to paint on the seat and now says it cannot, which is a capability this
195// verb never honestly had.
196
197/// The display names a verifier is never handed, whatever the hand was started with.
198pub const SEAT_VARS: &[&str] = &["DISPLAY", "WAYLAND_DISPLAY", "XDG_SESSION_TYPE"];
199
200/// The name that tells `dev/display.mjs` nobody is at the keyboard for this run.
201pub const UNATTENDED_VAR: &str = "DAIMOND_UNATTENDED";
202
203/// The environment a verifier is spawned with, from the hand's own.
204///
205/// Separate and pure so a test can put an environment to it without spawning
206/// anything -- the mistake `dev/verify_harness.mjs` exists because of was a rule
207/// that had no test because testing it meant opening a window.
208///
209/// # Arguments
210/// * `env` - The hand's own environment, name to value.
211pub fn unattended_env(env: &BTreeMap<String, String>) -> BTreeMap<String, String> {
212 let mut out = env.clone();
213 for v in SEAT_VARS {
214 out.remove(*v);
215 }
216 out.insert(fmt!("{}", UNATTENDED_VAR), fmt!("1"));
217 out
218}
219
220/// The hand's own environment, as a map.
221fn own_env() -> BTreeMap<String, String> {
222 std::env::vars().collect()
223}
224
225/// Is this a name a verifier may be looked up by?
226///
227/// Deliberately narrower than a file name. Lower case, digits and underscore
228/// are the whole alphabet every verifier in the tree is spelled with, and they
229/// leave nothing for a path, a flag or a shell to be made of: no dot, so `..`
230/// cannot be written; no slash; no dash, so nothing can begin with one and be
231/// read as an option.
232///
233/// # Arguments
234/// * `name` - The short name, as the model wrote it.
235pub fn name_ok(name: &str) -> bool {
236 !name.is_empty()
237 && name.len() <= NAME_MAX
238 && name.chars().all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '_')
239}
240
241/// Why a file is not the commit's.
242#[derive(Clone, Copy, Debug, Eq, PartialEq)]
243pub enum Change {
244 Untracked, // git has never heard of the path
245 Edited, // git has, and the working copy is not what the commit holds
246 Linked, // the path is a symlink, so its bytes are wherever it points
247}
248
249impl Change {
250 /// The clause the refusal and the report both put after the file's name.
251 pub fn phrase(&self) -> &'static str {
252 match self {
253 Self::Untracked => "git has never heard of it",
254 Self::Edited => "it is tracked, and the working copy is not what the commit \
255 holds -- staging a change is not committing it",
256 Self::Linked => "it is a symbolic link, so its bytes are wherever it points \
257 and the commit says nothing about them",
258 }
259 }
260}
261
262/// Whose bytes these are, as the repository can answer it.
263///
264/// **The question is content and not membership**, and the difference is the
265/// whole of `REVIEW.md` §1.21. `git ls-files --error-unmatch` answers about the
266/// INDEX: append one comment to a tracked verifier and it still exits 0, while
267/// `git diff --quiet HEAD` on the same file exits 1. The claim the unfenced run
268/// rests on is *these bytes came with the checkout*, and only the second question
269/// answers it.
270#[derive(Clone, Debug, Eq, PartialEq)]
271pub enum Provenance {
272 /// The working file is byte for byte what the commit holds.
273 Committed,
274 /// It is not, and this is how.
275 Changed(Change),
276 /// Nothing could be asked, and this is why.
277 ///
278 /// Refused rather than assumed either way. Four verifiers were once
279 /// promoted into `dev/` and left untracked, and `dev/gate.sh` builds its
280 /// tree with `git worktree add` -- a clean checkout of a commit -- so the
281 /// suite ran none of them while reporting a pass.
282 Unknown(String),
283}
284
285impl Provenance {
286 /// May a file of this provenance run outside the command fence?
287 pub fn committed(&self) -> bool {
288 matches!(self, Self::Committed)
289 }
290
291 /// The phrase the report carries.
292 pub fn phrase(&self) -> String {
293 match self {
294 Self::Committed => fmt!("byte for byte the commit's"),
295 Self::Changed(c) => fmt!("NOT THE COMMIT'S -- {}", c.phrase()),
296 Self::Unknown(w) => fmt!("provenance unknown ({})", w),
297 }
298 }
299}
300
301/// One verifier, as the tree actually holds it.
302#[derive(Clone, Debug)]
303pub struct Script {
304 /// The short name it was looked up by.
305 pub name: String,
306 /// The file name, taken from the directory entry and never from the caller.
307 pub file: String,
308 /// The absolute path, built from the directory and that entry.
309 pub path: PathBuf,
310 /// Every break the file declares, in the order it declares them.
311 pub breaks: Vec<String>,
312 /// Whose bytes these are, as the repository can answer it.
313 pub prov: Provenance,
314}
315
316/// Every break name a verifier's own source declares.
317///
318/// The declaration is a comment, because that is how this tree has always
319/// spelled it -- `dev/breakcheck.mjs` already reads the same text for the same
320/// purpose:
321///
322/// ```text
323/// // node dev/verify_about.mjs --break nobutton # 1: no About button in the bar
324/// ```
325///
326/// So the set is derived from the file rather than kept beside it, and a break
327/// that has been deleted stops being offered the moment it goes. A `--break`
328/// followed by anything but a lower-case letter -- `<name>`, `${BREAK}`, a
329/// closing quote -- is the file talking about the flag rather than declaring a
330/// value, and is passed over.
331///
332/// # Arguments
333/// * `src` - The verifier's source.
334pub fn declared_breaks(src: &str) -> Vec<String> {
335 let flag = "--break";
336 let b = src.as_bytes();
337 let mut out: Vec<String> = Vec::new();
338 let mut i = 0usize;
339 while let Some(hit) = src[i..].find(flag) {
340 let mut j = i + hit + flag.len();
341 i = j;
342 // The separator, which is the whole of what tells a declaration from a mention.
343 match b.get(j) {
344 Some(b' ') | Some(b'\t') | Some(b'=') => j += 1,
345 _ => continue,
346 }
347 while let Some(b' ') | Some(b'\t') = b.get(j) {
348 j += 1;
349 }
350 let start = j;
351 while let Some(c) = b.get(j) {
352 if c.is_ascii_lowercase() || c.is_ascii_digit() || *c == b'_' {
353 j += 1;
354 } else {
355 break;
356 }
357 }
358 if j == start {
359 continue;
360 }
361 let word = &src[start..j];
362 // A leading digit or underscore is not a break name in this tree, and taking one
363 // would offer the model a flag it cannot use.
364 match word.chars().next() {
365 Some(c) if c.is_ascii_lowercase() => (),
366 _ => continue,
367 }
368 if word.len() <= NAME_MAX && !out.iter().any(|w| w == word) {
369 out.push(fmt!("{}", word));
370 }
371 }
372 out
373}
374
375/// Every verifier the tree holds, by short name.
376///
377/// # Arguments
378/// * `root` - The granted root.
379pub fn catalogue(root: &Path) -> Outcome<Vec<String>> {
380 let dir = root.join(DEV_DIR);
381 let rd = res!(std::fs::read_dir(&dir).map_err(|e| err!(e,
382 "The verifiers live in '{}', which could not be read.", dir.display();
383 IO, Missing)));
384 let mut out = Vec::new();
385 for entry in rd {
386 let entry = match entry {
387 Ok(e) => e,
388 Err(_) => continue,
389 };
390 let name = entry.file_name();
391 let name = match name.to_str() {
392 Some(s) => s,
393 None => continue,
394 };
395 if name.starts_with(PREFIX) && name.ends_with(SUFFIX) {
396 let short = &name[PREFIX.len()..name.len() - SUFFIX.len()];
397 if name_ok(short) {
398 out.push(fmt!("{}", short));
399 }
400 }
401 }
402 out.sort();
403 Ok(out)
404}
405
406/// Whether this machine has any verifier to run at all.
407///
408/// What the `verify:` capability in the handshake is computed from, so the page
409/// can tell a model "not on this computer" rather than letting it discover the
410/// same thing one refusal at a time.
411///
412/// # Arguments
413/// * `root` - The granted root.
414pub fn available(root: &Path) -> bool {
415 match catalogue(root) {
416 Ok(v) => !v.is_empty(),
417 Err(_) => false,
418 }
419}
420
421/// The capability entry the handshake carries.
422///
423/// # Arguments
424/// * `root` - The granted root.
425pub fn cap(root: &Path) -> String {
426 match available(root) {
427 true => fmt!("verify:dev"),
428 false => fmt!("verify:none"),
429 }
430}
431
432/// The verifier a name refers to, or the sentence saying why there is none.
433///
434/// **Every string that leaves here came from the file system or from the file.**
435/// `file` is the directory entry's own name, `path` is that entry joined onto
436/// the directory, and `breaks` is parsed out of the source. The caller's `name`
437/// is used to *match* and is never used to *build*, which is the whole of the
438/// argument that nothing the model wrote can reach a process.
439///
440/// # Arguments
441/// * `root` - The granted root.
442/// * `name` - The short name, as the model wrote it.
443pub fn resolve(root: &Path, name: &str) -> Result<Script, String> {
444 if !name_ok(name) {
445 return Err(fmt!(
446 "Refused: '{}' is not a verifier name. A name is lower-case letters, digits and \
447 underscores -- 'graph' for dev/verify_graph.mjs. This verb takes a NAME and not a \
448 path or a command line: there is nothing here that would run a path you wrote.",
449 trim_for_message(name)));
450 }
451 let dir = root.join(DEV_DIR);
452 let wanted = fmt!("{}{}{}", PREFIX, name, SUFFIX);
453 let rd = match std::fs::read_dir(&dir) {
454 Ok(r) => r,
455 Err(e) => return Err(fmt!(
456 "Refused: this computer's granted folder has no readable '{}' directory ({}), so \
457 there are no verifiers on it to run.", dir.display(), e)),
458 };
459 let mut found: Option<std::ffi::OsString> = None;
460 for entry in rd {
461 let entry = match entry {
462 Ok(e) => e,
463 Err(_) => continue,
464 };
465 if entry.file_name() == std::ffi::OsStr::new(&wanted) {
466 found = Some(entry.file_name());
467 break;
468 }
469 }
470 let file = match found {
471 Some(f) => f,
472 None => return Err(no_such(root, name, &wanted)),
473 };
474 let path = dir.join(&file);
475 match std::fs::metadata(&path) {
476 Ok(m) if m.is_file() => (),
477 Ok(_) => return Err(fmt!(
478 "Refused: '{}' is not a file, so there is nothing to run.", path.display())),
479 Err(e) => return Err(fmt!(
480 "Refused: '{}' could not be read ({}), so there is nothing to run.",
481 path.display(), e)),
482 }
483 let src = match std::fs::read_to_string(&path) {
484 Ok(s) => s,
485 Err(e) => return Err(fmt!(
486 "Refused: '{}' could not be read as text ({}), so its breaks cannot be listed and \
487 it will not be run blind.", path.display(), e)),
488 };
489 let file_str = match file.to_str() {
490 Some(s) => fmt!("{}", s),
491 None => return Err(fmt!(
492 "Refused: the name of '{}' is not UTF-8, so it cannot be reported honestly.",
493 path.display())),
494 };
495 Ok(Script {
496 name: fmt!("{}", name),
497 breaks: declared_breaks(&src),
498 prov: provenance(root, &file_str),
499 file: file_str,
500 path,
501 })
502}
503
504/// The sentence for a name that names nothing, with the near misses in it.
505fn no_such(root: &Path, name: &str, wanted: &str) -> String {
506 let near = match catalogue(root) {
507 Ok(all) => {
508 let mut n: Vec<String> = all.into_iter()
509 .filter(|c| c.contains(name) || name.contains(c.as_str()))
510 .take(8)
511 .collect();
512 n.sort();
513 n
514 },
515 Err(_) => Vec::new(),
516 };
517 let tail = match near.is_empty() {
518 true => fmt!("Ask for a file listing of 'dev' to see what there is."),
519 false => fmt!("Names close to it: {}.", near.join(", ")),
520 };
521 fmt!(
522 "Refused: there is no 'dev/{}' in the folder this hand was granted, so '{}' names no \
523 verifier. {} Nothing was run.", wanted, trim_for_message(name), tail)
524}
525
526/// A caller's string, cut short and stripped of control characters, for a message.
527///
528/// A refusal quotes what the model asked for so it can see its own mistake, and
529/// a refusal is written to the journal -- so a hundred kilobytes of newlines
530/// must not become a hundred kilobytes of journal.
531fn trim_for_message(s: &str) -> String {
532 let mut out = String::new();
533 for c in s.chars() {
534 if out.chars().count() >= 40 {
535 out.push('…');
536 break;
537 }
538 match c.is_control() {
539 true => out.push('·'),
540 false => out.push(c),
541 }
542 }
543 out
544}
545
546/// Whose bytes `dev/<file>` holds, asked of git rather than assumed.
547///
548/// Three questions and the order matters. Is there a repository here at all --
549/// because a granted folder that is not one can vouch for nothing, and the
550/// permissive reading of that was half of `REVIEW.md` §1.21's reproduction. Is
551/// the path a symlink -- because a committed link's target is an ordinary file
552/// inside the granted root that any command may rewrite, which is §1.1's shape
553/// wearing §1.21's clothes. And last, does the working file differ from the
554/// commit: `git diff --quiet HEAD` compares the WORKING TREE with `HEAD`, so it
555/// catches a staged change and an unstaged one alike, and it applies the
556/// repository's own end-of-line and filter settings, which a hash of the bytes
557/// taken here would not.
558///
559/// Every argument vector is fixed but for one directory entry's own name, and
560/// there is no shell, so `--` before it is belt to the braces.
561///
562/// # Arguments
563/// * `root` - The granted root, which is where git is asked.
564/// * `file` - The verifier's file name, from the directory entry.
565pub fn provenance(root: &Path, file: &str) -> Provenance {
566 let git = match on_path("git") {
567 Some(p) => p,
568 None => return Provenance::Unknown(fmt!(
569 "there is no git on this hand's PATH, so nothing here can say whose code this is")),
570 };
571 let rel = fmt!("{}/{}", DEV_DIR, file);
572 let ask = |args: &[&str]| -> std::io::Result<std::process::ExitStatus> {
573 std::process::Command::new(&git)
574 .arg("-C").arg(root)
575 .args(args)
576 .stdin(Stdio::null())
577 .stdout(Stdio::null())
578 .stderr(Stdio::null())
579 .status()
580 };
581 match ask(&["rev-parse", "--git-dir"]) {
582 Ok(s) if s.success() => (),
583 Ok(_) => return Provenance::Unknown(fmt!(
584 "the folder this hand was granted is not a git repository, so there is no commit \
585 to compare this file with")),
586 Err(e) => return Provenance::Unknown(fmt!("git could not be run: {}", e)),
587 }
588 // Asked without following the link, which is the whole point: `metadata` above
589 // followed it and answered about the target.
590 match std::fs::symlink_metadata(root.join(&rel)) {
591 Ok(m) if m.file_type().is_symlink() => return Provenance::Changed(Change::Linked),
592 Ok(_) => (),
593 Err(e) => return Provenance::Unknown(fmt!(
594 "'{}' could not be looked at ({})", rel, e)),
595 }
596 match ask(&["ls-files", "--error-unmatch", "--", &rel]) {
597 Ok(s) if s.success() => (),
598 Ok(_) => return Provenance::Changed(Change::Untracked),
599 Err(e) => return Provenance::Unknown(fmt!("git could not be run: {}", e)),
600 }
601 // `git diff` says 0 for no difference and 1 for a difference; anything else is git
602 // failing to answer, and an unborn HEAD in a repository with no commits is exactly
603 // that. A failure to answer is never read as agreement.
604 match ask(&["diff", "--quiet", "HEAD", "--", &rel]) {
605 Ok(s) if s.success() => Provenance::Committed,
606 Ok(s) if s.code() == Some(1) => Provenance::Changed(Change::Edited),
607 Ok(s) => Provenance::Unknown(fmt!(
608 "git could not compare '{}' with the commit (it exited {}); a repository with no \
609 commit in it yet is the usual reason", rel, match s.code() {
610 Some(c) => fmt!("{}", c),
611 None => fmt!("on a signal"),
612 })),
613 Err(e) => Provenance::Unknown(fmt!("git could not be run: {}", e)),
614 }
615}
616
617/// The sentence a verifier that is not the commit's is refused with, or `None`.
618///
619/// **This is the gate `REVIEW.md` §1.21 was open for**, and it is asked fresh
620/// rather than read off the [`Script`] resolved earlier: a verifier sequence
621/// runs for minutes, a background command started by an earlier turn can rewrite
622/// a file while it does, and a check taken once at the start would be answering
623/// about bytes that are no longer there. [`conduct`] asks before anything runs
624/// and [`once`] asks again before each spawn, so the window between the answer
625/// and the `execve` is as small as this side can make it.
626///
627/// **The sentence is written to be acted on in one call.** A refusal a model
628/// cannot converge on costs the run anyway -- so it hands over the command that
629/// runs the same file INSIDE the fence, which is where a command a model wrote
630/// belongs and is enough for the nine verifiers in ten here that only read the
631/// tree; it says what that cannot do, so a fenced failure is not misread as the
632/// verifier being broken; and it names the two things that reach the unfenced
633/// run, one of which is a person.
634///
635/// # Arguments
636/// * `root` - The granted root.
637/// * `file` - The verifier's file name, from the directory entry.
638pub fn unfenced_refusal(root: &Path, file: &str) -> Option<String> {
639 let prov = provenance(root, file);
640 let why = match &prov {
641 Provenance::Committed => return None,
642 Provenance::Changed(c) => fmt!("{}", c.phrase()),
643 Provenance::Unknown(w) => fmt!("{}", w),
644 };
645 Some(fmt!(
646 "Refused: dev/{} is not this repository's committed code -- {}. Nothing was run. A \
647 verifier is the one thing this hand runs OUTSIDE the command fence, and the whole of \
648 the reason is that its bytes came with the checkout instead of from a model; this \
649 hand cannot tell your edit from anybody else's, so it will not run one unfenced. \
650 RUN IT YOURSELF INSTEAD: 'run' with [\"node\",\"dev/{}\"] runs this same file inside \
651 the fence every command gets, which is all a verifier that reads the tree needs. It \
652 is not enough for one that drives a browser or opens a port -- the fence refuses \
653 both, so a failure there is the fence and not your script. To get the unfenced run, \
654 commit the file and ask again, or ask the person to run it themselves.",
655 file, why, file))
656}
657
658/// The first entry on `PATH` that is a runnable file with this name.
659///
660/// The hand's own `PATH`, which is the browser's, since Chrome hands a native
661/// messaging host its own environment. A name and never a path: this is asked
662/// only about `node` and `git`, both of which are written down here.
663///
664/// # Arguments
665/// * `prog` - The program's bare name.
666pub fn on_path(prog: &str) -> Option<PathBuf> {
667 let path = match std::env::var("PATH") {
668 Ok(p) => p,
669 Err(_) => return None,
670 };
671 for dir in path.split(':') {
672 if dir.is_empty() {
673 continue;
674 }
675 let cand = Path::new(dir).join(prog);
676 if runnable(&cand) {
677 return Some(cand);
678 }
679 }
680 None
681}
682
683/// Is this a file the kernel would execute?
684#[cfg(unix)]
685fn runnable(p: &Path) -> bool {
686 use std::os::unix::fs::PermissionsExt;
687 match std::fs::metadata(p) {
688 Ok(m) => m.is_file() && (m.permissions().mode() & 0o111) != 0,
689 Err(_) => false,
690 }
691}
692
693/// Is this a file the kernel would execute?
694#[cfg(not(unix))]
695fn runnable(p: &Path) -> bool {
696 match std::fs::metadata(p) {
697 Ok(m) => m.is_file(),
698 Err(_) => false,
699 }
700}
701
702/// Every file under `dev/shots` this sequence wrote, workspace-relative.
703///
704/// Named rather than returned. A daimon reads a picture by asking `file_read`
705/// for it with `"as":"image"`, or by dispatching a worker who can see -- so what
706/// the report owes it is the PATH, and handing back the bytes would spend a
707/// context window on an image the model may not be able to look at anyway.
708///
709/// Compared by modification time against the moment the sequence started, so an
710/// old screenshot from last week's run is not claimed as this run's evidence.
711///
712/// # Arguments
713/// * `root` - The granted root.
714/// * `since` - When the sequence began.
715pub fn shots_since(root: &Path, since: std::time::SystemTime) -> Vec<String> {
716 let mut out = Vec::new();
717 let base = root.join(SHOTS_DIR);
718 let mut todo = vec![base.clone()];
719 while let Some(dir) = todo.pop() {
720 if out.len() >= SHOTS_MAX {
721 break;
722 }
723 let rd = match std::fs::read_dir(&dir) {
724 Ok(r) => r,
725 Err(_) => continue,
726 };
727 for entry in rd {
728 let entry = match entry {
729 Ok(e) => e,
730 Err(_) => continue,
731 };
732 let path = entry.path();
733 let meta = match entry.metadata() {
734 Ok(m) => m,
735 Err(_) => continue,
736 };
737 if meta.is_dir() {
738 todo.push(path);
739 continue;
740 }
741 let fresh = match meta.modified() {
742 Ok(t) => t >= since,
743 Err(_) => false,
744 };
745 if !fresh {
746 continue;
747 }
748 if let Ok(rel) = path.strip_prefix(root) {
749 out.push(fmt!("{}", rel.display()));
750 }
751 }
752 }
753 out.sort();
754 out.truncate(SHOTS_MAX);
755 out
756}
757
758// ┌───────────────────────────────────────────────────────────────┐
759// │ Reading what a verifier said │
760// └───────────────────────────────────────────────────────────────┘
761
762/// Every check a run reported, by name, and whether it passed every time it
763/// appeared.
764///
765/// A `BTreeMap` because two runs are compared and the comparison must not depend
766/// on the order the lines arrived in; a name that appears twice folds to the
767/// worse of the two, since a check that failed once is not a check that passed.
768pub type Checks = BTreeMap<String, bool>;
769
770/// The checks a run's output reports.
771///
772/// The convention is one line per check, and it is the same in all 265 files of
773/// `dev/`, because they all copy the same four-line helper:
774///
775/// ```text
776/// console.log((pass ? ' ok ' : ' FAIL ') + name + (detail ? ' — ' + detail : ''));
777/// ```
778///
779/// The detail is cut off before the name is kept. It carries counts, paths and
780/// timings that differ between two runs of the same passing check, and comparing
781/// it would make every break look as though it had bitten.
782///
783/// # Arguments
784/// * `text` - Everything the run printed, on both streams.
785pub fn parse_checks(text: &str) -> Checks {
786 let mut out: Checks = BTreeMap::new();
787 for line in text.lines() {
788 let t = line.trim_start();
789 let (pass, rest) = if let Some(r) = t.strip_prefix("ok ") {
790 (true, r)
791 } else if let Some(r) = t.strip_prefix("FAIL ") {
792 (false, r)
793 } else if let Some(r) = t.strip_prefix("PASS ") {
794 (true, r)
795 } else {
796 continue;
797 };
798 let head = match rest.find(" — ") {
799 Some(i) => &rest[..i],
800 None => rest,
801 };
802 let name = head.trim();
803 if name.is_empty() {
804 continue;
805 }
806 let e = out.entry(fmt!("{}", name)).or_insert(true);
807 *e = *e && pass;
808 }
809 out
810}
811
812/// The text of a run with the parts that legitimately differ taken out.
813///
814/// Used for one detail only -- saying that a break's output was *identical* to
815/// the clean run's, which is the most damning thing that can be said about an
816/// instrument. A verifier usually announces its own break in a banner, so a raw
817/// comparison would never find two runs identical and the worst case would never
818/// be reported. Lines mentioning the flag are therefore dropped, along with
819/// blank lines and trailing space.
820///
821/// # Arguments
822/// * `text` - Everything the run printed.
823pub fn normalise(text: &str) -> String {
824 let mut out = String::with_capacity(text.len());
825 for line in text.lines() {
826 if line.contains("--break") || line.contains("BREAK") || line.contains("break '") {
827 continue;
828 }
829 let t = line.trim_end();
830 if t.is_empty() {
831 continue;
832 }
833 out.push_str(t);
834 out.push('\n');
835 }
836 out
837}
838
839/// What one run of a verifier did.
840#[derive(Clone, Debug)]
841pub struct Pass {
842 /// `clean`, or the break's name.
843 pub label: String,
844 /// The exit status, or -1 where there was none.
845 pub exit: i32,
846 /// Whether the budget killed it.
847 pub timed: bool,
848 /// The checks it reported.
849 pub checks: Checks,
850 /// Its output with the varying parts removed, for the identity comparison.
851 pub norm: String,
852 /// Up to a few failing lines, verbatim, for the report.
853 pub fails: Vec<String>,
854 /// How long it took, in milliseconds.
855 pub ms: u64,
856}
857
858impl Pass {
859 /// How many checks passed.
860 pub fn passed(&self) -> usize {
861 self.checks.values().filter(|v| **v).count()
862 }
863
864 /// How many checks failed.
865 pub fn failed(&self) -> usize {
866 self.checks.values().filter(|v| !**v).count()
867 }
868}
869
870/// What a break did to the checks that passed clean.
871#[derive(Clone, Debug, Eq, PartialEq)]
872pub enum Bite {
873 /// At least one check that passed clean failed under it.
874 Red {
875 /// Which ones, up to a handful.
876 names: Vec<String>,
877 },
878 /// No check that passed clean failed under it. A lying instrument.
879 None {
880 /// Whether its output was the clean run's, banner aside.
881 same: bool,
882 /// Checks that passed clean and did not appear at all.
883 vanished: usize,
884 },
885 /// It never ran, and this is why.
886 Unrun {
887 /// The reason, in a phrase.
888 why: String,
889 },
890}
891
892/// What a break did, compared with the clean run.
893///
894/// **Deliberately the weaker claim.** A break declares in a comment which check
895/// it aims at, and a comment is not something to hold a verdict on -- so what is
896/// asserted is that *some* check which passed clean now fails. A break that
897/// reddens a different check than it meant to is still an instrument that has
898/// been seen to move; a break that reddens nothing is not.
899///
900/// # Arguments
901/// * `clean` - The clean run.
902/// * `brk` - The run under the break.
903pub fn bite(clean: &Pass, brk: &Pass) -> Bite {
904 let mut newly: Vec<String> = Vec::new();
905 let mut vanished: usize = 0;
906 for (name, ok) in clean.checks.iter() {
907 if !*ok {
908 continue; // it was already failing; it can say nothing about this break
909 }
910 match brk.checks.get(name) {
911 Some(true) => (),
912 Some(false) => newly.push(fmt!("{}", name)),
913 None => vanished += 1,
914 }
915 }
916 if !newly.is_empty() {
917 newly.truncate(6);
918 return Bite::Red { names: newly };
919 }
920 Bite::None {
921 same: clean.norm == brk.norm,
922 vanished,
923 }
924}
925
926// ┌───────────────────────────────────────────────────────────────┐
927// │ The verdict, which cannot be given without its bad half │
928// └───────────────────────────────────────────────────────────────┘
929
930/// What a whole sequence proved.
931///
932/// **An enum with no field accessors on purpose.** The count of checks that
933/// passed is reachable only by matching, and every arm that carries it also
934/// carries what is wrong with it -- so there is no expression in this program
935/// that yields "27 passed" without yielding, in the same breath, the number of
936/// breaks that proved nothing or the fact that none was run. That is the whole
937/// design of this type, and a struct with three public fields would not have it.
938#[derive(Clone, Debug, Eq, PartialEq)]
939pub enum Verdict {
940 /// The clean run and at least one break were run.
941 Proven {
942 /// Checks that passed clean.
943 passed: usize,
944 /// Checks that failed clean.
945 failed: usize,
946 /// Breaks that reddened a check which passed clean.
947 red: Vec<String>,
948 /// Breaks that reddened nothing. **The lying-instrument count.**
949 dead: Vec<String>,
950 /// Breaks the budget or the machine never let run.
951 unrun: Vec<String>,
952 },
953 /// Only the clean run was made, so nothing here is known to be able to fail.
954 Unproven {
955 /// Checks that passed clean.
956 passed: usize,
957 /// Checks that failed clean.
958 failed: usize,
959 /// The breaks the file declares and this run did not use.
960 declared: Vec<String>,
961 },
962}
963
964impl Verdict {
965 /// The one-line trailer, which is the sentence a model repeats.
966 ///
967 /// Both arms are here so that neither can be written without the other being
968 /// looked at, and the `Unproven` arm names itself in the words the model
969 /// should carry back: *proves nothing*.
970 pub fn trailer(&self) -> String {
971 match self {
972 Self::Proven { passed, failed, red, dead, unrun } => {
973 let mut s = fmt!(
974 "{} {} checks passed, {} failed, {} breaks confirmed red, {} breaks proved \
975 nothing", TRAILER, passed, failed, red.len(), dead.len());
976 if !unrun.is_empty() {
977 s.push_str(&fmt!(", {} breaks never ran", unrun.len()));
978 }
979 s.push(']');
980 s
981 },
982 Self::Unproven { passed, failed, declared } => fmt!(
983 "{} {} checks passed, {} failed -- NOT PROVEN: no break was run, so no check \
984 here has been shown to be able to fail. This is not evidence that anything \
985 works. {} declared breaks were skipped; run again without 'clean_only' to prove \
986 the instrument.]",
987 TRAILER, passed, failed, declared.len()),
988 }
989 }
990
991 /// The exit status this verdict deserves.
992 pub fn exit(&self) -> i32 {
993 match self {
994 Self::Proven { failed, dead, unrun, .. } => {
995 if *failed > 0 {
996 EXIT_FAILED
997 } else if !dead.is_empty() || !unrun.is_empty() {
998 EXIT_UNPROVEN
999 } else {
1000 EXIT_PROVED
1001 }
1002 },
1003 Self::Unproven { failed, .. } => match *failed > 0 {
1004 true => EXIT_FAILED,
1005 false => EXIT_UNPROVEN,
1006 },
1007 }
1008 }
1009
1010 /// The paragraph under the trailer that says what to do about it.
1011 pub fn advice(&self) -> String {
1012 match self {
1013 Self::Proven { failed, dead, unrun, .. } => {
1014 let mut s = String::new();
1015 if *failed > 0 {
1016 s.push_str(
1017 "Checks failed in the CLEAN run: that is the code, not the instrument. \
1018 Read the failing lines above before running anything again.\n");
1019 }
1020 if !dead.is_empty() {
1021 s.push_str(&fmt!(
1022 "A break that proved nothing means the check it aims at cannot be made \
1023 to fail, so that check is measuring nothing and its pass is worth \
1024 nothing. Treat these as unmeasured and say so: {}.\n",
1025 dead.join(", ")));
1026 }
1027 if !unrun.is_empty() {
1028 s.push_str(&fmt!(
1029 "These breaks never ran, so this sequence is an incomplete proof: {}.\n",
1030 unrun.join(", ")));
1031 }
1032 if s.is_empty() {
1033 s.push_str(
1034 "Every declared break reddened a check that passed clean, so these \
1035 checks have each been seen to fail. That is what makes the pass above \
1036 evidence rather than an assertion.\n");
1037 }
1038 s
1039 },
1040 Self::Unproven { declared, .. } => {
1041 let mut s = fmt!(
1042 "THIS RUN PROVES NOTHING. A check that has never been observed failing is \
1043 not a check, and no check here was observed failing. Do not report the \
1044 passing count as evidence.\n");
1045 if !declared.is_empty() {
1046 s.push_str(&fmt!(
1047 "The breaks this verifier declares, and which were not run: {}.\n",
1048 declared.join(", ")));
1049 }
1050 s
1051 },
1052 }
1053 }
1054}
1055
1056// ┌───────────────────────────────────────────────────────────────┐
1057// │ Which breaks to run │
1058// └───────────────────────────────────────────────────────────────┘
1059
1060/// The breaks a sequence will attempt, or the sentence saying why it cannot.
1061///
1062/// # Arguments
1063/// * `script` - The resolved verifier.
1064/// * `want` - What the caller asked for.
1065pub fn chosen(script: &Script, want: &crate::wire::Breaks) -> Result<Vec<String>, String> {
1066 match want {
1067 crate::wire::Breaks::None => Ok(Vec::new()),
1068 crate::wire::Breaks::All => Ok(script.breaks.clone()),
1069 crate::wire::Breaks::One(asked) => {
1070 if !name_ok(asked) {
1071 return Err(fmt!(
1072 "Refused: '{}' is not a break name. A break is lower-case letters, digits \
1073 and underscores, and it must be one this verifier declares.",
1074 trim_for_message(asked)));
1075 }
1076 // Matched against the file's own declarations, and the string that goes on to
1077 // the command line is the DECLARED copy, never the caller's.
1078 match script.breaks.iter().find(|b| b.as_str() == asked.as_str()) {
1079 Some(b) => Ok(vec![b.clone()]),
1080 None => Err(match script.breaks.is_empty() {
1081 true => fmt!(
1082 "Refused: dev/{} declares no breaks at all, so '{}' is not one of \
1083 them and there is no way to prove any of its checks. Run it without a \
1084 break and treat the result as unproven, or give the verifier a break \
1085 mode first.", script.file, trim_for_message(asked)),
1086 false => fmt!(
1087 "Refused: dev/{} does not declare a break called '{}'. The ones it \
1088 declares are: {}. A break has to be one the verifier itself knows how \
1089 to apply; naming any other string would run the file unchanged and \
1090 report a pass that means nothing.",
1091 script.file, trim_for_message(asked), script.breaks.join(", ")),
1092 }),
1093 }
1094 },
1095 }
1096}
1097
1098// ┌───────────────────────────────────────────────────────────────┐
1099// │ The record │
1100// └───────────────────────────────────────────────────────────────┘
1101
1102/// The journal, and whether it is still being written.
1103///
1104/// A handle rather than the whole dispatcher, so the sequence can be run from a
1105/// task of its own without carrying `main.rs`'s furniture into this module.
1106#[derive(Clone)]
1107pub struct Ledger {
1108 jr: Arc<Mutex<Journal>>,
1109 sound: Arc<AtomicBool>,
1110}
1111
1112impl Ledger {
1113 /// # Arguments
1114 /// * `jr` - The record.
1115 /// * `sound` - Whether it is still being written.
1116 pub fn new(jr: Arc<Mutex<Journal>>, sound: Arc<AtomicBool>) -> Self {
1117 Self { jr, sound }
1118 }
1119
1120 /// Writes one event, and remembers a failure exactly as the dispatcher does.
1121 pub fn record(&self, ev: &Event) -> Outcome<()> {
1122 let done = {
1123 let mut g = lock_mutex!(self.jr);
1124 g.append(ev)
1125 };
1126 match done {
1127 Ok(_) => Ok(()),
1128 Err(e) => {
1129 self.sound.store(false, Ordering::SeqCst);
1130 Err(e)
1131 },
1132 }
1133 }
1134
1135 /// Is the record still sound?
1136 pub fn sound(&self) -> bool {
1137 self.sound.load(Ordering::SeqCst)
1138 }
1139}
1140
1141// ┌───────────────────────────────────────────────────────────────┐
1142// │ Running the sequence │
1143// └───────────────────────────────────────────────────────────────┘
1144
1145/// Everything one sequence needs.
1146pub struct Job {
1147 /// The caller's identifier, echoed on every response.
1148 pub id: String,
1149 /// The granted root, which is where the verifier runs.
1150 pub root: PathBuf,
1151 /// The verifier.
1152 pub script: Script,
1153 /// The breaks to attempt, already checked against the file's declarations.
1154 pub breaks: Vec<String>,
1155 /// Node, found on the hand's own `PATH`.
1156 pub node: PathBuf,
1157 /// The whole sequence's wall-clock budget.
1158 pub budget: Duration,
1159 /// The record.
1160 pub ledger: Ledger,
1161}
1162
1163/// The argument vector one run takes.
1164///
1165/// Three elements at most, and every one of them is either a constant of this
1166/// program, a path built from a directory entry, or a break name parsed out of
1167/// the verifier's own source. **There is no shell**: this vector is handed to
1168/// `execve` through [`Command`], so a semicolon or a `$(…)` in any element would
1169/// be an argument and not a command. A model's string is not in it at all.
1170///
1171/// # Arguments
1172/// * `job` - The sequence.
1173/// * `brk` - The break, or `None` for the clean run.
1174pub fn argv_for(job: &Job, brk: Option<&str>) -> Vec<String> {
1175 let mut v = vec![
1176 fmt!("{}", job.node.display()),
1177 fmt!("{}", job.script.path.display()),
1178 ];
1179 if let Some(b) = brk {
1180 v.push(fmt!("--break"));
1181 v.push(fmt!("{}", b));
1182 }
1183 v
1184}
1185
1186/// Reads a stream to its end, keeping at most `cap` bytes.
1187///
1188/// The rest is drained rather than left, because a child whose pipe is full
1189/// stops running and a run that never ends is a budget spent on nothing.
1190async fn drain<R>(mut r: R, cap: usize) -> (Vec<u8>, u64)
1191where
1192 R: tokio::io::AsyncRead + Unpin,
1193{
1194 let mut kept = Vec::new();
1195 let mut total = 0u64;
1196 let mut buf = [0u8; 32 * 1024];
1197 loop {
1198 let n = match r.read(&mut buf).await {
1199 Ok(0) => break,
1200 Ok(n) => n,
1201 Err(_) => break,
1202 };
1203 total = total.saturating_add(n as u64);
1204 if kept.len() < cap {
1205 let room = cap - kept.len();
1206 let take = room.min(n);
1207 kept.extend_from_slice(&buf[..take]);
1208 }
1209 }
1210 (kept, total)
1211}
1212
1213/// Runs the verifier once, clean or under one break.
1214///
1215/// # Arguments
1216/// * `job` - The sequence.
1217/// * `brk` - The break, or `None`.
1218/// * `left` - What is left of the budget.
1219/// * `announce` - Where to send [`Resp::Started`], for the first run only.
1220async fn once(
1221 job: &Job,
1222 brk: Option<&str>,
1223 left: Duration,
1224 announce: Option<&Sender<Resp>>,
1225)
1226 -> Result<Pass, String>
1227{
1228 let label = match brk {
1229 Some(b) => fmt!("{}", b),
1230 None => fmt!("clean"),
1231 };
1232 let argv = argv_for(job, brk);
1233 // The record is written before the process exists, exactly as a command's is, and with the
1234 // REAL argument vector -- so a reader of the journal sees the node invocation rather than a
1235 // verb they would have to trust this module's account of.
1236 if !job.ledger.sound() {
1237 return Err(fmt!("the hand's journal cannot be written"));
1238 }
1239 let ev = Event::from_req(&Req::Exec {
1240 id: fmt!("{}#{}", job.id, label),
1241 argv: argv.clone(),
1242 cwd: fmt!("{}", job.root.display()),
1243 env: Vec::new(),
1244 stdin: None,
1245 timeout_ms: left.as_millis() as u64,
1246 capture: Capture::Both,
1247 // What the process may touch, as intent. `mechs` below is what was actually in force,
1248 // and it says `fence:none` -- because this verb runs a TRACKED SCRIPT outside the
1249 // command fence on purpose, and a record that hid that would be worse than no record.
1250 fence: FenceSpec {
1251 rw: vec![fmt!("{}", job.root.display())],
1252 ro: Vec::new(),
1253 deny: Vec::new(),
1254 net: true,
1255 },
1256 toolkits: Vec::new(),
1257 }, &[
1258 fmt!("fence:none"),
1259 fmt!("verify:tracked-script"),
1260 fmt!("net:open"),
1261 ]);
1262 if let Some(ev) = ev {
1263 if let Err(e) = job.ledger.record(&ev) {
1264 return Err(fmt!("it could not be written to the journal ({})", e.msgs().join(" ")));
1265 }
1266 }
1267
1268 // Asked again, as late as anything on this side can be. The sequence has been
1269 // running for minutes by the time a late break starts, and a command an earlier turn
1270 // left in the background can rewrite a file in that time.
1271 if let Some(no) = unfenced_refusal(&job.root, &job.script.file) {
1272 // The sentence opens with "Refused:" and every caller of `once` puts its own
1273 // word in front of what comes back from here, so the word is said once.
1274 let said = match no.strip_prefix("Refused: ") {
1275 Some(t) => t,
1276 None => no.as_str(),
1277 };
1278 return Err(fmt!(
1279 "it stopped being the commit's while the sequence was running. {}", said));
1280 }
1281
1282 let started = std::time::Instant::now();
1283 let mut cmd = Command::new(&job.node);
1284 for a in argv.iter().skip(1) {
1285 cmd.arg(a);
1286 }
1287 cmd.current_dir(&job.root);
1288 // Cleared and rebuilt rather than trimmed, because `Command` has no way to
1289 // ask what it inherited: the map is the hand's own environment with the seat
1290 // taken out and the mark put in, which is the same thing said once.
1291 cmd.env_clear();
1292 cmd.envs(unattended_env(&own_env()));
1293 cmd.stdin(Stdio::null());
1294 cmd.stdout(Stdio::piped());
1295 cmd.stderr(Stdio::piped());
1296 cmd.kill_on_drop(true);
1297 let mut child = match cmd.spawn() {
1298 Ok(c) => c,
1299 Err(e) => return Err(fmt!("node could not be started ({})", e)),
1300 };
1301 // The page starts a 30-second clock at the request and stops it at `Started`, so the
1302 // first child's real process id goes out the moment there is one.
1303 if let Some(tx) = announce {
1304 let _ = tx.send(Resp::Started {
1305 id: fmt!("{}", job.id),
1306 pid: child.id().unwrap_or(0),
1307 }).await;
1308 }
1309 let out_pipe = child.stdout.take();
1310 let err_pipe = child.stderr.take();
1311 // BOTH STREAMS AT ONCE, and it has to be. A pipe holds 64 KB; a child that fills stderr
1312 // while this side is still waiting for stdout to reach its end stops running, and the
1313 // deadlock lasts until the budget kills it. Every verifier in dev/ writes to both.
1314 let reading = async {
1315 let o = async {
1316 match out_pipe {
1317 Some(p) => drain(p, OUT_MAX).await,
1318 None => (Vec::new(), 0),
1319 }
1320 };
1321 let e = async {
1322 match err_pipe {
1323 Some(p) => drain(p, OUT_MAX).await,
1324 None => (Vec::new(), 0),
1325 }
1326 };
1327 tokio::join!(o, e)
1328 };
1329 let waiting = async {
1330 let status = child.wait().await;
1331 status
1332 };
1333 let both = async { tokio::join!(waiting, reading) };
1334 let (status, (out, err), timed) = match tokio::time::timeout(left, both).await {
1335 Ok((s, pair)) => (s, pair, false),
1336 Err(_) => {
1337 // The budget is up. The child is asked to go; anything it started of its own
1338 // (a browser, a server) is not this module's to find, and the report says so.
1339 (Err(std::io::Error::new(std::io::ErrorKind::TimedOut, "budget spent")),
1340 ((Vec::new(), 0u64), (Vec::new(), 0u64)),
1341 true)
1342 },
1343 };
1344 let exit = match &status {
1345 Ok(s) => s.code().unwrap_or(-1),
1346 Err(_) => -1,
1347 };
1348 let text = fmt!("{}{}",
1349 String::from_utf8_lossy(&out.0),
1350 String::from_utf8_lossy(&err.0));
1351 let checks = parse_checks(&text);
1352 let mut fails: Vec<String> = text.lines()
1353 .filter(|l| l.trim_start().starts_with("FAIL "))
1354 .map(|l| fmt!("{}", l.trim_end()))
1355 .collect();
1356 fails.truncate(12);
1357 let ms = started.elapsed().as_millis() as u64;
1358 // The ending is recorded as a command's is.
1359 let ended = Event::Ended {
1360 id: fmt!("{}#{}", job.id, label),
1361 exit,
1362 timed_out: timed,
1363 killed: false,
1364 out_bytes: out.1,
1365 err_bytes: err.1,
1366 };
1367 if let Err(e) = job.ledger.record(&ended) {
1368 eprintln!("daimond-hand: a verify run's ending was not journalled: {}", e);
1369 }
1370 if timed {
1371 return Err(fmt!("it ran past the budget and was killed"));
1372 }
1373 Ok(Pass {
1374 label,
1375 exit,
1376 timed,
1377 norm: normalise(&text),
1378 checks,
1379 fails,
1380 ms,
1381 })
1382}
1383
1384/// Runs the clean pass and every chosen break, and answers with all three
1385/// numbers.
1386///
1387/// **There is no path out of this function that yields the passing count
1388/// alone.** It returns a [`Verdict`], whose arms each carry the bad half, and
1389/// the report below is composed from that one value.
1390///
1391/// # Arguments
1392/// * `job` - The sequence.
1393/// * `tx` - Where progress and the report are sent.
1394pub async fn conduct(job: Job, tx: Sender<Resp>) -> Outcome<()> {
1395 // BEFORE THE JOURNAL AND BEFORE THE FIRST PROCESS. `REVIEW.md` §1.21: everything
1396 // below this line runs a script with no fence on it at all, and the only thing that
1397 // makes that defensible is that the script is the commit's. Asked here rather than
1398 // in `main.rs` because this function is the crate's public way in, and a gate in the
1399 // dispatcher is a gate the reproduction walked straight past.
1400 if let Some(no) = unfenced_refusal(&job.root, &job.script.file) {
1401 let _ = tx.send(Resp::Refused { id: fmt!("{}", job.id), reason: no }).await;
1402 return Ok(());
1403 }
1404 let mut left = job.budget;
1405 let mut seq_err = 0u64;
1406 let began = std::time::SystemTime::now();
1407
1408 let say_err = |line: String, seq: &mut u64| {
1409 let n = *seq;
1410 *seq = seq.saturating_add(1);
1411 Resp::Chunk {
1412 id: fmt!("{}", job.id),
1413 stream: Stream::Err,
1414 seq: n,
1415 data: line,
1416 }
1417 };
1418
1419 // The clean run first, always. Its checks are the baseline every break is compared with,
1420 // so a sequence that could not make it has nothing to compare anything against.
1421 let note = say_err(fmt!(" .. clean run of dev/{}\n", job.script.file), &mut seq_err);
1422 let _ = tx.send(note).await;
1423 let clean = match once(&job, None, left, Some(&tx)).await {
1424 Ok(p) => p,
1425 Err(w) => {
1426 let _ = tx.send(Resp::Refused {
1427 id: fmt!("{}", job.id),
1428 reason: fmt!(
1429 "Refused: the clean run of dev/{} did not happen -- {}. Nothing was \
1430 measured, so there is no result to report.", job.script.file, w),
1431 }).await;
1432 return Ok(());
1433 },
1434 };
1435 left = left.saturating_sub(Duration::from_millis(clean.ms));
1436
1437 let mut passes: Vec<(Pass, Option<Bite>)> = vec![(clean.clone(), None)];
1438 let mut red: Vec<String> = Vec::new();
1439 let mut dead: Vec<String> = Vec::new();
1440 let mut unrun: Vec<String> = Vec::new();
1441
1442 for b in job.breaks.iter() {
1443 if left < Duration::from_millis(RUN_FLOOR_MS) {
1444 unrun.push(fmt!("{}", b));
1445 continue;
1446 }
1447 let note = say_err(fmt!(" .. --break {}\n", b), &mut seq_err);
1448 let _ = tx.send(note).await;
1449 match once(&job, Some(b.as_str()), left, None).await {
1450 Ok(p) => {
1451 left = left.saturating_sub(Duration::from_millis(p.ms));
1452 let what = bite(&clean, &p);
1453 match &what {
1454 Bite::Red { .. } => red.push(fmt!("{}", b)),
1455 Bite::None { .. } => dead.push(fmt!("{}", b)),
1456 Bite::Unrun { .. } => unrun.push(fmt!("{}", b)),
1457 }
1458 passes.push((p, Some(what)));
1459 },
1460 Err(w) => {
1461 unrun.push(fmt!("{}", b));
1462 passes.push((Pass {
1463 label: fmt!("{}", b),
1464 exit: -1,
1465 timed: false,
1466 checks: Checks::new(),
1467 norm: String::new(),
1468 fails: Vec::new(),
1469 ms: 0,
1470 }, Some(Bite::Unrun { why: w })));
1471 },
1472 }
1473 }
1474
1475 // The one place a verdict is made, and it is made from what actually ran.
1476 let verdict = match job.breaks.is_empty() {
1477 true => Verdict::Unproven {
1478 passed: clean.passed(),
1479 failed: clean.failed(),
1480 declared: job.script.breaks.clone(),
1481 },
1482 false => Verdict::Proven {
1483 passed: clean.passed(),
1484 failed: clean.failed(),
1485 red,
1486 dead,
1487 unrun,
1488 },
1489 };
1490 let shots = shots_since(&job.root, began);
1491 let text = report(&job.script, &passes, &verdict, &shots);
1492 let bytes = text.len() as u64;
1493 let _ = tx.send(Resp::Chunk {
1494 id: fmt!("{}", job.id),
1495 stream: Stream::Out,
1496 seq: 0,
1497 data: text,
1498 }).await;
1499 let _ = tx.send(Resp::Ended {
1500 id: fmt!("{}", job.id),
1501 exit: verdict.exit(),
1502 timed_out: false,
1503 killed: false,
1504 out_bytes: bytes,
1505 err_bytes: seq_err,
1506 }).await;
1507 Ok(())
1508}
1509
1510/// The whole report, which is the only thing the model reads.
1511///
1512/// # Arguments
1513/// * `script` - The verifier.
1514/// * `passes` - The clean run first, then each break with what it did.
1515/// * `verdict` - What the sequence proved.
1516/// * `shots` - Pictures this sequence wrote, workspace-relative.
1517pub fn report(
1518 script: &Script,
1519 passes: &[(Pass, Option<Bite>)],
1520 verdict: &Verdict,
1521 shots: &[String],
1522)
1523 -> String
1524{
1525 let mut s = String::new();
1526 s.push_str(&fmt!("dev/{} — {} run{}, {}\n",
1527 script.file,
1528 passes.len(),
1529 match passes.len() { 1 => "", _ => "s" },
1530 script.prov.phrase()));
1531 s.push_str(
1532 "Run OUTSIDE the command fence, because these bytes are the commit's rather than a \
1533 command anybody's model wrote -- that was checked against the commit before each run, \
1534 and a verifier that differs from it is refused rather than reported. It reaches this \
1535 machine as any script you run yourself would.\n\n");
1536 for (p, what) in passes.iter() {
1537 match what {
1538 None => {
1539 s.push_str(&fmt!("CLEAN {} passed, {} failed, exit {}, {} ms\n",
1540 p.passed(), p.failed(), p.exit, p.ms));
1541 for f in p.fails.iter() {
1542 s.push_str(&fmt!(" {}\n", f));
1543 }
1544 },
1545 Some(Bite::Red { names }) => {
1546 s.push_str(&fmt!(
1547 "BREAK {:<10} {} passed, {} failed, exit {}, {} ms RED: {} went red\n",
1548 p.label, p.passed(), p.failed(), p.exit, p.ms, names.join("; ")));
1549 },
1550 Some(Bite::None { same, vanished }) => {
1551 let how = match (*same, *vanished) {
1552 (true, _) => fmt!("its output was the clean run's, its own banner aside"),
1553 (_, 0) => fmt!("the output moved, and no check that passed clean failed"),
1554 (_, n) => fmt!(
1555 "no check went red; {} checks that passed clean did not appear at all, \
1556 so it may have aborted rather than measured", n),
1557 };
1558 s.push_str(&fmt!(
1559 "BREAK {:<10} {} passed, {} failed, exit {}, {} ms PROVED NOTHING: {}\n",
1560 p.label, p.passed(), p.failed(), p.exit, p.ms, how));
1561 },
1562 Some(Bite::Unrun { why }) => {
1563 s.push_str(&fmt!(
1564 "BREAK {:<10} NEVER RAN: {}\n", p.label, why));
1565 },
1566 }
1567 }
1568 if !shots.is_empty() {
1569 s.push_str(&fmt!("\n{} pictures were written; read one with file_read and \"as\":\"image\", \
1570 or hand the path to a worker who can see:\n", shots.len()));
1571 for p in shots.iter() {
1572 s.push_str(&fmt!(" {}\n", p));
1573 }
1574 }
1575 s.push('\n');
1576 s.push_str(&verdict.advice());
1577 s.push('\n');
1578 s.push_str(&verdict.trailer());
1579 s.push('\n');
1580 if s.len() > REPORT_MAX {
1581 // The trailer is the one line that must survive, since it carries all three numbers
1582 // and the far end refuses a result without it.
1583 let tail = fmt!("\n[the report was cut here]\n{}\n", verdict.trailer());
1584 let room = REPORT_MAX.saturating_sub(tail.len());
1585 let mut cut = room;
1586 while cut > 0 && !s.is_char_boundary(cut) {
1587 cut -= 1;
1588 }
1589 s.truncate(cut);
1590 s.push_str(&tail);
1591 }
1592 s
1593}
1594
1595#[cfg(test)]
1596mod tests {
1597 use super::*;
1598
1599 use crate::wire::Breaks;
1600
1601 use std::fs;
1602
1603 /// Where the fixtures go.
1604 ///
1605 /// Under the home cache and never `/tmp`: that is a tmpfs here, its pages are
1606 /// charged to whoever wrote them, and filling it has taken this machine down
1607 /// before.
1608 ///
1609 /// # Arguments
1610 /// * `name` - Unique to the calling test, so tests do not share state.
1611 fn tree(name: &str) -> Outcome<PathBuf> {
1612 let home = match std::env::var("HOME") {
1613 Ok(h) => h,
1614 Err(e) => return Err(err!(
1615 "The verify tests need HOME to know where to put fixtures: {}", e;
1616 Test, Configuration)),
1617 };
1618 let root = PathBuf::from(home).join(".cache/daimond-hand-verify-tests").join(name);
1619 if root.exists() {
1620 res!(fs::remove_dir_all(&root).map_err(|e| err!(e, "clearing {:?}", root; Test, IO)));
1621 }
1622 res!(fs::create_dir_all(root.join(DEV_DIR))
1623 .map_err(|e| err!(e, "making {:?}", root; Test, IO)));
1624 Ok(root)
1625 }
1626
1627 /// Lays a verifier down in a fixture tree.
1628 fn put(root: &Path, name: &str, src: &str) -> Outcome<()> {
1629 let p = root.join(DEV_DIR).join(fmt!("{}{}{}", PREFIX, name, SUFFIX));
1630 res!(fs::write(&p, src).map_err(|e| err!(e, "writing {:?}", p; Test, IO)));
1631 Ok(())
1632 }
1633
1634 // ── The alphabet ────────────────────────────────────────────────
1635
1636 #[test]
1637 fn a_name_is_a_name_and_never_a_path() {
1638 assert!(name_ok("graph"));
1639 assert!(name_ok("a11y_aria"));
1640 assert!(name_ok("relay_e2e"));
1641 // Everything a path, a flag or a shell would be made of.
1642 for bad in [
1643 "",
1644 "dev/verify_graph.mjs",
1645 "../etc/passwd",
1646 "graph.mjs",
1647 "-rf",
1648 "graph;rm",
1649 "graph ",
1650 "Graph",
1651 "graph$(id)",
1652 "graph\nrm",
1653 ] {
1654 assert!(!name_ok(bad), "{:?} was accepted as a verifier name", bad);
1655 }
1656 assert!(!name_ok(&"a".repeat(NAME_MAX + 1)), "a name past the cap was accepted");
1657 }
1658
1659 // ── Nobody is at the keyboard ───────────────────────────────────
1660
1661 #[test]
1662 fn a_verifier_is_handed_no_display_however_the_hand_got_one() {
1663 // The hand as the browser starts it: a native messaging host inherits the
1664 // browser's environment, and the browser is on the owner's screen.
1665 let mine: BTreeMap<String, String> = [
1666 ("DISPLAY", ":0"),
1667 ("WAYLAND_DISPLAY", "wayland-0"),
1668 ("XDG_SESSION_TYPE", "wayland"),
1669 ("HOME", "/home/jason"),
1670 ("PATH", "/usr/bin"),
1671 ].iter().map(|(k, v)| (fmt!("{}", k), fmt!("{}", v))).collect();
1672 let out = unattended_env(&mine);
1673 for v in SEAT_VARS {
1674 assert!(!out.contains_key(*v),
1675 "{} reached a verifier, so a headed one could paint on the seat", v);
1676 }
1677 assert_eq!(Some(&fmt!("1")), out.get(UNATTENDED_VAR),
1678 "the run was not marked unattended, so dev/display.mjs would allow :0");
1679 // Everything else is untouched: a verifier with no HOME dies on the first
1680 // line of any script in dev/, which is B13's neighbour in the same file.
1681 assert_eq!(Some(&fmt!("/home/jason")), out.get("HOME"));
1682 assert_eq!(Some(&fmt!("/usr/bin")), out.get("PATH"));
1683 }
1684
1685 #[test]
1686 fn a_hand_with_no_display_still_says_nobody_is_looking() {
1687 // The mark is not conditional on there having been something to strip. An
1688 // absent DISPLAY and an unattended run are different sentences in
1689 // `dev/display.mjs`, and only the second one names the seat.
1690 let mine: BTreeMap<String, String> =
1691 [(fmt!("HOME"), fmt!("/home/jason"))].into_iter().collect();
1692 let out = unattended_env(&mine);
1693 assert_eq!(Some(&fmt!("1")), out.get(UNATTENDED_VAR));
1694 assert_eq!(2, out.len(), "something else was added: {:?}", out);
1695 }
1696
1697 #[test]
1698 fn a_verifier_cannot_ask_to_be_watched() {
1699 // The mark WINS over an inherited one. Nothing sets this name today, and a
1700 // day when something does is a day when the last word has to be the hand's.
1701 let mine: BTreeMap<String, String> = [
1702 (fmt!("{}", UNATTENDED_VAR), fmt!("")),
1703 (fmt!("DISPLAY"), fmt!(":0")),
1704 ].into_iter().collect();
1705 let out = unattended_env(&mine);
1706 assert_eq!(Some(&fmt!("1")), out.get(UNATTENDED_VAR),
1707 "an inherited empty mark survived, and an empty mark is read as absent");
1708 assert!(!out.contains_key("DISPLAY"));
1709 }
1710
1711 // ── What a verifier declares ────────────────────────────────────
1712
1713 #[test]
1714 fn the_breaks_come_out_of_the_files_own_source() {
1715 let src = "\
1716// EACH CHECK IS PROVED AGAINST BROKEN CODE FIRST. `--break <name>` serves a
1717// node dev/verify_about.mjs --break nobutton # 1: no About button
1718// node dev/verify_about.mjs --break oldbadge # 1: the badge is still there
1719// node dev/verify_about.mjs --break=stretch # 3: the splash is squashed
1720const BREAK = process.argv.indexOf('--break');
1721if (BREAK) console.log(`running with --break ${BREAK}`);
1722";
1723 assert_eq!(vec![fmt!("nobutton"), fmt!("oldbadge"), fmt!("stretch")], declared_breaks(src),
1724 "the placeholder, the quoted flag and the interpolation must not be taken as names");
1725 }
1726
1727 #[test]
1728 fn a_file_that_declares_nothing_declares_nothing() {
1729 assert!(declared_breaks("console.log(' ok nothing');").is_empty());
1730 // The flag mentioned but never given a value.
1731 assert!(declared_breaks("// node dev/verify_x.mjs --break <name>\n").is_empty());
1732 }
1733
1734 // ── Looking one up ──────────────────────────────────────────────
1735
1736 #[test]
1737 fn a_name_that_is_not_there_is_refused_with_the_near_misses() -> Outcome<()> {
1738 let root = res!(tree("absent"));
1739 res!(put(&root, "graph", "// node dev/verify_graph.mjs --break nolinks\n"));
1740 res!(put(&root, "graphedit", "//\n"));
1741 let e = match resolve(&root, "graphs") {
1742 Err(e) => e,
1743 Ok(s) => return Err(err!("'graphs' resolved to {:?}", s.file; Test)),
1744 };
1745 assert!(e.starts_with("Refused:"), "{}", e);
1746 assert!(e.contains("graph"), "the near misses are not named: {}", e);
1747 Ok(())
1748 }
1749
1750 #[test]
1751 fn a_path_is_refused_before_the_directory_is_read() -> Outcome<()> {
1752 let root = res!(tree("path"));
1753 for bad in ["../../etc/passwd", "dev/verify_graph.mjs", "graph.mjs"] {
1754 let e = match resolve(&root, bad) {
1755 Err(e) => e,
1756 Ok(_) => return Err(err!("{:?} resolved to something", bad; Test)),
1757 };
1758 assert!(e.contains("not a verifier name"), "{}", e);
1759 }
1760 Ok(())
1761 }
1762
1763 #[test]
1764 fn what_reaches_the_command_line_came_from_the_directory() -> Outcome<()> {
1765 let root = res!(tree("argv"));
1766 res!(put(&root, "graph", "// node dev/verify_graph.mjs --break nolinks\n"));
1767 let script = match resolve(&root, "graph") {
1768 Ok(s) => s,
1769 Err(e) => return Err(err!("{}", e; Test)),
1770 };
1771 let job = Job {
1772 id: fmt!("v1"),
1773 root: root.clone(),
1774 script: script.clone(),
1775 breaks: vec![fmt!("nolinks")],
1776 node: PathBuf::from("/usr/bin/node"),
1777 budget: Duration::from_millis(1000),
1778 ledger: Ledger::new(
1779 Arc::new(Mutex::new(res!(crate::journal::Journal::open(
1780 crate::journal::Cfg::at(root.join("journal")))))),
1781 Arc::new(AtomicBool::new(true))),
1782 };
1783 let argv = argv_for(&job, Some("nolinks"));
1784 assert_eq!(4, argv.len(), "{:?}", argv);
1785 assert_eq!(fmt!("{}", root.join(DEV_DIR).join("verify_graph.mjs").display()), argv[1]);
1786 assert_eq!(fmt!("--break"), argv[2]);
1787 assert_eq!(fmt!("nolinks"), argv[3]);
1788 Ok(())
1789 }
1790
1791 #[test]
1792 fn a_break_the_file_does_not_declare_is_refused_and_the_real_ones_are_listed()
1793 -> Outcome<()>
1794 {
1795 let root = res!(tree("undeclared"));
1796 res!(put(&root, "graph",
1797 "// node dev/verify_graph.mjs --break nolinks\n\
1798 // node dev/verify_graph.mjs --break stale\n"));
1799 let script = match resolve(&root, "graph") {
1800 Ok(s) => s,
1801 Err(e) => return Err(err!("{}", e; Test)),
1802 };
1803 assert_eq!(vec![fmt!("nolinks"), fmt!("stale")], script.breaks);
1804 let e = match chosen(&script, &Breaks::One(fmt!("invented"))) {
1805 Err(e) => e,
1806 Ok(v) => return Err(err!("an undeclared break was accepted: {:?}", v; Test)),
1807 };
1808 assert!(e.contains("nolinks") && e.contains("stale"),
1809 "the refusal does not list what it does declare: {}", e);
1810 // And the declared one is taken, as the file spells it.
1811 assert_eq!(Ok(vec![fmt!("stale")]), chosen(&script, &Breaks::One(fmt!("stale"))));
1812 assert_eq!(Ok(vec![fmt!("nolinks"), fmt!("stale")]), chosen(&script, &Breaks::All));
1813 assert_eq!(Ok(Vec::new()), chosen(&script, &Breaks::None));
1814 Ok(())
1815 }
1816
1817 #[test]
1818 fn a_verifier_with_no_breaks_cannot_be_asked_for_one() -> Outcome<()> {
1819 let root = res!(tree("nobreaks"));
1820 res!(put(&root, "bare", "console.log(' ok something');\n"));
1821 let script = match resolve(&root, "bare") {
1822 Ok(s) => s,
1823 Err(e) => return Err(err!("{}", e; Test)),
1824 };
1825 let e = match chosen(&script, &Breaks::One(fmt!("anything"))) {
1826 Err(e) => e,
1827 Ok(_) => return Err(err!("a break was accepted from a file with none"; Test)),
1828 };
1829 assert!(e.contains("declares no breaks"), "{}", e);
1830 Ok(())
1831 }
1832
1833 // ── Reading a run ───────────────────────────────────────────────
1834
1835 #[test]
1836 fn the_check_lines_are_read_and_the_detail_is_cut_off() {
1837 let out = "\
1838 ok the badge counts — 3 of 3
1839 FAIL the links are drawn — 0 found
1840 ok the badge counts — 4 of 4
1841";
1842 let c = parse_checks(out);
1843 assert_eq!(Some(&true), c.get("the badge counts"));
1844 assert_eq!(Some(&false), c.get("the links are drawn"));
1845 assert_eq!(2, c.len(), "{:?}", c);
1846 }
1847
1848 #[test]
1849 fn one_failure_beats_a_pass_of_the_same_name() {
1850 let c = parse_checks(" ok x\n FAIL x\n ok x\n");
1851 assert_eq!(Some(&false), c.get("x"), "a name that failed once must not read as passing");
1852 }
1853
1854 // ── What a break did ────────────────────────────────────────────
1855
1856 fn pass_of(label: &str, out: &str) -> Pass {
1857 Pass {
1858 label: fmt!("{}", label),
1859 exit: 0,
1860 timed: false,
1861 checks: parse_checks(out),
1862 norm: normalise(out),
1863 fails: Vec::new(),
1864 ms: 1,
1865 }
1866 }
1867
1868 #[test]
1869 fn a_break_that_reddens_a_check_is_red() {
1870 let clean = pass_of("clean", " ok a\n ok b\n");
1871 let brk = pass_of("nolinks", " ok a\n FAIL b\n");
1872 assert_eq!(Bite::Red { names: vec![fmt!("b")] }, bite(&clean, &brk));
1873 }
1874
1875 #[test]
1876 fn a_break_whose_output_is_the_clean_runs_proves_nothing_and_says_which() {
1877 let clean = pass_of("clean", " ok a\n ok b\n");
1878 let brk = pass_of("dead", "*** RUNNING UNDER --break dead ***\n ok a\n ok b\n");
1879 assert_eq!(Bite::None { same: true, vanished: 0 }, bite(&clean, &brk),
1880 "the break's own banner must not hide that nothing else changed");
1881 }
1882
1883 #[test]
1884 fn a_break_that_moves_the_output_and_reddens_nothing_still_proves_nothing() {
1885 let clean = pass_of("clean", " ok a\n ok b\nfound 3 links\n");
1886 let brk = pass_of("dead", " ok a\n ok b\nfound 9 links\n");
1887 assert_eq!(Bite::None { same: false, vanished: 0 }, bite(&clean, &brk),
1888 "a changed number is not a reddened check");
1889 }
1890
1891 #[test]
1892 fn a_check_already_failing_clean_says_nothing_about_a_break() {
1893 let clean = pass_of("clean", " ok a\n FAIL b\n");
1894 let brk = pass_of("dead", " ok a\n FAIL b\n");
1895 assert_eq!(Bite::None { same: true, vanished: 0 }, bite(&clean, &brk),
1896 "a break must not be credited with a failure that was already there");
1897 }
1898
1899 #[test]
1900 fn a_break_that_aborts_is_not_counted_red() {
1901 let clean = pass_of("clean", " ok a\n ok b\n ok c\n");
1902 let brk = pass_of("crash", " ok a\n");
1903 assert_eq!(Bite::None { same: false, vanished: 2 }, bite(&clean, &brk),
1904 "a run that stopped early has not shown a check able to fail");
1905 }
1906
1907 // ── The verdict, and the number that cannot be dropped ──────────
1908
1909 #[test]
1910 fn every_proven_trailer_carries_all_three_numbers() {
1911 let v = Verdict::Proven {
1912 passed: 27,
1913 failed: 0,
1914 red: vec![fmt!("nolinks"), fmt!("stale")],
1915 dead: vec![fmt!("nosc")],
1916 unrun: Vec::new(),
1917 };
1918 let t = v.trailer();
1919 assert!(t.starts_with(TRAILER), "{}", t);
1920 assert!(t.contains("27 checks passed"), "{}", t);
1921 assert!(t.contains("2 breaks confirmed red"), "{}", t);
1922 assert!(t.contains("1 breaks proved nothing"), "{}", t);
1923 assert_eq!(EXIT_UNPROVEN, v.exit(), "a dead break is not a clean bill of health");
1924 assert!(v.advice().contains("nosc"), "the dead break is not named: {}", v.advice());
1925 }
1926
1927 #[test]
1928 fn a_clean_only_run_says_it_proves_nothing_in_the_words_a_model_repeats() {
1929 let v = Verdict::Unproven {
1930 passed: 27,
1931 failed: 0,
1932 declared: vec![fmt!("nolinks"), fmt!("stale")],
1933 };
1934 let t = v.trailer();
1935 assert!(t.contains("NOT PROVEN"), "{}", t);
1936 assert!(t.contains("not evidence") || t.contains("NOT PROVEN"), "{}", t);
1937 assert_eq!(EXIT_UNPROVEN, v.exit());
1938 assert!(v.advice().contains("PROVES NOTHING"), "{}", v.advice());
1939 assert!(v.advice().contains("nolinks"), "the skipped breaks are not named");
1940 }
1941
1942 #[test]
1943 fn a_sequence_where_every_break_bit_is_the_only_clean_bill() {
1944 let v = Verdict::Proven {
1945 passed: 5,
1946 failed: 0,
1947 red: vec![fmt!("a"), fmt!("b")],
1948 dead: Vec::new(),
1949 unrun: Vec::new(),
1950 };
1951 assert_eq!(EXIT_PROVED, v.exit());
1952 assert!(v.trailer().contains("0 breaks proved nothing"), "{}", v.trailer());
1953 }
1954
1955 #[test]
1956 fn a_break_the_budget_never_reached_is_said_and_not_hidden() {
1957 let v = Verdict::Proven {
1958 passed: 5,
1959 failed: 0,
1960 red: vec![fmt!("a")],
1961 dead: Vec::new(),
1962 unrun: vec![fmt!("b")],
1963 };
1964 assert!(v.trailer().contains("1 breaks never ran"), "{}", v.trailer());
1965 assert_eq!(EXIT_UNPROVEN, v.exit());
1966 }
1967
1968 #[test]
1969 fn a_failing_clean_run_is_the_code_and_the_report_says_so() {
1970 let v = Verdict::Proven {
1971 passed: 20,
1972 failed: 3,
1973 red: vec![fmt!("a")],
1974 dead: Vec::new(),
1975 unrun: Vec::new(),
1976 };
1977 assert_eq!(EXIT_FAILED, v.exit());
1978 assert!(v.advice().contains("CLEAN run"), "{}", v.advice());
1979 }
1980
1981 // ── The report ──────────────────────────────────────────────────
1982
1983 #[test]
1984 fn the_report_always_ends_with_the_trailer() -> Outcome<()> {
1985 let root = res!(tree("report"));
1986 res!(put(&root, "graph", "// node dev/verify_graph.mjs --break nolinks\n"));
1987 let script = match resolve(&root, "graph") {
1988 Ok(s) => s,
1989 Err(e) => return Err(err!("{}", e; Test)),
1990 };
1991 let clean = pass_of("clean", " ok a\n ok b\n");
1992 let brk = pass_of("nolinks", " ok a\n FAIL b\n");
1993 let what = bite(&clean, &brk);
1994 let v = Verdict::Proven {
1995 passed: clean.passed(),
1996 failed: clean.failed(),
1997 red: vec![fmt!("nolinks")],
1998 dead: Vec::new(),
1999 unrun: Vec::new(),
2000 };
2001 let text = report(&script, &[(clean, None), (brk, Some(what))], &v, &[]);
2002 let last = match text.lines().filter(|l| !l.is_empty()).last() {
2003 Some(l) => l,
2004 None => return Err(err!("the report is empty"; Test)),
2005 };
2006 assert!(last.starts_with(TRAILER), "the last line is {:?}", last);
2007 assert!(text.contains("RED: b went red"), "{}", text);
2008 assert!(text.contains("OUTSIDE the command fence"),
2009 "the report does not say where it ran");
2010 Ok(())
2011 }
2012
2013 #[test]
2014 fn a_report_too_long_to_send_keeps_its_trailer() -> Outcome<()> {
2015 let root = res!(tree("long"));
2016 res!(put(&root, "big", "//\n"));
2017 let script = match resolve(&root, "big") {
2018 Ok(s) => s,
2019 Err(e) => return Err(err!("{}", e; Test)),
2020 };
2021 let mut clean = pass_of("clean", " ok a\n");
2022 clean.fails = (0..40_000).map(|i| fmt!("FAIL a very long check name number {}", i))
2023 .collect();
2024 let v = Verdict::Unproven { passed: 1, failed: 0, declared: Vec::new() };
2025 let text = report(&script, &[(clean, None)], &v, &[]);
2026 assert!(text.len() <= REPORT_MAX, "{} bytes", text.len());
2027 assert!(text.trim_end().ends_with(']'), "the trailer was cut off");
2028 assert!(text.contains(TRAILER), "the trailer is gone");
2029 assert!(text.contains("NOT PROVEN"), "the unproven label was cut off");
2030 Ok(())
2031 }
2032
2033 #[test]
2034 fn the_capability_says_whether_this_folder_has_verifiers_at_all() -> Outcome<()> {
2035 let bare = res!(tree("cap_bare"));
2036 assert_eq!(fmt!("verify:none"), cap(&bare));
2037 res!(put(&bare, "one", "//\n"));
2038 assert_eq!(fmt!("verify:dev"), cap(&bare));
2039 Ok(())
2040 }
2041
2042 // ── Whose code is this? ─────────────────────────────────────────
2043 //
2044 // `REVIEW.md` §1.21. A verifier runs OUTSIDE the command fence, and the
2045 // whole of the argument for that is that its bytes came with the checkout.
2046 // These tests aim at the argument rather than at the code: the fixture
2047 // verifier's entire body writes a file OUTSIDE the granted root, so the
2048 // marker's existence is the measurement. A provenance that may run gets
2049 // its marker; every other provenance must not.
2050
2051 /// Runs one git command in a fixture tree.
2052 ///
2053 /// # Arguments
2054 /// * `root` - The fixture.
2055 /// * `args` - The argument vector after `git -C <root>`.
2056 fn git(root: &Path, args: &[&str]) -> Outcome<()> {
2057 let git = res!(on_path("git").ok_or_else(|| err!(
2058 "These tests ask git what the repository holds, and there is no git on PATH.";
2059 Test, Missing)));
2060 let out = res!(std::process::Command::new(git)
2061 .arg("-C").arg(root)
2062 .args(args)
2063 .output()
2064 .map_err(|e| err!(e, "running git {:?}", args; Test, IO)));
2065 if !out.status.success() {
2066 return Err(err!("git {:?} failed in {:?}: {}",
2067 args, root, String::from_utf8_lossy(&out.stderr); Test, IO));
2068 }
2069 Ok(())
2070 }
2071
2072 /// A verifier whose whole body writes one file, at an absolute path.
2073 ///
2074 /// It prints a check line as well, so that a run which is allowed reports
2075 /// something a `Pass` can be built from and the fixture exercises the real
2076 /// path rather than an empty one.
2077 ///
2078 /// # Arguments
2079 /// * `marker` - Where it writes, which the tests place outside the root.
2080 fn escaper(marker: &Path) -> String {
2081 fmt!(
2082 "import fs from 'node:fs';\n\
2083 fs.writeFileSync({:?}, 'the verifier ran unfenced\\n');\n\
2084 console.log(' ok it ran');\n",
2085 fmt!("{}", marker.display()))
2086 }
2087
2088 /// What one attempt at a fixture verifier did.
2089 struct Attempt {
2090 refused: Option<String>, // the sentence, where the hand declined
2091 escaped: bool, // whether the marker outside the root appeared
2092 }
2093
2094 /// Resolves and conducts a fixture verifier, and says what came of it.
2095 ///
2096 /// Deliberately the crate's own public API and not the dispatcher's: a gate
2097 /// that lives in `main.rs` is a gate this function would walk past, and the
2098 /// reproduction in `REVIEW.md` §1.21 walked past exactly that.
2099 ///
2100 /// # Arguments
2101 /// * `root` - The fixture.
2102 /// * `name` - The verifier's short name.
2103 /// * `marker` - The path the verifier writes, outside `root`.
2104 async fn attempt(root: &Path, name: &str, marker: &Path) -> Outcome<Attempt> {
2105 if marker.exists() {
2106 res!(fs::remove_file(marker).map_err(|e| err!(e, "clearing {:?}", marker; Test, IO)));
2107 }
2108 let node = res!(on_path("node").ok_or_else(|| err!(
2109 "Every verifier is a Node script and there is no node on PATH, so this test would \
2110 prove nothing about whether one ran.";
2111 Test, Missing)));
2112 let script = match resolve(root, name) {
2113 Ok(s) => s,
2114 Err(w) => return Ok(Attempt { refused: Some(w), escaped: marker.exists() }),
2115 };
2116 let stem = match root.file_name().and_then(|n| n.to_str()) {
2117 Some(n) => fmt!("{}", n),
2118 None => fmt!("{}", name),
2119 };
2120 let jdir = match root.parent() {
2121 Some(p) => p.join(fmt!("{}-journal", stem)),
2122 None => root.join("journal"),
2123 };
2124 if jdir.exists() {
2125 res!(fs::remove_dir_all(&jdir).map_err(|e| err!(e, "clearing {:?}", jdir; Test, IO)));
2126 }
2127 let jr = res!(Journal::open(crate::journal::Cfg::at(&jdir)));
2128 let job = Job {
2129 id: fmt!("probe"),
2130 root: root.to_path_buf(),
2131 script,
2132 breaks: Vec::new(),
2133 node,
2134 budget: Duration::from_millis(60_000),
2135 ledger: Ledger::new(Arc::new(Mutex::new(jr)), Arc::new(AtomicBool::new(true))),
2136 };
2137 let (tx, mut rx) = tokio::sync::mpsc::channel(64);
2138 // Boxed, so the whole sequence's future lives on the heap. Inlined, `conduct`
2139 // holds `once` holds `drain`'s buffers, and a debug build of that overflows a
2140 // test thread's stack before the first process starts.
2141 res!(Box::pin(conduct(job, tx)).await);
2142 let mut refused = None;
2143 while let Some(r) = rx.recv().await {
2144 if let Resp::Refused { reason, .. } = r {
2145 refused = Some(reason);
2146 }
2147 }
2148 Ok(Attempt { refused, escaped: marker.exists() })
2149 }
2150
2151 #[tokio::test]
2152 async fn a_verifier_the_checkout_brought_runs_outside_the_fence() -> Outcome<()> {
2153 let root = res!(tree("prov_committed"));
2154 let marker = root.join("..").join("prov_committed.escaped");
2155 let marker = PathBuf::from(fmt!("{}", marker.display()));
2156 res!(git(&root, &["init", "-q"]));
2157 res!(put(&root, "probe", &escaper(&marker)));
2158 res!(git(&root, &["add", "dev/verify_probe.mjs"]));
2159 res!(git(&root, &["-c", "user.name=T", "-c", "user.email=t@t",
2160 "commit", "-q", "--no-verify", "-m", "probe"]));
2161 let a = res!(attempt(&root, "probe", &marker).await);
2162 assert!(a.refused.is_none(), "the committed case must run: {:?}", a.refused);
2163 // The half that keeps this pair honest. Without it a guard that refused
2164 // everything would leave the three tests below green while making the verb
2165 // useless, which is the failure `REVIEW.md` was written about.
2166 assert!(a.escaped,
2167 "a verifier that IS the commit runs outside the fence, and this one wrote nothing \
2168 outside the root -- so the three tests below prove nothing about a fence");
2169 Ok(())
2170 }
2171
2172 /// The verb, over this very tree, driven by hand.
2173 ///
2174 /// `#[ignore]`d, so it is never part of a suite: it runs a real verifier from
2175 /// the repository this crate sits in, which starts browsers and takes minutes.
2176 /// It is here because the fixtures above cannot find what it finds. Run over
2177 /// `verify_reflux` on 2026-08-25 it turned up two defects in a file that had
2178 /// passed standalone twelve times -- a break declared by a sentence that named
2179 /// ANOTHER file's break, and an X server killed hard enough to keep its lock --
2180 /// and neither is visible from a fixture, because a fixture has no header prose
2181 /// and starts no X server.
2182 ///
2183 /// LIVE=reflux cargo test --manifest-path hand/Cargo.toml --lib \
2184 /// verify::tests::live_walk -- --ignored --nocapture
2185 #[tokio::test]
2186 #[ignore]
2187 async fn live_walk_over_this_very_tree() -> Outcome<()> {
2188 let root = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("..");
2189 let name = std::env::var("LIVE").unwrap_or(fmt!("harness"));
2190 let script = match resolve(&root, &name) {
2191 Ok(s) => s,
2192 Err(w) => { println!("REFUSED: {}", w); return Ok(()); },
2193 };
2194 println!("script {:?}\nprov {}\nbreaks {:?}",
2195 script.path, script.prov.phrase(), script.breaks);
2196 let node = res!(on_path("node").ok_or_else(|| err!(
2197 "Every verifier is a Node script and there is no node on PATH."; Test, Missing)));
2198 let jdir = res!(tree("live_walk")).join("journal");
2199 let jr = res!(Journal::open(crate::journal::Cfg::at(&jdir)));
2200 let job = Job {
2201 id: fmt!("live"), root: root.clone(), breaks: script.breaks.clone(), script,
2202 node, budget: Duration::from_millis(1_800_000),
2203 ledger: Ledger::new(Arc::new(Mutex::new(jr)), Arc::new(AtomicBool::new(true))),
2204 };
2205 let (tx, mut rx) = tokio::sync::mpsc::channel(64);
2206 res!(Box::pin(conduct(job, tx)).await);
2207 while let Some(r) = rx.recv().await {
2208 match r {
2209 Resp::Chunk { data, .. } => println!("{}", data),
2210 Resp::Refused { reason, .. } => println!("REFUSED: {}", reason),
2211 Resp::Ended { exit, .. } => println!("exit {}", exit),
2212 _ => {},
2213 }
2214 }
2215 Ok(())
2216 }
2217
2218 #[tokio::test]
2219 async fn the_spawn_itself_hands_on_no_display() -> Outcome<()> {
2220 // The pure function above can be right and never called, which is this
2221 // repository's own recorded failure shape -- `dev/BLOCKERS.md` keeps a list
2222 // of things built and unreachable. So this one goes through `conduct`,
2223 // `once` and a real `execve`, and asks the child what it was actually given.
2224 let root = res!(tree("prov_seat"));
2225 let marker = root.join("..").join("prov_seat.saw");
2226 let marker = PathBuf::from(fmt!("{}", marker.display()));
2227 res!(git(&root, &["init", "-q"]));
2228 res!(put(&root, "probe", &fmt!(
2229 "import fs from 'node:fs';\n\
2230 const e = process.env;\n\
2231 fs.writeFileSync({:?}, `DISPLAY=${{e.DISPLAY ?? '<unset>'}} \
2232 WAYLAND_DISPLAY=${{e.WAYLAND_DISPLAY ?? '<unset>'}} \
2233 DAIMOND_UNATTENDED=${{e.DAIMOND_UNATTENDED ?? '<unset>'}} \
2234 HOME=${{e.HOME ? 'set' : '<unset>'}}`);\n\
2235 console.log(' ok it ran');\n",
2236 fmt!("{}", marker.display()))));
2237 res!(git(&root, &["add", "dev/verify_probe.mjs"]));
2238 res!(git(&root, &["-c", "user.name=T", "-c", "user.email=t@t",
2239 "commit", "-q", "--no-verify", "-m", "probe"]));
2240 let mine = std::env::var("DISPLAY").unwrap_or_default();
2241 let a = res!(attempt(&root, "probe", &marker).await);
2242 assert!(a.refused.is_none(), "the committed case must run: {:?}", a.refused);
2243 let saw = res!(fs::read_to_string(&marker)
2244 .map_err(|e| err!(e, "the probe wrote nothing at {:?}", marker; Test, IO)));
2245 assert!(saw.contains("DISPLAY=<unset>"),
2246 "a verifier was handed a display. This process holds DISPLAY={:?}, and what \
2247 reached the child was: {}", mine, saw);
2248 assert!(saw.contains("WAYLAND_DISPLAY=<unset>"),
2249 "a verifier was handed the compositor, which Chromium prefers over DISPLAY \
2250 and which is the owner's own screen: {}", saw);
2251 assert!(saw.contains("DAIMOND_UNATTENDED=1"),
2252 "the run was not marked unattended, so dev/display.mjs would allow :0: {}", saw);
2253 // The half that keeps the three above honest: an environment cleared and not
2254 // rebuilt would satisfy every one of them and break every verifier in dev/.
2255 assert!(saw.contains("HOME=set"),
2256 "the environment was cleared and not rebuilt, so every script under dev/ \
2257 dies on its first line: {}", saw);
2258 Ok(())
2259 }
2260
2261 #[tokio::test]
2262 async fn a_verifier_git_has_never_heard_of_does_not_run() -> Outcome<()> {
2263 let root = res!(tree("prov_untracked"));
2264 let marker = PathBuf::from(fmt!("{}", root.join("..").join("prov_untracked.escaped").display()));
2265 res!(git(&root, &["init", "-q"]));
2266 res!(put(&root, "probe", &escaper(&marker)));
2267 let a = res!(attempt(&root, "probe", &marker).await);
2268 assert!(!a.escaped, "an untracked verifier ran and wrote outside the granted root");
2269 let no = res!(a.refused.ok_or_else(|| err!(
2270 "nothing was refused, so the run happened"; Test, Invalid)));
2271 assert!(no.contains("dev/verify_probe.mjs"), "the refusal must name the file: {}", no);
2272 Ok(())
2273 }
2274
2275 #[tokio::test]
2276 async fn a_tracked_verifier_the_model_has_edited_does_not_run() -> Outcome<()> {
2277 let root = res!(tree("prov_edited"));
2278 let marker = PathBuf::from(fmt!("{}", root.join("..").join("prov_edited.escaped").display()));
2279 res!(git(&root, &["init", "-q"]));
2280 res!(put(&root, "probe", "console.log(' ok it ran');\n"));
2281 res!(git(&root, &["add", "dev/verify_probe.mjs"]));
2282 res!(git(&root, &["-c", "user.name=T", "-c", "user.email=t@t",
2283 "commit", "-q", "--no-verify", "-m", "probe"]));
2284 // Now the model edits it. `git ls-files --error-unmatch` goes on saying yes.
2285 res!(put(&root, "probe", &escaper(&marker)));
2286 let a = res!(attempt(&root, "probe", &marker).await);
2287 assert!(!a.escaped, "an edited verifier ran and wrote outside the granted root");
2288 assert!(a.refused.is_some(), "nothing was refused, so the run happened");
2289 // And the whole of §1.21's second half: staging it is not committing it.
2290 res!(git(&root, &["add", "dev/verify_probe.mjs"]));
2291 let b = res!(attempt(&root, "probe", &marker).await);
2292 assert!(!b.escaped,
2293 "'git add' bought the model an unfenced run, so the check is about the index and \
2294 not about the bytes");
2295 assert!(b.refused.is_some(), "a staged edit was accepted");
2296 Ok(())
2297 }
2298
2299 #[tokio::test]
2300 async fn a_folder_with_no_git_in_it_cannot_vouch_for_anything() -> Outcome<()> {
2301 let root = res!(tree("prov_nogit"));
2302 let marker = PathBuf::from(fmt!("{}", root.join("..").join("prov_nogit.escaped").display()));
2303 res!(put(&root, "probe", &escaper(&marker)));
2304 let a = res!(attempt(&root, "probe", &marker).await);
2305 assert!(!a.escaped,
2306 "a granted folder with no repository in it ran a verifier outside the fence, and \
2307 nothing there could have said whose code it was");
2308 assert!(a.refused.is_some(), "nothing was refused, so the run happened");
2309 Ok(())
2310 }
2311
2312 #[tokio::test]
2313 async fn the_refusal_hands_the_daimon_the_fenced_route() -> Outcome<()> {
2314 let root = res!(tree("prov_sentence"));
2315 let marker = PathBuf::from(fmt!("{}", root.join("..").join("prov_sentence.escaped").display()));
2316 res!(git(&root, &["init", "-q"]));
2317 res!(put(&root, "probe", &escaper(&marker)));
2318 let a = res!(attempt(&root, "probe", &marker).await);
2319 let no = res!(a.refused.ok_or_else(|| err!("nothing was refused"; Test, Invalid)));
2320 // A refusal a model cannot converge on costs the run anyway. Each of these is a
2321 // thing the daimon must be able to DO next, and the sentence is asserted for
2322 // meaning rather than for length.
2323 // The argv form and not a command line, because that is what 'run' takes: a
2324 // refusal that hands a model a shell string teaches it the one spelling the
2325 // tool refuses.
2326 assert!(no.contains(r#"["node","dev/verify_probe.mjs"]"#),
2327 "the refusal must hand over the argv that runs it fenced: {}", no);
2328 assert!(no.contains("'run'"),
2329 "the refusal must name the tool that command goes to: {}", no);
2330 assert!(no.contains("commit"),
2331 "the refusal must say what makes the unfenced run available: {}", no);
2332 assert!(no.contains("browser"),
2333 "the refusal must say what the fenced route cannot do, or a daimon will read the \
2334 fenced run's failure as the verifier being broken: {}", no);
2335 Ok(())
2336 }
2337
2338 #[test]
2339 fn a_verifier_that_is_a_symlink_is_not_the_commit_whatever_git_says() -> Outcome<()> {
2340 let root = res!(tree("prov_link"));
2341 res!(git(&root, &["init", "-q"]));
2342 // The target is an ordinary file inside the granted root, which every fenced
2343 // command may rewrite. Committing the LINK commits the name and not the bytes.
2344 let target = root.join("notes.mjs");
2345 res!(fs::write(&target, "console.log(' ok it ran');\n")
2346 .map_err(|e| err!(e, "writing {:?}", target; Test, IO)));
2347 #[cfg(unix)]
2348 res!(std::os::unix::fs::symlink("../notes.mjs",
2349 root.join(DEV_DIR).join("verify_probe.mjs"))
2350 .map_err(|e| err!(e, "linking"; Test, IO)));
2351 res!(git(&root, &["add", "-A"]));
2352 res!(git(&root, &["-c", "user.name=T", "-c", "user.email=t@t",
2353 "commit", "-q", "--no-verify", "-m", "probe"]));
2354 assert_eq!(Provenance::Changed(Change::Linked),
2355 provenance(&root, "verify_probe.mjs"),
2356 "a committed symlink read as the commit's own bytes");
2357 let no = res!(unfenced_refusal(&root, "verify_probe.mjs").ok_or_else(|| err!(
2358 "a committed symlink was allowed to run unfenced"; Test, Invalid)));
2359 assert!(no.contains("symbolic link"), "{}", no);
2360 Ok(())
2361 }
2362
2363 #[test]
2364 fn membership_is_not_content_and_this_is_where_the_two_part() -> Outcome<()> {
2365 let root = res!(tree("prov_parts"));
2366 res!(git(&root, &["init", "-q"]));
2367 res!(put(&root, "probe", "console.log(' ok one');\n"));
2368 res!(git(&root, &["add", "dev/verify_probe.mjs"]));
2369 res!(git(&root, &["-c", "user.name=T", "-c", "user.email=t@t",
2370 "commit", "-q", "--no-verify", "-m", "probe"]));
2371 assert_eq!(Provenance::Committed, provenance(&root, "verify_probe.mjs"));
2372 assert!(unfenced_refusal(&root, "verify_probe.mjs").is_none());
2373
2374 // One appended comment. This is the case `REVIEW.md` §1.21 turns on: the index
2375 // goes on saying yes, and the bytes are the model's.
2376 res!(put(&root, "probe", "console.log(' ok one');\n// and now mine\n"));
2377 let git_path = res!(on_path("git").ok_or_else(|| err!("no git"; Test, Missing)));
2378 let listed = res!(std::process::Command::new(&git_path)
2379 .arg("-C").arg(&root)
2380 .args(["ls-files", "--error-unmatch", "--", "dev/verify_probe.mjs"])
2381 .stdout(Stdio::null()).stderr(Stdio::null())
2382 .status().map_err(|e| err!(e, "ls-files"; Test, IO)));
2383 assert!(listed.success(),
2384 "the premise of this test is that git goes on naming an edited file");
2385 assert_eq!(Provenance::Changed(Change::Edited), provenance(&root, "verify_probe.mjs"),
2386 "the check followed the index rather than the bytes");
2387 Ok(())
2388 }
2389
2390 #[test]
2391 fn the_report_says_whose_bytes_it_ran() -> Outcome<()> {
2392 let script = Script {
2393 name: fmt!("probe"),
2394 file: fmt!("verify_probe.mjs"),
2395 path: PathBuf::from("/w/dev/verify_probe.mjs"),
2396 breaks: Vec::new(),
2397 prov: Provenance::Committed,
2398 };
2399 let clean = pass_of("clean", " ok one\n");
2400 let txt = report(&script, &[(clean, None)],
2401 &Verdict::Unproven { passed: 1, failed: 0, declared: Vec::new() }, &[]);
2402 let first = res!(txt.lines().next().ok_or_else(|| err!("empty report"; Test, Invalid)));
2403 assert!(first.contains("byte for byte the commit's"),
2404 "the report's first line must say whose bytes ran: {}", first);
2405 Ok(())
2406 }
2407
2408 #[tokio::test]
2409 async fn a_verifier_that_rewrites_itself_gets_no_second_run() -> Outcome<()> {
2410 let root = res!(tree("prov_swap"));
2411 // Committed, and what it does when it runs is append to its own source. So the
2412 // clean run is the commit's and the break run is not, which is the race the
2413 // re-ask before every spawn exists for: no test can hold a file still for the
2414 // minutes a real sequence takes, and a verifier that changes itself is the same
2415 // event arriving on schedule.
2416 let src = "// node dev/verify_probe.mjs --break swap # 1: the break\n\
2417 import fs from 'node:fs';\n\
2418 console.log(' ok it ran');\n\
2419 fs.appendFileSync('dev/verify_probe.mjs', '// and now it is mine\\n');\n";
2420 res!(git(&root, &["init", "-q"]));
2421 res!(put(&root, "probe", src));
2422 res!(git(&root, &["add", "dev/verify_probe.mjs"]));
2423 res!(git(&root, &["-c", "user.name=T", "-c", "user.email=t@t",
2424 "commit", "-q", "--no-verify", "-m", "probe"]));
2425 let script = res!(resolve(&root, "probe").map_err(|w| err!("{}", w; Test, Invalid)));
2426 assert_eq!(vec![fmt!("swap")], script.breaks);
2427 assert_eq!(Provenance::Committed, script.prov);
2428 let node = res!(on_path("node").ok_or_else(|| err!("no node"; Test, Missing)));
2429 let jdir = root.join("..").join("prov_swap-journal");
2430 let jdir = PathBuf::from(fmt!("{}", jdir.display()));
2431 if jdir.exists() {
2432 res!(fs::remove_dir_all(&jdir).map_err(|e| err!(e, "clearing"; Test, IO)));
2433 }
2434 let jr = res!(Journal::open(crate::journal::Cfg::at(&jdir)));
2435 let job = Job {
2436 id: fmt!("swap"),
2437 root: root.clone(),
2438 script,
2439 breaks: vec![fmt!("swap")],
2440 node,
2441 budget: Duration::from_millis(60_000),
2442 ledger: Ledger::new(Arc::new(Mutex::new(jr)), Arc::new(AtomicBool::new(true))),
2443 };
2444 let (tx, mut rx) = tokio::sync::mpsc::channel(64);
2445 res!(Box::pin(conduct(job, tx)).await);
2446 let mut out = String::new();
2447 while let Some(r) = rx.recv().await {
2448 if let Resp::Chunk { stream: Stream::Out, data, .. } = r {
2449 out.push_str(&data);
2450 }
2451 }
2452 assert!(out.contains("BREAK swap"), "the break is not in the report: {}", out);
2453 assert!(out.contains("NEVER RAN"),
2454 "the break ran against a file the clean run had rewritten: {}", out);
2455 assert!(out.contains("stopped being the commit's"),
2456 "the report does not say why the break never ran: {}", out);
2457 Ok(())
2458 }
2459}