oxedyne/daimond/dev/verify_conformance.mjs
101 KiB, 1 run
created by r2519314175:307, which is this file's identity for as long as the history lasts, whatever it is later renamed to
download · who wrote it · its history
| 1 | // verify_conformance.mjs — does the live Oregami forge answer the Improve-panel |
| 2 | // contract, on the repository the product actually names? |
| 3 | // |
| 4 | // THE AUTHORITY IS NOT THIS FILE. It is |
| 5 | // ~/usr/complement/projects/oxedyne/projects/ore/doc/design/improve_panel_contract.md |
| 6 | // and where the two disagree, the contract is right and this file is a bug. Every |
| 7 | // check below carries the sentence it rests on, printed beside it when it fails and |
| 8 | // listed in full by `--cites`. |
| 9 | // |
| 10 | // ── WHY IT EXISTS ──────────────────────────────────────────────────────────────── |
| 11 | // |
| 12 | // A conformance run against the live forge was performed on 2026-08-13 and reported |
| 13 | // "31 passed, 0 failed". THE SCRIPT WAS NEVER COMMITTED. It is in no tree, so the |
| 14 | // number has been quoted forward through three handovers as though a standing |
| 15 | // instrument produced it, and nobody since has been able to re-run it, read what it |
| 16 | // asserted, or point at what it was aimed at. A result nobody can reproduce is a |
| 17 | // claim, not evidence. This file is the instrument those numbers claimed. |
| 18 | // |
| 19 | // It is also the instrument the same day proved was needed. Contract §3.3: the panel |
| 20 | // had never worked in production, the forge held three repositories and the panel's |
| 21 | // was not one of them, every route answered `absent` correctly, and thirty-one checks |
| 22 | // stayed green — because they were pointed somewhere else. |
| 23 | // |
| 24 | // ── THE SUBJECT RULE, §3.3, WHICH IS WHY THIS FILE IS SHAPED AS IT IS ──────────── |
| 25 | // |
| 26 | // "So the rule: the suite takes its subject from the client's own configured |
| 27 | // constant, and pointing it anywhere else is an explicit act. A subject supplied |
| 28 | // on the command line will be pointed at whatever passes, and it will be pointed |
| 29 | // there by the person with the most reason to want a green result." |
| 30 | // |
| 31 | // So THERE IS NO `--account` AND NO `--repo`. The subject is read out of |
| 32 | // `www/js/improve.js`, from the same two lines the browser reads it from, and if |
| 33 | // those lines cannot be found this file exits 2 rather than guessing. Rename the |
| 34 | // constant in the client and this instrument stops, loudly, instead of quietly |
| 35 | // measuring something else. |
| 36 | // |
| 37 | // ── THE SPLIT, ruled by the Oregami session 2026-08-15 (§3.3) ──────────────────── |
| 38 | // |
| 39 | // "Reads take the constant; writes take a sandbox derived from it. ... the read |
| 40 | // checks resolve the client's constant and assert it answers; the success paths of |
| 41 | // the write checks go to a sandbox; and the write checks aimed at the product's |
| 42 | // repository are the refusal paths, which resolve the route and the credential |
| 43 | // rule and change no state. The sandbox is a second compiled-in constant and never |
| 44 | // a command-line argument, for the reason the subject is not one." |
| 45 | // |
| 46 | // READ checks -> the client's constant. A read leaves nothing behind, and |
| 47 | // SOME check has to ask about the repository the product |
| 48 | // actually names. That is the check the 31 never made. |
| 49 | // WRITE checks that pass -> `oxedyne/conformance`, the sandbox. A second compiled-in |
| 50 | // constant, below, for the same reason the subject is not |
| 51 | // an argument. |
| 52 | // WRITE checks aimed at -> the REFUSAL paths only: bad credential, credential in |
| 53 | // the client's constant the body, absent `d`, wrong role. They resolve the route |
| 54 | // and exercise the credential rule against the real target |
| 55 | // and change no state. |
| 56 | // |
| 57 | // The reason for the split, and it is worth keeping in view: a proposal is a file |
| 58 | // beside the repository rather than an Ore operation (§8), so a probe write COULD be |
| 59 | // removed afterwards — and a backlog that has been tidied is worse evidence than one |
| 60 | // that was never polluted. |
| 61 | // |
| 62 | // The refusal probes are still writes, and a forge that wrongly accepted one would |
| 63 | // leave a record. So every probe body this file sends to the subject says so in its |
| 64 | // own title: if one ever appears in the backlog, it names the fault that put it |
| 65 | // there rather than looking like a tester's report. |
| 66 | // |
| 67 | // ── WHAT IT TALKS TO ───────────────────────────────────────────────────────────── |
| 68 | // |
| 69 | // THE FORGE, DIRECTLY, over loopback — not `/api/improve`. §3's routes are the |
| 70 | // forge's, and the gateway in front of them answers 401 to anything without a |
| 71 | // Daimond session (§3, "A public read needs no voice, but it does need a Daimond |
| 72 | // session"), which would stand between this instrument and its subject. So this file |
| 73 | // says NOTHING about the gateway half of the path; that is `dev/verify_improve.mjs`'s |
| 74 | // lane, and neither run substitutes for the other. |
| 75 | // |
| 76 | // Every request is built and read by hand over a socket (`node:net` / `node:tls`) |
| 77 | // rather than through `fetch`. Not fussiness: undici NORMALISES header values, so a |
| 78 | // credential of three spaces reaches the wire as an empty one, and three of §3.1's |
| 79 | // credential spellings would silently become one probe repeated. An instrument that |
| 80 | // cannot put the bytes on the wire must not report as though it did. The same client |
| 81 | // is what lets `?format=json&x=%zz` be sent literally, which §3.2's raw-query rule |
| 82 | // cannot be tested without. |
| 83 | // |
| 84 | // ── WHAT THE ORCHESTRATOR MUST SUPPLY ──────────────────────────────────────────── |
| 85 | // |
| 86 | // ORE_FORGE Base URL of the forge, e.g. `http://127.0.0.1:8430`. Optional: |
| 87 | // without it the default is read from `gateway/app.jdat`'s |
| 88 | // `forge_host` and `forge_port`, which is where the gateway |
| 89 | // itself reads the forge's address from. |
| 90 | // ORE_VOICE A secret for a voice provisioned on the SUBJECT repository with |
| 91 | // at least `pull`. Read-only in effect: it is used for the voiced |
| 92 | // READ checks and for refusal probes whose ordering is only |
| 93 | // defined once the credential is good. It never makes a write to |
| 94 | // the subject that could succeed. |
| 95 | // ORE_VOICE_READER A secret for a voice on the SUBJECT repository whose role is |
| 96 | // BELOW `pull`. Without it `unpermitted` cannot be provoked and |
| 97 | // that check is skipped rather than faked. |
| 98 | // ORE_VOICE_SANDBOX A secret for a voice with at least `pull` on the SANDBOX |
| 99 | // repository. Without it every write-success check is skipped. |
| 100 | // |
| 101 | // A secret may instead sit in a file, one `NAME=value` per line, `#` starting a |
| 102 | // comment, passed as `--voices <path>`; the environment wins over the file. NO |
| 103 | // CREDENTIAL IS WRITTEN IN THIS FILE, as a default or otherwise (CLAUDE.md, and the |
| 104 | // leak of 2026-07-10). Nothing this file prints carries a secret either: every line |
| 105 | // of output goes through `redact()`. |
| 106 | // |
| 107 | // Each supplied voice is CHECKED FOR RECOGNITION before it is relied on, by a read |
| 108 | // that must not answer `unknown`. A credential the forge does not know would turn |
| 109 | // every check that used it red for a reason that is not the forge's — which §3.3 |
| 110 | // names as the failure that cost fourteen checks in one run. |
| 111 | // |
| 112 | // ORE_FORGE=http://127.0.0.1:8430 ORE_VOICE=... node dev/verify_conformance.mjs |
| 113 | // node dev/verify_conformance.mjs --voices /run/ore/voices.env |
| 114 | // node dev/verify_conformance.mjs --cites # every check and its sentence |
| 115 | // node dev/verify_conformance.mjs --why-not # the reason it cannot run, or silence |
| 116 | // |
| 117 | // ── STANDING ONE UP LOCALLY, WHICH IS WRITTEN DOWN BECAUSE IT WAS NOT ──────────── |
| 118 | // |
| 119 | // Run with nothing prepared, this file dies at A1 on `ECONNREFUSED 127.0.0.1:8430`, |
| 120 | // and three sessions in a row read that as "unrunnable". It is not: it is a forge |
| 121 | // that is not up, and the whole of §3.3's argument is that a result nobody can |
| 122 | // reproduce is a claim rather than evidence. So, once, in full. `oregami` is at |
| 123 | // `~/usr/code/web/apps/oxedyne/oregami`; `$B` is its binary. |
| 124 | // |
| 125 | // ./server start # the forge, on :8430 |
| 126 | // ./server stop # `voice` and `create` will not |
| 127 | // # run while `serve` holds the |
| 128 | // # records, and say so |
| 129 | // $ORE init # anywhere; prints a signing key |
| 130 | // $B create oxedyne/daimond --owner <key> --public --app . |
| 131 | // $B voice oxedyne/daimond <person> pull --app . # -> ORE_VOICE |
| 132 | // $B voice oxedyne/conformance <person> pull --app . # -> ORE_VOICE_SANDBOX |
| 133 | // ./server start |
| 134 | // |
| 135 | // `oxedyne/daimond` is NAMED in the forge's `app.jdat` and is not on disk until |
| 136 | // somebody creates it, which is the 2026-08-14 fault in miniature: the panel names |
| 137 | // a repository the forge does not serve, A1 says so, and every check aimed |
| 138 | // elsewhere still passes. A1 going red on a bare local forge is this file working. |
| 139 | // |
| 140 | // THE SUBJECT MUST THEN HOLD A FEW PROPOSALS or a dozen read checks skip and B4 |
| 141 | // fails on nothing to read: `POST /oxedyne/daimond/proposals` with the subject |
| 142 | // voice, three times, is enough. Those are LOCAL FIXTURE proposals in a LOCAL |
| 143 | // repository; the §3.3 rule against writing to the subject is about the product's |
| 144 | // real backlog on the live forge, and nothing here may be run against that. |
| 145 | // |
| 146 | // Measured on 2026-08-17 with exactly the above: 59 passed, 0 failed, 12 skipped. |
| 147 | // |
| 148 | // ORE_VOICE_READER CANNOT BE PROVISIONED ON THIS FORGE, so E13 is a permanent skip |
| 149 | // rather than a fixture somebody should go and build. It wants a voice the forge |
| 150 | // recognises whose role is BELOW `pull`, and oregami's ladder is Pull, Push, Admin |
| 151 | // (`oregami/src/voice.rs`, `Role::covers`) -- `pull` IS the floor, so there is no |
| 152 | // such role to grant. Provoking `unpermitted` needs the forge to grow one, or a |
| 153 | // route with a higher floor than the vote's. Said here so the next reader does not |
| 154 | // spend an afternoon looking for the flag. |
| 155 | // |
| 156 | // G6 READ BY EYE, once, since a throttle finally arrived: the sentence is "Too many |
| 157 | // requests from this voice. Wait, then try again." It names the voice and does not |
| 158 | // name which allowance was spent, which is what §3.1 requires. Recorded rather than |
| 159 | // turned into a check, because the check would have to invent the word list it |
| 160 | // searched for -- §3.3's third and worst shape. |
| 161 | // |
| 162 | // ── EVERY CHECK IS PROVED RED, AND THE MEANS SHIPS IN THIS FILE ────────────────── |
| 163 | // |
| 164 | // A CHECK THAT WILL NOT GO RED IS A FINDING, NOT A FIXTURE PROBLEM. |
| 165 | // |
| 166 | // Chase it until it is explained. A break that stays green usually means a different |
| 167 | // rule is quietly holding the property and the check under test never ran at all. |
| 168 | // |
| 169 | // A suite that cites a sentence per check and never reddens is indistinguishable from |
| 170 | // a suite that works, which is the failure one level above the one this file exists to |
| 171 | // fix: "31 passed" was a true sentence about the wrong repository and nobody could |
| 172 | // tell. So `--break <name>` corrupts THE ANSWER between the socket and the assertions |
| 173 | // — the response, not the source, because the forge is not ours to damage and the |
| 174 | // thing under test here is what this file does with what it is handed. |
| 175 | // |
| 176 | // node dev/verify_conformance.mjs --break wrongstatus # -> G3 |
| 177 | // node dev/verify_conformance.mjs --break nokey # -> A2 |
| 178 | // node dev/verify_conformance.mjs --break novotes # -> A7 (and A8 becomes a skip) |
| 179 | // node dev/verify_conformance.mjs --break nullvotes # -> A8 |
| 180 | // node dev/verify_conformance.mjs --break leak403 # -> D5 AND D6, on purpose |
| 181 | // node dev/verify_conformance.mjs --break bodyvoice # -> E9 |
| 182 | // node dev/verify_conformance.mjs --break trimmed # -> E2 AND A10, on purpose |
| 183 | // node dev/verify_conformance.mjs --break mineleak # -> A9 |
| 184 | // node dev/verify_conformance.mjs --break emptyceiling # -> B4 |
| 185 | // node dev/verify_conformance.mjs --break charset # -> G1 |
| 186 | // node dev/verify_conformance.mjs --break noncanonical # -> G5 |
| 187 | // node dev/verify_conformance.mjs --break clamped # -> B6 AND B7, on purpose |
| 188 | // |
| 189 | // A run with `--break` EXPECTS to fail: it exits 0 when something reddened and 1 when |
| 190 | // nothing did, so "the break changed nothing" is itself a failing run. |
| 191 | // |
| 192 | // EACH BREAK IS SCOPED TO SURVIVE EVERY CHECK BUT THE ONE IT PROVES, and the sharper |
| 193 | // form of that rule is the one that bit this file in the writing: A BREAK CAUGHT BY AN |
| 194 | // EARLIER, CHEAPER CHECK PROVES NOTHING ABOUT THE LATER ONE. `wrongstatus` is the |
| 195 | // worked example. Every refusal check first read BOTH the token and the status, so a |
| 196 | // broken status turned fifteen checks red and said nothing about the one that owns the |
| 197 | // property. The fix was not a cleverer break, it was to split the property: a local |
| 198 | // check now asserts the TOKEN alone, and §3.1's token-to-status table is asserted once, |
| 199 | // centrally, over every refusal the run saw (G3). One property, one check, one break. |
| 200 | // |
| 201 | // The same reasoning shapes the rest. `nokey` drops a field and re-canonicalises the |
| 202 | // bytes, so G5 stays green and only A2 moves; had it left the text alone, G5 would have |
| 203 | // reddened too and A2 would have been credited with catching something it never saw. |
| 204 | // `novotes` and `nullvotes` are a pair for the same reason: A7 asks whether the key is |
| 205 | // there and A8 asks what it holds, so removing the key must not be the same break as |
| 206 | // emptying it. `mineleak` adds `mine` only where NO voice header was sent at all, so |
| 207 | // A10 — which sends a header of spaces — stays green and A9 alone moves. |
| 208 | // |
| 209 | // THREE BREAKS ARE DELIBERATELY NOT ISOLATED, and each is one forge behaviour that |
| 210 | // violates two sentences, so no cleverer break could separate them: |
| 211 | // |
| 212 | // `clamped` reddens B6 and B7. B6 says a limit outside 1..200 is refused and B7 says |
| 213 | // it is not served clamped. |
| 214 | // `leak403` reddens D5 and D6. D5 is the privacy rule to an unvoiced caller and D6 |
| 215 | // is the same rule to one holding a voice; §3 requires `absent` to EVERY |
| 216 | // caller, so a 403 breaks both halves at once. D6 only reddens when |
| 217 | // `ORE_VOICE` is supplied -- without it D6 is a skip and D5 moves alone, |
| 218 | // which is what this line said until 2026-08-17, when a run with a voice |
| 219 | // was made for the first time. |
| 220 | // `trimmed` reddens E2 and A10. §3.1's whitespace rule has a read half (A10) and a |
| 221 | // write half (E2), and a header of spaces read as a credential breaks the |
| 222 | // two together. `mineleak` is the break that leaves A10 green, on purpose. |
| 223 | // |
| 224 | // Written down rather than tidied away, because a break whose reach is not stated is a |
| 225 | // break whose reach is not known -- and an UNDERSTATED reach is worse than a wide one, |
| 226 | // since the extra check it reddens gets silent credit for catching something else. |
| 227 | // |
| 228 | // ── SKIPS ARE PRINTED AT THE SAME WEIGHT AS PASSES ─────────────────────────────── |
| 229 | // |
| 230 | // "37 passed" with four silent skips reads as coverage, and a silent skip is the next |
| 231 | // absence nobody can see — §3.3's own defect class, one level up. So every skip names |
| 232 | // the clause it could not test, the summary line carries them, and a run with skips |
| 233 | // cannot be mistaken for a full pass at a glance. The exit code still turns on failures |
| 234 | // alone; the reading of the run turns on both. |
| 235 | // |
| 236 | // ── THE TEST FOR EVERY CHECK BELOW, WHICH IS MECHANICAL (§3.3) ─────────────────── |
| 237 | // |
| 238 | // A CONFORMANCE CHECK MAY NOT ASSERT A DISTINCTION THE CONTRACT DOES NOT DEFINE. |
| 239 | // IF YOU CANNOT CITE THE SENTENCE, YOU INVENTED THE RULE. |
| 240 | // |
| 241 | // A clause that NAMES a category without DEFINING one does not count. On meeting an |
| 242 | // undefined clause the move is to report the gap and skip the check, never to guess: |
| 243 | // a guess in a conformance suite is indistinguishable from a requirement by the time |
| 244 | // anyone reads it, which is how a regular expression in a mock came to arbitrate a |
| 245 | // question the contract had never answered. Every skip taken for that reason is |
| 246 | // printed, counted, and listed again under GAPS at the end. |
| 247 | // |
| 248 | // ── TWO TOKENS THIS FILE DOES NOT PROVOKE, AND SAYS SO RATHER THAN FAKING ──────── |
| 249 | // |
| 250 | // `unsupported` (405). §3.1: "unsupported is unreachable from any honest client, and |
| 251 | // it is not dead vocabulary to be deleted." The token answers a verb this surface |
| 252 | // documents nowhere, and the panel sends no such verb; a probe would be asserting |
| 253 | // the door's behaviour rather than the surface's. §3.1 already records the fact from |
| 254 | // the forge's side (`serve.rs:153`), and warns that "a reader who finds no test |
| 255 | // naturally reaching this token will conclude it can go". This paragraph is that |
| 256 | // reader's answer. If the orchestrator wants the row covered anyway, a `DELETE` on |
| 257 | // any route provokes it and changes nothing; it is left out on purpose, not missed. |
| 258 | // |
| 259 | // `internal` (500). A fault inside the forge, which cannot be induced from outside |
| 260 | // without breaking the forge, and breaking a live forge to see it say so is not a |
| 261 | // conformance check. |
| 262 | // |
| 263 | // `throttled` (429) is not PROVOKED either, and that is §2's requirement rather than |
| 264 | // shyness: earning one on purpose takes a flood, and §2.1 puts a global failure |
| 265 | // backstop behind exactly that. Its SHAPE is still checked — a throttle that arrives |
| 266 | // for any reason is read by G1..G4 like any other refusal. |
| 267 | // |
| 268 | // IT DOES ARRIVE. This suite writes two records per run under one voice, and oregami's |
| 269 | // allowance is twenty writes a voice a minute (`oregami/src/limit.rs`, `VOICE_CAP`), so |
| 270 | // several runs in quick succession spend it. On 2026-08-17 the fourth consecutive run |
| 271 | // reported `FAIL F1 ... 429 error=throttled`: a red check whose cause was the forge |
| 272 | // obeying §2, which is the opposite of what a red line there means. Two things were |
| 273 | // wrong and both are fixed below — the F-series now SKIPS on a throttle and names the |
| 274 | // allowance, and `keep()` files a refused write in the refusal ledger, which it did not |
| 275 | // do, so the sentence above about G1..G4 reading it was false for exactly the answer |
| 276 | // most likely to need it. |
| 277 | |
| 278 | import fs from 'node:fs'; |
| 279 | import net from 'node:net'; |
| 280 | import path from 'node:path'; |
| 281 | import tls from 'node:tls'; |
| 282 | import { fileURLToPath } from 'node:url'; |
| 283 | |
| 284 | const HERE = path.dirname(fileURLToPath(import.meta.url)); |
| 285 | const ROOT = path.join(HERE, '..'); |
| 286 | |
| 287 | |
| 288 | // ┌───────────────────────────────────────────────────────────────────────────────┐ |
| 289 | // │ THE SUBJECT — the client's own constant, and nothing else │ |
| 290 | // └───────────────────────────────────────────────────────────────────────────────┘ |
| 291 | |
| 292 | /// Where the panel names its repository. Read here so that the suite and the browser |
| 293 | /// are reading the same two lines. |
| 294 | const CLIENT = path.join(ROOT, 'www', 'js', 'improve.js'); |
| 295 | |
| 296 | /// Pull one `var NAME = '...';` out of the client, or stop. |
| 297 | /// |
| 298 | /// Exactly one match is required. A missing constant means the client has been |
| 299 | /// changed underneath this file, and a second match means the file no longer has one |
| 300 | /// place where the repository is named -- both are reasons to stop rather than pick. |
| 301 | function constantOf(src, name) { |
| 302 | const re = new RegExp("^\\s*var\\s+" + name + "\\s*=\\s*'([^']*)'\\s*;", 'gm'); |
| 303 | const hits = [...src.matchAll(re)]; |
| 304 | if (hits.length !== 1) { |
| 305 | console.error(`www/js/improve.js holds ${hits.length} definitions of ${name}, and this ` |
| 306 | + 'suite takes its subject from exactly one. §3.3: the subject is the client\'s own ' |
| 307 | + 'configured constant, so a suite that cannot read it must stop rather than guess.'); |
| 308 | process.exit(2); |
| 309 | } |
| 310 | return hits[0][1]; |
| 311 | } |
| 312 | |
| 313 | const CLIENT_SRC = (() => { |
| 314 | try { return fs.readFileSync(CLIENT, 'utf8'); } |
| 315 | catch (e) { |
| 316 | console.error(`Could not read ${CLIENT}: ${e && e.message}. The subject lives there and ` |
| 317 | + 'nowhere else, so there is nothing to run against.'); |
| 318 | process.exit(2); |
| 319 | } |
| 320 | })(); |
| 321 | |
| 322 | /// The repository the product names. §3.3: never an argument. |
| 323 | const SUBJECT = { |
| 324 | account: constantOf(CLIENT_SRC, 'ACCOUNT'), |
| 325 | repo: constantOf(CLIENT_SRC, 'REPO'), |
| 326 | }; |
| 327 | |
| 328 | /// Where the write checks that are expected to SUCCEED go. §3.3: "The sandbox is a |
| 329 | /// second compiled-in constant and never a command-line argument, for the reason the |
| 330 | /// subject is not one." Change it here, in a commit somebody reviews, or not at all. |
| 331 | const SANDBOX = { account: 'oxedyne', repo: 'conformance' }; |
| 332 | |
| 333 | /// The header a voice travels in. §4 names it. A literal, not an import: a rename on |
| 334 | /// either side must fail this suite rather than travel through it. |
| 335 | const HDR = 'x-ore-voice'; |
| 336 | |
| 337 | /// §3.1's nine tokens and their statuses, transcribed from the table and from nothing |
| 338 | /// else. A tenth token, or a status that disagrees with this map, is a finding. |
| 339 | const TABLE = { |
| 340 | absent: 404, |
| 341 | unvoiced: 401, |
| 342 | unknown: 401, |
| 343 | unpermitted: 403, |
| 344 | throttled: 429, |
| 345 | malformed: 400, |
| 346 | no_proposal: 404, |
| 347 | unsupported: 405, |
| 348 | internal: 500, |
| 349 | }; |
| 350 | |
| 351 | /// The three reasons §3.1 admits on a `throttled`, and the only refusal that carries |
| 352 | /// `because` at all. |
| 353 | const BECAUSE = ['address', 'voice', 'failing']; |
| 354 | |
| 355 | /// §3's four spellings of `state`. |
| 356 | const STATES = ['open', 'accepted', 'declined', 'done']; |
| 357 | |
| 358 | /// §3's fields on every proposal record, from the listing example. §9 adds `votes`, |
| 359 | /// and `mine` when a voice was carried; those are asserted separately, because a forge |
| 360 | /// that has not shipped §9 must still pass everything §3 asks for. |
| 361 | const FIELDS = ['number', 'title', 'state', 'author', 'comments', 'opened', 'changed', |
| 362 | 'mark', 'build']; |
| 363 | |
| 364 | /// What a refusal must be served as. §3.2: "A refusal is served as `application/json`, |
| 365 | /// no `charset`". |
| 366 | const JSON_CT = 'application/json'; |
| 367 | |
| 368 | /// The largest listing §3 admits, and the size of one when nothing says. |
| 369 | const CEILING = 200; |
| 370 | const DEFAULT_PAGE = 50; |
| 371 | |
| 372 | /// A title that names itself, on every probe body sent to the SUBJECT. Every one of |
| 373 | /// those probes is expected to be refused; if the forge ever accepts one, the record |
| 374 | /// it leaves says what it is instead of reading like a tester's report. |
| 375 | const PROBE_TITLE = 'conformance probe — this request should have been REFUSED'; |
| 376 | const PROBE_BODY = 'Sent by dev/verify_conformance.mjs to check a refusal path. If you ' |
| 377 | + 'are reading this in the backlog, the forge accepted a request the contract says it ' |
| 378 | + 'must refuse, and the check that sent it will have gone red in the same run.'; |
| 379 | |
| 380 | |
| 381 | // ┌───────────────────────────────────────────────────────────────────────────────┐ |
| 382 | // │ ARGUMENTS — none of which is a subject │ |
| 383 | // └───────────────────────────────────────────────────────────────────────────────┘ |
| 384 | |
| 385 | const argv = process.argv.slice(2); |
| 386 | const flag = (n) => argv.includes('--' + n); |
| 387 | const opt = (n, d) => { |
| 388 | const i = argv.indexOf('--' + n); |
| 389 | if (i < 0 || i + 1 >= argv.length) return d; |
| 390 | const v = String(argv[i + 1]); |
| 391 | return v.startsWith('--') ? d : v; |
| 392 | }; |
| 393 | |
| 394 | for (const a of argv) { |
| 395 | if (a === '--account' || a === '--repo') { |
| 396 | console.error(`${a} does not exist here, deliberately. §3.3: the subject is the ` |
| 397 | + 'client\'s own configured constant, because "a subject supplied on the command ' |
| 398 | + 'line will be pointed at whatever passes, and it will be pointed there by the ' |
| 399 | + 'person with the most reason to want a green result." Edit www/js/improve.js if ' |
| 400 | + 'the product\'s repository has changed.'); |
| 401 | process.exit(2); |
| 402 | } |
| 403 | } |
| 404 | |
| 405 | /// Which deliberate corruption is in force, if any. |
| 406 | const BREAK = String(opt('break', '')); |
| 407 | |
| 408 | /// Secrets from a file, one `NAME=value` per line. The environment wins over it, and |
| 409 | /// nothing here is ever printed. |
| 410 | function voicesFrom(file) { |
| 411 | const out = {}; |
| 412 | let text = ''; |
| 413 | try { text = fs.readFileSync(file, 'utf8'); } |
| 414 | catch (e) { |
| 415 | console.error(`--voices ${file} could not be read: ${e && e.message}`); |
| 416 | process.exit(2); |
| 417 | } |
| 418 | for (const line of text.split(/\r?\n/)) { |
| 419 | const s = line.trim(); |
| 420 | if (!s || s.startsWith('#')) continue; |
| 421 | const i = s.indexOf('='); |
| 422 | if (i < 0) continue; |
| 423 | out[s.slice(0, i).trim()] = s.slice(i + 1).trim().replace(/^["']|["']$/g, ''); |
| 424 | } |
| 425 | return out; |
| 426 | } |
| 427 | |
| 428 | const FILE_VOICES = opt('voices', '') ? voicesFrom(opt('voices', '')) : {}; |
| 429 | const secretOf = (name) => String(process.env[name] || FILE_VOICES[name] || ''); |
| 430 | |
| 431 | /// The three voices, each one a secret and each one optional. What is missing costs |
| 432 | /// checks, and those checks are skipped by name. |
| 433 | const V = { |
| 434 | subject: secretOf('ORE_VOICE'), |
| 435 | reader: secretOf('ORE_VOICE_READER'), |
| 436 | sandbox: secretOf('ORE_VOICE_SANDBOX'), |
| 437 | }; |
| 438 | |
| 439 | /// Nothing printed by this file carries a secret, whatever a check puts in its detail |
| 440 | /// string. Cheaper than remembering, and the rule it keeps is CLAUDE.md's. |
| 441 | const SECRETS = Object.values(V).filter(s => s.length > 0); |
| 442 | function redact(s) { |
| 443 | let out = String(s == null ? '' : s); |
| 444 | for (const sec of SECRETS) { |
| 445 | if (sec) out = out.split(sec).join('«voice»'); |
| 446 | } |
| 447 | return out; |
| 448 | } |
| 449 | |
| 450 | /// Where the forge is. The address is not the subject -- it names WHERE to ask, not |
| 451 | /// WHAT to ask about -- so it may come from the environment. It is printed at the top |
| 452 | /// of every run and in the summary, so the evidence carries what it was pointed at. |
| 453 | function forgeBase() { |
| 454 | const env = String(process.env.ORE_FORGE || '').trim(); |
| 455 | if (env) return { url: env.replace(/\/$/, ''), from: 'ORE_FORGE' }; |
| 456 | try { |
| 457 | const jdat = fs.readFileSync(path.join(ROOT, 'gateway', 'app.jdat'), 'utf8'); |
| 458 | const host = /"forge_host"\s*:\s*"([^"]+)"/.exec(jdat); |
| 459 | const port = /"forge_port"\s*:\s*"([^"]+)"/.exec(jdat); |
| 460 | if (host && port) { |
| 461 | return { url: `http://${host[1]}:${port[1]}`, from: 'gateway/app.jdat' }; |
| 462 | } |
| 463 | } catch (e) { /* fall through to the failure below */ } |
| 464 | console.error('No ORE_FORGE, and gateway/app.jdat names no forge_host and forge_port. ' |
| 465 | + 'There is nothing to ask.'); |
| 466 | process.exit(2); |
| 467 | } |
| 468 | |
| 469 | const BASE = forgeBase(); |
| 470 | const URLB = new URL(BASE.url); |
| 471 | const PREFIX = URLB.pathname.replace(/\/$/, ''); |
| 472 | const IS_TLS = URLB.protocol === 'https:'; |
| 473 | const PORT = Number(URLB.port || (IS_TLS ? 443 : 80)); |
| 474 | |
| 475 | // `--why-not`: can this run be made at all, and if not, in one sentence, why. |
| 476 | // |
| 477 | // A SKIP IS NOT A PASS AND A FAILURE TO CONNECT IS NOT A CONFORMANCE RESULT. On the |
| 478 | // 2026-08-17 gate this file appeared among fifty-six reds, indistinguishable from a |
| 479 | // forge that had answered wrongly, when what had happened was that no forge was |
| 480 | // running -- and its own output said so at length and better than the summary line |
| 481 | // that quoted it. `dev/run_all.sh` asks this question before running anything, on |
| 482 | // the same principle every other derivation in that file follows: ask the verifier, |
| 483 | // never a list somebody has to remember to edit. The address is resolved here, by |
| 484 | // the code that will do the asking, so the two can never disagree about where the |
| 485 | // forge was meant to be. |
| 486 | // |
| 487 | // Silence means it can run. Anything printed is the reason it cannot, and the suite |
| 488 | // prints that as a SKIP naming the absent thing. |
| 489 | if (flag('why-not')) { |
| 490 | const reachable = await new Promise((res) => { |
| 491 | const sock = net.connect({ host: URLB.hostname, port: PORT }); |
| 492 | const done = (v) => { sock.destroy(); res(v); }; |
| 493 | sock.setTimeout(3000); |
| 494 | sock.on('connect', () => done(true)); |
| 495 | sock.on('timeout', () => done(false)); |
| 496 | sock.on('error', () => done(false)); |
| 497 | }); |
| 498 | if (!reachable) { |
| 499 | console.log(`no Oregami forge answering at ${BASE.url} (${BASE.from}) — this run would ` |
| 500 | + 'say NOTHING about the repository the product names, which is the whole reason ' |
| 501 | + 'the file exists (§3.3). Stand one up as its header sets out, or point ORE_FORGE ' |
| 502 | + 'at one. It is NOT a pass.'); |
| 503 | } |
| 504 | process.exit(0); |
| 505 | } |
| 506 | |
| 507 | |
| 508 | // ┌───────────────────────────────────────────────────────────────────────────────┐ |
| 509 | // │ THE CLIENT — one request, built by hand, so the bytes are the bytes │ |
| 510 | // └───────────────────────────────────────────────────────────────────────────────┘ |
| 511 | |
| 512 | /// How long one exchange may take before the run gives up on it. |
| 513 | const TIMEOUT_MS = 20000; |
| 514 | |
| 515 | /// Send one request and read the whole answer. |
| 516 | /// |
| 517 | /// # Arguments |
| 518 | /// * `method` - The verb, written literally into the request line. |
| 519 | /// * `target` - The request-target, INCLUDING the query, exactly as it should appear |
| 520 | /// on the wire. Nothing here re-encodes it: §3.2's raw-query rule cannot be tested |
| 521 | /// through a client that tidies the query up. |
| 522 | /// * `opts` - `headers` as `[name, value]` pairs written verbatim, and `body` as a |
| 523 | /// string. |
| 524 | function ask(method, target, opts = {}) { |
| 525 | const headers = opts.headers || []; |
| 526 | const body = opts.body == null ? null : String(opts.body); |
| 527 | const host = URLB.port ? `${URLB.hostname}:${URLB.port}` : URLB.hostname; |
| 528 | let head = `${method} ${PREFIX}${target} HTTP/1.1\r\n` |
| 529 | + `Host: ${host}\r\n` |
| 530 | + 'User-Agent: daimond-conformance/1\r\n' |
| 531 | + 'Connection: close\r\n'; |
| 532 | for (const [n, v] of headers) head += `${n}: ${v}\r\n`; |
| 533 | if (body != null) head += `Content-Length: ${Buffer.byteLength(body)}\r\n`; |
| 534 | head += '\r\n'; |
| 535 | |
| 536 | return new Promise((resolve, reject) => { |
| 537 | const chunks = []; |
| 538 | const sock = IS_TLS |
| 539 | ? tls.connect({ host: URLB.hostname, port: PORT, servername: URLB.hostname }) |
| 540 | : net.connect({ host: URLB.hostname, port: PORT }); |
| 541 | sock.setTimeout(TIMEOUT_MS); |
| 542 | // Once, and on the right event: a TLS socket emits both `connect` and |
| 543 | // `secureConnect`, and writing on each would send the request twice. |
| 544 | sock.once(IS_TLS ? 'secureConnect' : 'connect', |
| 545 | () => sock.write(head + (body == null ? '' : body))); |
| 546 | sock.on('data', (d) => chunks.push(d)); |
| 547 | sock.on('timeout', () => { sock.destroy(); reject(new Error('the forge did not answer in time')); }); |
| 548 | sock.on('error', (e) => reject(e)); |
| 549 | sock.on('close', () => { |
| 550 | try { |
| 551 | // The one place a break may touch anything: the answer, on its way in, |
| 552 | // before the first assertion reads it. |
| 553 | const voiced = headers.find(([n]) => n.toLowerCase() === HDR); |
| 554 | resolve(damage({ |
| 555 | method, target, body, |
| 556 | voice: voiced ? voiced[1] : null, |
| 557 | r: parse(Buffer.concat(chunks)), |
| 558 | })); |
| 559 | } catch (e) { reject(e); } |
| 560 | }); |
| 561 | }); |
| 562 | } |
| 563 | |
| 564 | /// Split a raw HTTP/1.1 answer into its status, its headers and its body. |
| 565 | function parse(raw) { |
| 566 | const cut = raw.indexOf('\r\n\r\n'); |
| 567 | if (cut < 0) throw new Error('the answer had no header block'); |
| 568 | const head = raw.slice(0, cut).toString('latin1'); |
| 569 | let rest = raw.slice(cut + 4); |
| 570 | const lines = head.split('\r\n'); |
| 571 | const status = Number((/^HTTP\/1\.[01]\s+(\d{3})/.exec(lines[0]) || [])[1] || 0); |
| 572 | const headers = {}; |
| 573 | for (const line of lines.slice(1)) { |
| 574 | const i = line.indexOf(':'); |
| 575 | if (i > 0) headers[line.slice(0, i).trim().toLowerCase()] = line.slice(i + 1).trim(); |
| 576 | } |
| 577 | if (/chunked/i.test(headers['transfer-encoding'] || '')) rest = dechunk(rest); |
| 578 | const len = headers['content-length'] ? Number(headers['content-length']) : null; |
| 579 | const bytes = (len != null && len <= rest.length) ? rest.slice(0, len) : rest; |
| 580 | const text = bytes.toString('utf8'); |
| 581 | let json = null; |
| 582 | try { json = JSON.parse(text); } catch (e) { /* not JSON, which is sometimes the finding */ } |
| 583 | return { status, headers, type: headers['content-type'] || '', bytes, text, json }; |
| 584 | } |
| 585 | |
| 586 | /// Undo chunked transfer coding. |
| 587 | function dechunk(buf) { |
| 588 | const out = []; |
| 589 | let i = 0; |
| 590 | for (;;) { |
| 591 | const nl = buf.indexOf('\r\n', i); |
| 592 | if (nl < 0) break; |
| 593 | const n = parseInt(buf.slice(i, nl).toString('latin1').split(';')[0], 16); |
| 594 | if (!Number.isFinite(n) || n === 0) break; |
| 595 | out.push(buf.slice(nl + 2, nl + 2 + n)); |
| 596 | i = nl + 2 + n + 2; |
| 597 | } |
| 598 | return Buffer.concat(out); |
| 599 | } |
| 600 | |
| 601 | /// A form-encoded body, which is what every write on this surface carries (§9: "The |
| 602 | /// body is form-encoded, like every other write on this surface."). |
| 603 | const form = (o) => new URLSearchParams(o).toString(); |
| 604 | const FORM_CT = ['Content-Type', 'application/x-www-form-urlencoded']; |
| 605 | |
| 606 | /// A GET, optionally carrying a voice. `secret` of `null` sends NO header at all, |
| 607 | /// which is a different request from one sending an empty header (§3.1). |
| 608 | const get = (target, secret = null) => |
| 609 | ask('GET', target, { headers: secret === null ? [] : [[HDR, secret]] }); |
| 610 | |
| 611 | /// A POST with a form body, optionally carrying a voice. |
| 612 | const post = (target, body, secret = null) => |
| 613 | ask('POST', target, { |
| 614 | headers: secret === null ? [FORM_CT] : [FORM_CT, [HDR, secret]], |
| 615 | body: typeof body === 'string' ? body : form(body), |
| 616 | }); |
| 617 | |
| 618 | /// The two routes, for a repository. |
| 619 | const listOf = (r) => `/${r.account}/${r.repo}/proposals`; |
| 620 | |
| 621 | |
| 622 | // ┌───────────────────────────────────────────────────────────────────────────────┐ |
| 623 | // │ CANONICAL JSON — written here, not imported from anything under test │ |
| 624 | // └───────────────────────────────────────────────────────────────────────────────┘ |
| 625 | |
| 626 | /// RFC 8785 canonical form of a parsed value. |
| 627 | /// |
| 628 | /// Keys sort by UTF-16 code unit, which is what JavaScript's own string ordering is |
| 629 | /// and what §3.2 says `json_canonical` does. Numbers go through `JSON.stringify`, |
| 630 | /// which agrees with RFC 8785 over the integers this data holds -- §3 records that the |
| 631 | /// encoder refuses floats, byte strings, non-string keys and integers beyond 2^53 - 1, |
| 632 | /// so nothing on this wire reaches the cases where the two could part. |
| 633 | function canon(v) { |
| 634 | if (v === null || typeof v === 'boolean' || typeof v === 'string') return JSON.stringify(v); |
| 635 | if (typeof v === 'number') { |
| 636 | if (!Number.isFinite(v)) throw new Error('a non-finite number is not canonical'); |
| 637 | return JSON.stringify(v); |
| 638 | } |
| 639 | if (Array.isArray(v)) return '[' + v.map(canon).join(',') + ']'; |
| 640 | return '{' + Object.keys(v).sort() |
| 641 | .map(k => JSON.stringify(k) + ':' + canon(v[k])).join(',') + '}'; |
| 642 | } |
| 643 | |
| 644 | |
| 645 | // ┌───────────────────────────────────────────────────────────────────────────────┐ |
| 646 | // │ THE BREAKS — a check that will not go red is a finding, not a fixture problem │ |
| 647 | // └───────────────────────────────────────────────────────────────────────────────┘ |
| 648 | // |
| 649 | // Each one corrupts THE ANSWER on its way in, between `parse` and the first assertion |
| 650 | // that reads it. The forge is not ours to damage and it is not what is under test |
| 651 | // here: what is under test is whether these checks would notice. |
| 652 | // |
| 653 | // Each is scoped to redden exactly one check. Where it cannot be, the reach is written |
| 654 | // beside it. Where a break turns a neighbour into a SKIP rather than a failure that is |
| 655 | // written down too, because a check that quietly stops running is the absence this |
| 656 | // whole suite is about. |
| 657 | |
| 658 | /// Re-emit a mutated body as canonical bytes, so that a break aimed at a field does not |
| 659 | /// trip the canonical-bytes check instead and take the credit. |
| 660 | function recanon(r) { |
| 661 | r.text = canon(r.json); |
| 662 | r.bytes = Buffer.from(r.text, 'utf8'); |
| 663 | } |
| 664 | |
| 665 | /// Is this the subject's full listing — the one A1, A2, A7, A8 and A9 all read? |
| 666 | /// |
| 667 | /// The `from=` exclusion is not tidiness: B4 asks for the same limit with a ceiling on |
| 668 | /// it, and a break that also corrupted THAT answer would be reaching a second check for |
| 669 | /// no reason. Proved by driving each break over a set of representative answers rather |
| 670 | /// than by reading the predicate, which is how the reach was found in the first place. |
| 671 | const bigList = (c) => c.method === 'GET' |
| 672 | && c.target.startsWith(`/${SUBJECT.account}/${SUBJECT.repo}/proposals?`) |
| 673 | && c.target.includes(`limit=${CEILING}`) |
| 674 | && !c.target.includes('from='); |
| 675 | |
| 676 | const BREAKS = { |
| 677 | /// A token and its status put out of step. §3.1's table is asserted once, centrally, |
| 678 | /// so this must move G3 and nothing else -- which is the only reason the local checks |
| 679 | /// read the token alone. |
| 680 | wrongstatus: (c) => { |
| 681 | // ONE refusal, not every `malformed` one: the smallest corruption that can move |
| 682 | // the check under test is the one that says most about it. |
| 683 | if (c.target.includes('from=0') && c.r.json && c.r.json.error === 'malformed') { |
| 684 | c.r.status = 422; |
| 685 | } |
| 686 | }, |
| 687 | |
| 688 | /// A field missing from the listing record, with the bytes re-canonicalised behind it. |
| 689 | nokey: (c) => { |
| 690 | if (bigList(c) && c.r.json && Array.isArray(c.r.json.proposals)) { |
| 691 | for (const p of c.r.json.proposals) delete p.mark; |
| 692 | recanon(c.r); |
| 693 | } |
| 694 | }, |
| 695 | |
| 696 | /// `votes` absent where §9 guarantees the zero object. A8 has nothing left to read and |
| 697 | /// SKIPS, which is written here rather than discovered. |
| 698 | novotes: (c) => { |
| 699 | if (bigList(c) && c.r.json && Array.isArray(c.r.json.proposals)) { |
| 700 | for (const p of c.r.json.proposals) delete p.votes; |
| 701 | recanon(c.r); |
| 702 | } |
| 703 | }, |
| 704 | |
| 705 | /// A zero tally emitted as null. The key is still there, so A7 stays green and only |
| 706 | /// A8 -- which is the check that owns "never null" -- moves. |
| 707 | nullvotes: (c) => { |
| 708 | if (bigList(c) && c.r.json && Array.isArray(c.r.json.proposals)) { |
| 709 | for (const p of c.r.json.proposals) { |
| 710 | if (p.votes && p.votes.for === 0 && p.votes.against === 0) p.votes = null; |
| 711 | } |
| 712 | recanon(c.r); |
| 713 | } |
| 714 | }, |
| 715 | |
| 716 | /// 403 where `absent` is required. The token moves with the status, so G3 stays green |
| 717 | /// and D5 -- the privacy check -- is the only one that can catch this. |
| 718 | leak403: (c) => { |
| 719 | if (c.target.includes('no-such-repository')) { |
| 720 | c.r.status = 403; |
| 721 | c.r.json = { error: 'unpermitted', said: 'That voice may not see this repository.' }; |
| 722 | recanon(c.r); |
| 723 | } |
| 724 | }, |
| 725 | |
| 726 | /// A credential in the body accepted where §4 requires `malformed`. Only where NO |
| 727 | /// header was sent, so E10 -- both doors at once -- is untouched. |
| 728 | bodyvoice: (c) => { |
| 729 | if (c.method === 'POST' && c.voice === null && (c.body || '').includes('voice=')) { |
| 730 | c.r.status = 200; |
| 731 | c.r.json = { |
| 732 | number: 0, title: 'accepted', state: 'open', author: 'nobody', body: '', |
| 733 | comments: 0, opened: 0, changed: 0, mark: null, build: null, |
| 734 | discussion: [], votes: { for: 0, against: 0 }, mine: null, |
| 735 | }; |
| 736 | recanon(c.r); |
| 737 | } |
| 738 | }, |
| 739 | |
| 740 | /// A header of spaces read as a credential. Spaces only, so E3's tabs and E4's empty |
| 741 | /// header stay exactly right and E2 alone moves. |
| 742 | trimmed: (c) => { |
| 743 | if (typeof c.voice === 'string' && c.voice.length > 0 && /^ +$/.test(c.voice)) { |
| 744 | c.r.status = 401; |
| 745 | c.r.json = { error: 'unknown', said: 'That voice is not recognised here.' }; |
| 746 | recanon(c.r); |
| 747 | } |
| 748 | }, |
| 749 | |
| 750 | /// `mine` on a read that carried no voice. Only where the header was ABSENT, so A10 -- |
| 751 | /// which sends a header of spaces -- stays green. |
| 752 | mineleak: (c) => { |
| 753 | if (bigList(c) && c.voice === null && c.r.json && Array.isArray(c.r.json.proposals)) { |
| 754 | for (const p of c.r.json.proposals) p.mine = null; |
| 755 | recanon(c.r); |
| 756 | } |
| 757 | }, |
| 758 | |
| 759 | /// A `from` above the highest answered with an empty page instead of the newest one. |
| 760 | emptyceiling: (c) => { |
| 761 | const m = /[?&]from=(\d+)/.exec(c.target); |
| 762 | if (m && Number(m[1]) >= 1000 && c.r.json && Array.isArray(c.r.json.proposals)) { |
| 763 | c.r.json.proposals = []; |
| 764 | recanon(c.r); |
| 765 | } |
| 766 | }, |
| 767 | |
| 768 | /// A charset on a refusal's media type. The type before the semicolon is untouched, so |
| 769 | /// C2 -- which reads that -- stays green and G1 alone moves. |
| 770 | charset: (c) => { |
| 771 | if (c.r.json && typeof c.r.json.error === 'string') { |
| 772 | c.r.type = c.r.type.split(';')[0] + '; charset=utf-8'; |
| 773 | } |
| 774 | }, |
| 775 | |
| 776 | /// The right values in the wrong bytes. `json` is left exactly as it was, so every |
| 777 | /// check that reads a value stays green and G5 alone moves. |
| 778 | noncanonical: (c) => { |
| 779 | if (bigList(c) && c.r.json) c.r.text = JSON.stringify(c.r.json, null, 1); |
| 780 | }, |
| 781 | |
| 782 | /// An over-large limit served instead of refused. REACH: B6 and B7 together, and it |
| 783 | /// cannot be otherwise -- one behaviour violates both sentences. |
| 784 | clamped: (c) => { |
| 785 | if (c.target.includes(`limit=${CEILING + 1}`)) { |
| 786 | c.r.status = 200; |
| 787 | c.r.json = { total: 0, proposals: [] }; |
| 788 | recanon(c.r); |
| 789 | } |
| 790 | }, |
| 791 | }; |
| 792 | |
| 793 | if (BREAK && !(BREAK in BREAKS)) { |
| 794 | console.error(`--break ${BREAK} is not one of: ${Object.keys(BREAKS).join(', ')}`); |
| 795 | process.exit(2); |
| 796 | } |
| 797 | |
| 798 | /// Apply the break in force, if any, to one answer. |
| 799 | function damage(ctx) { |
| 800 | if (BREAK) BREAKS[BREAK](ctx); |
| 801 | return ctx.r; |
| 802 | } |
| 803 | |
| 804 | |
| 805 | // ┌───────────────────────────────────────────────────────────────────────────────┐ |
| 806 | // │ BOOKKEEPING — a skip is a hole in the evidence and has to look like one │ |
| 807 | // └───────────────────────────────────────────────────────────────────────────────┘ |
| 808 | |
| 809 | const ok = [], bad = [], skipped = []; |
| 810 | /// Every check's citation, by id, so `--cites` can print the map and a failure can |
| 811 | /// print the sentence it rests on. |
| 812 | const CITES = {}; |
| 813 | /// Every JSON refusal this run has seen, whatever provoked it. The F-series reads them |
| 814 | /// all at the end: the shape of a refusal is one property, not fifteen. |
| 815 | const refusals = []; |
| 816 | /// Every JSON body carrying a record, for the canonical-bytes check. |
| 817 | const bodies = []; |
| 818 | /// Contract gaps met on the way, each one a check not made. |
| 819 | const GAPS = []; |
| 820 | |
| 821 | /// Record one check. |
| 822 | /// |
| 823 | /// # Arguments |
| 824 | /// * `id` - Its short name, stable so a handover can refer to one. |
| 825 | /// * `cite` - The sentence from the authority that requires it. Not decoration: §3.3's |
| 826 | /// rule is that a check which cannot cite one has invented its rule. |
| 827 | /// * `name` - What is being asserted, in words. |
| 828 | /// * `pass` - Whether it held. |
| 829 | /// * `detail` - What was actually seen. |
| 830 | function check(id, cite, name, pass, detail) { |
| 831 | CITES[id] = cite; |
| 832 | (pass ? ok : bad).push(id); |
| 833 | console.log((pass ? ' ok ' : ' FAIL ') + id + ' ' + name |
| 834 | + (detail ? ' — ' + redact(detail) : '')); |
| 835 | if (!pass) console.log(' ' + cite); |
| 836 | } |
| 837 | |
| 838 | /// A check that could not run, and why. |
| 839 | /// |
| 840 | /// Counted, printed at the same weight as a pass, and NAMED IN THE SUMMARY. §3.3's |
| 841 | /// whole subject is evidence that is not there looking like evidence that is, and |
| 842 | /// "37 passed" beside four silent skips is that defect one level up. Every skip |
| 843 | /// therefore carries `clause`: the thing this run says nothing about, in a few words, |
| 844 | /// so the summary line can say it without anybody opening the file. |
| 845 | function skip(id, clause, why, cite) { |
| 846 | if (cite) CITES[id] = cite; |
| 847 | skipped.push({ id, clause, why, undefined_: false }); |
| 848 | console.log(' skip ' + id + ' [' + clause + '] — ' + redact(why)); |
| 849 | } |
| 850 | |
| 851 | /// A clause the authority does not define. Recorded as a GAP, and the check it would |
| 852 | /// have carried is skipped rather than guessed: §3.3, "on meeting an undefined clause |
| 853 | /// the move is to report the gap and skip the check, never to guess". |
| 854 | function gap(id, clause, what) { |
| 855 | GAPS.push({ id, clause, what }); |
| 856 | skipped.push({ id, clause, why: what, undefined_: true }); |
| 857 | console.log(' skip ' + id + ' [UNDEFINED: ' + clause + '] — ' + redact(what)); |
| 858 | } |
| 859 | |
| 860 | /// Keep a refusal for the F-series, and hand it back for a local assertion. |
| 861 | function note(where, r) { |
| 862 | if (r.json && typeof r.json === 'object' && typeof r.json.error === 'string') { |
| 863 | refusals.push({ where, r }); |
| 864 | } |
| 865 | return r; |
| 866 | } |
| 867 | |
| 868 | /// Keep a record-bearing body for the canonical-bytes check, and a refused one for the |
| 869 | /// refusal ledger as well. |
| 870 | /// |
| 871 | /// A write CAN be refused — a throttle is the refusal that actually happens — and until |
| 872 | /// 2026-08-17 an answer arriving here went into `bodies` alone, so G1..G4 never read the |
| 873 | /// one refusal a run was most likely to meet while a comment above claimed they did. |
| 874 | /// G5 reads the two ledgers by identity, so an answer in both is still read once. |
| 875 | function keep(where, r) { |
| 876 | if (r.json && typeof r.json === 'object') bodies.push({ where, r }); |
| 877 | return note(where, r); |
| 878 | } |
| 879 | |
| 880 | /// Whether an answer is the refusal it should be — THE TOKEN, AND NOT THE STATUS. |
| 881 | /// |
| 882 | /// The status is §3.1's table's property and it is asserted once, over every refusal |
| 883 | /// the run saw, by G3. That split is not tidiness. When these two were read together, |
| 884 | /// a single wrong status turned fifteen checks red and said nothing at all about the |
| 885 | /// check that owns the table -- a break caught by an earlier, cheaper check leaves the |
| 886 | /// later one still untested. |
| 887 | const refused = (r, token) => !!(r.json && r.json.error === token); |
| 888 | /// What an answer actually was, for a detail string. |
| 889 | const shown = (r) => `${r.status} ${r.type || '(no type)'} ${r.json && r.json.error |
| 890 | ? `error=${r.json.error}` : r.text.slice(0, 70).replace(/\s+/g, ' ')}`; |
| 891 | |
| 892 | |
| 893 | // ┌───────────────────────────────────────────────────────────────────────────────┐ |
| 894 | // │ THE RUN │ |
| 895 | // └───────────────────────────────────────────────────────────────────────────────┘ |
| 896 | |
| 897 | console.log(`subject ${SUBJECT.account}/${SUBJECT.repo} ` |
| 898 | + `read from www/js/improve.js — the client's own constant, §3.3`); |
| 899 | console.log(`sandbox ${SANDBOX.account}/${SANDBOX.repo} ` |
| 900 | + 'compiled into dev/verify_conformance.mjs, never an argument, §3.3'); |
| 901 | console.log(`forge ${BASE.url} (${BASE.from})`); |
| 902 | console.log(`voices subject:${V.subject ? 'yes' : 'NO'} ` |
| 903 | + `reader:${V.reader ? 'yes' : 'NO'} sandbox:${V.sandbox ? 'yes' : 'NO'}`); |
| 904 | if (BREAK) { |
| 905 | console.log(`BROKEN --break ${BREAK}: the answers are being corrupted on the way in, and ` |
| 906 | + 'this run is EXPECTED to fail. Nothing red here is the forge\'s.'); |
| 907 | } |
| 908 | console.log(''); |
| 909 | |
| 910 | const R = listOf(SUBJECT); |
| 911 | const S = listOf(SANDBOX); |
| 912 | |
| 913 | /// Whether the read lane ran at all. Without it a green summary would mean nothing, |
| 914 | /// so it decides the exit code as much as the failures do. |
| 915 | let readLane = false; |
| 916 | |
| 917 | try { |
| 918 | await reads(); |
| 919 | await writeRefusals(); |
| 920 | await sandboxWrites(); |
| 921 | await refusalShapes(); |
| 922 | } catch (e) { |
| 923 | check('RUN', 'The run must complete; a suite that stopped early has measured nothing ' |
| 924 | + 'below the point it stopped.', 'the run completed', false, |
| 925 | String((e && e.stack) || e)); |
| 926 | } |
| 927 | |
| 928 | finish(); |
| 929 | |
| 930 | |
| 931 | // ┌───────────────────────────────────────────────────────────────────────────────┐ |
| 932 | // │ A. READS — against the client's own constant, which is the point │ |
| 933 | // └───────────────────────────────────────────────────────────────────────────────┘ |
| 934 | |
| 935 | async function reads() { |
| 936 | // ── A1: the subject answers. This is the check the thirty-one never made. §3.3: |
| 937 | // "the read checks resolve the client's constant and assert it answers". |
| 938 | let list; |
| 939 | try { |
| 940 | list = keep('A1', await get(`${R}?format=json&limit=${CEILING}`)); |
| 941 | } catch (e) { |
| 942 | check('A1', '§3.3: "the read checks resolve the client\'s constant and assert it ' |
| 943 | + 'answers".', `${SUBJECT.account}/${SUBJECT.repo} answers a listing`, false, |
| 944 | String(e && e.message)); |
| 945 | skip('A2..D7', 'every read, path and privacy rule', |
| 946 | 'the subject did not answer at all — the socket failed, so not one of the read ' |
| 947 | + 'checks below was made'); |
| 948 | return; |
| 949 | } |
| 950 | const alive = list.status === 200 && list.json && Array.isArray(list.json.proposals) |
| 951 | && typeof list.json.total === 'number'; |
| 952 | check('A1', '§3.3: "the read checks resolve the client\'s constant and assert it answers"; ' |
| 953 | + '§3: "GET /<account>/<repo>/proposals?format=json".', |
| 954 | `${SUBJECT.account}/${SUBJECT.repo} answers a listing`, alive, shown(list)); |
| 955 | if (!alive) { |
| 956 | note('A1', list); |
| 957 | skip('A2..C3', 'the subject\'s records, selectors and surface rules', |
| 958 | 'the subject did not answer a listing, so no record, no selector and no ' |
| 959 | + 'surface rule can be measured on it. THIS IS THE 2026-08-14 FAULT: the panel names ' |
| 960 | + 'a repository the forge does not serve, and every check aimed elsewhere still passes.'); |
| 961 | // The path and privacy rules do not need the subject's contents, so they still run: |
| 962 | // they are the checks that say WHICH kind of nothing this is. |
| 963 | await paths(null); |
| 964 | return; |
| 965 | } |
| 966 | readLane = true; |
| 967 | |
| 968 | const props = list.json.proposals; |
| 969 | const nums = props.map(p => p && p.number).filter(n => typeof n === 'number'); |
| 970 | const top = nums.length ? Math.max(...nums) : 0; |
| 971 | const total = list.json.total; |
| 972 | |
| 973 | // ── A2: the record's fields. §3's listing example. |
| 974 | if (!props.length) { |
| 975 | skip('A2', 'the record\'s fields', |
| 976 | 'the subject holds no proposals, so there is no record to read'); |
| 977 | } else { |
| 978 | const missing = []; |
| 979 | for (const p of props) { |
| 980 | for (const f of FIELDS) if (!(f in p)) missing.push(`#${p.number}:${f}`); |
| 981 | } |
| 982 | check('A2', '§3: the listing example carries number, title, state, author, comments, ' |
| 983 | + 'opened, changed, mark and build on every record.', |
| 984 | 'every listing record carries §3\'s fields', missing.length === 0, |
| 985 | missing.slice(0, 6).join(' ')); |
| 986 | } |
| 987 | |
| 988 | // ── A3: `comments` is a count, on both routes. §3's own heading. |
| 989 | const detail = props.length |
| 990 | ? keep('A3', await get(`${R}/${top}?format=json`)) |
| 991 | : null; |
| 992 | if (!detail) { |
| 993 | skip('A3', '`comments` on the detail route', |
| 994 | 'the subject holds no proposals, so the detail route has nothing to read'); |
| 995 | skip('A4', 'the detail shape', |
| 996 | 'the subject holds no proposals, so the detail route has nothing to read'); |
| 997 | } else { |
| 998 | const listNum = props.every(p => typeof p.comments === 'number'); |
| 999 | const oneNum = detail.json && typeof detail.json.comments === 'number'; |
| 1000 | check('A3', '§3: "`comments` is a count everywhere ... One key, one type, on every ' |
| 1001 | + 'route."', '`comments` is a number on the listing and on the detail route', |
| 1002 | listNum && oneNum, |
| 1003 | `listing ${listNum}, detail ${detail.json ? typeof detail.json.comments : 'no body'}`); |
| 1004 | |
| 1005 | // ── A4: the detail shape adds `body` and the discussion. §3's second example. |
| 1006 | const d = detail.json || {}; |
| 1007 | const shape = typeof d.body === 'string' && Array.isArray(d.discussion); |
| 1008 | check('A4', '§3, "One proposal": the detail record is the listing record "plus `body` ' |
| 1009 | + 'and the discussion".', 'the detail route adds `body` and `discussion`', shape, |
| 1010 | shown(detail)); |
| 1011 | |
| 1012 | // ── A5: a discussion entry's keys, `when` and not `at`. §3.2 names the divergence. |
| 1013 | const entry = (Array.isArray(d.discussion) ? d.discussion : []).find(Boolean); |
| 1014 | if (!entry) { |
| 1015 | skip('A5', 'a discussion entry\'s keys', |
| 1016 | 'no proposal read here carries a discussion entry, so its keys cannot be ' |
| 1017 | + 'read. Nothing is asserted about a shape that was not seen.'); |
| 1018 | } else { |
| 1019 | check('A5', '§3: a discussion entry is {"author":..,"said":..,"when":..}; §3.2: "The ' |
| 1020 | + 'discussion entry\'s timestamp key is `when` on the wire and `at` on disk."', |
| 1021 | 'a discussion entry carries author, said and when', |
| 1022 | typeof entry.author === 'string' && typeof entry.said === 'string' |
| 1023 | && typeof entry.when === 'number', Object.keys(entry).join(',')); |
| 1024 | } |
| 1025 | } |
| 1026 | |
| 1027 | // ── A6: `state` is one of four. §3. |
| 1028 | if (props.length) { |
| 1029 | const rogue = props.filter(p => !STATES.includes(p.state)).map(p => `#${p.number}=${p.state}`); |
| 1030 | check('A6', '§3: "`state` is one of `open`, `accepted`, `declined`, `done`."', |
| 1031 | 'every record\'s state is one of the four', rogue.length === 0, rogue.slice(0, 5).join(' ')); |
| 1032 | } else { |
| 1033 | skip('A6', 'the state vocabulary', 'the subject holds no proposals'); |
| 1034 | } |
| 1035 | |
| 1036 | // ── A7/A8: §9's tally. The panel keys on the KEY's presence to know whether the vote |
| 1037 | // route has shipped, so its absence is a fact worth reporting exactly. |
| 1038 | // |
| 1039 | // A7 asks whether the KEY is there and A8 asks what it HOLDS, deliberately split: |
| 1040 | // "never absent" and "never null" are two sentences and one check could not be |
| 1041 | // proved red against both without a break that reddened it twice over. |
| 1042 | const votesPresent = props.length > 0 && props.every(p => 'votes' in p); |
| 1043 | if (!props.length) { |
| 1044 | skip('A7', '`votes` on every record', 'the subject holds no proposals'); |
| 1045 | skip('A8', 'the zero object', 'the subject holds no proposals'); |
| 1046 | } else { |
| 1047 | check('A7', '§9: "`votes` appears on both the listing and the detail route, same key, ' |
| 1048 | + 'same shape"; "the zero object, never null and never absent". The panel keys on ' |
| 1049 | + 'this key\'s presence to know whether the vote route has shipped.', |
| 1050 | '`votes` is on every record', votesPresent, |
| 1051 | votesPresent ? JSON.stringify(props[0].votes) |
| 1052 | : 'ABSENT — §9\'s vote route has not shipped on this forge'); |
| 1053 | if (!votesPresent) { |
| 1054 | skip('A8', 'the zero object', |
| 1055 | 'no record carried a `votes` key at all, so what it holds cannot be read. ' |
| 1056 | + 'A7 is the check that says so.'); |
| 1057 | } else { |
| 1058 | const shapes = props.filter(p => !(p.votes && typeof p.votes === 'object' |
| 1059 | && typeof p.votes.for === 'number' && typeof p.votes.against === 'number')) |
| 1060 | .map(p => `#${p.number}=${JSON.stringify(p.votes)}`); |
| 1061 | const zero = props.find(p => p.votes && p.votes.for === 0 && p.votes.against === 0); |
| 1062 | check('A8', '§9: "`votes` on a proposal nobody has voted on is {"for":0,"against":0}" ' |
| 1063 | + '-- the zero object, never null and never absent. A client forced to test for ' |
| 1064 | + 'the key before drawing a control would draw nothing on the record that most ' |
| 1065 | + 'needs one."', |
| 1066 | 'and it is a {for,against} object — the zero object when nobody has voted', |
| 1067 | shapes.length === 0 && (!zero || canon(zero.votes) === '{"against":0,"for":0}'), |
| 1068 | shapes.length ? shapes.slice(0, 4).join(' ') |
| 1069 | : (zero ? JSON.stringify(zero.votes) : 'no unvoted proposal here to read')); |
| 1070 | } |
| 1071 | } |
| 1072 | |
| 1073 | // ── A9: `mine` is ABSENT when the read carried no voice. §9. |
| 1074 | if (!props.length) { |
| 1075 | skip('A9', '`mine` absent when unvoiced', 'the subject holds no proposals'); |
| 1076 | } else { |
| 1077 | const present = props.filter(p => 'mine' in p).map(p => `#${p.number}`); |
| 1078 | check('A9', '§9: "When the request carries no voice, `mine` is absent rather than null, ' |
| 1079 | + 'so \'has not voted\' and \'was not asked\' cannot be confused."', |
| 1080 | '`mine` is absent from an unvoiced read', present.length === 0, present.slice(0, 5).join(' ')); |
| 1081 | } |
| 1082 | |
| 1083 | // ── A10: a header holding only whitespace IS the headerless case, so the read is |
| 1084 | // served and `mine` stays absent. Two sentences, composed: §3.1's trim rule says |
| 1085 | // such a header presented no credential, and §3.1 says a headerless GET of a |
| 1086 | // public repository is a 200. |
| 1087 | const spaced = keep('A10', await get(`${R}?format=json&limit=1`, ' ')); |
| 1088 | check('A10', '§3.1: "A credential is *not presented* when the header is absent, or when it ' |
| 1089 | + 'carries nothing once whitespace is stripped." with "a public read needs no voice, so ' |
| 1090 | + 'a headerless GET is a 200".', |
| 1091 | 'a whitespace-only voice header reads as no header at all', |
| 1092 | spaced.status === 200 && !!spaced.json |
| 1093 | && (spaced.json.proposals || []).every(p => !('mine' in p)), shown(spaced)); |
| 1094 | |
| 1095 | // ── A11: a wrong credential on a READ is `unknown`, not a public read. §3.2. |
| 1096 | const badRead = note('A11', await get(`${R}?format=json&limit=1`, '.')); |
| 1097 | check('A11', '§3.2: "A read carrying a wrong credential is `unknown`, not a public read."; ' |
| 1098 | + '§3.1: "\'.\' unknown <- one character is a credential".', |
| 1099 | 'a read carrying a wrong credential is `unknown`', |
| 1100 | refused(badRead, 'unknown'), shown(badRead)); |
| 1101 | |
| 1102 | // ── A12: `mine` when the read IS voiced, on BOTH routes. Needs a recognised voice. |
| 1103 | const known = V.subject ? await recognised(R, V.subject) : null; |
| 1104 | if (!V.subject) { |
| 1105 | skip('A12', '`mine` on a voiced read', |
| 1106 | 'no ORE_VOICE supplied, so no read can carry a voice'); |
| 1107 | } else if (known !== true) { |
| 1108 | skip('A12', '`mine` on a voiced read', |
| 1109 | 'the ORE_VOICE supplied is not recognised on the subject (the forge answered ' |
| 1110 | + '`unknown`), so a check using it would fail for a reason that is not the forge\'s'); |
| 1111 | } else if (!props.length) { |
| 1112 | skip('A12', '`mine` on a voiced read', 'the subject holds no proposals'); |
| 1113 | } else { |
| 1114 | const vl = keep('A12', await get(`${R}?format=json&limit=${CEILING}`, V.subject)); |
| 1115 | const vd = keep('A12', await get(`${R}/${top}?format=json`, V.subject)); |
| 1116 | const okMine = (p) => 'mine' in p && (p.mine === 1 || p.mine === -1 || p.mine === null); |
| 1117 | const listMine = !!vl.json && (vl.json.proposals || []).length > 0 |
| 1118 | && (vl.json.proposals || []).every(okMine); |
| 1119 | const detailMine = !!vd.json && okMine(vd.json); |
| 1120 | check('A12', '§9: "When the request carries a voice, the record also carries `mine`, one ' |
| 1121 | + 'of `1`, `-1` or `null` -- on the listing exactly as on the detail route."', |
| 1122 | '`mine` is present on both routes when the read is voiced', |
| 1123 | listMine && detailMine, `listing ${listMine}, detail ${detailMine}`); |
| 1124 | } |
| 1125 | |
| 1126 | // ── B1: order is newest first. §3.2. |
| 1127 | if (nums.length < 2) { |
| 1128 | skip('B1', 'newest first', |
| 1129 | 'fewer than two proposals, so an order cannot be read off one record'); |
| 1130 | } else { |
| 1131 | let desc = true; |
| 1132 | for (let i = 1; i < nums.length; i++) if (nums[i] >= nums[i - 1]) desc = false; |
| 1133 | check('B1', '§3.2: "Order is newest first. Descending by number".', |
| 1134 | 'the listing runs newest first', desc, nums.slice(0, 6).join(',')); |
| 1135 | } |
| 1136 | |
| 1137 | // ── B2: absent `from` starts at the newest. §3.2. |
| 1138 | if (!nums.length) { |
| 1139 | skip('B2', 'the newest page by default', 'the subject holds no proposals'); |
| 1140 | } else { |
| 1141 | const first = props[0] && props[0].number; |
| 1142 | check('B2', '§3.2: "`from` absent means no ceiling: start at the newest."', |
| 1143 | 'an unbounded listing starts at the highest number', first === top, |
| 1144 | `first ${first}, highest ${top}`); |
| 1145 | } |
| 1146 | |
| 1147 | // ── B3: `from` is a ceiling that counts down. §3.2. |
| 1148 | if (nums.length < 2) { |
| 1149 | skip('B3', '`from` as a ceiling', |
| 1150 | 'fewer than two proposals, so a ceiling cannot be told from an offset'); |
| 1151 | } else { |
| 1152 | const mid = nums.slice().sort((a, b) => b - a)[1]; // the second highest |
| 1153 | const r = keep('B3', await get(`${R}?format=json&from=${mid}&limit=${CEILING}`)); |
| 1154 | const got = r.json && Array.isArray(r.json.proposals) |
| 1155 | ? r.json.proposals.map(p => p.number) : []; |
| 1156 | check('B3', '§3.2: "`from` is the number to *start at and count down from*"; "Say `from` ' |
| 1157 | + 'is a ceiling, and that no ceiling is the newest."', |
| 1158 | '`from` is a ceiling and nothing above it comes back', |
| 1159 | got.length > 0 && got.every(n => n <= mid) && got[0] === mid, |
| 1160 | `from=${mid} gave ${got.slice(0, 6).join(',')}`); |
| 1161 | } |
| 1162 | |
| 1163 | // ── B4: a `from` above the highest is the newest page, not an empty one. §3.2. |
| 1164 | { |
| 1165 | const r = keep('B4', await get(`${R}?format=json&from=${top + 1000}&limit=${CEILING}`)); |
| 1166 | const got = r.json && Array.isArray(r.json.proposals) ? r.json.proposals : []; |
| 1167 | check('B4', '§3.2: "A `from` above the highest number is the newest page, not an empty ' |
| 1168 | + 'one. It is a ceiling; nothing is above it."', |
| 1169 | 'a `from` above the highest is the newest page', got.length > 0 && got[0].number === top, |
| 1170 | `${got.length} records, first ${got[0] && got[0].number}, highest ${top}`); |
| 1171 | } |
| 1172 | |
| 1173 | // ── B5: `from=0` is malformed and is NOT the newest page. §3.2's reversal, and the |
| 1174 | // reason it matters is that the alias makes a paging client loop for ever. |
| 1175 | { |
| 1176 | const r = note('B5', await get(`${R}?format=json&from=0`)); |
| 1177 | check('B5', '§3.2: "`from=0` is `malformed`, for exactly the reason `limit=0` is." The ' |
| 1178 | + 'alias to "newest" was withdrawn because it makes the parameter non-monotonic at ' |
| 1179 | + 'its own boundary and the obvious paging loop never terminates.', |
| 1180 | '`from=0` is `malformed`', refused(r, 'malformed'), shown(r)); |
| 1181 | } |
| 1182 | |
| 1183 | // ── B6/B7: the limit's two ends, refused and not clamped. §3.2. |
| 1184 | { |
| 1185 | const zero = note('B6', await get(`${R}?format=json&limit=0`)); |
| 1186 | const over = note('B6', await get(`${R}?format=json&limit=${CEILING + 1}`)); |
| 1187 | const neg = note('B6', await get(`${R}?format=json&limit=-1`)); |
| 1188 | check('B6', '§3.2: "`limit` must be 1 to 200, and anything else is `malformed`. It is not ' |
| 1189 | + 'clamped."', 'a limit outside 1..200 is `malformed`', |
| 1190 | refused(zero, 'malformed') && refused(over, 'malformed') && refused(neg, 'malformed'), |
| 1191 | `limit=0 ${shown(zero)}; limit=201 ${shown(over)}; limit=-1 ${shown(neg)}`); |
| 1192 | check('B7', '§3.2: "Clamping looks kinder and is worse: a client asking for 500 silently ' |
| 1193 | + 'receives 200 for ever and never learns it asked wrongly."', |
| 1194 | 'an over-large limit is refused rather than served clamped', |
| 1195 | !(over.status === 200), shown(over)); |
| 1196 | } |
| 1197 | |
| 1198 | // ── B8: the ends that are IN range are served. §3. |
| 1199 | { |
| 1200 | const one = keep('B8', await get(`${R}?format=json&limit=1`)); |
| 1201 | const max = keep('B8', await get(`${R}?format=json&limit=${CEILING}`)); |
| 1202 | const n1 = one.json && Array.isArray(one.json.proposals) ? one.json.proposals.length : -1; |
| 1203 | check('B8', '§3: "?limit=<n> how many, default 50, maximum 200".', |
| 1204 | 'limit=1 and limit=200 are both served', one.status === 200 && max.status === 200 |
| 1205 | && n1 <= 1, `limit=1 gave ${n1} record(s), limit=200 ${max.status}`); |
| 1206 | } |
| 1207 | |
| 1208 | // ── B9: the default page is fifty. Only visible on a repository holding more. |
| 1209 | if (total <= DEFAULT_PAGE) { |
| 1210 | skip('B9', 'the default page of fifty', |
| 1211 | `the subject holds ${total} proposals, which is not more than the default page ` |
| 1212 | + `of ${DEFAULT_PAGE}, so a default cannot be told from "everything there is"`); |
| 1213 | } else { |
| 1214 | const r = keep('B9', await get(`${R}?format=json`)); |
| 1215 | const n = r.json && Array.isArray(r.json.proposals) ? r.json.proposals.length : -1; |
| 1216 | check('B9', '§3: "?limit=<n> how many, default 50, maximum 200".', |
| 1217 | 'a listing with no limit carries fifty', n === DEFAULT_PAGE, `${n} records`); |
| 1218 | } |
| 1219 | |
| 1220 | // ── B10: `total` is after `state` and before `from` and `limit`. §3. |
| 1221 | { |
| 1222 | const one = keep('B10', await get(`${R}?format=json&limit=1`)); |
| 1223 | const same = one.json && one.json.total === total; |
| 1224 | check('B10', '§3: "The response carries `total` -- the count *after* `state` is applied ' |
| 1225 | + 'and before `from` and `limit`".', |
| 1226 | '`total` ignores `limit`', !!same, `unbounded ${total}, limit=1 ${one.json && one.json.total}`); |
| 1227 | } |
| 1228 | |
| 1229 | // ── B11: `state=` empty means all. §3.2. |
| 1230 | { |
| 1231 | const r = keep('B11', await get(`${R}?format=json&state=&limit=${CEILING}`)); |
| 1232 | check('B11', '§3.2: "`state=` with an empty value means all, which is what a form\'s ' |
| 1233 | + '\'any\' option sends."', 'an empty `state` means all', |
| 1234 | r.status === 200 && !!r.json && r.json.total === total, |
| 1235 | `${r.status}, total ${r.json && r.json.total} against ${total}`); |
| 1236 | } |
| 1237 | |
| 1238 | // ── B12: `state=open` selects. §3. |
| 1239 | { |
| 1240 | const r = keep('B12', await get(`${R}?format=json&state=open&limit=${CEILING}`)); |
| 1241 | const got = r.json && Array.isArray(r.json.proposals) ? r.json.proposals : null; |
| 1242 | check('B12', '§3: "?state=open one of open, accepted, declined, done; omit for all".', |
| 1243 | '`state=open` returns only open proposals and a total no larger than all', |
| 1244 | !!got && got.every(p => p.state === 'open') && r.json.total <= total, |
| 1245 | `${got ? got.length : '-'} records, total ${r.json && r.json.total} of ${total}`); |
| 1246 | } |
| 1247 | |
| 1248 | // ── B13: an out-of-vocabulary state. NOT ASSERTED. |
| 1249 | gap('B13', 'an out-of-vocabulary `state` value', |
| 1250 | 'the authority says `state` is "one of open, accepted, declined, done" and never says ' |
| 1251 | + 'what a value outside that set does — refuse it as `malformed`, ignore it, or return ' |
| 1252 | + 'nothing. A mock asserts a closed vocabulary; the contract does not define one.'); |
| 1253 | |
| 1254 | // ── C1: a `format` that is not exactly `json` is NOT refused. §3.2. |
| 1255 | { |
| 1256 | const r = await get(`${R}?format=jsonn`); |
| 1257 | const isRefusal = !!(r.json && typeof r.json.error === 'string'); |
| 1258 | check('C1', '§3.2: "A `format` value that is not exactly `json` falls through to the HTML ' |
| 1259 | + 'answer and is not refused."', 'format=jsonn is not refused', !isRefusal, |
| 1260 | shown(r)); |
| 1261 | } |
| 1262 | |
| 1263 | // ── C2: the RAW QUERY decides the surface, even when the query will not parse. §3.2. |
| 1264 | { |
| 1265 | const r = note('C2', await get(`${R}?format=json&x=%zz`)); |
| 1266 | const isJson = !!r.json && typeof r.json.error === 'string' |
| 1267 | && r.type.split(';')[0].trim() === JSON_CT; |
| 1268 | check('C2', '§3.2: "THE RAW QUERY DECIDES THE SURFACE, and it decides it even when the ' |
| 1269 | + 'query will not parse. ... the first `format` parameter, compared against `json` ' |
| 1270 | + 'without decoding. A client that wrote it plainly is answered in JSON whatever else ' |
| 1271 | + 'is wrong with the request."', |
| 1272 | 'an undecodable query holding a legible format=json is refused IN JSON', isJson, |
| 1273 | shown(r)); |
| 1274 | check('C2b', '§3.1 table: "`malformed` 400 The request itself did not parse."', |
| 1275 | 'and that refusal is `malformed`', refused(r, 'malformed'), shown(r)); |
| 1276 | } |
| 1277 | |
| 1278 | // ── C3: a query fault refuses only the surface that asked for it. §3.2. |
| 1279 | { |
| 1280 | const r = await get(`${R}?limit=999`); |
| 1281 | const isRefusal = !!(r.json && typeof r.json.error === 'string'); |
| 1282 | check('C3', '§3.2: "A query fault refuses only the surface that asked for it. A page asked ' |
| 1283 | + 'for with `?limit=999` and no `format` is still a page."', |
| 1284 | 'a bad parameter with no `format` is not a JSON refusal', !isRefusal, shown(r)); |
| 1285 | } |
| 1286 | |
| 1287 | await paths(top); |
| 1288 | } |
| 1289 | |
| 1290 | /// The path rules and the privacy rule. Split out because they do not need the |
| 1291 | /// subject's contents and must still run when the subject answers nothing. |
| 1292 | async function paths(top) { |
| 1293 | // ── D1..D4: §3.2's four-line table, verbatim. |
| 1294 | const wat = note('D1', await get(`${R}/wat?format=json`)); |
| 1295 | check('D1', '§3.2: "proposals/wat no_proposal".', |
| 1296 | '`proposals/wat` is `no_proposal`', refused(wat, 'no_proposal'), shown(wat)); |
| 1297 | |
| 1298 | const watVote = note('D2', await get(`${R}/wat/vote?format=json`)); |
| 1299 | check('D2', '§3.2: "proposals/wat/vote no_proposal" — "being non-numeric, is `no_proposal` ' |
| 1300 | + 'on the number before the tail is considered".', |
| 1301 | '`proposals/wat/vote` is `no_proposal`', refused(watVote, 'no_proposal'), shown(watVote)); |
| 1302 | |
| 1303 | const watDeep = note('D3', await get(`${R}/wat/deeper?format=json`)); |
| 1304 | check('D3', '§3.2: "proposals/wat/deeper no_proposal <- the number, not the tail". "The ' |
| 1305 | + 'number is judged before the tail ALWAYS, and not only when the tail is `/vote`."', |
| 1306 | '`proposals/wat/deeper` is `no_proposal` — the number is judged first', |
| 1307 | refused(watDeep, 'no_proposal'), shown(watDeep)); |
| 1308 | |
| 1309 | const n = top || 1; |
| 1310 | const numDeep = note('D4', await get(`${R}/${n}/deeper?format=json`)); |
| 1311 | check('D4', '§3.2: "proposals/1/deeper absent <- the number is fine; the route is not". ' |
| 1312 | + '"`absent` means you asked for a route this surface does not have."', |
| 1313 | `\`proposals/${n}/deeper\` is \`absent\` — the route, not the number`, |
| 1314 | refused(numDeep, 'absent'), shown(numDeep)); |
| 1315 | |
| 1316 | // ── D5/D6: privacy. 404 `absent`, never 403, to every caller. |
| 1317 | const nowhere = `/${SUBJECT.account}/no-such-repository-${Date.now().toString(36)}/proposals`; |
| 1318 | const gone = note('D5', await get(`${nowhere}?format=json`)); |
| 1319 | check('D5', '§3.1 table: "`absent` 404 No repository here that you may see."; "Privacy is ' |
| 1320 | + 'unchanged and must stay unchanged: A private repository answers 404 on the JSON routes ' |
| 1321 | + 'exactly as on the HTML ones, never 403."', |
| 1322 | 'a repository that is not there is `absent`, and never a 403', |
| 1323 | refused(gone, 'absent') && gone.status !== 403, shown(gone)); |
| 1324 | |
| 1325 | if (!V.subject) { |
| 1326 | skip('D6', 'privacy to a caller holding a voice', |
| 1327 | 'no ORE_VOICE supplied, so the voiced half of the privacy rule cannot be asked'); |
| 1328 | } else { |
| 1329 | const voiced = note('D6', await get(`${nowhere}?format=json`, V.subject)); |
| 1330 | check('D6', '§3: "a repository the web does not publish answers `absent` to **every** ' |
| 1331 | + 'caller, including one holding a provisioned voice with `pull` on it. That is ' |
| 1332 | + 'intended."', |
| 1333 | 'and it is `absent` to a caller holding a voice too', |
| 1334 | refused(voiced, 'absent') && voiced.status !== 403, shown(voiced)); |
| 1335 | } |
| 1336 | |
| 1337 | // ── D7: private vs absent, which cannot be probed from here. |
| 1338 | gap('D7', 'a private repository against one that is not there', |
| 1339 | 'the authority requires the two to be indistinguishable (§3.1, §3, §9). Telling them ' |
| 1340 | + 'apart from outside is precisely what the rule forbids, so a check would need the NAME ' |
| 1341 | + 'of a private repository handed to it — and a subject supplied from outside is what ' |
| 1342 | + '§3.3 forbids. The property is verified on the forge\'s side by reading `Found::find`, ' |
| 1343 | + 'not here.'); |
| 1344 | } |
| 1345 | |
| 1346 | |
| 1347 | // ┌───────────────────────────────────────────────────────────────────────────────┐ |
| 1348 | // │ E. WRITE REFUSALS — aimed at the subject, and they change no state │ |
| 1349 | // └───────────────────────────────────────────────────────────────────────────────┘ |
| 1350 | |
| 1351 | /// Is this secret one the forge knows on this repository? Established separately |
| 1352 | /// rather than inferred, per §3.3: "when you cannot tell absence from presence, stop |
| 1353 | /// trying to compare and go and establish existence separately." |
| 1354 | async function recognised(route, secret) { |
| 1355 | const r = await get(`${route}?format=json&limit=1`, secret); |
| 1356 | if (r.json && r.json.error === 'unknown') return false; |
| 1357 | if (r.status === 200) return true; |
| 1358 | return null; // something else is wrong; the caller skips rather than guesses |
| 1359 | } |
| 1360 | |
| 1361 | async function writeRefusals() { |
| 1362 | // A repository that does not resolve is `absent` before a credential is looked at |
| 1363 | // (§9, "Privacy is decided before the credential"), so every probe below would earn |
| 1364 | // `absent` and report A1's failure a second time under thirteen other names. |
| 1365 | if (!readLane) { |
| 1366 | skip('E1..E13', 'the credential rule on the subject', |
| 1367 | 'the subject did not answer, and a repository that does not resolve is refused ' |
| 1368 | + '`absent` before a credential is looked at (§9), so every credential probe would ' |
| 1369 | + 'be reporting A1\'s failure again under another name'); |
| 1370 | return; |
| 1371 | } |
| 1372 | |
| 1373 | const probe = form({ title: PROBE_TITLE, body: PROBE_BODY }); |
| 1374 | |
| 1375 | // ── E1..E4: the four spellings that are NO credential. §3.1's trim table. Sent over a |
| 1376 | // socket this file writes itself, because a client library normalises header values |
| 1377 | // and would collapse three of these into one probe. |
| 1378 | const spellings = [ |
| 1379 | ['E1', null, 'an absent header', '§3.1: "absent header unvoiced".'], |
| 1380 | ['E2', ' ', 'a header of spaces', '§3.1: "header of spaces unvoiced".'], |
| 1381 | ['E3', '\t\t', 'a header of tabs', '§3.1: "header of tabs unvoiced".'], |
| 1382 | ['E4', '', 'an explicitly empty header', '§3.1: "header explicitly empty unvoiced".'], |
| 1383 | ]; |
| 1384 | for (const [id, secret, what, cite] of spellings) { |
| 1385 | const r = note(id, await post(`${R}?format=json`, probe, secret)); |
| 1386 | check(id, cite + ' "The rule is a trim, and nothing more. A credential is *not presented* ' |
| 1387 | + 'when the header is absent, or when it carries nothing once whitespace is stripped."', |
| 1388 | `a write with ${what} is \`unvoiced\``, refused(r, 'unvoiced'), shown(r)); |
| 1389 | } |
| 1390 | |
| 1391 | // ── E5..E7: present, unrecognised, and therefore `unknown` — including the single |
| 1392 | // character, which is the distinction §3.1 says no ordinary test would find. |
| 1393 | const junk = [ |
| 1394 | ['E5', '.', 'a single full stop', |
| 1395 | '§3.1: "\\".\\" unknown <- one character is a credential"; "A single non-whitespace ' |
| 1396 | + 'character is a credential. The test is emptiness after trimming and nothing else -- ' |
| 1397 | + 'not plausibility, not a length floor, not a character class."'], |
| 1398 | ['E6', '%%%', 'three percent signs', |
| 1399 | '§3.1: "\\"%%%\\", \\"café-voice\\" unknown".'], |
| 1400 | ['E7', 'not a credential', 'a sentence', |
| 1401 | '§3.1: "\\"not a credential\\" unknown".'], |
| 1402 | ]; |
| 1403 | for (const [id, secret, what, cite] of junk) { |
| 1404 | const r = note(id, await post(`${R}?format=json`, probe, secret)); |
| 1405 | check(id, cite, `a write carrying ${what} is \`unknown\``, |
| 1406 | refused(r, 'unknown'), shown(r)); |
| 1407 | } |
| 1408 | |
| 1409 | // ── E8: the non-ASCII spelling §3.1's table names. Sent as UTF-8 bytes in the header, |
| 1410 | // which is outside RFC 9110's field-value character set; if the forge cannot read it |
| 1411 | // the check is SKIPPED rather than counted, because the probe would be asserting the |
| 1412 | // transport rather than the rule. |
| 1413 | { |
| 1414 | const r = note('E8', await post(`${R}?format=json`, probe, 'café-voice')); |
| 1415 | if (r.json && r.json.error === 'unknown') { |
| 1416 | check('E8', '§3.1: "\\"%%%\\", \\"café-voice\\" unknown".', |
| 1417 | 'a non-ASCII credential is `unknown`', refused(r, 'unknown'), shown(r)); |
| 1418 | } else if (r.json && r.json.error === 'malformed') { |
| 1419 | skip('E8', 'a non-ASCII credential', |
| 1420 | 'the forge answered `malformed` to a header carrying UTF-8 bytes. That is a ' |
| 1421 | + 'statement about the transport, which §3.1\'s table does not legislate, so this ' |
| 1422 | + 'probe is not counted either way. ' + shown(r)); |
| 1423 | } else { |
| 1424 | check('E8', '§3.1: "\\"%%%\\", \\"café-voice\\" unknown".', |
| 1425 | 'a non-ASCII credential is `unknown`', false, shown(r)); |
| 1426 | } |
| 1427 | } |
| 1428 | |
| 1429 | // ── E9: THE TWO DOORS, and the precedence. §4 and §3.2. The FORM field spelling, which |
| 1430 | // is the one the HTML pages read and therefore the one a body can actually carry in. |
| 1431 | { |
| 1432 | const body = form({ title: PROBE_TITLE, body: PROBE_BODY, voice: 'a-secret-in-the-body' }); |
| 1433 | const r = note('E9', await post(`${R}?format=json`, body, null)); |
| 1434 | check('E9', '§4: "The rule is about the FORM field ... Under `?format=json` the header is ' |
| 1435 | + 'authoritative, and a `voice` field in the body is refused -- `malformed`, not ' |
| 1436 | + 'ignored."; §3.2: "Precedence: the two-doors check runs BEFORE the credential is ' |
| 1437 | + 'looked up. A body carrying a `voice` field with no header is `malformed`, not ' |
| 1438 | + '`unvoiced`."', |
| 1439 | 'a `voice` FORM field with no header is `malformed`, not `unvoiced`', |
| 1440 | refused(r, 'malformed'), shown(r)); |
| 1441 | } |
| 1442 | |
| 1443 | // ── E10: both doors at once, with a credential the forge knows. |
| 1444 | const knownSubject = V.subject ? await recognised(R, V.subject) : null; |
| 1445 | if (!V.subject) { |
| 1446 | skip('E10', 'both doors on one request', |
| 1447 | 'no ORE_VOICE supplied, so "both doors on one request" cannot be sent with a ' |
| 1448 | + 'header the forge would otherwise accept'); |
| 1449 | } else if (knownSubject !== true) { |
| 1450 | skip('E10', 'both doors on one request', |
| 1451 | 'the ORE_VOICE supplied is not recognised on the subject, so a refusal here ' |
| 1452 | + 'could not be told from the refusal a wrong credential earns anyway'); |
| 1453 | } else { |
| 1454 | const body = form({ title: PROBE_TITLE, body: PROBE_BODY, voice: 'a-secret-in-the-body' }); |
| 1455 | const r = note('E10', await post(`${R}?format=json`, body, V.subject)); |
| 1456 | check('E10', '§4: "So: form field on the HTML surface, header on the machine surface, ' |
| 1457 | + 'never both on one request." "Refusing rather than ignoring keeps the confusion ' |
| 1458 | + 'loud."', |
| 1459 | 'a good header AND a `voice` form field is still `malformed`', |
| 1460 | refused(r, 'malformed'), shown(r)); |
| 1461 | } |
| 1462 | |
| 1463 | // ── E11/E12: §9's `d`. Sent with a recognised voice, because the contract fixes the |
| 1464 | // precedence of the two-doors check and of privacy, and fixes NO ordering between a |
| 1465 | // malformed body and an unrecognised credential — so an unvoiced probe here could be |
| 1466 | // answered either way honestly and the check would be asserting an undefined rule. |
| 1467 | // |
| 1468 | // The number is one that EXISTS, for the same reason: the authority does not say |
| 1469 | // whether a malformed body or an unknown proposal number is reported first, so a |
| 1470 | // probe at a made-up number could be answered `no_proposal` perfectly honestly. |
| 1471 | const canVote = knownSubject === true && await voteRouteExists(); |
| 1472 | const head = await get(`${R}?format=json&limit=1`); |
| 1473 | const someN = (head.json && Array.isArray(head.json.proposals) && head.json.proposals[0]) |
| 1474 | ? head.json.proposals[0].number : null; |
| 1475 | if (someN === null) { |
| 1476 | skip('E11', 'a vote with no `d`', |
| 1477 | 'the subject holds no proposal to vote on, and a vote at ' |
| 1478 | + 'a number that does not exist could be answered `no_proposal` rather than ' |
| 1479 | + '`malformed` without contradicting anything the authority says'); |
| 1480 | skip('E12', 'a vote of `d=2`', 'the subject holds no proposal to vote on'); |
| 1481 | } else if (!V.subject || knownSubject !== true) { |
| 1482 | skip('E11', 'a vote with no `d`', |
| 1483 | 'needs a recognised ORE_VOICE: with a bad credential the contract does not say ' |
| 1484 | + 'whether `malformed` or the credential refusal comes first (GAP, below)'); |
| 1485 | skip('E12', 'a vote of `d=2`', |
| 1486 | 'needs a recognised ORE_VOICE, for the same reason as E11'); |
| 1487 | } else if (!canVote) { |
| 1488 | skip('E11', 'a vote with no `d`', |
| 1489 | '§9\'s vote route does not answer on this forge, so a vote cannot be sent'); |
| 1490 | skip('E12', 'a vote of `d=2`', |
| 1491 | '§9\'s vote route does not answer on this forge, so a vote cannot be sent'); |
| 1492 | } else { |
| 1493 | const nowt = note('E11', await post(`${R}/${someN}/vote?format=json`, '', V.subject)); |
| 1494 | check('E11', '§9: "A vote with no `d` at all is `malformed`, never a withdrawal: treating a ' |
| 1495 | + 'lost field as an instruction to delete would turn a dropped parameter into silent ' |
| 1496 | + 'data loss."', 'a vote with no `d` is `malformed`', |
| 1497 | refused(nowt, 'malformed'), shown(nowt)); |
| 1498 | |
| 1499 | const two = note('E12', await post(`${R}/${someN}/vote?format=json`, 'd=2', V.subject)); |
| 1500 | check('E12', '§9: "Any `d` that is not `1`, `-1` or `0` is `malformed` too."', |
| 1501 | 'a vote of `d=2` is `malformed`', refused(two, 'malformed'), shown(two)); |
| 1502 | } |
| 1503 | |
| 1504 | // ── E13: the wrong role. `unpermitted` is the one refusal that needs a voice the forge |
| 1505 | // knows and a role it will not accept, so it needs a second credential or nothing. |
| 1506 | const knownReader = V.reader ? await recognised(R, V.reader) : null; |
| 1507 | if (!V.reader) { |
| 1508 | skip('E13', 'the role floor, `unpermitted`', |
| 1509 | 'no ORE_VOICE_READER supplied, so `unpermitted` cannot be provoked. It needs a ' |
| 1510 | + 'voice the forge RECOGNISES whose role is below `pull`; anything else earns ' |
| 1511 | + '`unknown` and would prove nothing'); |
| 1512 | } else if (knownReader !== true) { |
| 1513 | skip('E13', 'the role floor, `unpermitted`', |
| 1514 | 'the ORE_VOICE_READER supplied is not recognised on the subject, so a 403 could ' |
| 1515 | + 'not be told from the 401 an unknown credential earns'); |
| 1516 | } else if (!canVote || someN === null) { |
| 1517 | skip('E13', 'the role floor, `unpermitted`', '§9\'s vote route does not answer on this ' |
| 1518 | + 'forge, or there is no proposal to vote on. A vote is the one write whose body the ' |
| 1519 | + 'authority defines, so without it the role floor cannot be probed without inventing ' |
| 1520 | + 'a body shape (see the E16 gap)'); |
| 1521 | } else { |
| 1522 | const r = note('E13', await post(`${R}/${someN}/vote?format=json`, 'd=1', V.reader)); |
| 1523 | check('E13', '§9: "Voting needs the `pull` role — the same floor as opening a proposal and ' |
| 1524 | + 'commenting on one."; §3.1 table: "`unpermitted` 403 Recognised, but this role may ' |
| 1525 | + 'not do this."', |
| 1526 | 'a voice below `pull` voting is `unpermitted`', refused(r, 'unpermitted'), shown(r)); |
| 1527 | } |
| 1528 | |
| 1529 | // ── E14: a JSON-encoded write body. NOT ASSERTED. |
| 1530 | gap('E14', 'a JSON-encoded write body', |
| 1531 | '§9 says "The body is form-encoded, like every other write on this surface", which states ' |
| 1532 | + 'what a client sends and not what the forge does with anything else. Nothing in the ' |
| 1533 | + 'authority says a JSON body is refused. A mock asserts `malformed`; that is the ' |
| 1534 | + 'invented-rule shape §3.3 warns about, so this file does not send the probe.'); |
| 1535 | |
| 1536 | // ── E15: the ordering gap E11 and E12 lean on, recorded whether or not they ran. |
| 1537 | gap('E15', 'precedence between a malformed body and the credential', |
| 1538 | 'the authority fixes two precedences — the two-doors check before the credential (§3.2) ' |
| 1539 | + 'and privacy before the credential (§9) — and fixes no others. Whether a malformed body ' |
| 1540 | + 'or an unrecognised credential is reported first is undefined, so every malformed-body ' |
| 1541 | + 'probe here carries a credential the forge knows.'); |
| 1542 | |
| 1543 | // ── E16: the write body's own field names, which the authority never gives. |
| 1544 | gap('E16', 'the write body\'s field names', |
| 1545 | '§4 and §9 give the write ROUTES and, for a vote, the field `d`. Neither names the fields ' |
| 1546 | + 'that open a proposal or add a comment. `title`, `body`, `build` and `said` are read out ' |
| 1547 | + 'of www/js/improve.js and gateway/src/handlers/improve.rs, which is what the product ' |
| 1548 | + 'sends, so the F-series asserts that the forge accepts THAT — not that the authority ' |
| 1549 | + 'requires it. If the forge ever renames one, this suite reports a failure the contract ' |
| 1550 | + 'cannot adjudicate.'); |
| 1551 | } |
| 1552 | |
| 1553 | /// Whether §9's vote route answers at all on this forge. Established from the panel's |
| 1554 | /// own tell: §9 records that "the absence of the key means the forge has not got there |
| 1555 | /// yet, and the zero object means it has and nobody has voted". |
| 1556 | async function voteRouteExists() { |
| 1557 | const r = await get(`${R}?format=json&limit=1`); |
| 1558 | const p = r.json && Array.isArray(r.json.proposals) ? r.json.proposals[0] : null; |
| 1559 | return !!(p && 'votes' in p); |
| 1560 | } |
| 1561 | |
| 1562 | |
| 1563 | // ┌───────────────────────────────────────────────────────────────────────────────┐ |
| 1564 | // │ F. WRITES THAT SHOULD SUCCEED — into the sandbox, never the product's backlog │ |
| 1565 | // └───────────────────────────────────────────────────────────────────────────────┘ |
| 1566 | |
| 1567 | async function sandboxWrites() { |
| 1568 | if (!V.sandbox) { |
| 1569 | skip('F1..F9', 'every write that should succeed', |
| 1570 | `no ORE_VOICE_SANDBOX supplied, so nothing is written to ` |
| 1571 | + `${SANDBOX.account}/${SANDBOX.repo} and every write-success check is unmade. ` |
| 1572 | + 'THE SUBJECT IS NOT A FALLBACK: a probe proposal in the product\'s own backlog is ' |
| 1573 | + 'exactly what the sandbox exists to prevent (§3.3).'); |
| 1574 | return; |
| 1575 | } |
| 1576 | |
| 1577 | // Listed WITH the sandbox voice, because F9 needs to know which proposals this voice |
| 1578 | // has already voted on and `mine` is absent from an unvoiced read (§9). |
| 1579 | let list; |
| 1580 | try { list = await get(`${S}?format=json&limit=${CEILING}`, V.sandbox); } |
| 1581 | catch (e) { |
| 1582 | skip('F1..F9', 'every write that should succeed', |
| 1583 | `the sandbox ${SANDBOX.account}/${SANDBOX.repo} could not be reached: ` |
| 1584 | + String(e && e.message)); |
| 1585 | return; |
| 1586 | } |
| 1587 | if (!(list.status === 200 && list.json && Array.isArray(list.json.proposals))) { |
| 1588 | skip('F1..F9', 'every write that should succeed', |
| 1589 | `the sandbox ${SANDBOX.account}/${SANDBOX.repo} does not answer a listing ` |
| 1590 | + `(${shown(list)}). Create it on the forge, public, with the ORE_VOICE_SANDBOX voice ` |
| 1591 | + 'provisioned at `pull` or better. Nothing is written to the subject instead.'); |
| 1592 | return; |
| 1593 | } |
| 1594 | if (await recognised(S, V.sandbox) !== true) { |
| 1595 | skip('F1..F9', 'every write that should succeed', |
| 1596 | 'the ORE_VOICE_SANDBOX supplied is not recognised on the sandbox, so every ' |
| 1597 | + 'write below would be refused for a reason that is not the forge\'s'); |
| 1598 | return; |
| 1599 | } |
| 1600 | |
| 1601 | // ── F1: opening one. §9's "What a write returns". |
| 1602 | // |
| 1603 | // The FIELD NAMES here are the client's (`www/js/improve.js`: title, body, build) and |
| 1604 | // NOT the authority's, which names no write-body fields anywhere. That is a real gap |
| 1605 | // and it is recorded below; what this check therefore asserts is that the forge |
| 1606 | // accepts what the client actually sends, which is the property the product needs. |
| 1607 | const opened = keep('F1', await post(`${S}?format=json`, |
| 1608 | form({ |
| 1609 | title: `conformance run ${new Date().toISOString()}`, |
| 1610 | body: 'Opened by dev/verify_conformance.mjs against the sandbox. Safe to delete.', |
| 1611 | build: 'daimond-conformance', |
| 1612 | }), V.sandbox)); |
| 1613 | // A throttle here is the forge spending this voice's allowance, not a forge that fails |
| 1614 | // the contract, so F1 is not asserted at all. `throttled` is checked BEFORE the shape, |
| 1615 | // because a red F1 reading "429 error=throttled" says the opposite of what it means. |
| 1616 | if (refused(opened, 'throttled')) { |
| 1617 | skip('F1..F9', 'every write that should succeed', |
| 1618 | 'the sandbox voice\'s write allowance is spent, so the forge answered `throttled` ' |
| 1619 | + 'and NOTHING below was written. That is §2\'s throttle working: oregami allows ' |
| 1620 | + 'twenty writes a voice a minute and this suite spends two a run, so several runs ' |
| 1621 | + 'in quick succession earn one. It recovers on its own — wait a minute and run ' |
| 1622 | + 'again. The throttle\'s shape was read by G1..G4 like any other refusal.'); |
| 1623 | return; |
| 1624 | } |
| 1625 | const rec = opened.json; |
| 1626 | const isDetail = !!rec && !rec.error && typeof rec.number === 'number' |
| 1627 | && typeof rec.body === 'string' && Array.isArray(rec.discussion) |
| 1628 | && FIELDS.every(f => f in rec); |
| 1629 | check('F1', '§9, "What a write returns": "Every write under `?format=json` returns the detail ' |
| 1630 | + 'shape of the record it changed — a new proposal, a comment, a decision and a vote ' |
| 1631 | + 'alike."; §3, "One proposal", for what that shape is.', |
| 1632 | 'opening a proposal answers with the detail shape', isDetail, shown(opened)); |
| 1633 | if (!isDetail) { |
| 1634 | skip('F2..F9', 'everything after the sandbox write', |
| 1635 | 'the sandbox write did not answer a record, so nothing below it can be measured. ' |
| 1636 | + 'If the answer was `malformed`, read gap E16: the authority names no write-body ' |
| 1637 | + 'field names and these are the client\'s.'); |
| 1638 | return; |
| 1639 | } |
| 1640 | const n = rec.number; |
| 1641 | |
| 1642 | // ── F2/F3: what a brand-new proposal carries. §9. |
| 1643 | check('F2', '§9: "`votes` on a proposal nobody has voted on is {"for":0,"against":0}" -- the ' |
| 1644 | + 'zero object, never null and never absent."', |
| 1645 | 'a proposal nobody has voted on carries the zero object', |
| 1646 | !!rec.votes && canon(rec.votes) === '{"against":0,"for":0}', JSON.stringify(rec.votes)); |
| 1647 | check('F3', '§9: "When the request carries a voice, the record also carries `mine`, one of ' |
| 1648 | + '`1`, `-1` or `null`."', 'and `mine`, present and null, on a write that carried a voice', |
| 1649 | 'mine' in rec && rec.mine === null, `mine=${JSON.stringify(rec.mine)}`); |
| 1650 | |
| 1651 | // ── F4: commenting. The `said` field is the client's spelling; same gap as F1. |
| 1652 | const said = `A comment from the conformance run at ${new Date().toISOString()}.`; |
| 1653 | const commented = keep('F4', await post(`${S}/${n}?format=json`, form({ said }), V.sandbox)); |
| 1654 | // The allowance counts proposals and comments together, so it can run out BETWEEN the |
| 1655 | // two writes of one run. Seen once, as a red F4 under an unrelated `--break`, which is |
| 1656 | // how a throttle gets credited to whichever break happened to be in force. |
| 1657 | if (refused(commented, 'throttled')) { |
| 1658 | skip('F4..F9', 'commenting, and every write after it', |
| 1659 | 'the sandbox voice\'s write allowance ran out between the proposal and the comment ' |
| 1660 | + '— oregami counts the two against one allowance of twenty a minute — so the ' |
| 1661 | + 'comment was refused `throttled` and nothing after it was driven. Not a ' |
| 1662 | + 'conformance failure; wait a minute and run again.'); |
| 1663 | return; |
| 1664 | } |
| 1665 | const cr = commented.json; |
| 1666 | check('F4', '§9: "Every write under `?format=json` returns the detail shape of the record it ' |
| 1667 | + 'changed"; §3: "`comments` is a count everywhere; the discussion is `discussion`."', |
| 1668 | 'a comment answers the detail shape, with the count up by one and the entry in the discussion', |
| 1669 | !!cr && !cr.error && cr.comments === rec.comments + 1 |
| 1670 | && Array.isArray(cr.discussion) && cr.discussion.some(d => d && d.said === said), |
| 1671 | shown(commented)); |
| 1672 | |
| 1673 | // ── F5..F8: §9's votes, in the one order that tells them apart. |
| 1674 | if (!(rec.votes && await voteRouteOn(S))) { |
| 1675 | skip('F5..F8', 'casting, repeating, moving and withdrawing a vote', |
| 1676 | '§9\'s vote route does not answer on this forge, so casting, repeating, ' |
| 1677 | + 'moving and withdrawing cannot be driven. §9 itself records that a forge without it ' |
| 1678 | + 'carries no `votes` key.'); |
| 1679 | } else { |
| 1680 | const cast = keep('F5', await post(`${S}/${n}/vote?format=json`, 'd=1', V.sandbox)); |
| 1681 | // Votes have their own, far larger allowance — two hundred a minute against the |
| 1682 | // twenty for proposals and comments — so this is the unlikely one. Guarded anyway, |
| 1683 | // because the cost of not guarding it is a red line blaming the forge for §2. |
| 1684 | if (refused(cast, 'throttled')) { |
| 1685 | skip('F5..F9', 'casting, repeating, moving and withdrawing a vote, and the ' |
| 1686 | + '`changed` bump', |
| 1687 | 'the sandbox voice\'s VOTE allowance is spent, so the first vote was refused ' |
| 1688 | + '`throttled`. Not a conformance failure; wait a minute and run again.'); |
| 1689 | return; |
| 1690 | } |
| 1691 | check('F5', '§9: "A vote\'s answer therefore carries the new `votes` and the caller\'s own ' |
| 1692 | + '`mine`, so a panel needs no second request to redraw the control the tester just ' |
| 1693 | + 'tapped."', |
| 1694 | 'a vote answers the record with the new tally and the caller\'s own `mine`', |
| 1695 | !!cast.json && cast.json.votes && cast.json.votes.for === 1 |
| 1696 | && cast.json.votes.against === 0 && cast.json.mine === 1, shown(cast)); |
| 1697 | |
| 1698 | const again = keep('F6', await post(`${S}/${n}/vote?format=json`, 'd=1', V.sandbox)); |
| 1699 | check('F6', '§9: "A second `POST` carrying the same `d` is idempotent and not an ' |
| 1700 | + 'increment."', 'the same vote again does not increment', |
| 1701 | !!again.json && again.json.votes && again.json.votes.for === 1 |
| 1702 | && again.json.mine === 1, shown(again)); |
| 1703 | |
| 1704 | const moved = keep('F7', await post(`${S}/${n}/vote?format=json`, 'd=-1', V.sandbox)); |
| 1705 | check('F7', '§9: "the opposite `d` moves the vote."', |
| 1706 | 'the opposite vote moves rather than adds', |
| 1707 | !!moved.json && moved.json.votes && moved.json.votes.for === 0 |
| 1708 | && moved.json.votes.against === 1 && moved.json.mine === -1, shown(moved)); |
| 1709 | |
| 1710 | const drop = keep('F8', await post(`${S}/${n}/vote?format=json`, 'd=0', V.sandbox)); |
| 1711 | check('F8', '§9: "`0` withdraws"; "A withdrawal by a voice that never voted is a 200 no-op ' |
| 1712 | + 'with `mine` null. It asked for a state and that state is what it gets."', |
| 1713 | 'a withdrawal empties the tally and leaves `mine` null', |
| 1714 | !!drop.json && drop.json.votes && canon(drop.json.votes) === '{"against":0,"for":0}' |
| 1715 | && drop.json.mine === null, shown(drop)); |
| 1716 | |
| 1717 | const again0 = keep('F8b', await post(`${S}/${n}/vote?format=json`, 'd=0', V.sandbox)); |
| 1718 | check('F8b', '§9: "A withdrawal by a voice that never voted is a 200 no-op with `mine` ' |
| 1719 | + 'null."', 'and withdrawing when nothing is held is a no-op, not a refusal', |
| 1720 | !!again0.json && !again0.json.error && again0.json.mine === null, shown(again0)); |
| 1721 | |
| 1722 | // ── F9: a vote bumps `changed`. Only visible on a record that was not written in |
| 1723 | // this run: on a proposal opened seconds ago, a forge that never touched |
| 1724 | // `changed` would be indistinguishable from one that did. §3.3's own shape. |
| 1725 | // It must also be one this voice has NOT voted on, or an idempotent repeat is |
| 1726 | // what gets sent and the authority does not say whether a vote that changed |
| 1727 | // nothing still bumps `changed`. |
| 1728 | const old = (list.json.proposals || []).find(p => |
| 1729 | typeof p.changed === 'number' |
| 1730 | && p.changed < Math.floor(Date.now() / 1000) - 300 |
| 1731 | && p.mine === null); |
| 1732 | if (!old) { |
| 1733 | skip('F9', '`changed` bumped by a vote', |
| 1734 | 'the sandbox holds no proposal older than five minutes that this voice has ' |
| 1735 | + 'not voted on, and a `changed` bump on a record written seconds ago cannot be ' |
| 1736 | + 'told from one that was never touched. A later run will have one.'); |
| 1737 | } else { |
| 1738 | const before = old.changed; |
| 1739 | const bumped = keep('F9', await post(`${S}/${old.number}/vote?format=json`, 'd=1', |
| 1740 | V.sandbox)); |
| 1741 | check('F9', '§9: "A vote bumps `changed`. The record changed."', |
| 1742 | 'a vote on an older proposal bumps `changed`', |
| 1743 | !!bumped.json && typeof bumped.json.changed === 'number' |
| 1744 | && bumped.json.changed > before, `${before} -> ${bumped.json && bumped.json.changed}`); |
| 1745 | // Leave the tally as it was found. `changed` stays bumped, which is what the |
| 1746 | // check just proved and is the sandbox's business. |
| 1747 | await post(`${S}/${old.number}/vote?format=json`, 'd=0', V.sandbox); |
| 1748 | } |
| 1749 | } |
| 1750 | } |
| 1751 | |
| 1752 | /// Whether the sandbox's records carry §9's tally, read the same way as the subject's. |
| 1753 | async function voteRouteOn(route) { |
| 1754 | const r = await get(`${route}?format=json&limit=1`); |
| 1755 | const p = r.json && Array.isArray(r.json.proposals) ? r.json.proposals[0] : null; |
| 1756 | return !!(p && 'votes' in p); |
| 1757 | } |
| 1758 | |
| 1759 | |
| 1760 | // ┌───────────────────────────────────────────────────────────────────────────────┐ |
| 1761 | // │ G. THE SHAPE OF EVERY REFUSAL SEEN — one property, read over the whole run │ |
| 1762 | // └───────────────────────────────────────────────────────────────────────────────┘ |
| 1763 | |
| 1764 | async function refusalShapes() { |
| 1765 | if (!refusals.length) { |
| 1766 | skip('G1..G4', 'the shape of a refusal', |
| 1767 | 'no refusal was seen in this run at all, which means the probes above did ' |
| 1768 | + 'not reach the forge — read their failures rather than this line'); |
| 1769 | return; |
| 1770 | } |
| 1771 | |
| 1772 | // ── G1: the media type, with no charset parameter. §3.2. |
| 1773 | { |
| 1774 | const wrong = refusals.filter(({ r }) => { |
| 1775 | const parts = r.type.split(';').map(s => s.trim()); |
| 1776 | return parts[0].toLowerCase() !== JSON_CT || parts.length > 1; |
| 1777 | }).map(({ where, r }) => `${where}:${r.type || '(none)'}`); |
| 1778 | check('G1', '§3.2: "A refusal is served as `application/json`, no `charset` — JSON is UTF-8 ' |
| 1779 | + 'by definition and the media type has no such parameter."', |
| 1780 | `all ${refusals.length} refusals are application/json with no charset`, |
| 1781 | wrong.length === 0, wrong.slice(0, 5).join(' ')); |
| 1782 | } |
| 1783 | |
| 1784 | // ── G2: the token is one of the nine, and `said` is there for a person to read. §3.1. |
| 1785 | { |
| 1786 | const wrong = refusals.filter(({ r }) => |
| 1787 | !(r.json.error in TABLE) || typeof r.json.said !== 'string' || !r.json.said.trim()) |
| 1788 | .map(({ where, r }) => `${where}:${r.json.error}`); |
| 1789 | check('G2', '§3.1: "`error` is the stable token a client branches on; `said` is the ' |
| 1790 | + 'sentence a person reads"; the table lists nine tokens and "nine tokens are emitted ' |
| 1791 | + 'and nine are now listed."', |
| 1792 | 'every refusal names one of the nine tokens and carries a sentence', |
| 1793 | wrong.length === 0, wrong.slice(0, 5).join(' ')); |
| 1794 | } |
| 1795 | |
| 1796 | // ── G3: token and status agree with §3.1's table. |
| 1797 | { |
| 1798 | const wrong = refusals.filter(({ r }) => TABLE[r.json.error] !== r.status) |
| 1799 | .map(({ where, r }) => `${where}:${r.json.error}=${r.status}, expected ${TABLE[r.json.error]}`); |
| 1800 | check('G3', '§3.1\'s table: absent 404, unvoiced 401, unknown 401, unpermitted 403, ' |
| 1801 | + 'throttled 429, malformed 400, no_proposal 404, unsupported 405, internal 500.', |
| 1802 | 'every refusal\'s status is the one its token carries in the table', |
| 1803 | wrong.length === 0, wrong.slice(0, 4).join('; ')); |
| 1804 | } |
| 1805 | |
| 1806 | // ── G4: `because` appears only on `throttled`. §3.1. |
| 1807 | { |
| 1808 | const wrong = refusals.filter(({ r }) => |
| 1809 | ('because' in r.json) && (r.json.error !== 'throttled' |
| 1810 | || !BECAUSE.includes(r.json.because))) |
| 1811 | .map(({ where, r }) => `${where}:${r.json.error}/${r.json.because}`); |
| 1812 | check('G4', '§3.1: "`because` appears only on `throttled`." and the table: "`throttled` 429 ' |
| 1813 | + '`because` is `address`, `voice` or `failing`."', |
| 1814 | '`because` appears only on `throttled`, and only as one of its three reasons', |
| 1815 | wrong.length === 0, wrong.slice(0, 5).join(' ')); |
| 1816 | } |
| 1817 | |
| 1818 | // ── G5: canonical bytes, over every JSON body this run received. |
| 1819 | { |
| 1820 | // One answer may sit in both ledgers -- a refusal is a body too -- so it is read |
| 1821 | // once, by identity, rather than counted twice in the total this check reports. |
| 1822 | const seen = new Map(); |
| 1823 | for (const b of bodies.concat(refusals)) if (!seen.has(b.r)) seen.set(b.r, b); |
| 1824 | const all = [...seen.values()]; |
| 1825 | const wrong = []; |
| 1826 | for (const { where, r } of all) { |
| 1827 | let want = null; |
| 1828 | try { want = canon(r.json); } catch (e) { wrong.push(`${where}:${e.message}`); continue; } |
| 1829 | if (want !== r.text) wrong.push(`${where}:${r.text.slice(0, 40)}`); |
| 1830 | } |
| 1831 | check('G5', '§3: "The body is RFC 8785 canonical JSON, produced by `Dat::json_canonical()`"; ' |
| 1832 | + '§3.2: "`json_canonical` sorts keys by UTF-16 code unit"; §3.1: "Under `?format=json` ' |
| 1833 | + 'every answer is JSON, refusals included."; §9: a write "returns the detail shape of ' |
| 1834 | + 'the record it changed" — the same representation through the same encoder.', |
| 1835 | `all ${all.length} JSON bodies are canonical bytes`, wrong.length === 0, |
| 1836 | wrong.slice(0, 3).join(' | ')); |
| 1837 | } |
| 1838 | |
| 1839 | // ── G6: the throttle sentence. NOT ASSERTED. |
| 1840 | gap('G6', 'what the throttle sentence may not name', |
| 1841 | '§3.1 requires that "The throttle sentence must not name which allowance was spent", and ' |
| 1842 | + 'the authority names no allowances for the forge — so a mechanical check would have to ' |
| 1843 | + 'invent the word list it searches for, which is §3.3\'s third and worst shape. Read the ' |
| 1844 | + 'sentence by eye when a throttle is next seen.'); |
| 1845 | skip('G7', '`unsupported`', |
| 1846 | '`unsupported` (405) is not provoked: §3.1 records that it "is unreachable from any ' |
| 1847 | + 'honest client", and a probe would assert the door\'s verb guard rather than this ' |
| 1848 | + 'surface. It is NOT dead vocabulary — §3.1 says so explicitly — and this line is why no ' |
| 1849 | + 'check reaches it.'); |
| 1850 | skip('G8', '`internal`', |
| 1851 | '`internal` (500) is not provoked: a fault inside the forge cannot be induced from ' |
| 1852 | + 'outside, and breaking a live forge to watch it say so is not a conformance check.'); |
| 1853 | // A throttle is never provoked on purpose, but one CAN arrive — spending a voice's write |
| 1854 | // allowance over several quick runs is enough — and a line saying none was seen when one |
| 1855 | // was is the stale report this file exists to avoid. So say which it was. |
| 1856 | const throttles = refusals.filter(({ r }) => r.json.error === 'throttled'); |
| 1857 | skip('G9', '`throttled`', throttles.length |
| 1858 | ? `${throttles.length} throttle(s) arrived unasked — at ${throttles.map(t => t.where) |
| 1859 | .join(', ')} — and G1..G4 read their shape like any other refusal. NO CHECK ` |
| 1860 | + 'PROVOKES one: earning it on purpose takes a flood, §2 shows what a flood on this ' |
| 1861 | + 'path costs everybody, and §2.1 puts a failure backstop behind it. What is unmade ' |
| 1862 | + 'is the write series the throttle stopped, named on its own line above.' |
| 1863 | : '`throttled` (429) is not provoked: earning one takes a flood, §2 shows what a ' |
| 1864 | + 'flood on this path costs everybody, and §2.1 puts a failure backstop behind it. ' |
| 1865 | + 'Its shape is still read by G1..G4 if one arrives.'); |
| 1866 | } |
| 1867 | |
| 1868 | |
| 1869 | // ┌───────────────────────────────────────────────────────────────────────────────┐ |
| 1870 | // │ THE REPORT │ |
| 1871 | // └───────────────────────────────────────────────────────────────────────────────┘ |
| 1872 | |
| 1873 | /// One line naming a set of clauses, folded so the same clause twice reads once. |
| 1874 | /// |
| 1875 | /// A DECLARATION rather than a `const` arrow, because `finish()` is called at module |
| 1876 | /// top level well above this point. As an arrow it sat in the temporal dead zone and |
| 1877 | /// threw on every run that had anything to skip -- the instrument crashing precisely |
| 1878 | /// when it had gaps to report. |
| 1879 | function named(list) { |
| 1880 | return [...new Set(list.map(s => s.clause))].join('; '); |
| 1881 | } |
| 1882 | |
| 1883 | function finish() { |
| 1884 | const undef = skipped.filter(s => s.undefined_); |
| 1885 | const unasked = skipped.filter(s => !s.undefined_); |
| 1886 | |
| 1887 | // THE SUMMARY LINE NAMES WHAT WAS NOT ASKED. "37 passed" beside four silent skips |
| 1888 | // reads as coverage, and a skipped surface reads as a clean one -- which is §3.3's |
| 1889 | // own defect class arriving one level up, in the instrument rather than the subject. |
| 1890 | console.log(`\n${ok.length} passed, ${bad.length} failed, ${skipped.length} skipped` |
| 1891 | + (undef.length ? ` (undefined: ${named(undef)})` : '') |
| 1892 | + (unasked.length ? ` (unasked: ${named(unasked)})` : '')); |
| 1893 | |
| 1894 | if (unasked.length) { |
| 1895 | console.log('\nNOT ASKED — each of these is a surface this run says nothing about:'); |
| 1896 | for (const s of unasked) console.log(` ${s.id} ${s.clause} — ${redact(s.why)}`); |
| 1897 | } |
| 1898 | if (GAPS.length) { |
| 1899 | console.log('\nUNDEFINED IN THE AUTHORITY — reported rather than guessed (§3.3):'); |
| 1900 | for (const g of GAPS) console.log(` ${g.id} ${g.clause}\n ${redact(g.what)}`); |
| 1901 | } |
| 1902 | if (flag('cites')) { |
| 1903 | console.log('\nEvery check and the sentence it rests on:'); |
| 1904 | for (const id of Object.keys(CITES).sort()) console.log(` ${id} ${CITES[id]}`); |
| 1905 | } |
| 1906 | |
| 1907 | console.log(`\nsubject ${SUBJECT.account}/${SUBJECT.repo} at ${BASE.url}` |
| 1908 | + ` sandbox ${SANDBOX.account}/${SANDBOX.repo}`); |
| 1909 | if (!readLane) { |
| 1910 | console.log('THE SUBJECT DID NOT ANSWER. Whatever passed above, this run says nothing ' |
| 1911 | + 'about the repository the product names — which is the whole reason this file ' |
| 1912 | + 'exists (§3.3).'); |
| 1913 | } |
| 1914 | |
| 1915 | // A break EXPECTS a failure. A break that changed nothing is a finding about the |
| 1916 | // check it was aimed at, not a clean run, so it exits non-zero and says which. |
| 1917 | if (BREAK) { |
| 1918 | console.log(bad.length |
| 1919 | ? `\nbreak '${BREAK}' turned red: ${bad.join(', ')}` |
| 1920 | : `\nBREAK '${BREAK}' CHANGED NOTHING. The check it targets did not run, or a ` |
| 1921 | + 'different rule is quietly holding the property. That is a finding, not a ' |
| 1922 | + 'fixture problem — chase it until it is explained.'); |
| 1923 | process.exit(bad.length ? 0 : 1); |
| 1924 | } |
| 1925 | |
| 1926 | // A run that failed nothing because it asked nothing is the failure this suite is |
| 1927 | // named after, so an unanswered subject is an exit code and not a remark. |
| 1928 | process.exit(bad.length || !readLane ? 1 : 0); |
| 1929 | } |