Oregami
Repositories/oxedyne/daimond

oxedyne/daimond/dev/verify_predrop.mjs

16.7 KiB, 1 run

created by r2519314175:595, which is this file's identity for as long as the history lasts, whatever it is later renamed to

download · who wrote it · its history

1// verify_predrop.mjs — a turn that dies BEFORE THE FIRST TOKEN is handed back, whatever
2// the browser calls a dead fetch.
3//
4// THE DEFECT. `dev/verify_dropped.mjs` proves the mid-stream case: three words arrive, the
5// socket dies, the turn comes back badged with a Continue button. The case BEFORE the first
6// token did not work, and it did not work on one platform only.
7//
8// A rejected `fetch` reaches JavaScript as the browser's own sentence. Chromium says
9// `TypeError: Failed to fetch`; WebKit says `TypeError: Load failed`. `isUnreachable` in
10// www/js/daimond.js held the first and not the second, so on iOS a turn that died before any
11// token fell through to the terminal branch of `runTurn`'s catch: an error line was written
12// and `J.clearTurn` deleted the turn from the write-ahead log. No badge, no Continue, and the
13// whole recovery apparatus built for exactly this case never ran. `failureClass` had the same
14// gap, which is why telemetry counted these as `other` and the numbers never showed it.
15//
16// Mid-stream breaks were unaffected: their text carries `read stream chunk failed`, which is
17// the `err!` text itself and crosses the wasm boundary. THAT ASYMMETRY IS WHAT THIS FILE
18// MEASURES.
19//
20// AND WHY THE REGEX IS NOT THE FIX. `TransportErr` (src/llm.rs) keeps `reason` and `err`
21// apart, and only `err` used to leave the module -- so `could not reach the provider`, which
22// is Daimond's own wording and identical on every browser, never reached the classifier that
23// quoted it. `TransportErr::crossed` now puts the reason in front of the error. Adding
24// `Load failed` to the regex fixes iOS; carrying the reason means the next browser's wording
25// does not have to be discovered from a bug report.
26//
27// A SECOND DEFECT, found while proving the first. `_unloading` was set by `pagehide` and
28// never cleared. `pagehide` fires when a tab enters the back/forward cache, which is what iOS
29// does to a home-screen PWA on every app switch -- so ONE switch away and back put the tab in
30// "we are unloading" mode for the rest of the sitting, and after that a dropped turn showed
31// the user nothing at all: no error, no badge, no button, the spinner simply stopping. Check 5
32// is that one, and it is the check that is closest to the symptom actually reported.
33//
34// HOW THE FAILURE IS PRODUCED. `window.fetch` is replaced in the page with one that rejects
35// with `new TypeError('Load failed')` -- WebKit's own wording, verbatim -- for requests to the
36// provider, and only while armed. The engine reaches `fetch` through the window object at call
37// time, so the wasm sees the replacement. This is a simulation of Safari and is honest about
38// it: what it proves is that the APP classifies WebKit's sentence correctly. Whether iOS
39// produces that sentence in the field is a question for a device, and the owner is testing it
40// on one.
41//
42// FOUR PROPERTIES, and one more:
43//
44// 1. THE TURN ENDS rather than hanging on the road. (The retry ladder runs first -- eight
45// attempts over up to 120 s -- so this is not a five-second wait.)
46// 2. AND IT IS HANDED BACK, badged, with a Continue button.
47// 3. AND THE BADGE SAYS THE ROAD WENT, not that the browser closed. Four sentences exist for
48// exactly this distinction and picking the wrong one is a small lie.
49// 4. AND THE TURN IS STILL IN THE WRITE-AHEAD LOG. This is the one the user cannot see and
50// the one that cost them the turn: the terminal branch calls `J.clearTurn`.
51// 5. AND A PAGE THAT WAS BACKGROUNDED AND CAME BACK STILL SAYS SO.
52// 6. AND THE SCREEN WAS KEPT AWAKE WHILE THE TURN RAN, AND ONLY WHILE IT RAN.
53//
54// PROVED AGAINST BROKEN CODE FIRST:
55//
56// node dev/verify_predrop.mjs --break wording # the shipped regex: 2-4 fail
57// node dev/verify_predrop.mjs --break bfcache # `_unloading` never cleared: 5 fails
58// node dev/verify_predrop.mjs --break nolock # the wake lock never taken: 6 fails
59// node dev/verify_predrop.mjs # and then, clean
60//
61// `--break wording` restores `isUnreachable` exactly as it stood on 2026-08-27, and it is the
62// one break whose result CHANGES with the engine. Measured in world 17 on 2026-08-28:
63//
64// old JS classifier, old engine 6 of 15 fail: an error line, nothing badged, no
65// Continue, and `0 turn(s) open` -- the write-ahead
66// entry deleted by `clearTurn`
67// old JS classifier, new engine all 15 pass
68//
69// Nothing about the JavaScript differs between those two runs. What differs is that
70// `TransportErr::crossed` puts `could not reach the provider` in front of the browser's
71// sentence, and the 2026-08-27 regex already quoted that phrase -- it had simply never
72// arrived. That is the argument for having done the Rust half rather than adding a string,
73// and running this file both ways across the rebuild is how it was made.
74//
75// eval "$(bash dev/world.sh 17 --up)"
76// node dev/verify_predrop.mjs
77//
78// Needs dev/serve.mjs and the mock. No gateway. The engine matters -- see above.
79import fs from 'node:fs';
80import path from 'node:path';
81import { fileURLToPath } from 'node:url';
82import { open, newChat, scratch, shot, signInAs } from './harness.mjs';
83
84const HERE = path.dirname(fileURLToPath(import.meta.url));
85const WWW = path.join(HERE, '..', 'www');
86
87const BREAK = (() => {
88 const i = process.argv.indexOf('--break');
89 return i > 0 ? String(process.argv[i + 1] || '') : '';
90})();
91
92// The three lines that decide it, each quoted whole so a move breaks this file loudly
93// rather than letting it patch nothing and report a pass.
94const ANCHORS = {
95 // The classifier, as it reads now.
96 wording: [
97 '\t\treturn CLIENT_ROAD.test(s) || BROWSER_ROAD.test(s);\n',
98 // Exactly the two patterns of 2026-08-27, before `Load failed` was in either of
99 // them and before the client\'s own reason could reach here at all.
100 '\t\treturn /Failed to fetch|network\\s*error|ERR_CONNECTION|ENOTFOUND|ECONNREFUSED|refused|dns/i.test(s)\n'
101 + '\t\t\t|| /could not reach|the stream broke|read stream chunk failed/i.test(s);\n',
102 ],
103 // The bfcache restore.
104 bfcache: [
105 "\twindow.addEventListener('pageshow', function () { _unloading = false; });\n",
106 "\twindow.addEventListener('pageshow', function () { /* broken: nothing clears it */ });\n",
107 ],
108 // The lock itself.
109 nolock: [
110 '\t\t\thold: function () { held++; ask(); },\n',
111 '\t\t\thold: function () { /* broken: never asked for */ },\n',
112 ],
113};
114if (BREAK && !ANCHORS[BREAK]) {
115 console.error(`unknown break '${BREAK}'; one of: ${Object.keys(ANCHORS).join(', ')}`);
116 process.exit(2);
117}
118
119let bad = 0;
120const check = (pass, name, detail) => {
121 if (!pass) bad++;
122 console.log((pass ? ' ok ' : ' FAIL ') + name + (detail ? ' — ' + detail : ''));
123};
124
125const SRC = fs.readFileSync(path.join(WWW, 'js/daimond.js'), 'utf8');
126for (const [k, [from]] of Object.entries(ANCHORS)) {
127 if (SRC.split(from).length !== 2) {
128 console.error(`the line the '${k}' break patches is not in js/daimond.js exactly once; `
129 + 'the anchor has moved and the break would patch nothing');
130 process.exit(2);
131 }
132}
133
134// WebKit's own sentence for a fetch that never got a response, verbatim.
135// https://trackjs.com/javascript-errors/load-failed/
136const WEBKIT_WORDING = 'Load failed';
137
138const s = await open({
139 name: 'predrop',
140 profile: scratch('pw', 'predrop' + (BREAK ? '-' + BREAK : '')),
141 route: async (page) => {
142 if (BREAK) {
143 const [from, to] = ANCHORS[BREAK];
144 const body = SRC.replace(from, to);
145 await page.route('**/js/daimond.js', (r) => r.fulfill({
146 status: 200, contentType: 'application/javascript', body,
147 }));
148 }
149 // The Safari failure, armed from the test rather than from the mock: what is being
150 // simulated is the BROWSER's behaviour, not the provider's, so it belongs in the
151 // browser. Installed before any script runs, and inert until `__loadFail` is set.
152 await page.addInitScript((wording) => {
153 const real = window.fetch.bind(window);
154 window.__loadFailCount = 0;
155 window.fetch = function (input, init) {
156 const url = typeof input === 'string' ? input
157 : (input && input.url) || String(input || '');
158 if (window.__loadFail && /\/v1\/chat\/completions/.test(url)) {
159 window.__loadFailCount++;
160 // A TypeError with no response and no status, which is the whole of
161 // what a page gets when a fetch dies before its headers.
162 return Promise.reject(new TypeError(wording));
163 }
164 return real(input, init);
165 };
166 }, WEBKIT_WORDING);
167 },
168});
169const { page: p } = s;
170if (BREAK) console.log(`\n*** RUNNING UNDER --break ${BREAK}: failures below are the point ***\n`);
171
172/// What the thread holds, and what is offered about the last answer in it.
173const thread = () => p.evaluate(() => {
174 const inter = [...document.querySelectorAll('#chat-output .chat-msg.interrupted')].pop() || null;
175 return {
176 interrupted: !!inter,
177 // By role rather than by its words, so a translation does not decide this file.
178 continues: !!(inter && inter.querySelector('.turn-interrupted button')),
179 badge: inter ? ((inter.querySelector('.ti-label') || {}).textContent || '') : '',
180 errors: [...document.querySelectorAll('#chat-output .chat-msg-error, #chat-output .error-log')].length,
181 users: [...document.querySelectorAll('#chat-output .chat-msg-user .chat-msg-content')]
182 .map(e => e.textContent),
183 tries: window.__loadFailCount || 0,
184 };
185});
186
187/// Wait for the composer to offer Send again — the app's own "the turn is over".
188///
189/// The bound is above `RetryPolicy::default()`'s whole budget (eight attempts,
190/// `max_total_wait_ms` 120 s), because a fetch that never gets a response is
191/// retryable and the ladder runs to the end of it before the turn dies.
192const settle = async (label, timeout = 200000) => {
193 const t0 = Date.now();
194 while (Date.now() - t0 < timeout) {
195 const busy = await p.evaluate(() => {
196 const b = document.getElementById('chat-send');
197 return !!b && (b.classList.contains('stop') || b.disabled);
198 });
199 if (!busy) return { ended: true, ms: Date.now() - t0 };
200 await p.waitForTimeout(250);
201 }
202 console.log(` note the ${label} turn was still running after ${timeout} ms`);
203 return { ended: false, ms: Date.now() - t0 };
204};
205
206/// Turn ids still open in the write-ahead log.
207const journalled = () => p.evaluate(async () => {
208 if (!window.DaimondJournal) return null;
209 const rec = await DaimondJournal.recover();
210 return (rec.turns || []).map(t => t.turnId);
211});
212
213try {
214 await newChat(s);
215
216 // ── 1-4 and 6. The road never opens ──────────────────────────
217 //
218 // Armed AFTER the model is connected, so only the turn's own request dies.
219 await p.evaluate(() => { window.__loadFail = true; });
220 await p.fill('#chat-input', '@text hello');
221
222 // The lock is taken by the turn and given back by it, so both halves are read around
223 // the send rather than after it.
224 const lockBefore = await p.evaluate(() => window.DaimondWake && DaimondWake.state());
225 await p.click('#chat-send');
226 // Sampled while the retry ladder is still climbing, which is the only window there is.
227 await p.waitForTimeout(1500);
228 const lockDuring = await p.evaluate(() => window.DaimondWake && DaimondWake.state());
229
230 const end = await settle('pre-token');
231 check(end.ended, 'THE TURN THAT NEVER REACHED THE PROVIDER ENDS rather than hanging',
232 `${end.ms} ms`);
233
234 const t1 = await thread();
235 await shot(s, 'predrop-interrupted');
236 check(t1.tries > 0,
237 'the instrument fired — the engine really did meet a rejected fetch',
238 `${t1.tries} attempt(s) refused with ${JSON.stringify(WEBKIT_WORDING)}`);
239 check(t1.interrupted,
240 "A TURN THAT DIED BEFORE THE FIRST TOKEN IS HANDED BACK, not written off",
241 t1.interrupted ? '' : `${t1.errors} error line(s) and nothing marked interrupted`);
242 check(t1.continues,
243 'and the Continue button is on it',
244 t1.continues ? '' : (t1.interrupted ? 'no button in .turn-interrupted'
245 : 'nothing was marked interrupted'));
246 // The badge that says the ROAD went. `turn.offline_early` in www/i18n/en.js; matched on a
247 // distinctive clause rather than the whole sentence so a rewording does not fail this.
248 check(/connection dropped/i.test(t1.badge),
249 'AND THE BADGE SAYS THE CONNECTION DROPPED, not that the browser closed',
250 JSON.stringify(t1.badge.slice(0, 90)));
251 check(t1.users.length === 1,
252 'and the prompt is still in the thread, once',
253 JSON.stringify(t1.users.join('|').slice(0, 80)));
254
255 check(t1.errors === 0,
256 'AND IT IS SAID ONCE — no red line telling the user to check their base URL',
257 `${t1.errors} error line(s) beside the badge`);
258
259 // THE PROPERTY THE USER ACTUALLY HAS, and the one the deleted journal entry cost them:
260 // close the app, come back, and the turn is still there to be continued. Everything above
261 // is true of a sitting; this is true of a device.
262 await p.reload({ waitUntil: 'domcontentloaded' });
263 await p.waitForTimeout(1200);
264 await signInAs(s, 'predrop');
265 await p.waitForTimeout(1200);
266 await p.locator('#session-list .chat-box').first().click({ force: true });
267 await p.waitForSelector('#chat-input', { state: 'visible', timeout: 10000 });
268 await p.waitForTimeout(600);
269 const t1r = await thread();
270 await shot(s, 'predrop-after-reload');
271 check(t1r.interrupted && t1r.continues,
272 'AND IT IS STILL THERE AFTER A RELOAD, badge and button',
273 t1r.interrupted ? '' : `${t1r.errors} error line(s), nothing interrupted`);
274 check(t1r.users.length === 1,
275 'with the prompt kept, once',
276 JSON.stringify(t1r.users.join('|').slice(0, 80)));
277
278 // 6. The screen was kept awake for the turn, and given back at the end of it.
279 const lockAfter = await p.evaluate(() => window.DaimondWake && DaimondWake.state());
280 check(!!lockBefore && lockBefore.held === 0 && !!lockDuring && lockDuring.held === 1,
281 'THE SCREEN WAKE LOCK IS HELD FOR THE DURATION OF A TURN',
282 `held ${lockBefore && lockBefore.held} before, ${lockDuring && lockDuring.held} during`);
283 check(!!lockAfter && lockAfter.held === 0 && !lockAfter.locked,
284 'and released when the turn ends, by whatever door it left',
285 `held ${lockAfter && lockAfter.held}, lock ${lockAfter && lockAfter.locked}`
286 + (lockAfter && !lockAfter.supported ? ' (API absent here: the count is the assertion)' : ''));
287
288 // ── 5. A page that was backgrounded and came back ────────────
289 //
290 // `pagehide` then `pageshow` is what iOS does to a home-screen PWA on an app switch.
291 // After it, a dropped turn must still say so.
292 await newChat(s);
293 // Re-armed: the reload above ran the init script again, which leaves the stub in place
294 // and disarmed.
295 await p.evaluate(() => {
296 window.__loadFail = true;
297 window.dispatchEvent(new PageTransitionEvent('pagehide', { persisted: true }));
298 window.dispatchEvent(new PageTransitionEvent('pageshow', { persisted: true }));
299 });
300 await p.fill('#chat-input', '@text after the switch');
301 await p.click('#chat-send');
302 const end2 = await settle('after-bfcache');
303 const t2 = await thread();
304 await shot(s, 'predrop-after-bfcache');
305 // UNDER `--break bfcache` THIS FAILS WITH NOTHING ON SCREEN AT ALL, which is the shape
306 // worth recognising: no badge and no error line, because `_unloading` sent the catch down
307 // its first branch. Under `--break wording` it fails with an error line instead. The count
308 // is reported so the two are told apart from the output.
309 check(end2.ended && t2.interrupted && t2.continues,
310 'A TAB THAT WENT INTO THE BACKGROUND AND CAME BACK STILL HANDS THE TURN BACK',
311 t2.interrupted ? '' : `${t2.errors} error line(s), nothing interrupted`);
312
313 // ── The control. A provider refusal is still terminal ────────
314 //
315 // THE CHECK THAT MAKES THE OTHERS MEAN SOMETHING. Without it, "everything is
316 // recoverable" would pass every check above and classify nothing.
317 await p.evaluate(() => { window.__loadFail = false; });
318 await newChat(s);
319 // What is open BEFORE the refusal, so the check below is about this turn and not about
320 // the interrupted ones above it -- which are open, correctly, and stay open.
321 const openBefore = await journalled();
322 await p.fill('#chat-input', '@err 400');
323 await p.click('#chat-send');
324 const end3 = await settle('400');
325 const t3 = await thread();
326 check(end3.ended, 'a provider refusal ends too', `${end3.ms} ms`);
327 check(!t3.interrupted && t3.errors > 0,
328 'BUT A PROVIDER REFUSAL IS STILL TERMINAL — a 400 is not offered back',
329 t3.interrupted ? 'a 400 was badged interrupted and offers a Continue that cannot work'
330 : `${t3.errors} error line(s)`);
331 const openAfter = await journalled();
332 check(Array.isArray(openAfter) && Array.isArray(openBefore)
333 && openAfter.length === openBefore.length,
334 'AND ITS JOURNAL ENTRY IS PRUNED — a terminal turn is not left to be recovered',
335 openBefore === null ? 'no journal'
336 : `${openBefore.length} turn(s) open before it, ${openAfter && openAfter.length} after`);
337} finally {
338 await s.close();
339}
340
341console.log(bad ? `\n${bad} check(s) FAILED` : '\nall checks passed');
342process.exit(bad ? 1 : 0);