oxedyne/daimond — hand/install/README.md

13.9 KiB, 1 run

created by r2519314175:901, which is this file's identity for as long as the history lasts, whatever it is later renamed to

download · who wrote it · its history

1	# Installing Daimond's machine hand
2
3	Daimond runs in a web page. A web page cannot start a program — there is no
4	flag, no setting, and no future browser API that will change that. So if Daimond
5	is to run a build or a test on your computer, the part that actually starts the
6	program has to live outside the page, and something has to introduce the two.
7
8	That something is your browser. Chrome will start a small program on an
9	extension's behalf and connect the two by a pipe, but only if a file on this
10	computer names both of them.
11
12	Two files, once. One says which program the browser may start. The other
13	says which folder that program may work in. Nothing listens on a port, nothing
14	runs in the background, and there is no password to keep. That is the point of
15	doing it this way: a background service on a port would be reachable by any page
16	you visit, and its only defence would be a secret you had pasted somewhere. Here
17	there is nothing to find and nothing to steal — the browser is the doorman.
18
19	## Before you start: your browser must not be a snap or a flatpak
20
21	A snap or flatpak browser cannot run Machine Operations, and this is worth
22	checking first because everything below will otherwise appear to succeed.
23
24	The confinement extends to the programs the browser starts. The hand is one of
25	them, so it may only see the files in `$HOME` that are not hidden — and its
26	journal is at `~/.local/share/daimond/hand/journal`, behind one that is. The hand
27	exits before it can open the journal it would have used to say so, and the
28	browser reports only "Native host has exited".
29
30	`install.sh` finds these profiles and refuses them by name. The fix is a
31	Chromium-family browser installed from a `.deb`. Moving the journal is not a fix:
32	the browser hands the hand its own environment, so `DAIMOND_HAND_JOURNAL_DIR`
33	never reaches it.
34
35	## What you need
36
37	- The Daimond Hands extension, in this repository at `ext/`.
38	- A folder you are content for Daimond to work in. That folder bounds
39	everything any command can read or write, so not your home directory.
40	- A .deb Chromium-family browser, started at least once — the profile
41	directory appears on first run, not when the package is installed.
42
43	## Do this
44
45	Two commands, from the top of the Daimond repository, then four things in the
46	browser.
47
48	### 1. Build it
49
50	```sh
51	cargo build --release --manifest-path hand/Cargo.toml
52	```
53
54	`--manifest-path`, not `-p`: the hand is its own cargo workspace, so `-p` fails.
55
56	### 2. Grant a folder and register the hand
57
58	```sh
59	hand/install/install.sh --workspace ~/work
60	```
61
62	Replace `~/work` with the folder you chose; it must already exist. That one
63	command creates the journal directory at mode `700`, writes your folder into
64	`root.txt` beside it, and writes the host manifest into every usable browser
65	profile it finds. It builds nothing, downloads nothing, starts nothing, and needs
66	no root.
67
68	Two things worth knowing rather than discovering:
69
70	- The hand never guesses the folder. Without `root.txt` it refuses to serve a
71	page at all, because a guessed folder is a guess about what a command may
72	touch.
73	- `DAIMOND_HAND_ROOT` does the same job and is a trap. It takes precedence,
74	and it will not work for a browser started from a desktop launcher, because
75	the browser hands the hand its own environment. Use the file.
76
77	### 3. Load the extension, and restart the browser
78
79	`install.sh` prints the path. At `chrome://extensions`, turn on Developer mode,
80	click "Load unpacked", and choose `ext/`. Then restart the browser: it reads the
81	registration only at startup.
82
83	### 4. In Daimond, open the folder and allow the first command
84
85	Both are decisions and neither is automated. The first time Daimond wants to run
86	something a window opens and asks you; there is no browser permission for "may
87	run programs on this computer", so that window is the approval, and the
88	extension remembers your answer itself. Until you allow it, nothing runs.
89
90	## When something is wrong, run this
91
92	```sh
93	hand/install/install.sh --check
94	```
95
96	One line per thing that has to be true — browser, registration, binary, journal
97	directory, granted folder, fence, extension — and the fix printed under each that
98	is not. It changes nothing. This is the first thing to run, before reading
99	anything else here.
100
101	Two more, both harmless:
102
103	```sh
104	hand/target/release/daimond-hand --report
105	```
106
107	prints what the fence can enforce on this kernel and, at greater length, what
108	it cannot. Read the second list: on a kernel below Linux 7.1 it includes a way
109	out of the fence entirely.
110
111	```sh
112	hand/target/release/daimond-hand < /dev/null
113	```
114
115	starts the hand the way your browser will. `the page closed the pipe` means it is
116	configured and ready; anything else is a sentence naming the path, the cause and
117	the fix.
118
119	## What a command can actually reach
120
121	Worth knowing before the first thing a daimon runs fails in a way that looks
122	like a broken tool.
123
124	A command runs inside a kernel fence, and the fence is built out of the folder
125	you granted. On top of that the hand adds two things:
126
127	- The system paths a program needs in order to be a program: `/usr`, `/bin`,
128	`/sbin`, `/lib*`, `/etc`, `/opt` and the harmless `/dev` devices, all
129	read-only. Without them nothing can start at all — not even `cat`.
130	- A private temporary directory, writable, with `TMPDIR` pointing at it. It
131	is the one place outside your folder that a command may write, it is removed
132	when the run ends, and no other run can reach it.
133
134	Everything else is refused by the kernel, not by a check somebody wrote. Reading
135	a file one folder outside the grant comes back `Permission denied`, and the
136	daimon is shown that refusal rather than the file.
137
138	The consequence for build tools. A toolchain installed under your home
139	directory — `~/.cargo`, `~/.rustup`, `~/.nvm`, a `node_modules` you keep
140	elsewhere — is not reachable, and today there is no way to grant it: every
141	path in the fence is built by joining a workspace-relative name onto the folder
142	you granted, so nothing outside that folder can be named at all. A daimon can
143	run anything under `/usr/bin` and anything inside your folder. If you want it to
144	run `cargo test`, the toolchain has to be inside the folder you granted.
145
146	## Variants
147
148	### If you keep the binary somewhere else
149
150	```sh
151	hand/install/install.sh /path/to/daimond-hand
152	```
153
154	### If your browser keeps its profile somewhere unusual
155
156	```sh
157	hand/install/install.sh --dir /path/to/profile/NativeMessagingHosts /path/to/daimond-hand
158	```
159
160	A browser started with `--user-data-dir` reads `<that dir>/NativeMessagingHosts`
161	and nothing else, which is what this is for.
162
163	`--dir` writes into a snap or flatpak profile too, with the warning above, on the
164	grounds that naming a directory explicitly means you know better. Expect the hand
165	to disconnect on the first command.
166
167	### To see what it would do, without doing it
168
169	```sh
170	hand/install/install.sh --list
171	hand/install/install.sh --help
172	```
173
174	### To check the script itself
175
176	```sh
177	hand/install/install.sh --selftest
178	```
179
180	Builds throwaway home directories that look like a snap profile, a flatpak
181	profile, a `.deb` profile and nothing at all, runs itself against each, and
182	asserts what it says and its exit status. Nothing of yours is touched.
183
184	## To take it back
185
186	Three different things, and all three are worth knowing about.
187
188	Withdraw the permission. Click the Daimond Hands icon in the toolbar.
189	"Running commands on this computer" is listed there beside the sites you have
190	approved, with a Revoke button. Revoking stops anything that is running at that
191	moment — not at the end of the current build, immediately.
192
193	Take away the folder. Delete `root.txt`. The hand then refuses to serve at
194	all, whatever the browser says, and the refusal is a whole sentence rather than
195	a silent nothing.
196
197	Remove the registration.
198
199	```sh
200	hand/install/uninstall.sh # every browser found
201	hand/install/uninstall.sh --dir DIR # one directory
202	```
203
204	That deletes the file `install.sh` wrote, so the browser can no longer start the
205	hand at all. The binary is left where it is; the script did not put it there.
206
207	## When it does not work
208
209	Run `hand/install/install.sh --check` first. What follows is what each symptom
210	usually turns out to be.
211
212	"Daimond's machine hand is not installed on this computer." The browser found
213	no file naming the host: `install.sh` has not been run, or was run for a
214	different browser, or the browser has not been restarted since.
215
216	"The machine hand disconnected without finishing (Native host has exited.)"
217	on the first command, before anything could have crashed. The hand read its
218	configuration, refused to serve, and exited. Two causes account for nearly all of
219	it: a snap or flatpak browser, which cannot reach the journal at all; or no
220	`root.txt`. `--check` distinguishes them. So does
221	`hand/target/release/daimond-hand < /dev/null`, which prints the refusal as a
222	sentence — the same sentence the browser puts in its own log, since Chrome
223	captures a native host's standard error.
224
225	"…did not say which folder it was granted." The hand answered but named no
226	root: `root.txt` is missing, empty, or names a folder that does not exist.
227
228	"Installed but will not talk to this extension." The file exists but names a
229	different extension. This happens if you loaded the extension without the pinned
230	key in `ext/manifest.json`, which gives it a different id. Find the real id at
231	`chrome://extensions`, then:
232
233	```sh
234	DAIMOND_HAND_EXT_ID=<the id you see> hand/install/install.sh
235	```
236
237	"Disconnected without finishing", mid-command. The hand stopped part-way.
238	Either it crashed, or it produced a single message over 1 MB, which Chrome
239	silently refuses to deliver and answers by cutting the connection. The hand
240	chunks its output well below that, so this should be a crash; its journal will
241	say.
242
243	A command comes back `Permission denied` on a file you expected it to read.
244	That is the fence, working. See "What a command can actually reach".
245
246	## The files here
247
248	\| File \| What it is \|
249	\|---\|---\|
250	\| `install.sh` \| Writes the host manifest into each usable browser's directory, and with `--workspace` the journal directory and `root.txt` too. `--check` diagnoses an install; `--selftest` tests the script. It holds the table of browsers, so it is the one file to edit when a path changes. \|
251	\| `uninstall.sh` \| Removes the manifests again. Reads the directory list from `install.sh --paths` rather than keeping a copy, because a stale copy would silently forget the browsers it had not heard of. \|
252	\| `com.oxedyne.daimond.hand.json` \| The manifest, for reading, and for registering by hand on a platform the script does not cover yet. `install.sh` writes its own copy rather than editing this one, so the path and the extension id are decided in exactly one place. \|
253	\| `mock_host.py` \| A stand-in hand for testing, which speaks the real protocol and runs nothing. See below. \|
254
255	## Other platforms
256
257	Linux is implemented. The others are not, and the paths are written down in
258	`install.sh` rather than left to be rediscovered:
259
260	- macOS uses the same file in a different directory
261	(`~/Library/Application Support/Google/Chrome/NativeMessagingHosts/`), and
262	additionally needs the binary signed and notarised before Gatekeeper will let
263	a browser run it. That is a packaging job, not a path.
264	- Windows has no directory at all: the manifest is found through a registry
265	key under `HKCU\Software\Google\Chrome\NativeMessagingHosts\`, whose default
266	value is the absolute path to the JSON file, which may live anywhere.
267
268	There is no fence on either yet, and the app refuses a command it cannot
269	contain, so a hand on those platforms would install and then refuse everything.
270
271	## Testing without the binary
272
273	`mock_host.py` speaks the real framing — a 4-byte native-endian length prefix
274	and UTF-8 JSON — and the real messages, and runs nothing at all. It exists
275	because the failures worth testing are ones a correct hand never produces: a gap
276	in the output sequence, a message over Chrome's 1 MB limit, and a host that dies
277	mid-command.
278
279	Register it like any other binary:
280
281	```sh
282	hand/install/install.sh hand/install/mock_host.py
283	```
284
285	It reads `mock_cfg.json`, beside it, if there is one — Chrome gives a native
286	messaging host no arguments of its own, so a file is the only way in:
287
288	```json
289	{ "chunks": 3, "gap": false, "huge": false, "crash": false, "delay_ms": 0 }
290	```
291
292	and appends everything it sees and says to `mock_host.log`. Neither file belongs
293	in a commit.
294
295	## The three end-to-end tests
296
297	Each needs nothing running, writes its host manifest into a throwaway browser
298	profile of its own, and never touches yours. Headed, so under a virtual display:
299
300	```sh
301	xvfb-run -a -s "-screen 0 1400x900x24" node dev/verify_hand.mjs
302	xvfb-run -a -s "-screen 0 1400x900x24" node dev/verify_handrun.mjs
303	xvfb-run -a -s "-screen 0 1400x900x24" node dev/verify_handreal.mjs
304	```
305
306	\| \| Browser \| Extension \| Host \| Command \|
307	\|---\|---\|---\|---\|---\|
308	\| `verify_hand` \| real \| real \| mock \| none \|
309	\| `verify_handrun` \| real \| real \| mock \| none \|
310	\| `verify_handreal` \| real \| real \| real \| real \|
311
312	The first two use the mock deliberately: they test the order of the output, a
313	gap in it, the 1 MB disconnect, a crash mid-command, a page that goes away with
314	a command running, the missing-host message, and revoking from the popup — and a
315	correct hand does none of those things, so a correct hand cannot be used to test
316	them. `verify_handrun` goes on to check what the model was shown, which is the
317	only thing that matters about a tool result.
318
319	`verify_handreal` is the one that proves the join. It builds the binary,
320	registers it with `install.sh`, writes a `root.txt`, clicks Allow for real, and
321	then has a daimon run real commands: it asserts that a nonce written to disk a
322	moment earlier comes back through the model, that a real non-zero exit arrives as
323	non-zero, that a file outside the fence is refused by the kernel and its
324	contents never reach the model, that the journal on disk names what was run, and
325	that a real `cargo test` compiles and runs inside the fence. It takes about
326	twelve seconds after the build.