oxedyne/daimond — www/assets/typst/VENDORED.md

oxedyne/daimond/www/assets/typst/VENDORED.md

5.0 KiB, 1 run

created by r2519314175:1057, which is this file's identity for as long as the history lasts, whatever it is later renamed to

download · who wrote it · its history

1	# Typst packages, carried in the bundle
2
3	`#import "@preview/cetz:0.3.4"` names a package in Typst Universe. The command-line
4	compiler downloads it and caches it under `~/.cache/typst/packages`; the compiler in
5	this page has no network at all, so until now every document with a live cetz diagram
6	was refused outright — including the author's own 281-page book, whose template opens
7	with that exact line.
8
9	These packs are how that is answered. Five packages ship inside the app, as sealed
10	files, and `src/wasm/typst.rs` hands their sources to the compiler as ordinary files.
11
12	\| pack \| files \| bytes \| why it is here \|
13	\|---\|---\|---\|---\|
14	\| `preview/cetz/0.3.4.pack` \| 38 \| 289,319 \| the book imports it \|
15	\| `preview/oxifmt/0.2.1.pack` \| 3 \| 21,812 \| cetz 0.3.4 and 0.3.2 both import it \|
16	\| `preview/cetz/0.3.2.pack` \| 38 \| 287,832 \| cetz-plot 0.1.1 imports this version, not 0.3.4 \|
17	\| `preview/cetz-plot/0.1.1.pack` \| 27 \| 184,702 \| charts and plots on top of cetz \|
18	\| `preview/fletcher/0.5.7.pack` \| 13 \| 143,367 \| arrow diagrams; imports cetz 0.3.4 \|
19
20	927 KB in total, fetched only when a document actually imports one. A session that
21	typesets nothing, or typesets a document with no packages in it, pays nothing.
22
23	## Why not `www/vendor/`
24
25	Because `www/vendor/` is gitignored, excluded from the public mirror by
26	`dev/publish.mjs`, and excluded from the bundle fingerprint by `verify/lib.mjs`. A
27	live view was once shipped with no renderer behind it for exactly that reason. These
28	files are under `www/assets/`, which is committed, crosses to the mirror, is cached by
29	the service worker as part of the shell, and **is inside `www/manifest.json` and the
30	transparency chain**. What compiled the book is provably what shipped.
31
32	## Why the whole closure and not the named package
33
34	cetz 0.3.4's `src/deps.typ` is one line: `#import "@preview/oxifmt:0.2.1"`. Ship cetz
35	without oxifmt and the import resolves, then fails — and *a package that resolves while
36	its dependency does not is indistinguishable from one that never resolved*. The same
37	trap caught the first run of `dev/probe_typstpkg.mjs`.
38
39	So the closure is walked, not assumed:
40
41	- `refresh.sh` carries the set a human chose. It is a wish list, not a closure.
42	- `src/wasm/typst.rs` derives the closure at compile time from the package **sources
43	themselves** — it reads each pack's `.typ` files and follows every `@…` import it
44	finds, recursively. A dependency cannot be silently absent, because nothing consults
45	a hand-written list of dependencies.
46	- A dependency that is not carried produces its own refusal, in different words
47	from an unavailable package, saying that the document is not at fault.
48
49	`cetz-plot` is why this matters in practice: its `src/cetz.typ` imports
50	`@preview/cetz:0.3.2`, not the 0.3.4 the book uses. cetz is therefore vendored twice.
51	Nobody would have guessed that from the package's name or its version number.
52
53	## The pack format
54
55	One file per package, so one HTTP request loads one package and the service worker
56	caches it whole.
57
58	```
59	DAIMOND TYPST PACK 1
60	namespace preview
61	name cetz
62	version 0.3.4
63	entrypoint src/lib.typ
64	file 7651 LICENSE
65	file 1953 src/aabb.typ
66	…
67	<one blank line>
68	<the file bodies, concatenated, in the order listed>
69	```
70
71	ASCII header, blank line, raw bytes. Lengths rather than delimiters, so a source
72	containing anything at all cannot end a file early. `packs/INDEX` lists every pack with
73	its file count and its byte length, and `src/wasm/typst.rs` **`include_str!`s that
74	INDEX** — so the wasm knows what this build carries without asking the network, and a
75	pack that does not match its INDEX line is reported as a broken deployment rather than
76	as a missing package.
77
78	## Refreshing
79
80	```bash
81	bash www/assets/typst/refresh.sh # from ~/.cache/typst/packages
82	```
83
84	Nothing is downloaded: the local typst package cache is read, and a package that is not
85	in it stops the script rather than producing a set with a hole in it. The script is
86	served alongside what it produced, so the recipe is inside the same seal as the result.
87
88	`packs/INDEX` is compiled into the wasm. After a refresh the wasm must be rebuilt
89	and the bundle resealed, or the wasm and the served packs describe different sets.
90
91	## What is taken, and what is left
92
93	Sources (`*.typ`), the `typst.toml` that names the entrypoint, and the licence. Not the
94	README, not the galleries, not the built PDFs — none of them is read by a compile and
95	together they are most of what a package directory weighs. The licences ship because
96	this is other people's LGPL and MIT work travelling inside our bundle.
97
98	## The accepted cost
99
100	A document that asks for a version not carried here still fails. It fails with a
101	sentence that says which versions are carried and that the fix is to vendor another
102	one — but it fails. That was the deliberate choice: the alternative is fetching from a
103	registry at compile time, which would make "this compiler runs inside the page with no
104	network at all" false, and would put outside the seal the one thing the whole
105	verification story rests on.