oxedyne/fe2o3 — fe2o3_graphics/tests/mp4

oxedyne/fe2o3/fe2o3_graphics/tests/mp4_frag.rs

31.5 KiB, 38 runs

created by r1870400018:21816, which is this file's identity for as long as the history lasts, whatever it is later renamed to

download · who wrote it · its history

1	//! Does a film written as fragments keep its sound against its picture?
2	//!
3	//! A whole-file MP4 states its timing once, in a sample table the writer builds
4	//! with every sample in hand. A fragmented one states it over and over: the
5	//! `moov` at the front carries no samples at all, and each `moof` after it says
6	//! where its run begins on each track's timeline and how long every sample in it
7	//! lasts. Nothing in the file cross-checks a fragment against the one before it,
8	//! and no decoder ever complains about the arithmetic, so a writer that loses a
9	//! tick a fragment on one track produces a film that opens, reports the right
10	//! shape, decodes every frame -- and walks its sound away from its picture. That
11	//! is the characteristic failure of a bad remux and the reason this test exists.
12	//!
13	//! So the film is written by calling `head` once and `next` several times, the
14	//! parts are concatenated, and a different program is asked five questions
15	//! in order, each a separate named failure:
16	//!
17	//! 1. Shape. Two streams; the picture h264 at the source's size; the sound
18	//! aac; and the packet counts equal to what was handed to the writer.
19	//! 2. Decode. `ffmpeg -v error` says nothing at all. A container written
20	//! wrongly still opens and still counts right, and only a decode settles
21	//! whether the bytes in the `mdat` were the frames the `moof` described.
22	//! 3. When the pictures are shown. The written times are read back and held
23	//! against the source's, allowing exactly ONE constant difference across all
24	//! of them: the whole track may be delayed -- that is how a negative
25	//! composition offset is avoided -- but the intervals may not change.
26	//! 4. When the sounds are heard. The same check on `a:0`, and the one that
27	//! matters most, because sound drifting away from picture is invisible to
28	//! checks 1 and 2 and is exactly what a viewer notices first.
29	//! 5. The two tracks against each other. Both may be delayed; they may not
30	//! be delayed by different amounts, which is a film whose sound is out and
31	//! which every check above passes happily.
32	//!
33	//! # Proving the checks
34	//!
35	//! A check is only proved by a break it catches that everything before it
36	//! misses. A break that dies at the decode has said nothing about the timing
37	//! comparison that would have run after it. So each of these is chosen to
38	//! survive every earlier question and fail exactly one. They are applied by
39	//! `BREAK_FRAG=<name>` to the samples handed to the writer, leaving what the
40	//! test expects untouched, and an unknown name is an error rather than a quiet
41	//! nothing.
42	//!
43	//! - `picture-one-tick` proves check 3. One picture sample's composition
44	//! offset, in the middle of a fragment, is raised by a single tick. The
45	//! sample count does not move, so check 1 passes. A millisecond on a picture
46	//! shown forty milliseconds from its neighbours reorders nothing and leaves
47	//! every offset non-negative, so ffmpeg decodes it without a word and check 2
48	//! passes. The picture times then need two constants where one is allowed,
49	//! and check 3 fires. The sound is untouched and the moved sample is not a
50	//! fragment's first, so checks 4 and 5 would have seen nothing.
51	//! - `sound-one-tick` proves check 4. The same single tick, on one sound
52	//! sample's composition offset in the middle of a fragment. Counts, decode
53	//! and the whole picture track are as they were, and the moved sample is not
54	//! a fragment's first, so check 5 is left with nothing to find either.
55	//! - `sound-one-tick-dur` is the fallback for check 4 where a writer drops
56	//! composition offsets on a sound track -- which is itself a defect worth
57	//! knowing about. It raises one sound sample's duration by a tick, which
58	//! moves every sample after it and therefore disturbs check 5 at the later
59	//! fragments as well. Check 4 still fires first, but this break proves less
60	//! than the one above and is here only so a stuck run has a way forward.
61	//! - `sound-delayed` proves check 5. Every sound sample's composition
62	//! offset is raised by [`DELAY`] milliseconds. The sound track is then
63	//! internally perfect: the same intervals, one constant away from the source,
64	//! which is precisely what check 4 permits. Counts and decode are untouched.
65	//! Only check 5, which holds the two tracks against one another, can see that
66	//! the sound now sits a fifth of a second from where the picture puts it.
67	//!
68	//! One caveat, for whoever changes the writer: these comparisons read the
69	//! packet times ffprobe reports, and those move with an edit list. The writer
70	//! today writes none, so a track's times are its media times and checks 3 to 5
71	//! are exact. If a compensating `elst` is ever added to one track and not the
72	//! other, check 5 will fail on a film that is in fact correct, and it must be
73	//! told about the compensation rather than loosened.
74	//!
75	//! Point `MKV_CORPUS` at a directory of films. Output goes under
76	//! `~/.cache/ochre-remux-probe`, never `/tmp`, which is a tmpfs here and is
77	//! charged to the memory budget of whoever writes to it.
78	//!
79	//! [Written with AI entirely](https://need2know.ai/entirely-ai/code)\
80	//! Anthropic Claude
81
82	use oxedyne_fe2o3_core::prelude::*;
83	use oxedyne_fe2o3_graphics::{
84	matroska::{Clusters, Matroska, TrackKind},
85	mp4::{Codec, Fragments, Media, Sample, Stream, composition_offsets},
86	};
87
88	use std::{
89	env,
90	fs::{self, File},
91	io::Read,
92	path::{Path, PathBuf},
93	process::Command,
94	};
95
96	const HEAD: usize = 1024 * 1024; // read before the clusters, to find the tracks
97	const WINDOW: usize = 256 * 1024; // the window the frame reader is held to
98	const FRAMES: usize = 300; // pictures repackaged, deciding how long the written film is
99
100	// How many fragments the film is cut into: more than a handful, because one
101	// fragment carrying everything would exercise none of the bookkeeping that runs
102	// between them, which is the whole subject.
103	const FRAGS: usize = 6;
104	const MIN_FRAGS: usize = 4; // fewer and fragmentation is not being tested, so the film is skipped
105
106	// The timescale both tracks are written in, in ticks a second. Milliseconds,
107	// matching the unit Matroska stamps its frames in, so no time is rescaled and no
108	// rounding is introduced anywhere between the source and the comparison. A sound
109	// track is more usually written on its sampling rate, and a remuxer in earnest
110	// should be; here the point is to compare times exactly, and a rate of 44100
111	// does not divide a millisecond.
112	const SCALE: u32 = 1000;
113
114	// How far sound-delayed moves the sound, in ticks -- glaring to a viewer and
115	// invisible to every check but the last.
116	const DELAY: i32 = 200;
117
118	#[test]
119	fn a_fragmented_film_carries_both_streams() -> Outcome<()> {
120	let dir = match env::var("MKV_CORPUS") {
121	Ok(d) => PathBuf::from(d),
122	Err(_) => {
123	println!("MKV_CORPUS is not set, so nothing was fragmented and this \
124	test proves nothing.");
125	return Ok(());
126	},
127	};
128	if Command::new("ffprobe").arg("-version").output().is_err() {
129	println!("ffprobe is not installed, so there is no oracle and this test \
130	proves nothing.");
131	return Ok(());
132	}
133	if Command::new("ffmpeg").arg("-version").output().is_err() {
134	println!("ffmpeg is not installed, so nothing can be decoded and this \
135	test proves nothing.");
136	return Ok(());
137	}
138
139	let out = match env::var("HOME") {
140	Ok(h) => PathBuf::from(h).join(".cache/ochre-remux-probe"),
141	Err(_) => return Err(err!("HOME is not set, so there is nowhere to \
142	write the film."; Invalid, Init)),
143	};
144	res!(fs::create_dir_all(&out));
145
146	let mut films = Vec::new();
147	res!(gather(&dir, &mut films));
148	films.sort();
149
150	let want = num_from_env("MKV_FRAMES", FRAMES);
151	let frags = num_from_env("MKV_FRAGS", FRAGS).max(MIN_FRAGS);
152	let mut done = 0usize;
153	let mut skipped = 0usize;
154	// What was actually held against something, so that a run which compared
155	// nothing cannot read as a run that passed.
156	let mut compared = 0usize;
157	let mut joins = 0usize;
158
159	for film in films.iter() {
160	if done >= num_from_env("MKV_FILES", 2) {
161	break;
162	}
163	let made = match repackage(film, want, frags) {
164	Ok(Some(v)) => v,
165	// A film without both an h264 picture and an aac sound is not what
166	// this test is about, and is skipped by name rather than failed.
167	Ok(None) => { skipped += 1; continue },
168	Err(e) => {
169	println!("{}: could not be fragmented: {}", film.display(), e);
170	skipped += 1;
171	continue;
172	},
173	};
174	let path = out.join(format!("frag-{}.mp4", done));
175	res!(fs::write(&path, &made.bytes));
176
177	// 1. Does another program agree it is a film of the right shape?
178	let streams = res!(stream_count(&path));
179	if streams != 2 {
180	return Err(err!(
181	"{}: two streams were written and ffprobe finds {} in the film.",
182	film.display(), streams; Invalid, Mismatch));
183	}
184	let pic = res!(probe_fields(&path, "v:0",
185	"stream=codec_name,width,height,nb_read_packets"));
186	let name = res!(pick(&pic, "codec_name"));
187	if name != "h264" {
188	return Err(err!(
189	"{}: the written film's picture says it holds {} rather than \
190	h264.", film.display(), name; Invalid, Mismatch));
191	}
192	let w = res!(number(&pic, "width"));
193	let h = res!(number(&pic, "height"));
194	if w != made.w as i64 \|\| h != made.h as i64 {
195	return Err(err!(
196	"{}: the written film is {} by {} and the source is {} by {}.",
197	film.display(), w, h, made.w, made.h; Invalid, Mismatch));
198	}
199	let n = res!(number(&pic, "nb_read_packets"));
200	if n != made.vid.len() as i64 {
201	return Err(err!(
202	"{}: {} pictures were written over {} fragments and ffprobe \
203	counts {}.",
204	film.display(), made.vid.len(), made.starts.len(), n;
205	Invalid, Mismatch));
206	}
207	let snd = res!(probe_fields(&path, "a:0",
208	"stream=codec_name,channels,nb_read_packets"));
209	let name = res!(pick(&snd, "codec_name"));
210	if name != "aac" {
211	return Err(err!(
212	"{}: the written film's sound says it holds {} rather than aac.",
213	film.display(), name; Invalid, Mismatch));
214	}
215	let chans = res!(number(&snd, "channels"));
216	if chans != made.chans as i64 {
217	return Err(err!(
218	"{}: {} channels of sound were written and ffprobe finds {}.",
219	film.display(), made.chans, chans; Invalid, Mismatch));
220	}
221	let n = res!(number(&snd, "nb_read_packets"));
222	if n != made.aud.len() as i64 {
223	return Err(err!(
224	"{}: {} sounds were written over {} fragments and ffprobe counts \
225	{}.", film.display(), made.aud.len(), made.starts.len(), n;
226	Invalid, Mismatch));
227	}
228
229	// 2. Does it actually decode, both streams of it, without a murmur?
230	res!(decodes_silently(&path, film));
231
232	// 3. Are the pictures shown when the source said they were?
233	let back = res!(times_of(&path, "v:0", film));
234	if back.len() != made.vid.len() {
235	return Err(err!(
236	"{}: {} picture times were written and {} read back.",
237	film.display(), made.vid.len(), back.len(); Invalid, Mismatch));
238	}
239	let vshift = back[0] - made.vid[0];
240	for i in 0..back.len() {
241	if back[i] - made.vid[i] != vshift {
242	return Err(err!(
243	"{}: picture {} is shown at {} and was written at {}, a \
244	difference of {} against the film's {}. The pictures are not \
245	where they were put.",
246	film.display(), i, back[i], made.vid[i],
247	back[i] - made.vid[i], vshift; Invalid, Mismatch));
248	}
249	compared += 1;
250	}
251
252	// 4. Is the sound heard when the source said it was?
253	//
254	// The reason the test exists. A sound track's timing is bookkeeping and
255	// nothing else -- there is no reordering in it and no offset any decoder
256	// would object to -- so an error here passes every question above and
257	// arrives as a film whose voices do not match its mouths.
258	let backa = res!(times_of(&path, "a:0", film));
259	if backa.len() != made.aud.len() {
260	return Err(err!(
261	"{}: {} sound times were written and {} read back.",
262	film.display(), made.aud.len(), backa.len(); Invalid, Mismatch));
263	}
264	let ashift = backa[0] - made.aud[0];
265	for j in 0..backa.len() {
266	if backa[j] - made.aud[j] != ashift {
267	return Err(err!(
268	"{}: sound {} is heard at {} and was written at {}, a \
269	difference of {} against the track's {}. The sound has \
270	drifted from where it was put, by {} ticks so far.",
271	film.display(), j, backa[j], made.aud[j],
272	backa[j] - made.aud[j], ashift,
273	(backa[j] - made.aud[j]) - ashift; Invalid, Mismatch));
274	}
275	compared += 1;
276	}
277
278	// 5. Do the two tracks stay in step with each other?
279	//
280	// Checks 3 and 4 each allow the track they look at to be delayed as a
281	// whole, and separately they are right to: a delay is how a negative
282	// offset is avoided. Together they let both tracks be delayed by
283	// DIFFERENT amounts, which is a film whose sound is out and which
284	// nothing above can see. So the two delays are held against each other,
285	// at every fragment, so that a failure names where the tracks parted.
286	for (k, (vi, ai)) in made.starts.iter().enumerate() {
287	let gap = (back[vi] - made.vid[vi]) - (backa[ai] - made.aud[ai]);
288	if gap != 0 {
289	return Err(err!(
290	"{}: at fragment {} the sound sits {} ticks from where the \
291	picture puts it. The picture is delayed {} ticks and the \
292	sound {}, and a film's two tracks must be delayed by the \
293	same amount or not at all.",
294	film.display(), k, gap, back[vi] - made.vid[vi],
295	backa[ai] - made.aud[ai]; Invalid, Mismatch));
296	}
297	joins += 1;
298	}
299
300	println!("{}: {} pictures and {} sounds over {} fragments, all decode, \
301	both tracks where they were put (picture delayed {} ticks, sound \
302	starting {} ticks after it)",
303	film.display(), made.vid.len(), made.aud.len(), made.starts.len(),
304	made.shift, made.lead);
305	done += 1;
306	}
307
308	println!("{} films found; {} fragmented, {} skipped; {} times compared and \
309	{} fragment joins checked",
310	films.len(), done, skipped, compared, joins);
311	if done == 0 {
312	return Err(err!(
313	"{} films were found and not one held both an h264 picture and an \
314	aac sound that could be fragmented, so this run proves nothing.",
315	films.len(); Invalid, Mismatch));
316	}
317	if compared == 0 \|\| joins == 0 {
318	return Err(err!(
319	"{} films were fragmented and no time was compared, so this run \
320	proves nothing.", done; Invalid, Mismatch));
321	}
322	Ok(())
323	}
324
325	// --------------------------------------------------------------- the writing
326
327	/// A fragmented film written out, and everything the checks hold it against.
328	struct Made {
329	bytes: Vec<u8>,
330	w: u16,
331	h: u16,
332	chans: u16, // channels of sound, as the source states them
333	vid: Vec<i64>, // picture times written, in decode order, on its own timeline
334	aud: Vec<i64>, // the same for the sound, on the sound track's own timeline
335	starts: Vec<(usize, usize)>, // each fragment's first picture and first sound sample
336	shift: i64, // how far the picture track is delayed by its offsets
337	// How far the first sound sample sits after the first picture sample on the
338	// source's clock. The one misalignment this test introduces itself, and it is
339	// under a frame of sound: each track begins at its own decode time nought, and
340	// the sound frame nearest the first picture is rarely on the same instant. It
341	// is carried here because the expectations are built with it and check 5
342	// therefore holds regardless of it.
343	lead: i64,
344	}
345
346	/// Reads a film's picture and sound frames and writes them as a fragmented MP4,
347	/// decoding nothing.
348	///
349	/// `None` where the film does not hold both an h264 picture and an aac sound
350	/// with the configuration records a repackaging needs, which is not a defect in
351	/// the writer and is skipped rather than failed.
352	fn repackage(path: &Path, want: usize, frags: usize) -> Outcome<Option<Made>> {
353	let mut head = vec![0u8; HEAD];
354	let mut file = res!(File::open(path));
355	let n = res!(file.read(&mut head));
356	head.truncate(n);
357	let mkv = res!(Matroska::read(&head));
358
359	// The picture. Its configuration record moves across verbatim -- this is the
360	// whole trick -- because the `avcC` an MP4 sample entry wants and the
361	// `CodecPrivate` a Matroska track entry carries are the same bytes.
362	let vt = match mkv.tracks().iter().find(\|t\| t.kind() == Some(TrackKind::Video)) {
363	Some(t) => t,
364	None => return Ok(None),
365	};
366	if vt.codec() != "V_MPEG4/ISO/AVC" \|\| vt.private().is_empty() {
367	return Ok(None);
368	}
369	let (w, h) = vt.size();
370	if w == 0 \|\| h == 0 \|\| w > u16::MAX as u32 \|\| h > u16::MAX as u32 {
371	return Ok(None);
372	}
373
374	// The sound. The same move: `CodecPrivate` for `A_AAC` is the
375	// `AudioSpecificConfig` an `esds` states, and a film that carries none of it
376	// would need the configuration derived, which is not this test's subject.
377	let at = match mkv.tracks().iter().find(\|t\| {
378	t.kind() == Some(TrackKind::Audio) && t.codec().starts_with("A_AAC")
379	}) {
380	Some(t) => t,
381	None => return Ok(None),
382	};
383	if at.private().is_empty() {
384	return Ok(None);
385	}
386	let chans = at.channels();
387	let rate = at.rate();
388	if chans == 0 \|\| chans > 8 \|\| !rate.is_finite() \|\| rate < 8000.0 \|\| rate > 192_000.0 {
389	return Ok(None);
390	}
391	let chans = chans as u16;
392	let rate = rate.round() as u32;
393
394	// One pass over the clusters, taking both streams as they interleave. The
395	// sound is taken only while the picture is still wanted, so the two spans
396	// end together rather than the sound running on past the last picture.
397	let vnum = vt.number();
398	let anum = at.number();
399	let step = if vt.frame_nanos() > 0 {
400	(vt.frame_nanos() / 1_000_000).max(1) as u32
401	} else {
402	40
403	};
404	let mut file = res!(File::open(path));
405	let mut cl = Clusters::new(&mkv);
406	let mut buf: Vec<u8> = Vec::new();
407	let mut vraw: Vec<(Vec<u8>, bool, i64)> = Vec::new();
408	let mut araw: Vec<(Vec<u8>, i64)> = Vec::new();
409	let mut eof = false;
410
411	while vraw.len() < want {
412	while buf.len() < WINDOW && !eof {
413	let mut chunk = vec![0u8; WINDOW];
414	let got = res!(file.read(&mut chunk));
415	if got == 0 {
416	eof = true;
417	break;
418	}
419	chunk.truncate(got);
420	buf.extend_from_slice(&chunk);
421	}
422	if buf.is_empty() {
423	break;
424	}
425	let fed = res!(cl.feed(&buf, &mut \|frame\| {
426	if vraw.len() < want {
427	if frame.track == vnum {
428	vraw.push((frame.data.to_vec(), frame.key, frame.time));
429	} else if frame.track == anum {
430	araw.push((frame.data.to_vec(), frame.time));
431	}
432	}
433	Ok(())
434	}));
435	buf.drain(..fed.used);
436	if fed.used == 0 {
437	if eof {
438	break;
439	}
440	if fed.want > buf.len() {
441	let mut chunk = vec![0u8; fed.want - buf.len()];
442	let got = res!(file.read(&mut chunk));
443	if got == 0 {
444	break;
445	}
446	chunk.truncate(got);
447	buf.extend_from_slice(&chunk);
448	}
449	}
450	}
451
452	// A film must begin at a sync sample; anything before the first one cannot
453	// be decoded by a reader starting here and is dropped rather than written.
454	let first = match vraw.iter().position(\|(_, k, _)\| *k) {
455	Some(i) => i,
456	None => return Ok(None),
457	};
458	let vkept: Vec<(Vec<u8>, bool, i64)> = vraw.drain(..).skip(first).collect();
459	if vkept.len() < frags * 2 {
460	return Ok(None);
461	}
462	// The instant the film starts, which both tracks are measured from.
463	let t0 = vkept[0].2;
464	let last = match vkept.last() {
465	Some((_, _, t)) => *t,
466	None => return Ok(None),
467	};
468	let akept: Vec<(Vec<u8>, i64)> = araw.drain(..)
469	.filter(\|(_, t)\| t >= t0 && t <= last)
470	.collect();
471	if akept.len() < frags * 2 {
472	return Ok(None);
473	}
474
475	// The picture arrives in decode order carrying the times it is SHOWN, and
476	// MP4 wants the times it is decoded plus the difference. Every duration here
477	// is the same, so the decoding times are `i * step` and the offsets are what
478	// is left -- which for a film with B-pictures is not nought, and which
479	// delays the whole track by one constant to keep them all non-negative.
480	let vrel: Vec<i64> = vkept.iter().map(\|(_, _, t)\| *t - t0).collect();
481	let vdurs: Vec<u32> = vec![step; vkept.len()];
482	let voffs = res!(composition_offsets(&vrel, &vdurs));
483	let mut vid = Vec::with_capacity(vkept.len());
484	for i in 0..vkept.len() {
485	vid.push((i as i64) * step as i64 + voffs[i] as i64);
486	}
487	let shift = vid[0] - vrel[0];
488
489	// The sound is not reordered, so its offsets are nought and its durations
490	// are simply the gaps between the times the source states. Taking them from
491	// the source rather than from the sampling rate is deliberate: 1024 samples
492	// at 44100 is not a whole number of milliseconds, and a duration rounded
493	// once a frame is exactly the slow drift this test is looking for.
494	let arel: Vec<i64> = akept.iter().map(\|(_, t)\| *t - t0).collect();
495	let lead = arel[0];
496	let mut adurs: Vec<u32> = Vec::with_capacity(arel.len());
497	for j in 0..arel.len() {
498	let d = if j + 1 < arel.len() {
499	arel[j + 1] - arel[j]
500	} else {
501	match adurs.last() {
502	Some(d) => *d as i64,
503	None => 0,
504	}
505	};
506	// A stream whose stamps do not advance is one this test cannot reason
507	// about, and guessing at it would prove nothing.
508	if d <= 0 \|\| d > u32::MAX as i64 {
509	return Ok(None);
510	}
511	adurs.push(d as u32);
512	}
513	let aud: Vec<i64> = arel.iter().map(\|t\| *t - lead).collect();
514
515	// Where the fragments are cut. A fragment beginning at a sync sample is what
516	// makes one seekable, so the cuts are put on sync samples where the film has
517	// enough of them and spread evenly where it does not.
518	let starts = res!(cuts(&vkept, &arel, step, frags));
519	if starts.len() < MIN_FRAGS {
520	return Ok(None);
521	}
522
523	// The samples, built whole and then broken up, so that a break is applied to
524	// one film rather than to one fragment.
525	let mut pic: Vec<Sample> = Vec::with_capacity(vkept.len());
526	for (i, (data, key, _)) in vkept.into_iter().enumerate() {
527	let s = if key { Sample::key(data, step) } else { Sample::delta(data, step) };
528	pic.push(s.shown_after(voffs[i]));
529	}
530	let mut snd: Vec<Sample> = Vec::with_capacity(akept.len());
531	for (j, (data, _)) in akept.into_iter().enumerate() {
532	snd.push(Sample::key(data, adurs[j]));
533	}
534	res!(apply_break(&mut pic, &mut snd, &starts));
535
536	let streams = vec![
537	Stream {
538	media: Media::Picture { w: w as u16, h: h as u16 },
539	timescale: SCALE,
540	codec: Codec::Avc(vt.private().to_vec()),
541	// Both tracks begin at nought here: the times handed to the writer
542	// are already rebased on the first kept frame of each.
543	start: 0,
544	},
545	Stream {
546	media: Media::Sound { channels: chans, rate },
547	timescale: SCALE,
548	codec: Codec::Aac(at.private().to_vec()),
549	start: 0,
550	},
551	];
552	let mut film = res!(Fragments::new(streams));
553	let mut bytes = res!(film.head());
554	for k in 0..starts.len() {
555	let (vi, ai) = starts[k];
556	let (vj, aj) = match starts.get(k + 1) {
557	Some((v, a)) => (v, a),
558	None => (pic.len() + vi, snd.len() + ai),
559	};
560	// The samples are drained from the front, so what is left always begins
561	// at this fragment and the counts are differences of the boundaries.
562	let vrun: Vec<Sample> = pic.drain(..vj - vi).collect();
563	let arun: Vec<Sample> = snd.drain(..aj - ai).collect();
564	req!(vrun.is_empty(), false, "Fragment {} carries no picture.", k);
565	req!(arun.is_empty(), false, "Fragment {} carries no sound.", k);
566	let part = res!(film.next(vec![(0, vrun), (1, arun)]));
567	bytes.extend_from_slice(&part);
568	}
569	req!(pic.len(), 0, "Pictures were left over after the last fragment.");
570	req!(snd.len(), 0, "Sounds were left over after the last fragment.");
571
572	Ok(Some(Made {
573	bytes,
574	w: w as u16,
575	h: h as u16,
576	chans,
577	vid,
578	aud,
579	starts,
580	shift,
581	lead,
582	}))
583	}
584
585	/// Where the fragments begin: for each, the index of its first picture sample
586	/// and of its first sound sample.
587	///
588	/// The sound follows the picture rather than being cut on its own count, so that
589	/// a fragment holds the sound of the pictures in it -- which is what a reader
590	/// playing one fragment at a time needs, and what makes the two runs in a `moof`
591	/// describe the same stretch of film.
592	fn cuts(
593	pics: &[(Vec<u8>, bool, i64)],
594	arel: &[i64],
595	step: u32,
596	frags: usize,
597	)
598	-> Outcome<Vec<(usize, usize)>>
599	{
600	let n = pics.len();
601	let target = (n + frags - 1) / frags;
602	if target == 0 {
603	return Err(err!("A film of {} pictures cannot be cut into {} fragments.",
604	n, frags; Invalid, Input, Range));
605	}
606	let mut vstarts = vec![0usize];
607	for (i, (_, key, _)) in pics.iter().enumerate() {
608	if !*key {
609	continue;
610	}
611	let prev = match vstarts.last() {
612	Some(v) => *v,
613	None => 0,
614	};
615	// The last fragment is left something to carry.
616	if i >= prev + target && n - i >= 2 {
617	vstarts.push(i);
618	}
619	}
620	// A film whose sync samples are too far apart is still worth fragmenting; a
621	// fragment that does not begin at one is legal and merely not seekable.
622	if vstarts.len() < MIN_FRAGS {
623	vstarts = (0..frags).map(\|k\| k * n / frags).collect();
624	vstarts.dedup();
625	}
626
627	let mut out = Vec::with_capacity(vstarts.len());
628	for (k, i) in vstarts.iter().enumerate() {
629	if k == 0 {
630	out.push((0usize, 0usize));
631	continue;
632	}
633	// The fragment's picture begins at this decode time on the film's clock,
634	// and the sound it carries is the sound from there on.
635	let at = (i as i64) step as i64;
636	let j = match arel.iter().position(\|t\| *t >= at) {
637	Some(j) => j,
638	None => arel.len(),
639	};
640	let prev = match out.last() {
641	Some((_, a)) => *a,
642	None => 0,
643	};
644	// A fragment with no sound in it would leave check 5 nothing to read at
645	// that join, so the cut is dropped rather than written empty.
646	if j <= prev \|\| j >= arel.len() {
647	continue;
648	}
649	out.push((*i, j));
650	}
651	Ok(out)
652	}
653
654	/// Applies the break named by `BREAK_FRAG` to the samples about to be written.
655	///
656	/// The expectations the checks hold the film against are built before this runs
657	/// and are not touched by it, so a break moves the film and not the yardstick.
658	/// An unrecognised name is an error: a mistyped break that quietly did nothing
659	/// would look exactly like a check that passed.
660	fn apply_break(
661	pic: &mut [Sample],
662	snd: &mut [Sample],
663	starts: &[(usize, usize)],
664	)
665	-> Outcome<()>
666	{
667	let name = match env::var("BREAK_FRAG") {
668	Ok(v) => v,
669	Err(_) => return Ok(()),
670	};
671	if name.trim().is_empty() {
672	return Ok(());
673	}
674	// Well inside the second fragment, so that no break sits on a boundary that
675	// a later check reads and none of them are proved by the wrong question.
676	let (vi, ai) = match starts.get(1) {
677	Some(v) => *v,
678	None => return Err(err!(
679	"BREAK_FRAG={} needs a film of more than one fragment.", name;
680	Invalid, Input)),
681	};
682	let v = vi + 3;
683	let a = ai + 3;
684	if v + 1 >= pic.len() \|\| a + 1 >= snd.len() {
685	return Err(err!(
686	"BREAK_FRAG={} needs a longer film than this one.", name;
687	Invalid, Input, Range));
688	}
689	match name.as_str() {
690	"picture-one-tick" => pic[v].off += 1,
691	"sound-one-tick" => snd[a].off += 1,
692	"sound-one-tick-dur" => snd[a].dur += 1,
693	"sound-delayed" => {
694	for s in snd.iter_mut() {
695	s.off += DELAY;
696	}
697	},
698	other => return Err(err!(
699	"BREAK_FRAG={} names no break this test knows. The breaks are \
700	picture-one-tick, sound-one-tick, sound-one-tick-dur and \
701	sound-delayed.", other; Invalid, Input)),
702	}
703	println!("BREAK_FRAG={} was applied, so this run is expected to FAIL.", name);
704	Ok(())
705	}
706
707	// ----------------------------------------------------------- asking ffprobe
708
709	/// The `key=value` fields ffprobe reports for one stream of a film.
710	///
711	/// Read as keys rather than as a bare comma-separated line, because the order of
712	/// the columns is ffprobe's business and a silent reordering would compare the
713	/// width against the height.
714	fn probe_fields(path: &Path, select: &str, entries: &str)
715	-> Outcome<Vec<(String, String)>>
716	{
717	let out = res!(Command::new("ffprobe")
718	.args([
719	"-v", "error",
720	"-select_streams", select,
721	"-count_packets",
722	"-show_entries", entries,
723	"-of", "default=noprint_wrappers=1",
724	])
725	.arg(path)
726	.output());
727	if !out.status.success() {
728	return Err(err!(
729	"ffprobe would not read stream {} of {}: {}",
730	select, path.display(), String::from_utf8_lossy(&out.stderr).trim();
731	Invalid, Mismatch));
732	}
733	let text = String::from_utf8_lossy(&out.stdout);
734	let mut fields = Vec::new();
735	for line in text.lines() {
736	if let Some(at) = line.find('=') {
737	fields.push((
738	line[..at].trim().to_string(),
739	line[at + 1..].trim().to_string(),
740	));
741	}
742	}
743	if fields.is_empty() {
744	return Err(err!(
745	"ffprobe found no stream {} in {} at all.", select, path.display();
746	Invalid, Mismatch));
747	}
748	Ok(fields)
749	}
750
751	/// One field of what ffprobe said.
752	fn pick(fields: &[(String, String)], key: &str) -> Outcome<String> {
753	for (k, v) in fields {
754	if k == key {
755	return Ok(v.clone());
756	}
757	}
758	Err(err!("ffprobe said nothing about `{}`.", key; Missing))
759	}
760
761	/// One field of what ffprobe said, as a number.
762	fn number(fields: &[(String, String)], key: &str) -> Outcome<i64> {
763	let v = res!(pick(fields, key));
764	match v.parse::<i64>() {
765	Ok(n) => Ok(n),
766	Err(_) => Err(err!(
767	"ffprobe gives `{}` as {}, which is not a number.", key, v;
768	Invalid, Mismatch)),
769	}
770	}
771
772	fn stream_count(path: &Path) -> Outcome<usize> {
773	let out = res!(Command::new("ffprobe")
774	.args([
775	"-v", "error",
776	"-show_entries", "stream=index",
777	"-of", "csv=p=0",
778	])
779	.arg(path)
780	.output());
781	if !out.status.success() {
782	return Err(err!(
783	"ffprobe would not read {}: {}",
784	path.display(), String::from_utf8_lossy(&out.stderr).trim();
785	Invalid, Mismatch));
786	}
787	let text = String::from_utf8_lossy(&out.stdout);
788	Ok(text.lines().filter(\|l\| !l.trim().is_empty()).count())
789	}
790
791	/// Decodes the whole film and refuses any word ffmpeg has to say about it.
792	///
793	/// The stronger half of the shape check: a container whose fragments are wrong
794	/// still opens and still reports a plausible packet count, and only a decode
795	/// says whether the bytes handed to the decoder were the frames. Anything at all
796	/// on stderr is a failure -- a warning about a broken `trun` is a broken `trun`
797	/// -- and it is quoted so the fault can be read rather than guessed at.
798	fn decodes_silently(path: &Path, film: &Path) -> Outcome<()> {
799	let out = res!(Command::new("ffmpeg")
800	.args(["-v", "error", "-i"])
801	.arg(path)
802	.args(["-f", "null", "-"])
803	.output());
804	let errs = String::from_utf8_lossy(&out.stderr);
805	if !errs.trim().is_empty() {
806	return Err(err!(
807	"{}: ffmpeg complained while decoding the fragmented film: {}",
808	film.display(), errs.trim(); Invalid, Mismatch));
809	}
810	if !out.status.success() {
811	return Err(err!(
812	"{}: ffmpeg gave up on the fragmented film without saying why.",
813	film.display(); Invalid, Mismatch));
814	}
815	Ok(())
816	}
817
818	/// The presentation times of one stream of a written film, in decode order.
819	fn times_of(path: &Path, select: &str, film: &Path) -> Outcome<Vec<i64>> {
820	let out = res!(Command::new("ffprobe")
821	.args([
822	"-v", "error",
823	"-select_streams", select,
824	"-show_entries", "packet=pts",
825	"-of", "csv=p=0",
826	])
827	.arg(path)
828	.output());
829	if !out.status.success() {
830	return Err(err!(
831	"{}: ffprobe would not read the times of stream {}.",
832	film.display(), select; Invalid, Mismatch));
833	}
834	let text = String::from_utf8_lossy(&out.stdout);
835	let mut times = Vec::new();
836	for line in text.lines() {
837	let v = line.trim();
838	if v.is_empty() {
839	continue;
840	}
841	match v.parse::<i64>() {
842	Ok(t) => times.push(t),
843	// `N/A` here is a packet the reader could not place, which is the
844	// fault this test is about and not a reason to read past it.
845	Err(_) => return Err(err!(
846	"{}: stream {} has a packet whose time reads `{}`.",
847	film.display(), select, v; Invalid, Mismatch)),
848	}
849	}
850	if times.is_empty() {
851	return Err(err!(
852	"{}: stream {} of the written film holds no packet times.",
853	film.display(), select; Invalid, Mismatch));
854	}
855	Ok(times)
856	}
857
858	// --------------------------------------------------------------- the corpus
859
860	fn gather(dir: &Path, out: &mut Vec<PathBuf>) -> Outcome<()> {
861	let entries = match fs::read_dir(dir) {
862	Ok(e) => e,
863	Err(_) => return Ok(()),
864	};
865	for entry in entries.flatten() {
866	let path = entry.path();
867	let meta = match fs::metadata(&path) {
868	Ok(m) => m,
869	Err(_) => continue,
870	};
871	if meta.is_dir() {
872	res!(gather(&path, out));
873	} else if meta.len() > 0
874	&& path.extension().map(\|e\| e.eq_ignore_ascii_case("mkv")).unwrap_or(false)
875	{
876	out.push(path);
877	}
878	}
879	Ok(())
880	}
881
882	fn num_from_env(key: &str, or: usize) -> usize {
883	match env::var(key) {
884	Ok(v) => v.parse::<usize>().unwrap_or(or),
885	Err(_) => or,
886	}
887	}