Reference

Tests

30 unit tests in 8 files, all offline: no model downloads, no GPU, no network. They protect the evaluation code, which is where a silent bug would quietly corrupt every reported number.

python -m unittest discover -s tests -v

Expected: Ran 30 tests and OK in about two seconds. Use -s tests: the folder is not a package, so python -m unittest tests.test_x will not import.

What passing means It means the metric arithmetic, dataset handling, config guards, plotting and the denoise, hallucination and bootstrap logic behave as specified on constructed inputs. It does not mean the models are accurate; model quality is measured by the evaluation, not by the tests.

Coverage by file

File	Tests	Covers
`test_evaluate.py`	5	Edit distance and Unicode policy; quality score and RTF; dataset loading and validation; paired comparison and corpus weighting; batch reporting and failure handling
`test_pipeline.py`	4	Stage hand-off and the empty-transcript guard; config paths and validation; audio mono mix, resampling and missing file; LLM returns only the continuation and enforces the input budget
`test_prepare_data.py`	3	Folder round trip and seeded subset reproducibility; manifest column mapping and explicit placeholders; invalid sources and duplicate audio
`test_plot_results.py`	3	JSON and CSV round trip; missing scatter pairs and optional latency; invalid data is not silently plotted
`test_verify_task2.py`	5	Config guards; dependency version constraints are enforced; directory failures and independent mock status; three real evaluator pairs and Markdown output; wrong metric results raise instead of passing
`test_denoise.py`	3	Noise-power reduction on a bursty tone; input validation; short-signal pass-through
`test_hallucination.py`	4	Key-fact extraction for both templates; three-way classification; sum digit form counts but only as a whole number; rates over report rows
`test_bootstrap.py`	3	Identical systems give zero difference and interval; consistent improvement excludes zero; determinism for a seed and input validation
Total	30

Notable details

Metrics are checked against hand-computed values

test_evaluate.py and the self-check inside verify_task2.py use fixed cases with known answers (perfect match, substitution, deletion, insertion, empty reference, Unicode normalization), so a regression in the edit-distance code shows up as a mismatch rather than a plausible-looking number.

Models are replaced by stand-ins

Pipeline and evaluation tests inject simple fake ASR and LLM objects, and the verifier test patches from_pretrained to fail if a download is attempted, so they run anywhere. The wrappers' audio handling is tested on tiny generated WAV files.

Bad input must fail loudly

Several tests assert errors rather than success: malformed datasets, duplicate IDs, missing audio, wrong-shaped plot data, a wrong metric value in the verifier, an out-of-range denoise parameter, a mismatched bootstrap pair. The project's rule is that invalid data should never be silently dropped or plotted.

A test that was wrong first

The first denoise test used a constant sine tone, and failed (error went up, not down). The cause was the algorithm's assumption that each frequency bin has quiet frames, which a sustained tone does not. The test was corrected to use a bursty tone with gaps, which resembles speech, rather than weakening the assertion. See the extension page.

Growth of the suite

Stage	Tests
Original suite (pipeline, evaluate, prepare, plot, verify)	20
+ denoise	23
+ hallucination classifier	27
+ bootstrap	30

Not covered by tests

Real model inference (that is the evaluation itself).
synthesize_corpus.py' TTS step, which needs a model download. Its pure helpers (build_qa_pairs, add_noise) are not unit-tested either.
The LaTeX reports and this documentation site.