Dataset

No recorded Telugu corpus exists locally, so the evaluation set is synthesized: 129 template questions with exact reference answers, spoken by an open Telugu TTS model, in a clean and a noisy version.

Overview

Question templates

29 state and country capitals

Pattern: question "<place> రాజధాని ఏది?" (What is the capital of <place>?), answer "<place> రాజధాని <capital>." This follows the example pattern given in the requirements. IDs are cap-001 to cap-029.

100 addition problems

Every ordered pair (a, b) with a and b from 1 to 10, spoken with Telugu number words, so sums run from 2 to 20. IDs are sum-001 to sum-100. Pattern: question "<a> మరియు <b> కలిపితే ఎంత వస్తుంది?" (what do a and b make together?), answer "<a> మరియు <b> కలిపితే <sum> వస్తుంది."

Number words are spelled out (not digits) so the TTS pronounces them naturally. Both orders of each pair appear, which adds variety without new vocabulary.

The noisy condition

Zero-mean Gaussian noise is added to the clean waveform at a fixed signal-to-noise ratio:

noise_power = signal_power / 10^(snr_db / 10)
noisy = clean + Normal(0, sqrt(noise_power))
if max(|noisy|) > 1: noisy = noisy / max(|noisy|)     # avoid clipping

At 0 dB, noise power equals signal power. The noise generator is seeded (numpy seed 42), so the same noisy waveform is produced for the same clean waveform.

File formats

Synthesized corpus folder

corpus_clean/ and corpus_noisy/ use prepare_data.py's sidecar format, one triple per question:

corpus_clean/
  question_cap-001.wav             audio (sample rate = the TTS model's rate)
  question_cap-001.txt             reference transcript = the spoken question
  question_cap-001.response.txt    reference answer
  ...
  question_sum-100.{wav,txt,response.txt}

Evaluator manifest

prepare_data.py --limit 0 turns a folder into data/test_clean.json / data/test_noisy.json plus a .metadata.json. Each row:

{
  "id": "te-000001",
  "audio_path": "../corpus_clean/question_cap-001.wav",
  "ground_truth_transcript": "...",
  "ground_truth_response": "...",
  "response_source": "provided"
}

• id is te-NNNNNN by position in the sorted source listing (the folder format carries no ID of its own). The original cap-/sum- label survives in the audio file name, which hallucination.py relies on.
• audio_path is relative to the manifest's directory.
• response_source is provided for every row here. A placeholder mode exists for plumbing tests only; it fills missing answers with a fixed string and marks the dataset as not valid for response-quality scoring. None is used here (0 placeholders).
• Selection is reproducible (seeded) and validation stops on missing audio, duplicate audio or IDs, or empty labels instead of silently dropping samples.

Clean and noisy are not matched recordings

The noisy corpus is not the clean audio plus noise. The TTS model's duration prediction is not seeded, and the noisy set was regenerated once at the harsher SNR, so each condition is an independent rendering of the same text. Consequences:

• The clean-vs-noisy WER gap includes natural TTS rendering variance, not noise alone. The paired bootstrap over questions still shows the gap is real.
• The baseline-vs-denoised comparison is exactly matched: both runs read the identical corpus_noisy/ files and differ only in the asr.denoise flag. The extension's conclusion rests on that comparison.