Reference
Limitations
What these results and analyses do not show. Each item says how much it matters and what would fix it.
Part I: analysis and proposal
| Limitation | Weight | What would address it |
|---|---|---|
| No experiment was run. The yielding-controller hypothesis is specified but untested; no gain, dataset size, parameter count or runtime is claimed. | Fundamental | Implement the controller, collect and annotate the data, run the constrained comparison on the proposal page. |
| Statements about the models rest on two reports. Later releases or code may differ, and reports omit many quantities (see what is unspecified). | Moderate | Check against released code and checkpoints. |
| The three limitations are ranked by judgment, not measured. | Moderate | Component-level timing and event-level evaluation of the real systems. |
| Implementation access is unverified: whether Moshi's temporal context and PAD probabilities are exposed; what suppressing audio does to its internal state; Fisher licensing. | Moderate | Read the implementation; check licenses. |
| Novelty is unchecked. Turn prediction already exists (TurnGPT, Voice Activity Projection). Whether a speech generator's hidden state has been used for action selection is not searched. | Scope | A wider literature search. No priority claim is made. |
Part II: the Telugu pipeline
| Limitation | Weight | What would address it |
|---|---|---|
| Synthetic speech only. All audio is TTS output. Nothing here speaks to accents, prosody, disfluency, speaking rate or microphone channels. | Fundamental | Evaluate on real recorded Telugu (an exported FLEURS or IndicVoices subset can be fed through prepare_data.py as a manifest). |
| Small models, by memory necessity. A larger ASR/LLM pair would likely change both WER and, above all, factual accuracy. The 0/129 result is specific to Qwen2.5-1.5B. | Fundamental | A GPU with room for Qwen2.5-7B-Instruct (or quantization) and Whisper-large-v3. |
| Templated questions. Only capitals and additions, short utterances, one fact each. | Moderate | Open-ended or multi-turn questions with a human judge or semantic scorer. |
| Clean and noisy are not matched recordings. TTS duration prediction was not seeded, so the clean-vs-noisy gap includes rendering variance. The baseline-vs-denoised comparison is matched. | Moderate | Seed TTS once, save the clean waveforms, add noise to those exact files. |
| Quality is a lexical proxy. Word-overlap F1 does not understand paraphrase or inflection. The hallucination test is a substring heuristic, so its rate is an upper bound. | Moderate | A semantic or human-judged correctness score. evaluate.py accepts a custom scorer. |
| One deterministic run per condition. Intervals reflect question sampling only, not run-to-run, speaker or recording variance. | Moderate | More question sets, speakers and TTS voices. |
| The noise level was chosen to create a bottleneck. 0 dB white Gaussian noise is harsher and simpler than most real conditions, and 8 dB showed no effect. | Disclosed | Realistic noise types (babble, reverberation, codec artifacts) at realistic SNRs. |
| The denoiser was tried at two settings. A negative result for this algorithm at these parameters is not a statement about denoising in general (learned enhancement models were not tried). | Scope | A neural speech-enhancement front end; or fine-tuning the recognizer on noisy speech. |
The ASR checkpoint is unpinned. vasista22/whisper-telugu-small resolved to two different commits for config and weights during the session, so pinning one broke loading. A future upstream change could shift results. | Reproducibility | Pin a consistent commit once the upstream repository stabilizes, or vendor the weights locally. |
| Latency is hardware-specific. RTF depends on this laptop GPU, the 256-token cap and greedy decoding. It is not streaming first-audio latency. | Scope | Report on target hardware; add a streaming first-token measurement. |
What can and cannot be claimed
Supported
The pipeline runs end to end. WER is 0.293 clean. Noise raises WER by a real, interval-backed amount. Denoising does not measurably change it. From speech, the 1.5B LLM never gave the right fact.
Suggestive
Aggressive spectral subtraction is worse than none (25-question slice only). The LLM, not ASR, is the main lever for quality.
Not shown
That the yielding controller works. That results hold on real speech, other languages, bigger models, or other noise. That cascades are slower than end-to-end systems.