Neural Audio Codec
What determines the rate–distortion ceiling of scalar-quantized speech codecs? (M.Sc. thesis)
EntroCodec — a causal, streaming-capable neural speech codec built as the instrument for a controlled empirical study, not an attempt to beat state-of-the-art systems. M.Sc. thesis at TU Ilmenau, Faculty of Electrical Engineering and Information Technology — supervised by Prof. Gerald Schuller (TU Ilmenau), co-supervised by Andreas Brendel (Fraunhofer IIS, Erlangen).
The question
When a neural speech codec uses scalar quantization and a fixed, non-learned entropy coder, what actually sets its quality ceiling — quantizer resolution, or the information content the training objective places in the latent?
Approach
An eight-phase controlled curriculum, not a hyperparameter sweep: each phase loads the previous phase’s checkpoint and changes exactly one aspect of the training objective, holding architecture and data fixed. That single-variable-at-a-time design is what makes the result attributable to a specific cause rather than an aggregate correlation.
A causal transformer encoder/decoder (~38M params) compresses 16kHz speech through a 32-dimension bottleneck, 3-bit uniform-quantized and zlib-coded, to ~5.9 kbps. zlib is used deliberately for its lack of learned adaptivity — because it only exploits generic statistical redundancy, its achieved compression ratio is an unbiased probe of the latent’s own Shannon entropy.
Finding
Latent entropy sets the ceiling — not quantizer resolution:
- Entropy and quality move together, in both directions, across the training curriculum. The deliberate counter-experiment (Phase D-VAE: a KL term that directly penalizes latent entropy, with no change to the reconstruction loss) drops entropy from ~1.5 to 1.09 bits — and quality drops with it. That’s the piece that turns correlation into evidence.
- Adding quantization bits stops helping. Sweeping Phase G’s trained weights from 1-bit to 6-bit at inference (no retraining): 1→3-bit produces real gains, but 3→6-bit triples bitrate (5.87 → 15.18 kbps) while STOI moves only 0.793 → 0.806.
- Causality isn’t why a gap to EnCodec remains. A non-causal ablation (bidirectional attention) changes almost nothing (PESQ 1.269 vs. 1.279) — the direction of the tiny delta flips per speaker, the signature of measurement noise rather than a real effect. The paper’s argument is that EnCodec’s adversarial training produces a fundamentally different latent-shaping signal than any reconstruction-based loss used here can supply.
Results
Evaluated on LibriSpeech test-clean (5 speakers, 5-second clips, 16 kHz mono):
| Codec | Bitrate | PESQ-WB | STOI |
|---|---|---|---|
| AAC | ~16 kbps | 1.641 | 0.855 |
| EnCodec 6.0 kbps (Meta) | 6.0 kbps | 2.842 | 0.922 |
| EntroCodec — Phase G (best) | 5.9 kbps | 1.279 | 0.766 |
The contribution is the controlled evidence for why scalar-quantization codecs hit their quality ceiling, not closing the gap to systems with structurally different architectures (EnCodec uses residual vector quantization and adversarial training, neither replicated here).
Known limitations — disclosed
- A
torch.triu/torch.trilbug made the intended 200-frame sliding-attention window a no-op — the model actually trained on full unbounded causal attention across the ~1,995-frame chunk. Reported metrics reflect this actual behavior. - No positional encoding — temporal order comes from causal convolutions and the causal attention mask only.
- Dropout was never active in any training run; regularization came only from noise augmentation and the D-VAE phase’s KL term.
Stack
Python, PyTorch, torchaudio, causal transformers, scalar quantization + zlib entropy coding, PESQ, STOI, LibriSpeech.
Status
The 20 CP research project is complete — project Exposé submitted, manuscript in preparation. The M.Sc. thesis (30 CP) extends it, testing whether the entropy-quality coupling generalizes to music, other latent widths, and vector quantization.