Speech Recognition

Play back a recorded sentence and the silences land in the wrong places. Words do not come with gaps.

Continuous speech is one smear of sound, and the pauses you think you hear are mostly not there. Statistical decoders handled that by treating the audio as the output of hidden states, states you never observe directly, and asking which sequence of them most likely produced the sound that arrived. The 1987 stochastic segment model pushed the same logic further, time-warping each variable-length phonetic segment to a fixed length and scoring it under one joint density. Against a hidden Markov phonetic model it cut word error rate by about a third on speaker-dependent continuous speech. The search stayed affordable through dynamic programming, a bookkeeping trick that reuses partial answers instead of retrying every possible path. Nothing here knows meaning, only that some sounds follow other sounds often, which is also what makes forced alignment possible: lining a transcript up against audio frame by frame.

Strangers sat in quiet rooms at MIT and read scripted sentences into a microphone. That was federal money at work.

Nobody publishes a career on recordings they made in their own hallway, because no rival lab can check the result. DARPA funding turned collection into infrastructure: TIMIT, then Resource Management, ATIS, and the CSR Wall Street Journal sets, designed once and shipped to every institution in the field. TIMIT alone supplied 6300 recorded utterances from 630 American English speakers as a shared benchmark. That is what made word error rates comparable between groups instead of anecdotal. Progress on large-vocabulary continuous speech became a public scoreboard rather than a set of private claims.

Say "ee" then "ah" and feel your tongue drop. The tube changed shape, so the sound changed loudness bands.

Those bands are formants: the frequencies a given mouth shape amplifies, visible as bright stripes on a spectrogram. Phonetics work going back to 1972 tied them, plus voice onset time (the lag between a consonant's release and the voice starting), to specific articulations. That gave decoders features grounded in anatomy rather than guessed spectral patterns. It also explained a failure mode. Children have shorter vocal tracts and different pitch, so a recognizer trained on adults degrades badly on them, and the repair is anatomical: normalize for vocal tract length, then modify pitch and speaking rate until the adult recordings look more like the children's.

Switchboard is telephone calls between strangers, badly recorded and fast. For two decades it was where recognizers went to look bad.

The 2012 review by Hinton, Deng, Yu, Dahl, Mohamed and seven co-authors did not replace the decoder. It replaced the scoring layer, swapping Gaussian mixtures (each sound modeled as a blend of bell curves) for a deep network that learns its own features from the audio. On large-vocabulary Switchboard speech, that swap moved word error rate from 27.4% to 18.5%, a gap of 8.9 percentage points. The statistics stayed statistics, just with a far better estimate of how likely each frame of sound was.