Streaming ASR interfaces - #2377

What does this PR do?

The goal of this PR is to provide high-level interfaces for streaming inference using the model introduced by #2140, and to provide some shared infrastructure for future streaming models.

I have hosted a LibriSpeech streaming ASR model at https://huggingface.co/sdelangen/speechbrain-asr-conformer-test/tree/main. There is currently no model card, but it will be possible to migrate this model under the speechbrain org once this is merged.

Introduced interfaces

The StreamingASR inference interface was introduced.

Features:

• Streaming decode of long audio files (if ffmpeg is installed), see demos
• Streaming decode of live streams (if ffmpeg is installed), see demos
• Chunkwise decoding support (with an exposed lower-level transcribe_chunk/decode_chunk API)
• Batching support for the lower-level interfaces which makes it suitable as a basis for transcription servers
• Relatively model agnostic

Feature extraction

In the current conformer-transducer model, we have some feature extraction logic that stacks filter banks and 2 layers of down-sampling convolutions.
Ignoring normalization, we can consider feature extraction as a "filter" with a specific window size and stride.

Dynamic chunk training does not change how feature extraction is performed: it is applied over the entire batch at once, and is unaware of chunking.
This is mostly for simplicity, flexibility and training performance reasons.

Thus, when streaming, we have to be careful about how we chunk up our signal: for any given chunk, we must provide some past and future frames so that the transition across chunks remains as close to training as possible.

Streaming feature extraction abstraction

This logic is abstracted by speechbrain.lobes.features.StreamingFeatureWrapper as introduced by this PR. When provided the "filter properties" of our feature extractor (window size and stride), it is able to process a stream of audio chunk by chunk while abstracting padding and left context saving details¹.

Automatically determining filter properties

Those filter properties remain to be provided by the model developer. For this, we introduce speechbrain.utils.filter_analysis. It allows modelling simple filter properties, and provides utilities to "stack" them, i.e. model what the properties of filter2(filter1(x)) are.

In our case, we are essentially trying to model conv_layer_2(conv_layer_1(fbanks(waveform)))².

This allows us to determine the filter properties of our feature extraction easily and requires little to no adjustment when changing feature extraction hyperparameters.
This PR introduces a get_filter_propreties method for some modules, so all that remains is providing StreamingFeatureWrapper our feature extractor module list.

¹ This comes with a latency cost which depends on the window size and stride. However, that overhead is generally insignificant in practice as long as the window size and stride are kept reasonable.

² ignoring normalization. Not considering normalization specifically in our chunking process does not appear to have a significant effect on WER.

Example programs

from argparse import ArgumentParser
import logging

parser = ArgumentParser()
parser.add_argument("audio_path")
parser.add_argument("--model-source", required=True)
parser.add_argument("--device", default="cpu")
parser.add_argument("--ip", default="127.0.0.1")
parser.add_argument("--port", default=9431)
parser.add_argument("--chunk-size", default=24, type=int)
parser.add_argument("--left-context-chunks", default=4, type=int)
parser.add_argument("--num-threads", default=None, type=int)
parser.add_argument("--verbose", "-v", default=False, action="store_true")
args = parser.parse_args()

if args.verbose:
    logging.getLogger().setLevel(logging.INFO)

logging.info("Loading libraries")

from speechbrain.inference.ASR import StreamingASR
from speechbrain.utils.dynamic_chunk_training import DynChunkTrainConfig
import torch

device = args.device

if args.num_threads is not None:
    torch.set_num_threads(args.num_threads)

logging.info(f"Loading model from \"{args.model_source}\" onto device {device}")

asr = StreamingASR.from_hparams(args.model_source, run_opts={"device": device})
config = DynChunkTrainConfig(args.chunk_size, args.left_context_chunks)

logging.info(f"Starting stream from URI \"{args.audio_path}\"")

for text_chunk in asr.transcribe_file_streaming(args.audio_path, config):
    print(text_chunk, flush=True, end="")

TODOs

Not sure how to handle things like rescoring that could change entire sentences, but let's consider it out-of-scope for StreamingASR.

PR review