Describe the bug
I see a similar issue was raised before but didn't have enough detail to answer it. #827
When running the following code:
from speechbrain.pretrained import EncoderDecoderASR
asr_model = EncoderDecoderASR.from_hparams(source="speechbrain/asr-transformer-transformerlm-librispeech", savedir="pretrained_models/asr-transformer-transformerlm-librispeech")
trans = asr_model.transcribe_file("my/local/path/file.wav")
I get this error:
RuntimeError Traceback (most recent call last)
Cell In[10], line 3
1 with torch.no_grad():
2 wav_lens = rel_length.to(asr_model.device)
----> 3 encoder_out = asr_model.encode_batch(batch, rel_length)
File [c:\Users\francisw\Documents\repos\speech-to-text\.venv\Lib\site-packages\speechbrain\pretrained\interfaces.py:604](file:///C:/Users/francisw/Documents/repos/speech-to-text/.venv/Lib/site-packages/speechbrain/pretrained/interfaces.py:604), in EncoderDecoderASR.encode_batch(self, wavs, wav_lens)
602 wavs = wavs.float()
603 wavs, wav_lens = wavs.to(self.device), wav_lens.to(self.device)
--> 604 encoder_out = self.mods.encoder(wavs, wav_lens)
605 return encoder_out
File [c:\Users\francisw\Documents\repos\speech-to-text\.venv\Lib\site-packages\torch\nn\modules\module.py:1501](file:///C:/Users/francisw/Documents/repos/speech-to-text/.venv/Lib/site-packages/torch/nn/modules/module.py:1501), in Module._call_impl(self, *args, **kwargs)
1496 # If we don't have any hooks, we want to skip the rest of the logic in
1497 # this function, and just call forward.
1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1499 or _global_backward_pre_hooks or _global_backward_hooks
1500 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1501 return forward_call(*args, **kwargs)
1502 # Do not call functions when jit is used
1503 full_backward_hooks, non_full_backward_hooks = [], []
File [c:\Users\francisw\Documents\repos\speech-to-text\.venv\Lib\site-packages\speechbrain\nnet\containers.py:191](file:///C:/Users/francisw/Documents/repos/speech-to-text/.venv/Lib/site-packages/speechbrain/nnet/containers.py:191), in LengthsCapableSequential.forward(self, x, lengths)
189 x = layer(x, lengths=lengths)
190 else:
--> 191 x = layer(x)
192 if isinstance(x, tuple):
193 x = x[0]
File [c:\Users\francisw\Documents\repos\speech-to-text\.venv\Lib\site-packages\torch\nn\modules\module.py:1501](file:///C:/Users/francisw/Documents/repos/speech-to-text/.venv/Lib/site-packages/torch/nn/modules/module.py:1501), in Module._call_impl(self, *args, **kwargs)
1496 # If we don't have any hooks, we want to skip the rest of the logic in
1497 # this function, and just call forward.
1498 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
1499 or _global_backward_pre_hooks or _global_backward_hooks
1500 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1501 return forward_call(*args, **kwargs)
1502 # Do not call functions when jit is used
1503 full_backward_hooks, non_full_backward_hooks = [], []
File [c:\Users\francisw\Documents\repos\speech-to-text\.venv\Lib\site-packages\speechbrain\lobes\models\transformer\TransformerASR.py:348](file:///C:/Users/francisw/Documents/repos/speech-to-text/.venv/Lib/site-packages/speechbrain/lobes/models/transformer/TransformerASR.py:348), in EncoderWrapper.forward(self, x, wav_lens)
346 def forward(self, x, wav_lens=None):
347 """ Processes the input tensor x and returns an output tensor."""
--> 348 x = self.transformer.encode(x, wav_lens)
349 return x
File [c:\Users\francisw\Documents\repos\speech-to-text\.venv\Lib\site-packages\speechbrain\lobes\models\transformer\TransformerASR.py:301](file:///C:/Users/francisw/Documents/repos/speech-to-text/.venv/Lib/site-packages/speechbrain/lobes/models/transformer/TransformerASR.py:301), in TransformerASR.encode(self, src, wav_len)
298 pos_embs_source = self.positional_encoding(src)
300 elif self.positional_encoding_type == "fixed_abs_sine":
--> 301 src = src + self.positional_encoding(src)
302 pos_embs_source = None
304 encoder_out, _ = self.encoder(
305 src=src,
306 src_key_padding_mask=src_key_padding_mask,
307 pos_embs=pos_embs_source,
308 )
RuntimeError: The size of tensor a (140590) must match the size of tensor b (2500) at non-singleton dimension 1
I'm not sure this is necessarily a bug or just that I need to do some additional pre-processing of the wav file that I'm using.
When I run the lines leading up to the encoder_out = asr_model.encode_batch(batch, rel_length) line, the dimension of the batch that I'm passing to encode_batch is [1, 89977000].
waveform = asr_model.load_audio("my/local/path/file.wav")
batch = waveform.unsqueeze(0)
rel_length = torch.tensor([1.0])
print(batch.shape)
torch.Size([1, 89977000])
When inspecting the properties of my wave file, I see it has a bit rate of 705kbps. As I understand it, load_file should normalize this to meet the sampling used in the pre-trained model, but I think this part seems to be having issues? At least that's my best guess.
Any help would be greatly appreciated, thanks!
Expected behaviour
The audio file should be transcribed to text.
To Reproduce
No response
Versions
No response
Relevant log output
No response
Additional context
No response
Describe the bug
I see a similar issue was raised before but didn't have enough detail to answer it. #827
When running the following code:
I get this error:
I'm not sure this is necessarily a bug or just that I need to do some additional pre-processing of the wav file that I'm using.
When I run the lines leading up to the
encoder_out = asr_model.encode_batch(batch, rel_length)line, the dimension of the batch that I'm passing toencode_batchis [1, 89977000].When inspecting the properties of my wave file, I see it has a bit rate of 705kbps. As I understand it,
load_fileshould normalize this to meet the sampling used in the pre-trained model, but I think this part seems to be having issues? At least that's my best guess.Any help would be greatly appreciated, thanks!
Expected behaviour
The audio file should be transcribed to text.
To Reproduce
No response
Versions
No response
Relevant log output
No response
Additional context
No response