Conversation
…tected
`infer_device` guessed the running device with `torch.cuda.is_available()`,
falling back to "cpu" on any other accelerator. On machines with a non-CUDA
accelerator (e.g. Ascend NPU via torch_npu, or any PrivateUse1 backend
registered through the torch.accelerator API), this forces Brain and the
pretrained interfaces onto the CPU even though an accelerator is present,
and multi-process runs then all pile onto the CPU.
PyTorch 2.x exposes `torch.accelerator.is_available()` and
`torch.accelerator.current_accelerator()` as the device-agnostic equivalent:
they return "cuda" on CUDA machines (identical behavior) and "npu"/"mps"...
where a corresponding backend is available.
- `infer_device` now uses the accelerator API when available, keeping the
LOCAL_RANK suffix behavior for multi-process runs.
- `current_accelerator()` returns a `torch.device` on newer versions, so the
result is stringified before appending the rank suffix.
- Adds unit tests for both the plain and LOCAL_RANK paths.
Verified on Ascend 910B (torch 2.14.0a0 + torch_npu):
- old code returns "cpu" while an NPU is available
- new code returns "npu"; `torch.nn.Linear(2,2).to("npu")` and a forward
pass work; with LOCAL_RANK=1 the device is "npu:1"
- on CUDA machines behavior is unchanged ("cuda"[:rank])
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
speechbrain.utils.distributed.infer_device()guesses the running device withtorch.cuda.is_available(), falling back to"cpu"on any other accelerator. On machines with a non-CUDA accelerator (e.g. Ascend NPU viatorch_npu, or any backend registered through thetorch.acceleratorAPI), this forcesBrainand the pretrained interfaces (speechbrain.inference.interfaces) onto the CPU even though an accelerator is present, and multi-process runs all pile onto the CPU instead of using one accelerator per rank.Root cause
Only CUDA is considered; every other accelerator is invisible to the guess.
Fix
Use the device-agnostic PyTorch accelerator API, which returns
"cuda"on CUDA machines (identical behavior) and the actual accelerator type (e.g."npu","mps") where available:current_accelerator()returns atorch.deviceon newer versions, so the result is stringified before appending the rank suffix (the function's return type staysstr).Adds unit tests for both the plain and
LOCAL_RANKpaths.Note:
torch.acceleratoris available since PyTorch 2.4, while SpeechBrain pinstorch>=2.1.0— happy to guard this with agetattrfallback if you prefer keeping 2.1–2.3 compatibility.Verification
Verified on Ascend 910B (aarch64, torch 2.14.0a0 + torch_npu 2.14.0):
"cpu"while an NPU is available (the bug)"npu";torch.nn.Linear(2, 2).to("npu")plus a forward pass run correctlyLOCAL_RANK=1the returned device is"npu:1""cuda"/"cuda:<rank>")