Handle empty PyTorch video predictions without crashing - #3488
Open
deruyter92 wants to merge 20 commits into
Open
deruyter92 wants to merge 20 commits into
deruyter92 wants to merge 20 commits into
Conversation
Handle the valid case where PyTorch video inference returns zero predictions.
videos with zero detections would fail after serialization on `np.stack([])` Return an empty DataFrame with the expected DLC column schema instead and add a regression test for the empty-output case.
2 tasks
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Empty HDF exports remain unreadable, multi-animal schemas are incomplete, and one regression assertion fails.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Prevents PyTorch video analysis from crashing when inference produces no predictions.
Changes:
- Safely computes frame-key widths for empty predictions.
- Adds empty DataFrame export handling and clearer warnings.
- Adds regression tests for empty-prediction paths.
File summaries
| File | Description |
|---|---|
deeplabcut/pose_estimation_pytorch/apis/videos.py |
Handles empty predictions during serialization and warning generation. |
tests/pose_estimation_pytorch/apis/test_videos.py |
Adds empty-output regression tests. |
Review details
- Files reviewed: 2/2 changed files
- Comments generated: 3
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- add informative warning - add guard in dataframe export: raise ValueError if no predictions
deruyter92
marked this pull request as ready for review
September 9, 2026 06:33
Ensure inference emits results for images whenever a prediction exists, even when no detections are produced and thus no queue item is generated. This avoids dropping frames with valid predictions but empty detection outputs.
Fixes a mismatch in `analyze_videos` where the DataFrame export step still used `predictions` instead of `ctd_predictions`. This ensures saved video analysis results use the corrected prediction set from the current processing path.
When a video produces no pose predictions, analysis now writes all-NaN placeholder predictions so downstream HDF5 export and filtering can continue instead of failing on an empty result set. The DataFrame export path was adjusted to handle these fallback frames correctly and raises clearer errors only for genuinely invalid empty input. FMPOSE 3D inference now also raises a clear RuntimeError when no 2D poses are detected for a video.
Move frame key zero-padding logic into `runners.shelving.frame_key_width` and use it from both video output generation and shelf writing. This keeps frame key formatting consistent across full-data exports and on-the-fly shelving, while preserving the default width when the frame count is unknown.
Refines `video_inference` logging so empty results distinguish unreadable videos from detector misses, and keeps re-encoding guidance for partial reads. Expands tests to cover zero-prediction, partial-prediction, and shelf-writer cases.
Simplifies `analyze_videos` by always passing the inference `predictions` directly to `create_df_from_prediction` when `save_as_df` is enabled, and removes the special-case branch that generated all-NaN placeholder rows when no predictions were produced. The unused `_nan_predictions` helper and its warning/error handling were also deleted.
Use the CTD prediction list when deciding whether to write the dataframe, so `.h5` export still happens based on the converted CTD results instead of an unrelated variable. The warning now also explains that the CTD pickle had no frames when export is skipped.
Add an explanatory comment in `analyze_videos` documenting why `ctd_predictions` can be empty, and why the code warns and skips dataframe creation instead of raising. This captures edge cases like zero-frame pickle output and `save_as_df=False`.
Refines the test docstring in `test_runners_inference.py` to describe the async behavior more clearly by comparing it to the sequential path without implying a regression pin.
ShelfWriter now always derives its frame key width through `frame_key_width()` instead of keeping a separate default. This removes duplicated fallback logic and keeps frame index formatting consistent with the shared helper.
Collaborator
|
@deruyter92 Sorry this stayed as-is, the latest commits were the review in a sense; can I do anything else here? Should I review once more? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes a set of related bugs in the PyTorch
analyze_videosoutput path when inference produces no predictions, which can happen legitimately for clips where no animal is detected.Problem
video_inference()can return an empty predictions list when no detections are produced across the whole clip:_generate_output_data()computes frame-key width withnp.log10(len(predictions)), which crashes withOverflowErrorwhenlen(predictions) == 0.np.stack(...)is called on the empty list + a misleading warning message is emmitted).Changes
Related issues
Current issue:
Previous related issues: