Skip to content

Handle empty PyTorch video predictions without crashing - #3488

Open
deruyter92 wants to merge 20 commits into
devfrom
jaap/fix-empty-video-predictions
Open

deruyter92 wants to merge 20 commits into
devfrom
jaap/fix-empty-video-predictions

Conversation

@deruyter92

@deruyter92 deruyter92 commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

This PR fixes a set of related bugs in the PyTorch analyze_videos output path when inference produces no predictions, which can happen legitimately for clips where no animal is detected.

Problem

  1. For top-down inference, video_inference() can return an empty predictions list when no detections are produced across the whole clip: _generate_output_data() computes frame-key width with np.log10(len(predictions)), which crashes with OverflowError when len(predictions) == 0.
  2. That case is currently not handled consistently downstream. (e.g. np.stack(...) is called on the empty list + a misleading warning message is emmitted).

Changes

  • Guard frame-key width generation for zero predictions.
  • No DataFrames are exported when empty predictions. A clear warning is emitted.
  • Add regression coverage for empty-prediction output paths.

Related issues
Current issue:

Previous related issues:

Handle the valid case where PyTorch video inference returns zero predictions.
@deruyter92
deruyter92 changed the base branch from main to dev September 8, 2026 07:31
videos with zero
detections would fail after serialization on `np.stack([])`

Return an
empty DataFrame with the expected DLC column schema instead and add a
regression test for the empty-output case.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Empty HDF exports remain unreadable, multi-animal schemas are incomplete, and one regression assertion fails.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Prevents PyTorch video analysis from crashing when inference produces no predictions.

Changes:

  • Safely computes frame-key widths for empty predictions.
  • Adds empty DataFrame export handling and clearer warnings.
  • Adds regression tests for empty-prediction paths.
File summaries
File Description
deeplabcut/pose_estimation_pytorch/apis/videos.py Handles empty predictions during serialization and warning generation.
tests/pose_estimation_pytorch/apis/test_videos.py Adds empty-output regression tests.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 3
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread deeplabcut/pose_estimation_pytorch/apis/videos.py Outdated
Comment thread tests/pose_estimation_pytorch/apis/test_videos.py
Comment thread deeplabcut/pose_estimation_pytorch/apis/videos.py Outdated
@deruyter92
deruyter92 marked this pull request as ready for review September 9, 2026 06:33
@deruyter92
deruyter92 requested a review from C-Achard September 9, 2026 06:34
Ensure inference emits results for images whenever a prediction exists, even when no detections are produced and thus no queue item is generated. This avoids dropping frames with valid predictions but empty detection outputs.
Fixes a mismatch in `analyze_videos` where the DataFrame export step still used `predictions` instead of `ctd_predictions`. This ensures saved video analysis results use the corrected prediction set from the current processing path.
When a video produces no pose predictions, analysis now writes all-NaN placeholder predictions so downstream HDF5 export and filtering can continue instead of failing on an empty result set. The DataFrame export path was adjusted to handle these fallback frames correctly and raises clearer errors only for genuinely invalid empty input. FMPOSE 3D inference now also raises a clear RuntimeError when no 2D poses are detected for a video.
Move frame key zero-padding logic into `runners.shelving.frame_key_width` and use it from both video output generation and shelf writing. This keeps frame key formatting consistent across full-data exports and on-the-fly shelving, while preserving the default width when the frame count is unknown.
Refines `video_inference` logging so empty results distinguish unreadable videos from detector misses, and keeps re-encoding guidance for partial reads. Expands tests to cover zero-prediction, partial-prediction, and shelf-writer cases.
Simplifies `analyze_videos` by always passing the inference `predictions` directly to `create_df_from_prediction` when `save_as_df` is enabled, and removes the special-case branch that generated all-NaN placeholder rows when no predictions were produced. The unused `_nan_predictions` helper and its warning/error handling were also deleted.
Use the CTD prediction list when deciding whether to write the dataframe, so `.h5` export still happens based on the converted CTD results instead of an unrelated variable. The warning now also explains that the CTD pickle had no frames when export is skipped.
Add an explanatory comment in `analyze_videos` documenting why `ctd_predictions` can be empty, and why the code warns and skips dataframe creation instead of raising. This captures edge cases like zero-frame pickle output and `save_as_df=False`.
@deruyter92 deruyter92 added 3.1 and removed 3.0.2 labels Sep 9, 2026
Refines the test docstring in `test_runners_inference.py` to describe the async behavior more clearly by comparing it to the sequential path without implying a regression pin.
ShelfWriter now always derives its frame key width through `frame_key_width()` instead of keeping a separate default. This removes duplicated fallback logic and keeps frame index formatting consistent with the shared helper.
@C-Achard

Copy link
Copy Markdown
Collaborator

@deruyter92 Sorry this stayed as-is, the latest commits were the review in a sense; can I do anything else here? Should I review once more?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

analyze_videos (PyTorch): OverflowError in _generate_output_data when a video yields zero detections (empty predictions)

3 participants