[Sync] merge to main for 3.0.2 release - bug fixes; knowledge indexing; future compatibility & dependencies - #3495
Merged
Merged
Conversation
Introduce a new utility module that centralizes colormap access, listing, registration, and unregistration across Matplotlib API versions. The new helpers prefer modern `matplotlib.colormaps` APIs when available and fall back to legacy `matplotlib.cm`/`pyplot` behavior, with deprecation markers on legacy paths to support the migration.
Adds a `stacklevel` argument to `deeplabcut.core.deprecation.deprecated()` and uses it when emitting `DLCDeprecationWarning`. This keeps the default behavior (`stacklevel=2`) while allowing wrapper/legacy shim layers to bump the level so warnings point to the real caller.
Replaced direct `plt.cm.get_cmap(...)` calls across visualization and plotting modules with `deeplabcut.utils.matplotlib_future_mode.get_colormap`. This centralizes colormap access, keeps behavior consistent across TensorFlow/PyTorch/GUI/3D paths, and aligns with Matplotlib future-mode compatibility.
Under pandas 3.0 Copy-on-Write, .to_numpy()/.values return read-only arrays for single-dtype selections; in-place mutation raises ValueError. Use copy=True at the five affected sites. Pin unchanged.
Follow-up to the pandas 3.0 CoW sweep (#3362). Three more sites extract a NumPy array from a pandas object and mutate it in place; under pandas 3 CoW these can be read-only views, raising "assignment destination is read-only". - refine_training_dataset/tracklets.py: self.data backs self.xy/self.prob, which swap_tracklets and the refine GUI mutate in place (HDF load path; flagged in review). - utils/make_labeled_video.py: coords is a view into xyp, masked in place in both the first-frame and per-frame branches. - pose_estimation_tensorflow/.../pose_multianimal_imgaug.py: a single-row float Series is read-only; kpts is masked in place when mask_kpts_below_thresh is set. Co-authored-by: Cursor <cursoragent@cursor.com>
Guards against the in-place write on the next line producing a read-only view when the intermediate frame becomes single-block.
…t of fixed eval_interval=10 for the detector as in base config
…_names Previously it duplicated the legacy lookup inline (list(plt.colormaps())) instead of calling the @deprecated-decorated _legacy_get_colormap_names(), so the deprecation warning never fired on old Matplotlib installs, unlike register_colormap/unregister_colormap.
_legacy_register_colormap/_legacy_unregister_colormap/_legacy_get_colormap_names now report their public wrapper names (e.g. register_colormap) in deprecation warnings instead of the private helper name.
update all import sites accordingly
Extend ProjectConfig.normalize_legacy_empty_vales to treat a bare/null YAML value as unset for fields whose type doesn't accept None (e.g. multianimalproject). This is required to universally support legacy config templates with unset fields.
Mark the parameter as deprecated in utils.read_config. This parameter was only needed for supporting empty values in legacy project configs. Instead, currently a more strict policy is enforced: null/empty is loaded as None, and fails validation if not supported. since f7305aa None-type support specifically for ProjectConfigs is tolerated using an after validator, which makes the current `ignore_empty` parameter obsolete
DictViewer passes the config path straight to QLabel, which only accepts str. Since the pathlib migration ConfigEditor hands it a Path, so opening "Edit config.yaml" raises TypeError. Reproduced on PySide6 6.7.3 and 6.9.3; it went unnoticed because the GUI tests are skipped where PySide6 is absent.
The deferred reload timer was duplicated in ManageProject and AnalyzeVideos, and each of the five tabs opened ConfigEditor slightly differently. They now share DefaultTab._open_config_editor, which keeps the dialog referenced and parents it to the main window. That fixes two latent bugs in the pose_cfg / inference_cfg tabs, where the dialog was a bare local: nothing kept it alive after the method returned, and without a parent ConfigEditor skipped its screen-relative sizing. Only the project config asks for a reload, since rebuilding the tabs is wrong for the other files.
Cover the dialog outliving the call that opened it, the single deferred reload after saving the project config, the absence of a reload for any other config, and the no-op when no project is loaded.
Add pre-validation in `plot_evaluation_results` to correctly split and compare ground-truth vs prediction individuals and bodyparts before reshaping arrays. When counts or bodypart sets mismatch, the function now logs a warning and skips that image instead of proceeding with invalid dimensions.
Adds a new test module for `deeplabcut.utils.visualization` focused on evaluation plotting behavior. The tests cover single-animal label handling across scorers, coordinate arrangement for both plotting modes, mismatch/malformed-data skip paths with expected messages, invalid mode validation, output filename conventions in `save_labeled_frame`, and bounding-box linestyle behavior in `make_multianimal_labeled_image`.
Tightened `plot_evaluation_results` by constraining `mode` with a `Literal` type and adding an explicit runtime `ValueError` for unsupported values. The plotting branch logic now treats non-`bodypart` mode consistently as `individual`, and warning messages for individual-count/reshape issues were rewritten to be clearer and more specific. Tests were updated to match the new warning text, and a matplotlib cleanup fixture was added to close figures after each test.
fix broken numpy 2.0 compat in crossvalutils and predict_multianimal
…Fitter Under numpy 2 numba types np.linalg.eig output as complex128, so the jitted _fit fails to type T @ a1. The fit needs the real part; taking it explicitly keeps the same result on numpy 1. Covered by tests/test_trackingutils.py::test_ellipse_fitter, which fails before and passes after on numpy 2.5.3 with numba 0.67.0.
deruyter92
marked this pull request as ready for review
September 9, 2026 13:01
fix(trackingutils): take the real part of eig eigenvectors in EllipseFitter
Expose an `overwrite` option in the compatibility `analyze_videos` wrapper, document its PyTorch-only behavior, and pass it through so callers can re-run analysis even when prediction files already exist.
To avoid silently re-running on stale outputs when overwrite is true
This change adds a regression test to ensure the compatibility wrapper passes the `overwrite` flag through to the PyTorch `analyze_videos` API. It fixes the bug where the flag was effectively pinned to `False` when using `Engine.PYTORCH`.
* Fix NumPy torch snapshot loading Add NumPy 2.0 compatibility for PyTorch snapshot loading by registering the correct scalar reconstruction helper and keeping NumPy float metrics loadable without changing weights_only. If registration fails, emit a warning explaining the fallback and the impact on snapshot loading. * Ensure affinity stays a float Add explicit float typing and return values for `Assembly.affinity`, and cast the affinity value before storing it in PyTorch pose tensors. This keeps the inference path type-consistent and avoids integer zero values when no links are present. * Clarify NumPy safe globals workaround Expand the snapshot-loading helper docstring to explain why NumPy reconstructors must be allowlisted for legacy metrics, note the NumPy version split more clearly, and mention that newer Torch releases may make the workaround unnecessary. * Raise numby, add opencv - opencv was used but never declared as a dependency - numba .054 is not compatible with our numpy lower bound now * Update uv.lock * Add regression tests for Assembly.affinity type * Fix numba minimum version spec Correct the `numba` dependency in `pyproject.toml` from `>=0.6` to `>=0.60` so the declared minimum version matches the intended release series. * Update uv.lock * Test tweak: avoid typecasting Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Remove redundant float typecast Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com> --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Jaap de Ruyter van Steveninck <32810691+deruyter92@users.noreply.github.com>
Add a `strict_cells` workflow input and use it to make only `matrix.strict` canary lanes fail the job outright. The intelligent testing workflow now enables strict cells for pushes and same-repo PRs while keeping fork PRs non-blocking, and the reporting step skips annotations for lanes that already fail normally.
* release upper bounds for numpy and matplotlib * keep np upper bound for tensorflow extras * add numpy-1 to canary lane * Fix intelligent testing matrix JSON Add the missing comma between the `matplotlib` and `numpy-1` matrix entries in `.github/workflows/intelligent-testing.yml` so the GitHub Actions matrix parses correctly. * Expand intelligent test matrix Broaden the intelligent-testing workflow to include strict dependency-resolution checks for Python 3.10 and 3.12, plus NumPy 1/2 and Matplotlib upgrade variants. This improves compatibility coverage and validates resolver behavior beyond the previous narrower matrix. * Adjust matplotlib and TF numpy constraints Restore exclusion of matplotlib 3.7 and 3.7.1 from the base dependencies, and drop the redundant numpy cap from the `tf-cu12` extra because TensorFlow 2.18 already declares the required numpy range. * Update uv.lock * Fix missing ceiling in tf * Update uv.lock * update pyproject.toml and uv.lock. Only declare np upperbound when required * Restore numba/opencv floors from #3508 after restack --------- Co-authored-by: C-Achard <cyril.achard@epfl.ch>
…ments Follow-up hardening for the LLM knowledge index
Fix hard-coded overwrite kwarg in pytorch `analyze_videos`
Merged
MMathisLab
approved these changes
Sep 15, 2026
MMathisLab
left a comment
Member
There was a problem hiding this comment.
Looks amazing -- great work @deruyter92 @C-Achard
C-Achard
approved these changes
Sep 15, 2026
AlexEMG
approved these changes
Sep 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Merge the current
devsnapshot tomainin preparation for 3.0.2 release.Caution
This does not bump the project version yet. Do not make a PyPI release before merging the version bump in #3496
Milestones
Dependencies & Future compatibility
Adapt old usage patterns of libraries that changed their API, allowing us to release the upper bounds of our dependency stack and improve broad compatibility.
.to_numpy()/.valuesmutationsAgentic interaction
Improve the access to domain-specific knowledge for agents that interact with DeepLabCut.
AGENTS.mdfor AI-assisted developmentBug fixes
DefaultTabCOCOLoader- makemax_individualsauthorative instead of inferring fromtrain.json#3435 — FixCOCOLoader: treatmodel_cfg.metadata/max_individualsas authoritativebox_score_thresh: nullanalyze_videos#3514 — Fix hard-coded overwrite=False in PyTorch analyze_videos / auto-track pathtfextra is absentNone-type validation for legacy ProjectConfig templates Removeignore_empty#3439 — Legacy ProjectConfigNone/empty handling; removeignore_emptystr/Pathconversionscreate_videos#3468 — Latent bugs / misleading errors in GUIcreate_videosWandbLoggerConfigso extra wandb kwargs validate (scoop intowandb_kwargs)export_modeldumping unnormalized objects in pickle #3473 —export_modelpickle: normalize typed config (normalize=True)config_pathinMainWindowconfig tests #3480 — GUI tests: useconfig_pathafter renametoolsimports under a plainpytestinvocation #3484 — Fixtoolsimports under plainpytest(addpythonpath = ["."])examples/README.mdFeatures / enhancements
detector_eval_intervaltotrain_from_coco(Refresh core API)DefaultIfNoneconfig validator (null→ field default); used for detector threshwandb.init)