Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
f3a0c4d
[WIP] Add DeepLabCut-live GUI documentation
C-Achard Feb 6, 2026
897ff8e
Revert indent changes
C-Achard Feb 6, 2026
a49939e
Update _toc.yml
C-Achard Feb 6, 2026
d3ebacc
Merge branch 'cy/fix-toc-jb1.0.4post1' into cy/dlclivegui2.0-docs
C-Achard Feb 6, 2026
a9b413e
Move dlc-live previous docs
C-Achard Feb 10, 2026
febcab8
Reorganize and update DLC-Live GUI docs
C-Achard Feb 10, 2026
bc1903c
Add conda install instructions and clarify env
C-Achard Feb 10, 2026
53dc969
Revise dlclivegui overview and UI guide
C-Achard Feb 10, 2026
b089715
Delete myst.yml
C-Achard Feb 10, 2026
d8f0cf2
Reorganize camera backend docs & update TOC
C-Achard Feb 10, 2026
9a8faaa
Update overview.md
C-Achard Feb 10, 2026
f0f6d33
Update camera backends docs and TOC
C-Achard Feb 10, 2026
139b508
Move timestamp_format into user_guide/misc
C-Achard Feb 10, 2026
60341fe
Remove dlc-live-gui README and features
C-Achard Feb 10, 2026
31b7efc
Fix camera docs link and add OpenCV note
C-Achard Feb 10, 2026
77f7dba
Docs: normalize DeepLabCut-live-GUI branding
C-Achard Feb 10, 2026
b1aad42
Merge branch 'main' into cy/dlclivegui2.0-docs
C-Achard Feb 10, 2026
b83bc3a
Docs: Aravis backend config & link updates
C-Achard Feb 11, 2026
2b6d8a4
docs: improve Aravis backend documentation
C-Achard Feb 11, 2026
dde2a84
Update camera backend docs and add install tabs
C-Achard Feb 11, 2026
4a8db74
docs: clarify video timestamps and recorder notes
C-Achard Feb 11, 2026
b77f8db
Update DeepLabCut-live-GUI install guide for DLClive 1.1.0 version
C-Achard Feb 16, 2026
e63bd17
Update install.md
C-Achard Feb 16, 2026
3f63d8d
Add Basler camera backend documentation
C-Achard Feb 17, 2026
932758e
Specify supported Python versions
C-Achard Feb 17, 2026
7a08b30
Create gentl_backend.md
C-Achard Feb 17, 2026
13f8b60
Update aravis_backend.md
C-Achard Feb 17, 2026
144c450
Add timestamp note; update visualization section
C-Achard Feb 17, 2026
fc2b7a7
Add recording path note and undock tip
C-Achard Feb 17, 2026
b90f4aa
Create opencv_backend.md
C-Achard Feb 17, 2026
ad4bad5
Tidy up camera docs
C-Achard Feb 17, 2026
0523771
Create modelzoo_downloads.md
C-Achard Feb 17, 2026
3656c89
Update opencv_backend.md
C-Achard Feb 17, 2026
5d7f554
Update modelzoo_downloads.md
C-Achard Feb 17, 2026
44182b5
Add misc landing page and update TOC & install
C-Achard Feb 17, 2026
d26b74e
Merge branch 'main' into cy/dlclivegui2.0-docs
C-Achard Feb 17, 2026
8ee492c
Fix typo
C-Achard Feb 17, 2026
fb280af
Merge branch 'cy/dlclivegui2.0-docs' of https://github.com/C-Achard/D…
C-Achard Feb 17, 2026
5c1defb
Update index.md
C-Achard Feb 20, 2026
da08158
docs(dlc-live-gui): update install quickstart
C-Achard Feb 20, 2026
7fa3488
Update main_window_100226.png
C-Achard Feb 20, 2026
a31b107
Add GPU install note and tidy backend install blocks
C-Achard Feb 20, 2026
d034450
docs(dlc-live-gui): refine docs and install quickstart
C-Achard Feb 26, 2026
f77bbf3
Add Sphinx cross-reference anchors for backends
C-Achard Feb 26, 2026
733f2bc
Docs: update camera backend guidance
C-Achard Feb 26, 2026
148e470
Improve OpenCV backend docs structure
C-Achard Feb 26, 2026
42d0d16
Refactor Aravis backend docs layout
C-Achard Feb 27, 2026
3b05b82
docs: restructure Basler backend user guide
C-Achard Feb 27, 2026
c3588a9
docs: improve GenTL backend user guide
C-Achard Feb 27, 2026
094176e
Relocate frame-size note to overview docs
C-Achard Feb 27, 2026
5cc5403
Add detector_name to modelzoo export example
C-Achard Feb 27, 2026
ebb7a55
Mention detectors and add detector_name
C-Achard Mar 2, 2026
1a9d788
Add model zoo download/export note
C-Achard Mar 3, 2026
0a0b68b
Clarify TensorFlow note in modelzoo docs
C-Achard Mar 3, 2026
a806ed9
Merge branch 'main' into cy/dlclivegui2.0-docs
deruyter92 Mar 9, 2026
968475f
Update install.md
C-Achard Mar 9, 2026
23b0275
Fix quickstart directory and package name
C-Achard Mar 9, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions _toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,21 @@ parts:

- caption: DeepLabCut-Live!
chapters:
- file: docs/deeplabcutlive
- file: docs/dlc-live/deeplabcutlive
- file: docs/dlc-live/dlc-live-gui/index
sections:
- file: docs/dlc-live/dlc-live-gui/quickstart/install
- file: docs/dlc-live/dlc-live-gui/user_guide/overview
- file: docs/dlc-live/dlc-live-gui/user_guide/cameras_backends/camera_support
sections:
- file: docs/dlc-live/dlc-live-gui/user_guide/cameras_backends/opencv_backend
- file: docs/dlc-live/dlc-live-gui/user_guide/cameras_backends/basler_backend
- file: docs/dlc-live/dlc-live-gui/user_guide/cameras_backends/aravis_backend
- file: docs/dlc-live/dlc-live-gui/user_guide/cameras_backends/gentl_backend
- file: docs/dlc-live/dlc-live-gui/user_guide/misc/misc_landing
sections:
- file: docs/dlc-live/dlc-live-gui/user_guide/misc/modelzoo_downloads
- file: docs/dlc-live/dlc-live-gui/user_guide/misc/timestamp_format

- caption: "🦄 DeepLabCut Model Zoo"
chapters:
Expand All @@ -107,4 +121,4 @@ parts:

- caption: Citations for DeepLabCut
chapters:
- file: docs/citation
- file: docs/citation
17 changes: 9 additions & 8 deletions docs/ModelZoo.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# The DeepLabCut Model Zoo!
(file:model-zoo)=
# The DeepLabCut Model Zoo!

![image](https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/8957c690-4f27-4430-8581-4161fd58d052/68747470733a2f2f696d616765732e73717561726573706163652d63646e2e636f6d2f636f6e74656e742f76312f3537663664353163396637343536366635356563663237312f313631363439323337333730302d50474f41433732494f4236415545343756544a582f6b6531375a77644742546f646449.png?format=450w)

Expand All @@ -11,7 +12,7 @@ first proper [SuperAnimal Foundation Models](#about-the-superanimal-models) publ

- (1) a collection of models that are trained on diverse data across (typically) large datasets, which means you do not need to train models yourself, rather you can use them in your research applications.
- (2) a contribution website for community crowd sourcing of expertly labeled keypoints to improve models! You can get involved here: [contrib.deeplabcut.org](https://contrib.deeplabcut.org/).
- (3) a no-install DeepLabCut that you can use on ♾[Google Colab](https://github.com/DeepLabCut/DeepLabCut/blob/main/examples/COLAB/COLAB_DEMO_SuperAnimal.ipynb),
- (3) a no-install DeepLabCut that you can use on ♾[Google Colab](https://github.com/DeepLabCut/DeepLabCut/blob/main/examples/COLAB/COLAB_DEMO_SuperAnimal.ipynb),
test our models in 🕸[the browser](https://contrib.deeplabcut.org/), or on our 🤗[HuggingFace](https://huggingface.co/spaces/DeepLabCut/DeepLabCutModelZoo-SuperAnimals) app!
- (4) new methods to make SuperAnimal Foundation Models that combine data across different labs/datasets, keypoints, animals/species, and use on your data!

Expand All @@ -22,7 +23,7 @@ pip install deeplabcut[gui,modelzoo]

## About the SuperAnimal Models

Animal pose estimation is critical in applications ranging from neuroscience to veterinary medicine. However, reliable inference of animal poses currently requires domain knowledge and labeling effort. To ease access to high-performance animal pose estimation models across diverse environments and species, we present a new paradigm for pre-training and fine-tuning that provides excellent zero-shot (no training required) performance on two major classes of animal pose data: quadrupeds and lab mice.
Animal pose estimation is critical in applications ranging from neuroscience to veterinary medicine. However, reliable inference of animal poses currently requires domain knowledge and labeling effort. To ease access to high-performance animal pose estimation models across diverse environments and species, we present a new paradigm for pre-training and fine-tuning that provides excellent zero-shot (no training required) performance on two major classes of animal pose data: quadrupeds and lab mice.

To provide the community with easy access to such high performance models across diverse environments and species, we present a new paradigm for building pre-trained animal pose models -- which we call SuperAnimal models -- and the ability to use them for transfer learning (e.g., fine-tune them if needed).

Expand All @@ -31,7 +32,7 @@ To provide the community with easy access to such high performance models across



### SuperAnimal-Quadruped:
### SuperAnimal-Quadruped:


- `superanimal_quadruped_x` models aim to work across a large range of quadruped animals, from horses, dogs, sheep, rodents, to elephants. The camera perspective is orthogonal to the animal ("side view"), and most of the data includes the animals face (thus the front and side of the animal). You will note we have several variants that differ in speed vs. performance, so please do test them out on your data to see which is best suited for your application. Also note we have a "video adaptation" feature, which lets you adapt your data to the model in a self-supervised way. No labeling needed!
Expand Down Expand Up @@ -63,7 +64,7 @@ To provide the community with easy access to such high performance models across
- `superanimal_topviewmouse_dlcrnet` is a bottom-up model that predicts all keypoints then groups them into individuals. This can be faster, but more error prone.
- `superanimal_topviewmouse` -> This is the same as `superanimal_topviewmouse_dlcrnet`, this was the old naming and being depreciated.
- For all models, they are automatically downloaded to modelzoo/checkpoints when used.

- Here are example images of what the model is trained on:
![SA-TVM](https://user-images.githubusercontent.com/28102185/209957260-c0db72e0-4fdf-434c-8579-34bc5f27f907.png)

Expand All @@ -76,9 +77,9 @@ To provide the community with easy access to such high performance models across

### Practical example: Using SuperAnimal models for inference without training.

You can simply call the model and run video inference.
You can simply call the model and run video inference.

To note, a good step is typically to use our self-supervised video adaptation method to reduce jitter. In the `deeplabcut.video_inference_superanimal` simply function set the `video_adapt` option to __True__. Be aware, that enabling this option will (minimally) extend the processing time.
To note, a good step is typically to use our self-supervised video adaptation method to reduce jitter. In the `deeplabcut.video_inference_superanimal` simply function set the `video_adapt` option to __True__. Be aware, that enabling this option will (minimally) extend the processing time.

```python
import deeplabcut
Expand Down Expand Up @@ -114,7 +115,7 @@ deeplabcut.video_inference_superanimal([video_path],
```

### Practical example: Using transfer learning with superanimal weights.
In the `deeplabcut.train_network` function, the `superanimal_transfer_learning` option plays a pivotal role. If it's set to __True__, it uses a new decoding layer and allows you to use superanimal weights in any project, no matter the number of keypoints. However, if it's set to __False__, you are doing fine-tuning. So, make sure your dataset has the right number of keypoints.
In the `deeplabcut.train_network` function, the `superanimal_transfer_learning` option plays a pivotal role. If it's set to __True__, it uses a new decoding layer and allows you to use superanimal weights in any project, no matter the number of keypoints. However, if it's set to __False__, you are doing fine-tuning. So, make sure your dataset has the right number of keypoints.

Specifically:
* `superanimal_quadruped_x` uses 39 keypoints
Expand Down
13 changes: 0 additions & 13 deletions docs/deeplabcutlive.md

This file was deleted.

18 changes: 18 additions & 0 deletions docs/dlc-live/deeplabcutlive.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
(deeplabcut-live)=
# Running DeepLabCut models in real-time

We provide two additional packages that allow you to record and stream camera data and run DeepLabCut models in real-time.

You can get an in-depth overview of this work in [Kane et al, 2020 eLife](https://elifesciences.org/articles/61909).

## DLC-Live! Software SDK

You may find more information on the DLC-Live! software SDK on GitHub:

- [DLC-Live! Software](https://github.com/DeepLabCut/DeepLabCut-live)

## DLC-Live! GUI

For the DLC-Live! Camera GUI, we provide documentation in the section below:

- {doc}`DLC-Live! GUI <./dlc-live-gui/index>`
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
75 changes: 75 additions & 0 deletions docs/dlc-live/dlc-live-gui/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# DeepLabCut-live-GUI

A graphical application for **real-time pose estimation with DeepLabCut** using one or more cameras.

This GUI is designed for **scientists and experimenters** who want to preview, run inference, and record synchronized video with pose overlays—without writing code.

## Table of Contents

- {doc}`Installation <./quickstart/install>`
- {doc}`Overview <./user_guide/overview>`
- {doc}`Camera setup and backends <./user_guide/cameras_backends/camera_support>`
- {doc}`Timestamp format and synchronization <./user_guide/misc/timestamp_format>`

```{caution}
Please be aware of the {ref}`sec:dlclivegui-index-limitations`
```

---

## Description

### What this software does

- **Live camera preview** from one or multiple cameras
- **Real-time pose inference** using DeepLabCut Live models
- **Multi-camera support** with tiled display
- **Video recording** (raw or with pose and bounding-box overlays)
- **Session-based data organization** with reproducible naming
- **Optional processor plugins** to extend behavior (e.g. remote control, triggers)

The application is built with **PySide6 (Qt)** and is intended for interactive experimental use rather than offline batch processing.
<!-- TODO mention and link to DLC live standalone -->

### Typical workflow

1. **Install** the application and required camera backends
2. **Configure cameras** (single or multi-camera)
3. **Select a DeepLabCut Live model**
4. **Start preview** and verify frame rate
5. **Run pose inference** on a selected camera
6. **Record video** (optionally with overlays)
- With **organized results** by session and run

Each of these steps is covered in the *{doc}`Quickstart <quickstart/install>`*
and *{doc}`User Guide <user_guide/overview>`* sections of this documentation.

### Who this is for

- Neuroscience and behavior labs
- Experimentalists running real-time tracking
- Anyone who wants a **GUI-first** workflow for DeepLabCut Live

---
(sec:dlclivegui-index-limitations)=
## Current limitations

Before getting started, be aware of the following constraints:

- Pose inference runs on **one selected camera at a time** (even in multi-camera mode)
- Camera synchronization depends on **backend capabilities and hardware**
- OpenCV controls for resolution and FPS are "best effort" and may not work with all cameras.
Expect inconsistencies when setting certain frame rates or resolutions as resolution depends on the device driver.
- DeepLabCut Live models must be **exported and compatible** with the selected backend
- Some SuperAnimal models from {ref}`file:model-zoo` may not work out of the box.<br>This is currently a known issue for:
- SuperHuman model (missing detector)
- **Performance** depends on camera resolution, frame rate, GPU availability, and codec choice
- Expect bottlenecks with heavy models, multiple high-resolution cameras, or CPU-only inference.

---

## Feedback, issues, and contributions

> *This project is under active development. Feedback from real experimental use is highly valued.*
>
> [Please report issues, suggest features, or contribute to the codebase on GitHub !](https://github.com/DeepLabCut/DeepLabCut-live-GUI)
Loading
Loading