diff --git a/README.md b/README.md index 56525620db..28ef4e09a8 100644 --- a/README.md +++ b/README.md @@ -48,8 +48,7 @@ [![Gitter](https://badges.gitter.im/DeepLabCut/community.svg)](https://gitter.im/DeepLabCut/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge) [![Twitter Follow](https://img.shields.io/twitter/follow/DeepLabCut.svg?label=DeepLabCut&style=social)](https://twitter.com/DeepLabCut) [![Generic badge](https://img.shields.io/badge/Contributions-Welcome-brightgreen.svg)](CONTRIBUTING.md) - - +[![CZI's Essential Open Source Software for Science](https://chanzuckerberg.github.io/open-science/badges/CZI-EOSS.svg)](https://czi.co/EOSS) @@ -128,84 +127,9 @@ This is an actively developed package and we welcome community development and i | The DeepLabCut [AI Residency Program](https://www.deeplabcutairesidency.org/) | To come and work with us next summerπŸ‘ | Annually | DLC Team | -## References: - -If you use this code or data we kindly ask that you please [cite Mathis et al, 2018](https://www.nature.com/articles/s41593-018-0209-y) and, if you use the Python package (DeepLabCut2.x) please also cite [Nath, Mathis et al, 2019](https://doi.org/10.1038/s41596-019-0176-0). If you utilize the MobileNetV2s or EfficientNets please cite [Mathis, Biasi et al. 2021](https://openaccess.thecvf.com/content/WACV2021/papers/Mathis_Pretraining_Boosts_Out-of-Domain_Robustness_for_Pose_Estimation_WACV_2021_paper.pdf). If you use versions 2.2beta+ or 2.2rc1+, please cite [Lauer et al. 2022](https://www.nature.com/articles/s41592-022-01443-0). - -DOIs (#ProTip, for helping you find citations for software, check out [CiteAs.org](http://citeas.org/)!): - -- Mathis et al 2018: [10.1038/s41593-018-0209-y](https://doi.org/10.1038/s41593-018-0209-y) -- Nath, Mathis et al 2019: [10.1038/s41596-019-0176-0](https://doi.org/10.1038/s41596-019-0176-0) -- Lauer et al 2022: [10.1038/s41592-022-01443-0](https://doi.org/10.1038/s41592-022-01443-0) - - -Please check out the following references for more details: - - @article{Mathisetal2018, - title = {DeepLabCut: markerless pose estimation of user-defined body parts with deep learning}, - author = {Alexander Mathis and Pranav Mamidanna and Kevin M. Cury and Taiga Abe and Venkatesh N. Murthy and Mackenzie W. Mathis and Matthias Bethge}, - journal = {Nature Neuroscience}, - year = {2018}, - url = {https://www.nature.com/articles/s41593-018-0209-y}} - - @article{NathMathisetal2019, - title = {Using DeepLabCut for 3D markerless pose estimation across species and behaviors}, - author = {Nath*, Tanmay and Mathis*, Alexander and Chen, An Chi and Patel, Amir and Bethge, Matthias and Mathis, Mackenzie W}, - journal = {Nature Protocols}, - year = {2019}, - url = {https://doi.org/10.1038/s41596-019-0176-0}} - - @InProceedings{Mathis_2021_WACV, - author = {Mathis, Alexander and Biasi, Thomas and Schneider, Steffen and Yuksekgonul, Mert and Rogers, Byron and Bethge, Matthias and Mathis, Mackenzie W.}, - title = {Pretraining Boosts Out-of-Domain Robustness for Pose Estimation}, - booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)}, - month = {January}, - year = {2021}, - pages = {1859-1868}} - - @article{Lauer2022MultianimalPE, - title={Multi-animal pose estimation, identification and tracking with DeepLabCut}, - author={Jessy Lauer and Mu Zhou and Shaokai Ye and William Menegas and Steffen Schneider and Tanmay Nath and Mohammed Mostafizur Rahman and Valentina Di Santo and Daniel Soberanes and Guoping Feng and Venkatesh N. Murthy and George Lauder and Catherine Dulac and M. Mathis and Alexander Mathis}, - journal={Nature Methods}, - year={2022}, - volume={19}, - pages={496 - 504}} - - @article{insafutdinov2016eccv, - title = {DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model}, - author = {Eldar Insafutdinov and Leonid Pishchulin and Bjoern Andres and Mykhaylo Andriluka and Bernt Schiele}, - booktitle = {ECCV'16}, - url = {http://arxiv.org/abs/1605.03170}} - -Review & Educational articles: - - @article{Mathis2020DeepLT, - title={Deep learning tools for the measurement of animal behavior in neuroscience}, - author={Mackenzie W. Mathis and Alexander Mathis}, - journal={Current Opinion in Neurobiology}, - year={2020}, - volume={60}, - pages={1-11}} - - @article{Mathis2020Primer, - title={A Primer on Motion Capture with Deep Learning: Principles, Pitfalls, and Perspectives}, - author={Alexander Mathis and Steffen Schneider and Jessy Lauer and Mackenzie W. Mathis}, - journal={Neuron}, - year={2020}, - volume={108}, - pages={44-65}} - -Other open-access pre-prints related to our work on DeepLabCut: - - @article{MathisWarren2018speed, - author = {Mathis, Alexander and Warren, Richard A.}, - title = {On the inference speed and video-compression robustness of DeepLabCut}, - year = {2018}, - doi = {10.1101/457242}, - publisher = {Cold Spring Harbor Laboratory}, - URL = {https://www.biorxiv.org/content/early/2018/10/30/457242}, - eprint = {https://www.biorxiv.org/content/early/2018/10/30/457242.full.pdf}, - journal = {bioRxiv}} +## References \& Citations: + +Please see our [dedicated page](https://deeplabcut.github.io/DeepLabCut/docs/citation.html) on how to **cite DeepLabCut** πŸ™ and our sugestions for your Methods section! ## License: @@ -276,3 +200,7 @@ importing a project into the new data format for DLC 2.0 - August 2018: NVIDIA AI Developer News: [AI Enables Markerless Animal Tracking](https://news.developer.nvidia.com/ai-enables-markerless-animal-tracking/) - July 2018: Ed Yong covered DeepLabCut and interviewed several users for the [Atlantic](https://www.theatlantic.com/science/archive/2018/07/deeplabcut-tracking-animal-movements/564338). - April 2018: first DeepLabCut preprint on [arXiv.org](https://arxiv.org/abs/1804.03142) + + ## Funding + + We are grateful for the follow support over the years! This software project was supported in part by the Essential Open Source Software for Science (EOSS) program at Chan Zuckerberg Initiative (cycles 1, 3, 3-DEI, 4), and jointly with the Kavli Foundation for EOSS Cycle 6! We also thank the Rowland Institute at Harvard for funding from 2017-2020, and EPFL from 2020-present. diff --git a/_toc.yml b/_toc.yml index 0cb5c2cb83..471655454f 100644 --- a/_toc.yml +++ b/_toc.yml @@ -29,7 +29,7 @@ parts: chapters: - file: docs/quick-start/single_animal_quick_guide - file: docs/quick-start/tutorial_maDLC -- caption: Beginner's Guide to DeepLabCut +- caption: πŸš€ Beginner's Guide to DeepLabCut chapters: - file: docs/beginner-guides/beginners-guide - file: docs/beginner-guides/manage-project @@ -42,15 +42,16 @@ parts: - caption: DeepLabCut-Live! chapters: - file: docs/deeplabcutlive -- caption: DeepLabCut Model Zoo +- caption: πŸ¦„ DeepLabCut Model Zoo chapters: - file: docs/ModelZoo - file: docs/recipes/UsingModelZooPupil - file: docs/recipes/MegaDetectorDLCLive -- caption: Cookbook (detailed helper guides) +- caption: πŸ§‘β€πŸ³ Cookbook (detailed helper guides) chapters: - file: docs/tutorial - file: docs/convert_maDLC + - file: docs/recipes/OtherData - file: docs/recipes/io - file: docs/recipes/nn - file: docs/recipes/post @@ -61,11 +62,15 @@ parts: - file: docs/recipes/flip_and_rotate - file: docs/recipes/pose_cfg_file_breakdown - file: docs/recipes/publishing_notebooks_into_the_DLC_main_cookbook -- caption: DeepLabCut Benchmark +- caption: DeepLabCut Benchmarking chapters: - file: docs/benchmark + - file: docs/pytorch/Benchmarking_shuffle_guide - caption: Mission & Contribute chapters: - file: docs/MISSION_AND_VALUES - file: docs/roadmap - file: docs/Governance +- caption: Citations for DeepLabCut + chapters: + - file: docs/citation diff --git a/deeplabcut/pose_estimation_tensorflow/predict_videos.py b/deeplabcut/pose_estimation_tensorflow/predict_videos.py index e868ae515a..45a29b160e 100644 --- a/deeplabcut/pose_estimation_tensorflow/predict_videos.py +++ b/deeplabcut/pose_estimation_tensorflow/predict_videos.py @@ -1461,6 +1461,7 @@ def _convert_detections_to_tracklets( greedy=greedy, pcutoff=inference_cfg.get("pcutoff", 0.1), min_affinity=inference_cfg.get("pafthreshold", 0.05), + min_n_links=inference_cfg["minimalnumberofconnections"] ) if calibrate: trainingsetfolder = auxiliaryfunctions.get_training_set_folder(cfg) @@ -1753,6 +1754,7 @@ def convert_detections2tracklets( min_affinity=inferencecfg.get("pafthreshold", 0.05), window_size=window_size, identity_only=identity_only, + min_n_links=inferencecfg["minimalnumberofconnections"] ) assemblies_filename = dataname.split(".h5")[0] + "_assemblies.pickle" if not os.path.exists(assemblies_filename) or overwrite: diff --git a/docs/UseOverviewGuide.md b/docs/UseOverviewGuide.md index 6d619c7f65..778b5d2c0e 100644 --- a/docs/UseOverviewGuide.md +++ b/docs/UseOverviewGuide.md @@ -4,7 +4,7 @@ Below we will first outline what you need to get started, the different ways you can use DeepLabCut, and then the full workflow. Note, we highly recommend you also read and follow our [Nature Protocols paper](https://www.nature.com/articles/s41596-019-0176-0), which is (still) fully relevant to standard DeepLabCut. ```{Hint} -πŸ’‘πŸ“š If you are new to Python and DeepLabCut, you might consider checking our [beginner guide](https://deeplabcut.github.io/DeepLabCut/docs/beginners-guide.html) once you are ready to jump into using the DeepLabCut App! +πŸ’‘πŸ“š If you are new to Python and DeepLabCut, you might consider checking our [beginner guide](https://deeplabcut.github.io/DeepLabCut/docs/beginner-guides/beginners-guide.html) once you are ready to jump into using the DeepLabCut App! ``` diff --git a/docs/citation.md b/docs/citation.md new file mode 100644 index 0000000000..c427b1e223 --- /dev/null +++ b/docs/citation.md @@ -0,0 +1,138 @@ +# How to Cite DeepLabCut + +Thank you for using DeepLabCut! Here are our recommendations for citing and documenting your use of DeepLabCut in your Methods section: + + +If you use this code or data we kindly ask that you please [cite Mathis et al, 2018](https://www.nature.com/articles/s41593-018-0209-y) +and, if you use the Python package (DeepLabCut2.x+) please also cite [Nath, Mathis et al, 2019](https://doi.org/10.1038/s41596-019-0176-0). +If you utilize the MobileNetV2s or EfficientNets please cite [Mathis, Biasi et al. 2021](https://openaccess.thecvf.com/content/WACV2021/papers/Mathis_Pretraining_Boosts_Out-of-Domain_Robustness_for_Pose_Estimation_WACV_2021_paper.pdf). +If you use multi-animal versions 2.2beta+ or 2.2rc1+, please cite [Lauer et al. 2022](https://www.nature.com/articles/s41592-022-01443-0). +If you use our SuperAnimal models, please cite [Ye et al. 2024](https://www.nature.com/articles/s41467-024-48792-2). + +DOIs (#ProTip, for helping you find citations for software, check out [CiteAs.org](http://citeas.org/)!): + +- Mathis et al 2018: [10.1038/s41593-018-0209-y](https://doi.org/10.1038/s41593-018-0209-y) +- Nath, Mathis et al 2019: [10.1038/s41596-019-0176-0](https://doi.org/10.1038/s41596-019-0176-0) +- Lauer et al 2022: [10.1038/s41592-022-01443-0](https://doi.org/10.1038/s41592-022-01443-0) +- Ye et al 2024: [10.1038/s41467-024-48792-2](https://www.nature.com/articles/s41467-024-48792-2) + +## Formatted citations: + + @article{Mathisetal2018, + title = {DeepLabCut: markerless pose estimation of user-defined body parts with deep learning}, + author = {Alexander Mathis and Pranav Mamidanna and Kevin M. Cury and Taiga Abe and Venkatesh N. Murthy and Mackenzie W. Mathis and Matthias Bethge}, + journal = {Nature Neuroscience}, + year = {2018}, + url = {https://www.nature.com/articles/s41593-018-0209-y}} + + @article{NathMathisetal2019, + title = {Using DeepLabCut for 3D markerless pose estimation across species and behaviors}, + author = {Nath*, Tanmay and Mathis*, Alexander and Chen, An Chi and Patel, Amir and Bethge, Matthias and Mathis, Mackenzie W}, + journal = {Nature Protocols}, + year = {2019}, + url = {https://doi.org/10.1038/s41596-019-0176-0}} + + @InProceedings{Mathis_2021_WACV, + author = {Mathis, Alexander and Biasi, Thomas and Schneider, Steffen and Yuksekgonul, Mert and Rogers, Byron and Bethge, Matthias and Mathis, Mackenzie W.}, + title = {Pretraining Boosts Out-of-Domain Robustness for Pose Estimation}, + booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)}, + month = {January}, + year = {2021}, + pages = {1859-1868}} + + @article{Lauer2022MultianimalPE, + title={Multi-animal pose estimation, identification and tracking with DeepLabCut}, + author={Jessy Lauer and Mu Zhou and Shaokai Ye and William Menegas and Steffen Schneider and Tanmay Nath and Mohammed Mostafizur Rahman and Valentina Di Santo and Daniel Soberanes and Guoping Feng and Venkatesh N. Murthy and George Lauder and Catherine Dulac and M. Mathis and Alexander Mathis}, + journal={Nature Methods}, + year={2022}, + volume={19}, + pages={496 - 504}} + + @article{Ye2024SuperAnimal, + title={SuperAnimal pretrained pose estimation models for behavioral analysis}, + author={Shaokai Ye and Anastasiia Filippova and Jessy Lauer and Steffen Schneider and Maxime Vidal and and Tian Qiu and Alexander Mathis and Mackenzie W. Mathis}, + journal={Nature Communications}, + year={2024}, + volume={15}} + + +### Review & Educational articles: + + @article{Mathis2020DeepLT, + title={Deep learning tools for the measurement of animal behavior in neuroscience}, + author={Mackenzie W. Mathis and Alexander Mathis}, + journal={Current Opinion in Neurobiology}, + year={2020}, + volume={60}, + pages={1-11}} + + @article{Mathis2020Primer, + title={A Primer on Motion Capture with Deep Learning: Principles, Pitfalls, and Perspectives}, + author={Alexander Mathis and Steffen Schneider and Jessy Lauer and Mackenzie W. Mathis}, + journal={Neuron}, + year={2020}, + volume={108}, + pages={44-65}} + +### Other open-access pre-prints related to our work on DeepLabCut: + + @article{MathisWarren2018speed, + author = {Mathis, Alexander and Warren, Richard A.}, + title = {On the inference speed and video-compression robustness of DeepLabCut}, + year = {2018}, + doi = {10.1101/457242}, + publisher = {Cold Spring Harbor Laboratory}, + URL = {https://www.biorxiv.org/content/early/2018/10/30/457242}, + eprint = {https://www.biorxiv.org/content/early/2018/10/30/457242.full.pdf}, + journal = {bioRxiv}} + + + +## Methods Suggestion: + +For body part tracking we used DeepLabCut (version 2.X.X)* [Mathis et al, 2018, Nath et al, 2019, Lauer et al. 2022]. Specifically, we labeled X number of frames taken from X videos/animals (then X% was used for training (default is 95%). We used a X-based neural network (i.e. X = ResNet-50, ResNet-101, MobileNetV2-0.35, MobileNetV2-0.5, MobileNetV2-0.75, MobileNetV2-1***) with default parameters* for X number of training iterations. We validated with X number of shuffles, and found the test error was: X pixels, train: X pixels (image size was X by X). We then used a p-cutoff of X (i.e. 0.9) to condition the X,Y coordinates for future analysis. This network was then used to analyze videos from similar experimental settings. + +> Mathis, A. et al. Deeplabcut: markerless pose estimation +> of user-defined body parts with deep learning. Nature +> Neuroscience 21, 1281–1289 (2018). + +> Nath, T. et al. Using deeplabcut for 3d markerless pose +> estimation across species and behaviors. Nature Protocols +> 14, 2152–2176 (2019). + +*If any defaults were changed in *`pose_config.yaml`*, mention them here. + +i.e. common things one might change: +* the loader (options are `default`, `imgaug`, `tensorpack`, `deterministic`). +* the `post_dist_threshold` (default is 17 and determines training resolution). +* optimizer: do you use the default `SGD` or `ADAM`? + +*** here, you could add additional citations. +If you use ResNets, consider citing Insafutdinov et al 2016 & He et al 2016. If you use the MobileNetV2s consider citing Mathis et al 2019, and Sandler et al, 2018. + + +> Mathis, A. et al. Pretraining boosts out-of-domain robustness for pose estimation +> arXiv 1909.11229 (2019) + +> Insafutdinov, E., Pishchulin, L., Andres, B., Andriluka, +> M. & Schiele, B. DeeperCut: A deeper, stronger, and +> faster multi-person pose estimation model. In European +> Conference on Computer Vision, 34–50 (Springer, 2016). + +> Sandler, M., Howard, A., Zhu, M., Zhmoginov, A. & +> Chen, L.-C. Mobilenetv2: Inverted residuals and linear +> bottlenecks. In Proceedings of the IEEE Conference +> on Computer Vision and Pattern Recognition, 4510–4520 +> (2018). + +> He, K., Zhang, X., Ren, S. & Sun, J. Deep residual +> learning for image recognition. In Proceedings of the +> IEEE conference on computer vision and pattern recognition, +> 770–778 (2016). URL https://arxiv.org/abs/ +> 1512.03385. + +## Graphics + +We also have the network graphic freely available on SciDraw.io if you'd like to use it! https://scidraw.io/drawing/290 + +You are welcome to use our logo in your works as well. diff --git a/docs/pytorch/Benchmarking_shuffle_guide.md b/docs/pytorch/Benchmarking_shuffle_guide.md new file mode 100644 index 0000000000..61c42107d1 --- /dev/null +++ b/docs/pytorch/Benchmarking_shuffle_guide.md @@ -0,0 +1,135 @@ +# DeepLabCut Benchmarking - User Guide + +## Reasoning for benchmarking models in DLC (across DLC versions and architectures) + +DeepLabCut 3.0+ introduced using PyTorch πŸ”₯ as a deep learning engine (and TensorFlow will be depreciated). +It is of importance for replicability of data analysis to benchmark existing models created using DeepLabCut versions +prior to 3.0 against new models created in DeepLabCut 3.0+ and later versions. + +When comparing different models, it's important to use the same train-test data +split to ensure fair comparisons. If the models are trained on different datasets, +their performance metrics can't be accurately compared. This is crucial when +comparing the performance of models with different architectures or different +sets of hyperparameters. For example, if we compare the RMSE of a model on an +"easy" test image with the RMSE of another model on a "hard" test image, it +doesn't determine whether a model is better than the other because the +architecture performs better or because the training images were "better" to +learn from. Thus, we not only need to compare the models based on metrics +computed on the same test images, but also train them on an identical fixed +training set in order to "decouple" the dataset from the model architecture. + +Creating a model using the same data split can be carried out using a GUI or +using code, and this guide outlines the steps for both. + +## Important files & folders + +``` +dlc-project +| +|___dlc-models-pytorch +| |__ iterationX +| |__ shuffleX +| |__ pytorch_config.yaml +| +|___training-datasets +| |__ metadata.yaml +| +|___config.yaml +``` + +## Benchmarking a TensorFlow model against a PyTorch model + +### Creating a shuffle + +Creating a new shuffle with the same train/test split as an existing one: +#### In the DeepLabCut GUI +1. Front page > Load project > Open project folder > choose *config.yaml* +2. Select *'Create training dataset'* tab +3. Tick *Use an existing data split* option + + ![create_from_existing]() +4. Click 'View existing shuffles': + - This is used to view the indices of shuffles created for a project to determine which index is available to assign to a new shuffle. + - The elements described in this window are: + - train_fraction: The fraction of the dataset used for training. + - index: The index of the shuffle. + - split: The data split for the shuffle. The integer value on its own does not +hold any meaning, but this "split" value indicates which shuffles have the same split +(as their results can then be compared) + - engine: Whether it is a PyTorch or TensorFlow shuffle + + ![view_existing_sh]() +5. Choose the index of the training shuffle to replicate. Let us assume we want +to replicate the train-test split from OpenfieldOct30-trainset95shuffle3, in which +`split: 3`. In this case, we insert in the *'From shuffle'* menu + + ![choose_existing_index]() +6. To create this new dataset, set the shuffle option to an un-used shuffle +(here 4) + + ![choose_new_index]() +7. Click *'Create training dataset'* and move on to *'train network'*. Shuffle should be +set to the new shuffle entered at the previous step (in this case, 4) + + ![create_from_existing]() +8. To view/edit the specifications of the model you created, you can go to `pytoch_config.yaml` file at: + ``` + dlc-project + | + |___ dlc-models-pytorch + |__ iterationX + |__ shuffleX + |__ pytorch_config.yaml + ``` + +#### In Code + +With the `deeplabcut` module in Python, use the +`create_training_dataset_from_existing_split()` method to create new shuffles from +existing ones (e.g. TensorFlow shuffles). + +Similarly, here, we create a new shuffle '4' from the existing shuffle '3'. + +```python +import deeplabcut +from deeplabcut.core.engine import Engine + +config = "path/to/project/config.yaml" + +training_dataset = deeplabcut.create_training_dataset_from_existing_split( + config=config, + from_shuffle=3, + from_trainsetindex=0, + shuffles=[4], + net_type="resnet_50", +) +``` + +We can then train our new PyTorch model with the same data split as the +TensorFlow model. + +```python +deeplabcut.train_network(config, shuffle=4, engine=Engine.PYTORCH, batch_size=8) +``` + +Once trained we can evaluate our model using + +```python +deeplabcut.evaluate_network(config, Shuffles=[4], snapshotindex="all") +``` +Now, we can compare performances with peace of mind! + +#### Good practices: naming shuffles created from existing ones + +In a setting where one has multiple TensorFlow models and intends to benchmark +their performances against new PyTorch models, it is good practice to follow +a naming pattern for the shuffles we create. + +Say we have TensorFlow shuffles 0, 1, and 2. We can create new PyTorch shuffles +from them by naming them 1000, 1001, and 1002. This allows us to quickly +recognize that the shuffles belonging to the 100x range are PyTorch shuffles +and that shuffle 1001, for example, has the same data split as TensorFlow +shuffle 1. This way, the comparison can be more straightforward and guaranteed +to be correct! + +This was contributed by the [2024 DLC AI Residents](https://www.deeplabcutairesidency.org/our-team)! diff --git a/docs/pytorch/assets/img1.png b/docs/pytorch/assets/img1.png new file mode 100644 index 0000000000..dde3d96115 Binary files /dev/null and b/docs/pytorch/assets/img1.png differ diff --git a/docs/pytorch/assets/img2.png b/docs/pytorch/assets/img2.png new file mode 100644 index 0000000000..6e86649dd8 Binary files /dev/null and b/docs/pytorch/assets/img2.png differ diff --git a/docs/pytorch/assets/img3.png b/docs/pytorch/assets/img3.png new file mode 100644 index 0000000000..39c516bfb7 Binary files /dev/null and b/docs/pytorch/assets/img3.png differ diff --git a/docs/pytorch/assets/img4.png b/docs/pytorch/assets/img4.png new file mode 100644 index 0000000000..a231130b5f Binary files /dev/null and b/docs/pytorch/assets/img4.png differ diff --git a/docs/pytorch/assets/img5.png b/docs/pytorch/assets/img5.png new file mode 100644 index 0000000000..4e7a44ca0d Binary files /dev/null and b/docs/pytorch/assets/img5.png differ diff --git a/docs/recipes/OtherData.md b/docs/recipes/OtherData.md new file mode 100644 index 0000000000..1d9e648d70 --- /dev/null +++ b/docs/recipes/OtherData.md @@ -0,0 +1,33 @@ +# How to use data labeled outside of DeepLabCut +- and/or if you merge projects across scorers (see below): + + + +## Using data labeled elsewhere: + +Some users may have annotation data in different formats, yet want to use the DLC pipeline. In this case, you need to convert the data to our format. Simply, you can format your data in an excel sheet (.csv file) or pandas array (.h5 file). + +Here is a guide to do this via the ".csv" route: (the pandas array route is identical, just format the pandas array in the same way). + +**Step 1**: create a project as describe in the user guide: https://github.com/AlexEMG/DeepLabCut/blob/master/docs/UseOverviewGuide.md#create-a-new-project + +**Step 2**: edit the ``config.yaml`` file to include the body part names, please take care that spelling, spacing, and capitalization are IDENTICAL to the "labeled data body part names". + +**Step 3**: Please inspect the excel formatted sheet (.csv) from our [demo project](https://github.com/AlexEMG/DeepLabCut/tree/master/examples/Reaching-Mackenzie-2018-08-30/labeled-data/reachingvideo1) +- i.e. this file: https://github.com/AlexEMG/DeepLabCut/blob/master/examples/Reaching-Mackenzie-2018-08-30/labeled-data/reachingvideo1/CollectedData_Mackenzie.csv + +**Step 4**: Edit the .csv file such that it contains the X, Y pixel coordinates, the body part names, the scorer name as well as the relative path to the image: e.g. /labeled-data/somefolder/img017.jpg +Then make sure the scorer name, and body parts are the same in the config.yaml file. + +Also add for each folder a video to the `video_set` in the config.yaml file. This can also be a dummy variable, but should be e.g. +C://somefolder.avi if the folder is called somefolder. See demo config.yaml file for proper formatting. + +**Step 5**: When you are done, run ``deeplabcut.convertcsv2h5('path_to_config.yaml', scorer= 'experimenter')`` + + - The scorer name must be identical to the input name for experimenter that you used when you created the project. This will automatically update "Mackenzie" to your name in the example demo notebook. + +## If you merge projects: + +**Step 1**: rename the CSV files to be the target name. + +**Step 2**: run and pass the target name ``deeplabcut.convertcsv2h5('path_to_config.yaml', scorer= 'experimenter')``. This will overwrite the H5 file so the data is all merged under the target name. diff --git a/docs/standardDeepLabCut_UserGuide.md b/docs/standardDeepLabCut_UserGuide.md index 884b38a4f8..39188b3d48 100644 --- a/docs/standardDeepLabCut_UserGuide.md +++ b/docs/standardDeepLabCut_UserGuide.md @@ -401,7 +401,7 @@ dynamic: triple containing (state, detectiontreshold, margin) If the state is true, then dynamic cropping will be performed. That means that if an object is detected (i.e., any body part > detectiontreshold), then object boundaries are computed according to the smallest/largest x position and smallest/largest y position of all body parts. This window is expanded by the margin and from then on only the posture within this crop is analyzed (until the object is lost; i.e.,