Skip to content

Effnet docs release - #1074

Merged
AlexEMG merged 13 commits into
masterfrom
effnet-docs-release
Jan 22, 2021
Merged

Effnet docs release#1074
AlexEMG merged 13 commits into
masterfrom
effnet-docs-release

Conversation

@AlexEMG

@AlexEMG AlexEMG commented Jan 6, 2021

Copy link
Copy Markdown
Member

This is wip to update the docs & release 2.1.10.

This is also benchmarked. My results were (Run with DLC 2.1.9 in Jan 2021) for 100 k iterations.

**ResNet50
Imgaug:
Results for 100000 training iterations: 95 1 train error: 2.13 pixels. Test error: 2.22 pixels.
With pcutoff of 0.4 train error: 2.13 pixels. Test error: 2.22 pixels

Scalecrop:
Results for 100000 training iterations: 95 2 train error: 1.47 pixels. Test error: 1.77 pixels.
With pcutoff of 0.4 train error: 1.47 pixels. Test error: 1.77 pixels

Tensorpack:
Results for 100000 training iterations: 95 3 train error: 2.09 pixels. Test error: 2.36 pixels.
With pcutoff of 0.4 train error: 2.09 pixels. Test error: 2.36 pixels

**EffNet-b3
Imgaug:
Results for 100000 training iterations: 95 4 train error: 2.39 pixels. Test error: 2.57 pixels.
With pcutoff of 0.4 train error: 2.39 pixels. Test error: 2.57 pixels

Scalecrop:
Results for 100000 training iterations: 95 5 train error: 2.26 pixels. Test error: 2.24 pixels.
With pcutoff of 0.4 train error: 2.26 pixels. Test error: 2.24 pixels

Tensorpack:
Results for 100000 training iterations: 95 6 train error: 1.65 pixels. Test error: 2.24 pixels.
With pcutoff of 0.4 train error: 1.65 pixels. Test error: 2.24 pixels

****NOTE: for the EfficientNet we have not optimized the learning rate schedule for batch size 1 and large images. Here we showed rigorously for different datasets and models that EffNets outperform ResNets and MobNets as expected from the imagenet performance: https://openaccess.thecvf.com/content/WACV2021/html/Mathis_Pretraining_Boosts_Out-of-Domain_Robustness_for_Pose_Estimation_WACV_2021_paper.html

(thereby we cross-validated the learning rate schedule for each model, you might want to do this as well on your data!).****

@AlexEMG
AlexEMG requested a review from MMathisLab January 6, 2021 22:51
@AlexEMG AlexEMG added the enhancement New feature or request label Jan 6, 2021
@AlexEMG
AlexEMG requested a review from jeylau January 6, 2021 22:51
@jeylau

jeylau commented Jan 7, 2021

Copy link
Copy Markdown
Contributor

@AlexEMG, why rolling back predict_.py?

AlexEMG and others added 2 commits January 22, 2021 20:56
* updated maDLC Colab Notebook (#1073)

* Created using Colaboratory

* Created using Colaboratory

* Created using Colaboratory

* Some fixes (#1080)

* Fix batchwise video analysis

* Fix grayscale-to-RGB FutureWarning

* Minor 3D test script fix

Co-authored-by: jeylau <30733203+jeylau@users.noreply.github.com>

Co-authored-by: Mackenzie Mathis <mathis@rowland.harvard.edu>
Co-authored-by: jeylau <30733203+jeylau@users.noreply.github.com>
@AlexEMG

AlexEMG commented Jan 22, 2021

Copy link
Copy Markdown
Member Author

Jan 2020:
MobileNetV2 0.35
Results for 10000 training iterations: 95 1 train error: 5.79 pixels. Test error: 5.63 pixels.
With pcutoff of 0.4 train error: 5.79 pixels. Test error: 5.63 pixels

ResNet 50
Results for 10000 training iterations: 95 2 train error: 3.61 pixels. Test error: 3.7 pixels.
With pcutoff of 0.4 train error: 3.61 pixels. Test error: 3.7 pixels

EffNet-b3
Results for 10000 training iterations: 95 3 train error: 6.86 pixels. Test error: 6.63 pixels.
With pcutoff of 0.4 train error: 6.86 pixels. Test error: 6.63 pixels

@AlexEMG
AlexEMG merged commit 4ae9541 into master Jan 22, 2021
@MMathisLab

Copy link
Copy Markdown
Member

@AlexEMG it should be 2.1.10, not 2.1.9.10^

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants