Skip to content

IndexError: list index out of range when using analyze_videos #2449

Closed
Closed
IndexError: list index out of range when using analyze_videos#2449
Assignees
n-poulsen
Labels
bugSomething isn't workingmaDLC

Description

@trexbone
trexbone
opened on Nov 17, 2023

Is there an existing issue for this?

  • I have searched the existing issues

Bug description

Hi!
This is my first time using python and DeepLabCut, so any suggestions or comments are greatly appreciated! When I was trying to analyze the video file, it was running fine until it failed halfway. Then I tried the suggestion in #1527 but the same error occured again. Due to the capacity of the gpu, I am running on batchsize=1.

Operating System

Ubuntu 20.04.6 LTS

DeepLabCut version

2.3.8

DeepLabCut mode

multi animal

Device type

gpu

Steps To Reproduce

  1. Enter conda environment by conda activate DEEPLABCUTAUD
  2. Switch to ipython
  3. import deeplabcut
  4. Analyze videos
  5. Error appears after analyzing two videos

Relevant log output

In [28]: deeplabcut.analyze_videos('/home/fishlab4/Fish_audrey-Audrey-2023-10-31/config.yaml',['/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos'],videotype='.MP4',auto_track=True,allow_growth=True)
Using snapshot-500000 for model /home/fishlab4/Fish_audrey-Audrey-2023-10-31/dlc-models/iteration-0/Fish_audreyOct31-trainset95shuffle1
/home/fishlab4/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/tensorflow/python/keras/engine/base_layer_v1.py:1694: UserWarning: `layer.apply` is deprecated and will be removed in a future version. Please use `layer.__call__` method instead.
  warnings.warn('`layer.apply` is deprecated and '
Activating extracting of PAFs
2023-11-15 11:33:07.319245: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1371 MB memory:  -> device: 0, name: Quadro P620, pci bus id: 0000:18:00.0, compute capability: 6.1
Analyzing all the videos in the directory...
Starting to analyze %  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Non-Resident1.MP4
Loading  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Non-Resident1.MP4
Duration of video [s]:  712.71 , recorded with  59.94 fps!
Overall # of frames:  42720  found with (before cropping) frame dimensions:  2704 1520
Starting to extract posture from the video(s) with batchsize: 1
                                                                                                                                                                                                          2023-11-15 11:33:09.072472: W tensorflow/core/common_runtime/bfc_allocator.cc:290] Allocator (GPU_0_bfc) ran out of memory trying to allocate 592.19MiB with freed_by_count=0. The caller indicates that this is not a failure, but this may mean that there could be performance gains if more memory were available.
  0%|                                                                                                                                                                          | 0/42720 [4:56:08<?, ?it/s]
 34%|████████████████████████████████████████████████████▉                                                                                                       | 14493/42720 [4:50:29<9:24:09,  1.20s/it 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 42720/42720 [14:15:16<00:00,  1.20s/it]
Video Analyzed. Saving results in /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos...██████████████████████████████████████████████████████████████████████████| 42720/42720 [14:15:16<00:00,  1.20s/it]
Using snapshot-500000 for model /home/fishlab4/Fish_audrey-Audrey-2023-10-31/dlc-models/iteration-0/Fish_audreyOct31-trainset95shuffle1
Processing...  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Non-Resident1.MP4
Analyzing /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Non-Resident1DLC_dlcrnetms5_Fish_audreyOct31shuffle1_500000.h5
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 42720/42720 [00:22<00:00, 1871.29it/s]
42720it [00:51, 826.14it/s] 
The tracklets were created (i.e., under the hood deeplabcut.convert_detections2tracklets was run). Now you can 'refine_tracklets' in the GUI, or run 'deeplabcut.stitch_tracklets'.
Processing...  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Non-Resident1.MP4
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2595/2595 [00:22<00:00, 115.13it/s]
Starting to analyze %  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident1.MP4
Loading  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident1.MP4
Duration of video [s]:  711.71 , recorded with  59.94 fps!
Overall # of frames:  42660  found with (before cropping) frame dimensions:  2704 1520
Starting to extract posture from the video(s) with batchsize: 1
 63%|█████████████████████████████████████████████████████████████████████████████████████████████████▊                                                          | 26746/42660 [8:55:39<5:17:41,  1.20s/it]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 42660/42660 [14:16:36<00:00,  1.20s/it]
Video Analyzed. Saving results in /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos...
Using snapshot-500000 for model /home/fishlab4/Fish_audrey-Audrey-2023-10-31/dlc-models/iteration-0/Fish_audreyOct31-trainset95shuffle1
Processing...  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident1.MP4
Analyzing /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident1DLC_dlcrnetms5_Fish_audreyOct31shuffle1_500000.h5
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 42660/42660 [00:20<00:00, 2042.56it/s]
42660it [00:41, 1019.74it/s]
The tracklets were created (i.e., under the hood deeplabcut.convert_detections2tracklets was run). Now you can 'refine_tracklets' in the GUI, or run 'deeplabcut.stitch_tracklets'.
Processing...  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident1.MP4
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1490/1490 [00:14<00:00, 100.95it/s]
Starting to analyze %  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Non-Resident1.MP4
Loading  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Non-Resident1.MP4
Duration of video [s]:  712.71 , recorded with  59.94 fps!
Overall # of frames:  42720  found with (before cropping) frame dimensions:  2704 1520
Starting to extract posture from the video(s) with batchsize: 1
  4%|██████▍                                                                                                                                                       | 1728/42720 [34:31<13:42:12,  1.20s/it]---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[28], line 1
----> 1 deeplabcut.analyze_videos('/home/fishlab4/Fish_audrey-Audrey-2023-10-31/config.yaml',['/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos'],videotype='.MP4',auto_track=True,allow_growth=True)

File ~/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/predict_videos.py:628, in analyze_videos(config, videos, videotype, shuffle, trainingsetindex, gputouse, save_as_csv, in_random_order, destfolder, batchsize, cropping, TFGPUinference, dynamic, modelprefix, robust_nframes, allow_growth, use_shelve, auto_track, n_tracks, calibrate, identity_only, use_openvino)
    623 from deeplabcut.pose_estimation_tensorflow.predict_multianimal import (
    624     AnalyzeMultiAnimalVideo,
    625 )
    627 for video in Videos:
--> 628     AnalyzeMultiAnimalVideo(
    629         video,
    630         DLCscorer,
    631         trainFraction,
    632         cfg,
    633         dlc_cfg,
    634         sess,
    635         inputs,
    636         outputs,
    637         destfolder,
    638         robust_nframes=robust_nframes,
    639         use_shelve=use_shelve,
    640     )
    641     if auto_track:  # tracker type is taken from default in cfg
    642         convert_detections2tracklets(
    643             config,
    644             [video],
   (...)
    651             identity_only=identity_only,
    652         )

File ~/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/predict_multianimal.py:184, in AnalyzeMultiAnimalVideo(video, DLCscorer, trainFraction, cfg, dlc_cfg, sess, inputs, outputs, destfolder, robust_nframes, use_shelve)
    172     PredicteData, nframes = GetPoseandCostsF(
    173         cfg,
    174         dlc_cfg,
   (...)
    181         shelf_path,
    182     )
    183 else:
--> 184     PredicteData, nframes = GetPoseandCostsS(
    185         cfg,
    186         dlc_cfg,
    187         sess,
    188         inputs,
    189         outputs,
    190         vid,
    191         nframes,
    192         shelf_path,
    193     )
    195 stop = time.time()
    197 if cfg["cropping"] == True:

File ~/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/predict_multianimal.py:511, in GetPoseandCostsS(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, shelf_path)
    503         frame = rgba2rgb(frame)
    504     dets = predict.predict_batched_peaks_and_costs(
    505         dlc_cfg,
    506         np.expand_dims(frame, axis=0),
   (...)
    509         outputs,
    510     )
--> 511     db[key] = dets[0]
    512     del dets
    513 elif counter >= nframes:

IndexError: list index out of range

In[30] deeplabcut.analyze_videos('/home/fishlab4/Fish_audrey-Audrey-2023-10-31/config.yaml',['/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Non-Resident1.MP4','/home/fishlab4/Fish_audr
    ...: ey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Predator1.MP4','/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Resident1.MP4','/home/fishlab4/Fish_audrey-Audrey-2023-10-
    ...: 31/videos/LD1_(Tank51)_Non-Resident2.MP4'],videotype='.MP4',auto_track=True)
Using snapshot-500000 for model /home/fishlab4/Fish_audrey-Audrey-2023-10-31/dlc-models/iteration-0/Fish_audreyOct31-trainset95shuffle1
/home/fishlab4/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/tensorflow/python/keras/engine/base_layer_v1.py:1694: UserWarning: `layer.apply` is deprecated and will be removed in a future version. Please use `layer.__call__` method instead.
  warnings.warn('`layer.apply` is deprecated and '
Activating extracting of PAFs
2023-11-17 12:16:38.828896: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 1371 MB memory:  -> device: 0, name: Quadro P620, pci bus id: 0000:18:00.0, compute capability: 6.1
Starting to analyze %  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Non-Resident1.MP4
Loading  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Non-Resident1.MP4
Duration of video [s]:  712.71 , recorded with  59.94 fps!
Overall # of frames:  42720  found with (before cropping) frame dimensions:  2704 1520
Starting to extract posture from the video(s) with batchsize: 1
                                                                                                                                                                                                          ---------------------------------------------------------------------------                                                                                         | 1728/42720 [34:33<13:36:07,  1.19s/it]
IndexError                                Traceback (most recent call last)
Cell In[30], line 1
----> 1 deeplabcut.analyze_videos('/home/fishlab4/Fish_audrey-Audrey-2023-10-31/config.yaml',['/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Non-Resident1.MP4','/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Predator1.MP4','/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Resident1.MP4','/home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident2.MP4'],videotype='.MP4',auto_track=True)

File ~/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/predict_videos.py:628, in analyze_videos(config, videos, videotype, shuffle, trainingsetindex, gputouse, save_as_csv, in_random_order, destfolder, batchsize, cropping, TFGPUinference, dynamic, modelprefix, robust_nframes, allow_growth, use_shelve, auto_track, n_tracks, calibrate, identity_only, use_openvino)
    623 from deeplabcut.pose_estimation_tensorflow.predict_multianimal import (
    624     AnalyzeMultiAnimalVideo,
    625 )
    627 for video in Videos:
--> 628     AnalyzeMultiAnimalVideo(
    629         video,
    630         DLCscorer,
    631         trainFraction,
    632         cfg,
    633         dlc_cfg,
    634         sess,
    635         inputs,
    636         outputs,
    637         destfolder,
    638         robust_nframes=robust_nframes,
    639         use_shelve=use_shelve,
    640     )
    641     if auto_track:  # tracker type is taken from default in cfg
    642         convert_detections2tracklets(
    643             config,
    644             [video],
   (...)
    651             identity_only=identity_only,
    652         )

File ~/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/predict_multianimal.py:184, in AnalyzeMultiAnimalVideo(video, DLCscorer, trainFraction, cfg, dlc_cfg, sess, inputs, outputs, destfolder, robust_nframes, use_shelve)
    172     PredicteData, nframes = GetPoseandCostsF(
    173         cfg,
    174         dlc_cfg,
   (...)
    181         shelf_path,
    182     )
    183 else:
--> 184     PredicteData, nframes = GetPoseandCostsS(
    185         cfg,
    186         dlc_cfg,
    187         sess,
    188         inputs,
    189         outputs,
    190         vid,
    191         nframes,
    192         shelf_path,
    193     )
    195 stop = time.time()
    197 if cfg["cropping"] == True:

File ~/anaconda3/envs/DEEPLABCUTAUD/lib/python3.9/site-packages/deeplabcut/pose_estimation_tensorflow/predict_multianimal.py:511, in GetPoseandCostsS(cfg, dlc_cfg, sess, inputs, outputs, cap, nframes, shelf_path)
    503         frame = rgba2rgb(frame)
    504     dets = predict.predict_batched_peaks_and_costs(
    505         dlc_cfg,
    506         np.expand_dims(frame, axis=0),
   (...)
    509         outputs,
    510     )
--> 511     db[key] = dets[0]
    512     del dets
    513 elif counter >= nframes:

IndexError: list index out of range

Anything else?

Below is my config file

  # Project definitions (do not edit)
Task: Fish_audrey
scorer: Audrey
date: Oct31
multianimalproject: true
identity: true

    # Project path (change when moving around)
project_path: /home/fishlab4/Fish_audrey-Audrey-2023-10-31

    # Annotation data set configuration (and individual video cropping parameters)
video_sets:
  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Non-Resident1.MP4:
    crop: 0, 2704, 0, 1520
  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Predator1.MP4:
    crop: 0, 2704, 0, 1520
  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/L.bicolor1_(Tank23)_Resident1.MP4:
    crop: 0, 2704, 0, 1520
  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident1.MP4:
    crop: 0, 2704, 0, 1520
  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Non-Resident2.MP4:
    crop: 0, 2704, 0, 1520
  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Predator1.MP4:
    crop: 0, 2704, 0, 1520
  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LD1_(Tank51)_Resident1.MP4:
    crop: 0, 2704, 0, 1520
  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Non-Resident1.MP4:
    crop: 0, 2704, 0, 1520
  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Predator1.MP4:
    crop: 0, 2704, 0, 1520
  /home/fishlab4/Fish_audrey-Audrey-2023-10-31/videos/LQ1_Tank51_Resident1.MP4:
    crop: 0, 2704, 0, 1520
individuals:
- Cleaner
- ClientPredator
- ClientResident
- ClientNonResident
uniquebodyparts: []
multianimalbodyparts:
- HeadTerminalMouth
- HeadBasisUp
- HeadBasisDown
- TailBottomCorner
- TailUpperCorner
- TailJunctionBody
- BodyMidUp
- BodyMidDown
bodyparts: MULTI!

    # Fraction of video to start/stop when extracting frames for labeling/refinement
start: 0
stop: 1
numframes2pick: 20

    # Plotting configuration
skeleton:
 - bodypart1
 - bodypart2
 - bodypart2
 - bodypart3
 - bodypart1
 - bodypart3
skeleton_color: black
pcutoff: 0.6
dotsize: 12
alphavalue: 0.7
colormap: rainbow

    # Training,Evaluation and Analysis configuration
TrainingFraction:
- 0.95
iteration: 0
default_net_type: dlcrnet_ms5
default_augmenter: multi-animal-imgaug
default_track_method: ellipse
snapshotindex: -1
batch_size: 1

    # Cropping Parameters (for analysis and outlier frame detection)
cropping: false
    #if cropping is true for analysis, then set the values here:
x1: 0
x2: 640
y1: 277
y2: 624

    # Refinement configuration (parameters from annotation dataset configuration also relevant in this stage)
corner2move2:
- 50
- 50
move2corner: true

Code of Conduct

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingmaDLC

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions