PaddleVideo训练stgcn

Question

使用PaddleVideo训练stgcn模型问题：
【paddle版本为develop版本】
报错信息如下：
[08/31 21:47:28] model_name : STGCN
W0831 21:47:29.443328 4185452 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 8.6, Driver API Version: 11.6, Runtime API Version: 11.6
W0831 21:47:29.455291 4185452 gpu_resources.cc:149] device: 0, cuDNN Version: 8.6.
[08/31 21:47:29] Loading data, it will take some moment...
[08/31 21:47:29] Data Loaded!
[08/31 21:47:29] Loading data, it will take some moment...
[08/31 21:47:30] Data Loaded!
[08/31 21:47:30] Training in fp32 mode.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 10. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 10. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 10. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 10. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 10. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 10. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 10. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 10. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 10. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 10. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 11. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 11. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 11. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 11. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 11. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 11. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 11. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 11. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 11. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 11. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 21. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 23. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 23. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 23. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 23. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 23. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 23. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 23. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 23. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 23. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 23. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 22. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 26. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 13. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Error: ../paddle/phi/kernels/gpu/cross_entropy_kernel.cu:1010 Assertion false failed. The value of label expected >= 0 and < 10, or == -100, but got 19. Please check label value.
Traceback (most recent call last):
File "main.py", line 143, in
main()
File "main.py", line 139, in main
profiler_options=args.profiler_options)
File "PaddleVideo/paddlevideo/tasks/train.py", line 237, in train_model
outputs = model(data, mode='train')
File "/home/dp504/miniconda3/envs/paddle/lib/python3.7/site-packages/paddle/nn/layer/layers.py", line 1240, in call
return self.forward(*inputs, **kwargs)
File "PaddleVideo/paddlevideo/modeling/framework/recognizers/base.py", line 49, in forward
return self.train_step(data_batch)
File "PaddleVideo/paddlevideo/modeling/framework/recognizers/recognizer_gcn.py", line 54, in train_step
loss_metrics = self.head.loss(cls_score, label, if_top5=self.if_top5)
File "/media/dp504/36675aac-be99-45b0-abf0-9c74c037c1b7/PycharmProjects/swf/PaddleVideo/paddlevideo/modeling/heads/base.py", line 85, in loss
loss = self.loss_func(scores, labels, **kwargs)
File "/home/dp504/miniconda3/envs/paddle/lib/python3.7/site-packages/paddle/nn/layer/layers.py", line 1240, in call
return self.forward(*inputs, **kwargs)
File "PaddleVideo/paddlevideo/modeling/losses/base.py", line 49, in forward
return self._forward(*args, **kwargs) * self.loss_weight
File "PaddleVideo/paddlevideo/modeling/losses/cross_entropy_loss.py", line 35, in _forward
loss = F.cross_entropy(score, labels, **kwargs)
File "/home/dp504/miniconda3/envs/paddle/lib/python3.7/site-packages/paddle/nn/functional/loss.py", line 2788, in cross_entropy
if paddle.count_nonzero(is_ignore) > 0: # ignore label
File "/home/dp504/miniconda3/envs/paddle/lib/python3.7/site-packages/paddle/fluid/dygraph/tensor_patch_methods.py", line 681, in bool
return self.nonzero()
File "/home/dp504/miniconda3/envs/paddle/lib/python3.7/site-packages/paddle/fluid/dygraph/tensor_patch_methods.py", line 678, in nonzero
return bool(np.array(self) > 0)
File "/home/dp504/miniconda3/envs/paddle/lib/python3.7/site-packages/paddle/fluid/dygraph/tensor_patch_methods.py", line 704, in array
array = self.numpy(False)
OSError: (External) CUDA error(719), unspecified launch failure.
[Hint: 'cudaErrorLaunchFailure'. An exception occurred on the device while executing a kernel. Common causes include dereferencing an invalid device pointerand accessing out of bounds shared memory. Less common cases can be system specific - more information about these cases canbe found in the system specific user guide. This leaves the process in an inconsistent state and any further CUDA work willreturn the same error. To continue using CUDA, the process must be terminated and relaunched.] (at ../paddle/phi/backends/gpu/cuda/cuda_info.cc:272)

[1]+ Killed python

Answer 1

【以下回答由 GPT 生成】

回答

根据报错信息，可以得知训练过程中出现了标签值（label）的错误。具体来说，错误是发生在计算交叉熵（cross entropy）损失函数时出现的。

这个错误的原因是训练数据中的标签值出现了超出范围的数值，导致交叉熵的计算失败。交叉熵损失函数需要标签值在一个指定的范围内（通常是0到类别数量之间）。

为了解决这个问题，你可以按照以下步骤进行：

1. 首先，检查训练数据中标签值的范围，确保它们在合理的范围内。对于多分类问题，标签值应该从0开始递增，并且不应该超过类别数量减1。
2. 如果发现标签值超出了合理范围，你可以尝试一些方法来调整它们。例如，可以使用np.clip函数将标签值限制在合理范围内。
3. 如果你已经确认数据中的标签值是正确的，并且仍然遇到这个错误，请检查你的模型是否正确配置。确保模型的输出维度和标签值的维度匹配。
4. 如果以上步骤都没有解决问题，可能是PaddleVideo库中的一个bug导致的。你可以尝试查看PaddleVideo的GitHub仓库，看是否有其他用户报告过类似的问题，并寻求官方的支持或提交一个issue来解决问题。

希望这些步骤能够帮助你解决问题！如果你还有其他问题，请随时提问。

如果你已经解决了该问题, 非常希望你能够分享一下解决方案, 写成博客, 将相关链接放在评论区, 以帮助更多的人 ^-^