torch.gather()结果无法复现

Question

深度学习模型中已经固定了随机数种子，使用torch.gather()函数仍然会导致结果不可复现。（使用 torch.use_deterministic_algorithms(True)会抛出异常）。网上关于torch.gather() 确定性的内容较少，请问这是否是一个通用的问题，以及如何解决？

异常信息：RuntimeError: scatter_add_cuda_kernel does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True)'. You can turn off determinism just for this operation, or you can use the 'warn_only=True' option, if that's acceptable for your application. You can also file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation.

Answer 1

• 文章：torch.gather()用法详解 中也许有你想要的答案，请看下吧
• 同时，你还可以查看手册：pytorch all_gather() (in module torch.distributed) 中的内容