#问题
#尝试了以下两种方法
(1)pip install sentencepiece
(2)uninstall transformers → pip install --no-cache-dir transformers sentencepiece → Use_fast= False like this: tokenizer = AutoTokenizer.from_pretrained(“XXXXX”, use_fast=False)
#结果
通通无效T_T,有时候真的想报警,jiu jiu hai zi 吧!