在做word embedding时遇到的问题,有人知道咋解决吗

Question

UnicodeDecodeError: 'gbk' codec can't decode byte 0x97 in position 15: illegal multibyte sequence

Answer 1

# 读取目标文件
    try:
        with open(file_path, 'r', encoding='UTF-8') as f:
            str_ori = f.read()
    except:
        with open(file_path, 'r', encoding='gb18030', errors='ignore') as f:
            str_ori = f.read()
    finally:
        if f in locals():
            f.close()