基于ubuntu的Mamba配置环境失败问题

Question

在基于ubuntu配置Mamba + lm_eval测试环境的时候遇到了AttributeError: 'MambaEvalWrapper' object has no attribute 'add_bos_token'问题，请帮忙看下。具体搭建环境过程和错误信息如下：

1、搭建环境方法
1.1 主要参照链接 https://blog.csdn.net/weixin_46524818/article/details/136427276 的方法搭建
1.2 由于没有外网翻墙，因此将下述文件中的链接替换为镜像链接
/home/ubuntu/anaconda3/envs/DevEnv/lib/python3.11/site-packages/huggingface_hub/constants.py
/home/ubuntu/anaconda3/envs/DevEnv/lib/python3.11/site-packages/datasets/config.py
~/.cache/huggingface/modules/datasets_modules/datasets/EleutherAI--lambada_openai/57baddecfa09d1790541ef07274c5666abfbe9d2ccd0cd46013cd557b0343095/lambada_openai.py
1.3 当前已经能成功执行 import mamba_ssm

2、执行测试套件方法
按照mamba 的 github ( https://github.com/state-spaces/mamba ) 所写，使用下述命令执行测试
python evals/lm_harness_eval.py --model mamba --model_args pretrained=state-spaces/mamba-130m --tasks lambada_openai --device cuda --batch_size 64

3、当前conda虚拟机已经安装的软件包列表

(MambaEnv) ubuntu@ubuntu-virtual-machine:~/BlackOp/mamba-1.0.1$ conda list
# packages in environment at /home/ubuntu/anaconda3/envs/MambaEnv:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             5.1                       1_gnu  
absl-py                   2.1.0                    pypi_0    pypi
accelerate                0.30.1                   pypi_0    pypi
aiohttp                   3.9.5                    pypi_0    pypi
aiosignal                 1.3.1                    pypi_0    pypi
argparse                  1.4.0                    pypi_0    pypi
async-timeout             4.0.3                    pypi_0    pypi
attrs                     23.2.0                   pypi_0    pypi
automat                   22.10.0                  pypi_0    pypi
buildtools                1.0.6                    pypi_0    pypi
bzip2                     1.0.8                h5eee18b_6  
ca-certificates           2024.3.11            h06a4308_0  
causal-conv1d             1.1.1                    pypi_0    pypi
certifi                   2022.12.7                pypi_0    pypi
chardet                   5.2.0                    pypi_0    pypi
charset-normalizer        2.1.1                    pypi_0    pypi
click                     8.1.7                    pypi_0    pypi
colorama                  0.4.6                    pypi_0    pypi
constantly                23.10.4                  pypi_0    pypi
dataproperty              1.0.1                    pypi_0    pypi
datasets                  2.19.1                   pypi_0    pypi
dill                      0.3.8                    pypi_0    pypi
docopt                    0.6.2                    pypi_0    pypi
einops                    0.8.0                    pypi_0    pypi
evaluate                  0.4.2                    pypi_0    pypi
filelock                  3.13.1                   pypi_0    pypi
frozenlist                1.4.1                    pypi_0    pypi
fsspec                    2024.2.0                 pypi_0    pypi
furl                      2.1.3                    pypi_0    pypi
greenlet                  3.0.3                    pypi_0    pypi
huggingface-hub           0.23.0                   pypi_0    pypi
hyperlink                 21.0.0                   pypi_0    pypi
idna                      3.4                      pypi_0    pypi
incremental               22.10.0                  pypi_0    pypi
jinja2                    3.1.3                    pypi_0    pypi
joblib                    1.4.2                    pypi_0    pypi
jsonlines                 4.0.0                    pypi_0    pypi
ld_impl_linux-64          2.38                 h1181459_1  
libffi                    3.4.4                h6a678d5_1  
libgcc-ng                 11.2.0               h1234567_1  
libgomp                   11.2.0               h1234567_1  
libstdcxx-ng              11.2.0               h1234567_1  
libuuid                   1.41.5               h5eee18b_0  
lm-eval                   0.4.2                    pypi_0    pypi
lxml                      5.2.2                    pypi_0    pypi
mamba-ssm                 1.0.1                    pypi_0    pypi
markupsafe                2.1.5                    pypi_0    pypi
mbstrdecoder              1.1.3                    pypi_0    pypi
more-itertools            10.2.0                   pypi_0    pypi
mpmath                    1.3.0                    pypi_0    pypi
multidict                 6.0.5                    pypi_0    pypi
multiprocess              0.70.16                  pypi_0    pypi
ncurses                   6.4                  h6a678d5_0  
networkx                  3.2.1                    pypi_0    pypi
ninja                     1.11.1.1                 pypi_0    pypi
nltk                      3.8.1                    pypi_0    pypi
numexpr                   2.10.0                   pypi_0    pypi
numpy                     1.26.3                   pypi_0    pypi
openssl                   3.0.13               h7f8727e_1  
orderedmultidict          1.0.1                    pypi_0    pypi
packaging                 23.2            py310h06a4308_0  
pandas                    2.2.2                    pypi_0    pypi
pathvalidate              3.2.0                    pypi_0    pypi
peft                      0.11.1                   pypi_0    pypi
pillow                    10.2.0                   pypi_0    pypi
pip                       24.0            py310h06a4308_0  
portalocker               2.8.2                    pypi_0    pypi
psutil                    5.9.8                    pypi_0    pypi
pyarrow                   16.1.0                   pypi_0    pypi
pyarrow-hotfix            0.6                      pypi_0    pypi
pybind11                  2.12.0                   pypi_0    pypi
pytablewriter             1.2.0                    pypi_0    pypi
python                    3.10.13              h955ad1f_0  
python-dateutil           2.9.0.post0              pypi_0    pypi
pytz                      2024.1                   pypi_0    pypi
pyyaml                    6.0.1                    pypi_0    pypi
readline                  8.2                  h5eee18b_0  
redo                      2.0.4                    pypi_0    pypi
regex                     2024.5.15                pypi_0    pypi
requests                  2.28.1                   pypi_0    pypi
rouge-score               0.1.2                    pypi_0    pypi
sacrebleu                 2.4.2                    pypi_0    pypi
safetensors               0.4.3                    pypi_0    pypi
scikit-learn              1.4.2                    pypi_0    pypi
scipy                     1.13.0                   pypi_0    pypi
setuptools                69.5.1          py310h06a4308_0  
simplejson                3.19.2                   pypi_0    pypi
six                       1.16.0                   pypi_0    pypi
sqlalchemy                2.0.30                   pypi_0    pypi
sqlite                    3.45.3               h5eee18b_0  
sqlitedict                2.1.0                    pypi_0    pypi
sympy                     1.12                     pypi_0    pypi
tabledata                 1.3.3                    pypi_0    pypi
tabulate                  0.9.0                    pypi_0    pypi
tcolorpy                  0.1.6                    pypi_0    pypi
threadpoolctl             3.5.0                    pypi_0    pypi
tk                        8.6.14               h39e8969_0  
tokenizers                0.19.1                   pypi_0    pypi
torch                     2.1.1+cu118              pypi_0    pypi
torchaudio                2.1.1+cu118              pypi_0    pypi
torchvision               0.16.1+cu118             pypi_0    pypi
tqdm                      4.66.4                   pypi_0    pypi
tqdm-multiprocess         0.0.11                   pypi_0    pypi
transformers              4.41.0                   pypi_0    pypi
triton                    2.1.0                    pypi_0    pypi
twisted                   24.3.0                   pypi_0    pypi
typepy                    1.3.2                    pypi_0    pypi
typing-extensions         4.9.0                    pypi_0    pypi
tzdata                    2024.1                   pypi_0    pypi
urllib3                   1.26.13                  pypi_0    pypi
wheel                     0.43.0          py310h06a4308_0  
word2number               1.1                      pypi_0    pypi
xxhash                    3.4.1                    pypi_0    pypi
xz                        5.4.6                h5eee18b_1  
yarl                      1.9.4                    pypi_0    pypi
zlib                      1.2.13               h5eee18b_1  
zope-interface            6.4                      pypi_0    pypi
zstandard                 0.22.0                   pypi_0    pypi

4、具体的报错信息

(MambaEnv) ubuntu@ubuntu-virtual-machine:~/BlackOp/mamba-1.0.1$ python evals/lm_harness_eval.py --model mamba --tasks lambada_openai --device cuda --batch_size 64
2024-05-22:23:09:46,316 INFO     [__main__.py:251] Verbosity set to INFO
2024-05-22:23:09:50,248 INFO     [__main__.py:335] Selected Tasks: ['lambada_openai']
2024-05-22:23:09:50,249 INFO     [__main__.py:336] Loading selected tasks...
2024-05-22:23:09:50,249 INFO     [evaluator.py:131] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234
model_name = state-spaces/mamba-2.8b, CONFIG_NAME = config.json
path_or_repo_id = state-spaces/mamba-2.8b, filename = config.json, subfolder = , cache_dir = /home/ubuntu/.cache/huggingface/hub
resolved_archive_file = /home/ubuntu/.cache/huggingface/hub/models--state-spaces--mamba-2.8b/snapshots/e886be8192cbb383b01559a3877dfd5e6bfb3e55/config.json
path_or_repo_id = state-spaces/mamba-2.8b, filename = pytorch_model.bin, subfolder = , cache_dir = /home/ubuntu/.cache/huggingface/hub
pytorch_model.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 11.1G/11.1G [45:42<00:00, 3.31MB/s]
path_or_repo_id = EleutherAI/gpt-neox-20b, filename = tokenizer_config.json, subfolder = , cache_dir = /home/ubuntu/.cache/huggingface/hub
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
path = EleutherAI/lambada_openai, data_dir = None, cache_dir = None, download_config = None
url = https://hf-mirror.com/api/datasets/EleutherAI/lambada_openai
2024-05-22:23:55:58,120 WARNING  [task.py:322] [Task: lambada_openai] has_training_docs and has_validation_docs are False, using test_docs as fewshot_docs but this is not recommended.
2024-05-22:23:55:58,120 WARNING  [task.py:322] [Task: lambada_openai] has_training_docs and has_validation_docs are False, using test_docs as fewshot_docs but this is not recommended.
2024-05-22:23:55:58,170 INFO     [task.py:395] Building contexts for lambada_openai on rank 0...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5153/5153 [00:05<00:00, 926.66it/s]
2024-05-22:23:56:03,780 INFO     [evaluator.py:362] Running loglikelihood requests
Traceback (most recent call last):
  File "/home/ubuntu/BlackOp/mamba-1.0.1/evals/lm_harness_eval.py", line 39, in <module>
    cli_evaluate()
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/__main__.py", line 342, in cli_evaluate
    results = evaluator.simple_evaluate(
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/utils.py", line 288, in _wrapper
    return fn(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/evaluator.py", line 234, in simple_evaluate
    results = evaluate(
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/utils.py", line 288, in _wrapper
    return fn(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/evaluator.py", line 373, in evaluate
    resps = getattr(lm, reqtype)(cloned_reqs)
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/api/model.py", line 325, in loglikelihood
    context_enc, continuation_enc = self._encode_pair(context, continuation)
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/api/model.py", line 301, in _encode_pair
    whole_enc = self.tok_encode(context + continuation)
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/models/huggingface.py", line 669, in tok_encode
    add_special_tokens = False or self.add_bos_token
AttributeError: 'MambaEvalWrapper' object has no attribute 'add_bos_token'
(MambaEnv) ubuntu@ubuntu-virtual-machine:~/BlackOp/mamba-1.0.1$ python evals/lm_harness_eval.py --model mamba --tasks lambada_openai --device cuda --batch_size 64
2024-05-23:00:38:35,606 INFO     [__main__.py:251] Verbosity set to INFO
2024-05-23:00:38:39,375 INFO     [__main__.py:335] Selected Tasks: ['lambada_openai']
2024-05-23:00:38:39,375 INFO     [__main__.py:336] Loading selected tasks...
2024-05-23:00:38:39,376 INFO     [evaluator.py:131] Setting random seed to 0 | Setting numpy seed to 1234 | Setting torch manual seed to 1234
model_name = state-spaces/mamba-2.8b, CONFIG_NAME = config.json
path_or_repo_id = state-spaces/mamba-2.8b, filename = config.json, subfolder = , cache_dir = /home/ubuntu/.cache/huggingface/hub
resolved_archive_file = /home/ubuntu/.cache/huggingface/hub/models--state-spaces--mamba-2.8b/snapshots/e886be8192cbb383b01559a3877dfd5e6bfb3e55/config.json
path_or_repo_id = state-spaces/mamba-2.8b, filename = pytorch_model.bin, subfolder = , cache_dir = /home/ubuntu/.cache/huggingface/hub
path_or_repo_id = EleutherAI/gpt-neox-20b, filename = tokenizer_config.json, subfolder = , cache_dir = /home/ubuntu/.cache/huggingface/hub
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
path = EleutherAI/lambada_openai, data_dir = None, cache_dir = None, download_config = None
url = https://hf-mirror.com/api/datasets/EleutherAI/lambada_openai
2024-05-23:00:38:52,952 WARNING  [task.py:322] [Task: lambada_openai] has_training_docs and has_validation_docs are False, using test_docs as fewshot_docs but this is not recommended.
2024-05-23:00:38:52,952 WARNING  [task.py:322] [Task: lambada_openai] has_training_docs and has_validation_docs are False, using test_docs as fewshot_docs but this is not recommended.
2024-05-23:00:38:53,010 INFO     [task.py:395] Building contexts for lambada_openai on rank 0...
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5153/5153 [00:05<00:00, 922.83it/s]
2024-05-23:00:38:58,643 INFO     [evaluator.py:362] Running loglikelihood requests
Traceback (most recent call last):
  File "/home/ubuntu/BlackOp/mamba-1.0.1/evals/lm_harness_eval.py", line 39, in <module>
    cli_evaluate()
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/__main__.py", line 342, in cli_evaluate
    results = evaluator.simple_evaluate(
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/utils.py", line 288, in _wrapper
    return fn(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/evaluator.py", line 234, in simple_evaluate
    results = evaluate(
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/utils.py", line 288, in _wrapper
    return fn(*args, **kwargs)
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/evaluator.py", line 373, in evaluate
    resps = getattr(lm, reqtype)(cloned_reqs)
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/api/model.py", line 325, in loglikelihood
    context_enc, continuation_enc = self._encode_pair(context, continuation)
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/api/model.py", line 301, in _encode_pair
    whole_enc = self.tok_encode(context + continuation)
  File "/home/ubuntu/anaconda3/envs/MambaEnv/lib/python3.10/site-packages/lm_eval/models/huggingface.py", line 669, in tok_encode
    add_special_tokens = False or self.add_bos_token
AttributeError: 'MambaEvalWrapper' object has no attribute 'add_bos_token'

Answer 1

以下内容由CHATGPT及阿里嘎多学长共同生成、有用望采纳：

---

根据您提供的信息，以下是解决这个问题的具体步骤：

1. 检查MambaEvalWrapper类：首先，您需要找到MambaEvalWrapper类的定义，查看是否有add_bos_token属性。如果这个类是自定义的，您可能需要检查本地的代码库。

2. 查看lm_eval库的文档：访问lm_eval的官方文档或GitHub页面，查找关于MambaEvalWrapper和add_bos_token的信息。确认是否有关于这个属性的文档说明，以及它是在哪个版本中引入的。

3. 检查lm_eval库的版本：通过运行pip show lm_eval或conda list lm_eval来检查当前安装的lm_eval库的版本。如果版本过旧，可能需要更新到包含add_bos_token属性的版本。

4. 更新lm_eval库：如果确认是因为版本问题导致的属性缺失，您可以尝试更新lm_eval库：

   pip install --upgrade lm_eval
   

   或者如果您使用的是conda环境：

   conda update lm_eval
   

5. 检查代码中的使用情况：如果add_bos_token属性确实存在于新版本中，检查您的代码是否正确地引用了这个属性。如果代码中有硬编码的属性名，确保它们与库中的定义相匹配。

6. 尝试重新运行测试：更新库后，重新运行您的测试套件，查看问题是否解决。

7. 检查代码修改：如果您之前修改了huggingface_hub或datasets库的配置文件以使用镜像链接，确保这些修改没有影响到lm_eval库的功能。

8. 查看错误日志：如果问题依旧存在，仔细查看错误日志，看看是否有其他相关的错误信息可以提供线索。

9. 寻求社区帮助：如果以上步骤都不能解决问题，您可以考虑在相关的技术社区（如Stack Overflow）或lm_eval的GitHub问题跟踪器上提问，提供详细的错误信息和您已经尝试的解决步骤。

10. 联系库的维护者：如果问题看起来像是库的一个bug，或者您在社区中没有找到解决方案，您可以考虑直接联系库的维护者。

请注意，由于我无法访问外部链接或执行代码，以上步骤是基于您提供的信息和一般性问题解决策略提出的建议。实际操作时可能需要根据具体情况进行调整。