执行pip install -e .时报错： Preparing metadata (setup.py) ... error error: subprocess-exited-with-error

Question

Obtaining file:///F:/code/Valley-main
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Installing backend dependencies ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting transformers@ git+https://github.com/huggingface/transformers.git@cae78c46 (from valley==0.1.0)
  Cloning https://github.com/huggingface/transformers.git (to revision cae78c46) to c:\users\lenovo\appdata\local\temp\pip-install-65d2dap2\transformers_367533cc333b472298e3b012c4de1e0a
  Running command git clone --filter=blob:none --quiet https://github.com/huggingface/transformers.git 'C:\Users\lenovo\AppData\Local\Temp\pip-install-65d2dap2\transformers_367533cc333b472298e3b012c4de1e0a'
  WARNING: Did not find branch or tag 'cae78c46', assuming revision or ref.
  Running command git checkout -q cae78c46
  Resolved https://github.com/huggingface/transformers.git to commit cae78c46
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting accelerate (from valley==0.1.0)
  Using cached accelerate-0.29.3-py3-none-any.whl.metadata (18 kB)
Collecting einops (from valley==0.1.0)
  Using cached einops-0.7.0-py3-none-any.whl.metadata (13 kB)
Collecting fastapi (from valley==0.1.0)
  Using cached fastapi-0.110.2-py3-none-any.whl.metadata (24 kB)
Collecting gradio==3.23 (from valley==0.1.0)
  Using cached gradio-3.23.0-py3-none-any.whl.metadata (14 kB)
Collecting markdown2[all] (from valley==0.1.0)
  Using cached markdown2-2.4.13-py2.py3-none-any.whl.metadata (2.0 kB)
Requirement already satisfied: numpy in d:\anaconda\envs\valley\lib\site-packages (from valley==0.1.0) (1.26.4)
Requirement already satisfied: requests in d:\anaconda\envs\valley\lib\site-packages (from valley==0.1.0) (2.31.0)
Requirement already satisfied: sentencepiece in d:\anaconda\envs\valley\lib\site-packages (from valley==0.1.0) (0.1.99)
Collecting tokenizers==0.12.1 (from valley==0.1.0)
  Using cached tokenizers-0.12.1-cp310-cp310-win_amd64.whl.metadata (6.5 kB)
Requirement already satisfied: torch in d:\anaconda\envs\valley\lib\site-packages (from valley==0.1.0) (2.2.2)
Requirement already satisfied: torchvision in d:\anaconda\envs\valley\lib\site-packages (from valley==0.1.0) (0.17.2)
Collecting uvicorn (from valley==0.1.0)
  Using cached uvicorn-0.29.0-py3-none-any.whl.metadata (6.3 kB)
Collecting ninja (from valley==0.1.0)
  Using cached ninja-1.11.1.1-py2.py3-none-win_amd64.whl.metadata (5.4 kB)
Requirement already satisfied: decord in d:\anaconda\envs\valley\lib\site-packages (from valley==0.1.0) (0.6.0)
Requirement already satisfied: scikit-image in d:\anaconda\envs\valley\lib\site-packages (from valley==0.1.0) (0.23.2)
Collecting opencv-python-headless (from valley==0.1.0)
  Using cached opencv_python_headless-4.9.0.80-cp37-abi3-win_amd64.whl.metadata (20 kB)
Collecting peft (from valley==0.1.0)
  Using cached peft-0.10.0-py3-none-any.whl.metadata (13 kB)
Collecting deepspeed (from valley==0.1.0)
  Using cached deepspeed-0.14.1.tar.gz (1.3 MB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [12 lines of output]
      fatal: not a git repository (or any of the parent directories): .git
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "C:\Users\lenovo\AppData\Local\Temp\pip-install-65d2dap2\deepspeed_39dae13ffc9746af98eb1f62a7c221c8\setup.py", line 222, in <module>
          create_dir_symlink('..\\..\\csrc', '.\\deepspeed\\ops\\csrc')
        File "C:\Users\lenovo\AppData\Local\Temp\pip-install-65d2dap2\deepspeed_39dae13ffc9746af98eb1f62a7c221c8\setup.py", line 214, in create_dir_symlink
          os.remove(dest)
      PermissionError: [WinError 5] 拒绝访问。: '.\\deepspeed\\ops\\csrc'
      [WARNING] Torch did not find cuda available, if cross-compiling or running with cpu only you can ignore this message. Adding compute capability for Pascal, Volta, and Turing (compute capabilities 6.0, 6.1, 6.2)        
      DS_BUILD_OPS=1
      Install Ops={'deepspeed_not_implemented': 1, 'deepspeed_ccl_comm': 1, 'cpu_adam': 1, 'fused_adam': 1}
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Answer 1

引自免费微信小程序：皆我百晓生

看起来你在尝试安装一个名为Valley的项目，并且在执行pip install -e .时遇到了错误。这个错误是由于在构建deepspeed包时出现了问题，具体来说是权限问题和可能的非Git仓库目录。

为了解决这个问题，可以按照以下步骤进行操作：

1. 确保你有正确的权限来执行命令。在Windows上，试着右键点击命令提示符并选择“以管理员身份运行”。

2. 如果你在尝试克隆的huggingface/transformers库的版本（cae78c46）有问题，你可以考虑更新到最新版本或者使用一个已知稳定的工作提交版本。首先，删除临时文件夹中的transformers项目：

   rmdir /S /Q %USERPROFILE%\AppData\Local\Temp\pip-install-65d2dap2\transformers_367533cc333b472298e3b012c4de1e0a
   

3. 清除pip缓存以确保获取最新的元数据：

   pip cache purge
   

4. 尝试重新安装Valley，但这次不使用特定的deepspeed版本，这样它会安装最新可用的版本：

   pip install -e .
   

如果这还不能解决问题，那么可能是deepspeed的安装过程中需要创建一个符号链接，但是由于某些原因没有足够的权限。你可以尝试手动创建这个链接，或者在安全模式下运行命令提示符，以获得更高的权限。

如果你仍然遇到问题，特别是关于not a git repository的错误，确认你当前的路径是一个有效的Git工作树。如果不是，你可能需要从GitHub克隆整个Valley项目，然后在项目的根目录下执行pip install -e .。

如果你对上述操作不熟悉，建议联系项目作者或查看项目文档，以便得到更具体的指导。如果问题仍然存在，可能需要检查系统环境设置，例如环境变量、Python路径等，以确保所有依赖项都正确配置。