weixin_40008566
2020-12-30 03:36 阅读 8

Setup fails to install shap due to missing Microsoft Visual C++

Describe the bug

Setup for installing fairlearn throws an error as it's unable to install shap

Steps/Code to Reproduce


In a new conda env, run these:
git clone git.com:fairlearn/fairlearn.git
pip install -e .
pip install -r requirements.txt

Expected Results

Successful install and no errors

Actual Results

Building wheels for collected packages: shap Building wheel for shap (setup.py) ... error ERROR: Command errored out with exit status 1: command: 'C:\Users\arjsingh\Miniconda3\envs\fair_learn\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\arjsingh\AppData\Local\Temp\pip-install-h00nraer\shap\setup.py'"'"'; file='"'"'C:\Users\arjsingh\AppData\Local\Temp\pip-install-h00nraer\shap\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\arjsingh\AppData\Local\Temp\pip-wheel-f_zgw3dc' cwd: C:\Users\arjsingh\AppData\Local\Temp\pip-install-h00nraer\shap\ Complete output (67 lines): running bdist_wheel running build running build_py creating build creating build\lib.win-amd64-3.7 creating build\lib.win-amd64-3.7\shap copying shap\common.py -> build\lib.win-amd64-3.7\shap copying shap\datasets.py -> build\lib.win-amd64-3.7\shap copying shap__init__.py -> build\lib.win-amd64-3.7\shap creating build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\additive.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\bruteforce.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\explainer.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\gradient.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\kernel.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\linear.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\mimic.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\partition.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\permutation.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\pytree.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\sampling.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\tf_utils.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\tree.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers__init__.py -> build\lib.win-amd64-3.7\shap\explainers creating build\lib.win-amd64-3.7\shap\explainers\other copying shap\explainers\other\coefficent.py -> build\lib.win-amd64-3.7\shap\explainers\other copying shap\explainers\other\lime.py -> build\lib.win-amd64-3.7\shap\explainers\other copying shap\explainers\other\maple.py -> build\lib.win-amd64-3.7\shap\explainers\other copying shap\explainers\other\random.py -> build\lib.win-amd64-3.7\shap\explainers\other copying shap\explainers\other\treegain.py -> build\lib.win-amd64-3.7\shap\explainers\other copying shap\explainers\other__init__.py -> build\lib.win-amd64-3.7\shap\explainers\other creating build\lib.win-amd64-3.7\shap\explainers\deep copying shap\explainers\deep\deep_pytorch.py -> build\lib.win-amd64-3.7\shap\explainers\deep copying shap\explainers\deep\deep_tf.py -> build\lib.win-amd64-3.7\shap\explainers\deep copying shap\explainers\deep__init__.py -> build\lib.win-amd64-3.7\shap\explainers\deep creating build\lib.win-amd64-3.7\shap\plots copying shap\plots\bar.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\colorconv.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\colors.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\decision.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\dependence.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\embedding.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\force.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\force_matplotlib.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\image.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\monitoring.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\partial_dependence.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\summary.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\text.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\waterfall.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots__init__.py -> build\lib.win-amd64-3.7\shap\plots creating build\lib.win-amd64-3.7\shap\benchmark copying shap\benchmark\experiments.py -> build\lib.win-amd64-3.7\shap\benchmark copying shap\benchmark\measures.py -> build\lib.win-amd64-3.7\shap\benchmark copying shap\benchmark\methods.py -> build\lib.win-amd64-3.7\shap\benchmark copying shap\benchmark\metrics.py -> build\lib.win-amd64-3.7\shap\benchmark copying shap\benchmark\models.py -> build\lib.win-amd64-3.7\shap\benchmark copying shap\benchmark\plots.py -> build\lib.win-amd64-3.7\shap\benchmark copying shap\benchmark__init__.py -> build\lib.win-amd64-3.7\shap\benchmark creating build\lib.win-amd64-3.7\shap\plots\resources copying shap\plots\resources\bundle.js -> build\lib.win-amd64-3.7\shap\plots\resources copying shap\plots\resources\logoSmallGray.png -> build\lib.win-amd64-3.7\shap\plots\resources copying shap\tree_shap.h -> build\lib.win-amd64-3.7\shap running build_ext numpy.get_include() C:\Users\arjsingh\Miniconda3\envs\fair_learn\lib\site-packages\numpy\core\include building 'shap._cext' extension error: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/

ERROR: Failed building wheel for shap Running setup.py clean for shap Failed to build shap Installing collected packages: shap, psutil, memory-profiler, tempeh, xlrd, qtpy, qtconsole, jupyter-console, jupyter, nbval, sphinxcontrib-devhelp, sphinxcontrib-jsmath, imagesize, alabaster, sphinxcontrib-qthelp, sphinxcontrib-serializinghtml, sphinxcontrib-htmlhelp, babel, sphinxcontrib-applehelp, sphinx, sphinx-gallery, pydata-sphinx-theme Running setup.py install for shap ... error ERROR: Command errored out with exit status 1: command: 'C:\Users\arjsingh\Miniconda3\envs\fair_learn\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\arjsingh\AppData\Local\Temp\pip-install-h00nraer\shap\setup.py'"'"'; file='"'"'C:\Users\arjsingh\AppData\Local\Temp\pip-install-h00nraer\shap\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\arjsingh\AppData\Local\Temp\pip-record-cp20g2kq\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\arjsingh\Miniconda3\envs\fair_learn\Include\shap' cwd: C:\Users\arjsingh\AppData\Local\Temp\pip-install-h00nraer\shap\ Complete output (67 lines): running install running build running build_py creating build creating build\lib.win-amd64-3.7 creating build\lib.win-amd64-3.7\shap copying shap\common.py -> build\lib.win-amd64-3.7\shap copying shap\datasets.py -> build\lib.win-amd64-3.7\shap copying shap__init__.py -> build\lib.win-amd64-3.7\shap creating build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\additive.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\bruteforce.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\explainer.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\gradient.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\kernel.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\linear.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\mimic.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\partition.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\permutation.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\pytree.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\sampling.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\tf_utils.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers\tree.py -> build\lib.win-amd64-3.7\shap\explainers copying shap\explainers__init__.py -> build\lib.win-amd64-3.7\shap\explainers creating build\lib.win-amd64-3.7\shap\explainers\other copying shap\explainers\other\coefficent.py -> build\lib.win-amd64-3.7\shap\explainers\other copying shap\explainers\other\lime.py -> build\lib.win-amd64-3.7\shap\explainers\other copying shap\explainers\other\maple.py -> build\lib.win-amd64-3.7\shap\explainers\other copying shap\explainers\other\random.py -> build\lib.win-amd64-3.7\shap\explainers\other copying shap\explainers\other\treegain.py -> build\lib.win-amd64-3.7\shap\explainers\other copying shap\explainers\other__init__.py -> build\lib.win-amd64-3.7\shap\explainers\other creating build\lib.win-amd64-3.7\shap\explainers\deep copying shap\explainers\deep\deep_pytorch.py -> build\lib.win-amd64-3.7\shap\explainers\deep copying shap\explainers\deep\deep_tf.py -> build\lib.win-amd64-3.7\shap\explainers\deep copying shap\explainers\deep__init__.py -> build\lib.win-amd64-3.7\shap\explainers\deep creating build\lib.win-amd64-3.7\shap\plots copying shap\plots\bar.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\colorconv.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\colors.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\decision.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\dependence.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\embedding.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\force.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\force_matplotlib.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\image.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\monitoring.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\partial_dependence.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\summary.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\text.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots\waterfall.py -> build\lib.win-amd64-3.7\shap\plots copying shap\plots__init__.py -> build\lib.win-amd64-3.7\shap\plots creating build\lib.win-amd64-3.7\shap\benchmark copying shap\benchmark\experiments.py -> build\lib.win-amd64-3.7\shap\benchmark copying shap\benchmark\measures.py -> build\lib.win-amd64-3.7\shap\benchmark copying shap\benchmark\methods.py -> build\lib.win-amd64-3.7\shap\benchmark copying shap\benchmark\metrics.py -> build\lib.win-amd64-3.7\shap\benchmark copying shap\benchmark\models.py -> build\lib.win-amd64-3.7\shap\benchmark copying shap\benchmark\plots.py -> build\lib.win-amd64-3.7\shap\benchmark copying shap\benchmark__init__.py -> build\lib.win-amd64-3.7\shap\benchmark creating build\lib.win-amd64-3.7\shap\plots\resources copying shap\plots\resources\bundle.js -> build\lib.win-amd64-3.7\shap\plots\resources copying shap\plots\resources\logoSmallGray.png -> build\lib.win-amd64-3.7\shap\plots\resources copying shap\tree_shap.h -> build\lib.win-amd64-3.7\shap running build_ext numpy.get_include() C:\Users\arjsingh\Miniconda3\envs\fair_learn\lib\site-packages\numpy\core\include building 'shap._cext' extension error: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/ ---------------------------------------- ERROR: Command errored out with exit status 1: 'C:\Users\arjsingh\Miniconda3\envs\fair_learn\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\arjsingh\AppData\Local\Temp\pip-install-h00nraer\shap\setup.py'"'"'; file='"'"'C:\Users\arjsingh\AppData\Local\Temp\pip-install-h00nraer\shap\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\arjsingh\AppData\Local\Temp\pip-record-cp20g2kq\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\arjsingh\Miniconda3\envs\fair_learn\Include\shap' Check the logs for full command output.

Screenshots

Versions

该提问来源于开源项目:fairlearn/fairlearn

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享

11条回答 默认 最新

  • weixin_40008566 weixin_40008566 2020-12-30 03:36

    Possibly happening because tempeh depends on shap and one of the Jupyter Notebooks requires tempeh

    点赞 评论 复制链接分享
  • weixin_40002238 weixin_40002238 2020-12-30 03:36

    While this is definitely an issue worth investigating this also raises the dependency on tempeh as an issue. We could easily remove that dependency if we add the remaining datasets that we use from there directly to fairlearn.datasets.

    点赞 评论 复制链接分享
  • weixin_40002238 weixin_40002238 2020-12-30 03:36

    I tried reproing and couldn't with these steps:

    
    conda create -n shap_issue_env python=3.7
    conda activate shap_issue_env
    pip install -e .
    pip install -r .\requirements.txt
    

    Now it could obviously be that I have C++ installed on my machine from another project and that isn't restricted by conda envs. I don't recall manually installing that, so another idea would be that it's there for everyone and you may have manually uninstalled it?

    Could you tell me - the python version you used - the exact commands to create the conda env (perhaps there's some significant difference) - whether you see any C++ installed under "Apps & features". I have the following, for example:

    I'll look into removing tempeh from requirements.txt since that's bothered me for a while...

    点赞 评论 复制链接分享
  • weixin_39747615 weixin_39747615 2020-12-30 03:36

    Microsoft still doesn't allow redistribution of the VS compilers :/ As a result, setting up C++ compilers on Windows is still very challenging.

    You could check the python's guide or scikit-learn's guide on how to setup a windows machine. It may help.

    点赞 评论 复制链接分享
  • weixin_39716703 weixin_39716703 2020-12-30 03:36

    And of course the Windows agents in our build pools all come with Visual Studio: https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted (and at work, one of the first things I do on a newly imaged machine is install VS).

    Is there an easy way of detecting packages with potentially problematic dependencies like this?

    点赞 评论 复制链接分享
  • weixin_39747615 weixin_39747615 2020-12-30 03:36

    Their setup.py says they support python 3.6 only, but their CI checks for python 3.5, 3.6, and 3.7. They can't do 3.8 since tensorflow is always lagging behind in terms of supporting latest python releases.

    Also, for some reason they don't even include the 3.7 wheel in their release: https://pypi.org/project/shap/#files and that's why installing on a 3.7 environment tries to compile the package from source.

    In general, it's important to be very careful with dependencies since it's really easy to get into situations where dependencies cannot be easily satisfied.

    点赞 评论 复制链接分享
  • weixin_39580715 weixin_39580715 2020-12-30 03:36

    I ran into the same issue with 3.7 Python 3.6 works without any issues.

    Installing this did not work

    error: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/

    Installing MSVC v142 - VS 2019 C++ x64/x86 build tools (v14.26) and Windows 10 SDK (10.0.19041.0) allows Shap to compile successfully.

    点赞 评论 复制链接分享
  • weixin_40002238 weixin_40002238 2020-12-30 03:36

    Well on the plus side we don't depend on it, and we can soon remove the dependencies for tempeh. has a PR out to replace the COMPAS dataset from tempeh with the openml version of it, and then the only remaining one should be the law school dataset.

    点赞 评论 复制链接分享
  • weixin_39716703 weixin_39716703 2020-12-30 03:36

    While we should be off this particular issue for shap soon, I have a feeling we're going to hit the same underlying problem again. Indeed, I was asked to redo the ADO pipelines for interpret-community and interpret-text based on the work I did for Fairlearn, and.... those had an interesting mix of conda and pip in their setup. Then on the AzureML side, another problem got traced to pip not bothering to check the package version against the requirement in setup.py if the package was already installed.

    have you any suggestions for managing this form of dependency hell? When we supported Python 3.5, we needed a separate requirements-3.5.txt and we could go back to that.... presumably with similarly suffixed conda environment files if/when we add that in? Spawning off specialised files like that strikes me as non-ideal. One of the interpret repos has a script to generate the conda environment file, but that just moves the maintenance headaches to that script (and in its current incarnation, doesn't actually support any Python version than 3.6.8).

    点赞 评论 复制链接分享
  • weixin_40002238 weixin_40002238 2020-12-30 03:36

    -ms I like the idea outlined in #516 which is about splitting.

    点赞 评论 复制链接分享
  • weixin_39716703 weixin_39716703 2020-12-30 03:36

    I don't see shap in our requirements files any more... this should be fixed?

    Update: No; tempeh depends on it. However, only one notebook depends on tempeh

    点赞 评论 复制链接分享

相关推荐