[Bug] Virtual Provider Concretization Saga

Question

One bug leads to another here. The first bug... See in provider_index.py, SpackYAMLError.__init__() needs more arguments. Any idea what to pass to it?

    def from_yaml(stream):
        try:
            yfile = yaml.load(stream)
        except MarkedYAMLError, e:
            raise spack.spec.SpackYAMLError(
                "error parsing YAML ProviderIndex cache:", str(e))

        if not isinstance(yfile, dict):
            raise spack.spec.SpackYAMLError(
                "YAML ProviderIndex was not a dict.")

        if 'provider_index' not in yfile:
            raise spack.spec.SpackYAMLError(
                "YAML ProviderIndex does not start with 'provider_index'")

该提问来源于开源项目：spack/spack

Answer 1

The code above raised an exception, of course, because of some other problem. In this case, the problem was that yfile is None. That was coming from the code:

            with spack.user_cache.read_transaction(key) as f:
                self._provider_index = ProviderIndex.from_yaml(f)

which was reading the file: /home/rpfische/.spack/cache/providers/builtin-index.yaml. In this case, my builtin-index.yaml file existed but was of size 0. There was nothing else in the entire ~/.spack/cache directory. See stacktrace:

Traceback (most recent call last):
  File "/home/rpfische/spack2/bin/spack", line 161, in main
    return_val = command(parser, args)
  File "/gpfsm/dnb53/rpfische/spack2/lib/spack/spack/cmd/install.py", line 83, in install
    specs = spack.cmd.parse_specs(args.packages, concretize=True)
  File "/gpfsm/dnb53/rpfische/spack2/lib/spack/spack/cmd/__init__.py", line 106, in parse_specs
    spec.concretize()  # implies normalize
  File "/gpfsm/dnb53/rpfische/spack2/lib/spack/spack/spec.py", line 1252, in concretize
    self._expand_virtual_packages(),
  File "/gpfsm/dnb53/rpfische/spack2/lib/spack/spack/spec.py", line 1169, in _expand_virtual_packages
    spec)
  File "/gpfsm/dnb53/rpfische/spack2/lib/spack/spack/concretize.py", line 118, in choose_virtual_or_external
    candidates = self._valid_virtuals_and_externals(spec)
  File "/gpfsm/dnb53/rpfische/spack2/lib/spack/spack/concretize.py", line 62, in _valid_virtuals_and_externals
    providers = spack.repo.providers_for(spec)
  File "/gpfsm/dnb53/rpfische/spack2/lib/spack/spack/repository.py", line 75, in converter
    return function(self, spec_like, *args, **kwargs)
  File "/gpfsm/dnb53/rpfische/spack2/lib/spack/spack/repository.py", line 248, in providers_for
    providers = self.provider_index.providers_for(vpkg_spec)
  File "/gpfsm/dnb53/rpfische/spack2/lib/spack/spack/repository.py", line 242, in provider_index
    self._provider_index.merge(repo.provider_index)
  File "/gpfsm/dnb53/rpfische/spack2/lib/spack/spack/repository.py", line 654, in provider_index
    self._update_provider_index()
  File "/gpfsm/dnb53/rpfische/spack2/lib/spack/spack/repository.py", line 635, in _update_provider_index
    self._provider_index = ProviderIndex.from_yaml(f)
  File "/gpfsm/dnb53/rpfische/spack2/lib/spack/spack/provider_index.py", line 205, in from_yaml
    "YAML ProviderIndex was not a dict, but of type %s instead" % type(yfile), '')

I moved ~/.spack/cache out of the way, and now things went further. Read on...