weixin_39942191 2020-11-20 20:32
浏览 0

importing six breaks pickling

Originally reported by: Anselm Kruis (Bitbucket: akruis, GitHub: akruis)

The python2.7 library module pickle contains the function whichmodule(func, funcname). This function eventually inspects every value in sys.modules, triggering imports, that might fail.

This way importing six in one module can break pickling in a completely unrelated module.

An example trace-back on Linux:


  File "/scr/fg2ef/kruis/slp276/lib/python2.7/pickle.py", line 854, in whichmodule
    if name != '__main__' and getattr(module, funcname, None) is func:
  File "/scr/fg2ef/kruis/slp276_six/lib/python2.7/site-packages/six.py", line 116, in __getattr__
    _module = self._resolve()
  File "/scr/fg2ef/kruis/slp276_six/lib/python2.7/site-packages/six.py", line 105, in _resolve
    return _import_module(self.mod)
  File "/scr/fg2ef/kruis/slp276_six/lib/python2.7/site-packages/six.py", line 76, in _import_module
    __import__(name)
ImportError: No module named _winreg

Code of function whichmodule:


def whichmodule(func, funcname):
    """Figure out the module in which a function occurs.

    Search sys.modules for the module.
    Cache in classmap.
    Return a module name.
    If the function cannot be found, return "__main__".
    """
    # Python functions should always get an __module__ from their globals.
    mod = getattr(func, "__module__", None)
    if mod is not None:
        return mod
    if func in classmap:
        return classmap[func]

    for name, module in sys.modules.items():
        if module is None:
            continue # skip dummy package entries
        if name != '__main__' and getattr(module, funcname, None) is func:
            break
    else:
        name = '__main__'
    classmap[func] = name
    return name

Similar code exists in the C-function whichmodule of the extension module Modules/cPickle.c. Therefore it is probably impossible to fix this problem by monkey-patching the Python library.

Idea for a fix: don't populate sys.modules during the import of six itself, but use a sys.meta_path finder (see PEP302) to add six-modules on demand.

  • Bitbucket: https://bitbucket.org/gutworth/six/issue/63

该提问来源于开源项目:benjaminp/six

  • 写回答

9条回答 默认 最新

  • weixin_39942191 2020-11-20 20:32
    关注

    Original comment by Anselm Kruis (Bitbucket: akruis, GitHub: akruis):

    Many thanks. And sorry for not testing the reload case.

    评论

报告相同问题?