V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
推荐学习书目
Learn Python the Hard Way
Python Sites
PyPI - Python Package Index
http://diveintopython.org/toc/index.html
Pocoo
值得关注的项目
PyPy
Celery
Jinja2
Read the Docs
gevent
pyenv
virtualenv
Stackless Python
Beautiful Soup
结巴中文分词
Green Unicorn
Sentry
Shovel
Pyflakes
pytest
Python 编程
pep8 Checker
Styles
PEP 8
Google Python Style Guide
Code Style from The Hitchhiker's Guide
v2exblog
V2EX  ›  Python

今天封楼被困公司,研究了一下 ProcessPoolExecutor,发现了一个 Python 的 Bug,分享给各位,网上似乎没解决方案

  •  
  •   v2exblog · 2022-04-25 22:30:36 +08:00 · 3713 次点击
    这是一个创建于 693 天前的主题,其中的信息可能已经有所发展或是发生改变。

    用的 python3.7.13 。报错信息如下兄弟们:

    很奇怪为什么不能 raise key-value 格式的自定义错误 ,

    要是改成 raise MyError( 1, 'desc') 就没问题了,这是不是 python 的 bug?

    # 这是代码,直接,可以运行
    from concurrent.futures.process import ProcessPoolExecutor
    
    
    class MyError(Exception):
        def __init__(self, code: int, desc: str):
            self.code = code
            self.desc = desc
    
    
    def foo(i, j):
        print(i, j)
        raise MyError(code=1, desc='desc')
    
    
    def main():
        with ProcessPoolExecutor(3) as executor:
            f = executor.submit(foo, 1, 11)
            e = f.exception()
            print(e)
    
            f.result()
    
    
    if __name__ == '__main__':
        main()
    

    下面是输出结果:

    1 11
    A process in the process pool was terminated abruptly while the future was running or pending.
    concurrent.futures.process._RemoteTraceback: 
    '''
    Traceback (most recent call last):
      File "/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/process.py", line 367, in _queue_management_worker
        result_item = result_reader.recv()
      File "/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py", line 251, in recv
        return _ForkingPickler.loads(buf.getbuffer())
    TypeError: __init__() missing 2 required positional arguments: 'code' and 'desc'
    '''
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "/Users/v2exblog/demo/aaa.py", line 25, in <module>
        main()
      File "/Users/v2exblog/demo/aaa.py", line 21, in main
        f.result()
      File "/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/_base.py", line 428, in result
        return self.__get_result()
      File "/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
        raise self._exception
    concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
    
    3 条回复    2022-04-26 11:04:12 +08:00
    thinkershare
        1
    thinkershare  
       2022-04-25 22:45:28 +08:00
    封送反序列化的问题, 问题出在 raise 这里, 使用命名参数构造的对象无法按照预期方式反序列化, 错误对象都是构建成功了的.
    thinkershare
        2
    thinkershare  
       2022-04-25 22:47:09 +08:00
    新版本的 python 没啥问题, 已经修复了
    julyclyde
        3
    julyclyde  
       2022-04-26 11:04:12 +08:00
    required positional arguments

    它认为__init__里边那俩是位置参数?
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   我们的愿景   ·   实用小工具   ·   3480 人在线   最高记录 6543   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 21ms · UTC 11:14 · PVG 19:14 · LAX 04:14 · JFK 07:14
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.