dongzhi8487
2018-02-12 19:12
浏览 80

uWSGI +构建Go .so无法正常工作

Problem: .so(shared object) as library in python works well when python calls it and fails in uWSGI-running python(Django) application.

More info: I've build Go module with go build -buildmode=c-shared -o output.so input.go to call it in Python with

from ctypes import cdll

lib = cdll.LoadLibrary('path_to_library/output.so')

When django project is served via uWSGI the request handler that calling Go library freezes, causing future 504 in Nginx. After getting in "so called freeze", uWSGI is locked there and only restarting helps to enliven app. No logs AT ALL! It just freezes.

Everything works correctly when i run in python interpreter on the same machine.

My thoughts: i've tried to debug this and put a lot of log messages in library, but it won't give much info because everything is fine with library(because it works in interpreter). Library loads correctly, because some log messages that i've putted in library. I think it some sort of uWSGI limitation. I don't think putting uwsgi.ini file is somehow helpful.

Additional info:

What limitations can be in uWSGI in that type of shared object work and if there a way to overcome them?

图片转代码服务由CSDN问答提供 功能建议

问题 :. so(共享库)作为python中的库,当 python 调用它,并在运行uWSGI的python(Django)应用程序中失败。

更多信息:我已经使用 go build -buildmode = c-shared -o output.so input.go ,以

 从ctypes在Python中调用import cdll 
 
lib =  cdll.LoadLibrary('path_to_library / output.so')
   
 
 

通过uWSGI为Django项目提供服务时,调用 Go库的请求处理程序 冻结,导致将来在Nginx中出现504。 进入“冻结”状态后,uWSGI被锁定在那里,只有重新启动才能使应用程序活跃起来。 根本没有日志! 只是冻结。

当我在同一台计算机上的python解释器中运行时,一切都正常运行。

我的想法:我已尝试调试此问题并将大量日志消息存储在库中,但是由于库的一切都很好(因为它可以在解释器中工作),因此它不会提供太多信息。 库正确加载,因为我已将某些日志消息放入库中。 我认为这是uWSGI的某种局限性。 我认为放置uwsgi.ini文件不会有所帮助。

其他信息:
  • 转到依赖项

    • fmt
    • github.com/360EntSecGroup-Skylar/excelize
    • log < / code>
    • encoding / json
    • OS:CentOS 6.9

    • Python:Python 3.6.2

    • uWSGI:2.0.15 \ n

      在这种共享对象工作中,uWSGI有哪些限制?是否有克服这些限制的方法?

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 邀请回答

3条回答 默认 最新

  • dsfdsf21321 2018-02-21 20:41
    已采纳

    Attempts and thoughts

    I tried to avoid separation of shared library from my python code, since it requires support of at least one more process, and i would have to rewrite some of the library to create new api.

    As @Lu.nemec kindly noted that:

    Go has its own runtime, thread management and may not work well with uWSGI which handles threads/processes in a different way

    Since uWSGI is the problem i started seaching for a solution there. One of the hopes was installing GCCGO uWSGI plugin somehow solve that problem. But even it's hard to install on old OSes, because it lacks of pre-builded plugins and manual build haven't gone very well, it haven't helped, nothing changes, it still freezes.

    And then i thought that i wan't to disable coroutines and that type of stuff that differs from uWSGI and one of the changes that i am able to do is to set GOMAXPROCS

    GOMAXPROCS sets the maximum number of CPUs that can be executing simultaneously and returns the previous setting. If n < 1, it does not change the current setting. The number of logical CPUs on the local machine can be queried with NumCPU. This call will go away when the scheduler improves.

    And it worked like a charm!!!

    The solution

    import (
        ...
        "runtime"
    )
    ...
    //export yourFunc
    func yourFunc(yourArgs argType) {
        runtime.GOMAXPROCS(1)
        ...
    }
    
    点赞 打赏 评论
  • douqu2481 2018-02-12 19:51

    Firstly, are you absolutely positive you need to call Go as a library from uWSGI process?

    uWSGI are usually for interpreted languages such as PHP, Python, Ruby and others. It bootstraps the interpreter and manages the master/worker processes to handle requests. It seems strange to be using it on Go library.

    You mentioned having nginx as your webserver, why not just use your Go program as the http server (which it does great) and call it from nginx directly using it's URL:

    location /name/ {
        proxy_pass http://127.0.0.1/go_url/;
    }
    

    See nginx docs.

    If you really want to use Go as a python imported library via a .so module, you have to be aware Go has its own runtime, thread management and may not work well with uWSGI which handles threads/processes in a different way. In this case I'm unable to help you, since I never actually tried this.

    If you could clarify your question with what are you actually tring to do, we might me able to answer more helpfully.

    点赞 打赏 评论
  • dongxkbjiaofea1163 2018-03-04 10:34

    My previous answer works in some cases. HOWEVER, when i tried to run the same project on another server with same OS, same Python, same uWSGI (version, plugins, config files), same requirements, same .so file, it freezed the save way as i described in answer.

    I personally didn't want to run this as separate process and bind it to socket/port and create API for communicating with shared library.

    The solution:

    Required only separate process. Run with celery.

    Some caveats:

    1.You cannot run task with task.apply() since it would be run in main application, not in celery:

    result = task.apply_async()
    while result.ready():
        time.wait(5)
    

    2.You neeed to run celery with solo execution pool

    celery -A app worker -P solo
    
    点赞 打赏 评论

相关推荐 更多相似问题