uWSGI +构建Go .so无法正常工作

问题</ strong> :. so(共享库)作为python中的库,当 python </ code>调用它,并在运行uWSGI的python(Django)应用程序中失败。</ p>

更多信息</ strong>:我已经使用 go build -buildmode = c-shared -o output.so input.go </ code>,以</ p>

 从ctypes在Python中调用import cdll 

lib = cdll.LoadLibrary('path_to_library / output.so')
</ code> </ pre>

通过uWSGI为Django项目提供服务时,调用 Go库</ em>的请求处理程序 冻结,导致将来在Nginx中出现504。 进入“冻结”状态后,uWSGI被锁定在那里,只有重新启动才能使应用程序活跃起来。 根本没有日志! 只是冻结。</ p>

当我在同一台计算机上的python解释器中运行时,一切都正常运行。</ p>

我的想法</ em>:我已尝试调试此问题并将大量日志消息存储在库中,但是由于库的一切都很好(因为它可以在解释器中工作),因此它不会提供太多信息。 库正确加载,因为我已将某些日志消息放入库中。 我认为这是uWSGI的某种局限性。 我认为放置uwsgi.ini文件不会有所帮助。</ p>

其他信息:</ h3>


  • 转到依赖项</ em> </ p>:</ p>


    • fmt </ code> </ li>
    • github.com/360EntSecGroup-Skylar/excelize </ code> </ li>
    • log < / code> </ li>
    • encoding / json </ code> </ li>
      </ ul> </ li>
    • OS:CentOS 6.9 </ p> </ li>

    • Python:Python 3.6.2 </ p> </ li>

    • uWSGI:2.0.15 </ p> </ li> \ n </ ul>

      在这种共享对象工作中,uWSGI有哪些限制?是否有克服这些限制的方法?</ p>
      </ div>

展开原文

原文

Problem: .so(shared object) as library in python works well when python calls it and fails in uWSGI-running python(Django) application.

More info: I've build Go module with go build -buildmode=c-shared -o output.so input.go to call it in Python with

from ctypes import cdll

lib = cdll.LoadLibrary('path_to_library/output.so')

When django project is served via uWSGI the request handler that calling Go library freezes, causing future 504 in Nginx. After getting in "so called freeze", uWSGI is locked there and only restarting helps to enliven app. No logs AT ALL! It just freezes.

Everything works correctly when i run in python interpreter on the same machine.

My thoughts: i've tried to debug this and put a lot of log messages in library, but it won't give much info because everything is fine with library(because it works in interpreter). Library loads correctly, because some log messages that i've putted in library. I think it some sort of uWSGI limitation. I don't think putting uwsgi.ini file is somehow helpful.

Additional info:

What limitations can be in uWSGI in that type of shared object work and if there a way to overcome them?

3个回答

Attempts and thoughts

I tried to avoid separation of shared library from my python code, since it requires support of at least one more process, and i would have to rewrite some of the library to create new api.

As @Lu.nemec kindly noted that:

Go has its own runtime, thread management and may not work well with uWSGI which handles threads/processes in a different way

Since uWSGI is the problem i started seaching for a solution there. One of the hopes was installing GCCGO uWSGI plugin somehow solve that problem. But even it's hard to install on old OSes, because it lacks of pre-builded plugins and manual build haven't gone very well, it haven't helped, nothing changes, it still freezes.

And then i thought that i wan't to disable coroutines and that type of stuff that differs from uWSGI and one of the changes that i am able to do is to set GOMAXPROCS

GOMAXPROCS sets the maximum number of CPUs that can be executing simultaneously and returns the previous setting. If n < 1, it does not change the current setting. The number of logical CPUs on the local machine can be queried with NumCPU. This call will go away when the scheduler improves.

And it worked like a charm!!!

The solution

import (
    ...
    "runtime"
)
...
//export yourFunc
func yourFunc(yourArgs argType) {
    runtime.GOMAXPROCS(1)
    ...
}



首先,您绝对肯定需要从uWSGI流程中调用Go作为一个库吗?</ p>

uWSGI通常用于解释语言,例如PHP,Python,Ruby和其他语言。 它引导解释器并管理主/工作进程以处理请求。 在Go库上使用它似乎很奇怪。</ p>

您提到将nginx用作Web服务器,为什么不只将Go程序用作http服务器(它做的很好)并调用它 从nginx直接使用其URL:</ p>

 位置/ name / {
proxy_pass http://127.0.0.1/go_url/;
}
</ code> < / pre>

请参阅nginx文档。 </ p>

如果您真的想通过.so模块将Go用作python导入的库,则必须意识到Go具有自己的运行时,线程管理并且可能不适用于uWSGI 以不同的方式处理线程/进程。 在这种情况下,由于我从未真正尝试过此操作,因此无法为您提供帮助。</ p>

如果您可以通过实际操作来澄清自己的问题,我们也许可以回答 更有帮助。</ p>
</ div>

展开原文

原文

Firstly, are you absolutely positive you need to call Go as a library from uWSGI process?

uWSGI are usually for interpreted languages such as PHP, Python, Ruby and others. It bootstraps the interpreter and manages the master/worker processes to handle requests. It seems strange to be using it on Go library.

You mentioned having nginx as your webserver, why not just use your Go program as the http server (which it does great) and call it from nginx directly using it's URL:

location /name/ {
    proxy_pass http://127.0.0.1/go_url/;
}

See nginx docs.

If you really want to use Go as a python imported library via a .so module, you have to be aware Go has its own runtime, thread management and may not work well with uWSGI which handles threads/processes in a different way. In this case I'm unable to help you, since I never actually tried this.

If you could clarify your question with what are you actually tring to do, we might me able to answer more helpfully.

dongluobei9359
dongluobei9359 好吧,在这种情况下,请尝试在自己的uWSGI进程中运行Go库,但是一组只能有1个worker。 我不确定Go的运行时如何与uWSGI worker管理一起使用。 同样,它不应使用辅助线程,而应使用进程,因为Go处理自己的线程。
2 年多之前 回复
douchen2011
douchen2011 谢谢回复。 我不想将此库与主应用程序分开。 我不想为这个小的Go库创建另一个进程,以供nginx监听并处理身份验证和其他内容。
2 年多之前 回复



我的上一个答案有效 在某些情况下。 但是,当我尝试在具有相同操作系统,相同Python,相同uWSGI(版本,插件,配置文件),相同要求,相同.so文件的另一台服务器上运行相同项目时,它冻结了我在答案中所述的保存方式。 </ p>

我个人不想将其作为单独的进程运行并将其绑定到套接字/端口并创建用于与共享库通信的API。 </ p>

解决方案:</ h2>

仅需要单独的过程。 使用 celery </ code>运行。</ p>

一些警告:</ h2>

1。您不能使用 task.apply运行任务。 )</ code>,因为它将在主应用程序中运行,而不是在 celery </ code>中运行:</ p>

  result = task.apply_async()
同时会得到结果。 ready():
time.wait(5)
</ code> </ pre>

2。您需要使用 solo </ code> 执行池</ p>

  celery-应用程序工作者-P独奏
</ code> </ pre>
</ div>

展开原文

原文

My previous answer works in some cases. HOWEVER, when i tried to run the same project on another server with same OS, same Python, same uWSGI (version, plugins, config files), same requirements, same .so file, it freezed the save way as i described in answer.

I personally didn't want to run this as separate process and bind it to socket/port and create API for communicating with shared library.

The solution:

Required only separate process. Run with celery.

Some caveats:

1.You cannot run task with task.apply() since it would be run in main application, not in celery:

result = task.apply_async()
while result.ready():
    time.wait(5)

2.You neeed to run celery with solo execution pool

celery -A app worker -P solo

Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
立即提问
相关内容推荐