douzhongpi9734 2017-07-26 16:26
浏览 58
已采纳

在Golang Web应用中管理长流程

I'm trying to implement the following functionality in Golang.

I have a web page with a form, used to upload a .csv file. Gorilla mux is used to route to a handler which takes the file and parses it, does a bunch of operations on the data and at the end produces a report with number of lines parsed, # of rejected lines, etc.

My problem is that even though it works on my machine, on a server Apache will time out before I can get to the end of it all: the file upload itself isn't the issue, but I have to wait for transformations on the data to complete.

I've tried to use Gorilla websocket to get feedback from the process (incrementing number of lines parsed and treated, for instance) and keep the connection open, but this is a POST request, and Gorilla websocket won't upgrade from http to websocket unless one has a GET request.

I'm not even sure I'm on the right track with websockets for doing this type of thing.

I can have a goroutine for the processing itself and return the handler before the goroutine completes, but then how do I show the result of the process in the UI?

So at this stage my question boils down to: what would be the best way, in Golang, when you need to:

  • upload a file,
  • wait for a long process to complete
  • and display the result in a web page?

A clue as to the right direction to go in would be much appreciated.

  • 写回答

1条回答 默认 最新

  • dsajkdadsa14222 2017-07-27 13:36
    关注

    You've stumbled onto a non-trivial problem. There are a lot of possible solutions, with different user experiences, implementation complexities, and side effects. This is a pretty big topic so this answer is intended mostly as a starting point for further research.

    The Simplest Option

    First, pretty much regardless of solution, you're going to have to give each long-running task a unique ID that the browser can use to get status updates later. The task runner itself can just flag jobs as complete, or it can periodically issue progress updates if you want to present progress to the user.

    The easiest to implement is likely to have your form submission immediately respond with a page, with the task ID included in the URL, whose handler checks the task status and either a) returns a page with "still working" or something to that effect and auto-refreshes after a few seconds, or b) returns a page saying "completed" and does not refresh. This isn't terribly difficult to implement, but it's not particularly smooth, either. If this is a simple internal-use project with simple UX and operational requirements, I'd just do this. Otherwise, further down the rabbit hole we go!

    Live Updates

    You could do live updates without reloading the page by a few different methods:

    • Regular AJAX requests to check the status of the task, updating the UI based on the response. This would have a REST-style handler on the back end.
    • You can use WebSockets to do the same thing over a single connection.
    • You can use HTTP long-polling to simulate WebSocket-like behavior, but this has generally been supplanted by WebSockets.

    Either option will require both a handler to serve the status update information and some JavaScript wizardry on the front-end to call the handler, parse the response, and update the page.

    Side-Effects

    Depending on the scale and requirements of this service, there are some side-effects to consider; mainly that a long-running task is effectively a kind of application state, making your application stateful, which has some severe operational downsides when it comes to availability, scaling, and deployment. If you're running multiple load-balanced instances you'll have to use sticky sessions or share task status between instances somehow.

    The most common way to handle long-running tasks at scale is to separate the worker from the web application, using some kind of work queue (either in a database or a dedicated message broker like Rabbit or Kafka) to manage the tasks. This makes it a little more complicated to get status updates because you're working across processes, but it gives you a lot more flexibility operationally.

    I'm guessing this is a more complicated answer than you expected to "requests are timing out", but this is a case of a trivial issue with a non-trivial solution. You're certainly not alone in tackling this issue; researching handling long-running tasks in web applications will yield a ton of information you can leverage.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥20 matlab计算中误差
  • ¥15 对于相关问题的求解与代码
  • ¥15 ubuntu子系统密码忘记
  • ¥15 信号傅里叶变换在matlab上遇到的小问题请求帮助
  • ¥15 保护模式-系统加载-段寄存器
  • ¥15 电脑桌面设定一个区域禁止鼠标操作
  • ¥15 求NPF226060磁芯的详细资料
  • ¥15 使用R语言marginaleffects包进行边际效应图绘制
  • ¥20 usb设备兼容性问题
  • ¥15 错误(10048): “调用exui内部功能”库命令的参数“参数4”不能接受空数据。怎么解决啊