douzhongpi9734 2017-07-26 16:26
浏览 58
已采纳

在Golang Web应用中管理长流程

I'm trying to implement the following functionality in Golang.

I have a web page with a form, used to upload a .csv file. Gorilla mux is used to route to a handler which takes the file and parses it, does a bunch of operations on the data and at the end produces a report with number of lines parsed, # of rejected lines, etc.

My problem is that even though it works on my machine, on a server Apache will time out before I can get to the end of it all: the file upload itself isn't the issue, but I have to wait for transformations on the data to complete.

I've tried to use Gorilla websocket to get feedback from the process (incrementing number of lines parsed and treated, for instance) and keep the connection open, but this is a POST request, and Gorilla websocket won't upgrade from http to websocket unless one has a GET request.

I'm not even sure I'm on the right track with websockets for doing this type of thing.

I can have a goroutine for the processing itself and return the handler before the goroutine completes, but then how do I show the result of the process in the UI?

So at this stage my question boils down to: what would be the best way, in Golang, when you need to:

  • upload a file,
  • wait for a long process to complete
  • and display the result in a web page?

A clue as to the right direction to go in would be much appreciated.

  • 写回答

1条回答 默认 最新

  • dsajkdadsa14222 2017-07-27 13:36
    关注

    You've stumbled onto a non-trivial problem. There are a lot of possible solutions, with different user experiences, implementation complexities, and side effects. This is a pretty big topic so this answer is intended mostly as a starting point for further research.

    The Simplest Option

    First, pretty much regardless of solution, you're going to have to give each long-running task a unique ID that the browser can use to get status updates later. The task runner itself can just flag jobs as complete, or it can periodically issue progress updates if you want to present progress to the user.

    The easiest to implement is likely to have your form submission immediately respond with a page, with the task ID included in the URL, whose handler checks the task status and either a) returns a page with "still working" or something to that effect and auto-refreshes after a few seconds, or b) returns a page saying "completed" and does not refresh. This isn't terribly difficult to implement, but it's not particularly smooth, either. If this is a simple internal-use project with simple UX and operational requirements, I'd just do this. Otherwise, further down the rabbit hole we go!

    Live Updates

    You could do live updates without reloading the page by a few different methods:

    • Regular AJAX requests to check the status of the task, updating the UI based on the response. This would have a REST-style handler on the back end.
    • You can use WebSockets to do the same thing over a single connection.
    • You can use HTTP long-polling to simulate WebSocket-like behavior, but this has generally been supplanted by WebSockets.

    Either option will require both a handler to serve the status update information and some JavaScript wizardry on the front-end to call the handler, parse the response, and update the page.

    Side-Effects

    Depending on the scale and requirements of this service, there are some side-effects to consider; mainly that a long-running task is effectively a kind of application state, making your application stateful, which has some severe operational downsides when it comes to availability, scaling, and deployment. If you're running multiple load-balanced instances you'll have to use sticky sessions or share task status between instances somehow.

    The most common way to handle long-running tasks at scale is to separate the worker from the web application, using some kind of work queue (either in a database or a dedicated message broker like Rabbit or Kafka) to manage the tasks. This makes it a little more complicated to get status updates because you're working across processes, but it gives you a lot more flexibility operationally.

    I'm guessing this is a more complicated answer than you expected to "requests are timing out", but this is a case of a trivial issue with a non-trivial solution. You're certainly not alone in tackling this issue; researching handling long-running tasks in web applications will yield a ton of information you can leverage.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥15 在获取boss直聘的聊天的时候只能获取到前40条聊天数据
  • ¥20 关于URL获取的参数,无法执行二选一查询
  • ¥15 液位控制,当液位超过高限时常开触点59闭合,直到液位低于低限时,断开
  • ¥15 marlin编译错误,如何解决?
  • ¥15 有偿四位数,节约算法和扫描算法
  • ¥15 VUE项目怎么运行,系统打不开
  • ¥50 pointpillars等目标检测算法怎么融合注意力机制
  • ¥20 Vs code Mac系统 PHP Debug调试环境配置
  • ¥60 大一项目课,微信小程序
  • ¥15 求视频摘要youtube和ovp数据集