Golang go-workers自定义日志中间件？

Question

I'm building a Golang app that implements a Sidekiq-compatible jrallison/go-workers work queue and a custom logging wrapper around Sirupsen/logrus for marshaled JSON logs.

Now, all of my app (except for go-workers so far) uses my logger-wrapper in a central place to ensure that 100% of its output is JSON compatible.

Note that lines #1 and #2 are proper JSON from our central logger, but when go-workers initializes we see line #3 come from the wrong logger in plain text.

{"db":{"Mapper":{}},"instance_id":"1","level":"info","msg":"Db: Connected to MySQL","time":"2015-05-27T04:15:15-04:00"}
{"concurrency":10,"instance_id":"1","level":"info","msg":"Worker: Commencing work","time":"2015-05-27T04:15:15-04:00"}
workers: 2015/05/27 04:15:15.211217 processing queue contact with 10 workers.

And when we send the signal to close the program, we see first on line #1 the wrong logger in plain text, followed by the proper JSON from our central logger on line #2.

^C
workers: 2015/05/27 04:15:17.197504 quitting queue contact (waiting for 0 / 10 workers).
{"instance_id":"1","level":"info","msg":"Closed correctly","time":"2015-05-27T04:15:17-04:00"}

I cannot seem to get this custom MiddlewareLogging to replace the go-workers default logging middleware.

func (a *App) ConfigureWorkers() {
  workers.Middleware = workers.NewMiddleware(
    &WorkMiddleware{ App: a },
    )
}

type WorkMiddleware struct{
  App *App
}

func (m *WorkMiddleware) Call(queue string, message *workers.Msg, next func() bool) (acknowledge bool) {
  // before each message is processed:
  job_json := message.Args().GetIndex(0).MustString()
  job := ContactJob{}
  err := json.Unmarshal([]byte(job_json), &job)
  if err != nil {
  m.App.Log.WithFields( log.Fields{
    "job_json": job_json,
    }).Fatal("Worker: Could not Unmarshal job JSON")
    return
  }
  SetMessageJob(message, job)
  start := time.Now()
  m.App.Log.WithFields( log.Fields{
    "job_id": message.Jid(),
    "queue": queue,
    "args": message.Args(),
    }).Print("Work: Job Starting")
  defer func() {
    if e := recover(); e != nil {
      buf := make([]byte, 4096)
      buf = buf[:runtime.Stack(buf, false)]
      m.App.Log.WithFields( log.Fields{
          "job_id": message.Jid(),
          "queue": queue,
          "duration": time.Since(start),
          "error": e,
          "stack": buf,
        }).Fatal("Work: Job Failed")
    }
  }()
  acknowledge = next()
  result := GetMessageResult(message)
  m.App.Log.WithFields( log.Fields{
    "job_id": message.Jid(),
    "result": result,
    "queue": queue,
    "duration": time.Since(start),
    }).Print("Work: Job Done")
  return
}

Is it actually possible to replace the default go-workers logging middleware for those lines?

Answer 1

Quoting jrallison from github.com/jrallison/go-workers/issues/50:

It looks like you're replacing the worker logging middleware (which logs information about each and every message processed by your workers).

The only logging I see in your output is the processing/quitting log entries from the manager which are logged on startup and shutdown.

https://github.com/jrallison/go-workers/blob/571e6b3b7be959e99024ec12b83b2ad261b06ff7/manager.go#L47

If you'd like these to not be logged, you may want to replace workers.Logger with an object that meets the WorkersLogger interface, but doesn't actually log anything (... or perhaps for your use-case, logs it in JSON format):

Issue is resolved by my final code:

func (a *App) ConfigureWorkers() {
  workers.Middleware = workers.NewMiddleware( &WorkMiddleware{ App: a } )
  workers.Logger = &WorkersLogger{ App: a }
}

type WorkersLogger struct {
  App *App
}

func (l *WorkersLogger) Println(args ...interface{}) {
  l.App.Log.WithFields( log.Fields{
    "instanceId": l.App.Id,
    }).Println(args...)
}

func (l *WorkersLogger) Printf(fmt string, args ...interface{}) {
  l.App.Log.WithFields( log.Fields{
    "instanceId": l.App.Id,
    }).Printf(fmt, args...)
}