在Google App Engine上从main.go传递出去后，上下文值发生了变化

Question

There's a problem where I don't know why a context.Context was changed once I pass it to a different package on Google App Engine.

The following code works fine when running on App Engine:

package main

import (
    "net/http"
    "log"

    "google.golang.org/appengine"
)

func main() {
    http.HandleFunc("/", myHandler)

    appengine.Main()
}

func myHandler(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()
    account, err := appengine.ServiceAccount(ctx)
    if err != nil {
        log.Println("[myHandler] error:", err)
    } else {
        log.Println("[myHandler] ServiceAccount:", account)
    }   

    w.Write([]byte("ok"))
}

I could retrieve the ServiceAccount successfully when accessing /, and everything was good.

However, when I passed the context from main.go to another package, the function call didn't work. The following was added to main.go:

import (
    // other stuff
    "github.com/adlerhsieh/q_context/handlers"
)

func main() {
    http.HandleFunc("/", myHandler)

    appengine.Main()
}

func myHandler(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()
    account, err := appengine.ServiceAccount(ctx)
    if err != nil {
        log.Println("[myHandler] error:", err)
    } else {
        log.Println("[myHandler] ServiceAccount:", account)
    }   

    handlers.AnotherFunc(ctx) // <--- added this

    w.Write([]byte("ok"))
}

Another package:

package handlers

import (
    "log"
    "context"

    "google.golang.org/appengine"
)

func AnotherFunc(ctx context.Context) {
    account, err := appengine.ServiceAccount(ctx)
    if err != nil {
        log.Println("[AnotherFunc] error:", err)
    } else {
        log.Println("[AnotherFunc] ServiceAccount:", account)
    }
}

When I ran it on App Engine, the log said:

2019/09/04 09:36:30 [myHandler] ServiceAccount: [e-mail]
2019/09/04 09:36:30 [AnotherFunc] error: not an App Engine context

The function calls are the same, but just in different packages. I dug in the package itself and found that it uses the key here (which leads to here) to setup the context. And here to check whether that value was setup properly. However, that value seem to be modified/changed so that the second function call couldn't get it. Even if I omitted the first function call and went straight to the second one, it still has the same error.

Any idea why context object was modified when passing to another package?

The following is my app.yaml:

runtime: go111
service: default
instance_class: F1
automatic_scaling:
  min_idle_instances: 0
  max_idle_instances: automatic
  min_pending_latency: automatic
  max_pending_latency: automatic
  max_concurrent_requests: 30

handlers:
- url: /.*
  script: auto
  login: admin

nobuild_files:
- vendor

env_variables:
  ENV: 'dev'
  GO111MODULE: 'off'

Here is the GitHub repo link.

Thank you!

Answer 1

It turns out that my code actually worked. It's because of some other operation error.

However, I'll just post the issue that actually caused it so it can help those who have the same issue.

With the new go111 runtime, it treats packages from non-root directory or its subdirectories as a different type of package. This caused the problem with "not an App Engine context". I'll just call it an "outcast" package for now (cause I'm not entirely sure why's that).

For example:

- appengine
  - main.go
  - handlers
    - handlers.go <-- this is a regular package

- appengine
  - main.go
- handlers
  - handlers.go < -- this is an outcast package

An outcast package would have issues handling context.Context generated from App Engine, as pointed out in my question.

The mechanism of App Engine knowing that the context is created from App Engine, is using a built-in value that can only be retrieved from its internal package (with an un-exported pointer-string key). When passing the context to an outcast package, we can no longer retrieve the value from the context. It's still a mystery for me that why the value disappeared, but it's probably because of some Go compiling mechanism.

The solution would be moving the main.go to the top-level directory in the project, so that there would be no outcast package anywhere.