如何使用Go脚本在Google云存储中获取我的图片（base64）

Question

I have been looking for an example GAE script in go to get my image that I got from the resulted screenshot of PageSpeed Insights and saved it as json_decode object using Kohana/Cache to Google Cloud Storage (GCS).

The reason of this method is simply because I found this Kohana model is the most convenient way writing files to GCS, although I am seeking also other way like this to write files to GCS using Blobstore to serve them while the Go API Files has been deprecate as documented here.

Here is the form of stored object containing the screenshot image data (base64) which is saved as public in default application bucket with object name images/thumb/mythumb.jpg:

stdClass Object
(
    [screenshot] => stdClass Object
        (
            [data] => _9j_4AAQSkZJRgABAQAAAQABAAD_...= // base64 data
            [height] => 240
            [mime_type] => image/jpeg
            [width] => 320
        )

    [otherdata] => Array
        (
            [..] => ..
            [..] => ..
        )

)

I want to get this image that set as public using my customized url as below that to be proceed through go module and also I need it to be expired in a certain time because I have managed to update the image content itself regularly:

http://myappId.appspot.com/image/thumb/mythumb.jpg

I have set in disptach.yaml to send all image request to my go module as below:

- url: "*/images/*"
  module: go

and set the handler in go.yaml to proceed the image request as below:

handlers:
- url: /images/thumb/.*
  script: _go_app

- url: /images
  static_dir: images

Using this directive I have got that all /images/ request (other than /images/thumb/ request) serve images from the static directory and that /images/thumb/mythumb.jpg goes to the module application.

So left what code I have to use (see ????) in my application file named thumb.go as below:

package thumb

import(
    //what to import
    ????
    ????
)

const (
    googleAccessID            = "<serviceAccountEmail>@developer.gserviceaccount.com"
    serviceAccountPEMFilename = "YOUR_SERVICE_ACCOUNT_KEY.pem"
    bucket                    = "myappId.appspot.com"
)

var (
    expiration = time.Now().Add(time.Second * 60) //expire in 60 seconds
)

func init() {
    http.HandleFunc("/images/thumb/", handleThumb)
}

func handleThumb(w http.ResponseWriter, r *http.Request) {
    ctx := cloud.NewContext(appengine.AppID(c), hc)
    ???? //what code to get the string of 'mythumb.jpg' from url
    ???? //what code to get the image stored data from GCS
    ???? //what code to encoce base64 data
    w.Header().Set("Content-Type", "image/jpeg;")    
    fmt.Fprintf(w, "%v", mythumb.jpg)
}

I have taken many codes from some examples like this, this or this but could not get one works so far. I have also tried a sample from this which is almost close to my case but also found no luck.

So in generally t was mainly due to lack on what are the correct code to be put on the line that I marked by ???? as well the relevant library or path to be imported. I have also checked the GCS permission if something have been missing as described here and here.

I shall thank you much for your help and advise.

Answer 1

From what I've read in your description, it seems that the only relevant parts are the ???? lines in the actual Go code. Let me know if that's not the case.

First ????: "what code to get the string of 'mythumb.jpg' from url"?

From reading the code, you're looking to extract mythumb.jpg from a url like http://localhost/images/thumb/mythumb.jpg. A working example is available at the Writing Web Applications tutorial:

package main

import (
    "fmt"
    "net/http"
)

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
}

func main() {
    http.HandleFunc("/", handler)
    http.ListenAndServe(":8080", nil)
}

Such that

http://localhost:8080/monkeys

Prints

Hi there, I love monkeys!

Second ????: "what code to get the image stored data from GCS"?

The API method you're probably looking to use is storage.objects.get.

You did link to one of the JSON API Go Examples for Google Cloud Storage, which is a good general reference, but is not related to the problem you're trying to solve. That particular example is put together for Client-side applications (hence the redirectURL = "urn:ietf:wg:oauth:2.0:oob" line). Additionally, this sample uses deprecated/out-of-date oauth2 and storage packages.

One of the cleanest (and non-deprecated) ways to do this for an application which wants to access its own buckets on behalf of itself would be to use the golang/oauth2 and Google APIs Client Library for Go packages.

An example of how to authenticate with JSON Web Token auth with the golang/oauth2 package is available in the repo:

func ExampleJWTConfig() {
    conf := &jwt.Config{
        Email: "[e-mail]",
        // The contents of your RSA private key or your PEM file
        // that contains a private key.
        // If you have a p12 file instead, you
        // can use `openssl` to export the private key into a pem file.
        //
        //    $ openssl pkcs12 -in key.p12 -out key.pem -nodes
        //
        // It only supports PEM containers with no passphrase.
        PrivateKey: []byte("-----BEGIN RSA PRIVATE KEY-----..."),
        Subject:    "[e-mail]",
        TokenURL:   "https://provider.com/o/oauth2/token",
    }
    // Initiate an http.Client, the following GET request will be
    // authorized and authenticated on the behalf of [e-mail].
    client := conf.Client(oauth2.NoContext)
    client.Get("...")
}

Next, instead of using the oauth2 client directly, use that client with the Google APIs Client Library for Go mentioned earlier:

service, err := storage.New(client)
if err != nil {
    fatalf(service, "Failed to create service %v", err)
}

Notice the similarity to the out-of-date JSON API Go Examples?

In your handler, you'll want to go out and get the related object using func ObjectsService.Get. Assuming that you know the name of the object and bucket, that is.

Straight from the previous example, you can use code similar to what's below to retrieve the download link:

if res, err := service.Objects.Get(bucketName, objectName).Do(); err == nil {
    fmt.Printf("The media download link for %v/%v is %v.

", bucketName, res.Name, res.MediaLink)
} else {
    fatalf(service, "Failed to get %s/%s: %s.", bucketName, objectName, err)
}

Then, fetch the file, or do whatever you want with it. Full example:

import (
    "golang.org/x/oauth2"
    "golang.org/x/oauth2/jwt"
    "google.golang.org/api/storage/v1"
    "fmt"
)

...

const (
    bucketName = "YOUR_BUCKET_NAME"
    objectName = "mythumb.jpg"
)

func main() {
    conf := &jwt.Config{
        Email: "[e-mail]",
        PrivateKey: []byte("-----BEGIN RSA PRIVATE KEY-----..."),
        Subject:    "[e-mail]",
        TokenURL:   "https://provider.com/o/oauth2/token",
     }

    client := conf.Client(oauth2.NoContext)

    service, err := storage.New(client)
    if err != nil {
        fatalf(service, "Failed to create service %v", err)
    }

    if res, err := service.Objects.Get(bucketName, objectName).Do(); err == nil {
        fmt.Printf("The media download link for %v/%v is %v.

", bucketName, res.Name, res.MediaLink)
    } else {
        fatalf(service, "Failed to get %s/%s: %s.", bucketName, objectName, err)
    }

    // Go fetch the file, etc.
}

Third ????: "what code to encoce base64 data"?

Pretty simple with the encoding/base64 package. SO simple, that they've included an example:

package main

import (
    "encoding/base64"
    "fmt"
)

func main() {
    data := []byte("any + old & data")
    str := base64.StdEncoding.EncodeToString(data)
    fmt.Println(str)
}

Hope that helps.