无法使golang和包bigquery正常工作以加载到大查询

Question

I am trying to figure out how to get a simple bq load command to work with https://godoc.org/cloud.google.com/go/bigquery#Table.LoaderFrom

Running it manually it looks like this:

bq load --source_format=AVRO --ignore_unknown_values --replace=true mydataset.mytable gs://mybucket/table/*

And running it in my golang with exec.Command() successfully looks like this:

exec.Command("bq", "load", "--source_format=AVRO", "--ignore_unknown_values",
             "--replace=true", "mydataset.mytable",
             "gs://mybucket/table/*")

However, I cannot get this program to run without a segmentation fault when trying to get the load and job.wait to run successfully it seems to be getting a segmentation violation at the job.wait line of the program

package main

import (
    "context"
        "log"
        "cloud.google.com/go/bigquery"
)

func main(){
    ctx := context.Background()
    client, err := bigquery.NewClient(ctx, "my-project-id")
    if err != nil {
        // TODO: Handle error.
    }
    gcsRef := bigquery.NewGCSReference("gs://mybucket/table/*")
    gcsRef.SourceFormat = "AVRO"
    gcsRef.IgnoreUnknownValues = true
    // TODO: set other options on the GCSReference.
    ds := client.Dataset("mydataset")
    loader := ds.Table("mytable").LoaderFrom(gcsRef)
    // TODO: set other options on the Loader.
    job, err := loader.Run(ctx)
    if err != nil {
        // TODO: Handle error.
    }
    status, err := job.Wait(ctx) //seg faults right here
    if err != nil {
        // TODO: Handle error.
    }
    if status.Err() != nil {
        // TODO: Handle error.
    }
}

Answer 1

The panic is probably coming from a nil pointer reference to the job variable.

I would suggest including a log.Fatal(err)

In all of your err!= nil blocks.

This will help get you closer to why job is not being assigned correctly.

When you're writing one off scripts like this one in go log.Fatal is a great way to exit the program and print exactly what the issue is.

With go you're always trying to bubble errors up the stack to determine if the code should continue to execute, if things can be recovered, or if it's just a fatal thing and you should end the program.

For more info on the logging package checkout here: https://golang.org/pkg/log/

If you're just starting out learning go here are some awesome resources that can help give you ideas on how different types of programs can be designed.

https://github.com/dashpradeep99/https-github.com-miguellgt-books/tree/master/go

Best,

Christopher