dongyou9373 2015-05-22 11:42
浏览 49
已采纳

使用Golang将嵌套数据插入BigQuery

I can insert a flat object into BigQuery using Golang - how I can insert nested data into a table?

My BigQuery schema looks like this (from the example):

[{
    "name": "kind",
    "mode": "nullable",
    "type": "string"
  },
  {
    "name": "fullName",
    "type": "string",
    "mode": "required"
  },
  { "name": "visit",
    "type": "record",
    "mode": "repeated",
    "fields": [
    {
       "name": "time",
       "type": "timestamp",
       "mode": "nullable"
    },
    {
       "name": "duration",
       "type": "integer",
       "mode": "nullable"
    }
   ]
  }
]

My first attempt to insert looked like this (example):

func ExampleInsert(f string,) {

  jsonRow := make(map[string]bigquery.JsonValue)

  bq, _ := bigquery.New(client)
  request := new(bigquery.TableDataInsertAllRequest)

  rows := make([]*bigquery.TableDataInsertAllRequestRows, 1)

  jsonRow["kind"] = bigquery.JsonValue(kind)
  jsonRow["visit_duration"] = bigquery.JsonValue(duration)

  rows[i] = new(bigquery.TableDataInsertAllRequestRows)
  rows[i].Json = jsonRow

  bq.Tabledata.InsertAll(projectID, "visits", "visitsv4", request)
  ...
}

Which flattens and inserts without problems. I'm just using visit_duration

But, I need to loop through a slice and add to the visits record. I've tried to build a visit object (without a loop to test) and add that to the row but it's not inserting and I get no errors:

func ExampleInsert(f string,) {

  jsonRow := make(map[string]bigquery.JsonValue)

  bq, _ := bigquery.New(client)
  request := new(bigquery.TableDataInsertAllRequest)

  rows := make([]*bigquery.TableDataInsertAllRequestRows, 1)

  jsonRow["kind"] = bigquery.JsonValue(kind)

  visits := make([]*bigquery.TableDataInsertAllRequestRows, 1)

  jsonVisit := make(map[string]bigquery.JsonValue)
  jsonVisit["duration"] = rand.Intn(1000)
  visits[0] = new(bigquery.TableDataInsertAllRequestRows)
  visits[0].Json = jsonVisit

  jsonRow["visit"] = visits

  rows[i] = new(bigquery.TableDataInsertAllRequestRows)
  rows[i].Json = jsonRow

  bq.Tabledata.InsertAll(projectID, "visits", "visitsv4", request)

  _, err := Call.Do()
}

---[SOLUTION]----

As suggested in the comments, I have also tried creating a slice and then appending the visit:

var visits []bigquery.JsonValue
visit := make(map[string]bigquery.JsonValue)
visit["duration"] = rand.Intn(100)
visits = append(visits, visit)

jsonRow["visit"] = visits

I can confirm this does in fact work :) For those of you reading this, the reason it wasn't initially, even after adding a slice, was because I had copied the table. In doing so, I had also flattened the results. Watch out.

  • 写回答

1条回答 默认 最新

  • dqsw7529 2015-05-22 13:14
    关注

    Visits should be a slice of bigquery.JsonValue I am not sure why you have used: TableDataInsertAllRequestRows that should be only used one time for the payload descriptor.

    var visits []bigquery.JsonValue
    visit := make(map[string]bigquery.JsonValue)
    visit["duration"] = rand.Intn(100)
    visits = append(visits, visit)
    
    jsonRow["visit"] = visits
    

    ps. also make sure you have your schema flatten

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥15 寻一个支付宝扫码远程授权登录的软件助手app
  • ¥15 解riccati方程组
  • ¥15 display:none;样式在嵌套结构中的已设置了display样式的元素上不起作用?
  • ¥30 用arduino开发esp32控制ps2手柄一直报错
  • ¥15 使用rabbitMQ 消息队列作为url源进行多线程爬取时,总有几个url没有处理的问题。
  • ¥15 Ubuntu在安装序列比对软件STAR时出现报错如何解决
  • ¥50 树莓派安卓APK系统签名
  • ¥65 汇编语言除法溢出问题
  • ¥15 Visual Studio问题
  • ¥20 求一个html代码,有偿