doupaoshu8334 2019-05-02 10:22
浏览 35

bigquery表分区以编程方式使用php

I am trying to create partition on big-query when load data using php script but not getting right solution.

<?php
require_once '/v/vv/vv/common.php';
require_once '/vv/vv/vv/vendor/autoload.php';

use Google\Cloud\BigQuery\BigQueryClient;
use Google\Cloud\Core\ExponentialBackoff;
/** Uncomment and populate these variables in your code */
$projectId  = 'id';
$datasetId  = 'ids';

$bigQuery = new BigQueryClient([
   'projectId' => $projectId,
]);
$dataset = $bigQuery->dataset($datasetId);
$table_id='test_table';
$table = $dataset->table($table_id);
// create the import job
$schema = [
  'fields' => [
        ['name' => 'date', 'type' => 'Date','mode'=>'required'],
        ['name' => 'name', 'type' => 'string'],
        ['name' => 'post_abbr', 'type' => 'string']
  ],
 'timePartitioning'=>['type'=>'DAY','filed'=>'date']
]
  • 写回答

1条回答 默认 最新

  • doupu9251 2019-08-06 13:05
    关注

    You are mixing the schema definition with timePartitioning. Here is a snippet of code:

    use Google\Cloud\BigQuery\BigQueryClient;
    
    $source = 'miData.csv';
    $datasetId  = 'miDataset';
    $table_id='miTable';
    
    $bigQuery = new BigQueryClient();
    $dataset = $bigQuery->dataset($datasetId);
    $table = $dataset->table($table_id);
    
    $loadConfig = $table->load(fopen($source, 'r'));
    
    $loadConfig->sourceFormat('CSV');
    
    // create the import job
    $schema = [
      'fields' => [
            ['name' => 'miDate', 'type' => 'DATE','mode'=>'required'],
            ['name' => 'name', 'type' => 'STRING'],
            ['name' => 'post_abbr', 'type' => 'STRING']
      ]
    ];
    
    $loadConfig->schema($schema);
    
    $timePartitioning = ['type'=>'DAY','filed'=>'miDate'];
    $loadConfig->timePartitioning($timePartitioning);
    
    $loadConfig->createDisposition('CREATE_IF_NEEDED');
    $loadConfig->writeDisposition('WRITE_TRUNCATE');
    
    
    $job = $table->runJob($loadConfig);
    

    For more information about the available options for load jobs check the API documentation and the PHP loadJobConfiguration.

    评论

报告相同问题?

悬赏问题

  • ¥15 数学建模招标中位数问题
  • ¥15 phython路径名过长报错 不知道什么问题
  • ¥15 深度学习中模型转换该怎么实现
  • ¥15 HLs设计手写数字识别程序编译通不过
  • ¥15 Stata外部命令安装问题求帮助!
  • ¥15 从键盘随机输入A-H中的一串字符串,用七段数码管方法进行绘制。提交代码及运行截图。
  • ¥15 TYPCE母转母,插入认方向
  • ¥15 如何用python向钉钉机器人发送可以放大的图片?
  • ¥15 matlab(相关搜索:紧聚焦)
  • ¥15 基于51单片机的厨房煤气泄露检测报警系统设计