java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429) at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:765)
at org.apac
我想读取一个csv文件,并将其写入到BigQuery使用阿帕奇光束数据流。为此,我需要将数据以字典的形式呈现给BigQuery。如何使用apachebeam转换数据以实现此目的?我的输入csv文件有两列,我想在BigQuery中创建一个后续的两列的表。我知道如何在BigQuery中创建数据,这很简单,我不知道的是如何将csv转换成字典。# Standard imports
import apache_beam as beam
java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429) at java.util.concurrent.FutureTask.get(FutureTask.java:191) at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:765) at org.apache</em
我正在开发一个数据流管道,它正在从谷歌云存储读取协议的文件,并解析它,并试图写入到BigQuery表。当不是时,它工作得很好。行的大小约为20k,但如果不是。行数约为200k,则失败。阶段,并给出以下错误: Error message from worker: java.lang.RuntimeException: Failed to create job with prefix beam_bq_job_LOAD_testjobpackageorg.apache.beam.sdk.io.gcp.b
java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429) at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:803)
at org.apac
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:231)
at org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl.startJob(BigQueryServicesImpl.java:20