作者:FF小小女人 | 来源:互联网 | 2022-12-10 13:12
我正在尝试使用内置的数据流模板从PUBSUB加载Big Query中的以下数据:
{
"current_speed": "19.09",
"_east": "-87.654561",
"_last_updt": "2018-07-17 15:50:54.0",
"_region_id": "1",
"_north": "42.026444",
"_south": "41.997946",
"region": "Rogers Park - West Ridge",
"_west": "-87.709645",
"_description": "North of Devon. Kedzie to Lake Shore"
}
但我一直收到这个错误:
"读取数据时出错,错误消息:无法解析JSON:字符串意外结束;字符串意外结束;预期键"
我实际上需要加载更大的数据集,如下所示:
[{
"current_speed": "19.09",
"_east": "-87.654561",
"_last_updt": "2018-07-17 15:50:54.0",
"_region_id": "1",
"_north": "42.026444",
"_south": "41.997946",
"region": "Rogers Park - West Ridge",
"_west": "-87.709645",
"_description": "North of Devon. Kedzie to Lake Shore"
}, {
"current_speed": "25.23",
"_east": "-87.747456",
"_last_updt": "2018-07-17 15:50:54.0",
"_region_id": "2",
"_north": "42.0190998",
"_south": "41.960669",
"region": "Far North West",
"_west": "-87.84621",
"_description": "North of Montrose. East River to Cicero"
}
]
但是我得到了这个错误:
读取数据时出错,错误消息:无法解析JSON:启动新阵列时找不到对象.BeginArray返回false; 解析器在字符串结束之前终止
我在这做错了什么?
1> 小智..:
是的,BigQuery只接受换行符分隔的JSON,这意味着每行一个完整的JSON对象.在将对象合并到一行之前,BigQuery读取"{",它是对象的开头,并期望读取一个键,但行结束,因此您会看到错误消息"expected key".
对于多个JSON对象,只需将它们放在每一行中.不要将它们包含在数组中.BigQuery期望每一行都以一个对象"{"开头.如果您将"["作为第一个字符,您将看到第二条错误消息,这意味着BigQuery读取数组但不读取对象内部.
2> Felipe Hoffa..:
要将JSON转换为新行分隔的JSON(BigQuery提取的格式),可以使用jq
:
$ cat a.json
[{
"key01": "value01",
"key02": "value02",
"keyN": "valueN"
},
{
"key01": "value01",
"key02": "value02",
"keyN": "valueN"
},
{
"key01": "value01",
"key02": "value02",
"keyN": "valueN"
}
]
$ cat a.json | jq -c '.[]'
{"key01":"value01","key02":"value02","keyN":"valueN"}
{"key01":"value01","key02":"value02","keyN":"valueN"}
{"key01":"value01","key02":"value02","keyN":"valueN"}
(请参阅/sf/ask/17360801/)