Flink SQL 1.13处理数组类型数据(一行变多行)
解析源数据中嵌套的数组,数据一行变多行。Flink 版本: 1.13。
·
Flink SQL 1.13处理数组类型数据(一行变多行)
Flink 版本: 1.13
需求:解析源数据中嵌套的数组。
Kafka消息样例:
{
"create_by": "charlie",
"create_date": "2022-08-08 18:24:23",
"data_list": [
{
"name": "zhangsan",
"age": 21
}
]
}
Kafka connector 参考:https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/connectors/table/kafka/
对应Flink SQL中的create table DDL为:
CREATE TABLE kafka_source (
create_by STRING,
create_date TIMESTAMP(0),
data_list ARRAY<ROW<
name STRING,
age INT
>>
) WITH (
'connector' = 'kafka',
'topic' = 'kafka-topic-name',
'properties.group.id' = 'flink-test',
'properties.bootstrap.servers' ='localhost:9092',
'format'='json',
'scan.startup.mode'='earliest-offset'
);
参考 https://blog.csdn.net/xianpanjia4616/article/details/114811852
然后新建一个view来处理嵌套的数组,展开数组:
create view view_fields as
select
name,
age
from kafka_source, UNNEST(data_list) as T(
name,
age
);
更多推荐
已为社区贡献1条内容
所有评论(0)