1
0

[HUDI-1296] Support Metadata Table in Spark Datasource (#4789)

* Bootstrapping initial support for Metadata Table in Spark Datasource

- Consolidated Avro/Row conversion utilities to center around Spark's AvroDeserializer ; removed duplication
- Bootstrapped HoodieBaseRelation
- Updated HoodieMergeOnReadRDD to be able to handle Metadata Table
- Modified MOR relations to be able to read different Base File formats (Parquet, HFile)
This commit is contained in:
Alexey Kudinkin
2022-02-24 13:23:13 -08:00
committed by GitHub
parent 521338b4d9
commit 85e8a5c4de
56 changed files with 1634 additions and 1010 deletions

View File

@@ -20,7 +20,43 @@
"type": "record",
"name": "User",
"fields": [
{"name": "field1", "type": ["null", "string"], "default": null},
{"name": "createTime", "type": ["null", "long"], "default": null}
{
"name": "field1",
"type": [
"null",
"string"
],
"default": null
},
{
"name": "createTime",
"type": [
"null",
"long"
],
"default": null
},
{
"name": "createTimeString",
"type": [
"null",
"string"
],
"default": null
},
{
"name": "createTimeDecimal",
"type": [
"null",
{
"name": "decimalFixed",
"type": "fixed",
"logicalType": "decimal",
"precision": 20,
"scale": 4,
"size": 10
}
]
}
]
}