[HUDI-1296] Support Metadata Table in Spark Datasource (#4789)
* Bootstrapping initial support for Metadata Table in Spark Datasource - Consolidated Avro/Row conversion utilities to center around Spark's AvroDeserializer ; removed duplication - Bootstrapped HoodieBaseRelation - Updated HoodieMergeOnReadRDD to be able to handle Metadata Table - Modified MOR relations to be able to read different Base File formats (Parquet, HFile)
This commit is contained in:
@@ -20,7 +20,43 @@
|
||||
"type": "record",
|
||||
"name": "User",
|
||||
"fields": [
|
||||
{"name": "field1", "type": ["null", "string"], "default": null},
|
||||
{"name": "createTime", "type": ["null", "long"], "default": null}
|
||||
{
|
||||
"name": "field1",
|
||||
"type": [
|
||||
"null",
|
||||
"string"
|
||||
],
|
||||
"default": null
|
||||
},
|
||||
{
|
||||
"name": "createTime",
|
||||
"type": [
|
||||
"null",
|
||||
"long"
|
||||
],
|
||||
"default": null
|
||||
},
|
||||
{
|
||||
"name": "createTimeString",
|
||||
"type": [
|
||||
"null",
|
||||
"string"
|
||||
],
|
||||
"default": null
|
||||
},
|
||||
{
|
||||
"name": "createTimeDecimal",
|
||||
"type": [
|
||||
"null",
|
||||
{
|
||||
"name": "decimalFixed",
|
||||
"type": "fixed",
|
||||
"logicalType": "decimal",
|
||||
"precision": 20,
|
||||
"scale": 4,
|
||||
"size": 10
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user