Files

Aditya Tiwari 051ad0b033 [HUDI-3130] Fixing Hive getSchema for RT tables addressing different partitions having different schemas (#4468 )

* Fixing Hive getSchema for RT tables

* Addressing feedback

* temp diff

* fixing tests after spark datasource read support for metadata table is merged to master

* Adding multi-partition schema evolution tests to HoodieRealTimeRecordReader

Co-authored-by: Aditya Tiwari <aditya.tiwari@flipkart.com>
Co-authored-by: sivabalan <n.siva.b@gmail.com>

2022-03-06 07:51:35 +05:30

hudi-spark

[HUDI-3445] Support Clustering Command Based on Call Procedure Command for Spark SQL (#4901 )

2022-03-04 09:33:16 +08:00

hudi-spark2

[HUDI-1296] Support Metadata Table in Spark Datasource (#4789 )

2022-02-24 16:23:13 -05:00

hudi-spark2-common

[HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation (#4514 )

2022-01-14 13:42:35 +08:00

hudi-spark3

[HUDI-1296] Support Metadata Table in Spark Datasource (#4789 )

2022-02-24 16:23:13 -05:00

hudi-spark3-common

[HUDI-1296] Support Metadata Table in Spark Datasource (#4789 )

2022-02-24 16:23:13 -05:00

hudi-spark3.1.x

[HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation (#4514 )

2022-01-14 13:42:35 +08:00

hudi-spark-common

[HUDI-3130] Fixing Hive getSchema for RT tables addressing different partitions having different schemas (#4468 )

2022-03-06 07:51:35 +05:30

pom.xml

[HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation (#4514 )

2022-01-14 13:42:35 +08:00

README.md

[HUDI-3172] Refactor hudi existing modules to make more code reuse in V2 Implementation (#4514 )

2022-01-14 13:42:35 +08:00

README.md

Description of the relationship between each module

This repo contains the code that integrate Hudi with Spark. The repo is split into the following modules

hudi-spark hudi-spark2 hudi-spark3 hudi-spark3.1.x hudi-spark2-common hudi-spark3-common hudi-spark-common

hudi-spark is the module that contains the code that both spark2 & spark3 version would share, also contains the antlr4 file that supports spark sql on spark 2.x version.
hudi-spark2 is the module that contains the code that compatible with spark 2.x versions.
hudi-spark3 is the module that contains the code that compatible with spark 3.2.0(and above) versions。
hudi-spark3.1.x is the module that contains the code that compatible with spark3.1.x and spark3.0.x version.
hudi-spark2-common is the module that contains the code that would be reused between spark2.x versions, right now the module has no class since hudi only supports spark 2.4.4 version, and it acts as the placeholder when packaging hudi-spark-bundle module.
hudi-spark3-common is the module that contains the code that would be reused between spark3.x versions.
hudi-spark-common is the module that contains the code that would be reused between spark2.x and spark3.x versions.