pengzhiwei
aacb8be521
[HUDI-1415] Read Hoodie Table As Spark DataSource Table ( #2283 )
2021-04-20 14:21:38 -07:00
Danny Chan
ab4a7b0b4a
[HUDI-1788] Insert overwrite (table) for Flink writer ( #2808 )
...
Supports `INSERT OVERWRITE` and `INSERT OVERWRITE TABLE` for Flink
writer.
2021-04-14 10:23:37 +08:00
pengzhiwei
684622c7c9
[HUDI-1591] Implement Spark's FileIndex for Hudi to support queries via Hudi DataSource using non-globbed table path and partition pruning ( #2651 )
2021-04-01 11:12:28 -07:00
garyli1019
6e803e08b1
Moving to 0.9.0-SNAPSHOT on master branch.
2021-03-24 21:37:14 +08:00
Volodymyr Burenin
900de34e45
[HUDI-1650] Custom avro kafka deserializer. ( #2619 )
...
* Custom avro kafka deserializer
Co-authored-by: volodymyr.burenin <volodymyr.burenin@cloudkitchens.com >
Co-authored-by: Sivabalan Narayanan <sivabala@uber.com >
2021-03-20 00:51:08 -07:00
Sivabalan Narayanan
b038623ed3
[HUDI 1615] Fixing null schema in bulk_insert row writer path ( #2653 )
...
* [HUDI-1615] Avoid passing in null schema from row writing/deltastreamer
* Fixing null schema in bulk insert row writer path
* Fixing tests
Co-authored-by: vc <vinoth@apache.org >
2021-03-16 09:44:11 -07:00
liujinhui
8c2197ae5e
[HUDI-1269] Make whether the failure of connect hive affects hudi ingest process configurable ( #2443 )
...
Co-authored-by: Sivabalan Narayanan <sivabala@uber.com >
2021-02-25 10:09:32 -05:00
n3nash
ffcfb58bac
[HUDI-1486] Remove inline inflight rollback in hoodie writer ( #2359 )
...
1. Refactor rollback and move cleaning failed commits logic into cleaner
2. Introduce hoodie heartbeat to ascertain failed commits
3. Fix test cases
2021-02-19 20:12:22 -08:00
Sivabalan Narayanan
c9fcf964b2
[HUDI-1315] Adding builder for HoodieTableMetaClient initialization ( #2534 )
2021-02-20 09:54:26 +08:00
teeyog
26da4f5462
[HUDI-1526] Translate the api partitionBy in spark datasource to hoodie.datasource.write.partitionpath.field ( #2431 )
2021-02-10 12:07:54 -05:00
pengzhiwei
0d8a4d0a56
[HUDI-1550] Honor ordering field for MOR Spark datasource reader ( #2497 )
2021-02-01 21:04:27 +08:00
Vinoth Chandar
3719e7b388
Moving to 0.8.0-SNAPSHOT on master branch.
2021-01-20 11:31:22 -08:00
Sivabalan Narayanan
b9c2856d16
[HUDI-1535] Fix 0.7.0 snapshot ( #2456 )
...
* Revert "[MINOR] Bumping snapshot version to 0.7.0 (#2435 )"
This reverts commit a43e191d6c
2021-01-19 12:20:43 -08:00
Sivabalan Narayanan
a43e191d6c
[MINOR] Bumping snapshot version to 0.7.0 ( #2435 )
2021-01-16 09:56:28 -05:00
Gary Li
c5e8a024f6
[HUDI-1418] Set up flink client unit test infra ( #2281 )
2020-12-31 08:57:22 +08:00
wenningd
286055ce34
[HUDI-1451] Support bulk insert v2 with Spark 3.0.0 ( #2328 )
...
Co-authored-by: Wenning Ding <wenningd@amazon.com >
- Added support for bulk insert v2 with datasource v2 api in Spark 3.0.0.
2020-12-25 09:43:34 -05:00
Sivabalan Narayanan
33d338f392
[HUDI-115] Adding DefaultHoodieRecordPayload to honor ordering with combineAndGetUpdateValue ( #2311 )
...
* Added ability to pass in `properties` to payload methods, so they can perform table/record specific merges
* Added default methods so existing payload classes are backwards compatible.
* Adding DefaultHoodiePayload to honor ordering while merging two records
* Fixing default payload based on feedback
2020-12-19 19:19:42 -08:00
wenningd
fce1453fa6
[HUDI-1040] Make Hudi support Spark 3 ( #2208 )
...
* Fix flaky MOR unit test
* Update Spark APIs to make it be compatible with both spark2 & spark3
* Refactor bulk insert v2 part to make Hudi be able to compile with Spark3
* Add spark3 profile to handle fasterxml & spark version
* Create hudi-spark-common module & refactor hudi-spark related modules
Co-authored-by: Wenning Ding <wenningd@amazon.com >
2020-12-09 15:52:23 -08:00
