吴祥平
408663c42b
[HUDI-3912] Fix lose data when rollback in flink async compact ( #5357 )
...
* stop add event when has failed compact event
Co-authored-by: wxp <wxp4532@outlook.com >
2022-04-20 19:23:39 +08:00
Danny Chan
7a9e411e9d
[HUDI-3917] Flink write task hangs if last checkpoint has no data input ( #5360 )
2022-04-20 12:48:24 +08:00
董可伦
b8e465fdfc
[MINOR] Fix typos in log4j-surefire.properties ( #5212 )
2022-04-15 13:33:37 -07:00
Danny Chan
6f9b02decb
[HUDI-3870] Add timeout rollback for flink online compaction ( #5314 )
2022-04-13 20:05:48 +08:00
Danny Chan
0281725c6b
[MINOR] Inline the partition path logic into the builder ( #5310 )
2022-04-13 16:54:39 +05:30
Danny Chan
43de2b4702
[HUDI-3868] Disable the sort input for flink streaming append mode ( #5309 )
2022-04-13 14:21:08 +08:00
Sagar Sumit
df87095ef0
[HUDI-3454] Fix partition name in all code paths for LogRecordScanner ( #5252 )
...
* Depend on FSUtils#getRelativePartitionPath(basePath, logFilePath.getParent)
to get the partition.
* If the list of log file paths in the split is empty, then fallback to usual behaviour.
2022-04-08 09:59:36 +05:30
xiarixiaoyao
531381faff
[HUDI-3096] fixed the bug that the cow table(contains decimalType) write by flink cannot be read by spark. ( #4421 )
2022-04-07 17:21:25 +08:00
Danny Chan
e33149be9a
[HUDI-3808] Flink bulk_insert timestamp(3) can not be read by Spark ( #5236 )
2022-04-07 15:17:39 +08:00
Raymond Xu
e96f08f355
Moving to 0.12.0-SNAPSHOT on master branch.
2022-04-06 15:24:10 +08:00
todd5167
eef3f9c74a
[HUDI-3771] flink supports sync table information to aws glue ( #5202 )
2022-04-02 21:16:10 +08:00
Bo Cui
17d11f4839
[MINOR] Repeated execution of update status ( #5089 )
2022-03-30 17:30:06 -04:00
Danny Chan
b9fbada2f2
[minor] Follow 3178, fix the flink metadata table compaction ( #5175 )
2022-03-30 20:45:29 +08:00
Danny Chan
5c1b482a1b
[HUDI-3741] Fix flink bucket index bulk insert generates too many small files ( #5164 )
2022-03-30 08:18:36 +08:00
Danny Chan
3bf9c5ffe8
[HUDI-3728] Set the sort operator parallelism for flink bucket bulk insert ( #5154 )
2022-03-29 09:52:35 +08:00
Shawy Geng
2e2d08cb72
[HUDI-3539] Flink bucket index bucketID bootstrap optimization. ( #5093 )
...
* [HUDI-3539] Flink bucket index bucketID bootstrap optimization.
Co-authored-by: gengxiaoyu <gengxiaoyu@bytedance.com >
2022-03-28 19:50:36 +08:00
Danny Chan
4d940bbf8a
[HUDI-3716] OOM occurred when use bulk_insert cow table with flink BUCKET index ( #5135 )
2022-03-27 09:13:58 +08:00
Zhaojing Yu
483ee843e6
[HUDI-3703] Reset taskID in restoreWriteMetadata ( #5122 )
2022-03-25 10:18:28 +08:00
Danny Chan
5e86cdd1e9
[HUDI-3701] Flink bulk_insert support bucket hash index ( #5118 )
2022-03-25 09:01:42 +08:00
Danny Chan
a1c42fcc07
[minor] Checks the data block type for archived timeline ( #5106 )
2022-03-24 14:10:43 +08:00
wxp4532
26e5d2e6fc
[HUDI-3559] Flink bucket index with COW table throws NoSuchElementException
...
Actually method FlinkWriteHelper#deduplicateRecords does not guarantee the records sequence, but there is a
implicit constraint: all the records in one bucket should have the same bucket type(instant time here),
the BucketStreamWriteFunction breaks the rule and fails to comply with this constraint.
close apache/hudi#5018
2022-03-21 17:34:54 +08:00
Danny Chan
799c78e688
[HUDI-3665] Support flink multiple versions ( #5072 )
2022-03-21 10:34:50 +08:00
