lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

[HUDI-1902] Global index for flink writer (#2958)

Browse Source 
Supports deduplication for record keys with different partition path.
This commit is contained in:
Danny Chan
2021-05-18 13:55:38 +08:00
committed by GitHub
parent fcedbfcb58
commit 46a2399a45
10 changed files with 345 additions and 89 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
hudi-flink/src/test/resources/test_source_4.data Normal file
View File

@@ -0,0 +1,8 @@
{"uuid": "id1", "name": "Danny", "age": 24, "ts": "1970-01-01T00:00:01", "partition": "par1"}
{"uuid": "id1", "name": "Stephen", "age": 34, "ts": "1970-01-01T00:00:02", "partition": "par1"}
{"uuid": "id1", "name": "Julian", "age": 54, "ts": "1970-01-01T00:00:03", "partition": "par2"}
{"uuid": "id1", "name": "Fabian", "age": 32, "ts": "1970-01-01T00:00:04", "partition": "par2"}
{"uuid": "id1", "name": "Sophia", "age": 18, "ts": "1970-01-01T00:00:05", "partition": "par3"}
{"uuid": "id1", "name": "Jane", "age": 19, "ts": "1970-01-01T00:00:06", "partition": "par3"}
{"uuid": "id1", "name": "Ella", "age": 38, "ts": "1970-01-01T00:00:07", "partition": "par4"}
{"uuid": "id1", "name": "Phoebe", "age": 52, "ts": "1970-01-01T00:00:08", "partition": "par4"}