Commit Graph

Select branches

Hide Pull Requests

0.13.0

Mono Color

• a62a6cff32 [MINOR] Refactor hive sync tool to reduce duplicate code (#3276) vinoyang 2021-07-15 23:54:38 +08:00
• 23a4a96eb4 [HUDI-2153] Fix BucketAssignFunction Context NullPointerException moranyuwen 2021-07-14 17:38:34 +08:00
• d024439764 [HUDI-2029] Implement compression for DiskBasedMap in Spillable Map (#3128) rmahindra123 2021-07-14 19:57:38 -07:00
• 75040ee9e5 [HUDI-2149] Ensure and Audit docs for every configuration class in the codebase (#3272) vinoth chandar 2021-07-14 10:56:08 -07:00
• c1810f210e [MINOR] Correct the logs of enable/not-enable async cleaner service. (#3271) zhangyue19921010 2021-07-15 00:08:29 +08:00
• 2debb9b3ed [HUDI-1828] Update unit tests to support ORC as the base file format (#3237) Jintao Guan 2021-07-14 09:05:42 -07:00
• 93967404a7 [HUDI-2180] Fix Compile Error For Spark3 (#3274) pengzhiwei 2021-07-15 00:02:28 +08:00
• 52524b659d [HUDI-2165] Support Transformer for HoodieFlinkStreamer (#3270) vinoyang 2021-07-14 23:01:52 +08:00
• 632bfd1a65 Merge pull request #3268 from yuzhaojing/HUDI-2171 Danny Chan 2021-07-14 17:01:30 +08:00
• ac75bda929 [HUDI-1969] Support reading logs for MOR Hive rt table (#3033) Danny Chan 2021-07-14 14:43:30 +08:00
• f0a2f378ea Merge pull request #3120 from pengzhiwei2018/dev_metasync pengzhiwei 2021-07-13 22:37:20 +08:00
• 7395a56dfb [HUDI-2168] Fix for AccessControlException for anonymous user (#3264) Vinay Patil 2021-07-13 18:26:51 +05:30
• aff1a1ed29 [HUDI-2171] Add parallelism conf for bootstrap operator 喻兆靖 2021-07-13 17:55:12 +08:00
• b0089b894a [MINOR] Fix EXTERNAL_RECORD_AND_SCHEMA_TRANSFORMATION config (#3250) Sagar Sumit 2021-07-13 09:54:40 +05:30
• c8a2033c27 [HUDI-2144]Bug-Fix:Offline clustering(HoodieClusteringJob) will cause insert action losing data (#3240) zhangyue19921010 2021-07-13 09:14:17 +08:00
• ca440ccf88 [HUDI-2107] Support Read Log Only MOR Table For Spark (#3193) pengzhiwei 2021-07-12 17:31:23 +08:00
• ffa934182a [HUDI-2045] Support Read Hoodie As DataSource Table For Flink And DeltaStreamer pengzhiwei 2021-06-21 14:13:25 +08:00
• 5804ad8e32 [HUDI-1483] Support async clustering for deltastreamer and Spark streaming (#3142) Sagar Sumit 2021-07-12 00:13:38 +05:30
• 9b01d2a045 [HUDI-2142] Support setting bucket assign parallelism for flink write task (#3239) swuferhong 2021-07-10 15:43:36 +08:00
• 942a024e74 [HUDI-2143] Tweak the default compaction target IO to 500GB when flink async compaction is off (#3238) Danny Chan 2021-07-10 15:40:30 +08:00
• 783c9cb369 [HUDI-2087] Support Append only in Flink stream (#3252) yuzhaojing 2021-07-10 14:49:35 +08:00
• 7c6eebf98c [MINOR] Fix some wrong assert reasons (#3248) vinoyang 2021-07-10 14:35:40 +08:00
• 3b2a4f2b6b [HUDI-2147] Remove unused class AvroConvertor in hudi-flink (#3243) wangxianghu 2021-07-10 10:16:33 +08:00
• b4562e86e4 Revert "[HUDI-2087] Support Append only in Flink stream (#3174)" (#3251) vinoth chandar 2021-07-09 11:20:09 -07:00
• 371526789d [HUDI-2087] Support Append only in Flink stream (#3174) yuzhaojing 2021-07-09 16:06:32 +08:00
• 047d956e01 [HUDI-2136] Fix conflict when flink-sql-connector-hive and hudi-flink-bundle are both in flink lib (#3227) swuferhong 2021-07-09 10:10:21 +08:00
• c50c24908a [MINOR] Fix build broken from #3186 (#3245) vinoth chandar 2021-07-08 14:23:52 -07:00
• de07e61382 [HUDI-2099]hive lock which state is WATING should be released, otherwise this hive lock will be locked forever (#3186) xiarixiaoyao 2021-07-08 22:30:48 +08:00
• 8c0dbaa9b3 [HUDI-2009] Fixing extra commit metadata in row writer path (#3075) Sivabalan Narayanan 2021-07-08 03:07:27 -04:00
• 1d3cd06572 [HUDI-2134]Add generics to avoif forced conversion in BaseSparkCommitActionExecutor#partition (#3232) Yungthuis 2021-07-08 13:31:38 +08:00
• 16e90d30ea [HUDI-1105] Adding dedup support for Bulk Insert w/ Rows (#2206) Sivabalan Narayanan 2021-07-07 17:38:26 -04:00
• 8f7ad8b178 [HUDI-2069] Refactored String constants (#3172) Sebastian Bernauer 2021-07-07 20:22:00 +02:00
• ea9e5d0e8b [HUDI-1104] Adding support for UserDefinedPartitioners and SortModes to BulkInsert with Rows (#3149) Sivabalan Narayanan 2021-07-07 11:15:25 -04:00
• 55ecbc662e [HUDI-2115] FileSlices in the filegroup is not descending by timestamp (#3206) Shawy Geng 2021-07-07 22:24:36 +08:00
• 990820476a [HUDI-2140] Fixed the unit test TestHoodieBackedMetadata.testOnlyValidPartitionsAdded. (#3234) Prashant Wason 2021-07-06 23:50:27 -07:00
• 221ddd9bf3 [HUDI-2016] Fixed bootstrap of Metadata Table when some actions are in progress. (#3083) Prashant Wason 2021-07-06 08:08:46 -07:00
• 6e24434682 [HUDI-2113] Fix integration testing failure caused by sql results out of order (#3204) Shawy Geng 2021-07-06 15:35:12 +08:00
• f2621da32f [HUDI-2093] Fix empty avro schema path caused by duplicate parameters (#3177) wangxianghu 2021-07-06 15:14:30 +08:00
• 60e0254e67 [HUDI-1996] Adding functionality to allow the providing of basic auth creds for confluent cloud schema registry (#3097) Randal Boyle 2021-07-06 07:40:23 +01:00
• 2b21ae1775 [HUDI-2046] Loaded too many classes like sun/reflect/GeneratedSerializationConstructorAccessor in JVM metaspace (#3121) dwshmilyss 2021-07-06 14:36:55 +08:00
• 05d6e18190 [HUDI-2055] Added deltastreamer metric for time of lastSync (#3129) Sebastian Bernauer 2021-07-06 08:34:46 +02:00
• 1d6978cde4 [HUDI-2135] Add compaction schedule option for flink (#3226) Danny Chan 2021-07-06 14:11:20 +08:00
• a4dcbb5c5a [HUDI-2028] Implement RockDbBasedMap as an alternate to DiskBasedMap in ExternalSpillableMap (#3194) rmahindra123 2021-07-05 23:03:41 -07:00
• a0f598d371 [HUDI-2089]fix the bug that metatable cannot support non_partition table (#3182) xiarixiaoyao 2021-07-06 11:14:05 +08:00
• 0bd20827ab [HUDI-2133] Support hive1 metadata sync for flink writer (#3225) swuferhong 2021-07-06 11:01:57 +08:00
• bc313727e3 [HUDI-2106] Fix flink batch compaction bug while user don't set compaction tasks (#3192) swuferhong 2021-07-06 09:10:37 +08:00
• 32bd8ce088 [HUDI-2132] Make coordinator events as POJO for efficient serialization (#3223) Danny Chan 2021-07-06 09:02:38 +08:00
• 650c4455c6 [HUDI-2122] Improvement in packaging insert into smallfiles (#3213) wangxianghu 2021-07-06 00:30:57 +08:00
• 287d2dd79c [HUDI-2131] Exception Throw Out When MergeInto With Decimal Type Field (#3224) pengzhiwei 2021-07-05 22:28:57 +08:00
• e6ee7bdb51 [HUDI-2129] StreamerUtil.medianInstantTime should return a valid date time string (#3221) Danny Chan 2021-07-05 20:56:24 +08:00
• 2cecb75187 [HUDI-2058]support incremental query for insert_overwrite_table/insert_overwrite operation on cow table (#3139) xiarixiaoyao 2021-07-05 18:54:05 +08:00
• 2033d35dc3 [HUDI-2127] Initialize the maxMemorySizeInBytes in log scanner (#3220) Shawy Geng 2021-07-05 11:53:18 +08:00
• 98ec017bc8 [HUDI-2126] The coordinator send events to write function when there are no data for the checkpoint (#3219) Danny Chan 2021-07-05 11:34:18 +08:00
• 6a71412f78 [HUDI-2116] Support batch synchronization of partition datas to hive metastore to avoid oom problem (#3209) xiarixiaoyao 2021-07-04 22:30:36 +08:00
• 62a1ad8b3a [HUDI-1930] Bootstrap support configure KeyGenerator by type (#3170) wangxianghu 2021-07-03 20:27:37 +08:00
• 4f215e2938 [HUDI-2057] CTAS Generate An External Table When Create Managed Table (#3146) pengzhiwei 2021-07-03 15:55:36 +08:00
• 7173d1338a [HUDI-2124] A Grafana dashboard for HUDI. (#3216) Prashant Wason 2021-07-02 18:48:37 -07:00
• 70d9c2e747 [HUDI-2123] Exception When Merge With Null-Value Field (#3214) pengzhiwei 2021-07-02 22:46:52 +08:00
• d424fe6072 [HUDI-2121] Add operator uid for flink stateful operators (#3212) Danny Chan 2021-07-02 19:44:32 +08:00
• ac65189458 [HUDI-2114] Spark Query MOR Table Written By Flink Return Incorrect Timestamp Value (#3208) pengzhiwei 2021-07-02 17:39:57 +08:00
• 7462fdefc3 [HUDI-2112] Support reading pure logs file group for flink batch reader after compaction (#3202) Danny Chan 2021-07-02 16:29:22 +08:00
• 6403547431 [HUDI-2051] Enable Hive Sync When Spark Enable Hive Meta For Spark Sql (#3126) pengzhiwei 2021-07-02 16:08:36 +08:00
• 6eca06d074 [HUDI-2105] Compaction Failed For MergeInto MOR Table (#3190) pengzhiwei 2021-07-01 23:40:14 +08:00
• b376cefc3e [MINOR] Add Documentation to KEYGENERATOR_TYPE_PROP (#3196) wangxianghu 2021-07-01 18:48:59 +08:00
• b34d53fa9c [HUDI-2088] Missing Partition Fields And PreCombineField In Hoodie Properties For Table Written By Flink (#3171) pengzhiwei 2021-07-01 17:25:18 +08:00
• d07def1290 [MINOR] Fix broken build due to FlinkOptions (#3198) vinoth chandar 2021-06-30 20:34:58 -07:00
• 7895a3586e [MINOR] Update .asf.yaml to codify notification settings, turn on jira comments, gh discussions (#3164) vinoth chandar 2021-06-30 14:56:56 -07:00
• d412fb2fe6 [HUDI-89] Add configOption & refactor all configs based on that (#2833) wenningd 2021-06-30 14:26:30 -07:00
• 07e93de8b4 [HUDI-2052] Support load logFile in BootstrapFunction (#3134) yuzhaojing 2021-06-30 20:37:00 +08:00
• 94f0f40fec [HUDI-1944] Support Hudi to read from committed offset (#3175) Vinay Patil 2021-06-30 14:11:28 +05:30
• 1cbf43b6e7 [HUDI-2103] Add rebalance before index bootstrap (#3185) yuzhaojing 2021-06-30 16:40:55 +08:00
• 5564c7ec01 [HUDI-2006] Adding more yaml templates to test suite (#3073) Sivabalan Narayanan 2021-06-29 23:05:46 -04:00
• 202887b8ca [HUDI-2092] Fix NPE caused by FlinkStreamerConfig#writePartitionUrlEncode null value (#3176) wangxianghu 2021-06-30 09:21:06 +08:00
• f665db071f [HUDI-2085] Support specify compaction paralleism and compaction target io for flink batch compaction (#3169) swuferhong 2021-06-29 22:53:01 +08:00
• 5a7d1b3d6c [HUDI-2097] Fix Flink unable to read commit metadata error (#3180) swuferhong 2021-06-29 22:43:47 +08:00
• b8a8f572d6 [HUDI-2094] Supports hive style partitioning for flink writer (#3178) Danny Chan 2021-06-29 15:34:26 +08:00
• 0749cc826a [HUDI-2081] Move schema util tests out from TestHiveSyncTool (#3166) Raymond Xu 2021-06-28 20:23:46 -07:00
• 37b7c65d8a [HUDI-2084] Resend the uncommitted write metadata when start up (#3168) yuzhaojing 2021-06-29 08:53:52 +08:00
• 039aeb6dce [HUDI-1910] Commit Offset to Kafka after successful Hudi commit (#3092) Vinay Patil 2021-06-28 19:22:05 +05:30
• 34fc8a8880 [HUDI-2067] Sync FlinkOptions config to FlinkStreamerConfig (#3151) Vinay Patil 2021-06-28 16:56:08 +05:30
• 9e61dad597 [MINOR] Drop duplicate keygenerator class configuration setting (#3167) wangxianghu 2021-06-28 17:11:32 +08:00
• d24341d10c [HUDI-2074] Use while loop instead of recursive call in MergeOnReadInputFormat#MergeIterator to avoid StackOverflow (#3159) Danny Chan 2021-06-28 16:03:10 +08:00
• e99a6b031b [HUDI-2073] Fix the bug of hoodieClusteringJob never quit (#3157) zhangyue19921010 2021-06-27 13:03:41 +08:00
• f73bedd374 [MINOR] Remove unused methods (#3152) wangxianghu 2021-06-26 13:19:26 +08:00
• ed1a5daa9a [HUDI-2060] Added tests for KafkaOffsetGen (#3136) Vinay Patil 2021-06-25 22:07:47 +05:30
• 23dbc09a0d [MINOR] Removing un-used files and references (#3150) n3nash 2021-06-24 22:17:40 -07:00
• 0fb8556b0d Add ability to provide multi-region (global) data consistency across HMS in different regions (#2542) s-sanjay 2021-06-25 08:56:26 +05:30
• e64fe55054 [HUDI-2068] Skip the assign state for SmallFileAssign when the state can not assign initially (#3148) Danny Chan 2021-06-25 08:57:56 +08:00
• 218f2a6df8 [HUDI-2062] Catch FileNotFoundException in WriteProfiles #getCommitMetadata Safely (#3138) yuzhaojing 2021-06-25 08:54:59 +08:00
• b32855545b [HUDI-2069] Fix KafkaAvroSchemaDeserializer to not rely on reflection (#3111) Sebastian Bernauer 2021-06-24 15:08:21 +02:00
• 84dd3ca18b [HUDI-2053] Insert Static Partition With DateType Return Incorrect Partition Value (#3133) pengzhiwei 2021-06-24 19:09:37 +08:00
• 7e50f9a5a6 [HUDI-2061] Incorrect Schema Inference For Schema Evolved Table (#3137) pengzhiwei 2021-06-24 13:48:01 +08:00
• e039e0ff6d [HUDI-2064] Fix TestHoodieBackedMetadata#testOnlyValidPartitionsAdded (#3141) leesf 2021-06-24 07:37:55 +08:00
• 380518e232 [HUDI-2038] Support rollback inflight compaction instances for CompactionPlanOperator (#3105) yuzhaojing 2021-06-23 20:58:52 +08:00
• 43b9c1fa1c [HUDI-1826] Add ORC support in HoodieSnapshotExporter (#3130) Vaibhav Sinha 2021-06-23 14:34:25 +05:30
• 2687eab8f0 [HUDI-2054] Remove the duplicate name for flink write pipeline (#3135) Danny Chan 2021-06-23 14:49:38 +08:00
• 3fb59dda83 [HUDI-1988] FinalizeWrite() been executed twice in AbstractHoodieWriteClient$commitstats (#3050) swuferhong 2021-06-23 13:57:09 +08:00
• 11e64b2db0 [HUDI-1717] Metadata Reader should merge all the un-synced but complete instants from the dataset timeline. (#3082) Prashant Wason 2021-06-22 08:52:18 -07:00
• 062d5baf84 [HUDI-2013] Removed option to fallback to file listing when Metadata Table is enabled. (#3079) Prashant Wason 2021-06-22 08:41:52 -07:00
• 69c0d9e2d0 [HUDI-1883] Support Truncate Table For Hoodie (#3098) pengzhiwei 2021-06-22 22:33:20 +08:00