Commit Graph

Select branches

Hide Pull Requests

0.13.0

Mono Color

• 62b392b49c [HUDI-1343] Add standard schema postprocessor which would rewrite the schema using spark-avro conversion (#2192) liujinhui 2020-12-04 11:28:34 +08:00
• 1f0d5c077e [HUDI-1349] spark sql support overwrite use insert_overwrite_table (#2196) lw0090 2020-12-04 04:26:21 +08:00
• 78fd122594 [HUDI-1196] Update HoodieKey when deduplicating records with global index (#2248) rmpifer 2020-12-01 13:50:46 -08:00
• ac23d2587f [HUDI-1357] Added a check to validate records are not lost during merges. (#2216) Prashant Wason 2020-12-01 13:44:57 -08:00
• b826c53e33 [HUDI-1373] Add Support for OpenJ9 JVM (#2231) Guy Khazma 2020-12-01 23:19:40 +02:00
• 36ce5bcd92 [HUDI-1424] Write Type changed to BULK_INSERT when set ENABLE_ROW_WRITER_OPT_KEY=true (#2289) pengzhiwei 2020-11-30 23:07:21 +08:00
• 3d5e9fee7f [MINOR] refactor code in HoodieMergeHandle (#2272) leesf 2020-11-28 21:47:05 +08:00
• 56866a11fe [HUDI-1392] lose partition info when using spark parameter basePath (#2243) steven zhang 2020-11-25 11:55:33 +08:00
• 0ebef1c0a0 [HUDI-1358] Fix leaks in DiskBasedMap and LazyFileIterable (#2249) Balaji Varadarajan 2020-11-23 10:56:26 -08:00
• 751e4ee882 [HUDI-1396] Fix for preventing bootstrap datasource jobs from hanging via spark-submit (#2253) wenningd 2020-11-23 10:43:24 -08:00
• d9411c38db [HUDI-1364] Add HoodieJavaEngineContext to hudi-java-client (#2222) Shen Hong 2020-11-24 02:06:28 +08:00
• 971f028aaf [HUDI-1393] Add compaction action in archive command (#2246) hongdd 2020-11-23 16:53:01 +08:00
• 537502a8ef [MINOR] Add apacheflink label (#2268) wangxianghu 2020-11-22 10:41:11 +08:00
• c8d5ea2752 [MINOR] clean up and add comments to flink client (#2261) Gary Li 2020-11-19 15:27:52 +08:00
• d7af8caa45 [HUDI-1384] Decoupling hive jdbc dependency when HIVE_USE_JDBC_OPT_KEY set false (#2241) pengzhiwei 2020-11-19 13:44:03 +08:00
• a23230c8c2 [HUDI-1400] Replace Operation enum with WriteOperationType (#2259) wangxianghu 2020-11-19 13:40:04 +08:00
• 4d05680038 [HUDI-1327] Introduce base implemetation of hudi-flink-client (#2176) wangxianghu 2020-11-18 17:57:11 +08:00
• 430d4b428e [HUDI-1377] remove duplicate code (#2235) Karl_Wang 2020-11-11 02:08:08 +08:00
• 42b6aeca28 [HUDI-1358] Fix Memory Leak in HoodieLogFormatWriter (#2217) Balaji Varadarajan 2020-11-09 19:26:13 -08:00
• 0364498ae3 [HUDI-1375] Fix bug in HoodieAvroUtils.removeMetadataFields() method (#2232) wenningd 2020-11-05 17:30:17 -08:00
• 33ec88fc38 [HUDI-1352] Add FileSystemView APIs to query pending clustering operations (#2202) satishkotha 2020-11-05 08:49:58 -08:00
• 5f5c15b0d9 [HUDI-892] RealtimeParquetInputFormat skip adding projection columns if there are no log files (#2190) lw0090 2020-11-03 12:00:12 +08:00
• d160abb437 [HUDI-912] Refactor and relocate KeyGenerator to support more engines (#2200) wangxianghu 2020-11-03 05:12:51 +08:00
• 59f995a3f5 Use RateLimiter instead of sleep. Repartition WriteStatus to optimize Hbase index writes (#1484) Venkatesh Rudraraju 2020-11-02 08:33:27 -08:00
• a205dd10fa [HUDI-1338] Adding Delete support to test suite framework (#2172) Sivabalan Narayanan 2020-11-01 00:15:41 -04:00
• 6310a2307a [HUDI-1351] Improvements to the hudi test suite for scalability and repeated testing. (#2197) Prashant Wason 2020-10-29 06:50:37 -07:00
• 736a940854 [HUDI-1274] Make hive synchronization supports hourly partition (#2122) liujinhui 2020-10-29 11:29:50 +08:00
• e109a61803 1. Fix merge on read DAG to make docker demo pass (#2092) n3nash 2020-10-28 19:34:26 -07:00
• e206ddd431 [MINOR] Private the NoArgsConstructor of SparkMergeHelper and code clean (#2194) wangxianghu 2020-10-26 12:22:11 +08:00
• 8545ea3856 [HUDI-1118] Cleanup rollback files residing in .hoodie folder (#2205) lw0090 2020-10-26 12:04:56 +08:00
• 49e855c348 [HUDI-1326] Added an API to force publish metrics and flush them. (#2152) Prashant Wason 2020-10-24 16:47:24 -07:00
• 14c4611857 [MINOR] Fix caller to SparkBulkInsertCommitActionExecutor (#2195) Raymond Xu 2020-10-21 19:50:10 -07:00
• 49407169ac [HUDI-1209] Properties File must be optional when running deltastreamer (#2085) Shen Hong 2020-10-22 08:49:28 +08:00
• e4931744eb [HUDI-1200] fixed NPE in CustomKeyGenerator (#2093) Pratyaksh Sharma 2020-10-21 12:06:25 +05:30
• af5ef4d49d [HUDI-1330] handle prefix filtering at directory level (#2157) Ho Tien Vu 2020-10-21 14:20:19 +08:00
• fd269ddeb0 [MINOR] Make sure factory method is used to instanciate DFSPathSelector (#2187) Ho Tien Vu 2020-10-20 17:52:31 +08:00
• 6490b029dd [HUDI-1345] Remove Hbase and htrace relocation from utilities bundle (#2185) Bhavani Sudha Saktheeswaran 2020-10-19 14:11:08 -07:00
• 4d80e1e221 [HUDI-284] add more test for UpdateSchemaEvolution (#2127) lw0090 2020-10-19 22:38:04 +08:00
• 35d406de40 [HUDI-1344] IBM Cloud Object Storage Support (#2182) Guy Khazma 2020-10-18 12:24:53 +03:00
• ec6267c303 [HUDI-307] add test to check timestamp date decimal type write and read consistent (#2177) lw0090 2020-10-18 17:18:50 +08:00
• a44f66869f [HUDI-1289] Remove relocation of pattern for hbase dependencies and add shading of guava in hadoop, spark, and presto bundles (#2147) rmpifer 2020-10-14 17:04:35 -07:00
• 7fa641ea9a [HUDI-1302] Add support for timestamp field in HiveSync (#2129) satishkotha 2020-10-13 22:58:00 -07:00
• c7d962efff [HUDI-1328] Introduce HoodieFlinkEngineContext to hudi-flink-client (#2161) wangxianghu 2020-10-14 09:30:49 +08:00
• b66c3ef23a [HUDI-1298] Add better error messages when IOException occurs during log file reading (#2133) lw0090 2020-10-13 15:45:10 +08:00
• 0d407342ef [HUDI-1304] Add unit test for testing compaction on replaced file groups (#2150) satishkotha 2020-10-12 16:48:29 -07:00
• c5e10d668f [HUDI-995] Migrate HoodieTestUtils APIs to HoodieTestTable (#2167) Raymond Xu 2020-10-11 23:39:10 -07:00
• c0472d3317 [HUDI-1184] Fix the support of hbase index partition path change (#1978) hj2016 2020-10-12 10:05:57 +08:00
• b58daf29ba [MINOR] remove unused generics type (#2163) dugenkui 2020-10-12 09:38:42 +08:00
• 2126f13e13 [HUDI-791] Replace null by Option in Delta Streamer (#2171) lw0090 2020-10-12 09:29:57 +08:00
• 032bc3b08f [MINOR] NPE Optimization for Option (#2158) dugenkui 2020-10-12 08:55:41 +08:00
• d4d4c8c899 [MINOR] Fix typo and others (#2164) dugenkui 2020-10-12 08:52:44 +08:00
• 86db4da33c [HUDI-1339] delete useless import in hudi-spark module (#2173) lw0090 2020-10-12 08:10:52 +08:00
• 585ce0094d [HUDI-1301] use spark INCREMENTAL mode query hudi dataset support schema version. (#2125) lw0090 2020-10-10 20:53:41 +08:00
• eafd7bf289 [MINOR] Fix wrong javadoc and refactor some naming issues (#2156) vinoyang 2020-10-10 06:09:26 +08:00
• 00271af64e [MINOR] Fix typo (#2159) dugenkui 2020-10-10 05:52:55 +08:00
• 1d1d91d444 [HUDI-995] Migrate HoodieTestUtils APIs to HoodieTestTable (#2143) Raymond Xu 2020-10-08 19:21:27 -07:00
• 788d236c44 [HUDI-1303] Some improvements for the HUDI Test Suite. (#2128) Prashant Wason 2020-10-07 05:33:51 -07:00
• 524193eb4b [HUDI-603]: DeltaStreamer can now fetch schema before every run in continuous mode (#1566) Pratyaksh Sharma 2020-10-07 09:04:03 +05:30
• fed01cd3c9 [MINOR] Update spark master default to yarn (#2148) rmpifer 2020-10-05 15:22:28 -07:00
• fdae388626 [HUDI-1203] add port configuration for EmbeddedTimelineService (#2142) lw0090 2020-10-06 02:36:54 +08:00
• b335459c80 [HUDI-1208] Ordering Field should be optional when precombine is turned off (#2088) Shen Hong 2020-10-05 02:34:21 +08:00
• 080ba3ed54 [HUDI-1199] relocated jetty in hudi-utilities-bundle pom (#1990) Pratyaksh Sharma 2020-10-04 23:52:01 +05:30
• 6c610b91ef [HUDI-1305] Added an API to shutdown and remove the metrics reporter. (#2132) Prashant Wason 2020-10-04 09:30:04 -07:00
• 1f7add9291 [HUDI-1089] Refactor hudi-client to support multi-engine (#1827) Mathieu 2020-10-02 05:25:29 +08:00
• 5aaaf8bff1 [MINOR] Change the log level of the dag scheduler for the test suite (#2134) vinoyang 2020-09-30 17:17:44 +08:00
• a99e93bed5 [HUDI-1072] Introduce REPLACE top level action. Implement insert_overwrite operation on top of replace action (#2048) satishkotha 2020-09-29 17:04:25 -07:00
• 32c9cad52c [HUDI-840] Avoid blank file created by HoodieLogFormatWriter (#1567) hongdd 2020-09-29 23:02:15 +08:00
• 20b9b399c9 [HUDI-1233] Deltastreamer Kafka consumption delay reporting indicators (#2074) liujinhui 2020-09-29 13:44:31 +08:00
• c0c0095fa9 [MINOR] Reformat prepare_integration_suite script (#2126) vinoyang 2020-09-29 05:12:57 +08:00
• a86f5574ed [HUDI-1192] Make create hive database automatically configurable (#1968) liujinhui 2020-09-27 14:10:13 +08:00
• b0f1b736f8 [MINOR] Fix checkstyle (#2117) leesf 2020-09-26 22:25:19 +08:00
• 1be0b06ef8 [HUDI-995] Migrate HoodieTestUtils APIs to HoodieTestTable (#2112) Raymond Xu 2020-09-26 06:21:47 -07:00
• ae68b2b355 [MINOR] fix typos (#2116) dugenkui 2020-09-26 20:40:33 +08:00
• 1dd6635fbb [MINOR] Fix ClassCastException when use QuickstartUtils generate data (#2105) Mathieu 2020-09-26 01:13:39 +08:00
• 2eaba0962a [HUDI-544] Archived commits command code cleanup (#1242) hongdd 2020-09-26 00:36:41 +08:00
• 6837118c21 [MINOR] Improve description (#2113) dugenkui 2020-09-25 22:21:37 +08:00
• 83d2e03cf7 [MINOR] Adding scripts to checkout and push to PRs (#2109) vinoth chandar 2020-09-24 15:01:32 -07:00
• d37977b310 [MINOR] Remove useless config for bootstrap integ testing (#2102) wenningd 2020-09-22 13:29:59 -07:00
• fcc497eff1 [HUDI-1268] fix UpgradeDowngrade fs Rename issue for hdfs and aliyun oss (#2099) lw0090 2020-09-23 00:57:20 +08:00
• 8087016504 [HUDI-1213] Set Default for the bootstrap config : hoodie.bootstrap.full.input.provider (#2087) Kaiux 2020-09-22 18:28:19 +08:00
• c8e19e2def [HUDI-801] Adding a way to post process schema after it is fetched (#1524) Alexander Filipchik 2020-09-19 11:18:36 -07:00
• 7c45894f43 [HUDI-995] Migrate HoodieTestUtils APIs to HoodieTestTable (#2094) Raymond Xu 2020-09-19 02:55:24 -07:00
• 73e5b4c7bb [HUDI-796] Add deduping logic for upserts case (#1558) Pratyaksh Sharma 2020-09-18 17:07:52 +05:30
• bf65269f66 [HUDI-1230] Fix for preventing MOR datasource jobs from hanging via spark-submit (#2046) Udit Mehrotra 2020-09-17 20:03:35 -07:00
• 3201665295 [HUDI-995] Use HoodieTestTable in more classes (#2079) Raymond Xu 2020-09-17 09:29:07 -07:00
• 581d54097c [HUDI-1143] Change timestamp field in HoodieTestDataGenerator from double to long shenh062326 2020-09-06 16:00:45 +08:00
• 6c84ef20ac [HUDI-1282] Check whether the topic exists before deltastrmer consumes Kafka (#2090) liujinhui 2020-09-16 10:43:52 +08:00
• 5e61454a6c [HUDI-802] AWSDmsTransformer does not handle insert and delete of a row in a single batch correctly (#2084) Balaji Varadarajan 2020-09-11 16:11:42 -07:00
• a1cff8abae [HUDI-1255] Add new Payload(OverwriteNonDefaultsWithLatestAvroPayload) for updating specified fields in storage (#2056) Karl-WangSK 2020-09-10 12:54:21 +08:00
• 063a98fc2b [HUDI-1254] TypedProperties can not get values by initializing an existing properties (#2059) linshan-ma 2020-09-09 23:42:41 +08:00
• fec7cd3c97 [HUDI-1130] hudi-test-suite support for schema evolution (can be triggered on any insert/upsert DAG node). Balajee Nagasubramaniam 2020-08-24 15:56:07 -07:00
• 53d1e55110 Test Suite should work with Docker + Unit Tests Abhishek Modi 2020-09-01 20:12:22 -07:00
• 2fee087f0f [HUDI-1181] Fix decimal type display issue for record key field (#1953) wenningd 2020-09-08 17:50:54 -07:00
• e3cf34dff9 Merge pull request #2077 from chuangehh/typofix Gary Li 2020-09-08 00:02:08 -07:00
• 51b16bd36f [MINOR] fix typo chuangehh 2020-09-08 11:55:38 +08:00
• fe7c9e71eb [MINOR] Fix BindException when running tests of shared machines. (#2070) Prashant Wason 2020-09-07 19:30:45 -07:00
• 83e39e2b17 [HUDI-781] Add HoodieWriteableTestTable (#2040) Raymond Xu 2020-09-07 02:54:36 -07:00
• 6537af2676 [HUDI-1153] Spark DataSource and Streaming Write must fail when operation type is misconfigured (#2014) Sreeram Ramji 2020-09-04 09:08:30 -07:00
• 8d19ebfd0f [HUDI-993] Let delete API use "hoodie.delete.shuffle.parallelism" (#1703) Dongwook 2020-09-01 09:55:31 -07:00
• 48a58c98a1 [MINOR] fix get classname for hive sync (#2008) Gary Li 2020-08-31 16:26:10 -07:00