lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

Commit Graph

  • bc0325f6ea [HUDI-1522] Add a new pipeline for Flink writer (#2430) Danny Chan 2021-01-28 08:53:13 +08:00
  • 7b2e658ac0 [MINOR] Add Jira URL and Mailing List (#2404) wangxianghu 2021-01-28 08:48:42 +08:00
  • 2ee1c3fb0c [HUDI-1234] Insert new records to data files without merging for "Insert" operation. (#2111) SteNicholas 2021-01-28 02:09:51 +08:00
  • a54550d94f [MINOR]Fix NPE when using HoodieFlinkStreamer with multi parallelism (#2492) luokey 2021-01-27 21:00:20 +08:00
  • c8ee40f8ae [MINOR] Update doap with 0.7.0 release (#2491) vinoth chandar 2021-01-26 09:28:22 -08:00
  • c4afd179c1 [HUDI-1476] Introduce unit test infra for java client (#2478) Shen Hong 2021-01-25 03:17:19 +08:00
  • 81836f0309 Removing spring repos from pom (#2481) vinoth chandar 2021-01-24 07:42:52 -08:00
  • 84df26323d [MINOR] Use skipTests flag for skip.hudi-spark2.unit.tests property (#2477) Raymond Xu 2021-01-24 05:36:41 -08:00
  • e302c6bc12 [HUDI-1453] Fix NPE using HoodieFlinkStreamer to etl data from kafka to hudi (#2474) wangxianghu 2021-01-23 10:27:40 +08:00
  • d3ea0f957e [HOTFIX] Revert upgrade flink verison to 1.12.0 (#2473) wangxianghu 2021-01-23 02:55:46 +08:00
  • 048633da1a [MINOR] Improve code readability,remove the continue keyword (#2459) cooper 2021-01-22 13:47:14 +08:00
  • 748dcc9aae [MINOR] Remove InstantGeneratorOperator parallelism limit in HoodieFlinkStreamer and update docs (#2471) wangxianghu 2021-01-22 13:46:25 +08:00
  • 641abe8ab7 [HUDI-1332] Introduce FlinkHoodieBloomIndex to hudi-flink-client (#2375) Xiang Yang 2021-01-22 10:36:28 +08:00
  • b64d22e047 [HUDI-1511] InstantGenerateOperator support multiple parallelism (#2434) luokey 2021-01-22 09:17:50 +08:00
  • 976420c49a [HUDI-1512] Fix spark 2 unit tests failure with Spark 3 (#2412) wenningd 2021-01-21 07:04:28 -08:00
  • 81ccb0c71a [MINOR] Make a separate travis CI job for hudi-utilities (#2469) vinoth chandar 2021-01-20 21:46:05 -08:00
  • 5e30fc1b2b [MINOR] Disabling problematic tests temporarily to stabilize CI (#2468) vinoth chandar 2021-01-20 14:24:34 -08:00
  • 3719e7b388 Moving to 0.8.0-SNAPSHOT on master branch. Vinoth Chandar 2021-01-20 11:31:22 -08:00
  • 244f6def9c [MINOR] Fix dataSource cannot use hoodie.datasource.hive_sync.auto_create_database (#2444) liujinhui 2021-01-20 22:58:18 +08:00
  • c931dc5406 [MINOR] Remove redundant judgments (#2466) teeyog 2021-01-20 20:41:09 +08:00
  • 5ca0625b27 [HUDI 1308] Harden RFC-15 Implementation based on production testing (#2441) vinoth chandar 2021-01-19 21:20:28 -08:00
  • e23967b9e9 [HUDI-1540] Fixing commons codec shading in spark bundle (#2460) Sivabalan Narayanan 2021-01-20 00:00:13 -05:00
  • 91b9cb53d3 [MINOR] Fixing setting defaults for index config (#2457) Sivabalan Narayanan 2021-01-19 18:16:25 -05:00
  • b9c2856d16 [HUDI-1535] Fix 0.7.0 snapshot (#2456) Sivabalan Narayanan 2021-01-19 15:20:43 -05:00
  • a38612b10f [HUDI-1532] Fixed suboptimal implementation of a magic sequence search (#2440) Volodymyr Burenin 2021-01-19 01:07:27 -06:00
  • 684e12e9fc [HUDI-1529] Add block size to the FileStatus objects returned from metadata table to avoid too many file splits (#2451) Udit Mehrotra 2021-01-18 07:29:53 -08:00
  • 3d1d5d00b0 [HUDI-1533] Make SerializableSchema work for large schemas and add ability to sortBy numeric values (#2453) satishkotha 2021-01-17 12:36:55 -08:00
  • a43e191d6c [MINOR] Bumping snapshot version to 0.7.0 (#2435) Sivabalan Narayanan 2021-01-16 09:56:28 -05:00
  • 749f657856 [HUDI-1509]: Reverting LinkedHashSet changes to combine fields from oldSchema and newSchema in favor of using only new schema for record rewriting (#2424) n3nash 2021-01-14 12:47:50 -08:00
  • e926c1a45c HUDI-1525 fix test hbase index (#2436) n3nash 2021-01-12 23:30:21 -08:00
  • e3d3677b7e [HUDI-1502] MOR rollback and restore support for metadata sync (#2421) Sivabalan Narayanan 2021-01-11 16:23:13 -05:00
  • de42adc230 [HUDI-1520] add configure for spark sql overwrite use INSERT_OVERWRITE_TABLE (#2428) lw0090 2021-01-12 01:07:47 +08:00
  • 7ce3ac778e [HUDI-1479] Use HoodieEngineContext to parallelize fetching of partiton paths (#2417) Udit Mehrotra 2021-01-10 21:19:52 -08:00
  • 23e93d05c0 [MINOR] fix spark 3 build for incremental query on MOR (#2425) Gary Li 2021-01-10 13:08:55 +08:00
  • 368c1a8f5c [HUDI-1399] support a independent clustering spark job to asynchronously clustering (#2379) lw0090 2021-01-10 09:30:16 +08:00
  • 65866c45ec [HUDI-1276] [HUDI-1459] Make Clustering/ReplaceCommit and Metadata table be compatible (#2422) vinoth chandar 2021-01-09 16:53:34 -08:00
  • 79ec7b4894 [HUDI-920] Support Incremental query for MOR table (#1938) Gary Li 2021-01-10 00:02:08 +08:00
  • 1a836f9a84 [HUDI-1514] Avoid raw type use for parameter of Transformer interface (#2420) puyvqi 2021-01-09 02:36:13 -08:00
  • c151147819 [MINOR] Sync HUDI-1196 to FlinkWriteHelper (#2415) Trevor 2021-01-09 07:57:56 +08:00
  • 17df517b81 [HUDI-1510] Move HoodieEngineContext and its dependencies to hudi-common (#2410) Udit Mehrotra 2021-01-07 11:34:06 -08:00
  • 5ff8e88d58 [HUDI-1513] Introduce WriteClient#preWrite() and relocate metadata table syncing (#2413) vinoth chandar 2021-01-06 23:08:30 -08:00
  • b593f10629 [MINOR] Rename unit test package of hudi-spark3 from scala to java (#2411) wangxianghu 2021-01-06 23:07:24 +08:00
  • 2c4868e770 [HUDI-1507] Change timeline utils to support reading replacecommit metadata (#2407) satishkotha 2021-01-06 04:55:14 -08:00
  • da2919a75f [HUDI-1383] Fixing sorting of partition vals for hive sync computation (#2402) Sivabalan Narayanan 2021-01-06 07:49:44 -05:00
  • 47c5e518a7 [HUDI-1506] Fix wrong exception thrown in HoodieAvroUtils (#2405) wangxianghu 2021-01-06 19:49:17 +08:00
  • 698694a157 [HUDI-1498] Read clustering plan from requested file for inflight instant (#2389) satishkotha 2021-01-04 10:36:44 -08:00
  • 31e674eb57 [HUDI-1504] Allow log files generated during restore/rollback to be synced as well Vinoth Chandar 2021-01-04 01:15:49 -08:00
  • 1a0579ca7d [HUDI-1312] [RFC-15] Support for metadata listing for snapshot queries through Hive/SparkSQL (#2366) rmpifer 2020-12-29 13:09:55 -08:00
  • 4b94529aaf [HUDI-1325] [RFC-15] Merge updates of unsynced instants to metadata table (apache#2342) Ryan Pifer 2020-12-31 08:57:13 -08:00
  • 2bd4a68731 [HUDI-1469] Faster initialization of metadata table using parallelized listing. (#2343) Prashant Wason 2020-12-31 01:40:12 -08:00
  • 4e64226844 [HUDI-1450] Use metadata table for listing in HoodieROTablePathFilter (apache#2326) Udit Mehrotra 2020-12-31 01:20:02 -08:00
  • 298808baaf [HUDI-842] Implementation of HUDI RFC-15. Prashant Wason 2020-12-30 18:29:55 -08:00
  • c3e9243ea1 [MINOR] Add maven profile to support skipping shade sources jars (#2358) jshmchenxi 2021-01-04 12:19:48 +08:00
  • ff8313caf1 [HUDI-1423] Support delete in hudi-java-client (#2353) Shen Hong 2021-01-03 20:38:45 +08:00
  • a23aa41a1a [MINOR] Sync UpsertPartitioner modify of HUDI-1398 to flink/java (#2390) steven zhang 2020-12-31 11:06:41 +08:00
  • c5e8a024f6 [HUDI-1418] Set up flink client unit test infra (#2281) Gary Li 2020-12-31 08:57:22 +08:00
  • 605b617cfa [HUDI-1434] fix incorrect log file path in HoodieWriteStat (#2300) Gary Li 2020-12-31 06:22:15 +08:00
  • ef28763f08 [MINOR] Update report_coverage.sh (#2396) wangxianghu 2020-12-30 19:47:04 +08:00
  • c6bf952332 [HUDI-1493] Fixed schema compatibility check for fields. (#2350) Prashant Wason 2020-12-29 17:02:21 -08:00
  • e33a8f733c [HUDI-1147] Modify GenericRecordFullPayloadGenerator to generate vali… (#2045) Balajee Nagasubramaniam 2020-12-29 13:33:19 -08:00
  • da51aa64fc [HUDI-1474] Add additional unit tests to TestHBaseIndex (#2349) Balajee Nagasubramaniam 2020-12-28 20:04:38 -08:00
  • b83d1d3e61 [HUDI-1484] Escape the partition value in HiveSyncTool (#2363) pengzhiwei 2020-12-29 12:02:36 +08:00
  • 4c17528de0 [HUDI-1398] Align insert file size for reducing IO (#2256) steven zhang 2020-12-29 11:52:35 +08:00
  • 0ecdec348e [MINOR] Remove the duplicate code in AbstractHoodieWriteClient.startCommit (#2385) Danny Chan 2020-12-29 10:49:24 +08:00
  • 76faf59652 [HUDI-1495] Upgrade Flink version to 1.12.0 (#2384) Danny Chan 2020-12-29 10:15:43 +08:00
  • e177466fd2 [HUDI-1350] Support Partition level delete API in HUDI (#2254) lw0090 2020-12-29 07:01:06 +08:00
  • 6cdf59d92b [HUDI-1354] Block updates and replace on file groups in clustering (#2275) lw0090 2020-12-28 12:30:29 +08:00
  • 9e6889a8ce [HUDI-1481] add structured streaming and delta streamer clustering unit test (#2360) lw0090 2020-12-28 12:27:09 +08:00
  • 8cf6a7223f [HUDI-1331] Adding support for validating entire dataset and long running tests in test suite framework (#2168) Sivabalan Narayanan 2020-12-26 12:29:24 -05:00
  • 3ec9270e8e [HUDI-1490] Incremental Query should work even when there are partitions that have no incremental changes (#2371) Balaji Varadarajan 2020-12-26 09:17:49 -08:00
  • e807bb895e [HUDI-1487] fix unit test testCopyOnWriteStorage random failed (#2364) lw0090 2020-12-26 01:54:23 +08:00
  • 286055ce34 [HUDI-1451] Support bulk insert v2 with Spark 3.0.0 (#2328) wenningd 2020-12-25 06:43:34 -08:00
  • 89f482eaf2 [HUDI-1489] Fix null pointer exception when reading updated written bootstrap table (#2370) wenningd 2020-12-23 11:26:24 -08:00
  • 38b9264dd0 [HUDI-1488] Fix Test Case Failure in TestHBaseIndex (#2365) pengzhiwei 2020-12-23 16:47:38 +08:00
  • 01ad449ad6 [HUDI-1485] Fix Deletes issued without any prior commits exception (#2361) wangxianghu 2020-12-22 23:10:19 +08:00
  • f8ccb2872d [HUDI-1471] Make QuickStartUtils generate deletes according to specific ts (#2357) wangxianghu 2020-12-22 21:14:18 +08:00
  • 959afb8ba4 Merge pull request #2263 from satishkotha/sk/clustering satishkotha 2020-12-21 19:18:18 -08:00
  • 6dc03b65bf [HUDI-1075] Implement simple clustering strategies to create ClusteringPlan and to run the plan Satish Kotha 2020-11-07 17:08:55 -08:00
  • 0c821fecc2 [MINOR] Pass root exception to HoodieKeyGeneratorException for more information (#2354) jshmchenxi 2020-12-22 09:02:23 +08:00
  • e4e2fbc3bb [HUDI-1419] Add base implementation for hudi java client (#2286) Shen Hong 2020-12-20 11:25:27 +08:00
  • 33d338f392 [HUDI-115] Adding DefaultHoodieRecordPayload to honor ordering with combineAndGetUpdateValue (#2311) Sivabalan Narayanan 2020-12-19 22:19:42 -05:00
  • 5388c7f7a3 [HUDI-1470] Use the latest writer schema, when reading from existing parquet files in the hudi-test-suite (#2344) Balajee Nagasubramaniam 2020-12-18 03:18:52 -08:00
  • 8b5d6f9430 [HUDI-1437] support more accurate spark JobGroup for better performance tracking (#2322) lw0090 2020-12-18 07:20:13 +08:00
  • 14d5d1100c [HUDI-1406] Add date partition based source input selector for Delta streamer (#2264) Bhavani Sudha Saktheeswaran 2020-12-17 03:59:30 -08:00
  • 4ddfc61d70 [MINOR] Make QuickstartUtil generate random timestamp instead of 0 (#2340) wangxianghu 2020-12-17 18:00:23 +08:00
  • 6a6b772c49 [MINOR] Fix error information in exception (#2341) ChangLi 2020-12-16 19:37:01 +08:00
  • 26cdc457f6 [HUDI-1376] Drop Hudi metadata cols at the beginning of Spark datasource writing (#2233) wenningd 2020-12-15 16:20:48 -08:00
  • 93d9c25aee [MINOR] Improve code readability by passing in the fileComparisonsRDD in bloom index (#2319) Danny Chan 2020-12-15 14:35:24 +08:00
  • 069a1dcf24 [HUDI-1435] Fix bug in Marker File Reconciliation for Non-Partitioned datasets (#2301) Balaji Varadarajan 2020-12-14 22:24:12 -08:00
  • facde4c16f [HUDI-1448] Hudi dla sync support skip rt table syncing (#2324) lw0090 2020-12-14 23:25:10 +08:00
  • 11bc1fe6f4 [HUDI-1428] Clean old fileslice is invalid (#2292) steven zhang 2020-12-13 22:28:53 +08:00
  • 236d1b0dec [HUDI-1439] Remove scala dependency from hudi-client-common (#2306) Shen Hong 2020-12-11 16:36:37 +08:00
  • 6cf25d5c8a [MINOR] Minor improve in IncrementalRelation (#2314) wangxianghu 2020-12-10 20:16:00 +08:00
  • 4bc45a391a [HUDI-1445] Refactor AbstractHoodieLogRecordScanner to use Builder (#2313) Danny Chan 2020-12-10 20:02:02 +08:00
  • bd9cceccb5 [HUDI-1395] Fix partition path using FSUtils (#2312) Raymond Xu 2020-12-09 18:19:19 -08:00
  • 007014c1ef [MINOR] Throw an exception when keyGenerator initialization failed (#2307) wangxianghu 2020-12-10 09:56:19 +08:00
  • fce1453fa6 [HUDI-1040] Make Hudi support Spark 3 (#2208) wenningd 2020-12-09 15:52:23 -08:00
  • 3a91d26d62 fix typo (#2308) jshmchenxi 2020-12-08 22:28:20 +08:00
  • de2fbeac33 [HUDI-1412] Make HoodieWriteConfig support setting different default … (#2278) wangxianghu 2020-12-07 09:29:53 +08:00
  • 319b7a58e4 [HUDI-1427] Fix FileAlreadyExistsException when set HOODIE_AUTO_COMMIT_PROP to true (#2295) pengzhiwei 2020-12-05 08:07:25 +08:00