lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

Commit Graph

  • 5a94043f38 [HUDI-2343]Fix the exception for mergeInto when the primaryKey and preCombineField of source table and target table differ in case only (#3517) 董可伦 2021-09-21 22:11:52 +08:00
  • 5091ab7311 [HUDI-2444] Fixing delete files corner cases wrt cleaning and rollback when applying changes to metadata (#3678) Sivabalan Narayanan 2021-09-20 11:05:31 -04:00
  • 3354fac42f [HUDI-2449] Incremental read for Flink (#3686) Danny Chan 2021-09-19 09:06:46 +08:00
  • c7a5c8273b [HUDI-2267] Update docs and infra test configs, add support for graphite (#3482) Satish M 2021-09-17 19:40:15 +05:30
  • 3a150ee181 [HUDI-2447] Extract common business logic & Fix typo (#3683) 董可伦 2021-09-17 19:45:22 +08:00
  • 61d0096088 [HUDI-2434] Make periodSeconds of GraphiteReporter configurable (#3667) liujinhui 2021-09-17 19:39:55 +08:00
  • 8a652171cf [MINOR] Fix typo,'compatiblity' corrected to 'compatibility' (#3675) 董可伦 2021-09-17 09:43:23 +08:00
  • 57d5da68aa [HUDI-2330][HUDI-2335] Adding support for merge-on-read tables (#3679) vinoth chandar 2021-09-16 15:24:34 -07:00
  • b8dad628e5 [HUDI-2422] Adding rollback plan and rollback requested instant (#3651) Sivabalan Narayanan 2021-09-16 11:16:06 -04:00
  • 4deaa30c8d [HUDI-2404] Add metrics-jmx to spark and flink bundles (#3632) Sarah Witt 2021-09-16 09:53:16 -04:00
  • 2791fb9a96 [HUDI-2423] Separate some config logic from HoodieMetricsConfig into HoodieMetricsGraphiteConfig HoodieMetricsJmxConfig (#3652) liujinhui 2021-09-16 15:08:10 +08:00
  • 2d5ac55195 [HUDI-2355][Bug]Archive service executed after cleaner finished. (#3545) zhangyue19921010 2021-09-16 07:00:04 +08:00
  • 916f12b7dd [HUDI-2433] Refactor rollback actions in hudi-client module (#3664) Y Ethan Guo 2021-09-15 15:52:43 -07:00
  • 86a7351c39 [MINOR] Delete Redundant code (#3661) liujinhui 2021-09-15 14:46:11 +08:00
  • 76554aa31a [MINOR] Add document for DataSourceReadOptions (#3653) liujinhui 2021-09-15 14:33:43 +08:00
  • 627f20f9c5 [HUDI-2430] Make decimal compatible with hudi for flink writer (#3658) Danny Chan 2021-09-15 12:04:46 +08:00
  • d90fd1f68c [MINOR] Update Kafka connect sink readme Vinoth Chandar 2021-09-14 10:36:37 -07:00
  • 9735f4b8ef [HUDI-2428] Fix protocol and other issues after stress testing Hudi Kafka Connect (#3656) rmahindra123 2021-09-14 07:14:58 -07:00
  • 5d60491f5b [HUDI-2388] Add DAG nodes for Spark SQL in integration test suite (#3583) Y Ethan Guo 2021-09-13 08:53:13 -07:00
  • 35a04c43a5 [HUDI-2425] TestHoodieMultiTableDeltaStreamer CI failed due to exception (#3654) liujinhui 2021-09-13 21:57:04 +08:00
  • 89651c9408 [HUDI-2421] Catch the throwable when scheduling the cleaning task for flink writer (#3650) Danny Chan 2021-09-13 20:43:44 +08:00
  • 9f3c4a2a7f [HUDI-2410] Fix getDefaultBootstrapIndexClass logical error (#3633) liujinhui 2021-09-13 16:10:17 +08:00
  • c79017cb74 [HUDI-2397] Add --enable-sync parameter (#3608) K.I. (Dennis) Jung 2021-09-13 15:34:49 +09:00
  • 280f66e0f8 [MINOR] Fix the default parallelism of write task (#3649) Danny Chan 2021-09-13 11:41:49 +08:00
  • 4f991ee352 [HUDI-2398] Collect event time for inserts in DefaultHoodieRecordPayload (#3602) Ankush Kanungo 2021-09-11 20:27:40 -07:00
  • 9d5c3e5cb9 [HUDI-2415] Add more info log for flink streaming reader (#3642) Danny Chan 2021-09-12 10:00:17 +08:00
  • 6228b17a3d [MINOR] Fix typo, 'requried' corrected to 'required' (#3643) 董可伦 2021-09-11 15:46:24 +08:00
  • dbcf60f370 [MINOR] fix typo (#3640) 董可伦 2021-09-11 15:45:49 +08:00
  • b30c5bdaef [HUDI-2412] Add timestamp based partitioning for flink writer (#3638) Danny Chan 2021-09-11 13:17:16 +08:00
  • 06240417e9 [HUDI-2354] Fix TimelineServer error because of replacecommit archive (#3536) zhangyue19921010 2021-09-11 12:26:04 +08:00
  • e528dd798a [HUDI-2394] Implement Kafka Sink Protocol for Hudi for Ingesting Immutable Data (#3592) rmahindra123 2021-09-10 18:20:26 -07:00
  • bd1d2d4952 [MINOR] Add avro schema evolution test with (non)nullable column and with(out) default value (#3639) Sagar Sumit 2021-09-10 19:33:35 +05:30
  • cf15431852 [HUDI-2393] Add yamls for large scale testing (#3594) Sagar Sumit 2021-09-10 18:32:01 +05:30
  • 44b9bc145e [HUDI-2411] Remove unnecessary method overriden and note (#3636) wangxianghu 2021-09-10 18:58:34 +08:00
  • 512ca42d14 [MINOR] Correct the comment for the parallelism of tasks in FlinkOptions (#3634) SteNicholas 2021-09-10 13:42:11 +08:00
  • 56d08fbe70 [HUDI-2351] Extract common FS and IO utils for marker mechanism (#3529) Y Ethan Guo 2021-09-09 11:45:28 -07:00
  • 57c8113ee1 [HUDI-2408] Deprecate FunctionalTestHarness to avoid init DFS (#3628) Raymond Xu 2021-09-09 08:29:04 -07:00
  • 4abcb4f659 [MINOR] Remove unused variables (#3631) Wei 2021-09-09 23:21:16 +08:00
  • 3c4eb60913 Add the document to the PUSHGATEWAY configuration item (#3627) liujinhui 2021-09-09 15:53:58 +08:00
  • db2ab9a150 [HUDI-2403] Add metadata table listing for flink query source (#3618) Danny Chan 2021-09-08 14:52:39 +08:00
  • 81acb4cafe [MINOR] Remove commenting from Github, JIRA bridge (#3620) vinoth chandar 2021-09-07 21:54:58 -07:00
  • cf3a2ead32 [HUDI-2401] Load archived instants for flink streaming reader (#3610) Danny Chan 2021-09-08 10:43:54 +08:00
  • ea59a7ff5f [HUDI-2080] Move to ubuntu-18.04 for Azure CI (#3409) vinoth chandar 2021-09-07 09:44:30 -07:00
  • eb5e7eec0a MINOR_CHECKSTYLE (#3616) liujinhui 2021-09-07 18:19:39 +08:00
  • cf002b6918 [HUDI-2079] Make CLI command tests functional (#3601) Raymond Xu 2021-09-06 15:53:53 -07:00
  • f218693f5d [MINOR] Fixing some functional tests by moving to right packages (#3596) Sivabalan Narayanan 2021-09-06 00:07:55 -04:00
  • 7592ddd776 [HUDI-2399] Rebalance CI jobs for shorter wait time (#3604) Raymond Xu 2021-09-05 09:25:57 -07:00
  • e9bf1c1186 [HUDI-2380] The default archive folder should be 'archived' (#3568) Danny Chan 2021-09-04 15:53:55 +08:00
  • 073c318d9f [HUDI-1989] Disable HDFSParquetImporter related tests (#3597) Raymond Xu 2021-09-03 20:08:11 -07:00
  • 6bd3ca98d6 [HUDI-1989] Fix flakiness in TestHoodieMergeOnReadTable (#3574) Raymond Xu 2021-09-03 13:17:17 -07:00
  • 11398e8480 [MINOR] Skip checkstyle and rat in Azure (#3593) Raymond Xu 2021-09-03 09:18:18 -07:00
  • 79b896f071 [HUDI-2392] Do not send partition delete record when changelog mode enabled (#3586) Danny Chan 2021-09-02 20:58:12 +08:00
  • 7a1bd225ca [HUDI-2376] Add pipeline for Append mode (#3573) yuzhaojing 2021-09-02 16:32:40 +08:00
  • 21fd6edfe7 [HUDI-2384] Change log file size config to long (#3577) Shawy Geng 2021-09-02 11:14:09 +08:00
  • 38c9b85aa8 [HUDI-2280] Use GitHub Actions to build different scala spark versions (#3556) Raymond Xu 2021-09-01 08:51:00 -07:00
  • f66e1ce9bf [HUDI-2379] Include the pending compaction file groups for flink (#3567) Danny Chan 2021-09-01 16:47:52 +08:00
  • d59c8044f8 [HUDI-2378] Add configs for common and pre validate (#3564) rmahindra123 2021-08-30 20:28:35 -07:00
  • bf5a52e51b [HUDI-2320] Add support ByteArrayDeserializer in AvroKafkaSource (#3502) 董可伦 2021-08-30 10:01:15 +08:00
  • 57668d02a0 [HUDI-2371] Improvement flink streaming reader (#3552) Danny Chan 2021-08-28 20:16:54 +08:00
  • 69cbcc9516 Merge pull request #3541 from rahil-c/rahil-c/HUDI-2359 wenningd 2021-08-27 16:28:51 -07:00
  • 562e28f079 [HUDI-2365]Optimizing overwriteField method with Objects.equals (#3542) 董可伦 2021-08-27 17:17:22 +08:00
  • 9850e90e2e [HUDI-2229] Refact HoodieFlinkStreamer to reuse the pipeline of HoodieTableSink (#3495) mikewu 2021-08-27 10:14:04 +08:00
  • 55a80a817d [HUDI-2264] Refactor HoodieSparkSqlWriterSuite to add setup and teardown (#3544) Satish M 2021-08-26 19:31:48 +05:30
  • 0f39137ba8 [HUDI-2321] Use the caller classloader for ReflectionUtils (#3535) Danny Chan 2021-08-26 21:00:30 +08:00
  • 73fdcf37df [HUDI-2368] Catch Throwable in BoundedInMemoryExecutor (#3546) yuzhaojing 2021-08-26 20:34:05 +08:00
  • cc5256a7d8 [HUDI-2357] MERGE INTO doesn't work for tables created using CTAS (#3534) pengzhiwei 2021-08-26 16:54:41 +08:00
  • be57e42200 [HUDI-2366] fix too many logs (#3543) ayachi_nene 2021-08-26 16:45:52 +08:00
  • 694300477f [HUDI-2359] Add basic "hoodie_is_deleted" unit tests to TestDataSource classes Rahil Chertara 2021-08-22 21:55:11 -07:00
  • 486bc7dc3b [MINOR] Update DOAP with 0.9.0 Release (#3537) Udit Mehrotra 2021-08-25 13:57:05 -07:00
  • a60fab3a5c [HUDI-2352] The upgrade downgrade action of flink writer should be singleton (#3531) Danny Chan 2021-08-25 10:56:14 +08:00
  • 04ede8eecf [HUDI-2262] reduce build warnings (#3481) Satish M 2021-08-24 22:36:38 +05:30
  • 15bf01dcb7 [HUDI-2349] Adding spark delete node to integ test suite (#3528) Sivabalan Narayanan 2021-08-24 10:58:47 -04:00
  • de94787a85 [HUDI-2345] Hoodie columns sort partitioner for bulk insert (#3523) zhangyue19921010 2021-08-24 21:45:17 +08:00
  • 05e6f44d53 [MINOR] Fix BatchBootstrapOperator initialization (#3520) Danny Chan 2021-08-22 13:03:22 +08:00
  • be8c1e499f Support referencing subquery with column aliases by table alias in merge into (#3380) 董可伦 2021-08-21 21:53:16 +08:00
  • ab3fbb8895 [HUDI-2342] Optimize Bootstrap operator (#3516) yuzhaojing 2021-08-21 20:03:03 +08:00
  • c7c517f14c [HUDI-2340] Merge the data set for flink bounded source when changelog mode turns off (#3513) Danny Chan 2021-08-21 07:21:35 +08:00
  • e39d0a2f28 Keep non-conflicting names for common configs between DataSourceOptions and HoodieWriteConfig (#3511) Udit Mehrotra 2021-08-20 02:42:59 -07:00
  • 49829f8822 [HUDI-2339] Create Table If Not Exists Failed After Alter Table (#3510) pengzhiwei 2021-08-20 14:21:10 +08:00
  • c350d05dd3 Restore 0.8.0 config keys with deprecated annotation (#3506) Udit Mehrotra 2021-08-19 13:36:40 -07:00
  • 37c29e75dc [HUDI-2322] Use correct meta columns while preparing dataset for bulk insert (#3504) Sagar Sumit 2021-08-19 21:37:12 +05:30
  • 9762e4c08c [MINOR] Some cosmetic changes for Flink (#3503) Danny Chan 2021-08-19 23:21:20 +08:00
  • 7dddd54406 [MINOR] Fixing release validation script (#3493) Sivabalan Narayanan 2021-08-19 07:54:56 -04:00
  • b15c3cf9a9 [HUDI-1363] Include _hoodie_operation meta column in removeMetadataFields (#3501) Sagar Sumit 2021-08-19 16:33:54 +05:30
  • 1fed44af84 [HUDI-2316] Support Flink batch upsert (#3494) swuferhong 2021-08-19 17:15:26 +08:00
  • b7a0d76fc9 [HUDI-2167] HoodieCompactionConfig get HoodieCleaningPolicy NullPointerException leiqiang 2021-07-13 11:26:12 +08:00
  • 5ee35a0a92 HUDI-1674 (#3488) liujinhui 2021-08-18 13:45:48 +08:00
  • 99663d370b [HUDI-2301] fix FileSliceMetrics utils bug (#3487) ayachi_nene 2021-08-18 02:09:53 +08:00
  • 66f951322a [HUDI-2191] Bump flink version to 1.13.1 (#3291) Danny Chan 2021-08-16 18:14:05 +08:00
  • 4d508ef673 [MINOR] Fix SelectPackages in HoodieSparkFunctionalTestSuite (#3476) Raymond Xu 2021-08-15 10:17:00 -07:00
  • 09e625becd [HOT-FIX] Add apache license to spark_command.txt.template (#3477) Udit Mehrotra 2021-08-15 04:08:55 -07:00
  • 3e301196bf Moving to 0.10.0-SNAPSHOT on master branch. Udit Mehrotra 2021-08-14 18:51:09 -07:00
  • 23dca6c237 [HUDI-2268] Add upgrade and downgrade to and from 0.9.0 (#3470) Y Ethan Guo 2021-08-14 17:20:23 -07:00
  • 18e6b79947 [MINOR] Adding back all old default val members to DataSourceOptions (#3474) vinoth chandar 2021-08-14 14:49:22 -07:00
  • 5cc96e85c1 [HUDI-1897] Deltastreamer source for AWS S3 (#3433) Sagar Sumit 2021-08-14 17:55:10 +05:30
  • 9056c68744 [HUDI-2305] Add MARKERS.type and fix marker-based rollback (#3472) Y Ethan Guo 2021-08-14 05:18:49 -07:00
  • b7da6cb33d [HUDI-2307] When using delete_partition with ds should not rely on the primary key (#3469) liujinhui 2021-08-14 14:53:39 +08:00
  • 8eed440694 [HUDI-2119] Ensure the rolled-back instance was previously synced to the Metadata Table when syncing a Rollback Instant. (#3210) Prashant Wason 2021-08-13 21:23:34 -07:00
  • 642b1b671d [HUDI-2151] Flipping defaults (#3452) Sivabalan Narayanan 2021-08-13 19:29:22 -04:00
  • 9689278014 [HUDI-1363] Provide option to drop partition columns (#3465) Sagar Sumit 2021-08-13 22:31:26 +05:30