lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

Commit Graph

  • 5db37c255b [HUDI-2047] Ignore FileNotFoundException in WriteProfiles #getWritePathsOfInstant (#3125) yuzhaojing 2021-06-22 14:18:46 +08:00
  • 7bd517a82f [HUDI-2031] JVM occasionally crashes during compaction when spark speculative execution is enabled (#3093) Rong Ma 2021-06-22 09:09:51 +08:00
  • cb5cd35991 [HUDI-2043] HoodieDefaultTimeline$filterPendingCompactionTImeline() method have wrong filter condition (#3109) swuferhong 2021-06-22 08:53:54 +08:00
  • 4fd8a88b7e [HUDI-1776] Support AlterCommand For Hoodie (#3086) pengzhiwei 2021-06-21 22:58:43 +08:00
  • f8d9242372 [HUDI-2050] Support rollback inflight compaction instances for batch flink compactor (#3124) swuferhong 2021-06-21 20:32:48 +08:00
  • adf167991a [HUDI-2049] StreamWriteFunction should wait for the next inflight instant time before flushing (#3123) Danny Chan 2021-06-21 20:15:27 +08:00
  • 429e9fb5fe [HUDI-1248] Increase timeout for deltaStreamerTestRunner in TestHoodieDeltaStreamer (#3110) Sagar Sumit 2021-06-21 10:12:12 +05:30
  • e41f13fe7b [MINOR] Put Azure cache tasks first (#3118) Raymond Xu 2021-06-20 14:36:39 -07:00
  • c08fbb4268 [MINOR] Remove unused module (#3116) Wei 2021-06-20 03:06:47 +08:00
  • 1cbdb49816 [HUDI-251] Adds JDBC source support for DeltaStreamer (#2915) Sagar Sumit 2021-06-19 19:42:11 +05:30
  • 7865da1e15 [MINOR] Fix Javadoc wrong references (#3115) Wei 2021-06-19 12:51:54 +08:00
  • 53396061cc [MINOR] Fix wrong package name (#3114) Wei 2021-06-19 11:50:01 +08:00
  • cdb9b48170 [HUDI-2040] Make flink writer as exactly-once by default (#3106) Danny Chan 2021-06-18 13:55:23 +08:00
  • aa6342c3c9 [HUDI-2036] Move the compaction plan scheduling out of flink writer coordinator (#3101) Danny Chan 2021-06-18 09:35:09 +08:00
  • b9e28e5292 [HUDI-2033] ClassCastException Throw When PreCombineField Is String Type (#3099) pengzhiwei 2021-06-17 23:21:20 +08:00
  • 67c3124352 [HUDI-2032] Make keygen class and keygen type optional for FlinkStreamerConfig (#3104) vinoyang 2021-06-17 21:22:13 +08:00
  • f97dd25d41 [HUDI-2019] Set up the file system view storage config for singleton embedded server write config every time (#3102) yuzhaojing 2021-06-17 20:28:03 +08:00
  • ad53cf450e [HUDI-1879] Fix RO Tables Returning Snapshot Result (#2925) pengzhiwei 2021-06-17 19:18:21 +08:00
  • 6763b45dd4 [HUDI-2030] Add metadata cache to WriteProfile to reduce IO (#3090) Danny Chan 2021-06-17 19:10:34 +08:00
  • 0b57483a8e [HUDI-2015] Fix flink operator uid to allow multiple pipelines in one job (#3091) Danny Chan 2021-06-17 09:08:19 +08:00
  • 5ce64a81bd Fix the filter condition is missing in the judgment condition of compaction instance (#3025) swuferhong 2021-06-17 05:28:53 +08:00
  • d519c74626 [HUDI-2008] Avoid the raw type usage in some classes under hudi-utilities module (#3076) Wei 2021-06-16 22:37:29 +08:00
  • 8b0a502c4f [HUDI-2014] Support flink hive sync in batch mode (#3081) swuferhong 2021-06-16 14:29:16 +08:00
  • 61efc6af79 [HUDI-2022] Release writer for append handle #close (#3087) yuzhaojing 2021-06-16 09:18:38 +08:00
  • 910fe4842c [MINOR] Rename broken codecov file (#3088) vinoth chandar 2021-06-15 18:05:50 -07:00
  • b8fe5b91d5 [HUDI-764] [HUDI-765] ORC reader writer Implementation (#2999) Jintao Guan 2021-06-15 15:21:43 -07:00
  • cb642ceb75 [HUDI-1999] Refresh the base file view cache for WriteProfile (#3067) Danny Chan 2021-06-15 23:18:38 +08:00
  • f922837064 [HUDI-1950] Fix Azure CI failure in TestParquetUtils (#2984) Raymond Xu 2021-06-15 03:45:17 -07:00
  • 515ce8eb36 [MINOR] Fixed the log which should only be printed when the Metadata Table is disabled. (#3080) Prashant Wason 2021-06-15 01:18:15 -07:00
  • 769dd2d7c9 [HUDI-2004] Move CheckpointUtils test cases to independant class (#3072) Vinay Patil 2021-06-14 14:44:59 +05:30
  • 7d9f9d7d82 [HUDI-1991] Fixing drop dups exception in bulk insert row writer path (#3055) Sivabalan Narayanan 2021-06-13 21:55:52 -04:00
  • 6e78682cea [HUDI-2000] Release file writer for merge handle #close (#3068) yuzhaojing 2021-06-13 18:09:48 +08:00
  • 0c4f2fdc15 [HUDI-1984] Support independent flink hudi compaction function (#3046) swuferhong 2021-06-13 15:04:46 +08:00
  • ba728d822f [HUDI-2002] Modify HiveIncrementalPuller log level to ERROR (#3070) Wei 2021-06-13 01:21:43 +08:00
  • 673d62f3c3 [MINOR] Add Tencent Cloud HDFS storage support for hudi (#3064) Xuedong Luan 2021-06-11 09:16:51 +08:00
  • 9e4114dd46 [HUDI-1790] Added SqlSource to fetch data from any partitions for backfill use case (#2896) Vinoth Govindarajan 2021-06-10 15:03:07 -07:00
  • 125415a8b8 [HUDI-1994] Release the new records iterator for append handle #close (#3058) Danny Chan 2021-06-10 19:09:23 +08:00
  • e0108e972e [MINOR] Add Baidu BOS storage support for hudi (#3061) JunZhang 2021-06-10 15:51:36 +08:00
  • a8b10e9067 [MINOR] Remove boxing (#3062) Wei 2021-06-10 13:03:32 +08:00
  • afbafe7046 [HUDI-1992] Release the new records map for merge handle #close (#3056) Danny Chan 2021-06-09 21:12:56 +08:00
  • 728089a888 delete duplicate bootstrap function (#3052) yuzhaojing 2021-06-09 19:29:57 +08:00
  • e8fcf04b57 [HUDI-1987] Fix non partition table hive meta sync for flink writer (#3049) Danny Chan 2021-06-09 14:20:04 +08:00
  • a6f5fc5967 [HUDI-1986] Skip creating marker files for flink merge handle (#3047) Danny Chan 2021-06-09 14:17:28 +08:00
  • 11360f707e [HUDI-1892] Fix NPE when avro field value is null (#3051) Vinay Patil 2021-06-09 03:42:18 +05:30
  • 75d663f65d [HUDI-1980] Optimize the code to prevent other exceptions from causing resources not to be closed (#3038) Wei 2021-06-08 21:58:34 +08:00
  • 7261f08507 [HUDI-1929] Support configure KeyGenerator by type (#2993) wangxianghu 2021-06-08 21:26:10 +08:00
  • f760ec543e [HUDI-1659] Basic Implement Of Spark Sql Support For Hoodie (#2645) pengzhiwei 2021-06-08 14:24:32 +08:00
  • cf83f10f5b add BootstrapFunction to support index bootstrap (#3024) yuzhaojing 2021-06-08 13:55:25 +08:00
  • 57611d10b5 [HUDI-1743] Added support for SqlFileBasedTransformer (#2747) Vinoth Govindarajan 2021-06-07 18:48:27 -07:00
  • 919590988a [HUDI-1914] Add fetching latest schema to table command in hudi-cli (#2964) Sivabalan Narayanan 2021-06-07 19:04:35 -04:00
  • 441076b2cc [HUDI-1950] Move TestHiveMetastoreBasedLockProvider to functional (#3043) Raymond Xu 2021-06-07 15:38:59 -07:00
  • f3d7b49bfe [HUDI-1148] Remove Hadoop Conf Logs (#3040) Vinay Patil 2021-06-08 03:19:55 +05:30
  • 0d0dc6fb07 [HUDI-1909] Skip Commits with empty files (#3045) Vinay Patil 2021-06-07 19:28:19 +05:30
  • 08464a6a5b [HUDI-1931] BucketAssignFunction use ValueState instead of MapState (#3026) Danny Chan 2021-06-06 10:40:15 +08:00
  • 2a7e1e091e [HUDI-1942] Add Default value for HIVE_AUTO_CREATE_DATABASE_OPT_KEY in HoodieSparkSqlWriter (#3036) Vinay Patil 2021-06-06 03:32:26 +05:30
  • dab13f7473 [HUDI-1979] Optimize logic to improve code readability (#3037) Wei 2021-06-05 19:40:45 +08:00
  • c2383ee904 [HUDI-1967] Fix the NPE for MOR Hive rt table query (#3032) Danny Chan 2021-06-05 16:06:34 +08:00
  • cf90f17732 [HUDI-1281] Add deltacommit to ActionType (#3018) Vinay Patil 2021-06-05 11:00:48 +05:30
  • c4a2ad2702 [HUDI-1954] only reset bucket when flush bucket success (#3029) yuzhaojing 2021-06-05 11:48:08 +08:00
  • d02c0e5387 [MINOR] Resolve build issue arising from inaccessible pentaho jar (#3034) vinoth chandar 2021-06-04 12:28:44 -07:00
  • a658328001 [HUDI-1961] Add a debezium json integration test case for flink (#3030) Danny Chan 2021-06-04 15:15:32 +08:00
  • 870e97b5f8 [MINOR] Remove unused method in DataSourceUtils (#3031) wangxianghu 2021-06-04 01:24:51 +08:00
  • f6eee77636 [MINOR] Remove the implementation of Serializable from HoodieException (#3020) Wei 2021-06-03 19:46:33 +08:00
  • ad72691d24 [HUDI-1957] Fix flink timeline service lack jetty dependency (#3028) swuferhong 2021-06-03 19:39:31 +08:00
  • 86007e9a13 [HUDI-1953] Fix NPE due to not set the output type of the operator (#3023) taylorliao 2021-06-03 14:20:57 +08:00
  • 05a9830e86 [HUDI-1952] Fix hive3 meta sync for flink writer (#3021) swuferhong 2021-06-02 14:12:03 +08:00
  • 7fa2f8ea82 [HUDI-1921] Add target io option for flink compaction (#2980) Danny Chan 2021-06-02 10:10:35 +08:00
  • bf1cfb5635 [HUDI-1949] Refactor BucketAssigner to make it more efficient (#3017) Danny Chan 2021-06-02 09:12:35 +08:00
  • 83c31e356f [HUDI-1927] Improve HoodieFlinkStreamer (#3019) taylorliao 2021-06-01 18:35:14 +08:00
  • 83b0301c1a [HUDI-1943] Lose properties when hoodieWriteConfig initializtion (#3006) hk__lrzy 2021-06-01 16:09:48 +08:00
  • e6a71ea544 [MINOR] Access the static member getLastHeartbeatTime via the class instead (#3015) Wei 2021-05-31 18:54:05 +08:00
  • 219b92c8ae [MINOR] The collection can use forEach() directly (#3016) Wei 2021-05-31 18:52:30 +08:00
  • 34ab756a40 [HUDI-1948] Shade kryo-shaded jar for hudi flink bundle (#3014) Danny Chan 2021-05-31 17:39:19 +08:00
  • 7a63175a70 fix the grammer err of the comment (#3013) Yao WANG 2021-05-31 11:44:25 +08:00
  • d965b0550f [MINOR] 'return' is unnecessary as the last statement in a 'void' method (#3012) Wei 2021-05-31 11:43:10 +08:00
  • dcd7c331dc [HUDI-1879] Support Partition Prune For MergeOnRead Snapshot Table (#2926) pengzhiwei 2021-05-29 22:50:24 +08:00
  • 0709c62a6b [HUDI-1800] Exclude file slices in pending compaction when performing small file sizing (#2902) rmpifer 2021-05-29 05:06:01 -07:00
  • 974b476180 [HUDI-1940] Add SqlQueryBasedTransformer unit test (#3004) wangxianghu 2021-05-28 22:30:30 +08:00
  • bc18c39835 [FLINK-1923] Exactly-once write for flink writer (#3002) yuzhaojing 2021-05-28 14:58:21 +08:00
  • 7fed7352bd [HUDI-1865] Make embedded time line service singleton (#2899) Danny Chan 2021-05-27 13:38:33 +08:00
  • 4eb6ef8144 [HUDI-1935] Updated Logger statement (#2996) Vinay Patil 2021-05-26 12:34:58 +05:30
  • 112732db81 [HUDI-1922] Bulk insert with row writer supports mor table (#2981) leesf 2021-05-26 00:40:22 +08:00
  • afa6bc0b10 [HUDI-1723] Fix path selector listing files with the same mod date (#2845) Raymond Xu 2021-05-25 07:19:10 -07:00
  • e7020748b5 [HUDI-1920] Set archived as the default value of HOODIE_ARCHIVELOG_FOLDER_PROP_NAME (#2978) wangxianghu 2021-05-25 16:29:55 +08:00
  • aba1eadbfc [HUDI-1919] Type mismatch when streaming read copy_on_write table using flink (#2986) Town 2021-05-25 11:36:43 +08:00
  • 369a849337 [HUDI-1873] collect() call causing issues with very large upserts (#2907) mpouttu 2021-05-23 22:29:01 -07:00
  • 6539813733 [MINOR] Update the javadoc of EngineType (#2979) wangxianghu 2021-05-22 19:44:08 +08:00
  • 685f77b5dd [HUDI-1740] Fix insert-overwrite API archival (#2784) Susu Dong 2021-05-22 05:52:13 +09:00
  • 99b14a78e3 [HUDI-1918] Fix incorrect keyBy field cause serious data skew, to avoid multiple subtasks write to a partition at the same time (#2972) zhangminglei 2021-05-21 21:59:47 +08:00
  • a96034d38d [HUDI-1888] Fix NPE when the nested partition path field has null value (#2957) Y Ethan Guo 2021-05-21 05:28:11 -07:00
  • 7c213f9f26 [HUDI-1917] Remove the metadata sync logic in HoodieFlinkWriteClient#preWrite because it is not thread safe (#2971) Danny Chan 2021-05-21 11:29:54 +08:00
  • 081061e14b [HUDI-1719] hive on spark/mr,Incremental query of the mor table, the partition field is incorrect (#2720) xiarixiaoyao 2021-05-20 23:00:08 +08:00
  • 928b09ea0b [HUDI-1871] Fix hive conf for Flink writer hive meta sync (#2968) swuferhong 2021-05-20 17:03:52 +08:00
  • 9b01d2f864 [HUDI-1915] Fix the file id for write data buffer before flushing (#2966) Danny Chan 2021-05-20 10:20:08 +08:00
  • ced068e1ee [MINOR] Remove unused method in BaseSparkCommitActionExecutor (#2965) wangxianghu 2021-05-20 10:18:07 +08:00
  • fe3f5c2d56 [HUDI-1913] Using streams instead of loops for input/output (#2962) zhangminglei 2021-05-19 09:13:38 +08:00
  • 5d1f592395 [HUDI-1806] Honoring skipROSuffix in spark ds (#2882) Sivabalan Narayanan 2021-05-18 19:11:39 -04:00
  • 7d2971d4e2 [HUDI-1911] Reuse the partition path and file group id for flink write data buffer (#2961) Danny Chan 2021-05-18 17:47:22 +08:00
  • 46a2399a45 [HUDI-1902] Global index for flink writer (#2958) Danny Chan 2021-05-18 13:55:38 +08:00
  • fcedbfcb58 [MINOR][hudi-client] Code-cleanup,remove redundant variable declarations (#2956) Roc Marshal 2021-05-17 13:34:42 +08:00