lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

Commit Graph

  • 64df98fc4a [HUDI-164] Fixes incorrect averageBytesPerRecord Bhavani Sudha Saktheeswaran 2019-08-30 16:29:23 -07:00
  • 93bc5e2153 HUDI-243 Rename HoodieInputFormat and HoodieRealtimeInputFormat to HoodieParquetInputFormat and HoodieParquetRealtimeInputFormat Balaji Varadarajan 2019-09-11 11:31:07 -07:00
  • d0b9b56b7d [HUDI-143] Excluding javax.* from utilities and spark bundles Vinoth Chandar 2019-09-02 19:46:10 -07:00
  • 7a973a6944 [HUDI-159] Redesigning bundles for lighter-weight integrations vinoth chandar 2019-09-02 16:15:55 -07:00
  • 0e6f078ec4 Fix logging in HoodieSparkSqlWriter Mehrotra 2019-09-05 13:47:24 -07:00
  • 07a0ea87ab [hotfix] fix typo leesf 2019-09-06 07:25:28 +08:00
  • 821e0dcffc [HUDI-236] Failed to close stream leesf 2019-09-03 15:07:23 +08:00
  • 555dd55c16 Support nested ordering fields Alex Filipchik 2019-08-29 16:22:01 -07:00
  • 8b150a3c6b [HUDI-230] Add missing Apache License in some files leesf 2019-08-30 17:09:47 +08:00
  • 376b59ae5f [HUDI-227] : DeltaStreamer Improvements : Commit empty input batch with progressing checkpoints and allow users to override configs through properties. Original PR : PR-805 and PR-806 (#863) Balaji Varadarajan 2019-08-30 09:13:34 -07:00
  • a6908ef44d HUDI-170 Updating hoodie record before inserting it into ExternalSpillableMap (#866) Balaji Varadarajan 2019-08-30 09:03:37 -07:00
  • 40dd4dd637 [HUDI-229] Fix mvn notice:generate issue in windows leesf 2019-08-30 12:36:21 +08:00
  • 5c2da6051e [HUDI-225] Create Hudi Timeline Server Fat Jar leesf 2019-08-29 16:26:34 +08:00
  • 5f9fa82f47 HUDI-124 : Exclude jdk.tools from hadoop-common and update Notice files (#858) Balaji Varadarajan 2019-08-28 16:20:47 -07:00
  • 00cfe72c5d [hotfix] change hoodie-timeline-*.jar to hudi-timeline-*.jar leesf 2019-08-28 16:42:18 +08:00
  • b44f8521f2 [HUDI-222] Rename main class path to org.apache.hudi.timeline.service.TimelineService in run_server.sh leesf 2019-08-28 16:14:15 +08:00
  • 41dbac6903 Fixed unit test Alex Filipchik 2019-08-27 19:19:52 -07:00
  • b5d4da7958 Addressing comments Alex Filipchik 2019-08-27 15:31:04 -07:00
  • baea4f3b82 Ignore dublicate of a compaction file Alex Filipchik 2019-08-27 14:44:54 -07:00
  • e0ab89b3ac [HUDI-223] Adding a way to infer target schema from the dataset after the transformation (#854) Alexander Filipchik 2019-08-28 04:48:38 -07:00
  • 78e0721507 [HUDI-159] Precursor cleanup to reduce build warnings Vinoth Chandar 2019-08-26 17:41:00 -07:00
  • c265b4948f HUDI-128 Preparing POM for release and snapshot builds (#851) Balaji Varadarajan 2019-08-26 08:52:36 -07:00
  • cd090871a1 [HUDI-159]: Pom cleanup and removal of com.twitter.parquet vinoth chandar 2019-08-25 05:34:51 -07:00
  • 6edf0b9def [HUDI-68] Pom cleanup & demo automation (#846) vinoth chandar 2019-08-22 20:18:50 -07:00
  • 92eed6aca8 [HUDI-82] Adds Presto integration in Docker demo (#847) Bhavani Sudha Saktheeswaran 2019-08-22 19:40:36 -07:00
  • 1b79ef7672 HUDI-212: Specify Charset to UTF-8 for IOUtils.toString (#837) leesf 2019-08-16 23:27:19 +08:00
  • 8f5e7ad5d9 [HUDI-205] Let checkstyle ban Java and Guava Optional instead of using Option provided by Hudi (#834) vinoyang 2019-08-14 08:13:52 +08:00
  • 4787076c6d HUDI-204 : Make MOR rollback idempotent and disable using rolling stats for small file selection (#833) Balaji Varadarajan 2019-08-13 17:13:30 -07:00
  • 8d37fbf0db Adding GPG Keys Nishith Agarwal 2019-08-12 11:03:25 -07:00
  • a4f9d7575f HUDI-123 Rename code packages/constants to org.apache.hudi (#830) Balaji Varadarajan 2019-08-11 17:48:17 -07:00
  • 722b6be04a [HUDI-153] Use com.uber.hoodie.common.util.Option instead of Java and Guava Optional yanghua 2019-08-06 14:20:42 +08:00
  • d288e32833 HUDI-171 delete tmp file in addShutDownHook garyli1019 2019-07-17 21:58:01 -07:00
  • ec965892b0 HUDI-149 - Remove platform dependencies and update NOTICE plugin Balaji Varadarajan 2019-08-01 08:54:09 -07:00
  • a066865bd6 - Adding HoodieCombineHiveInputFormat for COW tables (#811) n3nash 2019-08-03 08:44:01 -07:00
  • 1a29d46a57 - Fix realtime queries by removing COLUMN_ID and COLUMN_NAME cache in inputformat (#814) n3nash 2019-08-02 16:06:34 -07:00
  • 86b5fcdd33 Cache RDD to avoid recomputing data ingestion. Return result RDD after updating index so that this step is not skipped by chained actions on the same RDD venkatr 2019-07-24 17:55:38 -07:00
  • 8139ffd94c HUDI-197 Hive Sync and othe CLIs using bundle picking sources jar instead of binary jar Balaji Varadarajan 2019-08-02 05:12:47 -07:00
  • 8ddfa2ecda HUDI-178 : Add keys for vinoth to KEYS file vinothchandar 2019-08-02 04:11:53 -07:00
  • 69d2afd0a9 Update Keys with anchee@apache.org Anbu Cheeralan 2019-08-01 14:05:06 -04:00
  • 171901a9d0 Fix typo in hoodie-presto-bundle (#818) Luke Zhu 2019-08-01 08:51:57 -07:00
  • 6e0ff3a235 Generate Source Jars for bundle packages (#810) Balaji Varadarajan 2019-07-30 18:17:14 -07:00
  • e20b77be3b HUDI-92 : Making deltastreamer with DistributedTestSource also run locally Vinoth Chandar 2019-07-19 04:53:28 -07:00
  • 68464c7d02 [HUDI-181] Fix the Bold markdown grammar issue of README file (#808) vinoyang 2019-07-30 18:47:53 +08:00
  • e0648de2ef HUDI-175 - add an option to manually override the DeltaStreamer checkpoint (#798) eisig 2019-07-30 01:40:02 +08:00
  • 9265c7cc36 Add balaji gpg key to KEYS file Balaji Varadarajan 2019-07-28 20:05:29 -07:00
  • 83dab21ae1 Allow HoodieWrapperFileSystem to wrap other proxy file-system implementations with no getScheme implementation (#793) Balaji Varadarajan 2019-07-24 21:31:46 -07:00
  • 0b451b3a58 HUDI-140 : GCS: Log File Reading not working due to difference in seek() behavior for EOF Balaji Varadarajan 2019-07-15 17:29:34 -07:00
  • 9857c4b21c add jssc.stop() (#797) eisig 2019-07-19 20:01:45 +08:00
  • 6efa16317c Fixing default value for avro 1.7 which assumes NULL value instead of a jsonnode that is null (#792) n3nash 2019-07-17 03:25:54 -07:00
  • 3d408ee96b HUDI-168 Ensure getFileStatus calls for files getting written is done after close() is called (#788) Balaji Varadarajan 2019-07-16 17:33:34 -07:00
  • c0593e7a13 fix HoodieLogFileReader (#787) eisig 2019-07-16 04:25:55 +08:00
  • ae3c02fb3f HUDI-162 : File System view must be built with correct timeline actions Balaji Varadarajan 2019-07-01 18:19:12 -07:00
  • 5823c1ebd7 HUDI-138 - Meta Files handling also need to support consistency guard Balaji Varadarajan 2019-06-20 18:05:01 -07:00
  • 621c246fa9 [HUDI-161] Remove --key-generator-class CLI arg in HoodieDeltaStreamer and use key generator class specified in datasource properties. (#781) Yihua Guo 2019-07-12 13:45:49 -07:00
  • 11c4121f73 Fixed TableNotFoundException when write with structured streaming (#778) Ho Tien Vu 2019-07-13 00:17:16 +08:00
  • 62ecb2da62 when column type is decimal, should add precision and scale (#753) Thinking Chen 2019-07-09 07:13:22 +08:00
  • 9f18a1ca80 Fixing bugs found during running hoodie demo (#760) Balaji Varadarajan 2019-06-28 17:49:23 -07:00
  • e48e35385a Added preemptive check for 'spark.scheduler.mode' Ho Tien Vu 2019-06-25 00:42:31 +08:00
  • 17e878f721 adding support for complex keys (#728) Jaimin Shah 2019-06-21 12:55:06 +05:30
  • 1b61eb45e0 Adding support for optional skipping single archiving failures Ron Barabash 2019-06-19 09:46:53 +03:00
  • 66c7fa2322 Reword confusing message and reducing the severity level Balaji Varadarajan 2019-06-19 14:15:02 -07:00
  • 8223127611 Add maprfs to storage schemes Balaji Varadarajan 2019-06-20 12:34:14 -07:00
  • 2c40e8419e Ensure TableMetaClient and FileSystem instances have exclusive copy of Configuration Balaji Varadarajan 2019-06-20 08:34:56 -07:00
  • a0d7ab2384 HUDI-70 : Making DeltaStreamer run in continuous mode with concurrent compaction Balaji Varadarajan 2019-05-15 13:21:55 -07:00
  • 3a210ef08e Disable Notice Plugin Balaji Varadarajan 2019-06-18 11:28:20 -07:00
  • a1483f2c5f HUDI-148 Small File selection logic for MOR must skip fileIds selected for pending compaction correctly Balaji Varadarajan 2019-06-08 12:40:08 -07:00
  • 8c9980f4f5 Update README.md vinoth chandar 2019-06-17 18:19:34 -07:00
  • 8e08d498c9 Reading baseCommitTime from the latest file slice as opposed to the tagged record value Nishith Agarwal 2019-06-14 14:22:48 -07:00
  • 129e433641 - Ugrading to Hive 2.x - Eliminating in-memory deltaRecordsMap - Use writerSchema to generate generic record needed by custom payloads - changes to make tests work with hive 2.x Nishith Agarwal 2019-05-10 13:09:09 -07:00
  • cd7623e216 All Opened hoodie clients in tests needs to be closed TestMergeOnReadTable must use embedded timeline server Balaji Varadarajan 2019-06-12 18:28:49 -07:00
  • 136f8478a3 TestMergeOnReadTable must use embedded timeline server Balaji Varadarajan 2019-06-12 18:28:49 -07:00
  • 04fc86b43d Turn on embedded server for all client tests Balaji Varadarajan 2019-06-12 16:50:13 -07:00
  • 1c943ab230 Ensure log files are consistently ordered when scanning Balaji Varadarajan 2019-06-11 19:06:06 -07:00
  • b791473a6d Introduce HoodieReadHandle abstraction into index Vinoth Chandar 2019-05-21 15:37:38 -07:00
  • 51d122b5c3 Close Hoodie Clients which are opened to properly shutdown embedded timeline service Balaji Varadarajan 2019-06-11 12:58:58 -07:00
  • 065173211e HUDI-147 Compaction Inflight Rollback not deleting Marker directory Balaji Varadarajan 2019-06-08 01:45:19 -07:00
  • 479908fd20 HUDI-125 : Change License for all source files and update RAT configurations Balaji Varadarajan 2019-06-08 22:31:47 -07:00
  • 30b0f2636f Changes related to Licensing work 1. Go through dependencies list one round to ensure compliance. Generated current NOTICE list in all submodules (other apache projects like flink does this). To be on conservative side regarding licensing, NOTICE.txt lists all dependencies including transitive. Pending Compliance questions reported in https://issues.apache.org/jira/browse/LEGAL-461 2. Automate generating NOTICE.txt files to allow future package compliance issues be identified early as part of code-review process. 3. Added NOTICE.txt and LICENSE.txt to all HUDI jars Balaji Varadarajan 2019-06-05 18:17:55 -07:00
  • 173e0b6be4 exlude fasterxml and parquet from presto bundle guanjianhui 2019-06-06 16:51:16 +08:00
  • b325cbff10 set codehaus.jackson modules to the same version 1.9.13 guanjianhui 2019-06-03 10:16:06 +08:00
  • 45e65cc2f7 Auto generated Slack Channel Notifications setup Balaji Varadarajan 2019-06-05 19:34:11 -07:00
  • 5ae34db764 Replace Non-Compliant dnl.utils package with Apache 2.0 licensed alternative Balaji Varadarajan 2019-06-06 17:15:45 -07:00
  • a0391b7c01 LogFile comparator must handle log file names without write token for backwards compatibility Balaji Varadarajan 2019-06-05 18:18:44 -07:00
  • 66893bfef2 fix spark-shell add jar problem Thinking 2019-06-02 21:25:21 +08:00
  • 7b4a28ecf8 Move depedency repos to https urls Vinoth Chandar 2019-05-31 19:21:41 -07:00
  • acd74129cd Create hoodie-utilities-bundle to host the shaded jar Vinoth Chandar 2019-05-30 20:06:26 -07:00
  • a5e2439514 Turn off noisy test Vinoth Chandar 2019-05-30 20:21:54 -07:00
  • 3b916ec1af Add support for maven deploy plugin to make snapshot releases Vinoth Chandar 2019-05-30 18:53:56 -07:00
  • 6b5abb5d92 fix maven pom guanjianhui 2019-05-22 11:44:32 +08:00
  • d860fb18b6 HUDI-139 Compaction running twice due to duplicate "map" transformation while finalizing compaction Balaji Varadarajan 2019-05-29 12:00:35 -07:00
  • 66c0b81b49 [maven-release-plugin] prepare for next development iteration vinothchandar 2019-05-28 19:17:26 -07:00
  • 227785c022 [maven-release-plugin] prepare release hoodie-0.4.7 vinothchandar 2019-05-28 19:17:15 -07:00
  • a1f287d359 Release notes for 0.4.7 Vinoth Chandar 2019-05-28 18:26:40 -07:00
  • 93f8f12a30 HUDI-135 - Skip Meta folder when looking for partitions Balaji Varadarajan 2019-05-28 12:54:23 -07:00
  • 33f5208c1e Only inflight commit timeline (.commit/.deltacommit) must be used when checking for sanity during compaction scheduling Balaji Varadarajan 2019-05-28 15:17:54 -07:00
  • 9c8f8212ef HUDI-134 - Disable inline compaction for Hoodie Demo Balaji Varadarajan 2019-05-27 20:44:19 -07:00
  • d0d2fa0337 Reduce logging in unit-test runs Balaji Varadarajan 2019-05-24 22:20:10 -07:00
  • f2d91a455e default implementation for HBase index qps allocator (#685) Venkat 2019-05-24 18:43:46 -07:00
  • 99b0c72aa6 HUDI-131 Zero FIle Listing in Compactor run Balaji Varadarajan 2019-05-24 16:51:09 -07:00
  • 4074c5eb23 Fixed HUDI-116 : Handle duplicate record keys across partitions Vinoth Chandar 2019-05-21 18:59:10 -07:00