lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

Commit Graph

  • be0b1f3e57 Adding global indexing to HbaseIndex implementation - Adding tests or HbaseIndex - Enabling global index functionality Nishith Agarwal 2017-12-15 21:29:02 -08:00
  • 15e669c60c Incorporating code review feedback for finalizeWrite for COW #4 Jian Xu 2018-01-30 11:18:00 -08:00
  • 3736243fb3 Rebases with latest upstream Jian Xu 2018-01-18 12:39:30 -08:00
  • 363e35bb0f Add finalizeWrite support for HoodieMergeHandle Jian Xu 2018-01-18 11:29:10 -08:00
  • acae6586f3 Incorporating code review feedback for finalizeWrite for COW #3 Jian Xu 2017-12-21 09:58:51 -08:00
  • 37f2cdd7e4 Incorporating code review feedback for finalizeWrite for COW #2 Jian Xu 2017-12-14 13:36:32 -08:00
  • 2fe4fef625 Incorporating code review feedback for finalizeWrite for COW Jian Xu 2017-12-06 14:35:44 -08:00
  • c874248f23 Add FinalizeWrite in HoodieCreateHandle for COW tables Jian Xu 2017-11-29 16:59:28 -08:00
  • e10100fe32 Reducing list status calls from listing logfile versions, some associated refactoring Nishith Agarwal 2018-01-23 15:10:43 -08:00
  • 937ae322ba Reducing memory footprint required in HoodieAvroDataBlock and HoodieAppendHandle Nishith Agarwal 2017-12-15 14:03:06 -08:00
  • 85d32930cd Update Gemfile.lock Vinoth Chandar 2018-01-18 00:05:22 -08:00
  • 21ce846f18 Remove stateful fs member from HoodieTestUtils & FSUtils vinothchandar 2018-01-03 16:05:30 -08:00
  • cf7f7aabb9 Nicer handling of timeline archival for Cloud storage vinothchandar 2018-01-03 04:32:21 -08:00
  • 0cd186c899 Multi FS Support Vinoth Chandar 2017-12-10 23:31:54 -08:00
  • 44839b88c6 Removing compaction action type and associated compaction timeline operations, replace with commit action type Nishith Agarwal 2017-12-05 00:58:53 -08:00
  • a1c0d0dbad Update README.md vinoth chandar 2017-12-10 07:50:37 -08:00
  • 4aed5c7338 Adding a new Partition/Time based compaction strategy Nishith Agarwal 2017-10-25 13:30:15 -07:00
  • 051f600b7f Enable hive sync even if there is no compaction commit Nishith Agarwal 2017-11-30 17:21:34 -08:00
  • 9b610f82c7 Separating out compaction() API Nishith Agarwal 2017-11-13 10:36:33 -08:00
  • e45679f5e2 Reformatting code per Google Code Style all over Vinoth Chandar 2017-11-12 22:54:56 -08:00
  • 5a62480a92 Update docs on code style setup Vinoth Chandar 2017-11-12 22:21:21 -08:00
  • abe964bebd Implementing custom payload/merge hooks abstractions for application specific merge logic Nishith Agarwal 2017-09-26 11:16:35 -07:00
  • c7d63a7622 1) Separated rollback as a table operation 2) Implement rollback for MOR Nishith Agarwal 2017-08-17 17:31:55 -07:00
  • e1fe3ab937 [maven-release-plugin] prepare for next development iteration Vinoth Chandar 2017-10-02 22:42:54 -07:00
  • 50139fe904 [maven-release-plugin] prepare release hoodie-0.4.0 Vinoth Chandar 2017-10-02 22:42:32 -07:00
  • 3768ad45fb Release notes for 0.4.0 Vinoth Chandar 2017-10-02 22:24:21 -07:00
  • 274aaf49fe Incorporating code review feedback for DataSource Vinoth Chandar 2017-10-02 20:29:07 -07:00
  • 64e0573aca Adding hoodie-spark to support Spark Datasource for Hoodie Vinoth Chandar 2017-08-28 01:28:08 -07:00
  • c98ee057fc capture record metadata before deflating for record counting Kaushik Devarajaiah 2017-08-22 14:48:04 -07:00
  • f2980052cd Revert effects of PR #259 Vinoth Chandar 2017-09-28 07:24:39 -07:00
  • 9f98ae643b Adding canIndexLogFiles(), isImplicitWithStorage(), isGlobal() to HoodieIndex Vinoth Chandar 2017-09-28 09:57:28 -07:00
  • 6230e15191 Update deprecated hash function Eric Sayle 2017-09-18 16:14:09 -07:00
  • 7e9a4a89dd Use getFileStatus to get single FileStatus for single file Jian Xu 2017-09-06 15:30:15 -07:00
  • 5c639c0b05 Adding support for UserDefinedBulkInsertPartitioner Omkar Joshi 2017-09-08 16:09:39 -07:00
  • ec40d04d51 Fixing UpsertPartitioner to ensure that input records are deterministically assigned to output partitions Omkar Joshi 2017-09-02 14:59:10 -07:00
  • e2d13c6305 Fix build failing issues Nishith Agarwal 2017-08-14 11:57:38 -07:00
  • 63f1b12355 adding ability to read archived files written in log format Nishith Agarwal 2017-08-25 14:12:19 -07:00
  • e484e91807 adding new config to separate shuffle and write parallelism Nishith Agarwal 2017-07-26 11:49:22 -07:00
  • b1cf097b0c Add nested fields support for MOR tables Jian Xu 2017-06-30 17:03:44 -07:00
  • 6a3c94aaa3 suppressing logs (under 4MB) for jenkins Nishith Agarwal 2017-08-14 11:45:36 -07:00
  • 5ee4ac40ae Use CompletedFileSystemView instead of CompactedView considering deltacommits Nishith Agarwal 2017-07-25 17:25:41 -07:00
  • 45dd8980c3 Temporary fix for build break after rebase Vinoth Chandar 2017-08-04 17:00:40 -07:00
  • 86209640f7 Adding range based pruning to bloom index Vinoth Chandar 2017-07-14 09:29:16 -07:00
  • 0b26b60a5c fix for cleaning log files(mor) Nishith Agarwal 2017-06-29 00:32:11 -07:00
  • 19c22b231e 1. Use HoodieLogFormat to archive commits and other actions 2. Introduced avro schema for commits and compactions and an avro wrapper schema Nishith Agarwal 2017-06-20 23:50:23 -07:00
  • 616c9a68c3 Enabled deletes in merge_on_read Nishith Agarwal 2017-07-07 12:50:01 -07:00
  • cf1dde0323 Add recent talks/presentations to documentation Vinoth Chandar 2017-07-08 22:45:22 -07:00
  • e8b3ddd7cb Add note on community engagement to committership guidelines Vinoth Chandar 2017-07-08 21:30:06 -07:00
  • 7d3963b4ab Pushing master to 0.4.0 as we continue to make minor releases over 0.3.8 (MVP for MOR) Prasanna Rajaperumal 2017-06-30 11:41:23 -07:00
  • 3eba812a1b [maven-release-plugin] prepare for next development iteration Nishith Agarwal 2017-06-30 11:17:07 -07:00
  • 06d44daea3 [maven-release-plugin] prepare release hoodie-0.3.9 Nishith Agarwal 2017-06-30 11:16:58 -07:00
  • 348250d960 Using FsUtils instead of Files API to extract file extension Nishith Agarwal 2017-06-29 16:42:58 -07:00
  • e5d9b818bc Sync Tool registers 2 tables, RO and RT Tables Nishith Agarwal 2017-06-22 20:33:33 -07:00
  • 5cc071f74e Savepoint should not create a hole in the commit timeline Prasanna Rajaperumal 2017-06-22 15:00:27 -07:00
  • 29b906b763 Fix TimestampBasedKeyGenerator when DATE_STRING is used for partitionpath.field Jian Xu 2017-06-27 12:00:37 -07:00
  • 754ab88a2d Introduce ReadOptimizedView & RealtimeView out of TableFileSystemView Vinoth Chandar 2017-06-19 17:16:45 -07:00
  • c00f1a9ed9 Refactoring HoodieTableFileSystemView using FileGroups/FileSlices Vinoth Chandar 2017-06-19 00:34:37 -07:00
  • 23e7badd8a Rename IO Handles & introduce stub for BucketedIndex Vinoth Chandar 2017-05-14 09:44:44 -07:00
  • b0a2a23372 Adding Nishith to Contributors list prazanna 2017-06-20 15:48:43 -07:00
  • 649475c5cb Adding Kaushik to contributors list prazanna 2017-06-20 15:47:05 -07:00
  • 3aa8083913 Correct clean bug that causes clean failure when partitionPaths are empty Kaushik Devarajaiah 2017-06-20 00:16:54 -07:00
  • 7ef76a4de0 Adding Kathy Ge to the contributors list prazanna 2017-06-16 12:52:54 -07:00
  • 52c507f83e Writes relative paths to .commit files gekath 2017-06-13 13:35:38 -04:00
  • db7311f85e Writes relative paths to .commit files instead of absolute paths gekath 2017-06-02 11:28:47 -04:00
  • 0ed3fac5e3 [maven-release-plugin] prepare for next development iteration Prasanna Rajaperumal 2017-06-16 11:03:17 -07:00
  • 45732e440c [maven-release-plugin] prepare release hoodie-0.3.8 Prasanna Rajaperumal 2017-06-16 10:59:58 -07:00
  • e44f9b889b Added CHANGELOG.md and updated community contributions guideline Prasanna Rajaperumal 2017-06-16 10:47:25 -07:00
  • 4b26be9f61 Fixes to RealtimeInputFormat and RealtimeRecordReader and update documentation for HiveSyncTool Prasanna Rajaperumal 2017-06-15 05:40:59 -07:00
  • 521555c576 Parallelize file version deletes during clean and related tests Kaushik Devarajaiah 2017-05-29 15:36:12 -07:00
  • dda28c0b4b Rollback inflight commits as well when rolling back to savepoint Prasanna Rajaperumal 2017-06-13 12:09:36 -07:00
  • db6150c5ef Refactor hoodie-hive Prasanna Rajaperumal 2017-05-19 23:47:27 -07:00
  • c192dd60b4 Change from deprecated closeQuietly to try with resources Danny Chen 2017-05-31 17:41:27 -07:00
  • 36fa6f66ac Merge pull request #180 from n3nash/record-reader-with-compaction n3nash 2017-06-02 11:49:32 -07:00
  • ba050973e3 updated HoodieRealtimeRecordReader to use HoodieCompactedLogRecordScanner, added test for recordreader Nishith Agarwal 2017-05-31 15:06:22 -07:00
  • 933cc8071f [maven-release-plugin] prepare for next development iteration Prasanna Rajaperumal 2017-05-24 14:02:50 -07:00
  • bebae06b5b [maven-release-plugin] prepare release hoodie-0.3.7 Prasanna Rajaperumal 2017-05-24 14:02:41 -07:00
  • bae98efeee Delete other instant files (.clean) as well during commit archival Prasanna Rajaperumal 2017-05-18 09:57:44 -07:00
  • e1d13f2bc8 https://repository.cloudera.com/artifactory/repo/ has been changed to https://repository.cloudera.com/artifactory/public/ prazanna 2017-05-23 12:05:01 -07:00
  • 240c91241b Implement HoodieLogFormat replacing Avro as the default log format Prasanna Rajaperumal 2017-05-17 12:52:36 -07:00
  • 3c984447da view scheme added Nishith Agarwal 2017-05-18 15:10:28 -07:00
  • 70dd7a25ea Clean should not create a .inflight file Prasanna Rajaperumal 2017-05-19 23:48:31 -07:00
  • 7014670795 Update contributor list Vinoth Chandar 2017-05-18 10:45:26 -07:00
  • 43a55b09fd Add GCS to supported filesystems Zeeshan Qureshi 2017-03-16 12:19:13 -04:00
  • 21e334592f Update java version to 8 in travis.yml prazanna 2017-05-17 13:43:11 -07:00
  • 1b0a027942 Update community.md with committership guidelines vinoth chandar 2017-05-04 17:25:54 -07:00
  • b4e787ce1d Update docs Vinoth Chandar 2017-04-26 14:03:06 -07:00
  • da17c5c607 Introduce getCommitsAndCompactionsTimeline() explicitly & adjust usage across code base Vinoth Chandar 2017-04-26 13:36:49 -07:00
  • bae0528013 Cleanup calls to HoodieTimeline.compareTimeStamps Vinoth Chandar 2017-04-26 11:00:50 -07:00
  • 7b1446548f Initial impl of HoodieRealtimeInputFormat Vinoth Chandar 2017-04-24 08:19:02 -07:00
  • 9f526396a0 Add support for merge_on_read tables to HoodieClientExample Vinoth Chandar 2017-04-04 05:17:38 -07:00
  • 7bca428a0a Test to check if properties set are properly propogated Prasanna Rajaperumal 2017-04-28 12:47:14 -07:00
  • 3f97bdcccf Test to check if properties set are properly propogated Prasanna Rajaperumal 2017-04-28 12:40:58 -07:00
  • c3258039f0 [maven-release-plugin] prepare for next development iteration Prasanna Rajaperumal 2017-04-27 11:00:56 -07:00
  • de1bdad756 [maven-release-plugin] prepare release hoodie-0.3.6 Prasanna Rajaperumal 2017-04-27 11:00:45 -07:00
  • 8974e11161 Make sure properties set in HoodieWriteConfig is propogated down to individual configs. Fix a race condition which lets InputFormat to think file size is 0 when it is actually not Prasanna Rajaperumal 2017-04-27 10:37:30 -07:00
  • 91b088f29f Implement Compaction policy abstraction. Implement LogSizeBased Bounded IO Compaction as the default strategy Prasanna Rajaperumal 2017-04-04 12:37:28 -07:00
  • 82b211d2e6 Rebase with generic partition support Vinoth Chandar 2017-04-03 18:23:47 -07:00
  • 848814bece Adding docs for deltastreamer, hivesync tool usage Vinoth Chandar 2017-04-03 16:59:44 -07:00
  • 542d622e49 Adding HiveSyncTool to sync hoodie dataset schema/partitions to Hive Vinoth Chandar 2017-04-03 14:44:43 -07:00
  • 2b6322318c CR feedback Vinoth Chandar 2017-04-03 18:17:56 -07:00