lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity
180 Commits 1 Branch 0 Tags
e8b3ddd7cbf4232d3e02fa9bcb71c100a5aea26f
Commit Graph

180 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Vinoth Chandar
e8b3ddd7cb Add note on community engagement to committership guidelines 2017-07-08 22:47:15 -07:00
Prasanna Rajaperumal
7d3963b4ab Pushing master to 0.4.0 as we continue to make minor releases over 0.3.8 (MVP for MOR) 2017-06-30 11:41:23 -07:00
Nishith Agarwal
3eba812a1b [maven-release-plugin] prepare for next development iteration 2017-06-30 11:17:07 -07:00
Nishith Agarwal
06d44daea3 [maven-release-plugin] prepare release hoodie-0.3.9 2017-06-30 11:16:58 -07:00
Nishith Agarwal
348250d960 Using FsUtils instead of Files API to extract file extension 2017-06-29 19:26:31 -07:00
Nishith Agarwal
e5d9b818bc Sync Tool registers 2 tables, RO and RT Tables 2017-06-28 15:41:36 -07:00
Prasanna Rajaperumal
5cc071f74e Savepoint should not create a hole in the commit timeline 2017-06-27 16:36:09 -07:00
Jian Xu
29b906b763 Fix TimestampBasedKeyGenerator when DATE_STRING is used for partitionpath.field 2017-06-27 13:02:06 -07:00
Vinoth Chandar
754ab88a2d Introduce ReadOptimizedView & RealtimeView out of TableFileSystemView 
- Usage now marks code as clearly using either RO or RT views, for future evolution
  - Tests on all of FileGroups and FileSlices
 2017-06-22 17:16:13 -07:00
Vinoth Chandar
c00f1a9ed9 Refactoring HoodieTableFileSystemView using FileGroups/FileSlices 
- Merged all filter* and get* methods
 - new constructor takes filestatus[]
 - All existing tests pass
 - FileGroup is all files that belong to a fileID within a partition
 - FileSlice is a generation of data and log files, starting at a base commit
 2017-06-22 17:16:13 -07:00
Vinoth Chandar
23e7badd8a Rename IO Handles & introduce stub for BucketedIndex 
- UpdateHandle -> MergeHandle, InsertHandle -> CreateHandle
 - Also bunch of code cleanup in different places
 2017-06-22 17:16:13 -07:00
prazanna
b0a2a23372 Adding Nishith to Contributors list 2017-06-20 15:48:43 -07:00
prazanna
649475c5cb Adding Kaushik to contributors list 2017-06-20 15:47:05 -07:00
Kaushik Devarajaiah
3aa8083913 Correct clean bug that causes clean failure when partitionPaths are empty 2017-06-20 15:45:32 -07:00
prazanna
7ef76a4de0 Adding Kathy Ge to the contributors list 2017-06-16 12:52:54 -07:00
gekath
52c507f83e Writes relative paths to .commit files 
Handle case where path is read in as null from commit file

Merged with updated release
 2017-06-16 12:51:19 -07:00
gekath
db7311f85e Writes relative paths to .commit files instead of absolute paths 
Clean up code

Removed commented out code

Fixed merge conflict with master
 2017-06-16 12:51:19 -07:00
Prasanna Rajaperumal
0ed3fac5e3 [maven-release-plugin] prepare for next development iteration 2017-06-16 11:03:17 -07:00
Prasanna Rajaperumal
45732e440c [maven-release-plugin] prepare release hoodie-0.3.8 2017-06-16 10:59:58 -07:00
Prasanna Rajaperumal
e44f9b889b Added CHANGELOG.md and updated community contributions guideline 2017-06-16 10:48:37 -07:00
Prasanna Rajaperumal
4b26be9f61 Fixes to RealtimeInputFormat and RealtimeRecordReader and update documentation for HiveSyncTool 2017-06-15 18:21:07 -07:00
Kaushik Devarajaiah
521555c576 Parallelize file version deletes during clean and related tests 2017-06-15 18:20:42 -07:00
Prasanna Rajaperumal
dda28c0b4b Rollback inflight commits as well when rolling back to savepoint 2017-06-14 11:03:27 -07:00
Prasanna Rajaperumal
db6150c5ef Refactor hoodie-hive 2017-06-09 13:06:33 -07:00
Danny Chen
c192dd60b4 Change from deprecated closeQuietly to try with resources 2017-06-05 19:11:53 -07:00
n3nash
36fa6f66ac Merge pull request #180 from n3nash/record-reader-with-compaction 
updated HoodieRealtimeRecordReader to use HoodieCompactedLogRecordSca…
 2017-06-02 11:49:32 -07:00
Nishith Agarwal
ba050973e3 updated HoodieRealtimeRecordReader to use HoodieCompactedLogRecordScanner, added test for recordreader 2017-06-02 11:33:59 -07:00
Prasanna Rajaperumal
933cc8071f [maven-release-plugin] prepare for next development iteration 2017-05-24 14:02:50 -07:00
Prasanna Rajaperumal
bebae06b5b [maven-release-plugin] prepare release hoodie-0.3.7 2017-05-24 14:02:41 -07:00
Prasanna Rajaperumal
bae98efeee Delete other instant files (.clean) as well during commit archival 2017-05-24 13:51:49 -07:00
prazanna
e1d13f2bc8 https://repository.cloudera.com/artifactory/repo/ has been changed to https://repository.cloudera.com/artifactory/public/ 2017-05-23 12:05:01 -07:00
Prasanna Rajaperumal
240c91241b Implement HoodieLogFormat replacing Avro as the default log format 2017-05-23 08:35:11 -07:00
Nishith Agarwal
3c984447da view scheme added 2017-05-22 12:27:40 -07:00
Prasanna Rajaperumal
70dd7a25ea Clean should not create a .inflight file 2017-05-22 10:48:35 -07:00
Vinoth Chandar
7014670795 Update contributor list 2017-05-18 10:48:42 -07:00
Zeeshan Qureshi
43a55b09fd Add GCS to supported filesystems 2017-05-18 10:30:34 -07:00
prazanna
21e334592f Update java version to 8 in travis.yml 2017-05-17 13:43:11 -07:00
vinoth chandar
1b0a027942 Update community.md with committership guidelines 2017-05-04 17:25:54 -07:00
Vinoth Chandar
b4e787ce1d Update docs 2017-05-01 21:48:27 -07:00
Vinoth Chandar
da17c5c607 Introduce getCommitsAndCompactionsTimeline() explicitly & adjust usage across code base 2017-05-01 21:48:27 -07:00
Vinoth Chandar
bae0528013 Cleanup calls to HoodieTimeline.compareTimeStamps 2017-05-01 21:48:27 -07:00
Vinoth Chandar
7b1446548f Initial impl of HoodieRealtimeInputFormat 
- Works end-end for flat schemas
 - Schema evolution & hardening remains
 - HoodieClientExample can now write mor tables as well
 2017-05-01 21:48:27 -07:00
Vinoth Chandar
9f526396a0 Add support for merge_on_read tables to HoodieClientExample 2017-05-01 21:48:27 -07:00
Prasanna Rajaperumal
7bca428a0a Test to check if properties set are properly propogated 2017-04-28 12:47:14 -07:00
Prasanna Rajaperumal
3f97bdcccf Test to check if properties set are properly propogated 2017-04-28 12:40:58 -07:00
Prasanna Rajaperumal
c3258039f0 [maven-release-plugin] prepare for next development iteration 2017-04-27 11:00:56 -07:00
Prasanna Rajaperumal
de1bdad756 [maven-release-plugin] prepare release hoodie-0.3.6 2017-04-27 11:00:45 -07:00
Prasanna Rajaperumal
8974e11161 Make sure properties set in HoodieWriteConfig is propogated down to individual configs. Fix a race condition which lets InputFormat to think file size is 0 when it is actually not 2017-04-27 10:52:25 -07:00
Prasanna Rajaperumal
91b088f29f Implement Compaction policy abstraction. Implement LogSizeBased Bounded IO Compaction as the default strategy 2017-04-20 16:59:06 -07:00
Vinoth Chandar
82b211d2e6 Rebase with generic partition support 2017-04-03 21:27:49 -07:00