lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity
718 Commits 1 Branch 0 Tags
5af3dc6aedcfc8d7d4cd5da21300d9cfb92c16ba
Commit Graph

718 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
hongdd
5af3dc6aed [HUDI-331]Fix java docs for all public apis in HoodieWriteClient (#1111) 2020-01-09 16:00:53 +08:00
Bhavani Sudha Saktheeswaran
d09eacdc13 [HUDI-25] Optimize HoodieInputformat.listStatus() for faster Hive incremental queries on Hoodie 
Summary:
    - InputPathHandler class classifies  inputPaths into incremental, non incremental and non hoodie paths.
    - Incremental queries leverage HoodieCommitMetadata to get partitions that are affected and only lists those partitions as opposed to listing all partitions
    - listStatus() processes each category separately
 2020-01-08 14:53:05 -08:00
Y Ethan Guo
480fc7869d [HUDI-319] Add a new maven profile to generate unified Javadoc for all Java and Scala classes (#1195) 
* Add javadoc build command in README, links to javadoc plugin and rename profile.
* Make java version configurable in one place.
 2020-01-08 10:38:09 -08:00
Wenning Ding
aba83876e7 Update deprecated HBase API 2020-01-08 10:26:47 -08:00
lamber-ken
9884972a3a [MINOR] Remove old jekyll config file (#1198) 2020-01-07 16:45:19 -08:00
vinoth chandar
9706f659db [HUDI-508] Standardizing on "Table" instead of "Dataset" across code (#1197) 
- Docs were talking about storage types before, cWiki moved to "Table"
 - Most of code already has HoodieTable, HoodieTableMetaClient - correct naming
 - Replacing renaming use of dataset across code/comments
 - Few usages in comments and use of Spark SQL DataSet remain unscathed
 2020-01-07 12:52:32 -08:00
Balaji Varadarajan
8306f749a2 [HUDI-417] Refactor HoodieWriteClient so that commit logic can be shareable by both bootstrap and normal write operations (#1166) 2020-01-06 20:11:48 -08:00
Abhishek Modi
b5df6723a2 [HUDI-464] Use Hive Exec Core for tests (#1125) 2020-01-06 16:32:55 -08:00
lamber-ken
75c3f630d4 [HUDI-405] Remove HIVE_ASSUME_DATE_PARTITION_OPT_KEY config from DataSource 2020-01-06 14:25:38 -08:00
yuehan124
c78092d2d3 [HUDI-501] Execute docker/setup_demo.sh in any directory 2020-01-06 10:26:06 -08:00
Pratyaksh Sharma
8f935e779a [HUDI-406]: added default partition path in TimestampBasedKeyGenerator 2020-01-06 09:38:06 -08:00
hongdd
2d5b79d96f [HUDI-438] Merge duplicated code fragment in HoodieSparkSqlWriter (#1114) 2020-01-06 22:51:22 +08:00
hejinbiao123
b9fab0b933 Revert "[HUDI-455] Redo hudi-client log statements using SLF4J (#1145)" (#1181) 
This reverts commit e637d9ed26.
 2020-01-06 21:13:29 +08:00
lamber-ken
28ccf8c521 [HUDI-484] Fix NPE when reading IncrementalPull.sqltemplate in HiveIncrementalPuller (#1167) 2020-01-04 23:53:47 -08:00
Sivabalan Narayanan
7031445eb3 [HUDI-377] Adding Delete() support to DeltaStreamer (#1073) 
- Provides ability to perform hard deletes by writing delete marker records into the source data
- if the record contains a special field _hoodie_delete_marker set to true, deletes are performed
 2020-01-04 11:07:31 -08:00
SteNicholas
726ae47ce2 [MINOR]Optimize hudi-client module (#1139) 2020-01-04 10:57:08 -08:00
SteNicholas
a733f4ef72 [MINOR] Optimize hudi-cli module (#1136) 2020-01-04 09:05:50 -08:00
Pratyaksh Sharma
290278fc6c [HUDI-118]: Options provided for passing properties to Cleaner, compactor and importer commands 2020-01-03 16:00:57 -08:00
hongdongdong
ff1113f3b7 [HUDI-492]Fix show env all in hudi-cli 2020-01-03 15:50:20 -08:00
lamber-ken
e1e5fe3324 [MINOR] Fix error usage of String.format (#1169) 2020-01-02 09:11:15 +08:00
Pratyaksh Sharma
dde21e7315 [HUDI-402]: code clean up in test cases 2019-12-31 11:10:49 -08:00
Suneel Marthi
98c0d8cf60 Merge pull request #1160 from smarthi/HUDI-343 
[HUDI-343]: Create a DOAP file for Hudi
 2019-12-31 04:21:31 -05:00
Suneel Marthi
47c1f746e2 [HUDI-343]: Create a DOAP file for Hudi 2019-12-31 03:49:21 -05:00
Suneel Marthi
add4b1e329 Merge pull request #1143 from BigDataArtisans/outoflimit 
[MINOR] Fix out of limits for results
 2019-12-31 02:08:54 -05:00
lamber-ken
619f501054 Clean up code 2019-12-31 13:59:26 +08:00
hejinbiao123
e637d9ed26 [HUDI-455] Redo hudi-client log statements using SLF4J (#1145) 
* [HUDI-455] Redo hudi-client log statements using SLF4J
 2019-12-31 13:49:34 +08:00
lamber-ken
ab6ae5cebb [HUDI-482] Fix missing @Override annotation on methods (#1156) 
* [HUDI-482] Fix missing @Override annotation on methods
 2019-12-31 11:44:56 +08:00
dengziming
2a823f32ee [MINOR]: alter some wrong params which bring fatal exception 2019-12-30 16:50:12 -08:00
lamber-ken
e4ea7a2971 Update comment 2019-12-29 19:03:56 -08:00
lamber-ken
8440482977 Fix empty content clean plan 2019-12-29 19:03:56 -08:00
lamber-ken
2f254163d4 Skip setting commit metadata 2019-12-29 19:03:56 -08:00
lamber-ken
179837e8ef Fix checkstyle 2019-12-29 19:03:56 -08:00
lamber-ken
58c5bed40a [HUDI-453] Fix throw failed to archive commits error when writing data to MOR/COW table 2019-12-29 19:03:56 -08:00
Mathieu
01c25d6aff [MINOR] Update the java doc of HoodieTableType (#1148) 2019-12-29 09:57:19 +08:00
lamber-ken
74b00d1f95 trigger rebuild 2019-12-27 02:06:26 +08:00
lamber-ken
36c0e6bae1 [MINOR] Fix out of limits for results 2019-12-27 01:16:24 +08:00
lamber-ken
bb90dedfc8 [MINOR] Fix out of limits for results 2019-12-27 01:13:47 +08:00
yungthuis66
f20a130e3a [MINOR] typo fix (#1142) 2019-12-26 09:03:43 -08:00
lamber-ken
842eabb27f [HUDI-470] Fix NPE when print result via hudi-cli (#1138) 2019-12-26 15:40:38 +08:00
SteNicholas
def18a5086 [MINOR] optimize hudi timeline service (#1137) 2019-12-25 14:40:25 -08:00
Mathieu
3c811ec29b [MINOR] fix typos 2019-12-25 20:26:16 +08:00
hongdd
8affdf8bcb [HUDI-416] Improve hint information for cli (#1110) 2019-12-25 20:19:12 +08:00
Sivabalan Narayanan
9c4217a3e1 [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom (#1091) 
* Fixing Index look up to return partitions for a given key along with fileId with Global Bloom
* Addressing some of the comments
* Fixing test in TestHoodieGlobalBloomIndex to test the fix
 2019-12-24 20:56:30 -08:00
dengziming
94aec965f5 [minor] Fix few typos in the java docs (#1132) 2019-12-24 20:44:11 -08:00
comsir
dd06660183 [MINOR] fix typo 2019-12-24 20:40:00 -08:00
Mathieu
41f36770e0 [MINOR] fix typo 2019-12-25 06:48:15 +08:00
lamber-ken
4b1b3fc28c [MINOR] Set info servity for ImportOrder temporarily (#1127) 
- Now we need fix import check error manually, disable the rule temporarily before finding a better solution.
 2019-12-24 19:07:04 +08:00
Thinking Chen
8172197c35 Fix Error: java.lang.IllegalArgumentException: Can not create a Path from an empty string in HoodieCopyOnWrite#deleteFilesFunc (#1126) 
same link in https://github.com/apache/incubator-hudi/pull/771
this time is in HoodieCopyOnWrite deleteFilesFunc method
 2019-12-24 14:29:28 +08:00
vinoth chandar
350b0ecb4d [HUDI-311] : Support for AWS Database Migration Service in DeltaStreamer 
- Add a transformer class, that adds `Op` fiels if not found in input frame
 - Add a payload implementation, that issues deletes when Op=D
 - Remove Parquet as a top level source type, consolidate with RowSource
 - Made delta streamer work without a property file, simply using overridden cli options
 - Unit tests for transformer/payload classes
 2019-12-23 20:56:55 -08:00
lamber-ken
313fab5fd1 [HUDI-444] Refactor the codes based on scala codestyle ReturnChecker rule (#1121) 2019-12-24 07:05:54 +08:00