lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity
777 Commits 1 Branch 0 Tags
462fd025563b0ae8a4d4f28d366a9bbfca070d3f
Commit Graph

777 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
openopen2
a44c61b813 [HUDI-502] provide a custom time zone definition for TimestampBasedKeyGenerator (#1188) 2020-01-12 15:45:23 -08:00
harveyyue
971c7d41bd [HUDI-322] DeltaSteamer should pick checkpoints off only deltacommits for MOR tables 2020-01-12 15:11:47 -08:00
Udit Mehrotra
ad50008a59 [HUDI-91][HUDI-12]Migrate to spark 2.4.4, migrate to spark-avro library instead of databricks-avro, add support for Decimal/Date types 
- Upgrade Spark to 2.4.4, Parquet to 1.10.1, Avro to 1.8.2
- Remove spark-avro from hudi-spark-bundle. Users need to provide --packages org.apache.spark:spark-avro:2.4.4 when running spark-shell or spark-submit
- Replace com.databricks:spark-avro with org.apache.spark:spark-avro
- Shade avro in hudi-hadoop-mr-bundle to make sure it does not conflict with hive's avro version.
 2020-01-12 15:03:11 -08:00
lamber-ken
d9675c4ec0 [HUDI-522] Use the same version jcommander uniformly (#1214) 2020-01-12 10:48:52 -08:00
lamber-ken
017ee8e661 [MINOR] Fix partition typo (#1209) 2020-01-12 09:35:55 +08:00
lamber-ken
e103165083 [CLEAN] replace utf-8 constant with StandardCharsets.UTF_8 2020-01-10 16:23:29 -08:00
Thinking
b95367d82a [HUDI-469] Fix: HoodieCommitMetadata only show first commit insert rows. 2020-01-10 16:17:11 -08:00
leesf
04afac977d [HUDI-248] CLI doesn't allow rolling back a Delta commit 2020-01-10 16:10:35 -08:00
pratyakshsharma
3c90d252cc [HUDI-114]: added option to overwrite payload implementation in hoodie.properties file 2020-01-09 22:34:40 -08:00
hongdd
5af3dc6aed [HUDI-331]Fix java docs for all public apis in HoodieWriteClient (#1111) 2020-01-09 16:00:53 +08:00
Bhavani Sudha Saktheeswaran
d09eacdc13 [HUDI-25] Optimize HoodieInputformat.listStatus() for faster Hive incremental queries on Hoodie 
Summary:
    - InputPathHandler class classifies  inputPaths into incremental, non incremental and non hoodie paths.
    - Incremental queries leverage HoodieCommitMetadata to get partitions that are affected and only lists those partitions as opposed to listing all partitions
    - listStatus() processes each category separately
 2020-01-08 14:53:05 -08:00
Y Ethan Guo
480fc7869d [HUDI-319] Add a new maven profile to generate unified Javadoc for all Java and Scala classes (#1195) 
* Add javadoc build command in README, links to javadoc plugin and rename profile.
* Make java version configurable in one place.
 2020-01-08 10:38:09 -08:00
Wenning Ding
aba83876e7 Update deprecated HBase API 2020-01-08 10:26:47 -08:00
lamber-ken
9884972a3a [MINOR] Remove old jekyll config file (#1198) 2020-01-07 16:45:19 -08:00
vinoth chandar
9706f659db [HUDI-508] Standardizing on "Table" instead of "Dataset" across code (#1197) 
- Docs were talking about storage types before, cWiki moved to "Table"
 - Most of code already has HoodieTable, HoodieTableMetaClient - correct naming
 - Replacing renaming use of dataset across code/comments
 - Few usages in comments and use of Spark SQL DataSet remain unscathed
 2020-01-07 12:52:32 -08:00
Balaji Varadarajan
8306f749a2 [HUDI-417] Refactor HoodieWriteClient so that commit logic can be shareable by both bootstrap and normal write operations (#1166) 2020-01-06 20:11:48 -08:00
Abhishek Modi
b5df6723a2 [HUDI-464] Use Hive Exec Core for tests (#1125) 2020-01-06 16:32:55 -08:00
lamber-ken
75c3f630d4 [HUDI-405] Remove HIVE_ASSUME_DATE_PARTITION_OPT_KEY config from DataSource 2020-01-06 14:25:38 -08:00
yuehan124
c78092d2d3 [HUDI-501] Execute docker/setup_demo.sh in any directory 2020-01-06 10:26:06 -08:00
Pratyaksh Sharma
8f935e779a [HUDI-406]: added default partition path in TimestampBasedKeyGenerator 2020-01-06 09:38:06 -08:00
hongdd
2d5b79d96f [HUDI-438] Merge duplicated code fragment in HoodieSparkSqlWriter (#1114) 2020-01-06 22:51:22 +08:00
hejinbiao123
b9fab0b933 Revert "[HUDI-455] Redo hudi-client log statements using SLF4J (#1145)" (#1181) 
This reverts commit e637d9ed26.
 2020-01-06 21:13:29 +08:00
lamber-ken
28ccf8c521 [HUDI-484] Fix NPE when reading IncrementalPull.sqltemplate in HiveIncrementalPuller (#1167) 2020-01-04 23:53:47 -08:00
Sivabalan Narayanan
7031445eb3 [HUDI-377] Adding Delete() support to DeltaStreamer (#1073) 
- Provides ability to perform hard deletes by writing delete marker records into the source data
- if the record contains a special field _hoodie_delete_marker set to true, deletes are performed
 2020-01-04 11:07:31 -08:00
SteNicholas
726ae47ce2 [MINOR]Optimize hudi-client module (#1139) 2020-01-04 10:57:08 -08:00
SteNicholas
a733f4ef72 [MINOR] Optimize hudi-cli module (#1136) 2020-01-04 09:05:50 -08:00
Pratyaksh Sharma
290278fc6c [HUDI-118]: Options provided for passing properties to Cleaner, compactor and importer commands 2020-01-03 16:00:57 -08:00
hongdongdong
ff1113f3b7 [HUDI-492]Fix show env all in hudi-cli 2020-01-03 15:50:20 -08:00
lamber-ken
e1e5fe3324 [MINOR] Fix error usage of String.format (#1169) 2020-01-02 09:11:15 +08:00
Pratyaksh Sharma
dde21e7315 [HUDI-402]: code clean up in test cases 2019-12-31 11:10:49 -08:00
Suneel Marthi
98c0d8cf60 Merge pull request #1160 from smarthi/HUDI-343 
[HUDI-343]: Create a DOAP file for Hudi
 2019-12-31 04:21:31 -05:00
Suneel Marthi
47c1f746e2 [HUDI-343]: Create a DOAP file for Hudi 2019-12-31 03:49:21 -05:00
Suneel Marthi
add4b1e329 Merge pull request #1143 from BigDataArtisans/outoflimit 
[MINOR] Fix out of limits for results
 2019-12-31 02:08:54 -05:00
lamber-ken
619f501054 Clean up code 2019-12-31 13:59:26 +08:00
hejinbiao123
e637d9ed26 [HUDI-455] Redo hudi-client log statements using SLF4J (#1145) 
* [HUDI-455] Redo hudi-client log statements using SLF4J
 2019-12-31 13:49:34 +08:00
lamber-ken
ab6ae5cebb [HUDI-482] Fix missing @Override annotation on methods (#1156) 
* [HUDI-482] Fix missing @Override annotation on methods
 2019-12-31 11:44:56 +08:00
dengziming
2a823f32ee [MINOR]: alter some wrong params which bring fatal exception 2019-12-30 16:50:12 -08:00
lamber-ken
e4ea7a2971 Update comment 2019-12-29 19:03:56 -08:00
lamber-ken
8440482977 Fix empty content clean plan 2019-12-29 19:03:56 -08:00
lamber-ken
2f254163d4 Skip setting commit metadata 2019-12-29 19:03:56 -08:00
lamber-ken
179837e8ef Fix checkstyle 2019-12-29 19:03:56 -08:00
lamber-ken
58c5bed40a [HUDI-453] Fix throw failed to archive commits error when writing data to MOR/COW table 2019-12-29 19:03:56 -08:00
Mathieu
01c25d6aff [MINOR] Update the java doc of HoodieTableType (#1148) 2019-12-29 09:57:19 +08:00
lamber-ken
74b00d1f95 trigger rebuild 2019-12-27 02:06:26 +08:00
lamber-ken
36c0e6bae1 [MINOR] Fix out of limits for results 2019-12-27 01:16:24 +08:00
lamber-ken
bb90dedfc8 [MINOR] Fix out of limits for results 2019-12-27 01:13:47 +08:00
yungthuis66
f20a130e3a [MINOR] typo fix (#1142) 2019-12-26 09:03:43 -08:00
lamber-ken
842eabb27f [HUDI-470] Fix NPE when print result via hudi-cli (#1138) 2019-12-26 15:40:38 +08:00
SteNicholas
def18a5086 [MINOR] optimize hudi timeline service (#1137) 2019-12-25 14:40:25 -08:00
Mathieu
3c811ec29b [MINOR] fix typos 2019-12-25 20:26:16 +08:00