lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity
777 Commits 1 Branch 0 Tags
462fd025563b0ae8a4d4f28d366a9bbfca070d3f
Commit Graph

777 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
hongdd
8affdf8bcb [HUDI-416] Improve hint information for cli (#1110) 2019-12-25 20:19:12 +08:00
Sivabalan Narayanan
9c4217a3e1 [HUDI-389] Fixing Index look up to return right partitions for a given key along with fileId with Global Bloom (#1091) 
* Fixing Index look up to return partitions for a given key along with fileId with Global Bloom
* Addressing some of the comments
* Fixing test in TestHoodieGlobalBloomIndex to test the fix
 2019-12-24 20:56:30 -08:00
dengziming
94aec965f5 [minor] Fix few typos in the java docs (#1132) 2019-12-24 20:44:11 -08:00
comsir
dd06660183 [MINOR] fix typo 2019-12-24 20:40:00 -08:00
Mathieu
41f36770e0 [MINOR] fix typo 2019-12-25 06:48:15 +08:00
lamber-ken
4b1b3fc28c [MINOR] Set info servity for ImportOrder temporarily (#1127) 
- Now we need fix import check error manually, disable the rule temporarily before finding a better solution.
 2019-12-24 19:07:04 +08:00
Thinking Chen
8172197c35 Fix Error: java.lang.IllegalArgumentException: Can not create a Path from an empty string in HoodieCopyOnWrite#deleteFilesFunc (#1126) 
same link in https://github.com/apache/incubator-hudi/pull/771
this time is in HoodieCopyOnWrite deleteFilesFunc method
 2019-12-24 14:29:28 +08:00
vinoth chandar
350b0ecb4d [HUDI-311] : Support for AWS Database Migration Service in DeltaStreamer 
- Add a transformer class, that adds `Op` fiels if not found in input frame
 - Add a payload implementation, that issues deletes when Op=D
 - Remove Parquet as a top level source type, consolidate with RowSource
 - Made delta streamer work without a property file, simply using overridden cli options
 - Unit tests for transformer/payload classes
 2019-12-23 20:56:55 -08:00
lamber-ken
313fab5fd1 [HUDI-444] Refactor the codes based on scala codestyle ReturnChecker rule (#1121) 2019-12-24 07:05:54 +08:00
lamber-ken
b284091783 [HUDI-386] Refactor hudi scala checkstyle rules (#1099) 2019-12-22 07:30:07 +08:00
YanJia-Gary-Li
36b3b6f5dd [HUDI-415] Get commit time when Spark start (#1113) 2019-12-19 22:19:06 -08:00
Sivabalan Narayanan
14881e99e0 [HUDI-106] Adding support for DynamicBloomFilter (#976) 
- Introduced configs for bloom filter type
- Implemented dynamic bloom filter with configurable max number of keys
- BloomFilterFactory abstractions; Defaults to current simple bloom filter
 2019-12-17 19:06:24 -08:00
lamber-ken
7498ca71cb [MINOR] Add slack invite icon in README (#1108) 2019-12-17 15:15:03 +08:00
Balaji Varadarajan
9a1f698eef [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset 2019-12-15 21:26:30 -08:00
hongdd
8963a68e6a [HUDI-398]Add spark env set/get for spark launcher (#1096) 2019-12-14 14:13:00 +08:00
lamber-ken
f324057b6e [MINOR] Unify Lists import (#1103) 2019-12-12 21:42:18 -08:00
lamber-ken
a405d3873b [MINOR] replace scala map add operator (#1093) 
replace ++: with ++
 2019-12-12 11:29:17 +08:00
lamber-ken
ba514cfea0 [MINOR] Remove redundant plus operator (#1097) 2019-12-12 05:42:05 +08:00
Pratyaksh Sharma
3790b75e05 [HUDI-368] code clean up in TestAsyncCompaction class (#1050) 2019-12-11 05:52:41 +08:00
lamber-ken
24a09c775f [HUDI-387] Fix NPE when create savepoint via hudi-cli (#1085) 2019-12-10 08:00:53 -08:00
Nisheet Kumar
8df4b83017 [HUDI-390] Add backtick character in hive queries to support hive identifier as tablename (#1090) 2019-12-10 07:42:33 -08:00
lamber-ken
d447e2d751 [checkstyle] Unify LOG form (#1092) 2019-12-10 19:23:38 +08:00
lamber-ken
70a1040998 [MINOR] Beautify the cli banner (#1089) 
* Add one empty line
* replace Cli to CLI
* replace Hoodie to Apache Hudi
 2019-12-09 13:24:42 -08:00
Wenning Ding
e555aa516d [HUDI-353] Add hive style partitioning path 2019-12-09 12:29:53 -08:00
lamber-ken
63e330b17c [MINOR] add *.log to .gitignore file (#1086) 2019-12-07 09:51:28 -08:00
lamber-ken
2745b7552f [HUDI-379] Refactor the codes based on new JavadocStyle code style rule (#1079) 2019-12-06 12:59:28 +08:00
lamber-ken
c06d89b648 [HUDI-378] Refactor the rest codes based on new ImportOrder code style rule (#1078) 2019-12-05 17:25:03 +08:00
lamber-ken
b3e0ebbc4a [checkstyle] Add ConstantName java checkstyle rule (#1066) 
* add SimplifyBooleanExpression java checkstyle rule
* collapse empty tags in scalastyle file
 2019-12-04 18:59:15 +08:00
Gurudatt Kulkarni
b2d9638bea [HUDI-365] Refactor hudi-cli based on new ImportOrder code style rule (#1076) 2019-12-04 15:10:40 +08:00
vinoyang
84602c8882 [HUDI-355] Refactor hudi-common based on new comment and code style rules (#1049) 
[HUDI-355] Refactor hudi-common based on new comment and code style rules
 2019-12-03 20:49:13 -08:00
leesf
98ab33bb6e [HUDI-294] Delete Paths written in Cleaner plan needs to be relative to partition-path (#1062) 
[HUDI-294] Delete Paths written in Cleaner plan needs to be relative to partition-path
 2019-12-03 10:11:03 -08:00
leesf
845e261658 [MINOR] Update some urls from http to https in the README file (#1074) 2019-12-03 17:35:27 +08:00
lamber-ken
89f0968097 [DOCS] Update the build source link (#1071) 2019-12-03 09:56:09 +08:00
lamber-ken
ff688107fa [DOCS] Update Hudi Readme (#1058) 
- Add build status 
- Clean up layout
 2019-12-02 09:25:43 -08:00
lamber-ken
784e3ad0b6 [HUDI-370] Refactor hudi-common based on new ImportOrder code style rule (#1063) 2019-12-02 06:59:09 +08:00
Gurudatt Kulkarni
75132c139f [HUDI-357] Refactor hudi-cli based on new comment and code style rules (#1051) 2019-11-30 11:12:41 -08:00
hongdd
b65a897856 [HUDI-374] Unable to generateUpdates in QuickstartUtils (#1059) 2019-11-30 11:11:00 -08:00
lamber-ken
024230fbd2 [HUDI-372] Support the shortName for Hudi DataSource (#1054) 
- Ability to do `spark.write.format("hudi")...`
 2019-11-30 08:02:33 -08:00
ForwardXu
0b52ae3ac2 [HUDI-209] Implement JMX metrics reporter (#1045) 2019-11-28 19:17:34 +08:00
lamber-ken
da8d1334ee [HUDI-373] Refactor hudi-client based on new ImportOrder code style rule (#1056) 2019-11-28 09:25:56 +08:00
谢磊
f9139c0f61 [HUDI-366] Refactor some module codes based on new ImportOrder code style rule (#1055) 
[HUDI-366] Refactor hudi-hadoop-mr / hudi-timeline-service / hudi-spark / hudi-integ-test / hudi- utilities based on new ImportOrder code style rule
 2019-11-27 21:32:43 +08:00
谢磊
b77fad39b5 [HUDI-364] Refactor hudi-hive based on new ImportOrder code style rule (#1048) 
[HUDI-364] Refactor hudi-hive based on new ImportOrder code style rule
 2019-11-27 16:30:37 +08:00
wenningd
d6e83e8f49 [HUDI-325] Fix Hive partition error for updated HDFS Hudi table (#1001) 2019-11-26 21:18:39 -08:00
bschell
60fed21dc7 [HUDI-327] Add null/empty checks to key generators (#1040) 
* Adds null and empty checks to all key generators. 
* Also improves error messaging for key generator issues.
 2019-11-26 02:37:16 -08:00
Pratyaksh Sharma
2a4cfb47c7 [HUDI-340]: made max events to read from kafka source configurable (#1039) 2019-11-26 18:34:02 +08:00
hongdd
a7e07cd910 [HUDI-359] Add hudi-env for hudi-cli module (#1042) 2019-11-25 13:25:42 -08:00
谢磊
212282c8aa [HUDI-358] Add Java-doc and importOrder checkstyle rule (#1043) 
- import groups are separated by one blank line
- org.apache.hudi.* at the top location
 2019-11-25 11:36:23 -08:00
hongdd
44823041a3 [HUDI-362] Adds a check for the existence of field (#1047) 2019-11-25 11:31:07 -08:00
filippo balicchia
845a0509b3 [MINOR] Some minor optimizations in HoodieJavaStreamingApp (#1046) 2019-11-25 18:49:13 +08:00
Sivabalan Narayanan
c3355109b1 [HUDI-328] Adding delete api to HoodieWriteClient (#1004) 
[HUDI-328]  Adding delete api to HoodieWriteClient and Spark DataSource
 2019-11-22 15:05:25 -08:00