lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity
801 Commits 1 Branch 0 Tags
323fffad0dccf0ca5cb4e533e7fbcd2a2bcfaf3b
Commit Graph

90 Commits

Author SHA1 Message Date
lamber-ken
83c8ad5a38 [HUDI-625] Fixing performance issues around DiskBasedMap & kryo (#1352) 2020-02-24 22:40:37 -08:00
Suneel Marthi
078d4825d9 [HUDI-624]: Split some of the code from PR for HUDI-479 (#1344) 2020-02-21 14:22:21 +08:00
Nishith Agarwal
185ff646ad Refactoring getter to avoid double extrametadata in json representation 2020-02-20 09:52:02 -08:00
Suneel Marthi
f9d2f66dc1 [HUDI-622]: Remove VisibleForTesting annotation and import from code (#1343) 
* HUDI:622: Remove VisibleForTesting annotation and import from code
 2020-02-20 15:17:53 +08:00
Suneel Marthi
b8f9d0ec45 [HUDI-615]: Add some methods and test cases for StringUtils. (#1338) 2020-02-17 14:13:33 +08:00
Suneel Marthi
24e73816b2 [MINOR] Code Cleanup, remove redundant code (#1337) 2020-02-15 22:03:29 +08:00
lamber-ken
d2c872ede4 [HUDI-605] Avoid calculating the size of schema redundantly (#1317) 2020-02-12 19:40:52 +08:00
Balajee Nagasubramaniam
1fb0b001a3 [HUDI-570] - Improve test coverage for FSUtils.java 2020-02-05 14:25:24 -08:00
Satish Kotha
462fd02556 [HUDI-571] Add 'commits show archived' command to CLI 2020-02-05 11:25:34 -08:00
lamber-ken
46842f4e92 [MINOR] Remove the declaration of thrown RuntimeException (#1305) 2020-02-05 23:23:20 +08:00
Prashant Wason
4de0fcfcb5 [HUDI-566] Added new test cases for class HoodieTimeline, HoodieDefaultTimeline and HoodieActiveTimeline. 2020-02-04 18:55:04 -08:00
Suneel Marthi
594da28fbf [HUDI-595] code cleanup, refactoring code out of PR# 1159 (#1302) 2020-02-04 21:52:03 +08:00
Suneel Marthi
5b7bb142dc [HUDI-583] Code Cleanup, remove redundant code, and other changes (#1237) 2020-02-02 18:03:44 +08:00
Prashant Wason
f27c7a16c6 [HUDI-564] Added new test cases for HoodieLogFormat and HoodieLogFormatVersion. 2020-01-30 13:53:18 -08:00
Balajee Nagasubramaniam
6f34be1b8d HUDI-117 Close file handle before throwing an exception due to append failure. 
Add test cases to handle/verify stage failure scenarios.
 2020-01-29 15:28:51 -08:00
hmatu
362a9b9aab [MINOR] Remove junit-dep dependency 2020-01-28 10:31:55 -08:00
Mathieu
b6e2993ceb [MINOR] Update the javadoc of HoodieTableMetaClient#scanFiles (#1263) 
[MINOR] Update the javadoc of HoodieTableMetaClient#scanFiles
 2020-01-21 15:50:40 +08:00
leesf
6e59c1c777 Moving to 0.5.2-SNAPSHOT on master branch. 2020-01-20 10:51:33 -08:00
Balaji Varadarajan
ba54a7e973 [HUDI-559] : Make the timeline layout version default to be null version 2020-01-20 00:02:55 -08:00
leesf
5471d8f0c2 [MINOR] Add toString method to TimelineLayoutVersion to make it more readable (#1244) 2020-01-17 20:22:55 -05:00
Balaji Varadarajan
923e2b4a1e [HUDI-535] Ensure Compaction Plan is always written in .aux folder to avoid 0.5.0/0.5.1 reader-writer compatibility issues (#1229) 2020-01-17 10:56:35 -08:00
vinoth chandar
c2c0f6b13d [HUDI-509] Renaming code in sync with cWiki restructuring (#1212) 
- Storage Type replaced with Table Type (remaining instances)
 - View types replaced with query types;
 - ReadOptimized view referred as Snapshot Query
 - TableFileSystemView sub interfaces renamed to BaseFileOnly and Slice Views
 - HoodieDataFile renamed to HoodieBaseFile
 - Hive Sync tool will register RO tables for MOR with a `_ro` suffix
 - Datasource/Deltastreamer options renamed accordingly
 - Support fallback to old config values as well, so migration is painless
 - Config for controlling _ro suffix addition
 - Renaming DataFile to BaseFile across DTOs, HoodieFileSlice and AbstractTableFileSystemView
 2020-01-16 23:58:47 -08:00
lamber-ken
8a3a50309b [MINOR] Fix missing @Override annotation on BufferedRandomAccessFile method (#1236) 2020-01-16 11:14:39 -08:00
Balajee Nagasubramaniam
dd09abb56d [HUDI-335] Improvements to DiskBasedMap used by ExternalSpillableMap, for write and random/sequential read paths, by introducing bufferedRandmomAccessFile 2020-01-15 16:45:45 -08:00
lamber-ken
9b2944a9a2 [MINOR] Refactor unnecessary boxing inside TypedProperties code (#1227) 2020-01-14 19:27:53 -08:00
openopen2
a44c61b813 [HUDI-502] provide a custom time zone definition for TimestampBasedKeyGenerator (#1188) 2020-01-12 15:45:23 -08:00
lamber-ken
017ee8e661 [MINOR] Fix partition typo (#1209) 2020-01-12 09:35:55 +08:00
lamber-ken
e103165083 [CLEAN] replace utf-8 constant with StandardCharsets.UTF_8 2020-01-10 16:23:29 -08:00
Thinking
b95367d82a [HUDI-469] Fix: HoodieCommitMetadata only show first commit insert rows. 2020-01-10 16:17:11 -08:00
pratyakshsharma
3c90d252cc [HUDI-114]: added option to overwrite payload implementation in hoodie.properties file 2020-01-09 22:34:40 -08:00
vinoth chandar
9706f659db [HUDI-508] Standardizing on "Table" instead of "Dataset" across code (#1197) 
- Docs were talking about storage types before, cWiki moved to "Table"
 - Most of code already has HoodieTable, HoodieTableMetaClient - correct naming
 - Replacing renaming use of dataset across code/comments
 - Few usages in comments and use of Spark SQL DataSet remain unscathed
 2020-01-07 12:52:32 -08:00
Pratyaksh Sharma
dde21e7315 [HUDI-402]: code clean up in test cases 2019-12-31 11:10:49 -08:00
lamber-ken
ab6ae5cebb [HUDI-482] Fix missing @Override annotation on methods (#1156) 
* [HUDI-482] Fix missing @Override annotation on methods
 2019-12-31 11:44:56 +08:00
lamber-ken
e4ea7a2971 Update comment 2019-12-29 19:03:56 -08:00
lamber-ken
8440482977 Fix empty content clean plan 2019-12-29 19:03:56 -08:00
Mathieu
01c25d6aff [MINOR] Update the java doc of HoodieTableType (#1148) 2019-12-29 09:57:19 +08:00
hongdd
8affdf8bcb [HUDI-416] Improve hint information for cli (#1110) 2019-12-25 20:19:12 +08:00
dengziming
94aec965f5 [minor] Fix few typos in the java docs (#1132) 2019-12-24 20:44:11 -08:00
comsir
dd06660183 [MINOR] fix typo 2019-12-24 20:40:00 -08:00
vinoth chandar
350b0ecb4d [HUDI-311] : Support for AWS Database Migration Service in DeltaStreamer 
- Add a transformer class, that adds `Op` fiels if not found in input frame
 - Add a payload implementation, that issues deletes when Op=D
 - Remove Parquet as a top level source type, consolidate with RowSource
 - Made delta streamer work without a property file, simply using overridden cli options
 - Unit tests for transformer/payload classes
 2019-12-23 20:56:55 -08:00
Sivabalan Narayanan
14881e99e0 [HUDI-106] Adding support for DynamicBloomFilter (#976) 
- Introduced configs for bloom filter type
- Implemented dynamic bloom filter with configurable max number of keys
- BloomFilterFactory abstractions; Defaults to current simple bloom filter
 2019-12-17 19:06:24 -08:00
Balaji Varadarajan
9a1f698eef [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset 2019-12-15 21:26:30 -08:00
lamber-ken
ba514cfea0 [MINOR] Remove redundant plus operator (#1097) 2019-12-12 05:42:05 +08:00
lamber-ken
d447e2d751 [checkstyle] Unify LOG form (#1092) 2019-12-10 19:23:38 +08:00
lamber-ken
2745b7552f [HUDI-379] Refactor the codes based on new JavadocStyle code style rule (#1079) 2019-12-06 12:59:28 +08:00
lamber-ken
c06d89b648 [HUDI-378] Refactor the rest codes based on new ImportOrder code style rule (#1078) 2019-12-05 17:25:03 +08:00
lamber-ken
b3e0ebbc4a [checkstyle] Add ConstantName java checkstyle rule (#1066) 
* add SimplifyBooleanExpression java checkstyle rule
* collapse empty tags in scalastyle file
 2019-12-04 18:59:15 +08:00
vinoyang
84602c8882 [HUDI-355] Refactor hudi-common based on new comment and code style rules (#1049) 
[HUDI-355] Refactor hudi-common based on new comment and code style rules
 2019-12-03 20:49:13 -08:00
leesf
98ab33bb6e [HUDI-294] Delete Paths written in Cleaner plan needs to be relative to partition-path (#1062) 
[HUDI-294] Delete Paths written in Cleaner plan needs to be relative to partition-path
 2019-12-03 10:11:03 -08:00
lamber-ken
784e3ad0b6 [HUDI-370] Refactor hudi-common based on new ImportOrder code style rule (#1063) 2019-12-02 06:59:09 +08:00