lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity
805 Commits 1 Branch 0 Tags
acf359c834bc1d9b9c4ea64d362ea20c7410c70a
Commit Graph

805 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ramachandran M S
acf359c834 [HUDI-627] Aggregate code coverage and publish to codecov.io during CI (#1347) 2020-02-27 13:54:20 -08:00
vinoth chandar
71170fafe7 [HUDI-554] Cleanup package structure in hudi-client (#1346) 
- Just package, class moves and renames with the following intent
 - `client` now has all the various client classes, that do the transaction management
 - `func` renamed to `execution` and some helpers moved to `client/utils`
 - All compaction code under `io` now under `table/compact`
 - Rollback code under `table/rollback` and in general all code for individual operations under `table`
 - `exception` `config`, `metrics` left untouched
 - Moved the tests also accordingly
 - Fixed some flaky tests
 2020-02-27 08:05:58 -08:00
lamber-ken
cacd9a3322 [HUDI-636] Fix could not get sources warnings while compiling (#1357) 2020-02-26 21:36:47 -08:00
Ramachandran M S
b7f35be452 [HUDI-618] Adding unit tests for PriorityBasedFileSystemView (#1345) 
[HUDI-618] Adding unit tests for PriorityBasedFileSystemView
 2020-02-26 10:55:02 -08:00
lamber-ken
323fffad0d [HUDI-606] Improve execute build_local_docker_images.sh script 2020-02-26 19:38:19 +08:00
lamber-ken
11fb2c2614 [HUDI-580] Fix incorrect license header in files 2020-02-25 08:54:26 -08:00
lamber-ken
83c8ad5a38 [HUDI-625] Fixing performance issues around DiskBasedMap & kryo (#1352) 2020-02-24 22:40:37 -08:00
YanJia-Gary-Li
4e7fcde4a6 [HUDI-597] Enable incremental pulling from defined partitions (#1348) 2020-02-24 11:46:30 -08:00
Suneel Marthi
078d4825d9 [HUDI-624]: Split some of the code from PR for HUDI-479 (#1344) 2020-02-21 14:22:21 +08:00
Nishith Agarwal
185ff646ad Refactoring getter to avoid double extrametadata in json representation 2020-02-20 09:52:02 -08:00
Suneel Marthi
f9d2f66dc1 [HUDI-622]: Remove VisibleForTesting annotation and import from code (#1343) 
* HUDI:622: Remove VisibleForTesting annotation and import from code
 2020-02-20 15:17:53 +08:00
amitsingh-10
c2b08cdfc9 [HUDI-617] Add support for types implementing CharSequence (#1339) 
- Data types extending CharSequence implement a #toString method which provides an easy way to convert them to String. 
- For example, org.apache.avro.util.Utf8 is easily convertible into String if we use the toString() method. It's better to make the support more generic to support a wider range of data types as partitionKey.
 2020-02-18 11:19:44 -08:00
Mathieu
8c6138cb01 [MINOR] Add javadoc to SchedulerConfGenerator and code clean (#1340) 2020-02-18 11:15:02 -08:00
Sivabalan Narayanan
00493235f5 [HUDI-108] Removing 2GB spark partition limitations in HoodieBloomIndex with spark 2.4.4 (#1315) 2020-02-18 11:12:20 -08:00
Suneel Marthi
b8f9d0ec45 [HUDI-615]: Add some methods and test cases for StringUtils. (#1338) 2020-02-17 14:13:33 +08:00
Suneel Marthi
24e73816b2 [MINOR] Code Cleanup, remove redundant code (#1337) 2020-02-15 22:03:29 +08:00
wangxianghu
aaa6cf9a98 [MINOR] Fix some typos 2020-02-15 09:49:25 +08:00
Satish Kotha
20ed2516d3 [HUDI-571] Add show archived compaction(s) to CLI 2020-02-14 10:58:28 -08:00
openopen2
dfbee673ef [HUDI-514] A schema provider to get metadata through Jdbc (#1200) 2020-02-13 18:06:06 -08:00
Mathieu
175de0db7b [MINOR] Fix typo (#1331) 2020-02-13 10:46:10 -08:00
lamber-ken
01c868ab86 [HUDI-574] Fix CLI counts small file inserts as updates (#1321) 2020-02-13 22:20:58 +08:00
Satish Kotha
63b42166b1 CLI - add option to print additional commit metadata 2020-02-12 14:11:24 -08:00
lamber-ken
d2c872ede4 [HUDI-605] Avoid calculating the size of schema redundantly (#1317) 2020-02-12 19:40:52 +08:00
lamber-ken
e5a69ed6af [HUDI-478] Fix too many files with unapproved license when execute build_local_docker_images (#1323) 2020-02-11 16:34:00 -08:00
RickyRajinder
578efa2a07 [HUDI-582] Update NOTICE year 2020-02-11 22:19:31 +08:00
Mathieu
5fdf5a1927 [HUDI-560] Remove legacy IdentityTransformer (#1264) 2020-02-10 10:04:58 +08:00
Prashant Wason
d26dc0b229 [HUDI-587] Fixed generation of jacoco coverage reports. 
surefire plugin's argLine is moved into a property. This configuration allows jacoco plugin to modify the argLine to insert it's Java Agent's configuration during pre-unit-test stage.
 2020-02-05 14:27:03 -08:00
Balajee Nagasubramaniam
1fb0b001a3 [HUDI-570] - Improve test coverage for FSUtils.java 2020-02-05 14:25:24 -08:00
Satish Kotha
462fd02556 [HUDI-571] Add 'commits show archived' command to CLI 2020-02-05 11:25:34 -08:00
Raymond Xu
c1516df8ac [HUDI-499] Allow update partition path with GLOBAL_BLOOM (#1187) 
* Handle partition path update by deleting a record from the old partition and
  insert into the new one
* Add a new configuration "hoodie.bloom.index.update.partition.path" to
  enable the behavior
* Add a new unit test case for global bloom index
 2020-02-05 09:33:33 -08:00
lamber-ken
46842f4e92 [MINOR] Remove the declaration of thrown RuntimeException (#1305) 2020-02-05 23:23:20 +08:00
lamber-ken
425e3e6c78 [HUDI-585] Optimize the steps of building with scala-2.12 (#1293) 2020-02-05 23:13:10 +08:00
Prashant Wason
4de0fcfcb5 [HUDI-566] Added new test cases for class HoodieTimeline, HoodieDefaultTimeline and HoodieActiveTimeline. 2020-02-04 18:55:04 -08:00
Suneel Marthi
594da28fbf [HUDI-595] code cleanup, refactoring code out of PR# 1159 (#1302) 2020-02-04 21:52:03 +08:00
dengziming
347e297ac1 [HUDI-596] Close KafkaConsumer every time (#1303) 2020-02-03 23:42:21 -08:00
Satish Kotha
d07ac588ac Increase test coverage for HoodieReadClient 2020-02-03 18:11:57 -08:00
Suneel Marthi
fcf9e4aded [MINOR] Updated DOAP with 0.5.1 release (#1301) 2020-02-02 22:41:30 +08:00
Suneel Marthi
0026234011 [MINOR] Updated DOAP with 0.5.1 release (#1300) 2020-02-02 15:13:24 +01:00
Suneel Marthi
5b7bb142dc [HUDI-583] Code Cleanup, remove redundant code, and other changes (#1237) 2020-02-02 18:03:44 +08:00
Prashant Wason
f27c7a16c6 [HUDI-564] Added new test cases for HoodieLogFormat and HoodieLogFormatVersion. 2020-01-30 13:53:18 -08:00
leesf
652224edc8 [HUDI-578] Trim recordKeyFields and partitionPathFields in ComplexKeyGenerator (#1281) 
* [HUDI-578] Trim recordKeyFields and partitionPathFields in ComplexKeyGenerator

* add tests
 2020-01-29 16:26:26 -08:00
Balajee Nagasubramaniam
6f34be1b8d HUDI-117 Close file handle before throwing an exception due to append failure. 
Add test cases to handle/verify stage failure scenarios.
 2020-01-29 15:28:51 -08:00
lamber-ken
c06ec8bfc7 [MINOR] Fix assigning to configuration more times (#1291) 2020-01-29 17:18:35 -05:00
hmatu
362a9b9aab [MINOR] Remove junit-dep dependency 2020-01-28 10:31:55 -08:00
lamber-ken
56a4e0d56c [MINOR] Fix invalid issue url & quickstart url (#1282) 2020-01-27 16:53:03 +08:00
lamber-ken
cdb028f1f3 [MINOR] Fix missing groupId / version property of dependency 2020-01-25 09:19:55 -08:00
Bhavani Sudha Saktheeswaran
1e79cbc259 [HUDI-549] update Github README with instructions to build with Scala 2.12 (#1275) 2020-01-22 23:43:50 -08:00
leesf
fc8d4a71ad [MINOR] fix license issue (#1273) 2020-01-23 02:03:49 +08:00
leesf
ed54eb20a5 [MINOR] Add missing licenses (#1271) 2020-01-22 08:06:45 -05:00
Balaji Varadarajan
8e3d81cff1 [MINOR] Change deploy_staging_jars script to take in scala version (#1270) 2020-01-22 14:28:30 +08:00