lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

[HUDI-808] Support cleaning bootstrap source data (#1870)

Browse Source 
Co-authored-by: Wenning Ding <wenningd@amazon.com>
Co-authored-by: Balaji Varadarajan <vbalaji@apache.org>
This commit is contained in:
wenningd
2020-08-11 01:43:46 -07:00
committed by GitHub
parent 626f78f6f6
commit 8b928e9bca
23 changed files with 772 additions and 173 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
hudi-spark/src/test/java/org/apache/hudi/client/TestBootstrap.java
View File

@@ -551,7 +551,7 @@ public class TestBootstrap extends HoodieClientTestBase {
}
public static Dataset<Row> generateTestRawTripDataset(double timestamp, int from, int to, List<String> partitionPaths,
JavaSparkContext jsc, SQLContext sqlContext) {
JavaSparkContext jsc, SQLContext sqlContext) {
boolean isPartitioned = partitionPaths != null && !partitionPaths.isEmpty();
final List<String> records = new ArrayList<>();
IntStream.range(from, to).forEach(i -> {