Nicer handling of timeline archival for Cloud storage
- When append() is not supported, rollover to new file always (instead of failing) - Provide way to configure archive log folder (avoids small files inside .hoodie) - Datasets written via Spark datasource archive to .hoodie/archived - HoodieClientExample will now retain only 2,3 commits to exercise archival path during dev cycles - Few tweaks to code structure around CommitArchiveLog
This commit is contained in:
committed by
vinoth chandar
parent
0cd186c899
commit
cf7f7aabb9
@@ -72,6 +72,7 @@ public class HoodieRealtimeRecordReaderTest {
|
||||
jobConf = new JobConf();
|
||||
fs = FSUtils
|
||||
.getFs(basePath.getRoot().getAbsolutePath(), HoodieTestUtils.getDefaultHadoopConf());
|
||||
HoodieTestUtils.fs = fs;
|
||||
}
|
||||
|
||||
@Rule
|
||||
|
||||
Reference in New Issue
Block a user