lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

[HUDI-1292] Created a config to enable/disable syncing of metadata table. (#3427)

Browse Source 
* [HUDI-1292] Created a config to enable/disable syncing of metadata table.

- Metadata Table should only be synced from a single pipeline to prevent conflicts.
- Skip syncing metadata table for clustering and compaction
- Renamed useFileListingMetadata

Co-authored-by: Vinoth Chandar <vinoth@apache.org>
This commit is contained in:
Prashant Wason
2021-08-12 15:45:57 -07:00
committed by GitHub
parent b651336454
commit 76bc686a77
11 changed files with 88 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/table/action/commit/JavaInsertOverwriteTableCommitActionExecutor.java
View File

@@ -52,7 +52,7 @@ public class JavaInsertOverwriteTableCommitActionExecutor<T extends HoodieRecord
protected Map<String, List<String>> getPartitionToReplacedFileIds(HoodieWriteMetadata<List<WriteStatus>> writeResult) {
Map<String, List<String>> partitionToExistingFileIds = new HashMap<>();
List<String> partitionPaths = FSUtils.getAllPartitionPaths(context,
table.getMetaClient().getBasePath(), config.useFileListingMetadata(),
table.getMetaClient().getBasePath(), config.isMetadataTableEnabled(),
config.getFileListingMetadataVerify(), config.shouldAssumeDatePartitioning());
if (partitionPaths != null && partitionPaths.size() > 0) {