[HUDI-1292] Created a config to enable/disable syncing of metadata table. (#3427)
* [HUDI-1292] Created a config to enable/disable syncing of metadata table. - Metadata Table should only be synced from a single pipeline to prevent conflicts. - Skip syncing metadata table for clustering and compaction - Renamed useFileListingMetadata Co-authored-by: Vinoth Chandar <vinoth@apache.org>
This commit is contained in:
@@ -447,6 +447,11 @@ public class SparkRDDWriteClient<T extends HoodieRecordPayload> extends
|
||||
|
||||
@Override
|
||||
public void syncTableMetadata() {
|
||||
if (!config.getMetadataConfig().enableSync()) {
|
||||
LOG.info("Metadata table sync is disabled in the config.");
|
||||
return;
|
||||
}
|
||||
|
||||
// Open up the metadata table again, for syncing
|
||||
try (HoodieTableMetadataWriter writer = SparkHoodieBackedTableMetadataWriter.create(hadoopConf, config, context)) {
|
||||
LOG.info("Successfully synced to metadata table");
|
||||
|
||||
Reference in New Issue
Block a user