[HUDI-2332] Add clustering and compaction in Kafka Connect Sink (#3857)
* [HUDI-2332] Add clustering and compaction in Kafka Connect Sink * Disable validation check on instant time for compaction and adjust configs * Add javadocs * Add clustering and compaction config * Fix transaction causing missing records in the target table * Add debugging logs * Fix kafka offset sync in participant * Adjust how clustering and compaction are configured in kafka-connect * Fix clustering strategy * Remove irrelevant changes from other published PRs * Update clustering logic and others * Update README * Fix test failures * Fix indentation * Fix clustering config * Add JavaCustomColumnsSortPartitioner and make async compaction enabled by default * Add test for JavaCustomColumnsSortPartitioner * Add more changes after IDE sync * Update README with clarification * Fix clustering logic after rebasing * Remove unrelated changes
This commit is contained in:
@@ -164,7 +164,7 @@ public class HoodieCompactor {
|
||||
// Get schema.
|
||||
SparkRDDWriteClient client =
|
||||
UtilHelpers.createHoodieClient(jsc, cfg.basePath, "", cfg.parallelism, Option.of(cfg.strategyClassName), props);
|
||||
if (cfg.compactionInstantTime == null) {
|
||||
if (StringUtils.isNullOrEmpty(cfg.compactionInstantTime)) {
|
||||
throw new IllegalArgumentException("No instant time is provided for scheduling compaction. "
|
||||
+ "Please specify the compaction instant time by using --instant-time.");
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user