lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

Fixing number of insert buckets to be generated by rounding off to the closest greater integer

Browse Source
This commit is contained in:
Nishith Agarwal
2018-11-13 15:42:36 -08:00
committed by vinoth chandar
parent 1362942aa3
commit d0fde47458
2 changed files with 42 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
hoodie-client/src/main/java/com/uber/hoodie/table/HoodieCopyOnWriteTable.java
View File

@@ -726,7 +726,7 @@ public class HoodieCopyOnWriteTable<T extends HoodieRecordPayload> extends Hoodi
insertRecordsPerBucket = config.getParquetMaxFileSize() / averageRecordSize;
}
int insertBuckets = (int) Math.max(totalUnassignedInserts / insertRecordsPerBucket, 1L);
int insertBuckets = (int) Math.ceil((1.0 * totalUnassignedInserts) / insertRecordsPerBucket);
logger.info(
"After small file assignment: unassignedInserts => " + totalUnassignedInserts
+ ", totalInsertBuckets => " + insertBuckets + ", recordsPerBucket => "