lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

[MINOR] Change MINI_BATCH_SIZE to 2048 (#4862)

Browse Source 
ParquetColumnarRowSplitReader#batchSize is 2048, so Changing MINI_BATCH_SIZE to 2048 will reduce memory cache.
This commit is contained in:
Bo Cui
2022-02-28 10:45:28 +08:00
committed by GitHub
parent d5444ff7ff
commit 193215201c
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
hudi-flink/src/main/java/org/apache/hudi/source/StreamReadOperator.java
View File

@@ -64,7 +64,7 @@ public class StreamReadOperator extends AbstractStreamOperator<RowData>
private static final Logger LOG = LoggerFactory.getLogger(StreamReadOperator.class); private static final Logger LOG = LoggerFactory.getLogger(StreamReadOperator.class);
private static final int MINI_BATCH_SIZE = 1000; private static final int MINI_BATCH_SIZE = 2048;
// It's the same thread that runs this operator and checkpoint actions. Use this executor to schedule only // It's the same thread that runs this operator and checkpoint actions. Use this executor to schedule only
// splits for subsequent reading, so that a new checkpoint could be triggered without blocking a long time // splits for subsequent reading, so that a new checkpoint could be triggered without blocking a long time