lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

[HUDI-3141] Metadata merged log record reader - avoiding NullPointerException when records by keys (#4505)

Browse Source 
- HoodieMetadataMergedLogRecordReader#getRecordsByKeys() and its parent class methods
   are not thread safe. When multiple queries come in for gettting log records
   by keys, they all operate on the same log record reader instance provided by
   HoodieBackedTableMetadata#openReadersIfNeeded() and they trip over each other
   as they clear/put/get the same class memeber records.

 - The fix is to streamline the mutatation to class member records. Making
   HoodieMetadataMergedLogRecordReader#getRecordsByKeys() a synchronized method
to avoid concurrent log records readers getting into NPE.
This commit is contained in:
Manoj Govindassamy
2022-01-04 13:41:33 -08:00
committed by GitHub
parent aaf5727495
commit bf4e3d63e7
1 changed files with 4 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataMergedLogRecordReader.java
View File

@@ -120,7 +120,10 @@ public class HoodieMetadataMergedLogRecordReader extends HoodieMergedLogRecordSc
return Collections.singletonList(Pair.of(key, Option.ofNullable((HoodieRecord) records.get(key))));
}
public List<Pair<String, Option<HoodieRecord<HoodieMetadataPayload>>>> getRecordsByKeys(List<String> keys) {
public synchronized List<Pair<String, Option<HoodieRecord<HoodieMetadataPayload>>>> getRecordsByKeys(List<String> keys) {
// Following operations have to be atomic, otherwise concurrent
// readers would race with each other and could crash when
// processing log block records as part of scan.
records.clear();
scan(Option.of(keys));
List<Pair<String, Option<HoodieRecord<HoodieMetadataPayload>>>> metadataRecords = new ArrayList<>();