1
0

[HUDI-3764] Allow loading external configs while querying Hudi tables with Spark (#4915)

Currently when doing Hudi queries w/ Spark, it won't 
load the external configurations. Say if customers enabled 
metadata listing in their global config file, then this would 
let them actually query w/o metadata feature enabled. 
This PR fixes this issue and allows loading global 
configs during the Hudi reading phase.

Co-authored-by: Wenning Ding <wenningd@amazon.com>
This commit is contained in:
wenningd
2022-07-21 02:42:17 -07:00
committed by GitHub
parent de37774e12
commit c7fe3fd01d
3 changed files with 35 additions and 17 deletions

View File

@@ -196,6 +196,12 @@ public class DFSPropertiesConfiguration {
return globalProps;
}
// test only
public static TypedProperties addToGlobalProps(String key, String value) {
GLOBAL_PROPS.put(key, value);
return GLOBAL_PROPS;
}
public TypedProperties getProps() {
return new TypedProperties(hoodieConfig.getProps());
}