Documentation update + helper method for WriteConfig builder
This commit is contained in:
committed by
vinoth chandar
parent
dce35ff0d7
commit
e0fc4ec38e
@@ -12,5 +12,5 @@ issues or pull requests against this repo. Before you do so, please sign the
|
||||
[Uber CLA](https://docs.google.com/a/uber.com/forms/d/1pAwS_-dA1KhPlfxzYLBqK6rsSWwRwH95OCCZrcsY5rk/viewform).
|
||||
Also, be sure to write unit tests for your bug fix or feature to show that it works as expected.
|
||||
|
||||
|
||||
If you want to participate in day-day conversations, please join our [slack group](https://hoodielib.slack.com/x-147852474016-157730502112/signup).
|
||||
|
||||
|
||||
@@ -21,6 +21,8 @@ summary: "Here we list all possible configurations and what they mean"
|
||||
<span style="color:grey">HoodieWriteClient.insert and HoodieWriteClient.upsert returns a persisted RDD[WriteStatus], this is because the Client can choose to inspect the WriteStatus and choose and commit or not based on the failures. This is a configuration for the storage level for this RDD </span>
|
||||
- [withAutoCommit](#withAutoCommit) (autoCommit = true) <br/>
|
||||
<span style="color:grey">Should HoodieWriteClient autoCommit after insert and upsert. The client can choose to turn off auto-commit and commit on a "defined success condition"</span>
|
||||
- [withAssumeDatePartitioning](#withAssumeDatePartitioning) (assumeDatePartitioning = false) <br/>
|
||||
<span style="color:grey">Should HoodieWriteClient assume the data is partitioned by dates, i.e three levels from base path. This is a stop-gap to support tables created by versions < 0.3.1. Will be removed eventually </span>
|
||||
|
||||
- [withIndexConfig](#withIndexConfig) (HoodieIndexConfig) <br/>
|
||||
<span style="color:grey">Hoodie uses a index to help find the FileID which contains an incoming record key. This is pluggable to have a external index (HBase) or use the default bloom filter stored in the Parquet files</span>
|
||||
|
||||
@@ -327,6 +327,11 @@ public class HoodieWriteConfig extends DefaultHoodieConfig {
|
||||
return this;
|
||||
}
|
||||
|
||||
public Builder withAssumeDatePartitioning(boolean assumeDatePartitioning) {
|
||||
props.setProperty(HOODIE_ASSUME_DATE_PARTITIONING_PROP, String.valueOf(assumeDatePartitioning));
|
||||
return this;
|
||||
}
|
||||
|
||||
public HoodieWriteConfig build() {
|
||||
HoodieWriteConfig config = new HoodieWriteConfig(props);
|
||||
// Check for mandatory properties
|
||||
|
||||
Reference in New Issue
Block a user