lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

Documentation update + helper method for WriteConfig builder

Browse Source
This commit is contained in:
Vinoth Chandar
2017-03-26 17:51:51 -07:00
committed by vinoth chandar
parent dce35ff0d7
commit e0fc4ec38e
3 changed files with 8 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/community.md
View File

@@ -12,5 +12,5 @@ issues or pull requests against this repo. Before you do so, please sign the
[Uber CLA](https://docs.google.com/a/uber.com/forms/d/1pAwS_-dA1KhPlfxzYLBqK6rsSWwRwH95OCCZrcsY5rk/viewform). [Uber CLA](https://docs.google.com/a/uber.com/forms/d/1pAwS_-dA1KhPlfxzYLBqK6rsSWwRwH95OCCZrcsY5rk/viewform).
Also, be sure to write unit tests for your bug fix or feature to show that it works as expected. Also, be sure to write unit tests for your bug fix or feature to show that it works as expected.
If you want to participate in day-day conversations, please join our [slack group](https://hoodielib.slack.com/x-147852474016-157730502112/signup).

2
docs/configurations.md
View File

@@ -21,6 +21,8 @@ summary: "Here we list all possible configurations and what they mean"
<span style="color:grey">HoodieWriteClient.insert and HoodieWriteClient.upsert returns a persisted RDD[WriteStatus], this is because the Client can choose to inspect the WriteStatus and choose and commit or not based on the failures. This is a configuration for the storage level for this RDD </span> <span style="color:grey">HoodieWriteClient.insert and HoodieWriteClient.upsert returns a persisted RDD[WriteStatus], this is because the Client can choose to inspect the WriteStatus and choose and commit or not based on the failures. This is a configuration for the storage level for this RDD </span>
- [withAutoCommit](#withAutoCommit) (autoCommit = true) <br/> - [withAutoCommit](#withAutoCommit) (autoCommit = true) <br/>
<span style="color:grey">Should HoodieWriteClient autoCommit after insert and upsert. The client can choose to turn off auto-commit and commit on a "defined success condition"</span> <span style="color:grey">Should HoodieWriteClient autoCommit after insert and upsert. The client can choose to turn off auto-commit and commit on a "defined success condition"</span>
- [withAssumeDatePartitioning](#withAssumeDatePartitioning) (assumeDatePartitioning = false) <br/>
<span style="color:grey">Should HoodieWriteClient assume the data is partitioned by dates, i.e three levels from base path. This is a stop-gap to support tables created by versions < 0.3.1. Will be removed eventually </span>
- [withIndexConfig](#withIndexConfig) (HoodieIndexConfig) <br/> - [withIndexConfig](#withIndexConfig) (HoodieIndexConfig) <br/>
<span style="color:grey">Hoodie uses a index to help find the FileID which contains an incoming record key. This is pluggable to have a external index (HBase) or use the default bloom filter stored in the Parquet files</span> <span style="color:grey">Hoodie uses a index to help find the FileID which contains an incoming record key. This is pluggable to have a external index (HBase) or use the default bloom filter stored in the Parquet files</span>

5
hoodie-client/src/main/java/com/uber/hoodie/config/HoodieWriteConfig.java
View File

@@ -327,6 +327,11 @@ public class HoodieWriteConfig extends DefaultHoodieConfig {
return this; return this;
} }
public Builder withAssumeDatePartitioning(boolean assumeDatePartitioning) {
props.setProperty(HOODIE_ASSUME_DATE_PARTITIONING_PROP, String.valueOf(assumeDatePartitioning));
return this;
}
public HoodieWriteConfig build() { public HoodieWriteConfig build() {
HoodieWriteConfig config = new HoodieWriteConfig(props); HoodieWriteConfig config = new HoodieWriteConfig(props);
// Check for mandatory properties // Check for mandatory properties