lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

formatting for docs

Browse Source
This commit is contained in:
Yash Sharma
2017-03-28 10:30:50 +11:00
committed by vinoth chandar
parent bca7e7dae4
commit e3b273e9fd
1 changed files with 8 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
docs/s3_filesystem.md
View File

@@ -8,12 +8,15 @@ summary: In this page, we go over how to configure hoodie with S3 filesystem.
--- ---
Hoodie works with HDFS by default. There is an experimental work going on Hoodie-S3 compatibility. Hoodie works with HDFS by default. There is an experimental work going on Hoodie-S3 compatibility.
## S3 configs ## AWS configs
There are two configurations required for Hoodie-S3 compatibility: There are two configurations required for Hoodie-S3 compatibility:
- Adding AWS Credentials for Hoodie - Adding AWS Credentials for Hoodie
- Adding required Jars to classpath - Adding required Jars to classpath
### AWS Credentials
Add the required configs in your core-site.xml from where Hoodie can fetch them. Replace the `fs.defaultFS` with your S3 bucket name and Hoodie should be able to read/write from the bucket. Add the required configs in your core-site.xml from where Hoodie can fetch them. Replace the `fs.defaultFS` with your S3 bucket name and Hoodie should be able to read/write from the bucket.
``` ```
@@ -48,7 +51,10 @@ Add the required configs in your core-site.xml from where Hoodie can fetch them.
</property> </property>
``` ```
AWS hadoop libraries to add to your classpath - ### AWS Libs
AWS hadoop libraries to add to our classpath
- com.amazonaws:aws-java-sdk:1.10.34 - com.amazonaws:aws-java-sdk:1.10.34
- org.apache.hadoop:hadoop-aws:2.7.3 - org.apache.hadoop:hadoop-aws:2.7.3