lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

[HUDI-92] Provide reasonable names for Spark DAG stages in HUDI. (#1289)

Browse Source
This commit is contained in:
Prashant Wason
2020-07-19 10:29:25 -07:00
committed by GitHub
parent 1aae437257
commit b71f25f210
25 changed files with 79 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
README.md
View File

@@ -92,6 +92,18 @@ spark-2.4.4-bin-hadoop2.7/bin/spark-shell \
--conf 'spark.serializer=org.apache.spark.serializer.KryoSerializer'
```
## Running Tests
All tests can be run with maven
```
mvn test
```
To run tests with spark event logging enabled, define the Spark event log directory. This allows visualizing test DAG and stages using Spark History Server UI.
```
mvn test -DSPARK_EVLOG_DIR=/path/for/spark/event/log
```
## Quickstart
Please visit [https://hudi.apache.org/docs/quick-start-guide.html](https://hudi.apache.org/docs/quick-start-guide.html) to quickly explore Hudi's capabilities using spark-shell.