lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

[HUDI-3843] Make flink profiles build with scala-2.11 (#5279)

Browse Source
This commit is contained in:
Raymond Xu
2022-04-12 08:33:48 -07:00
committed by GitHub
parent d16740976e
commit 84783b9779
2 changed files with 16 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
README.md
View File

@@ -101,6 +101,18 @@ mvn clean package -DskipTests -Pspark2.4,scala-2.12
Starting from versions 0.11, Hudi no longer requires `spark-avro` to be specified using `--packages`
### Build with different Flink versions
The default Flink version supported is 1.14. Refer to the table below for building with different Flink and Scala versions.
| Maven build options | Expected Flink bundle jar name | Notes |
|:---------------------------|:-------------------------------|:------------------------------------------------|
| (empty) | hudi-flink1.14-bundle_2.11 | For Flink 1.14 and Scala 2.11 (default options) |
| `-Dflink1.14` | hudi-flink1.14-bundle_2.11 | For Flink 1.14 and Scala 2.11 (same as default) |
| `-Dflink1.14 -Dscala-2.12` | hudi-flink1.14-bundle_2.12 | For Flink 1.14 and Scala 2.12 |
| `-Dflink1.13` | hudi-flink1.13-bundle_2.11 | For Flink 1.13 and Scala 2.11 |
| `-Dflink1.13 -Dscala-2.12` | hudi-flink1.13-bundle_2.12 | For Flink 1.13 and Scala 2.12 |
## Running Tests
Unit tests can be run with maven profile `unit-tests`.

4
scripts/release/deploy_staging_jars.sh
View File

@@ -37,7 +37,11 @@ if [ "$#" -gt "1" ]; then
fi
declare -a ALL_VERSION_OPTS=(
"-Dscala-2.11 -Dspark2 -Dflink1.13" # for legacy bundle name
"-Dscala-2.12 -Dspark2 -Dflink1.13" # for legacy bundle name
"-Dscala-2.12 -Dspark3 -Dflink1.14" # for legacy bundle name
"-Dscala-2.11 -Dspark2.4 -Dflink1.13"
"-Dscala-2.11 -Dspark2.4 -Dflink1.14"
"-Dscala-2.12 -Dspark2.4 -Dflink1.13"
"-Dscala-2.12 -Dspark3.1 -Dflink1.14"
"-Dscala-2.12 -Dspark3.2 -Dflink1.14"