1
0

[HUDI-1870] Add more Spark CI build tasks (#4022)

* [HUDI-1870] Add more Spark CI build tasks

- build for spark3.0.x
- build for spark-shade-unbundle-avro
- fix build failures
  - delete unnecessary assertion for spark 3.0.x
  - use AvroConversionUtils#convertAvroSchemaToStructType instead of calling SchemaConverters#toSqlType directly to solve the compilation failures with spark-shade-unbundle-avro (#5)

Co-authored-by: Yann <biyan900116@gmail.com>
This commit is contained in:
Raymond Xu
2021-11-22 02:16:45 -08:00
committed by GitHub
parent 8281cbf762
commit 02f7ca2b05
10 changed files with 48 additions and 32 deletions

View File

@@ -50,9 +50,5 @@ public class TestReflectUtil extends HoodieClientTestBase {
Assertions.assertTrue(
((UnresolvedRelation)newStatment.table()).multipartIdentifier().contains("test_reflect_util"));
if (!spark.version().startsWith("3.0")) {
Assertions.assertTrue(newStatment.userSpecifiedCols().isEmpty());
}
}
}