1
0

[HUDI-4018][HUDI-4027] Adding integ test yamls for immutable use-cases. Added delete partition support to integ tests (#5501)

- Added pure immutable test yamls to integ test framework. Added SparkBulkInsertNode as part of it.
- Added delete_partition support to integ test framework using spark-datasource.
- Added a single yaml to test all non core write operations (insert overwrite, insert overwrite table and delete partitions)
- Added tests for 4 concurrent spark datasource writers (multi-writer tests).
- Fixed readme w/ sample commands for multi-writer.
This commit is contained in:
Sivabalan Narayanan
2022-05-12 21:01:55 -04:00
committed by GitHub
parent ecd47e7aae
commit 0cec955fa2
26 changed files with 970 additions and 46 deletions

View File

@@ -17,7 +17,6 @@ dag_name: simple-deltastreamer.yaml
dag_rounds: 1
dag_intermittent_delay_mins: 1
dag_content:
first_insert:
config:
record_size: 1000
@@ -91,4 +90,4 @@ dag_content:
validate_hive: false
delete_input_data: false
type: ValidateDatasetNode
deps: third_upsert
deps: third_upsert