Sivabalan Narayanan
a818020f72
[HUDI-2530] Adding async compaction support to integ test suite framework ( #3750 )
2021-10-08 11:30:48 -04:00
Satish M
c7a5c8273b
[HUDI-2267] Update docs and infra test configs, add support for graphite ( #3482 )
...
Co-authored-by: Sivabalan Narayanan <n.siva.b@gmail.com >
2021-09-17 10:10:15 -04:00
Y Ethan Guo
5d60491f5b
[HUDI-2388] Add DAG nodes for Spark SQL in integration test suite ( #3583 )
...
- Fixed validation in integ test suite for both deltastreamer and write client path.
Co-authored-by: Sivabalan Narayanan <n.siva.b@gmail.com >
2021-09-13 11:53:13 -04:00
Sagar Sumit
cf15431852
[HUDI-2393] Add yamls for large scale testing ( #3594 )
2021-09-10 09:02:01 -04:00
Sivabalan Narayanan
15bf01dcb7
[HUDI-2349] Adding spark delete node to integ test suite ( #3528 )
2021-08-24 10:58:47 -04:00
Udit Mehrotra
09e625becd
[HOT-FIX] Add apache license to spark_command.txt.template ( #3477 )
2021-08-15 07:08:55 -04:00
Sivabalan Narayanan
5564c7ec01
[HUDI-2006] Adding more yaml templates to test suite ( #3073 )
2021-06-29 23:05:46 -04:00
Sivabalan Narayanan
ac72470e10
[HUDI-1851] Adding test suite long running automate scripts for docker ( #2880 )
2021-05-11 01:26:01 -07:00
Gary Li
050626ad6c
[MINOR] Add Missing Apache License to test files ( #2736 )
2021-03-29 07:17:23 -07:00
Sivabalan Narayanan
d5f202821b
Adding fixes to test suite framework. Adding clustering node and validate async operations node. ( #2400 )
2021-02-12 09:29:21 -08:00
Sivabalan Narayanan
8cf6a7223f
[HUDI-1331] Adding support for validating entire dataset and long running tests in test suite framework ( #2168 )
...
* trigger rebuild
* [HUDI-1156] Remove unused dependencies from HoodieDeltaStreamerWrapper Class (#1927 )
* Adding support for validating records and long running tests in test sutie framework
* Adding partial validate node
* Fixing spark session initiation in Validate nodes
* Fixing validation
* Adding hive table validation to ValidateDatasetNode
* Rebasing with latest commits from master
* Addressing feedback
* Addressing comments
Co-authored-by: lamber-ken <lamberken@163.com >
Co-authored-by: linshan-ma <mabin194046@163.com >
2020-12-26 09:29:24 -08:00
Sivabalan Narayanan
a205dd10fa
[HUDI-1338] Adding Delete support to test suite framework ( #2172 )
...
- Adding Delete support to test suite.
Added DeleteNode
Added support to generate delete records
2020-11-01 00:15:41 -04:00
n3nash
e109a61803
1. Fix merge on read DAG to make docker demo pass ( #2092 )
...
1. Fix merge on read DAG to make docker demo pass (#2092 )
2. Fix repeat_count, rollback node
2020-10-28 22:34:26 -04:00
shenh062326
581d54097c
[HUDI-1143] Change timestamp field in HoodieTestDataGenerator from double to long
2020-09-15 20:58:29 -07:00
Abhishek Modi
53d1e55110
Test Suite should work with Docker + Unit Tests
2020-09-08 22:41:14 -07:00
Dongwook
8d19ebfd0f
[HUDI-993] Let delete API use "hoodie.delete.shuffle.parallelism" ( #1703 )
...
For Delete API, "hoodie.delete.shuffle.parallelism" isn't used as opposed to "hoodie.upsert.shuffle.parallelism" is used for upsert, this creates the performance difference between delete by upsert API with "EmptyHoodieRecordPayload" and delete API for certain cases.
This patch makes the following fixes in this regard.
- Let deduplicateKeys method use "hoodie.delete.shuffle.parallelism"
- Repartition inputRDD as "hoodie.delete.shuffle.parallelism" in case "hoodie.combine.before.delete=false"
2020-09-01 12:55:31 -04:00
Nishith Agarwal
2fc2b01d86
[HUDI-394] Provide a basic implementation of test suite
2020-07-30 21:21:15 -07:00
