lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

[HUDI-312] Make docker hdfs cluster ephemeral. This is needed to fix flakiness in integration tests. Also, Fix DeltaStreamer hanging issue due to uncaught exception

Browse Source
This commit is contained in:
Balaji Varadarajan
2019-10-31 06:57:25 -07:00
committed by Balaji Varadarajan
parent 144ea4eedf
commit a6390aefc4
6 changed files with 16 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docker/setup_demo.sh
View File

@@ -18,16 +18,10 @@
# limitations under the License.
################################################################################
# Create host mount directory and copy
mkdir -p /tmp/hadoop_name
mkdir -p /tmp/hadoop_data
WS_ROOT=`dirname $PWD`
# restart cluster
HUDI_WS=${WS_ROOT} docker-compose -f compose/docker-compose_hadoop284_hive233_spark231.yml down
HUDI_WS=${WS_ROOT} docker-compose -f compose/docker-compose_hadoop284_hive233_spark231.yml pull
rm -rf /tmp/hadoop_data/*
rm -rf /tmp/hadoop_name/*
sleep 5
HUDI_WS=${WS_ROOT} docker-compose -f compose/docker-compose_hadoop284_hive233_spark231.yml up -d
sleep 15