6 Commits

Author	SHA1	Message	Date
jiale.tan	bf65219b73	feat(SparkDataSource): add structured streaming	2018-12-04 16:33:00 -08:00
jiale.tan	1628d044ac	feat(SparkDataSource): add additional feature to drop later arriving dups	2018-10-16 11:52:50 -07:00
Balaji Varadarajan	f3418e4718	Docker Container Build and Run setup with foundations for adding docker integration tests. Docker images built with Hadoop 2.8.4 Hive 2.3.3 and Spark 2.3.1 and published to docker-hub ... Look at quickstart document for how to setup docker and run demo	2018-10-02 09:28:21 +05:30
Vinoth Chandar	a5359662be	Moving depedencies off cdh to apache + Hive2 support ... - Tests redone in the process - Main changes are to RealtimeRecordReader and how it treats maps/arrays - Make hive sync work with Hive 1/2 and CDH environments - Fixes to make corner cases for Hive queries - Spark Hive integration - Working version across Apache and CDH versions - Known Issue - https://github.com/uber/hudi/issues/439	2018-09-11 11:03:30 +05:30
Vinoth Chandar	274aaf49fe	Incorporating code review feedback for DataSource	2017-10-02 20:44:53 -07:00
Vinoth Chandar	64e0573aca	Adding hoodie-spark to support Spark Datasource for Hoodie ... - Write with COW/MOR paths work fully - Read with RO view works on both storages* - Incremental view supported on COW - Refactored out HoodieReadClient methods, to just contain key based access - HoodieDataSourceHelpers class can be now used to construct inputs to datasource - Tests in hoodie-client using new helpers and mechanisms - Basic tests around save modes & insert/upserts (more to follow) - Bumped up scala to 2.11, since 2.10 is deprecated & complains with scalatest - Updated documentation to describe usage - New sample app written using the DataSource API	2017-10-02 20:44:53 -07:00