lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity
557 Commits 1 Branch 0 Tags
2ea8b0c3f1eeb19f4dc1e9946331c8fd93e6daab
Commit Graph

557 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Balaji Varadarajan
2ea8b0c3f1 [HUDI-279] Fix regression in Schema Evolution due to PR-755 2019-09-25 22:53:43 -07:00
Xing Pan
bf05f95413 [HUDI-269] Limit sync frequency (#921) 
* [HUDI-269] Throttle DeltaStreamer sync runs
 2019-09-24 05:30:35 -07:00
vinoth chandar
1e6dd1ca1d [MINOR] : Further updates to .mailmap for vinoth (#919) 2019-09-22 17:26:04 -07:00
PanXing
635154c439 [MINOR] support reading cfg file in another s3 bucket (#914) 2019-09-22 06:47:23 -07:00
vinoyang
f020d029c4 HUDI-267 Refactor bad method name HoodieTestUtils#initTableType and HoodieTableMetaClient#initializePathAsHoodieDataset (#916) 2019-09-21 09:05:02 -07:00
Nishith Agarwal
1104f9526f HUDI-180 : Adding support for hive registration using metastore along with JDBC 2019-09-21 08:04:34 -07:00
Balaji Varadarajan
69ca45b2da Fix mailmap to allow mapping of defunct emailIds to active emailIds 2019-09-19 00:17:56 -07:00
Balaji Varadarajan
2e75a6973d Adding mailmap to allow mapping of defunct emailIds to active emailIds 2019-09-19 00:12:20 -07:00
Balaji Varadarajan
4bda742a93 Updating release notes and preparing for 0.5.0-incubating-rc2 release 2019-09-17 12:41:54 -07:00
Vinoth Chandar
e217db56ab [HUDI-254]: Bundle and shade databricks/avro with spark bundle 
- spark 2.4 onwards, spark has built in support. shading to avoid conflicts
 - spark 2.3 still needs this bundled, so that dropping bundle into jars folder would work
 2019-09-17 12:38:51 -07:00
Balaji Varadarajan
c1e7d0e5a6 [HUDI-121] Update Release notes and fix master version 2019-09-17 09:50:30 -07:00
Balaji Varadarajan
2c6da09d9d [HUDI-257] Fix Bloom Index unit-test failures 2019-09-17 09:41:15 -07:00
Balaji Varadarajan
96a46d8701 [HUDI-252] Identify packages which are under non-apache license in LICENSE file 2019-09-16 21:56:12 -07:00
Balaji Varadarajan
629698d54f [HUDI-252] Add Disclaimer and cleanup NOTICE and LICENSE files in hudi 2019-09-16 13:08:26 -07:00
Taher Koitawala
c0f42afa35 [HUDI-62] Index Lookup Timer added to HoodieWriteClient 2019-09-16 11:12:52 -07:00
Balaji Varadarajan
63cc455d9c [HUDI-121] ASF Release : Ensure NOTICE.txt is generated as part of cutting release branch 2019-09-13 13:58:29 -07:00
Balaji Varadarajan
7190c022bb [HUDI-249] Updating Notice files 2019-09-13 13:50:58 -07:00
Balaji Varadarajan
1a61d6bc4d [HUDI-249] Fix RELEASE-NOTES to match correct state 2019-09-13 12:48:38 -07:00
Vinoth Chandar
b1446be2b4 [HUDI-244] : Hive sync should escape partition field name 
- now supports field names beginning with '_' for e.g
 2019-09-13 12:41:52 -07:00
Balaji Varadarajan
3ee16b5439 [HUDI-250] Ensure Hudi CLI wrapper works with non snapshot jars too. Also Fix bug in cut_release_branch script 2019-09-13 12:40:32 -07:00
Nishith Agarwal
0b032b2761 Fix requested eompaction rollback during restore command 2019-09-13 12:40:13 -07:00
Balaji Varadarajan
d2525c31b7 Moving to 0.6.0-SNAPSHOT on master branch. 2019-09-13 09:58:29 -07:00
Balaji Varadarajan
36c74c8a16 [HUDI-249] Fix cut_release_branch script 2019-09-13 09:54:39 -07:00
Balaji Varadarajan
117a68d503 Add missing commit in release notes 2019-09-13 09:45:19 -07:00
Balaji Varadarajan
58623631d4 [HUDI-249] Update Release-notes. Add sign-artifacts to POM and release related scripts. Add missing license headers 2019-09-13 08:41:29 -07:00
yanghua
895d732a14 refactor code 2019-09-12 05:15:07 -07:00
yanghua
5f04241fce refactor code: add docs and init/cleanup resource group for hoodie client test base 2019-09-12 05:15:07 -07:00
yanghua
80c27f2351 Optimize hoodie client after implementat auto closeable interface 2019-09-12 05:15:07 -07:00
yanghua
90bfb900aa revert setting jsc spark configuration 2019-09-12 05:15:07 -07:00
yanghua
6f2b166005 [HUDI-217] Provide a unified resource management class to standardize the resource allocation and release for hudi client test cases 2019-09-12 05:15:07 -07:00
Bhavani Sudha Saktheeswaran
64df98fc4a [HUDI-164] Fixes incorrect averageBytesPerRecord 
When number of records written is zero, averageBytesPerRecord results in a huge size (division by zero and ceiled to Long.MAX_VALUE) causing OOM. This commit fixes this issue by reverse traversing the commits until a more reasonable average record size can be computed and if that is not possible returns the default configured record size.
 2019-09-11 15:20:25 -07:00
Balaji Varadarajan
93bc5e2153 HUDI-243 Rename HoodieInputFormat and HoodieRealtimeInputFormat to HoodieParquetInputFormat and HoodieParquetRealtimeInputFormat 2019-09-11 14:03:01 -07:00
Vinoth Chandar
d0b9b56b7d [HUDI-143] Excluding javax.* from utilities and spark bundles 
- Plus minor code review comments
 2019-09-11 11:08:27 -07:00
vinoth chandar
7a973a6944 [HUDI-159] Redesigning bundles for lighter-weight integrations 
- Documented principles applied for redesign at packaging/README.md
 - No longer depends on incl commons-codec, commons-io, commons-pool, commons-dbcp, commons-lang, commons-logging, avro-mapred
 - Introduce new FileIOUtils & added checkstyle rule for illegal import of above
 - Parquet, Avro dependencies moved to provided scope to enable being picked up from Hive/Spark/Presto instead
 - Pickup jackson jars for Hive sync tool from HIVE_HOME & unbundling jackson everywhere
 - Remove hive-jdbc standalone jar from being bundled in Spark/Hive/Utilities bundles
 - 6.5x reduced number of classes across bundles
 2019-09-11 11:08:27 -07:00
Mehrotra
0e6f078ec4 Fix logging in HoodieSparkSqlWriter 2019-09-07 07:51:11 -07:00
leesf
07a0ea87ab [hotfix] fix typo 2019-09-06 08:31:30 -07:00
leesf
821e0dcffc [HUDI-236] Failed to close stream 2019-09-03 19:24:11 -07:00
Alex Filipchik
555dd55c16 Support nested ordering fields 2019-08-30 13:41:16 -07:00
leesf
8b150a3c6b [HUDI-230] Add missing Apache License in some files 2019-08-30 09:38:28 -07:00
Balaji Varadarajan
376b59ae5f [HUDI-227] : DeltaStreamer Improvements : Commit empty input batch with progressing checkpoints and allow users to override configs through properties. Original PR : PR-805 and PR-806 (#863) 2019-08-30 09:13:34 -07:00
Balaji Varadarajan
a6908ef44d HUDI-170 Updating hoodie record before inserting it into ExternalSpillableMap (#866) 2019-08-30 09:03:37 -07:00
leesf
40dd4dd637 [HUDI-229] Fix mvn notice:generate issue in windows 2019-08-30 00:16:24 -07:00
leesf
5c2da6051e [HUDI-225] Create Hudi Timeline Server Fat Jar 2019-08-29 20:03:06 -07:00
Balaji Varadarajan
5f9fa82f47 HUDI-124 : Exclude jdk.tools from hadoop-common and update Notice files (#858) 2019-08-28 16:20:47 -07:00
leesf
00cfe72c5d [hotfix] change hoodie-timeline-*.jar to hudi-timeline-*.jar 2019-08-28 13:59:33 -07:00
leesf
b44f8521f2 [HUDI-222] Rename main class path to org.apache.hudi.timeline.service.TimelineService in run_server.sh 2019-08-28 13:59:33 -07:00
Alex Filipchik
41dbac6903 Fixed unit test 2019-08-28 06:19:43 -07:00
Alex Filipchik
b5d4da7958 Addressing comments 2019-08-28 06:19:43 -07:00
Alex Filipchik
baea4f3b82 Ignore dublicate of a compaction file 2019-08-28 06:19:43 -07:00
Alexander Filipchik
e0ab89b3ac [HUDI-223] Adding a way to infer target schema from the dataset after the transformation (#854) 
- Adding a way to decouple target and source schema providers
- Adding flattening transformer
 2019-08-28 04:48:38 -07:00