lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

Commit Graph

  • 6461927eac [HUDI-960] Implementation of the HFile base and log file format. (#1804) Prashant Wason 2020-08-31 08:05:59 -07:00
  • 6df8f88d86 [HUDI-1252] Remove unused class NoOpBulkInsertPartitioner in DataSourceTestUtils (#2054) Mathieu 2020-08-31 18:03:10 +08:00
  • 6b417d1a86 [HUDI-1225] Fix: Avro Date logical type not handled correctly when converting to Spark Row (#2047) Thinking Chen 2020-08-29 16:16:42 +08:00
  • 0360bef217 [MINOR] Improve helper methods in TestCleaner (#2052) Raymond Xu 2020-08-28 23:06:25 -07:00
  • 4dbeabffa3 [HUDI-1228] Add utility method to query extra metadata Satish Kotha 2020-08-25 21:48:28 -07:00
  • fa81248247 [HUDI-531] Add java doc for hudi test suite general classes (#1900) Mathieu 2020-08-28 08:44:40 +08:00
  • 3a578d7402 [HUDI-1056] Fix release validate script for rc_num and release_type (#2025) Sivabalan Narayanan 2020-08-26 12:26:33 -04:00
  • dedc4517dd [HUDI-978] Specify version information for each component separately (#1772) hongdd 2020-08-26 21:08:09 +08:00
  • f468c20c6c [HUDI-1226] Fix ComplexKeyGenerator for non-partitioned tables Satish Kotha 2020-08-25 13:37:48 -07:00
  • df8f099c99 [HUDI-532] Add java doc for the test classes of hudi test suite (#1901) Mathieu 2020-08-26 08:49:01 +08:00
  • 7e68c42eb1 [HUDI-1223] Remove unused UpdateHandler class in HoodieCopyOnWriteTable (#2032) Mathieu 2020-08-26 08:46:19 +08:00
  • cc555ba188 [HUDI-1133] Tune buffer sizes for the diskbased external spillable map Balajee Nagasubramaniam 2020-08-25 08:21:21 -07:00
  • 492ddcbb06 [HUDI-1191] Add incremental meta client API to query partitions modified in a time window Satish Kotha 2020-08-13 17:14:25 -07:00
  • 6a4dc7384c [HUDI-1218] Introduce BulkInsertSortMode as Independent class (#2021) Trevor 2020-08-25 19:04:13 +08:00
  • 218d4a6836 [HUDI-1135] Make timeline server timeout settings configurable. Prashant Wason 2020-08-24 14:10:48 -07:00
  • 9b1f16b604 [HUDI-1136] Add back findInstantsAfterOrEquals to the HoodieTimeline class. Prashant Wason 2020-08-24 14:24:50 -07:00
  • f7e02aa8a3 [MINOR] Update DOAP with 0.6.0 Release (#2024) Bhavani Sudha Saktheeswaran 2020-08-24 14:47:38 -07:00
  • ea983ff912 [HUDI-1137] Add option to configure different path selector Satish Kotha 2020-08-24 11:11:10 -07:00
  • 111a9753a0 [MINOR] Update README.md (#2010) Raymond Xu 2020-08-24 09:28:29 -07:00
  • f8dcd5334e [HUDI-1217] Improve avroToBytes method of HoodieAvroUtils (#2018) Mathieu 2020-08-24 17:33:28 +08:00
  • 35b21855da [HUDI-1150] Fix unable to parse input partition field :1 exception when using TimestampBasedKeyGenerator(#1920) Mathieu 2020-08-23 19:56:50 +08:00
  • 7291607ae3 [MINOR] Remove unused log code in HoodieReadClient (#2000) Trevor 2020-08-22 21:45:50 +08:00
  • 1d09c02f1c [HUDI-1083] Optimization in determining insert bucket location for a given key (#1868) Shen Hong 2020-08-22 19:41:39 +08:00
  • bfdce7b082 [HUDI-1193](Upgrade http dependency version) (#1970) liujinhui 2020-08-21 20:24:04 +08:00
  • 3a2ae16961 [HUDI-781] Introduce HoodieTestTable for test preparation (#1997) Raymond Xu 2020-08-20 20:46:33 -07:00
  • 34c8c9e3ea [MINOR] Move HoodieUpgradeDowngradeException to exception package (#1993) Mathieu 2020-08-20 23:12:20 +08:00
  • b883b6d268 [HUDI-1122] Introduce a kafka implementation of hoodie write commit ca… (#1886) Mathieu 2020-08-20 23:00:59 +08:00
  • bd7814dadf [HUDI-1206] Remove unused variable in Compactor (#1994) Mathieu 2020-08-20 18:18:36 +08:00
  • a2312fa1b7 [HUDI-1177]: fixed TaskNotSerializableException in TimestampBasedKeyGenerator (#1987) Pratyaksh Sharma 2020-08-20 06:13:34 +05:30
  • 1137b0b343 Fix HBASE index MOR tables not considering record index valid Ryan Pifer 2020-08-13 15:39:37 -07:00
  • 6fa371a79c [MINOR] Fix release script for onetime uploading of gpgkeys (#1949) Bhavani Sudha Saktheeswaran 2020-08-18 21:29:52 -07:00
  • 824f23bcb8 [HUDI-1197] Fix import issue that fails scala 2.12 build (#1976) Bhavani Sudha Saktheeswaran 2020-08-18 08:41:16 -07:00
  • bedbb825e0 [HUDI-1025] Meter RPC calls in HoodieWrapperFileSystem (#1916) Abhishek Modi 2020-08-18 07:42:05 -07:00
  • 4226d75144 Moving to 0.6.1-SNAPSHOT on master branch. Bhavani Sudha Saktheeswaran 2020-08-14 12:54:15 -07:00
  • b8f4a30efd Fix Integration test flakiness in HoodieJavaStreamingApp (#1967) Balaji Varadarajan 2020-08-14 01:42:15 -07:00
  • 9bde6d616c [HUDI-1190] Introduce @PublicAPIClass and @PublicAPIMethod annotations to mark public APIs (#1965) vinoth chandar 2020-08-13 23:28:17 -07:00
  • 379cf0786f [HUDI-1013] Adding Bulk Insert V2 implementation (#1834) Sivabalan Narayanan 2020-08-13 03:33:39 -04:00
  • 8d04268264 [HUDI-1174] Changes for bootstrapped tables to work with presto (#1944) Udit Mehrotra 2020-08-12 17:51:31 -07:00
  • 8b928e9bca [HUDI-808] Support cleaning bootstrap source data (#1870) wenningd 2020-08-11 01:43:46 -07:00
  • 626f78f6f6 Revert "[HUDI-781] Introduce HoodieTestTable for test preparation (#1871)" Balaji Varadarajan 2020-08-10 22:13:02 -07:00
  • 9c24151929 [HUDI-1175] Commenting out testsuite tests from Integration tests until we investigate the CI flakiness (#1945) Sivabalan Narayanan 2020-08-11 00:00:57 -04:00
  • b2e703d442 [HUDI-781] Introduce HoodieTestTable for test preparation (#1871) Raymond Xu 2020-08-10 18:44:03 -07:00
  • 934f00b689 [HUDI-1173] fix hudi-prometheus pom dependency (#1942) liujinhui 2020-08-11 09:06:17 +08:00
  • 858eda85d7 [HUDI-1098] Adding OptimisticConsistencyGuard to be used during FinalizeWrite (#1912) Sivabalan Narayanan 2020-08-09 20:51:37 -04:00
  • ff53e8f0b6 [HUDI-1014] Adding Upgrade and downgrade infra for smooth transitioning from list based rollback to marker based rollback (#1858) Sivabalan Narayanan 2020-08-09 18:32:43 -04:00
  • e4a2d98f79 [HUDI-426] Bootstrap datasource integration (#1702) Udit Mehrotra 2020-08-09 14:06:13 -07:00
  • c24c528fb7 [HUDI-1156] Remove unused dependencies from HoodieDeltaStreamerWrapper Class (#1927) linshan-ma 2020-08-09 17:09:28 +08:00
  • 6b349b7711 [HUDI-210] Hudi Supports Prometheus Pushgateway (#1931) liujinhui 2020-08-09 15:29:54 +08:00
  • 3c949d2ff5 [MINOR] Fix path to hudi-hive-sync-bundle jars from run_sync_tool.sh (#1937) Bhavani Sudha Saktheeswaran 2020-08-08 21:45:10 -07:00
  • 9fe2d2b14a [HUDI-427] [HUDI-971] Implement CLI support for performing bootstrap (#1869) wenningd 2020-08-08 12:37:29 -07:00
  • 5ee676e34f [MINOR] Move a test method to Transformations (#1934) Raymond Xu 2020-08-08 03:25:55 -07:00
  • 1072f2748a [HUDI-1026] Removed slf4j dependency from HoodieClientTestHarness (#1928) cheshta2904 2020-08-08 09:37:22 +05:30
  • 8b66524090 [MINOR] Remove unused import (#1932) Yungthuis 2020-08-08 12:04:31 +08:00
  • 4f74a84607 [HUDI-69] Support Spark Datasource for MOR table - RDD approach (#1848) Gary Li 2020-08-07 00:28:14 -07:00
  • ab453f2623 [HUDI-999] [RFC-12] Parallelize fetching of source data files/partitions (#1924) Udit Mehrotra 2020-08-06 23:44:57 -07:00
  • b51646dcc7 [HUDI-1151] Fix NPE when no new data in kafka using HoodieDeltaStreamer (#1921) Mathieu 2020-08-07 00:03:20 +08:00
  • 51ea27d665 [HUDI-875] Abstract hudi-sync-common, and support hudi-hive-sync, hudi-dla-sync (#1810) lw0090 2020-08-06 12:34:55 +08:00
  • c21209cb58 [HUDI-1149] Added a console metrics reporter and associated unit tests. Prashant Wason 2020-08-04 17:14:36 -07:00
  • 9bcd3221fd [HUDI-1144] Speedup spark read queries by caching metaclient in HoodieROPathFilter (#1919) Balaji Varadarajan 2020-08-05 09:19:10 -07:00
  • 7a2429f5ba [HUDI-575] Spark Streaming with async compaction support (#1752) Balaji Varadarajan 2020-08-05 07:50:15 -07:00
  • 61e027fadd [MINOR] Adding timeout for each command execution in docker and capture output. This will help get stdout/stderr of stuck commands (#1918) Balaji Varadarajan 2020-08-05 07:46:34 -07:00
  • 217a84192c [HUDI-1140] Fix Jcommander issue for --hoodie-conf in DeltaStreamer (#1898) Sreeram Ramji 2020-08-04 21:42:51 -07:00
  • d3711a2641 [HUDI-525] lack of insert info in delta_commit inflight liujianhui 2020-04-12 22:11:38 +08:00
  • ab11ba43e1 [REVERT] "[HUDI-1058] Make delete marker configurable (#1819)" (#1914) Sivabalan Narayanan 2020-08-04 18:20:38 -04:00
  • 539621bd33 [HUDI-242] Support for RFC-12/Bootstrapping of external datasets to hudi (#1876) vinoth chandar 2020-08-03 20:19:21 -07:00
  • 266bce12b3 [MINOR] Fixing usage of right config value for parallelism to dedup in Bulk Insert (#1905) Sivabalan Narayanan 2020-08-03 13:38:36 -04:00
  • 433d7d2c98 [HUDI-1058] Make delete marker configurable (#1819) Shen Hong 2020-08-03 23:06:31 +08:00
  • 8aa9142de8 [MINOR] Prevent scalatest plugin from running in non-UTs (#1897) Raymond Xu 2020-08-02 20:33:58 -07:00
  • 4ebd2db05b [MINOR] Suppressing full hive log and fetching only exceptions with context (#1903) Bhavani Sudha Saktheeswaran 2020-08-02 19:44:51 -07:00
  • 30dcd5cf06 [MINOR] Remove redundant import in hudi-integ-test (#1899) Mathieu 2020-08-02 21:30:23 +08:00
  • 10e4268792 [HUDI-995] Use Transformations, Assertions and SchemaTestUtil (#1884) Raymond Xu 2020-08-01 05:57:18 -07:00
  • e79fbc07fe [HUDI-1054] Several performance fixes during finalizing writes (#1768) Udit Mehrotra 2020-07-31 20:10:28 -07:00
  • 727f1df62c [MINOR] Suppressing spark logs for hudi-integ and hudi-utilities (#1894) n3nash 2020-07-31 19:01:25 -07:00
  • ccd70a7e48 [HUDI-472] Introduce configurations and new modes of sorting for bulk_insert (#1149) Y Ethan Guo 2020-07-31 06:52:42 -07:00
  • 2fc2b01d86 [HUDI-394] Provide a basic implementation of test suite Nishith Agarwal 2019-11-01 12:40:09 -07:00
  • d5b593b7d9 [MINOR] change log.info to log.debug (#1883) Bhavani Sudha Saktheeswaran 2020-07-28 09:49:03 -07:00
  • b2763f433b [MINOR] Fixing default index parallelism for simple index (#1882) Sivabalan Narayanan 2020-07-28 11:22:09 -04:00
  • 5e7931b1f9 [MINOR] Fix master compilation failure (#1881) Udit Mehrotra 2020-07-27 23:02:58 -07:00
  • fa419213f6 [HUDI-703] Add test for HoodieSyncCommand (#1774) hongdd 2020-07-28 08:31:43 +08:00
  • ca36c44cb3 [HUDI-995] Move TestRawTripPayload and HoodieTestDataGenerator to hudi-common (#1873) Raymond Xu 2020-07-27 04:21:45 -07:00
  • 0cb24e4a2d [MINOR] Use HoodieActiveTimeline.COMMIT_FORMATTER (#1874) Raymond Xu 2020-07-24 18:48:56 -07:00
  • 467d097dae [MINOR] Add Databricks File System to StorageSchemes (#1877) Gary Li 2020-07-24 18:47:09 -07:00
  • c3279cd598 [HUDI-1082] Fix minor bug in deciding the insert buckets (#1838) Shen Hong 2020-07-23 20:31:49 +08:00
  • da106803b6 [HUDI-1037] Introduce a write committed callback hook and given a default http callback implementation (#1842) Mathieu 2020-07-23 19:07:05 +08:00
  • f61cd1086a [HUDI-985] Introduce rerun ci bot (#1693) lamber-ken 2020-07-23 13:59:24 +08:00
  • c39778c150 [HUDI-1113] Add user define metrics reporter (#1851) zherenyu831 2020-07-23 14:46:36 +09:00
  • 3dd189ec7d [MINOR] Fix checkstyle issue on TestHoodieClientOnCopyOnWriteStorage (#1865) vinoth chandar 2020-07-22 21:54:45 -07:00
  • a8bd76c299 [HUDI-1029] In inline compaction mode, previously failed compactions needs to be retried before new compactions (#1857) vinoth chandar 2020-07-22 21:22:06 -07:00
  • 9bd37ef291 [MINOR] Fix flaky testUpsertsUpdatePartitionPath* tests (#1863) vinoth chandar 2020-07-22 19:52:34 -07:00
  • 5b6026ba43 [HUDI-802] Fixing deletes for inserts in same batch in write path (#1792) Sivabalan Narayanan 2020-07-22 22:39:57 -04:00
  • 12ef8c9249 [HUDI-708] Add temps show and unit test for TempViewCommand (#1770) hongdd 2020-07-23 08:43:46 +08:00
  • 743ef322b8 [HUDI-871] Add support for Tencent Cloud Object Storage(COS) (#1855) DeyinZhong 2020-07-22 17:40:19 +08:00
  • 5e7ab11e2e [HUDI-994] Move TestHoodieIndex test cases to unit tests (#1850) Raymond Xu 2020-07-21 10:23:43 -07:00
  • 1ec89e9a94 [HUDI-839] Introducing support for rollbacks using marker files (#1756) lw0090 2020-07-21 13:41:42 +08:00
  • b71f25f210 [HUDI-92] Provide reasonable names for Spark DAG stages in HUDI. (#1289) Prashant Wason 2020-07-19 10:29:25 -07:00
  • 1aae437257 [HUDI-1102] Add common useful Spark related and Table path detection utilities (#1841) Udit Mehrotra 2020-07-18 16:16:32 -07:00
  • bf1d36fa63 [HUDI-1087] Handle decimal type for realtime record reader with SparkSQL (#1831) wenningd 2020-07-15 07:30:58 -07:00
  • b399b4ad43 [HUDI-996] Add functional test in hudi-client (#1824) Raymond Xu 2020-07-14 17:28:50 -07:00
  • f5dc8ca733 [HUDI-994] Split TestHBaseIndex to unit tests (#1818) Raymond Xu 2020-07-13 20:32:01 -07:00
  • 21bb1b505a [HUDI-1068] Fixing deletes in global bloom when update partition path is set (#1793) Sivabalan Narayanan 2020-07-13 22:34:07 -04:00