lanyuanxiaoyao/hudi
1
0
Fork 0
Code Activity

[HUDI-1295] Hash ID generator util for Hudi table columns, partition and files (#3884)

Browse Source 
* [HUDI-1295] Hash ID generator util for Hudi table columns, partition and files

- Adding a new utility class HashID to generate 32,64,128 bits hashes for any
  given message of string or byte array type. This class internally uses
  MessageDigest and xxhash libraries.

- Adding stateful hash holders for Hudi table columns, partition and files to
  pass around for metaindex and to convert to base64encoded strings whenever
  needed
This commit is contained in:
Manoj Govindassamy
2021-10-29 16:19:38 -07:00
committed by GitHub
parent 5b1992a92d
commit f6326693ef
8 changed files with 571 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
hudi-common/pom.xml
View File

@@ -248,5 +248,13 @@
</exclusion>
</exclusions>
</dependency>
<!-- LZ4 Hash Utils -->
<dependency>
<groupId>org.lz4</groupId>
<artifactId>lz4-java</artifactId>
<version>1.8.0</version>
</dependency>
</dependencies>
</project>