28dafa774e
- Add a new action called INDEX, whose state transition is described in the RFC. - Changes in timeline to support the new action. - Add an index planner in ScheduleIndexActionExecutor. - Add index plan executor in RunIndexActionExecutor. - Add 3 APIs in HoodieTableMetadataWriter; a) scheduleIndex: will generate an index plan based on latest completed instant, initialize file groups and add a requested INDEX instant, b) index: executes the index plan and also takes care of writes that happened after indexing was requested, c) dropIndex: will drop index by removing the given metadata partition. - Add 2 new table configs to serve as the source of truth for inflight and completed indexes. - Support upgrade/downgrade taking care of the newly added configs. - Add tool to trigger indexing in HoodieIndexer. - Handle corner cases related to partial failures. - Abort gracefully after deleting partition and instant. - Handle other actions in timeline to consider before catching up
49 lines
1.3 KiB
JSON
49 lines
1.3 KiB
JSON
/*
|
|
* Licensed to the Apache Software Foundation (ASF) under one
|
|
* or more contributor license agreements. See the NOTICE file
|
|
* distributed with this work for additional information
|
|
* regarding copyright ownership. The ASF licenses this file
|
|
* to you under the Apache License, Version 2.0 (the
|
|
* "License"); you may not use this file except in compliance
|
|
* with the License. You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software
|
|
* distributed under the License is distributed on an "AS IS" BASIS,
|
|
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
* See the License for the specific language governing permissions and
|
|
* limitations under the License.
|
|
*/
|
|
{
|
|
"namespace": "org.apache.hudi.avro.model",
|
|
"type": "record",
|
|
"name": "HoodieIndexPartitionInfo",
|
|
"fields": [
|
|
{
|
|
"name": "version",
|
|
"type": [
|
|
"int",
|
|
"null"
|
|
],
|
|
"default": 1
|
|
},
|
|
{
|
|
"name": "metadataPartitionPath",
|
|
"type": [
|
|
"null",
|
|
"string"
|
|
],
|
|
"default": null
|
|
},
|
|
{
|
|
"name": "indexUptoInstant",
|
|
"type": [
|
|
"null",
|
|
"string"
|
|
],
|
|
"default": null
|
|
}
|
|
]
|
|
}
|