feature(web): 增加独立小工具
直接通过HDFS查询时间线，有时候保留现场会直接copy一个hdfs目录，不遵循生产表的逻辑，只能通过hdfs路经查询
This commit is contained in:
@@ -91,6 +91,9 @@ public class TimelineService {
|
||||
) throws IOException {
|
||||
Configuration configuration = new Configuration();
|
||||
FileSystem fileSystem = FileSystem.get(configuration);
|
||||
if (!fileSystem.exists(new Path(hdfs))) {
|
||||
throw new IOException(StrUtil.format("Path {} is not exists", hdfs));
|
||||
}
|
||||
HoodieTableMetaClient client = HoodieTableMetaClient.builder()
|
||||
.setConf(configuration)
|
||||
.setBasePath(hdfs)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package com.lanyuanxiaoyao.service.web.controller;
|
||||
|
||||
import cn.hutool.core.util.ObjectUtil;
|
||||
import cn.hutool.core.util.StrUtil;
|
||||
import com.lanyuanxiaoyao.service.configuration.entity.AmisResponse;
|
||||
import com.lanyuanxiaoyao.service.configuration.entity.PageResponse;
|
||||
import com.lanyuanxiaoyao.service.configuration.entity.hudi.HudiInstant;
|
||||
@@ -67,4 +68,38 @@ public class HudiController extends BaseController {
|
||||
PageResponse<HudiInstant> response = hudiService.timelineList(queryMap);
|
||||
return responseCrudData(response.getData(), response.getTotal());
|
||||
}
|
||||
|
||||
@GetMapping("/timeline/list_hdfs")
|
||||
public AmisResponse timelineHdfs(
|
||||
@RequestParam(value = "page", defaultValue = "1") Integer page,
|
||||
@RequestParam(value = "count", defaultValue = "10") Integer count,
|
||||
@RequestParam(value = "order", defaultValue = "timestamp") String order,
|
||||
@RequestParam(value = "direction", defaultValue = "desc") String direction,
|
||||
@RequestParam("hdfs") String hdfs,
|
||||
@RequestParam(value = "filter_type", required = false) List<String> filterType,
|
||||
@RequestParam(value = "filter_action", required = false) List<String> filterAction,
|
||||
@RequestParam(value = "filter_state", required = false) List<String> filterState
|
||||
) {
|
||||
MutableMap<String, Object> queryMap = Maps.mutable.empty();
|
||||
queryMap.put("page", page);
|
||||
queryMap.put("count", count);
|
||||
if (StrUtil.isNotBlank(order) && StrUtil.isNotBlank(direction)) {
|
||||
queryMap.put("order", order);
|
||||
queryMap.put("direction", direction);
|
||||
}
|
||||
if (ObjectUtil.isNotEmpty(hdfs)) {
|
||||
queryMap.put("hdfs", hdfs);
|
||||
}
|
||||
if (ObjectUtil.isNotEmpty(filterType)) {
|
||||
queryMap.put("filter_type", filterType);
|
||||
}
|
||||
if (ObjectUtil.isNotEmpty(filterAction)) {
|
||||
queryMap.put("filter_action", filterAction);
|
||||
}
|
||||
if (ObjectUtil.isNotEmpty(filterState)) {
|
||||
queryMap.put("filter_state", filterState);
|
||||
}
|
||||
PageResponse<HudiInstant> response = hudiService.timelineHdfsList(queryMap);
|
||||
return responseCrudData(response.getData(), response.getTotal());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -8,13 +8,13 @@ function crudCommonOptions() {
|
||||
}
|
||||
}
|
||||
|
||||
function paginationCommonOptions(perPage = true) {
|
||||
function paginationCommonOptions(perPage = true, maxButtons = 5) {
|
||||
let option = {
|
||||
type: 'pagination',
|
||||
layout: [
|
||||
'pager'
|
||||
],
|
||||
maxButtons: 5,
|
||||
maxButtons: maxButtons,
|
||||
showPageInput: false,
|
||||
perPageAvailable: [10, 15, 20, 50, 100, 200],
|
||||
}
|
||||
@@ -740,6 +740,52 @@ function flinkJobDialog() {
|
||||
}
|
||||
}
|
||||
|
||||
function timelineColumns() {
|
||||
return [
|
||||
{
|
||||
name: 'timestamp',
|
||||
label: '时间点',
|
||||
width: 150,
|
||||
sortable: true,
|
||||
},
|
||||
{
|
||||
name: 'action',
|
||||
label: '类型',
|
||||
width: 100,
|
||||
...mappingField('action', hudiTimelineActionMapping),
|
||||
filterable: filterableField(hudiTimelineActionMapping, false),
|
||||
},
|
||||
{
|
||||
name: 'state',
|
||||
label: ' 状态',
|
||||
width: 80,
|
||||
align: 'center',
|
||||
...mappingField('state', hudiTimelineStateMapping),
|
||||
filterable: filterableField(hudiTimelineStateMapping, true),
|
||||
},
|
||||
{
|
||||
name: 'fileTime',
|
||||
label: ' 文件时间',
|
||||
width: 150,
|
||||
align: 'center',
|
||||
type: 'tpl',
|
||||
tpl: "${DATETOSTR(DATE(fileTime), 'YYYY-MM-DD HH:mm:ss')}",
|
||||
},
|
||||
{
|
||||
name: 'fileName',
|
||||
label: '文件名',
|
||||
},
|
||||
{
|
||||
name: 'type',
|
||||
label: '来源',
|
||||
width: 60,
|
||||
align: 'center',
|
||||
...mappingField('type', hudiTimelineTypeMapping),
|
||||
filterable: filterableField(hudiTimelineTypeMapping, true),
|
||||
},
|
||||
]
|
||||
}
|
||||
|
||||
function tableMetaDialog() {
|
||||
return {
|
||||
title: 'Table 详情',
|
||||
@@ -920,49 +966,7 @@ function tableMetaDialog() {
|
||||
footerToolbar: [
|
||||
paginationCommonOptions(),
|
||||
],
|
||||
columns: [
|
||||
{
|
||||
name: 'timestamp',
|
||||
label: '时间点',
|
||||
width: 150,
|
||||
sortable: true,
|
||||
},
|
||||
{
|
||||
name: 'action',
|
||||
label: '类型',
|
||||
width: 100,
|
||||
...mappingField('action', hudiTimelineActionMapping),
|
||||
filterable: filterableField(hudiTimelineActionMapping, false),
|
||||
},
|
||||
{
|
||||
name: 'state',
|
||||
label: ' 状态',
|
||||
width: 80,
|
||||
align: 'center',
|
||||
...mappingField('state', hudiTimelineStateMapping),
|
||||
filterable: filterableField(hudiTimelineStateMapping, true),
|
||||
},
|
||||
{
|
||||
name: 'fileTime',
|
||||
label: ' 文件时间',
|
||||
width: 150,
|
||||
align: 'center',
|
||||
type: 'tpl',
|
||||
tpl: "${DATETOSTR(DATE(fileTime), 'YYYY-MM-DD HH:mm:ss')}",
|
||||
},
|
||||
{
|
||||
name: 'fileName',
|
||||
label: '文件名',
|
||||
},
|
||||
{
|
||||
name: 'type',
|
||||
label: '来源',
|
||||
width: 60,
|
||||
align: 'center',
|
||||
...mappingField('type', hudiTimelineTypeMapping),
|
||||
filterable: filterableField(hudiTimelineTypeMapping, true),
|
||||
},
|
||||
],
|
||||
columns: timelineColumns(),
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
63
web/components/tool-tab.js
Normal file
63
web/components/tool-tab.js
Normal file
@@ -0,0 +1,63 @@
|
||||
function toolTab() {
|
||||
return {
|
||||
title: `小工具`,
|
||||
tab: [
|
||||
{
|
||||
type: 'form',
|
||||
title: '查询时间线',
|
||||
mode: 'horizontal',
|
||||
horizontal: {
|
||||
leftFixed: 'md',
|
||||
},
|
||||
actions: [
|
||||
{
|
||||
type: 'submit',
|
||||
label: '查询',
|
||||
actionType: 'dialog',
|
||||
dialog: {
|
||||
title: 'Hudi 表时间线',
|
||||
actions: [],
|
||||
size: 'lg',
|
||||
body: {
|
||||
type: 'crud',
|
||||
api: {
|
||||
method: 'get',
|
||||
url: '${base}/hudi/timeline/list_hdfs',
|
||||
data: {
|
||||
page: '${page|default:undefined}',
|
||||
count: '${perPage|default:undefined}',
|
||||
order: '${orderBy|default:undefined}',
|
||||
direction: '${orderDir|default:undefined}',
|
||||
hdfs: '${hdfs|default:undefined}',
|
||||
filter_type: "${type|default:active}",
|
||||
filter_action: '${action|default:undefined}',
|
||||
filter_state: '${state|default:undefined}',
|
||||
},
|
||||
},
|
||||
...crudCommonOptions(),
|
||||
perPage: 50,
|
||||
headerToolbar: [
|
||||
"reload",
|
||||
paginationCommonOptions(undefined, 10),
|
||||
],
|
||||
footerToolbar: [
|
||||
paginationCommonOptions(undefined, 10),
|
||||
],
|
||||
columns: timelineColumns(),
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
body: [
|
||||
{
|
||||
type: 'input-text',
|
||||
name: 'hdfs',
|
||||
label: '表HDFS路经',
|
||||
required: true,
|
||||
}
|
||||
]
|
||||
},
|
||||
{type: 'divider'},
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -44,6 +44,7 @@
|
||||
<script src="components/log-tab.js"></script>
|
||||
<script src="components/version-tab.js"></script>
|
||||
<script src="components/overview-tab.js"></script>
|
||||
<script src="components/tool-tab.js"></script>
|
||||
<script type="text/javascript">
|
||||
(function () {
|
||||
let amis = amisRequire('amis/embed')
|
||||
@@ -67,6 +68,7 @@
|
||||
yarnTab('b1,b5,a4', '压缩 b1 b5 a4', 'datalake,ten_iap.datalake', 'Compaction'),
|
||||
cloudTab(),
|
||||
yarnClusterTab(),
|
||||
toolTab(),
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user