lanyuanxiaoyao/DiAL
1
0
Fork 0
Code Activity

refactor: 将 checker normalize 职责下沉到各 runner 目录

Browse Source 
- 新增 CheckerDefinition.normalize 必需方法，typecheck 兜底遗漏实现
- 新增 expect/normalize.ts 共享 helper（compactExpect、normalizeValue、
  normalizeContent、normalizeKeyed）
- 为 HTTP、Cmd、DB、TCP、UDP、ICMP、LLM、WS、DNS 各新增独立 normalize.ts
- 简化 normalizer.ts：删除所有 checker type switch，改为 registry 委托
- 修复 DNS authoring 简写 bug：durationMs、valueCount、result 等字段
  现可通过完整加载链路
- 新增 DNS 回归测试和 registry 级合同测试
- 更新 docs/development/checker.md：补充 normalize 规范、文件结构、
  测试要求和 checklist
This commit is contained in:
兰缘小妖
2026-05-25 16:16:41 +08:00
parent c1db793073
commit 77c6015b3a
26 changed files with 565 additions and 194 deletions
Download Patch File Download Diff File Expand all files Collapse all files

78
docs/development/checker.md
View File

@@ -9,7 +9,7 @@ Checker 是 DiAL 的核心扩展单元。每个 checker 是 `src/server/checker/
## 设计原则
- 每个 checker 必须自包含在 `src/server/checker/runner/<type>/`。
- checker 专属类型、schema、validate、execute、expect 和协议辅助逻辑放在同一目录。
- checker 专属类型、schema、validate、execute、expect、normalize 和协议辅助逻辑放在同一目录。
- 注册只修改 `src/server/checker/runner/index.ts`，中间层不新增 type switch。
- schema 层只描述契约，语义规则放入 `validate.ts`。
- `resolve()` 只做默认值填充、路径解析和单位转换，不执行校验。
@@ -29,19 +29,20 @@ checkerRegistry
└── store.ts
```
注册后，中间层通过 registry 自动委托 schema 生成、契约校验、配置 resolve、执行和序列化。新增 checker 不应在中间层新增 `switch/case` 或类型分支。
注册后，中间层通过 registry 自动委托 schema 生成、契约校验、配置 normalize、配置 resolve、执行和序列化。新增 checker 不应在中间层新增 `switch/case` 或类型分支。
## 标准文件结构
| 文件 | 职责 |
| ------------- | ----------------------------------------------------- |
| `index.ts` | 模块入口，re-export Checker 类 |
| `types.ts` | Checker 专属类型 |
| `schema.ts` | TypeBox 契约 schema，包含 config 和 expect |
| `validate.ts` | 启动期语义校验 |
| `execute.ts` | Checker 类，实现 resolve、execute、serialize |
| `expect.ts` | Checker 专用断言函数 |
| 其他文件 | 协议解析、编码、provider 适配、平台命令封装等专属逻辑 |
| 文件 | 职责 |
| -------------- | ------------------------------------------------------- |
| `index.ts` | 模块入口，re-export Checker 类 |
| `types.ts` | Checker 专属类型 |
| `schema.ts` | TypeBox 契约 schema，包含 config 和 expect |
| `validate.ts` | 启动期语义校验 |
| `normalize.ts` | Checker 专属 authoring expect 归一化 |
| `execute.ts` | Checker 类，实现 normalize、resolve、execute、serialize |
| `expect.ts` | Checker 专用断言函数 |
| 其他文件 | 协议解析、编码、provider 适配、平台命令封装等专属逻辑 |
## 类型定义
@@ -86,6 +87,38 @@ checker 必须提供 `CheckerSchemas`，包含 Authoring 和 Normalized 两套 c
| `validateJsonPath` | 校验项目支持的 JSONPath 子集 |
| `isJsonValue` | 判断合法 JSON value |
## normalize 规范
`normalize()` 在 `CheckerDefinition` 中定义为必需方法，负责将 authoring expect DSL 转换为 normalized 形态。输入为变量已解析后的 target，输出为适配 normalized schema 的 target。该方法在 `resolve()` 和 normalized contract 校验之前执行。
在 `normalize.ts` 中实现 `normalizeTargetExpect` 函数，`execute.ts` 中的 `normalize` 方法委托到该函数。
共享 normalize helper 位于 `src/server/checker/expect/normalize.ts`：
| 函数 | 用途 |
| ------------------ | -------------------------------------------------------- |
| `compactExpect` | 合并两个 expect record，过滤 undefined 字段 |
| `normalizeValue` | ValueMatcher 原始值简写展开为 `{equals: value}` |
| `normalizeContent` | ContentExpectations 简写展开为 normalized 形态 |
| `normalizeKeyed` | KeyedExpectations 对象形态展开为 `[{key, matcher}]` 数组 |
```typescript
import { compactExpect, normalizeContent, normalizeKeyed, normalizeValue } from "../../expect/normalize";
export function normalizeTargetExpect(target: RawTargetConfig): RawTargetConfig {
if (target.expect === undefined || !isPlainObject(target.expect)) return target;
const raw = target.expect as Record<string, unknown>;
return {
...target,
expect: compactExpect(raw, {
/* checker 专属字段映射 */
}),
};
}
```
expect 字段的归一化规则：ValueMatcher 字段调用 `normalizeValue()`，ContentExpectations 字段调用 `normalizeContent()`，KeyedExpectations 字段调用 `normalizeKeyed()`，boolean/enum/array 等非断言模型字段直接透传。
## resolve 规范
`resolve()` 只做内置默认值填充、路径解析、单位转换，不执行校验。输入已经通过 Normalized schema 和语义校验，expect 已是 normalized 形态。
@@ -132,14 +165,15 @@ const resolvedExpect: ResolvedXxxExpectConfig = expect
测试文件放在 `tests/server/checker/runner/<type>/`，结构镜像源文件。
| 测试类别 | 覆盖内容 |
| ------------ | ---------------------------------------- |
| 契约测试 | TypeBox schema 与 JSON Schema 导出一致性 |
| 语义校验测试 | 合法和非法配置 |
| resolve 测试 | 默认值合并、路径解析、单位转换 |
| execute 测试 | 成功、失败、超时、expect 组合 |
| 注册测试 | registry 注册行为 |
| 配置加载测试 | 含新 checker 的 YAML 完整加载流程 |
| 测试类别 | 覆盖内容 |
| -------------- | ---------------------------------------------------- |
| 契约测试 | TypeBox schema 与 JSON Schema 导出一致性 |
| 语义校验测试 | 合法和非法配置 |
| normalize 测试 | authoring expect 简写展开和 normalized contract 通过 |
| resolve 测试 | 默认值合并、路径解析、单位转换 |
| execute 测试 | 成功、失败、超时、expect 组合 |
| 注册测试 | registry 注册行为 |
| 配置加载测试 | 含新 checker 的 YAML 完整加载流程 |
## 文档和 schema 更新
@@ -169,11 +203,11 @@ bun run check
## 完成检查清单
```text
□ checker 类型、schema、validate、resolve、execute、serialize 已实现
□ checker 类型、schema、validate、normalize、resolve、execute、serialize 已实现
□ checker 已在 runner/index.ts 注册
□ 配置契约、语义校验和 JSON Schema 导出已同步
□ probes.example.yaml 已添加或更新示例
□ tests/server/checker/runner/<type>/ 已覆盖契约、校验、resolve、execute、注册和配置加载
□ tests/server/checker/runner/<type>/ 已覆盖契约、校验、normalize、resolve、execute、注册和配置加载
□ docs/user/checkers/<type>.md 已添加或更新
□ docs/user/checkers/README.md 已添加或更新
□ 文档影响分析已完成，必要文档已同步
@@ -184,4 +218,4 @@ bun run check
## 更新触发条件
修改 checker 开发机制、目录结构、schema/validate/resolve/execute/expect 约定、测试要求、验证命令或文档同步 checklist 时，必须更新本文档。
修改 checker 开发机制、目录结构、schema/validate/normalize/resolve/execute/expect 约定、测试要求、验证命令或文档同步 checklist 时，必须更新本文档。

2
src/server/checker/config-loader.ts
View File

@@ -60,7 +60,7 @@ export async function loadConfig(configPath: string): Promise<ResolvedConfig> {
throw new Error("配置文件内容为空或格式无效");
}
const normalizeResult = normalizeAuthoringConfig(parsed);
const normalizeResult = normalizeAuthoringConfig(parsed, checkerRegistry);
if (normalizeResult.issues.length > 0) {
throwConfigIssues(dedupeIssues(normalizeResult.issues));
}

50
src/server/checker/expect/normalize.ts Normal file
View File

@@ -0,0 +1,50 @@
import { isPlainObject } from "es-toolkit";
import { resolveContentExpectations } from "./content";
import { CONTENT_EXTRACTOR_KEY_SET, MATCHER_KEY_SET } from "./keys";
import { isValueMatcherObject, isValueMatcherPrimitive, resolveValueExpectation } from "./value";
type ExpectRecord = Record<string, unknown>;
export function compactExpect(original: ExpectRecord, overrides: ExpectRecord): ExpectRecord {
const result: ExpectRecord = {};
for (const [key, value] of Object.entries(original)) {
if (value !== undefined) result[key] = value;
}
for (const [key, value] of Object.entries(overrides)) {
if (value !== undefined) result[key] = value;
}
return result;
}
export function normalizeContent(value: unknown): unknown {
if (value === undefined) return undefined;
if (!Array.isArray(value)) return value;
return (value as unknown[]).map((entry): unknown => {
if (!canNormalizeContentEntry(entry)) return entry;
const resolved = resolveContentExpectations([entry] as never);
return resolved?.[0];
});
}
export function normalizeKeyed(value: unknown): unknown {
if (value === undefined) return undefined;
if (!isPlainObject(value)) return value;
return Object.entries(value as ExpectRecord).map(([key, item]) => ({ key, matcher: normalizeValue(item) }));
}
export function normalizeValue(value: unknown): unknown {
if (value === undefined) return undefined;
if (isValueMatcherPrimitive(value) || isValueMatcherObject(value)) return resolveValueExpectation(value);
return value;
}
function canNormalizeContentEntry(value: unknown): boolean {
if (!isPlainObject(value)) return false;
const keys = Object.keys(value);
const extractorKeys = keys.filter((key) => CONTENT_EXTRACTOR_KEY_SET.has(key));
const matcherKeys = keys.filter((key) => MATCHER_KEY_SET.has(key));
if (extractorKeys.length === 0) return matcherKeys.length > 0 && matcherKeys.length === keys.length;
if (extractorKeys.length !== 1 || matcherKeys.length > 0 || keys.length !== 1) return false;
return isPlainObject((value as ExpectRecord)[extractorKeys[0]!]);
}

183
src/server/checker/normalizer.ts
View File

@@ -1,17 +1,17 @@
import { isPlainObject } from "es-toolkit";
import type { CheckerRegistry } from "./runner/registry";
import type { ConfigValidationIssue } from "./schema/issues";
import type { AuthoringProbeConfig, NormalizedProbeConfig } from "./schema/types";
import type { RawTargetConfig } from "./types";
import { resolveContentExpectations } from "./expect/content";
import { CONTENT_EXTRACTOR_KEY_SET, MATCHER_KEY_SET } from "./expect/keys";
import { isValueMatcherObject, isValueMatcherPrimitive, resolveValueExpectation } from "./expect/value";
import { checkerRegistry } from "./runner";
import { resolveVariables } from "./variables";
type ExpectRecord = Record<string, unknown>;
export function normalizeAuthoringConfig(config: unknown): {
export function normalizeAuthoringConfig(
config: unknown,
registry: CheckerRegistry = checkerRegistry,
): {
config: unknown;
issues: ConfigValidationIssue[];
} {
@@ -23,177 +23,20 @@ export function normalizeAuthoringConfig(config: unknown): {
const normalized = { ...(variableResult.config as Record<string, unknown>) };
delete normalized["variables"];
if (Array.isArray(normalized["targets"])) {
normalized["targets"] = normalized["targets"].map((target) => normalizeTarget(target));
normalized["targets"] = normalized["targets"].map((target) => normalizeTarget(target, registry));
}
return { config: normalized, issues: variableResult.issues };
}
function canNormalizeContentEntry(value: unknown): boolean {
if (!isPlainObject(value)) return false;
const keys = Object.keys(value);
const extractorKeys = keys.filter((key) => CONTENT_EXTRACTOR_KEY_SET.has(key));
const matcherKeys = keys.filter((key) => MATCHER_KEY_SET.has(key));
if (extractorKeys.length === 0) return matcherKeys.length > 0 && matcherKeys.length === keys.length;
if (extractorKeys.length !== 1 || matcherKeys.length > 0 || keys.length !== 1) return false;
return isPlainObject((value as ExpectRecord)[extractorKeys[0]!]);
}
function compact(original: ExpectRecord, overrides: ExpectRecord): ExpectRecord {
const result: ExpectRecord = {};
for (const [key, value] of Object.entries(original)) {
if (value !== undefined) result[key] = value;
}
for (const [key, value] of Object.entries(overrides)) {
if (value !== undefined) result[key] = value;
}
return result;
}
function normalizeCommandExpect(raw: ExpectRecord): ExpectRecord {
return compact(raw, {
durationMs: normalizeValue(raw["durationMs"]),
exitCode: raw["exitCode"],
stderr: normalizeContent(raw["stderr"]),
stdout: normalizeContent(raw["stdout"]),
});
}
function normalizeContent(value: unknown): unknown {
if (value === undefined) return undefined;
if (!Array.isArray(value)) return value;
return (value as unknown[]).map((entry): unknown => {
if (!canNormalizeContentEntry(entry)) return entry;
const resolved = resolveContentExpectations([entry] as never);
return resolved?.[0];
});
}
function normalizeDbExpect(raw: ExpectRecord): ExpectRecord {
return compact(raw, {
durationMs: normalizeValue(raw["durationMs"]),
result: normalizeContent(raw["result"]),
rowCount: normalizeValue(raw["rowCount"]),
rows: Array.isArray(raw["rows"]) ? raw["rows"].map((row) => normalizeKeyed(row)) : raw["rows"],
});
}
function normalizeExpect(type: string, expect: unknown): unknown {
if (!isPlainObject(expect)) return expect;
const raw = expect as ExpectRecord;
switch (type) {
case "cmd":
return normalizeCommandExpect(raw);
case "db":
return normalizeDbExpect(raw);
case "http":
return normalizeHttpExpect(raw);
case "icmp":
return normalizeIcmpExpect(raw);
case "llm":
return normalizeLlmExpect(raw);
case "tcp":
return normalizeTcpExpect(raw);
case "udp":
return normalizeUdpExpect(raw);
case "ws":
return normalizeWsExpect(raw);
default:
return expect;
}
}
function normalizeHttpExpect(raw: ExpectRecord): ExpectRecord {
return compact(raw, {
body: normalizeContent(raw["body"]),
durationMs: normalizeValue(raw["durationMs"]),
headers: normalizeKeyed(raw["headers"]),
status: raw["status"],
});
}
function normalizeIcmpExpect(raw: ExpectRecord): ExpectRecord {
return compact(raw, {
alive: raw["alive"],
avgLatencyMs: normalizeValue(raw["avgLatencyMs"]),
durationMs: normalizeValue(raw["durationMs"]),
maxLatencyMs: normalizeValue(raw["maxLatencyMs"]),
packetLossPercent: normalizeValue(raw["packetLossPercent"]),
});
}
function normalizeKeyed(value: unknown): unknown {
if (value === undefined) return undefined;
if (!isPlainObject(value)) return value;
return Object.entries(value as ExpectRecord).map(([key, item]) => ({ key, matcher: normalizeValue(item) }));
}
function normalizeLlmExpect(raw: ExpectRecord): ExpectRecord {
return compact(raw, {
durationMs: normalizeValue(raw["durationMs"]),
finishReason: normalizeValue(raw["finishReason"]),
headers: normalizeKeyed(raw["headers"]),
output: normalizeContent(raw["output"]),
rawFinishReason: normalizeValue(raw["rawFinishReason"]),
status: raw["status"],
stream: isPlainObject(raw["stream"])
? compact(raw["stream"] as ExpectRecord, {
completed: (raw["stream"] as ExpectRecord)["completed"],
firstTokenMs: normalizeValue((raw["stream"] as ExpectRecord)["firstTokenMs"]),
})
: raw["stream"],
usage: isPlainObject(raw["usage"])
? compact(raw["usage"] as ExpectRecord, {
inputTokens: normalizeValue((raw["usage"] as ExpectRecord)["inputTokens"]),
outputTokens: normalizeValue((raw["usage"] as ExpectRecord)["outputTokens"]),
totalTokens: normalizeValue((raw["usage"] as ExpectRecord)["totalTokens"]),
})
: raw["usage"],
});
}
function normalizeTarget(target: unknown): unknown {
function normalizeTarget(target: unknown, registry: CheckerRegistry): unknown {
if (!isPlainObject(target)) return target;
const result = { ...(target as RawTargetConfig) };
if (result.expect !== undefined) {
result.expect = normalizeExpect(result.type, result.expect);
}
return result;
}
function normalizeTcpExpect(raw: ExpectRecord): ExpectRecord {
return compact(raw, {
banner: normalizeContent(raw["banner"]),
connected: raw["connected"],
durationMs: normalizeValue(raw["durationMs"]),
});
}
function normalizeUdpExpect(raw: ExpectRecord): ExpectRecord {
return compact(raw, {
durationMs: normalizeValue(raw["durationMs"]),
responded: raw["responded"],
response: normalizeContent(raw["response"]),
responseSize: normalizeValue(raw["responseSize"]),
sourceHost: normalizeValue(raw["sourceHost"]),
sourcePort: normalizeValue(raw["sourcePort"]),
});
}
function normalizeValue(value: unknown): unknown {
if (value === undefined) return undefined;
if (isValueMatcherPrimitive(value) || isValueMatcherObject(value)) return resolveValueExpectation(value);
return value;
}
function normalizeWsExpect(raw: ExpectRecord): ExpectRecord {
return compact(raw, {
connected: raw["connected"],
connectTimeMs: normalizeValue(raw["connectTimeMs"]),
durationMs: normalizeValue(raw["durationMs"]),
handshakeHeaders: normalizeKeyed(raw["handshakeHeaders"]),
message: normalizeContent(raw["message"]),
});
const type = result.type;
if (typeof type !== "string") return result;
const checker = registry?.tryGet(type);
if (!checker) return result;
return checker.normalize(result);
}
export type { AuthoringProbeConfig, NormalizedProbeConfig };

5
src/server/checker/runner/cmd/execute.ts
View File

@@ -10,6 +10,7 @@ import { errorFailure } from "../../expect/failure";
import { checkValueExpectation } from "../../expect/value";
import { parseSize } from "../../utils";
import { checkExitCode } from "./expect";
import { normalizeTargetExpect } from "./normalize";
import { commandCheckerSchemas } from "./schema";
import { validateCommandConfig } from "./validate";
@@ -202,6 +203,10 @@ export class CommandChecker implements CheckerDefinition<ResolvedCommandTarget>
};
}
normalize(target: RawTargetConfig): RawTargetConfig {
return normalizeTargetExpect(target);
}
resolve(target: RawTargetConfig, context: ResolveContext): ResolvedCommandTarget {
const t = target as RawTargetConfig & { cmd: CommandTargetConfig; type: "cmd" };

19
src/server/checker/runner/cmd/normalize.ts Normal file
View File

@@ -0,0 +1,19 @@
import { isPlainObject } from "es-toolkit";
import type { RawTargetConfig } from "../../types";
import { compactExpect, normalizeContent, normalizeValue } from "../../expect/normalize";
export function normalizeTargetExpect(target: RawTargetConfig): RawTargetConfig {
if (target.expect === undefined || !isPlainObject(target.expect)) return target;
const raw = target.expect as Record<string, unknown>;
return {
...target,
expect: compactExpect(raw, {
durationMs: normalizeValue(raw["durationMs"]),
exitCode: raw["exitCode"],
stderr: normalizeContent(raw["stderr"]),
stdout: normalizeContent(raw["stdout"]),
}),
};
}

5
src/server/checker/runner/db/execute.ts
View File

@@ -9,6 +9,7 @@ import { checkContentExpectations } from "../../expect/content";
import { errorFailure } from "../../expect/failure";
import { checkValueExpectation } from "../../expect/value";
import { checkRowCount, checkRows } from "./expect";
import { normalizeTargetExpect } from "./normalize";
import { dbCheckerSchemas } from "./schema";
import { validateDbConfig } from "./validate";
@@ -223,6 +224,10 @@ export class DbChecker implements CheckerDefinition<ResolvedDbTarget> {
}
}
normalize(target: RawTargetConfig): RawTargetConfig {
return normalizeTargetExpect(target);
}
resolve(target: RawTargetConfig, context: ResolveContext): ResolvedDbTarget {
const t = target as RawTargetConfig & { db: DbTargetConfig; type: "db" };

19
src/server/checker/runner/db/normalize.ts Normal file
View File

@@ -0,0 +1,19 @@
import { isPlainObject } from "es-toolkit";
import type { RawTargetConfig } from "../../types";
import { compactExpect, normalizeContent, normalizeKeyed, normalizeValue } from "../../expect/normalize";
export function normalizeTargetExpect(target: RawTargetConfig): RawTargetConfig {
if (target.expect === undefined || !isPlainObject(target.expect)) return target;
const raw = target.expect as Record<string, unknown>;
return {
...target,
expect: compactExpect(raw, {
durationMs: normalizeValue(raw["durationMs"]),
result: normalizeContent(raw["result"]),
rowCount: normalizeValue(raw["rowCount"]),
rows: Array.isArray(raw["rows"]) ? raw["rows"].map((row) => normalizeKeyed(row)) : raw["rows"],
}),
};
}

5
src/server/checker/runner/dns/execute.ts
View File

@@ -25,6 +25,7 @@ import {
checkTtlMin,
checkValueCount,
} from "./expect";
import { normalizeTargetExpect } from "./normalize";
import { dnsCheckerSchemas } from "./schema";
import { queryDns } from "./transport";
import { validateDnsConfig } from "./validate";
@@ -83,6 +84,10 @@ export class DnsChecker implements CheckerDefinition<ResolvedDnsTarget> {
}
}
normalize(target: RawTargetConfig): RawTargetConfig {
return normalizeTargetExpect(target);
}
resolve(target: RawTargetConfig, context: ResolveContext): ResolvedDnsTarget {
const dns = target["dns"] as DnsServerConfig | DnsSystemConfig;

28
src/server/checker/runner/dns/normalize.ts Normal file
View File

@@ -0,0 +1,28 @@
import { isPlainObject } from "es-toolkit";
import type { RawTargetConfig } from "../../types";
import { compactExpect, normalizeContent, normalizeValue } from "../../expect/normalize";
export function normalizeTargetExpect(target: RawTargetConfig): RawTargetConfig {
if (target.expect === undefined || !isPlainObject(target.expect)) return target;
const raw = target.expect as Record<string, unknown>;
return {
...target,
expect: compactExpect(raw, {
answerCount: normalizeValue(raw["answerCount"]),
authenticatedData: raw["authenticatedData"],
authoritative: raw["authoritative"],
durationMs: normalizeValue(raw["durationMs"]),
rcode: raw["rcode"],
recursionAvailable: raw["recursionAvailable"],
responded: raw["responded"],
result: normalizeContent(raw["result"]),
truncated: raw["truncated"],
ttlMax: normalizeValue(raw["ttlMax"]),
ttlMin: normalizeValue(raw["ttlMin"]),
valueCount: normalizeValue(raw["valueCount"]),
values: raw["values"],
}),
};
}

5
src/server/checker/runner/http/execute.ts
View File

@@ -10,6 +10,7 @@ import { checkHeaderExpectations } from "../../expect/headers";
import { checkStatusCode } from "../../expect/status";
import { checkValueExpectation, displayValueExpectation } from "../../expect/value";
import { parseSize } from "../../utils";
import { normalizeTargetExpect } from "./normalize";
import { httpCheckerSchemas } from "./schema";
import { validateHttpConfig } from "./validate";
@@ -172,6 +173,10 @@ export class HttpChecker implements CheckerDefinition<ResolvedHttpTarget> {
}
}
normalize(target: RawTargetConfig): RawTargetConfig {
return normalizeTargetExpect(target);
}
resolve(target: RawTargetConfig, context: ResolveContext): ResolvedHttpTarget {
const t = target as RawTargetConfig & { http: HttpTargetConfig; type: "http" };

19
src/server/checker/runner/http/normalize.ts Normal file
View File

@@ -0,0 +1,19 @@
import { isPlainObject } from "es-toolkit";
import type { RawTargetConfig } from "../../types";
import { compactExpect, normalizeContent, normalizeKeyed, normalizeValue } from "../../expect/normalize";
export function normalizeTargetExpect(target: RawTargetConfig): RawTargetConfig {
if (target.expect === undefined || !isPlainObject(target.expect)) return target;
const raw = target.expect as Record<string, unknown>;
return {
...target,
expect: compactExpect(raw, {
body: normalizeContent(raw["body"]),
durationMs: normalizeValue(raw["durationMs"]),
headers: normalizeKeyed(raw["headers"]),
status: raw["status"],
}),
};
}

5
src/server/checker/runner/icmp/execute.ts
View File

@@ -8,6 +8,7 @@ import { errorFailure } from "../../expect/failure";
import { checkValueExpectation } from "../../expect/value";
import { buildPingCommand } from "./command";
import { checkAlive, checkAvgLatency, checkMaxLatency, checkPacketLoss } from "./expect";
import { normalizeTargetExpect } from "./normalize";
import { parsePingOutput } from "./parse";
import { icmpCheckerSchemas } from "./schema";
import { validatePingConfig } from "./validate";
@@ -153,6 +154,10 @@ export class IcmpChecker implements CheckerDefinition<ResolvedPingTarget> {
};
}
normalize(target: RawTargetConfig): RawTargetConfig {
return normalizeTargetExpect(target);
}
resolve(target: RawTargetConfig, context: ResolveContext): ResolvedPingTarget {
const t = target as RawTargetConfig & { icmp: PingTargetConfig; type: "icmp" };

20
src/server/checker/runner/icmp/normalize.ts Normal file
View File

@@ -0,0 +1,20 @@
import { isPlainObject } from "es-toolkit";
import type { RawTargetConfig } from "../../types";
import { compactExpect, normalizeValue } from "../../expect/normalize";
export function normalizeTargetExpect(target: RawTargetConfig): RawTargetConfig {
if (target.expect === undefined || !isPlainObject(target.expect)) return target;
const raw = target.expect as Record<string, unknown>;
return {
...target,
expect: compactExpect(raw, {
alive: raw["alive"],
avgLatencyMs: normalizeValue(raw["avgLatencyMs"]),
durationMs: normalizeValue(raw["durationMs"]),
maxLatencyMs: normalizeValue(raw["maxLatencyMs"]),
packetLossPercent: normalizeValue(raw["packetLossPercent"]),
}),
};
}

5
src/server/checker/runner/llm/execute.ts
View File

@@ -8,6 +8,7 @@ import type { LlmTargetConfig, ResolvedLlmExpectConfig, ResolvedLlmTarget } from
import { errorFailure } from "../../expect/failure";
import { checkValueExpectation } from "../../expect/value";
import { runExpects } from "./expect";
import { normalizeTargetExpect } from "./normalize";
import {
buildObservationFromApiCallError,
buildObservationFromGenerateText,
@@ -127,6 +128,10 @@ export class LlmChecker implements CheckerDefinition<ResolvedLlmTarget> {
}
}
normalize(target: RawTargetConfig): RawTargetConfig {
return normalizeTargetExpect(target);
}
resolve(target: RawTargetConfig, context: ResolveContext): ResolvedLlmTarget {
const t = target as RawTargetConfig & { llm: LlmTargetConfig; type: "llm" };

34
src/server/checker/runner/llm/normalize.ts Normal file
View File

@@ -0,0 +1,34 @@
import { isPlainObject } from "es-toolkit";
import type { RawTargetConfig } from "../../types";
import { compactExpect, normalizeContent, normalizeKeyed, normalizeValue } from "../../expect/normalize";
export function normalizeTargetExpect(target: RawTargetConfig): RawTargetConfig {
if (target.expect === undefined || !isPlainObject(target.expect)) return target;
const raw = target.expect as Record<string, unknown>;
return {
...target,
expect: compactExpect(raw, {
durationMs: normalizeValue(raw["durationMs"]),
finishReason: normalizeValue(raw["finishReason"]),
headers: normalizeKeyed(raw["headers"]),
output: normalizeContent(raw["output"]),
rawFinishReason: normalizeValue(raw["rawFinishReason"]),
status: raw["status"],
stream: isPlainObject(raw["stream"])
? compactExpect(raw["stream"] as Record<string, unknown>, {
completed: (raw["stream"] as Record<string, unknown>)["completed"],
firstTokenMs: normalizeValue((raw["stream"] as Record<string, unknown>)["firstTokenMs"]),
})
: raw["stream"],
usage: isPlainObject(raw["usage"])
? compactExpect(raw["usage"] as Record<string, unknown>, {
inputTokens: normalizeValue((raw["usage"] as Record<string, unknown>)["inputTokens"]),
outputTokens: normalizeValue((raw["usage"] as Record<string, unknown>)["outputTokens"]),
totalTokens: normalizeValue((raw["usage"] as Record<string, unknown>)["totalTokens"]),
})
: raw["usage"],
}),
};
}

5
src/server/checker/runner/tcp/execute.ts
View File

@@ -8,6 +8,7 @@ import { errorFailure } from "../../expect/failure";
import { checkValueExpectation } from "../../expect/value";
import { parseSize } from "../../utils";
import { checkBanner, checkConnected } from "./expect";
import { normalizeTargetExpect } from "./normalize";
import { tcpCheckerSchemas } from "./schema";
import { validateTcpConfig } from "./validate";
@@ -203,6 +204,10 @@ export class TcpChecker implements CheckerDefinition<ResolvedTcpTarget> {
}
}
normalize(target: RawTargetConfig): RawTargetConfig {
return normalizeTargetExpect(target);
}
resolve(target: RawTargetConfig, context: ResolveContext): ResolvedTcpTarget {
const t = target as RawTargetConfig & { tcp: TcpTargetConfig; type: "tcp" };

18
src/server/checker/runner/tcp/normalize.ts Normal file
View File

@@ -0,0 +1,18 @@
import { isPlainObject } from "es-toolkit";
import type { RawTargetConfig } from "../../types";
import { compactExpect, normalizeContent, normalizeValue } from "../../expect/normalize";
export function normalizeTargetExpect(target: RawTargetConfig): RawTargetConfig {
if (target.expect === undefined || !isPlainObject(target.expect)) return target;
const raw = target.expect as Record<string, unknown>;
return {
...target,
expect: compactExpect(raw, {
banner: normalizeContent(raw["banner"]),
connected: raw["connected"],
durationMs: normalizeValue(raw["durationMs"]),
}),
};
}

1
src/server/checker/runner/types.ts
View File

@@ -13,6 +13,7 @@ export interface CheckerDefinition<TResolved extends ResolvedTargetBase = Resolv
buildDetail(observation: Record<string, unknown>): null | string;
readonly configKey: string;
execute(target: TResolved, ctx: CheckerContext): Promise<CheckResult>;
normalize(target: RawTargetConfig): RawTargetConfig;
resolve(target: RawTargetConfig, context: ResolveContext): TResolved;
readonly schemas: CheckerSchemas;
serialize(target: TResolved): { config: string; target: string };

5
src/server/checker/runner/udp/execute.ts
View File

@@ -9,6 +9,7 @@ import { checkValueExpectation } from "../../expect/value";
import { parseSize } from "../../utils";
import { decodePayload, encodeResponse } from "./encoding";
import { checkResponded, checkResponseSize, checkResponseText, checkSourceHost, checkSourcePort } from "./expect";
import { normalizeTargetExpect } from "./normalize";
import { udpCheckerSchemas } from "./schema";
import { validateUdpConfig } from "./validate";
@@ -295,6 +296,10 @@ export class UdpChecker implements CheckerDefinition<ResolvedUdpTarget> {
}
}
normalize(target: RawTargetConfig): RawTargetConfig {
return normalizeTargetExpect(target);
}
resolve(target: RawTargetConfig, context: ResolveContext): ResolvedUdpTarget {
const t = target as RawTargetConfig & { type: "udp"; udp: UdpTargetConfig };

21
src/server/checker/runner/udp/normalize.ts Normal file
View File

@@ -0,0 +1,21 @@
import { isPlainObject } from "es-toolkit";
import type { RawTargetConfig } from "../../types";
import { compactExpect, normalizeContent, normalizeValue } from "../../expect/normalize";
export function normalizeTargetExpect(target: RawTargetConfig): RawTargetConfig {
if (target.expect === undefined || !isPlainObject(target.expect)) return target;
const raw = target.expect as Record<string, unknown>;
return {
...target,
expect: compactExpect(raw, {
durationMs: normalizeValue(raw["durationMs"]),
responded: raw["responded"],
response: normalizeContent(raw["response"]),
responseSize: normalizeValue(raw["responseSize"]),
sourceHost: normalizeValue(raw["sourceHost"]),
sourcePort: normalizeValue(raw["sourcePort"]),
}),
};
}

5
src/server/checker/runner/ws/execute.ts
View File

@@ -8,6 +8,7 @@ import { errorFailure } from "../../expect/failure";
import { checkValueExpectation } from "../../expect/value";
import { parseSize } from "../../utils";
import { checkConnected, checkHandshakeHeaders, checkMessage } from "./expect";
import { normalizeTargetExpect } from "./normalize";
import { wsCheckerSchemas } from "./schema";
import { validateWsConfig } from "./validate";
@@ -281,6 +282,10 @@ export class WsChecker implements CheckerDefinition<ResolvedWsTarget> {
}
}
normalize(target: RawTargetConfig): RawTargetConfig {
return normalizeTargetExpect(target);
}
resolve(target: RawTargetConfig, context: ResolveContext): ResolvedWsTarget {
const t = target as RawTargetConfig & { type: "ws"; ws: WsTargetConfig };

20
src/server/checker/runner/ws/normalize.ts Normal file
View File

@@ -0,0 +1,20 @@
import { isPlainObject } from "es-toolkit";
import type { RawTargetConfig } from "../../types";
import { compactExpect, normalizeContent, normalizeKeyed, normalizeValue } from "../../expect/normalize";
export function normalizeTargetExpect(target: RawTargetConfig): RawTargetConfig {
if (target.expect === undefined || !isPlainObject(target.expect)) return target;
const raw = target.expect as Record<string, unknown>;
return {
...target,
expect: compactExpect(raw, {
connected: raw["connected"],
connectTimeMs: normalizeValue(raw["connectTimeMs"]),
durationMs: normalizeValue(raw["durationMs"]),
handshakeHeaders: normalizeKeyed(raw["handshakeHeaders"]),
message: normalizeContent(raw["message"]),
}),
};
}

112
tests/server/checker/config-contract/validate.test.ts
View File

@@ -1,6 +1,7 @@
import Ajv from "ajv";
import { describe, expect, test } from "bun:test";
import { normalizeAuthoringConfig } from "../../../../src/server/checker/normalizer";
import { createDefaultCheckerRegistry } from "../../../../src/server/checker/runner";
import { createProbeConfigJsonSchema } from "../../../../src/server/checker/schema/export";
import { formatConfigIssues, issue } from "../../../../src/server/checker/schema/issues";
@@ -274,4 +275,115 @@ describe("config contract", () => {
}),
).toBe(false);
});
test("所有 checker 的 authoring ValueMatcher 简写经 normalize 后通过 normalized contract 校验", () => {
const authoringShorthandExamples: Record<string, object> = {
cmd: {
targets: [
{
cmd: { exec: "echo hello" },
expect: { durationMs: 1000 },
id: "cmd-test",
type: "cmd",
},
],
},
db: {
targets: [
{
db: { url: "sqlite://:memory:" },
expect: { durationMs: 2000 },
id: "db-test",
type: "db",
},
],
},
dns: {
targets: [
{
dns: { name: "example.com", resolver: "system" },
expect: { durationMs: 500 },
id: "dns-test",
type: "dns",
},
],
},
http: {
targets: [
{
expect: { durationMs: 5000 },
http: { url: "https://example.com" },
id: "http-test",
type: "http",
},
],
},
icmp: {
targets: [
{
expect: { packetLossPercent: 0 },
icmp: { host: "example.com" },
id: "icmp-test",
type: "icmp",
},
],
},
llm: {
targets: [
{
expect: { durationMs: 10000 },
id: "llm-test",
llm: {
model: "gpt-4o-mini",
prompt: "ping",
provider: "openai",
url: "https://example.com/v1/chat/completions",
},
type: "llm",
},
],
},
tcp: {
targets: [
{
expect: { durationMs: 3000 },
id: "tcp-test",
tcp: { host: "example.com", port: 80 },
type: "tcp",
},
],
},
udp: {
targets: [
{
expect: { durationMs: 1000 },
id: "udp-test",
type: "udp",
udp: { host: "example.com", port: 53 },
},
],
},
ws: {
targets: [
{
expect: { durationMs: 5000 },
id: "ws-test",
type: "ws",
ws: { url: "wss://example.com/ws" },
},
],
},
};
for (const [type, config] of Object.entries(authoringShorthandExamples)) {
const normalizeResult = normalizeAuthoringConfig(config, createDefaultCheckerRegistry());
expect(normalizeResult.issues).toHaveLength(0);
const contract = validateProbeConfigContract(normalizeResult.config, createDefaultCheckerRegistry());
expect(contract.config).not.toBeNull();
expect(
contract.issues,
`Checker "${type}" authoring shorthand should pass normalized contract, got issues: ${JSON.stringify(contract.issues.map((i) => `${i.path}: ${i.message}`))}`,
).toHaveLength(0);
}
});
});

87
tests/server/checker/runner/dns/normalize.test.ts Normal file
View File

@@ -0,0 +1,87 @@
import { describe, expect, test } from "bun:test";
import { normalizeAuthoringConfig } from "../../../../../src/server/checker/normalizer";
import { checkerRegistry } from "../../../../../src/server/checker/runner";
import { validateProbeConfigContract } from "../../../../../src/server/checker/schema/validate";
describe("DNS normalize", () => {
test("ValueMatcher 简写被展开", () => {
const result = normalizeAuthoringConfig(
{
targets: [
{
dns: { name: "example.com", resolver: "server", server: "8.8.8.8" },
expect: { durationMs: 1000, valueCount: { gte: 1 } },
id: "dns-test",
type: "dns",
},
],
},
checkerRegistry,
);
expect(result.issues).toHaveLength(0);
const target = (result.config as { targets: Array<{ expect: Record<string, unknown> }> }).targets[0]!;
expect(target.expect["durationMs"]).toEqual({ equals: 1000 });
expect(target.expect["valueCount"]).toEqual({ gte: 1 });
});
test("ContentExpectations 简写被展开", () => {
const result = normalizeAuthoringConfig(
{
targets: [
{
dns: { name: "example.com", resolver: "server", server: "8.8.8.8" },
expect: { result: [{ contains: "NOERROR" }] },
id: "dns-test",
type: "dns",
},
],
},
checkerRegistry,
);
expect(result.issues).toHaveLength(0);
const target = (result.config as { targets: Array<{ expect: Record<string, unknown> }> }).targets[0]!;
const resultExpect = target.expect["result"] as Array<Record<string, unknown>>;
expect(resultExpect[0]!["kind"]).toBe("value");
expect((resultExpect[0]!["matcher"] as Record<string, unknown>)["contains"]).toBe("NOERROR");
});
test("DNS authoring 简写经 normalize 后通过 normalized contract 校验", () => {
const result = normalizeAuthoringConfig(
{
targets: [
{
dns: { name: "example.com", resolver: "server", server: "8.8.8.8" },
expect: { durationMs: 1000, result: [{ contains: "NOERROR" }], valueCount: { gte: 1 } },
id: "dns-test",
type: "dns",
},
],
},
checkerRegistry,
);
expect(result.issues).toHaveLength(0);
const contract = validateProbeConfigContract(result.config, checkerRegistry);
expect(contract.config).not.toBeNull();
expect(contract.issues).toHaveLength(0);
});
test("DNS system 模式 ValueMatcher 简写被展开", () => {
const result = normalizeAuthoringConfig(
{
targets: [
{
dns: { name: "example.com", resolver: "system" },
expect: { durationMs: 500, valueCount: { gte: 1 } },
id: "dns-system-test",
type: "dns",
},
],
},
checkerRegistry,
);
expect(result.issues).toHaveLength(0);
const target = (result.config as { targets: Array<{ expect: Record<string, unknown> }> }).targets[0]!;
expect(target.expect["durationMs"]).toEqual({ equals: 500 });
});
});

3
tests/server/checker/runner/registry.test.ts
View File

@@ -2,7 +2,7 @@ import { Type } from "@sinclair/typebox";
import { describe, expect, test } from "bun:test";
import type { Checker } from "../../../../src/server/checker/runner/types";
import type { CheckResult, ResolvedTargetBase } from "../../../../src/server/checker/types";
import type { CheckResult, RawTargetConfig, ResolvedTargetBase } from "../../../../src/server/checker/types";
import { createDefaultCheckerRegistry } from "../../../../src/server/checker/runner";
import { CheckerRegistry } from "../../../../src/server/checker/runner/registry";
@@ -12,6 +12,7 @@ function createChecker(type: string): Checker {
buildDetail: () => null,
configKey: type,
execute: () => Promise.resolve<CheckResult>({} as unknown as CheckResult),
normalize: (t: RawTargetConfig) => t,
resolve: () => ({}) as unknown as ResolvedTargetBase,
schemas: {
authoring: {