lanyuanxiaoyao/DiAL
1
0
Fork 0
Code Activity

refactor: 统一 expect 断言体系，引入共享 ValueMatcher/ContentRules/KeyValueExpect 模型

Browse Source 
- 引入共享 ValueMatcher（equals/contains/regex/exists/empty/gt/gte/lt/lte）
- 引入共享 ContentRules 数组（direct/json/css/xpath 提取器）
- 引入共享 KeyValueExpect（动态键值断言，字面量等价 equals）
- maxDurationMs → durationMs: ValueMatcher（所有 checker）
- match → regex（固定无 flags）
- Ping max* → packetLossPercent/avgLatencyMs/maxLatencyMs（ValueMatcher）
- LLM finishReason/rawFinishReason → ValueMatcher
- DB 新增 result: ContentRules
- TCP banner → ContentRules 数组
- 删除旧模块：operator.ts、validate-operator.ts、duration.ts、body.ts、text.ts、output.ts
- 更新全部 checker schema/validate/expect/execute
- 更新 probe-config.schema.json、probes.example.yaml
- 更新 README.md、DEVELOPMENT.md（含 expect 字段选择规范）
- 同步 10 个 delta specs 到主 specs，归档 change
This commit is contained in:
兰缘小妖
2026-05-19 14:24:27 +08:00
parent 349896bd02
commit 7a635a0a9f
85 changed files with 4290 additions and 2028 deletions
Download Patch File Download Diff File Expand all files Collapse all files

143
DEVELOPMENT.md
View File

@@ -39,7 +39,7 @@ src/
variables.ts 配置 variables 提取、target 字符串变量替换和 unresolved-variable issue 生成 variables.ts 配置 variables 提取、target 字符串变量替换和 unresolved-variable issue 生成
schema/ TypeBox + Ajv 配置契约、schema fragments、issue 渲染和 schema 导出入口 schema/ TypeBox + Ajv 配置契约、schema fragments、issue 渲染和 schema 导出入口
builder.ts 全量 JSON Schema 组装（遍历 registry 生成） builder.ts 全量 JSON Schema 组装（遍历 registry 生成）
fragments.ts 共享 TypeBox schema 片段（duration、size、operator 等） fragments.ts 共享 TypeBox schema 片段（duration、size、ValueMatcher、ContentRules、KeyValueExpect 等）
validate.ts Ajv 契约校验入口 validate.ts Ajv 契约校验入口
issues.ts 校验问题类型与渲染 issues.ts 校验问题类型与渲染
types.ts schema 层类型 types.ts schema 层类型
@@ -48,22 +48,24 @@ src/
engine.ts 调度引擎（按 interval 分组的 es-toolkit groupBy + Semaphore 并发控制 + 数据清理） engine.ts 调度引擎（按 interval 分组的 es-toolkit groupBy + Semaphore 并发控制 + 数据清理）
utils.ts 共享工具函数（parseSize、parseDuration） utils.ts 共享工具函数（parseSize、parseDuration）
expect/ 共享 expect 断言基础设施（跨 checker 复用） expect/ 共享 expect 断言基础设施（跨 checker 复用）
types.ts ExpectResult 共享断言类型 types.ts ExpectResult、ValueMatcher、ContentRules、KeyValueExpect 类型
failure.ts 失败信息构造（errorFailure、mismatchFailure、truncateActual） failure.ts 失败信息构造（errorFailure、mismatchFailure、truncateActual）
operator.ts 操作符系统（applyOperator、evaluateJsonPath） matcher.ts ValueMatcher 执行、JSONPath 提取、字面量 equals 快捷语义
duration.ts 耗时断言（checkDuration） content.ts ContentRules 执行（direct/json/css/xpath）
validate-operator.ts 操作符语义校验（validateOperatorObject、isJsonValue、isPlainRecord） key-value.ts KeyValueExpect 执行（动态键与 key 规范化）
validate-matcher.ts matcher/content/key-value 语义校验
redos.ts regex ReDoS 风险检测
runner/ Checker 统一抽象与注册机制 runner/ Checker 统一抽象与注册机制
types.ts CheckerDefinition、CheckerContext、CheckerSchemas、ResolveContext types.ts CheckerDefinition、CheckerContext、CheckerSchemas、ResolveContext
registry.ts CheckerRegistry 注册中心 registry.ts CheckerRegistry 注册中心
index.ts 注册入口（显式数组 + 循环注册） index.ts 注册入口（显式数组 + 循环注册）
http/ HTTP Checker（自包含模块，含 types/schema/execute/expect/validate/body） http/ HTTP Checker（自包含模块，含 types/schema/execute/expect/validate）
cmd/ Cmd Checker（自包含模块，含 types/schema/execute/expect/validate/text） cmd/ Cmd Checker（自包含模块，含 types/schema/execute/expect/validate）
db/ DB Checker（自包含模块，含 types/schema/execute/expect/validate） db/ DB Checker（自包含模块，含 types/schema/execute/expect/validate）
tcp/ TCP Checker（自包含模块，含 types/schema/execute/expect/validate） tcp/ TCP Checker（自包含模块，含 types/schema/execute/expect/validate）
icmp/ Ping Checker（自包含模块，含 types/schema/execute/expect/validate/parse） icmp/ Ping Checker（自包含模块，含 types/schema/execute/expect/validate/parse）
udp/ UDP Checker（自包含模块，含 types/schema/execute/expect/validate/encoding） udp/ UDP Checker（自包含模块，含 types/schema/execute/expect/validate/encoding）
llm/ LLM Checker（自包含模块，含 types/schema/execute/expect/validate/output/provider/observation） llm/ LLM Checker（自包含模块，含 types/schema/execute/expect/validate/provider/observation）
shared/ shared/
api.ts 前后端共享 TypeScript 类型 api.ts 前后端共享 TypeScript 类型
web/ React 前端 Dashboard（通过 Bun HTML import 集成） web/ React 前端 Dashboard（通过 Bun HTML import 集成）
@@ -271,7 +273,7 @@ checkerRegistry（单例）
| `validate.ts` | 启动期语义校验（JSON Schema 无法表达的规则） | | `validate.ts` | 启动期语义校验（JSON Schema 无法表达的规则） |
| `execute.ts` | Checker 类：resolve（默认值合并 + 解析）、execute（执行检查）、serialize（DB 持久化） | | `execute.ts` | Checker 类：resolve（默认值合并 + 解析）、execute（执行检查）、serialize（DB 持久化） |
| `expect.ts` | Checker 专用断言函数 | | `expect.ts` | Checker 专用断言函数 |
| `*.ts` | 其他 checker 专属逻辑（如 http/body.ts、cmd/text.ts） | | `*.ts` | 其他 checker 专属逻辑（如协议解析、编码、provider 适配、平台命令封装） |
#### 1.7.2 步骤一：创建 Checker 目录与类型 #### 1.7.2 步骤一：创建 Checker 目录与类型
@@ -291,15 +293,15 @@ checkerRegistry（单例）
**可复用的共享 fragments**（来自 `schema/fragments.ts`）： **可复用的共享 fragments**（来自 `schema/fragments.ts`）：
| Fragment | 用途 | | Fragment | 用途 |
| ---------------------------- | -------------------------------------------------------- | | ------------------------------ | -------------------------------------------------------- |
| `durationSchema` | 时长字符串（`"30s"`、`"5m"`、`"2h"`、`"7d"`、`"500ms"`） | | `durationSchema` | 时长字符串（`"30s"`、`"5m"`、`"2h"`、`"7d"`、`"500ms"`） |
| `sizeSchema` | 大小单位（字符串如 `"10MB"` 或整数） | | `sizeSchema` | 大小单位（字符串如 `"10MB"` 或整数） |
| `statusCodePatternSchema` | 状态码（`100`-`599` 或 `"2xx"`） | | `statusCodePatternSchema` | 状态码（`100`-`599` 或 `"2xx"`） |
| `stringMapSchema` | `Record<string, string>`（用于 headers / env） | | `stringMapSchema` | `Record<string, string>`（用于 headers / env） |
| `createBodyRulesSchema()` | body 规则数组（json/css/xpath/contains/regex） | | `createValueMatcherSchema()` | `ValueMatcher` 对象（equals/contains/regex/数值比较等） |
| `createTextRulesSchema()` | 文本规则数组（stdout/stderr） | | `createContentRulesSchema()` | `ContentRules` 数组（direct/json/css/xpath 内容规则） |
| `createPureOperatorSchema()` | 操作符对象 | | `createKeyValueExpectSchema()` | 动态键 `KeyValueExpect`（headers、DB rows 列值） |
| `operatorProperties()` | 所有操作符字段的 Record | | `matcherProperties()` | matcher 字段 Record，供 extractor schema 复用 |
**注意**：默认对象策略为 `additionalProperties: false`。只有明确的动态键值表（如 `http.headers`、`cmd.env`）可以开放任意键名。 **注意**：默认对象策略为 `additionalProperties: false`。只有明确的动态键值表（如 `http.headers`、`cmd.env`）可以开放任意键名。
@@ -311,12 +313,15 @@ checkerRegistry（单例）
export function validateTcpConfig(input: CheckerValidationInput): ConfigValidationIssue[]; export function validateTcpConfig(input: CheckerValidationInput): ConfigValidationIssue[];
``` ```
**共享校验工具**（`expect/validate-operator.ts`）： **共享校验工具**（`expect/validate-matcher.ts`）：
| 函数 | 用途 | | 函数 | 用途 |
| --------------------------------------------------------- | ---------------------- | | ------------------------------------------------------ | ----------------------------------------- |
| `validateOperatorObject(ops, path, targetName, options?)` | 校验操作符对象 | | `validateValueMatcher(value, path, targetName, opts?)` | 校验 matcher 字段、类型、regex 和组合语义 |
| `isJsonValue(value)` | 判断是否为合法 JSON 值 | | `validateContentRules(rules, path, targetName)` | 校验 ContentRules 数组、extractor 互斥性 |
| `validateKeyValueExpect(value, path, targetName)` | 校验动态键值断言 |
| `validateJsonPath(path, rulePath, targetName)` | 校验项目支持的 JSONPath 子集 |
| `isJsonValue(value)` | 判断是否为合法 JSON value |
#### 1.7.5 步骤四：实现 Checker 类 #### 1.7.5 步骤四：实现 Checker 类
@@ -352,14 +357,17 @@ TcpChecker implements Checker
**可用的共享断言工具**（`checker/expect/`）： **可用的共享断言工具**（`checker/expect/`）：
| 模块 | 函数 | 用途 | | 模块 | 函数 | 用途 |
| ---------------------- | ----------------------------------------------------- | ------------------------------------- | | --------------------- | ----------------------------------------------------- | ------------------------------------- |
| `failure.ts` | `errorFailure(phase, path, msg)` | 构造错误类型 failure | | `failure.ts` | `errorFailure(phase, path, msg)` | 构造错误类型 failure |
| `failure.ts` | `mismatchFailure(phase, path, expected, actual, msg)` | 构造不匹配类型 failure | | `failure.ts` | `mismatchFailure(phase, path, expected, actual, msg)` | 构造不匹配类型 failure |
| `failure.ts` | `truncateActual(value, maxLen?)` | 截断过长的 actual 值（默认 200 字符） | | `failure.ts` | `truncateActual(value, maxLen?)` | 截断过长的 actual 值（默认 200 字符） |
| `duration.ts` | `checkDuration(ms, maxMs?)` | 耗时断言 | | `matcher.ts` | `applyMatcher(actual, matcher, options?)` | 执行 ValueMatcher AND 匹配 |
| `operator.ts` | `applyOperator(actual, operator)` | 执行单个操作符比较 | | `matcher.ts` | `checkValueMatcher(actual, matcher, options)` | 执行 matcher 并返回 `ExpectResult` |
| `operator.ts` | `evaluateJsonPath(json, path)` | JSONPath 提取 | | `matcher.ts` | `checkExpectValue(actual, expected)` | 执行字面量 equals 或 matcher |
| `validate-operator.ts` | `validateOperatorObject(ops, path, name)` | 操作符语义校验 | | `matcher.ts` | `evaluateJsonPath(json, path)` | JSONPath 提取 |
| `content.ts` | `checkContentRules(source, rules, options)` | 执行 ContentRules 数组 |
| `key-value.ts` | `checkKeyValueExpect(actual, expected, options)` | 执行动态键值断言 |
| `validate-matcher.ts` | `validateValueMatcher/ContentRules/KeyValueExpect` | matcher/content/key-value 语义校验 |
**Checker 专属断言**（如需要）放在同目录的 `expect.ts` 中，参考 `http/expect.ts`（checkStatus、checkHeaders）和 `cmd/expect.ts`（checkExitCode）。 **Checker 专属断言**（如需要）放在同目录的 `expect.ts` 中，参考 `http/expect.ts`（checkStatus、checkHeaders）和 `cmd/expect.ts`（checkExitCode）。
@@ -486,39 +494,77 @@ TcpChecker implements Checker
### 1.10 expect 断言系统 ### 1.10 expect 断言系统
两层模型：**观测值收集** → **规则校验**。共享断言基础设施位于 `checker/expect/`，checker 专属断言位于各自目录。 两层模型：**观测值收集** → **规则校验**。共享断言基础设施位于 `checker/expect/`，checker 专属状态断言位于各自目录。
**HTTP 校验流程**： **共享模型**：
| 模型 | 用途 | 典型字段 |
| ---------------- | ---------------------------------------------- | -------------------------------------------------------------------- |
| `ValueMatcher` | 单个值、数字指标和字符串元数据断言 | `durationMs`、`rowCount`、`usage.totalTokens`、`finishReason` |
| `ContentRules` | 返回内容或半结构化内容断言，必须是数组 | `body`、`stdout`、`stderr`、`banner`、`response`、`output`、`result` |
| `KeyValueExpect` | 动态键值断言，字面量等价于 `{ equals: value }` | `headers`、DB `rows[]` 中的列值 |
`ValueMatcher` 支持 `equals`、`contains`、`regex`、`empty`、`exists`、`gte`、`lte`、`gt`、`lt`。一个 matcher 对象内多个字段为 AND 语义；`exists: false` 不能和其他 matcher 组合；`equals` 使用 `es-toolkit/isEqual` 做 JSON 深度相等；`regex` 固定为无 flags 的 `new RegExp(pattern).test(String(actual))`。
`ContentRules` 数组按顺序快速失败。数组项可以是直接 matcher，也可以是 `{ json: {...} }`、`{ css: {...} }`、`{ xpath: {...} }` 提取器规则；一条规则不能混用直接 matcher 和 extractor，多个 extractor 也不能共存。Extractor 未配置 matcher 时等价于 `exists: true`。对对象或数组源执行直接 `contains`/`regex` 时会先 JSON 序列化，`equals` 仍对原始结构做深度相等。
启动期语义校验统一由 `expect/validate-matcher.ts` 负责，会校验空 matcher、未知字段、字段类型、`exists:false` 组合、ContentRules 互斥性、JSONPath 子集、XPath 可编译性、regex 可编译性和 ReDoS 风险。旧字段 `match`、`maxDurationMs`、Ping 的 `max*` 阈值字段不再支持。
**快速失败顺序**：
| Checker | 顺序 |
| ---------- | -------------------------------------------------------------------------------------------------------------------------- |
| HTTP | `status → headers → body → durationMs` |
| Cmd | `exitCode → durationMs → stdout → stderr` |
| DB | `durationMs → rowCount → rows → result` |
| TCP | `connected → banner → durationMs` |
| UDP | `responded → responseSize → response → sourceHost → sourcePort → durationMs` |
| Ping | `alive → packetLossPercent → avgLatencyMs → maxLatencyMs → durationMs` |
| LLM http | `status → headers → output → finishReason → rawFinishReason → usage → durationMs` |
| LLM stream | `status → headers → stream.completed → stream.firstTokenMs → output → finishReason → rawFinishReason → usage → durationMs` |
HTTP checker 的 `durationMs` 覆盖完整执行（含重定向、响应体读取、解码和 expect 校验）。status 或 headers 失败时不读取 body；有 body 规则时，在读取 body 前可先检查是否已超过 `durationMs` 阈值，避免无意义读取。
**expect 字段选择规范**：
新增或修改 checker 的 expect 字段时，按以下决策树选择合适的断言模型：
``` ```
HttpChecker.execute → 收集观测(statusCode/headers) expect 字段
→ status → headers → (early duration) → body(按需) → (final duration) │
→ 首个失败即停止，返回 CheckFailure ├─ 状态类结果，结果集合小且稳定
│ └─ enum / boolean
│ HTTP/LLM status、Cmd exitCode、TCP connected、
│ UDP responded、Ping alive
│
├─ 数字指标 / 字符串元数据
│ └─ ValueMatcher
│ durationMs、rowCount、responseSize、sourceHost、sourcePort、
│ packetLossPercent、avgLatencyMs、maxLatencyMs、
│ finishReason、rawFinishReason、usage.*、stream.firstTokenMs
│
└─ 返回内容 / 半结构化内容 / 不完全确定的值
├─ 内容断言 → ContentRules（数组）
│ HTTP body、Cmd stdout/stderr、TCP banner、
│ UDP response、LLM output、DB result
│
└─ 键值断言 → KeyValueExpect（动态键对象）
HTTP/LLM headers、DB rows[] 中的列值
``` ```
HTTP checker 的 `durationMs` 覆盖完整执行（含重定向、响应体读取、解码和 expect 校验）。status 或 headers 失败时不读取 body；进入 body 前若已超过 `maxDurationMs`，直接返回 duration failure。 选择原则：
**Cmd 校验流程**： 1. **状态类字段使用 enum 或 boolean**。结果集合小且稳定时（如 HTTP status 200/2xx、exitCode 0），枚举和布尔比 matcher 更贴近协议语义，配置也更直观。不要为了统一而把状态类字段改成 ValueMatcher。
``` 2. **单值数字指标和字符串元数据使用 ValueMatcher**。观测值是一个明确的标量（耗时、行数、丢包率、finish reason），但阈值不确定时，使用 `{ lte: 100 }` 或 `{ regex: "^(stop|end)$" }` 等 matcher 表达。
CommandChecker.execute → 收集观测(exitCode/stdout/stderr/durationMs)
→ exitCode → duration → stdout → stderr
→ 首个失败即停止
```
**Body 规则类型**（`runner/http/body.ts`）： 3. **返回内容使用 ContentRules 数组**。观测值是文本、JSON、HTML 或 XML 内容，且可能需要多步提取或多条规则时，使用 ContentRules。即使只有一条规则也必须写成数组形式（`[{ contains: "ok" }]`），不支持对象快捷写法。
- `contains`：文本包含匹配 4. **键值对使用 KeyValueExpect**。观测值是动态键值表（如 headers），且需要对每个键独立断言时使用。字面量值自动等价于 `{ equals: value }`。
- `regex`：正则表达式匹配（注意：body 正则字段为 `regex`，不是 `match`，启动期会拒绝嵌套量词等 ReDoS 风险模式）
- `json`：JSONPath 提取 + 操作符比较（使用 `es-toolkit/isPlainObject` 区分纯值和操作符）
- `css`：cheerio CSS 选择器 + 操作符比较
- `xpath`：XPath 节点提取 + 操作符比较
**文本规则**（`runner/cmd/text.ts`）：stdout/stderr 文本匹配，支持 `contains`、`match`（正则）、操作符比较 5. **不要混用模型**。一个 expect 字段只能对应一种断言模型。例如 `finishReason` 是单值字符串元数据，用 ValueMatcher 而非 ContentRules（ContentRules 的 json/css/xpath 提取器对单字符串无意义，且会增加数组包装的配置冗余）。
**操作符**（`expect/operator.ts`）：`equals`（深度比较，`es-toolkit/isEqual`）、`contains`、`match`（正则，启动期通过 `expect/redos.ts` 拒绝 ReDoS 风险模式）、`empty`（`isNil`+`isEmptyObject`）、`exists`、`gte`/`lte`/`gt`/`lt` 6. **failure phase 命名遵循去单位后缀规则**。数字指标字段的 phase 去掉单位后缀（`durationMs` → `duration`、`packetLossPercent` → `packetLoss`、`avgLatencyMs` → `avgLatency`），不带单位后缀的字段直接使用字段名（`rowCount` → `rowCount`、`finishReason` → `finishReason`）。
启动期语义校验会对 HTTP body `regex` 规则和所有 `match` operator 执行静态 ReDoS 检测，常见的嵌套量词模式如 `(a+)+`、`(\\d+)*x` 会被拒绝，避免运行期正则在大响应体上阻塞事件循环。
### 1.11 错误模式 ### 1.11 错误模式
@@ -529,12 +575,7 @@ CommandChecker.execute → 收集观测(exitCode/stdout/stderr/durationMs)
### 1.12 测试规范 ### 1.12 测试规范
- 测试目录 `tests/` 镜像 `src/` 目录结构，但共享模块的测试集中放在 `tests/server/checker/runner/shared/` 下 - 测试目录 `tests/` 镜像 `src/` 目录结构，但共享 expect 模块的测试集中放在 `tests/server/checker/runner/shared/` 下，覆盖 `failure.ts`、`matcher.ts`、`content.ts`、`key-value.ts`、`validate-matcher.ts` 和 `redos.ts`
- `tests/server/checker/runner/shared/failure.test.ts` ↔ `src/server/checker/expect/failure.ts`
- `tests/server/checker/runner/shared/duration.test.ts` ↔ `src/server/checker/expect/duration.ts`
- `tests/server/checker/runner/shared/operator.test.ts` ↔ `src/server/checker/expect/operator.ts`
- `tests/server/checker/runner/shared/body.test.ts` ↔ `src/server/checker/runner/http/body.ts`
- `tests/server/checker/runner/shared/text.test.ts` ↔ `src/server/checker/runner/cmd/text.ts`
- 使用 `bun:test` 框架（`describe`/`test`/`expect`），测试数据库用临时目录 + `tmpdir()` - 使用 `bun:test` 框架（`describe`/`test`/`expect`），测试数据库用临时目录 + `tmpdir()`
- 新增 store 方法必须编写单元测试；新增 API 端点必须在 `app.test.ts` 中添加集成测试 - 新增 store 方法必须编写单元测试；新增 API 端点必须在 `app.test.ts` 中添加集成测试
- 测试后清理：`afterAll` 中 `store.close()` + `rm(tempDir, { recursive: true })` - 测试后清理：`afterAll` 中 `store.close()` + `rm(tempDir, { recursive: true })`

72
README.md
View File

@@ -93,7 +93,8 @@ targets:
url: "https://www.baidu.com" url: "https://www.baidu.com"
expect: expect:
status: [200] status: [200]
maxDurationMs: 10000 durationMs:
lte: 10000
- id: "json-api" - id: "json-api"
name: "${env_name} JSON API 示例" name: "${env_name} JSON API 示例"
@@ -130,7 +131,8 @@ targets:
url: "${sqlite_url}" url: "${sqlite_url}"
query: "SELECT COUNT(*) as cnt FROM users WHERE status = 'active'" query: "SELECT COUNT(*) as cnt FROM users WHERE status = 'active'"
expect: expect:
maxDurationMs: 5000 durationMs:
lte: 5000
rowCount: { gte: 1 } rowCount: { gte: 1 }
rows: rows:
- cnt: { gte: 0 } - cnt: { gte: 0 }
@@ -142,7 +144,8 @@ targets:
host: "127.0.0.1" host: "127.0.0.1"
port: 6379 port: 6379
expect: expect:
maxDurationMs: 3000 durationMs:
lte: 3000
- id: "udp-heartbeat" - id: "udp-heartbeat"
name: "UDP 心跳检测" name: "UDP 心跳检测"
@@ -154,7 +157,8 @@ targets:
expect: expect:
response: response:
- contains: "PONG" - contains: "PONG"
maxDurationMs: 100 durationMs:
lte: 100
- id: "gateway-ping" - id: "gateway-ping"
name: "网关 ICMP 可达" name: "网关 ICMP 可达"
@@ -165,10 +169,14 @@ targets:
packetSize: 56 packetSize: 56
expect: expect:
alive: true alive: true
maxPacketLoss: 10 packetLossPercent:
maxAvgLatencyMs: 100 lte: 10
maxMaxLatencyMs: 300 avgLatencyMs:
maxDurationMs: 5000 lte: 100
maxLatencyMs:
lte: 300
durationMs:
lte: 5000
``` ```
### 配置说明 ### 配置说明
@@ -293,28 +301,34 @@ Ping checker 通过系统 `ping` 命令执行 ICMP 探测，支持 Linux、macOS
#### expect — 期望校验 #### expect — 期望校验
| 字段 | 适用类型 | 说明 | | 字段 | 适用类型 | 说明 |
| ------------------- | -------- | ---------------------------------------------------------------- | | ------------------- | -------- | ---------------------------------------------------------------------- |
| `status` | HTTP | 可接受的状态码列表，支持精确码和范围（如 `"2xx"`）；默认 `[200]` | | `status` | HTTP/LLM | 可接受的状态码列表，支持精确码和范围（如 `"2xx"`）；默认 `[200]` |
| `exitCode` | Cmd | 可接受的退出码列表；未指定时不校验 | | `exitCode` | Cmd | 可接受的退出码列表；未指定时默认 `[0]` |
| `headers` | HTTP | 响应头校验 | | `headers` | HTTP/LLM | 响应头校验，使用动态键名和 `KeyValueExpect` |
| `maxDurationMs` | 全部 | 最大耗时阈值（毫秒） | | `durationMs` | 全部 | 完整执行耗时校验，使用 `ValueMatcher`，如 `{ lte: 1000 }` |
| `output` | LLM | 模型输出校验（数组：`equals`/`contains`/`regex`/`json`） | | `output` | LLM | 模型输出校验，使用 `ContentRules` 数组 |
| `finishReason` | LLM | 期望的 finish reason 字符串 | | `finishReason` | LLM | finish reason 校验，使用 `ValueMatcher` |
| `rawFinishReason` | LLM | 期望的原始 finish reason 字符串 | | `rawFinishReason` | LLM | 原始 finish reason 校验，使用 `ValueMatcher` |
| `usage` | LLM | Token usage 校验（`inputTokens`/`outputTokens`/`totalTokens`） | | `usage` | LLM | Token usage 校验（`inputTokens`/`outputTokens`/`totalTokens` matcher） |
| `stream` | LLM | 流式断言（`completed`、`firstTokenMs`，仅 `mode: stream`） | | `stream` | LLM | 流式断言（`completed`、`firstTokenMs` matcher，仅 `mode: stream`） |
| `body` | HTTP | 响应体校验（数组，可组合使用，见下方） | | `body` | HTTP | 响应体校验，使用 `ContentRules` 数组 |
| `stdout` / `stderr` | Cmd | 输出校验（数组，每项一个操作符对象） | | `stdout` / `stderr` | Cmd | 输出校验，使用 `ContentRules` 数组 |
| `rowCount` | DB | 查询返回行数校验（操作符对象） | | `rowCount` | DB | 查询返回行数校验，使用 `ValueMatcher` |
| `rows` | DB | 查询结果逐行校验（数组，列名→操作符映射） | | `rows` | DB | 查询结果逐行校验，数组内每行为列名到 `KeyValueExpect` 的映射 |
| `result` | DB | 完整查询结果 `{ rows, rowCount }` 校验，使用 `ContentRules` 数组 |
| `connected` | TCP | 期望连接结果，`true`（默认）可达或 `false` 期望不可达 | | `connected` | TCP | 期望连接结果，`true`（默认）可达或 `false` 期望不可达 |
| `banner` | TCP | Banner 文本校验（操作符对象，需开启 `tcp.readBanner`） | | `banner` | TCP | Banner 内容校验，使用 `ContentRules` 数组，需开启 `tcp.readBanner` |
| `responded` | UDP | 期望是否收到响应，默认 `true` |
| `response` | UDP | 响应内容校验，使用 `ContentRules` 数组 |
| `responseSize` | UDP | 响应字节数校验，使用 `ValueMatcher` |
| `sourceHost` | UDP | 响应来源地址校验，使用 `ValueMatcher` |
| `sourcePort` | UDP | 响应来源端口校验，使用 `ValueMatcher` |
| `alive` | Ping | 期望主机可达性，默认 `true` | | `alive` | Ping | 期望主机可达性，默认 `true` |
| `maxPacketLoss` | Ping | 最大丢包率百分比，范围 `0-100` | | `packetLossPercent` | Ping | 丢包率百分比校验，范围 `0-100`，使用 `ValueMatcher` |
| `maxAvgLatencyMs` | Ping | 最大平均延迟（毫秒） | | `avgLatencyMs` | Ping | 平均延迟校验，使用 `ValueMatcher` |
| `maxMaxLatencyMs` | Ping | 最大单次延迟（毫秒） | | `maxLatencyMs` | Ping | 最大单次延迟校验，使用 `ValueMatcher` |
**body 校验项**（数组中可混合使用）： **ContentRules 校验项**（`body`、`stdout`、`stderr`、`banner`、`response`、`output`、`result` 均使用数组）：
- `contains` — 响应体包含指定文本 - `contains` — 响应体包含指定文本
- `regex` — 正则匹配（启动期会拒绝存在 ReDoS 风险的模式） - `regex` — 正则匹配（启动期会拒绝存在 ReDoS 风险的模式）
@@ -322,7 +336,9 @@ Ping checker 通过系统 `ping` 命令执行 ICMP 探测，支持 Linux、macOS
- `css` — CSS 选择器提取 HTML 元素（`selector` 必填，`attr` 可选提取属性） - `css` — CSS 选择器提取 HTML 元素（`selector` 必填，`attr` 可选提取属性）
- `xpath` — XPath 提取 XML/HTML 节点（`path` 必填，如 `/html/body/h1/text()`） - `xpath` — XPath 提取 XML/HTML 节点（`path` 必填，如 `/html/body/h1/text()`）
**比较操作符**：`equals`（默认）、`contains`、`match`（正则）、`empty`、`exists`、`gte`、`lte`、`gt`、`lt` **ValueMatcher 字段**：`equals`、`contains`、`regex`、`empty`、`exists`、`gte`、`lte`、`gt`、`lt`。`equals` 支持 JSON 深度相等；`regex` 固定使用无 flags 正则；提取器未配置 matcher 时等价于 `exists: true`。
旧字段 `maxDurationMs`、`maxPacketLoss`、`maxAvgLatencyMs`、`maxMaxLatencyMs` 和旧正则字段 `match` 已移除，请分别改用 `durationMs`、Ping matcher 字段和 `regex`。
**大小说明**：`maxBodyBytes` 和 `maxOutputBytes` 支持 `KB`、`MB`、`GB` 单位，也可直接使用数字。 **大小说明**：`maxBodyBytes` 和 `maxOutputBytes` 支持 `KB`、`MB`、`GB` 单位，也可直接使用数字。

42
openspec/specs/checker-runner-abstraction/spec.md
View File

@@ -145,35 +145,31 @@
- **THEN** store SHALL 对每个 target 调用对应 checker 的 `serialize()` 方法获取 `{ target, config }` - **THEN** store SHALL 对每个 target 调用对应 checker 的 `serialize()` 方法获取 `{ target, config }`
### Requirement: 共享 expect 断言函数 ### Requirement: 共享 expect 断言函数
系统 SHALL 在 `src/server/checker/expect/` 中提供可被多个 checker 复用的 expect 函数。checker 专用的 expect 函数 SHALL 保留在各自子包内。仅被单个 checker 使用的断言模块 SHALL 位于该 checker 目录内。 系统 SHALL 在 `src/server/checker/expect/` 中提供可被多个 checker 复用的 expect 基础设施。共享基础设施 SHALL 包含 matcher、content rules、key-value expect、failure 构造和 ReDoS 校验。checker 专用的状态类断言 SHALL 保留在对应 checker 目录，或在多个 checker 复用时移动到共享模块。仅被单个 checker 使用且不属于通用 matcher/content/key-value 模型的断言模块 SHALL 位于该 checker 目录内。
#### Scenario: 共享 duration 断言 #### Scenario: 共享 ValueMatcher 断言
- **WHEN** 任何 checker 需要校验执行耗时 - **WHEN** 任何 checker 需要对数字、字符串、布尔或 JSON value 执行 matcher 匹配
- **THEN** SHALL 调用 `expect/duration.ts` 中的 `checkDuration(durationMs, maxDurationMs?)`，返回统一的 `ExpectResult` - **THEN** SHALL 调用共享 matcher 工具执行 `equals`、`contains`、`regex`、`exists`、`empty`、`gt`、`gte`、`lt` 和 `lte` 语义
#### Scenario: 共享 operator 断言 #### Scenario: 共享 ContentRules 断言
- **WHEN** 任何 checker 需要对值执行 operator 匹配 - **WHEN** HTTP body、LLM output、Cmd stdout/stderr、UDP response 或 TCP banner 需要执行返回内容校验
- **THEN** SHALL 调用 `expect/operator.ts` 中的 `applyOperator(actual, op)` - **THEN** SHALL 调用共享 content rules 工具，而不是在 checker 目录内复制 contains/regex/json/css/xpath 逻辑
#### Scenario: 共享 KeyValueExpect 断言
- **WHEN** HTTP 或 LLM checker 需要校验响应 headers，或 DB checker 需要校验 rows 中的列值
- **THEN** SHALL 调用共享 key-value expect 工具，并按调用方规则决定 key 是否大小写敏感
#### Scenario: 共享 regex ReDoS 校验
- **WHEN** 任一 matcher 或 content rule 配置 `regex`
- **THEN** SHALL 调用共享 ReDoS 校验工具在启动期拒绝危险正则
#### Scenario: 共享 failure 构造 #### Scenario: 共享 failure 构造
- **WHEN** 任何 checker 需要构造 CheckFailure 对象 - **WHEN** 任何 checker 需要构造 CheckFailure 对象
- **THEN** SHALL 调用 `expect/failure.ts` 中的 `errorFailure()` 或 `mismatchFailure()` - **THEN** SHALL 调用 `expect/failure.ts` 中的 `errorFailure()` 或 `mismatchFailure()`，并保留 actual 截断策略
#### Scenario: HTTP body 断言位于 HTTP 目录 #### Scenario: HTTP 专用 status 断言
- **WHEN** HTTP checker 需要对响应体执行 contains/regex/json/css/xpath 规则校验 - **WHEN** HTTP 或 LLM checker 需要校验响应状态码
- **THEN** SHALL 调用 `runner/http/body.ts` 中的 `checkBodyExpect(body, rules)` - **THEN** SHALL 复用同一 status 断言函数，支持精确状态码和 `1xx` 到 `5xx` 范围模式
#### Scenario: Cmd text 断言位于 Cmd 目录
- **WHEN** Cmd checker 需要对 stdout/stderr 执行文本规则校验
- **THEN** SHALL 调用 `runner/cmd/text.ts` 中的 `checkTextRules(text, rules, phase)`
#### Scenario: HTTP 专用 expect
- **WHEN** HTTP checker 需要校验响应状态码和响应头
- **THEN** SHALL 调用 `runner/http/expect.ts` 中的 `checkStatus()` 和 `checkHeaders()`
#### Scenario: Cmd 专用 expect
- **WHEN** Cmd checker 需要校验退出码
- **THEN** SHALL 调用 `runner/cmd/expect.ts` 中的 `checkExitCode()`
### Requirement: 超时控制由引擎注入 signal ### Requirement: 超时控制由引擎注入 signal
Checker 实现的 `execute()` MUST 使用 `ctx.signal` 感知超时，不得自行创建 `AbortController` 或 `setTimeout` 用于超时控制。Cmd checker 和 ping checker 可在 signal abort 时 `proc.kill()` 以确保子进程被终止。 Checker 实现的 `execute()` MUST 使用 `ctx.signal` 感知超时，不得自行创建 `AbortController` 或 `setTimeout` 用于超时控制。Cmd checker 和 ping checker 可在 signal abort 时 `proc.kill()` 以确保子进程被终止。

54
openspec/specs/cmd-checker/spec.md
View File

@@ -55,7 +55,7 @@
- **THEN** 系统 MUST 停止收集输出并终止该检查，记录 `matched=false`，并在 failure 中写入输出超限信息 - **THEN** 系统 MUST 停止收集输出并终止该检查，记录 `matched=false`，并在 failure 中写入输出超限信息
### Requirement: cmd expect 校验 ### Requirement: cmd expect 校验
系统 SHALL 支持 cmd 专用 expect，包括 `exitCode`、`stdout` 和 `stderr`，并按 exitCode、duration、stdout、stderr 的阶段顺序快速失败。 系统 SHALL 支持 cmd 专用 expect，包括 `exitCode`、`durationMs`、`stdout` 和 `stderr`，并按 exitCode、durationMs、stdout、stderr 的阶段顺序快速失败。`exitCode` SHALL 保持有限整数数组语义，未配置时默认 `[0]`。`durationMs` SHALL 使用共享 `ValueMatcher` 校验完整命令执行耗时。`stdout` 和 `stderr` MUST 使用共享 `ContentRules` 数组，直接 matcher 作用于对应输出文本，`json` extractor SHALL 支持对 JSON CLI 输出执行 JSONPath 断言。
#### Scenario: 默认 exitCode 成功语义 #### Scenario: 默认 exitCode 成功语义
- **WHEN** cmd target 未显式配置 `expect.exitCode` - **WHEN** cmd target 未显式配置 `expect.exitCode`
@@ -67,75 +67,67 @@
#### Scenario: exitCode 不匹配快速失败 #### Scenario: exitCode 不匹配快速失败
- **WHEN** cmd target 配置 `expect.exitCode: [0]` 且实际 exit code 为 1 - **WHEN** cmd target 配置 `expect.exitCode: [0]` 且实际 exit code 为 1
- **THEN** 系统 SHALL 立即返回 `matched=false`，并在 failure 中写入 phase=`exitCode`、path=`expect.exitCode`、expected 和 actual - **THEN** 系统 SHALL 立即返回 `matched=false`，并在 failure 中写入 phase=`exitCode`、path=`exitCode`、expected 和 actual
#### Scenario: durationMs 校验
- **WHEN** cmd target 配置 `expect.durationMs: {lte: 1000}` 且实际执行耗时为 1500ms
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `duration`
#### Scenario: stdout 按配置顺序校验 #### Scenario: stdout 按配置顺序校验
- **WHEN** cmd target 配置 `expect.stdout` 为两个规则，第一条通过且第二条失败 - **WHEN** cmd target 配置 `expect.stdout` 为两个 ContentRules，第一条通过且第二条失败
- **THEN** 系统 SHALL 先执行第一条 stdout 规则，再执行第二条，并将 failure.path 指向失败的 `expect.stdout[1]` - **THEN** 系统 SHALL 先执行第一条 stdout 规则，再执行第二条，并将 failure.path 指向失败的 `stdout[1]`
#### Scenario: stderr 校验为空 #### Scenario: stderr 校验为空
- **WHEN** cmd target 配置 `expect.stderr: [{empty: true}]` 且实际 stderr 为空字符串 - **WHEN** cmd target 配置 `expect.stderr: [{empty: true}]` 且实际 stderr 为空字符串
- **THEN** 系统 SHALL 判定 stderr 阶段通过 - **THEN** 系统 SHALL 判定 stderr 阶段通过
#### Scenario: stdout JSON 输出校验
- **WHEN** cmd target 输出 stdout 为 `{"status":"ok"}` 且配置 `expect.stdout: [{json: {path: "$.status", equals: "ok"}}]`
- **THEN** 系统 SHALL 判定 stdout 阶段通过
#### Scenario: stdout 失败后不检查 stderr #### Scenario: stdout 失败后不检查 stderr
- **WHEN** cmd target 同时配置 stdout 和 stderr 规则，且 stdout 规则失败 - **WHEN** cmd target 同时配置 stdout 和 stderr 规则，且 stdout 规则失败
- **THEN** 系统 SHALL 快速失败并 MUST NOT 继续执行 stderr 规则 - **THEN** 系统 SHALL 快速失败并 MUST NOT 继续执行 stderr 规则
### Requirement: cmd checker 启动期配置校验 ### Requirement: cmd checker 启动期配置校验
系统 SHALL 在启动期对 cmd checker 的配置契约和语义执行严格校验。Cmd target 的 `cmd` 分组 SHALL 只允许 `exec`、`args`、`cwd`、`env`、`maxOutputBytes` 字段；Cmd expect SHALL 只允许 `exitCode`、`maxDurationMs`、`stdout`、`stderr` 字段。未知字段、非法类型和不可编译正则 MUST 导致启动期配置错误。`expect.exitCode` SHALL 保留原有有限整数数组语义，不限制到特定平台范围。 系统 SHALL 在启动期对 cmd checker 的配置契约和语义执行严格校验。Cmd target 的 `cmd` 分组 SHALL 只允许 `exec`、`args`、`cwd`、`env`、`maxOutputBytes` 字段。Cmd expect SHALL 只允许 `exitCode`、`durationMs`、`stdout`、`stderr` 字段。未知字段、非法类型、不可编译正则和 ReDoS 风险正则 MUST 导致启动期配置错误。`expect.exitCode` SHALL 保留原有有限整数数组语义，不限制到特定平台范围。
#### Scenario: cmd args 类型非法 #### Scenario: cmd args 类型非法
- **WHEN** YAML 中 cmd target 配置 `cmd.args` 不是字符串数组 - **WHEN** YAML 中 cmd target 配置 `cmd.args` 不是字符串数组
- **THEN** 系统 SHALL 以配置错误退出，提示 cmd.args 格式错误 - **THEN** 系统 SHALL 以配置错误退出，提示 cmd.args 格式错误
#### Scenario: cmd cwd 类型非法
- **WHEN** YAML 中 cmd target 配置 `cmd.cwd` 不是字符串
- **THEN** 系统 SHALL 以配置错误退出，提示 cmd.cwd 必须为字符串
#### Scenario: cmd env 值类型非法 #### Scenario: cmd env 值类型非法
- **WHEN** YAML 中 cmd target 配置 `cmd.env`，且任一环境变量值不是字符串 - **WHEN** YAML 中 cmd target 配置 `cmd.env`，且任一环境变量值不是字符串
- **THEN** 系统 SHALL 以配置错误退出，提示 cmd.env 对应变量值必须为字符串 - **THEN** 系统 SHALL 以配置错误退出，提示 cmd.env 对应变量值必须为字符串
#### Scenario: cmd maxOutputBytes 非法
- **WHEN** YAML 中 cmd target 或 defaults.cmd 配置的 `maxOutputBytes` 不是合法 size 值
- **THEN** 系统 SHALL 以配置错误退出，提示 maxOutputBytes 格式错误
#### Scenario: cmd 分组未知字段失败
- **WHEN** YAML 中 cmd target 的 `cmd` 分组包含 `shell: true` 等未知字段
- **THEN** 系统 SHALL 以配置错误退出，提示 cmd 分组包含未知字段
#### Scenario: cmd expect exitCode 类型非法 #### Scenario: cmd expect exitCode 类型非法
- **WHEN** YAML 中 cmd target 配置 `expect.exitCode` 不是整数数组 - **WHEN** YAML 中 cmd target 配置 `expect.exitCode` 不是整数数组
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.exitCode 必须为整数数组 - **THEN** 系统 SHALL 以配置错误退出，提示 expect.exitCode 必须为整数数组
#### Scenario: cmd expect exitCode 不限制平台范围 #### Scenario: cmd expect durationMs 非法
- **WHEN** YAML 中 cmd target 配置 `expect.exitCode` 为有限整数数组 - **WHEN** YAML 中 cmd target 配置 `expect.durationMs` 不是合法 `ValueMatcher`
- **THEN** 系统 SHALL 接受该数组，不额外限制为 0-255 等平台相关范围 - **THEN** 系统 SHALL 以配置错误退出，提示 expect.durationMs 格式错误
#### Scenario: cmd expect maxDurationMs 非法 #### Scenario: stdout 必须为 ContentRules 数组
- **WHEN** YAML 中 cmd target 配置 `expect.maxDurationMs` 不是非负有限数字
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.maxDurationMs 格式错误
#### Scenario: stdout 必须为规则数组
- **WHEN** YAML 中 cmd target 配置 `expect.stdout` 但其值不是数组 - **WHEN** YAML 中 cmd target 配置 `expect.stdout` 但其值不是数组
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.stdout 必须为数组 - **THEN** 系统 SHALL 以配置错误退出，提示 expect.stdout 必须为数组
#### Scenario: stderr 必须为规则数组 #### Scenario: stderr 必须为 ContentRules 数组
- **WHEN** YAML 中 cmd target 配置 `expect.stderr` 但其值不是数组 - **WHEN** YAML 中 cmd target 配置 `expect.stderr` 但其值不是数组
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.stderr 必须为数组 - **THEN** 系统 SHALL 以配置错误退出，提示 expect.stderr 必须为数组
#### Scenario: stdout text rule 空对象非法 #### Scenario: stdout text rule 空对象非法
- **WHEN** YAML 中 cmd target 配置 `expect.stdout: [{}]` - **WHEN** YAML 中 cmd target 配置 `expect.stdout: [{}]`
- **THEN** 系统 SHALL 以配置错误退出，提示 stdout 规则必须包含至少一个合法 operator - **THEN** 系统 SHALL 以配置错误退出，提示 stdout 规则必须包含至少一个合法 matcher 或 extractor
#### Scenario: stderr text rule 未知字段非法 #### Scenario: stderr text rule 未知字段非法
- **WHEN** YAML 中 cmd target 配置 `expect.stderr: [{foo: "bar"}]` - **WHEN** YAML 中 cmd target 配置 `expect.stderr: [{foo: "bar"}]`
- **THEN** 系统 SHALL 以配置错误退出，提示 stderr 规则包含未知 operator - **THEN** 系统 SHALL 以配置错误退出，提示 stderr 规则包含未知 matcher 或未知 extractor
#### Scenario: stdout match 正则非法 #### Scenario: stdout regex 正则非法
- **WHEN** YAML 中 cmd target 配置 `expect.stdout: [{match: "[invalid"}]` - **WHEN** YAML 中 cmd target 配置 `expect.stdout: [{regex: "[invalid"}]`
- **THEN** 系统 SHALL 在启动期配置校验失败，而不是延迟到运行期抛错 - **THEN** 系统 SHALL 在启动期配置校验失败，而不是延迟到运行期抛错
#### Scenario: cmd expect 未知字段失败 #### Scenario: cmd expect 未知字段失败
- **WHEN** YAML 中 cmd target 的 expect 包含 `status: [200]` 或其他非 cmd expect 字段 - **WHEN** YAML 中 cmd target 的 expect 包含 `status: [200]`、`maxDurationMs: 1000` 或其他非 cmd expect 字段
- **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段 - **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段

60
openspec/specs/db-checker/spec.md
View File

@@ -75,11 +75,11 @@
- **THEN** 系统 SHALL 立即关闭数据库连接 - **THEN** 系统 SHALL 立即关闭数据库连接
### Requirement: db expect 校验 ### Requirement: db expect 校验
系统 SHALL 支持 db 专用 expect，包括 `maxDurationMs`、`rowCount` 和 `rows`，按 duration、rowCount、rows 的阶段顺序快速失败。 系统 SHALL 支持 db 专用 expect，包括 `durationMs`、`rowCount`、`rows` 和 `result`，按 durationMs、rowCount、rows、result 的阶段顺序快速失败。`durationMs` 和 `rowCount` SHALL 使用共享 `ValueMatcher`。`rows` SHALL 保留按行索引匹配列值的语义，类型为 `Array<KeyValueExpect>`（外层数组按行索引，内层每个元素为一个 `KeyValueExpect` 表达该行的列值断言），每个行规则中列值字面量等价于 `{equals: <literal>}`。`result` MUST 使用共享 `ContentRules` 数组，对查询结果对象 `{ rows, rowCount }` 执行断言。
#### Scenario: maxDurationMs 校验 #### Scenario: durationMs 校验
- **WHEN** db target 配置 `expect.maxDurationMs: 3000` 且实际执行耗时 4000ms - **WHEN** db target 配置 `expect.durationMs: {lte: 3000}` 且实际执行耗时 4000ms
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `"duration"` - **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `duration`
#### Scenario: rowCount 校验通过 #### Scenario: rowCount 校验通过
- **WHEN** db target 配置 `expect.rowCount: { gte: 1 }` 且查询返回 5 行 - **WHEN** db target 配置 `expect.rowCount: { gte: 1 }` 且查询返回 5 行
@@ -87,13 +87,13 @@
#### Scenario: rowCount 校验失败 #### Scenario: rowCount 校验失败
- **WHEN** db target 配置 `expect.rowCount: { gte: 1 }` 且查询返回 0 行 - **WHEN** db target 配置 `expect.rowCount: { gte: 1 }` 且查询返回 0 行
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `"rowCount"`，path 为 `"rowCount"`，expected 为 `{ gte: 1 }`，actual 为 0 - **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `rowCount`，path 为 `rowCount`，expected 为 `{ gte: 1 }`，actual 为 0
#### Scenario: rows 按索引匹配列值（operator 形式） #### Scenario: rows 按索引匹配列值 matcher 形式
- **WHEN** db target 配置 `expect.rows: [{ cnt: { gte: 100 } }]` 且查询首行 cnt 列值为 50 - **WHEN** db target 配置 `expect.rows: [{ cnt: { gte: 100 } }]` 且查询首行 cnt 列值为 50
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `"row"`，path 为 `"rows[0].cnt"` - **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `row`，path 为 `rows[0].cnt`
#### Scenario: rows 按索引匹配列值（字面量形式） #### Scenario: rows 按索引匹配列值字面量形式
- **WHEN** db target 配置 `expect.rows: [{ status: "active" }]` 且查询首行 status 列值为 `"active"` - **WHEN** db target 配置 `expect.rows: [{ status: "active" }]` 且查询首行 status 列值为 `"active"`
- **THEN** 系统 SHALL 判定该行该列通过（字面量等价于 `{ equals: "active" }`） - **THEN** 系统 SHALL 判定该行该列通过（字面量等价于 `{ equals: "active" }`）
@@ -103,25 +103,33 @@
#### Scenario: rows 结果行数不足 #### Scenario: rows 结果行数不足
- **WHEN** db target 配置 `expect.rows` 包含 3 个元素但查询仅返回 2 行 - **WHEN** db target 配置 `expect.rows` 包含 3 个元素但查询仅返回 2 行
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `"row"`，message 说明结果行数不足 - **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `row`，message 说明结果行数不足
#### Scenario: 无 query 时 expect 被忽略 #### Scenario: result JSONPath 校验
- **WHEN** db target 未配置 `db.query` 但配置了 `expect.rowCount` - **WHEN** db target 查询返回首行 `{status: "active"}` 且配置 `expect.result: [{json: {path: "$.rows[0].status", equals: "active"}}]`
- **THEN** 系统 SHALL 忽略 expect 中的 rowCount 和 rows 断言（仅 maxDurationMs 生效） - **THEN** 系统 SHALL 基于 `{rows, rowCount}` 结果对象执行 JSONPath，并判定 result 阶段通过
#### Scenario: result rowCount 校验
- **WHEN** db target 查询返回 2 行且配置 `expect.result: [{json: {path: "$.rowCount", equals: 2}}]`
- **THEN** 系统 SHALL 判定 result 阶段通过
#### Scenario: 无 query 时结果类 expect 被忽略
- **WHEN** db target 未配置 `db.query` 但配置了 `expect.rowCount`、`expect.rows` 或 `expect.result`
- **THEN** 系统 SHALL 忽略这些查询结果断言（仅 `durationMs` 生效）
#### Scenario: 快速失败顺序 #### Scenario: 快速失败顺序
- **WHEN** db target 同时配置 maxDurationMs、rowCount 和 rows - **WHEN** db target 同时配置 durationMs、rowCount、rows 和 result
- **THEN** 系统 SHALL 按 duration → rowCount → rows 顺序校验，任一阶段失败立即返回 - **THEN** 系统 SHALL 按 durationMs → rowCount → rows → result 顺序校验，任一阶段失败立即返回
### Requirement: db checker 启动期配置校验 ### Requirement: db checker 启动期配置校验
系统 SHALL 在启动期对 db checker 的配置契约和语义执行严格校验。Db target 的 `db` 分组 SHALL 只允许 `url` 和 `query` 字段。Db expect SHALL 只允许 `maxDurationMs`、`rowCount` 和 `rows` 字段。 系统 SHALL 在启动期对 db checker 的配置契约和语义执行严格校验。Db target 的 `db` 分组 SHALL 只允许 `url` 和 `query` 字段。Db expect SHALL 只允许 `durationMs`、`rowCount`、`rows` 和 `result` 字段。未知字段、非法 matcher、非法 ContentRules、非法 regex 和 ReDoS 风险正则 MUST 导致启动期配置错误。
#### Scenario: db expect maxDurationMs 非法 #### Scenario: db expect durationMs 非法
- **WHEN** YAML 中 db target 配置 `expect.maxDurationMs` 不是非负有限数字 - **WHEN** YAML 中 db target 配置 `expect.durationMs` 不是合法 `ValueMatcher`
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.maxDurationMs 格式错误 - **THEN** 系统 SHALL 以配置错误退出，提示 expect.durationMs 格式错误
#### Scenario: db expect rowCount 非法 #### Scenario: db expect rowCount 非法
- **WHEN** YAML 中 db target 配置 `expect.rowCount` 不是合法的 operator 对象 - **WHEN** YAML 中 db target 配置 `expect.rowCount` 不是合法 `ValueMatcher`
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.rowCount 格式错误 - **THEN** 系统 SHALL 以配置错误退出，提示 expect.rowCount 格式错误
#### Scenario: db expect rows 非法 #### Scenario: db expect rows 非法
@@ -129,13 +137,17 @@
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.rows 必须为对象数组 - **THEN** 系统 SHALL 以配置错误退出，提示 expect.rows 必须为对象数组
#### Scenario: db expect rows 元素列值非法 #### Scenario: db expect rows 元素列值非法
- **WHEN** YAML 中 db target 配置 `expect.rows: [{ cnt: { foo: 1 } }]`，其中 foo 不是合法 operator - **WHEN** YAML 中 db target 配置 `expect.rows: [{ cnt: { foo: 1 } }]`，其中 foo 不是合法 matcher
- **THEN** 系统 SHALL 以配置错误退出，提示 rows 中包含未知 operator - **THEN** 系统 SHALL 以配置错误退出，提示 rows 中包含未知 matcher
#### Scenario: db expect result 非法
- **WHEN** YAML 中 db target 配置 `expect.result` 不是合法 ContentRules 数组
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.result 格式错误
#### Scenario: db expect 未知字段失败 #### Scenario: db expect 未知字段失败
- **WHEN** YAML 中 db target 的 expect 包含 `status: [200]` 或其他非 db expect 字段 - **WHEN** YAML 中 db target 的 expect 包含 `status: [200]`、`maxDurationMs: 1000` 或其他非 db expect 字段
- **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段 - **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段
#### Scenario: db expect rows 中 match 正则非法 #### Scenario: db expect rows 中 regex 正则非法
- **WHEN** YAML 中 db target 配置 `expect.rows: [{ name: { match: "[invalid" } }]` - **WHEN** YAML 中 db target 配置 `expect.rows: [{ name: { regex: "[invalid" } }]`
- **THEN** 系统 SHALL 在启动期配置校验失败，而不是延迟到运行期抛错 - **THEN** 系统 SHALL 在启动期配置校验失败，而不是延迟到运行期抛错

154
openspec/specs/expect-body-checkers/spec.md
View File

@@ -5,35 +5,23 @@
## Requirements ## Requirements
### Requirement: 响应体多种校验方法 ### Requirement: 响应体多种校验方法
系统 SHALL 支持对 HTTP 响应体进行五种可组合的校验方法：contains（子串）、regex（正则）、json（JSONPath）、css（CSS 选择器）、xpath（XPath）。这些方法 MUST 配置在 `expect.body` 有序数组中。 系统 SHALL 支持通过共享 `ContentRules` 对 HTTP 响应体进行有序内容校验。`expect.body` MUST 为规则数组。每个规则 SHALL 为直接 `ValueMatcher`，或 `json`、`css`、`xpath` extractor 规则之一。直接 matcher SHALL 作用于完整响应体文本。`json` SHALL 解析响应体为 JSON 并用 JSONPath 子集提取值。`css` SHALL 使用 CSS selector 从 HTML 中提取元素文本或属性。`xpath` SHALL 使用 XPath 从 XML/HTML 中提取节点值。Extractor 未配置 matcher 时 SHALL 等价于 `exists: true`。
#### Scenario: contains 子串匹配 #### Scenario: contains 子串匹配
- **WHEN** HTTP target 配置 `expect.body: [{contains: "healthy"}]`，且响应体包含 `"healthy"` - **WHEN** HTTP target 配置 `expect.body: [{contains: "healthy"}]`，且响应体包含 `"healthy"`
- **THEN** 系统 SHALL 判定该 body 规则通过 - **THEN** 系统 SHALL 判定该 body 规则通过
#### Scenario: contains 不匹配
- **WHEN** HTTP target 配置 `expect.body: [{contains: "healthy"}]`，且响应体不包含该文本
- **THEN** 系统 SHALL 判定 matched 为 false，并记录该规则的 failure.path
#### Scenario: regex 正则匹配 #### Scenario: regex 正则匹配
- **WHEN** HTTP target 配置 `expect.body: [{regex: '"status"\\s*:\\s*"ok"'}]`，且响应体匹配该正则 - **WHEN** HTTP target 配置 `expect.body: [{regex: '"status"\\s*:\\s*"ok"'}]`，且响应体匹配该正则
- **THEN** 系统 SHALL 判定该 body 规则通过 - **THEN** 系统 SHALL 判定该 body 规则通过
#### Scenario: regex 不匹配
- **WHEN** HTTP target 配置 regex body 规则，且响应体不匹配该正则
- **THEN** 系统 SHALL 判定 matched 为 false，并记录该规则的 failure.path
#### Scenario: json JSONPath 等值匹配 #### Scenario: json JSONPath 等值匹配
- **WHEN** HTTP target 配置 `expect.body: [{json: {path: "$.status", equals: "ok"}}]`，且响应 JSON 中 `$.status` 值为 `"ok"` - **WHEN** HTTP target 配置 `expect.body: [{json: {path: "$.status", equals: "ok"}}]`，且响应 JSON 中 `$.status` 值为 `"ok"`
- **THEN** 系统 SHALL 判定该 body 规则通过 - **THEN** 系统 SHALL 判定该 body 规则通过
#### Scenario: json JSONPath 值不匹配 #### Scenario: json JSONPath 存在性匹配
- **WHEN** HTTP target 配置 json body 规则，且提取值不符合期望 - **WHEN** HTTP target 配置 `expect.body: [{json: {path: "$.status"}}]`，且响应 JSON 中存在 `$.status`
- **THEN** 系统 SHALL 判定 matched 为 false，并记录包含 JSONPath 的 failure.path - **THEN** 系统 SHALL 将该规则按 `exists: true` 语义判定通过
#### Scenario: json 解析失败
- **WHEN** HTTP target 配置了 json body 规则但响应体不是合法 JSON
- **THEN** 系统 SHALL 判定 matched 为 false
#### Scenario: css 选择器匹配 #### Scenario: css 选择器匹配
- **WHEN** HTTP target 配置 `expect.body: [{css: {selector: "div#health", equals: "OK"}}]`，且 HTML 中存在 `div#health` 元素文本为 `"OK"` - **WHEN** HTTP target 配置 `expect.body: [{css: {selector: "div#health", equals: "OK"}}]`，且 HTML 中存在 `div#health` 元素文本为 `"OK"`
@@ -43,20 +31,16 @@
- **WHEN** HTTP target 配置 css 规则带 `attr: "content"` 用于提取属性，且属性值匹配期望 - **WHEN** HTTP target 配置 css 规则带 `attr: "content"` 用于提取属性，且属性值匹配期望
- **THEN** 系统 SHALL 判定该 body 规则通过 - **THEN** 系统 SHALL 判定该 body 规则通过
#### Scenario: css 选择器无匹配元素
- **WHEN** HTTP target 配置了 css 选择器但 HTML 中无匹配元素
- **THEN** 系统 SHALL 判定 matched 为 false
#### Scenario: xpath 表达式匹配 #### Scenario: xpath 表达式匹配
- **WHEN** HTTP target 配置 `expect.body: [{xpath: {path: "/root/status/text()", equals: "ok"}}]`，且 XML 中 `/root/status` 节点文本为 `"ok"` - **WHEN** HTTP target 配置 `expect.body: [{xpath: {path: "/root/status/text()", equals: "ok"}}]`，且 XML 中 `/root/status` 节点文本为 `"ok"`
- **THEN** 系统 SHALL 判定该 body 规则通过 - **THEN** 系统 SHALL 判定该 body 规则通过
#### Scenario: xpath 表达式无匹配节点 #### Scenario: 提取器无匹配目标失败
- **WHEN** HTTP target 配置了 xpath 表达式但 XML 中无匹配节点 - **WHEN** HTTP target 配置了 json、css 或 xpath 规则且对应路径、元素或节点不存在，并且规则未配置 `exists: false`
- **THEN** 系统 SHALL 判定 matched 为 false - **THEN** 系统 SHALL 判定 matched 为 false
### Requirement: 多种 body 校验方法 AND 组合 ### Requirement: 多种 body 校验方法 AND 组合
系统 SHALL 支持在 `expect.body` 数组中同时配置多种 body 校验方法，所有方法均通过时 matched 方为 true。 系统 SHALL 支持在 `expect.body` 数组中同时配置多条内容规则，所有规则均通过时 matched 方为 true。系统 SHALL 按数组顺序执行规则，任一规则失败后 MUST NOT 继续执行后续规则。
#### Scenario: 多种方法全部通过 #### Scenario: 多种方法全部通过
- **WHEN** HTTP target 的 `expect.body` 数组依次配置 contains、json、regex，且全部通过 - **WHEN** HTTP target 的 `expect.body` 数组依次配置 contains、json、regex，且全部通过
@@ -66,54 +50,50 @@
- **WHEN** HTTP target 的 `expect.body` 数组第一条 contains 不通过，后续还有 json 规则 - **WHEN** HTTP target 的 `expect.body` 数组第一条 contains 不通过，后续还有 json 规则
- **THEN** 系统 SHALL 判定 matched 为 false，且不再检查后续 json 规则 - **THEN** 系统 SHALL 判定 matched 为 false，且不再检查后续 json 规则
#### Scenario: 直接 matcher 多字段组合
- **WHEN** HTTP target 配置 `expect.body: [{contains: "ok", regex: "status"}]`，且响应体同时满足 contains 和 regex
- **THEN** 系统 SHALL 判定该规则通过
### Requirement: 操作符系统 ### Requirement: 操作符系统
系统 SHALL 支持对提取值和文本值使用以下操作符进行比较：equals（默认等值）、contains（子串包含）、match（正则匹配）、empty（空值判断）、exists（存在性判断）、gte/lte/gt/lt（数值比较）。 系统 SHALL 支持通过共享 `ValueMatcher` 对提取值和文本值进行比较：`equals`（深度等值）、`contains`（子串包含）、`regex`（正则匹配）、`empty`（空值判断）、`exists`（存在性判断）、`gte`/`lte`/`gt`/`lt`（数值比较）。系统 MUST NOT 支持旧 `match` 字段。
#### Scenario: 标量值隐式 equals #### Scenario: equals 匹配 JSON value
- **WHEN** 配置的期望值为标量（字符串/数字/布尔/null），如 `equals: "ok"` - **WHEN** 配置 `{equals: {status: "ok"}}`，且实际值为相同 JSON object
- **THEN** 系统 SHALL 使用 equals 操作符，对实际值做严格相等比较 - **THEN** 系统 SHALL 使用深度相等判定通过
#### Scenario: 显式 contains 操作符 #### Scenario: 显式 contains matcher
- **WHEN** 配置 `{contains: "success"}`，且实际值包含 `"success"` - **WHEN** 配置 `{contains: "success"}`，且实际值字符串化后包含 `"success"`
- **THEN** 系统 SHALL 判定该规则通过 - **THEN** 系统 SHALL 判定该规则通过
#### Scenario: 显式 match 操作符 #### Scenario: 显式 regex matcher
- **WHEN** 配置 `{match: '\\d+\\.\\d+\\.\\d+'}`，且实际值匹配该正则 - **WHEN** 配置 `{regex: '\\d+\\.\\d+\\.\\d+'}`，且实际值字符串化后匹配该正则
- **THEN** 系统 SHALL 判定该规则通过 - **THEN** 系统 SHALL 判定该规则通过
#### Scenario: empty 操作符判断为空 #### Scenario: empty matcher 判断为空
- **WHEN** 配置 `{empty: true}`，且实际值为空数组 `[]` - **WHEN** 配置 `{empty: true}`，且实际值为空数组 `[]`
- **THEN** 系统 SHALL 判定该规则通过 - **THEN** 系统 SHALL 判定该规则通过
#### Scenario: empty 操作符判断非空 #### Scenario: exists matcher 判断不存在
- **WHEN** 配置 `{empty: false}`，且实际值为 `[1, 2]` - **WHEN** 配置 `{exists: false}`，且实际值为 `undefined`
- **THEN** 系统 SHALL 判定该规则通过 - **THEN** 系统 SHALL 判定该规则通过
#### Scenario: exists 操作符判断存在 #### Scenario: 数值比较 matcher
- **WHEN** 配置 `{exists: false}`，且实际值不存在
- **THEN** 系统 SHALL 判定该规则通过
#### Scenario: gte 数值比较
- **WHEN** 配置 `{gte: 10}`，且实际值为 `15`（数字）
- **THEN** 系统 SHALL 判定该规则通过
#### Scenario: gt/lt 数值比较
- **WHEN** 配置 `{gt: 0, lt: 1000}`，且实际值为 `500` - **WHEN** 配置 `{gt: 0, lt: 1000}`，且实际值为 `500`
- **THEN** 系统 SHALL 对同一字段进行多操作符复合比较，全部通过则该规则通过 - **THEN** 系统 SHALL 对同一字段进行多 matcher 复合比较，全部通过则该规则通过
### Requirement: 响应头校验 ### Requirement: 响应头校验
系统 SHALL 支持通过 `expect.headers` 配置对 HTTP 响应头进行键值规则校验，header 名称匹配 MUST 不区分大小写。 系统 SHALL 支持通过共享 `KeyValueExpect` 配置 `expect.headers` 对 HTTP 响应头进行键值规则校验，header 名称匹配 MUST 不区分大小写。header 期望值 MAY 为字符串字面量或 `ValueMatcher`。字符串字面量 SHALL 等价于 `{equals: <value>}`。
#### Scenario: 响应头匹配 #### Scenario: 响应头字面量匹配
- **WHEN** HTTP target 配置 `expect.headers: {"Content-Type": {contains: "application/json"}}`，且响应包含该 header 且值匹配 - **WHEN** HTTP target 配置 `expect.headers: {"Content-Type": "application/json"}`，且响应包含该 header 且值精确匹配
- **THEN** 系统 SHALL 判定 headers 阶段通过 - **THEN** 系统 SHALL 判定 headers 阶段通过
#### Scenario: 响应头不匹配 #### Scenario: 响应头 matcher 匹配
- **WHEN** HTTP target 配置 `expect.headers: {"Content-Type": {equals: "application/json"}}`，且响应 header 值为 `"text/html"` - **WHEN** HTTP target 配置 `expect.headers: {"Content-Type": {contains: "application/json"}}`，且响应 header 值包含该文本
- **THEN** 系统 SHALL 判定 matched 为 false - **THEN** 系统 SHALL 判定 headers 阶段通过
#### Scenario: 响应头缺失 #### Scenario: 响应头缺失
- **WHEN** HTTP target 配置了某个 header 但响应中不存在该 header - **WHEN** HTTP target 配置了某个 header 但响应中不存在该 header，且未配置 `exists: false`
- **THEN** 系统 SHALL 判定 matched 为 false - **THEN** 系统 SHALL 判定 matched 为 false
### Requirement: 结构化 expect 失败信息 ### Requirement: 结构化 expect 失败信息
@@ -175,79 +155,43 @@
- **THEN** 系统 SHALL 在启动期配置校验失败 - **THEN** 系统 SHALL 在启动期配置校验失败
### Requirement: HTTP expect 规则启动期校验 ### Requirement: HTTP expect 规则启动期校验
系统 SHALL 在启动期校验 HTTP expect 中已支持字段的类型、格式、未知字段和可编译表达式。HTTP expect、body rule、json/css/xpath rule 和 operator 对象中的未知字段 SHALL 导致启动期配置失败。每个 body rule 对象 MUST 恰好包含 contains、regex、json、css、xpath 中的一种规则类型。纯 operator 对象 MUST 至少包含一个已知 operator；body 提取规则可以不配置 operator，并以路径、元素或节点存在作为通过语义。`equals` operator SHALL 支持任意 JSON value，包括数组和对象。系统 SHALL 在启动期对 regex body 规则和 match operator 的正则表达式进行 ReDoS 安全检测，含有嵌套量词等危险模式的正则 SHALL 导致启动期配置失败。 系统 SHALL 在启动期校验 HTTP expect 中已支持字段的类型、格式、未知字段和可编译表达式。HTTP expect SHALL 只允许 `status`、`headers`、`body` 和 `durationMs` 字段。`expect.body` MUST 为 `ContentRules` 数组。直接 `ValueMatcher` 对象 MUST 至少包含一个合法 matcher。Extractor 规则 MUST 只包含 `json`、`css`、`xpath` 中的一种 extractor。Extractor 内部可以不配置 matcher，并 SHALL 在运行期以存在性作为通过语义。`equals` matcher SHALL 支持任意 JSON value，包括数组和对象。系统 SHALL 在启动期对所有 `regex` 执行静态 ReDoS 检测。
#### Scenario: body rule 使用 regex 字段 #### Scenario: body rule 使用 regex 字段
- **WHEN** HTTP target 配置 `expect.body: [{regex: "ok|healthy"}]` 且 regex 可编译且无 ReDoS 风险 - **WHEN** HTTP target 配置 `expect.body: [{regex: "ok|healthy"}]` 且 regex 可编译且无 ReDoS 风险
- **THEN** 系统 SHALL 接受该配置，并在运行期按 regex body 规则匹配响应体 - **THEN** 系统 SHALL 接受该配置，并在运行期按 regex 规则匹配响应体
#### Scenario: body rule 不支持 match 字段 #### Scenario: body rule 不支持 match 字段
- **WHEN** HTTP target 配置 `expect.body: [{match: "ok"}]` 且该规则没有 contains、regex、json、css、xpath 任一支持字段 - **WHEN** HTTP target 配置 `expect.body: [{match: "ok"}]`
- **THEN** 系统 SHALL 在启动期配置校验失败，提示 `match` 是未知字段或不支持字段
#### Scenario: body rule 多 extractor 非法
- **WHEN** HTTP target 的同一条 body rule 同时配置 `json` 和 `css`
- **THEN** 系统 SHALL 在启动期配置校验失败 - **THEN** 系统 SHALL 在启动期配置校验失败
#### Scenario: body rule 忽略未知字段 → body rule 未知字段启动失败 #### Scenario: matcher regex 正则非法
- **WHEN** HTTP target 配置 `expect.body: [{contains: "ok", note: "ignored"}]` - **WHEN** HTTP target 的 expect.headers、body 直接 matcher 或 extractor 内部 matcher 配置了不可编译的 regex
- **THEN** 系统 SHALL 在启动期配置校验失败，提示 `note` 是未知字段
#### Scenario: body rule 多支持字段非法
- **WHEN** HTTP target 的同一条 body rule 同时配置 contains 和 regex
- **THEN** 系统 SHALL 在启动期配置校验失败 - **THEN** 系统 SHALL 在启动期配置校验失败
#### Scenario: operator match 正则非法 #### Scenario: matcher 数值比较类型非法
- **WHEN** HTTP target 的 expect.headers、json、css 或 xpath operator 配置了不可编译的 match 正则 - **WHEN** HTTP target 的 matcher 配置 gt、gte、lt 或 lte，且对应值不是有限数字
- **THEN** 系统 SHALL 在启动期配置校验失败 - **THEN** 系统 SHALL 在启动期配置校验失败
#### Scenario: operator 数值比较类型非法 #### Scenario: matcher 布尔类型非法
- **WHEN** HTTP target 的 expect operator 配置 gt、gte、lt 或 lte，且对应值不是有限数字 - **WHEN** HTTP target 的 matcher 配置 empty 或 exists，且对应值不是布尔值
- **THEN** 系统 SHALL 在启动期配置校验失败
#### Scenario: operator 布尔类型非法
- **WHEN** HTTP target 的 expect operator 配置 empty 或 exists，且对应值不是布尔值
- **THEN** 系统 SHALL 在启动期配置校验失败 - **THEN** 系统 SHALL 在启动期配置校验失败
#### Scenario: JSONPath 子集非法 #### Scenario: JSONPath 子集非法
- **WHEN** HTTP target 的 json body rule path 不符合系统支持的 JSONPath 子集 - **WHEN** HTTP target 的 json body rule path 不符合系统支持的 JSONPath 子集
- **THEN** 系统 SHALL 在启动期配置校验失败 - **THEN** 系统 SHALL 在启动期配置校验失败
#### Scenario: operator 未知字段非法 #### Scenario: matcher 未知字段非法
- **WHEN** HTTP target 的 expect operator 配置了 `foo: "bar"` 等未知 operator 字段 - **WHEN** HTTP target 的 matcher 配置了 `foo: "bar"` 等未知字段
- **THEN** 系统 SHALL 在启动期配置校验失败 - **THEN** 系统 SHALL 在启动期配置校验失败
#### Scenario: equals 支持对象 #### Scenario: durationMs matcher 非法
- **WHEN** HTTP target 配置 `expect.body: [{json: {path: "$.payload", equals: {status: "ok"}}}]` - **WHEN** HTTP target 配置 `expect.durationMs` 不是合法 `ValueMatcher` 或其中数值 matcher 不是有限数字
- **THEN** 系统 SHALL 接受该配置，并在运行期使用深度相等比较提取值和对象期望 - **THEN** 系统 SHALL 在启动期配置校验失败
#### Scenario: equals 支持数组
- **WHEN** HTTP target 配置 `expect.body: [{json: {path: "$.items", equals: ["a", "b"]}}]`
- **THEN** 系统 SHALL 接受该配置，并在运行期使用深度相等比较提取值和数组期望
#### Scenario: 纯 operator 对象不能为空
- **WHEN** HTTP target 的 `expect.headers` 中某个 header 期望配置为空对象 `{}`
- **THEN** 系统 SHALL 在启动期配置校验失败，要求显式配置至少一个 operator
#### Scenario: json rule 允许存在性语义
- **WHEN** HTTP target 配置 `expect.body: [{json: {path: "$.status"}}]`
- **THEN** 系统 SHALL 接受该配置，并在运行期以 JSONPath 值存在作为通过语义
#### Scenario: css rule 未知字段非法
- **WHEN** HTTP target 配置 `expect.body: [{css: {selector: "h1", unknown: true}}]`
- **THEN** 系统 SHALL 在启动期配置校验失败，提示未知字段
#### Scenario: xpath rule 未知字段非法
- **WHEN** HTTP target 配置 `expect.body: [{xpath: {path: "/html/body", unknown: true}}]`
- **THEN** 系统 SHALL 在启动期配置校验失败，提示未知字段
#### Scenario: regex body 规则含嵌套量词启动失败
- **WHEN** HTTP target 配置 `expect.body: [{regex: "(a+)+$"}]`
- **THEN** 系统 SHALL 在启动期配置校验失败，提示正则存在 ReDoS 风险
#### Scenario: match operator 含嵌套量词启动失败
- **WHEN** HTTP target 的 expect operator 配置 `{match: "(\\d+)*x"}`
- **THEN** 系统 SHALL 在启动期配置校验失败，提示正则存在 ReDoS 风险
#### Scenario: 安全正则通过校验
- **WHEN** HTTP target 配置 `expect.body: [{regex: "\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}"}]`
- **THEN** 系统 SHALL 接受该配置（无嵌套量词，无 ReDoS 风险）
### Requirement: HTTP body 运行期失败结构化 ### Requirement: HTTP body 运行期失败结构化
系统 SHALL 将 HTTP body 运行期失败记录为结构化 CheckFailure，并保留与具体规则相关的 phase 和 path。响应内容不符合配置 SHALL 记录为 mismatch；响应内容无法按配置解析或解码 SHALL 记录为 error。 系统 SHALL 将 HTTP body 运行期失败记录为结构化 CheckFailure，并保留与具体规则相关的 phase 和 path。响应内容不符合配置 SHALL 记录为 mismatch；响应内容无法按配置解析或解码 SHALL 记录为 error。

193
openspec/specs/expect-rule-system/spec.md Normal file
View File

@@ -0,0 +1,193 @@
## Purpose
定义共享 expect 断言规则系统的核心概念和基础设施：ValueMatcher 统一匹配器、ContentRules 内容规则数组、KeyValueExpect 键值规则、以及相关的启动期校验和失败路径规范。
## Requirements
### Requirement: ValueMatcher 统一匹配器
系统 SHALL 提供共享 `ValueMatcher` 作为所有非状态类 expect 的基础匹配结构。`ValueMatcher` SHALL 支持 `equals`、`contains`、`regex`、`exists`、`empty`、`gt`、`gte`、`lt` 和 `lte` 字段。`equals` MUST 支持任意 JSON value，并使用深度相等比较。`contains` 和 `regex` SHALL 将实际值转换为字符串后匹配。`gt`、`gte`、`lt` 和 `lte` SHALL 将实际值转换为有限数字后比较，无法转换为有限数字时 SHALL 判定不匹配。一个 `ValueMatcher` 对象包含多个 matcher 字段时，系统 SHALL 要求全部 matcher 均通过。
#### Scenario: equals 匹配对象
- **WHEN** 实际值为 `{status: "ok", count: 1}` 且 matcher 为 `{equals: {status: "ok", count: 1}}`
- **THEN** 系统 SHALL 使用深度相等判定该 matcher 通过
#### Scenario: contains 字符串化匹配
- **WHEN** 实际值为 `"service ready"` 且 matcher 为 `{contains: "ready"}`
- **THEN** 系统 SHALL 判定该 matcher 通过
#### Scenario: 数字范围组合匹配
- **WHEN** 实际值为 `50` 且 matcher 为 `{gte: 0, lte: 100}`
- **THEN** 系统 SHALL 判定该 matcher 通过
#### Scenario: 多 matcher 快速失败
- **WHEN** 实际值为 `"healthy"` 且 matcher 为 `{contains: "health", regex: "^ready$"}`
- **THEN** 系统 SHALL 在任一 matcher 不满足时判定该 matcher 对象不通过
### Requirement: ValueMatcher 启动期校验
系统 SHALL 在启动期对所有 `ValueMatcher` 对象执行严格的类型和语义校验。`contains` MUST 为 string。`equals` MAY 为任意 JSON value。`exists` 和 `empty` MUST 为 boolean。`gt`、`gte`、`lt` 和 `lte` MUST 为有限数字（`Number.isFinite`）。`regex` MUST 为可编译的 string pattern，并通过 ReDoS 风险校验。ValueMatcher 对象 MUST 至少包含一个合法 matcher 字段，空对象 `{}` SHALL 导致启动期配置错误。ValueMatcher 对象 MUST NOT 包含未知字段，任何不属于 `equals`、`contains`、`regex`、`exists`、`empty`、`gt`、`gte`、`lt`、`lte` 的字段 SHALL 导致启动期配置错误。
#### Scenario: 空 matcher 对象被拒绝
- **WHEN** YAML 配置中任一 matcher 对象为空 `{}`
- **THEN** 系统 SHALL 在启动期配置校验失败，提示 matcher 必须包含至少一个合法字段
#### Scenario: 未知 matcher 字段被拒绝
- **WHEN** YAML 配置中任一 matcher 对象包含 `foo: "bar"` 等未知字段
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段未知
#### Scenario: 数值 matcher 非有限数字被拒绝
- **WHEN** YAML 配置中任一 matcher 的 `gt`、`gte`、`lt` 或 `lte` 值为 `NaN`、`Infinity` 或非数字类型
- **THEN** 系统 SHALL 在启动期配置校验失败，提示数值 matcher 必须为有限数字
#### Scenario: 布尔 matcher 非布尔值被拒绝
- **WHEN** YAML 配置中任一 matcher 的 `exists` 或 `empty` 值不是布尔类型
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段必须为布尔值
### Requirement: empty matcher 语义
`empty: true` SHALL 在以下情况判定通过：实际值为 `null`、`undefined`、空字符串 `""`、空数组 `[]` 或空对象 `{}`。`empty: false` SHALL 在以上条件均不满足时判定通过。数字 `0` 和布尔 `false` SHALL NOT 被视为 empty。
#### Scenario: null 视为 empty
- **WHEN** 实际值为 `null` 且 matcher 为 `{empty: true}`
- **THEN** 系统 SHALL 判定该 matcher 通过
#### Scenario: 空字符串视为 empty
- **WHEN** 实际值为 `""` 且 matcher 为 `{empty: true}`
- **THEN** 系统 SHALL 判定该 matcher 通过
#### Scenario: 空数组视为 empty
- **WHEN** 实际值为 `[]` 且 matcher 为 `{empty: true}`
- **THEN** 系统 SHALL 判定该 matcher 通过
#### Scenario: 空对象视为 empty
- **WHEN** 实际值为 `{}` 且 matcher 为 `{empty: true}`
- **THEN** 系统 SHALL 判定该 matcher 通过
#### Scenario: 数字 0 不视为 empty
- **WHEN** 实际值为 `0` 且 matcher 为 `{empty: true}`
- **THEN** 系统 SHALL 判定该 matcher 不通过
#### Scenario: 布尔 false 不视为 empty
- **WHEN** 实际值为 `false` 且 matcher 为 `{empty: true}`
- **THEN** 系统 SHALL 判定该 matcher 不通过
### Requirement: exists 与其他 matcher 的组合语义
当 `ValueMatcher` 同时包含 `exists: false` 和其他非存在性 matcher（如 `contains`、`regex`、`equals` 等）时，系统 SHALL 在启动期配置校验失败，提示 `exists: false` 不能与其他 matcher 组合使用。`exists: true` MAY 与其他 matcher 组合，语义为先确认存在再执行其他 matcher。
#### Scenario: exists false 与 contains 组合被拒绝
- **WHEN** YAML 配置中 matcher 为 `{exists: false, contains: "foo"}`
- **THEN** 系统 SHALL 在启动期配置校验失败，提示 `exists: false` 不能与其他 matcher 组合
#### Scenario: exists true 与 contains 组合允许
- **WHEN** 实际值为 `"hello foo"` 且 matcher 为 `{exists: true, contains: "foo"}`
- **THEN** 系统 SHALL 判定该 matcher 通过
### Requirement: regex 字段语义
系统 SHALL 使用 `regex` 作为唯一正则 matcher 字段。`regex` 值 MUST 为可编译的字符串 pattern。运行期 SHALL 固定使用无 flags 的 `new RegExp(pattern).test(String(actual))` 执行匹配。系统 MUST NOT 支持旧 `match` 字段。系统 SHALL 在启动期对所有 `regex` pattern 执行可编译校验和 ReDoS 风险校验。
#### Scenario: regex 任意位置匹配
- **WHEN** 实际值为 `"api status ok"` 且 matcher 为 `{regex: "status"}`
- **THEN** 系统 SHALL 判定该 matcher 通过，因为无 flags 的 JavaScript 正则仍会搜索整个字符串中的第一次匹配
#### Scenario: regex 完整匹配由用户声明锚点
- **WHEN** 实际值为 `"OK\n"` 且 matcher 为 `{regex: "^OK$"}`
- **THEN** 系统 SHALL 判定该 matcher 不通过，因为系统 MUST NOT 默认启用 multiline flags
#### Scenario: match 字段启动失败
- **WHEN** YAML 配置中任一 matcher 对象包含 `match: "ok"`
- **THEN** 系统 SHALL 在启动期配置校验失败，提示 `match` 是未知字段或不支持字段
#### Scenario: regex ReDoS 风险启动失败
- **WHEN** YAML 配置中任一 `regex` 为 `"(a+)+$"`
- **THEN** 系统 SHALL 在启动期配置校验失败，提示正则存在 ReDoS 风险
### Requirement: ContentRules 内容规则数组
系统 SHALL 提供共享 `ContentRules` 表达返回内容断言。`ContentRules` MUST 为有序数组，数组项 SHALL 为直接 `ValueMatcher`，或 `json`、`css`、`xpath` 三类 extractor 规则之一。系统 SHALL 按数组顺序执行全部规则，任一规则失败时 SHALL 立即停止并返回该规则的 failure。系统 MUST NOT 支持内容字段的非数组对象快捷写法。
#### Scenario: 直接 matcher 内容规则
- **WHEN** 内容字段配置 `[{contains: "ready"}, {regex: "listening on \\d+"}]` 且原始内容同时满足两条规则
- **THEN** 系统 SHALL 判定该内容字段通过
#### Scenario: 内容规则数组快速失败
- **WHEN** 内容字段配置三条规则且第二条规则失败
- **THEN** 系统 SHALL 返回第二条规则的 failure，并 MUST NOT 执行第三条规则
#### Scenario: 内容字段必须为数组
- **WHEN** YAML 中内容字段配置为 `{contains: "ok"}` 而不是数组
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该内容字段必须为规则数组
### Requirement: ContentRule 互斥性约束
一条 `ContentRule` MUST 为直接 `ValueMatcher` 或恰好一个 extractor（`json`、`css`、`xpath` 之一）。系统 MUST NOT 允许同一条规则同时包含多个 extractor。直接 `ValueMatcher` 规则 MUST NOT 包含 `json`、`css`、`xpath` 字段。系统 SHALL 在启动期对违反互斥性的规则报错。
#### Scenario: 多 extractor 被拒绝
- **WHEN** YAML 中内容规则为 `{json: {path: "$.a"}, css: {selector: "div"}}`
- **THEN** 系统 SHALL 在启动期配置校验失败，提示一条规则不能同时包含多个 extractor
#### Scenario: 直接 matcher 混入 extractor 被拒绝
- **WHEN** YAML 中内容规则为 `{contains: "ok", json: {path: "$.a"}}`
- **THEN** 系统 SHALL 在启动期配置校验失败，提示直接 matcher 不能与 extractor 混用
### Requirement: 空 ContentRules 数组语义
`ContentRules` 空数组 `[]` SHALL 被系统接受为合法配置。运行期空数组 SHALL 等价于无规则，即该内容字段的断言直接通过。
#### Scenario: 空 body 数组通过
- **WHEN** HTTP target 配置 `expect.body: []` 且响应体为任意内容
- **THEN** 系统 SHALL 判定 body 阶段通过
### Requirement: ContentRules 非字符串值序列化
当 `ContentRules` 的观测源为非字符串值（如对象或数组）时，直接 `ValueMatcher` 的 `contains` 和 `regex` SHALL 先将值 JSON 序列化为字符串后匹配。`equals` SHALL 直接在原始结构化值上使用深度相等比较，不进行序列化。
#### Scenario: 对象序列化后 contains 匹配
- **WHEN** ContentRules 观测源为 `{status: "ok"}` 且规则为 `{contains: "ok"}`
- **THEN** 系统 SHALL 将对象 JSON 序列化后执行 contains 匹配
#### Scenario: 对象 equals 不序列化
- **WHEN** ContentRules 观测源为 `{status: "ok"}` 且规则为 `{equals: {status: "ok"}}`
- **THEN** 系统 SHALL 直接在结构化值上使用深度相等比较
### Requirement: ContentRules 提取器
系统 SHALL 支持在 `ContentRules` 中使用 `json`、`css` 和 `xpath` extractor。`json.path` MUST 使用现有 JSONPath 子集。`css.selector` MUST 为非空字符串，并 MAY 配置 `attr` 提取属性值。`xpath.path` MUST 为非空字符串，并 SHALL 在启动期进行可编译校验。Extractor 内部 MAY 包含任意 `ValueMatcher` 字段。Extractor 规则未配置任何 matcher 时 SHALL 等价于 `exists: true`。
#### Scenario: json extractor 数字比较
- **WHEN** 原始内容为 JSON 字符串 `{"count": 2}` 且规则为 `{json: {path: "$.count", gte: 1}}`
- **THEN** 系统 SHALL 解析 JSON、提取 `$.count` 并判定该规则通过
#### Scenario: json extractor 存在性默认语义
- **WHEN** 原始内容为 JSON 字符串 `{"user": {"id": null}}` 且规则为 `{json: {path: "$.user.id"}}`
- **THEN** 系统 SHALL 将该规则视为 `{json: {path: "$.user.id", exists: true}}` 并判定通过
#### Scenario: css attr 存在性默认语义
- **WHEN** 原始内容包含 `<meta name="status" content="ok">` 且规则为 `{css: {selector: "meta[name=status]", attr: "content"}}`
- **THEN** 系统 SHALL 在属性存在时判定该规则通过
#### Scenario: xpath 无匹配节点失败
- **WHEN** XML 内容中不存在 XPath 指向的节点，且规则为 `{xpath: {path: "/root/status"}}`
- **THEN** 系统 SHALL 判定该规则不通过并生成 phase 对应内容字段的 mismatch failure
### Requirement: KeyValueExpect 键值规则
系统 SHALL 提供共享 `KeyValueExpect` 表达键值型观测值断言。`KeyValueExpect` SHALL 为动态键对象，每个键对应的值 MAY 为 `ValueMatcher` 或 JSON 字面量。字面量值 SHALL 等价于 `{equals: <literal>}`。调用方 MAY 指定 key 规范化策略；HTTP 与 LLM headers MUST 使用大小写不敏感的 key 匹配。
#### Scenario: headers 字面量快捷写法
- **WHEN** 响应 headers 中 `content-type` 为 `application/json`，且配置为 `headers: {Content-Type: "application/json"}`
- **THEN** 系统 SHALL 按大小写不敏感 key 匹配并使用 equals 语义判定通过
#### Scenario: headers matcher 写法
- **WHEN** 响应 headers 中 `content-type` 为 `application/json; charset=utf-8`，且配置为 `headers: {Content-Type: {contains: "application/json"}}`
- **THEN** 系统 SHALL 判定该 header 规则通过
#### Scenario: 缺失键 exists false
- **WHEN** 观测键值表中不存在 `x-debug`，且配置为 `{x-debug: {exists: false}}`
- **THEN** 系统 SHALL 判定该键规则通过
### Requirement: 结构化失败路径
系统 SHALL 在共享 matcher、content 和 key-value 断言失败时生成结构化 `CheckFailure`。failure SHALL 包含 `kind`、`phase`、`path`、`message`，并在 mismatch 场景包含 `expected` 和 `actual`。内容规则 failure path SHALL 包含数组下标，key-value failure path SHALL 包含键名，extractor failure path SHALL 包含 extractor 类型和 path/selector 信息。
#### Scenario: ContentRules 失败路径
- **WHEN** `expect.body[1].json` 规则失败
- **THEN** failure.path SHALL 指向 `body[1].json($.path)` 或等价可定位路径，failure.phase SHALL 为 `body`
#### Scenario: KeyValueExpect 失败路径
- **WHEN** `expect.headers.Content-Type` 不匹配
- **THEN** failure.path SHALL 指向 `headers.Content-Type`，failure.phase SHALL 为 `headers`
#### Scenario: actual 截断
- **WHEN** matcher 失败时 actual 字符串长度超过 200 字符
- **THEN** 系统 SHALL 使用现有截断策略保存 failure.actual，避免历史记录写入过长内容

58
openspec/specs/icmp-checker/spec.md
View File

@@ -106,7 +106,7 @@
- **THEN** 系统 SHALL 记录 `matched=false`，failure 的 kind 为 `error`，phase 为 `ping`，path 为 `parse`，message 包含 "无法解析 ping 输出" - **THEN** 系统 SHALL 记录 `matched=false`，failure 的 kind 为 `error`，phase 为 `ping`，path 为 `parse`，message 包含 "无法解析 ping 输出"
### Requirement: ping expect 校验 ### Requirement: ping expect 校验
系统 SHALL 支持 ping 专属 expect，包括 `alive`、`maxPacketLoss`、`maxAvgLatencyMs`、`maxMaxLatencyMs` 和 `maxDurationMs`，并按 alive、packetLoss、avgLatency、maxLatency、duration 的阶段顺序快速失败。 系统 SHALL 支持 ping 专属 expect，包括 `alive`、`packetLossPercent`、`avgLatencyMs`、`maxLatencyMs` 和 `durationMs`，并按 alive、packetLossPercent、avgLatencyMs、maxLatencyMs、durationMs 的阶段顺序快速失败。`alive` SHALL 保持布尔状态语义，未配置时默认 `true`。`packetLossPercent` SHALL 表示 0 到 100 的丢包率百分比，并使用共享 `ValueMatcher`。`avgLatencyMs`、`maxLatencyMs` 和 `durationMs` SHALL 使用共享 `ValueMatcher`。
#### Scenario: 默认 alive 成功语义 #### Scenario: 默认 alive 成功语义
- **WHEN** ping target 未显式配置 `expect.alive` - **WHEN** ping target 未显式配置 `expect.alive`
@@ -124,57 +124,53 @@
- **WHEN** ping target 配置 `expect.alive: false`，且目标主机不可达 - **WHEN** ping target 配置 `expect.alive: false`，且目标主机不可达
- **THEN** 系统 SHALL 判定 alive 阶段通过（`matched=true`） - **THEN** 系统 SHALL 判定 alive 阶段通过（`matched=true`）
#### Scenario: 反向 alive 断言失败 #### Scenario: packetLossPercent 校验通过
- **WHEN** ping target 配置 `expect.alive: false`，但目标主机可达 - **WHEN** ping target 配置 `expect.packetLossPercent: {lte: 10}`，且实际丢包率为 0%
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `alive` - **THEN** 系统 SHALL 判定 packetLossPercent 阶段通过
#### Scenario: maxPacketLoss 校验通过 #### Scenario: packetLossPercent 校验失败
- **WHEN** ping target 配置 `expect.maxPacketLoss: 10`，且实际丢包率为 0% - **WHEN** ping target 配置 `expect.packetLossPercent: {lte: 10}`，且实际丢包率为 33%
- **THEN** 系统 SHALL 判定 packetLoss 阶段通过
#### Scenario: maxPacketLoss 校验失败
- **WHEN** ping target 配置 `expect.maxPacketLoss: 10`，且实际丢包率为 33%
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `packetLoss` - **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `packetLoss`
#### Scenario: maxAvgLatencyMs 校验通过 #### Scenario: avgLatencyMs 校验通过
- **WHEN** ping target 配置 `expect.maxAvgLatencyMs: 200`，且实际平均延迟为 12ms - **WHEN** ping target 配置 `expect.avgLatencyMs: {lte: 200}`，且实际平均延迟为 12ms
- **THEN** 系统 SHALL 判定 avgLatency 阶段通过 - **THEN** 系统 SHALL 判定 avgLatency 阶段通过
#### Scenario: maxAvgLatencyMs 校验失败 #### Scenario: avgLatencyMs 校验失败
- **WHEN** ping target 配置 `expect.maxAvgLatencyMs: 100`，且实际平均延迟为 156ms - **WHEN** ping target 配置 `expect.avgLatencyMs: {lte: 100}`，且实际平均延迟为 156ms
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `avgLatency` - **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `avgLatency`
#### Scenario: maxMaxLatencyMs 校验通过 #### Scenario: maxLatencyMs 校验通过
- **WHEN** ping target 配置 `expect.maxMaxLatencyMs: 500`，且实际最大延迟为 340ms - **WHEN** ping target 配置 `expect.maxLatencyMs: {lte: 500}`，且实际最大延迟为 340ms
- **THEN** 系统 SHALL 判定 maxLatency 阶段通过 - **THEN** 系统 SHALL 判定 maxLatency 阶段通过
#### Scenario: maxMaxLatencyMs 校验失败 #### Scenario: maxLatencyMs 校验失败
- **WHEN** ping target 配置 `expect.maxMaxLatencyMs: 200`，且实际最大延迟为 340ms - **WHEN** ping target 配置 `expect.maxLatencyMs: {lte: 200}`，且实际最大延迟为 340ms
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `maxLatency` - **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `maxLatency`
#### Scenario: maxDurationMs 校验 #### Scenario: durationMs 校验
- **WHEN** ping target 配置 `expect.maxDurationMs: 5000`，且完整执行耗时超过 5000ms - **WHEN** ping target 配置 `expect.durationMs: {lte: 5000}`，且完整执行耗时超过 5000ms
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `duration` - **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `duration`
#### Scenario: alive=false 时跳过延迟断言 #### Scenario: alive=false 时跳过延迟断言
- **WHEN** ping target 配置 `expect.alive: true` 和 `expect.maxAvgLatencyMs: 100`，且目标不可达 - **WHEN** ping target 配置 `expect.alive: true` 和 `expect.avgLatencyMs: {lte: 100}`，且目标不可达
- **THEN** 系统 SHALL 在 alive 阶段即返回失败，不执行后续延迟断言 - **THEN** 系统 SHALL 在 alive 阶段即返回失败，不执行后续延迟断言
#### Scenario: ping expect 未知字段失败 #### Scenario: ping expect 未知字段失败
- **WHEN** YAML 中 ping target 的 expect 包含 `status: [200]` 或其他非 ping expect 字段 - **WHEN** YAML 中 ping target 的 expect 包含 `status: [200]`、`maxPacketLoss`、`maxAvgLatencyMs`、`maxMaxLatencyMs`、`maxDurationMs` 或其他非 ping expect 字段
- **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段 - **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段
#### Scenario: maxPacketLoss 类型非法 #### Scenario: packetLossPercent 类型非法
- **WHEN** YAML 中 ping target 的 `expect.maxPacketLoss` 不是 0 到 100 之间的数字 - **WHEN** YAML 中 ping target 的 `expect.packetLossPercent` 不是合法 `ValueMatcher`，或其数值范围无法用于 0 到 100 的百分比断言
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.maxPacketLoss 必须为 0-100 的数字 - **THEN** 系统 SHALL 以配置错误退出，提示 expect.packetLossPercent 格式错误
#### Scenario: maxAvgLatencyMs 类型非法 #### Scenario: avgLatencyMs 类型非法
- **WHEN** YAML 中 ping target 的 `expect.maxAvgLatencyMs` 不是非负有限数字 - **WHEN** YAML 中 ping target 的 `expect.avgLatencyMs` 不是合法 `ValueMatcher`
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.maxAvgLatencyMs 格式错误 - **THEN** 系统 SHALL 以配置错误退出，提示 expect.avgLatencyMs 格式错误
#### Scenario: maxMaxLatencyMs 类型非法 #### Scenario: maxLatencyMs 类型非法
- **WHEN** YAML 中 ping target 的 `expect.maxMaxLatencyMs` 不是非负有限数字 - **WHEN** YAML 中 ping target 的 `expect.maxLatencyMs` 不是合法 `ValueMatcher`
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.maxMaxLatencyMs 格式错误 - **THEN** 系统 SHALL 以配置错误退出，提示 expect.maxLatencyMs 格式错误
### Requirement: ping statusDetail 摘要 ### Requirement: ping statusDetail 摘要
系统 SHALL 在 ping 执行成功后生成结构化 statusDetail 摘要，展示关键指标。 系统 SHALL 在 ping 执行成功后生成结构化 statusDetail 摘要，展示关键指标。

46
openspec/specs/llm-checker/spec.md
View File

@@ -108,7 +108,7 @@ LLM checker SHALL 在 SDK 调用结果和 expect 断言之间构建 `LlmCheckObs
- **THEN** LLM checker SHALL 返回 `phase: "request"` 的 error failure - **THEN** LLM checker SHALL 返回 `phase: "request"` 的 error failure
### Requirement: LLM Expect 断言 ### Requirement: LLM Expect 断言
LLM checker SHALL 支持 `expect.status`、`expect.headers`、`expect.output`、`expect.finishReason`、`expect.rawFinishReason`、`expect.usage.inputTokens`、`expect.usage.outputTokens`、`expect.usage.totalTokens`、`expect.stream.completed`、`expect.stream.firstTokenMs` 和 `expect.maxDurationMs`。`expect.status` 和 `expect.headers` 的运行期断言 SHALL 复用 `src/server/checker/runner/http/expect.ts` 中的 `checkStatus` 和 `checkHeaders` 函数。LLM checker MUST 按固定顺序快速失败，非流式顺序为 status、headers、output、finishReason、rawFinishReason、usage、duration；流式顺序为 status、headers、stream.completed、stream.firstTokenMs、output、finishReason、rawFinishReason、usage、duration。 LLM checker SHALL 支持 `expect.status`、`expect.headers`、`expect.output`、`expect.finishReason`、`expect.rawFinishReason`、`expect.usage.inputTokens`、`expect.usage.outputTokens`、`expect.usage.totalTokens`、`expect.stream.completed`、`expect.stream.firstTokenMs` 和 `expect.durationMs`。`expect.status` SHALL 保持 HTTP 状态码数组语义并复用 HTTP status 断言。`expect.headers` SHALL 使用共享 `KeyValueExpect` 且 header key 大小写不敏感。`expect.output` MUST 使用共享 `ContentRules`。`expect.finishReason` 和 `expect.rawFinishReason` SHALL 使用共享 `ValueMatcher`。`expect.usage.*`、`expect.stream.firstTokenMs` 和 `expect.durationMs` SHALL 使用共享 `ValueMatcher`。LLM checker MUST 按固定顺序快速失败，非流式顺序为 status、headers、output、finishReason、rawFinishReason、usage、durationMs；流式顺序为 status、headers、stream.completed、stream.firstTokenMs、output、finishReason、rawFinishReason、usage、durationMs。
#### Scenario: 默认 status 断言 #### Scenario: 默认 status 断言
- **WHEN** LLM target 未配置 `expect.status` - **WHEN** LLM target 未配置 `expect.status`
@@ -118,13 +118,25 @@ LLM checker SHALL 支持 `expect.status`、`expect.headers`、`expect.output`、
- **WHEN** observing fetch 捕获的响应 headers 满足 `expect.headers` 配置 - **WHEN** observing fetch 捕获的响应 headers 满足 `expect.headers` 配置
- **THEN** LLM checker SHALL 判定 headers 断言通过 - **THEN** LLM checker SHALL 判定 headers 断言通过
#### Scenario: expect headers 不匹配 #### Scenario: output ContentRules 通过
- **WHEN** observing fetch 捕获的响应 headers 不满足 `expect.headers` 中的某项配置 - **WHEN** LLM 输出文本满足 `expect.output` 中配置的全部 ContentRules
- **THEN** LLM checker SHALL 返回 `phase: "headers"` 的 mismatch failure - **THEN** LLM checker SHALL 判定 output 阶段通过
#### Scenario: 全部 expect 通过 #### Scenario: finishReason ValueMatcher 通过
- **WHEN** LLM checker 构建出的 observation 满足所有已配置 expect - **WHEN** observation.finishReason 为 `stop` 且 target 配置 `expect.finishReason: {equals: "stop"}`
- **THEN** 检查结果 SHALL 为 `matched=true` 且 `failure=null` - **THEN** LLM checker SHALL 判定 finishReason 阶段通过
#### Scenario: rawFinishReason regex 通过
- **WHEN** observation.rawFinishReason 为 `end_turn` 且 target 配置 `expect.rawFinishReason: {regex: "^(stop|end_turn)$"}`
- **THEN** LLM checker SHALL 判定 rawFinishReason 阶段通过
#### Scenario: usage matcher 通过
- **WHEN** observation.usage.totalTokens 为 14 且 target 配置 `expect.usage.totalTokens: {lte: 20}`
- **THEN** LLM checker SHALL 判定 usage 阶段通过
#### Scenario: durationMs matcher 失败
- **WHEN** LLM target 配置 `expect.durationMs: {lte: 1000}` 且实际执行耗时为 1500ms
- **THEN** LLM checker SHALL 返回 phase=`duration` 的 mismatch failure
#### Scenario: 首个 expect 失败 #### Scenario: 首个 expect 失败
- **WHEN** 多个 LLM expect 中某个较早顺序的断言失败 - **WHEN** 多个 LLM expect 中某个较早顺序的断言失败
@@ -139,7 +151,7 @@ LLM checker SHALL 支持 `expect.status`、`expect.headers`、`expect.output`、
- **THEN** LLM checker SHALL 因 `outputText` 缺失返回 `phase: "output"` 的 mismatch failure - **THEN** LLM checker SHALL 因 `outputText` 缺失返回 `phase: "output"` 的 mismatch failure
### Requirement: LLM Output 规则 ### Requirement: LLM Output 规则
LLM checker SHALL 支持 `expect.output` 有序规则数组，每个规则 MUST 仅包含 `equals`、`contains`、`regex` 或 `json` 中的一种。`equals` SHALL 对原始输出字符串做严格相等比较。`contains` SHALL 判断原始输出是否包含子串。`regex` SHALL 对原始输出执行正则匹配。`json` SHALL 将原始输出解析为 JSON，并用现有 JSONPath 子集和 operator 校验提取值。 LLM checker SHALL 使用共享 `ContentRules` 校验 `expect.output`。每个 output rule SHALL 为直接 `ValueMatcher`，或 `json`、`css`、`xpath` extractor 规则之一。直接 matcher SHALL 作用于原始输出字符串。`equals` SHALL 对原始输出字符串做严格相等比较。`contains` SHALL 判断原始输出是否包含子串。`regex` SHALL 对原始输出执行无 flags 正则匹配。`json` SHALL 将原始输出解析为 JSON，并用现有 JSONPath 子集和 `ValueMatcher` 校验提取值。`json.equals` SHALL 支持任意 JSON value。`css` 和 `xpath` 在 schema 层面可用，但 LLM 输出通常为纯文本或 JSON，实际场景中仅 `json` 提取器有意义。
#### Scenario: 原始输出严格相等 #### Scenario: 原始输出严格相等
- **WHEN** `outputText` 为 `"OK\n"` 且 target 配置 `expect.output: [{ equals: "OK" }]` - **WHEN** `outputText` 为 `"OK\n"` 且 target 配置 `expect.output: [{ equals: "OK" }]`
@@ -150,19 +162,27 @@ LLM checker SHALL 支持 `expect.output` 有序规则数组，每个规则 MUST
- **THEN** LLM checker SHALL 判定该 output contains 规则通过 - **THEN** LLM checker SHALL 判定该 output contains 规则通过
#### Scenario: output regex 通过 #### Scenario: output regex 通过
- **WHEN** `outputText` 匹配配置的合法正则 - **WHEN** `outputText` 匹配配置的合法 regex
- **THEN** LLM checker SHALL 判定该 output regex 规则通过 - **THEN** LLM checker SHALL 判定该 output regex 规则通过
#### Scenario: output JSONPath 通过 #### Scenario: output JSONPath 字符串 equals 通过
- **WHEN** `outputText` 是 JSON 字符串且 JSONPath 提取值满足 operator - **WHEN** `outputText` 是 JSON 字符串且 JSONPath 提取字符串值满足 `equals`
- **THEN** LLM checker SHALL 判定该 output json 规则通过 - **THEN** LLM checker SHALL 判定该 output json 规则通过
#### Scenario: output JSONPath 对象 equals 通过
- **WHEN** `outputText` 是 JSON 字符串且 JSONPath 提取对象值满足 `equals`
- **THEN** LLM checker SHALL 使用深度相等判定该 output json 规则通过
#### Scenario: output JSONPath 存在性默认语义
- **WHEN** `outputText` 是 JSON 字符串且 target 配置 `expect.output: [{json: {path: "$.status"}}]`
- **THEN** LLM checker SHALL 将该规则按 `exists: true` 语义执行
#### Scenario: output 规则按顺序快速失败 #### Scenario: output 规则按顺序快速失败
- **WHEN** `expect.output` 包含多个规则且第一条规则失败 - **WHEN** `expect.output` 包含多个规则且第一条规则失败
- **THEN** LLM checker SHALL 返回第一条失败规则的 mismatch failure，不继续校验后续 output 规则 - **THEN** LLM checker SHALL 返回第一条失败规则的 mismatch failure，不继续校验后续 output 规则
### Requirement: LLM Stream 断言 ### Requirement: LLM Stream 断言
LLM checker SHALL 仅允许 `mode: stream` 使用 `expect.stream`。`expect.stream.completed` 未配置时，LLM checker SHALL 在 stream observation 路径使用默认 `true` 语义。`expect.stream.firstTokenMs` SHALL 仅统计第一个非空 `text-delta` 事件耗时，不统计 reasoning、tool call 或 source 事件。 LLM checker SHALL 仅允许 `mode: stream` 使用 `expect.stream`。`expect.stream.completed` 未配置时，LLM checker SHALL 在 stream observation 路径使用默认 `true` 语义。`expect.stream.firstTokenMs` SHALL 使用共享 `ValueMatcher`，并仅统计第一个非空 `text-delta` 事件耗时，不统计 reasoning、tool call 或 source 事件。
#### Scenario: stream completed 默认值 #### Scenario: stream completed 默认值
- **WHEN** target 配置 `llm.mode: stream` 且未配置 `expect.stream.completed` - **WHEN** target 配置 `llm.mode: stream` 且未配置 `expect.stream.completed`
@@ -173,7 +193,7 @@ LLM checker SHALL 仅允许 `mode: stream` 使用 `expect.stream`。`expect.stre
- **THEN** LLM checker SHALL 返回 `phase: "stream"` 的 failure - **THEN** LLM checker SHALL 返回 `phase: "stream"` 的 failure
#### Scenario: firstTokenMs 达标 #### Scenario: firstTokenMs 达标
- **WHEN** target 配置 `expect.stream.firstTokenMs` 且首个非空 text delta 耗时满足 operator - **WHEN** target 配置 `expect.stream.firstTokenMs: {lte: 1000}` 且首个非空 text delta 耗时满足 matcher
- **THEN** LLM checker SHALL 判定 firstTokenMs 断言通过 - **THEN** LLM checker SHALL 判定 firstTokenMs 断言通过
#### Scenario: firstTokenMs 缺失 #### Scenario: firstTokenMs 缺失

112
openspec/specs/probe-config/spec.md
View File

@@ -193,9 +193,9 @@
- **WHEN** YAML 中某个 HTTP target 的 `expect.status` 包含非整数或不在 100-599 范围内的数字 - **WHEN** YAML 中某个 HTTP target 的 `expect.status` 包含非整数或不在 100-599 范围内的数字
- **THEN** 系统 SHALL 以错误退出，提示该 target 的 status 数字不合法 - **THEN** 系统 SHALL 以错误退出，提示该 target 的 status 数字不合法
#### Scenario: maxDurationMs 非法 #### Scenario: durationMs matcher 非法
- **WHEN** YAML 中某个 target 的 `expect.maxDurationMs` 不是非负有限数字 - **WHEN** YAML 中某个 target 的 `expect.durationMs` 不是合法 `ValueMatcher`
- **THEN** 系统 SHALL 以错误退出，提示该 target 的 expect.maxDurationMs 格式错误 - **THEN** 系统 SHALL 以错误退出，提示该 target 的 expect.durationMs 格式错误
#### Scenario: ping target 缺少 host #### Scenario: ping target 缺少 host
- **WHEN** YAML 中某个 target 配置 `type: ping` 但缺少 `ping.host` - **WHEN** YAML 中某个 target 配置 `type: ping` 但缺少 `ping.host`
@@ -237,13 +237,13 @@
- **WHEN** YAML 中某个 HTTP target 的 body xpath 规则缺少 path，或 path 不是非空字符串，或可被现有 XPath 库静态判定为语法错误 - **WHEN** YAML 中某个 HTTP target 的 body xpath 规则缺少 path，或 path 不是非空字符串，或可被现有 XPath 库静态判定为语法错误
- **THEN** 系统 SHALL 以错误退出，提示该 body xpath path 不合法 - **THEN** 系统 SHALL 以错误退出，提示该 body xpath path 不合法
#### Scenario: expect operator 类型非法 #### Scenario: expect matcher 类型非法
- **WHEN** YAML 中某个 HTTP expect operator 的 match 不是可编译正则字符串，empty/exists 不是布尔值，或 gt/gte/lt/lte 不是有限数字 - **WHEN** YAML 中某个 expect matcher 的 regex 不是可编译正则字符串，empty/exists 不是布尔值，或 gt/gte/lt/lte 不是有限数字
- **THEN** 系统 SHALL 以错误退出，提示对应 operator 配置不合法 - **THEN** 系统 SHALL 以错误退出，提示对应 matcher 配置不合法
#### Scenario: expect operator 类型非法 #### Scenario: expect match 字段不再支持
- **WHEN** YAML 中某个 expect operator 的 match 不是可编译正则字符串，empty/exists 不是布尔值，或 gt/gte/lt/lte 不是有限数字 - **WHEN** YAML 中某个 expect matcher 配置 `match` 字段
- **THEN** 系统 SHALL 以错误退出，提示对应 operator 配置不合法 - **THEN** 系统 SHALL 以错误退出，提示 `match` 是未知字段，请使用 `regex`
#### Scenario: unknown 字段失败 #### Scenario: unknown 字段失败
- **WHEN** YAML 中任一结构化配置对象包含契约未声明的字段，且该对象不是明确允许动态键的对象 - **WHEN** YAML 中任一结构化配置对象包含契约未声明的字段，且该对象不是明确允许动态键的对象
@@ -301,18 +301,38 @@
- **THEN** 系统 SHALL 调用 `Bun.YAML.parse()` 将内容解析为配置对象 - **THEN** 系统 SHALL 调用 `Bun.YAML.parse()` 将内容解析为配置对象
### Requirement: expect 配置增强 ### Requirement: expect 配置增强
系统 SHALL 支持 typed target 的领域专用 expect 配置，包括 HTTP 的 `status`（支持精确数字和范围模式）、`headers`、`body`，cmd 的 `exitCode`、`stdout`、`stderr`，tcp 的 `connected`、`banner`，ping 的 `alive`、`maxPacketLoss`、`maxAvgLatencyMs`、`maxMaxLatencyMs`，udp 的 `responded`、`response`、`responseSize`、`sourceHost`、`sourcePort` 和 `maxDurationMs`，以及 llm 的 `status`、`headers`、`output`、`finishReason`、`rawFinishReason`、`usage`、`stream` 和 `maxDurationMs`。内容类 expect MUST 使用数组表达配置顺序。 系统 SHALL 支持 typed target 的领域专用 expect 配置，并通过共享 `ValueMatcher`、`ContentRules` 和 `KeyValueExpect` 表达可复用断言能力。状态类字段 SHALL 保持枚举或布尔语义，包括 HTTP/LLM 的 `status`（支持精确数字和范围模式）、cmd 的 `exitCode`、tcp 的 `connected`、ping 的 `alive` 和 udp 的 `responded`。数字指标字段 SHALL 使用 `ValueMatcher`，包括通用 `durationMs`、db 的 `rowCount`、udp 的 `responseSize`/`sourceHost`/`sourcePort`、ping 的 `packetLossPercent`/`avgLatencyMs`/`maxLatencyMs`、llm 的 usage token 与 stream 首 token 耗时。内容类字段 MUST 使用 `ContentRules` 数组表达配置顺序，包括 HTTP `body`、cmd `stdout`/`stderr`、tcp `banner`、udp `response`、llm `output` 和 db `result`。LLM `finishReason` 和 `rawFinishReason` SHALL 使用 `ValueMatcher`（非 ContentRules），因为它们是单值字符串元数据。键值类字段 SHALL 使用 `KeyValueExpect`，包括 HTTP/LLM `headers` 和 db `rows` 中的列值断言（db `rows` 的类型为 `Array<KeyValueExpect>`，外层数组按行索引，内层每个元素为 KeyValueExpect）。
#### Scenario: 解析 HTTP expect 配置 #### Scenario: 解析 HTTP expect 配置
- **WHEN** YAML 配置文件中 HTTP target 的 expect 包含 status、headers、body 规则数组及内部方法 - **WHEN** YAML 配置文件中 HTTP target 的 expect 包含 status、headers、body 规则数组和 durationMs matcher
- **THEN** 系统 SHALL 正确解析并存储为 HTTP target 的 expect 字段 - **THEN** 系统 SHALL 正确解析并存储为 HTTP target 的 expect 字段
#### Scenario: 解析 cmd expect 配置 #### Scenario: 解析 cmd expect 配置
- **WHEN** YAML 配置文件中 cmd target 的 expect 包含 exitCode、stdout 和 stderr 规则数组 - **WHEN** YAML 配置文件中 cmd target 的 expect 包含 exitCode、stdout、stderr 和 durationMs matcher
- **THEN** 系统 SHALL 正确解析并存储为 cmd target 的 expect 字段 - **THEN** 系统 SHALL 正确解析并存储为 cmd target 的 expect 字段
#### Scenario: 解析 body 有序规则数组 #### Scenario: 解析 db expect 配置
- **WHEN** YAML 中 HTTP target 配置 `expect.body` 为 contains、json、regex 三个数组项 - **WHEN** YAML 配置文件中 db target 的 expect 包含 durationMs、rowCount、rows 和 result
- **THEN** 系统 SHALL 正确解析并存储为 db target 的 expect 字段
#### Scenario: 解析 tcp expect 配置
- **WHEN** YAML 配置文件中 tcp target 的 expect 包含 connected、banner 规则数组和 durationMs matcher
- **THEN** 系统 SHALL 正确解析并存储为 tcp target 的 expect 字段
#### Scenario: 解析 ping expect 配置
- **WHEN** YAML 配置文件中 ping target 的 expect 包含 alive、packetLossPercent、avgLatencyMs、maxLatencyMs 和 durationMs matcher
- **THEN** 系统 SHALL 正确解析并存储为 ping target 的 expect 字段
#### Scenario: 解析 udp expect 配置
- **WHEN** YAML 配置文件中 udp target 的 expect 包含 responded、response、responseSize、sourceHost、sourcePort 和 durationMs matcher
- **THEN** 系统 SHALL 正确解析并存储为 udp target 的 expect 字段
#### Scenario: 解析 llm expect 配置
- **WHEN** YAML 配置文件中 llm target 的 expect 包含 status、headers、output、finishReason、rawFinishReason、usage、stream 和 durationMs matcher
- **THEN** 系统 SHALL 正确解析并存储为 llm target 的 expect 字段，并保留 output 内容规则数组顺序
#### Scenario: 解析有序 ContentRules 数组
- **WHEN** YAML 中任一内容类 expect 配置 contains、json、regex 三个数组项
- **THEN** 系统 SHALL 保留数组顺序，供执行阶段按配置顺序快速失败 - **THEN** 系统 SHALL 保留数组顺序，供执行阶段按配置顺序快速失败
#### Scenario: 不配置 HTTP status #### Scenario: 不配置 HTTP status
@@ -323,41 +343,37 @@
- **WHEN** HTTP target 配置 `expect.status: ["2xx"]` - **WHEN** HTTP target 配置 `expect.status: ["2xx"]`
- **THEN** 系统 SHALL 在执行 expect 时匹配所有 200-299 状态码 - **THEN** 系统 SHALL 在执行 expect 时匹配所有 200-299 状态码
#### Scenario: 配置 HTTP status 混合模式
- **WHEN** HTTP target 配置 `expect.status: ["2xx", 301]`
- **THEN** 系统 SHALL 在执行 expect 时匹配所有 200-299 状态码或精确匹配 301
#### Scenario: 不配置 cmd exitCode #### Scenario: 不配置 cmd exitCode
- **WHEN** cmd target 未配置 `expect.exitCode` - **WHEN** cmd target 未配置 `expect.exitCode`
- **THEN** 系统 SHALL 在执行 expect 时使用默认 `exitCode: [0]` 语义 - **THEN** 系统 SHALL 在执行 expect 时使用默认 `exitCode: [0]` 语义
#### Scenario: 不配置 expect #### Scenario: 不配置 expect
- **WHEN** target 未配置任何 expect 规则 - **WHEN** target 未配置任何 expect 规则
- **THEN** 系统 SHALL 正常处理，expect 字段为 undefined - **THEN** 系统 SHALL 正常处理，expect 字段为 undefined，并由各 checker 使用自身默认状态语义
#### Scenario: 解析 ping expect 配置 #### Scenario: 旧 maxDurationMs 字段不再支持
- **WHEN** YAML 配置文件中 ping target 的 expect 包含 alive、maxPacketLoss、maxAvgLatencyMs、maxMaxLatencyMs 和 maxDurationMs - **WHEN** YAML 中任一 target 配置 `expect.maxDurationMs`
- **THEN** 系统 SHALL 正确解析并存储为 ping target 的 expect 字段 - **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段未知，并要求使用 `expect.durationMs`
#### Scenario: 不配置 ping expect #### Scenario: 旧 match 字段不再支持
- **WHEN** ping target 未配置任何 expect 规则 - **WHEN** YAML 中任一 matcher 或内容规则配置 `match`
- **THEN** 系统 SHALL 正常处理，expect 字段为 undefined，执行时使用默认 alive=true 语义 - **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段未知，并要求使用 `regex`
#### Scenario: 解析 udp expect 配置 #### Scenario: durationMs matcher 配置
- **WHEN** YAML 配置文件中 udp target 的 expect 包含 responded、response、responseSize、sourceHost、sourcePort 和 maxDurationMs - **WHEN** YAML 中任一 target 配置 `expect.durationMs: {lte: 1000}`
- **THEN** 系统 SHALL 正确解析并存储为 udp target 的 expect 字段 - **THEN** 系统 SHALL 接受该配置并在运行期使用完整执行耗时进行 matcher 校验
#### Scenario: 不配置 udp expect #### Scenario: 动态 headers 字段允许
- **WHEN** udp target 未配置任何 expect 规则 - **WHEN** YAML 中 `http.headers`、`defaults.http.headers`、`llm.headers`、`defaults.llm.headers` 或 `expect.headers` 包含任意 header 名称，且对应值符合契约
- **THEN** 系统 SHALL 正常处理，expect 字段为 undefined，执行时使用默认 responded=true 语义 - **THEN** 系统 SHALL 接受这些动态 header 名称
#### Scenario: 解析 llm expect 配置 #### Scenario: ContentRules 字段必须为数组
- **WHEN** YAML 配置文件中 llm target 的 expect 包含 status、headers、output、finishReason、rawFinishReason、usage、stream 和 maxDurationMs - **WHEN** YAML 中任一内容类 expect 字段配置为非数组
- **THEN** 系统 SHALL 正确解析并存储为 llm target 的 expect 字段，并保留 output 规则数组顺序 - **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段必须为规则数组
#### Scenario: 不配置 llm expect #### Scenario: regex 字段非法
- **WHEN** llm target 未配置任何 expect 规则 - **WHEN** YAML 中任一 `regex` 不是字符串、不是可编译正则或存在 ReDoS 风险
- **THEN** 系统 SHALL 正常处理，expect 字段为 undefined，执行时使用默认 status=[200] 语义 - **THEN** 系统 SHALL 在启动期配置校验失败，提示对应 regex 不合法
### Requirement: 数据保留配置字段 ### Requirement: 数据保留配置字段
配置 schema 的 `runtime` 段 SHALL 支持 `retention` 字段，类型为字符串，格式为 `<数字><单位>`（单位：`d` 天、`h` 小时、`m` 分钟），用于指定历史数据保留时长。 配置 schema 的 `runtime` 段 SHALL 支持 `retention` 字段，类型为字符串，格式为 `<数字><单位>`（单位：`d` 天、`h` 小时、`m` 分钟），用于指定历史数据保留时长。
@@ -444,7 +460,7 @@
- **THEN** target schema SHALL 声明 `id` 的 minLength 为 1、maxLength 为 30，并声明 `name` 为可选字段，类型为 string 或 null，字符串的 minLength 为 1、maxLength 为 30 - **THEN** target schema SHALL 声明 `id` 的 minLength 为 1、maxLength 为 30，并声明 `name` 为可选字段，类型为 string 或 null，字符串的 minLength 为 1、maxLength 为 30
### Requirement: TCP 配置校验 ### Requirement: TCP 配置校验
系统 SHALL 在启动期对 tcp checker 的配置契约和语义执行严格校验。Tcp target 的 `tcp` 分组 SHALL 只允许 `host`、`port`、`readBanner`、`bannerReadTimeout` 和 `maxBannerBytes` 字段；Tcp expect SHALL 只允许 `connected`、`maxDurationMs` 和 `banner` 字段。未知字段、非法类型、非法端口、非法 size 和不可编译正则 MUST 导致启动期配置错误。 系统 SHALL 在启动期对 tcp checker 的配置契约和语义执行严格校验。Tcp target 的 `tcp` 分组 SHALL 只允许 `host`、`port`、`readBanner`、`bannerReadTimeout` 和 `maxBannerBytes` 字段；Tcp expect SHALL 只允许 `connected`、`durationMs` 和 `banner` 字段。`banner` MUST 为 `ContentRules` 数组。未知字段、非法类型、非法端口、非法 size 和不可编译正则 MUST 导致启动期配置错误。
#### Scenario: tcp host 类型非法 #### Scenario: tcp host 类型非法
- **WHEN** YAML 中 tcp target 的 `tcp.host` 不是非空字符串 - **WHEN** YAML 中 tcp target 的 `tcp.host` 不是非空字符串
@@ -471,11 +487,11 @@
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.connected 必须为布尔值 - **THEN** 系统 SHALL 以配置错误退出，提示 expect.connected 必须为布尔值
#### Scenario: tcp expect banner 非法 #### Scenario: tcp expect banner 非法
- **WHEN** YAML 中 tcp target 的 `expect.banner` 不是合法 operator 对象 - **WHEN** YAML 中 tcp target 的 `expect.banner` 不是合法 ContentRules 数组
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.banner 格式错误 - **THEN** 系统 SHALL 以配置错误退出，提示 expect.banner 格式错误
#### Scenario: tcp expect banner match 正则非法 #### Scenario: tcp expect banner regex 正则非法
- **WHEN** YAML 中 tcp target 配置 `expect.banner: { match: "[invalid" }` - **WHEN** YAML 中 tcp target 配置 `expect.banner: [{ regex: "[invalid" }]`
- **THEN** 系统 SHALL 在启动期配置校验失败，而不是延迟到运行期抛错 - **THEN** 系统 SHALL 在启动期配置校验失败，而不是延迟到运行期抛错
#### Scenario: tcp 分组未知字段失败 #### Scenario: tcp 分组未知字段失败
@@ -487,7 +503,7 @@
- **THEN** 系统 SHALL 以配置错误退出，提示 defaults.tcp 包含未知字段 - **THEN** 系统 SHALL 以配置错误退出，提示 defaults.tcp 包含未知字段
### Requirement: LLM 配置校验 ### Requirement: LLM 配置校验
系统 SHALL 在启动期对 llm checker 的配置契约和语义执行严格校验。LLM target 的 `llm` 分组 SHALL 只允许 `provider`、`url`、`model`、`prompt`、`mode`、`key`、`authToken`、`headers`、`ignoreSSL`、`options` 和 `providerOptions` 字段。`defaults.llm` 分组 SHALL 只允许 `mode`、`headers`、`ignoreSSL`、`options` 和 `providerOptions` 字段。LLM expect SHALL 只允许 `status`、`headers`、`output`、`finishReason`、`rawFinishReason`、`usage`、`stream` 和 `maxDurationMs` 字段。未知字段、非法 provider、非法 URL、非法 mode、非法认证组合、非法 options、非法 output 规则和 `mode: http` 下配置 `expect.stream` MUST 导致启动期配置错误。 系统 SHALL 在启动期对 llm checker 的配置契约和语义执行严格校验。LLM target 的 `llm` 分组 SHALL 只允许 `provider`、`url`、`model`、`prompt`、`mode`、`key`、`authToken`、`headers`、`ignoreSSL`、`options` 和 `providerOptions` 字段。`defaults.llm` 分组 SHALL 只允许 `mode`、`headers`、`ignoreSSL`、`options` 和 `providerOptions` 字段。LLM expect SHALL 只允许 `status`、`headers`、`output`、`finishReason`、`rawFinishReason`、`usage`、`stream` 和 `durationMs` 字段。`expect.output` MUST 为 `ContentRules` 数组。`expect.finishReason` 和 `expect.rawFinishReason` SHALL 使用 `ValueMatcher`。`expect.usage.*` 和 `expect.stream.firstTokenMs` SHALL 使用 `ValueMatcher`。未知字段、非法 provider、非法 URL、非法 mode、非法认证组合、非法 options、非法 output 规则和 `mode: http` 下配置 `expect.stream` MUST 导致启动期配置错误。
#### Scenario: llm provider 非法 #### Scenario: llm provider 非法
- **WHEN** YAML 中 llm target 的 `llm.provider` 不是 `openai`、`openai-responses` 或 `anthropic` - **WHEN** YAML 中 llm target 的 `llm.provider` 不是 `openai`、`openai-responses` 或 `anthropic`
@@ -538,12 +554,12 @@
- **THEN** 系统 SHALL 以配置错误退出，提示 llm 分组包含未知字段 - **THEN** 系统 SHALL 以配置错误退出，提示 llm 分组包含未知字段
#### Scenario: llm output 规则缺少支持字段 #### Scenario: llm output 规则缺少支持字段
- **WHEN** YAML 中 llm target 的 `expect.output` 数组项未包含 equals、contains、regex、json 任一支持字段 - **WHEN** YAML 中 llm target 的 `expect.output` 数组项未包含任何合法 ValueMatcher 字段或 extractor
- **THEN** 系统 SHALL 以配置错误退出，提示 output rule 缺少支持的规则类型 - **THEN** 系统 SHALL 以配置错误退出，提示 output rule 缺少支持的规则类型
#### Scenario: llm output 规则同时配置多个支持字段 #### Scenario: llm output 规则同时配置多个 extractor
- **WHEN** YAML 中 llm target 的同一条 output rule 同时包含 equals、contains、regex、json 中的多个支持字段 - **WHEN** YAML 中 llm target 的同一条 output rule 同时包含 json、css、xpath 中的多个 extractor
- **THEN** 系统 SHALL 以配置错误退出，提示每条 output rule 只能配置一种规则类型 - **THEN** 系统 SHALL 以配置错误退出，提示每条 output rule 只能配置一种 extractor
#### Scenario: llm output regex 非法 #### Scenario: llm output regex 非法
- **WHEN** YAML 中 llm target 的 output regex 规则不是字符串、不是可编译正则表达式或存在 ReDoS 风险 - **WHEN** YAML 中 llm target 的 output regex 规则不是字符串、不是可编译正则表达式或存在 ReDoS 风险
@@ -554,7 +570,7 @@
- **THEN** 系统 SHALL 以配置错误退出，提示该 output json path 不合法 - **THEN** 系统 SHALL 以配置错误退出，提示该 output json path 不合法
#### Scenario: llm expect usage 非法 #### Scenario: llm expect usage 非法
- **WHEN** YAML 中 llm target 的 `expect.usage.inputTokens`、`expect.usage.outputTokens` 或 `expect.usage.totalTokens` 不是合法 operator 对象 - **WHEN** YAML 中 llm target 的 `expect.usage.inputTokens`、`expect.usage.outputTokens` 或 `expect.usage.totalTokens` 不是合法 `ValueMatcher`
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.usage 格式错误 - **THEN** 系统 SHALL 以配置错误退出，提示 expect.usage 格式错误
#### Scenario: llm expect stream 仅允许 stream mode #### Scenario: llm expect stream 仅允许 stream mode
@@ -562,5 +578,5 @@
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.stream 仅支持 stream mode - **THEN** 系统 SHALL 以配置错误退出，提示 expect.stream 仅支持 stream mode
#### Scenario: llm expect stream firstTokenMs 非法 #### Scenario: llm expect stream firstTokenMs 非法
- **WHEN** YAML 中 llm target 的 `expect.stream.firstTokenMs` 不是合法 operator 对象 - **WHEN** YAML 中 llm target 的 `expect.stream.firstTokenMs` 不是合法 `ValueMatcher`
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.stream.firstTokenMs 格式错误 - **THEN** 系统 SHALL 以配置错误退出，提示 expect.stream.firstTokenMs 格式错误

22
openspec/specs/tcp-checker/spec.md
View File

@@ -86,28 +86,32 @@
- **THEN** `statusDetail` SHALL 展示截断后的 banner 摘要，避免 UI 和历史记录写入过长文本 - **THEN** `statusDetail` SHALL 展示截断后的 banner 摘要，避免 UI 和历史记录写入过长文本
### Requirement: tcp expect 校验 ### Requirement: tcp expect 校验
系统 SHALL 支持 tcp 专属 expect，包括 `connected`、`banner` 和 `maxDurationMs`，并按 connected、banner、duration 的阶段顺序快速失败。 系统 SHALL 支持 tcp 专属 expect，包括 `connected`、`banner` 和 `durationMs`，并按 connected、banner、durationMs 的阶段顺序快速失败。`connected` SHALL 保持布尔状态语义，未配置时默认 `true`。`banner` MUST 使用共享 `ContentRules` 数组，并仅在 `tcp.readBanner: true` 时允许配置。`durationMs` SHALL 使用共享 `ValueMatcher` 校验包含连接和 banner 读取在内的完整执行耗时。
#### Scenario: 默认 connected 成功语义 #### Scenario: 默认 connected 成功语义
- **WHEN** tcp target 未显式配置 `expect.connected` - **WHEN** tcp target 未显式配置 `expect.connected`
- **THEN** 系统 SHALL 使用默认 `expect.connected: true` 进行校验 - **THEN** 系统 SHALL 使用默认 `expect.connected: true` 进行校验
#### Scenario: maxDurationMs 校验 #### Scenario: durationMs 校验
- **WHEN** tcp target 配置 `expect.maxDurationMs: 100`，且完整执行耗时超过 100ms - **WHEN** tcp target 配置 `expect.durationMs: {lte: 100}`，且完整执行耗时超过 100ms
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `duration` - **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `duration`
#### Scenario: banner operator 校验通过 #### Scenario: banner ContentRules 校验通过
- **WHEN** tcp target 配置 `readBanner: true`、`expect.banner: { contains: "ESMTP" }`，且实际 banner 包含 `ESMTP` - **WHEN** tcp target 配置 `readBanner: true`、`expect.banner: [{contains: "ESMTP"}]`，且实际 banner 包含 `ESMTP`
- **THEN** 系统 SHALL 判定 banner 阶段通过 - **THEN** 系统 SHALL 判定 banner 阶段通过
#### Scenario: banner operator 校验失败 #### Scenario: banner regex 校验失败
- **WHEN** tcp target 配置 `readBanner: true`、`expect.banner: { contains: "ESMTP" }`，且实际 banner 不包含 `ESMTP` - **WHEN** tcp target 配置 `readBanner: true`、`expect.banner: [{regex: "^SSH-2\\.0"}]`，且实际 banner 不匹配该正则
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `banner`，path 为 `banner` - **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `banner`，path 指向失败的 banner 规则
#### Scenario: banner 多规则快速失败
- **WHEN** tcp target 配置两条 banner 规则且第一条失败
- **THEN** 系统 SHALL 返回第一条失败规则的 failure，并 MUST NOT 执行第二条规则
#### Scenario: expect.banner 未开启 readBanner #### Scenario: expect.banner 未开启 readBanner
- **WHEN** tcp target 配置 `expect.banner`，但 `tcp.readBanner` 未配置为 `true` - **WHEN** tcp target 配置 `expect.banner`，但 `tcp.readBanner` 未配置为 `true`
- **THEN** 系统 SHALL 在启动期配置校验失败，提示 banner 断言需要启用 tcp.readBanner - **THEN** 系统 SHALL 在启动期配置校验失败，提示 banner 断言需要启用 tcp.readBanner
#### Scenario: tcp expect 未知字段失败 #### Scenario: tcp expect 未知字段失败
- **WHEN** YAML 中 tcp target 的 expect 包含 `status: [200]` 或其他非 tcp expect 字段 - **WHEN** YAML 中 tcp target 的 expect 包含 `status: [200]`、`maxDurationMs: 1000` 或其他非 tcp expect 字段
- **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段 - **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段

28
openspec/specs/udp-checker/spec.md
View File

@@ -133,17 +133,21 @@
- **THEN** 系统 SHALL 以配置错误退出，提示 maxResponseBytes 格式错误 - **THEN** 系统 SHALL 以配置错误退出，提示 maxResponseBytes 格式错误
### Requirement: udp expect 校验 ### Requirement: udp expect 校验
系统 SHALL 支持 udp 专属 expect，包括 `responded`、`response`、`responseSize`、`sourceHost`、`sourcePort` 和 `maxDurationMs`，并按 responded、responseSize、response、sourceHost、sourcePort、duration 的阶段顺序快速失败。 系统 SHALL 支持 udp 专属 expect，包括 `responded`、`response`、`responseSize`、`sourceHost`、`sourcePort` 和 `durationMs`，并按 responded、responseSize、response、sourceHost、sourcePort、durationMs 的阶段顺序快速失败。`responded` SHALL 保持布尔状态语义，未配置时默认 `true`。`response` MUST 使用共享 `ContentRules` 数组，并作用于按 `udp.responseEncoding` 转换后的响应文本。`responseSize`、`sourceHost`、`sourcePort` 和 `durationMs` SHALL 使用共享 `ValueMatcher`。
#### Scenario: 默认 responded 成功语义 #### Scenario: 默认 responded 成功语义
- **WHEN** udp target 未显式配置 `expect.responded` - **WHEN** udp target 未显式配置 `expect.responded`
- **THEN** 系统 SHALL 使用默认 `expect.responded: true` 进行校验 - **THEN** 系统 SHALL 使用默认 `expect.responded: true` 进行校验
#### Scenario: response text rules 校验通过 #### Scenario: response ContentRules 校验通过
- **WHEN** udp target 配置 `expect.response: [{ contains: "PONG" }]`，且按 `responseEncoding` 转换后的响应文本包含 `PONG` - **WHEN** udp target 配置 `expect.response: [{ contains: "PONG" }]`，且按 `responseEncoding` 转换后的响应文本包含 `PONG`
- **THEN** 系统 SHALL 判定 response 阶段通过 - **THEN** 系统 SHALL 判定 response 阶段通过
#### Scenario: response text rules 校验失败 #### Scenario: response JSON 校验通过
- **WHEN** udp target 收到文本响应 `{"status":"ok"}` 且配置 `expect.response: [{json: {path: "$.status", equals: "ok"}}]`
- **THEN** 系统 SHALL 判定 response 阶段通过
#### Scenario: response ContentRules 校验失败
- **WHEN** udp target 配置 `expect.response: [{ contains: "PONG" }]`，但按 `responseEncoding` 转换后的响应文本不包含 `PONG` - **WHEN** udp target 配置 `expect.response: [{ contains: "PONG" }]`，但按 `responseEncoding` 转换后的响应文本不包含 `PONG`
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `response`，path 指向失败的 response 规则 - **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `response`，path 指向失败的 response 规则
@@ -151,28 +155,24 @@
- **WHEN** udp target 配置 `udp.responseEncoding: "hex"` 且收到字节内容 `PONG` - **WHEN** udp target 配置 `udp.responseEncoding: "hex"` 且收到字节内容 `PONG`
- **THEN** 系统 SHALL 将响应转换为小写 hex 字符串 `504f4e47` 后执行 `expect.response` 规则 - **THEN** 系统 SHALL 将响应转换为小写 hex 字符串 `504f4e47` 后执行 `expect.response` 规则
#### Scenario: responseEncoding 为 base64 #### Scenario: responseSize matcher 校验通过
- **WHEN** udp target 配置 `udp.responseEncoding: "base64"` 且收到字节内容 `PONG`
- **THEN** 系统 SHALL 将响应转换为 base64 字符串 `UE9ORw==` 后执行 `expect.response` 规则
#### Scenario: responseSize operator 校验通过
- **WHEN** udp target 配置 `expect.responseSize: { gte: 4 }`，且实际响应为 4 字节 - **WHEN** udp target 配置 `expect.responseSize: { gte: 4 }`，且实际响应为 4 字节
- **THEN** 系统 SHALL 判定 responseSize 阶段通过 - **THEN** 系统 SHALL 判定 responseSize 阶段通过
#### Scenario: responseSize operator 校验失败 #### Scenario: responseSize matcher 校验失败
- **WHEN** udp target 配置 `expect.responseSize: { gte: 4 }`，但实际响应为 2 字节 - **WHEN** udp target 配置 `expect.responseSize: { gte: 4 }`，但实际响应为 2 字节
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `responseSize` - **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `responseSize`
#### Scenario: sourceHost operator 校验 #### Scenario: sourceHost matcher 校验
- **WHEN** udp target 配置 `expect.sourceHost: { equals: "127.0.0.1" }`，且 Bun 回调中的来源地址为 `127.0.0.1` - **WHEN** udp target 配置 `expect.sourceHost: { equals: "127.0.0.1" }`，且 Bun 回调中的来源地址为 `127.0.0.1`
- **THEN** 系统 SHALL 判定 sourceHost 阶段通过 - **THEN** 系统 SHALL 判定 sourceHost 阶段通过
#### Scenario: sourcePort operator 校验 #### Scenario: sourcePort matcher 校验
- **WHEN** udp target 配置 `expect.sourcePort: { equals: 9000 }`，且 Bun 回调中的来源端口为 `9000` - **WHEN** udp target 配置 `expect.sourcePort: { equals: 9000 }`，且 Bun 回调中的来源端口为 `9000`
- **THEN** 系统 SHALL 判定 sourcePort 阶段通过 - **THEN** 系统 SHALL 判定 sourcePort 阶段通过
#### Scenario: maxDurationMs 校验 #### Scenario: durationMs 校验
- **WHEN** udp target 配置 `expect.maxDurationMs: 100`，且完整执行耗时超过 100ms - **WHEN** udp target 配置 `expect.durationMs: {lte: 100}`，且完整执行耗时超过 100ms
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `duration` - **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `duration`
#### Scenario: response 断言要求实际有响应 #### Scenario: response 断言要求实际有响应
@@ -184,7 +184,7 @@
- **THEN** 系统 SHALL 在启动期配置校验失败，提示响应来源断言需要 `expect.responded` 为 true - **THEN** 系统 SHALL 在启动期配置校验失败，提示响应来源断言需要 `expect.responded` 为 true
#### Scenario: udp expect 未知字段失败 #### Scenario: udp expect 未知字段失败
- **WHEN** YAML 中 udp target 的 expect 包含 `status: [200]` 或其他非 udp expect 字段 - **WHEN** YAML 中 udp target 的 expect 包含 `status: [200]`、`maxDurationMs: 1000` 或其他非 udp expect 字段
- **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段 - **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段
### Requirement: udp statusDetail 摘要 ### Requirement: udp statusDetail 摘要

2369
probe-config.schema.json
View File

File diff suppressed because it is too large Load Diff

49
probes.example.yaml
View File

@@ -34,7 +34,8 @@ targets:
url: "https://www.baidu.com" url: "https://www.baidu.com"
expect: expect:
status: [200] status: [200]
maxDurationMs: 5000 durationMs:
lte: 5000
- id: "httpbin-json" - id: "httpbin-json"
name: "${env_name} JSON API — 完整流水线" name: "${env_name} JSON API — 完整流水线"
@@ -51,7 +52,8 @@ targets:
headers: headers:
Content-Type: Content-Type:
contains: "application/json" contains: "application/json"
maxDurationMs: 8000 durationMs:
lte: 8000
body: body:
- json: - json:
path: "$.slideshow.title" path: "$.slideshow.title"
@@ -92,7 +94,7 @@ targets:
expect: expect:
exitCode: [0] exitCode: [0]
stdout: stdout:
- match: "^\\d+\\.\\d+\\.\\d+" - regex: "^\\d+\\.\\d+\\.\\d+"
- id: "bun-stdout-rules" - id: "bun-stdout-rules"
name: "多规则 stdout 顺序校验" name: "多规则 stdout 顺序校验"
@@ -104,7 +106,7 @@ targets:
expect: expect:
stdout: stdout:
- contains: "version:" - contains: "version:"
- match: "\\d+\\.\\d+\\.\\d+" - regex: "\\d+\\.\\d+\\.\\d+"
- contains: "healthy" - contains: "healthy"
- id: "bun-stderr" - id: "bun-stderr"
@@ -127,7 +129,8 @@ targets:
db: db:
url: "${sqlite_url}" url: "${sqlite_url}"
expect: expect:
maxDurationMs: 1000 durationMs:
lte: 1000
- id: "sqlite-query" - id: "sqlite-query"
name: "SQLite 内存数据库多列结果校验" name: "SQLite 内存数据库多列结果校验"
@@ -145,6 +148,10 @@ targets:
exists: true exists: true
role: role:
contains: "engineer" contains: "engineer"
result:
- json:
path: "$.rows[0].role"
equals: "engineer"
# ========== TCP targets ========== # ========== TCP targets ==========
@@ -156,7 +163,8 @@ targets:
host: "127.0.0.1" host: "127.0.0.1"
port: 6379 port: 6379
expect: expect:
maxDurationMs: 3000 durationMs:
lte: 3000
- id: "smtp-banner" - id: "smtp-banner"
name: "SMTP Banner 探测" name: "SMTP Banner 探测"
@@ -169,7 +177,7 @@ targets:
bannerReadTimeout: 3000 bannerReadTimeout: 3000
expect: expect:
banner: banner:
contains: "ESMTP" - contains: "ESMTP"
# ========== Ping targets ========== # ========== Ping targets ==========
@@ -183,10 +191,14 @@ targets:
packetSize: 56 packetSize: 56
expect: expect:
alive: true alive: true
maxPacketLoss: 10 packetLossPercent:
maxAvgLatencyMs: 100 lte: 10
maxMaxLatencyMs: 300 avgLatencyMs:
maxDurationMs: 5000 lte: 100
maxLatencyMs:
lte: 300
durationMs:
lte: 5000
# ========== UDP targets ========== # ========== UDP targets ==========
@@ -201,7 +213,8 @@ targets:
expect: expect:
response: response:
- contains: "PONG" - contains: "PONG"
maxDurationMs: 100 durationMs:
lte: 100
- id: "udp-binary-probe" - id: "udp-binary-probe"
name: "UDP 二进制协议探测" name: "UDP 二进制协议探测"
@@ -216,7 +229,8 @@ targets:
expect: expect:
responseSize: responseSize:
gte: 4 gte: 4
maxDurationMs: 200 durationMs:
lte: 200
- id: "udp-fire-and-forget" - id: "udp-fire-and-forget"
name: "UDP 发送验证（不等待响应）" name: "UDP 发送验证（不等待响应）"
@@ -242,7 +256,8 @@ targets:
expect: expect:
status: status:
- 200 - 200
finishReason: "stop" finishReason:
equals: "stop"
output: output:
- contains: "OK" - contains: "OK"
@@ -264,5 +279,7 @@ targets:
completed: true completed: true
firstTokenMs: firstTokenMs:
lte: 5000 lte: 5000
finishReason: "stop" finishReason:
maxDurationMs: 15000 equals: "stop"
durationMs:
lte: 15000

175
src/server/checker/expect/content.ts Normal file
View File

@@ -0,0 +1,175 @@
import { DOMParser } from "@xmldom/xmldom";
import * as cheerio from "cheerio";
import { isArray } from "es-toolkit/compat";
import * as xpath from "xpath";
import type { CheckFailure } from "../types";
import type {
ContentCssRule,
ContentJsonRule,
ContentRule,
ContentRules,
ContentXpathRule,
ExpectResult,
} from "./types";
import { errorFailure, mismatchFailure } from "./failure";
import { applyMatcher, evaluateJsonPath } from "./matcher";
type ParsedJsonResult = { error: string; ok: false } | { ok: true; value: unknown };
export function checkContentRules(
source: unknown,
rules: ContentRules | undefined,
options: { path?: string; phase: CheckFailure["phase"] },
): ExpectResult {
if (!rules || rules.length === 0) return { failure: null, matched: true };
const basePath = options.path ?? options.phase;
let parsedJson: ParsedJsonResult | undefined;
for (let i = 0; i < rules.length; i++) {
const rule = rules[i]!;
if ("json" in rule && parsedJson === undefined) {
parsedJson = parseJsonSource(source);
}
const result = checkSingleContentRule(source, rule, `${basePath}[${i}]`, options.phase, parsedJson);
if (!result.matched) return result;
}
return { failure: null, matched: true };
}
function checkCssRule(
source: unknown,
rule: ContentCssRule,
rulePath: string,
phase: CheckFailure["phase"],
): ExpectResult {
const { attr, selector, ...matcher } = rule;
const fullPath = `${rulePath}.css(${selector}${attr ? `@${attr}` : ""})`;
let $: cheerio.CheerioAPI;
try {
$ = cheerio.load(contentText(source));
} catch {
return { failure: errorFailure(phase, fullPath, "failed to parse HTML"), matched: false };
}
const el = $(selector).first();
const actual = el.length === 0 ? undefined : attr ? el.attr(attr) : el.text();
const effectiveMatcher = Object.keys(matcher).length === 0 ? { exists: true } : matcher;
if (!applyMatcher(actual, effectiveMatcher)) {
return {
failure: mismatchFailure(phase, fullPath, effectiveMatcher, actual, `css selector ${selector} mismatch`),
matched: false,
};
}
return { failure: null, matched: true };
}
function checkJsonRule(
rule: ContentJsonRule,
rulePath: string,
phase: CheckFailure["phase"],
parsedJson?: ParsedJsonResult,
): ExpectResult {
const { path, ...matcher } = rule;
const fullPath = `${rulePath}.json(${path})`;
if (!parsedJson?.ok) {
return { failure: errorFailure(phase, fullPath, parsedJson?.error ?? "content is not valid JSON"), matched: false };
}
const actual = evaluateJsonPath(parsedJson.value, path);
const effectiveMatcher = Object.keys(matcher).length === 0 ? { exists: true } : matcher;
if (!applyMatcher(actual, effectiveMatcher)) {
return {
failure: mismatchFailure(phase, fullPath, effectiveMatcher, actual, `json path ${path} mismatch`),
matched: false,
};
}
return { failure: null, matched: true };
}
function checkSingleContentRule(
source: unknown,
rule: ContentRule,
rulePath: string,
phase: CheckFailure["phase"],
parsedJson?: ParsedJsonResult,
): ExpectResult {
if ("json" in rule) return checkJsonRule(rule.json, rulePath, phase, parsedJson);
if ("css" in rule) return checkCssRule(source, rule.css, rulePath, phase);
if ("xpath" in rule) return checkXpathRule(source, rule.xpath, rulePath, phase);
if (!applyMatcher(source, rule, { stringifyNonString: true })) {
return {
failure: mismatchFailure(phase, rulePath, rule, source, `${phase} rule mismatch`),
matched: false,
};
}
return { failure: null, matched: true };
}
function checkXpathRule(
source: unknown,
rule: ContentXpathRule,
rulePath: string,
phase: CheckFailure["phase"],
): ExpectResult {
const { path, ...matcher } = rule;
const fullPath = `${rulePath}.xpath(${path})`;
let doc: ReturnType<DOMParser["parseFromString"]>;
try {
doc = new DOMParser().parseFromString(contentText(source), "text/xml");
} catch {
return { failure: errorFailure(phase, fullPath, "failed to parse XML/HTML"), matched: false };
}
const result = xpath.select(path, doc as unknown as Node);
const actual = xpathValue(result);
const effectiveMatcher = Object.keys(matcher).length === 0 ? { exists: true } : matcher;
if (!applyMatcher(actual, effectiveMatcher)) {
return {
failure: mismatchFailure(phase, fullPath, effectiveMatcher, actual, `xpath ${path} mismatch`),
matched: false,
};
}
return { failure: null, matched: true };
}
function contentText(source: unknown): string {
if (source === null || source === undefined) return "";
if (typeof source === "string") return source;
if (typeof source === "number" || typeof source === "boolean" || typeof source === "bigint") return String(source);
if (typeof source === "symbol") return source.description ?? "";
if (typeof source === "function") return source.name;
return JSON.stringify(source) ?? "";
}
function parseJsonSource(source: unknown): ParsedJsonResult {
if (typeof source !== "string") return { ok: true, value: source };
try {
return { ok: true, value: JSON.parse(source) as unknown };
} catch {
return { error: "content is not valid JSON", ok: false };
}
}
function xpathValue(result: unknown): unknown {
if (!isArray(result)) return result;
if (result.length === 0) return undefined;
const node = (result as unknown[])[0]!;
if (typeof node !== "object" || node === null) return node;
const asNode = node as Node;
return asNode.nodeValue ?? (asNode as unknown as Element).textContent ?? "";
}

20
src/server/checker/expect/duration.ts
View File

@@ -1,20 +0,0 @@
import type { ExpectResult } from "./types";
import { mismatchFailure } from "./failure";
export function checkDuration(durationMs: number, maxDurationMs?: number): ExpectResult {
if (maxDurationMs === undefined) return { failure: null, matched: true };
if (durationMs > maxDurationMs) {
return {
failure: mismatchFailure(
"duration",
"duration",
`<=${maxDurationMs}ms`,
durationMs,
`duration ${durationMs}ms > ${maxDurationMs}ms`,
),
matched: false,
};
}
return { failure: null, matched: true };
}

32
src/server/checker/expect/key-value.ts Normal file
View File

@@ -0,0 +1,32 @@
import type { CheckFailure } from "../types";
import type { ExpectResult, KeyValueExpect } from "./types";
import { mismatchFailure } from "./failure";
import { checkExpectValue } from "./matcher";
export function checkKeyValueExpect(
actual: Record<string, unknown>,
expected: KeyValueExpect | undefined,
options: { normalizeKey?: (key: string) => string; path?: string; phase: CheckFailure["phase"] },
): ExpectResult {
if (!expected) return { failure: null, matched: true };
const normalizeKey = options.normalizeKey ?? ((key: string) => key);
const basePath = options.path ?? options.phase;
const actualMap = new Map<string, unknown>();
for (const [key, value] of Object.entries(actual)) {
actualMap.set(normalizeKey(key), value);
}
for (const [key, expectedValue] of Object.entries(expected)) {
const actualValue = actualMap.get(normalizeKey(key));
if (!checkExpectValue(actualValue, expectedValue)) {
return {
failure: mismatchFailure(options.phase, `${basePath}.${key}`, expectedValue, actualValue, `${key} mismatch`),
matched: false,
};
}
}
return { failure: null, matched: true };
}

133
src/server/checker/expect/matcher.ts Normal file
View File

@@ -0,0 +1,133 @@
import { isEmptyObject, isEqual, isNil, isPlainObject } from "es-toolkit";
import { isArray } from "es-toolkit/compat";
import type { CheckFailure, JsonValue } from "../types";
import type { ExpectResult, ValueMatcher } from "./types";
import { mismatchFailure } from "./failure";
export const MatcherKeys = ["contains", "empty", "equals", "exists", "gt", "gte", "lt", "lte", "regex"] as const;
const MATCHER_KEY_SET = new Set<string>(MatcherKeys);
export function applyMatcher(
actual: unknown,
matcher: ValueMatcher,
options: { stringifyNonString?: boolean } = {},
): boolean {
for (const [key, expected] of Object.entries(matcher)) {
if (expected === undefined) continue;
switch (key) {
case "contains":
if (!stringValue(actual, options).includes(expected as string)) return false;
break;
case "empty": {
const empty = isEmptyValue(actual);
if (expected !== empty) return false;
break;
}
case "equals":
if (!isEqual(actual, expected)) return false;
break;
case "exists":
if (expected) {
if (actual === undefined) return false;
} else {
if (actual !== undefined) return false;
}
break;
case "gt":
if (!compareNumber(actual, expected as number, (left, right) => left > right)) return false;
break;
case "gte":
if (!compareNumber(actual, expected as number, (left, right) => left >= right)) return false;
break;
case "lt":
if (!compareNumber(actual, expected as number, (left, right) => left < right)) return false;
break;
case "lte":
if (!compareNumber(actual, expected as number, (left, right) => left <= right)) return false;
break;
case "regex":
if (!new RegExp(expected as string).test(stringValue(actual, options))) return false;
break;
}
}
return true;
}
export function checkExpectValue(actual: unknown, expected: JsonValue | ValueMatcher): boolean {
if (isValueMatcherObject(expected)) {
return applyMatcher(actual, expected);
}
return applyMatcher(actual, { equals: expected });
}
export function checkValueMatcher(
actual: unknown,
matcher: undefined | ValueMatcher,
options: { message?: string; path: string; phase: CheckFailure["phase"]; stringifyNonString?: boolean },
): ExpectResult {
if (matcher === undefined) return { failure: null, matched: true };
if (applyMatcher(actual, matcher, { stringifyNonString: options.stringifyNonString })) {
return { failure: null, matched: true };
}
return {
failure: mismatchFailure(
options.phase,
options.path,
matcher,
actual,
options.message ?? `${options.path} mismatch`,
),
matched: false,
};
}
export function evaluateJsonPath(json: unknown, path: string): unknown {
if (!path.startsWith("$.")) return undefined;
const segments = path.slice(2).split(".");
let current: unknown = json;
for (const seg of segments) {
const bracketMatch = /^(.+?)\[(\d+)\]$/.exec(seg);
if (bracketMatch) {
if (current === null || current === undefined) return undefined;
current = (current as Record<string, unknown>)[bracketMatch[1]!];
const idx = parseInt(bracketMatch[2]!, 10);
if (!isArray(current) || idx >= current.length) return undefined;
current = current[idx];
} else {
if (current === null || current === undefined) return undefined;
current = (current as Record<string, unknown>)[seg];
}
}
return current;
}
export function isValueMatcherObject(value: unknown): value is ValueMatcher {
return isPlainObject(value) && Object.keys(value).some((key) => MATCHER_KEY_SET.has(key));
}
function compareNumber(
actual: unknown,
expected: number,
compare: (actual: number, expected: number) => boolean,
): boolean {
const value = Number(actual);
return Number.isFinite(value) && compare(value, expected);
}
function isEmptyValue(value: unknown): boolean {
return isNil(value) || value === "" || (isArray(value) && value.length === 0) || isEmptyObject(value);
}
function stringValue(actual: unknown, options: { stringifyNonString?: boolean }): string {
if (!options.stringifyNonString || typeof actual === "string") return String(actual);
if (actual !== null && typeof actual === "object") return JSON.stringify(actual);
return String(actual);
}

80
src/server/checker/expect/operator.ts
View File

@@ -1,80 +0,0 @@
import { isEmptyObject, isEqual, isNil, isPlainObject } from "es-toolkit";
import { isArray } from "es-toolkit/compat";
import type { ExpectOperator, ExpectValue } from "../types";
const OPERATOR_KEYS = new Set(["contains", "empty", "equals", "exists", "gt", "gte", "lt", "lte", "match"]);
export function applyOperator(actual: unknown, op: ExpectOperator): boolean {
for (const [key, expected] of Object.entries(op)) {
if (expected === undefined) continue;
switch (key) {
case "contains":
if (!String(actual).includes(expected as string)) return false;
break;
case "empty": {
const isEmpty =
isNil(actual) || actual === "" || (isArray(actual) && actual.length === 0) || isEmptyObject(actual);
if (expected !== isEmpty) return false;
break;
}
case "equals":
if (!isEqual(actual, expected)) return false;
break;
case "exists":
if (expected) {
if (actual === undefined) return false;
} else {
if (actual !== undefined) return false;
}
break;
case "gt":
if (!(Number(actual) > (expected as number))) return false;
break;
case "gte":
if (!(Number(actual) >= (expected as number))) return false;
break;
case "lt":
if (!(Number(actual) < (expected as number))) return false;
break;
case "lte":
if (!(Number(actual) <= (expected as number))) return false;
break;
case "match":
if (!new RegExp(expected as string).test(String(actual))) return false;
break;
}
}
return true;
}
export function checkExpectValue(actual: unknown, expected: ExpectValue): boolean {
if (isPlainObject(expected) && Object.keys(expected).some((key) => OPERATOR_KEYS.has(key))) {
return applyOperator(actual, expected as ExpectOperator);
}
return applyOperator(actual, { equals: expected as Exclude<ExpectValue, ExpectOperator> });
}
export function evaluateJsonPath(json: unknown, path: string): unknown {
if (!path.startsWith("$.")) return undefined;
const segments = path.slice(2).split(".");
let current: unknown = json;
for (const seg of segments) {
const bracketMatch = /^(.+?)\[(\d+)\]$/.exec(seg);
if (bracketMatch) {
current = (current as Record<string, unknown>)?.[bracketMatch[1]!];
const idx = parseInt(bracketMatch[2]!, 10);
if (!isArray(current) || idx >= current.length) return undefined;
current = current[idx];
} else {
if (current === null || current === undefined) return undefined;
current = (current as Record<string, unknown>)[seg];
}
}
return current;
}

37
src/server/checker/expect/types.ts
View File

@@ -1,6 +1,41 @@
import type { CheckFailure } from "../types"; import type { CheckFailure, JsonValue } from "../types";
export interface ContentCssRule extends ValueMatcher {
attr?: string;
selector: string;
}
export interface ContentJsonRule extends ValueMatcher {
path: string;
}
export type ContentRule =
| ValueMatcher
| { css: ContentCssRule }
| { json: ContentJsonRule }
| { xpath: ContentXpathRule };
export type ContentRules = ContentRule[];
export interface ContentXpathRule extends ValueMatcher {
path: string;
}
export interface ExpectResult { export interface ExpectResult {
failure: CheckFailure | null; failure: CheckFailure | null;
matched: boolean; matched: boolean;
} }
export type KeyValueExpect = Record<string, JsonValue | ValueMatcher>;
export interface ValueMatcher {
contains?: string;
empty?: boolean;
equals?: JsonValue;
exists?: boolean;
gt?: number;
gte?: number;
lt?: number;
lte?: number;
regex?: string;
}

225
src/server/checker/expect/validate-matcher.ts Normal file
View File

@@ -0,0 +1,225 @@
import { DOMParser } from "@xmldom/xmldom";
import { isBoolean, isNumber, isPlainObject, isString } from "es-toolkit";
import { isArray } from "es-toolkit/compat";
import * as xpath from "xpath";
import type { ConfigValidationIssue } from "../schema/issues";
import type { JsonValue } from "../types";
import { issue, joinPath } from "../schema/issues";
import { isUnsafeRegex } from "./redos";
export const MatcherKeys = ["contains", "empty", "equals", "exists", "gt", "gte", "lt", "lte", "regex"] as const;
const MATCHER_KEY_SET = new Set<string>(MatcherKeys);
const EXTRACTOR_KEYS = ["css", "json", "xpath"] as const;
const EXTRACTOR_KEY_SET = new Set<string>(EXTRACTOR_KEYS);
export function isJsonValue(value: unknown): value is JsonValue {
if (value === null) return true;
if (isString(value) || isBoolean(value)) return true;
if (isNumber(value)) return Number.isFinite(value);
if (isArray(value)) return value.every(isJsonValue);
if (isPlainObject(value)) return Object.values(value).every(isJsonValue);
return false;
}
export function isPlainRecord(value: unknown): value is Record<string, unknown> {
return isPlainObject(value);
}
export function validateContentRules(rules: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isArray(rules)) return [issue("invalid-type", path, "必须为数组", targetName)];
return rules.flatMap((rule, index) => validateContentRule(rule, `${path}[${index}]`, targetName));
}
export function validateJsonPath(path: string, rulePath: string, targetName?: string): ConfigValidationIssue[] {
if (!path.startsWith("$.") || path.length <= 2) {
return [issue("invalid-jsonpath", joinPath(rulePath, "path"), '必须为以 "$." 开头的有效 JSONPath', targetName)];
}
const issues: ConfigValidationIssue[] = [];
const segments = path.slice(2).split(".");
for (const seg of segments) {
if (seg === "") {
issues.push(issue("invalid-jsonpath", joinPath(rulePath, "path"), "包含空段", targetName));
}
const bracketMatch = /^(.+?)\[(\d+)\]$/.exec(seg);
if (bracketMatch?.[1]!.trim() === "") {
issues.push(issue("invalid-jsonpath", joinPath(rulePath, "path"), "数组访问缺少属性名", targetName));
}
}
return issues;
}
export function validateKeyValueExpect(value: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isPlainRecord(value)) return [issue("invalid-type", path, "必须为对象", targetName)];
const issues: ConfigValidationIssue[] = [];
for (const [key, item] of Object.entries(value)) {
const itemPath = joinPath(path, key);
if (isPlainRecord(item)) {
issues.push(...validateValueMatcher(item, itemPath, targetName));
} else if (!isJsonValue(item)) {
issues.push(issue("invalid-type", itemPath, "必须为 JSON value 或 matcher 对象", targetName));
}
}
return issues;
}
export function validateValueMatcher(
matcher: unknown,
path: string,
targetName?: string,
options: { requireAtLeastOne?: boolean } = {},
): ConfigValidationIssue[] {
const requireAtLeastOne = options.requireAtLeastOne ?? true;
if (!isPlainRecord(matcher)) return [issue("invalid-type", path, "必须为 matcher 对象", targetName)];
const issues: ConfigValidationIssue[] = [];
let found = 0;
for (const [key, value] of Object.entries(matcher)) {
if (!MATCHER_KEY_SET.has(key)) {
issues.push(issue("unknown-matcher", joinPath(path, key), "是未知 matcher", targetName));
continue;
}
if (value === undefined) continue;
found++;
issues.push(...validateMatcherValue(key, value, joinPath(path, key), targetName));
}
if (requireAtLeastOne && found === 0) {
issues.push(issue("empty-matcher", path, "必须包含至少一个合法 matcher", targetName));
}
if (matcher["exists"] === false && found > 1) {
issues.push(issue("invalid-value", joinPath(path, "exists"), "exists:false 不能与其他 matcher 组合", targetName));
}
return issues;
}
function validateContentRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
const issues: ConfigValidationIssue[] = [];
const extractors = Object.keys(rule).filter((key) => EXTRACTOR_KEY_SET.has(key));
const directMatchers = Object.keys(rule).filter((key) => MATCHER_KEY_SET.has(key));
for (const key of Object.keys(rule)) {
if (!MATCHER_KEY_SET.has(key) && !EXTRACTOR_KEY_SET.has(key)) {
issues.push(issue("unknown-field", joinPath(path, key), "是未知字段", targetName));
}
}
if (extractors.length > 1) {
issues.push(issue("multiple-content-rules", path, "一条规则不能同时包含多个 extractor", targetName));
}
if (extractors.length === 1 && directMatchers.length > 0) {
issues.push(issue("invalid-content-rule", path, "直接 matcher 不能与 extractor 混用", targetName));
}
if (issues.length > 0) return issues;
if (extractors.length === 0) return validateValueMatcher(rule, path, targetName);
const extractor = extractors[0]!;
switch (extractor) {
case "css":
return validateCssRule(rule["css"], joinPath(path, "css"), targetName);
case "json":
return validateJsonRule(rule["json"], joinPath(path, "json"), targetName);
case "xpath":
return validateXpathRule(rule["xpath"], joinPath(path, "xpath"), targetName);
}
return [];
}
function validateCssRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
const issues: ConfigValidationIssue[] = [];
if (!isString(rule["selector"]) || rule["selector"].trim() === "") {
issues.push(issue("invalid-type", joinPath(path, "selector"), "必须为非空字符串", targetName));
}
if ("attr" in rule && !isString(rule["attr"])) {
issues.push(issue("invalid-type", joinPath(path, "attr"), "必须为字符串", targetName));
}
issues.push(...validateExtractorMatcher(rule, new Set(["attr", "selector"]), path, targetName));
return issues;
}
function validateExtractorMatcher(
rule: Record<string, unknown>,
allowedFields: Set<string>,
path: string,
targetName?: string,
): ConfigValidationIssue[] {
const matcher: Record<string, unknown> = {};
const issues: ConfigValidationIssue[] = [];
for (const [key, value] of Object.entries(rule)) {
if (allowedFields.has(key)) continue;
matcher[key] = value;
}
issues.push(...validateValueMatcher(matcher, path, targetName, { requireAtLeastOne: false }));
return issues;
}
function validateJsonRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
const issues: ConfigValidationIssue[] = [];
if (!isString(rule["path"])) {
issues.push(issue("invalid-type", joinPath(path, "path"), "必须为字符串", targetName));
} else {
issues.push(...validateJsonPath(rule["path"], path, targetName));
}
issues.push(...validateExtractorMatcher(rule, new Set(["path"]), path, targetName));
return issues;
}
function validateMatcherValue(key: string, value: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
switch (key) {
case "contains":
return isString(value) ? [] : [issue("invalid-type", path, "必须为字符串", targetName)];
case "empty":
case "exists":
return isBoolean(value) ? [] : [issue("invalid-type", path, "必须为布尔值", targetName)];
case "equals":
return isJsonValue(value) ? [] : [issue("invalid-type", path, "必须为 JSON value", targetName)];
case "gt":
case "gte":
case "lt":
case "lte":
return isNumber(value) && Number.isFinite(value)
? []
: [issue("invalid-type", path, "必须为有限数字", targetName)];
case "regex":
if (!isString(value)) return [issue("invalid-type", path, "必须为字符串", targetName)];
try {
new RegExp(value);
} catch {
return [issue("invalid-regex", path, "正则不合法", targetName)];
}
return isUnsafeRegex(value) ? [issue("unsafe-regex", path, "正则存在 ReDoS 风险", targetName)] : [];
default:
return [issue("unknown-matcher", path, "是未知 matcher", targetName)];
}
}
function validateXpathRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
const issues: ConfigValidationIssue[] = [];
if (!isString(rule["path"]) || rule["path"].trim() === "") {
issues.push(issue("invalid-type", joinPath(path, "path"), "必须为非空字符串", targetName));
} else {
try {
const doc = new DOMParser().parseFromString("<x/>", "text/xml");
xpath.select(rule["path"], doc as unknown as Node);
} catch {
issues.push(issue("invalid-xpath", joinPath(path, "path"), "xpath 不合法", targetName));
}
}
issues.push(...validateExtractorMatcher(rule, new Set(["path"]), path, targetName));
return issues;
}

84
src/server/checker/expect/validate-operator.ts
View File

@@ -1,84 +0,0 @@
import { isBoolean, isNumber, isPlainObject, isString } from "es-toolkit";
import { isArray } from "es-toolkit/compat";
import type { ConfigValidationIssue } from "../schema/issues";
import type { JsonValue } from "../types";
import { OperatorKeys } from "../schema/fragments";
import { issue, joinPath } from "../schema/issues";
import { isUnsafeRegex } from "./redos";
const OPERATOR_KEY_SET = new Set<string>(OperatorKeys);
export function isJsonValue(value: unknown): value is JsonValue {
if (value === null) return true;
if (isString(value) || isBoolean(value)) return true;
if (isNumber(value)) return Number.isFinite(value);
if (isArray(value)) return value.every(isJsonValue);
if (isPlainObject(value)) {
return Object.values(value).every(isJsonValue);
}
return false;
}
export function isPlainRecord(value: unknown): value is Record<string, unknown> {
return isPlainObject(value);
}
export function validateOperatorObject(
operators: unknown,
path: string,
targetName?: string,
options: { requireAtLeastOne: boolean } = { requireAtLeastOne: true },
): ConfigValidationIssue[] {
if (!isPlainRecord(operators)) return [issue("invalid-type", path, "必须为操作符对象", targetName)];
const issues: ConfigValidationIssue[] = [];
let found = 0;
for (const [key, value] of Object.entries(operators)) {
if (!OPERATOR_KEY_SET.has(key)) {
issues.push(issue("unknown-operator", joinPath(path, key), "是未知 operator", targetName));
continue;
}
if (value === undefined) continue;
found++;
issues.push(...validateOperatorValue(key, value, joinPath(path, key), targetName));
}
if (options.requireAtLeastOne && found === 0) {
issues.push(issue("empty-operator", path, "必须包含至少一个合法 operator", targetName));
}
return issues;
}
export function validateOperatorValue(
key: string,
value: unknown,
path: string,
targetName?: string,
): ConfigValidationIssue[] {
switch (key) {
case "contains":
return isString(value) ? [] : [issue("invalid-type", path, "必须为字符串", targetName)];
case "empty":
case "exists":
return isBoolean(value) ? [] : [issue("invalid-type", path, "必须为布尔值", targetName)];
case "equals":
return isJsonValue(value) ? [] : [issue("invalid-type", path, "必须为 JSON value", targetName)];
case "gt":
case "gte":
case "lt":
case "lte":
return isNumber(value) && Number.isFinite(value)
? []
: [issue("invalid-type", path, "必须为有限数字", targetName)];
case "match":
if (!isString(value)) return [issue("invalid-type", path, "必须为字符串", targetName)];
try {
new RegExp(value);
} catch {
return [issue("invalid-regex", path, "正则不合法", targetName)];
}
return isUnsafeRegex(value) ? [issue("unsafe-regex", path, "正则存在 ReDoS 风险", targetName)] : [];
default:
return [issue("unknown-operator", path, "是未知 operator", targetName)];
}
}

14
src/server/checker/runner/cmd/execute.ts
View File

@@ -5,12 +5,12 @@ import type { CheckResult, RawTargetConfig } from "../../types";
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types"; import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
import type { CommandExpectConfig, CommandTargetConfig, ResolvedCommandTarget } from "./types"; import type { CommandExpectConfig, CommandTargetConfig, ResolvedCommandTarget } from "./types";
import { checkDuration } from "../../expect/duration"; import { checkContentRules } from "../../expect/content";
import { errorFailure } from "../../expect/failure"; import { errorFailure } from "../../expect/failure";
import { checkValueMatcher } from "../../expect/matcher";
import { parseSize } from "../../utils"; import { parseSize } from "../../utils";
import { checkExitCode } from "./expect"; import { checkExitCode } from "./expect";
import { commandCheckerSchemas } from "./schema"; import { commandCheckerSchemas } from "./schema";
import { checkTextRules } from "./text";
import { validateCommandConfig } from "./validate"; import { validateCommandConfig } from "./validate";
export class CommandChecker implements CheckerDefinition<ResolvedCommandTarget> { export class CommandChecker implements CheckerDefinition<ResolvedCommandTarget> {
@@ -118,7 +118,11 @@ export class CommandChecker implements CheckerDefinition<ResolvedCommandTarget>
}; };
} }
const durationResult = checkDuration(durationMs, t.expect?.maxDurationMs); const durationResult = checkValueMatcher(durationMs, t.expect?.durationMs, {
message: "durationMs mismatch",
path: "durationMs",
phase: "duration",
});
if (!durationResult.matched) { if (!durationResult.matched) {
return { return {
durationMs, durationMs,
@@ -131,7 +135,7 @@ export class CommandChecker implements CheckerDefinition<ResolvedCommandTarget>
} }
if (t.expect?.stdout && t.expect.stdout.length > 0) { if (t.expect?.stdout && t.expect.stdout.length > 0) {
const stdoutResult = checkTextRules(outputResult.stdout, t.expect.stdout, "stdout"); const stdoutResult = checkContentRules(outputResult.stdout, t.expect.stdout, { path: "stdout", phase: "stdout" });
if (!stdoutResult.matched) { if (!stdoutResult.matched) {
return { return {
durationMs, durationMs,
@@ -145,7 +149,7 @@ export class CommandChecker implements CheckerDefinition<ResolvedCommandTarget>
} }
if (t.expect?.stderr && t.expect.stderr.length > 0) { if (t.expect?.stderr && t.expect.stderr.length > 0) {
const stderrResult = checkTextRules(outputResult.stderr, t.expect.stderr, "stderr"); const stderrResult = checkContentRules(outputResult.stderr, t.expect.stderr, { path: "stderr", phase: "stderr" });
if (!stderrResult.matched) { if (!stderrResult.matched) {
return { return {
durationMs, durationMs,

13
src/server/checker/runner/cmd/schema.ts
View File

@@ -2,7 +2,12 @@ import { Type } from "@sinclair/typebox";
import type { CheckerSchemas } from "../types"; import type { CheckerSchemas } from "../types";
import { createTextRulesSchema, sizeSchema, stringMapSchema } from "../../schema/fragments"; import {
createContentRulesSchema,
createValueMatcherSchema,
sizeSchema,
stringMapSchema,
} from "../../schema/fragments";
export const commandCheckerSchemas: CheckerSchemas = { export const commandCheckerSchemas: CheckerSchemas = {
config: Type.Object( config: Type.Object(
@@ -24,10 +29,10 @@ export const commandCheckerSchemas: CheckerSchemas = {
), ),
expect: Type.Object( expect: Type.Object(
{ {
durationMs: Type.Optional(createValueMatcherSchema()),
exitCode: Type.Optional(Type.Array(Type.Integer())), exitCode: Type.Optional(Type.Array(Type.Integer())),
maxDurationMs: Type.Optional(Type.Number({ minimum: 0 })), stderr: Type.Optional(createContentRulesSchema()),
stderr: Type.Optional(createTextRulesSchema()), stdout: Type.Optional(createContentRulesSchema()),
stdout: Type.Optional(createTextRulesSchema()),
}, },
{ additionalProperties: false }, { additionalProperties: false },
), ),

19
src/server/checker/runner/cmd/text.ts
View File

@@ -1,19 +0,0 @@
import type { ExpectResult } from "../../expect/types";
import type { TextRule } from "./types";
import { mismatchFailure } from "../../expect/failure";
import { applyOperator } from "../../expect/operator";
export function checkTextRules(text: string, rules: TextRule[], phase: string): ExpectResult {
for (let i = 0; i < rules.length; i++) {
const rule = rules[i]!;
const path = `${phase}[${i}]`;
if (!applyOperator(text, rule)) {
return {
failure: mismatchFailure(phase, path, rule, text, `${phase} rule at index ${i} mismatch`),
matched: false,
};
}
}
return { failure: null, matched: true };
}

11
src/server/checker/runner/cmd/types.ts
View File

@@ -1,4 +1,5 @@
import type { ExpectOperator, ResolvedTargetBase } from "../../types"; import type { ContentRules, ValueMatcher } from "../../expect/types";
import type { ResolvedTargetBase } from "../../types";
export interface CommandDefaultsConfig { export interface CommandDefaultsConfig {
cwd?: string; cwd?: string;
@@ -6,10 +7,10 @@ export interface CommandDefaultsConfig {
} }
export interface CommandExpectConfig { export interface CommandExpectConfig {
durationMs?: ValueMatcher;
exitCode?: number[]; exitCode?: number[];
maxDurationMs?: number; stderr?: ContentRules;
stderr?: TextRule[]; stdout?: ContentRules;
stdout?: TextRule[];
} }
export interface CommandTargetConfig { export interface CommandTargetConfig {
@@ -37,5 +38,3 @@ export interface ResolvedCommandTarget extends ResolvedTargetBase {
timeoutMs: number; timeoutMs: number;
type: "cmd"; type: "cmd";
} }
export type TextRule = ExpectOperator;

20
src/server/checker/runner/cmd/validate.ts
View File

@@ -1,10 +1,9 @@
import { isNumber, isPlainObject, isString } from "es-toolkit"; import { isNumber, isPlainObject, isString } from "es-toolkit";
import { isArray } from "es-toolkit/compat";
import type { ConfigValidationIssue } from "../../schema/issues"; import type { ConfigValidationIssue } from "../../schema/issues";
import type { CheckerValidationInput } from "../types"; import type { CheckerValidationInput } from "../types";
import { validateOperatorObject } from "../../expect/validate-operator"; import { validateContentRules, validateValueMatcher } from "../../expect/validate-matcher";
import { issue, joinPath } from "../../schema/issues"; import { issue, joinPath } from "../../schema/issues";
import { parseSize } from "../../utils"; import { parseSize } from "../../utils";
@@ -32,10 +31,6 @@ function getTargetName(target: Record<string, unknown>): string | undefined {
return isString(target["id"]) ? target["id"] : undefined; return isString(target["id"]) ? target["id"] : undefined;
} }
function isNonNegativeFiniteNumber(value: unknown): boolean {
return isNumber(value) && Number.isFinite(value) && value >= 0;
}
function isSizeInput(value: unknown): value is number | string { function isSizeInput(value: unknown): value is number | string {
return isNumber(value) || isString(value); return isNumber(value) || isString(value);
} }
@@ -47,13 +42,13 @@ function validateCommandExpect(target: Record<string, unknown>, path: string): C
const issues: ConfigValidationIssue[] = []; const issues: ConfigValidationIssue[] = [];
const expectPath = joinPath(path, "expect"); const expectPath = joinPath(path, "expect");
if (expect["stdout"] !== undefined) { if (expect["stdout"] !== undefined) {
issues.push(...validateTextRules(expect["stdout"], joinPath(expectPath, "stdout"), targetName)); issues.push(...validateContentRules(expect["stdout"], joinPath(expectPath, "stdout"), targetName));
} }
if (expect["stderr"] !== undefined) { if (expect["stderr"] !== undefined) {
issues.push(...validateTextRules(expect["stderr"], joinPath(expectPath, "stderr"), targetName)); issues.push(...validateContentRules(expect["stderr"], joinPath(expectPath, "stderr"), targetName));
} }
if (expect["maxDurationMs"] !== undefined && !isNonNegativeFiniteNumber(expect["maxDurationMs"])) { if (expect["durationMs"] !== undefined) {
issues.push(issue("invalid-type", joinPath(expectPath, "maxDurationMs"), "必须为非负有限数字", targetName)); issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
} }
return issues; return issues;
} }
@@ -87,8 +82,3 @@ function validateSizeValue(value: number | string, path: string, targetName?: st
return [issue("invalid-size", path, error instanceof Error ? error.message : "size 格式不合法", targetName)]; return [issue("invalid-size", path, error instanceof Error ? error.message : "size 格式不合法", targetName)];
} }
} }
function validateTextRules(rules: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isArray(rules)) return [issue("invalid-type", path, "必须为数组", targetName)];
return rules.flatMap((rule, index) => validateOperatorObject(rule, `${path}[${index}]`, targetName));
}

32
src/server/checker/runner/db/execute.ts
View File

@@ -6,8 +6,9 @@ import type { CheckResult, RawTargetConfig } from "../../types";
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types"; import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
import type { DbExpectConfig, DbTargetConfig, ResolvedDbTarget } from "./types"; import type { DbExpectConfig, DbTargetConfig, ResolvedDbTarget } from "./types";
import { checkDuration } from "../../expect/duration"; import { checkContentRules } from "../../expect/content";
import { errorFailure } from "../../expect/failure"; import { errorFailure } from "../../expect/failure";
import { checkValueMatcher } from "../../expect/matcher";
import { checkRowCount, checkRows } from "./expect"; import { checkRowCount, checkRows } from "./expect";
import { dbCheckerSchemas } from "./schema"; import { dbCheckerSchemas } from "./schema";
import { validateDbConfig } from "./validate"; import { validateDbConfig } from "./validate";
@@ -59,7 +60,11 @@ export class DbChecker implements CheckerDefinition<ResolvedDbTarget> {
// 无 query 时仅测试连接 // 无 query 时仅测试连接
if (!t.db.query) { if (!t.db.query) {
const durationMs = Math.round(performance.now() - start); const durationMs = Math.round(performance.now() - start);
const durationResult = checkDuration(durationMs, t.expect?.maxDurationMs); const durationResult = checkValueMatcher(durationMs, t.expect?.durationMs, {
message: "durationMs mismatch",
path: "durationMs",
phase: "duration",
});
if (!durationResult.matched) { if (!durationResult.matched) {
return { return {
durationMs, durationMs,
@@ -111,7 +116,11 @@ export class DbChecker implements CheckerDefinition<ResolvedDbTarget> {
} }
// duration 断言 // duration 断言
const durationResult = checkDuration(durationMs, t.expect?.maxDurationMs); const durationResult = checkValueMatcher(durationMs, t.expect?.durationMs, {
message: "durationMs mismatch",
path: "durationMs",
phase: "duration",
});
if (!durationResult.matched) { if (!durationResult.matched) {
return { return {
durationMs, durationMs,
@@ -125,7 +134,7 @@ export class DbChecker implements CheckerDefinition<ResolvedDbTarget> {
// rowCount 断言 // rowCount 断言
if (t.expect?.rowCount) { if (t.expect?.rowCount) {
const rowCountResult = checkRowCount(rows, t.expect.rowCount); const rowCountResult = checkRowCount(isArray(rows) ? rows.length : 0, t.expect.rowCount);
if (!rowCountResult.matched) { if (!rowCountResult.matched) {
return { return {
durationMs, durationMs,
@@ -153,6 +162,21 @@ export class DbChecker implements CheckerDefinition<ResolvedDbTarget> {
} }
} }
if (t.expect?.result && t.expect.result.length > 0) {
const rowCount = isArray(rows) ? rows.length : 0;
const resultCheck = checkContentRules({ rowCount, rows }, t.expect.result, { path: "result", phase: "result" });
if (!resultCheck.matched) {
return {
durationMs,
failure: resultCheck.failure,
matched: false,
statusDetail: `${rowCount} rows`,
targetId: t.id,
timestamp,
};
}
}
return { return {
durationMs, durationMs,
failure: null, failure: null,

36
src/server/checker/runner/db/expect.ts
View File

@@ -1,25 +1,21 @@
import { isPlainObject } from "es-toolkit"; import { isPlainObject } from "es-toolkit";
import { isArray } from "es-toolkit/compat"; import { isArray } from "es-toolkit/compat";
import type { ExpectResult } from "../../expect/types"; import type { ExpectResult, KeyValueExpect, ValueMatcher } from "../../expect/types";
import type { ExpectOperator, ExpectValue } from "../../types";
import { mismatchFailure } from "../../expect/failure"; import { mismatchFailure } from "../../expect/failure";
import { checkExpectValue } from "../../expect/operator"; import { checkKeyValueExpect } from "../../expect/key-value";
import { checkValueMatcher } from "../../expect/matcher";
export function checkRowCount(rows: unknown, op: ExpectOperator): ExpectResult { export function checkRowCount(actual: number, matcher: ValueMatcher): ExpectResult {
const actual = isArray(rows) ? rows.length : 0; return checkValueMatcher(actual, matcher, {
const matched = checkExpectValue(actual, op); message: `rowCount ${actual} 不满足条件`,
if (!matched) { path: "rowCount",
return { phase: "rowCount",
failure: mismatchFailure("rowCount", "rowCount", op, actual, `rowCount ${actual} 不满足条件`), });
matched: false,
};
}
return { failure: null, matched: true };
} }
export function checkRows(rows: unknown, rules: Array<Record<string, ExpectValue>>): ExpectResult { export function checkRows(rows: unknown, rules: KeyValueExpect[]): ExpectResult {
if (!isArray(rows)) { if (!isArray(rows)) {
return { return {
failure: mismatchFailure("row", "rows", rules, rows, "查询结果不是数组"), failure: mismatchFailure("row", "rows", rules, rows, "查询结果不是数组"),
@@ -44,16 +40,8 @@ export function checkRows(rows: unknown, rules: Array<Record<string, ExpectValue
}; };
} }
for (const [col, expected] of Object.entries(rule)) { const result = checkKeyValueExpect(row, rule, { path: `rows[${i}]`, phase: "row" });
const actual = row[col]; if (!result.matched) return result;
const matched = checkExpectValue(actual, expected);
if (!matched) {
return {
failure: mismatchFailure("row", `rows[${i}].${col}`, expected, actual, `rows[${i}].${col} mismatch`),
matched: false,
};
}
}
} }
return { failure: null, matched: true }; return { failure: null, matched: true };

33
src/server/checker/runner/db/schema.ts
View File

@@ -2,10 +2,7 @@ import { Type } from "@sinclair/typebox";
import type { CheckerSchemas } from "../types"; import type { CheckerSchemas } from "../types";
import { createPureOperatorSchema, jsonValueSchema, operatorProperties } from "../../schema/fragments"; import { createContentRulesSchema, createKeyValueExpectSchema, createValueMatcherSchema } from "../../schema/fragments";
// Db expect 允许行对象中的列值为字面量或 operator
const dbRowValueSchema = Type.Union([jsonValueSchema, createPureOperatorSchema()]);
export const dbCheckerSchemas: CheckerSchemas = { export const dbCheckerSchemas: CheckerSchemas = {
config: Type.Object( config: Type.Object(
@@ -22,31 +19,11 @@ export const dbCheckerSchemas: CheckerSchemas = {
defaults: Type.Object({}, { additionalProperties: false }), defaults: Type.Object({}, { additionalProperties: false }),
expect: Type.Object( expect: Type.Object(
{ {
maxDurationMs: Type.Optional(Type.Number({ minimum: 0 })), durationMs: Type.Optional(createValueMatcherSchema()),
rowCount: Type.Optional(createPureOperatorSchema()), result: Type.Optional(createContentRulesSchema()),
rows: Type.Optional( rowCount: Type.Optional(createValueMatcherSchema()),
Type.Array( rows: Type.Optional(Type.Array(createKeyValueExpectSchema())),
Type.Record(Type.String(), dbRowValueSchema, {
additionalProperties: false,
minProperties: 1,
}),
),
),
}, },
{ additionalProperties: false }, { additionalProperties: false },
), ),
}; };
// 导出用于 validate 的辅助类型
export const DbOperatorKeys = new Set<string>([
...Object.keys(operatorProperties()),
"contains",
"empty",
"equals",
"exists",
"gt",
"gte",
"lt",
"lte",
"match",
]);

10
src/server/checker/runner/db/types.ts
View File

@@ -1,9 +1,11 @@
import type { ExpectOperator, ExpectValue, ResolvedTargetBase } from "../../types"; import type { ContentRules, KeyValueExpect, ValueMatcher } from "../../expect/types";
import type { ResolvedTargetBase } from "../../types";
export interface DbExpectConfig { export interface DbExpectConfig {
maxDurationMs?: number; durationMs?: ValueMatcher;
rowCount?: ExpectOperator; result?: ContentRules;
rows?: Array<Record<string, ExpectValue>>; rowCount?: ValueMatcher;
rows?: KeyValueExpect[];
} }
export interface DbTargetConfig { export interface DbTargetConfig {

48
src/server/checker/runner/db/validate.ts
View File

@@ -1,11 +1,10 @@
import { isNumber, isPlainObject, isString } from "es-toolkit"; import { isPlainObject, isString } from "es-toolkit";
import { isArray } from "es-toolkit/compat"; import { isArray } from "es-toolkit/compat";
import type { ConfigValidationIssue } from "../../schema/issues"; import type { ConfigValidationIssue } from "../../schema/issues";
import type { CheckerValidationInput } from "../types"; import type { CheckerValidationInput } from "../types";
import { isUnsafeRegex } from "../../expect/redos"; import { validateContentRules, validateKeyValueExpect, validateValueMatcher } from "../../expect/validate-matcher";
import { validateOperatorObject } from "../../expect/validate-operator";
import { issue, joinPath } from "../../schema/issues"; import { issue, joinPath } from "../../schema/issues";
export function validateDbConfig(input: CheckerValidationInput): ConfigValidationIssue[] { export function validateDbConfig(input: CheckerValidationInput): ConfigValidationIssue[] {
@@ -21,7 +20,7 @@ export function validateDbConfig(input: CheckerValidationInput): ConfigValidatio
return issues; return issues;
} }
function collectRowOperators(rows: unknown[], path: string, targetName?: string): ConfigValidationIssue[] { function collectRowExpects(rows: unknown[], path: string, targetName?: string): ConfigValidationIssue[] {
const issues: ConfigValidationIssue[] = []; const issues: ConfigValidationIssue[] = [];
for (let i = 0; i < rows.length; i++) { for (let i = 0; i < rows.length; i++) {
const row = rows[i]!; const row = rows[i]!;
@@ -29,28 +28,7 @@ function collectRowOperators(rows: unknown[], path: string, targetName?: string)
issues.push(issue("invalid-type", `${path}[${i}]`, "必须为对象", targetName)); issues.push(issue("invalid-type", `${path}[${i}]`, "必须为对象", targetName));
continue; continue;
} }
for (const [col, value] of Object.entries(row)) { issues.push(...validateKeyValueExpect(row, `${path}[${i}]`, targetName));
const colPath = `${path}[${i}].${col}`;
if (isPlainObject(value) && Object.keys(value).some((k) => k === "match")) {
// 检查 match 正则
const valueRecord = value as Record<string, unknown>;
const match: unknown = valueRecord["match"];
if (isString(match)) {
try {
new RegExp(match);
} catch {
issues.push(issue("invalid-regex", colPath, "正则不合法", targetName));
}
if (isUnsafeRegex(match)) {
issues.push(issue("unsafe-regex", colPath, "正则存在 ReDoS 风险", targetName));
}
}
}
// 校验 operator 对象
if (isPlainObject(value)) {
issues.push(...validateOperatorObject(value, colPath, targetName, { requireAtLeastOne: false }));
}
}
} }
return issues; return issues;
} }
@@ -60,10 +38,6 @@ function getTargetName(target: Record<string, unknown>): string | undefined {
return isString(target["id"]) ? target["id"] : undefined; return isString(target["id"]) ? target["id"] : undefined;
} }
function isNonNegativeFiniteNumber(value: unknown): boolean {
return isNumber(value) && Number.isFinite(value) && value >= 0;
}
function validateDbExpect(target: Record<string, unknown>, path: string): ConfigValidationIssue[] { function validateDbExpect(target: Record<string, unknown>, path: string): ConfigValidationIssue[] {
const targetName = getTargetName(target); const targetName = getTargetName(target);
const expect = target["expect"]; const expect = target["expect"];
@@ -71,24 +45,28 @@ function validateDbExpect(target: Record<string, unknown>, path: string): Config
const issues: ConfigValidationIssue[] = []; const issues: ConfigValidationIssue[] = [];
const expectPath = joinPath(path, "expect"); const expectPath = joinPath(path, "expect");
if (expect["maxDurationMs"] !== undefined && !isNonNegativeFiniteNumber(expect["maxDurationMs"])) { if (expect["durationMs"] !== undefined) {
issues.push(issue("invalid-type", joinPath(expectPath, "maxDurationMs"), "必须为非负有限数字", targetName)); issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
} }
if (expect["rowCount"] !== undefined) { if (expect["rowCount"] !== undefined) {
issues.push(...validateOperatorObject(expect["rowCount"], joinPath(expectPath, "rowCount"), targetName)); issues.push(...validateValueMatcher(expect["rowCount"], joinPath(expectPath, "rowCount"), targetName));
} }
if (expect["rows"] !== undefined) { if (expect["rows"] !== undefined) {
if (!isArray(expect["rows"])) { if (!isArray(expect["rows"])) {
issues.push(issue("invalid-type", joinPath(expectPath, "rows"), "必须为数组", targetName)); issues.push(issue("invalid-type", joinPath(expectPath, "rows"), "必须为数组", targetName));
} else { } else {
issues.push(...collectRowOperators(expect["rows"], joinPath(expectPath, "rows"), targetName)); issues.push(...collectRowExpects(expect["rows"], joinPath(expectPath, "rows"), targetName));
} }
} }
if (expect["result"] !== undefined) {
issues.push(...validateContentRules(expect["result"], joinPath(expectPath, "result"), targetName));
}
// 检查未知字段 // 检查未知字段
const allowedKeys = new Set(["maxDurationMs", "rowCount", "rows"]); const allowedKeys = new Set(["durationMs", "result", "rowCount", "rows"]);
for (const key of Object.keys(expect)) { for (const key of Object.keys(expect)) {
if (!allowedKeys.has(key)) { if (!allowedKeys.has(key)) {
issues.push(issue("unknown-field", joinPath(expectPath, key), "是未知字段", targetName)); issues.push(issue("unknown-field", joinPath(expectPath, key), "是未知字段", targetName));

212
src/server/checker/runner/http/body.ts
View File

@@ -1,212 +0,0 @@
import { DOMParser } from "@xmldom/xmldom";
import * as cheerio from "cheerio";
import { isArray } from "es-toolkit/compat";
import * as xpath from "xpath";
import type { ExpectResult } from "../../expect/types";
import type { BodyRule, CssRule, JsonRule, XpathRule } from "./types";
import { errorFailure, mismatchFailure } from "../../expect/failure";
import { applyOperator, evaluateJsonPath } from "../../expect/operator";
type ParsedJsonResult = { error: string; ok: false } | { ok: true; value: unknown };
export function checkBodyExpect(body: string, rules?: BodyRule[]): ExpectResult {
if (!rules || rules.length === 0) return { failure: null, matched: true };
let parsedJson: ParsedJsonResult | undefined;
for (let i = 0; i < rules.length; i++) {
const rule = rules[i]!;
if ("json" in rule && parsedJson === undefined) {
parsedJson = parseJsonBody(body);
}
const result = checkSingleBodyRule(body, rule, i, parsedJson);
if (!result.matched) return result;
}
return { failure: null, matched: true };
}
function checkCssRule(body: string, rule: CssRule, rulePath: string): ExpectResult {
const { attr, selector, ...operators } = rule;
const fullPath = `${rulePath}.css(${selector}${attr ? `@${attr}` : ""})`;
let $: cheerio.CheerioAPI;
try {
$ = cheerio.load(body);
} catch {
return {
failure: errorFailure("body", fullPath, "failed to parse HTML"),
matched: false,
};
}
const el = $(selector);
if (operators.exists === false) {
if (el.length > 0) {
return {
failure: mismatchFailure("body", fullPath, false, true, `selector ${selector} exists`),
matched: false,
};
}
return { failure: null, matched: true };
}
if (el.length === 0) {
const expected = operators.exists === true ? true : "element found";
const actual = operators.exists === true ? false : "no match";
return {
failure: mismatchFailure("body", fullPath, expected, actual, `selector ${selector} not found`),
matched: false,
};
}
if (operators.exists === true) return { failure: null, matched: true };
const actual = attr ? el.attr(attr) : el.text();
const opKeys = Object.keys(operators);
if (opKeys.length === 0) {
if (actual === undefined) {
return {
failure: mismatchFailure("body", fullPath, "attr present", undefined, `attribute ${attr} not found`),
matched: false,
};
}
return { failure: null, matched: true };
}
const matched = applyOperator(actual ?? "", operators);
if (!matched) {
return {
failure: mismatchFailure("body", fullPath, operators, actual, `css selector ${selector} mismatch`),
matched: false,
};
}
return { failure: null, matched: true };
}
function checkJsonRule(body: string, rule: JsonRule, rulePath: string, parsedJson?: ParsedJsonResult): ExpectResult {
const { path, ...operators } = rule;
const fullPath = `${rulePath}.json(${path})`;
const jsonResult = parsedJson ?? parseJsonBody(body);
if (!jsonResult.ok) {
return {
failure: errorFailure("body", fullPath, jsonResult.error),
matched: false,
};
}
const actual = evaluateJsonPath(jsonResult.value, path);
const opKeys = Object.keys(operators);
if (opKeys.length === 0) {
if (actual === undefined) {
return {
failure: mismatchFailure("body", fullPath, "defined", actual, `path ${path} is undefined`),
matched: false,
};
}
return { failure: null, matched: true };
}
const matched = applyOperator(actual, operators);
if (!matched) {
return {
failure: mismatchFailure("body", fullPath, operators, actual, `json path ${path} mismatch`),
matched: false,
};
}
return { failure: null, matched: true };
}
function checkSingleBodyRule(body: string, rule: BodyRule, index: number, parsedJson?: ParsedJsonResult): ExpectResult {
const rulePath = `body[${index}]`;
if ("contains" in rule) {
const matched = body.includes(rule.contains);
if (!matched) {
return {
failure: mismatchFailure("body", rulePath, rule.contains, body, `body does not contain "${rule.contains}"`),
matched: false,
};
}
return { failure: null, matched: true };
}
if ("regex" in rule) {
const matched = new RegExp(rule.regex).test(body);
if (!matched) {
return {
failure: mismatchFailure("body", rulePath, `/${rule.regex}/`, body, `body does not match /${rule.regex}/`),
matched: false,
};
}
return { failure: null, matched: true };
}
if ("json" in rule) {
return checkJsonRule(body, rule.json, rulePath, parsedJson);
}
if ("css" in rule) {
return checkCssRule(body, rule.css, rulePath);
}
if ("xpath" in rule) {
return checkXpathRule(body, rule.xpath, rulePath);
}
return { failure: null, matched: true };
}
function checkXpathRule(body: string, rule: XpathRule, rulePath: string): ExpectResult {
const { path, ...operators } = rule;
const fullPath = `${rulePath}.xpath(${path})`;
let doc: ReturnType<DOMParser["parseFromString"]>;
try {
doc = new DOMParser().parseFromString(body, "text/xml");
} catch {
return {
failure: errorFailure("body", fullPath, "failed to parse XML/HTML"),
matched: false,
};
}
const nodes = xpath.select(path, doc as unknown as Node);
if (!nodes || !isArray(nodes) || nodes.length === 0) {
return {
failure: mismatchFailure("body", fullPath, "node found", "no match", `xpath ${path} not found`),
matched: false,
};
}
const node = nodes[0]!;
const actual = node.nodeValue ?? (node as unknown as Element).textContent ?? "";
const opKeys = Object.keys(operators);
if (opKeys.length === 0) {
return { failure: null, matched: true };
}
const matched = applyOperator(actual, operators);
if (!matched) {
return {
failure: mismatchFailure("body", fullPath, operators, actual, `xpath ${path} mismatch`),
matched: false,
};
}
return { failure: null, matched: true };
}
function parseJsonBody(body: string): ParsedJsonResult {
try {
return { ok: true, value: JSON.parse(body) as unknown };
} catch {
return { error: "body is not valid JSON", ok: false };
}
}

40
src/server/checker/runner/http/execute.ts
View File

@@ -5,10 +5,10 @@ import type { CheckResult, RawTargetConfig } from "../../types";
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types"; import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
import type { HttpExpectConfig, HttpTargetConfig, ResolvedHttpTarget } from "./types"; import type { HttpExpectConfig, HttpTargetConfig, ResolvedHttpTarget } from "./types";
import { checkDuration } from "../../expect/duration"; import { checkContentRules } from "../../expect/content";
import { errorFailure, mismatchFailure } from "../../expect/failure"; import { errorFailure, mismatchFailure } from "../../expect/failure";
import { checkValueMatcher } from "../../expect/matcher";
import { parseSize } from "../../utils"; import { parseSize } from "../../utils";
import { checkBodyExpect } from "./body";
import { checkHeaders, checkStatus } from "./expect"; import { checkHeaders, checkStatus } from "./expect";
import { httpCheckerSchemas } from "./schema"; import { httpCheckerSchemas } from "./schema";
import { validateHttpConfig } from "./validate"; import { validateHttpConfig } from "./validate";
@@ -54,7 +54,7 @@ export class HttpChecker implements CheckerDefinition<ResolvedHttpTarget> {
const hasBodyRules = !!(expect?.body && expect.body.length > 0); const hasBodyRules = !!(expect?.body && expect.body.length > 0);
const earlyTimeout = hasBodyRules ? checkEarlyTimeout(start, expect?.maxDurationMs) : null; const earlyTimeout = hasBodyRules ? checkEarlyTimeout(start, expect?.durationMs) : null;
if (earlyTimeout) { if (earlyTimeout) {
return makeResult(t, timestamp, earlyTimeout.elapsed, earlyTimeout.failure, statusCode); return makeResult(t, timestamp, earlyTimeout.elapsed, earlyTimeout.failure, statusCode);
} }
@@ -70,14 +70,18 @@ export class HttpChecker implements CheckerDefinition<ResolvedHttpTarget> {
return makeResult(t, timestamp, performance.now() - start, decodeResult.failure, statusCode); return makeResult(t, timestamp, performance.now() - start, decodeResult.failure, statusCode);
} }
const bodyResult = checkBodyExpect(decodeResult.text, expect.body); const bodyResult = checkContentRules(decodeResult.text, expect.body, { path: "body", phase: "body" });
if (!bodyResult.matched) { if (!bodyResult.matched) {
return makeResult(t, timestamp, performance.now() - start, bodyResult.failure, statusCode); return makeResult(t, timestamp, performance.now() - start, bodyResult.failure, statusCode);
} }
} }
const durationMs = Math.round(performance.now() - start); const durationMs = Math.round(performance.now() - start);
const durationResult = checkDuration(durationMs, expect?.maxDurationMs); const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
message: "durationMs mismatch",
path: "durationMs",
phase: "duration",
});
if (!durationResult.matched) { if (!durationResult.matched) {
return makeResult(t, timestamp, durationMs, durationResult.failure, statusCode); return makeResult(t, timestamp, durationMs, durationResult.failure, statusCode);
} }
@@ -190,23 +194,29 @@ function buildRedirectInit(init: RequestInit, statusCode: number, fromUrl: strin
function checkEarlyTimeout( function checkEarlyTimeout(
start: number, start: number,
maxDurationMs: number | undefined, durationMatcher: HttpExpectConfig["durationMs"] | undefined,
): null | { elapsed: number; failure: CheckResult["failure"] } { ): null | { elapsed: number; failure: CheckResult["failure"] } {
if (maxDurationMs === undefined) return null; if (durationMatcher === undefined) return null;
const limit = Math.min(
durationMatcher.lte ?? Number.POSITIVE_INFINITY,
durationMatcher.lt ?? Number.POSITIVE_INFINITY,
);
if (!Number.isFinite(limit)) return null;
const elapsed = performance.now() - start; const elapsed = performance.now() - start;
if (elapsed <= maxDurationMs) return null; if (durationMatcher.lt !== undefined ? elapsed < limit : elapsed <= limit) return null;
const durationMs = Math.round(elapsed); const durationMs = Math.round(elapsed);
const durationResult = checkValueMatcher(durationMs, durationMatcher, {
message: "durationMs mismatch",
path: "durationMs",
phase: "duration",
});
return { return {
elapsed, elapsed,
failure: mismatchFailure( failure:
"duration", durationResult.failure ??
"duration", mismatchFailure("duration", "durationMs", durationMatcher, durationMs, "durationMs mismatch"),
`<=${maxDurationMs}ms`,
durationMs,
`duration ${durationMs}ms > ${maxDurationMs}ms`,
),
}; };
} }

50
src/server/checker/runner/http/expect.ts
View File

@@ -1,48 +1,16 @@
import { isNumber, isString } from "es-toolkit"; import { isNumber } from "es-toolkit";
import type { ExpectResult } from "../../expect/types"; import type { ExpectResult, KeyValueExpect } from "../../expect/types";
import type { HeaderExpect } from "./types";
import { mismatchFailure } from "../../expect/failure"; import { mismatchFailure } from "../../expect/failure";
import { applyOperator } from "../../expect/operator"; import { checkKeyValueExpect } from "../../expect/key-value";
export function checkHeaders( export function checkHeaders(headers: Record<string, string>, headerExpects?: KeyValueExpect): ExpectResult {
headers: Record<string, string>, return checkKeyValueExpect(headers, headerExpects, {
headerExpects?: Record<string, HeaderExpect>, normalizeKey: (key) => key.toLowerCase(),
): ExpectResult { path: "headers",
if (!headerExpects) return { failure: null, matched: true }; phase: "headers",
});
for (const [key, expected] of Object.entries(headerExpects)) {
const actualValue = headers[key.toLowerCase()];
const path = `headers.${key}`;
if (isString(expected)) {
if (actualValue !== expected) {
return {
failure: mismatchFailure("headers", path, expected, actualValue, `header ${key} mismatch`),
matched: false,
};
}
} else {
if (actualValue === undefined) {
if (expected.exists !== false) {
return {
failure: mismatchFailure("headers", path, "defined", undefined, `header ${key} not found`),
matched: false,
};
}
continue;
}
if (!applyOperator(actualValue, expected)) {
return {
failure: mismatchFailure("headers", path, expected, actualValue, `header ${key} mismatch`),
matched: false,
};
}
}
}
return { failure: null, matched: true };
} }
export function checkStatus(statusCode: number, allowed: Array<number | string>): ExpectResult { export function checkStatus(statusCode: number, allowed: Array<number | string>): ExpectResult {

11
src/server/checker/runner/http/schema.ts
View File

@@ -3,8 +3,9 @@ import { Type } from "@sinclair/typebox";
import type { CheckerSchemas } from "../types"; import type { CheckerSchemas } from "../types";
import { import {
createBodyRulesSchema, createContentRulesSchema,
createHeaderExpectSchema, createKeyValueExpectSchema,
createValueMatcherSchema,
httpMethodSchema, httpMethodSchema,
sizeSchema, sizeSchema,
statusCodePatternSchema, statusCodePatternSchema,
@@ -33,9 +34,9 @@ export const httpCheckerSchemas: CheckerSchemas = {
), ),
expect: Type.Object( expect: Type.Object(
{ {
body: Type.Optional(createBodyRulesSchema()), body: Type.Optional(createContentRulesSchema()),
headers: Type.Optional(createHeaderExpectSchema()), durationMs: Type.Optional(createValueMatcherSchema()),
maxDurationMs: Type.Optional(Type.Number({ minimum: 0 })), headers: Type.Optional(createKeyValueExpectSchema()),
status: Type.Optional(Type.Array(statusCodePatternSchema)), status: Type.Optional(Type.Array(statusCodePatternSchema)),
}, },
{ additionalProperties: false }, { additionalProperties: false },

24
src/server/checker/runner/http/types.ts
View File

@@ -1,15 +1,5 @@
import type { ExpectOperator, ResolvedTargetBase } from "../../types"; import type { ContentRules, KeyValueExpect, ValueMatcher } from "../../expect/types";
import type { ResolvedTargetBase } from "../../types";
export type BodyRule =
| { contains: string }
| { css: CssRule }
| { json: JsonRule }
| { regex: string }
| { xpath: XpathRule };
export type CssRule = ExpectOperator & { attr?: string; selector: string };
export type HeaderExpect = ExpectOperator | string;
export interface HttpDefaultsConfig { export interface HttpDefaultsConfig {
headers?: Record<string, string>; headers?: Record<string, string>;
@@ -18,9 +8,9 @@ export interface HttpDefaultsConfig {
} }
export interface HttpExpectConfig { export interface HttpExpectConfig {
body?: BodyRule[]; body?: ContentRules;
headers?: Record<string, HeaderExpect>; durationMs?: ValueMatcher;
maxDurationMs?: number; headers?: KeyValueExpect;
status?: Array<number | string>; status?: Array<number | string>;
} }
@@ -34,8 +24,6 @@ export interface HttpTargetConfig {
url: string; url: string;
} }
export type JsonRule = ExpectOperator & { path: string };
export interface ResolvedHttpConfig { export interface ResolvedHttpConfig {
body?: string; body?: string;
headers: Record<string, string>; headers: Record<string, string>;
@@ -55,5 +43,3 @@ export interface ResolvedHttpTarget extends ResolvedTargetBase {
timeoutMs: number; timeoutMs: number;
type: "http"; type: "http";
} }
export type XpathRule = ExpectOperator & { path: string };

144
src/server/checker/runner/http/validate.ts
View File

@@ -1,24 +1,19 @@
import { DOMParser } from "@xmldom/xmldom";
import { isNumber, isString } from "es-toolkit"; import { isNumber, isString } from "es-toolkit";
import { isArray } from "es-toolkit/compat"; import { isArray } from "es-toolkit/compat";
import * as xpath from "xpath";
import type { ConfigValidationIssue } from "../../schema/issues"; import type { ConfigValidationIssue } from "../../schema/issues";
import type { CheckerValidationInput } from "../types"; import type { CheckerValidationInput } from "../types";
import { isUnsafeRegex } from "../../expect/redos"; import {
import { isPlainRecord, validateOperatorObject } from "../../expect/validate-operator"; isPlainRecord,
import { BodyRuleTypeKeys, OperatorKeys } from "../../schema/fragments"; validateContentRules,
validateKeyValueExpect,
validateValueMatcher,
} from "../../expect/validate-matcher";
import { issue, joinPath } from "../../schema/issues"; import { issue, joinPath } from "../../schema/issues";
import { parseSize } from "../../utils"; import { parseSize } from "../../utils";
const ALLOWED_PROTOCOLS = new Set(["http:", "https:"]); const ALLOWED_PROTOCOLS = new Set(["http:", "https:"]);
const OPERATOR_KEY_SET = new Set<string>(OperatorKeys);
export function validateBodyRules(body: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isArray(body)) return [issue("invalid-type", path, "必须为数组", targetName)];
return body.flatMap((rule, index) => validateSingleBodyRule(rule, `${path}[${index}]`, targetName));
}
export function validateHttpConfig(input: CheckerValidationInput): ConfigValidationIssue[] { export function validateHttpConfig(input: CheckerValidationInput): ConfigValidationIssue[] {
const issues: ConfigValidationIssue[] = []; const issues: ConfigValidationIssue[] = [];
@@ -57,55 +52,15 @@ export function validateJsonPath(path: string, rulePath: string, targetName?: st
return issues; return issues;
} }
function collectOperatorObject(
object: Record<string, unknown>,
allowedKeys: Set<string>,
path: string,
targetName?: string,
): { issues: ConfigValidationIssue[]; operators: Record<string, unknown> } {
const issues: ConfigValidationIssue[] = [];
const operators: Record<string, unknown> = {};
for (const [key, value] of Object.entries(object)) {
if (allowedKeys.has(key)) continue;
if (OPERATOR_KEY_SET.has(key)) {
operators[key] = value;
} else {
issues.push(issue("unknown-field", joinPath(path, key), "是未知字段", targetName));
}
}
return { issues, operators };
}
function getTargetName(target: Record<string, unknown>): string | undefined { function getTargetName(target: Record<string, unknown>): string | undefined {
if (isString(target["name"])) return target["name"]; if (isString(target["name"])) return target["name"];
return isString(target["id"]) ? target["id"] : undefined; return isString(target["id"]) ? target["id"] : undefined;
} }
function isNonNegativeFiniteNumber(value: unknown): boolean {
return isNumber(value) && Number.isFinite(value) && value >= 0;
}
function isSizeInput(value: unknown): value is number | string { function isSizeInput(value: unknown): value is number | string {
return isNumber(value) || isString(value); return isNumber(value) || isString(value);
} }
function validateCssRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
const issues: ConfigValidationIssue[] = [];
if (!isString(rule["selector"]) || rule["selector"].trim() === "") {
issues.push(issue("invalid-type", joinPath(path, "selector"), "必须为非空字符串", targetName));
}
if ("attr" in rule && !isString(rule["attr"])) {
issues.push(issue("invalid-type", joinPath(path, "attr"), "必须为字符串", targetName));
}
const result = collectOperatorObject(rule, new Set(["attr", "selector"]), path, targetName);
issues.push(
...result.issues,
...validateOperatorObject(result.operators, path, targetName, { requireAtLeastOne: false }),
);
return issues;
}
function validateHttpExpect(target: Record<string, unknown>, path: string): ConfigValidationIssue[] { function validateHttpExpect(target: Record<string, unknown>, path: string): ConfigValidationIssue[] {
const targetName = getTargetName(target); const targetName = getTargetName(target);
const expect = target["expect"]; const expect = target["expect"];
@@ -114,22 +69,19 @@ function validateHttpExpect(target: Record<string, unknown>, path: string): Conf
const expectPath = joinPath(path, "expect"); const expectPath = joinPath(path, "expect");
if (isPlainRecord(expect["headers"])) { if (isPlainRecord(expect["headers"])) {
for (const [key, value] of Object.entries(expect["headers"])) { issues.push(...validateKeyValueExpect(expect["headers"], joinPath(expectPath, "headers"), targetName));
if (isString(value)) continue;
issues.push(...validateOperatorObject(value, joinPath(joinPath(expectPath, "headers"), key), targetName));
}
} }
if (expect["body"] !== undefined) { if (expect["body"] !== undefined) {
issues.push(...validateBodyRules(expect["body"], joinPath(expectPath, "body"), targetName)); issues.push(...validateContentRules(expect["body"], joinPath(expectPath, "body"), targetName));
} }
if (isArray(expect["status"])) { if (isArray(expect["status"])) {
issues.push(...validateStatusValues(expect["status"], joinPath(expectPath, "status"), targetName)); issues.push(...validateStatusValues(expect["status"], joinPath(expectPath, "status"), targetName));
} }
if (expect["maxDurationMs"] !== undefined && !isNonNegativeFiniteNumber(expect["maxDurationMs"])) { if (expect["durationMs"] !== undefined) {
issues.push(issue("invalid-type", joinPath(expectPath, "maxDurationMs"), "必须为非负有限数字", targetName)); issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
} }
return issues; return issues;
@@ -172,61 +124,6 @@ function validateHttpTarget(target: Record<string, unknown>, path: string): Conf
return issues; return issues;
} }
function validateJsonRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
const issues: ConfigValidationIssue[] = [];
if (!isString(rule["path"])) {
issues.push(issue("invalid-type", joinPath(path, "path"), "必须为字符串", targetName));
} else {
issues.push(...validateJsonPath(rule["path"], path, targetName));
}
const result = collectOperatorObject(rule, new Set(["path"]), path, targetName);
issues.push(
...result.issues,
...validateOperatorObject(result.operators, path, targetName, { requireAtLeastOne: false }),
);
return issues;
}
function validateRegexRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isString(rule)) return [issue("invalid-type", path, "必须为字符串", targetName)];
try {
new RegExp(rule);
} catch {
return [issue("invalid-regex", path, "正则不合法", targetName)];
}
return isUnsafeRegex(rule) ? [issue("unsafe-regex", path, "正则存在 ReDoS 风险", targetName)] : [];
}
function validateSingleBodyRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
const found = BodyRuleTypeKeys.filter((type) => type in rule);
if (found.length === 0) return [issue("missing-body-rule", path, "缺少支持的规则类型", targetName)];
if (found.length > 1) return [issue("multiple-body-rules", path, "只能配置一种规则类型", targetName)];
const ruleType = found[0]!;
const issues: ConfigValidationIssue[] = [];
for (const key of Object.keys(rule)) {
if (key !== ruleType) issues.push(issue("unknown-field", joinPath(path, key), "是未知字段", targetName));
}
if (issues.length > 0) return issues;
switch (ruleType) {
case "contains":
return isString(rule["contains"])
? []
: [issue("invalid-type", joinPath(path, "contains"), "必须为字符串", targetName)];
case "css":
return validateCssRule(rule["css"], joinPath(path, "css"), targetName);
case "json":
return validateJsonRule(rule["json"], joinPath(path, "json"), targetName);
case "regex":
return validateRegexRule(rule["regex"], joinPath(path, "regex"), targetName);
case "xpath":
return validateXpathRule(rule["xpath"], joinPath(path, "xpath"), targetName);
}
}
function validateSizeValue(value: number | string, path: string, targetName?: string): ConfigValidationIssue[] { function validateSizeValue(value: number | string, path: string, targetName?: string): ConfigValidationIssue[] {
try { try {
parseSize(value); parseSize(value);
@@ -257,24 +154,3 @@ function validateStatusValues(values: unknown[], path: string, targetName?: stri
} }
return issues; return issues;
} }
function validateXpathRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
const issues: ConfigValidationIssue[] = [];
if (!isString(rule["path"]) || rule["path"].trim() === "") {
issues.push(issue("invalid-type", joinPath(path, "path"), "必须为非空字符串", targetName));
} else {
try {
const doc = new DOMParser().parseFromString("<x/>", "text/xml");
xpath.select(rule["path"], doc as unknown as Node);
} catch {
issues.push(issue("invalid-xpath", joinPath(path, "path"), "xpath 不合法", targetName));
}
}
const result = collectOperatorObject(rule, new Set(["path"]), path, targetName);
issues.push(
...result.issues,
...validateOperatorObject(result.operators, path, targetName, { requireAtLeastOne: false }),
);
return issues;
}

14
src/server/checker/runner/icmp/execute.ts
View File

@@ -4,8 +4,8 @@ import type { CheckResult, RawTargetConfig } from "../../types";
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types"; import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
import type { PingExpectConfig, PingStats, PingTargetConfig, ResolvedPingTarget } from "./types"; import type { PingExpectConfig, PingStats, PingTargetConfig, ResolvedPingTarget } from "./types";
import { checkDuration } from "../../expect/duration";
import { errorFailure } from "../../expect/failure"; import { errorFailure } from "../../expect/failure";
import { checkValueMatcher } from "../../expect/matcher";
import { buildPingCommand } from "./command"; import { buildPingCommand } from "./command";
import { checkAlive, checkAvgLatency, checkMaxLatency, checkPacketLoss } from "./expect"; import { checkAlive, checkAvgLatency, checkMaxLatency, checkPacketLoss } from "./expect";
import { parsePingOutput } from "./parse"; import { parsePingOutput } from "./parse";
@@ -140,13 +140,17 @@ function buildStatusDetail(stats: PingStats): string {
function checkStats(stats: PingStats, expect: PingExpectConfig | undefined, durationMs: number) { function checkStats(stats: PingStats, expect: PingExpectConfig | undefined, durationMs: number) {
const aliveResult = checkAlive(stats.alive, expect?.alive ?? true); const aliveResult = checkAlive(stats.alive, expect?.alive ?? true);
if (!aliveResult.matched) return aliveResult; if (!aliveResult.matched) return aliveResult;
const packetLossResult = checkPacketLoss(stats.packetLoss, expect?.maxPacketLoss); const packetLossResult = checkPacketLoss(stats.packetLoss, expect?.packetLossPercent);
if (!packetLossResult.matched) return packetLossResult; if (!packetLossResult.matched) return packetLossResult;
const avgLatencyResult = checkAvgLatency(stats.avgLatencyMs, expect?.maxAvgLatencyMs); const avgLatencyResult = checkAvgLatency(stats.avgLatencyMs, expect?.avgLatencyMs);
if (!avgLatencyResult.matched) return avgLatencyResult; if (!avgLatencyResult.matched) return avgLatencyResult;
const maxLatencyResult = checkMaxLatency(stats.maxLatencyMs, expect?.maxMaxLatencyMs); const maxLatencyResult = checkMaxLatency(stats.maxLatencyMs, expect?.maxLatencyMs);
if (!maxLatencyResult.matched) return maxLatencyResult; if (!maxLatencyResult.matched) return maxLatencyResult;
return checkDuration(durationMs, expect?.maxDurationMs); return checkValueMatcher(durationMs, expect?.durationMs, {
message: "durationMs mismatch",
path: "durationMs",
phase: "duration",
});
} }
function formatNumber(value: number): string { function formatNumber(value: number): string {

42
src/server/checker/runner/icmp/expect.ts
View File

@@ -1,6 +1,7 @@
import type { ExpectResult } from "../../expect/types"; import type { ExpectResult, ValueMatcher } from "../../expect/types";
import { mismatchFailure } from "../../expect/failure"; import { mismatchFailure } from "../../expect/failure";
import { checkValueMatcher } from "../../expect/matcher";
export function checkAlive(actual: boolean, expected: boolean): ExpectResult { export function checkAlive(actual: boolean, expected: boolean): ExpectResult {
if (actual === expected) return { failure: null, matched: true }; if (actual === expected) return { failure: null, matched: true };
@@ -16,29 +17,26 @@ export function checkAlive(actual: boolean, expected: boolean): ExpectResult {
}; };
} }
export function checkAvgLatency(actual: null | number, max: number | undefined): ExpectResult { export function checkAvgLatency(actual: null | number, matcher: undefined | ValueMatcher): ExpectResult {
if (max === undefined) return { failure: null, matched: true }; return checkValueMatcher(actual, matcher, {
if (actual !== null && actual <= max) return { failure: null, matched: true }; message: "平均延迟不满足条件",
return { path: "avgLatencyMs",
failure: mismatchFailure("avgLatency", "avgLatencyMs", `<=${max}ms`, actual, `平均延迟超过 ${max}ms`), phase: "avgLatency",
matched: false, });
};
} }
export function checkMaxLatency(actual: null | number, max: number | undefined): ExpectResult { export function checkMaxLatency(actual: null | number, matcher: undefined | ValueMatcher): ExpectResult {
if (max === undefined) return { failure: null, matched: true }; return checkValueMatcher(actual, matcher, {
if (actual !== null && actual <= max) return { failure: null, matched: true }; message: "最大延迟不满足条件",
return { path: "maxLatencyMs",
failure: mismatchFailure("maxLatency", "maxLatencyMs", `<=${max}ms`, actual, `最大延迟超过 ${max}ms`), phase: "maxLatency",
matched: false, });
};
} }
export function checkPacketLoss(actual: number, max: number | undefined): ExpectResult { export function checkPacketLoss(actual: number, matcher: undefined | ValueMatcher): ExpectResult {
if (max === undefined) return { failure: null, matched: true }; return checkValueMatcher(actual, matcher, {
if (actual <= max) return { failure: null, matched: true }; message: "丢包率不满足条件",
return { path: "packetLossPercent",
failure: mismatchFailure("packetLoss", "packetLoss", `<=${max}%`, actual, `丢包率 ${actual}% > ${max}%`), phase: "packetLoss",
matched: false, });
};
} }

10
src/server/checker/runner/icmp/schema.ts
View File

@@ -2,6 +2,8 @@ import { Type } from "@sinclair/typebox";
import type { CheckerSchemas } from "../types"; import type { CheckerSchemas } from "../types";
import { createValueMatcherSchema } from "../../schema/fragments";
export const icmpCheckerSchemas: CheckerSchemas = { export const icmpCheckerSchemas: CheckerSchemas = {
config: Type.Object( config: Type.Object(
{ {
@@ -15,10 +17,10 @@ export const icmpCheckerSchemas: CheckerSchemas = {
expect: Type.Object( expect: Type.Object(
{ {
alive: Type.Optional(Type.Boolean()), alive: Type.Optional(Type.Boolean()),
maxAvgLatencyMs: Type.Optional(Type.Number({ minimum: 0 })), avgLatencyMs: Type.Optional(createValueMatcherSchema()),
maxDurationMs: Type.Optional(Type.Number({ minimum: 0 })), durationMs: Type.Optional(createValueMatcherSchema()),
maxMaxLatencyMs: Type.Optional(Type.Number({ minimum: 0 })), maxLatencyMs: Type.Optional(createValueMatcherSchema()),
maxPacketLoss: Type.Optional(Type.Number({ maximum: 100, minimum: 0 })), packetLossPercent: Type.Optional(createValueMatcherSchema()),
}, },
{ additionalProperties: false }, { additionalProperties: false },
), ),

9
src/server/checker/runner/icmp/types.ts
View File

@@ -1,11 +1,12 @@
import type { ValueMatcher } from "../../expect/types";
import type { ResolvedTargetBase } from "../../types"; import type { ResolvedTargetBase } from "../../types";
export interface PingExpectConfig { export interface PingExpectConfig {
alive?: boolean; alive?: boolean;
maxAvgLatencyMs?: number; avgLatencyMs?: ValueMatcher;
maxDurationMs?: number; durationMs?: ValueMatcher;
maxMaxLatencyMs?: number; maxLatencyMs?: ValueMatcher;
maxPacketLoss?: number; packetLossPercent?: ValueMatcher;
} }
export interface PingStats { export interface PingStats {

19
src/server/checker/runner/icmp/validate.ts
View File

@@ -3,6 +3,7 @@ import { isNumber, isPlainObject, isString } from "es-toolkit";
import type { ConfigValidationIssue } from "../../schema/issues"; import type { ConfigValidationIssue } from "../../schema/issues";
import type { CheckerValidationInput } from "../types"; import type { CheckerValidationInput } from "../types";
import { validateValueMatcher } from "../../expect/validate-matcher";
import { issue, joinPath } from "../../schema/issues"; import { issue, joinPath } from "../../schema/issues";
export function validatePingConfig(input: CheckerValidationInput): ConfigValidationIssue[] { export function validatePingConfig(input: CheckerValidationInput): ConfigValidationIssue[] {
@@ -37,10 +38,6 @@ function getTargetName(target: Record<string, unknown>): string | undefined {
return isString(target["id"]) ? target["id"] : undefined; return isString(target["id"]) ? target["id"] : undefined;
} }
function isNonNegativeFiniteNumber(value: unknown): boolean {
return isNumber(value) && Number.isFinite(value) && value >= 0;
}
function validatePingExpect(target: Record<string, unknown>, path: string): ConfigValidationIssue[] { function validatePingExpect(target: Record<string, unknown>, path: string): ConfigValidationIssue[] {
const rawExpect = target["expect"]; const rawExpect = target["expect"];
if (rawExpect === undefined || rawExpect === null || !isPlainObject(rawExpect)) return []; if (rawExpect === undefined || rawExpect === null || !isPlainObject(rawExpect)) return [];
@@ -52,19 +49,13 @@ function validatePingExpect(target: Record<string, unknown>, path: string): Conf
if (expect["alive"] !== undefined && typeof expect["alive"] !== "boolean") { if (expect["alive"] !== undefined && typeof expect["alive"] !== "boolean") {
issues.push(issue("invalid-type", joinPath(expectPath, "alive"), "必须为布尔值", targetName)); issues.push(issue("invalid-type", joinPath(expectPath, "alive"), "必须为布尔值", targetName));
} }
if (expect["maxPacketLoss"] !== undefined) { for (const key of ["packetLossPercent", "avgLatencyMs", "maxLatencyMs", "durationMs"]) {
const value = expect["maxPacketLoss"]; if (expect[key] !== undefined) {
if (!isNumber(value) || !Number.isFinite(value) || value < 0 || value > 100) { issues.push(...validateValueMatcher(expect[key], joinPath(expectPath, key), targetName));
issues.push(issue("invalid-value", joinPath(expectPath, "maxPacketLoss"), "必须为 0-100 的数字", targetName));
}
}
for (const key of ["maxAvgLatencyMs", "maxMaxLatencyMs", "maxDurationMs"]) {
if (expect[key] !== undefined && !isNonNegativeFiniteNumber(expect[key])) {
issues.push(issue("invalid-type", joinPath(expectPath, key), "必须为非负有限数字", targetName));
} }
} }
const allowedKeys = new Set(["alive", "maxAvgLatencyMs", "maxDurationMs", "maxMaxLatencyMs", "maxPacketLoss"]); const allowedKeys = new Set(["alive", "avgLatencyMs", "durationMs", "maxLatencyMs", "packetLossPercent"]);
for (const key of Object.keys(expect)) { for (const key of Object.keys(expect)) {
if (!allowedKeys.has(key)) { if (!allowedKeys.has(key)) {
issues.push(issue("unknown-field", joinPath(expectPath, key), "是未知字段", targetName)); issues.push(issue("unknown-field", joinPath(expectPath, key), "是未知字段", targetName));

20
src/server/checker/runner/llm/execute.ts
View File

@@ -7,8 +7,8 @@ import type { CheckResult, RawTargetConfig } from "../../types";
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types"; import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
import type { LlmCheckObservation, LlmExpectConfig, LlmTargetConfig, ResolvedLlmTarget } from "./types"; import type { LlmCheckObservation, LlmExpectConfig, LlmTargetConfig, ResolvedLlmTarget } from "./types";
import { checkDuration } from "../../expect/duration";
import { errorFailure } from "../../expect/failure"; import { errorFailure } from "../../expect/failure";
import { checkValueMatcher } from "../../expect/matcher";
import { runExpects } from "./expect"; import { runExpects } from "./expect";
import { import {
buildObservationFromApiCallError, buildObservationFromApiCallError,
@@ -54,7 +54,11 @@ export class LlmChecker implements CheckerDefinition<ResolvedLlmTarget> {
}; };
} }
const durationResult = checkDuration(durationMs, expect?.maxDurationMs); const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
message: "durationMs mismatch",
path: "durationMs",
phase: "duration",
});
const expectResult = runExpects(observation, expect); const expectResult = runExpects(observation, expect);
const failure = expectResult.failure ?? durationResult.failure; const failure = expectResult.failure ?? durationResult.failure;
@@ -209,7 +213,11 @@ export class LlmChecker implements CheckerDefinition<ResolvedLlmTarget> {
); );
const durationMs = Math.round(performance.now() - start); const durationMs = Math.round(performance.now() - start);
const durationResult = checkDuration(durationMs, expect?.maxDurationMs); const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
message: "durationMs mismatch",
path: "durationMs",
phase: "duration",
});
const expectResult = runExpects(observation, expect); const expectResult = runExpects(observation, expect);
const failure = expectResult.failure ?? durationResult.failure; const failure = expectResult.failure ?? durationResult.failure;
@@ -251,7 +259,11 @@ export class LlmChecker implements CheckerDefinition<ResolvedLlmTarget> {
); );
const durationMs = Math.round(performance.now() - start); const durationMs = Math.round(performance.now() - start);
const durationResult = checkDuration(durationMs, expect?.maxDurationMs); const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
message: "durationMs mismatch",
path: "durationMs",
phase: "duration",
});
const expectResult = runExpects(observation, expect); const expectResult = runExpects(observation, expect);
const failure = expectResult.failure ?? durationResult.failure; const failure = expectResult.failure ?? durationResult.failure;

120
src/server/checker/runner/llm/expect.ts
View File

@@ -1,11 +1,10 @@
import type { ExpectResult } from "../../expect/types"; import type { ExpectResult } from "../../expect/types";
import type { LlmCheckObservation, LlmExpectConfig } from "./types"; import type { LlmCheckObservation, LlmExpectConfig, LlmUsageExpect } from "./types";
import { checkDuration } from "../../expect/duration"; import { checkContentRules } from "../../expect/content";
import { mismatchFailure } from "../../expect/failure"; import { mismatchFailure } from "../../expect/failure";
import { applyOperator } from "../../expect/operator"; import { checkValueMatcher } from "../../expect/matcher";
import { checkHeaders, checkStatus } from "../http/expect"; import { checkHeaders, checkStatus } from "../http/expect";
import { checkOutputRules } from "./output";
export function checkStreamExpect(observation: LlmCheckObservation, expect: LlmExpectConfig): ExpectResult { export function checkStreamExpect(observation: LlmCheckObservation, expect: LlmExpectConfig): ExpectResult {
if (!observation.stream || !expect.stream) return { failure: null, matched: true }; if (!observation.stream || !expect.stream) return { failure: null, matched: true };
@@ -25,18 +24,11 @@ export function checkStreamExpect(observation: LlmCheckObservation, expect: LlmE
} }
if (expect.stream.firstTokenMs && observation.stream.firstTokenMs !== null) { if (expect.stream.firstTokenMs && observation.stream.firstTokenMs !== null) {
if (!applyOperator(observation.stream.firstTokenMs, expect.stream.firstTokenMs)) { return checkValueMatcher(observation.stream.firstTokenMs, expect.stream.firstTokenMs, {
return { message: "stream.firstTokenMs mismatch",
failure: mismatchFailure( path: "stream.firstTokenMs",
"stream", phase: "stream",
"stream.firstTokenMs", });
expect.stream.firstTokenMs,
observation.stream.firstTokenMs,
"stream.firstTokenMs mismatch",
),
matched: false,
};
}
} else if (expect.stream.firstTokenMs && observation.stream.firstTokenMs === null) { } else if (expect.stream.firstTokenMs && observation.stream.firstTokenMs === null) {
return { return {
failure: mismatchFailure( failure: mismatchFailure(
@@ -75,37 +67,25 @@ export function runExpects(observation: LlmCheckObservation, expect: LlmExpectCo
if (!streamResult.matched) return streamResult; if (!streamResult.matched) return streamResult;
} }
const outputResult = checkOutputRules(observation.outputText, expect.output); const outputResult = checkContentRules(observation.outputText, expect.output, { path: "output", phase: "output" });
if (!outputResult.matched) return outputResult; if (!outputResult.matched) return outputResult;
if (expect.finishReason !== undefined) { if (expect.finishReason !== undefined) {
if (observation.finishReason !== expect.finishReason) { const result = checkValueMatcher(observation.finishReason, expect.finishReason, {
return { message: "finishReason mismatch",
failure: mismatchFailure( path: "finishReason",
"finishReason", phase: "finishReason",
"finishReason", });
expect.finishReason, if (!result.matched) return result;
observation.finishReason,
"finishReason mismatch",
),
matched: false,
};
}
} }
if (expect.rawFinishReason !== undefined) { if (expect.rawFinishReason !== undefined) {
if (observation.rawFinishReason !== expect.rawFinishReason) { const result = checkValueMatcher(observation.rawFinishReason, expect.rawFinishReason, {
return { message: "rawFinishReason mismatch",
failure: mismatchFailure( path: "rawFinishReason",
"rawFinishReason", phase: "rawFinishReason",
"rawFinishReason", });
expect.rawFinishReason, if (!result.matched) return result;
observation.rawFinishReason,
"rawFinishReason mismatch",
),
matched: false,
};
}
} }
if (expect.usage && observation.usage) { if (expect.usage && observation.usage) {
@@ -118,51 +98,31 @@ export function runExpects(observation: LlmCheckObservation, expect: LlmExpectCo
function checkUsageExpect( function checkUsageExpect(
usage: { inputTokens: number; outputTokens: number; totalTokens: number }, usage: { inputTokens: number; outputTokens: number; totalTokens: number },
expectUsage: { inputTokens?: unknown; outputTokens?: unknown; totalTokens?: unknown }, expectUsage: LlmUsageExpect,
): ExpectResult { ): ExpectResult {
if (expectUsage.inputTokens !== undefined) { if (expectUsage.inputTokens !== undefined) {
if (!applyOperator(usage.inputTokens, expectUsage.inputTokens as Parameters<typeof applyOperator>[1])) { const result = checkValueMatcher(usage.inputTokens, expectUsage.inputTokens, {
return { message: "usage.inputTokens mismatch",
failure: mismatchFailure( path: "usage.inputTokens",
"usage", phase: "usage",
"usage.inputTokens", });
expectUsage.inputTokens, if (!result.matched) return result;
usage.inputTokens,
"usage.inputTokens mismatch",
),
matched: false,
};
}
} }
if (expectUsage.outputTokens !== undefined) { if (expectUsage.outputTokens !== undefined) {
if (!applyOperator(usage.outputTokens, expectUsage.outputTokens as Parameters<typeof applyOperator>[1])) { const result = checkValueMatcher(usage.outputTokens, expectUsage.outputTokens, {
return { message: "usage.outputTokens mismatch",
failure: mismatchFailure( path: "usage.outputTokens",
"usage", phase: "usage",
"usage.outputTokens", });
expectUsage.outputTokens, if (!result.matched) return result;
usage.outputTokens,
"usage.outputTokens mismatch",
),
matched: false,
};
}
} }
if (expectUsage.totalTokens !== undefined) { if (expectUsage.totalTokens !== undefined) {
if (!applyOperator(usage.totalTokens, expectUsage.totalTokens as Parameters<typeof applyOperator>[1])) { const result = checkValueMatcher(usage.totalTokens, expectUsage.totalTokens, {
return { message: "usage.totalTokens mismatch",
failure: mismatchFailure( path: "usage.totalTokens",
"usage", phase: "usage",
"usage.totalTokens", });
expectUsage.totalTokens, if (!result.matched) return result;
usage.totalTokens,
"usage.totalTokens mismatch",
),
matched: false,
};
}
} }
return { failure: null, matched: true }; return { failure: null, matched: true };
} }
export { checkDuration };

83
src/server/checker/runner/llm/output.ts
View File

@@ -1,83 +0,0 @@
import type { ExpectResult } from "../../expect/types";
import type { OutputRule } from "./types";
import { mismatchFailure } from "../../expect/failure";
import { applyOperator, evaluateJsonPath } from "../../expect/operator";
export function checkOutputRules(outputText: null | string, rules: OutputRule[] | undefined): ExpectResult {
if (!rules || rules.length === 0) return { failure: null, matched: true };
for (const rule of rules) {
const result = checkSingleOutputRule(outputText, rule);
if (!result.matched) return result;
}
return { failure: null, matched: true };
}
function checkSingleOutputRule(outputText: null | string, rule: OutputRule): ExpectResult {
if ("equals" in rule) {
if (outputText === null || outputText !== rule.equals) {
return {
failure: mismatchFailure("output", "output", rule.equals, outputText, "output equals mismatch"),
matched: false,
};
}
return { failure: null, matched: true };
}
if ("contains" in rule) {
if (!outputText?.includes(rule.contains)) {
return {
failure: mismatchFailure(
"output",
"output",
`contains: ${rule.contains}`,
outputText,
"output contains mismatch",
),
matched: false,
};
}
return { failure: null, matched: true };
}
if ("regex" in rule) {
if (outputText === null || !new RegExp(rule.regex).test(outputText)) {
return {
failure: mismatchFailure("output", "output", `match: ${rule.regex}`, outputText, "output regex mismatch"),
matched: false,
};
}
return { failure: null, matched: true };
}
if ("json" in rule) {
if (outputText === null) {
return {
failure: mismatchFailure("output", "output", "valid JSON", null, "output is null, cannot parse JSON"),
matched: false,
};
}
let parsed: unknown;
try {
parsed = JSON.parse(outputText);
} catch {
return {
failure: mismatchFailure("output", "output", "valid JSON", outputText, "output is not valid JSON"),
matched: false,
};
}
const value = evaluateJsonPath(parsed, rule.json.path);
if (!applyOperator(value, rule.json)) {
return {
failure: mismatchFailure("output", "output", rule.json, value, "output json mismatch"),
matched: false,
};
}
return { failure: null, matched: true };
}
return { failure: null, matched: true };
}

53
src/server/checker/runner/llm/schema.ts
View File

@@ -3,8 +3,9 @@ import { Type } from "@sinclair/typebox";
import type { CheckerSchemas } from "../types"; import type { CheckerSchemas } from "../types";
import { import {
createHeaderExpectSchema, createContentRulesSchema,
createPureOperatorSchema, createKeyValueExpectSchema,
createValueMatcherSchema,
statusCodePatternSchema, statusCodePatternSchema,
stringMapSchema, stringMapSchema,
} from "../../schema/fragments"; } from "../../schema/fragments";
@@ -25,36 +26,6 @@ function createLlmOptionsSchema() {
); );
} }
function createLlmOutputRulesSchema() {
return Type.Array(
Type.Object(
{
contains: Type.Optional(Type.String()),
equals: Type.Optional(Type.String()),
json: Type.Optional(
Type.Object({ path: Type.String(), ...operatorProperties() }, { additionalProperties: false }),
),
regex: Type.Optional(Type.String()),
},
{ additionalProperties: false },
),
);
}
function operatorProperties() {
return {
contains: Type.Optional(Type.String()),
empty: Type.Optional(Type.Boolean()),
equals: Type.Optional(Type.Number()),
exists: Type.Optional(Type.Boolean()),
gt: Type.Optional(Type.Number()),
gte: Type.Optional(Type.Number()),
lt: Type.Optional(Type.Number()),
lte: Type.Optional(Type.Number()),
match: Type.Optional(Type.String()),
};
}
export const llmCheckerSchemas: CheckerSchemas = { export const llmCheckerSchemas: CheckerSchemas = {
config: Type.Object( config: Type.Object(
{ {
@@ -84,17 +55,17 @@ export const llmCheckerSchemas: CheckerSchemas = {
), ),
expect: Type.Object( expect: Type.Object(
{ {
finishReason: Type.Optional(Type.String()), durationMs: Type.Optional(createValueMatcherSchema()),
headers: Type.Optional(createHeaderExpectSchema()), finishReason: Type.Optional(createValueMatcherSchema()),
maxDurationMs: Type.Optional(Type.Number({ minimum: 0 })), headers: Type.Optional(createKeyValueExpectSchema()),
output: Type.Optional(createLlmOutputRulesSchema()), output: Type.Optional(createContentRulesSchema()),
rawFinishReason: Type.Optional(Type.String()), rawFinishReason: Type.Optional(createValueMatcherSchema()),
status: Type.Optional(Type.Array(statusCodePatternSchema)), status: Type.Optional(Type.Array(statusCodePatternSchema)),
stream: Type.Optional( stream: Type.Optional(
Type.Object( Type.Object(
{ {
completed: Type.Optional(Type.Boolean()), completed: Type.Optional(Type.Boolean()),
firstTokenMs: Type.Optional(createPureOperatorSchema()), firstTokenMs: Type.Optional(createValueMatcherSchema()),
}, },
{ additionalProperties: false }, { additionalProperties: false },
), ),
@@ -102,9 +73,9 @@ export const llmCheckerSchemas: CheckerSchemas = {
usage: Type.Optional( usage: Type.Optional(
Type.Object( Type.Object(
{ {
inputTokens: Type.Optional(createPureOperatorSchema()), inputTokens: Type.Optional(createValueMatcherSchema()),
outputTokens: Type.Optional(createPureOperatorSchema()), outputTokens: Type.Optional(createValueMatcherSchema()),
totalTokens: Type.Optional(createPureOperatorSchema()), totalTokens: Type.Optional(createValueMatcherSchema()),
}, },
{ additionalProperties: false }, { additionalProperties: false },
), ),

27
src/server/checker/runner/llm/types.ts
View File

@@ -1,6 +1,7 @@
import type { JSONObject } from "@ai-sdk/provider"; import type { JSONObject } from "@ai-sdk/provider";
import type { ExpectOperator, ResolvedTargetBase } from "../../types"; import type { ContentRules, KeyValueExpect, ValueMatcher } from "../../expect/types";
import type { ResolvedTargetBase } from "../../types";
export interface LlmCheckObservation { export interface LlmCheckObservation {
finishReason: null | string; finishReason: null | string;
@@ -23,11 +24,11 @@ export interface LlmDefaultsConfig {
} }
export interface LlmExpectConfig { export interface LlmExpectConfig {
finishReason?: string; durationMs?: ValueMatcher;
headers?: Record<string, ExpectOperator | string>; finishReason?: ValueMatcher;
maxDurationMs?: number; headers?: KeyValueExpect;
output?: OutputRule[]; output?: ContentRules;
rawFinishReason?: string; rawFinishReason?: ValueMatcher;
status?: Array<number | string>; status?: Array<number | string>;
stream?: LlmStreamExpect; stream?: LlmStreamExpect;
usage?: LlmUsageExpect; usage?: LlmUsageExpect;
@@ -56,7 +57,7 @@ export type LlmProvider = "anthropic" | "openai" | "openai-responses";
export interface LlmStreamExpect { export interface LlmStreamExpect {
completed?: boolean; completed?: boolean;
firstTokenMs?: ExpectOperator; firstTokenMs?: ValueMatcher;
} }
export interface LlmStreamObservation { export interface LlmStreamObservation {
@@ -79,9 +80,9 @@ export interface LlmTargetConfig {
} }
export interface LlmUsageExpect { export interface LlmUsageExpect {
inputTokens?: ExpectOperator; inputTokens?: ValueMatcher;
outputTokens?: ExpectOperator; outputTokens?: ValueMatcher;
totalTokens?: ExpectOperator; totalTokens?: ValueMatcher;
} }
export interface LlmUsageObservation { export interface LlmUsageObservation {
@@ -90,12 +91,6 @@ export interface LlmUsageObservation {
totalTokens: number; totalTokens: number;
} }
export interface OutputJsonRule extends ExpectOperator {
path: string;
}
export type OutputRule = { contains: string } | { equals: string } | { json: OutputJsonRule } | { regex: string };
export interface ResolvedLlmConfig { export interface ResolvedLlmConfig {
authToken?: string; authToken?: string;
headers: Record<string, string>; headers: Record<string, string>;

165
src/server/checker/runner/llm/validate.ts
View File

@@ -1,17 +1,20 @@
import { isBoolean, isNumber, isPlainObject, isString } from "es-toolkit"; import { isBoolean, isNumber, isString } from "es-toolkit";
import { isArray } from "es-toolkit/compat"; import { isArray } from "es-toolkit/compat";
import type { ConfigValidationIssue } from "../../schema/issues"; import type { ConfigValidationIssue } from "../../schema/issues";
import type { CheckerValidationInput } from "../types"; import type { CheckerValidationInput } from "../types";
import { isUnsafeRegex } from "../../expect/redos"; import {
import { isPlainRecord, validateOperatorObject } from "../../expect/validate-operator"; isPlainRecord,
validateContentRules,
validateKeyValueExpect,
validateValueMatcher,
} from "../../expect/validate-matcher";
import { issue, joinPath } from "../../schema/issues"; import { issue, joinPath } from "../../schema/issues";
const ALLOWED_PROVIDERS = new Set(["anthropic", "openai", "openai-responses"]);
const ALLOWED_PROTOCOLS = new Set(["http:", "https:"]);
const ALLOWED_MODES = new Set(["http", "stream"]); const ALLOWED_MODES = new Set(["http", "stream"]);
const OUTPUT_RULE_KEYS = ["contains", "equals", "json", "regex"] as const; const ALLOWED_PROTOCOLS = new Set(["http:", "https:"]);
const ALLOWED_PROVIDERS = new Set(["anthropic", "openai", "openai-responses"]);
export function validateLlmConfig(input: CheckerValidationInput): ConfigValidationIssue[] { export function validateLlmConfig(input: CheckerValidationInput): ConfigValidationIssue[] {
const issues: ConfigValidationIssue[] = []; const issues: ConfigValidationIssue[] = [];
@@ -37,10 +40,6 @@ function getTargetName(target: Record<string, unknown>): string | undefined {
return isString(target["id"]) ? target["id"] : undefined; return isString(target["id"]) ? target["id"] : undefined;
} }
function isNonNegativeFiniteNumber(value: unknown): boolean {
return isNumber(value) && Number.isFinite(value) && value >= 0;
}
function validateLlmDefaults(defaults: Record<string, unknown>, path: string): ConfigValidationIssue[] { function validateLlmDefaults(defaults: Record<string, unknown>, path: string): ConfigValidationIssue[] {
const issues: ConfigValidationIssue[] = []; const issues: ConfigValidationIssue[] = [];
@@ -77,30 +76,23 @@ function validateLlmExpect(
if (isArray(expect["status"])) { if (isArray(expect["status"])) {
issues.push(...validateStatusValues(expect["status"], joinPath(expectPath, "status"), targetName)); issues.push(...validateStatusValues(expect["status"], joinPath(expectPath, "status"), targetName));
} }
if (expect["headers"] !== undefined) {
if (isPlainRecord(expect["headers"])) { issues.push(...validateKeyValueExpect(expect["headers"], joinPath(expectPath, "headers"), targetName));
for (const [key, value] of Object.entries(expect["headers"])) {
if (isString(value)) continue;
issues.push(...validateOperatorObject(value, joinPath(joinPath(expectPath, "headers"), key), targetName));
} }
}
if (expect["output"] !== undefined) { if (expect["output"] !== undefined) {
issues.push(...validateOutputRules(expect["output"], joinPath(expectPath, "output"), targetName)); issues.push(...validateContentRules(expect["output"], joinPath(expectPath, "output"), targetName));
} }
if (expect["finishReason"] !== undefined) {
if (expect["finishReason"] !== undefined && !isString(expect["finishReason"])) { issues.push(...validateValueMatcher(expect["finishReason"], joinPath(expectPath, "finishReason"), targetName));
issues.push(issue("invalid-type", joinPath(expectPath, "finishReason"), "必须为字符串", targetName));
} }
if (expect["rawFinishReason"] !== undefined) {
if (expect["rawFinishReason"] !== undefined && !isString(expect["rawFinishReason"])) { issues.push(
issues.push(issue("invalid-type", joinPath(expectPath, "rawFinishReason"), "必须为字符串", targetName)); ...validateValueMatcher(expect["rawFinishReason"], joinPath(expectPath, "rawFinishReason"), targetName),
);
} }
if (expect["usage"] !== undefined) { if (expect["usage"] !== undefined) {
issues.push(...validateUsageExpect(expect["usage"], joinPath(expectPath, "usage"), targetName)); issues.push(...validateUsageExpect(expect["usage"], joinPath(expectPath, "usage"), targetName));
} }
if (expect["stream"] !== undefined) { if (expect["stream"] !== undefined) {
if (mode === "http") { if (mode === "http") {
issues.push( issues.push(
@@ -110,9 +102,22 @@ function validateLlmExpect(
issues.push(...validateStreamExpect(expect["stream"], joinPath(expectPath, "stream"), targetName)); issues.push(...validateStreamExpect(expect["stream"], joinPath(expectPath, "stream"), targetName));
} }
} }
if (expect["durationMs"] !== undefined) {
issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
}
if (expect["maxDurationMs"] !== undefined && !isNonNegativeFiniteNumber(expect["maxDurationMs"])) { const allowedKeys = new Set([
issues.push(issue("invalid-type", joinPath(expectPath, "maxDurationMs"), "必须为非负有限数字", targetName)); "durationMs",
"finishReason",
"headers",
"output",
"rawFinishReason",
"status",
"stream",
"usage",
]);
for (const key of Object.keys(expect)) {
if (!allowedKeys.has(key)) issues.push(issue("unknown-field", joinPath(expectPath, key), "是未知字段", targetName));
} }
return issues; return issues;
@@ -197,27 +202,21 @@ function validateLlmTarget(target: Record<string, unknown>, path: string): Confi
if (!isString(llm["model"]) || llm["model"].trim() === "") { if (!isString(llm["model"]) || llm["model"].trim() === "") {
issues.push(issue("required", joinPath(joinPath(path, "llm"), "model"), "必须为非空字符串", targetName)); issues.push(issue("required", joinPath(joinPath(path, "llm"), "model"), "必须为非空字符串", targetName));
} }
if (!isString(llm["prompt"]) || llm["prompt"].trim() === "") { if (!isString(llm["prompt"]) || llm["prompt"].trim() === "") {
issues.push(issue("required", joinPath(joinPath(path, "llm"), "prompt"), "必须为非空字符串", targetName)); issues.push(issue("required", joinPath(joinPath(path, "llm"), "prompt"), "必须为非空字符串", targetName));
} }
if (llm["mode"] !== undefined && !ALLOWED_MODES.has(llm["mode"] as string)) { if (llm["mode"] !== undefined && !ALLOWED_MODES.has(llm["mode"] as string)) {
issues.push(issue("invalid-type", joinPath(joinPath(path, "llm"), "mode"), "必须为 http 或 stream", targetName)); issues.push(issue("invalid-type", joinPath(joinPath(path, "llm"), "mode"), "必须为 http 或 stream", targetName));
} }
if (llm["headers"] !== undefined) { if (llm["headers"] !== undefined) {
issues.push(...validateStringMap(llm["headers"], joinPath(joinPath(path, "llm"), "headers"), targetName)); issues.push(...validateStringMap(llm["headers"], joinPath(joinPath(path, "llm"), "headers"), targetName));
} }
if (llm["ignoreSSL"] !== undefined && !isBoolean(llm["ignoreSSL"])) { if (llm["ignoreSSL"] !== undefined && !isBoolean(llm["ignoreSSL"])) {
issues.push(issue("invalid-type", joinPath(joinPath(path, "llm"), "ignoreSSL"), "必须为布尔值", targetName)); issues.push(issue("invalid-type", joinPath(joinPath(path, "llm"), "ignoreSSL"), "必须为布尔值", targetName));
} }
const provider = llm["provider"] as string | undefined; const provider = llm["provider"] as string | undefined;
if (llm["authToken"] !== undefined && provider !== "anthropic") {
if (llm["authToken"] !== undefined) {
if (provider !== "anthropic") {
issues.push( issues.push(
issue( issue(
"invalid-auth", "invalid-auth",
@@ -227,8 +226,6 @@ function validateLlmTarget(target: Record<string, unknown>, path: string): Confi
), ),
); );
} }
}
if ( if (
provider === "anthropic" && provider === "anthropic" &&
isString(llm["key"]) && isString(llm["key"]) &&
@@ -240,11 +237,9 @@ function validateLlmTarget(target: Record<string, unknown>, path: string): Confi
issue("auth-conflict", joinPath(joinPath(path, "llm"), "key"), "key 与 authToken 不能同时配置", targetName), issue("auth-conflict", joinPath(joinPath(path, "llm"), "key"), "key 与 authToken 不能同时配置", targetName),
); );
} }
if (llm["options"] !== undefined) { if (llm["options"] !== undefined) {
issues.push(...validateLlmOptions(llm["options"], joinPath(joinPath(path, "llm"), "options"), targetName)); issues.push(...validateLlmOptions(llm["options"], joinPath(joinPath(path, "llm"), "options"), targetName));
} }
if (llm["providerOptions"] !== undefined) { if (llm["providerOptions"] !== undefined) {
issues.push( issues.push(
...validateProviderOptions( ...validateProviderOptions(
@@ -261,76 +256,11 @@ function validateLlmTarget(target: Record<string, unknown>, path: string): Confi
return issues; return issues;
} }
function validateOutputJsonRule(value: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isPlainRecord(value)) return [issue("invalid-type", path, "必须为对象", targetName)];
const issues: ConfigValidationIssue[] = [];
if (!isString(value["path"]) || !value["path"].startsWith("$.") || value["path"].length <= 2) {
issues.push(issue("invalid-jsonpath", joinPath(path, "path"), '必须为以 "$." 开头的有效 JSONPath', targetName));
}
const operatorKeys = new Set(["path"]);
const operators: Record<string, unknown> = {};
for (const [key, val] of Object.entries(value)) {
if (operatorKeys.has(key)) continue;
operators[key] = val;
}
issues.push(...validateOperatorObject(operators, path, targetName, { requireAtLeastOne: false }));
return issues;
}
function validateOutputRegex(value: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isString(value)) return [issue("invalid-type", path, "必须为字符串", targetName)];
try {
new RegExp(value);
} catch {
return [issue("invalid-regex", path, "正则不合法", targetName)];
}
return isUnsafeRegex(value) ? [issue("unsafe-regex", path, "正则存在 ReDoS 风险", targetName)] : [];
}
function validateOutputRules(rules: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isArray(rules)) return [issue("invalid-type", path, "必须为数组", targetName)];
return rules.flatMap((rule, index) => validateSingleOutputRule(rule, `${path}[${index}]`, targetName));
}
function validateProviderOptions(value: unknown, path: string, targetName?: string): ConfigValidationIssue[] { function validateProviderOptions(value: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isPlainObject(value)) return [issue("invalid-type", path, "必须为 JSON object", targetName)]; if (!isPlainRecord(value)) return [issue("invalid-type", path, "必须为 JSON object", targetName)];
return []; return [];
} }
function validateSingleOutputRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
const found = OUTPUT_RULE_KEYS.filter((type) => type in rule);
if (found.length === 0) return [issue("missing-body-rule", path, "缺少支持的规则类型", targetName)];
if (found.length > 1) return [issue("multiple-body-rules", path, "只能配置一种规则类型", targetName)];
const ruleType = found[0]!;
const issues: ConfigValidationIssue[] = [];
for (const key of Object.keys(rule)) {
if (key !== ruleType) issues.push(issue("unknown-field", joinPath(path, key), "是未知字段", targetName));
}
if (issues.length > 0) return issues;
switch (ruleType) {
case "contains":
return isString(rule["contains"])
? []
: [issue("invalid-type", joinPath(path, "contains"), "必须为字符串", targetName)];
case "equals":
return isString(rule["equals"])
? []
: [issue("invalid-type", joinPath(path, "equals"), "必须为字符串", targetName)];
case "json":
return validateOutputJsonRule(rule["json"], joinPath(path, "json"), targetName);
case "regex":
return validateOutputRegex(rule["regex"], joinPath(path, "regex"), targetName);
}
}
function validateStatusValues(values: unknown[], path: string, targetName?: string): ConfigValidationIssue[] { function validateStatusValues(values: unknown[], path: string, targetName?: string): ConfigValidationIssue[] {
const issues: ConfigValidationIssue[] = []; const issues: ConfigValidationIssue[] = [];
for (let i = 0; i < values.length; i++) { for (let i = 0; i < values.length; i++) {
@@ -360,18 +290,22 @@ function validateStreamExpect(stream: unknown, path: string, targetName?: string
if (stream["completed"] !== undefined && !isBoolean(stream["completed"])) { if (stream["completed"] !== undefined && !isBoolean(stream["completed"])) {
issues.push(issue("invalid-type", joinPath(path, "completed"), "必须为布尔值", targetName)); issues.push(issue("invalid-type", joinPath(path, "completed"), "必须为布尔值", targetName));
} }
if (stream["firstTokenMs"] !== undefined) { if (stream["firstTokenMs"] !== undefined) {
issues.push(...validateOperatorObject(stream["firstTokenMs"], joinPath(path, "firstTokenMs"), targetName)); issues.push(...validateValueMatcher(stream["firstTokenMs"], joinPath(path, "firstTokenMs"), targetName));
}
const allowedKeys = new Set(["completed", "firstTokenMs"]);
for (const key of Object.keys(stream)) {
if (!allowedKeys.has(key)) issues.push(issue("unknown-field", joinPath(path, key), "是未知字段", targetName));
} }
return issues; return issues;
} }
function validateStringMap(value: unknown, path: string, targetName?: string): ConfigValidationIssue[] { function validateStringMap(value: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
if (!isPlainObject(value)) return [issue("invalid-type", path, "必须为对象", targetName)]; if (!isPlainRecord(value)) return [issue("invalid-type", path, "必须为对象", targetName)];
const issues: ConfigValidationIssue[] = []; const issues: ConfigValidationIssue[] = [];
for (const [key, val] of Object.entries(value as Record<string, unknown>)) { for (const [key, val] of Object.entries(value)) {
if (!isString(val)) { if (!isString(val)) {
issues.push(issue("invalid-type", joinPath(path, key), "必须为字符串", targetName)); issues.push(issue("invalid-type", joinPath(path, key), "必须为字符串", targetName));
} }
@@ -383,14 +317,15 @@ function validateUsageExpect(usage: unknown, path: string, targetName?: string):
if (!isPlainRecord(usage)) return [issue("invalid-type", path, "必须为对象", targetName)]; if (!isPlainRecord(usage)) return [issue("invalid-type", path, "必须为对象", targetName)];
const issues: ConfigValidationIssue[] = []; const issues: ConfigValidationIssue[] = [];
if (usage["inputTokens"] !== undefined) { for (const key of ["inputTokens", "outputTokens", "totalTokens"]) {
issues.push(...validateOperatorObject(usage["inputTokens"], joinPath(path, "inputTokens"), targetName)); if (usage[key] !== undefined) {
issues.push(...validateValueMatcher(usage[key], joinPath(path, key), targetName));
} }
if (usage["outputTokens"] !== undefined) {
issues.push(...validateOperatorObject(usage["outputTokens"], joinPath(path, "outputTokens"), targetName));
} }
if (usage["totalTokens"] !== undefined) {
issues.push(...validateOperatorObject(usage["totalTokens"], joinPath(path, "totalTokens"), targetName)); const allowedKeys = new Set(["inputTokens", "outputTokens", "totalTokens"]);
for (const key of Object.keys(usage)) {
if (!allowedKeys.has(key)) issues.push(issue("unknown-field", joinPath(path, key), "是未知字段", targetName));
} }
return issues; return issues;

8
src/server/checker/runner/tcp/execute.ts
View File

@@ -4,8 +4,8 @@ import type { CheckResult, RawTargetConfig } from "../../types";
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types"; import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
import type { ResolvedTcpTarget, TcpExpectConfig, TcpTargetConfig } from "./types"; import type { ResolvedTcpTarget, TcpExpectConfig, TcpTargetConfig } from "./types";
import { checkDuration } from "../../expect/duration";
import { errorFailure } from "../../expect/failure"; import { errorFailure } from "../../expect/failure";
import { checkValueMatcher } from "../../expect/matcher";
import { parseSize } from "../../utils"; import { parseSize } from "../../utils";
import { checkBanner, checkConnected } from "./expect"; import { checkBanner, checkConnected } from "./expect";
import { tcpCheckerSchemas } from "./schema"; import { tcpCheckerSchemas } from "./schema";
@@ -124,7 +124,11 @@ export class TcpChecker implements CheckerDefinition<ResolvedTcpTarget> {
} }
const durationMs = Math.round(performance.now() - start); const durationMs = Math.round(performance.now() - start);
const durationResult = checkDuration(durationMs, expect?.maxDurationMs); const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
message: "durationMs mismatch",
path: "durationMs",
phase: "duration",
});
if (!durationResult.matched) { if (!durationResult.matched) {
return { return {
durationMs, durationMs,

16
src/server/checker/runner/tcp/expect.ts
View File

@@ -1,18 +1,10 @@
import type { ExpectResult } from "../../expect/types"; import type { ContentRules, ExpectResult } from "../../expect/types";
import type { ExpectOperator } from "../../types";
import { checkContentRules } from "../../expect/content";
import { mismatchFailure } from "../../expect/failure"; import { mismatchFailure } from "../../expect/failure";
import { applyOperator } from "../../expect/operator";
export function checkBanner(banner: string, op: ExpectOperator): ExpectResult { export function checkBanner(banner: string, rules: ContentRules): ExpectResult {
const matched = applyOperator(banner, op); return checkContentRules(banner, rules, { path: "banner", phase: "banner" });
if (!matched) {
return {
failure: mismatchFailure("banner", "banner", op, banner, `banner 不满足条件`),
matched: false,
};
}
return { failure: null, matched: true };
} }
export function checkConnected(connected: boolean, expected: boolean): ExpectResult { export function checkConnected(connected: boolean, expected: boolean): ExpectResult {

6
src/server/checker/runner/tcp/schema.ts
View File

@@ -2,7 +2,7 @@ import { Type } from "@sinclair/typebox";
import type { CheckerSchemas } from "../types"; import type { CheckerSchemas } from "../types";
import { createPureOperatorSchema, sizeSchema } from "../../schema/fragments"; import { createContentRulesSchema, createValueMatcherSchema, sizeSchema } from "../../schema/fragments";
export const tcpCheckerSchemas: CheckerSchemas = { export const tcpCheckerSchemas: CheckerSchemas = {
config: Type.Object( config: Type.Object(
@@ -24,9 +24,9 @@ export const tcpCheckerSchemas: CheckerSchemas = {
), ),
expect: Type.Object( expect: Type.Object(
{ {
banner: Type.Optional(createPureOperatorSchema()), banner: Type.Optional(createContentRulesSchema()),
connected: Type.Optional(Type.Boolean()), connected: Type.Optional(Type.Boolean()),
maxDurationMs: Type.Optional(Type.Number({ minimum: 0 })), durationMs: Type.Optional(createValueMatcherSchema()),
}, },
{ additionalProperties: false }, { additionalProperties: false },
), ),

7
src/server/checker/runner/tcp/types.ts
View File

@@ -1,4 +1,5 @@
import type { ExpectOperator, ResolvedTargetBase } from "../../types"; import type { ContentRules, ValueMatcher } from "../../expect/types";
import type { ResolvedTargetBase } from "../../types";
export interface ResolvedTcpConfig { export interface ResolvedTcpConfig {
bannerReadTimeout: number; bannerReadTimeout: number;
@@ -24,9 +25,9 @@ export interface TcpDefaultsConfig {
} }
export interface TcpExpectConfig { export interface TcpExpectConfig {
banner?: ExpectOperator; banner?: ContentRules;
connected?: boolean; connected?: boolean;
maxDurationMs?: number; durationMs?: ValueMatcher;
} }
export interface TcpTargetConfig { export interface TcpTargetConfig {

10
src/server/checker/runner/tcp/validate.ts
View File

@@ -3,7 +3,7 @@ import { isNumber, isPlainObject, isString } from "es-toolkit";
import type { ConfigValidationIssue } from "../../schema/issues"; import type { ConfigValidationIssue } from "../../schema/issues";
import type { CheckerValidationInput } from "../types"; import type { CheckerValidationInput } from "../types";
import { validateOperatorObject } from "../../expect/validate-operator"; import { validateContentRules, validateValueMatcher } from "../../expect/validate-matcher";
import { issue, joinPath } from "../../schema/issues"; import { issue, joinPath } from "../../schema/issues";
export function validateTcpConfig(input: CheckerValidationInput): ConfigValidationIssue[] { export function validateTcpConfig(input: CheckerValidationInput): ConfigValidationIssue[] {
@@ -79,8 +79,8 @@ function validateTcpExpect(
issues.push(issue("invalid-type", joinPath(expectPath, "connected"), "必须为布尔值", targetName)); issues.push(issue("invalid-type", joinPath(expectPath, "connected"), "必须为布尔值", targetName));
} }
if (expect["maxDurationMs"] !== undefined && !isNonNegativeFiniteNumber(expect["maxDurationMs"])) { if (expect["durationMs"] !== undefined) {
issues.push(issue("invalid-type", joinPath(expectPath, "maxDurationMs"), "必须为非负有限数字", targetName)); issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
} }
if (expect["banner"] !== undefined) { if (expect["banner"] !== undefined) {
@@ -89,11 +89,11 @@ function validateTcpExpect(
issue("invalid-value", joinPath(expectPath, "banner"), "banner 断言需要启用 tcp.readBanner", targetName), issue("invalid-value", joinPath(expectPath, "banner"), "banner 断言需要启用 tcp.readBanner", targetName),
); );
} else { } else {
issues.push(...validateOperatorObject(expect["banner"], joinPath(expectPath, "banner"), targetName)); issues.push(...validateContentRules(expect["banner"], joinPath(expectPath, "banner"), targetName));
} }
} }
const allowedKeys = new Set(["banner", "connected", "maxDurationMs"]); const allowedKeys = new Set(["banner", "connected", "durationMs"]);
for (const key of Object.keys(expect)) { for (const key of Object.keys(expect)) {
if (!allowedKeys.has(key)) { if (!allowedKeys.has(key)) {
issues.push(issue("unknown-field", joinPath(expectPath, key), "是未知字段", targetName)); issues.push(issue("unknown-field", joinPath(expectPath, key), "是未知字段", targetName));

14
src/server/checker/runner/udp/execute.ts
View File

@@ -4,8 +4,8 @@ import type { CheckResult, RawTargetConfig } from "../../types";
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types"; import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
import type { ResolvedUdpTarget, UdpDefaultsConfig, UdpExpectConfig, UdpTargetConfig } from "./types"; import type { ResolvedUdpTarget, UdpDefaultsConfig, UdpExpectConfig, UdpTargetConfig } from "./types";
import { checkDuration } from "../../expect/duration";
import { errorFailure } from "../../expect/failure"; import { errorFailure } from "../../expect/failure";
import { checkValueMatcher } from "../../expect/matcher";
import { parseSize } from "../../utils"; import { parseSize } from "../../utils";
import { decodePayload, encodeResponse } from "./encoding"; import { decodePayload, encodeResponse } from "./encoding";
import { checkResponded, checkResponseSize, checkResponseText, checkSourceHost, checkSourcePort } from "./expect"; import { checkResponded, checkResponseSize, checkResponseText, checkSourceHost, checkSourcePort } from "./expect";
@@ -83,7 +83,11 @@ export class UdpChecker implements CheckerDefinition<ResolvedUdpTarget> {
if (!exchangeResult.responded) { if (!exchangeResult.responded) {
const durationMs = Math.round(performance.now() - start); const durationMs = Math.round(performance.now() - start);
const durationResult = checkDuration(durationMs, expect?.maxDurationMs); const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
message: "durationMs mismatch",
path: "durationMs",
phase: "duration",
});
if (!durationResult.matched) { if (!durationResult.matched) {
return { return {
durationMs, durationMs,
@@ -194,7 +198,11 @@ export class UdpChecker implements CheckerDefinition<ResolvedUdpTarget> {
} }
const durationMs = Math.round(performance.now() - start); const durationMs = Math.round(performance.now() - start);
const durationResult = checkDuration(durationMs, expect?.maxDurationMs); const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
message: "durationMs mismatch",
path: "durationMs",
phase: "duration",
});
if (!durationResult.matched) { if (!durationResult.matched) {
return { return {
durationMs, durationMs,

65
src/server/checker/runner/udp/expect.ts
View File

@@ -1,8 +1,8 @@
import type { ExpectResult } from "../../expect/types"; import type { ContentRules, ExpectResult, ValueMatcher } from "../../expect/types";
import type { ExpectOperator } from "../../types";
import { checkContentRules } from "../../expect/content";
import { mismatchFailure } from "../../expect/failure"; import { mismatchFailure } from "../../expect/failure";
import { applyOperator } from "../../expect/operator"; import { checkValueMatcher } from "../../expect/matcher";
export function checkResponded(responded: boolean, expected: boolean): ExpectResult { export function checkResponded(responded: boolean, expected: boolean): ExpectResult {
if (responded === expected) return { failure: null, matched: true }; if (responded === expected) return { failure: null, matched: true };
@@ -18,49 +18,30 @@ export function checkResponded(responded: boolean, expected: boolean): ExpectRes
}; };
} }
export function checkResponseSize(size: number, op: ExpectOperator): ExpectResult { export function checkResponseSize(size: number, matcher: ValueMatcher): ExpectResult {
const matched = applyOperator(size, op); return checkValueMatcher(size, matcher, {
if (!matched) { message: "响应大小不满足条件",
return { path: "responseSize",
failure: mismatchFailure("responseSize", "responseSize", op, size, "响应大小不满足条件"), phase: "responseSize",
matched: false, });
};
}
return { failure: null, matched: true };
} }
export function checkResponseText(text: string, rules: ExpectOperator[]): ExpectResult { export function checkResponseText(text: string, rules: ContentRules): ExpectResult {
for (let i = 0; i < rules.length; i++) { return checkContentRules(text, rules, { path: "response", phase: "response" });
const rule = rules[i]!;
const path = `response[${i}]`;
if (!applyOperator(text, rule)) {
return {
failure: mismatchFailure("response", path, rule, text, `response rule at index ${i} mismatch`),
matched: false,
};
}
}
return { failure: null, matched: true };
} }
export function checkSourceHost(actual: string, op: ExpectOperator): ExpectResult { export function checkSourceHost(actual: string, matcher: ValueMatcher): ExpectResult {
const matched = applyOperator(actual, op); return checkValueMatcher(actual, matcher, {
if (!matched) { message: "响应来源地址不满足条件",
return { path: "sourceHost",
failure: mismatchFailure("sourceHost", "sourceHost", op, actual, "响应来源地址不满足条件"), phase: "sourceHost",
matched: false, });
};
}
return { failure: null, matched: true };
} }
export function checkSourcePort(actual: number, op: ExpectOperator): ExpectResult { export function checkSourcePort(actual: number, matcher: ValueMatcher): ExpectResult {
const matched = applyOperator(actual, op); return checkValueMatcher(actual, matcher, {
if (!matched) { message: "响应来源端口不满足条件",
return { path: "sourcePort",
failure: mismatchFailure("sourcePort", "sourcePort", op, actual, "响应来源端口不满足条件"), phase: "sourcePort",
matched: false, });
};
}
return { failure: null, matched: true };
} }

12
src/server/checker/runner/udp/schema.ts
View File

@@ -2,7 +2,7 @@ import { Type } from "@sinclair/typebox";
import type { CheckerSchemas } from "../types"; import type { CheckerSchemas } from "../types";
import { createPureOperatorSchema, createTextRulesSchema, sizeSchema } from "../../schema/fragments"; import { createContentRulesSchema, createValueMatcherSchema, sizeSchema } from "../../schema/fragments";
export const udpCheckerSchemas: CheckerSchemas = { export const udpCheckerSchemas: CheckerSchemas = {
config: Type.Object( config: Type.Object(
@@ -26,12 +26,12 @@ export const udpCheckerSchemas: CheckerSchemas = {
), ),
expect: Type.Object( expect: Type.Object(
{ {
maxDurationMs: Type.Optional(Type.Number({ minimum: 0 })), durationMs: Type.Optional(createValueMatcherSchema()),
responded: Type.Optional(Type.Boolean()), responded: Type.Optional(Type.Boolean()),
response: Type.Optional(createTextRulesSchema()), response: Type.Optional(createContentRulesSchema()),
responseSize: Type.Optional(createPureOperatorSchema()), responseSize: Type.Optional(createValueMatcherSchema()),
sourceHost: Type.Optional(createPureOperatorSchema()), sourceHost: Type.Optional(createValueMatcherSchema()),
sourcePort: Type.Optional(createPureOperatorSchema()), sourcePort: Type.Optional(createValueMatcherSchema()),
}, },
{ additionalProperties: false }, { additionalProperties: false },
), ),

13
src/server/checker/runner/udp/types.ts
View File

@@ -1,4 +1,5 @@
import type { ExpectOperator, ResolvedTargetBase } from "../../types"; import type { ContentRules, ValueMatcher } from "../../expect/types";
import type { ResolvedTargetBase } from "../../types";
export interface ResolvedUdpConfig { export interface ResolvedUdpConfig {
encoding: UdpEncoding; encoding: UdpEncoding;
@@ -28,12 +29,12 @@ export interface UdpDefaultsConfig {
export type UdpEncoding = "base64" | "hex" | "text"; export type UdpEncoding = "base64" | "hex" | "text";
export interface UdpExpectConfig { export interface UdpExpectConfig {
maxDurationMs?: number; durationMs?: ValueMatcher;
responded?: boolean; responded?: boolean;
response?: ExpectOperator[]; response?: ContentRules;
responseSize?: ExpectOperator; responseSize?: ValueMatcher;
sourceHost?: ExpectOperator; sourceHost?: ValueMatcher;
sourcePort?: ExpectOperator; sourcePort?: ValueMatcher;
} }
export interface UdpTargetConfig { export interface UdpTargetConfig {

36
src/server/checker/runner/udp/validate.ts
View File

@@ -3,7 +3,7 @@ import { isNumber, isPlainObject, isString } from "es-toolkit";
import type { ConfigValidationIssue } from "../../schema/issues"; import type { ConfigValidationIssue } from "../../schema/issues";
import type { CheckerValidationInput } from "../types"; import type { CheckerValidationInput } from "../types";
import { validateOperatorObject } from "../../expect/validate-operator"; import { validateContentRules, validateValueMatcher } from "../../expect/validate-matcher";
import { issue, joinPath } from "../../schema/issues"; import { issue, joinPath } from "../../schema/issues";
const VALID_ENCODINGS = new Set(["base64", "hex", "text"]); const VALID_ENCODINGS = new Set(["base64", "hex", "text"]);
@@ -28,10 +28,6 @@ function getTargetName(target: Record<string, unknown>): string | undefined {
return isString(target["id"]) ? target["id"] : undefined; return isString(target["id"]) ? target["id"] : undefined;
} }
function isNonNegativeFiniteNumber(value: unknown): boolean {
return isNumber(value) && Number.isFinite(value) && value >= 0;
}
function validateEncoding(value: unknown, path: string, targetName: string | undefined): ConfigValidationIssue[] { function validateEncoding(value: unknown, path: string, targetName: string | undefined): ConfigValidationIssue[] {
if (value === undefined) return []; if (value === undefined) return [];
if (!isString(value) || !VALID_ENCODINGS.has(value)) { if (!isString(value) || !VALID_ENCODINGS.has(value)) {
@@ -48,22 +44,6 @@ function validateSize(value: unknown, path: string, targetName: string | undefin
return []; return [];
} }
function validateTextRulesArray(value: unknown, path: string, targetName: string | undefined): ConfigValidationIssue[] {
if (!Array.isArray(value)) {
return [issue("invalid-type", path, "必须为数组", targetName)];
}
const issues: ConfigValidationIssue[] = [];
for (let i = 0; i < value.length; i++) {
const rule: unknown = value[i];
if (!isPlainObject(rule)) {
issues.push(issue("invalid-type", joinPath(path, `[${i}]`), "必须为 operator 对象", targetName));
continue;
}
issues.push(...validateOperatorObject(rule, joinPath(path, `[${i}]`), targetName));
}
return issues;
}
function validateUdpDefaults(input: CheckerValidationInput): ConfigValidationIssue[] { function validateUdpDefaults(input: CheckerValidationInput): ConfigValidationIssue[] {
const issues: ConfigValidationIssue[] = []; const issues: ConfigValidationIssue[] = [];
const defaults = input.defaults["udp"]; const defaults = input.defaults["udp"];
@@ -99,24 +79,24 @@ function validateUdpExpect(target: Record<string, unknown>, path: string): Confi
issues.push(issue("invalid-type", joinPath(expectPath, "responded"), "必须为布尔值", targetName)); issues.push(issue("invalid-type", joinPath(expectPath, "responded"), "必须为布尔值", targetName));
} }
if (expect["maxDurationMs"] !== undefined && !isNonNegativeFiniteNumber(expect["maxDurationMs"])) { if (expect["durationMs"] !== undefined) {
issues.push(issue("invalid-type", joinPath(expectPath, "maxDurationMs"), "必须为非负有限数字", targetName)); issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
} }
if (expect["response"] !== undefined) { if (expect["response"] !== undefined) {
issues.push(...validateTextRulesArray(expect["response"], joinPath(expectPath, "response"), targetName)); issues.push(...validateContentRules(expect["response"], joinPath(expectPath, "response"), targetName));
} }
if (expect["responseSize"] !== undefined) { if (expect["responseSize"] !== undefined) {
issues.push(...validateOperatorObject(expect["responseSize"], joinPath(expectPath, "responseSize"), targetName)); issues.push(...validateValueMatcher(expect["responseSize"], joinPath(expectPath, "responseSize"), targetName));
} }
if (expect["sourceHost"] !== undefined) { if (expect["sourceHost"] !== undefined) {
issues.push(...validateOperatorObject(expect["sourceHost"], joinPath(expectPath, "sourceHost"), targetName)); issues.push(...validateValueMatcher(expect["sourceHost"], joinPath(expectPath, "sourceHost"), targetName));
} }
if (expect["sourcePort"] !== undefined) { if (expect["sourcePort"] !== undefined) {
issues.push(...validateOperatorObject(expect["sourcePort"], joinPath(expectPath, "sourcePort"), targetName)); issues.push(...validateValueMatcher(expect["sourcePort"], joinPath(expectPath, "sourcePort"), targetName));
} }
const respondedFalse = responded === false; const respondedFalse = responded === false;
@@ -143,7 +123,7 @@ function validateUdpExpect(target: Record<string, unknown>, path: string): Confi
} }
} }
const allowedKeys = new Set(["maxDurationMs", "responded", "response", "responseSize", "sourceHost", "sourcePort"]); const allowedKeys = new Set(["durationMs", "responded", "response", "responseSize", "sourceHost", "sourcePort"]);
for (const key of Object.keys(expect)) { for (const key of Object.keys(expect)) {
if (!allowedKeys.has(key)) { if (!allowedKeys.has(key)) {
issues.push(issue("unknown-field", joinPath(expectPath, key), "是未知字段", targetName)); issues.push(issue("unknown-field", joinPath(expectPath, key), "是未知字段", targetName));

14
src/server/checker/schema/builder.ts
View File

@@ -4,14 +4,24 @@ import { Type } from "@sinclair/typebox";
import type { CheckerDefinition } from "../runner/types"; import type { CheckerDefinition } from "../runner/types";
import { durationSchema, variableValueSchema } from "./fragments"; import {
createContentRulesSchema,
createKeyValueExpectSchema,
createValueMatcherSchema,
durationSchema,
variableValueSchema,
} from "./fragments";
export function createExternalProbeConfigSchema(checkers: CheckerDefinition[]): Record<string, unknown> { export function createExternalProbeConfigSchema(checkers: CheckerDefinition[]): Record<string, unknown> {
return { return {
...cloneSchema(createProbeConfigSchema(checkers, true)), ...cloneSchema(createProbeConfigSchema(checkers, true)),
$id: "https://dial.local/probe-config.schema.json", $id: "https://dial.local/probe-config.schema.json",
$schema: "http://json-schema.org/draft-07/schema#", $schema: "http://json-schema.org/draft-07/schema#",
definitions: {}, definitions: {
ContentRules: cloneSchema(createContentRulesSchema()),
KeyValueExpect: cloneSchema(createKeyValueExpectSchema()),
ValueMatcher: cloneSchema(createValueMatcherSchema()),
},
}; };
} }

36
src/server/checker/schema/fragments.ts
View File

@@ -6,9 +6,7 @@ import type { JsonValue } from "./types";
export const HTTP_METHODS = ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"] as const; export const HTTP_METHODS = ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"] as const;
export const BodyRuleTypeKeys = ["contains", "regex", "json", "css", "xpath"] as const; export const MatcherKeys = ["contains", "empty", "equals", "exists", "gt", "gte", "lt", "lte", "regex"] as const;
export const OperatorKeys = ["contains", "empty", "equals", "exists", "gt", "gte", "lt", "lte", "match"] as const;
export const durationSchema = Type.String(); export const durationSchema = Type.String();
@@ -41,51 +39,43 @@ export const stringMapSchema = Type.Unsafe<Record<string, string>>({
type: "object", type: "object",
}); });
export function createBodyRulesSchema(): TSchema { export function createContentRulesSchema(): TSchema {
return Type.Array( return Type.Array(
Type.Object( Type.Object(
{ {
contains: Type.Optional(Type.String()), ...matcherProperties(),
css: Type.Optional( css: Type.Optional(
Type.Object( Type.Object(
{ attr: Type.Optional(Type.String()), selector: Type.String({ minLength: 1 }), ...operatorProperties() }, { attr: Type.Optional(Type.String()), selector: Type.String({ minLength: 1 }), ...matcherProperties() },
{ additionalProperties: false }, { additionalProperties: false },
), ),
), ),
json: Type.Optional( json: Type.Optional(
Type.Object({ path: Type.String(), ...operatorProperties() }, { additionalProperties: false }), Type.Object({ path: Type.String(), ...matcherProperties() }, { additionalProperties: false }),
), ),
regex: Type.Optional(Type.String()),
xpath: Type.Optional( xpath: Type.Optional(
Type.Object( Type.Object({ path: Type.String({ minLength: 1 }), ...matcherProperties() }, { additionalProperties: false }),
{ path: Type.String({ minLength: 1 }), ...operatorProperties() },
{ additionalProperties: false },
),
), ),
}, },
{ additionalProperties: false }, { additionalProperties: false, minProperties: 1 },
), ),
); );
} }
export function createHeaderExpectSchema(): TSchema { export function createKeyValueExpectSchema(): TSchema {
return Type.Unsafe<Record<string, unknown>>({ return Type.Unsafe<Record<string, unknown>>({
additionalProperties: { additionalProperties: {
anyOf: [{ type: "string" }, createPureOperatorSchema()], anyOf: [jsonValueSchema, createValueMatcherSchema()],
}, },
type: "object", type: "object",
}); });
} }
export function createPureOperatorSchema(): TSchema { export function createValueMatcherSchema(): TSchema {
return Type.Object(operatorProperties(), { additionalProperties: false, minProperties: 1 }); return Type.Object(matcherProperties(), { additionalProperties: false, minProperties: 1 });
} }
export function createTextRulesSchema(): TSchema { export function matcherProperties(): Record<string, TSchema> {
return Type.Array(createPureOperatorSchema());
}
export function operatorProperties(): Record<string, TSchema> {
return { return {
contains: Type.Optional(Type.String()), contains: Type.Optional(Type.String()),
empty: Type.Optional(Type.Boolean()), empty: Type.Optional(Type.Boolean()),
@@ -95,6 +85,6 @@ export function operatorProperties(): Record<string, TSchema> {
gte: Type.Optional(Type.Number()), gte: Type.Optional(Type.Number()),
lt: Type.Optional(Type.Number()), lt: Type.Optional(Type.Number()),
lte: Type.Optional(Type.Number()), lte: Type.Optional(Type.Number()),
match: Type.Optional(Type.String()), regex: Type.Optional(Type.String()),
}; };
} }

14
src/server/checker/types.ts
View File

@@ -15,20 +15,6 @@ export interface EngineRuntimeConfig {
retention?: string; retention?: string;
} }
export interface ExpectOperator {
contains?: string;
empty?: boolean;
equals?: JsonValue;
exists?: boolean;
gt?: number;
gte?: number;
lt?: number;
lte?: number;
match?: string;
}
export type ExpectValue = ExpectOperator | JsonValue;
export type JsonValue = boolean | JsonValue[] | null | number | string | { [key: string]: JsonValue }; export type JsonValue = boolean | JsonValue[] | null | number | string | { [key: string]: JsonValue };
export interface ProbeConfig { export interface ProbeConfig {

72
tests/server/checker/config-loader.test.ts
View File

@@ -824,7 +824,8 @@ targets:
- json: - json:
path: "$.status" path: "$.status"
equals: "ok" equals: "ok"
maxDurationMs: 3000 durationMs:
lte: 3000
`, `,
); );
@@ -833,7 +834,7 @@ targets:
if (t.type === "http") { if (t.type === "http") {
expect(t.expect).toEqual({ expect(t.expect).toEqual({
body: [{ contains: "ok" }, { json: { equals: "ok", path: "$.status" } }], body: [{ contains: "ok" }, { json: { equals: "ok", path: "$.status" } }],
maxDurationMs: 3000, durationMs: { lte: 3000 },
status: [200, 201], status: [200, 201],
}); });
} }
@@ -853,10 +854,11 @@ targets:
exitCode: [0, 2] exitCode: [0, 2]
stdout: stdout:
- contains: "ok" - contains: "ok"
- match: "done" - regex: "done"
stderr: stderr:
- empty: true - empty: true
maxDurationMs: 5000 durationMs:
lte: 5000
`, `,
); );
@@ -864,10 +866,10 @@ targets:
const t = config.targets[0]!; const t = config.targets[0]!;
if (t.type === "cmd") { if (t.type === "cmd") {
expect(t.expect).toEqual({ expect(t.expect).toEqual({
durationMs: { lte: 5000 },
exitCode: [0, 2], exitCode: [0, 2],
maxDurationMs: 5000,
stderr: [{ empty: true }], stderr: [{ empty: true }],
stdout: [{ contains: "ok" }, { match: "done" }], stdout: [{ contains: "ok" }, { regex: "done" }],
}); });
} }
}); });
@@ -1074,7 +1076,7 @@ targets:
await expect(loadConfig(configPath)).rejects.toThrow("5xx"); await expect(loadConfig(configPath)).rejects.toThrow("5xx");
}); });
test("expect.maxDurationMs 负数抛出错误", async () => { test("expect.durationMs 非 matcher 抛出错误", async () => {
const configPath = join(tempDir, "neg-duration.yaml"); const configPath = join(tempDir, "neg-duration.yaml");
await writeFile( await writeFile(
configPath, configPath,
@@ -1085,11 +1087,11 @@ targets:
http: http:
url: "http://example.com" url: "http://example.com"
expect: expect:
maxDurationMs: -100 durationMs: -100
`, `,
); );
// eslint-disable-next-line @typescript-eslint/await-thenable // eslint-disable-next-line @typescript-eslint/await-thenable
await expect(loadConfig(configPath)).rejects.toThrow("maxDurationMs 必须为非负有限数字"); await expect(loadConfig(configPath)).rejects.toThrow("expect.durationMs 必须为 matcher 对象");
}); });
test("expect.body 非数组抛出错误", async () => { test("expect.body 非数组抛出错误", async () => {
@@ -1126,7 +1128,7 @@ targets:
`, `,
); );
// eslint-disable-next-line @typescript-eslint/await-thenable // eslint-disable-next-line @typescript-eslint/await-thenable
await expect(loadConfig(configPath)).rejects.toThrow("缺少支持的规则类型"); await expect(loadConfig(configPath)).rejects.toThrow("foo 是未知字段");
}); });
test("body rule 使用 match 字段（非支持）抛出错误", async () => { test("body rule 使用 match 字段（非支持）抛出错误", async () => {
@@ -1145,10 +1147,10 @@ targets:
`, `,
); );
// eslint-disable-next-line @typescript-eslint/await-thenable // eslint-disable-next-line @typescript-eslint/await-thenable
await expect(loadConfig(configPath)).rejects.toThrow("缺少支持的规则类型"); await expect(loadConfig(configPath)).rejects.toThrow("match 是未知字段");
}); });
test("body rule 多个支持字段抛出错误", async () => { test("body rule 直接 matcher 混入 extractor 抛出错误", async () => {
const configPath = join(tempDir, "bad-body-rule-multi.yaml"); const configPath = join(tempDir, "bad-body-rule-multi.yaml");
await writeFile( await writeFile(
configPath, configPath,
@@ -1161,11 +1163,12 @@ targets:
expect: expect:
body: body:
- contains: "ok" - contains: "ok"
regex: "ok" json:
path: "$.status"
`, `,
); );
// eslint-disable-next-line @typescript-eslint/await-thenable // eslint-disable-next-line @typescript-eslint/await-thenable
await expect(loadConfig(configPath)).rejects.toThrow("只能配置一种规则类型"); await expect(loadConfig(configPath)).rejects.toThrow("直接 matcher 不能与 extractor 混用");
}); });
test("body regex 非法正则抛出错误", async () => { test("body regex 非法正则抛出错误", async () => {
@@ -1228,7 +1231,7 @@ targets:
await expect(loadConfig(configPath)).rejects.toThrow("css.selector 必须为非空字符串"); await expect(loadConfig(configPath)).rejects.toThrow("css.selector 必须为非空字符串");
}); });
test("operator match 非法正则抛出错误", async () => { test("旧 match matcher 抛出错误", async () => {
const configPath = join(tempDir, "bad-op-match.yaml"); const configPath = join(tempDir, "bad-op-match.yaml");
await writeFile( await writeFile(
configPath, configPath,
@@ -1245,7 +1248,7 @@ targets:
`, `,
); );
// eslint-disable-next-line @typescript-eslint/await-thenable // eslint-disable-next-line @typescript-eslint/await-thenable
await expect(loadConfig(configPath)).rejects.toThrow("match 正则不合法"); await expect(loadConfig(configPath)).rejects.toThrow("match 是未知 matcher");
}); });
test("operator gte 非数字抛出错误", async () => { test("operator gte 非数字抛出错误", async () => {
@@ -1531,7 +1534,7 @@ targets:
stdout: stdout:
- {} - {}
`, `,
"stdout[0] 必须包含至少一个合法 operator", "stdout[0] 必须包含至少一个合法 matcher",
); );
}); });
@@ -1552,7 +1555,7 @@ targets:
); );
}); });
test("cmd stdout match 正则非法", async () => { test("cmd stdout 旧 match 字段非法", async () => {
await expectConfigError( await expectConfigError(
"bad-cmd-stdout-regex.yaml", "bad-cmd-stdout-regex.yaml",
`targets: `targets:
@@ -1565,7 +1568,7 @@ targets:
stdout: stdout:
- match: "[invalid" - match: "[invalid"
`, `,
"stdout[0].match 正则不合法", "stdout[0].match 是未知字段",
); );
}); });
@@ -1878,14 +1881,14 @@ targets:
readBanner: true readBanner: true
expect: expect:
banner: banner:
contains: "ESMTP" - contains: "ESMTP"
`, `,
); );
const config = await loadConfig(configPath); const config = await loadConfig(configPath);
const t = config.targets[0]! as ResolvedTcpTarget; const t = config.targets[0]! as ResolvedTcpTarget;
expect(t.tcp.readBanner).toBe(true); expect(t.tcp.readBanner).toBe(true);
expect(t.expect?.banner).toEqual({ contains: "ESMTP" }); expect(t.expect?.banner).toEqual([{ contains: "ESMTP" }]);
}); });
test("tcp expect.banner 未开启 readBanner 抛出错误", async () => { test("tcp expect.banner 未开启 readBanner 抛出错误", async () => {
@@ -1954,7 +1957,7 @@ targets:
); );
}); });
test("tcp expect connected 和 maxDurationMs", async () => { test("tcp expect connected 和 durationMs", async () => {
const configPath = join(tempDir, "tcp-expect-connected.yaml"); const configPath = join(tempDir, "tcp-expect-connected.yaml");
await writeFile( await writeFile(
configPath, configPath,
@@ -1966,14 +1969,15 @@ targets:
port: 80 port: 80
expect: expect:
connected: false connected: false
maxDurationMs: 5000 durationMs:
lte: 5000
`, `,
); );
const config = await loadConfig(configPath); const config = await loadConfig(configPath);
const t = config.targets[0]! as ResolvedTcpTarget; const t = config.targets[0]! as ResolvedTcpTarget;
expect(t.expect?.connected).toBe(false); expect(t.expect?.connected).toBe(false);
expect(t.expect?.maxDurationMs).toBe(5000); expect(t.expect?.durationMs).toEqual({ lte: 5000 });
}); });
test("解析最简 ping 配置", async () => { test("解析最简 ping 配置", async () => {
@@ -2011,10 +2015,14 @@ targets:
packetSize: 1472 packetSize: 1472
expect: expect:
alive: true alive: true
maxPacketLoss: 10 packetLossPercent:
maxAvgLatencyMs: 200 lte: 10
maxMaxLatencyMs: 500 avgLatencyMs:
maxDurationMs: 5000 lte: 200
maxLatencyMs:
lte: 500
durationMs:
lte: 5000
`, `,
); );
@@ -2023,10 +2031,10 @@ targets:
expect(t.ping).toEqual({ count: 5, host: "10.0.0.1", packetSize: 1472 }); expect(t.ping).toEqual({ count: 5, host: "10.0.0.1", packetSize: 1472 });
expect(t.expect).toEqual({ expect(t.expect).toEqual({
alive: true, alive: true,
maxAvgLatencyMs: 200, avgLatencyMs: { lte: 200 },
maxDurationMs: 5000, durationMs: { lte: 5000 },
maxMaxLatencyMs: 500, maxLatencyMs: { lte: 500 },
maxPacketLoss: 10, packetLossPercent: { lte: 10 },
}); });
}); });

4
tests/server/checker/runner/db/execute.test.ts
View File

@@ -74,9 +74,9 @@ describe("DbChecker", () => {
expect(result.failure!.message).toBeTruthy(); expect(result.failure!.message).toBeTruthy();
}); });
test("maxDurationMs 超时返回失败", async () => { test("durationMs 超时返回失败", async () => {
const result = await checker.execute( const result = await checker.execute(
makeTarget({ query: "SELECT 1" }, { expect: { maxDurationMs: -1 } }), makeTarget({ query: "SELECT 1" }, { expect: { durationMs: { lt: 0 } } }),
makeCtx(), makeCtx(),
); );
expect(result.matched).toBe(false); expect(result.matched).toBe(false);

18
tests/server/checker/runner/db/expect.test.ts
View File

@@ -4,35 +4,35 @@ import { checkRowCount, checkRows } from "../../../../../src/server/checker/runn
describe("checkRowCount", () => { describe("checkRowCount", () => {
test("空数组通过 rowCount gte 0", () => { test("空数组通过 rowCount gte 0", () => {
const result = checkRowCount([], { gte: 0 }); const result = checkRowCount(0, { gte: 0 });
expect(result.matched).toBe(true); expect(result.matched).toBe(true);
expect(result.failure).toBeNull(); expect(result.failure).toBeNull();
}); });
test("非数组视为 0 行", () => { test("0 行通过 gte 0", () => {
const result = checkRowCount(null, { gte: 0 }); const result = checkRowCount(0, { gte: 0 });
expect(result.matched).toBe(true); expect(result.matched).toBe(true);
}); });
test("rowCount gte 通过", () => { test("rowCount gte 通过", () => {
const result = checkRowCount([1, 2, 3], { gte: 3 }); const result = checkRowCount(3, { gte: 3 });
expect(result.matched).toBe(true); expect(result.matched).toBe(true);
}); });
test("rowCount gte 失败", () => { test("rowCount gte 失败", () => {
const result = checkRowCount([1, 2], { gte: 3 }); const result = checkRowCount(2, { gte: 3 });
expect(result.matched).toBe(false); expect(result.matched).toBe(false);
expect(result.failure!.phase).toBe("rowCount"); expect(result.failure!.phase).toBe("rowCount");
expect(result.failure!.path).toBe("rowCount"); expect(result.failure!.path).toBe("rowCount");
}); });
test("rowCount equals 通过", () => { test("rowCount equals 通过", () => {
const result = checkRowCount([1, 2, 3], { equals: 3 }); const result = checkRowCount(3, { equals: 3 });
expect(result.matched).toBe(true); expect(result.matched).toBe(true);
}); });
test("rowCount equals 失败", () => { test("rowCount equals 失败", () => {
const result = checkRowCount([1, 2, 3], { equals: 5 }); const result = checkRowCount(3, { equals: 5 });
expect(result.matched).toBe(false); expect(result.matched).toBe(false);
}); });
}); });
@@ -117,8 +117,8 @@ describe("checkRows", () => {
expect(result.matched).toBe(true); expect(result.matched).toBe(true);
}); });
test("match 正则匹配", () => { test("regex 正则匹配", () => {
const result = checkRows([{ code: "ABC-123" }], [{ code: { match: "^ABC-" } }]); const result = checkRows([{ code: "ABC-123" }], [{ code: { regex: "^ABC-" } }]);
expect(result.matched).toBe(true); expect(result.matched).toBe(true);
}); });

14
tests/server/checker/runner/db/validate.test.ts
View File

@@ -49,20 +49,20 @@ describe("validateDbConfig", () => {
expect(unknownError!.code).toBe("unknown-field"); expect(unknownError!.code).toBe("unknown-field");
}); });
test("expect.maxDurationMs 非数字返回错误", () => { test("expect.durationMs 非 matcher 返回错误", () => {
const result = validateDbConfig({ const result = validateDbConfig({
defaults: {}, defaults: {},
targets: [ targets: [
{ {
db: { url: "sqlite://:memory:" }, db: { url: "sqlite://:memory:" },
expect: { maxDurationMs: "invalid" }, expect: { durationMs: "invalid" },
id: "test", id: "test",
name: "test", name: "test",
type: "db", type: "db",
}, },
], ],
}); });
const durationError = result.find((e) => e.path.includes("expect.maxDurationMs")); const durationError = result.find((e) => e.path.includes("expect.durationMs"));
expect(durationError).toBeDefined(); expect(durationError).toBeDefined();
expect(durationError!.code).toBe("invalid-type"); expect(durationError!.code).toBe("invalid-type");
}); });
@@ -76,7 +76,7 @@ describe("validateDbConfig", () => {
}); });
const rowCountError = result.find((e) => e.path.includes("expect.rowCount")); const rowCountError = result.find((e) => e.path.includes("expect.rowCount"));
expect(rowCountError).toBeDefined(); expect(rowCountError).toBeDefined();
expect(rowCountError!.code).toBe("unknown-operator"); expect(rowCountError!.code).toBe("unknown-matcher");
}); });
test("expect.rows 不是数组返回错误", () => { test("expect.rows 不是数组返回错误", () => {
@@ -103,13 +103,13 @@ describe("validateDbConfig", () => {
expect(rowError!.code).toBe("invalid-type"); expect(rowError!.code).toBe("invalid-type");
}); });
test("expect.rows 中 match 正则非法返回错误", () => { test("expect.rows 中 regex 正则非法返回错误", () => {
const result = validateDbConfig({ const result = validateDbConfig({
defaults: {}, defaults: {},
targets: [ targets: [
{ {
db: { url: "sqlite://:memory:" }, db: { url: "sqlite://:memory:" },
expect: { rows: [{ name: { match: "[invalid" } }] }, expect: { rows: [{ name: { regex: "[invalid" } }] },
id: "test", id: "test",
name: "test", name: "test",
type: "db", type: "db",
@@ -137,7 +137,7 @@ describe("validateDbConfig", () => {
targets: [ targets: [
{ {
db: { query: "SELECT 1", url: "sqlite://:memory:" }, db: { query: "SELECT 1", url: "sqlite://:memory:" },
expect: { maxDurationMs: 5000, rowCount: { gte: 1 }, rows: [{ cnt: { gte: 1 } }] }, expect: { durationMs: { lte: 5000 }, rowCount: { gte: 1 }, rows: [{ cnt: { gte: 1 } }] },
id: "test", id: "test",
name: "test", name: "test",
type: "db", type: "db",

2
tests/server/checker/runner/http/expect.test.ts
View File

@@ -26,7 +26,7 @@ describe("checkHeaders", () => {
const headers = { "content-type": "application/json" }; const headers = { "content-type": "application/json" };
expect(checkHeaders(headers, { "content-type": { contains: "json" } }).matched).toBe(true); expect(checkHeaders(headers, { "content-type": { contains: "json" } }).matched).toBe(true);
expect(checkHeaders(headers, { "content-type": { match: "^application/" } }).matched).toBe(true); expect(checkHeaders(headers, { "content-type": { regex: "^application/" } }).matched).toBe(true);
expect(checkHeaders(headers, { "content-type": { contains: "xml" } }).matched).toBe(false); expect(checkHeaders(headers, { "content-type": { contains: "xml" } }).matched).toBe(false);
}); });

24
tests/server/checker/runner/http/runner.test.ts
View File

@@ -487,7 +487,7 @@ describe("HttpChecker", () => {
expect(result.durationMs!).toBeGreaterThanOrEqual(0); expect(result.durationMs!).toBeGreaterThanOrEqual(0);
}); });
test("expect.maxDurationMs 使用完整耗时", async () => { test("expect.durationMs 使用完整耗时", async () => {
const slowServer = Bun.serve({ const slowServer = Bun.serve({
async fetch() { async fetch() {
await new Promise((resolve) => setTimeout(resolve, 50)); await new Promise((resolve) => setTimeout(resolve, 50));
@@ -498,7 +498,7 @@ describe("HttpChecker", () => {
try { try {
const result = await checker.execute( const result = await checker.execute(
makeTarget({ makeTarget({
expect: { body: [{ contains: "x" }], maxDurationMs: 10 }, expect: { body: [{ contains: "x" }], durationMs: { lte: 10 } },
url: `http://localhost:${slowServer.port}/`, url: `http://localhost:${slowServer.port}/`,
}), }),
makeCtx(), makeCtx(),
@@ -521,7 +521,7 @@ describe("HttpChecker", () => {
test("body 失败优先于 duration 检查", async () => { test("body 失败优先于 duration 检查", async () => {
const result = await checker.execute( const result = await checker.execute(
makeTarget({ makeTarget({
expect: { body: [{ contains: "nonexistent" }], maxDurationMs: 999999 }, expect: { body: [{ contains: "nonexistent" }], durationMs: { lte: 999999 } },
url: `${baseUrl}/ok`, url: `${baseUrl}/ok`,
}), }),
makeCtx(), makeCtx(),
@@ -689,7 +689,7 @@ describe("HttpChecker", () => {
name: "test", name: "test",
type: "http", type: "http",
}); });
expect(errors).toContain("缺少支持的规则类型"); expect(errors).toContain("match 是未知字段");
}); });
test("非法 regex 启动校验失败", () => { test("非法 regex 启动校验失败", () => {
@@ -722,19 +722,19 @@ describe("HttpChecker", () => {
expect(errors).toContain("json.path"); expect(errors).toContain("json.path");
}); });
test("非法 operator match 启动校验失败", () => { test("旧 match matcher 启动校验失败", () => {
const errors = validateHttpTarget({ const errors = validateHttpTarget({
expect: { headers: { "x-test": { match: "[invalid" } } }, expect: { headers: { "x-test": { match: "[invalid" } } },
http: { url: "https://example.com" }, http: { url: "https://example.com" },
name: "test", name: "test",
type: "http", type: "http",
}); });
expect(errors).toContain("match 正则不合法"); expect(errors).toContain("match 是未知 matcher");
}); });
test("ReDoS operator match 启动校验失败", () => { test("ReDoS regex matcher 启动校验失败", () => {
const errors = validateHttpTarget({ const errors = validateHttpTarget({
expect: { headers: { "x-test": { match: "(\\d+)*x" } } }, expect: { headers: { "x-test": { regex: "(\\d+)*x" } } },
http: { url: "https://example.com" }, http: { url: "https://example.com" },
name: "test", name: "test",
type: "http", type: "http",
@@ -769,17 +769,17 @@ describe("HttpChecker", () => {
name: "test", name: "test",
type: "http", type: "http",
}); });
expect(errors).toContain("必须包含至少一个合法 operator"); expect(errors).toContain("必须包含至少一个合法 matcher");
}); });
test("body rule 多个支持字段启动失败", () => { test("body rule 多个支持字段启动失败", () => {
const errors = validateHttpTarget({ const errors = validateHttpTarget({
expect: { body: [{ contains: "ok", regex: "ok" }] }, expect: { body: [{ contains: "ok", json: { path: "$.status" } }] },
http: { url: "https://example.com" }, http: { url: "https://example.com" },
name: "test", name: "test",
type: "http", type: "http",
}); });
expect(errors).toContain("只能配置一种规则类型"); expect(errors).toContain("直接 matcher 不能与 extractor 混用");
}); });
test("body rule 缺少支持字段启动失败", () => { test("body rule 缺少支持字段启动失败", () => {
@@ -789,7 +789,7 @@ describe("HttpChecker", () => {
name: "test", name: "test",
type: "http", type: "http",
}); });
expect(errors).toContain("缺少支持的规则类型"); expect(errors).toContain("foo 是未知字段");
}); });
test("css selector 为空启动失败", () => { test("css selector 为空启动失败", () => {

7
tests/server/checker/runner/icmp/execute.test.ts
View File

@@ -60,7 +60,10 @@ rtt min/avg/max/mdev = 1.234/2.345/3.456/0.567 ms`);
test("alive 失败短路", async () => { test("alive 失败短路", async () => {
mockSpawn(`3 packets transmitted, 0 received, 100% packet loss, time 2003ms`); mockSpawn(`3 packets transmitted, 0 received, 100% packet loss, time 2003ms`);
const result = await checker.execute(makeTarget({ expect: { alive: true, maxAvgLatencyMs: 100 } }), makeCtx()); const result = await checker.execute(
makeTarget({ expect: { alive: true, avgLatencyMs: { lte: 100 } } }),
makeCtx(),
);
expect(result.matched).toBe(false); expect(result.matched).toBe(false);
expect(result.failure?.phase).toBe("alive"); expect(result.failure?.phase).toBe("alive");
expect(result.statusDetail).toBe("unreachable (0/3 received)"); expect(result.statusDetail).toBe("unreachable (0/3 received)");
@@ -75,7 +78,7 @@ rtt min/avg/max/mdev = 1.234/2.345/3.456/0.567 ms`);
test("packetLoss 断言失败", async () => { test("packetLoss 断言失败", async () => {
mockSpawn(`3 packets transmitted, 2 received, 33% packet loss, time 2003ms mockSpawn(`3 packets transmitted, 2 received, 33% packet loss, time 2003ms
rtt min/avg/max/mdev = 1.234/156.000/340.000/0.567 ms`); rtt min/avg/max/mdev = 1.234/156.000/340.000/0.567 ms`);
const result = await checker.execute(makeTarget({ expect: { maxPacketLoss: 10 } }), makeCtx()); const result = await checker.execute(makeTarget({ expect: { packetLossPercent: { lte: 10 } } }), makeCtx());
expect(result.matched).toBe(false); expect(result.matched).toBe(false);
expect(result.failure?.phase).toBe("packetLoss"); expect(result.failure?.phase).toBe("packetLoss");
expect(result.statusDetail).toContain("max 340ms"); expect(result.statusDetail).toContain("max 340ms");

12
tests/server/checker/runner/icmp/expect.test.ts
View File

@@ -16,22 +16,22 @@ describe("ping expect", () => {
}); });
test("packetLoss 通过和失败", () => { test("packetLoss 通过和失败", () => {
expect(checkPacketLoss(0, 10).matched).toBe(true); expect(checkPacketLoss(0, { lte: 10 }).matched).toBe(true);
const result = checkPacketLoss(33, 10); const result = checkPacketLoss(33, { lte: 10 });
expect(result.matched).toBe(false); expect(result.matched).toBe(false);
expect(result.failure?.phase).toBe("packetLoss"); expect(result.failure?.phase).toBe("packetLoss");
}); });
test("avgLatency 通过和失败", () => { test("avgLatency 通过和失败", () => {
expect(checkAvgLatency(12, 200).matched).toBe(true); expect(checkAvgLatency(12, { lte: 200 }).matched).toBe(true);
const result = checkAvgLatency(156, 100); const result = checkAvgLatency(156, { lte: 100 });
expect(result.matched).toBe(false); expect(result.matched).toBe(false);
expect(result.failure?.phase).toBe("avgLatency"); expect(result.failure?.phase).toBe("avgLatency");
}); });
test("maxLatency 通过和失败", () => { test("maxLatency 通过和失败", () => {
expect(checkMaxLatency(340, 500).matched).toBe(true); expect(checkMaxLatency(340, { lte: 500 }).matched).toBe(true);
const result = checkMaxLatency(340, 200); const result = checkMaxLatency(340, { lte: 200 });
expect(result.matched).toBe(false); expect(result.matched).toBe(false);
expect(result.failure?.phase).toBe("maxLatency"); expect(result.failure?.phase).toBe("maxLatency");
}); });

14
tests/server/checker/runner/icmp/validate.test.ts
View File

@@ -43,24 +43,24 @@ describe("validatePingConfig", () => {
expect(issues.some((item) => item.path.endsWith("expect.status"))).toBe(true); expect(issues.some((item) => item.path.endsWith("expect.status"))).toBe(true);
}); });
test("expect 数值非法", () => { test("expect 数值旧字段非法", () => {
const issues = validate({ expect: { maxPacketLoss: 101 }, id: "ping", ping: { host: "127.0.0.1" }, type: "ping" }); const issues = validate({ expect: { maxPacketLoss: 101 }, id: "ping", ping: { host: "127.0.0.1" }, type: "ping" });
expect(issues.some((item) => item.path.endsWith("expect.maxPacketLoss"))).toBe(true); expect(issues.some((item) => item.path.endsWith("expect.maxPacketLoss"))).toBe(true);
}); });
test("maxDurationMs 类型非法", () => { test("durationMs 类型非法", () => {
const issues = validate({ expect: { maxDurationMs: -1 }, id: "ping", ping: { host: "127.0.0.1" }, type: "ping" }); const issues = validate({ expect: { durationMs: -1 }, id: "ping", ping: { host: "127.0.0.1" }, type: "ping" });
expect(issues.some((item) => item.path.endsWith("expect.maxDurationMs"))).toBe(true); expect(issues.some((item) => item.path.endsWith("expect.durationMs"))).toBe(true);
}); });
test("maxAvgLatencyMs 类型非法", () => { test("avgLatencyMs 类型非法", () => {
const issues = validate({ const issues = validate({
expect: { maxAvgLatencyMs: "slow" }, expect: { avgLatencyMs: "slow" },
id: "ping", id: "ping",
ping: { host: "127.0.0.1" }, ping: { host: "127.0.0.1" },
type: "ping", type: "ping",
}); });
expect(issues.some((item) => item.path.endsWith("expect.maxAvgLatencyMs"))).toBe(true); expect(issues.some((item) => item.path.endsWith("expect.avgLatencyMs"))).toBe(true);
}); });
test("host 为空字符串", () => { test("host 为空字符串", () => {

2
tests/server/checker/runner/llm/execute.test.ts
View File

@@ -135,7 +135,7 @@ describe("LlmChecker execute - 非流式", () => {
}); });
test("finishReason expect 不匹配", async () => { test("finishReason expect 不匹配", async () => {
const result = await checker.execute(makeTarget(undefined, { finishReason: "length" }), makeCtx()); const result = await checker.execute(makeTarget(undefined, { finishReason: { equals: "length" } }), makeCtx());
expect(result.matched).toBe(false); expect(result.matched).toBe(false);
expect(result.failure?.phase).toBe("finishReason"); expect(result.failure?.phase).toBe("finishReason");
}); });

12
tests/server/checker/runner/llm/output-expect.test.ts
View File

@@ -2,8 +2,12 @@ import { describe, expect, test } from "bun:test";
import type { LlmCheckObservation } from "../../../../../src/server/checker/runner/llm/types"; import type { LlmCheckObservation } from "../../../../../src/server/checker/runner/llm/types";
import { checkContentRules } from "../../../../../src/server/checker/expect/content";
import { runExpects } from "../../../../../src/server/checker/runner/llm/expect"; import { runExpects } from "../../../../../src/server/checker/runner/llm/expect";
import { checkOutputRules } from "../../../../../src/server/checker/runner/llm/output";
function checkOutputRules(outputText: null | string, rules: Parameters<typeof checkContentRules>[1]) {
return checkContentRules(outputText, rules, { path: "output", phase: "output" });
}
function makeObservation(overrides?: Partial<LlmCheckObservation>): LlmCheckObservation { function makeObservation(overrides?: Partial<LlmCheckObservation>): LlmCheckObservation {
return { return {
@@ -72,7 +76,7 @@ describe("LLM runExpects", () => {
test("全部 expect 通过", () => { test("全部 expect 通过", () => {
const observation = makeObservation(); const observation = makeObservation();
const result = runExpects(observation, { const result = runExpects(observation, {
finishReason: "stop", finishReason: { equals: "stop" },
output: [{ contains: "OK" }], output: [{ contains: "OK" }],
status: [200], status: [200],
}); });
@@ -95,14 +99,14 @@ describe("LLM runExpects", () => {
test("finishReason 不匹配失败", () => { test("finishReason 不匹配失败", () => {
const observation = makeObservation(); const observation = makeObservation();
const result = runExpects(observation, { finishReason: "length" }); const result = runExpects(observation, { finishReason: { equals: "length" } });
expect(result.matched).toBe(false); expect(result.matched).toBe(false);
expect(result.failure?.phase).toBe("finishReason"); expect(result.failure?.phase).toBe("finishReason");
}); });
test("rawFinishReason 不匹配失败", () => { test("rawFinishReason 不匹配失败", () => {
const observation = makeObservation(); const observation = makeObservation();
const result = runExpects(observation, { rawFinishReason: "end_turn" }); const result = runExpects(observation, { rawFinishReason: { equals: "end_turn" } });
expect(result.matched).toBe(false); expect(result.matched).toBe(false);
expect(result.failure?.phase).toBe("rawFinishReason"); expect(result.failure?.phase).toBe("rawFinishReason");
}); });

8
tests/server/checker/runner/llm/schema-validate-resolve.test.ts
View File

@@ -206,15 +206,15 @@ describe("LlmChecker validate", () => {
defaults: {}, defaults: {},
targets: [makeRawTarget({ expect: { output: [{}] } })], targets: [makeRawTarget({ expect: { output: [{}] } })],
}); });
expect(issues.some((i) => i.code === "missing-body-rule")).toBe(true); expect(issues.some((i) => i.code === "empty-matcher")).toBe(true);
}); });
test("expect.output 同时多种规则类型报错", () => { test("expect.output 直接 matcher 混入 extractor 报错", () => {
const issues = validateLlmConfig({ const issues = validateLlmConfig({
defaults: {}, defaults: {},
targets: [makeRawTarget({ expect: { output: [{ contains: "y", equals: "x" }] } })], targets: [makeRawTarget({ expect: { output: [{ contains: "y", json: { path: "$.status" } }] } })],
}); });
expect(issues.some((i) => i.code === "multiple-body-rules")).toBe(true); expect(issues.some((i) => i.code === "invalid-content-rule")).toBe(true);
}); });
test("expect.output regex ReDoS 报错", () => { test("expect.output regex ReDoS 报错", () => {

8
tests/server/checker/runner/shared/body.test.ts
View File

@@ -1,6 +1,10 @@
import { describe, expect, test } from "bun:test"; import { describe, expect, test } from "bun:test";
import { checkBodyExpect } from "../../../../../src/server/checker/runner/http/body"; import { checkContentRules } from "../../../../../src/server/checker/expect/content";
function checkBodyExpect(body: string, rules?: Parameters<typeof checkContentRules>[1]) {
return checkContentRules(body, rules, { path: "body", phase: "body" });
}
describe("checkBodyExpect (BodyRule[])", () => { describe("checkBodyExpect (BodyRule[])", () => {
test("无规则返回匹配成功", () => { test("无规则返回匹配成功", () => {
@@ -57,7 +61,7 @@ describe("checkBodyExpect (BodyRule[])", () => {
test("json 操作符匹配", () => { test("json 操作符匹配", () => {
const body = JSON.stringify({ count: 42, version: "v2.1.0" }); const body = JSON.stringify({ count: 42, version: "v2.1.0" });
expect(checkBodyExpect(body, [{ json: { gte: 10, path: "$.count" } }]).matched).toBe(true); expect(checkBodyExpect(body, [{ json: { gte: 10, path: "$.count" } }]).matched).toBe(true);
expect(checkBodyExpect(body, [{ json: { match: "\\d+\\.\\d+\\.\\d+", path: "$.version" } }]).matched).toBe(true); expect(checkBodyExpect(body, [{ json: { path: "$.version", regex: "\\d+\\.\\d+\\.\\d+" } }]).matched).toBe(true);
expect(checkBodyExpect(body, [{ json: { gte: 100, path: "$.count" } }]).matched).toBe(false); expect(checkBodyExpect(body, [{ json: { gte: 100, path: "$.count" } }]).matched).toBe(false);
}); });

9
tests/server/checker/runner/shared/duration.test.ts
View File

@@ -1,6 +1,13 @@
import { describe, expect, test } from "bun:test"; import { describe, expect, test } from "bun:test";
import { checkDuration } from "../../../../../src/server/checker/expect/duration"; import { checkValueMatcher } from "../../../../../src/server/checker/expect/matcher";
function checkDuration(durationMs: number, maxDurationMs?: number) {
return checkValueMatcher(durationMs, maxDurationMs === undefined ? undefined : { lte: maxDurationMs }, {
path: "durationMs",
phase: "duration",
});
}
describe("checkDuration", () => { describe("checkDuration", () => {
test("未配置 maxDurationMs 返回匹配成功", () => { test("未配置 maxDurationMs 返回匹配成功", () => {

94
tests/server/checker/runner/shared/operator.test.ts
View File

@@ -1,6 +1,6 @@
import { describe, expect, test } from "bun:test"; import { describe, expect, test } from "bun:test";
import { applyOperator, checkExpectValue, evaluateJsonPath } from "../../../../../src/server/checker/expect/operator"; import { applyMatcher, checkExpectValue, evaluateJsonPath } from "../../../../../src/server/checker/expect/matcher";
describe("evaluateJsonPath", () => { describe("evaluateJsonPath", () => {
const obj = { const obj = {
@@ -55,81 +55,83 @@ describe("evaluateJsonPath", () => {
}); });
}); });
describe("applyOperator", () => { describe("applyMatcher", () => {
test("equals 操作符", () => { test("equals 操作符", () => {
expect(applyOperator("ok", { equals: "ok" })).toBe(true); expect(applyMatcher("ok", { equals: "ok" })).toBe(true);
expect(applyOperator("ok", { equals: "error" })).toBe(false); expect(applyMatcher("ok", { equals: "error" })).toBe(false);
expect(applyOperator(42, { equals: 42 })).toBe(true); expect(applyMatcher(42, { equals: 42 })).toBe(true);
expect(applyOperator(42, { equals: 41 })).toBe(false); expect(applyMatcher(42, { equals: 41 })).toBe(false);
expect(applyOperator(null, { equals: null })).toBe(true); expect(applyMatcher(null, { equals: null })).toBe(true);
expect(applyOperator(true, { equals: true })).toBe(true); expect(applyMatcher(true, { equals: true })).toBe(true);
}); });
test("equals 支持 JSON 对象和数组", () => { test("equals 支持 JSON 对象和数组", () => {
expect(applyOperator({ status: "ok" }, { equals: { status: "ok" } })).toBe(true); expect(applyMatcher({ status: "ok" }, { equals: { status: "ok" } })).toBe(true);
expect(applyOperator({ status: "ok" }, { equals: { status: "fail" } })).toBe(false); expect(applyMatcher({ status: "ok" }, { equals: { status: "fail" } })).toBe(false);
expect(applyOperator(["a", "b"], { equals: ["a", "b"] })).toBe(true); expect(applyMatcher(["a", "b"], { equals: ["a", "b"] })).toBe(true);
expect(applyOperator(["a", "b"], { equals: ["b", "a"] })).toBe(false); expect(applyMatcher(["a", "b"], { equals: ["b", "a"] })).toBe(false);
}); });
test("contains 操作符", () => { test("contains 操作符", () => {
expect(applyOperator("hello world", { contains: "hello" })).toBe(true); expect(applyMatcher("hello world", { contains: "hello" })).toBe(true);
expect(applyOperator("hello world", { contains: "missing" })).toBe(false); expect(applyMatcher("hello world", { contains: "missing" })).toBe(false);
expect(applyOperator(12345, { contains: "23" })).toBe(true); expect(applyMatcher(12345, { contains: "23" })).toBe(true);
}); });
test("match 操作符", () => { test("regex matcher", () => {
expect(applyOperator("v2.1.0", { match: "\\d+\\.\\d+\\.\\d+" })).toBe(true); expect(applyMatcher("v2.1.0", { regex: "\\d+\\.\\d+\\.\\d+" })).toBe(true);
expect(applyOperator("v2.1", { match: "\\d+\\.\\d+\\.\\d+" })).toBe(false); expect(applyMatcher("v2.1", { regex: "\\d+\\.\\d+\\.\\d+" })).toBe(false);
expect(applyOperator("abc123", { match: "^\\w+\\d+$" })).toBe(true); expect(applyMatcher("abc123", { regex: "^\\w+\\d+$" })).toBe(true);
}); });
test("empty 操作符", () => { test("empty 操作符", () => {
expect(applyOperator("", { empty: true })).toBe(true); expect(applyMatcher("", { empty: true })).toBe(true);
expect(applyOperator(null, { empty: true })).toBe(true); expect(applyMatcher(null, { empty: true })).toBe(true);
expect(applyOperator(undefined, { empty: true })).toBe(true); expect(applyMatcher(undefined, { empty: true })).toBe(true);
expect(applyOperator([], { empty: true })).toBe(true); expect(applyMatcher([], { empty: true })).toBe(true);
expect(applyOperator({}, { empty: true })).toBe(true); expect(applyMatcher({}, { empty: true })).toBe(true);
expect(applyOperator("ok", { empty: true })).toBe(false); expect(applyMatcher("ok", { empty: true })).toBe(false);
expect(applyOperator([1, 2], { empty: false })).toBe(true); expect(applyMatcher(0, { empty: true })).toBe(false);
expect(applyOperator([], { empty: false })).toBe(false); expect(applyMatcher(false, { empty: true })).toBe(false);
expect(applyMatcher([1, 2], { empty: false })).toBe(true);
expect(applyMatcher([], { empty: false })).toBe(false);
}); });
test("exists 操作符", () => { test("exists 操作符", () => {
expect(applyOperator("ok", { exists: true })).toBe(true); expect(applyMatcher("ok", { exists: true })).toBe(true);
expect(applyOperator(null, { exists: true })).toBe(true); expect(applyMatcher(null, { exists: true })).toBe(true);
expect(applyOperator(undefined, { exists: true })).toBe(false); expect(applyMatcher(undefined, { exists: true })).toBe(false);
expect(applyOperator(undefined, { exists: false })).toBe(true); expect(applyMatcher(undefined, { exists: false })).toBe(true);
expect(applyOperator("ok", { exists: false })).toBe(false); expect(applyMatcher("ok", { exists: false })).toBe(false);
}); });
test("gte 操作符", () => { test("gte 操作符", () => {
expect(applyOperator(10, { gte: 5 })).toBe(true); expect(applyMatcher(10, { gte: 5 })).toBe(true);
expect(applyOperator(5, { gte: 5 })).toBe(true); expect(applyMatcher(5, { gte: 5 })).toBe(true);
expect(applyOperator(3, { gte: 5 })).toBe(false); expect(applyMatcher(3, { gte: 5 })).toBe(false);
expect(applyOperator("10", { gte: 5 })).toBe(true); expect(applyMatcher("10", { gte: 5 })).toBe(true);
}); });
test("lte 操作符", () => { test("lte 操作符", () => {
expect(applyOperator(3, { lte: 5 })).toBe(true); expect(applyMatcher(3, { lte: 5 })).toBe(true);
expect(applyOperator(5, { lte: 5 })).toBe(true); expect(applyMatcher(5, { lte: 5 })).toBe(true);
expect(applyOperator(10, { lte: 5 })).toBe(false); expect(applyMatcher(10, { lte: 5 })).toBe(false);
}); });
test("gt 操作符", () => { test("gt 操作符", () => {
expect(applyOperator(10, { gt: 5 })).toBe(true); expect(applyMatcher(10, { gt: 5 })).toBe(true);
expect(applyOperator(5, { gt: 5 })).toBe(false); expect(applyMatcher(5, { gt: 5 })).toBe(false);
}); });
test("lt 操作符", () => { test("lt 操作符", () => {
expect(applyOperator(3, { lt: 5 })).toBe(true); expect(applyMatcher(3, { lt: 5 })).toBe(true);
expect(applyOperator(5, { lt: 5 })).toBe(false); expect(applyMatcher(5, { lt: 5 })).toBe(false);
}); });
test("多操作符 AND 组合", () => { test("多操作符 AND 组合", () => {
expect(applyOperator(7, { gte: 5, lte: 10 })).toBe(true); expect(applyMatcher(7, { gte: 5, lte: 10 })).toBe(true);
expect(applyOperator(3, { gte: 5, lte: 10 })).toBe(false); expect(applyMatcher(3, { gte: 5, lte: 10 })).toBe(false);
expect(applyOperator(15, { gte: 5, lte: 10 })).toBe(false); expect(applyMatcher(15, { gte: 5, lte: 10 })).toBe(false);
}); });
}); });

8
tests/server/checker/runner/shared/text.test.ts
View File

@@ -1,6 +1,10 @@
import { describe, expect, test } from "bun:test"; import { describe, expect, test } from "bun:test";
import { checkTextRules } from "../../../../../src/server/checker/runner/cmd/text"; import { checkContentRules } from "../../../../../src/server/checker/expect/content";
function checkTextRules(text: string, rules: Parameters<typeof checkContentRules>[1], phase: string) {
return checkContentRules(text, rules, { path: phase, phase });
}
describe("checkTextRules", () => { describe("checkTextRules", () => {
test("无规则返回匹配成功", () => { test("无规则返回匹配成功", () => {
@@ -24,7 +28,7 @@ describe("checkTextRules", () => {
test("多条规则全部通过", () => { test("多条规则全部通过", () => {
const r = checkTextRules( const r = checkTextRules(
"version: 3.2.1, build: ok", "version: 3.2.1, build: ok",
[{ contains: "version" }, { match: "\\d+\\.\\d+\\.\\d+" }], [{ contains: "version" }, { regex: "\\d+\\.\\d+\\.\\d+" }],
"stdout", "stdout",
); );
expect(r.matched).toBe(true); expect(r.matched).toBe(true);

14
tests/server/checker/runner/tcp/execute.test.ts
View File

@@ -155,8 +155,8 @@ describe("TcpChecker execute", () => {
expect(result.failure!.phase).toBe("connected"); expect(result.failure!.phase).toBe("connected");
}); });
test("maxDurationMs 超时返回失败", async () => { test("durationMs 超时返回失败", async () => {
const result = await checker.execute(makeTarget({}, { expect: { maxDurationMs: -1 } }), makeCtx()); const result = await checker.execute(makeTarget({}, { expect: { durationMs: { lt: 0 } } }), makeCtx());
expect(result.matched).toBe(false); expect(result.matched).toBe(false);
expect(result.failure!.phase).toBe("duration"); expect(result.failure!.phase).toBe("duration");
}); });
@@ -179,7 +179,7 @@ describe("TcpChecker execute", () => {
const result = await checker.execute( const result = await checker.execute(
makeTarget( makeTarget(
{ host: "127.0.0.1", port: bannerServerPort, readBanner: true }, { host: "127.0.0.1", port: bannerServerPort, readBanner: true },
{ expect: { banner: { contains: "ESMTP" } } }, { expect: { banner: [{ contains: "ESMTP" }] } },
), ),
makeCtx(), makeCtx(),
); );
@@ -190,14 +190,14 @@ describe("TcpChecker execute", () => {
const result = await checker.execute( const result = await checker.execute(
makeTarget( makeTarget(
{ host: "127.0.0.1", port: bannerServerPort, readBanner: true }, { host: "127.0.0.1", port: bannerServerPort, readBanner: true },
{ expect: { banner: { contains: "POSTFIX" } } }, { expect: { banner: [{ contains: "POSTFIX" }] } },
), ),
makeCtx(), makeCtx(),
); );
expect(result.matched).toBe(false); expect(result.matched).toBe(false);
expect(result.failure!.kind).toBe("mismatch"); expect(result.failure!.kind).toBe("mismatch");
expect(result.failure!.phase).toBe("banner"); expect(result.failure!.phase).toBe("banner");
expect(result.failure!.path).toBe("banner"); expect(result.failure!.path).toBe("banner[0]");
}); });
test("默认不读取 banner", async () => { test("默认不读取 banner", async () => {
@@ -346,14 +346,14 @@ describe("TcpChecker resolve", () => {
test("expect 配置解析", () => { test("expect 配置解析", () => {
const target = checker.resolve( const target = checker.resolve(
{ {
expect: { banner: { contains: "ESMTP" }, connected: false, maxDurationMs: 5000 }, expect: { banner: [{ contains: "ESMTP" }], connected: false, durationMs: { lte: 5000 } },
id: "t", id: "t",
tcp: { host: "127.0.0.1", port: 80, readBanner: true }, tcp: { host: "127.0.0.1", port: 80, readBanner: true },
type: "tcp", type: "tcp",
}, },
{ configDir: "/tmp", defaultIntervalMs: 30000, defaults: {}, defaultTimeoutMs: 10000 }, { configDir: "/tmp", defaultIntervalMs: 30000, defaults: {}, defaultTimeoutMs: 10000 },
); );
expect(target.expect).toEqual({ banner: { contains: "ESMTP" }, connected: false, maxDurationMs: 5000 }); expect(target.expect).toEqual({ banner: [{ contains: "ESMTP" }], connected: false, durationMs: { lte: 5000 } });
}); });
test("name 和 group 解析", () => { test("name 和 group 解析", () => {

14
tests/server/checker/runner/tcp/expect.test.ts
View File

@@ -32,34 +32,34 @@ describe("checkConnected", () => {
describe("checkBanner", () => { describe("checkBanner", () => {
test("contains 匹配", () => { test("contains 匹配", () => {
const result = checkBanner("220 smtp.example.com ESMTP", { contains: "ESMTP" }); const result = checkBanner("220 smtp.example.com ESMTP", [{ contains: "ESMTP" }]);
expect(result.matched).toBe(true); expect(result.matched).toBe(true);
}); });
test("contains 不匹配", () => { test("contains 不匹配", () => {
const result = checkBanner("220 smtp.example.com ESMTP", { contains: "POSTFIX" }); const result = checkBanner("220 smtp.example.com ESMTP", [{ contains: "POSTFIX" }]);
expect(result.matched).toBe(false); expect(result.matched).toBe(false);
expect(result.failure!.kind).toBe("mismatch"); expect(result.failure!.kind).toBe("mismatch");
expect(result.failure!.phase).toBe("banner"); expect(result.failure!.phase).toBe("banner");
}); });
test("match 正则匹配", () => { test("regex 正则匹配", () => {
const result = checkBanner("220 smtp.example.com ESMTP", { match: "^220" }); const result = checkBanner("220 smtp.example.com ESMTP", [{ regex: "^220" }]);
expect(result.matched).toBe(true); expect(result.matched).toBe(true);
}); });
test("空 banner 与 contains 空字符串", () => { test("空 banner 与 contains 空字符串", () => {
const result = checkBanner("", { contains: "" }); const result = checkBanner("", [{ contains: "" }]);
expect(result.matched).toBe(true); expect(result.matched).toBe(true);
}); });
test("多 operator 同时匹配", () => { test("多 operator 同时匹配", () => {
const result = checkBanner("220 ESMTP", { contains: "ESMTP", match: "^220" }); const result = checkBanner("220 ESMTP", [{ contains: "ESMTP", regex: "^220" }]);
expect(result.matched).toBe(true); expect(result.matched).toBe(true);
}); });
test("多 operator 部分不匹配", () => { test("多 operator 部分不匹配", () => {
const result = checkBanner("220 ESMTP", { contains: "ESMTP", match: "^250" }); const result = checkBanner("220 ESMTP", [{ contains: "ESMTP", regex: "^250" }]);
expect(result.matched).toBe(false); expect(result.matched).toBe(false);
}); });
}); });

14
tests/server/checker/runner/tcp/validate.test.ts
View File

@@ -68,7 +68,7 @@ describe("validateTcpConfig", () => {
const issues = validateTcpConfig( const issues = validateTcpConfig(
makeInput([ makeInput([
{ {
expect: { banner: { contains: "ESMTP" } }, expect: { banner: [{ contains: "ESMTP" }] },
id: "t1", id: "t1",
tcp: { host: "127.0.0.1", port: 25 }, tcp: { host: "127.0.0.1", port: 25 },
type: "tcp", type: "tcp",
@@ -82,7 +82,7 @@ describe("validateTcpConfig", () => {
const issues = validateTcpConfig( const issues = validateTcpConfig(
makeInput([ makeInput([
{ {
expect: { banner: { contains: "ESMTP" } }, expect: { banner: [{ contains: "ESMTP" }] },
id: "t1", id: "t1",
tcp: { host: "127.0.0.1", port: 25, readBanner: true }, tcp: { host: "127.0.0.1", port: 25, readBanner: true },
type: "tcp", type: "tcp",
@@ -106,18 +106,18 @@ describe("validateTcpConfig", () => {
expect(issues.some((i) => i.path.includes("connected"))).toBe(true); expect(issues.some((i) => i.path.includes("connected"))).toBe(true);
}); });
test("expect maxDurationMs 非数字", () => { test("expect durationMs 非 matcher", () => {
const issues = validateTcpConfig( const issues = validateTcpConfig(
makeInput([ makeInput([
{ {
expect: { maxDurationMs: "slow" }, expect: { durationMs: "slow" },
id: "t1", id: "t1",
tcp: { host: "127.0.0.1", port: 80 }, tcp: { host: "127.0.0.1", port: 80 },
type: "tcp", type: "tcp",
}, },
]), ]),
); );
expect(issues.some((i) => i.path.includes("maxDurationMs"))).toBe(true); expect(issues.some((i) => i.path.includes("durationMs"))).toBe(true);
}); });
test("expect 未知字段", () => { test("expect 未知字段", () => {
@@ -134,11 +134,11 @@ describe("validateTcpConfig", () => {
expect(issues.some((i) => i.message.includes("未知字段"))).toBe(true); expect(issues.some((i) => i.message.includes("未知字段"))).toBe(true);
}); });
test("expect.banner match 正则非法", () => { test("expect.banner regex 正则非法", () => {
const issues = validateTcpConfig( const issues = validateTcpConfig(
makeInput([ makeInput([
{ {
expect: { banner: { match: "[invalid" } }, expect: { banner: [{ regex: "[invalid" }] },
id: "t1", id: "t1",
tcp: { host: "127.0.0.1", port: 25, readBanner: true }, tcp: { host: "127.0.0.1", port: 25, readBanner: true },
type: "tcp", type: "tcp",

4
tests/server/checker/runner/udp/execute.test.ts
View File

@@ -250,7 +250,7 @@ describe("UdpChecker execute", () => {
} }
}); });
it("should fail when duration exceeds maxDurationMs", async () => { it("should fail when duration exceeds durationMs", async () => {
const server = await Bun.udpSocket({ const server = await Bun.udpSocket({
socket: { socket: {
data() { data() {
@@ -266,7 +266,7 @@ describe("UdpChecker execute", () => {
}); });
try { try {
const checker = new UdpChecker(); const checker = new UdpChecker();
const target = makeTarget({ port: server.port }, { maxDurationMs: 1, responded: false }); const target = makeTarget({ port: server.port }, { durationMs: { lte: 1 }, responded: false });
const controller = new AbortController(); const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 200); const timer = setTimeout(() => controller.abort(), 200);
const result = await checker.execute(target, { signal: controller.signal }); const result = await checker.execute(target, { signal: controller.signal });

4
tests/server/checker/runner/udp/expect.test.ts
View File

@@ -76,13 +76,13 @@ describe("checkResponseText", () => {
}); });
it("多条规则全部匹配", () => { it("多条规则全部匹配", () => {
const result = checkResponseText("hello world", [{ contains: "hello" }, { match: "^hello" }]); const result = checkResponseText("hello world", [{ contains: "hello" }, { regex: "^hello" }]);
expect(result.matched).toBe(true); expect(result.matched).toBe(true);
expect(result.failure).toBeNull(); expect(result.failure).toBeNull();
}); });
it("多条规则第二条失败 → 不匹配", () => { it("多条规则第二条失败 → 不匹配", () => {
const result = checkResponseText("hello world", [{ contains: "hello" }, { match: "^world" }]); const result = checkResponseText("hello world", [{ contains: "hello" }, { regex: "^world" }]);
expect(result.matched).toBe(false); expect(result.matched).toBe(false);
expect(result.failure!.phase).toBe("response"); expect(result.failure!.phase).toBe("response");
expect(result.failure!.path).toBe("response[1]"); expect(result.failure!.path).toBe("response[1]");

6
tests/server/checker/runner/udp/validate.test.ts
View File

@@ -213,12 +213,12 @@ describe("validateUdpConfig", () => {
expect(issues.some((i) => i.path.includes("responded") && i.message.includes("响应来源"))).toBe(true); expect(issues.some((i) => i.path.includes("responded") && i.message.includes("响应来源"))).toBe(true);
}); });
it("reports invalid-type for negative expect.maxDurationMs", () => { it("reports invalid-type for non-matcher expect.durationMs", () => {
const issues = validateUdpConfig( const issues = validateUdpConfig(
makeInput({ makeInput({
targets: [ targets: [
{ {
expect: { maxDurationMs: -100 }, expect: { durationMs: -100 },
id: "test", id: "test",
type: "udp", type: "udp",
udp: { host: "127.0.0.1", port: 53 }, udp: { host: "127.0.0.1", port: 53 },
@@ -226,7 +226,7 @@ describe("validateUdpConfig", () => {
], ],
}), }),
); );
expect(issues.some((i) => i.code === "invalid-type" && i.path.includes("maxDurationMs"))).toBe(true); expect(issues.some((i) => i.code === "invalid-type" && i.path.includes("durationMs"))).toBe(true);
}); });
it("reports invalid-type for non-boolean expect.responded", () => { it("reports invalid-type for non-boolean expect.responded", () => {

4
tests/server/checker/store.test.ts
View File

@@ -26,7 +26,7 @@ beforeAll(() => {
const httpTarget: ResolvedHttpTarget = { const httpTarget: ResolvedHttpTarget = {
description: null, description: null,
expect: { maxDurationMs: 3000, status: [200] }, expect: { durationMs: { lte: 3000 }, status: [200] },
group: "default", group: "default",
http: { http: {
headers: { Accept: "application/json" }, headers: { Accept: "application/json" },
@@ -106,7 +106,7 @@ describe("ProbeStore", () => {
expect(config.maxRedirects).toBe(0); expect(config.maxRedirects).toBe(0);
expect(t.interval_ms).toBe(30000); expect(t.interval_ms).toBe(30000);
expect(t.timeout_ms).toBe(10000); expect(t.timeout_ms).toBe(10000);
expect(JSON.parse(t.expect!)).toEqual({ maxDurationMs: 3000, status: [200] }); expect(JSON.parse(t.expect!)).toEqual({ durationMs: { lte: 3000 }, status: [200] });
}); });
test("cmd target 字段正确", () => { test("cmd target 字段正确", () => {