refactor: 统一 expect 断言体系，引入共享 ValueMatcher/ContentRules/KeyValueExpect 模型
- 引入共享 ValueMatcher（equals/contains/regex/exists/empty/gt/gte/lt/lte） - 引入共享 ContentRules 数组（direct/json/css/xpath 提取器） - 引入共享 KeyValueExpect（动态键值断言，字面量等价 equals） - maxDurationMs → durationMs: ValueMatcher（所有 checker） - match → regex（固定无 flags） - Ping max* → packetLossPercent/avgLatencyMs/maxLatencyMs（ValueMatcher） - LLM finishReason/rawFinishReason → ValueMatcher - DB 新增 result: ContentRules - TCP banner → ContentRules 数组 - 删除旧模块：operator.ts、validate-operator.ts、duration.ts、body.ts、text.ts、output.ts - 更新全部 checker schema/validate/expect/execute - 更新 probe-config.schema.json、probes.example.yaml - 更新 README.md、DEVELOPMENT.md（含 expect 字段选择规范） - 同步 10 个 delta specs 到主 specs，归档 change
This commit is contained in:
143
DEVELOPMENT.md
143
DEVELOPMENT.md
@@ -39,7 +39,7 @@ src/
|
||||
variables.ts 配置 variables 提取、target 字符串变量替换和 unresolved-variable issue 生成
|
||||
schema/ TypeBox + Ajv 配置契约、schema fragments、issue 渲染和 schema 导出入口
|
||||
builder.ts 全量 JSON Schema 组装（遍历 registry 生成）
|
||||
fragments.ts 共享 TypeBox schema 片段（duration、size、operator 等）
|
||||
fragments.ts 共享 TypeBox schema 片段（duration、size、ValueMatcher、ContentRules、KeyValueExpect 等）
|
||||
validate.ts Ajv 契约校验入口
|
||||
issues.ts 校验问题类型与渲染
|
||||
types.ts schema 层类型
|
||||
@@ -48,22 +48,24 @@ src/
|
||||
engine.ts 调度引擎（按 interval 分组的 es-toolkit groupBy + Semaphore 并发控制 + 数据清理）
|
||||
utils.ts 共享工具函数（parseSize、parseDuration）
|
||||
expect/ 共享 expect 断言基础设施（跨 checker 复用）
|
||||
types.ts ExpectResult 共享断言类型
|
||||
types.ts ExpectResult、ValueMatcher、ContentRules、KeyValueExpect 类型
|
||||
failure.ts 失败信息构造（errorFailure、mismatchFailure、truncateActual）
|
||||
operator.ts 操作符系统（applyOperator、evaluateJsonPath）
|
||||
duration.ts 耗时断言（checkDuration）
|
||||
validate-operator.ts 操作符语义校验（validateOperatorObject、isJsonValue、isPlainRecord）
|
||||
matcher.ts ValueMatcher 执行、JSONPath 提取、字面量 equals 快捷语义
|
||||
content.ts ContentRules 执行（direct/json/css/xpath）
|
||||
key-value.ts KeyValueExpect 执行（动态键与 key 规范化）
|
||||
validate-matcher.ts matcher/content/key-value 语义校验
|
||||
redos.ts regex ReDoS 风险检测
|
||||
runner/ Checker 统一抽象与注册机制
|
||||
types.ts CheckerDefinition、CheckerContext、CheckerSchemas、ResolveContext
|
||||
registry.ts CheckerRegistry 注册中心
|
||||
index.ts 注册入口（显式数组 + 循环注册）
|
||||
http/ HTTP Checker（自包含模块，含 types/schema/execute/expect/validate/body）
|
||||
cmd/ Cmd Checker（自包含模块，含 types/schema/execute/expect/validate/text）
|
||||
http/ HTTP Checker（自包含模块，含 types/schema/execute/expect/validate）
|
||||
cmd/ Cmd Checker（自包含模块，含 types/schema/execute/expect/validate）
|
||||
db/ DB Checker（自包含模块，含 types/schema/execute/expect/validate）
|
||||
tcp/ TCP Checker（自包含模块，含 types/schema/execute/expect/validate）
|
||||
icmp/ Ping Checker（自包含模块，含 types/schema/execute/expect/validate/parse）
|
||||
udp/ UDP Checker（自包含模块，含 types/schema/execute/expect/validate/encoding）
|
||||
llm/ LLM Checker（自包含模块，含 types/schema/execute/expect/validate/output/provider/observation）
|
||||
llm/ LLM Checker（自包含模块，含 types/schema/execute/expect/validate/provider/observation）
|
||||
shared/
|
||||
api.ts 前后端共享 TypeScript 类型
|
||||
web/ React 前端 Dashboard（通过 Bun HTML import 集成）
|
||||
@@ -271,7 +273,7 @@ checkerRegistry（单例）
|
||||
| `validate.ts` | 启动期语义校验（JSON Schema 无法表达的规则） |
|
||||
| `execute.ts` | Checker 类：resolve（默认值合并 + 解析）、execute（执行检查）、serialize（DB 持久化） |
|
||||
| `expect.ts` | Checker 专用断言函数 |
|
||||
| `*.ts` | 其他 checker 专属逻辑（如 http/body.ts、cmd/text.ts） |
|
||||
| `*.ts` | 其他 checker 专属逻辑（如协议解析、编码、provider 适配、平台命令封装） |
|
||||
|
||||
#### 1.7.2 步骤一：创建 Checker 目录与类型
|
||||
|
||||
@@ -291,15 +293,15 @@ checkerRegistry（单例）
|
||||
**可复用的共享 fragments**（来自 `schema/fragments.ts`）：
|
||||
|
||||
| Fragment | 用途 |
|
||||
| ---------------------------- | -------------------------------------------------------- |
|
||||
| ------------------------------ | -------------------------------------------------------- |
|
||||
| `durationSchema` | 时长字符串（`"30s"`、`"5m"`、`"2h"`、`"7d"`、`"500ms"`） |
|
||||
| `sizeSchema` | 大小单位（字符串如 `"10MB"` 或整数） |
|
||||
| `statusCodePatternSchema` | 状态码（`100`-`599` 或 `"2xx"`） |
|
||||
| `stringMapSchema` | `Record<string, string>`（用于 headers / env） |
|
||||
| `createBodyRulesSchema()` | body 规则数组（json/css/xpath/contains/regex） |
|
||||
| `createTextRulesSchema()` | 文本规则数组（stdout/stderr） |
|
||||
| `createPureOperatorSchema()` | 操作符对象 |
|
||||
| `operatorProperties()` | 所有操作符字段的 Record |
|
||||
| `createValueMatcherSchema()` | `ValueMatcher` 对象（equals/contains/regex/数值比较等） |
|
||||
| `createContentRulesSchema()` | `ContentRules` 数组（direct/json/css/xpath 内容规则） |
|
||||
| `createKeyValueExpectSchema()` | 动态键 `KeyValueExpect`（headers、DB rows 列值） |
|
||||
| `matcherProperties()` | matcher 字段 Record，供 extractor schema 复用 |
|
||||
|
||||
**注意**：默认对象策略为 `additionalProperties: false`。只有明确的动态键值表（如 `http.headers`、`cmd.env`）可以开放任意键名。
|
||||
|
||||
@@ -311,12 +313,15 @@ checkerRegistry（单例）
|
||||
export function validateTcpConfig(input: CheckerValidationInput): ConfigValidationIssue[];
|
||||
```
|
||||
|
||||
**共享校验工具**（`expect/validate-operator.ts`）：
|
||||
**共享校验工具**（`expect/validate-matcher.ts`）：
|
||||
|
||||
| 函数 | 用途 |
|
||||
| --------------------------------------------------------- | ---------------------- |
|
||||
| `validateOperatorObject(ops, path, targetName, options?)` | 校验操作符对象 |
|
||||
| `isJsonValue(value)` | 判断是否为合法 JSON 值 |
|
||||
| ------------------------------------------------------ | ----------------------------------------- |
|
||||
| `validateValueMatcher(value, path, targetName, opts?)` | 校验 matcher 字段、类型、regex 和组合语义 |
|
||||
| `validateContentRules(rules, path, targetName)` | 校验 ContentRules 数组、extractor 互斥性 |
|
||||
| `validateKeyValueExpect(value, path, targetName)` | 校验动态键值断言 |
|
||||
| `validateJsonPath(path, rulePath, targetName)` | 校验项目支持的 JSONPath 子集 |
|
||||
| `isJsonValue(value)` | 判断是否为合法 JSON value |
|
||||
|
||||
#### 1.7.5 步骤四：实现 Checker 类
|
||||
|
||||
@@ -352,14 +357,17 @@ TcpChecker implements Checker
|
||||
**可用的共享断言工具**（`checker/expect/`）：
|
||||
|
||||
| 模块 | 函数 | 用途 |
|
||||
| ---------------------- | ----------------------------------------------------- | ------------------------------------- |
|
||||
| --------------------- | ----------------------------------------------------- | ------------------------------------- |
|
||||
| `failure.ts` | `errorFailure(phase, path, msg)` | 构造错误类型 failure |
|
||||
| `failure.ts` | `mismatchFailure(phase, path, expected, actual, msg)` | 构造不匹配类型 failure |
|
||||
| `failure.ts` | `truncateActual(value, maxLen?)` | 截断过长的 actual 值（默认 200 字符） |
|
||||
| `duration.ts` | `checkDuration(ms, maxMs?)` | 耗时断言 |
|
||||
| `operator.ts` | `applyOperator(actual, operator)` | 执行单个操作符比较 |
|
||||
| `operator.ts` | `evaluateJsonPath(json, path)` | JSONPath 提取 |
|
||||
| `validate-operator.ts` | `validateOperatorObject(ops, path, name)` | 操作符语义校验 |
|
||||
| `matcher.ts` | `applyMatcher(actual, matcher, options?)` | 执行 ValueMatcher AND 匹配 |
|
||||
| `matcher.ts` | `checkValueMatcher(actual, matcher, options)` | 执行 matcher 并返回 `ExpectResult` |
|
||||
| `matcher.ts` | `checkExpectValue(actual, expected)` | 执行字面量 equals 或 matcher |
|
||||
| `matcher.ts` | `evaluateJsonPath(json, path)` | JSONPath 提取 |
|
||||
| `content.ts` | `checkContentRules(source, rules, options)` | 执行 ContentRules 数组 |
|
||||
| `key-value.ts` | `checkKeyValueExpect(actual, expected, options)` | 执行动态键值断言 |
|
||||
| `validate-matcher.ts` | `validateValueMatcher/ContentRules/KeyValueExpect` | matcher/content/key-value 语义校验 |
|
||||
|
||||
**Checker 专属断言**（如需要）放在同目录的 `expect.ts` 中，参考 `http/expect.ts`（checkStatus、checkHeaders）和 `cmd/expect.ts`（checkExitCode）。
|
||||
|
||||
@@ -486,39 +494,77 @@ TcpChecker implements Checker
|
||||
|
||||
### 1.10 expect 断言系统
|
||||
|
||||
两层模型：**观测值收集** → **规则校验**。共享断言基础设施位于 `checker/expect/`，checker 专属断言位于各自目录。
|
||||
两层模型：**观测值收集** → **规则校验**。共享断言基础设施位于 `checker/expect/`，checker 专属状态断言位于各自目录。
|
||||
|
||||
**HTTP 校验流程**：
|
||||
**共享模型**：
|
||||
|
||||
| 模型 | 用途 | 典型字段 |
|
||||
| ---------------- | ---------------------------------------------- | -------------------------------------------------------------------- |
|
||||
| `ValueMatcher` | 单个值、数字指标和字符串元数据断言 | `durationMs`、`rowCount`、`usage.totalTokens`、`finishReason` |
|
||||
| `ContentRules` | 返回内容或半结构化内容断言，必须是数组 | `body`、`stdout`、`stderr`、`banner`、`response`、`output`、`result` |
|
||||
| `KeyValueExpect` | 动态键值断言，字面量等价于 `{ equals: value }` | `headers`、DB `rows[]` 中的列值 |
|
||||
|
||||
`ValueMatcher` 支持 `equals`、`contains`、`regex`、`empty`、`exists`、`gte`、`lte`、`gt`、`lt`。一个 matcher 对象内多个字段为 AND 语义；`exists: false` 不能和其他 matcher 组合；`equals` 使用 `es-toolkit/isEqual` 做 JSON 深度相等；`regex` 固定为无 flags 的 `new RegExp(pattern).test(String(actual))`。
|
||||
|
||||
`ContentRules` 数组按顺序快速失败。数组项可以是直接 matcher，也可以是 `{ json: {...} }`、`{ css: {...} }`、`{ xpath: {...} }` 提取器规则；一条规则不能混用直接 matcher 和 extractor，多个 extractor 也不能共存。Extractor 未配置 matcher 时等价于 `exists: true`。对对象或数组源执行直接 `contains`/`regex` 时会先 JSON 序列化，`equals` 仍对原始结构做深度相等。
|
||||
|
||||
启动期语义校验统一由 `expect/validate-matcher.ts` 负责，会校验空 matcher、未知字段、字段类型、`exists:false` 组合、ContentRules 互斥性、JSONPath 子集、XPath 可编译性、regex 可编译性和 ReDoS 风险。旧字段 `match`、`maxDurationMs`、Ping 的 `max*` 阈值字段不再支持。
|
||||
|
||||
**快速失败顺序**：
|
||||
|
||||
| Checker | 顺序 |
|
||||
| ---------- | -------------------------------------------------------------------------------------------------------------------------- |
|
||||
| HTTP | `status → headers → body → durationMs` |
|
||||
| Cmd | `exitCode → durationMs → stdout → stderr` |
|
||||
| DB | `durationMs → rowCount → rows → result` |
|
||||
| TCP | `connected → banner → durationMs` |
|
||||
| UDP | `responded → responseSize → response → sourceHost → sourcePort → durationMs` |
|
||||
| Ping | `alive → packetLossPercent → avgLatencyMs → maxLatencyMs → durationMs` |
|
||||
| LLM http | `status → headers → output → finishReason → rawFinishReason → usage → durationMs` |
|
||||
| LLM stream | `status → headers → stream.completed → stream.firstTokenMs → output → finishReason → rawFinishReason → usage → durationMs` |
|
||||
|
||||
HTTP checker 的 `durationMs` 覆盖完整执行（含重定向、响应体读取、解码和 expect 校验）。status 或 headers 失败时不读取 body；有 body 规则时，在读取 body 前可先检查是否已超过 `durationMs` 阈值，避免无意义读取。
|
||||
|
||||
**expect 字段选择规范**：
|
||||
|
||||
新增或修改 checker 的 expect 字段时，按以下决策树选择合适的断言模型：
|
||||
|
||||
```
|
||||
HttpChecker.execute → 收集观测(statusCode/headers)
|
||||
→ status → headers → (early duration) → body(按需) → (final duration)
|
||||
→ 首个失败即停止，返回 CheckFailure
|
||||
expect 字段
|
||||
│
|
||||
├─ 状态类结果，结果集合小且稳定
|
||||
│ └─ enum / boolean
|
||||
│ HTTP/LLM status、Cmd exitCode、TCP connected、
|
||||
│ UDP responded、Ping alive
|
||||
│
|
||||
├─ 数字指标 / 字符串元数据
|
||||
│ └─ ValueMatcher
|
||||
│ durationMs、rowCount、responseSize、sourceHost、sourcePort、
|
||||
│ packetLossPercent、avgLatencyMs、maxLatencyMs、
|
||||
│ finishReason、rawFinishReason、usage.*、stream.firstTokenMs
|
||||
│
|
||||
└─ 返回内容 / 半结构化内容 / 不完全确定的值
|
||||
├─ 内容断言 → ContentRules（数组）
|
||||
│ HTTP body、Cmd stdout/stderr、TCP banner、
|
||||
│ UDP response、LLM output、DB result
|
||||
│
|
||||
└─ 键值断言 → KeyValueExpect（动态键对象）
|
||||
HTTP/LLM headers、DB rows[] 中的列值
|
||||
```
|
||||
|
||||
HTTP checker 的 `durationMs` 覆盖完整执行（含重定向、响应体读取、解码和 expect 校验）。status 或 headers 失败时不读取 body；进入 body 前若已超过 `maxDurationMs`，直接返回 duration failure。
|
||||
选择原则：
|
||||
|
||||
**Cmd 校验流程**：
|
||||
1. **状态类字段使用 enum 或 boolean**。结果集合小且稳定时（如 HTTP status 200/2xx、exitCode 0），枚举和布尔比 matcher 更贴近协议语义，配置也更直观。不要为了统一而把状态类字段改成 ValueMatcher。
|
||||
|
||||
```
|
||||
CommandChecker.execute → 收集观测(exitCode/stdout/stderr/durationMs)
|
||||
→ exitCode → duration → stdout → stderr
|
||||
→ 首个失败即停止
|
||||
```
|
||||
2. **单值数字指标和字符串元数据使用 ValueMatcher**。观测值是一个明确的标量（耗时、行数、丢包率、finish reason），但阈值不确定时，使用 `{ lte: 100 }` 或 `{ regex: "^(stop|end)$" }` 等 matcher 表达。
|
||||
|
||||
**Body 规则类型**（`runner/http/body.ts`）：
|
||||
3. **返回内容使用 ContentRules 数组**。观测值是文本、JSON、HTML 或 XML 内容，且可能需要多步提取或多条规则时，使用 ContentRules。即使只有一条规则也必须写成数组形式（`[{ contains: "ok" }]`），不支持对象快捷写法。
|
||||
|
||||
- `contains`：文本包含匹配
|
||||
- `regex`：正则表达式匹配（注意：body 正则字段为 `regex`，不是 `match`，启动期会拒绝嵌套量词等 ReDoS 风险模式）
|
||||
- `json`：JSONPath 提取 + 操作符比较（使用 `es-toolkit/isPlainObject` 区分纯值和操作符）
|
||||
- `css`：cheerio CSS 选择器 + 操作符比较
|
||||
- `xpath`：XPath 节点提取 + 操作符比较
|
||||
4. **键值对使用 KeyValueExpect**。观测值是动态键值表（如 headers），且需要对每个键独立断言时使用。字面量值自动等价于 `{ equals: value }`。
|
||||
|
||||
**文本规则**（`runner/cmd/text.ts`）：stdout/stderr 文本匹配，支持 `contains`、`match`（正则）、操作符比较
|
||||
5. **不要混用模型**。一个 expect 字段只能对应一种断言模型。例如 `finishReason` 是单值字符串元数据，用 ValueMatcher 而非 ContentRules（ContentRules 的 json/css/xpath 提取器对单字符串无意义，且会增加数组包装的配置冗余）。
|
||||
|
||||
**操作符**（`expect/operator.ts`）：`equals`（深度比较，`es-toolkit/isEqual`）、`contains`、`match`（正则，启动期通过 `expect/redos.ts` 拒绝 ReDoS 风险模式）、`empty`（`isNil`+`isEmptyObject`）、`exists`、`gte`/`lte`/`gt`/`lt`
|
||||
|
||||
启动期语义校验会对 HTTP body `regex` 规则和所有 `match` operator 执行静态 ReDoS 检测，常见的嵌套量词模式如 `(a+)+`、`(\\d+)*x` 会被拒绝，避免运行期正则在大响应体上阻塞事件循环。
|
||||
6. **failure phase 命名遵循去单位后缀规则**。数字指标字段的 phase 去掉单位后缀（`durationMs` → `duration`、`packetLossPercent` → `packetLoss`、`avgLatencyMs` → `avgLatency`），不带单位后缀的字段直接使用字段名（`rowCount` → `rowCount`、`finishReason` → `finishReason`）。
|
||||
|
||||
### 1.11 错误模式
|
||||
|
||||
@@ -529,12 +575,7 @@ CommandChecker.execute → 收集观测(exitCode/stdout/stderr/durationMs)
|
||||
|
||||
### 1.12 测试规范
|
||||
|
||||
- 测试目录 `tests/` 镜像 `src/` 目录结构，但共享模块的测试集中放在 `tests/server/checker/runner/shared/` 下
|
||||
- `tests/server/checker/runner/shared/failure.test.ts` ↔ `src/server/checker/expect/failure.ts`
|
||||
- `tests/server/checker/runner/shared/duration.test.ts` ↔ `src/server/checker/expect/duration.ts`
|
||||
- `tests/server/checker/runner/shared/operator.test.ts` ↔ `src/server/checker/expect/operator.ts`
|
||||
- `tests/server/checker/runner/shared/body.test.ts` ↔ `src/server/checker/runner/http/body.ts`
|
||||
- `tests/server/checker/runner/shared/text.test.ts` ↔ `src/server/checker/runner/cmd/text.ts`
|
||||
- 测试目录 `tests/` 镜像 `src/` 目录结构，但共享 expect 模块的测试集中放在 `tests/server/checker/runner/shared/` 下，覆盖 `failure.ts`、`matcher.ts`、`content.ts`、`key-value.ts`、`validate-matcher.ts` 和 `redos.ts`
|
||||
- 使用 `bun:test` 框架（`describe`/`test`/`expect`），测试数据库用临时目录 + `tmpdir()`
|
||||
- 新增 store 方法必须编写单元测试；新增 API 端点必须在 `app.test.ts` 中添加集成测试
|
||||
- 测试后清理：`afterAll` 中 `store.close()` + `rm(tempDir, { recursive: true })`
|
||||
|
||||
72
README.md
72
README.md
@@ -93,7 +93,8 @@ targets:
|
||||
url: "https://www.baidu.com"
|
||||
expect:
|
||||
status: [200]
|
||||
maxDurationMs: 10000
|
||||
durationMs:
|
||||
lte: 10000
|
||||
|
||||
- id: "json-api"
|
||||
name: "${env_name} JSON API 示例"
|
||||
@@ -130,7 +131,8 @@ targets:
|
||||
url: "${sqlite_url}"
|
||||
query: "SELECT COUNT(*) as cnt FROM users WHERE status = 'active'"
|
||||
expect:
|
||||
maxDurationMs: 5000
|
||||
durationMs:
|
||||
lte: 5000
|
||||
rowCount: { gte: 1 }
|
||||
rows:
|
||||
- cnt: { gte: 0 }
|
||||
@@ -142,7 +144,8 @@ targets:
|
||||
host: "127.0.0.1"
|
||||
port: 6379
|
||||
expect:
|
||||
maxDurationMs: 3000
|
||||
durationMs:
|
||||
lte: 3000
|
||||
|
||||
- id: "udp-heartbeat"
|
||||
name: "UDP 心跳检测"
|
||||
@@ -154,7 +157,8 @@ targets:
|
||||
expect:
|
||||
response:
|
||||
- contains: "PONG"
|
||||
maxDurationMs: 100
|
||||
durationMs:
|
||||
lte: 100
|
||||
|
||||
- id: "gateway-ping"
|
||||
name: "网关 ICMP 可达"
|
||||
@@ -165,10 +169,14 @@ targets:
|
||||
packetSize: 56
|
||||
expect:
|
||||
alive: true
|
||||
maxPacketLoss: 10
|
||||
maxAvgLatencyMs: 100
|
||||
maxMaxLatencyMs: 300
|
||||
maxDurationMs: 5000
|
||||
packetLossPercent:
|
||||
lte: 10
|
||||
avgLatencyMs:
|
||||
lte: 100
|
||||
maxLatencyMs:
|
||||
lte: 300
|
||||
durationMs:
|
||||
lte: 5000
|
||||
```
|
||||
|
||||
### 配置说明
|
||||
@@ -293,28 +301,34 @@ Ping checker 通过系统 `ping` 命令执行 ICMP 探测，支持 Linux、macOS
|
||||
#### expect — 期望校验
|
||||
|
||||
| 字段 | 适用类型 | 说明 |
|
||||
| ------------------- | -------- | ---------------------------------------------------------------- |
|
||||
| `status` | HTTP | 可接受的状态码列表，支持精确码和范围（如 `"2xx"`）；默认 `[200]` |
|
||||
| `exitCode` | Cmd | 可接受的退出码列表；未指定时不校验 |
|
||||
| `headers` | HTTP | 响应头校验 |
|
||||
| `maxDurationMs` | 全部 | 最大耗时阈值（毫秒） |
|
||||
| `output` | LLM | 模型输出校验（数组：`equals`/`contains`/`regex`/`json`） |
|
||||
| `finishReason` | LLM | 期望的 finish reason 字符串 |
|
||||
| `rawFinishReason` | LLM | 期望的原始 finish reason 字符串 |
|
||||
| `usage` | LLM | Token usage 校验（`inputTokens`/`outputTokens`/`totalTokens`） |
|
||||
| `stream` | LLM | 流式断言（`completed`、`firstTokenMs`，仅 `mode: stream`） |
|
||||
| `body` | HTTP | 响应体校验（数组，可组合使用，见下方） |
|
||||
| `stdout` / `stderr` | Cmd | 输出校验（数组，每项一个操作符对象） |
|
||||
| `rowCount` | DB | 查询返回行数校验（操作符对象） |
|
||||
| `rows` | DB | 查询结果逐行校验（数组，列名→操作符映射） |
|
||||
| ------------------- | -------- | ---------------------------------------------------------------------- |
|
||||
| `status` | HTTP/LLM | 可接受的状态码列表，支持精确码和范围（如 `"2xx"`）；默认 `[200]` |
|
||||
| `exitCode` | Cmd | 可接受的退出码列表；未指定时默认 `[0]` |
|
||||
| `headers` | HTTP/LLM | 响应头校验，使用动态键名和 `KeyValueExpect` |
|
||||
| `durationMs` | 全部 | 完整执行耗时校验，使用 `ValueMatcher`，如 `{ lte: 1000 }` |
|
||||
| `output` | LLM | 模型输出校验，使用 `ContentRules` 数组 |
|
||||
| `finishReason` | LLM | finish reason 校验，使用 `ValueMatcher` |
|
||||
| `rawFinishReason` | LLM | 原始 finish reason 校验，使用 `ValueMatcher` |
|
||||
| `usage` | LLM | Token usage 校验（`inputTokens`/`outputTokens`/`totalTokens` matcher） |
|
||||
| `stream` | LLM | 流式断言（`completed`、`firstTokenMs` matcher，仅 `mode: stream`） |
|
||||
| `body` | HTTP | 响应体校验，使用 `ContentRules` 数组 |
|
||||
| `stdout` / `stderr` | Cmd | 输出校验，使用 `ContentRules` 数组 |
|
||||
| `rowCount` | DB | 查询返回行数校验，使用 `ValueMatcher` |
|
||||
| `rows` | DB | 查询结果逐行校验，数组内每行为列名到 `KeyValueExpect` 的映射 |
|
||||
| `result` | DB | 完整查询结果 `{ rows, rowCount }` 校验，使用 `ContentRules` 数组 |
|
||||
| `connected` | TCP | 期望连接结果，`true`（默认）可达或 `false` 期望不可达 |
|
||||
| `banner` | TCP | Banner 文本校验（操作符对象，需开启 `tcp.readBanner`） |
|
||||
| `banner` | TCP | Banner 内容校验，使用 `ContentRules` 数组，需开启 `tcp.readBanner` |
|
||||
| `responded` | UDP | 期望是否收到响应，默认 `true` |
|
||||
| `response` | UDP | 响应内容校验，使用 `ContentRules` 数组 |
|
||||
| `responseSize` | UDP | 响应字节数校验，使用 `ValueMatcher` |
|
||||
| `sourceHost` | UDP | 响应来源地址校验，使用 `ValueMatcher` |
|
||||
| `sourcePort` | UDP | 响应来源端口校验，使用 `ValueMatcher` |
|
||||
| `alive` | Ping | 期望主机可达性，默认 `true` |
|
||||
| `maxPacketLoss` | Ping | 最大丢包率百分比，范围 `0-100` |
|
||||
| `maxAvgLatencyMs` | Ping | 最大平均延迟（毫秒） |
|
||||
| `maxMaxLatencyMs` | Ping | 最大单次延迟（毫秒） |
|
||||
| `packetLossPercent` | Ping | 丢包率百分比校验，范围 `0-100`，使用 `ValueMatcher` |
|
||||
| `avgLatencyMs` | Ping | 平均延迟校验，使用 `ValueMatcher` |
|
||||
| `maxLatencyMs` | Ping | 最大单次延迟校验，使用 `ValueMatcher` |
|
||||
|
||||
**body 校验项**（数组中可混合使用）：
|
||||
**ContentRules 校验项**（`body`、`stdout`、`stderr`、`banner`、`response`、`output`、`result` 均使用数组）：
|
||||
|
||||
- `contains` — 响应体包含指定文本
|
||||
- `regex` — 正则匹配（启动期会拒绝存在 ReDoS 风险的模式）
|
||||
@@ -322,7 +336,9 @@ Ping checker 通过系统 `ping` 命令执行 ICMP 探测，支持 Linux、macOS
|
||||
- `css` — CSS 选择器提取 HTML 元素（`selector` 必填，`attr` 可选提取属性）
|
||||
- `xpath` — XPath 提取 XML/HTML 节点（`path` 必填，如 `/html/body/h1/text()`）
|
||||
|
||||
**比较操作符**：`equals`（默认）、`contains`、`match`（正则）、`empty`、`exists`、`gte`、`lte`、`gt`、`lt`
|
||||
**ValueMatcher 字段**：`equals`、`contains`、`regex`、`empty`、`exists`、`gte`、`lte`、`gt`、`lt`。`equals` 支持 JSON 深度相等；`regex` 固定使用无 flags 正则；提取器未配置 matcher 时等价于 `exists: true`。
|
||||
|
||||
旧字段 `maxDurationMs`、`maxPacketLoss`、`maxAvgLatencyMs`、`maxMaxLatencyMs` 和旧正则字段 `match` 已移除，请分别改用 `durationMs`、Ping matcher 字段和 `regex`。
|
||||
|
||||
**大小说明**：`maxBodyBytes` 和 `maxOutputBytes` 支持 `KB`、`MB`、`GB` 单位，也可直接使用数字。
|
||||
|
||||
|
||||
@@ -145,35 +145,31 @@
|
||||
- **THEN** store SHALL 对每个 target 调用对应 checker 的 `serialize()` 方法获取 `{ target, config }`
|
||||
|
||||
### Requirement: 共享 expect 断言函数
|
||||
系统 SHALL 在 `src/server/checker/expect/` 中提供可被多个 checker 复用的 expect 函数。checker 专用的 expect 函数 SHALL 保留在各自子包内。仅被单个 checker 使用的断言模块 SHALL 位于该 checker 目录内。
|
||||
系统 SHALL 在 `src/server/checker/expect/` 中提供可被多个 checker 复用的 expect 基础设施。共享基础设施 SHALL 包含 matcher、content rules、key-value expect、failure 构造和 ReDoS 校验。checker 专用的状态类断言 SHALL 保留在对应 checker 目录，或在多个 checker 复用时移动到共享模块。仅被单个 checker 使用且不属于通用 matcher/content/key-value 模型的断言模块 SHALL 位于该 checker 目录内。
|
||||
|
||||
#### Scenario: 共享 duration 断言
|
||||
- **WHEN** 任何 checker 需要校验执行耗时
|
||||
- **THEN** SHALL 调用 `expect/duration.ts` 中的 `checkDuration(durationMs, maxDurationMs?)`，返回统一的 `ExpectResult`
|
||||
#### Scenario: 共享 ValueMatcher 断言
|
||||
- **WHEN** 任何 checker 需要对数字、字符串、布尔或 JSON value 执行 matcher 匹配
|
||||
- **THEN** SHALL 调用共享 matcher 工具执行 `equals`、`contains`、`regex`、`exists`、`empty`、`gt`、`gte`、`lt` 和 `lte` 语义
|
||||
|
||||
#### Scenario: 共享 operator 断言
|
||||
- **WHEN** 任何 checker 需要对值执行 operator 匹配
|
||||
- **THEN** SHALL 调用 `expect/operator.ts` 中的 `applyOperator(actual, op)`
|
||||
#### Scenario: 共享 ContentRules 断言
|
||||
- **WHEN** HTTP body、LLM output、Cmd stdout/stderr、UDP response 或 TCP banner 需要执行返回内容校验
|
||||
- **THEN** SHALL 调用共享 content rules 工具，而不是在 checker 目录内复制 contains/regex/json/css/xpath 逻辑
|
||||
|
||||
#### Scenario: 共享 KeyValueExpect 断言
|
||||
- **WHEN** HTTP 或 LLM checker 需要校验响应 headers，或 DB checker 需要校验 rows 中的列值
|
||||
- **THEN** SHALL 调用共享 key-value expect 工具，并按调用方规则决定 key 是否大小写敏感
|
||||
|
||||
#### Scenario: 共享 regex ReDoS 校验
|
||||
- **WHEN** 任一 matcher 或 content rule 配置 `regex`
|
||||
- **THEN** SHALL 调用共享 ReDoS 校验工具在启动期拒绝危险正则
|
||||
|
||||
#### Scenario: 共享 failure 构造
|
||||
- **WHEN** 任何 checker 需要构造 CheckFailure 对象
|
||||
- **THEN** SHALL 调用 `expect/failure.ts` 中的 `errorFailure()` 或 `mismatchFailure()`
|
||||
- **THEN** SHALL 调用 `expect/failure.ts` 中的 `errorFailure()` 或 `mismatchFailure()`，并保留 actual 截断策略
|
||||
|
||||
#### Scenario: HTTP body 断言位于 HTTP 目录
|
||||
- **WHEN** HTTP checker 需要对响应体执行 contains/regex/json/css/xpath 规则校验
|
||||
- **THEN** SHALL 调用 `runner/http/body.ts` 中的 `checkBodyExpect(body, rules)`
|
||||
|
||||
#### Scenario: Cmd text 断言位于 Cmd 目录
|
||||
- **WHEN** Cmd checker 需要对 stdout/stderr 执行文本规则校验
|
||||
- **THEN** SHALL 调用 `runner/cmd/text.ts` 中的 `checkTextRules(text, rules, phase)`
|
||||
|
||||
#### Scenario: HTTP 专用 expect
|
||||
- **WHEN** HTTP checker 需要校验响应状态码和响应头
|
||||
- **THEN** SHALL 调用 `runner/http/expect.ts` 中的 `checkStatus()` 和 `checkHeaders()`
|
||||
|
||||
#### Scenario: Cmd 专用 expect
|
||||
- **WHEN** Cmd checker 需要校验退出码
|
||||
- **THEN** SHALL 调用 `runner/cmd/expect.ts` 中的 `checkExitCode()`
|
||||
#### Scenario: HTTP 专用 status 断言
|
||||
- **WHEN** HTTP 或 LLM checker 需要校验响应状态码
|
||||
- **THEN** SHALL 复用同一 status 断言函数，支持精确状态码和 `1xx` 到 `5xx` 范围模式
|
||||
|
||||
### Requirement: 超时控制由引擎注入 signal
|
||||
Checker 实现的 `execute()` MUST 使用 `ctx.signal` 感知超时，不得自行创建 `AbortController` 或 `setTimeout` 用于超时控制。Cmd checker 和 ping checker 可在 signal abort 时 `proc.kill()` 以确保子进程被终止。
|
||||
|
||||
@@ -55,7 +55,7 @@
|
||||
- **THEN** 系统 MUST 停止收集输出并终止该检查，记录 `matched=false`，并在 failure 中写入输出超限信息
|
||||
|
||||
### Requirement: cmd expect 校验
|
||||
系统 SHALL 支持 cmd 专用 expect，包括 `exitCode`、`stdout` 和 `stderr`，并按 exitCode、duration、stdout、stderr 的阶段顺序快速失败。
|
||||
系统 SHALL 支持 cmd 专用 expect，包括 `exitCode`、`durationMs`、`stdout` 和 `stderr`，并按 exitCode、durationMs、stdout、stderr 的阶段顺序快速失败。`exitCode` SHALL 保持有限整数数组语义，未配置时默认 `[0]`。`durationMs` SHALL 使用共享 `ValueMatcher` 校验完整命令执行耗时。`stdout` 和 `stderr` MUST 使用共享 `ContentRules` 数组，直接 matcher 作用于对应输出文本，`json` extractor SHALL 支持对 JSON CLI 输出执行 JSONPath 断言。
|
||||
|
||||
#### Scenario: 默认 exitCode 成功语义
|
||||
- **WHEN** cmd target 未显式配置 `expect.exitCode`
|
||||
@@ -67,75 +67,67 @@
|
||||
|
||||
#### Scenario: exitCode 不匹配快速失败
|
||||
- **WHEN** cmd target 配置 `expect.exitCode: [0]` 且实际 exit code 为 1
|
||||
- **THEN** 系统 SHALL 立即返回 `matched=false`，并在 failure 中写入 phase=`exitCode`、path=`expect.exitCode`、expected 和 actual
|
||||
- **THEN** 系统 SHALL 立即返回 `matched=false`，并在 failure 中写入 phase=`exitCode`、path=`exitCode`、expected 和 actual
|
||||
|
||||
#### Scenario: durationMs 校验
|
||||
- **WHEN** cmd target 配置 `expect.durationMs: {lte: 1000}` 且实际执行耗时为 1500ms
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `duration`
|
||||
|
||||
#### Scenario: stdout 按配置顺序校验
|
||||
- **WHEN** cmd target 配置 `expect.stdout` 为两个规则，第一条通过且第二条失败
|
||||
- **THEN** 系统 SHALL 先执行第一条 stdout 规则，再执行第二条，并将 failure.path 指向失败的 `expect.stdout[1]`
|
||||
- **WHEN** cmd target 配置 `expect.stdout` 为两个 ContentRules，第一条通过且第二条失败
|
||||
- **THEN** 系统 SHALL 先执行第一条 stdout 规则，再执行第二条，并将 failure.path 指向失败的 `stdout[1]`
|
||||
|
||||
#### Scenario: stderr 校验为空
|
||||
- **WHEN** cmd target 配置 `expect.stderr: [{empty: true}]` 且实际 stderr 为空字符串
|
||||
- **THEN** 系统 SHALL 判定 stderr 阶段通过
|
||||
|
||||
#### Scenario: stdout JSON 输出校验
|
||||
- **WHEN** cmd target 输出 stdout 为 `{"status":"ok"}` 且配置 `expect.stdout: [{json: {path: "$.status", equals: "ok"}}]`
|
||||
- **THEN** 系统 SHALL 判定 stdout 阶段通过
|
||||
|
||||
#### Scenario: stdout 失败后不检查 stderr
|
||||
- **WHEN** cmd target 同时配置 stdout 和 stderr 规则，且 stdout 规则失败
|
||||
- **THEN** 系统 SHALL 快速失败并 MUST NOT 继续执行 stderr 规则
|
||||
|
||||
### Requirement: cmd checker 启动期配置校验
|
||||
系统 SHALL 在启动期对 cmd checker 的配置契约和语义执行严格校验。Cmd target 的 `cmd` 分组 SHALL 只允许 `exec`、`args`、`cwd`、`env`、`maxOutputBytes` 字段；Cmd expect SHALL 只允许 `exitCode`、`maxDurationMs`、`stdout`、`stderr` 字段。未知字段、非法类型和不可编译正则 MUST 导致启动期配置错误。`expect.exitCode` SHALL 保留原有有限整数数组语义，不限制到特定平台范围。
|
||||
系统 SHALL 在启动期对 cmd checker 的配置契约和语义执行严格校验。Cmd target 的 `cmd` 分组 SHALL 只允许 `exec`、`args`、`cwd`、`env`、`maxOutputBytes` 字段。Cmd expect SHALL 只允许 `exitCode`、`durationMs`、`stdout`、`stderr` 字段。未知字段、非法类型、不可编译正则和 ReDoS 风险正则 MUST 导致启动期配置错误。`expect.exitCode` SHALL 保留原有有限整数数组语义，不限制到特定平台范围。
|
||||
|
||||
#### Scenario: cmd args 类型非法
|
||||
- **WHEN** YAML 中 cmd target 配置 `cmd.args` 不是字符串数组
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 cmd.args 格式错误
|
||||
|
||||
#### Scenario: cmd cwd 类型非法
|
||||
- **WHEN** YAML 中 cmd target 配置 `cmd.cwd` 不是字符串
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 cmd.cwd 必须为字符串
|
||||
|
||||
#### Scenario: cmd env 值类型非法
|
||||
- **WHEN** YAML 中 cmd target 配置 `cmd.env`，且任一环境变量值不是字符串
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 cmd.env 对应变量值必须为字符串
|
||||
|
||||
#### Scenario: cmd maxOutputBytes 非法
|
||||
- **WHEN** YAML 中 cmd target 或 defaults.cmd 配置的 `maxOutputBytes` 不是合法 size 值
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 maxOutputBytes 格式错误
|
||||
|
||||
#### Scenario: cmd 分组未知字段失败
|
||||
- **WHEN** YAML 中 cmd target 的 `cmd` 分组包含 `shell: true` 等未知字段
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 cmd 分组包含未知字段
|
||||
|
||||
#### Scenario: cmd expect exitCode 类型非法
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.exitCode` 不是整数数组
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.exitCode 必须为整数数组
|
||||
|
||||
#### Scenario: cmd expect exitCode 不限制平台范围
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.exitCode` 为有限整数数组
|
||||
- **THEN** 系统 SHALL 接受该数组，不额外限制为 0-255 等平台相关范围
|
||||
#### Scenario: cmd expect durationMs 非法
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.durationMs` 不是合法 `ValueMatcher`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.durationMs 格式错误
|
||||
|
||||
#### Scenario: cmd expect maxDurationMs 非法
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.maxDurationMs` 不是非负有限数字
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.maxDurationMs 格式错误
|
||||
|
||||
#### Scenario: stdout 必须为规则数组
|
||||
#### Scenario: stdout 必须为 ContentRules 数组
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.stdout` 但其值不是数组
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.stdout 必须为数组
|
||||
|
||||
#### Scenario: stderr 必须为规则数组
|
||||
#### Scenario: stderr 必须为 ContentRules 数组
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.stderr` 但其值不是数组
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.stderr 必须为数组
|
||||
|
||||
#### Scenario: stdout text rule 空对象非法
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.stdout: [{}]`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 stdout 规则必须包含至少一个合法 operator
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 stdout 规则必须包含至少一个合法 matcher 或 extractor
|
||||
|
||||
#### Scenario: stderr text rule 未知字段非法
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.stderr: [{foo: "bar"}]`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 stderr 规则包含未知 operator
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 stderr 规则包含未知 matcher 或未知 extractor
|
||||
|
||||
#### Scenario: stdout match 正则非法
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.stdout: [{match: "[invalid"}]`
|
||||
#### Scenario: stdout regex 正则非法
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.stdout: [{regex: "[invalid"}]`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，而不是延迟到运行期抛错
|
||||
|
||||
#### Scenario: cmd expect 未知字段失败
|
||||
- **WHEN** YAML 中 cmd target 的 expect 包含 `status: [200]` 或其他非 cmd expect 字段
|
||||
- **WHEN** YAML 中 cmd target 的 expect 包含 `status: [200]`、`maxDurationMs: 1000` 或其他非 cmd expect 字段
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段
|
||||
|
||||
@@ -75,11 +75,11 @@
|
||||
- **THEN** 系统 SHALL 立即关闭数据库连接
|
||||
|
||||
### Requirement: db expect 校验
|
||||
系统 SHALL 支持 db 专用 expect，包括 `maxDurationMs`、`rowCount` 和 `rows`，按 duration、rowCount、rows 的阶段顺序快速失败。
|
||||
系统 SHALL 支持 db 专用 expect，包括 `durationMs`、`rowCount`、`rows` 和 `result`，按 durationMs、rowCount、rows、result 的阶段顺序快速失败。`durationMs` 和 `rowCount` SHALL 使用共享 `ValueMatcher`。`rows` SHALL 保留按行索引匹配列值的语义，类型为 `Array<KeyValueExpect>`（外层数组按行索引，内层每个元素为一个 `KeyValueExpect` 表达该行的列值断言），每个行规则中列值字面量等价于 `{equals: <literal>}`。`result` MUST 使用共享 `ContentRules` 数组，对查询结果对象 `{ rows, rowCount }` 执行断言。
|
||||
|
||||
#### Scenario: maxDurationMs 校验
|
||||
- **WHEN** db target 配置 `expect.maxDurationMs: 3000` 且实际执行耗时 4000ms
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `"duration"`
|
||||
#### Scenario: durationMs 校验
|
||||
- **WHEN** db target 配置 `expect.durationMs: {lte: 3000}` 且实际执行耗时 4000ms
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `duration`
|
||||
|
||||
#### Scenario: rowCount 校验通过
|
||||
- **WHEN** db target 配置 `expect.rowCount: { gte: 1 }` 且查询返回 5 行
|
||||
@@ -87,13 +87,13 @@
|
||||
|
||||
#### Scenario: rowCount 校验失败
|
||||
- **WHEN** db target 配置 `expect.rowCount: { gte: 1 }` 且查询返回 0 行
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `"rowCount"`，path 为 `"rowCount"`，expected 为 `{ gte: 1 }`，actual 为 0
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `rowCount`，path 为 `rowCount`，expected 为 `{ gte: 1 }`，actual 为 0
|
||||
|
||||
#### Scenario: rows 按索引匹配列值（operator 形式）
|
||||
#### Scenario: rows 按索引匹配列值 matcher 形式
|
||||
- **WHEN** db target 配置 `expect.rows: [{ cnt: { gte: 100 } }]` 且查询首行 cnt 列值为 50
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `"row"`，path 为 `"rows[0].cnt"`
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `row`，path 为 `rows[0].cnt`
|
||||
|
||||
#### Scenario: rows 按索引匹配列值（字面量形式）
|
||||
#### Scenario: rows 按索引匹配列值字面量形式
|
||||
- **WHEN** db target 配置 `expect.rows: [{ status: "active" }]` 且查询首行 status 列值为 `"active"`
|
||||
- **THEN** 系统 SHALL 判定该行该列通过（字面量等价于 `{ equals: "active" }`）
|
||||
|
||||
@@ -103,25 +103,33 @@
|
||||
|
||||
#### Scenario: rows 结果行数不足
|
||||
- **WHEN** db target 配置 `expect.rows` 包含 3 个元素但查询仅返回 2 行
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `"row"`，message 说明结果行数不足
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `row`，message 说明结果行数不足
|
||||
|
||||
#### Scenario: 无 query 时 expect 被忽略
|
||||
- **WHEN** db target 未配置 `db.query` 但配置了 `expect.rowCount`
|
||||
- **THEN** 系统 SHALL 忽略 expect 中的 rowCount 和 rows 断言（仅 maxDurationMs 生效）
|
||||
#### Scenario: result JSONPath 校验
|
||||
- **WHEN** db target 查询返回首行 `{status: "active"}` 且配置 `expect.result: [{json: {path: "$.rows[0].status", equals: "active"}}]`
|
||||
- **THEN** 系统 SHALL 基于 `{rows, rowCount}` 结果对象执行 JSONPath，并判定 result 阶段通过
|
||||
|
||||
#### Scenario: result rowCount 校验
|
||||
- **WHEN** db target 查询返回 2 行且配置 `expect.result: [{json: {path: "$.rowCount", equals: 2}}]`
|
||||
- **THEN** 系统 SHALL 判定 result 阶段通过
|
||||
|
||||
#### Scenario: 无 query 时结果类 expect 被忽略
|
||||
- **WHEN** db target 未配置 `db.query` 但配置了 `expect.rowCount`、`expect.rows` 或 `expect.result`
|
||||
- **THEN** 系统 SHALL 忽略这些查询结果断言（仅 `durationMs` 生效）
|
||||
|
||||
#### Scenario: 快速失败顺序
|
||||
- **WHEN** db target 同时配置 maxDurationMs、rowCount 和 rows
|
||||
- **THEN** 系统 SHALL 按 duration → rowCount → rows 顺序校验，任一阶段失败立即返回
|
||||
- **WHEN** db target 同时配置 durationMs、rowCount、rows 和 result
|
||||
- **THEN** 系统 SHALL 按 durationMs → rowCount → rows → result 顺序校验，任一阶段失败立即返回
|
||||
|
||||
### Requirement: db checker 启动期配置校验
|
||||
系统 SHALL 在启动期对 db checker 的配置契约和语义执行严格校验。Db target 的 `db` 分组 SHALL 只允许 `url` 和 `query` 字段。Db expect SHALL 只允许 `maxDurationMs`、`rowCount` 和 `rows` 字段。
|
||||
系统 SHALL 在启动期对 db checker 的配置契约和语义执行严格校验。Db target 的 `db` 分组 SHALL 只允许 `url` 和 `query` 字段。Db expect SHALL 只允许 `durationMs`、`rowCount`、`rows` 和 `result` 字段。未知字段、非法 matcher、非法 ContentRules、非法 regex 和 ReDoS 风险正则 MUST 导致启动期配置错误。
|
||||
|
||||
#### Scenario: db expect maxDurationMs 非法
|
||||
- **WHEN** YAML 中 db target 配置 `expect.maxDurationMs` 不是非负有限数字
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.maxDurationMs 格式错误
|
||||
#### Scenario: db expect durationMs 非法
|
||||
- **WHEN** YAML 中 db target 配置 `expect.durationMs` 不是合法 `ValueMatcher`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.durationMs 格式错误
|
||||
|
||||
#### Scenario: db expect rowCount 非法
|
||||
- **WHEN** YAML 中 db target 配置 `expect.rowCount` 不是合法的 operator 对象
|
||||
- **WHEN** YAML 中 db target 配置 `expect.rowCount` 不是合法 `ValueMatcher`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.rowCount 格式错误
|
||||
|
||||
#### Scenario: db expect rows 非法
|
||||
@@ -129,13 +137,17 @@
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.rows 必须为对象数组
|
||||
|
||||
#### Scenario: db expect rows 元素列值非法
|
||||
- **WHEN** YAML 中 db target 配置 `expect.rows: [{ cnt: { foo: 1 } }]`，其中 foo 不是合法 operator
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 rows 中包含未知 operator
|
||||
- **WHEN** YAML 中 db target 配置 `expect.rows: [{ cnt: { foo: 1 } }]`，其中 foo 不是合法 matcher
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 rows 中包含未知 matcher
|
||||
|
||||
#### Scenario: db expect result 非法
|
||||
- **WHEN** YAML 中 db target 配置 `expect.result` 不是合法 ContentRules 数组
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.result 格式错误
|
||||
|
||||
#### Scenario: db expect 未知字段失败
|
||||
- **WHEN** YAML 中 db target 的 expect 包含 `status: [200]` 或其他非 db expect 字段
|
||||
- **WHEN** YAML 中 db target 的 expect 包含 `status: [200]`、`maxDurationMs: 1000` 或其他非 db expect 字段
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段
|
||||
|
||||
#### Scenario: db expect rows 中 match 正则非法
|
||||
- **WHEN** YAML 中 db target 配置 `expect.rows: [{ name: { match: "[invalid" } }]`
|
||||
#### Scenario: db expect rows 中 regex 正则非法
|
||||
- **WHEN** YAML 中 db target 配置 `expect.rows: [{ name: { regex: "[invalid" } }]`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，而不是延迟到运行期抛错
|
||||
|
||||
@@ -5,35 +5,23 @@
|
||||
## Requirements
|
||||
|
||||
### Requirement: 响应体多种校验方法
|
||||
系统 SHALL 支持对 HTTP 响应体进行五种可组合的校验方法：contains（子串）、regex（正则）、json（JSONPath）、css（CSS 选择器）、xpath（XPath）。这些方法 MUST 配置在 `expect.body` 有序数组中。
|
||||
系统 SHALL 支持通过共享 `ContentRules` 对 HTTP 响应体进行有序内容校验。`expect.body` MUST 为规则数组。每个规则 SHALL 为直接 `ValueMatcher`，或 `json`、`css`、`xpath` extractor 规则之一。直接 matcher SHALL 作用于完整响应体文本。`json` SHALL 解析响应体为 JSON 并用 JSONPath 子集提取值。`css` SHALL 使用 CSS selector 从 HTML 中提取元素文本或属性。`xpath` SHALL 使用 XPath 从 XML/HTML 中提取节点值。Extractor 未配置 matcher 时 SHALL 等价于 `exists: true`。
|
||||
|
||||
#### Scenario: contains 子串匹配
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{contains: "healthy"}]`，且响应体包含 `"healthy"`
|
||||
- **THEN** 系统 SHALL 判定该 body 规则通过
|
||||
|
||||
#### Scenario: contains 不匹配
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{contains: "healthy"}]`，且响应体不包含该文本
|
||||
- **THEN** 系统 SHALL 判定 matched 为 false，并记录该规则的 failure.path
|
||||
|
||||
#### Scenario: regex 正则匹配
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{regex: '"status"\\s*:\\s*"ok"'}]`，且响应体匹配该正则
|
||||
- **THEN** 系统 SHALL 判定该 body 规则通过
|
||||
|
||||
#### Scenario: regex 不匹配
|
||||
- **WHEN** HTTP target 配置 regex body 规则，且响应体不匹配该正则
|
||||
- **THEN** 系统 SHALL 判定 matched 为 false，并记录该规则的 failure.path
|
||||
|
||||
#### Scenario: json JSONPath 等值匹配
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{json: {path: "$.status", equals: "ok"}}]`，且响应 JSON 中 `$.status` 值为 `"ok"`
|
||||
- **THEN** 系统 SHALL 判定该 body 规则通过
|
||||
|
||||
#### Scenario: json JSONPath 值不匹配
|
||||
- **WHEN** HTTP target 配置 json body 规则，且提取值不符合期望
|
||||
- **THEN** 系统 SHALL 判定 matched 为 false，并记录包含 JSONPath 的 failure.path
|
||||
|
||||
#### Scenario: json 解析失败
|
||||
- **WHEN** HTTP target 配置了 json body 规则但响应体不是合法 JSON
|
||||
- **THEN** 系统 SHALL 判定 matched 为 false
|
||||
#### Scenario: json JSONPath 存在性匹配
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{json: {path: "$.status"}}]`，且响应 JSON 中存在 `$.status`
|
||||
- **THEN** 系统 SHALL 将该规则按 `exists: true` 语义判定通过
|
||||
|
||||
#### Scenario: css 选择器匹配
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{css: {selector: "div#health", equals: "OK"}}]`，且 HTML 中存在 `div#health` 元素文本为 `"OK"`
|
||||
@@ -43,20 +31,16 @@
|
||||
- **WHEN** HTTP target 配置 css 规则带 `attr: "content"` 用于提取属性，且属性值匹配期望
|
||||
- **THEN** 系统 SHALL 判定该 body 规则通过
|
||||
|
||||
#### Scenario: css 选择器无匹配元素
|
||||
- **WHEN** HTTP target 配置了 css 选择器但 HTML 中无匹配元素
|
||||
- **THEN** 系统 SHALL 判定 matched 为 false
|
||||
|
||||
#### Scenario: xpath 表达式匹配
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{xpath: {path: "/root/status/text()", equals: "ok"}}]`，且 XML 中 `/root/status` 节点文本为 `"ok"`
|
||||
- **THEN** 系统 SHALL 判定该 body 规则通过
|
||||
|
||||
#### Scenario: xpath 表达式无匹配节点
|
||||
- **WHEN** HTTP target 配置了 xpath 表达式但 XML 中无匹配节点
|
||||
#### Scenario: 提取器无匹配目标失败
|
||||
- **WHEN** HTTP target 配置了 json、css 或 xpath 规则且对应路径、元素或节点不存在，并且规则未配置 `exists: false`
|
||||
- **THEN** 系统 SHALL 判定 matched 为 false
|
||||
|
||||
### Requirement: 多种 body 校验方法 AND 组合
|
||||
系统 SHALL 支持在 `expect.body` 数组中同时配置多种 body 校验方法，所有方法均通过时 matched 方为 true。
|
||||
系统 SHALL 支持在 `expect.body` 数组中同时配置多条内容规则，所有规则均通过时 matched 方为 true。系统 SHALL 按数组顺序执行规则，任一规则失败后 MUST NOT 继续执行后续规则。
|
||||
|
||||
#### Scenario: 多种方法全部通过
|
||||
- **WHEN** HTTP target 的 `expect.body` 数组依次配置 contains、json、regex，且全部通过
|
||||
@@ -66,54 +50,50 @@
|
||||
- **WHEN** HTTP target 的 `expect.body` 数组第一条 contains 不通过，后续还有 json 规则
|
||||
- **THEN** 系统 SHALL 判定 matched 为 false，且不再检查后续 json 规则
|
||||
|
||||
#### Scenario: 直接 matcher 多字段组合
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{contains: "ok", regex: "status"}]`，且响应体同时满足 contains 和 regex
|
||||
- **THEN** 系统 SHALL 判定该规则通过
|
||||
|
||||
### Requirement: 操作符系统
|
||||
系统 SHALL 支持对提取值和文本值使用以下操作符进行比较：equals（默认等值）、contains（子串包含）、match（正则匹配）、empty（空值判断）、exists（存在性判断）、gte/lte/gt/lt（数值比较）。
|
||||
系统 SHALL 支持通过共享 `ValueMatcher` 对提取值和文本值进行比较：`equals`（深度等值）、`contains`（子串包含）、`regex`（正则匹配）、`empty`（空值判断）、`exists`（存在性判断）、`gte`/`lte`/`gt`/`lt`（数值比较）。系统 MUST NOT 支持旧 `match` 字段。
|
||||
|
||||
#### Scenario: 标量值隐式 equals
|
||||
- **WHEN** 配置的期望值为标量（字符串/数字/布尔/null），如 `equals: "ok"`
|
||||
- **THEN** 系统 SHALL 使用 equals 操作符，对实际值做严格相等比较
|
||||
#### Scenario: equals 匹配 JSON value
|
||||
- **WHEN** 配置 `{equals: {status: "ok"}}`，且实际值为相同 JSON object
|
||||
- **THEN** 系统 SHALL 使用深度相等判定通过
|
||||
|
||||
#### Scenario: 显式 contains 操作符
|
||||
- **WHEN** 配置 `{contains: "success"}`，且实际值包含 `"success"`
|
||||
#### Scenario: 显式 contains matcher
|
||||
- **WHEN** 配置 `{contains: "success"}`，且实际值字符串化后包含 `"success"`
|
||||
- **THEN** 系统 SHALL 判定该规则通过
|
||||
|
||||
#### Scenario: 显式 match 操作符
|
||||
- **WHEN** 配置 `{match: '\\d+\\.\\d+\\.\\d+'}`，且实际值匹配该正则
|
||||
#### Scenario: 显式 regex matcher
|
||||
- **WHEN** 配置 `{regex: '\\d+\\.\\d+\\.\\d+'}`，且实际值字符串化后匹配该正则
|
||||
- **THEN** 系统 SHALL 判定该规则通过
|
||||
|
||||
#### Scenario: empty 操作符判断为空
|
||||
#### Scenario: empty matcher 判断为空
|
||||
- **WHEN** 配置 `{empty: true}`，且实际值为空数组 `[]`
|
||||
- **THEN** 系统 SHALL 判定该规则通过
|
||||
|
||||
#### Scenario: empty 操作符判断非空
|
||||
- **WHEN** 配置 `{empty: false}`，且实际值为 `[1, 2]`
|
||||
#### Scenario: exists matcher 判断不存在
|
||||
- **WHEN** 配置 `{exists: false}`，且实际值为 `undefined`
|
||||
- **THEN** 系统 SHALL 判定该规则通过
|
||||
|
||||
#### Scenario: exists 操作符判断存在
|
||||
- **WHEN** 配置 `{exists: false}`，且实际值不存在
|
||||
- **THEN** 系统 SHALL 判定该规则通过
|
||||
|
||||
#### Scenario: gte 数值比较
|
||||
- **WHEN** 配置 `{gte: 10}`，且实际值为 `15`（数字）
|
||||
- **THEN** 系统 SHALL 判定该规则通过
|
||||
|
||||
#### Scenario: gt/lt 数值比较
|
||||
#### Scenario: 数值比较 matcher
|
||||
- **WHEN** 配置 `{gt: 0, lt: 1000}`，且实际值为 `500`
|
||||
- **THEN** 系统 SHALL 对同一字段进行多操作符复合比较，全部通过则该规则通过
|
||||
- **THEN** 系统 SHALL 对同一字段进行多 matcher 复合比较，全部通过则该规则通过
|
||||
|
||||
### Requirement: 响应头校验
|
||||
系统 SHALL 支持通过 `expect.headers` 配置对 HTTP 响应头进行键值规则校验，header 名称匹配 MUST 不区分大小写。
|
||||
系统 SHALL 支持通过共享 `KeyValueExpect` 配置 `expect.headers` 对 HTTP 响应头进行键值规则校验，header 名称匹配 MUST 不区分大小写。header 期望值 MAY 为字符串字面量或 `ValueMatcher`。字符串字面量 SHALL 等价于 `{equals: <value>}`。
|
||||
|
||||
#### Scenario: 响应头匹配
|
||||
- **WHEN** HTTP target 配置 `expect.headers: {"Content-Type": {contains: "application/json"}}`，且响应包含该 header 且值匹配
|
||||
#### Scenario: 响应头字面量匹配
|
||||
- **WHEN** HTTP target 配置 `expect.headers: {"Content-Type": "application/json"}`，且响应包含该 header 且值精确匹配
|
||||
- **THEN** 系统 SHALL 判定 headers 阶段通过
|
||||
|
||||
#### Scenario: 响应头不匹配
|
||||
- **WHEN** HTTP target 配置 `expect.headers: {"Content-Type": {equals: "application/json"}}`，且响应 header 值为 `"text/html"`
|
||||
- **THEN** 系统 SHALL 判定 matched 为 false
|
||||
#### Scenario: 响应头 matcher 匹配
|
||||
- **WHEN** HTTP target 配置 `expect.headers: {"Content-Type": {contains: "application/json"}}`，且响应 header 值包含该文本
|
||||
- **THEN** 系统 SHALL 判定 headers 阶段通过
|
||||
|
||||
#### Scenario: 响应头缺失
|
||||
- **WHEN** HTTP target 配置了某个 header 但响应中不存在该 header
|
||||
- **WHEN** HTTP target 配置了某个 header 但响应中不存在该 header，且未配置 `exists: false`
|
||||
- **THEN** 系统 SHALL 判定 matched 为 false
|
||||
|
||||
### Requirement: 结构化 expect 失败信息
|
||||
@@ -175,79 +155,43 @@
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败
|
||||
|
||||
### Requirement: HTTP expect 规则启动期校验
|
||||
系统 SHALL 在启动期校验 HTTP expect 中已支持字段的类型、格式、未知字段和可编译表达式。HTTP expect、body rule、json/css/xpath rule 和 operator 对象中的未知字段 SHALL 导致启动期配置失败。每个 body rule 对象 MUST 恰好包含 contains、regex、json、css、xpath 中的一种规则类型。纯 operator 对象 MUST 至少包含一个已知 operator；body 提取规则可以不配置 operator，并以路径、元素或节点存在作为通过语义。`equals` operator SHALL 支持任意 JSON value，包括数组和对象。系统 SHALL 在启动期对 regex body 规则和 match operator 的正则表达式进行 ReDoS 安全检测，含有嵌套量词等危险模式的正则 SHALL 导致启动期配置失败。
|
||||
系统 SHALL 在启动期校验 HTTP expect 中已支持字段的类型、格式、未知字段和可编译表达式。HTTP expect SHALL 只允许 `status`、`headers`、`body` 和 `durationMs` 字段。`expect.body` MUST 为 `ContentRules` 数组。直接 `ValueMatcher` 对象 MUST 至少包含一个合法 matcher。Extractor 规则 MUST 只包含 `json`、`css`、`xpath` 中的一种 extractor。Extractor 内部可以不配置 matcher，并 SHALL 在运行期以存在性作为通过语义。`equals` matcher SHALL 支持任意 JSON value，包括数组和对象。系统 SHALL 在启动期对所有 `regex` 执行静态 ReDoS 检测。
|
||||
|
||||
#### Scenario: body rule 使用 regex 字段
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{regex: "ok|healthy"}]` 且 regex 可编译且无 ReDoS 风险
|
||||
- **THEN** 系统 SHALL 接受该配置，并在运行期按 regex body 规则匹配响应体
|
||||
- **THEN** 系统 SHALL 接受该配置，并在运行期按 regex 规则匹配响应体
|
||||
|
||||
#### Scenario: body rule 不支持 match 字段
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{match: "ok"}]` 且该规则没有 contains、regex、json、css、xpath 任一支持字段
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{match: "ok"}]`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示 `match` 是未知字段或不支持字段
|
||||
|
||||
#### Scenario: body rule 多 extractor 非法
|
||||
- **WHEN** HTTP target 的同一条 body rule 同时配置 `json` 和 `css`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败
|
||||
|
||||
#### Scenario: body rule 忽略未知字段 → body rule 未知字段启动失败
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{contains: "ok", note: "ignored"}]`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示 `note` 是未知字段
|
||||
|
||||
#### Scenario: body rule 多支持字段非法
|
||||
- **WHEN** HTTP target 的同一条 body rule 同时配置 contains 和 regex
|
||||
#### Scenario: matcher regex 正则非法
|
||||
- **WHEN** HTTP target 的 expect.headers、body 直接 matcher 或 extractor 内部 matcher 配置了不可编译的 regex
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败
|
||||
|
||||
#### Scenario: operator match 正则非法
|
||||
- **WHEN** HTTP target 的 expect.headers、json、css 或 xpath operator 配置了不可编译的 match 正则
|
||||
#### Scenario: matcher 数值比较类型非法
|
||||
- **WHEN** HTTP target 的 matcher 配置 gt、gte、lt 或 lte，且对应值不是有限数字
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败
|
||||
|
||||
#### Scenario: operator 数值比较类型非法
|
||||
- **WHEN** HTTP target 的 expect operator 配置 gt、gte、lt 或 lte，且对应值不是有限数字
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败
|
||||
|
||||
#### Scenario: operator 布尔类型非法
|
||||
- **WHEN** HTTP target 的 expect operator 配置 empty 或 exists，且对应值不是布尔值
|
||||
#### Scenario: matcher 布尔类型非法
|
||||
- **WHEN** HTTP target 的 matcher 配置 empty 或 exists，且对应值不是布尔值
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败
|
||||
|
||||
#### Scenario: JSONPath 子集非法
|
||||
- **WHEN** HTTP target 的 json body rule path 不符合系统支持的 JSONPath 子集
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败
|
||||
|
||||
#### Scenario: operator 未知字段非法
|
||||
- **WHEN** HTTP target 的 expect operator 配置了 `foo: "bar"` 等未知 operator 字段
|
||||
#### Scenario: matcher 未知字段非法
|
||||
- **WHEN** HTTP target 的 matcher 配置了 `foo: "bar"` 等未知字段
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败
|
||||
|
||||
#### Scenario: equals 支持对象
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{json: {path: "$.payload", equals: {status: "ok"}}}]`
|
||||
- **THEN** 系统 SHALL 接受该配置，并在运行期使用深度相等比较提取值和对象期望
|
||||
|
||||
#### Scenario: equals 支持数组
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{json: {path: "$.items", equals: ["a", "b"]}}]`
|
||||
- **THEN** 系统 SHALL 接受该配置，并在运行期使用深度相等比较提取值和数组期望
|
||||
|
||||
#### Scenario: 纯 operator 对象不能为空
|
||||
- **WHEN** HTTP target 的 `expect.headers` 中某个 header 期望配置为空对象 `{}`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，要求显式配置至少一个 operator
|
||||
|
||||
#### Scenario: json rule 允许存在性语义
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{json: {path: "$.status"}}]`
|
||||
- **THEN** 系统 SHALL 接受该配置，并在运行期以 JSONPath 值存在作为通过语义
|
||||
|
||||
#### Scenario: css rule 未知字段非法
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{css: {selector: "h1", unknown: true}}]`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示未知字段
|
||||
|
||||
#### Scenario: xpath rule 未知字段非法
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{xpath: {path: "/html/body", unknown: true}}]`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示未知字段
|
||||
|
||||
#### Scenario: regex body 规则含嵌套量词启动失败
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{regex: "(a+)+$"}]`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示正则存在 ReDoS 风险
|
||||
|
||||
#### Scenario: match operator 含嵌套量词启动失败
|
||||
- **WHEN** HTTP target 的 expect operator 配置 `{match: "(\\d+)*x"}`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示正则存在 ReDoS 风险
|
||||
|
||||
#### Scenario: 安全正则通过校验
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{regex: "\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}"}]`
|
||||
- **THEN** 系统 SHALL 接受该配置（无嵌套量词，无 ReDoS 风险）
|
||||
#### Scenario: durationMs matcher 非法
|
||||
- **WHEN** HTTP target 配置 `expect.durationMs` 不是合法 `ValueMatcher` 或其中数值 matcher 不是有限数字
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败
|
||||
|
||||
### Requirement: HTTP body 运行期失败结构化
|
||||
系统 SHALL 将 HTTP body 运行期失败记录为结构化 CheckFailure，并保留与具体规则相关的 phase 和 path。响应内容不符合配置 SHALL 记录为 mismatch；响应内容无法按配置解析或解码 SHALL 记录为 error。
|
||||
|
||||
193
openspec/specs/expect-rule-system/spec.md
Normal file
193
openspec/specs/expect-rule-system/spec.md
Normal file
@@ -0,0 +1,193 @@
|
||||
## Purpose
|
||||
|
||||
定义共享 expect 断言规则系统的核心概念和基础设施：ValueMatcher 统一匹配器、ContentRules 内容规则数组、KeyValueExpect 键值规则、以及相关的启动期校验和失败路径规范。
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: ValueMatcher 统一匹配器
|
||||
系统 SHALL 提供共享 `ValueMatcher` 作为所有非状态类 expect 的基础匹配结构。`ValueMatcher` SHALL 支持 `equals`、`contains`、`regex`、`exists`、`empty`、`gt`、`gte`、`lt` 和 `lte` 字段。`equals` MUST 支持任意 JSON value，并使用深度相等比较。`contains` 和 `regex` SHALL 将实际值转换为字符串后匹配。`gt`、`gte`、`lt` 和 `lte` SHALL 将实际值转换为有限数字后比较，无法转换为有限数字时 SHALL 判定不匹配。一个 `ValueMatcher` 对象包含多个 matcher 字段时，系统 SHALL 要求全部 matcher 均通过。
|
||||
|
||||
#### Scenario: equals 匹配对象
|
||||
- **WHEN** 实际值为 `{status: "ok", count: 1}` 且 matcher 为 `{equals: {status: "ok", count: 1}}`
|
||||
- **THEN** 系统 SHALL 使用深度相等判定该 matcher 通过
|
||||
|
||||
#### Scenario: contains 字符串化匹配
|
||||
- **WHEN** 实际值为 `"service ready"` 且 matcher 为 `{contains: "ready"}`
|
||||
- **THEN** 系统 SHALL 判定该 matcher 通过
|
||||
|
||||
#### Scenario: 数字范围组合匹配
|
||||
- **WHEN** 实际值为 `50` 且 matcher 为 `{gte: 0, lte: 100}`
|
||||
- **THEN** 系统 SHALL 判定该 matcher 通过
|
||||
|
||||
#### Scenario: 多 matcher 快速失败
|
||||
- **WHEN** 实际值为 `"healthy"` 且 matcher 为 `{contains: "health", regex: "^ready$"}`
|
||||
- **THEN** 系统 SHALL 在任一 matcher 不满足时判定该 matcher 对象不通过
|
||||
|
||||
### Requirement: ValueMatcher 启动期校验
|
||||
系统 SHALL 在启动期对所有 `ValueMatcher` 对象执行严格的类型和语义校验。`contains` MUST 为 string。`equals` MAY 为任意 JSON value。`exists` 和 `empty` MUST 为 boolean。`gt`、`gte`、`lt` 和 `lte` MUST 为有限数字（`Number.isFinite`）。`regex` MUST 为可编译的 string pattern，并通过 ReDoS 风险校验。ValueMatcher 对象 MUST 至少包含一个合法 matcher 字段，空对象 `{}` SHALL 导致启动期配置错误。ValueMatcher 对象 MUST NOT 包含未知字段，任何不属于 `equals`、`contains`、`regex`、`exists`、`empty`、`gt`、`gte`、`lt`、`lte` 的字段 SHALL 导致启动期配置错误。
|
||||
|
||||
#### Scenario: 空 matcher 对象被拒绝
|
||||
- **WHEN** YAML 配置中任一 matcher 对象为空 `{}`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示 matcher 必须包含至少一个合法字段
|
||||
|
||||
#### Scenario: 未知 matcher 字段被拒绝
|
||||
- **WHEN** YAML 配置中任一 matcher 对象包含 `foo: "bar"` 等未知字段
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段未知
|
||||
|
||||
#### Scenario: 数值 matcher 非有限数字被拒绝
|
||||
- **WHEN** YAML 配置中任一 matcher 的 `gt`、`gte`、`lt` 或 `lte` 值为 `NaN`、`Infinity` 或非数字类型
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示数值 matcher 必须为有限数字
|
||||
|
||||
#### Scenario: 布尔 matcher 非布尔值被拒绝
|
||||
- **WHEN** YAML 配置中任一 matcher 的 `exists` 或 `empty` 值不是布尔类型
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段必须为布尔值
|
||||
|
||||
### Requirement: empty matcher 语义
|
||||
`empty: true` SHALL 在以下情况判定通过：实际值为 `null`、`undefined`、空字符串 `""`、空数组 `[]` 或空对象 `{}`。`empty: false` SHALL 在以上条件均不满足时判定通过。数字 `0` 和布尔 `false` SHALL NOT 被视为 empty。
|
||||
|
||||
#### Scenario: null 视为 empty
|
||||
- **WHEN** 实际值为 `null` 且 matcher 为 `{empty: true}`
|
||||
- **THEN** 系统 SHALL 判定该 matcher 通过
|
||||
|
||||
#### Scenario: 空字符串视为 empty
|
||||
- **WHEN** 实际值为 `""` 且 matcher 为 `{empty: true}`
|
||||
- **THEN** 系统 SHALL 判定该 matcher 通过
|
||||
|
||||
#### Scenario: 空数组视为 empty
|
||||
- **WHEN** 实际值为 `[]` 且 matcher 为 `{empty: true}`
|
||||
- **THEN** 系统 SHALL 判定该 matcher 通过
|
||||
|
||||
#### Scenario: 空对象视为 empty
|
||||
- **WHEN** 实际值为 `{}` 且 matcher 为 `{empty: true}`
|
||||
- **THEN** 系统 SHALL 判定该 matcher 通过
|
||||
|
||||
#### Scenario: 数字 0 不视为 empty
|
||||
- **WHEN** 实际值为 `0` 且 matcher 为 `{empty: true}`
|
||||
- **THEN** 系统 SHALL 判定该 matcher 不通过
|
||||
|
||||
#### Scenario: 布尔 false 不视为 empty
|
||||
- **WHEN** 实际值为 `false` 且 matcher 为 `{empty: true}`
|
||||
- **THEN** 系统 SHALL 判定该 matcher 不通过
|
||||
|
||||
### Requirement: exists 与其他 matcher 的组合语义
|
||||
当 `ValueMatcher` 同时包含 `exists: false` 和其他非存在性 matcher（如 `contains`、`regex`、`equals` 等）时，系统 SHALL 在启动期配置校验失败，提示 `exists: false` 不能与其他 matcher 组合使用。`exists: true` MAY 与其他 matcher 组合，语义为先确认存在再执行其他 matcher。
|
||||
|
||||
#### Scenario: exists false 与 contains 组合被拒绝
|
||||
- **WHEN** YAML 配置中 matcher 为 `{exists: false, contains: "foo"}`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示 `exists: false` 不能与其他 matcher 组合
|
||||
|
||||
#### Scenario: exists true 与 contains 组合允许
|
||||
- **WHEN** 实际值为 `"hello foo"` 且 matcher 为 `{exists: true, contains: "foo"}`
|
||||
- **THEN** 系统 SHALL 判定该 matcher 通过
|
||||
|
||||
### Requirement: regex 字段语义
|
||||
系统 SHALL 使用 `regex` 作为唯一正则 matcher 字段。`regex` 值 MUST 为可编译的字符串 pattern。运行期 SHALL 固定使用无 flags 的 `new RegExp(pattern).test(String(actual))` 执行匹配。系统 MUST NOT 支持旧 `match` 字段。系统 SHALL 在启动期对所有 `regex` pattern 执行可编译校验和 ReDoS 风险校验。
|
||||
|
||||
#### Scenario: regex 任意位置匹配
|
||||
- **WHEN** 实际值为 `"api status ok"` 且 matcher 为 `{regex: "status"}`
|
||||
- **THEN** 系统 SHALL 判定该 matcher 通过，因为无 flags 的 JavaScript 正则仍会搜索整个字符串中的第一次匹配
|
||||
|
||||
#### Scenario: regex 完整匹配由用户声明锚点
|
||||
- **WHEN** 实际值为 `"OK\n"` 且 matcher 为 `{regex: "^OK$"}`
|
||||
- **THEN** 系统 SHALL 判定该 matcher 不通过，因为系统 MUST NOT 默认启用 multiline flags
|
||||
|
||||
#### Scenario: match 字段启动失败
|
||||
- **WHEN** YAML 配置中任一 matcher 对象包含 `match: "ok"`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示 `match` 是未知字段或不支持字段
|
||||
|
||||
#### Scenario: regex ReDoS 风险启动失败
|
||||
- **WHEN** YAML 配置中任一 `regex` 为 `"(a+)+$"`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示正则存在 ReDoS 风险
|
||||
|
||||
### Requirement: ContentRules 内容规则数组
|
||||
系统 SHALL 提供共享 `ContentRules` 表达返回内容断言。`ContentRules` MUST 为有序数组，数组项 SHALL 为直接 `ValueMatcher`，或 `json`、`css`、`xpath` 三类 extractor 规则之一。系统 SHALL 按数组顺序执行全部规则，任一规则失败时 SHALL 立即停止并返回该规则的 failure。系统 MUST NOT 支持内容字段的非数组对象快捷写法。
|
||||
|
||||
#### Scenario: 直接 matcher 内容规则
|
||||
- **WHEN** 内容字段配置 `[{contains: "ready"}, {regex: "listening on \\d+"}]` 且原始内容同时满足两条规则
|
||||
- **THEN** 系统 SHALL 判定该内容字段通过
|
||||
|
||||
#### Scenario: 内容规则数组快速失败
|
||||
- **WHEN** 内容字段配置三条规则且第二条规则失败
|
||||
- **THEN** 系统 SHALL 返回第二条规则的 failure，并 MUST NOT 执行第三条规则
|
||||
|
||||
#### Scenario: 内容字段必须为数组
|
||||
- **WHEN** YAML 中内容字段配置为 `{contains: "ok"}` 而不是数组
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该内容字段必须为规则数组
|
||||
|
||||
### Requirement: ContentRule 互斥性约束
|
||||
一条 `ContentRule` MUST 为直接 `ValueMatcher` 或恰好一个 extractor（`json`、`css`、`xpath` 之一）。系统 MUST NOT 允许同一条规则同时包含多个 extractor。直接 `ValueMatcher` 规则 MUST NOT 包含 `json`、`css`、`xpath` 字段。系统 SHALL 在启动期对违反互斥性的规则报错。
|
||||
|
||||
#### Scenario: 多 extractor 被拒绝
|
||||
- **WHEN** YAML 中内容规则为 `{json: {path: "$.a"}, css: {selector: "div"}}`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示一条规则不能同时包含多个 extractor
|
||||
|
||||
#### Scenario: 直接 matcher 混入 extractor 被拒绝
|
||||
- **WHEN** YAML 中内容规则为 `{contains: "ok", json: {path: "$.a"}}`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示直接 matcher 不能与 extractor 混用
|
||||
|
||||
### Requirement: 空 ContentRules 数组语义
|
||||
`ContentRules` 空数组 `[]` SHALL 被系统接受为合法配置。运行期空数组 SHALL 等价于无规则，即该内容字段的断言直接通过。
|
||||
|
||||
#### Scenario: 空 body 数组通过
|
||||
- **WHEN** HTTP target 配置 `expect.body: []` 且响应体为任意内容
|
||||
- **THEN** 系统 SHALL 判定 body 阶段通过
|
||||
|
||||
### Requirement: ContentRules 非字符串值序列化
|
||||
当 `ContentRules` 的观测源为非字符串值（如对象或数组）时，直接 `ValueMatcher` 的 `contains` 和 `regex` SHALL 先将值 JSON 序列化为字符串后匹配。`equals` SHALL 直接在原始结构化值上使用深度相等比较，不进行序列化。
|
||||
|
||||
#### Scenario: 对象序列化后 contains 匹配
|
||||
- **WHEN** ContentRules 观测源为 `{status: "ok"}` 且规则为 `{contains: "ok"}`
|
||||
- **THEN** 系统 SHALL 将对象 JSON 序列化后执行 contains 匹配
|
||||
|
||||
#### Scenario: 对象 equals 不序列化
|
||||
- **WHEN** ContentRules 观测源为 `{status: "ok"}` 且规则为 `{equals: {status: "ok"}}`
|
||||
- **THEN** 系统 SHALL 直接在结构化值上使用深度相等比较
|
||||
|
||||
### Requirement: ContentRules 提取器
|
||||
系统 SHALL 支持在 `ContentRules` 中使用 `json`、`css` 和 `xpath` extractor。`json.path` MUST 使用现有 JSONPath 子集。`css.selector` MUST 为非空字符串，并 MAY 配置 `attr` 提取属性值。`xpath.path` MUST 为非空字符串，并 SHALL 在启动期进行可编译校验。Extractor 内部 MAY 包含任意 `ValueMatcher` 字段。Extractor 规则未配置任何 matcher 时 SHALL 等价于 `exists: true`。
|
||||
|
||||
#### Scenario: json extractor 数字比较
|
||||
- **WHEN** 原始内容为 JSON 字符串 `{"count": 2}` 且规则为 `{json: {path: "$.count", gte: 1}}`
|
||||
- **THEN** 系统 SHALL 解析 JSON、提取 `$.count` 并判定该规则通过
|
||||
|
||||
#### Scenario: json extractor 存在性默认语义
|
||||
- **WHEN** 原始内容为 JSON 字符串 `{"user": {"id": null}}` 且规则为 `{json: {path: "$.user.id"}}`
|
||||
- **THEN** 系统 SHALL 将该规则视为 `{json: {path: "$.user.id", exists: true}}` 并判定通过
|
||||
|
||||
#### Scenario: css attr 存在性默认语义
|
||||
- **WHEN** 原始内容包含 `<meta name="status" content="ok">` 且规则为 `{css: {selector: "meta[name=status]", attr: "content"}}`
|
||||
- **THEN** 系统 SHALL 在属性存在时判定该规则通过
|
||||
|
||||
#### Scenario: xpath 无匹配节点失败
|
||||
- **WHEN** XML 内容中不存在 XPath 指向的节点，且规则为 `{xpath: {path: "/root/status"}}`
|
||||
- **THEN** 系统 SHALL 判定该规则不通过并生成 phase 对应内容字段的 mismatch failure
|
||||
|
||||
### Requirement: KeyValueExpect 键值规则
|
||||
系统 SHALL 提供共享 `KeyValueExpect` 表达键值型观测值断言。`KeyValueExpect` SHALL 为动态键对象，每个键对应的值 MAY 为 `ValueMatcher` 或 JSON 字面量。字面量值 SHALL 等价于 `{equals: <literal>}`。调用方 MAY 指定 key 规范化策略；HTTP 与 LLM headers MUST 使用大小写不敏感的 key 匹配。
|
||||
|
||||
#### Scenario: headers 字面量快捷写法
|
||||
- **WHEN** 响应 headers 中 `content-type` 为 `application/json`，且配置为 `headers: {Content-Type: "application/json"}`
|
||||
- **THEN** 系统 SHALL 按大小写不敏感 key 匹配并使用 equals 语义判定通过
|
||||
|
||||
#### Scenario: headers matcher 写法
|
||||
- **WHEN** 响应 headers 中 `content-type` 为 `application/json; charset=utf-8`，且配置为 `headers: {Content-Type: {contains: "application/json"}}`
|
||||
- **THEN** 系统 SHALL 判定该 header 规则通过
|
||||
|
||||
#### Scenario: 缺失键 exists false
|
||||
- **WHEN** 观测键值表中不存在 `x-debug`，且配置为 `{x-debug: {exists: false}}`
|
||||
- **THEN** 系统 SHALL 判定该键规则通过
|
||||
|
||||
### Requirement: 结构化失败路径
|
||||
系统 SHALL 在共享 matcher、content 和 key-value 断言失败时生成结构化 `CheckFailure`。failure SHALL 包含 `kind`、`phase`、`path`、`message`，并在 mismatch 场景包含 `expected` 和 `actual`。内容规则 failure path SHALL 包含数组下标，key-value failure path SHALL 包含键名，extractor failure path SHALL 包含 extractor 类型和 path/selector 信息。
|
||||
|
||||
#### Scenario: ContentRules 失败路径
|
||||
- **WHEN** `expect.body[1].json` 规则失败
|
||||
- **THEN** failure.path SHALL 指向 `body[1].json($.path)` 或等价可定位路径，failure.phase SHALL 为 `body`
|
||||
|
||||
#### Scenario: KeyValueExpect 失败路径
|
||||
- **WHEN** `expect.headers.Content-Type` 不匹配
|
||||
- **THEN** failure.path SHALL 指向 `headers.Content-Type`，failure.phase SHALL 为 `headers`
|
||||
|
||||
#### Scenario: actual 截断
|
||||
- **WHEN** matcher 失败时 actual 字符串长度超过 200 字符
|
||||
- **THEN** 系统 SHALL 使用现有截断策略保存 failure.actual，避免历史记录写入过长内容
|
||||
@@ -106,7 +106,7 @@
|
||||
- **THEN** 系统 SHALL 记录 `matched=false`，failure 的 kind 为 `error`，phase 为 `ping`，path 为 `parse`，message 包含 "无法解析 ping 输出"
|
||||
|
||||
### Requirement: ping expect 校验
|
||||
系统 SHALL 支持 ping 专属 expect，包括 `alive`、`maxPacketLoss`、`maxAvgLatencyMs`、`maxMaxLatencyMs` 和 `maxDurationMs`，并按 alive、packetLoss、avgLatency、maxLatency、duration 的阶段顺序快速失败。
|
||||
系统 SHALL 支持 ping 专属 expect，包括 `alive`、`packetLossPercent`、`avgLatencyMs`、`maxLatencyMs` 和 `durationMs`，并按 alive、packetLossPercent、avgLatencyMs、maxLatencyMs、durationMs 的阶段顺序快速失败。`alive` SHALL 保持布尔状态语义，未配置时默认 `true`。`packetLossPercent` SHALL 表示 0 到 100 的丢包率百分比，并使用共享 `ValueMatcher`。`avgLatencyMs`、`maxLatencyMs` 和 `durationMs` SHALL 使用共享 `ValueMatcher`。
|
||||
|
||||
#### Scenario: 默认 alive 成功语义
|
||||
- **WHEN** ping target 未显式配置 `expect.alive`
|
||||
@@ -124,57 +124,53 @@
|
||||
- **WHEN** ping target 配置 `expect.alive: false`，且目标主机不可达
|
||||
- **THEN** 系统 SHALL 判定 alive 阶段通过（`matched=true`）
|
||||
|
||||
#### Scenario: 反向 alive 断言失败
|
||||
- **WHEN** ping target 配置 `expect.alive: false`，但目标主机可达
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `alive`
|
||||
#### Scenario: packetLossPercent 校验通过
|
||||
- **WHEN** ping target 配置 `expect.packetLossPercent: {lte: 10}`，且实际丢包率为 0%
|
||||
- **THEN** 系统 SHALL 判定 packetLossPercent 阶段通过
|
||||
|
||||
#### Scenario: maxPacketLoss 校验通过
|
||||
- **WHEN** ping target 配置 `expect.maxPacketLoss: 10`，且实际丢包率为 0%
|
||||
- **THEN** 系统 SHALL 判定 packetLoss 阶段通过
|
||||
|
||||
#### Scenario: maxPacketLoss 校验失败
|
||||
- **WHEN** ping target 配置 `expect.maxPacketLoss: 10`，且实际丢包率为 33%
|
||||
#### Scenario: packetLossPercent 校验失败
|
||||
- **WHEN** ping target 配置 `expect.packetLossPercent: {lte: 10}`，且实际丢包率为 33%
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `packetLoss`
|
||||
|
||||
#### Scenario: maxAvgLatencyMs 校验通过
|
||||
- **WHEN** ping target 配置 `expect.maxAvgLatencyMs: 200`，且实际平均延迟为 12ms
|
||||
#### Scenario: avgLatencyMs 校验通过
|
||||
- **WHEN** ping target 配置 `expect.avgLatencyMs: {lte: 200}`，且实际平均延迟为 12ms
|
||||
- **THEN** 系统 SHALL 判定 avgLatency 阶段通过
|
||||
|
||||
#### Scenario: maxAvgLatencyMs 校验失败
|
||||
- **WHEN** ping target 配置 `expect.maxAvgLatencyMs: 100`，且实际平均延迟为 156ms
|
||||
#### Scenario: avgLatencyMs 校验失败
|
||||
- **WHEN** ping target 配置 `expect.avgLatencyMs: {lte: 100}`，且实际平均延迟为 156ms
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `avgLatency`
|
||||
|
||||
#### Scenario: maxMaxLatencyMs 校验通过
|
||||
- **WHEN** ping target 配置 `expect.maxMaxLatencyMs: 500`，且实际最大延迟为 340ms
|
||||
#### Scenario: maxLatencyMs 校验通过
|
||||
- **WHEN** ping target 配置 `expect.maxLatencyMs: {lte: 500}`，且实际最大延迟为 340ms
|
||||
- **THEN** 系统 SHALL 判定 maxLatency 阶段通过
|
||||
|
||||
#### Scenario: maxMaxLatencyMs 校验失败
|
||||
- **WHEN** ping target 配置 `expect.maxMaxLatencyMs: 200`，且实际最大延迟为 340ms
|
||||
#### Scenario: maxLatencyMs 校验失败
|
||||
- **WHEN** ping target 配置 `expect.maxLatencyMs: {lte: 200}`，且实际最大延迟为 340ms
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `maxLatency`
|
||||
|
||||
#### Scenario: maxDurationMs 校验
|
||||
- **WHEN** ping target 配置 `expect.maxDurationMs: 5000`，且完整执行耗时超过 5000ms
|
||||
#### Scenario: durationMs 校验
|
||||
- **WHEN** ping target 配置 `expect.durationMs: {lte: 5000}`，且完整执行耗时超过 5000ms
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `duration`
|
||||
|
||||
#### Scenario: alive=false 时跳过延迟断言
|
||||
- **WHEN** ping target 配置 `expect.alive: true` 和 `expect.maxAvgLatencyMs: 100`，且目标不可达
|
||||
- **WHEN** ping target 配置 `expect.alive: true` 和 `expect.avgLatencyMs: {lte: 100}`，且目标不可达
|
||||
- **THEN** 系统 SHALL 在 alive 阶段即返回失败，不执行后续延迟断言
|
||||
|
||||
#### Scenario: ping expect 未知字段失败
|
||||
- **WHEN** YAML 中 ping target 的 expect 包含 `status: [200]` 或其他非 ping expect 字段
|
||||
- **WHEN** YAML 中 ping target 的 expect 包含 `status: [200]`、`maxPacketLoss`、`maxAvgLatencyMs`、`maxMaxLatencyMs`、`maxDurationMs` 或其他非 ping expect 字段
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段
|
||||
|
||||
#### Scenario: maxPacketLoss 类型非法
|
||||
- **WHEN** YAML 中 ping target 的 `expect.maxPacketLoss` 不是 0 到 100 之间的数字
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.maxPacketLoss 必须为 0-100 的数字
|
||||
#### Scenario: packetLossPercent 类型非法
|
||||
- **WHEN** YAML 中 ping target 的 `expect.packetLossPercent` 不是合法 `ValueMatcher`，或其数值范围无法用于 0 到 100 的百分比断言
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.packetLossPercent 格式错误
|
||||
|
||||
#### Scenario: maxAvgLatencyMs 类型非法
|
||||
- **WHEN** YAML 中 ping target 的 `expect.maxAvgLatencyMs` 不是非负有限数字
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.maxAvgLatencyMs 格式错误
|
||||
#### Scenario: avgLatencyMs 类型非法
|
||||
- **WHEN** YAML 中 ping target 的 `expect.avgLatencyMs` 不是合法 `ValueMatcher`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.avgLatencyMs 格式错误
|
||||
|
||||
#### Scenario: maxMaxLatencyMs 类型非法
|
||||
- **WHEN** YAML 中 ping target 的 `expect.maxMaxLatencyMs` 不是非负有限数字
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.maxMaxLatencyMs 格式错误
|
||||
#### Scenario: maxLatencyMs 类型非法
|
||||
- **WHEN** YAML 中 ping target 的 `expect.maxLatencyMs` 不是合法 `ValueMatcher`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.maxLatencyMs 格式错误
|
||||
|
||||
### Requirement: ping statusDetail 摘要
|
||||
系统 SHALL 在 ping 执行成功后生成结构化 statusDetail 摘要，展示关键指标。
|
||||
|
||||
@@ -108,7 +108,7 @@ LLM checker SHALL 在 SDK 调用结果和 expect 断言之间构建 `LlmCheckObs
|
||||
- **THEN** LLM checker SHALL 返回 `phase: "request"` 的 error failure
|
||||
|
||||
### Requirement: LLM Expect 断言
|
||||
LLM checker SHALL 支持 `expect.status`、`expect.headers`、`expect.output`、`expect.finishReason`、`expect.rawFinishReason`、`expect.usage.inputTokens`、`expect.usage.outputTokens`、`expect.usage.totalTokens`、`expect.stream.completed`、`expect.stream.firstTokenMs` 和 `expect.maxDurationMs`。`expect.status` 和 `expect.headers` 的运行期断言 SHALL 复用 `src/server/checker/runner/http/expect.ts` 中的 `checkStatus` 和 `checkHeaders` 函数。LLM checker MUST 按固定顺序快速失败，非流式顺序为 status、headers、output、finishReason、rawFinishReason、usage、duration；流式顺序为 status、headers、stream.completed、stream.firstTokenMs、output、finishReason、rawFinishReason、usage、duration。
|
||||
LLM checker SHALL 支持 `expect.status`、`expect.headers`、`expect.output`、`expect.finishReason`、`expect.rawFinishReason`、`expect.usage.inputTokens`、`expect.usage.outputTokens`、`expect.usage.totalTokens`、`expect.stream.completed`、`expect.stream.firstTokenMs` 和 `expect.durationMs`。`expect.status` SHALL 保持 HTTP 状态码数组语义并复用 HTTP status 断言。`expect.headers` SHALL 使用共享 `KeyValueExpect` 且 header key 大小写不敏感。`expect.output` MUST 使用共享 `ContentRules`。`expect.finishReason` 和 `expect.rawFinishReason` SHALL 使用共享 `ValueMatcher`。`expect.usage.*`、`expect.stream.firstTokenMs` 和 `expect.durationMs` SHALL 使用共享 `ValueMatcher`。LLM checker MUST 按固定顺序快速失败，非流式顺序为 status、headers、output、finishReason、rawFinishReason、usage、durationMs；流式顺序为 status、headers、stream.completed、stream.firstTokenMs、output、finishReason、rawFinishReason、usage、durationMs。
|
||||
|
||||
#### Scenario: 默认 status 断言
|
||||
- **WHEN** LLM target 未配置 `expect.status`
|
||||
@@ -118,13 +118,25 @@ LLM checker SHALL 支持 `expect.status`、`expect.headers`、`expect.output`、
|
||||
- **WHEN** observing fetch 捕获的响应 headers 满足 `expect.headers` 配置
|
||||
- **THEN** LLM checker SHALL 判定 headers 断言通过
|
||||
|
||||
#### Scenario: expect headers 不匹配
|
||||
- **WHEN** observing fetch 捕获的响应 headers 不满足 `expect.headers` 中的某项配置
|
||||
- **THEN** LLM checker SHALL 返回 `phase: "headers"` 的 mismatch failure
|
||||
#### Scenario: output ContentRules 通过
|
||||
- **WHEN** LLM 输出文本满足 `expect.output` 中配置的全部 ContentRules
|
||||
- **THEN** LLM checker SHALL 判定 output 阶段通过
|
||||
|
||||
#### Scenario: 全部 expect 通过
|
||||
- **WHEN** LLM checker 构建出的 observation 满足所有已配置 expect
|
||||
- **THEN** 检查结果 SHALL 为 `matched=true` 且 `failure=null`
|
||||
#### Scenario: finishReason ValueMatcher 通过
|
||||
- **WHEN** observation.finishReason 为 `stop` 且 target 配置 `expect.finishReason: {equals: "stop"}`
|
||||
- **THEN** LLM checker SHALL 判定 finishReason 阶段通过
|
||||
|
||||
#### Scenario: rawFinishReason regex 通过
|
||||
- **WHEN** observation.rawFinishReason 为 `end_turn` 且 target 配置 `expect.rawFinishReason: {regex: "^(stop|end_turn)$"}`
|
||||
- **THEN** LLM checker SHALL 判定 rawFinishReason 阶段通过
|
||||
|
||||
#### Scenario: usage matcher 通过
|
||||
- **WHEN** observation.usage.totalTokens 为 14 且 target 配置 `expect.usage.totalTokens: {lte: 20}`
|
||||
- **THEN** LLM checker SHALL 判定 usage 阶段通过
|
||||
|
||||
#### Scenario: durationMs matcher 失败
|
||||
- **WHEN** LLM target 配置 `expect.durationMs: {lte: 1000}` 且实际执行耗时为 1500ms
|
||||
- **THEN** LLM checker SHALL 返回 phase=`duration` 的 mismatch failure
|
||||
|
||||
#### Scenario: 首个 expect 失败
|
||||
- **WHEN** 多个 LLM expect 中某个较早顺序的断言失败
|
||||
@@ -139,7 +151,7 @@ LLM checker SHALL 支持 `expect.status`、`expect.headers`、`expect.output`、
|
||||
- **THEN** LLM checker SHALL 因 `outputText` 缺失返回 `phase: "output"` 的 mismatch failure
|
||||
|
||||
### Requirement: LLM Output 规则
|
||||
LLM checker SHALL 支持 `expect.output` 有序规则数组，每个规则 MUST 仅包含 `equals`、`contains`、`regex` 或 `json` 中的一种。`equals` SHALL 对原始输出字符串做严格相等比较。`contains` SHALL 判断原始输出是否包含子串。`regex` SHALL 对原始输出执行正则匹配。`json` SHALL 将原始输出解析为 JSON，并用现有 JSONPath 子集和 operator 校验提取值。
|
||||
LLM checker SHALL 使用共享 `ContentRules` 校验 `expect.output`。每个 output rule SHALL 为直接 `ValueMatcher`，或 `json`、`css`、`xpath` extractor 规则之一。直接 matcher SHALL 作用于原始输出字符串。`equals` SHALL 对原始输出字符串做严格相等比较。`contains` SHALL 判断原始输出是否包含子串。`regex` SHALL 对原始输出执行无 flags 正则匹配。`json` SHALL 将原始输出解析为 JSON，并用现有 JSONPath 子集和 `ValueMatcher` 校验提取值。`json.equals` SHALL 支持任意 JSON value。`css` 和 `xpath` 在 schema 层面可用，但 LLM 输出通常为纯文本或 JSON，实际场景中仅 `json` 提取器有意义。
|
||||
|
||||
#### Scenario: 原始输出严格相等
|
||||
- **WHEN** `outputText` 为 `"OK\n"` 且 target 配置 `expect.output: [{ equals: "OK" }]`
|
||||
@@ -150,19 +162,27 @@ LLM checker SHALL 支持 `expect.output` 有序规则数组，每个规则 MUST
|
||||
- **THEN** LLM checker SHALL 判定该 output contains 规则通过
|
||||
|
||||
#### Scenario: output regex 通过
|
||||
- **WHEN** `outputText` 匹配配置的合法正则
|
||||
- **WHEN** `outputText` 匹配配置的合法 regex
|
||||
- **THEN** LLM checker SHALL 判定该 output regex 规则通过
|
||||
|
||||
#### Scenario: output JSONPath 通过
|
||||
- **WHEN** `outputText` 是 JSON 字符串且 JSONPath 提取值满足 operator
|
||||
#### Scenario: output JSONPath 字符串 equals 通过
|
||||
- **WHEN** `outputText` 是 JSON 字符串且 JSONPath 提取字符串值满足 `equals`
|
||||
- **THEN** LLM checker SHALL 判定该 output json 规则通过
|
||||
|
||||
#### Scenario: output JSONPath 对象 equals 通过
|
||||
- **WHEN** `outputText` 是 JSON 字符串且 JSONPath 提取对象值满足 `equals`
|
||||
- **THEN** LLM checker SHALL 使用深度相等判定该 output json 规则通过
|
||||
|
||||
#### Scenario: output JSONPath 存在性默认语义
|
||||
- **WHEN** `outputText` 是 JSON 字符串且 target 配置 `expect.output: [{json: {path: "$.status"}}]`
|
||||
- **THEN** LLM checker SHALL 将该规则按 `exists: true` 语义执行
|
||||
|
||||
#### Scenario: output 规则按顺序快速失败
|
||||
- **WHEN** `expect.output` 包含多个规则且第一条规则失败
|
||||
- **THEN** LLM checker SHALL 返回第一条失败规则的 mismatch failure，不继续校验后续 output 规则
|
||||
|
||||
### Requirement: LLM Stream 断言
|
||||
LLM checker SHALL 仅允许 `mode: stream` 使用 `expect.stream`。`expect.stream.completed` 未配置时，LLM checker SHALL 在 stream observation 路径使用默认 `true` 语义。`expect.stream.firstTokenMs` SHALL 仅统计第一个非空 `text-delta` 事件耗时，不统计 reasoning、tool call 或 source 事件。
|
||||
LLM checker SHALL 仅允许 `mode: stream` 使用 `expect.stream`。`expect.stream.completed` 未配置时，LLM checker SHALL 在 stream observation 路径使用默认 `true` 语义。`expect.stream.firstTokenMs` SHALL 使用共享 `ValueMatcher`，并仅统计第一个非空 `text-delta` 事件耗时，不统计 reasoning、tool call 或 source 事件。
|
||||
|
||||
#### Scenario: stream completed 默认值
|
||||
- **WHEN** target 配置 `llm.mode: stream` 且未配置 `expect.stream.completed`
|
||||
@@ -173,7 +193,7 @@ LLM checker SHALL 仅允许 `mode: stream` 使用 `expect.stream`。`expect.stre
|
||||
- **THEN** LLM checker SHALL 返回 `phase: "stream"` 的 failure
|
||||
|
||||
#### Scenario: firstTokenMs 达标
|
||||
- **WHEN** target 配置 `expect.stream.firstTokenMs` 且首个非空 text delta 耗时满足 operator
|
||||
- **WHEN** target 配置 `expect.stream.firstTokenMs: {lte: 1000}` 且首个非空 text delta 耗时满足 matcher
|
||||
- **THEN** LLM checker SHALL 判定 firstTokenMs 断言通过
|
||||
|
||||
#### Scenario: firstTokenMs 缺失
|
||||
|
||||
@@ -193,9 +193,9 @@
|
||||
- **WHEN** YAML 中某个 HTTP target 的 `expect.status` 包含非整数或不在 100-599 范围内的数字
|
||||
- **THEN** 系统 SHALL 以错误退出，提示该 target 的 status 数字不合法
|
||||
|
||||
#### Scenario: maxDurationMs 非法
|
||||
- **WHEN** YAML 中某个 target 的 `expect.maxDurationMs` 不是非负有限数字
|
||||
- **THEN** 系统 SHALL 以错误退出，提示该 target 的 expect.maxDurationMs 格式错误
|
||||
#### Scenario: durationMs matcher 非法
|
||||
- **WHEN** YAML 中某个 target 的 `expect.durationMs` 不是合法 `ValueMatcher`
|
||||
- **THEN** 系统 SHALL 以错误退出，提示该 target 的 expect.durationMs 格式错误
|
||||
|
||||
#### Scenario: ping target 缺少 host
|
||||
- **WHEN** YAML 中某个 target 配置 `type: ping` 但缺少 `ping.host`
|
||||
@@ -237,13 +237,13 @@
|
||||
- **WHEN** YAML 中某个 HTTP target 的 body xpath 规则缺少 path，或 path 不是非空字符串，或可被现有 XPath 库静态判定为语法错误
|
||||
- **THEN** 系统 SHALL 以错误退出，提示该 body xpath path 不合法
|
||||
|
||||
#### Scenario: expect operator 类型非法
|
||||
- **WHEN** YAML 中某个 HTTP expect operator 的 match 不是可编译正则字符串，empty/exists 不是布尔值，或 gt/gte/lt/lte 不是有限数字
|
||||
- **THEN** 系统 SHALL 以错误退出，提示对应 operator 配置不合法
|
||||
#### Scenario: expect matcher 类型非法
|
||||
- **WHEN** YAML 中某个 expect matcher 的 regex 不是可编译正则字符串，empty/exists 不是布尔值，或 gt/gte/lt/lte 不是有限数字
|
||||
- **THEN** 系统 SHALL 以错误退出，提示对应 matcher 配置不合法
|
||||
|
||||
#### Scenario: expect operator 类型非法
|
||||
- **WHEN** YAML 中某个 expect operator 的 match 不是可编译正则字符串，empty/exists 不是布尔值，或 gt/gte/lt/lte 不是有限数字
|
||||
- **THEN** 系统 SHALL 以错误退出，提示对应 operator 配置不合法
|
||||
#### Scenario: expect match 字段不再支持
|
||||
- **WHEN** YAML 中某个 expect matcher 配置 `match` 字段
|
||||
- **THEN** 系统 SHALL 以错误退出，提示 `match` 是未知字段，请使用 `regex`
|
||||
|
||||
#### Scenario: unknown 字段失败
|
||||
- **WHEN** YAML 中任一结构化配置对象包含契约未声明的字段，且该对象不是明确允许动态键的对象
|
||||
@@ -301,18 +301,38 @@
|
||||
- **THEN** 系统 SHALL 调用 `Bun.YAML.parse()` 将内容解析为配置对象
|
||||
|
||||
### Requirement: expect 配置增强
|
||||
系统 SHALL 支持 typed target 的领域专用 expect 配置，包括 HTTP 的 `status`（支持精确数字和范围模式）、`headers`、`body`，cmd 的 `exitCode`、`stdout`、`stderr`，tcp 的 `connected`、`banner`，ping 的 `alive`、`maxPacketLoss`、`maxAvgLatencyMs`、`maxMaxLatencyMs`，udp 的 `responded`、`response`、`responseSize`、`sourceHost`、`sourcePort` 和 `maxDurationMs`，以及 llm 的 `status`、`headers`、`output`、`finishReason`、`rawFinishReason`、`usage`、`stream` 和 `maxDurationMs`。内容类 expect MUST 使用数组表达配置顺序。
|
||||
系统 SHALL 支持 typed target 的领域专用 expect 配置，并通过共享 `ValueMatcher`、`ContentRules` 和 `KeyValueExpect` 表达可复用断言能力。状态类字段 SHALL 保持枚举或布尔语义，包括 HTTP/LLM 的 `status`（支持精确数字和范围模式）、cmd 的 `exitCode`、tcp 的 `connected`、ping 的 `alive` 和 udp 的 `responded`。数字指标字段 SHALL 使用 `ValueMatcher`，包括通用 `durationMs`、db 的 `rowCount`、udp 的 `responseSize`/`sourceHost`/`sourcePort`、ping 的 `packetLossPercent`/`avgLatencyMs`/`maxLatencyMs`、llm 的 usage token 与 stream 首 token 耗时。内容类字段 MUST 使用 `ContentRules` 数组表达配置顺序，包括 HTTP `body`、cmd `stdout`/`stderr`、tcp `banner`、udp `response`、llm `output` 和 db `result`。LLM `finishReason` 和 `rawFinishReason` SHALL 使用 `ValueMatcher`（非 ContentRules），因为它们是单值字符串元数据。键值类字段 SHALL 使用 `KeyValueExpect`，包括 HTTP/LLM `headers` 和 db `rows` 中的列值断言（db `rows` 的类型为 `Array<KeyValueExpect>`，外层数组按行索引，内层每个元素为 KeyValueExpect）。
|
||||
|
||||
#### Scenario: 解析 HTTP expect 配置
|
||||
- **WHEN** YAML 配置文件中 HTTP target 的 expect 包含 status、headers、body 规则数组及内部方法
|
||||
- **WHEN** YAML 配置文件中 HTTP target 的 expect 包含 status、headers、body 规则数组和 durationMs matcher
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 HTTP target 的 expect 字段
|
||||
|
||||
#### Scenario: 解析 cmd expect 配置
|
||||
- **WHEN** YAML 配置文件中 cmd target 的 expect 包含 exitCode、stdout 和 stderr 规则数组
|
||||
- **WHEN** YAML 配置文件中 cmd target 的 expect 包含 exitCode、stdout、stderr 和 durationMs matcher
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 cmd target 的 expect 字段
|
||||
|
||||
#### Scenario: 解析 body 有序规则数组
|
||||
- **WHEN** YAML 中 HTTP target 配置 `expect.body` 为 contains、json、regex 三个数组项
|
||||
#### Scenario: 解析 db expect 配置
|
||||
- **WHEN** YAML 配置文件中 db target 的 expect 包含 durationMs、rowCount、rows 和 result
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 db target 的 expect 字段
|
||||
|
||||
#### Scenario: 解析 tcp expect 配置
|
||||
- **WHEN** YAML 配置文件中 tcp target 的 expect 包含 connected、banner 规则数组和 durationMs matcher
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 tcp target 的 expect 字段
|
||||
|
||||
#### Scenario: 解析 ping expect 配置
|
||||
- **WHEN** YAML 配置文件中 ping target 的 expect 包含 alive、packetLossPercent、avgLatencyMs、maxLatencyMs 和 durationMs matcher
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 ping target 的 expect 字段
|
||||
|
||||
#### Scenario: 解析 udp expect 配置
|
||||
- **WHEN** YAML 配置文件中 udp target 的 expect 包含 responded、response、responseSize、sourceHost、sourcePort 和 durationMs matcher
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 udp target 的 expect 字段
|
||||
|
||||
#### Scenario: 解析 llm expect 配置
|
||||
- **WHEN** YAML 配置文件中 llm target 的 expect 包含 status、headers、output、finishReason、rawFinishReason、usage、stream 和 durationMs matcher
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 llm target 的 expect 字段，并保留 output 内容规则数组顺序
|
||||
|
||||
#### Scenario: 解析有序 ContentRules 数组
|
||||
- **WHEN** YAML 中任一内容类 expect 配置 contains、json、regex 三个数组项
|
||||
- **THEN** 系统 SHALL 保留数组顺序，供执行阶段按配置顺序快速失败
|
||||
|
||||
#### Scenario: 不配置 HTTP status
|
||||
@@ -323,41 +343,37 @@
|
||||
- **WHEN** HTTP target 配置 `expect.status: ["2xx"]`
|
||||
- **THEN** 系统 SHALL 在执行 expect 时匹配所有 200-299 状态码
|
||||
|
||||
#### Scenario: 配置 HTTP status 混合模式
|
||||
- **WHEN** HTTP target 配置 `expect.status: ["2xx", 301]`
|
||||
- **THEN** 系统 SHALL 在执行 expect 时匹配所有 200-299 状态码或精确匹配 301
|
||||
|
||||
#### Scenario: 不配置 cmd exitCode
|
||||
- **WHEN** cmd target 未配置 `expect.exitCode`
|
||||
- **THEN** 系统 SHALL 在执行 expect 时使用默认 `exitCode: [0]` 语义
|
||||
|
||||
#### Scenario: 不配置 expect
|
||||
- **WHEN** target 未配置任何 expect 规则
|
||||
- **THEN** 系统 SHALL 正常处理，expect 字段为 undefined
|
||||
- **THEN** 系统 SHALL 正常处理，expect 字段为 undefined，并由各 checker 使用自身默认状态语义
|
||||
|
||||
#### Scenario: 解析 ping expect 配置
|
||||
- **WHEN** YAML 配置文件中 ping target 的 expect 包含 alive、maxPacketLoss、maxAvgLatencyMs、maxMaxLatencyMs 和 maxDurationMs
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 ping target 的 expect 字段
|
||||
#### Scenario: 旧 maxDurationMs 字段不再支持
|
||||
- **WHEN** YAML 中任一 target 配置 `expect.maxDurationMs`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段未知，并要求使用 `expect.durationMs`
|
||||
|
||||
#### Scenario: 不配置 ping expect
|
||||
- **WHEN** ping target 未配置任何 expect 规则
|
||||
- **THEN** 系统 SHALL 正常处理，expect 字段为 undefined，执行时使用默认 alive=true 语义
|
||||
#### Scenario: 旧 match 字段不再支持
|
||||
- **WHEN** YAML 中任一 matcher 或内容规则配置 `match`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段未知，并要求使用 `regex`
|
||||
|
||||
#### Scenario: 解析 udp expect 配置
|
||||
- **WHEN** YAML 配置文件中 udp target 的 expect 包含 responded、response、responseSize、sourceHost、sourcePort 和 maxDurationMs
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 udp target 的 expect 字段
|
||||
#### Scenario: durationMs matcher 配置
|
||||
- **WHEN** YAML 中任一 target 配置 `expect.durationMs: {lte: 1000}`
|
||||
- **THEN** 系统 SHALL 接受该配置并在运行期使用完整执行耗时进行 matcher 校验
|
||||
|
||||
#### Scenario: 不配置 udp expect
|
||||
- **WHEN** udp target 未配置任何 expect 规则
|
||||
- **THEN** 系统 SHALL 正常处理，expect 字段为 undefined，执行时使用默认 responded=true 语义
|
||||
#### Scenario: 动态 headers 字段允许
|
||||
- **WHEN** YAML 中 `http.headers`、`defaults.http.headers`、`llm.headers`、`defaults.llm.headers` 或 `expect.headers` 包含任意 header 名称，且对应值符合契约
|
||||
- **THEN** 系统 SHALL 接受这些动态 header 名称
|
||||
|
||||
#### Scenario: 解析 llm expect 配置
|
||||
- **WHEN** YAML 配置文件中 llm target 的 expect 包含 status、headers、output、finishReason、rawFinishReason、usage、stream 和 maxDurationMs
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 llm target 的 expect 字段，并保留 output 规则数组顺序
|
||||
#### Scenario: ContentRules 字段必须为数组
|
||||
- **WHEN** YAML 中任一内容类 expect 字段配置为非数组
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段必须为规则数组
|
||||
|
||||
#### Scenario: 不配置 llm expect
|
||||
- **WHEN** llm target 未配置任何 expect 规则
|
||||
- **THEN** 系统 SHALL 正常处理，expect 字段为 undefined，执行时使用默认 status=[200] 语义
|
||||
#### Scenario: regex 字段非法
|
||||
- **WHEN** YAML 中任一 `regex` 不是字符串、不是可编译正则或存在 ReDoS 风险
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示对应 regex 不合法
|
||||
|
||||
### Requirement: 数据保留配置字段
|
||||
配置 schema 的 `runtime` 段 SHALL 支持 `retention` 字段，类型为字符串，格式为 `<数字><单位>`（单位：`d` 天、`h` 小时、`m` 分钟），用于指定历史数据保留时长。
|
||||
@@ -444,7 +460,7 @@
|
||||
- **THEN** target schema SHALL 声明 `id` 的 minLength 为 1、maxLength 为 30，并声明 `name` 为可选字段，类型为 string 或 null，字符串的 minLength 为 1、maxLength 为 30
|
||||
|
||||
### Requirement: TCP 配置校验
|
||||
系统 SHALL 在启动期对 tcp checker 的配置契约和语义执行严格校验。Tcp target 的 `tcp` 分组 SHALL 只允许 `host`、`port`、`readBanner`、`bannerReadTimeout` 和 `maxBannerBytes` 字段；Tcp expect SHALL 只允许 `connected`、`maxDurationMs` 和 `banner` 字段。未知字段、非法类型、非法端口、非法 size 和不可编译正则 MUST 导致启动期配置错误。
|
||||
系统 SHALL 在启动期对 tcp checker 的配置契约和语义执行严格校验。Tcp target 的 `tcp` 分组 SHALL 只允许 `host`、`port`、`readBanner`、`bannerReadTimeout` 和 `maxBannerBytes` 字段；Tcp expect SHALL 只允许 `connected`、`durationMs` 和 `banner` 字段。`banner` MUST 为 `ContentRules` 数组。未知字段、非法类型、非法端口、非法 size 和不可编译正则 MUST 导致启动期配置错误。
|
||||
|
||||
#### Scenario: tcp host 类型非法
|
||||
- **WHEN** YAML 中 tcp target 的 `tcp.host` 不是非空字符串
|
||||
@@ -471,11 +487,11 @@
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.connected 必须为布尔值
|
||||
|
||||
#### Scenario: tcp expect banner 非法
|
||||
- **WHEN** YAML 中 tcp target 的 `expect.banner` 不是合法 operator 对象
|
||||
- **WHEN** YAML 中 tcp target 的 `expect.banner` 不是合法 ContentRules 数组
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.banner 格式错误
|
||||
|
||||
#### Scenario: tcp expect banner match 正则非法
|
||||
- **WHEN** YAML 中 tcp target 配置 `expect.banner: { match: "[invalid" }`
|
||||
#### Scenario: tcp expect banner regex 正则非法
|
||||
- **WHEN** YAML 中 tcp target 配置 `expect.banner: [{ regex: "[invalid" }]`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，而不是延迟到运行期抛错
|
||||
|
||||
#### Scenario: tcp 分组未知字段失败
|
||||
@@ -487,7 +503,7 @@
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 defaults.tcp 包含未知字段
|
||||
|
||||
### Requirement: LLM 配置校验
|
||||
系统 SHALL 在启动期对 llm checker 的配置契约和语义执行严格校验。LLM target 的 `llm` 分组 SHALL 只允许 `provider`、`url`、`model`、`prompt`、`mode`、`key`、`authToken`、`headers`、`ignoreSSL`、`options` 和 `providerOptions` 字段。`defaults.llm` 分组 SHALL 只允许 `mode`、`headers`、`ignoreSSL`、`options` 和 `providerOptions` 字段。LLM expect SHALL 只允许 `status`、`headers`、`output`、`finishReason`、`rawFinishReason`、`usage`、`stream` 和 `maxDurationMs` 字段。未知字段、非法 provider、非法 URL、非法 mode、非法认证组合、非法 options、非法 output 规则和 `mode: http` 下配置 `expect.stream` MUST 导致启动期配置错误。
|
||||
系统 SHALL 在启动期对 llm checker 的配置契约和语义执行严格校验。LLM target 的 `llm` 分组 SHALL 只允许 `provider`、`url`、`model`、`prompt`、`mode`、`key`、`authToken`、`headers`、`ignoreSSL`、`options` 和 `providerOptions` 字段。`defaults.llm` 分组 SHALL 只允许 `mode`、`headers`、`ignoreSSL`、`options` 和 `providerOptions` 字段。LLM expect SHALL 只允许 `status`、`headers`、`output`、`finishReason`、`rawFinishReason`、`usage`、`stream` 和 `durationMs` 字段。`expect.output` MUST 为 `ContentRules` 数组。`expect.finishReason` 和 `expect.rawFinishReason` SHALL 使用 `ValueMatcher`。`expect.usage.*` 和 `expect.stream.firstTokenMs` SHALL 使用 `ValueMatcher`。未知字段、非法 provider、非法 URL、非法 mode、非法认证组合、非法 options、非法 output 规则和 `mode: http` 下配置 `expect.stream` MUST 导致启动期配置错误。
|
||||
|
||||
#### Scenario: llm provider 非法
|
||||
- **WHEN** YAML 中 llm target 的 `llm.provider` 不是 `openai`、`openai-responses` 或 `anthropic`
|
||||
@@ -538,12 +554,12 @@
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 llm 分组包含未知字段
|
||||
|
||||
#### Scenario: llm output 规则缺少支持字段
|
||||
- **WHEN** YAML 中 llm target 的 `expect.output` 数组项未包含 equals、contains、regex、json 任一支持字段
|
||||
- **WHEN** YAML 中 llm target 的 `expect.output` 数组项未包含任何合法 ValueMatcher 字段或 extractor
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 output rule 缺少支持的规则类型
|
||||
|
||||
#### Scenario: llm output 规则同时配置多个支持字段
|
||||
- **WHEN** YAML 中 llm target 的同一条 output rule 同时包含 equals、contains、regex、json 中的多个支持字段
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示每条 output rule 只能配置一种规则类型
|
||||
#### Scenario: llm output 规则同时配置多个 extractor
|
||||
- **WHEN** YAML 中 llm target 的同一条 output rule 同时包含 json、css、xpath 中的多个 extractor
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示每条 output rule 只能配置一种 extractor
|
||||
|
||||
#### Scenario: llm output regex 非法
|
||||
- **WHEN** YAML 中 llm target 的 output regex 规则不是字符串、不是可编译正则表达式或存在 ReDoS 风险
|
||||
@@ -554,7 +570,7 @@
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示该 output json path 不合法
|
||||
|
||||
#### Scenario: llm expect usage 非法
|
||||
- **WHEN** YAML 中 llm target 的 `expect.usage.inputTokens`、`expect.usage.outputTokens` 或 `expect.usage.totalTokens` 不是合法 operator 对象
|
||||
- **WHEN** YAML 中 llm target 的 `expect.usage.inputTokens`、`expect.usage.outputTokens` 或 `expect.usage.totalTokens` 不是合法 `ValueMatcher`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.usage 格式错误
|
||||
|
||||
#### Scenario: llm expect stream 仅允许 stream mode
|
||||
@@ -562,5 +578,5 @@
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.stream 仅支持 stream mode
|
||||
|
||||
#### Scenario: llm expect stream firstTokenMs 非法
|
||||
- **WHEN** YAML 中 llm target 的 `expect.stream.firstTokenMs` 不是合法 operator 对象
|
||||
- **WHEN** YAML 中 llm target 的 `expect.stream.firstTokenMs` 不是合法 `ValueMatcher`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.stream.firstTokenMs 格式错误
|
||||
|
||||
@@ -86,28 +86,32 @@
|
||||
- **THEN** `statusDetail` SHALL 展示截断后的 banner 摘要，避免 UI 和历史记录写入过长文本
|
||||
|
||||
### Requirement: tcp expect 校验
|
||||
系统 SHALL 支持 tcp 专属 expect，包括 `connected`、`banner` 和 `maxDurationMs`，并按 connected、banner、duration 的阶段顺序快速失败。
|
||||
系统 SHALL 支持 tcp 专属 expect，包括 `connected`、`banner` 和 `durationMs`，并按 connected、banner、durationMs 的阶段顺序快速失败。`connected` SHALL 保持布尔状态语义，未配置时默认 `true`。`banner` MUST 使用共享 `ContentRules` 数组，并仅在 `tcp.readBanner: true` 时允许配置。`durationMs` SHALL 使用共享 `ValueMatcher` 校验包含连接和 banner 读取在内的完整执行耗时。
|
||||
|
||||
#### Scenario: 默认 connected 成功语义
|
||||
- **WHEN** tcp target 未显式配置 `expect.connected`
|
||||
- **THEN** 系统 SHALL 使用默认 `expect.connected: true` 进行校验
|
||||
|
||||
#### Scenario: maxDurationMs 校验
|
||||
- **WHEN** tcp target 配置 `expect.maxDurationMs: 100`，且完整执行耗时超过 100ms
|
||||
#### Scenario: durationMs 校验
|
||||
- **WHEN** tcp target 配置 `expect.durationMs: {lte: 100}`，且完整执行耗时超过 100ms
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `duration`
|
||||
|
||||
#### Scenario: banner operator 校验通过
|
||||
- **WHEN** tcp target 配置 `readBanner: true`、`expect.banner: { contains: "ESMTP" }`，且实际 banner 包含 `ESMTP`
|
||||
#### Scenario: banner ContentRules 校验通过
|
||||
- **WHEN** tcp target 配置 `readBanner: true`、`expect.banner: [{contains: "ESMTP"}]`，且实际 banner 包含 `ESMTP`
|
||||
- **THEN** 系统 SHALL 判定 banner 阶段通过
|
||||
|
||||
#### Scenario: banner operator 校验失败
|
||||
- **WHEN** tcp target 配置 `readBanner: true`、`expect.banner: { contains: "ESMTP" }`，且实际 banner 不包含 `ESMTP`
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `banner`，path 为 `banner`
|
||||
#### Scenario: banner regex 校验失败
|
||||
- **WHEN** tcp target 配置 `readBanner: true`、`expect.banner: [{regex: "^SSH-2\\.0"}]`，且实际 banner 不匹配该正则
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `banner`，path 指向失败的 banner 规则
|
||||
|
||||
#### Scenario: banner 多规则快速失败
|
||||
- **WHEN** tcp target 配置两条 banner 规则且第一条失败
|
||||
- **THEN** 系统 SHALL 返回第一条失败规则的 failure，并 MUST NOT 执行第二条规则
|
||||
|
||||
#### Scenario: expect.banner 未开启 readBanner
|
||||
- **WHEN** tcp target 配置 `expect.banner`，但 `tcp.readBanner` 未配置为 `true`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示 banner 断言需要启用 tcp.readBanner
|
||||
|
||||
#### Scenario: tcp expect 未知字段失败
|
||||
- **WHEN** YAML 中 tcp target 的 expect 包含 `status: [200]` 或其他非 tcp expect 字段
|
||||
- **WHEN** YAML 中 tcp target 的 expect 包含 `status: [200]`、`maxDurationMs: 1000` 或其他非 tcp expect 字段
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段
|
||||
|
||||
@@ -133,17 +133,21 @@
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 maxResponseBytes 格式错误
|
||||
|
||||
### Requirement: udp expect 校验
|
||||
系统 SHALL 支持 udp 专属 expect，包括 `responded`、`response`、`responseSize`、`sourceHost`、`sourcePort` 和 `maxDurationMs`，并按 responded、responseSize、response、sourceHost、sourcePort、duration 的阶段顺序快速失败。
|
||||
系统 SHALL 支持 udp 专属 expect，包括 `responded`、`response`、`responseSize`、`sourceHost`、`sourcePort` 和 `durationMs`，并按 responded、responseSize、response、sourceHost、sourcePort、durationMs 的阶段顺序快速失败。`responded` SHALL 保持布尔状态语义，未配置时默认 `true`。`response` MUST 使用共享 `ContentRules` 数组，并作用于按 `udp.responseEncoding` 转换后的响应文本。`responseSize`、`sourceHost`、`sourcePort` 和 `durationMs` SHALL 使用共享 `ValueMatcher`。
|
||||
|
||||
#### Scenario: 默认 responded 成功语义
|
||||
- **WHEN** udp target 未显式配置 `expect.responded`
|
||||
- **THEN** 系统 SHALL 使用默认 `expect.responded: true` 进行校验
|
||||
|
||||
#### Scenario: response text rules 校验通过
|
||||
#### Scenario: response ContentRules 校验通过
|
||||
- **WHEN** udp target 配置 `expect.response: [{ contains: "PONG" }]`，且按 `responseEncoding` 转换后的响应文本包含 `PONG`
|
||||
- **THEN** 系统 SHALL 判定 response 阶段通过
|
||||
|
||||
#### Scenario: response text rules 校验失败
|
||||
#### Scenario: response JSON 校验通过
|
||||
- **WHEN** udp target 收到文本响应 `{"status":"ok"}` 且配置 `expect.response: [{json: {path: "$.status", equals: "ok"}}]`
|
||||
- **THEN** 系统 SHALL 判定 response 阶段通过
|
||||
|
||||
#### Scenario: response ContentRules 校验失败
|
||||
- **WHEN** udp target 配置 `expect.response: [{ contains: "PONG" }]`，但按 `responseEncoding` 转换后的响应文本不包含 `PONG`
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `response`，path 指向失败的 response 规则
|
||||
|
||||
@@ -151,28 +155,24 @@
|
||||
- **WHEN** udp target 配置 `udp.responseEncoding: "hex"` 且收到字节内容 `PONG`
|
||||
- **THEN** 系统 SHALL 将响应转换为小写 hex 字符串 `504f4e47` 后执行 `expect.response` 规则
|
||||
|
||||
#### Scenario: responseEncoding 为 base64
|
||||
- **WHEN** udp target 配置 `udp.responseEncoding: "base64"` 且收到字节内容 `PONG`
|
||||
- **THEN** 系统 SHALL 将响应转换为 base64 字符串 `UE9ORw==` 后执行 `expect.response` 规则
|
||||
|
||||
#### Scenario: responseSize operator 校验通过
|
||||
#### Scenario: responseSize matcher 校验通过
|
||||
- **WHEN** udp target 配置 `expect.responseSize: { gte: 4 }`，且实际响应为 4 字节
|
||||
- **THEN** 系统 SHALL 判定 responseSize 阶段通过
|
||||
|
||||
#### Scenario: responseSize operator 校验失败
|
||||
#### Scenario: responseSize matcher 校验失败
|
||||
- **WHEN** udp target 配置 `expect.responseSize: { gte: 4 }`，但实际响应为 2 字节
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `responseSize`
|
||||
|
||||
#### Scenario: sourceHost operator 校验
|
||||
#### Scenario: sourceHost matcher 校验
|
||||
- **WHEN** udp target 配置 `expect.sourceHost: { equals: "127.0.0.1" }`，且 Bun 回调中的来源地址为 `127.0.0.1`
|
||||
- **THEN** 系统 SHALL 判定 sourceHost 阶段通过
|
||||
|
||||
#### Scenario: sourcePort operator 校验
|
||||
#### Scenario: sourcePort matcher 校验
|
||||
- **WHEN** udp target 配置 `expect.sourcePort: { equals: 9000 }`，且 Bun 回调中的来源端口为 `9000`
|
||||
- **THEN** 系统 SHALL 判定 sourcePort 阶段通过
|
||||
|
||||
#### Scenario: maxDurationMs 校验
|
||||
- **WHEN** udp target 配置 `expect.maxDurationMs: 100`，且完整执行耗时超过 100ms
|
||||
#### Scenario: durationMs 校验
|
||||
- **WHEN** udp target 配置 `expect.durationMs: {lte: 100}`，且完整执行耗时超过 100ms
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `duration`
|
||||
|
||||
#### Scenario: response 断言要求实际有响应
|
||||
@@ -184,7 +184,7 @@
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示响应来源断言需要 `expect.responded` 为 true
|
||||
|
||||
#### Scenario: udp expect 未知字段失败
|
||||
- **WHEN** YAML 中 udp target 的 expect 包含 `status: [200]` 或其他非 udp expect 字段
|
||||
- **WHEN** YAML 中 udp target 的 expect 包含 `status: [200]`、`maxDurationMs: 1000` 或其他非 udp expect 字段
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段
|
||||
|
||||
### Requirement: udp statusDetail 摘要
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -34,7 +34,8 @@ targets:
|
||||
url: "https://www.baidu.com"
|
||||
expect:
|
||||
status: [200]
|
||||
maxDurationMs: 5000
|
||||
durationMs:
|
||||
lte: 5000
|
||||
|
||||
- id: "httpbin-json"
|
||||
name: "${env_name} JSON API — 完整流水线"
|
||||
@@ -51,7 +52,8 @@ targets:
|
||||
headers:
|
||||
Content-Type:
|
||||
contains: "application/json"
|
||||
maxDurationMs: 8000
|
||||
durationMs:
|
||||
lte: 8000
|
||||
body:
|
||||
- json:
|
||||
path: "$.slideshow.title"
|
||||
@@ -92,7 +94,7 @@ targets:
|
||||
expect:
|
||||
exitCode: [0]
|
||||
stdout:
|
||||
- match: "^\\d+\\.\\d+\\.\\d+"
|
||||
- regex: "^\\d+\\.\\d+\\.\\d+"
|
||||
|
||||
- id: "bun-stdout-rules"
|
||||
name: "多规则 stdout 顺序校验"
|
||||
@@ -104,7 +106,7 @@ targets:
|
||||
expect:
|
||||
stdout:
|
||||
- contains: "version:"
|
||||
- match: "\\d+\\.\\d+\\.\\d+"
|
||||
- regex: "\\d+\\.\\d+\\.\\d+"
|
||||
- contains: "healthy"
|
||||
|
||||
- id: "bun-stderr"
|
||||
@@ -127,7 +129,8 @@ targets:
|
||||
db:
|
||||
url: "${sqlite_url}"
|
||||
expect:
|
||||
maxDurationMs: 1000
|
||||
durationMs:
|
||||
lte: 1000
|
||||
|
||||
- id: "sqlite-query"
|
||||
name: "SQLite 内存数据库多列结果校验"
|
||||
@@ -145,6 +148,10 @@ targets:
|
||||
exists: true
|
||||
role:
|
||||
contains: "engineer"
|
||||
result:
|
||||
- json:
|
||||
path: "$.rows[0].role"
|
||||
equals: "engineer"
|
||||
|
||||
# ========== TCP targets ==========
|
||||
|
||||
@@ -156,7 +163,8 @@ targets:
|
||||
host: "127.0.0.1"
|
||||
port: 6379
|
||||
expect:
|
||||
maxDurationMs: 3000
|
||||
durationMs:
|
||||
lte: 3000
|
||||
|
||||
- id: "smtp-banner"
|
||||
name: "SMTP Banner 探测"
|
||||
@@ -169,7 +177,7 @@ targets:
|
||||
bannerReadTimeout: 3000
|
||||
expect:
|
||||
banner:
|
||||
contains: "ESMTP"
|
||||
- contains: "ESMTP"
|
||||
|
||||
# ========== Ping targets ==========
|
||||
|
||||
@@ -183,10 +191,14 @@ targets:
|
||||
packetSize: 56
|
||||
expect:
|
||||
alive: true
|
||||
maxPacketLoss: 10
|
||||
maxAvgLatencyMs: 100
|
||||
maxMaxLatencyMs: 300
|
||||
maxDurationMs: 5000
|
||||
packetLossPercent:
|
||||
lte: 10
|
||||
avgLatencyMs:
|
||||
lte: 100
|
||||
maxLatencyMs:
|
||||
lte: 300
|
||||
durationMs:
|
||||
lte: 5000
|
||||
|
||||
# ========== UDP targets ==========
|
||||
|
||||
@@ -201,7 +213,8 @@ targets:
|
||||
expect:
|
||||
response:
|
||||
- contains: "PONG"
|
||||
maxDurationMs: 100
|
||||
durationMs:
|
||||
lte: 100
|
||||
|
||||
- id: "udp-binary-probe"
|
||||
name: "UDP 二进制协议探测"
|
||||
@@ -216,7 +229,8 @@ targets:
|
||||
expect:
|
||||
responseSize:
|
||||
gte: 4
|
||||
maxDurationMs: 200
|
||||
durationMs:
|
||||
lte: 200
|
||||
|
||||
- id: "udp-fire-and-forget"
|
||||
name: "UDP 发送验证（不等待响应）"
|
||||
@@ -242,7 +256,8 @@ targets:
|
||||
expect:
|
||||
status:
|
||||
- 200
|
||||
finishReason: "stop"
|
||||
finishReason:
|
||||
equals: "stop"
|
||||
output:
|
||||
- contains: "OK"
|
||||
|
||||
@@ -264,5 +279,7 @@ targets:
|
||||
completed: true
|
||||
firstTokenMs:
|
||||
lte: 5000
|
||||
finishReason: "stop"
|
||||
maxDurationMs: 15000
|
||||
finishReason:
|
||||
equals: "stop"
|
||||
durationMs:
|
||||
lte: 15000
|
||||
|
||||
175
src/server/checker/expect/content.ts
Normal file
175
src/server/checker/expect/content.ts
Normal file
@@ -0,0 +1,175 @@
|
||||
import { DOMParser } from "@xmldom/xmldom";
|
||||
import * as cheerio from "cheerio";
|
||||
import { isArray } from "es-toolkit/compat";
|
||||
import * as xpath from "xpath";
|
||||
|
||||
import type { CheckFailure } from "../types";
|
||||
import type {
|
||||
ContentCssRule,
|
||||
ContentJsonRule,
|
||||
ContentRule,
|
||||
ContentRules,
|
||||
ContentXpathRule,
|
||||
ExpectResult,
|
||||
} from "./types";
|
||||
|
||||
import { errorFailure, mismatchFailure } from "./failure";
|
||||
import { applyMatcher, evaluateJsonPath } from "./matcher";
|
||||
|
||||
type ParsedJsonResult = { error: string; ok: false } | { ok: true; value: unknown };
|
||||
|
||||
export function checkContentRules(
|
||||
source: unknown,
|
||||
rules: ContentRules | undefined,
|
||||
options: { path?: string; phase: CheckFailure["phase"] },
|
||||
): ExpectResult {
|
||||
if (!rules || rules.length === 0) return { failure: null, matched: true };
|
||||
|
||||
const basePath = options.path ?? options.phase;
|
||||
let parsedJson: ParsedJsonResult | undefined;
|
||||
|
||||
for (let i = 0; i < rules.length; i++) {
|
||||
const rule = rules[i]!;
|
||||
if ("json" in rule && parsedJson === undefined) {
|
||||
parsedJson = parseJsonSource(source);
|
||||
}
|
||||
|
||||
const result = checkSingleContentRule(source, rule, `${basePath}[${i}]`, options.phase, parsedJson);
|
||||
if (!result.matched) return result;
|
||||
}
|
||||
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
function checkCssRule(
|
||||
source: unknown,
|
||||
rule: ContentCssRule,
|
||||
rulePath: string,
|
||||
phase: CheckFailure["phase"],
|
||||
): ExpectResult {
|
||||
const { attr, selector, ...matcher } = rule;
|
||||
const fullPath = `${rulePath}.css(${selector}${attr ? `@${attr}` : ""})`;
|
||||
|
||||
let $: cheerio.CheerioAPI;
|
||||
try {
|
||||
$ = cheerio.load(contentText(source));
|
||||
} catch {
|
||||
return { failure: errorFailure(phase, fullPath, "failed to parse HTML"), matched: false };
|
||||
}
|
||||
|
||||
const el = $(selector).first();
|
||||
const actual = el.length === 0 ? undefined : attr ? el.attr(attr) : el.text();
|
||||
const effectiveMatcher = Object.keys(matcher).length === 0 ? { exists: true } : matcher;
|
||||
|
||||
if (!applyMatcher(actual, effectiveMatcher)) {
|
||||
return {
|
||||
failure: mismatchFailure(phase, fullPath, effectiveMatcher, actual, `css selector ${selector} mismatch`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
function checkJsonRule(
|
||||
rule: ContentJsonRule,
|
||||
rulePath: string,
|
||||
phase: CheckFailure["phase"],
|
||||
parsedJson?: ParsedJsonResult,
|
||||
): ExpectResult {
|
||||
const { path, ...matcher } = rule;
|
||||
const fullPath = `${rulePath}.json(${path})`;
|
||||
|
||||
if (!parsedJson?.ok) {
|
||||
return { failure: errorFailure(phase, fullPath, parsedJson?.error ?? "content is not valid JSON"), matched: false };
|
||||
}
|
||||
|
||||
const actual = evaluateJsonPath(parsedJson.value, path);
|
||||
const effectiveMatcher = Object.keys(matcher).length === 0 ? { exists: true } : matcher;
|
||||
if (!applyMatcher(actual, effectiveMatcher)) {
|
||||
return {
|
||||
failure: mismatchFailure(phase, fullPath, effectiveMatcher, actual, `json path ${path} mismatch`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
function checkSingleContentRule(
|
||||
source: unknown,
|
||||
rule: ContentRule,
|
||||
rulePath: string,
|
||||
phase: CheckFailure["phase"],
|
||||
parsedJson?: ParsedJsonResult,
|
||||
): ExpectResult {
|
||||
if ("json" in rule) return checkJsonRule(rule.json, rulePath, phase, parsedJson);
|
||||
if ("css" in rule) return checkCssRule(source, rule.css, rulePath, phase);
|
||||
if ("xpath" in rule) return checkXpathRule(source, rule.xpath, rulePath, phase);
|
||||
|
||||
if (!applyMatcher(source, rule, { stringifyNonString: true })) {
|
||||
return {
|
||||
failure: mismatchFailure(phase, rulePath, rule, source, `${phase} rule mismatch`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
function checkXpathRule(
|
||||
source: unknown,
|
||||
rule: ContentXpathRule,
|
||||
rulePath: string,
|
||||
phase: CheckFailure["phase"],
|
||||
): ExpectResult {
|
||||
const { path, ...matcher } = rule;
|
||||
const fullPath = `${rulePath}.xpath(${path})`;
|
||||
|
||||
let doc: ReturnType<DOMParser["parseFromString"]>;
|
||||
try {
|
||||
doc = new DOMParser().parseFromString(contentText(source), "text/xml");
|
||||
} catch {
|
||||
return { failure: errorFailure(phase, fullPath, "failed to parse XML/HTML"), matched: false };
|
||||
}
|
||||
|
||||
const result = xpath.select(path, doc as unknown as Node);
|
||||
const actual = xpathValue(result);
|
||||
const effectiveMatcher = Object.keys(matcher).length === 0 ? { exists: true } : matcher;
|
||||
|
||||
if (!applyMatcher(actual, effectiveMatcher)) {
|
||||
return {
|
||||
failure: mismatchFailure(phase, fullPath, effectiveMatcher, actual, `xpath ${path} mismatch`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
function contentText(source: unknown): string {
|
||||
if (source === null || source === undefined) return "";
|
||||
if (typeof source === "string") return source;
|
||||
if (typeof source === "number" || typeof source === "boolean" || typeof source === "bigint") return String(source);
|
||||
if (typeof source === "symbol") return source.description ?? "";
|
||||
if (typeof source === "function") return source.name;
|
||||
return JSON.stringify(source) ?? "";
|
||||
}
|
||||
|
||||
function parseJsonSource(source: unknown): ParsedJsonResult {
|
||||
if (typeof source !== "string") return { ok: true, value: source };
|
||||
try {
|
||||
return { ok: true, value: JSON.parse(source) as unknown };
|
||||
} catch {
|
||||
return { error: "content is not valid JSON", ok: false };
|
||||
}
|
||||
}
|
||||
|
||||
function xpathValue(result: unknown): unknown {
|
||||
if (!isArray(result)) return result;
|
||||
if (result.length === 0) return undefined;
|
||||
|
||||
const node = (result as unknown[])[0]!;
|
||||
if (typeof node !== "object" || node === null) return node;
|
||||
const asNode = node as Node;
|
||||
return asNode.nodeValue ?? (asNode as unknown as Element).textContent ?? "";
|
||||
}
|
||||
@@ -1,20 +0,0 @@
|
||||
import type { ExpectResult } from "./types";
|
||||
|
||||
import { mismatchFailure } from "./failure";
|
||||
|
||||
export function checkDuration(durationMs: number, maxDurationMs?: number): ExpectResult {
|
||||
if (maxDurationMs === undefined) return { failure: null, matched: true };
|
||||
if (durationMs > maxDurationMs) {
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
"duration",
|
||||
"duration",
|
||||
`<=${maxDurationMs}ms`,
|
||||
durationMs,
|
||||
`duration ${durationMs}ms > ${maxDurationMs}ms`,
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
32
src/server/checker/expect/key-value.ts
Normal file
32
src/server/checker/expect/key-value.ts
Normal file
@@ -0,0 +1,32 @@
|
||||
import type { CheckFailure } from "../types";
|
||||
import type { ExpectResult, KeyValueExpect } from "./types";
|
||||
|
||||
import { mismatchFailure } from "./failure";
|
||||
import { checkExpectValue } from "./matcher";
|
||||
|
||||
export function checkKeyValueExpect(
|
||||
actual: Record<string, unknown>,
|
||||
expected: KeyValueExpect | undefined,
|
||||
options: { normalizeKey?: (key: string) => string; path?: string; phase: CheckFailure["phase"] },
|
||||
): ExpectResult {
|
||||
if (!expected) return { failure: null, matched: true };
|
||||
|
||||
const normalizeKey = options.normalizeKey ?? ((key: string) => key);
|
||||
const basePath = options.path ?? options.phase;
|
||||
const actualMap = new Map<string, unknown>();
|
||||
for (const [key, value] of Object.entries(actual)) {
|
||||
actualMap.set(normalizeKey(key), value);
|
||||
}
|
||||
|
||||
for (const [key, expectedValue] of Object.entries(expected)) {
|
||||
const actualValue = actualMap.get(normalizeKey(key));
|
||||
if (!checkExpectValue(actualValue, expectedValue)) {
|
||||
return {
|
||||
failure: mismatchFailure(options.phase, `${basePath}.${key}`, expectedValue, actualValue, `${key} mismatch`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
133
src/server/checker/expect/matcher.ts
Normal file
133
src/server/checker/expect/matcher.ts
Normal file
@@ -0,0 +1,133 @@
|
||||
import { isEmptyObject, isEqual, isNil, isPlainObject } from "es-toolkit";
|
||||
import { isArray } from "es-toolkit/compat";
|
||||
|
||||
import type { CheckFailure, JsonValue } from "../types";
|
||||
import type { ExpectResult, ValueMatcher } from "./types";
|
||||
|
||||
import { mismatchFailure } from "./failure";
|
||||
|
||||
export const MatcherKeys = ["contains", "empty", "equals", "exists", "gt", "gte", "lt", "lte", "regex"] as const;
|
||||
|
||||
const MATCHER_KEY_SET = new Set<string>(MatcherKeys);
|
||||
|
||||
export function applyMatcher(
|
||||
actual: unknown,
|
||||
matcher: ValueMatcher,
|
||||
options: { stringifyNonString?: boolean } = {},
|
||||
): boolean {
|
||||
for (const [key, expected] of Object.entries(matcher)) {
|
||||
if (expected === undefined) continue;
|
||||
|
||||
switch (key) {
|
||||
case "contains":
|
||||
if (!stringValue(actual, options).includes(expected as string)) return false;
|
||||
break;
|
||||
case "empty": {
|
||||
const empty = isEmptyValue(actual);
|
||||
if (expected !== empty) return false;
|
||||
break;
|
||||
}
|
||||
case "equals":
|
||||
if (!isEqual(actual, expected)) return false;
|
||||
break;
|
||||
case "exists":
|
||||
if (expected) {
|
||||
if (actual === undefined) return false;
|
||||
} else {
|
||||
if (actual !== undefined) return false;
|
||||
}
|
||||
break;
|
||||
case "gt":
|
||||
if (!compareNumber(actual, expected as number, (left, right) => left > right)) return false;
|
||||
break;
|
||||
case "gte":
|
||||
if (!compareNumber(actual, expected as number, (left, right) => left >= right)) return false;
|
||||
break;
|
||||
case "lt":
|
||||
if (!compareNumber(actual, expected as number, (left, right) => left < right)) return false;
|
||||
break;
|
||||
case "lte":
|
||||
if (!compareNumber(actual, expected as number, (left, right) => left <= right)) return false;
|
||||
break;
|
||||
case "regex":
|
||||
if (!new RegExp(expected as string).test(stringValue(actual, options))) return false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
export function checkExpectValue(actual: unknown, expected: JsonValue | ValueMatcher): boolean {
|
||||
if (isValueMatcherObject(expected)) {
|
||||
return applyMatcher(actual, expected);
|
||||
}
|
||||
return applyMatcher(actual, { equals: expected });
|
||||
}
|
||||
|
||||
export function checkValueMatcher(
|
||||
actual: unknown,
|
||||
matcher: undefined | ValueMatcher,
|
||||
options: { message?: string; path: string; phase: CheckFailure["phase"]; stringifyNonString?: boolean },
|
||||
): ExpectResult {
|
||||
if (matcher === undefined) return { failure: null, matched: true };
|
||||
if (applyMatcher(actual, matcher, { stringifyNonString: options.stringifyNonString })) {
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
options.phase,
|
||||
options.path,
|
||||
matcher,
|
||||
actual,
|
||||
options.message ?? `${options.path} mismatch`,
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
|
||||
export function evaluateJsonPath(json: unknown, path: string): unknown {
|
||||
if (!path.startsWith("$.")) return undefined;
|
||||
|
||||
const segments = path.slice(2).split(".");
|
||||
let current: unknown = json;
|
||||
|
||||
for (const seg of segments) {
|
||||
const bracketMatch = /^(.+?)\[(\d+)\]$/.exec(seg);
|
||||
if (bracketMatch) {
|
||||
if (current === null || current === undefined) return undefined;
|
||||
current = (current as Record<string, unknown>)[bracketMatch[1]!];
|
||||
const idx = parseInt(bracketMatch[2]!, 10);
|
||||
if (!isArray(current) || idx >= current.length) return undefined;
|
||||
current = current[idx];
|
||||
} else {
|
||||
if (current === null || current === undefined) return undefined;
|
||||
current = (current as Record<string, unknown>)[seg];
|
||||
}
|
||||
}
|
||||
|
||||
return current;
|
||||
}
|
||||
|
||||
export function isValueMatcherObject(value: unknown): value is ValueMatcher {
|
||||
return isPlainObject(value) && Object.keys(value).some((key) => MATCHER_KEY_SET.has(key));
|
||||
}
|
||||
|
||||
function compareNumber(
|
||||
actual: unknown,
|
||||
expected: number,
|
||||
compare: (actual: number, expected: number) => boolean,
|
||||
): boolean {
|
||||
const value = Number(actual);
|
||||
return Number.isFinite(value) && compare(value, expected);
|
||||
}
|
||||
|
||||
function isEmptyValue(value: unknown): boolean {
|
||||
return isNil(value) || value === "" || (isArray(value) && value.length === 0) || isEmptyObject(value);
|
||||
}
|
||||
|
||||
function stringValue(actual: unknown, options: { stringifyNonString?: boolean }): string {
|
||||
if (!options.stringifyNonString || typeof actual === "string") return String(actual);
|
||||
if (actual !== null && typeof actual === "object") return JSON.stringify(actual);
|
||||
return String(actual);
|
||||
}
|
||||
@@ -1,80 +0,0 @@
|
||||
import { isEmptyObject, isEqual, isNil, isPlainObject } from "es-toolkit";
|
||||
import { isArray } from "es-toolkit/compat";
|
||||
|
||||
import type { ExpectOperator, ExpectValue } from "../types";
|
||||
|
||||
const OPERATOR_KEYS = new Set(["contains", "empty", "equals", "exists", "gt", "gte", "lt", "lte", "match"]);
|
||||
|
||||
export function applyOperator(actual: unknown, op: ExpectOperator): boolean {
|
||||
for (const [key, expected] of Object.entries(op)) {
|
||||
if (expected === undefined) continue;
|
||||
|
||||
switch (key) {
|
||||
case "contains":
|
||||
if (!String(actual).includes(expected as string)) return false;
|
||||
break;
|
||||
case "empty": {
|
||||
const isEmpty =
|
||||
isNil(actual) || actual === "" || (isArray(actual) && actual.length === 0) || isEmptyObject(actual);
|
||||
if (expected !== isEmpty) return false;
|
||||
break;
|
||||
}
|
||||
case "equals":
|
||||
if (!isEqual(actual, expected)) return false;
|
||||
break;
|
||||
case "exists":
|
||||
if (expected) {
|
||||
if (actual === undefined) return false;
|
||||
} else {
|
||||
if (actual !== undefined) return false;
|
||||
}
|
||||
break;
|
||||
case "gt":
|
||||
if (!(Number(actual) > (expected as number))) return false;
|
||||
break;
|
||||
case "gte":
|
||||
if (!(Number(actual) >= (expected as number))) return false;
|
||||
break;
|
||||
case "lt":
|
||||
if (!(Number(actual) < (expected as number))) return false;
|
||||
break;
|
||||
case "lte":
|
||||
if (!(Number(actual) <= (expected as number))) return false;
|
||||
break;
|
||||
case "match":
|
||||
if (!new RegExp(expected as string).test(String(actual))) return false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
export function checkExpectValue(actual: unknown, expected: ExpectValue): boolean {
|
||||
if (isPlainObject(expected) && Object.keys(expected).some((key) => OPERATOR_KEYS.has(key))) {
|
||||
return applyOperator(actual, expected as ExpectOperator);
|
||||
}
|
||||
return applyOperator(actual, { equals: expected as Exclude<ExpectValue, ExpectOperator> });
|
||||
}
|
||||
|
||||
export function evaluateJsonPath(json: unknown, path: string): unknown {
|
||||
if (!path.startsWith("$.")) return undefined;
|
||||
|
||||
const segments = path.slice(2).split(".");
|
||||
let current: unknown = json;
|
||||
|
||||
for (const seg of segments) {
|
||||
const bracketMatch = /^(.+?)\[(\d+)\]$/.exec(seg);
|
||||
if (bracketMatch) {
|
||||
current = (current as Record<string, unknown>)?.[bracketMatch[1]!];
|
||||
const idx = parseInt(bracketMatch[2]!, 10);
|
||||
if (!isArray(current) || idx >= current.length) return undefined;
|
||||
current = current[idx];
|
||||
} else {
|
||||
if (current === null || current === undefined) return undefined;
|
||||
current = (current as Record<string, unknown>)[seg];
|
||||
}
|
||||
}
|
||||
|
||||
return current;
|
||||
}
|
||||
@@ -1,6 +1,41 @@
|
||||
import type { CheckFailure } from "../types";
|
||||
import type { CheckFailure, JsonValue } from "../types";
|
||||
|
||||
export interface ContentCssRule extends ValueMatcher {
|
||||
attr?: string;
|
||||
selector: string;
|
||||
}
|
||||
|
||||
export interface ContentJsonRule extends ValueMatcher {
|
||||
path: string;
|
||||
}
|
||||
|
||||
export type ContentRule =
|
||||
| ValueMatcher
|
||||
| { css: ContentCssRule }
|
||||
| { json: ContentJsonRule }
|
||||
| { xpath: ContentXpathRule };
|
||||
|
||||
export type ContentRules = ContentRule[];
|
||||
|
||||
export interface ContentXpathRule extends ValueMatcher {
|
||||
path: string;
|
||||
}
|
||||
|
||||
export interface ExpectResult {
|
||||
failure: CheckFailure | null;
|
||||
matched: boolean;
|
||||
}
|
||||
|
||||
export type KeyValueExpect = Record<string, JsonValue | ValueMatcher>;
|
||||
|
||||
export interface ValueMatcher {
|
||||
contains?: string;
|
||||
empty?: boolean;
|
||||
equals?: JsonValue;
|
||||
exists?: boolean;
|
||||
gt?: number;
|
||||
gte?: number;
|
||||
lt?: number;
|
||||
lte?: number;
|
||||
regex?: string;
|
||||
}
|
||||
|
||||
225
src/server/checker/expect/validate-matcher.ts
Normal file
225
src/server/checker/expect/validate-matcher.ts
Normal file
@@ -0,0 +1,225 @@
|
||||
import { DOMParser } from "@xmldom/xmldom";
|
||||
import { isBoolean, isNumber, isPlainObject, isString } from "es-toolkit";
|
||||
import { isArray } from "es-toolkit/compat";
|
||||
import * as xpath from "xpath";
|
||||
|
||||
import type { ConfigValidationIssue } from "../schema/issues";
|
||||
import type { JsonValue } from "../types";
|
||||
|
||||
import { issue, joinPath } from "../schema/issues";
|
||||
import { isUnsafeRegex } from "./redos";
|
||||
|
||||
export const MatcherKeys = ["contains", "empty", "equals", "exists", "gt", "gte", "lt", "lte", "regex"] as const;
|
||||
|
||||
const MATCHER_KEY_SET = new Set<string>(MatcherKeys);
|
||||
const EXTRACTOR_KEYS = ["css", "json", "xpath"] as const;
|
||||
const EXTRACTOR_KEY_SET = new Set<string>(EXTRACTOR_KEYS);
|
||||
|
||||
export function isJsonValue(value: unknown): value is JsonValue {
|
||||
if (value === null) return true;
|
||||
if (isString(value) || isBoolean(value)) return true;
|
||||
if (isNumber(value)) return Number.isFinite(value);
|
||||
if (isArray(value)) return value.every(isJsonValue);
|
||||
if (isPlainObject(value)) return Object.values(value).every(isJsonValue);
|
||||
return false;
|
||||
}
|
||||
|
||||
export function isPlainRecord(value: unknown): value is Record<string, unknown> {
|
||||
return isPlainObject(value);
|
||||
}
|
||||
|
||||
export function validateContentRules(rules: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isArray(rules)) return [issue("invalid-type", path, "必须为数组", targetName)];
|
||||
return rules.flatMap((rule, index) => validateContentRule(rule, `${path}[${index}]`, targetName));
|
||||
}
|
||||
|
||||
export function validateJsonPath(path: string, rulePath: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!path.startsWith("$.") || path.length <= 2) {
|
||||
return [issue("invalid-jsonpath", joinPath(rulePath, "path"), '必须为以 "$." 开头的有效 JSONPath', targetName)];
|
||||
}
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const segments = path.slice(2).split(".");
|
||||
for (const seg of segments) {
|
||||
if (seg === "") {
|
||||
issues.push(issue("invalid-jsonpath", joinPath(rulePath, "path"), "包含空段", targetName));
|
||||
}
|
||||
const bracketMatch = /^(.+?)\[(\d+)\]$/.exec(seg);
|
||||
if (bracketMatch?.[1]!.trim() === "") {
|
||||
issues.push(issue("invalid-jsonpath", joinPath(rulePath, "path"), "数组访问缺少属性名", targetName));
|
||||
}
|
||||
}
|
||||
return issues;
|
||||
}
|
||||
|
||||
export function validateKeyValueExpect(value: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(value)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
for (const [key, item] of Object.entries(value)) {
|
||||
const itemPath = joinPath(path, key);
|
||||
if (isPlainRecord(item)) {
|
||||
issues.push(...validateValueMatcher(item, itemPath, targetName));
|
||||
} else if (!isJsonValue(item)) {
|
||||
issues.push(issue("invalid-type", itemPath, "必须为 JSON value 或 matcher 对象", targetName));
|
||||
}
|
||||
}
|
||||
return issues;
|
||||
}
|
||||
|
||||
export function validateValueMatcher(
|
||||
matcher: unknown,
|
||||
path: string,
|
||||
targetName?: string,
|
||||
options: { requireAtLeastOne?: boolean } = {},
|
||||
): ConfigValidationIssue[] {
|
||||
const requireAtLeastOne = options.requireAtLeastOne ?? true;
|
||||
if (!isPlainRecord(matcher)) return [issue("invalid-type", path, "必须为 matcher 对象", targetName)];
|
||||
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
let found = 0;
|
||||
for (const [key, value] of Object.entries(matcher)) {
|
||||
if (!MATCHER_KEY_SET.has(key)) {
|
||||
issues.push(issue("unknown-matcher", joinPath(path, key), "是未知 matcher", targetName));
|
||||
continue;
|
||||
}
|
||||
if (value === undefined) continue;
|
||||
found++;
|
||||
issues.push(...validateMatcherValue(key, value, joinPath(path, key), targetName));
|
||||
}
|
||||
|
||||
if (requireAtLeastOne && found === 0) {
|
||||
issues.push(issue("empty-matcher", path, "必须包含至少一个合法 matcher", targetName));
|
||||
}
|
||||
|
||||
if (matcher["exists"] === false && found > 1) {
|
||||
issues.push(issue("invalid-value", joinPath(path, "exists"), "exists:false 不能与其他 matcher 组合", targetName));
|
||||
}
|
||||
|
||||
return issues;
|
||||
}
|
||||
|
||||
function validateContentRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const extractors = Object.keys(rule).filter((key) => EXTRACTOR_KEY_SET.has(key));
|
||||
const directMatchers = Object.keys(rule).filter((key) => MATCHER_KEY_SET.has(key));
|
||||
|
||||
for (const key of Object.keys(rule)) {
|
||||
if (!MATCHER_KEY_SET.has(key) && !EXTRACTOR_KEY_SET.has(key)) {
|
||||
issues.push(issue("unknown-field", joinPath(path, key), "是未知字段", targetName));
|
||||
}
|
||||
}
|
||||
|
||||
if (extractors.length > 1) {
|
||||
issues.push(issue("multiple-content-rules", path, "一条规则不能同时包含多个 extractor", targetName));
|
||||
}
|
||||
if (extractors.length === 1 && directMatchers.length > 0) {
|
||||
issues.push(issue("invalid-content-rule", path, "直接 matcher 不能与 extractor 混用", targetName));
|
||||
}
|
||||
if (issues.length > 0) return issues;
|
||||
|
||||
if (extractors.length === 0) return validateValueMatcher(rule, path, targetName);
|
||||
|
||||
const extractor = extractors[0]!;
|
||||
switch (extractor) {
|
||||
case "css":
|
||||
return validateCssRule(rule["css"], joinPath(path, "css"), targetName);
|
||||
case "json":
|
||||
return validateJsonRule(rule["json"], joinPath(path, "json"), targetName);
|
||||
case "xpath":
|
||||
return validateXpathRule(rule["xpath"], joinPath(path, "xpath"), targetName);
|
||||
}
|
||||
|
||||
return [];
|
||||
}
|
||||
|
||||
function validateCssRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
|
||||
if (!isString(rule["selector"]) || rule["selector"].trim() === "") {
|
||||
issues.push(issue("invalid-type", joinPath(path, "selector"), "必须为非空字符串", targetName));
|
||||
}
|
||||
if ("attr" in rule && !isString(rule["attr"])) {
|
||||
issues.push(issue("invalid-type", joinPath(path, "attr"), "必须为字符串", targetName));
|
||||
}
|
||||
issues.push(...validateExtractorMatcher(rule, new Set(["attr", "selector"]), path, targetName));
|
||||
return issues;
|
||||
}
|
||||
|
||||
function validateExtractorMatcher(
|
||||
rule: Record<string, unknown>,
|
||||
allowedFields: Set<string>,
|
||||
path: string,
|
||||
targetName?: string,
|
||||
): ConfigValidationIssue[] {
|
||||
const matcher: Record<string, unknown> = {};
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
for (const [key, value] of Object.entries(rule)) {
|
||||
if (allowedFields.has(key)) continue;
|
||||
matcher[key] = value;
|
||||
}
|
||||
issues.push(...validateValueMatcher(matcher, path, targetName, { requireAtLeastOne: false }));
|
||||
return issues;
|
||||
}
|
||||
|
||||
function validateJsonRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
|
||||
if (!isString(rule["path"])) {
|
||||
issues.push(issue("invalid-type", joinPath(path, "path"), "必须为字符串", targetName));
|
||||
} else {
|
||||
issues.push(...validateJsonPath(rule["path"], path, targetName));
|
||||
}
|
||||
issues.push(...validateExtractorMatcher(rule, new Set(["path"]), path, targetName));
|
||||
return issues;
|
||||
}
|
||||
|
||||
function validateMatcherValue(key: string, value: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
switch (key) {
|
||||
case "contains":
|
||||
return isString(value) ? [] : [issue("invalid-type", path, "必须为字符串", targetName)];
|
||||
case "empty":
|
||||
case "exists":
|
||||
return isBoolean(value) ? [] : [issue("invalid-type", path, "必须为布尔值", targetName)];
|
||||
case "equals":
|
||||
return isJsonValue(value) ? [] : [issue("invalid-type", path, "必须为 JSON value", targetName)];
|
||||
case "gt":
|
||||
case "gte":
|
||||
case "lt":
|
||||
case "lte":
|
||||
return isNumber(value) && Number.isFinite(value)
|
||||
? []
|
||||
: [issue("invalid-type", path, "必须为有限数字", targetName)];
|
||||
case "regex":
|
||||
if (!isString(value)) return [issue("invalid-type", path, "必须为字符串", targetName)];
|
||||
try {
|
||||
new RegExp(value);
|
||||
} catch {
|
||||
return [issue("invalid-regex", path, "正则不合法", targetName)];
|
||||
}
|
||||
return isUnsafeRegex(value) ? [issue("unsafe-regex", path, "正则存在 ReDoS 风险", targetName)] : [];
|
||||
default:
|
||||
return [issue("unknown-matcher", path, "是未知 matcher", targetName)];
|
||||
}
|
||||
}
|
||||
|
||||
function validateXpathRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
|
||||
if (!isString(rule["path"]) || rule["path"].trim() === "") {
|
||||
issues.push(issue("invalid-type", joinPath(path, "path"), "必须为非空字符串", targetName));
|
||||
} else {
|
||||
try {
|
||||
const doc = new DOMParser().parseFromString("<x/>", "text/xml");
|
||||
xpath.select(rule["path"], doc as unknown as Node);
|
||||
} catch {
|
||||
issues.push(issue("invalid-xpath", joinPath(path, "path"), "xpath 不合法", targetName));
|
||||
}
|
||||
}
|
||||
issues.push(...validateExtractorMatcher(rule, new Set(["path"]), path, targetName));
|
||||
return issues;
|
||||
}
|
||||
@@ -1,84 +0,0 @@
|
||||
import { isBoolean, isNumber, isPlainObject, isString } from "es-toolkit";
|
||||
import { isArray } from "es-toolkit/compat";
|
||||
|
||||
import type { ConfigValidationIssue } from "../schema/issues";
|
||||
import type { JsonValue } from "../types";
|
||||
|
||||
import { OperatorKeys } from "../schema/fragments";
|
||||
import { issue, joinPath } from "../schema/issues";
|
||||
import { isUnsafeRegex } from "./redos";
|
||||
|
||||
const OPERATOR_KEY_SET = new Set<string>(OperatorKeys);
|
||||
|
||||
export function isJsonValue(value: unknown): value is JsonValue {
|
||||
if (value === null) return true;
|
||||
if (isString(value) || isBoolean(value)) return true;
|
||||
if (isNumber(value)) return Number.isFinite(value);
|
||||
if (isArray(value)) return value.every(isJsonValue);
|
||||
if (isPlainObject(value)) {
|
||||
return Object.values(value).every(isJsonValue);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
export function isPlainRecord(value: unknown): value is Record<string, unknown> {
|
||||
return isPlainObject(value);
|
||||
}
|
||||
|
||||
export function validateOperatorObject(
|
||||
operators: unknown,
|
||||
path: string,
|
||||
targetName?: string,
|
||||
options: { requireAtLeastOne: boolean } = { requireAtLeastOne: true },
|
||||
): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(operators)) return [issue("invalid-type", path, "必须为操作符对象", targetName)];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
let found = 0;
|
||||
for (const [key, value] of Object.entries(operators)) {
|
||||
if (!OPERATOR_KEY_SET.has(key)) {
|
||||
issues.push(issue("unknown-operator", joinPath(path, key), "是未知 operator", targetName));
|
||||
continue;
|
||||
}
|
||||
if (value === undefined) continue;
|
||||
found++;
|
||||
issues.push(...validateOperatorValue(key, value, joinPath(path, key), targetName));
|
||||
}
|
||||
if (options.requireAtLeastOne && found === 0) {
|
||||
issues.push(issue("empty-operator", path, "必须包含至少一个合法 operator", targetName));
|
||||
}
|
||||
return issues;
|
||||
}
|
||||
|
||||
export function validateOperatorValue(
|
||||
key: string,
|
||||
value: unknown,
|
||||
path: string,
|
||||
targetName?: string,
|
||||
): ConfigValidationIssue[] {
|
||||
switch (key) {
|
||||
case "contains":
|
||||
return isString(value) ? [] : [issue("invalid-type", path, "必须为字符串", targetName)];
|
||||
case "empty":
|
||||
case "exists":
|
||||
return isBoolean(value) ? [] : [issue("invalid-type", path, "必须为布尔值", targetName)];
|
||||
case "equals":
|
||||
return isJsonValue(value) ? [] : [issue("invalid-type", path, "必须为 JSON value", targetName)];
|
||||
case "gt":
|
||||
case "gte":
|
||||
case "lt":
|
||||
case "lte":
|
||||
return isNumber(value) && Number.isFinite(value)
|
||||
? []
|
||||
: [issue("invalid-type", path, "必须为有限数字", targetName)];
|
||||
case "match":
|
||||
if (!isString(value)) return [issue("invalid-type", path, "必须为字符串", targetName)];
|
||||
try {
|
||||
new RegExp(value);
|
||||
} catch {
|
||||
return [issue("invalid-regex", path, "正则不合法", targetName)];
|
||||
}
|
||||
return isUnsafeRegex(value) ? [issue("unsafe-regex", path, "正则存在 ReDoS 风险", targetName)] : [];
|
||||
default:
|
||||
return [issue("unknown-operator", path, "是未知 operator", targetName)];
|
||||
}
|
||||
}
|
||||
@@ -5,12 +5,12 @@ import type { CheckResult, RawTargetConfig } from "../../types";
|
||||
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
|
||||
import type { CommandExpectConfig, CommandTargetConfig, ResolvedCommandTarget } from "./types";
|
||||
|
||||
import { checkDuration } from "../../expect/duration";
|
||||
import { checkContentRules } from "../../expect/content";
|
||||
import { errorFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { parseSize } from "../../utils";
|
||||
import { checkExitCode } from "./expect";
|
||||
import { commandCheckerSchemas } from "./schema";
|
||||
import { checkTextRules } from "./text";
|
||||
import { validateCommandConfig } from "./validate";
|
||||
|
||||
export class CommandChecker implements CheckerDefinition<ResolvedCommandTarget> {
|
||||
@@ -118,7 +118,11 @@ export class CommandChecker implements CheckerDefinition<ResolvedCommandTarget>
|
||||
};
|
||||
}
|
||||
|
||||
const durationResult = checkDuration(durationMs, t.expect?.maxDurationMs);
|
||||
const durationResult = checkValueMatcher(durationMs, t.expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
});
|
||||
if (!durationResult.matched) {
|
||||
return {
|
||||
durationMs,
|
||||
@@ -131,7 +135,7 @@ export class CommandChecker implements CheckerDefinition<ResolvedCommandTarget>
|
||||
}
|
||||
|
||||
if (t.expect?.stdout && t.expect.stdout.length > 0) {
|
||||
const stdoutResult = checkTextRules(outputResult.stdout, t.expect.stdout, "stdout");
|
||||
const stdoutResult = checkContentRules(outputResult.stdout, t.expect.stdout, { path: "stdout", phase: "stdout" });
|
||||
if (!stdoutResult.matched) {
|
||||
return {
|
||||
durationMs,
|
||||
@@ -145,7 +149,7 @@ export class CommandChecker implements CheckerDefinition<ResolvedCommandTarget>
|
||||
}
|
||||
|
||||
if (t.expect?.stderr && t.expect.stderr.length > 0) {
|
||||
const stderrResult = checkTextRules(outputResult.stderr, t.expect.stderr, "stderr");
|
||||
const stderrResult = checkContentRules(outputResult.stderr, t.expect.stderr, { path: "stderr", phase: "stderr" });
|
||||
if (!stderrResult.matched) {
|
||||
return {
|
||||
durationMs,
|
||||
|
||||
@@ -2,7 +2,12 @@ import { Type } from "@sinclair/typebox";
|
||||
|
||||
import type { CheckerSchemas } from "../types";
|
||||
|
||||
import { createTextRulesSchema, sizeSchema, stringMapSchema } from "../../schema/fragments";
|
||||
import {
|
||||
createContentRulesSchema,
|
||||
createValueMatcherSchema,
|
||||
sizeSchema,
|
||||
stringMapSchema,
|
||||
} from "../../schema/fragments";
|
||||
|
||||
export const commandCheckerSchemas: CheckerSchemas = {
|
||||
config: Type.Object(
|
||||
@@ -24,10 +29,10 @@ export const commandCheckerSchemas: CheckerSchemas = {
|
||||
),
|
||||
expect: Type.Object(
|
||||
{
|
||||
durationMs: Type.Optional(createValueMatcherSchema()),
|
||||
exitCode: Type.Optional(Type.Array(Type.Integer())),
|
||||
maxDurationMs: Type.Optional(Type.Number({ minimum: 0 })),
|
||||
stderr: Type.Optional(createTextRulesSchema()),
|
||||
stdout: Type.Optional(createTextRulesSchema()),
|
||||
stderr: Type.Optional(createContentRulesSchema()),
|
||||
stdout: Type.Optional(createContentRulesSchema()),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
|
||||
@@ -1,19 +0,0 @@
|
||||
import type { ExpectResult } from "../../expect/types";
|
||||
import type { TextRule } from "./types";
|
||||
|
||||
import { mismatchFailure } from "../../expect/failure";
|
||||
import { applyOperator } from "../../expect/operator";
|
||||
|
||||
export function checkTextRules(text: string, rules: TextRule[], phase: string): ExpectResult {
|
||||
for (let i = 0; i < rules.length; i++) {
|
||||
const rule = rules[i]!;
|
||||
const path = `${phase}[${i}]`;
|
||||
if (!applyOperator(text, rule)) {
|
||||
return {
|
||||
failure: mismatchFailure(phase, path, rule, text, `${phase} rule at index ${i} mismatch`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { ExpectOperator, ResolvedTargetBase } from "../../types";
|
||||
import type { ContentRules, ValueMatcher } from "../../expect/types";
|
||||
import type { ResolvedTargetBase } from "../../types";
|
||||
|
||||
export interface CommandDefaultsConfig {
|
||||
cwd?: string;
|
||||
@@ -6,10 +7,10 @@ export interface CommandDefaultsConfig {
|
||||
}
|
||||
|
||||
export interface CommandExpectConfig {
|
||||
durationMs?: ValueMatcher;
|
||||
exitCode?: number[];
|
||||
maxDurationMs?: number;
|
||||
stderr?: TextRule[];
|
||||
stdout?: TextRule[];
|
||||
stderr?: ContentRules;
|
||||
stdout?: ContentRules;
|
||||
}
|
||||
|
||||
export interface CommandTargetConfig {
|
||||
@@ -37,5 +38,3 @@ export interface ResolvedCommandTarget extends ResolvedTargetBase {
|
||||
timeoutMs: number;
|
||||
type: "cmd";
|
||||
}
|
||||
|
||||
export type TextRule = ExpectOperator;
|
||||
|
||||
@@ -1,10 +1,9 @@
|
||||
import { isNumber, isPlainObject, isString } from "es-toolkit";
|
||||
import { isArray } from "es-toolkit/compat";
|
||||
|
||||
import type { ConfigValidationIssue } from "../../schema/issues";
|
||||
import type { CheckerValidationInput } from "../types";
|
||||
|
||||
import { validateOperatorObject } from "../../expect/validate-operator";
|
||||
import { validateContentRules, validateValueMatcher } from "../../expect/validate-matcher";
|
||||
import { issue, joinPath } from "../../schema/issues";
|
||||
import { parseSize } from "../../utils";
|
||||
|
||||
@@ -32,10 +31,6 @@ function getTargetName(target: Record<string, unknown>): string | undefined {
|
||||
return isString(target["id"]) ? target["id"] : undefined;
|
||||
}
|
||||
|
||||
function isNonNegativeFiniteNumber(value: unknown): boolean {
|
||||
return isNumber(value) && Number.isFinite(value) && value >= 0;
|
||||
}
|
||||
|
||||
function isSizeInput(value: unknown): value is number | string {
|
||||
return isNumber(value) || isString(value);
|
||||
}
|
||||
@@ -47,13 +42,13 @@ function validateCommandExpect(target: Record<string, unknown>, path: string): C
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const expectPath = joinPath(path, "expect");
|
||||
if (expect["stdout"] !== undefined) {
|
||||
issues.push(...validateTextRules(expect["stdout"], joinPath(expectPath, "stdout"), targetName));
|
||||
issues.push(...validateContentRules(expect["stdout"], joinPath(expectPath, "stdout"), targetName));
|
||||
}
|
||||
if (expect["stderr"] !== undefined) {
|
||||
issues.push(...validateTextRules(expect["stderr"], joinPath(expectPath, "stderr"), targetName));
|
||||
issues.push(...validateContentRules(expect["stderr"], joinPath(expectPath, "stderr"), targetName));
|
||||
}
|
||||
if (expect["maxDurationMs"] !== undefined && !isNonNegativeFiniteNumber(expect["maxDurationMs"])) {
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, "maxDurationMs"), "必须为非负有限数字", targetName));
|
||||
if (expect["durationMs"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
}
|
||||
return issues;
|
||||
}
|
||||
@@ -87,8 +82,3 @@ function validateSizeValue(value: number | string, path: string, targetName?: st
|
||||
return [issue("invalid-size", path, error instanceof Error ? error.message : "size 格式不合法", targetName)];
|
||||
}
|
||||
}
|
||||
|
||||
function validateTextRules(rules: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isArray(rules)) return [issue("invalid-type", path, "必须为数组", targetName)];
|
||||
return rules.flatMap((rule, index) => validateOperatorObject(rule, `${path}[${index}]`, targetName));
|
||||
}
|
||||
|
||||
@@ -6,8 +6,9 @@ import type { CheckResult, RawTargetConfig } from "../../types";
|
||||
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
|
||||
import type { DbExpectConfig, DbTargetConfig, ResolvedDbTarget } from "./types";
|
||||
|
||||
import { checkDuration } from "../../expect/duration";
|
||||
import { checkContentRules } from "../../expect/content";
|
||||
import { errorFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { checkRowCount, checkRows } from "./expect";
|
||||
import { dbCheckerSchemas } from "./schema";
|
||||
import { validateDbConfig } from "./validate";
|
||||
@@ -59,7 +60,11 @@ export class DbChecker implements CheckerDefinition<ResolvedDbTarget> {
|
||||
// 无 query 时仅测试连接
|
||||
if (!t.db.query) {
|
||||
const durationMs = Math.round(performance.now() - start);
|
||||
const durationResult = checkDuration(durationMs, t.expect?.maxDurationMs);
|
||||
const durationResult = checkValueMatcher(durationMs, t.expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
});
|
||||
if (!durationResult.matched) {
|
||||
return {
|
||||
durationMs,
|
||||
@@ -111,7 +116,11 @@ export class DbChecker implements CheckerDefinition<ResolvedDbTarget> {
|
||||
}
|
||||
|
||||
// duration 断言
|
||||
const durationResult = checkDuration(durationMs, t.expect?.maxDurationMs);
|
||||
const durationResult = checkValueMatcher(durationMs, t.expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
});
|
||||
if (!durationResult.matched) {
|
||||
return {
|
||||
durationMs,
|
||||
@@ -125,7 +134,7 @@ export class DbChecker implements CheckerDefinition<ResolvedDbTarget> {
|
||||
|
||||
// rowCount 断言
|
||||
if (t.expect?.rowCount) {
|
||||
const rowCountResult = checkRowCount(rows, t.expect.rowCount);
|
||||
const rowCountResult = checkRowCount(isArray(rows) ? rows.length : 0, t.expect.rowCount);
|
||||
if (!rowCountResult.matched) {
|
||||
return {
|
||||
durationMs,
|
||||
@@ -153,6 +162,21 @@ export class DbChecker implements CheckerDefinition<ResolvedDbTarget> {
|
||||
}
|
||||
}
|
||||
|
||||
if (t.expect?.result && t.expect.result.length > 0) {
|
||||
const rowCount = isArray(rows) ? rows.length : 0;
|
||||
const resultCheck = checkContentRules({ rowCount, rows }, t.expect.result, { path: "result", phase: "result" });
|
||||
if (!resultCheck.matched) {
|
||||
return {
|
||||
durationMs,
|
||||
failure: resultCheck.failure,
|
||||
matched: false,
|
||||
statusDetail: `${rowCount} rows`,
|
||||
targetId: t.id,
|
||||
timestamp,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
durationMs,
|
||||
failure: null,
|
||||
|
||||
@@ -1,25 +1,21 @@
|
||||
import { isPlainObject } from "es-toolkit";
|
||||
import { isArray } from "es-toolkit/compat";
|
||||
|
||||
import type { ExpectResult } from "../../expect/types";
|
||||
import type { ExpectOperator, ExpectValue } from "../../types";
|
||||
import type { ExpectResult, KeyValueExpect, ValueMatcher } from "../../expect/types";
|
||||
|
||||
import { mismatchFailure } from "../../expect/failure";
|
||||
import { checkExpectValue } from "../../expect/operator";
|
||||
import { checkKeyValueExpect } from "../../expect/key-value";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
|
||||
export function checkRowCount(rows: unknown, op: ExpectOperator): ExpectResult {
|
||||
const actual = isArray(rows) ? rows.length : 0;
|
||||
const matched = checkExpectValue(actual, op);
|
||||
if (!matched) {
|
||||
return {
|
||||
failure: mismatchFailure("rowCount", "rowCount", op, actual, `rowCount ${actual} 不满足条件`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
export function checkRowCount(actual: number, matcher: ValueMatcher): ExpectResult {
|
||||
return checkValueMatcher(actual, matcher, {
|
||||
message: `rowCount ${actual} 不满足条件`,
|
||||
path: "rowCount",
|
||||
phase: "rowCount",
|
||||
});
|
||||
}
|
||||
|
||||
export function checkRows(rows: unknown, rules: Array<Record<string, ExpectValue>>): ExpectResult {
|
||||
export function checkRows(rows: unknown, rules: KeyValueExpect[]): ExpectResult {
|
||||
if (!isArray(rows)) {
|
||||
return {
|
||||
failure: mismatchFailure("row", "rows", rules, rows, "查询结果不是数组"),
|
||||
@@ -44,16 +40,8 @@ export function checkRows(rows: unknown, rules: Array<Record<string, ExpectValue
|
||||
};
|
||||
}
|
||||
|
||||
for (const [col, expected] of Object.entries(rule)) {
|
||||
const actual = row[col];
|
||||
const matched = checkExpectValue(actual, expected);
|
||||
if (!matched) {
|
||||
return {
|
||||
failure: mismatchFailure("row", `rows[${i}].${col}`, expected, actual, `rows[${i}].${col} mismatch`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
}
|
||||
const result = checkKeyValueExpect(row, rule, { path: `rows[${i}]`, phase: "row" });
|
||||
if (!result.matched) return result;
|
||||
}
|
||||
|
||||
return { failure: null, matched: true };
|
||||
|
||||
@@ -2,10 +2,7 @@ import { Type } from "@sinclair/typebox";
|
||||
|
||||
import type { CheckerSchemas } from "../types";
|
||||
|
||||
import { createPureOperatorSchema, jsonValueSchema, operatorProperties } from "../../schema/fragments";
|
||||
|
||||
// Db expect 允许行对象中的列值为字面量或 operator
|
||||
const dbRowValueSchema = Type.Union([jsonValueSchema, createPureOperatorSchema()]);
|
||||
import { createContentRulesSchema, createKeyValueExpectSchema, createValueMatcherSchema } from "../../schema/fragments";
|
||||
|
||||
export const dbCheckerSchemas: CheckerSchemas = {
|
||||
config: Type.Object(
|
||||
@@ -22,31 +19,11 @@ export const dbCheckerSchemas: CheckerSchemas = {
|
||||
defaults: Type.Object({}, { additionalProperties: false }),
|
||||
expect: Type.Object(
|
||||
{
|
||||
maxDurationMs: Type.Optional(Type.Number({ minimum: 0 })),
|
||||
rowCount: Type.Optional(createPureOperatorSchema()),
|
||||
rows: Type.Optional(
|
||||
Type.Array(
|
||||
Type.Record(Type.String(), dbRowValueSchema, {
|
||||
additionalProperties: false,
|
||||
minProperties: 1,
|
||||
}),
|
||||
),
|
||||
),
|
||||
durationMs: Type.Optional(createValueMatcherSchema()),
|
||||
result: Type.Optional(createContentRulesSchema()),
|
||||
rowCount: Type.Optional(createValueMatcherSchema()),
|
||||
rows: Type.Optional(Type.Array(createKeyValueExpectSchema())),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
};
|
||||
|
||||
// 导出用于 validate 的辅助类型
|
||||
export const DbOperatorKeys = new Set<string>([
|
||||
...Object.keys(operatorProperties()),
|
||||
"contains",
|
||||
"empty",
|
||||
"equals",
|
||||
"exists",
|
||||
"gt",
|
||||
"gte",
|
||||
"lt",
|
||||
"lte",
|
||||
"match",
|
||||
]);
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
import type { ExpectOperator, ExpectValue, ResolvedTargetBase } from "../../types";
|
||||
import type { ContentRules, KeyValueExpect, ValueMatcher } from "../../expect/types";
|
||||
import type { ResolvedTargetBase } from "../../types";
|
||||
|
||||
export interface DbExpectConfig {
|
||||
maxDurationMs?: number;
|
||||
rowCount?: ExpectOperator;
|
||||
rows?: Array<Record<string, ExpectValue>>;
|
||||
durationMs?: ValueMatcher;
|
||||
result?: ContentRules;
|
||||
rowCount?: ValueMatcher;
|
||||
rows?: KeyValueExpect[];
|
||||
}
|
||||
|
||||
export interface DbTargetConfig {
|
||||
|
||||
@@ -1,11 +1,10 @@
|
||||
import { isNumber, isPlainObject, isString } from "es-toolkit";
|
||||
import { isPlainObject, isString } from "es-toolkit";
|
||||
import { isArray } from "es-toolkit/compat";
|
||||
|
||||
import type { ConfigValidationIssue } from "../../schema/issues";
|
||||
import type { CheckerValidationInput } from "../types";
|
||||
|
||||
import { isUnsafeRegex } from "../../expect/redos";
|
||||
import { validateOperatorObject } from "../../expect/validate-operator";
|
||||
import { validateContentRules, validateKeyValueExpect, validateValueMatcher } from "../../expect/validate-matcher";
|
||||
import { issue, joinPath } from "../../schema/issues";
|
||||
|
||||
export function validateDbConfig(input: CheckerValidationInput): ConfigValidationIssue[] {
|
||||
@@ -21,7 +20,7 @@ export function validateDbConfig(input: CheckerValidationInput): ConfigValidatio
|
||||
return issues;
|
||||
}
|
||||
|
||||
function collectRowOperators(rows: unknown[], path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
function collectRowExpects(rows: unknown[], path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
for (let i = 0; i < rows.length; i++) {
|
||||
const row = rows[i]!;
|
||||
@@ -29,28 +28,7 @@ function collectRowOperators(rows: unknown[], path: string, targetName?: string)
|
||||
issues.push(issue("invalid-type", `${path}[${i}]`, "必须为对象", targetName));
|
||||
continue;
|
||||
}
|
||||
for (const [col, value] of Object.entries(row)) {
|
||||
const colPath = `${path}[${i}].${col}`;
|
||||
if (isPlainObject(value) && Object.keys(value).some((k) => k === "match")) {
|
||||
// 检查 match 正则
|
||||
const valueRecord = value as Record<string, unknown>;
|
||||
const match: unknown = valueRecord["match"];
|
||||
if (isString(match)) {
|
||||
try {
|
||||
new RegExp(match);
|
||||
} catch {
|
||||
issues.push(issue("invalid-regex", colPath, "正则不合法", targetName));
|
||||
}
|
||||
if (isUnsafeRegex(match)) {
|
||||
issues.push(issue("unsafe-regex", colPath, "正则存在 ReDoS 风险", targetName));
|
||||
}
|
||||
}
|
||||
}
|
||||
// 校验 operator 对象
|
||||
if (isPlainObject(value)) {
|
||||
issues.push(...validateOperatorObject(value, colPath, targetName, { requireAtLeastOne: false }));
|
||||
}
|
||||
}
|
||||
issues.push(...validateKeyValueExpect(row, `${path}[${i}]`, targetName));
|
||||
}
|
||||
return issues;
|
||||
}
|
||||
@@ -60,10 +38,6 @@ function getTargetName(target: Record<string, unknown>): string | undefined {
|
||||
return isString(target["id"]) ? target["id"] : undefined;
|
||||
}
|
||||
|
||||
function isNonNegativeFiniteNumber(value: unknown): boolean {
|
||||
return isNumber(value) && Number.isFinite(value) && value >= 0;
|
||||
}
|
||||
|
||||
function validateDbExpect(target: Record<string, unknown>, path: string): ConfigValidationIssue[] {
|
||||
const targetName = getTargetName(target);
|
||||
const expect = target["expect"];
|
||||
@@ -71,24 +45,28 @@ function validateDbExpect(target: Record<string, unknown>, path: string): Config
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const expectPath = joinPath(path, "expect");
|
||||
|
||||
if (expect["maxDurationMs"] !== undefined && !isNonNegativeFiniteNumber(expect["maxDurationMs"])) {
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, "maxDurationMs"), "必须为非负有限数字", targetName));
|
||||
if (expect["durationMs"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
}
|
||||
|
||||
if (expect["rowCount"] !== undefined) {
|
||||
issues.push(...validateOperatorObject(expect["rowCount"], joinPath(expectPath, "rowCount"), targetName));
|
||||
issues.push(...validateValueMatcher(expect["rowCount"], joinPath(expectPath, "rowCount"), targetName));
|
||||
}
|
||||
|
||||
if (expect["rows"] !== undefined) {
|
||||
if (!isArray(expect["rows"])) {
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, "rows"), "必须为数组", targetName));
|
||||
} else {
|
||||
issues.push(...collectRowOperators(expect["rows"], joinPath(expectPath, "rows"), targetName));
|
||||
issues.push(...collectRowExpects(expect["rows"], joinPath(expectPath, "rows"), targetName));
|
||||
}
|
||||
}
|
||||
|
||||
if (expect["result"] !== undefined) {
|
||||
issues.push(...validateContentRules(expect["result"], joinPath(expectPath, "result"), targetName));
|
||||
}
|
||||
|
||||
// 检查未知字段
|
||||
const allowedKeys = new Set(["maxDurationMs", "rowCount", "rows"]);
|
||||
const allowedKeys = new Set(["durationMs", "result", "rowCount", "rows"]);
|
||||
for (const key of Object.keys(expect)) {
|
||||
if (!allowedKeys.has(key)) {
|
||||
issues.push(issue("unknown-field", joinPath(expectPath, key), "是未知字段", targetName));
|
||||
|
||||
@@ -1,212 +0,0 @@
|
||||
import { DOMParser } from "@xmldom/xmldom";
|
||||
import * as cheerio from "cheerio";
|
||||
import { isArray } from "es-toolkit/compat";
|
||||
import * as xpath from "xpath";
|
||||
|
||||
import type { ExpectResult } from "../../expect/types";
|
||||
import type { BodyRule, CssRule, JsonRule, XpathRule } from "./types";
|
||||
|
||||
import { errorFailure, mismatchFailure } from "../../expect/failure";
|
||||
import { applyOperator, evaluateJsonPath } from "../../expect/operator";
|
||||
|
||||
type ParsedJsonResult = { error: string; ok: false } | { ok: true; value: unknown };
|
||||
|
||||
export function checkBodyExpect(body: string, rules?: BodyRule[]): ExpectResult {
|
||||
if (!rules || rules.length === 0) return { failure: null, matched: true };
|
||||
|
||||
let parsedJson: ParsedJsonResult | undefined;
|
||||
|
||||
for (let i = 0; i < rules.length; i++) {
|
||||
const rule = rules[i]!;
|
||||
if ("json" in rule && parsedJson === undefined) {
|
||||
parsedJson = parseJsonBody(body);
|
||||
}
|
||||
|
||||
const result = checkSingleBodyRule(body, rule, i, parsedJson);
|
||||
if (!result.matched) return result;
|
||||
}
|
||||
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
function checkCssRule(body: string, rule: CssRule, rulePath: string): ExpectResult {
|
||||
const { attr, selector, ...operators } = rule;
|
||||
const fullPath = `${rulePath}.css(${selector}${attr ? `@${attr}` : ""})`;
|
||||
|
||||
let $: cheerio.CheerioAPI;
|
||||
try {
|
||||
$ = cheerio.load(body);
|
||||
} catch {
|
||||
return {
|
||||
failure: errorFailure("body", fullPath, "failed to parse HTML"),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
|
||||
const el = $(selector);
|
||||
|
||||
if (operators.exists === false) {
|
||||
if (el.length > 0) {
|
||||
return {
|
||||
failure: mismatchFailure("body", fullPath, false, true, `selector ${selector} exists`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
if (el.length === 0) {
|
||||
const expected = operators.exists === true ? true : "element found";
|
||||
const actual = operators.exists === true ? false : "no match";
|
||||
return {
|
||||
failure: mismatchFailure("body", fullPath, expected, actual, `selector ${selector} not found`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
|
||||
if (operators.exists === true) return { failure: null, matched: true };
|
||||
|
||||
const actual = attr ? el.attr(attr) : el.text();
|
||||
const opKeys = Object.keys(operators);
|
||||
if (opKeys.length === 0) {
|
||||
if (actual === undefined) {
|
||||
return {
|
||||
failure: mismatchFailure("body", fullPath, "attr present", undefined, `attribute ${attr} not found`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
const matched = applyOperator(actual ?? "", operators);
|
||||
if (!matched) {
|
||||
return {
|
||||
failure: mismatchFailure("body", fullPath, operators, actual, `css selector ${selector} mismatch`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
function checkJsonRule(body: string, rule: JsonRule, rulePath: string, parsedJson?: ParsedJsonResult): ExpectResult {
|
||||
const { path, ...operators } = rule;
|
||||
const fullPath = `${rulePath}.json(${path})`;
|
||||
|
||||
const jsonResult = parsedJson ?? parseJsonBody(body);
|
||||
if (!jsonResult.ok) {
|
||||
return {
|
||||
failure: errorFailure("body", fullPath, jsonResult.error),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
|
||||
const actual = evaluateJsonPath(jsonResult.value, path);
|
||||
const opKeys = Object.keys(operators);
|
||||
|
||||
if (opKeys.length === 0) {
|
||||
if (actual === undefined) {
|
||||
return {
|
||||
failure: mismatchFailure("body", fullPath, "defined", actual, `path ${path} is undefined`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
const matched = applyOperator(actual, operators);
|
||||
if (!matched) {
|
||||
return {
|
||||
failure: mismatchFailure("body", fullPath, operators, actual, `json path ${path} mismatch`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
function checkSingleBodyRule(body: string, rule: BodyRule, index: number, parsedJson?: ParsedJsonResult): ExpectResult {
|
||||
const rulePath = `body[${index}]`;
|
||||
|
||||
if ("contains" in rule) {
|
||||
const matched = body.includes(rule.contains);
|
||||
if (!matched) {
|
||||
return {
|
||||
failure: mismatchFailure("body", rulePath, rule.contains, body, `body does not contain "${rule.contains}"`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
if ("regex" in rule) {
|
||||
const matched = new RegExp(rule.regex).test(body);
|
||||
if (!matched) {
|
||||
return {
|
||||
failure: mismatchFailure("body", rulePath, `/${rule.regex}/`, body, `body does not match /${rule.regex}/`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
if ("json" in rule) {
|
||||
return checkJsonRule(body, rule.json, rulePath, parsedJson);
|
||||
}
|
||||
|
||||
if ("css" in rule) {
|
||||
return checkCssRule(body, rule.css, rulePath);
|
||||
}
|
||||
|
||||
if ("xpath" in rule) {
|
||||
return checkXpathRule(body, rule.xpath, rulePath);
|
||||
}
|
||||
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
function checkXpathRule(body: string, rule: XpathRule, rulePath: string): ExpectResult {
|
||||
const { path, ...operators } = rule;
|
||||
const fullPath = `${rulePath}.xpath(${path})`;
|
||||
|
||||
let doc: ReturnType<DOMParser["parseFromString"]>;
|
||||
try {
|
||||
doc = new DOMParser().parseFromString(body, "text/xml");
|
||||
} catch {
|
||||
return {
|
||||
failure: errorFailure("body", fullPath, "failed to parse XML/HTML"),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
|
||||
const nodes = xpath.select(path, doc as unknown as Node);
|
||||
if (!nodes || !isArray(nodes) || nodes.length === 0) {
|
||||
return {
|
||||
failure: mismatchFailure("body", fullPath, "node found", "no match", `xpath ${path} not found`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
|
||||
const node = nodes[0]!;
|
||||
const actual = node.nodeValue ?? (node as unknown as Element).textContent ?? "";
|
||||
const opKeys = Object.keys(operators);
|
||||
|
||||
if (opKeys.length === 0) {
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
const matched = applyOperator(actual, operators);
|
||||
if (!matched) {
|
||||
return {
|
||||
failure: mismatchFailure("body", fullPath, operators, actual, `xpath ${path} mismatch`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
function parseJsonBody(body: string): ParsedJsonResult {
|
||||
try {
|
||||
return { ok: true, value: JSON.parse(body) as unknown };
|
||||
} catch {
|
||||
return { error: "body is not valid JSON", ok: false };
|
||||
}
|
||||
}
|
||||
@@ -5,10 +5,10 @@ import type { CheckResult, RawTargetConfig } from "../../types";
|
||||
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
|
||||
import type { HttpExpectConfig, HttpTargetConfig, ResolvedHttpTarget } from "./types";
|
||||
|
||||
import { checkDuration } from "../../expect/duration";
|
||||
import { checkContentRules } from "../../expect/content";
|
||||
import { errorFailure, mismatchFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { parseSize } from "../../utils";
|
||||
import { checkBodyExpect } from "./body";
|
||||
import { checkHeaders, checkStatus } from "./expect";
|
||||
import { httpCheckerSchemas } from "./schema";
|
||||
import { validateHttpConfig } from "./validate";
|
||||
@@ -54,7 +54,7 @@ export class HttpChecker implements CheckerDefinition<ResolvedHttpTarget> {
|
||||
|
||||
const hasBodyRules = !!(expect?.body && expect.body.length > 0);
|
||||
|
||||
const earlyTimeout = hasBodyRules ? checkEarlyTimeout(start, expect?.maxDurationMs) : null;
|
||||
const earlyTimeout = hasBodyRules ? checkEarlyTimeout(start, expect?.durationMs) : null;
|
||||
if (earlyTimeout) {
|
||||
return makeResult(t, timestamp, earlyTimeout.elapsed, earlyTimeout.failure, statusCode);
|
||||
}
|
||||
@@ -70,14 +70,18 @@ export class HttpChecker implements CheckerDefinition<ResolvedHttpTarget> {
|
||||
return makeResult(t, timestamp, performance.now() - start, decodeResult.failure, statusCode);
|
||||
}
|
||||
|
||||
const bodyResult = checkBodyExpect(decodeResult.text, expect.body);
|
||||
const bodyResult = checkContentRules(decodeResult.text, expect.body, { path: "body", phase: "body" });
|
||||
if (!bodyResult.matched) {
|
||||
return makeResult(t, timestamp, performance.now() - start, bodyResult.failure, statusCode);
|
||||
}
|
||||
}
|
||||
|
||||
const durationMs = Math.round(performance.now() - start);
|
||||
const durationResult = checkDuration(durationMs, expect?.maxDurationMs);
|
||||
const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
});
|
||||
if (!durationResult.matched) {
|
||||
return makeResult(t, timestamp, durationMs, durationResult.failure, statusCode);
|
||||
}
|
||||
@@ -190,23 +194,29 @@ function buildRedirectInit(init: RequestInit, statusCode: number, fromUrl: strin
|
||||
|
||||
function checkEarlyTimeout(
|
||||
start: number,
|
||||
maxDurationMs: number | undefined,
|
||||
durationMatcher: HttpExpectConfig["durationMs"] | undefined,
|
||||
): null | { elapsed: number; failure: CheckResult["failure"] } {
|
||||
if (maxDurationMs === undefined) return null;
|
||||
if (durationMatcher === undefined) return null;
|
||||
const limit = Math.min(
|
||||
durationMatcher.lte ?? Number.POSITIVE_INFINITY,
|
||||
durationMatcher.lt ?? Number.POSITIVE_INFINITY,
|
||||
);
|
||||
if (!Number.isFinite(limit)) return null;
|
||||
|
||||
const elapsed = performance.now() - start;
|
||||
if (elapsed <= maxDurationMs) return null;
|
||||
if (durationMatcher.lt !== undefined ? elapsed < limit : elapsed <= limit) return null;
|
||||
|
||||
const durationMs = Math.round(elapsed);
|
||||
const durationResult = checkValueMatcher(durationMs, durationMatcher, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
});
|
||||
return {
|
||||
elapsed,
|
||||
failure: mismatchFailure(
|
||||
"duration",
|
||||
"duration",
|
||||
`<=${maxDurationMs}ms`,
|
||||
durationMs,
|
||||
`duration ${durationMs}ms > ${maxDurationMs}ms`,
|
||||
),
|
||||
failure:
|
||||
durationResult.failure ??
|
||||
mismatchFailure("duration", "durationMs", durationMatcher, durationMs, "durationMs mismatch"),
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -1,48 +1,16 @@
|
||||
import { isNumber, isString } from "es-toolkit";
|
||||
import { isNumber } from "es-toolkit";
|
||||
|
||||
import type { ExpectResult } from "../../expect/types";
|
||||
import type { HeaderExpect } from "./types";
|
||||
import type { ExpectResult, KeyValueExpect } from "../../expect/types";
|
||||
|
||||
import { mismatchFailure } from "../../expect/failure";
|
||||
import { applyOperator } from "../../expect/operator";
|
||||
import { checkKeyValueExpect } from "../../expect/key-value";
|
||||
|
||||
export function checkHeaders(
|
||||
headers: Record<string, string>,
|
||||
headerExpects?: Record<string, HeaderExpect>,
|
||||
): ExpectResult {
|
||||
if (!headerExpects) return { failure: null, matched: true };
|
||||
|
||||
for (const [key, expected] of Object.entries(headerExpects)) {
|
||||
const actualValue = headers[key.toLowerCase()];
|
||||
const path = `headers.${key}`;
|
||||
|
||||
if (isString(expected)) {
|
||||
if (actualValue !== expected) {
|
||||
return {
|
||||
failure: mismatchFailure("headers", path, expected, actualValue, `header ${key} mismatch`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
} else {
|
||||
if (actualValue === undefined) {
|
||||
if (expected.exists !== false) {
|
||||
return {
|
||||
failure: mismatchFailure("headers", path, "defined", undefined, `header ${key} not found`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (!applyOperator(actualValue, expected)) {
|
||||
return {
|
||||
failure: mismatchFailure("headers", path, expected, actualValue, `header ${key} mismatch`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return { failure: null, matched: true };
|
||||
export function checkHeaders(headers: Record<string, string>, headerExpects?: KeyValueExpect): ExpectResult {
|
||||
return checkKeyValueExpect(headers, headerExpects, {
|
||||
normalizeKey: (key) => key.toLowerCase(),
|
||||
path: "headers",
|
||||
phase: "headers",
|
||||
});
|
||||
}
|
||||
|
||||
export function checkStatus(statusCode: number, allowed: Array<number | string>): ExpectResult {
|
||||
|
||||
@@ -3,8 +3,9 @@ import { Type } from "@sinclair/typebox";
|
||||
import type { CheckerSchemas } from "../types";
|
||||
|
||||
import {
|
||||
createBodyRulesSchema,
|
||||
createHeaderExpectSchema,
|
||||
createContentRulesSchema,
|
||||
createKeyValueExpectSchema,
|
||||
createValueMatcherSchema,
|
||||
httpMethodSchema,
|
||||
sizeSchema,
|
||||
statusCodePatternSchema,
|
||||
@@ -33,9 +34,9 @@ export const httpCheckerSchemas: CheckerSchemas = {
|
||||
),
|
||||
expect: Type.Object(
|
||||
{
|
||||
body: Type.Optional(createBodyRulesSchema()),
|
||||
headers: Type.Optional(createHeaderExpectSchema()),
|
||||
maxDurationMs: Type.Optional(Type.Number({ minimum: 0 })),
|
||||
body: Type.Optional(createContentRulesSchema()),
|
||||
durationMs: Type.Optional(createValueMatcherSchema()),
|
||||
headers: Type.Optional(createKeyValueExpectSchema()),
|
||||
status: Type.Optional(Type.Array(statusCodePatternSchema)),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
|
||||
@@ -1,15 +1,5 @@
|
||||
import type { ExpectOperator, ResolvedTargetBase } from "../../types";
|
||||
|
||||
export type BodyRule =
|
||||
| { contains: string }
|
||||
| { css: CssRule }
|
||||
| { json: JsonRule }
|
||||
| { regex: string }
|
||||
| { xpath: XpathRule };
|
||||
|
||||
export type CssRule = ExpectOperator & { attr?: string; selector: string };
|
||||
|
||||
export type HeaderExpect = ExpectOperator | string;
|
||||
import type { ContentRules, KeyValueExpect, ValueMatcher } from "../../expect/types";
|
||||
import type { ResolvedTargetBase } from "../../types";
|
||||
|
||||
export interface HttpDefaultsConfig {
|
||||
headers?: Record<string, string>;
|
||||
@@ -18,9 +8,9 @@ export interface HttpDefaultsConfig {
|
||||
}
|
||||
|
||||
export interface HttpExpectConfig {
|
||||
body?: BodyRule[];
|
||||
headers?: Record<string, HeaderExpect>;
|
||||
maxDurationMs?: number;
|
||||
body?: ContentRules;
|
||||
durationMs?: ValueMatcher;
|
||||
headers?: KeyValueExpect;
|
||||
status?: Array<number | string>;
|
||||
}
|
||||
|
||||
@@ -34,8 +24,6 @@ export interface HttpTargetConfig {
|
||||
url: string;
|
||||
}
|
||||
|
||||
export type JsonRule = ExpectOperator & { path: string };
|
||||
|
||||
export interface ResolvedHttpConfig {
|
||||
body?: string;
|
||||
headers: Record<string, string>;
|
||||
@@ -55,5 +43,3 @@ export interface ResolvedHttpTarget extends ResolvedTargetBase {
|
||||
timeoutMs: number;
|
||||
type: "http";
|
||||
}
|
||||
|
||||
export type XpathRule = ExpectOperator & { path: string };
|
||||
|
||||
@@ -1,24 +1,19 @@
|
||||
import { DOMParser } from "@xmldom/xmldom";
|
||||
import { isNumber, isString } from "es-toolkit";
|
||||
import { isArray } from "es-toolkit/compat";
|
||||
import * as xpath from "xpath";
|
||||
|
||||
import type { ConfigValidationIssue } from "../../schema/issues";
|
||||
import type { CheckerValidationInput } from "../types";
|
||||
|
||||
import { isUnsafeRegex } from "../../expect/redos";
|
||||
import { isPlainRecord, validateOperatorObject } from "../../expect/validate-operator";
|
||||
import { BodyRuleTypeKeys, OperatorKeys } from "../../schema/fragments";
|
||||
import {
|
||||
isPlainRecord,
|
||||
validateContentRules,
|
||||
validateKeyValueExpect,
|
||||
validateValueMatcher,
|
||||
} from "../../expect/validate-matcher";
|
||||
import { issue, joinPath } from "../../schema/issues";
|
||||
import { parseSize } from "../../utils";
|
||||
|
||||
const ALLOWED_PROTOCOLS = new Set(["http:", "https:"]);
|
||||
const OPERATOR_KEY_SET = new Set<string>(OperatorKeys);
|
||||
|
||||
export function validateBodyRules(body: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isArray(body)) return [issue("invalid-type", path, "必须为数组", targetName)];
|
||||
return body.flatMap((rule, index) => validateSingleBodyRule(rule, `${path}[${index}]`, targetName));
|
||||
}
|
||||
|
||||
export function validateHttpConfig(input: CheckerValidationInput): ConfigValidationIssue[] {
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
@@ -57,55 +52,15 @@ export function validateJsonPath(path: string, rulePath: string, targetName?: st
|
||||
return issues;
|
||||
}
|
||||
|
||||
function collectOperatorObject(
|
||||
object: Record<string, unknown>,
|
||||
allowedKeys: Set<string>,
|
||||
path: string,
|
||||
targetName?: string,
|
||||
): { issues: ConfigValidationIssue[]; operators: Record<string, unknown> } {
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const operators: Record<string, unknown> = {};
|
||||
for (const [key, value] of Object.entries(object)) {
|
||||
if (allowedKeys.has(key)) continue;
|
||||
if (OPERATOR_KEY_SET.has(key)) {
|
||||
operators[key] = value;
|
||||
} else {
|
||||
issues.push(issue("unknown-field", joinPath(path, key), "是未知字段", targetName));
|
||||
}
|
||||
}
|
||||
return { issues, operators };
|
||||
}
|
||||
|
||||
function getTargetName(target: Record<string, unknown>): string | undefined {
|
||||
if (isString(target["name"])) return target["name"];
|
||||
return isString(target["id"]) ? target["id"] : undefined;
|
||||
}
|
||||
|
||||
function isNonNegativeFiniteNumber(value: unknown): boolean {
|
||||
return isNumber(value) && Number.isFinite(value) && value >= 0;
|
||||
}
|
||||
|
||||
function isSizeInput(value: unknown): value is number | string {
|
||||
return isNumber(value) || isString(value);
|
||||
}
|
||||
|
||||
function validateCssRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
if (!isString(rule["selector"]) || rule["selector"].trim() === "") {
|
||||
issues.push(issue("invalid-type", joinPath(path, "selector"), "必须为非空字符串", targetName));
|
||||
}
|
||||
if ("attr" in rule && !isString(rule["attr"])) {
|
||||
issues.push(issue("invalid-type", joinPath(path, "attr"), "必须为字符串", targetName));
|
||||
}
|
||||
const result = collectOperatorObject(rule, new Set(["attr", "selector"]), path, targetName);
|
||||
issues.push(
|
||||
...result.issues,
|
||||
...validateOperatorObject(result.operators, path, targetName, { requireAtLeastOne: false }),
|
||||
);
|
||||
return issues;
|
||||
}
|
||||
|
||||
function validateHttpExpect(target: Record<string, unknown>, path: string): ConfigValidationIssue[] {
|
||||
const targetName = getTargetName(target);
|
||||
const expect = target["expect"];
|
||||
@@ -114,22 +69,19 @@ function validateHttpExpect(target: Record<string, unknown>, path: string): Conf
|
||||
const expectPath = joinPath(path, "expect");
|
||||
|
||||
if (isPlainRecord(expect["headers"])) {
|
||||
for (const [key, value] of Object.entries(expect["headers"])) {
|
||||
if (isString(value)) continue;
|
||||
issues.push(...validateOperatorObject(value, joinPath(joinPath(expectPath, "headers"), key), targetName));
|
||||
}
|
||||
issues.push(...validateKeyValueExpect(expect["headers"], joinPath(expectPath, "headers"), targetName));
|
||||
}
|
||||
|
||||
if (expect["body"] !== undefined) {
|
||||
issues.push(...validateBodyRules(expect["body"], joinPath(expectPath, "body"), targetName));
|
||||
issues.push(...validateContentRules(expect["body"], joinPath(expectPath, "body"), targetName));
|
||||
}
|
||||
|
||||
if (isArray(expect["status"])) {
|
||||
issues.push(...validateStatusValues(expect["status"], joinPath(expectPath, "status"), targetName));
|
||||
}
|
||||
|
||||
if (expect["maxDurationMs"] !== undefined && !isNonNegativeFiniteNumber(expect["maxDurationMs"])) {
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, "maxDurationMs"), "必须为非负有限数字", targetName));
|
||||
if (expect["durationMs"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
}
|
||||
|
||||
return issues;
|
||||
@@ -172,61 +124,6 @@ function validateHttpTarget(target: Record<string, unknown>, path: string): Conf
|
||||
return issues;
|
||||
}
|
||||
|
||||
function validateJsonRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
if (!isString(rule["path"])) {
|
||||
issues.push(issue("invalid-type", joinPath(path, "path"), "必须为字符串", targetName));
|
||||
} else {
|
||||
issues.push(...validateJsonPath(rule["path"], path, targetName));
|
||||
}
|
||||
const result = collectOperatorObject(rule, new Set(["path"]), path, targetName);
|
||||
issues.push(
|
||||
...result.issues,
|
||||
...validateOperatorObject(result.operators, path, targetName, { requireAtLeastOne: false }),
|
||||
);
|
||||
return issues;
|
||||
}
|
||||
|
||||
function validateRegexRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isString(rule)) return [issue("invalid-type", path, "必须为字符串", targetName)];
|
||||
try {
|
||||
new RegExp(rule);
|
||||
} catch {
|
||||
return [issue("invalid-regex", path, "正则不合法", targetName)];
|
||||
}
|
||||
return isUnsafeRegex(rule) ? [issue("unsafe-regex", path, "正则存在 ReDoS 风险", targetName)] : [];
|
||||
}
|
||||
|
||||
function validateSingleBodyRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
const found = BodyRuleTypeKeys.filter((type) => type in rule);
|
||||
if (found.length === 0) return [issue("missing-body-rule", path, "缺少支持的规则类型", targetName)];
|
||||
if (found.length > 1) return [issue("multiple-body-rules", path, "只能配置一种规则类型", targetName)];
|
||||
|
||||
const ruleType = found[0]!;
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
for (const key of Object.keys(rule)) {
|
||||
if (key !== ruleType) issues.push(issue("unknown-field", joinPath(path, key), "是未知字段", targetName));
|
||||
}
|
||||
if (issues.length > 0) return issues;
|
||||
|
||||
switch (ruleType) {
|
||||
case "contains":
|
||||
return isString(rule["contains"])
|
||||
? []
|
||||
: [issue("invalid-type", joinPath(path, "contains"), "必须为字符串", targetName)];
|
||||
case "css":
|
||||
return validateCssRule(rule["css"], joinPath(path, "css"), targetName);
|
||||
case "json":
|
||||
return validateJsonRule(rule["json"], joinPath(path, "json"), targetName);
|
||||
case "regex":
|
||||
return validateRegexRule(rule["regex"], joinPath(path, "regex"), targetName);
|
||||
case "xpath":
|
||||
return validateXpathRule(rule["xpath"], joinPath(path, "xpath"), targetName);
|
||||
}
|
||||
}
|
||||
|
||||
function validateSizeValue(value: number | string, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
try {
|
||||
parseSize(value);
|
||||
@@ -257,24 +154,3 @@ function validateStatusValues(values: unknown[], path: string, targetName?: stri
|
||||
}
|
||||
return issues;
|
||||
}
|
||||
|
||||
function validateXpathRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
if (!isString(rule["path"]) || rule["path"].trim() === "") {
|
||||
issues.push(issue("invalid-type", joinPath(path, "path"), "必须为非空字符串", targetName));
|
||||
} else {
|
||||
try {
|
||||
const doc = new DOMParser().parseFromString("<x/>", "text/xml");
|
||||
xpath.select(rule["path"], doc as unknown as Node);
|
||||
} catch {
|
||||
issues.push(issue("invalid-xpath", joinPath(path, "path"), "xpath 不合法", targetName));
|
||||
}
|
||||
}
|
||||
const result = collectOperatorObject(rule, new Set(["path"]), path, targetName);
|
||||
issues.push(
|
||||
...result.issues,
|
||||
...validateOperatorObject(result.operators, path, targetName, { requireAtLeastOne: false }),
|
||||
);
|
||||
return issues;
|
||||
}
|
||||
|
||||
@@ -4,8 +4,8 @@ import type { CheckResult, RawTargetConfig } from "../../types";
|
||||
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
|
||||
import type { PingExpectConfig, PingStats, PingTargetConfig, ResolvedPingTarget } from "./types";
|
||||
|
||||
import { checkDuration } from "../../expect/duration";
|
||||
import { errorFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { buildPingCommand } from "./command";
|
||||
import { checkAlive, checkAvgLatency, checkMaxLatency, checkPacketLoss } from "./expect";
|
||||
import { parsePingOutput } from "./parse";
|
||||
@@ -140,13 +140,17 @@ function buildStatusDetail(stats: PingStats): string {
|
||||
function checkStats(stats: PingStats, expect: PingExpectConfig | undefined, durationMs: number) {
|
||||
const aliveResult = checkAlive(stats.alive, expect?.alive ?? true);
|
||||
if (!aliveResult.matched) return aliveResult;
|
||||
const packetLossResult = checkPacketLoss(stats.packetLoss, expect?.maxPacketLoss);
|
||||
const packetLossResult = checkPacketLoss(stats.packetLoss, expect?.packetLossPercent);
|
||||
if (!packetLossResult.matched) return packetLossResult;
|
||||
const avgLatencyResult = checkAvgLatency(stats.avgLatencyMs, expect?.maxAvgLatencyMs);
|
||||
const avgLatencyResult = checkAvgLatency(stats.avgLatencyMs, expect?.avgLatencyMs);
|
||||
if (!avgLatencyResult.matched) return avgLatencyResult;
|
||||
const maxLatencyResult = checkMaxLatency(stats.maxLatencyMs, expect?.maxMaxLatencyMs);
|
||||
const maxLatencyResult = checkMaxLatency(stats.maxLatencyMs, expect?.maxLatencyMs);
|
||||
if (!maxLatencyResult.matched) return maxLatencyResult;
|
||||
return checkDuration(durationMs, expect?.maxDurationMs);
|
||||
return checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
});
|
||||
}
|
||||
|
||||
function formatNumber(value: number): string {
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import type { ExpectResult } from "../../expect/types";
|
||||
import type { ExpectResult, ValueMatcher } from "../../expect/types";
|
||||
|
||||
import { mismatchFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
|
||||
export function checkAlive(actual: boolean, expected: boolean): ExpectResult {
|
||||
if (actual === expected) return { failure: null, matched: true };
|
||||
@@ -16,29 +17,26 @@ export function checkAlive(actual: boolean, expected: boolean): ExpectResult {
|
||||
};
|
||||
}
|
||||
|
||||
export function checkAvgLatency(actual: null | number, max: number | undefined): ExpectResult {
|
||||
if (max === undefined) return { failure: null, matched: true };
|
||||
if (actual !== null && actual <= max) return { failure: null, matched: true };
|
||||
return {
|
||||
failure: mismatchFailure("avgLatency", "avgLatencyMs", `<=${max}ms`, actual, `平均延迟超过 ${max}ms`),
|
||||
matched: false,
|
||||
};
|
||||
export function checkAvgLatency(actual: null | number, matcher: undefined | ValueMatcher): ExpectResult {
|
||||
return checkValueMatcher(actual, matcher, {
|
||||
message: "平均延迟不满足条件",
|
||||
path: "avgLatencyMs",
|
||||
phase: "avgLatency",
|
||||
});
|
||||
}
|
||||
|
||||
export function checkMaxLatency(actual: null | number, max: number | undefined): ExpectResult {
|
||||
if (max === undefined) return { failure: null, matched: true };
|
||||
if (actual !== null && actual <= max) return { failure: null, matched: true };
|
||||
return {
|
||||
failure: mismatchFailure("maxLatency", "maxLatencyMs", `<=${max}ms`, actual, `最大延迟超过 ${max}ms`),
|
||||
matched: false,
|
||||
};
|
||||
export function checkMaxLatency(actual: null | number, matcher: undefined | ValueMatcher): ExpectResult {
|
||||
return checkValueMatcher(actual, matcher, {
|
||||
message: "最大延迟不满足条件",
|
||||
path: "maxLatencyMs",
|
||||
phase: "maxLatency",
|
||||
});
|
||||
}
|
||||
|
||||
export function checkPacketLoss(actual: number, max: number | undefined): ExpectResult {
|
||||
if (max === undefined) return { failure: null, matched: true };
|
||||
if (actual <= max) return { failure: null, matched: true };
|
||||
return {
|
||||
failure: mismatchFailure("packetLoss", "packetLoss", `<=${max}%`, actual, `丢包率 ${actual}% > ${max}%`),
|
||||
matched: false,
|
||||
};
|
||||
export function checkPacketLoss(actual: number, matcher: undefined | ValueMatcher): ExpectResult {
|
||||
return checkValueMatcher(actual, matcher, {
|
||||
message: "丢包率不满足条件",
|
||||
path: "packetLossPercent",
|
||||
phase: "packetLoss",
|
||||
});
|
||||
}
|
||||
|
||||
@@ -2,6 +2,8 @@ import { Type } from "@sinclair/typebox";
|
||||
|
||||
import type { CheckerSchemas } from "../types";
|
||||
|
||||
import { createValueMatcherSchema } from "../../schema/fragments";
|
||||
|
||||
export const icmpCheckerSchemas: CheckerSchemas = {
|
||||
config: Type.Object(
|
||||
{
|
||||
@@ -15,10 +17,10 @@ export const icmpCheckerSchemas: CheckerSchemas = {
|
||||
expect: Type.Object(
|
||||
{
|
||||
alive: Type.Optional(Type.Boolean()),
|
||||
maxAvgLatencyMs: Type.Optional(Type.Number({ minimum: 0 })),
|
||||
maxDurationMs: Type.Optional(Type.Number({ minimum: 0 })),
|
||||
maxMaxLatencyMs: Type.Optional(Type.Number({ minimum: 0 })),
|
||||
maxPacketLoss: Type.Optional(Type.Number({ maximum: 100, minimum: 0 })),
|
||||
avgLatencyMs: Type.Optional(createValueMatcherSchema()),
|
||||
durationMs: Type.Optional(createValueMatcherSchema()),
|
||||
maxLatencyMs: Type.Optional(createValueMatcherSchema()),
|
||||
packetLossPercent: Type.Optional(createValueMatcherSchema()),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
import type { ValueMatcher } from "../../expect/types";
|
||||
import type { ResolvedTargetBase } from "../../types";
|
||||
|
||||
export interface PingExpectConfig {
|
||||
alive?: boolean;
|
||||
maxAvgLatencyMs?: number;
|
||||
maxDurationMs?: number;
|
||||
maxMaxLatencyMs?: number;
|
||||
maxPacketLoss?: number;
|
||||
avgLatencyMs?: ValueMatcher;
|
||||
durationMs?: ValueMatcher;
|
||||
maxLatencyMs?: ValueMatcher;
|
||||
packetLossPercent?: ValueMatcher;
|
||||
}
|
||||
|
||||
export interface PingStats {
|
||||
|
||||
@@ -3,6 +3,7 @@ import { isNumber, isPlainObject, isString } from "es-toolkit";
|
||||
import type { ConfigValidationIssue } from "../../schema/issues";
|
||||
import type { CheckerValidationInput } from "../types";
|
||||
|
||||
import { validateValueMatcher } from "../../expect/validate-matcher";
|
||||
import { issue, joinPath } from "../../schema/issues";
|
||||
|
||||
export function validatePingConfig(input: CheckerValidationInput): ConfigValidationIssue[] {
|
||||
@@ -37,10 +38,6 @@ function getTargetName(target: Record<string, unknown>): string | undefined {
|
||||
return isString(target["id"]) ? target["id"] : undefined;
|
||||
}
|
||||
|
||||
function isNonNegativeFiniteNumber(value: unknown): boolean {
|
||||
return isNumber(value) && Number.isFinite(value) && value >= 0;
|
||||
}
|
||||
|
||||
function validatePingExpect(target: Record<string, unknown>, path: string): ConfigValidationIssue[] {
|
||||
const rawExpect = target["expect"];
|
||||
if (rawExpect === undefined || rawExpect === null || !isPlainObject(rawExpect)) return [];
|
||||
@@ -52,19 +49,13 @@ function validatePingExpect(target: Record<string, unknown>, path: string): Conf
|
||||
if (expect["alive"] !== undefined && typeof expect["alive"] !== "boolean") {
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, "alive"), "必须为布尔值", targetName));
|
||||
}
|
||||
if (expect["maxPacketLoss"] !== undefined) {
|
||||
const value = expect["maxPacketLoss"];
|
||||
if (!isNumber(value) || !Number.isFinite(value) || value < 0 || value > 100) {
|
||||
issues.push(issue("invalid-value", joinPath(expectPath, "maxPacketLoss"), "必须为 0-100 的数字", targetName));
|
||||
}
|
||||
}
|
||||
for (const key of ["maxAvgLatencyMs", "maxMaxLatencyMs", "maxDurationMs"]) {
|
||||
if (expect[key] !== undefined && !isNonNegativeFiniteNumber(expect[key])) {
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, key), "必须为非负有限数字", targetName));
|
||||
for (const key of ["packetLossPercent", "avgLatencyMs", "maxLatencyMs", "durationMs"]) {
|
||||
if (expect[key] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect[key], joinPath(expectPath, key), targetName));
|
||||
}
|
||||
}
|
||||
|
||||
const allowedKeys = new Set(["alive", "maxAvgLatencyMs", "maxDurationMs", "maxMaxLatencyMs", "maxPacketLoss"]);
|
||||
const allowedKeys = new Set(["alive", "avgLatencyMs", "durationMs", "maxLatencyMs", "packetLossPercent"]);
|
||||
for (const key of Object.keys(expect)) {
|
||||
if (!allowedKeys.has(key)) {
|
||||
issues.push(issue("unknown-field", joinPath(expectPath, key), "是未知字段", targetName));
|
||||
|
||||
@@ -7,8 +7,8 @@ import type { CheckResult, RawTargetConfig } from "../../types";
|
||||
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
|
||||
import type { LlmCheckObservation, LlmExpectConfig, LlmTargetConfig, ResolvedLlmTarget } from "./types";
|
||||
|
||||
import { checkDuration } from "../../expect/duration";
|
||||
import { errorFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { runExpects } from "./expect";
|
||||
import {
|
||||
buildObservationFromApiCallError,
|
||||
@@ -54,7 +54,11 @@ export class LlmChecker implements CheckerDefinition<ResolvedLlmTarget> {
|
||||
};
|
||||
}
|
||||
|
||||
const durationResult = checkDuration(durationMs, expect?.maxDurationMs);
|
||||
const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
});
|
||||
const expectResult = runExpects(observation, expect);
|
||||
const failure = expectResult.failure ?? durationResult.failure;
|
||||
|
||||
@@ -209,7 +213,11 @@ export class LlmChecker implements CheckerDefinition<ResolvedLlmTarget> {
|
||||
);
|
||||
|
||||
const durationMs = Math.round(performance.now() - start);
|
||||
const durationResult = checkDuration(durationMs, expect?.maxDurationMs);
|
||||
const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
});
|
||||
const expectResult = runExpects(observation, expect);
|
||||
const failure = expectResult.failure ?? durationResult.failure;
|
||||
|
||||
@@ -251,7 +259,11 @@ export class LlmChecker implements CheckerDefinition<ResolvedLlmTarget> {
|
||||
);
|
||||
|
||||
const durationMs = Math.round(performance.now() - start);
|
||||
const durationResult = checkDuration(durationMs, expect?.maxDurationMs);
|
||||
const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
});
|
||||
const expectResult = runExpects(observation, expect);
|
||||
const failure = expectResult.failure ?? durationResult.failure;
|
||||
|
||||
|
||||
@@ -1,11 +1,10 @@
|
||||
import type { ExpectResult } from "../../expect/types";
|
||||
import type { LlmCheckObservation, LlmExpectConfig } from "./types";
|
||||
import type { LlmCheckObservation, LlmExpectConfig, LlmUsageExpect } from "./types";
|
||||
|
||||
import { checkDuration } from "../../expect/duration";
|
||||
import { checkContentRules } from "../../expect/content";
|
||||
import { mismatchFailure } from "../../expect/failure";
|
||||
import { applyOperator } from "../../expect/operator";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { checkHeaders, checkStatus } from "../http/expect";
|
||||
import { checkOutputRules } from "./output";
|
||||
|
||||
export function checkStreamExpect(observation: LlmCheckObservation, expect: LlmExpectConfig): ExpectResult {
|
||||
if (!observation.stream || !expect.stream) return { failure: null, matched: true };
|
||||
@@ -25,18 +24,11 @@ export function checkStreamExpect(observation: LlmCheckObservation, expect: LlmE
|
||||
}
|
||||
|
||||
if (expect.stream.firstTokenMs && observation.stream.firstTokenMs !== null) {
|
||||
if (!applyOperator(observation.stream.firstTokenMs, expect.stream.firstTokenMs)) {
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
"stream",
|
||||
"stream.firstTokenMs",
|
||||
expect.stream.firstTokenMs,
|
||||
observation.stream.firstTokenMs,
|
||||
"stream.firstTokenMs mismatch",
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return checkValueMatcher(observation.stream.firstTokenMs, expect.stream.firstTokenMs, {
|
||||
message: "stream.firstTokenMs mismatch",
|
||||
path: "stream.firstTokenMs",
|
||||
phase: "stream",
|
||||
});
|
||||
} else if (expect.stream.firstTokenMs && observation.stream.firstTokenMs === null) {
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
@@ -75,37 +67,25 @@ export function runExpects(observation: LlmCheckObservation, expect: LlmExpectCo
|
||||
if (!streamResult.matched) return streamResult;
|
||||
}
|
||||
|
||||
const outputResult = checkOutputRules(observation.outputText, expect.output);
|
||||
const outputResult = checkContentRules(observation.outputText, expect.output, { path: "output", phase: "output" });
|
||||
if (!outputResult.matched) return outputResult;
|
||||
|
||||
if (expect.finishReason !== undefined) {
|
||||
if (observation.finishReason !== expect.finishReason) {
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
"finishReason",
|
||||
"finishReason",
|
||||
expect.finishReason,
|
||||
observation.finishReason,
|
||||
"finishReason mismatch",
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
const result = checkValueMatcher(observation.finishReason, expect.finishReason, {
|
||||
message: "finishReason mismatch",
|
||||
path: "finishReason",
|
||||
phase: "finishReason",
|
||||
});
|
||||
if (!result.matched) return result;
|
||||
}
|
||||
|
||||
if (expect.rawFinishReason !== undefined) {
|
||||
if (observation.rawFinishReason !== expect.rawFinishReason) {
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
"rawFinishReason",
|
||||
"rawFinishReason",
|
||||
expect.rawFinishReason,
|
||||
observation.rawFinishReason,
|
||||
"rawFinishReason mismatch",
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
const result = checkValueMatcher(observation.rawFinishReason, expect.rawFinishReason, {
|
||||
message: "rawFinishReason mismatch",
|
||||
path: "rawFinishReason",
|
||||
phase: "rawFinishReason",
|
||||
});
|
||||
if (!result.matched) return result;
|
||||
}
|
||||
|
||||
if (expect.usage && observation.usage) {
|
||||
@@ -118,51 +98,31 @@ export function runExpects(observation: LlmCheckObservation, expect: LlmExpectCo
|
||||
|
||||
function checkUsageExpect(
|
||||
usage: { inputTokens: number; outputTokens: number; totalTokens: number },
|
||||
expectUsage: { inputTokens?: unknown; outputTokens?: unknown; totalTokens?: unknown },
|
||||
expectUsage: LlmUsageExpect,
|
||||
): ExpectResult {
|
||||
if (expectUsage.inputTokens !== undefined) {
|
||||
if (!applyOperator(usage.inputTokens, expectUsage.inputTokens as Parameters<typeof applyOperator>[1])) {
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
"usage",
|
||||
"usage.inputTokens",
|
||||
expectUsage.inputTokens,
|
||||
usage.inputTokens,
|
||||
"usage.inputTokens mismatch",
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
const result = checkValueMatcher(usage.inputTokens, expectUsage.inputTokens, {
|
||||
message: "usage.inputTokens mismatch",
|
||||
path: "usage.inputTokens",
|
||||
phase: "usage",
|
||||
});
|
||||
if (!result.matched) return result;
|
||||
}
|
||||
if (expectUsage.outputTokens !== undefined) {
|
||||
if (!applyOperator(usage.outputTokens, expectUsage.outputTokens as Parameters<typeof applyOperator>[1])) {
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
"usage",
|
||||
"usage.outputTokens",
|
||||
expectUsage.outputTokens,
|
||||
usage.outputTokens,
|
||||
"usage.outputTokens mismatch",
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
const result = checkValueMatcher(usage.outputTokens, expectUsage.outputTokens, {
|
||||
message: "usage.outputTokens mismatch",
|
||||
path: "usage.outputTokens",
|
||||
phase: "usage",
|
||||
});
|
||||
if (!result.matched) return result;
|
||||
}
|
||||
if (expectUsage.totalTokens !== undefined) {
|
||||
if (!applyOperator(usage.totalTokens, expectUsage.totalTokens as Parameters<typeof applyOperator>[1])) {
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
"usage",
|
||||
"usage.totalTokens",
|
||||
expectUsage.totalTokens,
|
||||
usage.totalTokens,
|
||||
"usage.totalTokens mismatch",
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
const result = checkValueMatcher(usage.totalTokens, expectUsage.totalTokens, {
|
||||
message: "usage.totalTokens mismatch",
|
||||
path: "usage.totalTokens",
|
||||
phase: "usage",
|
||||
});
|
||||
if (!result.matched) return result;
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
export { checkDuration };
|
||||
|
||||
@@ -1,83 +0,0 @@
|
||||
import type { ExpectResult } from "../../expect/types";
|
||||
import type { OutputRule } from "./types";
|
||||
|
||||
import { mismatchFailure } from "../../expect/failure";
|
||||
import { applyOperator, evaluateJsonPath } from "../../expect/operator";
|
||||
|
||||
export function checkOutputRules(outputText: null | string, rules: OutputRule[] | undefined): ExpectResult {
|
||||
if (!rules || rules.length === 0) return { failure: null, matched: true };
|
||||
|
||||
for (const rule of rules) {
|
||||
const result = checkSingleOutputRule(outputText, rule);
|
||||
if (!result.matched) return result;
|
||||
}
|
||||
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
function checkSingleOutputRule(outputText: null | string, rule: OutputRule): ExpectResult {
|
||||
if ("equals" in rule) {
|
||||
if (outputText === null || outputText !== rule.equals) {
|
||||
return {
|
||||
failure: mismatchFailure("output", "output", rule.equals, outputText, "output equals mismatch"),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
if ("contains" in rule) {
|
||||
if (!outputText?.includes(rule.contains)) {
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
"output",
|
||||
"output",
|
||||
`contains: ${rule.contains}`,
|
||||
outputText,
|
||||
"output contains mismatch",
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
if ("regex" in rule) {
|
||||
if (outputText === null || !new RegExp(rule.regex).test(outputText)) {
|
||||
return {
|
||||
failure: mismatchFailure("output", "output", `match: ${rule.regex}`, outputText, "output regex mismatch"),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
if ("json" in rule) {
|
||||
if (outputText === null) {
|
||||
return {
|
||||
failure: mismatchFailure("output", "output", "valid JSON", null, "output is null, cannot parse JSON"),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(outputText);
|
||||
} catch {
|
||||
return {
|
||||
failure: mismatchFailure("output", "output", "valid JSON", outputText, "output is not valid JSON"),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
|
||||
const value = evaluateJsonPath(parsed, rule.json.path);
|
||||
if (!applyOperator(value, rule.json)) {
|
||||
return {
|
||||
failure: mismatchFailure("output", "output", rule.json, value, "output json mismatch"),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
@@ -3,8 +3,9 @@ import { Type } from "@sinclair/typebox";
|
||||
import type { CheckerSchemas } from "../types";
|
||||
|
||||
import {
|
||||
createHeaderExpectSchema,
|
||||
createPureOperatorSchema,
|
||||
createContentRulesSchema,
|
||||
createKeyValueExpectSchema,
|
||||
createValueMatcherSchema,
|
||||
statusCodePatternSchema,
|
||||
stringMapSchema,
|
||||
} from "../../schema/fragments";
|
||||
@@ -25,36 +26,6 @@ function createLlmOptionsSchema() {
|
||||
);
|
||||
}
|
||||
|
||||
function createLlmOutputRulesSchema() {
|
||||
return Type.Array(
|
||||
Type.Object(
|
||||
{
|
||||
contains: Type.Optional(Type.String()),
|
||||
equals: Type.Optional(Type.String()),
|
||||
json: Type.Optional(
|
||||
Type.Object({ path: Type.String(), ...operatorProperties() }, { additionalProperties: false }),
|
||||
),
|
||||
regex: Type.Optional(Type.String()),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
function operatorProperties() {
|
||||
return {
|
||||
contains: Type.Optional(Type.String()),
|
||||
empty: Type.Optional(Type.Boolean()),
|
||||
equals: Type.Optional(Type.Number()),
|
||||
exists: Type.Optional(Type.Boolean()),
|
||||
gt: Type.Optional(Type.Number()),
|
||||
gte: Type.Optional(Type.Number()),
|
||||
lt: Type.Optional(Type.Number()),
|
||||
lte: Type.Optional(Type.Number()),
|
||||
match: Type.Optional(Type.String()),
|
||||
};
|
||||
}
|
||||
|
||||
export const llmCheckerSchemas: CheckerSchemas = {
|
||||
config: Type.Object(
|
||||
{
|
||||
@@ -84,17 +55,17 @@ export const llmCheckerSchemas: CheckerSchemas = {
|
||||
),
|
||||
expect: Type.Object(
|
||||
{
|
||||
finishReason: Type.Optional(Type.String()),
|
||||
headers: Type.Optional(createHeaderExpectSchema()),
|
||||
maxDurationMs: Type.Optional(Type.Number({ minimum: 0 })),
|
||||
output: Type.Optional(createLlmOutputRulesSchema()),
|
||||
rawFinishReason: Type.Optional(Type.String()),
|
||||
durationMs: Type.Optional(createValueMatcherSchema()),
|
||||
finishReason: Type.Optional(createValueMatcherSchema()),
|
||||
headers: Type.Optional(createKeyValueExpectSchema()),
|
||||
output: Type.Optional(createContentRulesSchema()),
|
||||
rawFinishReason: Type.Optional(createValueMatcherSchema()),
|
||||
status: Type.Optional(Type.Array(statusCodePatternSchema)),
|
||||
stream: Type.Optional(
|
||||
Type.Object(
|
||||
{
|
||||
completed: Type.Optional(Type.Boolean()),
|
||||
firstTokenMs: Type.Optional(createPureOperatorSchema()),
|
||||
firstTokenMs: Type.Optional(createValueMatcherSchema()),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
@@ -102,9 +73,9 @@ export const llmCheckerSchemas: CheckerSchemas = {
|
||||
usage: Type.Optional(
|
||||
Type.Object(
|
||||
{
|
||||
inputTokens: Type.Optional(createPureOperatorSchema()),
|
||||
outputTokens: Type.Optional(createPureOperatorSchema()),
|
||||
totalTokens: Type.Optional(createPureOperatorSchema()),
|
||||
inputTokens: Type.Optional(createValueMatcherSchema()),
|
||||
outputTokens: Type.Optional(createValueMatcherSchema()),
|
||||
totalTokens: Type.Optional(createValueMatcherSchema()),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import type { JSONObject } from "@ai-sdk/provider";
|
||||
|
||||
import type { ExpectOperator, ResolvedTargetBase } from "../../types";
|
||||
import type { ContentRules, KeyValueExpect, ValueMatcher } from "../../expect/types";
|
||||
import type { ResolvedTargetBase } from "../../types";
|
||||
|
||||
export interface LlmCheckObservation {
|
||||
finishReason: null | string;
|
||||
@@ -23,11 +24,11 @@ export interface LlmDefaultsConfig {
|
||||
}
|
||||
|
||||
export interface LlmExpectConfig {
|
||||
finishReason?: string;
|
||||
headers?: Record<string, ExpectOperator | string>;
|
||||
maxDurationMs?: number;
|
||||
output?: OutputRule[];
|
||||
rawFinishReason?: string;
|
||||
durationMs?: ValueMatcher;
|
||||
finishReason?: ValueMatcher;
|
||||
headers?: KeyValueExpect;
|
||||
output?: ContentRules;
|
||||
rawFinishReason?: ValueMatcher;
|
||||
status?: Array<number | string>;
|
||||
stream?: LlmStreamExpect;
|
||||
usage?: LlmUsageExpect;
|
||||
@@ -56,7 +57,7 @@ export type LlmProvider = "anthropic" | "openai" | "openai-responses";
|
||||
|
||||
export interface LlmStreamExpect {
|
||||
completed?: boolean;
|
||||
firstTokenMs?: ExpectOperator;
|
||||
firstTokenMs?: ValueMatcher;
|
||||
}
|
||||
|
||||
export interface LlmStreamObservation {
|
||||
@@ -79,9 +80,9 @@ export interface LlmTargetConfig {
|
||||
}
|
||||
|
||||
export interface LlmUsageExpect {
|
||||
inputTokens?: ExpectOperator;
|
||||
outputTokens?: ExpectOperator;
|
||||
totalTokens?: ExpectOperator;
|
||||
inputTokens?: ValueMatcher;
|
||||
outputTokens?: ValueMatcher;
|
||||
totalTokens?: ValueMatcher;
|
||||
}
|
||||
|
||||
export interface LlmUsageObservation {
|
||||
@@ -90,12 +91,6 @@ export interface LlmUsageObservation {
|
||||
totalTokens: number;
|
||||
}
|
||||
|
||||
export interface OutputJsonRule extends ExpectOperator {
|
||||
path: string;
|
||||
}
|
||||
|
||||
export type OutputRule = { contains: string } | { equals: string } | { json: OutputJsonRule } | { regex: string };
|
||||
|
||||
export interface ResolvedLlmConfig {
|
||||
authToken?: string;
|
||||
headers: Record<string, string>;
|
||||
|
||||
@@ -1,17 +1,20 @@
|
||||
import { isBoolean, isNumber, isPlainObject, isString } from "es-toolkit";
|
||||
import { isBoolean, isNumber, isString } from "es-toolkit";
|
||||
import { isArray } from "es-toolkit/compat";
|
||||
|
||||
import type { ConfigValidationIssue } from "../../schema/issues";
|
||||
import type { CheckerValidationInput } from "../types";
|
||||
|
||||
import { isUnsafeRegex } from "../../expect/redos";
|
||||
import { isPlainRecord, validateOperatorObject } from "../../expect/validate-operator";
|
||||
import {
|
||||
isPlainRecord,
|
||||
validateContentRules,
|
||||
validateKeyValueExpect,
|
||||
validateValueMatcher,
|
||||
} from "../../expect/validate-matcher";
|
||||
import { issue, joinPath } from "../../schema/issues";
|
||||
|
||||
const ALLOWED_PROVIDERS = new Set(["anthropic", "openai", "openai-responses"]);
|
||||
const ALLOWED_PROTOCOLS = new Set(["http:", "https:"]);
|
||||
const ALLOWED_MODES = new Set(["http", "stream"]);
|
||||
const OUTPUT_RULE_KEYS = ["contains", "equals", "json", "regex"] as const;
|
||||
const ALLOWED_PROTOCOLS = new Set(["http:", "https:"]);
|
||||
const ALLOWED_PROVIDERS = new Set(["anthropic", "openai", "openai-responses"]);
|
||||
|
||||
export function validateLlmConfig(input: CheckerValidationInput): ConfigValidationIssue[] {
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
@@ -37,10 +40,6 @@ function getTargetName(target: Record<string, unknown>): string | undefined {
|
||||
return isString(target["id"]) ? target["id"] : undefined;
|
||||
}
|
||||
|
||||
function isNonNegativeFiniteNumber(value: unknown): boolean {
|
||||
return isNumber(value) && Number.isFinite(value) && value >= 0;
|
||||
}
|
||||
|
||||
function validateLlmDefaults(defaults: Record<string, unknown>, path: string): ConfigValidationIssue[] {
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
|
||||
@@ -77,30 +76,23 @@ function validateLlmExpect(
|
||||
if (isArray(expect["status"])) {
|
||||
issues.push(...validateStatusValues(expect["status"], joinPath(expectPath, "status"), targetName));
|
||||
}
|
||||
|
||||
if (isPlainRecord(expect["headers"])) {
|
||||
for (const [key, value] of Object.entries(expect["headers"])) {
|
||||
if (isString(value)) continue;
|
||||
issues.push(...validateOperatorObject(value, joinPath(joinPath(expectPath, "headers"), key), targetName));
|
||||
if (expect["headers"] !== undefined) {
|
||||
issues.push(...validateKeyValueExpect(expect["headers"], joinPath(expectPath, "headers"), targetName));
|
||||
}
|
||||
}
|
||||
|
||||
if (expect["output"] !== undefined) {
|
||||
issues.push(...validateOutputRules(expect["output"], joinPath(expectPath, "output"), targetName));
|
||||
issues.push(...validateContentRules(expect["output"], joinPath(expectPath, "output"), targetName));
|
||||
}
|
||||
|
||||
if (expect["finishReason"] !== undefined && !isString(expect["finishReason"])) {
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, "finishReason"), "必须为字符串", targetName));
|
||||
if (expect["finishReason"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["finishReason"], joinPath(expectPath, "finishReason"), targetName));
|
||||
}
|
||||
|
||||
if (expect["rawFinishReason"] !== undefined && !isString(expect["rawFinishReason"])) {
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, "rawFinishReason"), "必须为字符串", targetName));
|
||||
if (expect["rawFinishReason"] !== undefined) {
|
||||
issues.push(
|
||||
...validateValueMatcher(expect["rawFinishReason"], joinPath(expectPath, "rawFinishReason"), targetName),
|
||||
);
|
||||
}
|
||||
|
||||
if (expect["usage"] !== undefined) {
|
||||
issues.push(...validateUsageExpect(expect["usage"], joinPath(expectPath, "usage"), targetName));
|
||||
}
|
||||
|
||||
if (expect["stream"] !== undefined) {
|
||||
if (mode === "http") {
|
||||
issues.push(
|
||||
@@ -110,9 +102,22 @@ function validateLlmExpect(
|
||||
issues.push(...validateStreamExpect(expect["stream"], joinPath(expectPath, "stream"), targetName));
|
||||
}
|
||||
}
|
||||
if (expect["durationMs"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
}
|
||||
|
||||
if (expect["maxDurationMs"] !== undefined && !isNonNegativeFiniteNumber(expect["maxDurationMs"])) {
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, "maxDurationMs"), "必须为非负有限数字", targetName));
|
||||
const allowedKeys = new Set([
|
||||
"durationMs",
|
||||
"finishReason",
|
||||
"headers",
|
||||
"output",
|
||||
"rawFinishReason",
|
||||
"status",
|
||||
"stream",
|
||||
"usage",
|
||||
]);
|
||||
for (const key of Object.keys(expect)) {
|
||||
if (!allowedKeys.has(key)) issues.push(issue("unknown-field", joinPath(expectPath, key), "是未知字段", targetName));
|
||||
}
|
||||
|
||||
return issues;
|
||||
@@ -197,27 +202,21 @@ function validateLlmTarget(target: Record<string, unknown>, path: string): Confi
|
||||
if (!isString(llm["model"]) || llm["model"].trim() === "") {
|
||||
issues.push(issue("required", joinPath(joinPath(path, "llm"), "model"), "必须为非空字符串", targetName));
|
||||
}
|
||||
|
||||
if (!isString(llm["prompt"]) || llm["prompt"].trim() === "") {
|
||||
issues.push(issue("required", joinPath(joinPath(path, "llm"), "prompt"), "必须为非空字符串", targetName));
|
||||
}
|
||||
|
||||
if (llm["mode"] !== undefined && !ALLOWED_MODES.has(llm["mode"] as string)) {
|
||||
issues.push(issue("invalid-type", joinPath(joinPath(path, "llm"), "mode"), "必须为 http 或 stream", targetName));
|
||||
}
|
||||
|
||||
if (llm["headers"] !== undefined) {
|
||||
issues.push(...validateStringMap(llm["headers"], joinPath(joinPath(path, "llm"), "headers"), targetName));
|
||||
}
|
||||
|
||||
if (llm["ignoreSSL"] !== undefined && !isBoolean(llm["ignoreSSL"])) {
|
||||
issues.push(issue("invalid-type", joinPath(joinPath(path, "llm"), "ignoreSSL"), "必须为布尔值", targetName));
|
||||
}
|
||||
|
||||
const provider = llm["provider"] as string | undefined;
|
||||
|
||||
if (llm["authToken"] !== undefined) {
|
||||
if (provider !== "anthropic") {
|
||||
if (llm["authToken"] !== undefined && provider !== "anthropic") {
|
||||
issues.push(
|
||||
issue(
|
||||
"invalid-auth",
|
||||
@@ -227,8 +226,6 @@ function validateLlmTarget(target: Record<string, unknown>, path: string): Confi
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (
|
||||
provider === "anthropic" &&
|
||||
isString(llm["key"]) &&
|
||||
@@ -240,11 +237,9 @@ function validateLlmTarget(target: Record<string, unknown>, path: string): Confi
|
||||
issue("auth-conflict", joinPath(joinPath(path, "llm"), "key"), "key 与 authToken 不能同时配置", targetName),
|
||||
);
|
||||
}
|
||||
|
||||
if (llm["options"] !== undefined) {
|
||||
issues.push(...validateLlmOptions(llm["options"], joinPath(joinPath(path, "llm"), "options"), targetName));
|
||||
}
|
||||
|
||||
if (llm["providerOptions"] !== undefined) {
|
||||
issues.push(
|
||||
...validateProviderOptions(
|
||||
@@ -261,76 +256,11 @@ function validateLlmTarget(target: Record<string, unknown>, path: string): Confi
|
||||
return issues;
|
||||
}
|
||||
|
||||
function validateOutputJsonRule(value: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(value)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
|
||||
if (!isString(value["path"]) || !value["path"].startsWith("$.") || value["path"].length <= 2) {
|
||||
issues.push(issue("invalid-jsonpath", joinPath(path, "path"), '必须为以 "$." 开头的有效 JSONPath', targetName));
|
||||
}
|
||||
|
||||
const operatorKeys = new Set(["path"]);
|
||||
const operators: Record<string, unknown> = {};
|
||||
for (const [key, val] of Object.entries(value)) {
|
||||
if (operatorKeys.has(key)) continue;
|
||||
operators[key] = val;
|
||||
}
|
||||
issues.push(...validateOperatorObject(operators, path, targetName, { requireAtLeastOne: false }));
|
||||
|
||||
return issues;
|
||||
}
|
||||
|
||||
function validateOutputRegex(value: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isString(value)) return [issue("invalid-type", path, "必须为字符串", targetName)];
|
||||
try {
|
||||
new RegExp(value);
|
||||
} catch {
|
||||
return [issue("invalid-regex", path, "正则不合法", targetName)];
|
||||
}
|
||||
return isUnsafeRegex(value) ? [issue("unsafe-regex", path, "正则存在 ReDoS 风险", targetName)] : [];
|
||||
}
|
||||
|
||||
function validateOutputRules(rules: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isArray(rules)) return [issue("invalid-type", path, "必须为数组", targetName)];
|
||||
return rules.flatMap((rule, index) => validateSingleOutputRule(rule, `${path}[${index}]`, targetName));
|
||||
}
|
||||
|
||||
function validateProviderOptions(value: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainObject(value)) return [issue("invalid-type", path, "必须为 JSON object", targetName)];
|
||||
if (!isPlainRecord(value)) return [issue("invalid-type", path, "必须为 JSON object", targetName)];
|
||||
return [];
|
||||
}
|
||||
|
||||
function validateSingleOutputRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
|
||||
const found = OUTPUT_RULE_KEYS.filter((type) => type in rule);
|
||||
if (found.length === 0) return [issue("missing-body-rule", path, "缺少支持的规则类型", targetName)];
|
||||
if (found.length > 1) return [issue("multiple-body-rules", path, "只能配置一种规则类型", targetName)];
|
||||
|
||||
const ruleType = found[0]!;
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
|
||||
for (const key of Object.keys(rule)) {
|
||||
if (key !== ruleType) issues.push(issue("unknown-field", joinPath(path, key), "是未知字段", targetName));
|
||||
}
|
||||
if (issues.length > 0) return issues;
|
||||
|
||||
switch (ruleType) {
|
||||
case "contains":
|
||||
return isString(rule["contains"])
|
||||
? []
|
||||
: [issue("invalid-type", joinPath(path, "contains"), "必须为字符串", targetName)];
|
||||
case "equals":
|
||||
return isString(rule["equals"])
|
||||
? []
|
||||
: [issue("invalid-type", joinPath(path, "equals"), "必须为字符串", targetName)];
|
||||
case "json":
|
||||
return validateOutputJsonRule(rule["json"], joinPath(path, "json"), targetName);
|
||||
case "regex":
|
||||
return validateOutputRegex(rule["regex"], joinPath(path, "regex"), targetName);
|
||||
}
|
||||
}
|
||||
|
||||
function validateStatusValues(values: unknown[], path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
for (let i = 0; i < values.length; i++) {
|
||||
@@ -360,18 +290,22 @@ function validateStreamExpect(stream: unknown, path: string, targetName?: string
|
||||
if (stream["completed"] !== undefined && !isBoolean(stream["completed"])) {
|
||||
issues.push(issue("invalid-type", joinPath(path, "completed"), "必须为布尔值", targetName));
|
||||
}
|
||||
|
||||
if (stream["firstTokenMs"] !== undefined) {
|
||||
issues.push(...validateOperatorObject(stream["firstTokenMs"], joinPath(path, "firstTokenMs"), targetName));
|
||||
issues.push(...validateValueMatcher(stream["firstTokenMs"], joinPath(path, "firstTokenMs"), targetName));
|
||||
}
|
||||
|
||||
const allowedKeys = new Set(["completed", "firstTokenMs"]);
|
||||
for (const key of Object.keys(stream)) {
|
||||
if (!allowedKeys.has(key)) issues.push(issue("unknown-field", joinPath(path, key), "是未知字段", targetName));
|
||||
}
|
||||
|
||||
return issues;
|
||||
}
|
||||
|
||||
function validateStringMap(value: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainObject(value)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
if (!isPlainRecord(value)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
for (const [key, val] of Object.entries(value as Record<string, unknown>)) {
|
||||
for (const [key, val] of Object.entries(value)) {
|
||||
if (!isString(val)) {
|
||||
issues.push(issue("invalid-type", joinPath(path, key), "必须为字符串", targetName));
|
||||
}
|
||||
@@ -383,14 +317,15 @@ function validateUsageExpect(usage: unknown, path: string, targetName?: string):
|
||||
if (!isPlainRecord(usage)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
|
||||
if (usage["inputTokens"] !== undefined) {
|
||||
issues.push(...validateOperatorObject(usage["inputTokens"], joinPath(path, "inputTokens"), targetName));
|
||||
for (const key of ["inputTokens", "outputTokens", "totalTokens"]) {
|
||||
if (usage[key] !== undefined) {
|
||||
issues.push(...validateValueMatcher(usage[key], joinPath(path, key), targetName));
|
||||
}
|
||||
if (usage["outputTokens"] !== undefined) {
|
||||
issues.push(...validateOperatorObject(usage["outputTokens"], joinPath(path, "outputTokens"), targetName));
|
||||
}
|
||||
if (usage["totalTokens"] !== undefined) {
|
||||
issues.push(...validateOperatorObject(usage["totalTokens"], joinPath(path, "totalTokens"), targetName));
|
||||
|
||||
const allowedKeys = new Set(["inputTokens", "outputTokens", "totalTokens"]);
|
||||
for (const key of Object.keys(usage)) {
|
||||
if (!allowedKeys.has(key)) issues.push(issue("unknown-field", joinPath(path, key), "是未知字段", targetName));
|
||||
}
|
||||
|
||||
return issues;
|
||||
|
||||
@@ -4,8 +4,8 @@ import type { CheckResult, RawTargetConfig } from "../../types";
|
||||
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
|
||||
import type { ResolvedTcpTarget, TcpExpectConfig, TcpTargetConfig } from "./types";
|
||||
|
||||
import { checkDuration } from "../../expect/duration";
|
||||
import { errorFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { parseSize } from "../../utils";
|
||||
import { checkBanner, checkConnected } from "./expect";
|
||||
import { tcpCheckerSchemas } from "./schema";
|
||||
@@ -124,7 +124,11 @@ export class TcpChecker implements CheckerDefinition<ResolvedTcpTarget> {
|
||||
}
|
||||
|
||||
const durationMs = Math.round(performance.now() - start);
|
||||
const durationResult = checkDuration(durationMs, expect?.maxDurationMs);
|
||||
const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
});
|
||||
if (!durationResult.matched) {
|
||||
return {
|
||||
durationMs,
|
||||
|
||||
@@ -1,18 +1,10 @@
|
||||
import type { ExpectResult } from "../../expect/types";
|
||||
import type { ExpectOperator } from "../../types";
|
||||
import type { ContentRules, ExpectResult } from "../../expect/types";
|
||||
|
||||
import { checkContentRules } from "../../expect/content";
|
||||
import { mismatchFailure } from "../../expect/failure";
|
||||
import { applyOperator } from "../../expect/operator";
|
||||
|
||||
export function checkBanner(banner: string, op: ExpectOperator): ExpectResult {
|
||||
const matched = applyOperator(banner, op);
|
||||
if (!matched) {
|
||||
return {
|
||||
failure: mismatchFailure("banner", "banner", op, banner, `banner 不满足条件`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
export function checkBanner(banner: string, rules: ContentRules): ExpectResult {
|
||||
return checkContentRules(banner, rules, { path: "banner", phase: "banner" });
|
||||
}
|
||||
|
||||
export function checkConnected(connected: boolean, expected: boolean): ExpectResult {
|
||||
|
||||
@@ -2,7 +2,7 @@ import { Type } from "@sinclair/typebox";
|
||||
|
||||
import type { CheckerSchemas } from "../types";
|
||||
|
||||
import { createPureOperatorSchema, sizeSchema } from "../../schema/fragments";
|
||||
import { createContentRulesSchema, createValueMatcherSchema, sizeSchema } from "../../schema/fragments";
|
||||
|
||||
export const tcpCheckerSchemas: CheckerSchemas = {
|
||||
config: Type.Object(
|
||||
@@ -24,9 +24,9 @@ export const tcpCheckerSchemas: CheckerSchemas = {
|
||||
),
|
||||
expect: Type.Object(
|
||||
{
|
||||
banner: Type.Optional(createPureOperatorSchema()),
|
||||
banner: Type.Optional(createContentRulesSchema()),
|
||||
connected: Type.Optional(Type.Boolean()),
|
||||
maxDurationMs: Type.Optional(Type.Number({ minimum: 0 })),
|
||||
durationMs: Type.Optional(createValueMatcherSchema()),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { ExpectOperator, ResolvedTargetBase } from "../../types";
|
||||
import type { ContentRules, ValueMatcher } from "../../expect/types";
|
||||
import type { ResolvedTargetBase } from "../../types";
|
||||
|
||||
export interface ResolvedTcpConfig {
|
||||
bannerReadTimeout: number;
|
||||
@@ -24,9 +25,9 @@ export interface TcpDefaultsConfig {
|
||||
}
|
||||
|
||||
export interface TcpExpectConfig {
|
||||
banner?: ExpectOperator;
|
||||
banner?: ContentRules;
|
||||
connected?: boolean;
|
||||
maxDurationMs?: number;
|
||||
durationMs?: ValueMatcher;
|
||||
}
|
||||
|
||||
export interface TcpTargetConfig {
|
||||
|
||||
@@ -3,7 +3,7 @@ import { isNumber, isPlainObject, isString } from "es-toolkit";
|
||||
import type { ConfigValidationIssue } from "../../schema/issues";
|
||||
import type { CheckerValidationInput } from "../types";
|
||||
|
||||
import { validateOperatorObject } from "../../expect/validate-operator";
|
||||
import { validateContentRules, validateValueMatcher } from "../../expect/validate-matcher";
|
||||
import { issue, joinPath } from "../../schema/issues";
|
||||
|
||||
export function validateTcpConfig(input: CheckerValidationInput): ConfigValidationIssue[] {
|
||||
@@ -79,8 +79,8 @@ function validateTcpExpect(
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, "connected"), "必须为布尔值", targetName));
|
||||
}
|
||||
|
||||
if (expect["maxDurationMs"] !== undefined && !isNonNegativeFiniteNumber(expect["maxDurationMs"])) {
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, "maxDurationMs"), "必须为非负有限数字", targetName));
|
||||
if (expect["durationMs"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
}
|
||||
|
||||
if (expect["banner"] !== undefined) {
|
||||
@@ -89,11 +89,11 @@ function validateTcpExpect(
|
||||
issue("invalid-value", joinPath(expectPath, "banner"), "banner 断言需要启用 tcp.readBanner", targetName),
|
||||
);
|
||||
} else {
|
||||
issues.push(...validateOperatorObject(expect["banner"], joinPath(expectPath, "banner"), targetName));
|
||||
issues.push(...validateContentRules(expect["banner"], joinPath(expectPath, "banner"), targetName));
|
||||
}
|
||||
}
|
||||
|
||||
const allowedKeys = new Set(["banner", "connected", "maxDurationMs"]);
|
||||
const allowedKeys = new Set(["banner", "connected", "durationMs"]);
|
||||
for (const key of Object.keys(expect)) {
|
||||
if (!allowedKeys.has(key)) {
|
||||
issues.push(issue("unknown-field", joinPath(expectPath, key), "是未知字段", targetName));
|
||||
|
||||
@@ -4,8 +4,8 @@ import type { CheckResult, RawTargetConfig } from "../../types";
|
||||
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
|
||||
import type { ResolvedUdpTarget, UdpDefaultsConfig, UdpExpectConfig, UdpTargetConfig } from "./types";
|
||||
|
||||
import { checkDuration } from "../../expect/duration";
|
||||
import { errorFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { parseSize } from "../../utils";
|
||||
import { decodePayload, encodeResponse } from "./encoding";
|
||||
import { checkResponded, checkResponseSize, checkResponseText, checkSourceHost, checkSourcePort } from "./expect";
|
||||
@@ -83,7 +83,11 @@ export class UdpChecker implements CheckerDefinition<ResolvedUdpTarget> {
|
||||
|
||||
if (!exchangeResult.responded) {
|
||||
const durationMs = Math.round(performance.now() - start);
|
||||
const durationResult = checkDuration(durationMs, expect?.maxDurationMs);
|
||||
const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
});
|
||||
if (!durationResult.matched) {
|
||||
return {
|
||||
durationMs,
|
||||
@@ -194,7 +198,11 @@ export class UdpChecker implements CheckerDefinition<ResolvedUdpTarget> {
|
||||
}
|
||||
|
||||
const durationMs = Math.round(performance.now() - start);
|
||||
const durationResult = checkDuration(durationMs, expect?.maxDurationMs);
|
||||
const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
});
|
||||
if (!durationResult.matched) {
|
||||
return {
|
||||
durationMs,
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import type { ExpectResult } from "../../expect/types";
|
||||
import type { ExpectOperator } from "../../types";
|
||||
import type { ContentRules, ExpectResult, ValueMatcher } from "../../expect/types";
|
||||
|
||||
import { checkContentRules } from "../../expect/content";
|
||||
import { mismatchFailure } from "../../expect/failure";
|
||||
import { applyOperator } from "../../expect/operator";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
|
||||
export function checkResponded(responded: boolean, expected: boolean): ExpectResult {
|
||||
if (responded === expected) return { failure: null, matched: true };
|
||||
@@ -18,49 +18,30 @@ export function checkResponded(responded: boolean, expected: boolean): ExpectRes
|
||||
};
|
||||
}
|
||||
|
||||
export function checkResponseSize(size: number, op: ExpectOperator): ExpectResult {
|
||||
const matched = applyOperator(size, op);
|
||||
if (!matched) {
|
||||
return {
|
||||
failure: mismatchFailure("responseSize", "responseSize", op, size, "响应大小不满足条件"),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
export function checkResponseSize(size: number, matcher: ValueMatcher): ExpectResult {
|
||||
return checkValueMatcher(size, matcher, {
|
||||
message: "响应大小不满足条件",
|
||||
path: "responseSize",
|
||||
phase: "responseSize",
|
||||
});
|
||||
}
|
||||
|
||||
export function checkResponseText(text: string, rules: ExpectOperator[]): ExpectResult {
|
||||
for (let i = 0; i < rules.length; i++) {
|
||||
const rule = rules[i]!;
|
||||
const path = `response[${i}]`;
|
||||
if (!applyOperator(text, rule)) {
|
||||
return {
|
||||
failure: mismatchFailure("response", path, rule, text, `response rule at index ${i} mismatch`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
export function checkResponseText(text: string, rules: ContentRules): ExpectResult {
|
||||
return checkContentRules(text, rules, { path: "response", phase: "response" });
|
||||
}
|
||||
|
||||
export function checkSourceHost(actual: string, op: ExpectOperator): ExpectResult {
|
||||
const matched = applyOperator(actual, op);
|
||||
if (!matched) {
|
||||
return {
|
||||
failure: mismatchFailure("sourceHost", "sourceHost", op, actual, "响应来源地址不满足条件"),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
export function checkSourceHost(actual: string, matcher: ValueMatcher): ExpectResult {
|
||||
return checkValueMatcher(actual, matcher, {
|
||||
message: "响应来源地址不满足条件",
|
||||
path: "sourceHost",
|
||||
phase: "sourceHost",
|
||||
});
|
||||
}
|
||||
|
||||
export function checkSourcePort(actual: number, op: ExpectOperator): ExpectResult {
|
||||
const matched = applyOperator(actual, op);
|
||||
if (!matched) {
|
||||
return {
|
||||
failure: mismatchFailure("sourcePort", "sourcePort", op, actual, "响应来源端口不满足条件"),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
export function checkSourcePort(actual: number, matcher: ValueMatcher): ExpectResult {
|
||||
return checkValueMatcher(actual, matcher, {
|
||||
message: "响应来源端口不满足条件",
|
||||
path: "sourcePort",
|
||||
phase: "sourcePort",
|
||||
});
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@ import { Type } from "@sinclair/typebox";
|
||||
|
||||
import type { CheckerSchemas } from "../types";
|
||||
|
||||
import { createPureOperatorSchema, createTextRulesSchema, sizeSchema } from "../../schema/fragments";
|
||||
import { createContentRulesSchema, createValueMatcherSchema, sizeSchema } from "../../schema/fragments";
|
||||
|
||||
export const udpCheckerSchemas: CheckerSchemas = {
|
||||
config: Type.Object(
|
||||
@@ -26,12 +26,12 @@ export const udpCheckerSchemas: CheckerSchemas = {
|
||||
),
|
||||
expect: Type.Object(
|
||||
{
|
||||
maxDurationMs: Type.Optional(Type.Number({ minimum: 0 })),
|
||||
durationMs: Type.Optional(createValueMatcherSchema()),
|
||||
responded: Type.Optional(Type.Boolean()),
|
||||
response: Type.Optional(createTextRulesSchema()),
|
||||
responseSize: Type.Optional(createPureOperatorSchema()),
|
||||
sourceHost: Type.Optional(createPureOperatorSchema()),
|
||||
sourcePort: Type.Optional(createPureOperatorSchema()),
|
||||
response: Type.Optional(createContentRulesSchema()),
|
||||
responseSize: Type.Optional(createValueMatcherSchema()),
|
||||
sourceHost: Type.Optional(createValueMatcherSchema()),
|
||||
sourcePort: Type.Optional(createValueMatcherSchema()),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { ExpectOperator, ResolvedTargetBase } from "../../types";
|
||||
import type { ContentRules, ValueMatcher } from "../../expect/types";
|
||||
import type { ResolvedTargetBase } from "../../types";
|
||||
|
||||
export interface ResolvedUdpConfig {
|
||||
encoding: UdpEncoding;
|
||||
@@ -28,12 +29,12 @@ export interface UdpDefaultsConfig {
|
||||
export type UdpEncoding = "base64" | "hex" | "text";
|
||||
|
||||
export interface UdpExpectConfig {
|
||||
maxDurationMs?: number;
|
||||
durationMs?: ValueMatcher;
|
||||
responded?: boolean;
|
||||
response?: ExpectOperator[];
|
||||
responseSize?: ExpectOperator;
|
||||
sourceHost?: ExpectOperator;
|
||||
sourcePort?: ExpectOperator;
|
||||
response?: ContentRules;
|
||||
responseSize?: ValueMatcher;
|
||||
sourceHost?: ValueMatcher;
|
||||
sourcePort?: ValueMatcher;
|
||||
}
|
||||
|
||||
export interface UdpTargetConfig {
|
||||
|
||||
@@ -3,7 +3,7 @@ import { isNumber, isPlainObject, isString } from "es-toolkit";
|
||||
import type { ConfigValidationIssue } from "../../schema/issues";
|
||||
import type { CheckerValidationInput } from "../types";
|
||||
|
||||
import { validateOperatorObject } from "../../expect/validate-operator";
|
||||
import { validateContentRules, validateValueMatcher } from "../../expect/validate-matcher";
|
||||
import { issue, joinPath } from "../../schema/issues";
|
||||
|
||||
const VALID_ENCODINGS = new Set(["base64", "hex", "text"]);
|
||||
@@ -28,10 +28,6 @@ function getTargetName(target: Record<string, unknown>): string | undefined {
|
||||
return isString(target["id"]) ? target["id"] : undefined;
|
||||
}
|
||||
|
||||
function isNonNegativeFiniteNumber(value: unknown): boolean {
|
||||
return isNumber(value) && Number.isFinite(value) && value >= 0;
|
||||
}
|
||||
|
||||
function validateEncoding(value: unknown, path: string, targetName: string | undefined): ConfigValidationIssue[] {
|
||||
if (value === undefined) return [];
|
||||
if (!isString(value) || !VALID_ENCODINGS.has(value)) {
|
||||
@@ -48,22 +44,6 @@ function validateSize(value: unknown, path: string, targetName: string | undefin
|
||||
return [];
|
||||
}
|
||||
|
||||
function validateTextRulesArray(value: unknown, path: string, targetName: string | undefined): ConfigValidationIssue[] {
|
||||
if (!Array.isArray(value)) {
|
||||
return [issue("invalid-type", path, "必须为数组", targetName)];
|
||||
}
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
for (let i = 0; i < value.length; i++) {
|
||||
const rule: unknown = value[i];
|
||||
if (!isPlainObject(rule)) {
|
||||
issues.push(issue("invalid-type", joinPath(path, `[${i}]`), "必须为 operator 对象", targetName));
|
||||
continue;
|
||||
}
|
||||
issues.push(...validateOperatorObject(rule, joinPath(path, `[${i}]`), targetName));
|
||||
}
|
||||
return issues;
|
||||
}
|
||||
|
||||
function validateUdpDefaults(input: CheckerValidationInput): ConfigValidationIssue[] {
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const defaults = input.defaults["udp"];
|
||||
@@ -99,24 +79,24 @@ function validateUdpExpect(target: Record<string, unknown>, path: string): Confi
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, "responded"), "必须为布尔值", targetName));
|
||||
}
|
||||
|
||||
if (expect["maxDurationMs"] !== undefined && !isNonNegativeFiniteNumber(expect["maxDurationMs"])) {
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, "maxDurationMs"), "必须为非负有限数字", targetName));
|
||||
if (expect["durationMs"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
}
|
||||
|
||||
if (expect["response"] !== undefined) {
|
||||
issues.push(...validateTextRulesArray(expect["response"], joinPath(expectPath, "response"), targetName));
|
||||
issues.push(...validateContentRules(expect["response"], joinPath(expectPath, "response"), targetName));
|
||||
}
|
||||
|
||||
if (expect["responseSize"] !== undefined) {
|
||||
issues.push(...validateOperatorObject(expect["responseSize"], joinPath(expectPath, "responseSize"), targetName));
|
||||
issues.push(...validateValueMatcher(expect["responseSize"], joinPath(expectPath, "responseSize"), targetName));
|
||||
}
|
||||
|
||||
if (expect["sourceHost"] !== undefined) {
|
||||
issues.push(...validateOperatorObject(expect["sourceHost"], joinPath(expectPath, "sourceHost"), targetName));
|
||||
issues.push(...validateValueMatcher(expect["sourceHost"], joinPath(expectPath, "sourceHost"), targetName));
|
||||
}
|
||||
|
||||
if (expect["sourcePort"] !== undefined) {
|
||||
issues.push(...validateOperatorObject(expect["sourcePort"], joinPath(expectPath, "sourcePort"), targetName));
|
||||
issues.push(...validateValueMatcher(expect["sourcePort"], joinPath(expectPath, "sourcePort"), targetName));
|
||||
}
|
||||
|
||||
const respondedFalse = responded === false;
|
||||
@@ -143,7 +123,7 @@ function validateUdpExpect(target: Record<string, unknown>, path: string): Confi
|
||||
}
|
||||
}
|
||||
|
||||
const allowedKeys = new Set(["maxDurationMs", "responded", "response", "responseSize", "sourceHost", "sourcePort"]);
|
||||
const allowedKeys = new Set(["durationMs", "responded", "response", "responseSize", "sourceHost", "sourcePort"]);
|
||||
for (const key of Object.keys(expect)) {
|
||||
if (!allowedKeys.has(key)) {
|
||||
issues.push(issue("unknown-field", joinPath(expectPath, key), "是未知字段", targetName));
|
||||
|
||||
@@ -4,14 +4,24 @@ import { Type } from "@sinclair/typebox";
|
||||
|
||||
import type { CheckerDefinition } from "../runner/types";
|
||||
|
||||
import { durationSchema, variableValueSchema } from "./fragments";
|
||||
import {
|
||||
createContentRulesSchema,
|
||||
createKeyValueExpectSchema,
|
||||
createValueMatcherSchema,
|
||||
durationSchema,
|
||||
variableValueSchema,
|
||||
} from "./fragments";
|
||||
|
||||
export function createExternalProbeConfigSchema(checkers: CheckerDefinition[]): Record<string, unknown> {
|
||||
return {
|
||||
...cloneSchema(createProbeConfigSchema(checkers, true)),
|
||||
$id: "https://dial.local/probe-config.schema.json",
|
||||
$schema: "http://json-schema.org/draft-07/schema#",
|
||||
definitions: {},
|
||||
definitions: {
|
||||
ContentRules: cloneSchema(createContentRulesSchema()),
|
||||
KeyValueExpect: cloneSchema(createKeyValueExpectSchema()),
|
||||
ValueMatcher: cloneSchema(createValueMatcherSchema()),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -6,9 +6,7 @@ import type { JsonValue } from "./types";
|
||||
|
||||
export const HTTP_METHODS = ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"] as const;
|
||||
|
||||
export const BodyRuleTypeKeys = ["contains", "regex", "json", "css", "xpath"] as const;
|
||||
|
||||
export const OperatorKeys = ["contains", "empty", "equals", "exists", "gt", "gte", "lt", "lte", "match"] as const;
|
||||
export const MatcherKeys = ["contains", "empty", "equals", "exists", "gt", "gte", "lt", "lte", "regex"] as const;
|
||||
|
||||
export const durationSchema = Type.String();
|
||||
|
||||
@@ -41,51 +39,43 @@ export const stringMapSchema = Type.Unsafe<Record<string, string>>({
|
||||
type: "object",
|
||||
});
|
||||
|
||||
export function createBodyRulesSchema(): TSchema {
|
||||
export function createContentRulesSchema(): TSchema {
|
||||
return Type.Array(
|
||||
Type.Object(
|
||||
{
|
||||
contains: Type.Optional(Type.String()),
|
||||
...matcherProperties(),
|
||||
css: Type.Optional(
|
||||
Type.Object(
|
||||
{ attr: Type.Optional(Type.String()), selector: Type.String({ minLength: 1 }), ...operatorProperties() },
|
||||
{ attr: Type.Optional(Type.String()), selector: Type.String({ minLength: 1 }), ...matcherProperties() },
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
),
|
||||
json: Type.Optional(
|
||||
Type.Object({ path: Type.String(), ...operatorProperties() }, { additionalProperties: false }),
|
||||
Type.Object({ path: Type.String(), ...matcherProperties() }, { additionalProperties: false }),
|
||||
),
|
||||
regex: Type.Optional(Type.String()),
|
||||
xpath: Type.Optional(
|
||||
Type.Object(
|
||||
{ path: Type.String({ minLength: 1 }), ...operatorProperties() },
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
Type.Object({ path: Type.String({ minLength: 1 }), ...matcherProperties() }, { additionalProperties: false }),
|
||||
),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
{ additionalProperties: false, minProperties: 1 },
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
export function createHeaderExpectSchema(): TSchema {
|
||||
export function createKeyValueExpectSchema(): TSchema {
|
||||
return Type.Unsafe<Record<string, unknown>>({
|
||||
additionalProperties: {
|
||||
anyOf: [{ type: "string" }, createPureOperatorSchema()],
|
||||
anyOf: [jsonValueSchema, createValueMatcherSchema()],
|
||||
},
|
||||
type: "object",
|
||||
});
|
||||
}
|
||||
|
||||
export function createPureOperatorSchema(): TSchema {
|
||||
return Type.Object(operatorProperties(), { additionalProperties: false, minProperties: 1 });
|
||||
export function createValueMatcherSchema(): TSchema {
|
||||
return Type.Object(matcherProperties(), { additionalProperties: false, minProperties: 1 });
|
||||
}
|
||||
|
||||
export function createTextRulesSchema(): TSchema {
|
||||
return Type.Array(createPureOperatorSchema());
|
||||
}
|
||||
|
||||
export function operatorProperties(): Record<string, TSchema> {
|
||||
export function matcherProperties(): Record<string, TSchema> {
|
||||
return {
|
||||
contains: Type.Optional(Type.String()),
|
||||
empty: Type.Optional(Type.Boolean()),
|
||||
@@ -95,6 +85,6 @@ export function operatorProperties(): Record<string, TSchema> {
|
||||
gte: Type.Optional(Type.Number()),
|
||||
lt: Type.Optional(Type.Number()),
|
||||
lte: Type.Optional(Type.Number()),
|
||||
match: Type.Optional(Type.String()),
|
||||
regex: Type.Optional(Type.String()),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -15,20 +15,6 @@ export interface EngineRuntimeConfig {
|
||||
retention?: string;
|
||||
}
|
||||
|
||||
export interface ExpectOperator {
|
||||
contains?: string;
|
||||
empty?: boolean;
|
||||
equals?: JsonValue;
|
||||
exists?: boolean;
|
||||
gt?: number;
|
||||
gte?: number;
|
||||
lt?: number;
|
||||
lte?: number;
|
||||
match?: string;
|
||||
}
|
||||
|
||||
export type ExpectValue = ExpectOperator | JsonValue;
|
||||
|
||||
export type JsonValue = boolean | JsonValue[] | null | number | string | { [key: string]: JsonValue };
|
||||
|
||||
export interface ProbeConfig {
|
||||
|
||||
@@ -824,7 +824,8 @@ targets:
|
||||
- json:
|
||||
path: "$.status"
|
||||
equals: "ok"
|
||||
maxDurationMs: 3000
|
||||
durationMs:
|
||||
lte: 3000
|
||||
`,
|
||||
);
|
||||
|
||||
@@ -833,7 +834,7 @@ targets:
|
||||
if (t.type === "http") {
|
||||
expect(t.expect).toEqual({
|
||||
body: [{ contains: "ok" }, { json: { equals: "ok", path: "$.status" } }],
|
||||
maxDurationMs: 3000,
|
||||
durationMs: { lte: 3000 },
|
||||
status: [200, 201],
|
||||
});
|
||||
}
|
||||
@@ -853,10 +854,11 @@ targets:
|
||||
exitCode: [0, 2]
|
||||
stdout:
|
||||
- contains: "ok"
|
||||
- match: "done"
|
||||
- regex: "done"
|
||||
stderr:
|
||||
- empty: true
|
||||
maxDurationMs: 5000
|
||||
durationMs:
|
||||
lte: 5000
|
||||
`,
|
||||
);
|
||||
|
||||
@@ -864,10 +866,10 @@ targets:
|
||||
const t = config.targets[0]!;
|
||||
if (t.type === "cmd") {
|
||||
expect(t.expect).toEqual({
|
||||
durationMs: { lte: 5000 },
|
||||
exitCode: [0, 2],
|
||||
maxDurationMs: 5000,
|
||||
stderr: [{ empty: true }],
|
||||
stdout: [{ contains: "ok" }, { match: "done" }],
|
||||
stdout: [{ contains: "ok" }, { regex: "done" }],
|
||||
});
|
||||
}
|
||||
});
|
||||
@@ -1074,7 +1076,7 @@ targets:
|
||||
await expect(loadConfig(configPath)).rejects.toThrow("5xx");
|
||||
});
|
||||
|
||||
test("expect.maxDurationMs 负数抛出错误", async () => {
|
||||
test("expect.durationMs 非 matcher 抛出错误", async () => {
|
||||
const configPath = join(tempDir, "neg-duration.yaml");
|
||||
await writeFile(
|
||||
configPath,
|
||||
@@ -1085,11 +1087,11 @@ targets:
|
||||
http:
|
||||
url: "http://example.com"
|
||||
expect:
|
||||
maxDurationMs: -100
|
||||
durationMs: -100
|
||||
`,
|
||||
);
|
||||
// eslint-disable-next-line @typescript-eslint/await-thenable
|
||||
await expect(loadConfig(configPath)).rejects.toThrow("maxDurationMs 必须为非负有限数字");
|
||||
await expect(loadConfig(configPath)).rejects.toThrow("expect.durationMs 必须为 matcher 对象");
|
||||
});
|
||||
|
||||
test("expect.body 非数组抛出错误", async () => {
|
||||
@@ -1126,7 +1128,7 @@ targets:
|
||||
`,
|
||||
);
|
||||
// eslint-disable-next-line @typescript-eslint/await-thenable
|
||||
await expect(loadConfig(configPath)).rejects.toThrow("缺少支持的规则类型");
|
||||
await expect(loadConfig(configPath)).rejects.toThrow("foo 是未知字段");
|
||||
});
|
||||
|
||||
test("body rule 使用 match 字段（非支持）抛出错误", async () => {
|
||||
@@ -1145,10 +1147,10 @@ targets:
|
||||
`,
|
||||
);
|
||||
// eslint-disable-next-line @typescript-eslint/await-thenable
|
||||
await expect(loadConfig(configPath)).rejects.toThrow("缺少支持的规则类型");
|
||||
await expect(loadConfig(configPath)).rejects.toThrow("match 是未知字段");
|
||||
});
|
||||
|
||||
test("body rule 多个支持字段抛出错误", async () => {
|
||||
test("body rule 直接 matcher 混入 extractor 抛出错误", async () => {
|
||||
const configPath = join(tempDir, "bad-body-rule-multi.yaml");
|
||||
await writeFile(
|
||||
configPath,
|
||||
@@ -1161,11 +1163,12 @@ targets:
|
||||
expect:
|
||||
body:
|
||||
- contains: "ok"
|
||||
regex: "ok"
|
||||
json:
|
||||
path: "$.status"
|
||||
`,
|
||||
);
|
||||
// eslint-disable-next-line @typescript-eslint/await-thenable
|
||||
await expect(loadConfig(configPath)).rejects.toThrow("只能配置一种规则类型");
|
||||
await expect(loadConfig(configPath)).rejects.toThrow("直接 matcher 不能与 extractor 混用");
|
||||
});
|
||||
|
||||
test("body regex 非法正则抛出错误", async () => {
|
||||
@@ -1228,7 +1231,7 @@ targets:
|
||||
await expect(loadConfig(configPath)).rejects.toThrow("css.selector 必须为非空字符串");
|
||||
});
|
||||
|
||||
test("operator match 非法正则抛出错误", async () => {
|
||||
test("旧 match matcher 抛出错误", async () => {
|
||||
const configPath = join(tempDir, "bad-op-match.yaml");
|
||||
await writeFile(
|
||||
configPath,
|
||||
@@ -1245,7 +1248,7 @@ targets:
|
||||
`,
|
||||
);
|
||||
// eslint-disable-next-line @typescript-eslint/await-thenable
|
||||
await expect(loadConfig(configPath)).rejects.toThrow("match 正则不合法");
|
||||
await expect(loadConfig(configPath)).rejects.toThrow("match 是未知 matcher");
|
||||
});
|
||||
|
||||
test("operator gte 非数字抛出错误", async () => {
|
||||
@@ -1531,7 +1534,7 @@ targets:
|
||||
stdout:
|
||||
- {}
|
||||
`,
|
||||
"stdout[0] 必须包含至少一个合法 operator",
|
||||
"stdout[0] 必须包含至少一个合法 matcher",
|
||||
);
|
||||
});
|
||||
|
||||
@@ -1552,7 +1555,7 @@ targets:
|
||||
);
|
||||
});
|
||||
|
||||
test("cmd stdout match 正则非法", async () => {
|
||||
test("cmd stdout 旧 match 字段非法", async () => {
|
||||
await expectConfigError(
|
||||
"bad-cmd-stdout-regex.yaml",
|
||||
`targets:
|
||||
@@ -1565,7 +1568,7 @@ targets:
|
||||
stdout:
|
||||
- match: "[invalid"
|
||||
`,
|
||||
"stdout[0].match 正则不合法",
|
||||
"stdout[0].match 是未知字段",
|
||||
);
|
||||
});
|
||||
|
||||
@@ -1878,14 +1881,14 @@ targets:
|
||||
readBanner: true
|
||||
expect:
|
||||
banner:
|
||||
contains: "ESMTP"
|
||||
- contains: "ESMTP"
|
||||
`,
|
||||
);
|
||||
|
||||
const config = await loadConfig(configPath);
|
||||
const t = config.targets[0]! as ResolvedTcpTarget;
|
||||
expect(t.tcp.readBanner).toBe(true);
|
||||
expect(t.expect?.banner).toEqual({ contains: "ESMTP" });
|
||||
expect(t.expect?.banner).toEqual([{ contains: "ESMTP" }]);
|
||||
});
|
||||
|
||||
test("tcp expect.banner 未开启 readBanner 抛出错误", async () => {
|
||||
@@ -1954,7 +1957,7 @@ targets:
|
||||
);
|
||||
});
|
||||
|
||||
test("tcp expect connected 和 maxDurationMs", async () => {
|
||||
test("tcp expect connected 和 durationMs", async () => {
|
||||
const configPath = join(tempDir, "tcp-expect-connected.yaml");
|
||||
await writeFile(
|
||||
configPath,
|
||||
@@ -1966,14 +1969,15 @@ targets:
|
||||
port: 80
|
||||
expect:
|
||||
connected: false
|
||||
maxDurationMs: 5000
|
||||
durationMs:
|
||||
lte: 5000
|
||||
`,
|
||||
);
|
||||
|
||||
const config = await loadConfig(configPath);
|
||||
const t = config.targets[0]! as ResolvedTcpTarget;
|
||||
expect(t.expect?.connected).toBe(false);
|
||||
expect(t.expect?.maxDurationMs).toBe(5000);
|
||||
expect(t.expect?.durationMs).toEqual({ lte: 5000 });
|
||||
});
|
||||
|
||||
test("解析最简 ping 配置", async () => {
|
||||
@@ -2011,10 +2015,14 @@ targets:
|
||||
packetSize: 1472
|
||||
expect:
|
||||
alive: true
|
||||
maxPacketLoss: 10
|
||||
maxAvgLatencyMs: 200
|
||||
maxMaxLatencyMs: 500
|
||||
maxDurationMs: 5000
|
||||
packetLossPercent:
|
||||
lte: 10
|
||||
avgLatencyMs:
|
||||
lte: 200
|
||||
maxLatencyMs:
|
||||
lte: 500
|
||||
durationMs:
|
||||
lte: 5000
|
||||
`,
|
||||
);
|
||||
|
||||
@@ -2023,10 +2031,10 @@ targets:
|
||||
expect(t.ping).toEqual({ count: 5, host: "10.0.0.1", packetSize: 1472 });
|
||||
expect(t.expect).toEqual({
|
||||
alive: true,
|
||||
maxAvgLatencyMs: 200,
|
||||
maxDurationMs: 5000,
|
||||
maxMaxLatencyMs: 500,
|
||||
maxPacketLoss: 10,
|
||||
avgLatencyMs: { lte: 200 },
|
||||
durationMs: { lte: 5000 },
|
||||
maxLatencyMs: { lte: 500 },
|
||||
packetLossPercent: { lte: 10 },
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -74,9 +74,9 @@ describe("DbChecker", () => {
|
||||
expect(result.failure!.message).toBeTruthy();
|
||||
});
|
||||
|
||||
test("maxDurationMs 超时返回失败", async () => {
|
||||
test("durationMs 超时返回失败", async () => {
|
||||
const result = await checker.execute(
|
||||
makeTarget({ query: "SELECT 1" }, { expect: { maxDurationMs: -1 } }),
|
||||
makeTarget({ query: "SELECT 1" }, { expect: { durationMs: { lt: 0 } } }),
|
||||
makeCtx(),
|
||||
);
|
||||
expect(result.matched).toBe(false);
|
||||
|
||||
@@ -4,35 +4,35 @@ import { checkRowCount, checkRows } from "../../../../../src/server/checker/runn
|
||||
|
||||
describe("checkRowCount", () => {
|
||||
test("空数组通过 rowCount gte 0", () => {
|
||||
const result = checkRowCount([], { gte: 0 });
|
||||
const result = checkRowCount(0, { gte: 0 });
|
||||
expect(result.matched).toBe(true);
|
||||
expect(result.failure).toBeNull();
|
||||
});
|
||||
|
||||
test("非数组视为 0 行", () => {
|
||||
const result = checkRowCount(null, { gte: 0 });
|
||||
test("0 行通过 gte 0", () => {
|
||||
const result = checkRowCount(0, { gte: 0 });
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("rowCount gte 通过", () => {
|
||||
const result = checkRowCount([1, 2, 3], { gte: 3 });
|
||||
const result = checkRowCount(3, { gte: 3 });
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("rowCount gte 失败", () => {
|
||||
const result = checkRowCount([1, 2], { gte: 3 });
|
||||
const result = checkRowCount(2, { gte: 3 });
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.phase).toBe("rowCount");
|
||||
expect(result.failure!.path).toBe("rowCount");
|
||||
});
|
||||
|
||||
test("rowCount equals 通过", () => {
|
||||
const result = checkRowCount([1, 2, 3], { equals: 3 });
|
||||
const result = checkRowCount(3, { equals: 3 });
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("rowCount equals 失败", () => {
|
||||
const result = checkRowCount([1, 2, 3], { equals: 5 });
|
||||
const result = checkRowCount(3, { equals: 5 });
|
||||
expect(result.matched).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -117,8 +117,8 @@ describe("checkRows", () => {
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("match 正则匹配", () => {
|
||||
const result = checkRows([{ code: "ABC-123" }], [{ code: { match: "^ABC-" } }]);
|
||||
test("regex 正则匹配", () => {
|
||||
const result = checkRows([{ code: "ABC-123" }], [{ code: { regex: "^ABC-" } }]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
|
||||
@@ -49,20 +49,20 @@ describe("validateDbConfig", () => {
|
||||
expect(unknownError!.code).toBe("unknown-field");
|
||||
});
|
||||
|
||||
test("expect.maxDurationMs 非数字返回错误", () => {
|
||||
test("expect.durationMs 非 matcher 返回错误", () => {
|
||||
const result = validateDbConfig({
|
||||
defaults: {},
|
||||
targets: [
|
||||
{
|
||||
db: { url: "sqlite://:memory:" },
|
||||
expect: { maxDurationMs: "invalid" },
|
||||
expect: { durationMs: "invalid" },
|
||||
id: "test",
|
||||
name: "test",
|
||||
type: "db",
|
||||
},
|
||||
],
|
||||
});
|
||||
const durationError = result.find((e) => e.path.includes("expect.maxDurationMs"));
|
||||
const durationError = result.find((e) => e.path.includes("expect.durationMs"));
|
||||
expect(durationError).toBeDefined();
|
||||
expect(durationError!.code).toBe("invalid-type");
|
||||
});
|
||||
@@ -76,7 +76,7 @@ describe("validateDbConfig", () => {
|
||||
});
|
||||
const rowCountError = result.find((e) => e.path.includes("expect.rowCount"));
|
||||
expect(rowCountError).toBeDefined();
|
||||
expect(rowCountError!.code).toBe("unknown-operator");
|
||||
expect(rowCountError!.code).toBe("unknown-matcher");
|
||||
});
|
||||
|
||||
test("expect.rows 不是数组返回错误", () => {
|
||||
@@ -103,13 +103,13 @@ describe("validateDbConfig", () => {
|
||||
expect(rowError!.code).toBe("invalid-type");
|
||||
});
|
||||
|
||||
test("expect.rows 中 match 正则非法返回错误", () => {
|
||||
test("expect.rows 中 regex 正则非法返回错误", () => {
|
||||
const result = validateDbConfig({
|
||||
defaults: {},
|
||||
targets: [
|
||||
{
|
||||
db: { url: "sqlite://:memory:" },
|
||||
expect: { rows: [{ name: { match: "[invalid" } }] },
|
||||
expect: { rows: [{ name: { regex: "[invalid" } }] },
|
||||
id: "test",
|
||||
name: "test",
|
||||
type: "db",
|
||||
@@ -137,7 +137,7 @@ describe("validateDbConfig", () => {
|
||||
targets: [
|
||||
{
|
||||
db: { query: "SELECT 1", url: "sqlite://:memory:" },
|
||||
expect: { maxDurationMs: 5000, rowCount: { gte: 1 }, rows: [{ cnt: { gte: 1 } }] },
|
||||
expect: { durationMs: { lte: 5000 }, rowCount: { gte: 1 }, rows: [{ cnt: { gte: 1 } }] },
|
||||
id: "test",
|
||||
name: "test",
|
||||
type: "db",
|
||||
|
||||
@@ -26,7 +26,7 @@ describe("checkHeaders", () => {
|
||||
const headers = { "content-type": "application/json" };
|
||||
|
||||
expect(checkHeaders(headers, { "content-type": { contains: "json" } }).matched).toBe(true);
|
||||
expect(checkHeaders(headers, { "content-type": { match: "^application/" } }).matched).toBe(true);
|
||||
expect(checkHeaders(headers, { "content-type": { regex: "^application/" } }).matched).toBe(true);
|
||||
expect(checkHeaders(headers, { "content-type": { contains: "xml" } }).matched).toBe(false);
|
||||
});
|
||||
|
||||
|
||||
@@ -487,7 +487,7 @@ describe("HttpChecker", () => {
|
||||
expect(result.durationMs!).toBeGreaterThanOrEqual(0);
|
||||
});
|
||||
|
||||
test("expect.maxDurationMs 使用完整耗时", async () => {
|
||||
test("expect.durationMs 使用完整耗时", async () => {
|
||||
const slowServer = Bun.serve({
|
||||
async fetch() {
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
@@ -498,7 +498,7 @@ describe("HttpChecker", () => {
|
||||
try {
|
||||
const result = await checker.execute(
|
||||
makeTarget({
|
||||
expect: { body: [{ contains: "x" }], maxDurationMs: 10 },
|
||||
expect: { body: [{ contains: "x" }], durationMs: { lte: 10 } },
|
||||
url: `http://localhost:${slowServer.port}/`,
|
||||
}),
|
||||
makeCtx(),
|
||||
@@ -521,7 +521,7 @@ describe("HttpChecker", () => {
|
||||
test("body 失败优先于 duration 检查", async () => {
|
||||
const result = await checker.execute(
|
||||
makeTarget({
|
||||
expect: { body: [{ contains: "nonexistent" }], maxDurationMs: 999999 },
|
||||
expect: { body: [{ contains: "nonexistent" }], durationMs: { lte: 999999 } },
|
||||
url: `${baseUrl}/ok`,
|
||||
}),
|
||||
makeCtx(),
|
||||
@@ -689,7 +689,7 @@ describe("HttpChecker", () => {
|
||||
name: "test",
|
||||
type: "http",
|
||||
});
|
||||
expect(errors).toContain("缺少支持的规则类型");
|
||||
expect(errors).toContain("match 是未知字段");
|
||||
});
|
||||
|
||||
test("非法 regex 启动校验失败", () => {
|
||||
@@ -722,19 +722,19 @@ describe("HttpChecker", () => {
|
||||
expect(errors).toContain("json.path");
|
||||
});
|
||||
|
||||
test("非法 operator match 启动校验失败", () => {
|
||||
test("旧 match matcher 启动校验失败", () => {
|
||||
const errors = validateHttpTarget({
|
||||
expect: { headers: { "x-test": { match: "[invalid" } } },
|
||||
http: { url: "https://example.com" },
|
||||
name: "test",
|
||||
type: "http",
|
||||
});
|
||||
expect(errors).toContain("match 正则不合法");
|
||||
expect(errors).toContain("match 是未知 matcher");
|
||||
});
|
||||
|
||||
test("ReDoS operator match 启动校验失败", () => {
|
||||
test("ReDoS regex matcher 启动校验失败", () => {
|
||||
const errors = validateHttpTarget({
|
||||
expect: { headers: { "x-test": { match: "(\\d+)*x" } } },
|
||||
expect: { headers: { "x-test": { regex: "(\\d+)*x" } } },
|
||||
http: { url: "https://example.com" },
|
||||
name: "test",
|
||||
type: "http",
|
||||
@@ -769,17 +769,17 @@ describe("HttpChecker", () => {
|
||||
name: "test",
|
||||
type: "http",
|
||||
});
|
||||
expect(errors).toContain("必须包含至少一个合法 operator");
|
||||
expect(errors).toContain("必须包含至少一个合法 matcher");
|
||||
});
|
||||
|
||||
test("body rule 多个支持字段启动失败", () => {
|
||||
const errors = validateHttpTarget({
|
||||
expect: { body: [{ contains: "ok", regex: "ok" }] },
|
||||
expect: { body: [{ contains: "ok", json: { path: "$.status" } }] },
|
||||
http: { url: "https://example.com" },
|
||||
name: "test",
|
||||
type: "http",
|
||||
});
|
||||
expect(errors).toContain("只能配置一种规则类型");
|
||||
expect(errors).toContain("直接 matcher 不能与 extractor 混用");
|
||||
});
|
||||
|
||||
test("body rule 缺少支持字段启动失败", () => {
|
||||
@@ -789,7 +789,7 @@ describe("HttpChecker", () => {
|
||||
name: "test",
|
||||
type: "http",
|
||||
});
|
||||
expect(errors).toContain("缺少支持的规则类型");
|
||||
expect(errors).toContain("foo 是未知字段");
|
||||
});
|
||||
|
||||
test("css selector 为空启动失败", () => {
|
||||
|
||||
@@ -60,7 +60,10 @@ rtt min/avg/max/mdev = 1.234/2.345/3.456/0.567 ms`);
|
||||
|
||||
test("alive 失败短路", async () => {
|
||||
mockSpawn(`3 packets transmitted, 0 received, 100% packet loss, time 2003ms`);
|
||||
const result = await checker.execute(makeTarget({ expect: { alive: true, maxAvgLatencyMs: 100 } }), makeCtx());
|
||||
const result = await checker.execute(
|
||||
makeTarget({ expect: { alive: true, avgLatencyMs: { lte: 100 } } }),
|
||||
makeCtx(),
|
||||
);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("alive");
|
||||
expect(result.statusDetail).toBe("unreachable (0/3 received)");
|
||||
@@ -75,7 +78,7 @@ rtt min/avg/max/mdev = 1.234/2.345/3.456/0.567 ms`);
|
||||
test("packetLoss 断言失败", async () => {
|
||||
mockSpawn(`3 packets transmitted, 2 received, 33% packet loss, time 2003ms
|
||||
rtt min/avg/max/mdev = 1.234/156.000/340.000/0.567 ms`);
|
||||
const result = await checker.execute(makeTarget({ expect: { maxPacketLoss: 10 } }), makeCtx());
|
||||
const result = await checker.execute(makeTarget({ expect: { packetLossPercent: { lte: 10 } } }), makeCtx());
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("packetLoss");
|
||||
expect(result.statusDetail).toContain("max 340ms");
|
||||
|
||||
@@ -16,22 +16,22 @@ describe("ping expect", () => {
|
||||
});
|
||||
|
||||
test("packetLoss 通过和失败", () => {
|
||||
expect(checkPacketLoss(0, 10).matched).toBe(true);
|
||||
const result = checkPacketLoss(33, 10);
|
||||
expect(checkPacketLoss(0, { lte: 10 }).matched).toBe(true);
|
||||
const result = checkPacketLoss(33, { lte: 10 });
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("packetLoss");
|
||||
});
|
||||
|
||||
test("avgLatency 通过和失败", () => {
|
||||
expect(checkAvgLatency(12, 200).matched).toBe(true);
|
||||
const result = checkAvgLatency(156, 100);
|
||||
expect(checkAvgLatency(12, { lte: 200 }).matched).toBe(true);
|
||||
const result = checkAvgLatency(156, { lte: 100 });
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("avgLatency");
|
||||
});
|
||||
|
||||
test("maxLatency 通过和失败", () => {
|
||||
expect(checkMaxLatency(340, 500).matched).toBe(true);
|
||||
const result = checkMaxLatency(340, 200);
|
||||
expect(checkMaxLatency(340, { lte: 500 }).matched).toBe(true);
|
||||
const result = checkMaxLatency(340, { lte: 200 });
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("maxLatency");
|
||||
});
|
||||
|
||||
@@ -43,24 +43,24 @@ describe("validatePingConfig", () => {
|
||||
expect(issues.some((item) => item.path.endsWith("expect.status"))).toBe(true);
|
||||
});
|
||||
|
||||
test("expect 数值非法", () => {
|
||||
test("expect 数值旧字段非法", () => {
|
||||
const issues = validate({ expect: { maxPacketLoss: 101 }, id: "ping", ping: { host: "127.0.0.1" }, type: "ping" });
|
||||
expect(issues.some((item) => item.path.endsWith("expect.maxPacketLoss"))).toBe(true);
|
||||
});
|
||||
|
||||
test("maxDurationMs 类型非法", () => {
|
||||
const issues = validate({ expect: { maxDurationMs: -1 }, id: "ping", ping: { host: "127.0.0.1" }, type: "ping" });
|
||||
expect(issues.some((item) => item.path.endsWith("expect.maxDurationMs"))).toBe(true);
|
||||
test("durationMs 类型非法", () => {
|
||||
const issues = validate({ expect: { durationMs: -1 }, id: "ping", ping: { host: "127.0.0.1" }, type: "ping" });
|
||||
expect(issues.some((item) => item.path.endsWith("expect.durationMs"))).toBe(true);
|
||||
});
|
||||
|
||||
test("maxAvgLatencyMs 类型非法", () => {
|
||||
test("avgLatencyMs 类型非法", () => {
|
||||
const issues = validate({
|
||||
expect: { maxAvgLatencyMs: "slow" },
|
||||
expect: { avgLatencyMs: "slow" },
|
||||
id: "ping",
|
||||
ping: { host: "127.0.0.1" },
|
||||
type: "ping",
|
||||
});
|
||||
expect(issues.some((item) => item.path.endsWith("expect.maxAvgLatencyMs"))).toBe(true);
|
||||
expect(issues.some((item) => item.path.endsWith("expect.avgLatencyMs"))).toBe(true);
|
||||
});
|
||||
|
||||
test("host 为空字符串", () => {
|
||||
|
||||
@@ -135,7 +135,7 @@ describe("LlmChecker execute - 非流式", () => {
|
||||
});
|
||||
|
||||
test("finishReason expect 不匹配", async () => {
|
||||
const result = await checker.execute(makeTarget(undefined, { finishReason: "length" }), makeCtx());
|
||||
const result = await checker.execute(makeTarget(undefined, { finishReason: { equals: "length" } }), makeCtx());
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("finishReason");
|
||||
});
|
||||
|
||||
@@ -2,8 +2,12 @@ import { describe, expect, test } from "bun:test";
|
||||
|
||||
import type { LlmCheckObservation } from "../../../../../src/server/checker/runner/llm/types";
|
||||
|
||||
import { checkContentRules } from "../../../../../src/server/checker/expect/content";
|
||||
import { runExpects } from "../../../../../src/server/checker/runner/llm/expect";
|
||||
import { checkOutputRules } from "../../../../../src/server/checker/runner/llm/output";
|
||||
|
||||
function checkOutputRules(outputText: null | string, rules: Parameters<typeof checkContentRules>[1]) {
|
||||
return checkContentRules(outputText, rules, { path: "output", phase: "output" });
|
||||
}
|
||||
|
||||
function makeObservation(overrides?: Partial<LlmCheckObservation>): LlmCheckObservation {
|
||||
return {
|
||||
@@ -72,7 +76,7 @@ describe("LLM runExpects", () => {
|
||||
test("全部 expect 通过", () => {
|
||||
const observation = makeObservation();
|
||||
const result = runExpects(observation, {
|
||||
finishReason: "stop",
|
||||
finishReason: { equals: "stop" },
|
||||
output: [{ contains: "OK" }],
|
||||
status: [200],
|
||||
});
|
||||
@@ -95,14 +99,14 @@ describe("LLM runExpects", () => {
|
||||
|
||||
test("finishReason 不匹配失败", () => {
|
||||
const observation = makeObservation();
|
||||
const result = runExpects(observation, { finishReason: "length" });
|
||||
const result = runExpects(observation, { finishReason: { equals: "length" } });
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("finishReason");
|
||||
});
|
||||
|
||||
test("rawFinishReason 不匹配失败", () => {
|
||||
const observation = makeObservation();
|
||||
const result = runExpects(observation, { rawFinishReason: "end_turn" });
|
||||
const result = runExpects(observation, { rawFinishReason: { equals: "end_turn" } });
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("rawFinishReason");
|
||||
});
|
||||
|
||||
@@ -206,15 +206,15 @@ describe("LlmChecker validate", () => {
|
||||
defaults: {},
|
||||
targets: [makeRawTarget({ expect: { output: [{}] } })],
|
||||
});
|
||||
expect(issues.some((i) => i.code === "missing-body-rule")).toBe(true);
|
||||
expect(issues.some((i) => i.code === "empty-matcher")).toBe(true);
|
||||
});
|
||||
|
||||
test("expect.output 同时多种规则类型报错", () => {
|
||||
test("expect.output 直接 matcher 混入 extractor 报错", () => {
|
||||
const issues = validateLlmConfig({
|
||||
defaults: {},
|
||||
targets: [makeRawTarget({ expect: { output: [{ contains: "y", equals: "x" }] } })],
|
||||
targets: [makeRawTarget({ expect: { output: [{ contains: "y", json: { path: "$.status" } }] } })],
|
||||
});
|
||||
expect(issues.some((i) => i.code === "multiple-body-rules")).toBe(true);
|
||||
expect(issues.some((i) => i.code === "invalid-content-rule")).toBe(true);
|
||||
});
|
||||
|
||||
test("expect.output regex ReDoS 报错", () => {
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import { checkBodyExpect } from "../../../../../src/server/checker/runner/http/body";
|
||||
import { checkContentRules } from "../../../../../src/server/checker/expect/content";
|
||||
|
||||
function checkBodyExpect(body: string, rules?: Parameters<typeof checkContentRules>[1]) {
|
||||
return checkContentRules(body, rules, { path: "body", phase: "body" });
|
||||
}
|
||||
|
||||
describe("checkBodyExpect (BodyRule[])", () => {
|
||||
test("无规则返回匹配成功", () => {
|
||||
@@ -57,7 +61,7 @@ describe("checkBodyExpect (BodyRule[])", () => {
|
||||
test("json 操作符匹配", () => {
|
||||
const body = JSON.stringify({ count: 42, version: "v2.1.0" });
|
||||
expect(checkBodyExpect(body, [{ json: { gte: 10, path: "$.count" } }]).matched).toBe(true);
|
||||
expect(checkBodyExpect(body, [{ json: { match: "\\d+\\.\\d+\\.\\d+", path: "$.version" } }]).matched).toBe(true);
|
||||
expect(checkBodyExpect(body, [{ json: { path: "$.version", regex: "\\d+\\.\\d+\\.\\d+" } }]).matched).toBe(true);
|
||||
expect(checkBodyExpect(body, [{ json: { gte: 100, path: "$.count" } }]).matched).toBe(false);
|
||||
});
|
||||
|
||||
|
||||
@@ -1,6 +1,13 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import { checkDuration } from "../../../../../src/server/checker/expect/duration";
|
||||
import { checkValueMatcher } from "../../../../../src/server/checker/expect/matcher";
|
||||
|
||||
function checkDuration(durationMs: number, maxDurationMs?: number) {
|
||||
return checkValueMatcher(durationMs, maxDurationMs === undefined ? undefined : { lte: maxDurationMs }, {
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
});
|
||||
}
|
||||
|
||||
describe("checkDuration", () => {
|
||||
test("未配置 maxDurationMs 返回匹配成功", () => {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import { applyOperator, checkExpectValue, evaluateJsonPath } from "../../../../../src/server/checker/expect/operator";
|
||||
import { applyMatcher, checkExpectValue, evaluateJsonPath } from "../../../../../src/server/checker/expect/matcher";
|
||||
|
||||
describe("evaluateJsonPath", () => {
|
||||
const obj = {
|
||||
@@ -55,81 +55,83 @@ describe("evaluateJsonPath", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("applyOperator", () => {
|
||||
describe("applyMatcher", () => {
|
||||
test("equals 操作符", () => {
|
||||
expect(applyOperator("ok", { equals: "ok" })).toBe(true);
|
||||
expect(applyOperator("ok", { equals: "error" })).toBe(false);
|
||||
expect(applyOperator(42, { equals: 42 })).toBe(true);
|
||||
expect(applyOperator(42, { equals: 41 })).toBe(false);
|
||||
expect(applyOperator(null, { equals: null })).toBe(true);
|
||||
expect(applyOperator(true, { equals: true })).toBe(true);
|
||||
expect(applyMatcher("ok", { equals: "ok" })).toBe(true);
|
||||
expect(applyMatcher("ok", { equals: "error" })).toBe(false);
|
||||
expect(applyMatcher(42, { equals: 42 })).toBe(true);
|
||||
expect(applyMatcher(42, { equals: 41 })).toBe(false);
|
||||
expect(applyMatcher(null, { equals: null })).toBe(true);
|
||||
expect(applyMatcher(true, { equals: true })).toBe(true);
|
||||
});
|
||||
|
||||
test("equals 支持 JSON 对象和数组", () => {
|
||||
expect(applyOperator({ status: "ok" }, { equals: { status: "ok" } })).toBe(true);
|
||||
expect(applyOperator({ status: "ok" }, { equals: { status: "fail" } })).toBe(false);
|
||||
expect(applyOperator(["a", "b"], { equals: ["a", "b"] })).toBe(true);
|
||||
expect(applyOperator(["a", "b"], { equals: ["b", "a"] })).toBe(false);
|
||||
expect(applyMatcher({ status: "ok" }, { equals: { status: "ok" } })).toBe(true);
|
||||
expect(applyMatcher({ status: "ok" }, { equals: { status: "fail" } })).toBe(false);
|
||||
expect(applyMatcher(["a", "b"], { equals: ["a", "b"] })).toBe(true);
|
||||
expect(applyMatcher(["a", "b"], { equals: ["b", "a"] })).toBe(false);
|
||||
});
|
||||
|
||||
test("contains 操作符", () => {
|
||||
expect(applyOperator("hello world", { contains: "hello" })).toBe(true);
|
||||
expect(applyOperator("hello world", { contains: "missing" })).toBe(false);
|
||||
expect(applyOperator(12345, { contains: "23" })).toBe(true);
|
||||
expect(applyMatcher("hello world", { contains: "hello" })).toBe(true);
|
||||
expect(applyMatcher("hello world", { contains: "missing" })).toBe(false);
|
||||
expect(applyMatcher(12345, { contains: "23" })).toBe(true);
|
||||
});
|
||||
|
||||
test("match 操作符", () => {
|
||||
expect(applyOperator("v2.1.0", { match: "\\d+\\.\\d+\\.\\d+" })).toBe(true);
|
||||
expect(applyOperator("v2.1", { match: "\\d+\\.\\d+\\.\\d+" })).toBe(false);
|
||||
expect(applyOperator("abc123", { match: "^\\w+\\d+$" })).toBe(true);
|
||||
test("regex matcher", () => {
|
||||
expect(applyMatcher("v2.1.0", { regex: "\\d+\\.\\d+\\.\\d+" })).toBe(true);
|
||||
expect(applyMatcher("v2.1", { regex: "\\d+\\.\\d+\\.\\d+" })).toBe(false);
|
||||
expect(applyMatcher("abc123", { regex: "^\\w+\\d+$" })).toBe(true);
|
||||
});
|
||||
|
||||
test("empty 操作符", () => {
|
||||
expect(applyOperator("", { empty: true })).toBe(true);
|
||||
expect(applyOperator(null, { empty: true })).toBe(true);
|
||||
expect(applyOperator(undefined, { empty: true })).toBe(true);
|
||||
expect(applyOperator([], { empty: true })).toBe(true);
|
||||
expect(applyOperator({}, { empty: true })).toBe(true);
|
||||
expect(applyOperator("ok", { empty: true })).toBe(false);
|
||||
expect(applyOperator([1, 2], { empty: false })).toBe(true);
|
||||
expect(applyOperator([], { empty: false })).toBe(false);
|
||||
expect(applyMatcher("", { empty: true })).toBe(true);
|
||||
expect(applyMatcher(null, { empty: true })).toBe(true);
|
||||
expect(applyMatcher(undefined, { empty: true })).toBe(true);
|
||||
expect(applyMatcher([], { empty: true })).toBe(true);
|
||||
expect(applyMatcher({}, { empty: true })).toBe(true);
|
||||
expect(applyMatcher("ok", { empty: true })).toBe(false);
|
||||
expect(applyMatcher(0, { empty: true })).toBe(false);
|
||||
expect(applyMatcher(false, { empty: true })).toBe(false);
|
||||
expect(applyMatcher([1, 2], { empty: false })).toBe(true);
|
||||
expect(applyMatcher([], { empty: false })).toBe(false);
|
||||
});
|
||||
|
||||
test("exists 操作符", () => {
|
||||
expect(applyOperator("ok", { exists: true })).toBe(true);
|
||||
expect(applyOperator(null, { exists: true })).toBe(true);
|
||||
expect(applyOperator(undefined, { exists: true })).toBe(false);
|
||||
expect(applyOperator(undefined, { exists: false })).toBe(true);
|
||||
expect(applyOperator("ok", { exists: false })).toBe(false);
|
||||
expect(applyMatcher("ok", { exists: true })).toBe(true);
|
||||
expect(applyMatcher(null, { exists: true })).toBe(true);
|
||||
expect(applyMatcher(undefined, { exists: true })).toBe(false);
|
||||
expect(applyMatcher(undefined, { exists: false })).toBe(true);
|
||||
expect(applyMatcher("ok", { exists: false })).toBe(false);
|
||||
});
|
||||
|
||||
test("gte 操作符", () => {
|
||||
expect(applyOperator(10, { gte: 5 })).toBe(true);
|
||||
expect(applyOperator(5, { gte: 5 })).toBe(true);
|
||||
expect(applyOperator(3, { gte: 5 })).toBe(false);
|
||||
expect(applyOperator("10", { gte: 5 })).toBe(true);
|
||||
expect(applyMatcher(10, { gte: 5 })).toBe(true);
|
||||
expect(applyMatcher(5, { gte: 5 })).toBe(true);
|
||||
expect(applyMatcher(3, { gte: 5 })).toBe(false);
|
||||
expect(applyMatcher("10", { gte: 5 })).toBe(true);
|
||||
});
|
||||
|
||||
test("lte 操作符", () => {
|
||||
expect(applyOperator(3, { lte: 5 })).toBe(true);
|
||||
expect(applyOperator(5, { lte: 5 })).toBe(true);
|
||||
expect(applyOperator(10, { lte: 5 })).toBe(false);
|
||||
expect(applyMatcher(3, { lte: 5 })).toBe(true);
|
||||
expect(applyMatcher(5, { lte: 5 })).toBe(true);
|
||||
expect(applyMatcher(10, { lte: 5 })).toBe(false);
|
||||
});
|
||||
|
||||
test("gt 操作符", () => {
|
||||
expect(applyOperator(10, { gt: 5 })).toBe(true);
|
||||
expect(applyOperator(5, { gt: 5 })).toBe(false);
|
||||
expect(applyMatcher(10, { gt: 5 })).toBe(true);
|
||||
expect(applyMatcher(5, { gt: 5 })).toBe(false);
|
||||
});
|
||||
|
||||
test("lt 操作符", () => {
|
||||
expect(applyOperator(3, { lt: 5 })).toBe(true);
|
||||
expect(applyOperator(5, { lt: 5 })).toBe(false);
|
||||
expect(applyMatcher(3, { lt: 5 })).toBe(true);
|
||||
expect(applyMatcher(5, { lt: 5 })).toBe(false);
|
||||
});
|
||||
|
||||
test("多操作符 AND 组合", () => {
|
||||
expect(applyOperator(7, { gte: 5, lte: 10 })).toBe(true);
|
||||
expect(applyOperator(3, { gte: 5, lte: 10 })).toBe(false);
|
||||
expect(applyOperator(15, { gte: 5, lte: 10 })).toBe(false);
|
||||
expect(applyMatcher(7, { gte: 5, lte: 10 })).toBe(true);
|
||||
expect(applyMatcher(3, { gte: 5, lte: 10 })).toBe(false);
|
||||
expect(applyMatcher(15, { gte: 5, lte: 10 })).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import { checkTextRules } from "../../../../../src/server/checker/runner/cmd/text";
|
||||
import { checkContentRules } from "../../../../../src/server/checker/expect/content";
|
||||
|
||||
function checkTextRules(text: string, rules: Parameters<typeof checkContentRules>[1], phase: string) {
|
||||
return checkContentRules(text, rules, { path: phase, phase });
|
||||
}
|
||||
|
||||
describe("checkTextRules", () => {
|
||||
test("无规则返回匹配成功", () => {
|
||||
@@ -24,7 +28,7 @@ describe("checkTextRules", () => {
|
||||
test("多条规则全部通过", () => {
|
||||
const r = checkTextRules(
|
||||
"version: 3.2.1, build: ok",
|
||||
[{ contains: "version" }, { match: "\\d+\\.\\d+\\.\\d+" }],
|
||||
[{ contains: "version" }, { regex: "\\d+\\.\\d+\\.\\d+" }],
|
||||
"stdout",
|
||||
);
|
||||
expect(r.matched).toBe(true);
|
||||
|
||||
@@ -155,8 +155,8 @@ describe("TcpChecker execute", () => {
|
||||
expect(result.failure!.phase).toBe("connected");
|
||||
});
|
||||
|
||||
test("maxDurationMs 超时返回失败", async () => {
|
||||
const result = await checker.execute(makeTarget({}, { expect: { maxDurationMs: -1 } }), makeCtx());
|
||||
test("durationMs 超时返回失败", async () => {
|
||||
const result = await checker.execute(makeTarget({}, { expect: { durationMs: { lt: 0 } } }), makeCtx());
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.phase).toBe("duration");
|
||||
});
|
||||
@@ -179,7 +179,7 @@ describe("TcpChecker execute", () => {
|
||||
const result = await checker.execute(
|
||||
makeTarget(
|
||||
{ host: "127.0.0.1", port: bannerServerPort, readBanner: true },
|
||||
{ expect: { banner: { contains: "ESMTP" } } },
|
||||
{ expect: { banner: [{ contains: "ESMTP" }] } },
|
||||
),
|
||||
makeCtx(),
|
||||
);
|
||||
@@ -190,14 +190,14 @@ describe("TcpChecker execute", () => {
|
||||
const result = await checker.execute(
|
||||
makeTarget(
|
||||
{ host: "127.0.0.1", port: bannerServerPort, readBanner: true },
|
||||
{ expect: { banner: { contains: "POSTFIX" } } },
|
||||
{ expect: { banner: [{ contains: "POSTFIX" }] } },
|
||||
),
|
||||
makeCtx(),
|
||||
);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.kind).toBe("mismatch");
|
||||
expect(result.failure!.phase).toBe("banner");
|
||||
expect(result.failure!.path).toBe("banner");
|
||||
expect(result.failure!.path).toBe("banner[0]");
|
||||
});
|
||||
|
||||
test("默认不读取 banner", async () => {
|
||||
@@ -346,14 +346,14 @@ describe("TcpChecker resolve", () => {
|
||||
test("expect 配置解析", () => {
|
||||
const target = checker.resolve(
|
||||
{
|
||||
expect: { banner: { contains: "ESMTP" }, connected: false, maxDurationMs: 5000 },
|
||||
expect: { banner: [{ contains: "ESMTP" }], connected: false, durationMs: { lte: 5000 } },
|
||||
id: "t",
|
||||
tcp: { host: "127.0.0.1", port: 80, readBanner: true },
|
||||
type: "tcp",
|
||||
},
|
||||
{ configDir: "/tmp", defaultIntervalMs: 30000, defaults: {}, defaultTimeoutMs: 10000 },
|
||||
);
|
||||
expect(target.expect).toEqual({ banner: { contains: "ESMTP" }, connected: false, maxDurationMs: 5000 });
|
||||
expect(target.expect).toEqual({ banner: [{ contains: "ESMTP" }], connected: false, durationMs: { lte: 5000 } });
|
||||
});
|
||||
|
||||
test("name 和 group 解析", () => {
|
||||
|
||||
@@ -32,34 +32,34 @@ describe("checkConnected", () => {
|
||||
|
||||
describe("checkBanner", () => {
|
||||
test("contains 匹配", () => {
|
||||
const result = checkBanner("220 smtp.example.com ESMTP", { contains: "ESMTP" });
|
||||
const result = checkBanner("220 smtp.example.com ESMTP", [{ contains: "ESMTP" }]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("contains 不匹配", () => {
|
||||
const result = checkBanner("220 smtp.example.com ESMTP", { contains: "POSTFIX" });
|
||||
const result = checkBanner("220 smtp.example.com ESMTP", [{ contains: "POSTFIX" }]);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.kind).toBe("mismatch");
|
||||
expect(result.failure!.phase).toBe("banner");
|
||||
});
|
||||
|
||||
test("match 正则匹配", () => {
|
||||
const result = checkBanner("220 smtp.example.com ESMTP", { match: "^220" });
|
||||
test("regex 正则匹配", () => {
|
||||
const result = checkBanner("220 smtp.example.com ESMTP", [{ regex: "^220" }]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("空 banner 与 contains 空字符串", () => {
|
||||
const result = checkBanner("", { contains: "" });
|
||||
const result = checkBanner("", [{ contains: "" }]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("多 operator 同时匹配", () => {
|
||||
const result = checkBanner("220 ESMTP", { contains: "ESMTP", match: "^220" });
|
||||
const result = checkBanner("220 ESMTP", [{ contains: "ESMTP", regex: "^220" }]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("多 operator 部分不匹配", () => {
|
||||
const result = checkBanner("220 ESMTP", { contains: "ESMTP", match: "^250" });
|
||||
const result = checkBanner("220 ESMTP", [{ contains: "ESMTP", regex: "^250" }]);
|
||||
expect(result.matched).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -68,7 +68,7 @@ describe("validateTcpConfig", () => {
|
||||
const issues = validateTcpConfig(
|
||||
makeInput([
|
||||
{
|
||||
expect: { banner: { contains: "ESMTP" } },
|
||||
expect: { banner: [{ contains: "ESMTP" }] },
|
||||
id: "t1",
|
||||
tcp: { host: "127.0.0.1", port: 25 },
|
||||
type: "tcp",
|
||||
@@ -82,7 +82,7 @@ describe("validateTcpConfig", () => {
|
||||
const issues = validateTcpConfig(
|
||||
makeInput([
|
||||
{
|
||||
expect: { banner: { contains: "ESMTP" } },
|
||||
expect: { banner: [{ contains: "ESMTP" }] },
|
||||
id: "t1",
|
||||
tcp: { host: "127.0.0.1", port: 25, readBanner: true },
|
||||
type: "tcp",
|
||||
@@ -106,18 +106,18 @@ describe("validateTcpConfig", () => {
|
||||
expect(issues.some((i) => i.path.includes("connected"))).toBe(true);
|
||||
});
|
||||
|
||||
test("expect maxDurationMs 非数字", () => {
|
||||
test("expect durationMs 非 matcher", () => {
|
||||
const issues = validateTcpConfig(
|
||||
makeInput([
|
||||
{
|
||||
expect: { maxDurationMs: "slow" },
|
||||
expect: { durationMs: "slow" },
|
||||
id: "t1",
|
||||
tcp: { host: "127.0.0.1", port: 80 },
|
||||
type: "tcp",
|
||||
},
|
||||
]),
|
||||
);
|
||||
expect(issues.some((i) => i.path.includes("maxDurationMs"))).toBe(true);
|
||||
expect(issues.some((i) => i.path.includes("durationMs"))).toBe(true);
|
||||
});
|
||||
|
||||
test("expect 未知字段", () => {
|
||||
@@ -134,11 +134,11 @@ describe("validateTcpConfig", () => {
|
||||
expect(issues.some((i) => i.message.includes("未知字段"))).toBe(true);
|
||||
});
|
||||
|
||||
test("expect.banner match 正则非法", () => {
|
||||
test("expect.banner regex 正则非法", () => {
|
||||
const issues = validateTcpConfig(
|
||||
makeInput([
|
||||
{
|
||||
expect: { banner: { match: "[invalid" } },
|
||||
expect: { banner: [{ regex: "[invalid" }] },
|
||||
id: "t1",
|
||||
tcp: { host: "127.0.0.1", port: 25, readBanner: true },
|
||||
type: "tcp",
|
||||
|
||||
@@ -250,7 +250,7 @@ describe("UdpChecker execute", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("should fail when duration exceeds maxDurationMs", async () => {
|
||||
it("should fail when duration exceeds durationMs", async () => {
|
||||
const server = await Bun.udpSocket({
|
||||
socket: {
|
||||
data() {
|
||||
@@ -266,7 +266,7 @@ describe("UdpChecker execute", () => {
|
||||
});
|
||||
try {
|
||||
const checker = new UdpChecker();
|
||||
const target = makeTarget({ port: server.port }, { maxDurationMs: 1, responded: false });
|
||||
const target = makeTarget({ port: server.port }, { durationMs: { lte: 1 }, responded: false });
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), 200);
|
||||
const result = await checker.execute(target, { signal: controller.signal });
|
||||
|
||||
@@ -76,13 +76,13 @@ describe("checkResponseText", () => {
|
||||
});
|
||||
|
||||
it("多条规则全部匹配", () => {
|
||||
const result = checkResponseText("hello world", [{ contains: "hello" }, { match: "^hello" }]);
|
||||
const result = checkResponseText("hello world", [{ contains: "hello" }, { regex: "^hello" }]);
|
||||
expect(result.matched).toBe(true);
|
||||
expect(result.failure).toBeNull();
|
||||
});
|
||||
|
||||
it("多条规则第二条失败 → 不匹配", () => {
|
||||
const result = checkResponseText("hello world", [{ contains: "hello" }, { match: "^world" }]);
|
||||
const result = checkResponseText("hello world", [{ contains: "hello" }, { regex: "^world" }]);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.phase).toBe("response");
|
||||
expect(result.failure!.path).toBe("response[1]");
|
||||
|
||||
@@ -213,12 +213,12 @@ describe("validateUdpConfig", () => {
|
||||
expect(issues.some((i) => i.path.includes("responded") && i.message.includes("响应来源"))).toBe(true);
|
||||
});
|
||||
|
||||
it("reports invalid-type for negative expect.maxDurationMs", () => {
|
||||
it("reports invalid-type for non-matcher expect.durationMs", () => {
|
||||
const issues = validateUdpConfig(
|
||||
makeInput({
|
||||
targets: [
|
||||
{
|
||||
expect: { maxDurationMs: -100 },
|
||||
expect: { durationMs: -100 },
|
||||
id: "test",
|
||||
type: "udp",
|
||||
udp: { host: "127.0.0.1", port: 53 },
|
||||
@@ -226,7 +226,7 @@ describe("validateUdpConfig", () => {
|
||||
],
|
||||
}),
|
||||
);
|
||||
expect(issues.some((i) => i.code === "invalid-type" && i.path.includes("maxDurationMs"))).toBe(true);
|
||||
expect(issues.some((i) => i.code === "invalid-type" && i.path.includes("durationMs"))).toBe(true);
|
||||
});
|
||||
|
||||
it("reports invalid-type for non-boolean expect.responded", () => {
|
||||
|
||||
@@ -26,7 +26,7 @@ beforeAll(() => {
|
||||
|
||||
const httpTarget: ResolvedHttpTarget = {
|
||||
description: null,
|
||||
expect: { maxDurationMs: 3000, status: [200] },
|
||||
expect: { durationMs: { lte: 3000 }, status: [200] },
|
||||
group: "default",
|
||||
http: {
|
||||
headers: { Accept: "application/json" },
|
||||
@@ -106,7 +106,7 @@ describe("ProbeStore", () => {
|
||||
expect(config.maxRedirects).toBe(0);
|
||||
expect(t.interval_ms).toBe(30000);
|
||||
expect(t.timeout_ms).toBe(10000);
|
||||
expect(JSON.parse(t.expect!)).toEqual({ maxDurationMs: 3000, status: [200] });
|
||||
expect(JSON.parse(t.expect!)).toEqual({ durationMs: { lte: 3000 }, status: [200] });
|
||||
});
|
||||
|
||||
test("cmd target 字段正确", () => {
|
||||
|
||||
Reference in New Issue
Block a user