refactor: expect 类型模型重构，Raw/Resolved 双层分离与断言基础设施内聚
- 重命名 ContentRules→ContentExpectations, KeyValueExpect→KeyedExpectations - 新增 Raw/Resolved 双层模型：resolve 阶段物化为执行计划，store 持久化 Raw 快照 - HTTP body 按需读取：status/headers 失败或无 body expectation 时不读取 body - 新增 displayValueExpectation() 解包 failure.expected 用户可读展示 - 修复 checkEarlyTimeout 独立 lte/lt 检查，修复 KeyedExpectations JSON Schema - 新增 expect/value.ts(resolve/check/display)、keyed.ts、content.ts、headers.ts、status.ts - 删除旧 normalize.ts/matcher.ts/validate-matcher.ts/key-value.ts - 更新 DEVELOPMENT.md：expect 五层管线表、displayValueExpectation、1.7↔1.10 交叉引用 - 同步 13 个 main specs，归档 refactor-expect-type-model 变更（62/62 tasks）
This commit is contained in:
134
DEVELOPMENT.md
134
DEVELOPMENT.md
@@ -39,7 +39,7 @@ src/
|
||||
variables.ts 配置 variables 提取、target 字符串变量替换和 unresolved-variable issue 生成
|
||||
schema/ TypeBox + Ajv 配置契约、schema fragments、issue 渲染和 schema 导出入口
|
||||
builder.ts 全量 JSON Schema 组装（遍历 registry 生成）
|
||||
fragments.ts 共享 TypeBox schema 片段（duration、size、ValueMatcher、ContentRules、KeyValueExpect 等）
|
||||
fragments.ts 共享 TypeBox schema 片段（duration、size、ValueMatcher、ContentExpectations、KeyedExpectations 等）
|
||||
validate.ts Ajv 契约校验入口
|
||||
issues.ts 校验问题类型与渲染
|
||||
types.ts schema 层类型
|
||||
@@ -48,12 +48,14 @@ src/
|
||||
engine.ts 调度引擎（按 interval 分组的 es-toolkit groupBy + Semaphore 并发控制 + 数据清理）
|
||||
utils.ts 共享工具函数（parseSize、parseDuration）
|
||||
expect/ 共享 expect 断言基础设施（跨 checker 复用）
|
||||
types.ts ExpectResult、ValueMatcher、ContentRules、KeyValueExpect 类型
|
||||
types.ts Raw/Resolved ValueExpectation、ContentExpectations、KeyedExpectations、ExpectationResult 类型
|
||||
failure.ts 失败信息构造（errorFailure、mismatchFailure、truncateActual）
|
||||
matcher.ts ValueMatcher 执行、JSONPath 提取、字面量 equals 快捷语义
|
||||
content.ts ContentRules 执行（direct/json/css/xpath）
|
||||
key-value.ts KeyValueExpect 执行（动态键与 key 规范化）
|
||||
validate-matcher.ts matcher/content/key-value 语义校验
|
||||
value.ts ValueExpectation resolve（primitive→equals）和执行、JSONPath 提取
|
||||
content.ts Resolved ContentExpectations 执行（kind=value/json/css/xpath）和 Raw resolve
|
||||
keyed.ts Resolved KeyedExpectations 执行（顺序 + key 规范化）和 Raw resolve
|
||||
headers.ts HTTP/LLM 共享 header keyed expectation 包装（大小写不敏感）
|
||||
status.ts HTTP/LLM 共享 status code 断言（精确数值与 1xx-5xx 范围）
|
||||
validate.ts Raw value/content/keyed expectation 语义校验（不修改输入）
|
||||
redos.ts regex ReDoS 风险检测
|
||||
runner/ Checker 统一抽象与注册机制
|
||||
types.ts CheckerDefinition、CheckerContext、CheckerSchemas、ResolveContext
|
||||
@@ -266,9 +268,9 @@ checkerRegistry（单例）
|
||||
每个 checker 目录的标准文件结构：
|
||||
|
||||
| 文件 | 职责 |
|
||||
| ------------- | ------------------------------------------------------------------------------------- |
|
||||
| ------------- | ------------------------------------------------------------------------------------------ |
|
||||
| `index.ts` | 模块入口，re-export Checker 类 |
|
||||
| `types.ts` | Checker 专属类型（ResolvedXxxTarget、XxxTargetConfig、XxxExpectConfig 等） |
|
||||
| `types.ts` | Checker 专属类型（RawXxxTargetConfig、Raw/Resolved XxxExpectConfig、ResolvedXxxTarget 等） |
|
||||
| `schema.ts` | TypeBox 契约 schema（config / defaults / expect 三部分） |
|
||||
| `validate.ts` | 启动期语义校验（JSON Schema 无法表达的规则） |
|
||||
| `execute.ts` | Checker 类：resolve（默认值合并 + 解析）、execute（执行检查）、serialize（DB 持久化） |
|
||||
@@ -279,8 +281,8 @@ checkerRegistry（单例）
|
||||
|
||||
在 `src/server/checker/runner/tcp/types.ts` 中定义 checker 专属类型（参考 `http/types.ts`、`cmd/types.ts`）：
|
||||
|
||||
- `XxxTargetConfig` — YAML 原始配置类型
|
||||
- `XxxExpectConfig` — expect 字段类型
|
||||
- `RawXxxTargetConfig` — YAML 原始配置类型
|
||||
- `RawXxxExpectConfig` / `ResolvedXxxExpectConfig` — Raw expect 字段类型与运行期 Resolved expect 执行计划类型
|
||||
- `XxxDefaultsConfig` — defaults 专属字段类型
|
||||
- `ResolvedXxxTarget extends ResolvedTargetBase` — resolve 后的完整类型，含 `type: "xxx"` 字面量
|
||||
|
||||
@@ -293,14 +295,14 @@ checkerRegistry（单例）
|
||||
**可复用的共享 fragments**（来自 `schema/fragments.ts`）：
|
||||
|
||||
| Fragment | 用途 |
|
||||
| ------------------------------ | -------------------------------------------------------- |
|
||||
| ----------------------------------- | ----------------------------------------------------------- |
|
||||
| `durationSchema` | 时长字符串（`"30s"`、`"5m"`、`"2h"`、`"7d"`、`"500ms"`） |
|
||||
| `sizeSchema` | 大小单位（字符串如 `"10MB"` 或整数） |
|
||||
| `statusCodePatternSchema` | 状态码（`100`-`599` 或 `"2xx"`） |
|
||||
| `stringMapSchema` | `Record<string, string>`（用于 headers / env） |
|
||||
| `createValueMatcherSchema()` | `ValueMatcher` 对象（equals/contains/regex/数值比较等） |
|
||||
| `createContentRulesSchema()` | `ContentRules` 数组（direct/json/css/xpath 内容规则） |
|
||||
| `createKeyValueExpectSchema()` | 动态键 `KeyValueExpect`（headers、DB rows 列值） |
|
||||
| `createContentExpectationsSchema()` | `ContentExpectations` 数组（value/json/css/xpath 内容断言） |
|
||||
| `createKeyedExpectationsSchema()` | 动态键 `KeyedExpectations`（headers、DB rows 列值） |
|
||||
| `matcherProperties()` | matcher 字段 Record，供 extractor schema 复用 |
|
||||
|
||||
**注意**：默认对象策略为 `additionalProperties: false`。只有明确的动态键值表（如 `http.headers`、`cmd.env`）可以开放任意键名。
|
||||
@@ -313,13 +315,13 @@ checkerRegistry（单例）
|
||||
export function validateTcpConfig(input: CheckerValidationInput): ConfigValidationIssue[];
|
||||
```
|
||||
|
||||
**共享校验工具**（`expect/validate-matcher.ts`）：
|
||||
**共享校验工具**（`expect/validate.ts`）：
|
||||
|
||||
| 函数 | 用途 |
|
||||
| ------------------------------------------------------ | ----------------------------------------- |
|
||||
| `validateValueMatcher(value, path, targetName, opts?)` | 校验 matcher 字段、类型、regex 和组合语义 |
|
||||
| `validateContentRules(rules, path, targetName)` | 校验 ContentRules 数组、extractor 互斥性 |
|
||||
| `validateKeyValueExpect(value, path, targetName)` | 校验动态键值断言 |
|
||||
| -------------------------------------------------------------- | --------------------------------------------------- |
|
||||
| `validateRawValueExpectation(value, path, targetName, opts?)` | 校验 Raw `ValueExpectation`（primitive 或 matcher） |
|
||||
| `validateRawContentExpectations(value, path, targetName)` | 校验 Raw `ContentExpectations` 数组、extractor 互斥 |
|
||||
| `validateRawKeyedExpectations(value, path, targetName, opts?)` | 校验 Raw `KeyedExpectations`，可选大小写不敏感重复 |
|
||||
| `validateJsonPath(path, rulePath, targetName)` | 校验项目支持的 JSONPath 子集 |
|
||||
| `isJsonValue(value)` | 判断是否为合法 JSON value |
|
||||
|
||||
@@ -342,9 +344,47 @@ TcpChecker implements Checker
|
||||
**`resolve()` 规范**：
|
||||
|
||||
- 只做默认值合并、路径解析、单位转换，**不执行校验**
|
||||
- 若 checker 支持 expect，必须保留 `rawExpect` 为变量替换后的 Raw 配置快照，并生成只供 `execute()` 使用的 Resolved `expect`
|
||||
- 返回 `satisfies ResolvedXxxTarget` 确保类型正确
|
||||
- 通过 `context.defaults[this.configKey]` 访问 checker 专属默认值（需 `as` 断言为具体类型）
|
||||
|
||||
**expect 五层管线**：每种断言模型从定义到执行经过五个阶段，每层使用对应的共享函数：
|
||||
|
||||
| 断言模型 | 类型层（Raw） | Schema 层 | Validate 层 | Resolve 层 | Execute 层 |
|
||||
| --------------------- | ----------------------------------- | ----------------------------------- | ---------------------------------- | ------------------------------ | ---------------------------- |
|
||||
| `ValueExpectation` | `number \| ValueMatcher` | `createValueMatcherSchema()` | `validateRawValueExpectation()` | `resolveValueExpectation()` | `checkValueExpectation()` |
|
||||
| `ContentExpectations` | `(ValueMatcher \| ExtractorRule)[]` | `createContentExpectationsSchema()` | `validateRawContentExpectations()` | `resolveContentExpectations()` | `checkContentExpectations()` |
|
||||
| `KeyedExpectations` | `Record<string, ValueExpectation>` | `createKeyedExpectationsSchema()` | `validateRawKeyedExpectations()` | `resolveKeyedExpectations()` | `checkKeyedExpectations()` |
|
||||
|
||||
选择哪种模型参考 [1.10 expect 字段选择规范](#110-expect-断言系统)的决策树。
|
||||
|
||||
**resolve 中的标准模式**：
|
||||
|
||||
```typescript
|
||||
// resolve() 内：逐字段调用对应的 resolve 函数，未配置的字段保持 undefined
|
||||
const rawExpect = raw.expect ?? {};
|
||||
expect: {
|
||||
durationMs: rawExpect.durationMs != null ? resolveValueExpectation(rawExpect.durationMs) : undefined,
|
||||
body: rawExpect.body != null ? resolveContentExpectations(rawExpect.body) : undefined,
|
||||
headers: rawExpect.headers != null ? resolveKeyedExpectations(rawExpect.headers) : undefined,
|
||||
}
|
||||
```
|
||||
|
||||
**execute 中的标准模式**：
|
||||
|
||||
```typescript
|
||||
// execute() 内：按快速失败顺序依次检查，首个失败即返回
|
||||
const r = resolved.expect;
|
||||
if (r.durationMs) {
|
||||
const result = checkValueExpectation(elapsed, r.durationMs, { phase: "duration", path: "durationMs" });
|
||||
if (!result.matched) return { ..., failure: result.failure, matched: false };
|
||||
}
|
||||
if (r.body) {
|
||||
const result = checkContentExpectations(bodyText, r.body, { phase: "body", path: "body" });
|
||||
if (!result.matched) return { ..., failure: result.failure, matched: false };
|
||||
}
|
||||
```
|
||||
|
||||
**`execute()` 规范**：
|
||||
|
||||
- 始终记录 `timestamp`（ISO 字符串）和 `start = performance.now()`
|
||||
@@ -353,23 +393,29 @@ TcpChecker implements Checker
|
||||
- 成功时 `failure: null, matched: true`
|
||||
- 异常时使用 `errorFailure(phase, path, message)` 构造 failure
|
||||
- 不匹配时使用 `mismatchFailure(phase, path, expected, actual, message)` 构造 failure
|
||||
- `mismatchFailure` 的 `expected` 参数应传用户可读值，使用 `displayValueExpectation(matcher)` 解包单字段 `{ equals: x }` 为 `x`
|
||||
|
||||
**可用的共享断言工具**（`checker/expect/`）：
|
||||
|
||||
| 模块 | 函数 | 用途 |
|
||||
| --------------------- | ----------------------------------------------------- | ------------------------------------- |
|
||||
| ------------- | ------------------------------------------------------------------- | ---------------------------------------------------- |
|
||||
| `failure.ts` | `errorFailure(phase, path, msg)` | 构造错误类型 failure |
|
||||
| `failure.ts` | `mismatchFailure(phase, path, expected, actual, msg)` | 构造不匹配类型 failure |
|
||||
| `failure.ts` | `truncateActual(value, maxLen?)` | 截断过长的 actual 值（默认 200 字符） |
|
||||
| `matcher.ts` | `applyMatcher(actual, matcher, options?)` | 执行 ValueMatcher AND 匹配 |
|
||||
| `matcher.ts` | `checkValueMatcher(actual, matcher, options)` | 执行 matcher 并返回 `ExpectResult` |
|
||||
| `matcher.ts` | `checkExpectValue(actual, expected)` | 执行字面量 equals 或 matcher |
|
||||
| `matcher.ts` | `evaluateJsonPath(json, path)` | JSONPath 提取 |
|
||||
| `content.ts` | `checkContentRules(source, rules, options)` | 执行 ContentRules 数组 |
|
||||
| `key-value.ts` | `checkKeyValueExpect(actual, expected, options)` | 执行动态键值断言 |
|
||||
| `validate-matcher.ts` | `validateValueMatcher/ContentRules/KeyValueExpect` | matcher/content/key-value 语义校验 |
|
||||
| `value.ts` | `applyValueMatcher(actual, matcher, options?)` | 执行 Resolved `ValueMatcher` AND 匹配 |
|
||||
| `value.ts` | `checkValueExpectation(actual, matcher, options)` | 执行 matcher 并返回 `ExpectationResult` |
|
||||
| `value.ts` | `resolveValueExpectation(raw)` | Raw `ValueExpectation` → Resolved `ValueExpectation` |
|
||||
| `value.ts` | `displayValueExpectation(matcher)` | 解包单字段 `{ equals: x }` 为 `x`，用于 failure 展示 |
|
||||
| `value.ts` | `evaluateJsonPath(json, path)` | JSONPath 提取 |
|
||||
| `content.ts` | `checkContentExpectations(source, expectations, options)` | 执行 Resolved `ContentExpectations` |
|
||||
| `content.ts` | `resolveContentExpectations(raw)` | Raw → Resolved `ContentExpectations` |
|
||||
| `keyed.ts` | `checkKeyedExpectations(actual, expectations, options)` | 执行 Resolved `KeyedExpectations` |
|
||||
| `keyed.ts` | `resolveKeyedExpectations(raw)` | Raw Record → Resolved 有序数组 |
|
||||
| `headers.ts` | `checkHeaderExpectations(headers, expectations, options?)` | HTTP/LLM headers 大小写不敏感包装 |
|
||||
| `status.ts` | `checkStatusCode(actual, expected, phase, path)` | HTTP/LLM status code（精确数值与 1xx-5xx 范围） |
|
||||
| `validate.ts` | `validateRawValueExpectation/ContentExpectations/KeyedExpectations` | Raw expectation 语义校验（不修改输入） |
|
||||
|
||||
**Checker 专属断言**（如需要）放在同目录的 `expect.ts` 中，参考 `http/expect.ts`（checkStatus、checkHeaders）和 `cmd/expect.ts`（checkExitCode）。
|
||||
**Checker 专属断言**（如需要）放在同目录的 `expect.ts` 中，参考 `cmd/expect.ts`（checkExitCode）、`tcp/expect.ts`（checkConnected）、`udp/expect.ts`（checkResponded）和 `icmp/expect.ts`（checkAlive）。HTTP/LLM 复用的 status 与 headers 断言放在共享 expect 模块。
|
||||
|
||||
#### 1.7.6 步骤五：创建模块入口并注册
|
||||
|
||||
@@ -496,19 +542,21 @@ TcpChecker implements Checker
|
||||
|
||||
两层模型：**观测值收集** → **规则校验**。共享断言基础设施位于 `checker/expect/`，checker 专属状态断言位于各自目录。
|
||||
|
||||
**Raw vs Resolved**：用户 YAML 写的是 Raw 形态（primitive 简写、`{ json: { path, equals } }` 提取器对象、`Record<string, value>` 键值表），`config-loader` 的 resolve 阶段将其转换为 Resolved 形态供运行期执行（`{ equals: primitive }`、`{ kind, matcher, ... }` content 联合、`{ key, matcher }[]` 有序数组）。Store 持久化 Raw 快照（`rawExpect`），checker.execute 消费 Resolved `expect`。
|
||||
|
||||
**共享模型**：
|
||||
|
||||
| 模型 | 用途 | 典型字段 |
|
||||
| ---------------- | ---------------------------------------------- | -------------------------------------------------------------------- |
|
||||
| `ValueMatcher` | 单个值、数字指标和字符串元数据断言 | `durationMs`、`rowCount`、`usage.totalTokens`、`finishReason` |
|
||||
| `ContentRules` | 返回内容或半结构化内容断言，必须是数组 | `body`、`stdout`、`stderr`、`banner`、`response`、`output`、`result` |
|
||||
| `KeyValueExpect` | 动态键值断言，字面量等价于 `{ equals: value }` | `headers`、DB `rows[]` 中的列值 |
|
||||
| --------------------- | ---------------------------------------------- | -------------------------------------------------------------------- |
|
||||
| `ValueExpectation` | 单个值、数字指标和字符串元数据断言 | `durationMs`、`rowCount`、`usage.totalTokens`、`finishReason` |
|
||||
| `ContentExpectations` | 返回内容或半结构化内容断言，必须是数组 | `body`、`stdout`、`stderr`、`banner`、`response`、`output`、`result` |
|
||||
| `KeyedExpectations` | 动态键值断言，字面量等价于 `{ equals: value }` | `headers`、DB `rows[]` 中的列值 |
|
||||
|
||||
`ValueMatcher` 支持 `equals`、`contains`、`regex`、`empty`、`exists`、`gte`、`lte`、`gt`、`lt`。一个 matcher 对象内多个字段为 AND 语义；`exists: false` 不能和其他 matcher 组合；`equals` 使用 `es-toolkit/isEqual` 做 JSON 深度相等；`regex` 固定为无 flags 的 `new RegExp(pattern).test(String(actual))`。ValueMatcher expect 字段输入可使用 string、number、boolean 或 null 简写，语义校验入口会归一化为 `{ equals: value }`；数组和对象简写不支持，必须显式写成 `{ equals: ... }`。
|
||||
`ValueMatcher` 支持 `equals`、`contains`、`regex`、`empty`、`exists`、`gte`、`lte`、`gt`、`lt`。一个 matcher 对象内多个字段为 AND 语义；`exists: false` 不能和其他 matcher 组合；`equals` 使用 `es-toolkit/isEqual` 做 JSON 深度相等；`regex` 固定为无 flags 的 `new RegExp(pattern).test(String(actual))`。ValueExpectation Raw 输入可使用 string、number、boolean 或 null 简写，resolve 阶段归一化为 `{ equals: value }`；数组和对象简写不支持，必须显式写成 `{ equals: ... }`。
|
||||
|
||||
`ContentRules` 数组按顺序快速失败。数组项可以是直接 matcher，也可以是 `{ json: {...} }`、`{ css: {...} }`、`{ xpath: {...} }` 提取器规则；一条规则不能混用直接 matcher 和 extractor，多个 extractor 也不能共存。Extractor 未配置 matcher 时等价于 `exists: true`。对对象或数组源执行直接 `contains`/`regex` 时会先 JSON 序列化，`equals` 仍对原始结构做深度相等。
|
||||
`ContentExpectations` 数组按顺序快速失败。数组项可以是直接 matcher，也可以是 `{ json: {...} }`、`{ css: {...} }`、`{ xpath: {...} }` 提取器规则；一条规则不能混用直接 matcher 和 extractor，多个 extractor 也不能共存。Extractor 未配置 matcher 时等价于 `exists: true`。对对象或数组源执行直接 `contains`/`regex` 时会先 JSON 序列化，`equals` 仍对原始结构做深度相等。
|
||||
|
||||
启动期语义校验统一由 `expect/validate-matcher.ts` 负责，会校验空 matcher、未知字段、字段类型、`exists:false` 组合、ContentRules 互斥性、JSONPath 子集、XPath 可编译性、regex 可编译性和 ReDoS 风险。旧字段 `match`、`maxDurationMs`、ICMP 的 `max*` 阈值字段不再支持。
|
||||
启动期语义校验统一由 `expect/validate.ts` 负责，会校验空 matcher、未知字段、字段类型、`exists:false` 组合、ContentExpectations 互斥性、JSONPath 子集、XPath 可编译性、regex 可编译性、ReDoS 风险以及 HTTP/LLM headers 大小写归一化后重复 key。语义校验不修改 Raw 输入。旧字段 `match`、`maxDurationMs`、ICMP 的 `max*` 阈值字段不再支持。
|
||||
|
||||
**快速失败顺序**：
|
||||
|
||||
@@ -523,11 +571,11 @@ TcpChecker implements Checker
|
||||
| LLM http | `status → headers → output → finishReason → rawFinishReason → usage → durationMs` |
|
||||
| LLM stream | `status → headers → stream.completed → stream.firstTokenMs → output → finishReason → rawFinishReason → usage → durationMs` |
|
||||
|
||||
HTTP checker 的 `durationMs` 覆盖完整执行（含重定向、响应体读取、解码和 expect 校验）。status 或 headers 失败时不读取 body；有 body 规则时，在读取 body 前可先检查是否已超过 `durationMs` 阈值，避免无意义读取。
|
||||
HTTP checker 的 `durationMs` 覆盖完整执行（含重定向、按需响应体读取、解码和 expect 校验）。未配置 body expectation、status 失败或 headers 失败时不读取 body；有 body expectation 时，在读取 body 前可先检查 `durationMs` 上界 matcher 是否已不可能通过，避免无意义读取。
|
||||
|
||||
**expect 字段选择规范**：
|
||||
|
||||
新增或修改 checker 的 expect 字段时，按以下决策树选择合适的断言模型：
|
||||
新增或修改 checker 的 expect 字段时，按以下决策树选择合适的断言模型（选定后，各层的具体函数映射参考 [1.7.5 的五层管线表](#175-步骤四实现-checker-类)）：
|
||||
|
||||
```
|
||||
expect 字段
|
||||
@@ -544,11 +592,11 @@ expect 字段
|
||||
│ finishReason、rawFinishReason、usage.*、stream.firstTokenMs
|
||||
│
|
||||
└─ 返回内容 / 半结构化内容 / 不完全确定的值
|
||||
├─ 内容断言 → ContentRules（数组）
|
||||
├─ 内容断言 → ContentExpectations（数组）
|
||||
│ HTTP body、Cmd stdout/stderr、TCP banner、
|
||||
│ UDP response、LLM output、DB result
|
||||
│
|
||||
└─ 键值断言 → KeyValueExpect（动态键对象）
|
||||
└─ 键值断言 → KeyedExpectations（动态键对象）
|
||||
HTTP/LLM headers、DB rows[] 中的列值
|
||||
```
|
||||
|
||||
@@ -558,14 +606,16 @@ expect 字段
|
||||
|
||||
2. **单值数字指标和字符串元数据使用 ValueMatcher**。观测值是一个明确的标量（耗时、行数、丢包率、finish reason），但阈值不确定时，使用 `{ lte: 100 }` 或 `{ regex: "^(stop|end)$" }` 等 matcher 表达；精确匹配 primitive 可直接写 `100` 或 `"stop"`。
|
||||
|
||||
3. **返回内容使用 ContentRules 数组**。观测值是文本、JSON、HTML 或 XML 内容，且可能需要多步提取或多条规则时，使用 ContentRules。即使只有一条规则也必须写成数组形式（`[{ contains: "ok" }]`），不支持对象快捷写法。
|
||||
3. **返回内容使用 ContentExpectations 数组**。观测值是文本、JSON、HTML 或 XML 内容，且可能需要多步提取或多条规则时，使用 ContentExpectations。即使只有一条规则也必须写成数组形式（`[{ contains: "ok" }]`），不支持对象快捷写法。
|
||||
|
||||
4. **键值对使用 KeyValueExpect**。观测值是动态键值表（如 headers），且需要对每个键独立断言时使用。字面量值自动等价于 `{ equals: value }`。
|
||||
4. **键值对使用 KeyedExpectations**。观测值是动态键值表（如 headers），且需要对每个键独立断言时使用。字面量值自动等价于 `{ equals: value }`。
|
||||
|
||||
5. **不要混用模型**。一个 expect 字段只能对应一种断言模型。例如 `finishReason` 是单值字符串元数据，用 ValueMatcher 而非 ContentRules（ContentRules 的 json/css/xpath 提取器对单字符串无意义，且会增加数组包装的配置冗余）。
|
||||
5. **不要混用模型**。一个 expect 字段只能对应一种断言模型。例如 `finishReason` 是单值字符串元数据，用 ValueMatcher 而非 ContentExpectations（ContentExpectations 的 json/css/xpath 提取器对单字符串无意义，且会增加数组包装的配置冗余）。
|
||||
|
||||
6. **failure phase 命名遵循去单位后缀规则**。数字指标字段的 phase 去掉单位后缀（`durationMs` → `duration`、`packetLossPercent` → `packetLoss`、`avgLatencyMs` → `avgLatency`），不带单位后缀的字段直接使用字段名（`rowCount` → `rowCount`、`finishReason` → `finishReason`）。
|
||||
|
||||
7. **实现时参考 [1.7.5 五层管线](#175-步骤四实现-checker-类) 中的对应表**。决策树解决"选哪种模型"，五层管线表解决"每种模型从类型定义到执行分别调哪个函数"。
|
||||
|
||||
### 1.11 错误模式
|
||||
|
||||
- **API 错误**：`{ error: "描述", status: <code> }`，状态码 400/404/503
|
||||
@@ -575,7 +625,7 @@ expect 字段
|
||||
|
||||
### 1.12 测试规范
|
||||
|
||||
- 测试目录 `tests/` 镜像 `src/` 目录结构，但共享 expect 模块的测试集中放在 `tests/server/checker/runner/shared/` 下，覆盖 `failure.ts`、`matcher.ts`、`content.ts`、`key-value.ts`、`validate-matcher.ts` 和 `redos.ts`
|
||||
- 测试目录 `tests/` 镜像 `src/` 目录结构，但共享 expect 模块的测试集中放在 `tests/server/checker/runner/shared/` 下，覆盖 `failure.ts`、`value.ts`（operator）、`content.ts`（body/text）、`keyed.ts`（headers/duplicate-key）、`validate.ts`（shorthand）和 `redos.ts`
|
||||
- 使用 `bun:test` 框架（`describe`/`test`/`expect`），测试数据库用临时目录 + `tmpdir()`
|
||||
- 新增 store 方法必须编写单元测试；新增 API 端点必须在 `app.test.ts` 中添加集成测试
|
||||
- 测试后清理：`afterAll` 中 `store.close()` + `rm(tempDir, { recursive: true })`
|
||||
|
||||
34
README.md
34
README.md
@@ -16,7 +16,7 @@ DiAL 是一个自托管的拨测监控工具，支持 **HTTP**、**命令行**
|
||||
|
||||
- 多种拨测类型：HTTP（GET/POST/PUT 等）、Cmd（命令行执行）、DB（PostgreSQL/MySQL/SQLite）、TCP（端口可达性 + Banner 探测）、UDP（自定义 payload 请求-响应）、ICMP（存活检测、延迟、丢包率）、LLM（大模型服务应用层健康检查）
|
||||
- 丰富的校验规则：状态码、响应头、JSONPath、CSS 选择器、XPath、正则匹配、数值比较等
|
||||
- 结构化观测数据：检查结果保留 HTTP body 预览、TCP/UDP 响应摘要、ICMP 丢包率、CMD 输出预览、LLM token 用量等 observation，便于排障和后续分析
|
||||
- 结构化观测数据：检查结果保留按需读取的 HTTP body 预览、TCP/UDP 响应摘要、ICMP 丢包率、CMD 输出预览、LLM token 用量等 observation，便于排障和后续分析
|
||||
- 响应式 Dashboard：实时状态、可用率统计、耗时趋势图、手动/自动刷新
|
||||
- 多主题支持：系统、明亮、黑暗三种主题模式
|
||||
- 零外部依赖：数据存储使用 SQLite，无需额外数据库服务
|
||||
@@ -185,8 +185,8 @@ targets: # 拨测目标列表（必填）
|
||||
| 字段 | 说明 | 必填 | 默认值 |
|
||||
| ------------ | -------------------------------------------------- | ---- | ------- |
|
||||
| `status` | 可接受的状态码列表，支持精确码和范围（如 `"2xx"`） | 否 | `[200]` |
|
||||
| `headers` | 响应头校验，使用动态键名和 `KeyValueExpect` | 否 | |
|
||||
| `body` | 响应体校验，使用 `ContentRules` 数组 | 否 | |
|
||||
| `headers` | 响应头校验，使用动态键名和 `KeyedExpectations` | 否 | |
|
||||
| `body` | 响应体校验，使用 `ContentExpectations` 数组 | 否 | |
|
||||
| `durationMs` | 完整执行耗时校验，使用 `ValueMatcher` | 否 | |
|
||||
|
||||
**配置示例**
|
||||
@@ -235,10 +235,10 @@ targets: # 拨测目标列表（必填）
|
||||
**expect 校验项**
|
||||
|
||||
| 字段 | 说明 | 必填 | 默认值 |
|
||||
| ------------ | -------------------------------------- | ---- | ------ |
|
||||
| ------------ | --------------------------------------------- | ---- | ------ |
|
||||
| `exitCode` | 可接受的退出码列表 | 否 | `[0]` |
|
||||
| `stdout` | 标准输出校验，使用 `ContentRules` 数组 | 否 | |
|
||||
| `stderr` | 标准错误校验，使用 `ContentRules` 数组 | 否 | |
|
||||
| `stdout` | 标准输出校验，使用 `ContentExpectations` 数组 | 否 | |
|
||||
| `stderr` | 标准错误校验，使用 `ContentExpectations` 数组 | 否 | |
|
||||
| `durationMs` | 完整执行耗时校验，使用 `ValueMatcher` | 否 | |
|
||||
|
||||
**配置示例**
|
||||
@@ -270,10 +270,10 @@ targets: # 拨测目标列表（必填）
|
||||
**expect 校验项**
|
||||
|
||||
| 字段 | 说明 | 必填 | 默认值 |
|
||||
| ------------ | ---------------------------------------------------------------- | ---- | ------ |
|
||||
| ------------ | ----------------------------------------------------------------------- | ---- | ------ |
|
||||
| `rowCount` | 查询返回行数校验，使用 `ValueMatcher` | 否 | |
|
||||
| `rows` | 查询结果逐行校验，数组内每行为列名到 `KeyValueExpect` 的映射 | 否 | |
|
||||
| `result` | 完整查询结果 `{ rows, rowCount }` 校验，使用 `ContentRules` 数组 | 否 | |
|
||||
| `rows` | 查询结果逐行校验，数组内每行为列名到 `KeyedExpectations` 的映射 | 否 | |
|
||||
| `result` | 完整查询结果 `{ rows, rowCount }` 校验，使用 `ContentExpectations` 数组 | 否 | |
|
||||
| `durationMs` | 完整执行耗时校验，使用 `ValueMatcher` | 否 | |
|
||||
|
||||
**配置示例**
|
||||
@@ -310,9 +310,9 @@ targets: # 拨测目标列表（必填）
|
||||
**expect 校验项**
|
||||
|
||||
| 字段 | 说明 | 必填 | 默认值 |
|
||||
| ------------ | ------------------------------------------------------------------ | ---- | ------ |
|
||||
| ------------ | ------------------------------------------------------------------------- | ---- | ------ |
|
||||
| `connected` | 期望连接结果，`true` 可达或 `false` 期望不可达 | 否 | `true` |
|
||||
| `banner` | Banner 内容校验，使用 `ContentRules` 数组，需开启 `tcp.readBanner` | 否 | |
|
||||
| `banner` | Banner 内容校验，使用 `ContentExpectations` 数组，需开启 `tcp.readBanner` | 否 | |
|
||||
| `durationMs` | 完整执行耗时校验，使用 `ValueMatcher` | 否 | |
|
||||
|
||||
**配置示例**
|
||||
@@ -355,9 +355,9 @@ targets: # 拨测目标列表（必填）
|
||||
**expect 校验项**
|
||||
|
||||
| 字段 | 说明 | 必填 | 默认值 |
|
||||
| -------------- | -------------------------------------- | ---- | ------ |
|
||||
| -------------- | --------------------------------------------- | ---- | ------ |
|
||||
| `responded` | 期望是否收到响应 | 否 | `true` |
|
||||
| `response` | 响应内容校验，使用 `ContentRules` 数组 | 否 | |
|
||||
| `response` | 响应内容校验，使用 `ContentExpectations` 数组 | 否 | |
|
||||
| `responseSize` | 响应字节数校验，使用 `ValueMatcher` | 否 | |
|
||||
| `sourceHost` | 响应来源地址校验，使用 `ValueMatcher` | 否 | |
|
||||
| `sourcePort` | 响应来源端口校验，使用 `ValueMatcher` | 否 | |
|
||||
@@ -466,8 +466,8 @@ ICMP checker 通过系统 `ping` 命令执行 ICMP 探测，支持 Linux、macOS
|
||||
| 字段 | 说明 | 必填 | 默认值 |
|
||||
| ----------------- | ---------------------------------------------------------------------- | ---- | ------- |
|
||||
| `status` | 可接受的状态码列表，支持精确码和范围（如 `"2xx"`） | 否 | `[200]` |
|
||||
| `headers` | 响应头校验，使用动态键名和 `KeyValueExpect` | 否 | |
|
||||
| `output` | 模型输出校验，使用 `ContentRules` 数组 | 否 | |
|
||||
| `headers` | 响应头校验，使用动态键名和 `KeyedExpectations` | 否 | |
|
||||
| `output` | 模型输出校验，使用 `ContentExpectations` 数组 | 否 | |
|
||||
| `finishReason` | finish reason 校验，使用 `ValueMatcher` | 否 | |
|
||||
| `rawFinishReason` | 原始 finish reason 校验，使用 `ValueMatcher` | 否 | |
|
||||
| `usage` | Token usage 校验（`inputTokens`/`outputTokens`/`totalTokens` matcher） | 否 | |
|
||||
@@ -497,7 +497,7 @@ ICMP checker 通过系统 `ping` 命令执行 ICMP 探测，支持 Linux、macOS
|
||||
|
||||
### 通用校验规则
|
||||
|
||||
#### ContentRules 校验项
|
||||
#### ContentExpectations 校验项
|
||||
|
||||
`body`、`stdout`、`stderr`、`banner`、`response`、`output`、`result` 均使用数组：
|
||||
|
||||
@@ -511,7 +511,7 @@ ICMP checker 通过系统 `ping` 命令执行 ICMP 探测，支持 Linux、macOS
|
||||
|
||||
`equals`、`contains`、`regex`、`empty`、`exists`、`gte`、`lte`、`gt`、`lt`。`equals` 支持 JSON 深度相等；`regex` 固定使用无 flags 正则；提取器未配置 matcher 时等价于 `exists: true`。ValueMatcher expect 字段也可直接写 string、number、boolean 或 null，等价于 `{ equals: value }`；数组和对象必须显式写成 `{ equals: ... }`。
|
||||
|
||||
#### KeyValueExpect
|
||||
#### KeyedExpectations
|
||||
|
||||
`headers`、`rows` 中每行使用的校验结构，支持 ValueMatcher 的全部字段。
|
||||
|
||||
|
||||
@@ -47,30 +47,30 @@
|
||||
- **THEN** 该 checker 专属的断言函数 SHALL 定义在 `expect.ts` 中
|
||||
|
||||
### Requirement: 断言基础设施目录
|
||||
系统 SHALL 在 `src/server/checker/expect/` 目录中提供所有 checker 共享的断言基础设施。
|
||||
系统 SHALL 在 `src/server/checker/expect/` 目录中提供所有 checker 共享的断言基础设施。共享 expect 目录 SHALL 使用 Raw/Resolved expectation 术语和 value/content/keyed/status/headers 模块边界。
|
||||
|
||||
#### Scenario: expect 共享类型位置
|
||||
- **WHEN** 任何 checker 需要使用断言相关的共享类型（如 `ExpectResult`）
|
||||
- **WHEN** 任何 checker 需要使用断言相关的共享类型（如 `ExpectationResult`、`ValueExpectation`、`ContentExpectations` 或 `KeyedExpectations`）
|
||||
- **THEN** 这些类型 SHALL 从 `src/server/checker/expect/types.ts` 导入
|
||||
|
||||
#### Scenario: operator 断言引擎位置
|
||||
- **WHEN** 任何 checker 需要使用 `applyOperator`、`evaluateJsonPath` 或 `checkExpectValue`
|
||||
- **THEN** 这些函数 SHALL 从 `src/server/checker/expect/operator.ts` 导入
|
||||
#### Scenario: value 断言引擎位置
|
||||
- **WHEN** 任何 checker 需要使用 `applyValueMatcher`、`evaluateJsonPath`、`resolveValueExpectation` 或 `checkValueExpectation`
|
||||
- **THEN** 这些函数 SHALL 从 `src/server/checker/expect/value.ts` 导入
|
||||
|
||||
#### Scenario: duration 断言位置
|
||||
- **WHEN** 任何 checker 需要使用 `checkDuration`
|
||||
- **THEN** 该函数 SHALL 从 `src/server/checker/expect/duration.ts` 导入
|
||||
#### Scenario: content 和 keyed 断言位置
|
||||
- **WHEN** 任何 checker 需要执行内容数组或键值表 expectation
|
||||
- **THEN** SHALL 分别从 `src/server/checker/expect/content.ts` 和 `src/server/checker/expect/keyed.ts` 导入共享函数
|
||||
|
||||
#### Scenario: failure 构造器位置
|
||||
- **WHEN** 任何 checker 需要使用 `errorFailure` 或 `mismatchFailure`
|
||||
- **THEN** 这些函数 SHALL 从 `src/server/checker/expect/failure.ts` 导入
|
||||
|
||||
#### Scenario: operator 校验位置
|
||||
- **WHEN** 任何 checker 的 validate 需要使用 `validateOperatorObject`
|
||||
- **THEN** 该函数 SHALL 从 `src/server/checker/expect/validate-operator.ts` 导入
|
||||
#### Scenario: expectation 校验位置
|
||||
- **WHEN** 任何 checker 的 validate 需要校验 Raw value、Raw content 或 Raw keyed expectation
|
||||
- **THEN** 对应函数 SHALL 从 `src/server/checker/expect/validate.ts` 导入
|
||||
|
||||
#### Scenario: ExpectResult 类型位置
|
||||
- **WHEN** 任何 checker 需要使用 `ExpectResult` 类型
|
||||
#### Scenario: ExpectationResult 类型位置
|
||||
- **WHEN** 任何 checker 需要使用 `ExpectationResult` 类型
|
||||
- **THEN** 该类型 SHALL 从 `src/server/checker/expect/types.ts` 导入
|
||||
|
||||
### Requirement: Schema 目录结构
|
||||
@@ -130,15 +130,15 @@ checker 系统内的模块依赖 SHALL 遵循严格的分层方向。
|
||||
- **THEN** 开发者 SHALL 在 `runner/index.ts` 中添加一行 import 和一行数组项，无需修改其他文件
|
||||
|
||||
### Requirement: 公共类型文件瘦身
|
||||
顶层 `src/server/checker/types.ts` SHALL 仅保留跨 checker 共享的 base 类型和存储相关类型。
|
||||
顶层 `src/server/checker/types.ts` SHALL 仅保留跨 checker 共享的 base 类型和存储相关类型；expect 专属类型 SHALL 放在 `src/server/checker/expect/types.ts`。
|
||||
|
||||
#### Scenario: types.ts 不包含 checker 专属类型
|
||||
#### Scenario: types.ts 不包含 checker 或 expect 专属类型
|
||||
- **WHEN** 开发者查看顶层 `types.ts`
|
||||
- **THEN** 该文件 SHALL NOT 包含 `HttpTargetConfig`、`ResolvedHttpTarget`、`CommandExpectConfig`、`BodyRule`、`TextRule` 等 checker 专属类型
|
||||
- **THEN** 该文件 SHALL NOT 包含 `HttpTargetConfig`、`ResolvedHttpTarget`、`RawCommandExpectConfig`、`ContentExpectation` 等 checker 专属或 expect 专属类型
|
||||
|
||||
#### Scenario: types.ts 保留 base 类型
|
||||
- **WHEN** 开发者查看顶层 `types.ts`
|
||||
- **THEN** 该文件 SHALL 包含 `ResolvedTargetBase`、`RawTargetConfig`、`DefaultsConfig`、`CheckResult`、`ExpectOperator`、`CheckFailure`、`StoredTarget`、`StoredCheckResult`、`JsonValue` 等公共类型
|
||||
- **THEN** 该文件 SHALL 包含 `ResolvedTargetBase`、`RawTargetConfig`、`DefaultsConfig`、`CheckResult`、`CheckFailure`、`StoredTarget`、`StoredCheckResult`、`JsonValue` 等公共类型
|
||||
|
||||
#### Scenario: ResolvedTargetBase 替代联合类型
|
||||
- **WHEN** engine、store、config-loader 需要引用 resolved target 类型
|
||||
|
||||
@@ -146,38 +146,57 @@
|
||||
- **THEN** HttpChecker 的语义校验 SHALL 抛出校验错误，提示 URL 格式不合法
|
||||
|
||||
### Requirement: 存储序列化通过 registry 获取展示格式
|
||||
系统 SHALL 在 `ProbeStore.syncTargets()` 中通过 `checkerRegistry.get(t.type).serialize(t)` 获取每个 target 的展示摘要（`target` 列）和配置 JSON（`config` 列），替代 `buildTargetDisplay()` / `buildTargetConfig()` 中的类型分支。
|
||||
系统 SHALL 在 `ProbeStore.syncTargets()` 中通过 `checkerRegistry.get(t.type).serialize(t)` 获取每个 target 的展示摘要（`target` 列）和配置 JSON（`config` 列），替代 `buildTargetDisplay()` / `buildTargetConfig()` 中的类型分支。系统 SHALL 将 `targets.expect` 持久化为 resolved target 上的 Raw expect 快照，而不是运行期 Resolved expect 执行计划。
|
||||
|
||||
#### Scenario: 序列化委托 checker
|
||||
- **WHEN** store 同步 targets 表
|
||||
- **THEN** store SHALL 对每个 target 调用对应 checker 的 `serialize()` 方法获取 `{ target, config }`
|
||||
|
||||
#### Scenario: expect 持久化使用 rawExpect
|
||||
- **WHEN** store 同步带 expect 的 target 到 targets 表
|
||||
- **THEN** store SHALL 将 `rawExpect` 序列化写入 `targets.expect`，MUST NOT 将包含 `kind` 的 Resolved content expectation 写入该列
|
||||
|
||||
### Requirement: Checker resolve 生成 Raw 与 Resolved expect
|
||||
每个 checker 的 `resolve()` SHALL 在解析 checker 专属 target 配置时，同时保留变量替换后的 Raw expect 快照并生成运行期 Resolved expect 执行计划。Raw expect SHALL 用于配置快照持久化；Resolved expect SHALL 用于 checker `execute()`。`config-loader` SHALL 继续通过 registry 委托 checker resolve，MUST NOT 在中间层理解 checker 专属 expect 字段。
|
||||
|
||||
#### Scenario: resolve 输出双 expect 模型
|
||||
- **WHEN** config-loader 解析一个带 `expect.durationMs: 1000` 的 target
|
||||
- **THEN** 对应 checker 的 resolved target SHALL 包含 Raw expect 中的 `durationMs: 1000`，并在 Resolved expect 中包含 `{equals: 1000}` 形式的运行期 matcher
|
||||
|
||||
#### Scenario: 中间层不感知 checker expect 字段
|
||||
- **WHEN** 新增 checker 定义自己的 Raw/Resolved expect 字段
|
||||
- **THEN** config-loader SHALL 只调用该 checker 的 `validate()` 和 `resolve()`，不新增 checker 类型分支
|
||||
|
||||
### Requirement: 共享 expect 断言函数
|
||||
系统 SHALL 在 `src/server/checker/expect/` 中提供可被多个 checker 复用的 expect 基础设施。共享基础设施 SHALL 包含 matcher、content rules、key-value expect、failure 构造和 ReDoS 校验。checker 专用的状态类断言 SHALL 保留在对应 checker 目录，或在多个 checker 复用时移动到共享模块。仅被单个 checker 使用且不属于通用 matcher/content/key-value 模型的断言模块 SHALL 位于该 checker 目录内。
|
||||
系统 SHALL 在 `src/server/checker/expect/` 中提供可被多个 checker 复用的 expect 基础设施。共享基础设施 SHALL 包含 value expectation、content expectations、keyed expectations、status code 断言、headers keyed 断言、failure 构造和 ReDoS 校验。checker 专用的状态类断言 SHALL 保留在对应 checker 目录，或在多个 checker 复用时移动到共享模块。仅被单个 checker 使用且不属于通用 value/content/keyed/status/header 模型的断言模块 SHALL 位于该 checker 目录内。
|
||||
|
||||
#### Scenario: 共享 ValueMatcher 断言
|
||||
#### Scenario: 共享 ValueExpectation 断言
|
||||
- **WHEN** 任何 checker 需要对数字、字符串、布尔或 JSON value 执行 matcher 匹配
|
||||
- **THEN** SHALL 调用共享 matcher 工具执行 `equals`、`contains`、`regex`、`exists`、`empty`、`gt`、`gte`、`lt` 和 `lte` 语义
|
||||
- **THEN** SHALL 调用共享 value expectation 工具执行 `equals`、`contains`、`regex`、`exists`、`empty`、`gt`、`gte`、`lt` 和 `lte` 语义
|
||||
|
||||
#### Scenario: 共享 ContentRules 断言
|
||||
#### Scenario: 共享 ContentExpectations 断言
|
||||
- **WHEN** HTTP body、LLM output、Cmd stdout/stderr、UDP response 或 TCP banner 需要执行返回内容校验
|
||||
- **THEN** SHALL 调用共享 content rules 工具，而不是在 checker 目录内复制 contains/regex/json/css/xpath 逻辑
|
||||
- **THEN** SHALL 调用共享 content expectations 工具，而不是在 checker 目录内复制 contains/regex/json/css/xpath 逻辑
|
||||
|
||||
#### Scenario: 共享 KeyValueExpect 断言
|
||||
#### Scenario: 共享 KeyedExpectations 断言
|
||||
- **WHEN** HTTP 或 LLM checker 需要校验响应 headers，或 DB checker 需要校验 rows 中的列值
|
||||
- **THEN** SHALL 调用共享 key-value expect 工具，并按调用方规则决定 key 是否大小写敏感
|
||||
- **THEN** SHALL 调用共享 keyed expectations 工具，并按调用方规则决定 key 是否大小写敏感
|
||||
|
||||
#### Scenario: 共享 headers 断言
|
||||
- **WHEN** HTTP 或 LLM checker 需要校验响应 headers
|
||||
- **THEN** SHALL 调用共享 header expectation 包装函数，确保 header key 大小写不敏感
|
||||
|
||||
#### Scenario: 共享 regex ReDoS 校验
|
||||
- **WHEN** 任一 matcher 或 content rule 配置 `regex`
|
||||
- **WHEN** 任一 matcher 或 content expectation 配置 `regex`
|
||||
- **THEN** SHALL 调用共享 ReDoS 校验工具在启动期拒绝危险正则
|
||||
|
||||
#### Scenario: 共享 failure 构造
|
||||
- **WHEN** 任何 checker 需要构造 CheckFailure 对象
|
||||
- **THEN** SHALL 调用 `expect/failure.ts` 中的 `errorFailure()` 或 `mismatchFailure()`，并保留 actual 截断策略
|
||||
|
||||
#### Scenario: HTTP 专用 status 断言
|
||||
#### Scenario: 共享 status 断言
|
||||
- **WHEN** HTTP 或 LLM checker 需要校验响应状态码
|
||||
- **THEN** SHALL 复用同一 status 断言函数，支持精确状态码和 `1xx` 到 `5xx` 范围模式
|
||||
- **THEN** SHALL 复用共享 status code 断言函数，支持精确状态码和 `1xx` 到 `5xx` 范围模式
|
||||
|
||||
### Requirement: 超时控制由引擎注入 signal
|
||||
Checker 实现的 `execute()` MUST 使用 `ctx.signal` 感知超时，不得自行创建 `AbortController` 或 `setTimeout` 用于超时控制。Cmd checker 和 ping checker 可在 signal abort 时 `proc.kill()` 以确保子进程被终止。
|
||||
|
||||
@@ -55,11 +55,11 @@
|
||||
- **THEN** 系统 MUST 停止收集输出并终止该检查，记录 `matched=false`，并在 failure 中写入输出超限信息；observation SHALL 包含已截断输出预览和 error
|
||||
|
||||
### Requirement: cmd expect 校验
|
||||
系统 SHALL 支持 cmd 专用 expect，包括 `exitCode`、`durationMs`、`stdout` 和 `stderr`，并按 exitCode、durationMs、stdout、stderr 的阶段顺序快速失败。`exitCode` SHALL 保持有限整数数组语义，未配置时默认 `[0]`。`durationMs` SHALL 使用共享 `ValueMatcher` 校验完整命令执行耗时。`stdout` 和 `stderr` MUST 使用共享 `ContentRules` 数组，直接 matcher 作用于对应输出文本，`json` extractor SHALL 支持对 JSON CLI 输出执行 JSONPath 断言。
|
||||
系统 SHALL 支持 cmd 专用 expect，包括 `exitCode`、`durationMs`、`stdout` 和 `stderr`，并按 exitCode、durationMs、stdout、stderr 的阶段顺序快速失败。`exitCode` SHALL 保持有限整数数组语义，未配置时在 Resolved expect 中默认 `[0]`。`durationMs` SHALL 使用共享 `RawValueExpectation` 输入并在运行期使用 `ValueExpectation` 校验完整命令执行耗时。`stdout` 和 `stderr` MUST 使用共享 `RawContentExpectations` 数组输入并在运行期使用 `ContentExpectations`，直接 matcher 作用于对应输出文本，`json` extractor SHALL 支持对 JSON CLI 输出执行 JSONPath 断言。
|
||||
|
||||
#### Scenario: 默认 exitCode 成功语义
|
||||
- **WHEN** cmd target 未显式配置 `expect.exitCode`
|
||||
- **THEN** 系统 SHALL 使用默认 `expect.exitCode: [0]` 进行校验
|
||||
- **THEN** 系统 SHALL 在 Resolved cmd expect 中使用默认 `exitCode: [0]` 进行校验
|
||||
|
||||
#### Scenario: 显式 exitCode 校验
|
||||
- **WHEN** cmd target 配置 `expect.exitCode: [0, 2]` 且实际 exit code 为 2
|
||||
@@ -74,8 +74,8 @@
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `duration`
|
||||
|
||||
#### Scenario: stdout 按配置顺序校验
|
||||
- **WHEN** cmd target 配置 `expect.stdout` 为两个 ContentRules，第一条通过且第二条失败
|
||||
- **THEN** 系统 SHALL 先执行第一条 stdout 规则，再执行第二条，并将 failure.path 指向失败的 `stdout[1]`
|
||||
- **WHEN** cmd target 配置 `expect.stdout` 为两个 ContentExpectations，第一条通过且第二条失败
|
||||
- **THEN** 系统 SHALL 先执行第一条 stdout expectation，再执行第二条，并将 failure.path 指向失败的 `stdout[1]`
|
||||
|
||||
#### Scenario: stderr 校验为空
|
||||
- **WHEN** cmd target 配置 `expect.stderr: [{empty: true}]` 且实际 stderr 为空字符串
|
||||
@@ -86,11 +86,11 @@
|
||||
- **THEN** 系统 SHALL 判定 stdout 阶段通过
|
||||
|
||||
#### Scenario: stdout 失败后不检查 stderr
|
||||
- **WHEN** cmd target 同时配置 stdout 和 stderr 规则，且 stdout 规则失败
|
||||
- **THEN** 系统 SHALL 快速失败并 MUST NOT 继续执行 stderr 规则
|
||||
- **WHEN** cmd target 同时配置 stdout 和 stderr expectation，且 stdout expectation 失败
|
||||
- **THEN** 系统 SHALL 快速失败并 MUST NOT 继续执行 stderr expectation
|
||||
|
||||
### Requirement: cmd checker 启动期配置校验
|
||||
系统 SHALL 在启动期对 cmd checker 的配置契约和语义执行严格校验。Cmd target 的 `cmd` 分组 SHALL 只允许 `exec`、`args`、`cwd`、`env`、`maxOutputBytes` 字段。Cmd expect SHALL 只允许 `exitCode`、`durationMs`、`stdout`、`stderr` 字段。未知字段、非法类型、不可编译正则和 ReDoS 风险正则 MUST 导致启动期配置错误。`expect.exitCode` SHALL 保留原有有限整数数组语义，不限制到特定平台范围。
|
||||
系统 SHALL 在启动期对 cmd checker 的配置契约和语义执行严格校验。Cmd target 的 `cmd` 分组 SHALL 只允许 `exec`、`args`、`cwd`、`env`、`maxOutputBytes` 字段。Cmd expect SHALL 只允许 `exitCode`、`durationMs`、`stdout`、`stderr` 字段。未知字段、非法类型、不可编译正则和 ReDoS 风险正则 MUST 导致启动期配置错误。`expect.exitCode` SHALL 保留原有有限整数数组语义，不限制到特定平台范围。语义校验 MUST NOT 修改 Raw cmd expect 输入。
|
||||
|
||||
#### Scenario: cmd args 类型非法
|
||||
- **WHEN** YAML 中 cmd target 配置 `cmd.args` 不是字符串数组
|
||||
@@ -105,24 +105,24 @@
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.exitCode 必须为整数数组
|
||||
|
||||
#### Scenario: cmd expect durationMs 非法
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.durationMs` 不是合法 `ValueMatcher`
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.durationMs` 不是合法 `RawValueExpectation`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.durationMs 格式错误
|
||||
|
||||
#### Scenario: stdout 必须为 ContentRules 数组
|
||||
#### Scenario: stdout 必须为 ContentExpectations 数组
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.stdout` 但其值不是数组
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.stdout 必须为数组
|
||||
|
||||
#### Scenario: stderr 必须为 ContentRules 数组
|
||||
#### Scenario: stderr 必须为 ContentExpectations 数组
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.stderr` 但其值不是数组
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.stderr 必须为数组
|
||||
|
||||
#### Scenario: stdout text rule 空对象非法
|
||||
#### Scenario: stdout text expectation 空对象非法
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.stdout: [{}]`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 stdout 规则必须包含至少一个合法 matcher 或 extractor
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 stdout expectation 必须包含至少一个合法 matcher 或 extractor
|
||||
|
||||
#### Scenario: stderr text rule 未知字段非法
|
||||
#### Scenario: stderr text expectation 未知字段非法
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.stderr: [{foo: "bar"}]`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 stderr 规则包含未知 matcher 或未知 extractor
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 stderr expectation 包含未知 matcher 或未知 extractor
|
||||
|
||||
#### Scenario: stdout regex 正则非法
|
||||
- **WHEN** YAML 中 cmd target 配置 `expect.stdout: [{regex: "[invalid"}]`
|
||||
|
||||
@@ -75,7 +75,7 @@
|
||||
- **THEN** 系统 SHALL 立即关闭数据库连接
|
||||
|
||||
### Requirement: db expect 校验
|
||||
系统 SHALL 支持 db 专用 expect，包括 `durationMs`、`rowCount`、`rows` 和 `result`，按 durationMs、rowCount、rows、result 的阶段顺序快速失败。`durationMs` 和 `rowCount` SHALL 使用共享 `ValueMatcher`。`rows` SHALL 保留按行索引匹配列值的语义，类型为 `Array<KeyValueExpect>`（外层数组按行索引，内层每个元素为一个 `KeyValueExpect` 表达该行的列值断言），每个行规则中列值字面量等价于 `{equals: <literal>}`。`result` MUST 使用共享 `ContentRules` 数组，对查询结果对象 `{ rows, rowCount }` 执行断言。
|
||||
系统 SHALL 支持 db 专用 expect，包括 `durationMs`、`rowCount`、`rows` 和 `result`，按 durationMs、rowCount、rows、result 的阶段顺序快速失败。`durationMs` 和 `rowCount` SHALL 使用共享 `RawValueExpectation` 输入，并在 resolve 阶段转换为运行期 `ValueExpectation`。`rows` SHALL 保留按行索引匹配列值的语义，Raw 类型为 `Array<RawKeyedExpectations>`（外层数组按行索引，内层每个元素表达该行的列值断言），Resolved 类型为 `Array<KeyedExpectations>`。每个行规则中列值 primitive 字面量等价于 `{equals: <literal>}`。`result` MUST 使用共享 `RawContentExpectations` 数组输入，并在运行期以 `ContentExpectations` 对查询结果对象 `{ rows, rowCount }` 执行断言。
|
||||
|
||||
#### Scenario: durationMs 校验
|
||||
- **WHEN** db target 配置 `expect.durationMs: {lte: 3000}` 且实际执行耗时 4000ms
|
||||
@@ -95,7 +95,7 @@
|
||||
|
||||
#### Scenario: rows 按索引匹配列值字面量形式
|
||||
- **WHEN** db target 配置 `expect.rows: [{ status: "active" }]` 且查询首行 status 列值为 `"active"`
|
||||
- **THEN** 系统 SHALL 判定该行该列通过（字面量等价于 `{ equals: "active" }`）
|
||||
- **THEN** 系统 SHALL 在 resolve 阶段将该列值解析为 `{equals: "active"}` 并判定该行该列通过
|
||||
|
||||
#### Scenario: rows 只检查声明的列
|
||||
- **WHEN** db target 配置 `expect.rows: [{ cnt: { gte: 1 } }]` 且查询首行包含 cnt、name、age 三列
|
||||
@@ -122,14 +122,14 @@
|
||||
- **THEN** 系统 SHALL 按 durationMs → rowCount → rows → result 顺序校验，任一阶段失败立即返回
|
||||
|
||||
### Requirement: db checker 启动期配置校验
|
||||
系统 SHALL 在启动期对 db checker 的配置契约和语义执行严格校验。Db target 的 `db` 分组 SHALL 只允许 `url` 和 `query` 字段。Db expect SHALL 只允许 `durationMs`、`rowCount`、`rows` 和 `result` 字段。未知字段、非法 matcher、非法 ContentRules、非法 regex 和 ReDoS 风险正则 MUST 导致启动期配置错误。
|
||||
系统 SHALL 在启动期对 db checker 的配置契约和语义执行严格校验。Db target 的 `db` 分组 SHALL 只允许 `url` 和 `query` 字段。Db expect SHALL 只允许 `durationMs`、`rowCount`、`rows` 和 `result` 字段。未知字段、非法 ValueExpectation、非法 ContentExpectations、非法 regex 和 ReDoS 风险正则 MUST 导致启动期配置错误。语义校验 MUST NOT 修改 Raw db expect 输入。
|
||||
|
||||
#### Scenario: db expect durationMs 非法
|
||||
- **WHEN** YAML 中 db target 配置 `expect.durationMs` 不是合法 `ValueMatcher`
|
||||
- **WHEN** YAML 中 db target 配置 `expect.durationMs` 不是合法 `RawValueExpectation`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.durationMs 格式错误
|
||||
|
||||
#### Scenario: db expect rowCount 非法
|
||||
- **WHEN** YAML 中 db target 配置 `expect.rowCount` 不是合法 `ValueMatcher`
|
||||
- **WHEN** YAML 中 db target 配置 `expect.rowCount` 不是合法 `RawValueExpectation`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.rowCount 格式错误
|
||||
|
||||
#### Scenario: db expect rows 非法
|
||||
@@ -140,8 +140,12 @@
|
||||
- **WHEN** YAML 中 db target 配置 `expect.rows: [{ cnt: { foo: 1 } }]`，其中 foo 不是合法 matcher
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 rows 中包含未知 matcher
|
||||
|
||||
#### Scenario: db expect rows 对象值必须显式 equals
|
||||
- **WHEN** YAML 中 db target 配置 `expect.rows: [{ payload: { status: "ok" } }]` 且 payload 值不是合法 matcher 对象
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示对象值必须显式写成 `{equals: {status: "ok"}}`
|
||||
|
||||
#### Scenario: db expect result 非法
|
||||
- **WHEN** YAML 中 db target 配置 `expect.result` 不是合法 ContentRules 数组
|
||||
- **WHEN** YAML 中 db target 配置 `expect.result` 不是合法 ContentExpectations 数组
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.result 格式错误
|
||||
|
||||
#### Scenario: db expect 未知字段失败
|
||||
|
||||
@@ -5,54 +5,54 @@
|
||||
## Requirements
|
||||
|
||||
### Requirement: 响应体多种校验方法
|
||||
系统 SHALL 支持通过共享 `ContentRules` 对 HTTP 响应体进行有序内容校验。`expect.body` MUST 为规则数组。每个规则 SHALL 为直接 `ValueMatcher`，或 `json`、`css`、`xpath` extractor 规则之一。直接 matcher SHALL 作用于完整响应体文本。`json` SHALL 解析响应体为 JSON 并用 JSONPath 子集提取值。`css` SHALL 使用 CSS selector 从 HTML 中提取元素文本或属性。`xpath` SHALL 使用 XPath 从 XML/HTML 中提取节点值。Extractor 未配置 matcher 时 SHALL 等价于 `exists: true`。
|
||||
系统 SHALL 支持通过共享 `ContentExpectations` 对 HTTP 响应体进行有序内容校验。`expect.body` MUST 为 expectation 数组。每个 expectation SHALL 为直接 `ValueMatcher`，或 `json`、`css`、`xpath` extractor expectation 之一。直接 matcher SHALL 作用于完整响应体文本。`json` SHALL 解析响应体为 JSON 并用 JSONPath 子集提取值。`css` SHALL 使用 CSS selector 从 HTML 中提取元素文本或属性。`xpath` SHALL 使用 XPath 从 XML/HTML 中提取节点值。Extractor 未配置 matcher 时，resolve 阶段 SHALL 将其 matcher 物化为 `{exists: true}`。
|
||||
|
||||
#### Scenario: contains 子串匹配
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{contains: "healthy"}]`，且响应体包含 `"healthy"`
|
||||
- **THEN** 系统 SHALL 判定该 body 规则通过
|
||||
- **THEN** 系统 SHALL 判定该 body expectation 通过
|
||||
|
||||
#### Scenario: regex 正则匹配
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{regex: '"status"\\s*:\\s*"ok"'}]`，且响应体匹配该正则
|
||||
- **THEN** 系统 SHALL 判定该 body 规则通过
|
||||
- **THEN** 系统 SHALL 判定该 body expectation 通过
|
||||
|
||||
#### Scenario: json JSONPath 等值匹配
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{json: {path: "$.status", equals: "ok"}}]`，且响应 JSON 中 `$.status` 值为 `"ok"`
|
||||
- **THEN** 系统 SHALL 判定该 body 规则通过
|
||||
- **THEN** 系统 SHALL 判定该 body expectation 通过
|
||||
|
||||
#### Scenario: json JSONPath 存在性匹配
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{json: {path: "$.status"}}]`，且响应 JSON 中存在 `$.status`
|
||||
- **THEN** 系统 SHALL 将该规则按 `exists: true` 语义判定通过
|
||||
- **THEN** resolve 阶段 SHALL 将该 expectation 按 `exists: true` 语义物化并在运行期判定通过
|
||||
|
||||
#### Scenario: css 选择器匹配
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{css: {selector: "div#health", equals: "OK"}}]`，且 HTML 中存在 `div#health` 元素文本为 `"OK"`
|
||||
- **THEN** 系统 SHALL 判定该 body 规则通过
|
||||
- **THEN** 系统 SHALL 判定该 body expectation 通过
|
||||
|
||||
#### Scenario: css 选择器匹配属性值
|
||||
- **WHEN** HTTP target 配置 css 规则带 `attr: "content"` 用于提取属性，且属性值匹配期望
|
||||
- **THEN** 系统 SHALL 判定该 body 规则通过
|
||||
- **WHEN** HTTP target 配置 css expectation 带 `attr: "content"` 用于提取属性，且属性值匹配期望
|
||||
- **THEN** 系统 SHALL 判定该 body expectation 通过
|
||||
|
||||
#### Scenario: xpath 表达式匹配
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{xpath: {path: "/root/status/text()", equals: "ok"}}]`，且 XML 中 `/root/status` 节点文本为 `"ok"`
|
||||
- **THEN** 系统 SHALL 判定该 body 规则通过
|
||||
- **THEN** 系统 SHALL 判定该 body expectation 通过
|
||||
|
||||
#### Scenario: 提取器无匹配目标失败
|
||||
- **WHEN** HTTP target 配置了 json、css 或 xpath 规则且对应路径、元素或节点不存在，并且规则未配置 `exists: false`
|
||||
- **WHEN** HTTP target 配置了 json、css 或 xpath expectation 且对应路径、元素或节点不存在，并且 expectation 未配置 `exists: false`
|
||||
- **THEN** 系统 SHALL 判定 matched 为 false
|
||||
|
||||
### Requirement: 多种 body 校验方法 AND 组合
|
||||
系统 SHALL 支持在 `expect.body` 数组中同时配置多条内容规则，所有规则均通过时 matched 方为 true。系统 SHALL 按数组顺序执行规则，任一规则失败后 MUST NOT 继续执行后续规则。
|
||||
系统 SHALL 支持在 `expect.body` 数组中同时配置多条内容 expectation，所有 expectation 均通过时 matched 方为 true。系统 SHALL 按数组顺序执行 expectation，任一 expectation 失败后 MUST NOT 继续执行后续 expectation。
|
||||
|
||||
#### Scenario: 多种方法全部通过
|
||||
- **WHEN** HTTP target 的 `expect.body` 数组依次配置 contains、json、regex，且全部通过
|
||||
- **THEN** 系统 SHALL 判定 matched 为 true
|
||||
|
||||
#### Scenario: 多种方法任一失败
|
||||
- **WHEN** HTTP target 的 `expect.body` 数组第一条 contains 不通过，后续还有 json 规则
|
||||
- **THEN** 系统 SHALL 判定 matched 为 false，且不再检查后续 json 规则
|
||||
- **WHEN** HTTP target 的 `expect.body` 数组第一条 contains 不通过，后续还有 json expectation
|
||||
- **THEN** 系统 SHALL 判定 matched 为 false，且不再检查后续 json expectation
|
||||
|
||||
#### Scenario: 直接 matcher 多字段组合
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{contains: "ok", regex: "status"}]`，且响应体同时满足 contains 和 regex
|
||||
- **THEN** 系统 SHALL 判定该规则通过
|
||||
- **THEN** 系统 SHALL 判定该 expectation 通过
|
||||
|
||||
### Requirement: 操作符系统
|
||||
系统 SHALL 支持通过共享 `ValueMatcher` 对提取值和文本值进行比较：`equals`（深度等值）、`contains`（子串包含）、`regex`（正则匹配）、`empty`（空值判断）、`exists`（存在性判断）、`gte`/`lte`/`gt`/`lt`（数值比较）。系统 MUST NOT 支持旧 `match` 字段。
|
||||
@@ -82,7 +82,7 @@
|
||||
- **THEN** 系统 SHALL 对同一字段进行多 matcher 复合比较，全部通过则该规则通过
|
||||
|
||||
### Requirement: 响应头校验
|
||||
系统 SHALL 支持通过共享 `KeyValueExpect` 配置 `expect.headers` 对 HTTP 响应头进行键值规则校验，header 名称匹配 MUST 不区分大小写。header 期望值 MAY 为字符串字面量或 `ValueMatcher`。字符串字面量 SHALL 等价于 `{equals: <value>}`。
|
||||
系统 SHALL 支持通过共享 `KeyedExpectations` 配置 `expect.headers` 对 HTTP 响应头进行键值断言，header 名称匹配 MUST 不区分大小写。header 期望值 MUST 为 `RawValueExpectation`，primitive 字面量 SHALL 在 resolve 阶段等价为 `{equals: <value>}`。
|
||||
|
||||
#### Scenario: 响应头字面量匹配
|
||||
- **WHEN** HTTP target 配置 `expect.headers: {"Content-Type": "application/json"}`，且响应包含该 header 且值精确匹配
|
||||
@@ -155,18 +155,18 @@
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败
|
||||
|
||||
### Requirement: HTTP expect 规则启动期校验
|
||||
系统 SHALL 在启动期校验 HTTP expect 中已支持字段的类型、格式、未知字段和可编译表达式。HTTP expect SHALL 只允许 `status`、`headers`、`body` 和 `durationMs` 字段。`expect.body` MUST 为 `ContentRules` 数组。直接 `ValueMatcher` 对象 MUST 至少包含一个合法 matcher。Extractor 规则 MUST 只包含 `json`、`css`、`xpath` 中的一种 extractor。Extractor 内部可以不配置 matcher，并 SHALL 在运行期以存在性作为通过语义。`equals` matcher SHALL 支持任意 JSON value，包括数组和对象。系统 SHALL 在启动期对所有 `regex` 执行静态 ReDoS 检测。
|
||||
系统 SHALL 在启动期校验 HTTP expect 中已支持字段的类型、格式、未知字段和可编译表达式。HTTP expect SHALL 只允许 `status`、`headers`、`body` 和 `durationMs` 字段。`expect.body` MUST 为 `RawContentExpectations` 数组。直接 `ValueMatcher` 对象 MUST 至少包含一个合法 matcher。Extractor expectation MUST 只包含 `json`、`css`、`xpath` 中的一种 extractor。Extractor 内部可以不配置 matcher，并 SHALL 在 resolve 阶段以存在性 matcher 作为通过语义。`equals` matcher SHALL 支持任意 JSON value，包括数组和对象。系统 SHALL 在启动期对所有 `regex` 执行静态 ReDoS 检测。语义校验 MUST NOT 修改 Raw HTTP expect 输入。
|
||||
|
||||
#### Scenario: body rule 使用 regex 字段
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{regex: "ok|healthy"}]` 且 regex 可编译且无 ReDoS 风险
|
||||
- **THEN** 系统 SHALL 接受该配置，并在运行期按 regex 规则匹配响应体
|
||||
- **THEN** 系统 SHALL 接受该配置，并在运行期按 regex expectation 匹配响应体
|
||||
|
||||
#### Scenario: body rule 不支持 match 字段
|
||||
- **WHEN** HTTP target 配置 `expect.body: [{match: "ok"}]`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示 `match` 是未知字段或不支持字段
|
||||
|
||||
#### Scenario: body rule 多 extractor 非法
|
||||
- **WHEN** HTTP target 的同一条 body rule 同时配置 `json` 和 `css`
|
||||
- **WHEN** HTTP target 的同一条 body expectation 同时配置 `json` 和 `css`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败
|
||||
|
||||
#### Scenario: matcher regex 正则非法
|
||||
@@ -182,7 +182,7 @@
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败
|
||||
|
||||
#### Scenario: JSONPath 子集非法
|
||||
- **WHEN** HTTP target 的 json body rule path 不符合系统支持的 JSONPath 子集
|
||||
- **WHEN** HTTP target 的 json body expectation path 不符合系统支持的 JSONPath 子集
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败
|
||||
|
||||
#### Scenario: matcher 未知字段非法
|
||||
@@ -190,20 +190,20 @@
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败
|
||||
|
||||
#### Scenario: durationMs matcher 非法
|
||||
- **WHEN** HTTP target 配置 `expect.durationMs` 不是合法 `ValueMatcher` 或其中数值 matcher 不是有限数字
|
||||
- **WHEN** HTTP target 配置 `expect.durationMs` 不是合法 `RawValueExpectation` 或其中数值 matcher 不是有限数字
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败
|
||||
|
||||
### Requirement: HTTP body 运行期失败结构化
|
||||
系统 SHALL 将 HTTP body 运行期失败记录为结构化 CheckFailure，并保留与具体规则相关的 phase 和 path。响应内容不符合配置 SHALL 记录为 mismatch；响应内容无法按配置解析或解码 SHALL 记录为 error。
|
||||
系统 SHALL 将 HTTP body 运行期失败记录为结构化 CheckFailure，并保留与具体 expectation 相关的 phase 和 path。响应内容不符合配置 SHALL 记录为 mismatch；响应内容无法按配置解析或解码 SHALL 记录为 error。
|
||||
|
||||
#### Scenario: JSON 响应不是合法 JSON
|
||||
- **WHEN** HTTP target 配置 json body rule，但响应体不是合法 JSON
|
||||
- **THEN** 系统 SHALL 记录 `failure.kind="error"`、`failure.phase="body"`，且 failure.path SHALL 指向对应 json 规则
|
||||
- **WHEN** HTTP target 配置 json body expectation，但响应体不是合法 JSON
|
||||
- **THEN** 系统 SHALL 记录 `failure.kind="error"`、`failure.phase="body"`，且 failure.path SHALL 指向对应 json expectation
|
||||
|
||||
#### Scenario: CSS selector 无匹配元素
|
||||
- **WHEN** HTTP target 配置 css body rule，但响应 HTML 中无匹配元素
|
||||
- **THEN** 系统 SHALL 记录 `failure.kind="mismatch"`、`failure.phase="body"`，且 failure.path SHALL 指向对应 css 规则
|
||||
- **WHEN** HTTP target 配置 css body expectation，但响应 HTML 中无匹配元素
|
||||
- **THEN** 系统 SHALL 记录 `failure.kind="mismatch"`、`failure.phase="body"`，且 failure.path SHALL 指向对应 css expectation
|
||||
|
||||
#### Scenario: XPath 无匹配节点
|
||||
- **WHEN** HTTP target 配置 xpath body rule，但响应 XML/HTML 中无匹配节点
|
||||
- **THEN** 系统 SHALL 记录 `failure.kind="mismatch"`、`failure.phase="body"`，且 failure.path SHALL 指向对应 xpath 规则
|
||||
- **WHEN** HTTP target 配置 xpath body expectation，但响应 XML/HTML 中无匹配节点
|
||||
- **THEN** 系统 SHALL 记录 `failure.kind="mismatch"`、`failure.phase="body"`，且 failure.path SHALL 指向对应 xpath expectation
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
## Purpose
|
||||
|
||||
定义共享 expect 断言规则系统的核心概念和基础设施：ValueMatcher 统一匹配器、ContentRules 内容规则数组、KeyValueExpect 键值规则、以及相关的启动期校验和失败路径规范。
|
||||
定义共享 expect 断言规则系统的核心概念和基础设施：ValueMatcher 统一匹配器、ContentExpectations 内容断言数组、KeyedExpectations 键控断言数组、以及相关的启动期校验和失败路径规范。
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: ValueMatcher 统一匹配器
|
||||
系统 SHALL 提供共享 `ValueMatcher` 作为所有非状态类 expect 的基础匹配结构。`ValueMatcher` SHALL 支持 `equals`、`contains`、`regex`、`exists`、`empty`、`gt`、`gte`、`lt` 和 `lte` 字段。`equals` MUST 支持任意 JSON value，并使用深度相等比较。`contains` 和 `regex` SHALL 将实际值转换为字符串后匹配。`gt`、`gte`、`lt` 和 `lte` SHALL 将实际值转换为有限数字后比较，无法转换为有限数字时 SHALL 判定不匹配。一个 `ValueMatcher` 对象包含多个 matcher 字段时，系统 SHALL 要求全部 matcher 均通过。
|
||||
系统 SHALL 提供共享 `ValueMatcher` 作为所有非状态类 value expectation 的基础匹配结构。`ValueMatcher` SHALL 支持 `equals`、`contains`、`regex`、`exists`、`empty`、`gt`、`gte`、`lt` 和 `lte` 字段。`equals` MUST 支持任意 JSON value，并使用深度相等比较。`contains` 和 `regex` SHALL 将实际值转换为字符串后匹配。`gt`、`gte`、`lt` 和 `lte` SHALL 将实际值转换为有限数字后比较，无法转换为有限数字时 SHALL 判定不匹配。一个 `ValueMatcher` 对象包含多个 matcher 字段时，系统 SHALL 要求全部 matcher 均通过。
|
||||
|
||||
所有类型为 `ValueMatcher` 的 expect 字段 SHALL 同时接受 primitive 原始值（string / number / boolean / null）作为简写形式。原始值简写 SHALL 等价于 `{ equals: value }`。系统 SHALL 在语义校验入口将 primitive 原始值归一化为 `{ equals: value }` 对象形式，后续 resolve 和运行期逻辑 SHALL 仅处理 ValueMatcher 对象形式。数组和对象 MUST NOT 作为原始值简写；需要对数组或对象执行 equals 匹配时，配置 MUST 显式写成 `{ equals: value }`。
|
||||
所有类型为 `RawValueExpectation` 的 expect 字段 SHALL 同时接受 primitive 原始值（string / number / boolean / null）作为简写形式。原始值简写 SHALL 等价于 `{ equals: value }`。系统 SHALL 在 resolve 阶段将 primitive 原始值归一化为 `{ equals: value }` 对象形式，运行期逻辑 SHALL 仅处理 `ValueMatcher` 对象形式。数组和对象 MUST NOT 作为原始值简写；需要对数组或对象执行 equals 匹配时，配置 MUST 显式写成 `{ equals: value }`。
|
||||
|
||||
#### Scenario: equals 匹配对象
|
||||
- **WHEN** 实际值为 `{status: "ok", count: 1}` 且 matcher 为 `{equals: {status: "ok", count: 1}}`
|
||||
@@ -27,26 +27,26 @@
|
||||
|
||||
#### Scenario: 字符串原始值简写等价 equals
|
||||
- **WHEN** expect 字段配置为 `finishReason: "stop"` 且实际值为 `"stop"`
|
||||
- **THEN** 系统 SHALL 将 `"stop"` 归一化为 `{equals: "stop"}` 并判定通过
|
||||
- **THEN** 系统 SHALL 在 resolve 阶段将 `"stop"` 归一化为 `{equals: "stop"}` 并判定通过
|
||||
|
||||
#### Scenario: 数字原始值简写等价 equals
|
||||
- **WHEN** expect 字段配置为 `rowCount: 1` 且实际值为 `1`
|
||||
- **THEN** 系统 SHALL 将 `1` 归一化为 `{equals: 1}` 并判定通过
|
||||
- **THEN** 系统 SHALL 在 resolve 阶段将 `1` 归一化为 `{equals: 1}` 并判定通过
|
||||
|
||||
#### Scenario: 布尔原始值简写等价 equals
|
||||
- **WHEN** expect 字段配置为 ValueMatcher 类型且值为 `true`，实际值为 `true`
|
||||
- **THEN** 系统 SHALL 将 `true` 归一化为 `{equals: true}` 并判定通过
|
||||
- **WHEN** expect 字段配置为 RawValueExpectation 类型且值为 `true`，实际值为 `true`
|
||||
- **THEN** 系统 SHALL 在 resolve 阶段将 `true` 归一化为 `{equals: true}` 并判定通过
|
||||
|
||||
#### Scenario: null 原始值简写等价 equals
|
||||
- **WHEN** expect 字段配置为 ValueMatcher 类型且值为 `null`，实际值为 `null`
|
||||
- **THEN** 系统 SHALL 将 `null` 归一化为 `{equals: null}` 并判定通过
|
||||
- **WHEN** expect 字段配置为 RawValueExpectation 类型且值为 `null`，实际值为 `null`
|
||||
- **THEN** 系统 SHALL 在 resolve 阶段将 `null` 归一化为 `{equals: null}` 并判定通过
|
||||
|
||||
#### Scenario: 原始值简写不匹配
|
||||
- **WHEN** expect 字段配置为 `finishReason: "stop"` 且实际值为 `"error"`
|
||||
- **THEN** 系统 SHALL 判定不通过并生成 mismatch failure
|
||||
|
||||
### Requirement: ValueMatcher 启动期校验
|
||||
系统 SHALL 在启动期对所有 `ValueMatcher` 字段执行严格的类型和语义校验。校验 SHALL 同时接受 primitive 原始值和 ValueMatcher 对象两种形式。当输入为 primitive 原始值时，系统 SHALL 视为合法配置（等价于 `{equals: value}`），无需进一步校验 matcher 字段。当输入为 ValueMatcher 对象时，`contains` MUST 为 string。`equals` MAY 为任意 JSON value。`exists` 和 `empty` MUST 为 boolean。`gt`、`gte`、`lt` 和 `lte` MUST 为有限数字（`Number.isFinite`）。`regex` MUST 为可编译的 string pattern，并通过 ReDoS 风险校验。ValueMatcher 对象 MUST 至少包含一个合法 matcher 字段，空对象 `{}` SHALL 导致启动期配置错误。ValueMatcher 对象 MUST NOT 包含未知字段，任何不属于 `equals`、`contains`、`regex`、`exists`、`empty`、`gt`、`gte`、`lt`、`lte` 的字段 SHALL 导致启动期配置错误。
|
||||
系统 SHALL 在启动期对所有 `RawValueExpectation` 字段执行严格的类型和语义校验。校验 SHALL 同时接受 primitive 原始值和 `ValueMatcher` 对象两种形式，但 MUST NOT 修改输入对象。当输入为 primitive 原始值时，系统 SHALL 视为合法配置（等价于 `{equals: value}`），无需进一步校验 matcher 字段。当输入为 `ValueMatcher` 对象时，`contains` MUST 为 string。`equals` MAY 为任意 JSON value。`exists` 和 `empty` MUST 为 boolean。`gt`、`gte`、`lt` 和 `lte` MUST 为有限数字（`Number.isFinite`）。`regex` MUST 为可编译的 string pattern，并通过 ReDoS 风险校验。`ValueMatcher` 对象 MUST 至少包含一个合法 matcher 字段，空对象 `{}` SHALL 导致启动期配置错误。`ValueMatcher` 对象 MUST NOT 包含未知字段，任何不属于 `equals`、`contains`、`regex`、`exists`、`empty`、`gt`、`gte`、`lt`、`lte` 的字段 SHALL 导致启动期配置错误。
|
||||
|
||||
#### Scenario: 空 matcher 对象被拒绝
|
||||
- **WHEN** YAML 配置中任一 matcher 对象为空 `{}`
|
||||
@@ -65,23 +65,23 @@
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段必须为布尔值
|
||||
|
||||
#### Scenario: 字符串原始值校验通过
|
||||
- **WHEN** YAML 配置中 ValueMatcher 字段值为字符串 `"stop"`
|
||||
- **THEN** 系统 SHALL 接受该配置，视为 `{equals: "stop"}`
|
||||
- **WHEN** YAML 配置中 RawValueExpectation 字段值为字符串 `"stop"`
|
||||
- **THEN** 系统 SHALL 接受该配置，但 SHALL NOT 在语义校验阶段修改输入对象
|
||||
|
||||
#### Scenario: 数字原始值校验通过
|
||||
- **WHEN** YAML 配置中 ValueMatcher 字段值为数字 `5000`
|
||||
- **THEN** 系统 SHALL 接受该配置，视为 `{equals: 5000}`
|
||||
- **WHEN** YAML 配置中 RawValueExpectation 字段值为数字 `5000`
|
||||
- **THEN** 系统 SHALL 接受该配置，但 SHALL NOT 在语义校验阶段修改输入对象
|
||||
|
||||
#### Scenario: null 原始值校验通过
|
||||
- **WHEN** YAML 配置中 ValueMatcher 字段值为 `null`
|
||||
- **THEN** 系统 SHALL 接受该配置，视为 `{equals: null}`
|
||||
- **WHEN** YAML 配置中 RawValueExpectation 字段值为 `null`
|
||||
- **THEN** 系统 SHALL 接受该配置，但 SHALL NOT 在语义校验阶段修改输入对象
|
||||
|
||||
#### Scenario: 数组原始值被拒绝
|
||||
- **WHEN** YAML 配置中 ValueMatcher 字段值为数组 `[1, 2]`
|
||||
- **WHEN** YAML 配置中 RawValueExpectation 字段值为数组 `[1, 2]`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示必须为 primitive 原始值或 matcher 对象；如需数组 equals 匹配应写成 `{equals: [1, 2]}`
|
||||
|
||||
#### Scenario: 对象原始值必须显式 equals
|
||||
- **WHEN** YAML 配置中 ValueMatcher 字段值为对象 `{foo: "bar"}`，且 `foo` 不是合法 matcher 字段
|
||||
- **WHEN** YAML 配置中 RawValueExpectation 字段值为对象 `{foo: "bar"}`，且 `foo` 不是合法 matcher 字段
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示 `foo` 是未知 matcher；如需对象 equals 匹配应写成 `{equals: {foo: "bar"}}`
|
||||
|
||||
### Requirement: empty matcher 语义
|
||||
@@ -141,92 +141,108 @@
|
||||
- **WHEN** YAML 配置中任一 `regex` 为 `"(a+)+$"`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示正则存在 ReDoS 风险
|
||||
|
||||
### Requirement: ContentRules 内容规则数组
|
||||
系统 SHALL 提供共享 `ContentRules` 表达返回内容断言。`ContentRules` MUST 为有序数组，数组项 SHALL 为直接 `ValueMatcher`，或 `json`、`css`、`xpath` 三类 extractor 规则之一。系统 SHALL 按数组顺序执行全部规则，任一规则失败时 SHALL 立即停止并返回该规则的 failure。系统 MUST NOT 支持内容字段的非数组对象快捷写法。
|
||||
### Requirement: ContentExpectations 内容断言数组
|
||||
系统 SHALL 提供共享 `ContentExpectations` 表达返回内容断言。`ContentExpectations` MUST 为有序数组，数组项 SHALL 为直接 `ValueMatcher`，或 `json`、`css`、`xpath` 三类 extractor expectation 之一。系统 SHALL 在 resolve 阶段将 Raw content DSL 解析为带 `kind` 字段的 Resolved `ContentExpectation` 执行计划，并按数组顺序执行全部 expectation，任一 expectation 失败时 SHALL 立即停止并返回该 expectation 的 failure。系统 MUST NOT 支持内容字段的非数组对象快捷写法。
|
||||
|
||||
#### Scenario: 直接 matcher 内容规则
|
||||
- **WHEN** 内容字段配置 `[{contains: "ready"}, {regex: "listening on \\d+"}]` 且原始内容同时满足两条规则
|
||||
#### Scenario: 直接 matcher 内容 expectation
|
||||
- **WHEN** 内容字段配置 `[{contains: "ready"}, {regex: "listening on \\d+"}]` 且原始内容同时满足两条 expectation
|
||||
- **THEN** 系统 SHALL 判定该内容字段通过
|
||||
|
||||
#### Scenario: 内容规则数组快速失败
|
||||
- **WHEN** 内容字段配置三条规则且第二条规则失败
|
||||
- **THEN** 系统 SHALL 返回第二条规则的 failure，并 MUST NOT 执行第三条规则
|
||||
#### Scenario: 内容 expectation 数组快速失败
|
||||
- **WHEN** 内容字段配置三条 expectation 且第二条 expectation 失败
|
||||
- **THEN** 系统 SHALL 返回第二条 expectation 的 failure，并 MUST NOT 执行第三条 expectation
|
||||
|
||||
#### Scenario: 内容字段必须为数组
|
||||
- **WHEN** YAML 中内容字段配置为 `{contains: "ok"}` 而不是数组
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该内容字段必须为规则数组
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该内容字段必须为 expectation 数组
|
||||
|
||||
### Requirement: ContentRule 互斥性约束
|
||||
一条 `ContentRule` MUST 为直接 `ValueMatcher` 或恰好一个 extractor（`json`、`css`、`xpath` 之一）。系统 MUST NOT 允许同一条规则同时包含多个 extractor。直接 `ValueMatcher` 规则 MUST NOT 包含 `json`、`css`、`xpath` 字段。系统 SHALL 在启动期对违反互斥性的规则报错。
|
||||
#### Scenario: Resolved content expectation 使用 kind
|
||||
- **WHEN** Raw 内容字段包含直接 matcher、json extractor、css extractor 和 xpath extractor
|
||||
- **THEN** resolve 阶段 SHALL 分别生成 `kind="value"`、`kind="json"`、`kind="css"` 和 `kind="xpath"` 的 Resolved `ContentExpectation`
|
||||
|
||||
### Requirement: ContentExpectation 互斥性约束
|
||||
一条 Raw `ContentExpectation` MUST 为直接 `ValueMatcher` 或恰好一个 extractor（`json`、`css`、`xpath` 之一）。系统 MUST NOT 允许同一条 Raw expectation 同时包含多个 extractor。直接 `ValueMatcher` expectation MUST NOT 包含 `json`、`css`、`xpath` 字段。系统 SHALL 在启动期对违反互斥性的 Raw expectation 报错。
|
||||
|
||||
#### Scenario: 多 extractor 被拒绝
|
||||
- **WHEN** YAML 中内容规则为 `{json: {path: "$.a"}, css: {selector: "div"}}`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示一条规则不能同时包含多个 extractor
|
||||
- **WHEN** YAML 中内容 expectation 为 `{json: {path: "$.a"}, css: {selector: "div"}}`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示一条 expectation 不能同时包含多个 extractor
|
||||
|
||||
#### Scenario: 直接 matcher 混入 extractor 被拒绝
|
||||
- **WHEN** YAML 中内容规则为 `{contains: "ok", json: {path: "$.a"}}`
|
||||
- **WHEN** YAML 中内容 expectation 为 `{contains: "ok", json: {path: "$.a"}}`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示直接 matcher 不能与 extractor 混用
|
||||
|
||||
### Requirement: 空 ContentRules 数组语义
|
||||
`ContentRules` 空数组 `[]` SHALL 被系统接受为合法配置。运行期空数组 SHALL 等价于无规则，即该内容字段的断言直接通过。
|
||||
### Requirement: 空 ContentExpectations 数组语义
|
||||
`ContentExpectations` 空数组 `[]` SHALL 被系统接受为合法配置。运行期空数组 SHALL 等价于无内容 expectation，即该内容字段的断言直接通过。
|
||||
|
||||
#### Scenario: 空 body 数组通过
|
||||
- **WHEN** HTTP target 配置 `expect.body: []` 且响应体为任意内容
|
||||
- **THEN** 系统 SHALL 判定 body 阶段通过
|
||||
|
||||
### Requirement: ContentRules 非字符串值序列化
|
||||
当 `ContentRules` 的观测源为非字符串值（如对象或数组）时，直接 `ValueMatcher` 的 `contains` 和 `regex` SHALL 先将值 JSON 序列化为字符串后匹配。`equals` SHALL 直接在原始结构化值上使用深度相等比较，不进行序列化。
|
||||
### Requirement: ContentExpectations 非字符串值序列化
|
||||
当 `ContentExpectations` 的观测源为非字符串值（如对象或数组）时，直接 `ValueMatcher` 的 `contains` 和 `regex` SHALL 先将值 JSON 序列化为字符串后匹配。`equals` SHALL 直接在原始结构化值上使用深度相等比较，不进行序列化。
|
||||
|
||||
#### Scenario: 对象序列化后 contains 匹配
|
||||
- **WHEN** ContentRules 观测源为 `{status: "ok"}` 且规则为 `{contains: "ok"}`
|
||||
- **WHEN** ContentExpectations 观测源为 `{status: "ok"}` 且 expectation 为 `{contains: "ok"}`
|
||||
- **THEN** 系统 SHALL 将对象 JSON 序列化后执行 contains 匹配
|
||||
|
||||
#### Scenario: 对象 equals 不序列化
|
||||
- **WHEN** ContentRules 观测源为 `{status: "ok"}` 且规则为 `{equals: {status: "ok"}}`
|
||||
- **WHEN** ContentExpectations 观测源为 `{status: "ok"}` 且 expectation 为 `{equals: {status: "ok"}}`
|
||||
- **THEN** 系统 SHALL 直接在结构化值上使用深度相等比较
|
||||
|
||||
### Requirement: ContentRules 提取器
|
||||
系统 SHALL 支持在 `ContentRules` 中使用 `json`、`css` 和 `xpath` extractor。`json.path` MUST 使用现有 JSONPath 子集。`css.selector` MUST 为非空字符串，并 MAY 配置 `attr` 提取属性值。`xpath.path` MUST 为非空字符串，并 SHALL 在启动期进行可编译校验。Extractor 内部 MAY 包含任意 `ValueMatcher` 字段。Extractor 规则未配置任何 matcher 时 SHALL 等价于 `exists: true`。
|
||||
### Requirement: ContentExpectations 提取器
|
||||
系统 SHALL 支持在 `ContentExpectations` 中使用 `json`、`css` 和 `xpath` extractor。`json.path` MUST 使用现有 JSONPath 子集。`css.selector` MUST 为非空字符串，并 MAY 配置 `attr` 提取属性值。`xpath.path` MUST 为非空字符串，并 SHALL 在启动期进行可编译校验。Extractor 内部 MAY 包含任意 `ValueMatcher` 字段。Extractor expectation 未配置任何 matcher 时，resolve 阶段 SHALL 将其 Resolved matcher 物化为 `{ exists: true }`。
|
||||
|
||||
#### Scenario: json extractor 数字比较
|
||||
- **WHEN** 原始内容为 JSON 字符串 `{"count": 2}` 且规则为 `{json: {path: "$.count", gte: 1}}`
|
||||
- **THEN** 系统 SHALL 解析 JSON、提取 `$.count` 并判定该规则通过
|
||||
- **WHEN** 原始内容为 JSON 字符串 `{"count": 2}` 且 expectation 为 `{json: {path: "$.count", gte: 1}}`
|
||||
- **THEN** 系统 SHALL 解析 JSON、提取 `$.count` 并判定该 expectation 通过
|
||||
|
||||
#### Scenario: json extractor 存在性默认语义
|
||||
- **WHEN** 原始内容为 JSON 字符串 `{"user": {"id": null}}` 且规则为 `{json: {path: "$.user.id"}}`
|
||||
- **THEN** 系统 SHALL 将该规则视为 `{json: {path: "$.user.id", exists: true}}` 并判定通过
|
||||
- **WHEN** 原始内容为 JSON 字符串 `{"user": {"id": null}}` 且 expectation 为 `{json: {path: "$.user.id"}}`
|
||||
- **THEN** resolve 阶段 SHALL 将该 expectation 的 matcher 物化为 `{exists: true}` 并在运行期判定通过
|
||||
|
||||
#### Scenario: css attr 存在性默认语义
|
||||
- **WHEN** 原始内容包含 `<meta name="status" content="ok">` 且规则为 `{css: {selector: "meta[name=status]", attr: "content"}}`
|
||||
- **THEN** 系统 SHALL 在属性存在时判定该规则通过
|
||||
- **WHEN** 原始内容包含 `<meta name="status" content="ok">` 且 expectation 为 `{css: {selector: "meta[name=status]", attr: "content"}}`
|
||||
- **THEN** resolve 阶段 SHALL 将该 expectation 的 matcher 物化为 `{exists: true}` 并在属性存在时判定通过
|
||||
|
||||
#### Scenario: xpath 无匹配节点失败
|
||||
- **WHEN** XML 内容中不存在 XPath 指向的节点，且规则为 `{xpath: {path: "/root/status"}}`
|
||||
- **THEN** 系统 SHALL 判定该规则不通过并生成 phase 对应内容字段的 mismatch failure
|
||||
- **WHEN** XML 内容中不存在 XPath 指向的节点，且 expectation 为 `{xpath: {path: "/root/status"}}`
|
||||
- **THEN** 系统 SHALL 判定该 expectation 不通过并生成 phase 对应内容字段的 mismatch failure
|
||||
|
||||
### Requirement: KeyValueExpect 键值规则
|
||||
系统 SHALL 提供共享 `KeyValueExpect` 表达键值型观测值断言。`KeyValueExpect` SHALL 为动态键对象，每个键对应的值 MAY 为 `ValueMatcher` 或 JSON 字面量。字面量值 SHALL 等价于 `{equals: <literal>}`。调用方 MAY 指定 key 规范化策略；HTTP 与 LLM headers MUST 使用大小写不敏感的 key 匹配。
|
||||
### Requirement: KeyedExpectations 键控断言数组
|
||||
系统 SHALL 提供共享 `KeyedExpectations` 表达键值型观测值断言。Raw `KeyedExpectations` SHALL 为动态键对象，每个键对应的值 MUST 为 `RawValueExpectation`；数组和对象简写 MUST 被拒绝，数组或对象 equals 匹配 MUST 显式写为 `{equals: <value>}`。Resolved `KeyedExpectations` SHALL 为有序数组，每个元素包含原始 key 和已归一化的 `ValueExpectation` matcher。调用方 MAY 指定 key 规范化策略；HTTP 与 LLM headers MUST 使用大小写不敏感的 key 匹配，并 MUST 在启动期校验或 resolve 阶段拒绝归一化后重复的 header key。DB rows 不做 key 归一化，按查询结果列名大小写敏感匹配。
|
||||
|
||||
#### Scenario: headers 字面量快捷写法
|
||||
- **WHEN** 响应 headers 中 `content-type` 为 `application/json`，且配置为 `headers: {Content-Type: "application/json"}`
|
||||
- **THEN** 系统 SHALL 按大小写不敏感 key 匹配并使用 equals 语义判定通过
|
||||
- **THEN** resolve 阶段 SHALL 将该项解析为 keyed expectation `{key: "Content-Type", matcher: {equals: "application/json"}}`，运行期按大小写不敏感 key 匹配并判定通过
|
||||
|
||||
#### Scenario: headers matcher 写法
|
||||
- **WHEN** 响应 headers 中 `content-type` 为 `application/json; charset=utf-8`，且配置为 `headers: {Content-Type: {contains: "application/json"}}`
|
||||
- **THEN** 系统 SHALL 判定该 header 规则通过
|
||||
- **THEN** 系统 SHALL 判定该 header expectation 通过
|
||||
|
||||
#### Scenario: 缺失键 exists false
|
||||
- **WHEN** 观测键值表中不存在 `x-debug`，且配置为 `{x-debug: {exists: false}}`
|
||||
- **THEN** 系统 SHALL 判定该键规则通过
|
||||
- **THEN** 系统 SHALL 判定该 keyed expectation 通过
|
||||
|
||||
#### Scenario: keyed 对象值必须显式 equals
|
||||
- **WHEN** Raw keyed expectation 的某个值是对象 `{foo: "bar"}` 且未写在 `equals` 下
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示对象 equals 必须显式写成 `{equals: {foo: "bar"}}`
|
||||
|
||||
#### Scenario: keyed 数组值必须显式 equals
|
||||
- **WHEN** Raw keyed expectation 的某个值是数组 `["a"]` 且未写在 `equals` 下
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示数组 equals 必须显式写成 `{equals: ["a"]}`
|
||||
|
||||
#### Scenario: header 归一化重复 key 被拒绝
|
||||
- **WHEN** HTTP 或 LLM `expect.headers` 同时配置 `Content-Type` 和 `content-type`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示 header key 归一化后重复
|
||||
|
||||
### Requirement: 结构化失败路径
|
||||
系统 SHALL 在共享 matcher、content 和 key-value 断言失败时生成结构化 `CheckFailure`。failure SHALL 包含 `kind`、`phase`、`path`、`message`，并在 mismatch 场景包含 `expected` 和 `actual`。内容规则 failure path SHALL 包含数组下标，key-value failure path SHALL 包含键名，extractor failure path SHALL 包含 extractor 类型和 path/selector 信息。
|
||||
系统 SHALL 在共享 matcher、content 和 keyed expectation 断言失败时生成结构化 `CheckFailure`。failure SHALL 包含 `kind`、`phase`、`path`、`message`，并在 mismatch 场景包含 `expected` 和 `actual`。内容 expectation failure path SHALL 包含数组下标，keyed expectation failure path SHALL 包含键名，extractor failure path SHALL 包含 extractor 类型和 path/selector 信息。failure.expected SHOULD 使用用户可理解的 matcher 或 expectation 片段，MUST NOT 直接暴露 Resolved `kind` 执行计划；单字段 `equals` 包装 SHOULD 展示为原始 expected 值。
|
||||
|
||||
#### Scenario: ContentRules 失败路径
|
||||
- **WHEN** `expect.body[1].json` 规则失败
|
||||
#### Scenario: ContentExpectations 失败路径
|
||||
- **WHEN** `expect.body[1].json` expectation 失败
|
||||
- **THEN** failure.path SHALL 指向 `body[1].json($.path)` 或等价可定位路径，failure.phase SHALL 为 `body`
|
||||
|
||||
#### Scenario: KeyValueExpect 失败路径
|
||||
#### Scenario: KeyedExpectations 失败路径
|
||||
- **WHEN** `expect.headers.Content-Type` 不匹配
|
||||
- **THEN** failure.path SHALL 指向 `headers.Content-Type`，failure.phase SHALL 为 `headers`
|
||||
|
||||
|
||||
@@ -110,11 +110,11 @@
|
||||
- **THEN** 系统 SHALL 记录 `matched=false`，failure 的 kind 为 `error`，phase 为 `icmp`，path 为 `parse`，message 包含 "无法解析 icmp 输出"
|
||||
|
||||
### Requirement: icmp expect 校验
|
||||
系统 SHALL 支持 icmp 专属 expect，包括 `alive`、`packetLossPercent`、`avgLatencyMs`、`maxLatencyMs` 和 `durationMs`，并按 alive、packetLossPercent、avgLatencyMs、maxLatencyMs、durationMs 的阶段顺序快速失败。`alive` SHALL 保持布尔状态语义，未配置时默认 `true`。`packetLossPercent` SHALL 表示 0 到 100 的丢包率百分比，并使用共享 `ValueMatcher`。`avgLatencyMs`、`maxLatencyMs` 和 `durationMs` SHALL 使用共享 `ValueMatcher`。
|
||||
系统 SHALL 支持 icmp 专属 expect，包括 `alive`、`packetLossPercent`、`avgLatencyMs`、`maxLatencyMs` 和 `durationMs`，并按 alive、packetLossPercent、avgLatencyMs、maxLatencyMs、durationMs 的阶段顺序快速失败。`alive` SHALL 保持布尔状态语义，未配置时在 Resolved expect 中默认 `true`。`packetLossPercent` SHALL 表示 0 到 100 的丢包率百分比，并使用共享 `RawValueExpectation` 输入、运行期 `ValueExpectation` 执行。`avgLatencyMs`、`maxLatencyMs` 和 `durationMs` SHALL 使用共享 `RawValueExpectation` 输入、运行期 `ValueExpectation` 执行。
|
||||
|
||||
#### Scenario: 默认 alive 成功语义
|
||||
- **WHEN** icmp target 未显式配置 `expect.alive`
|
||||
- **THEN** 系统 SHALL 使用默认 `expect.alive: true` 进行校验
|
||||
- **THEN** 系统 SHALL 在 Resolved icmp expect 中使用默认 `alive: true` 进行校验
|
||||
|
||||
#### Scenario: alive 校验通过
|
||||
- **WHEN** icmp target 配置 `expect.alive: true`，且目标主机可达
|
||||
@@ -165,17 +165,21 @@
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段
|
||||
|
||||
#### Scenario: packetLossPercent 类型非法
|
||||
- **WHEN** YAML 中 icmp target 的 `expect.packetLossPercent` 不是合法 `ValueMatcher`，或其数值范围无法用于 0 到 100 的百分比断言
|
||||
- **WHEN** YAML 中 icmp target 的 `expect.packetLossPercent` 不是合法 `RawValueExpectation`，或其数值范围无法用于 0 到 100 的百分比断言
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.packetLossPercent 格式错误
|
||||
|
||||
#### Scenario: avgLatencyMs 类型非法
|
||||
- **WHEN** YAML 中 icmp target 的 `expect.avgLatencyMs` 不是合法 `ValueMatcher`
|
||||
- **WHEN** YAML 中 icmp target 的 `expect.avgLatencyMs` 不是合法 `RawValueExpectation`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.avgLatencyMs 格式错误
|
||||
|
||||
#### Scenario: maxLatencyMs 类型非法
|
||||
- **WHEN** YAML 中 icmp target 的 `expect.maxLatencyMs` 不是合法 `ValueMatcher`
|
||||
- **WHEN** YAML 中 icmp target 的 `expect.maxLatencyMs` 不是合法 `RawValueExpectation`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.maxLatencyMs 格式错误
|
||||
|
||||
#### Scenario: Raw icmp expect 不被校验阶段修改
|
||||
- **WHEN** YAML 中 icmp target 配置 `expect.durationMs: 5000`
|
||||
- **THEN** 语义校验 SHALL 接受该 Raw primitive 简写且 MUST NOT 将输入对象原地改写为 `{equals: 5000}`
|
||||
|
||||
### Requirement: icmp detail 摘要
|
||||
系统 SHALL 在 icmp API 序列化时从 observation 动态生成结构化 detail 摘要，展示关键指标。API registry type SHALL 仍为 `icmp`。
|
||||
|
||||
|
||||
@@ -108,18 +108,18 @@ LLM checker SHALL 在 SDK 调用结果和 expect 断言之间构建 `LlmCheckObs
|
||||
- **THEN** LLM checker SHALL 返回 `phase: "request"` 的 error failure
|
||||
|
||||
### Requirement: LLM Expect 断言
|
||||
LLM checker SHALL 支持 `expect.status`、`expect.headers`、`expect.output`、`expect.finishReason`、`expect.rawFinishReason`、`expect.usage.inputTokens`、`expect.usage.outputTokens`、`expect.usage.totalTokens`、`expect.stream.completed`、`expect.stream.firstTokenMs` 和 `expect.durationMs`。`expect.status` SHALL 保持 HTTP 状态码数组语义并复用 HTTP status 断言。`expect.headers` SHALL 使用共享 `KeyValueExpect` 且 header key 大小写不敏感。`expect.output` MUST 使用共享 `ContentRules`。`expect.finishReason` 和 `expect.rawFinishReason` SHALL 使用共享 `ValueMatcher`。`expect.usage.*`、`expect.stream.firstTokenMs` 和 `expect.durationMs` SHALL 使用共享 `ValueMatcher`。LLM checker MUST 按固定顺序快速失败，非流式顺序为 status、headers、output、finishReason、rawFinishReason、usage、durationMs；流式顺序为 status、headers、stream.completed、stream.firstTokenMs、output、finishReason、rawFinishReason、usage、durationMs。
|
||||
LLM checker SHALL 支持 `expect.status`、`expect.headers`、`expect.output`、`expect.finishReason`、`expect.rawFinishReason`、`expect.usage.inputTokens`、`expect.usage.outputTokens`、`expect.usage.totalTokens`、`expect.stream.completed`、`expect.stream.firstTokenMs` 和 `expect.durationMs`。`expect.status` SHALL 保持 HTTP 状态码数组语义并复用共享 status code 断言，未配置时在 Resolved expect 中物化默认 `[200]`。`expect.headers` SHALL 使用共享 `RawKeyedExpectations` 输入并在运行期使用 `KeyedExpectations`，header key 大小写不敏感。`expect.output` MUST 使用共享 `RawContentExpectations` 输入并在运行期使用 `ContentExpectations`。`expect.finishReason` 和 `expect.rawFinishReason` SHALL 使用共享 `RawValueExpectation` 输入并在运行期使用 `ValueExpectation`。`expect.usage.*`、`expect.stream.firstTokenMs` 和 `expect.durationMs` SHALL 使用共享 `RawValueExpectation` 输入并在运行期使用 `ValueExpectation`。LLM checker MUST 按固定顺序快速失败，非流式顺序为 status、headers、output、finishReason、rawFinishReason、usage、durationMs；流式顺序为 status、headers、stream.completed、stream.firstTokenMs、output、finishReason、rawFinishReason、usage、durationMs。
|
||||
|
||||
#### Scenario: 默认 status 断言
|
||||
- **WHEN** LLM target 未配置 `expect.status`
|
||||
- **THEN** LLM checker SHALL 使用默认 `status: [200]` 语义
|
||||
- **THEN** LLM checker SHALL 在 Resolved expect 中使用默认 `status: [200]` 语义
|
||||
|
||||
#### Scenario: expect headers 通过
|
||||
- **WHEN** observing fetch 捕获的响应 headers 满足 `expect.headers` 配置
|
||||
- **THEN** LLM checker SHALL 判定 headers 断言通过
|
||||
- **THEN** LLM checker SHALL 通过共享 header expectation 包装函数判定 headers 断言通过
|
||||
|
||||
#### Scenario: output ContentRules 通过
|
||||
- **WHEN** LLM 输出文本满足 `expect.output` 中配置的全部 ContentRules
|
||||
#### Scenario: output ContentExpectations 通过
|
||||
- **WHEN** LLM 输出文本满足 `expect.output` 中配置的全部 ContentExpectations
|
||||
- **THEN** LLM checker SHALL 判定 output 阶段通过
|
||||
|
||||
#### Scenario: finishReason ValueMatcher 通过
|
||||
@@ -175,18 +175,22 @@ LLM checker SHALL 使用共享 `ContentRules` 校验 `expect.output`。每个 ou
|
||||
|
||||
#### Scenario: output JSONPath 存在性默认语义
|
||||
- **WHEN** `outputText` 是 JSON 字符串且 target 配置 `expect.output: [{json: {path: "$.status"}}]`
|
||||
- **THEN** LLM checker SHALL 将该规则按 `exists: true` 语义执行
|
||||
- **THEN** resolve 阶段 SHALL 将该 expectation 的 matcher 物化为 `{exists: true}`，运行期按存在性语义执行
|
||||
|
||||
#### Scenario: output 规则按顺序快速失败
|
||||
- **WHEN** `expect.output` 包含多个规则且第一条规则失败
|
||||
- **THEN** LLM checker SHALL 返回第一条失败规则的 mismatch failure，不继续校验后续 output 规则
|
||||
- **WHEN** `expect.output` 包含多个 expectation 且第一条 expectation 失败
|
||||
- **THEN** LLM checker SHALL 返回第一条失败 expectation 的 mismatch failure，不继续校验后续 output expectation
|
||||
|
||||
### Requirement: LLM Stream 断言
|
||||
LLM checker SHALL 仅允许 `mode: stream` 使用 `expect.stream`。`expect.stream.completed` 未配置时，LLM checker SHALL 在 stream observation 路径使用默认 `true` 语义。`expect.stream.firstTokenMs` SHALL 使用共享 `ValueMatcher`，并仅统计第一个非空 `text-delta` 事件耗时，不统计 reasoning、tool call 或 source 事件。
|
||||
LLM checker SHALL 仅允许 `mode: stream` 使用 `expect.stream`。仅当用户配置了 `expect.stream` 且未配置 `expect.stream.completed` 时，resolve 阶段 SHALL 在 Resolved expect 中物化默认 `completed: true`；LLM checker MUST NOT 因为 `llm.mode: stream` 自动添加 `stream.completed` 断言。`expect.stream.firstTokenMs` SHALL 使用共享 `RawValueExpectation` 输入并在运行期使用 `ValueExpectation`，且仅统计第一个非空 `text-delta` 事件耗时，不统计 reasoning、tool call 或 source 事件。
|
||||
|
||||
#### Scenario: stream completed 默认值
|
||||
- **WHEN** target 配置 `llm.mode: stream` 且未配置 `expect.stream.completed`
|
||||
- **THEN** LLM checker SHALL 要求 SDK stream 正常完成
|
||||
- **WHEN** target 配置 `llm.mode: stream` 且配置 `expect.stream: {}`
|
||||
- **THEN** resolve 阶段 SHALL 在 Resolved expect 中物化 `stream.completed: true` 并要求 SDK stream 正常完成
|
||||
|
||||
#### Scenario: 未配置 expect.stream 不添加 completed
|
||||
- **WHEN** target 配置 `llm.mode: stream` 但未配置 `expect.stream`
|
||||
- **THEN** resolve 阶段 SHALL NOT 自动添加 `stream.completed` 断言
|
||||
|
||||
#### Scenario: stream error
|
||||
- **WHEN** `fullStream` 产生 error part
|
||||
|
||||
@@ -101,7 +101,7 @@
|
||||
|
||||
除 `headers`、`env`、`variables` 等明确声明为动态键值表的对象外，配置中的未知字段 SHALL 导致启动期配置错误。系统 MUST NOT 静默忽略未知字段。
|
||||
|
||||
所有 ValueMatcher 类型的 expect 字段 SHALL 在 JSON Schema 契约中声明为 `anyOf: [primitiveValue, matcherObject]` 联合类型，同时接受 primitive 原始值（string / number / boolean / null）和 ValueMatcher 对象。语义 validator SHALL 在校验 ValueMatcher 字段之前执行归一化，将 primitive 原始值转换为 `{equals: value}` 对象形式。数组和对象 MUST NOT 作为 ValueMatcher 原始值简写；需要对数组或对象执行 equals 匹配时，配置 MUST 显式写成 `{equals: value}`。
|
||||
所有 `RawValueExpectation` 类型的 expect 字段 SHALL 在 JSON Schema 契约中声明为 `anyOf: [primitiveValue, matcherObject]` 联合类型，同时接受 primitive 原始值（string / number / boolean / null）和 `ValueMatcher` 对象。语义 validator SHALL 在不修改输入的前提下校验 `RawValueExpectation`；primitive 原始值 SHALL 被视为合法 Raw 输入，并在 checker resolve 阶段转换为 `{equals: value}` 对象形式。数组和对象 MUST NOT 作为 `RawValueExpectation` 原始值简写；需要对数组或对象执行 equals 匹配时，配置 MUST 显式写成 `{equals: value}`。`RawKeyedExpectations` 的动态键值 schema SHALL 复用 `RawValueExpectation`，MUST NOT 通过额外 JSON value 分支接受数组或对象简写。
|
||||
|
||||
#### Scenario: target 缺少必填字段
|
||||
- **WHEN** YAML 中某个 target 缺少 id 或 type 字段
|
||||
@@ -196,20 +196,20 @@
|
||||
- **THEN** 系统 SHALL 以错误退出，提示该 target 的 status 数字不合法
|
||||
|
||||
#### Scenario: durationMs matcher 非法
|
||||
- **WHEN** YAML 中某个 target 的 `expect.durationMs` 不是合法 `ValueMatcher` 也不是 primitive 原始值
|
||||
- **WHEN** YAML 中某个 target 的 `expect.durationMs` 不是合法 `RawValueExpectation`
|
||||
- **THEN** 系统 SHALL 以错误退出，提示该 target 的 expect.durationMs 格式错误
|
||||
|
||||
#### Scenario: durationMs 原始值简写合法
|
||||
- **WHEN** YAML 中某个 target 配置 `expect.durationMs: 5000`
|
||||
- **THEN** 系统 SHALL 接受该配置，归一化为 `{equals: 5000}` 后校验通过
|
||||
- **THEN** 语义校验 SHALL 接受该 Raw primitive 简写且 MUST NOT 修改输入；checker resolve 阶段 SHALL 将其解析为 `{equals: 5000}`
|
||||
|
||||
#### Scenario: ValueMatcher 字段字符串简写合法
|
||||
- **WHEN** YAML 中某个 target 配置 `expect.finishReason: "stop"`
|
||||
- **THEN** 系统 SHALL 接受该配置，归一化为 `{equals: "stop"}` 后校验通过
|
||||
- **THEN** 语义校验 SHALL 接受该 Raw primitive 简写且 MUST NOT 修改输入；checker resolve 阶段 SHALL 将其解析为 `{equals: "stop"}`
|
||||
|
||||
#### Scenario: ValueMatcher 字段 null 简写合法
|
||||
- **WHEN** YAML 中某个 target 配置 ValueMatcher 字段值为 `null`
|
||||
- **THEN** 系统 SHALL 接受该配置，归一化为 `{equals: null}` 后校验通过
|
||||
- **THEN** 语义校验 SHALL 接受该 Raw primitive 简写且 MUST NOT 修改输入；checker resolve 阶段 SHALL 将其解析为 `{equals: null}`
|
||||
|
||||
#### Scenario: ValueMatcher 字段数组简写非法
|
||||
- **WHEN** YAML 中某个 target 配置 ValueMatcher 字段值为数组 `[1, 2]`
|
||||
@@ -223,40 +223,40 @@
|
||||
- **WHEN** YAML 中某个 target 配置 `type: icmp` 但缺少 `icmp.host`
|
||||
- **THEN** 系统 SHALL 以错误退出，提示该 target 缺少 icmp.host 字段
|
||||
|
||||
#### Scenario: ping expect 未知字段
|
||||
- **WHEN** YAML 中 ping target 的 expect 包含非 ping expect 字段
|
||||
#### Scenario: icmp expect 未知字段
|
||||
- **WHEN** YAML 中 icmp target 的 expect 包含非 icmp expect 字段
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect 包含未知字段
|
||||
|
||||
#### Scenario: HTTP expect headers 非法
|
||||
- **WHEN** YAML 中某个 HTTP target 的 `expect.headers` 不是对象，或某个 header 期望既不是字符串也不是合法 operator
|
||||
- **WHEN** YAML 中某个 HTTP target 的 `expect.headers` 不是对象，或某个 header 期望不是合法 `RawValueExpectation`
|
||||
- **THEN** 系统 SHALL 以错误退出，提示该 target 的 expect.headers 格式错误
|
||||
|
||||
#### Scenario: HTTP expect body 必须为数组
|
||||
- **WHEN** YAML 中某个 HTTP target 的 `expect.body` 已配置但不是数组
|
||||
- **THEN** 系统 SHALL 以错误退出，提示该 target 的 expect.body 必须为数组
|
||||
|
||||
#### Scenario: HTTP body rule 缺少支持字段
|
||||
#### Scenario: HTTP body expectation 缺少支持字段
|
||||
- **WHEN** YAML 中某个 HTTP target 的 `expect.body` 数组项未包含 contains、regex、json、css、xpath 任一支持字段
|
||||
- **THEN** 系统 SHALL 以错误退出，提示该 body rule 缺少支持的规则类型
|
||||
- **THEN** 系统 SHALL 以错误退出，提示该 body expectation 缺少支持的 expectation 类型
|
||||
|
||||
#### Scenario: HTTP body rule 同时配置多个支持字段
|
||||
- **WHEN** YAML 中某个 HTTP target 的同一条 body rule 同时包含 contains、regex、json、css、xpath 中的多个支持字段
|
||||
- **THEN** 系统 SHALL 以错误退出，提示每条 body rule 只能配置一种规则类型
|
||||
#### Scenario: HTTP body expectation 同时配置多个支持字段
|
||||
- **WHEN** YAML 中某个 HTTP target 的同一条 body expectation 同时包含 contains、regex、json、css、xpath 中的多个支持字段
|
||||
- **THEN** 系统 SHALL 以错误退出，提示每条 body expectation 只能配置一种 expectation 类型
|
||||
|
||||
#### Scenario: HTTP body regex 非法
|
||||
- **WHEN** YAML 中某个 HTTP target 的 body regex 规则不是字符串或不是可编译正则表达式
|
||||
- **WHEN** YAML 中某个 HTTP target 的 body regex expectation 不是字符串或不是可编译正则表达式
|
||||
- **THEN** 系统 SHALL 以错误退出，提示该 body regex 不合法
|
||||
|
||||
#### Scenario: HTTP body json path 非法
|
||||
- **WHEN** YAML 中某个 HTTP target 的 body json 规则缺少 path，或 path 不符合系统支持的 JSONPath 子集
|
||||
- **WHEN** YAML 中某个 HTTP target 的 body json expectation 缺少 path，或 path 不符合系统支持的 JSONPath 子集
|
||||
- **THEN** 系统 SHALL 以错误退出，提示该 body json path 不合法
|
||||
|
||||
#### Scenario: HTTP body css selector 非法
|
||||
- **WHEN** YAML 中某个 HTTP target 的 body css 规则缺少 selector，或 selector 不是非空字符串
|
||||
- **WHEN** YAML 中某个 HTTP target 的 body css expectation 缺少 selector，或 selector 不是非空字符串
|
||||
- **THEN** 系统 SHALL 以错误退出，提示该 body css selector 不合法
|
||||
|
||||
#### Scenario: HTTP body xpath path 非法
|
||||
- **WHEN** YAML 中某个 HTTP target 的 body xpath 规则缺少 path，或 path 不是非空字符串，或可被现有 XPath 库静态判定为语法错误
|
||||
- **WHEN** YAML 中某个 HTTP target 的 body xpath expectation 缺少 path，或 path 不是非空字符串，或可被现有 XPath 库静态判定为语法错误
|
||||
- **THEN** 系统 SHALL 以错误退出，提示该 body xpath path 不合法
|
||||
|
||||
#### Scenario: expect matcher 类型非法
|
||||
@@ -293,23 +293,27 @@
|
||||
|
||||
#### Scenario: 导出配置 JSON Schema
|
||||
- **WHEN** 仓库生成或检查配置契约
|
||||
- **THEN** 根目录 SHALL 存在 draft-07 `probe-config.schema.json`，且其内容 SHALL 与当前公共 fragments 和已注册 checker fragments 组装出的完整 schema 一致（包含 variables 段和 target 的 id/name 字段）。所有 ValueMatcher 字段的 schema SHALL 声明为 `anyOf: [primitiveValue, matcherObject]` 联合类型
|
||||
- **THEN** 根目录 SHALL 存在 draft-07 `probe-config.schema.json`，且其内容 SHALL 与当前公共 fragments 和已注册 checker fragments 组装出的完整 schema 一致（包含 variables 段和 target 的 id/name 字段）。所有 `RawValueExpectation` 字段的 schema SHALL 声明为 `anyOf: [primitiveValue, matcherObject]` 联合类型，`RawKeyedExpectations` 的 dynamic value schema SHALL 复用 `RawValueExpectation`
|
||||
|
||||
#### Scenario: JSON Schema ValueMatcher 接受原始值
|
||||
- **WHEN** 使用 JSON Schema 校验配置文件中 ValueMatcher 字段值为数字 `5000`
|
||||
- **THEN** JSON Schema 校验 SHALL 通过，因为 ValueMatcher schema 声明为 `anyOf: [primitiveValue, matcherObject]`
|
||||
#### Scenario: JSON Schema RawValueExpectation 接受原始值
|
||||
- **WHEN** 使用 JSON Schema 校验配置文件中 RawValueExpectation 字段值为数字 `5000`
|
||||
- **THEN** JSON Schema 校验 SHALL 通过，因为 RawValueExpectation schema 声明为 `anyOf: [primitiveValue, matcherObject]`
|
||||
|
||||
#### Scenario: JSON Schema ValueMatcher 接受 matcher 对象
|
||||
- **WHEN** 使用 JSON Schema 校验配置文件中 ValueMatcher 字段值为 `{lte: 5000}`
|
||||
#### Scenario: JSON Schema RawValueExpectation 接受 matcher 对象
|
||||
- **WHEN** 使用 JSON Schema 校验配置文件中 RawValueExpectation 字段值为 `{lte: 5000}`
|
||||
- **THEN** JSON Schema 校验 SHALL 通过
|
||||
|
||||
#### Scenario: JSON Schema ValueMatcher 拒绝数组原始值
|
||||
- **WHEN** 使用 JSON Schema 校验配置文件中 ValueMatcher 字段值为数组 `[1, 2]`
|
||||
#### Scenario: JSON Schema RawValueExpectation 拒绝数组原始值
|
||||
- **WHEN** 使用 JSON Schema 校验配置文件中 RawValueExpectation 字段值为数组 `[1, 2]`
|
||||
- **THEN** JSON Schema 校验 SHALL 失败，因为数组不属于 primitive 原始值或 matcher 对象
|
||||
|
||||
#### Scenario: JSON Schema ValueMatcher 接受 equals 数组对象
|
||||
- **WHEN** 使用 JSON Schema 校验配置文件中 ValueMatcher 字段值为 `{equals: [1, 2]}` 或 `{equals: {status: "ok"}}`
|
||||
#### Scenario: JSON Schema RawValueExpectation 接受 equals 数组对象
|
||||
- **WHEN** 使用 JSON Schema 校验配置文件中 RawValueExpectation 字段值为 `{equals: [1, 2]}` 或 `{equals: {status: "ok"}}`
|
||||
- **THEN** JSON Schema 校验 SHALL 通过，因为 `equals` 支持任意 JSON value
|
||||
|
||||
#### Scenario: JSON Schema RawKeyedExpectations 拒绝数组对象简写
|
||||
- **WHEN** 使用 JSON Schema 校验配置文件中 `expect.headers.X` 或 DB row 列值为数组 `["a"]` 或对象 `{nested: "ok"}`
|
||||
- **THEN** JSON Schema 校验 SHALL 失败，除非该值显式写在 `{equals: ...}` 下
|
||||
系统 SHALL 支持使用单位字符串配置读取上限，单位包括 `B`、`KB`、`MB` 和 `GB`。
|
||||
|
||||
#### Scenario: 解析 MB
|
||||
@@ -339,43 +343,45 @@
|
||||
- **THEN** 系统 SHALL 调用 `Bun.YAML.parse()` 将内容解析为配置对象
|
||||
|
||||
### Requirement: expect 配置增强
|
||||
系统 SHALL 支持 typed target 的领域专用 expect 配置，并通过共享 `ValueMatcher`、`ContentRules` 和 `KeyValueExpect` 表达可复用断言能力。状态类字段 SHALL 保持枚举或布尔语义，包括 HTTP/LLM 的 `status`（支持精确数字和范围模式）、cmd 的 `exitCode`、tcp 的 `connected`、ping 的 `alive` 和 udp 的 `responded`。数字指标字段 SHALL 使用 `ValueMatcher`，包括通用 `durationMs`、db 的 `rowCount`、udp 的 `responseSize`/`sourceHost`/`sourcePort`、ping 的 `packetLossPercent`/`avgLatencyMs`/`maxLatencyMs`、llm 的 usage token 与 stream 首 token 耗时。内容类字段 MUST 使用 `ContentRules` 数组表达配置顺序，包括 HTTP `body`、cmd `stdout`/`stderr`、tcp `banner`、udp `response`、llm `output` 和 db `result`。LLM `finishReason` 和 `rawFinishReason` SHALL 使用 `ValueMatcher`（非 ContentRules），因为它们是单值字符串元数据。键值类字段 SHALL 使用 `KeyValueExpect`，包括 HTTP/LLM `headers` 和 db `rows` 中的列值断言（db `rows` 的类型为 `Array<KeyValueExpect>`，外层数组按行索引，内层每个元素为 KeyValueExpect）。
|
||||
系统 SHALL 支持 typed target 的领域专用 expect 配置，并通过共享 `ValueMatcher`、`ContentExpectations` 和 `KeyedExpectations` 表达可复用断言能力。状态类字段 SHALL 保持枚举或布尔语义，包括 HTTP/LLM 的 `status`（支持精确数字和范围模式）、cmd 的 `exitCode`、tcp 的 `connected`、icmp 的 `alive` 和 udp 的 `responded`。value 类指标字段 SHALL 使用 `RawValueExpectation` 输入，并在 resolve 阶段归一化为运行期 `ValueExpectation`，包括通用 `durationMs`、db 的 `rowCount`、udp 的 `responseSize`/`sourceHost`/`sourcePort`、icmp 的 `packetLossPercent`/`avgLatencyMs`/`maxLatencyMs`、llm 的 usage token 与 stream 首 token 耗时。内容类字段 MUST 使用 `RawContentExpectations` 数组表达配置顺序，包括 HTTP `body`、cmd `stdout`/`stderr`、tcp `banner`、udp `response`、llm `output` 和 db `result`。LLM `finishReason` 和 `rawFinishReason` SHALL 使用 `RawValueExpectation`（非 ContentExpectations），因为它们是单值字符串元数据。键值类字段 SHALL 使用 `RawKeyedExpectations`，包括 HTTP/LLM `headers` 和 db `rows` 中的列值断言（db `rows` 的类型为 `Array<RawKeyedExpectations>`，外层数组按行索引，内层每个元素表达该行的列值断言）。
|
||||
|
||||
配置加载流程 SHALL 保留变量替换后的 Raw expect 作为用户配置快照，同时生成 Resolved expect 作为运行期执行计划。语义校验 SHALL 只读取 Raw expect 并报告问题，MUST NOT 原地归一化或修改 Raw expect。Store 持久化 SHALL 写入 Raw expect；checker execute SHALL 只消费 Resolved expect。
|
||||
|
||||
#### Scenario: 解析 HTTP expect 配置
|
||||
- **WHEN** YAML 配置文件中 HTTP target 的 expect 包含 status、headers、body 规则数组和 durationMs matcher
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 HTTP target 的 expect 字段
|
||||
- **WHEN** YAML 配置文件中 HTTP target 的 expect 包含 status、headers、body expectation 数组和 durationMs matcher
|
||||
- **THEN** 系统 SHALL 保留 Raw HTTP expect 快照，并生成包含默认 status、resolved keyed headers、resolved content body 和 resolved durationMs 的 HTTP Resolved expect
|
||||
|
||||
#### Scenario: 解析 cmd expect 配置
|
||||
- **WHEN** YAML 配置文件中 cmd target 的 expect 包含 exitCode、stdout、stderr 和 durationMs matcher
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 cmd target 的 expect 字段
|
||||
- **THEN** 系统 SHALL 保留 Raw cmd expect 快照，并生成包含默认 exitCode、resolved stdout/stderr content expectations 和 resolved durationMs 的 cmd Resolved expect
|
||||
|
||||
#### Scenario: 解析 db expect 配置
|
||||
- **WHEN** YAML 配置文件中 db target 的 expect 包含 durationMs、rowCount、rows 和 result
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 db target 的 expect 字段
|
||||
- **THEN** 系统 SHALL 保留 Raw db expect 快照，并生成包含 resolved rowCount、rows keyed expectations 和 result content expectations 的 db Resolved expect
|
||||
|
||||
#### Scenario: 解析 tcp expect 配置
|
||||
- **WHEN** YAML 配置文件中 tcp target 的 expect 包含 connected、banner 规则数组和 durationMs matcher
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 tcp target 的 expect 字段
|
||||
- **WHEN** YAML 配置文件中 tcp target 的 expect 包含 connected、banner expectation 数组和 durationMs matcher
|
||||
- **THEN** 系统 SHALL 保留 Raw tcp expect 快照，并生成包含默认 connected、resolved banner content expectations 和 resolved durationMs 的 tcp Resolved expect
|
||||
|
||||
#### Scenario: 解析 ping expect 配置
|
||||
- **WHEN** YAML 配置文件中 ping target 的 expect 包含 alive、packetLossPercent、avgLatencyMs、maxLatencyMs 和 durationMs matcher
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 ping target 的 expect 字段
|
||||
#### Scenario: 解析 icmp expect 配置
|
||||
- **WHEN** YAML 配置文件中 icmp target 的 expect 包含 alive、packetLossPercent、avgLatencyMs、maxLatencyMs 和 durationMs matcher
|
||||
- **THEN** 系统 SHALL 保留 Raw icmp expect 快照，并生成包含默认 alive 和 resolved 数值 expectations 的 icmp Resolved expect
|
||||
|
||||
#### Scenario: 解析 udp expect 配置
|
||||
- **WHEN** YAML 配置文件中 udp target 的 expect 包含 responded、response、responseSize、sourceHost、sourcePort 和 durationMs matcher
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 udp target 的 expect 字段
|
||||
- **THEN** 系统 SHALL 保留 Raw udp expect 快照，并生成包含默认 responded、resolved response content expectations 和 resolved value expectations 的 udp Resolved expect
|
||||
|
||||
#### Scenario: 解析 llm expect 配置
|
||||
- **WHEN** YAML 配置文件中 llm target 的 expect 包含 status、headers、output、finishReason、rawFinishReason、usage、stream 和 durationMs matcher
|
||||
- **THEN** 系统 SHALL 正确解析并存储为 llm target 的 expect 字段，并保留 output 内容规则数组顺序
|
||||
- **THEN** 系统 SHALL 保留 Raw llm expect 快照，并生成包含默认 status、resolved headers、output、finishReason、rawFinishReason、usage、stream 和 durationMs 的 llm Resolved expect，并保留 output 内容 expectation 数组顺序
|
||||
|
||||
#### Scenario: 解析有序 ContentRules 数组
|
||||
#### Scenario: 解析有序 ContentExpectations 数组
|
||||
- **WHEN** YAML 中任一内容类 expect 配置 contains、json、regex 三个数组项
|
||||
- **THEN** 系统 SHALL 保留数组顺序，供执行阶段按配置顺序快速失败
|
||||
- **THEN** 系统 SHALL 在 Raw expect 中保留数组顺序，并在 Resolved expect 中保留执行顺序，供执行阶段按配置顺序快速失败
|
||||
|
||||
#### Scenario: 不配置 HTTP status
|
||||
- **WHEN** HTTP target 未配置 `expect.status`
|
||||
- **THEN** 系统 SHALL 在执行 expect 时使用默认 `status: [200]` 语义
|
||||
- **THEN** 系统 SHALL 在 Resolved HTTP expect 中物化默认 `status: [200]` 语义
|
||||
|
||||
#### Scenario: 配置 HTTP status 范围模式
|
||||
- **WHEN** HTTP target 配置 `expect.status: ["2xx"]`
|
||||
@@ -383,19 +389,23 @@
|
||||
|
||||
#### Scenario: 不配置 cmd exitCode
|
||||
- **WHEN** cmd target 未配置 `expect.exitCode`
|
||||
- **THEN** 系统 SHALL 在执行 expect 时使用默认 `exitCode: [0]` 语义
|
||||
- **THEN** 系统 SHALL 在 Resolved cmd expect 中物化默认 `exitCode: [0]` 语义
|
||||
|
||||
#### Scenario: 不配置 expect
|
||||
- **WHEN** target 未配置任何 expect 规则
|
||||
- **THEN** 系统 SHALL 正常处理，expect 字段为 undefined，并由各 checker 使用自身默认状态语义
|
||||
- **THEN** 系统 SHALL 正常处理，Raw expect 快照为 undefined，Resolved expect 由各 checker 物化自身默认状态语义
|
||||
|
||||
#### Scenario: Raw expect 不被语义校验修改
|
||||
- **WHEN** YAML 中配置 `expect.durationMs: 1000`
|
||||
- **THEN** 语义校验 SHALL 接受该 Raw primitive 简写且 MUST NOT 将 Raw 输入原地修改为 `{equals: 1000}`
|
||||
|
||||
#### Scenario: 旧 maxDurationMs 字段不再支持
|
||||
- **WHEN** YAML 中任一 target 配置 `expect.maxDurationMs`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段未知，并要求使用 `expect.durationMs`
|
||||
|
||||
#### Scenario: 旧 match 字段不再支持
|
||||
- **WHEN** YAML 中任一 matcher 或内容规则配置 `match`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段未知，并要求使用 `regex`
|
||||
- **WHEN** YAML 中任一 matcher 或内容 expectation 配置 `match`
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段未知或不支持，并要求使用 `regex`
|
||||
|
||||
#### Scenario: durationMs matcher 配置
|
||||
- **WHEN** YAML 中任一 target 配置 `expect.durationMs: {lte: 1000}`
|
||||
@@ -405,14 +415,18 @@
|
||||
- **WHEN** YAML 中 `http.headers`、`defaults.http.headers`、`llm.headers`、`defaults.llm.headers` 或 `expect.headers` 包含任意 header 名称，且对应值符合契约
|
||||
- **THEN** 系统 SHALL 接受这些动态 header 名称
|
||||
|
||||
#### Scenario: ContentRules 字段必须为数组
|
||||
#### Scenario: ContentExpectations 字段必须为数组
|
||||
- **WHEN** YAML 中任一内容类 expect 字段配置为非数组
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段必须为规则数组
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示该字段必须为 expectation 数组
|
||||
|
||||
#### Scenario: regex 字段非法
|
||||
- **WHEN** YAML 中任一 `regex` 不是字符串、不是可编译正则或存在 ReDoS 风险
|
||||
- **THEN** 系统 SHALL 在启动期配置校验失败，提示对应 regex 不合法
|
||||
|
||||
#### Scenario: Store 持久化 Raw expect
|
||||
- **WHEN** 系统同步已解析 target 到 targets 表
|
||||
- **THEN** `targets.expect` SHALL 存储变量替换后的 Raw expect JSON，而不是包含 `kind` 或 resolved matcher 的运行期执行计划
|
||||
|
||||
### Requirement: 数据保留配置字段
|
||||
配置 schema 的 `runtime` 段 SHALL 支持 `retention` 字段，类型为字符串，格式为 `<数字><单位>`（单位：`d` 天、`h` 小时、`m` 分钟），用于指定历史数据保留时长。
|
||||
|
||||
@@ -498,7 +512,7 @@
|
||||
- **THEN** target schema SHALL 声明 `id` 的 minLength 为 1、maxLength 为 30，并声明 `name` 为可选字段，类型为 string 或 null，字符串的 minLength 为 1、maxLength 为 30
|
||||
|
||||
### Requirement: TCP 配置校验
|
||||
系统 SHALL 在启动期对 tcp checker 的配置契约和语义执行严格校验。Tcp target 的 `tcp` 分组 SHALL 只允许 `host`、`port`、`readBanner`、`bannerReadTimeout` 和 `maxBannerBytes` 字段；Tcp expect SHALL 只允许 `connected`、`durationMs` 和 `banner` 字段。`banner` MUST 为 `ContentRules` 数组。未知字段、非法类型、非法端口、非法 size 和不可编译正则 MUST 导致启动期配置错误。
|
||||
系统 SHALL 在启动期对 tcp checker 的配置契约和语义执行严格校验。Tcp target 的 `tcp` 分组 SHALL 只允许 `host`、`port`、`readBanner`、`bannerReadTimeout` 和 `maxBannerBytes` 字段；Tcp expect SHALL 只允许 `connected`、`durationMs` 和 `banner` 字段。`banner` MUST 为 `RawContentExpectations` 数组，`durationMs` SHALL 为 `RawValueExpectation`。未知字段、非法类型、非法端口、非法 size、非法 ContentExpectations 和不可编译正则 MUST 导致启动期配置错误。语义校验 MUST NOT 修改 Raw tcp expect 输入。
|
||||
|
||||
#### Scenario: tcp host 类型非法
|
||||
- **WHEN** YAML 中 tcp target 的 `tcp.host` 不是非空字符串
|
||||
@@ -525,7 +539,7 @@
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.connected 必须为布尔值
|
||||
|
||||
#### Scenario: tcp expect banner 非法
|
||||
- **WHEN** YAML 中 tcp target 的 `expect.banner` 不是合法 ContentRules 数组
|
||||
- **WHEN** YAML 中 tcp target 的 `expect.banner` 不是合法 `RawContentExpectations` 数组
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.banner 格式错误
|
||||
|
||||
#### Scenario: tcp expect banner regex 正则非法
|
||||
@@ -541,7 +555,7 @@
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 defaults.tcp 包含未知字段
|
||||
|
||||
### Requirement: LLM 配置校验
|
||||
系统 SHALL 在启动期对 llm checker 的配置契约和语义执行严格校验。LLM target 的 `llm` 分组 SHALL 只允许 `provider`、`url`、`model`、`prompt`、`mode`、`key`、`authToken`、`headers`、`ignoreSSL`、`options` 和 `providerOptions` 字段。`defaults.llm` 分组 SHALL 只允许 `mode`、`headers`、`ignoreSSL`、`options` 和 `providerOptions` 字段。LLM expect SHALL 只允许 `status`、`headers`、`output`、`finishReason`、`rawFinishReason`、`usage`、`stream` 和 `durationMs` 字段。`expect.output` MUST 为 `ContentRules` 数组。`expect.finishReason` 和 `expect.rawFinishReason` SHALL 使用 `ValueMatcher`。`expect.usage.*` 和 `expect.stream.firstTokenMs` SHALL 使用 `ValueMatcher`。未知字段、非法 provider、非法 URL、非法 mode、非法认证组合、非法 options、非法 output 规则和 `mode: http` 下配置 `expect.stream` MUST 导致启动期配置错误。
|
||||
系统 SHALL 在启动期对 llm checker 的配置契约和语义执行严格校验。LLM target 的 `llm` 分组 SHALL 只允许 `provider`、`url`、`model`、`prompt`、`mode`、`key`、`authToken`、`headers`、`ignoreSSL`、`options` 和 `providerOptions` 字段。`defaults.llm` 分组 SHALL 只允许 `mode`、`headers`、`ignoreSSL`、`options` 和 `providerOptions` 字段。LLM expect SHALL 只允许 `status`、`headers`、`output`、`finishReason`、`rawFinishReason`、`usage`、`stream` 和 `durationMs` 字段。`expect.output` MUST 为 `RawContentExpectations` 数组。`expect.finishReason`、`expect.rawFinishReason`、`expect.usage.*`、`expect.stream.firstTokenMs` 和 `expect.durationMs` SHALL 使用 `RawValueExpectation`。未知字段、非法 provider、非法 URL、非法 mode、非法认证组合、非法 options、非法 output expectation 和 `mode: http` 下配置 `expect.stream` MUST 导致启动期配置错误。语义校验 MUST NOT 修改 Raw llm expect 输入。
|
||||
|
||||
#### Scenario: llm provider 非法
|
||||
- **WHEN** YAML 中 llm target 的 `llm.provider` 不是 `openai`、`openai-responses` 或 `anthropic`
|
||||
@@ -591,24 +605,24 @@
|
||||
- **WHEN** YAML 中 llm target 配置 `api`、`providerName`、`baseURL`、`apiKey`、`messages`、`maxRetries`、`request`、`maxBodyBytes` 或 `maxStreamBytes`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 llm 分组包含未知字段
|
||||
|
||||
#### Scenario: llm output 规则缺少支持字段
|
||||
#### Scenario: llm output expectation 缺少支持字段
|
||||
- **WHEN** YAML 中 llm target 的 `expect.output` 数组项未包含任何合法 ValueMatcher 字段或 extractor
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 output rule 缺少支持的规则类型
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 output expectation 缺少支持的 expectation 类型
|
||||
|
||||
#### Scenario: llm output 规则同时配置多个 extractor
|
||||
- **WHEN** YAML 中 llm target 的同一条 output rule 同时包含 json、css、xpath 中的多个 extractor
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示每条 output rule 只能配置一种 extractor
|
||||
#### Scenario: llm output expectation 同时配置多个 extractor
|
||||
- **WHEN** YAML 中 llm target 的同一条 output expectation 同时包含 json、css、xpath 中的多个 extractor
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示每条 output expectation 只能配置一种 extractor
|
||||
|
||||
#### Scenario: llm output regex 非法
|
||||
- **WHEN** YAML 中 llm target 的 output regex 规则不是字符串、不是可编译正则表达式或存在 ReDoS 风险
|
||||
- **WHEN** YAML 中 llm target 的 output regex expectation 不是字符串、不是可编译正则表达式或存在 ReDoS 风险
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示该 output regex 不合法
|
||||
|
||||
#### Scenario: llm output json path 非法
|
||||
- **WHEN** YAML 中 llm target 的 output json 规则缺少 path，或 path 不符合系统支持的 JSONPath 子集
|
||||
- **WHEN** YAML 中 llm target 的 output json expectation 缺少 path，或 path 不符合系统支持的 JSONPath 子集
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示该 output json path 不合法
|
||||
|
||||
#### Scenario: llm expect usage 非法
|
||||
- **WHEN** YAML 中 llm target 的 `expect.usage.inputTokens`、`expect.usage.outputTokens` 或 `expect.usage.totalTokens` 不是合法 `ValueMatcher`
|
||||
- **WHEN** YAML 中 llm target 的 `expect.usage.inputTokens`、`expect.usage.outputTokens` 或 `expect.usage.totalTokens` 不是合法 `RawValueExpectation`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.usage 格式错误
|
||||
|
||||
#### Scenario: llm expect stream 仅允许 stream mode
|
||||
@@ -616,5 +630,5 @@
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.stream 仅支持 stream mode
|
||||
|
||||
#### Scenario: llm expect stream firstTokenMs 非法
|
||||
- **WHEN** YAML 中 llm target 的 `expect.stream.firstTokenMs` 不是合法 `ValueMatcher`
|
||||
- **WHEN** YAML 中 llm target 的 `expect.stream.firstTokenMs` 不是合法 `RawValueExpectation`
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 expect.stream.firstTokenMs 格式错误
|
||||
|
||||
@@ -30,11 +30,11 @@
|
||||
- **THEN** check_results 表的外键约束 SHALL 使用 `ON DELETE RESTRICT`，确保删除 target 时数据库层面阻止操作而非级联删除关联记录
|
||||
|
||||
### Requirement: targets 表同步
|
||||
系统 SHALL 在启动时将 YAML 配置中的目标列表同步到 SQLite targets 表，并持久化 target 类型、展示名称元信息、展示摘要、领域配置、调度配置、expect 配置、分组信息和目标说明。配置中不存在的 target SHALL 被标记为非活跃而非删除。
|
||||
系统 SHALL 在启动时将 YAML 配置中的目标列表同步到 SQLite targets 表，并持久化 target 类型、展示名称元信息、展示摘要、领域配置、调度配置、变量替换后的 Raw expect 配置快照、分组信息和目标说明。配置中不存在的 target SHALL 被标记为非活跃而非删除。`targets.expect` SHALL 存储 Raw expect JSON；系统 MUST NOT 将 Resolved expect 执行计划、`ContentExpectation.kind` union 或已归一化 matcher 包装结构写入该列。
|
||||
|
||||
#### Scenario: 首次同步目标
|
||||
- **WHEN** 数据库为空且 YAML 中定义了 N 个 typed target
|
||||
- **THEN** 系统 SHALL 将所有目标插入 targets 表，包含 name、description、type、target、config、interval_ms、timeout_ms、expect、grp 和 active=1，其中 name 和 description 均可为 NULL
|
||||
- **THEN** 系统 SHALL 将所有目标插入 targets 表，包含 name、description、type、target、config、interval_ms、timeout_ms、expect、grp 和 active=1，其中 name 和 description 均可为 NULL，expect 列保存变量替换后的 Raw expect JSON
|
||||
|
||||
#### Scenario: 配置变更后重新同步
|
||||
- **WHEN** YAML 配置发生变更（新增、删除或修改目标）后重启
|
||||
@@ -64,6 +64,14 @@
|
||||
- **WHEN** YAML target 配置 `description: null`
|
||||
- **THEN** targets 表 SHALL 将该目标的 description 存储为 NULL
|
||||
|
||||
#### Scenario: expect 列保存 Raw expect
|
||||
- **WHEN** target 配置 `expect.body: [{json: {path: "$.status"}}]` 和 `expect.durationMs: 1000`
|
||||
- **THEN** targets 表的 expect 列 SHALL 保存变量替换后的 Raw JSON，包含原始 `json.path` 和 `durationMs: 1000`，MUST NOT 保存 resolved `kind` 字段或 `{equals: 1000}` 执行计划
|
||||
|
||||
#### Scenario: 未配置 expect 写入 NULL
|
||||
- **WHEN** target 未配置任何 expect
|
||||
- **THEN** targets 表的 expect 列 SHALL 写入 NULL，即使 Resolved expect 中存在 checker 默认状态语义
|
||||
|
||||
### Requirement: check_results 表追加写入
|
||||
系统 SHALL 将每次检查结果追加写入 check_results 表，不更新或删除已有记录。
|
||||
|
||||
|
||||
@@ -121,11 +121,11 @@
|
||||
- **THEN** 系统 SHALL 正常记录执行结果并进入 expect 校验
|
||||
|
||||
### Requirement: expect 校验
|
||||
系统 SHALL 在 checker 执行完成后根据目标类型的 expect 配置校验观测结果，校验结果和首个失败原因记入 check result。HTTP checker 的 `durationMs` SHALL 表示完整 checker 执行耗时，包括重定向、响应体读取、响应体解码和 expect 校验。
|
||||
系统 SHALL 在 checker 执行完成后根据目标类型的 Resolved expect 执行计划校验观测结果，校验结果和首个失败原因记入 check result。HTTP checker 的 `durationMs` SHALL 表示完整 checker 执行耗时，包括重定向、响应体读取、响应体解码和 expect 校验。HTTP `expect.durationMs` SHALL 使用 `RawValueExpectation` 输入并在 resolve 阶段转换为运行期 `ValueExpectation`；旧 `expect.maxDurationMs` MUST NOT 再作为运行期耗时阈值使用。
|
||||
|
||||
#### Scenario: HTTP 默认状态码
|
||||
- **WHEN** HTTP target 未配置 `expect.status`
|
||||
- **THEN** 系统 SHALL 按默认 `status: [200]` 校验响应状态码
|
||||
- **THEN** 系统 SHALL 在 Resolved HTTP expect 中物化默认 `status: [200]` 并按该语义校验响应状态码
|
||||
|
||||
#### Scenario: 校验 HTTP 状态码精确值
|
||||
- **WHEN** HTTP target 配置了 `expect.status: [200, 201]`
|
||||
@@ -144,39 +144,39 @@
|
||||
- **THEN** 系统 SHALL 检查响应头是否符合指定规则，全部匹配时继续后续阶段
|
||||
|
||||
#### Scenario: 校验 HTTP 响应体
|
||||
- **WHEN** HTTP target 配置了有序 `expect.body` 规则数组
|
||||
- **THEN** 系统 SHALL 按数组顺序执行 body 规则，任一失败立即记录 failure 并停止后续规则
|
||||
- **WHEN** HTTP target 配置了有序 `expect.body` ContentExpectations 数组
|
||||
- **THEN** 系统 SHALL 按数组顺序执行 body expectations，任一失败立即记录 failure 并停止后续 expectation
|
||||
|
||||
#### Scenario: 校验 HTTP 完整耗时阈值
|
||||
- **WHEN** 目标配置了 `expect.maxDurationMs`，且 HTTP checker 完整执行（含重定向、body 读取、解码和 expect）后的 durationMs 超过阈值
|
||||
- **WHEN** 目标配置了 `expect.durationMs: {lte: 1000}`，且 HTTP checker 完整执行（含重定向、body 读取、解码和 expect）后的 durationMs 超过阈值
|
||||
- **THEN** 系统 SHALL 判定 duration 不匹配，记录完整 durationMs 和 duration failure
|
||||
|
||||
#### Scenario: HTTP body 前耗时已超阈值
|
||||
- **WHEN** HTTP target 配置了 body 校验和 `expect.maxDurationMs`，且进入 body 读取前的已耗时已超过阈值
|
||||
#### Scenario: HTTP body 前耗时已不可能满足 durationMs 上界
|
||||
- **WHEN** HTTP target 配置了 body 校验和 `expect.durationMs` 上界 matcher（如 `{lte: 1000}`），且进入 body 读取前的已耗时已使该 matcher 不可能通过
|
||||
- **THEN** 系统 SHALL 直接返回 duration failure，且 MUST NOT 读取 response body
|
||||
|
||||
#### Scenario: HTTP body 失败优先于后续 duration 检查
|
||||
- **WHEN** HTTP target 配置了 body 校验和 `expect.maxDurationMs`，body 阶段存在失败，且完整执行后 duration 也超过阈值
|
||||
- **WHEN** HTTP target 配置了 body 校验和 `expect.durationMs: {lte: 1000}`，body 阶段存在失败，且完整执行后 duration 也超过阈值
|
||||
- **THEN** 系统 SHALL 返回 body 阶段的失败（首个失败为准），durationMs SHALL 记录完整耗时
|
||||
|
||||
#### Scenario: HTTP 慢响应体计入耗时
|
||||
- **WHEN** HTTP target 配置了 body 校验和 `expect.maxDurationMs`，且响应头很快返回但响应体读取导致完整执行耗时超过阈值
|
||||
- **WHEN** HTTP target 配置了 body 校验和 `expect.durationMs: {lte: 1000}`，且响应头很快返回但响应体读取导致完整执行耗时超过阈值
|
||||
- **THEN** 系统 SHALL 判定 duration 不匹配并记录完整 durationMs
|
||||
|
||||
#### Scenario: 多条 expect 规则
|
||||
- **WHEN** 目标同时配置状态、duration、元数据和内容规则
|
||||
- **THEN** 系统 SHALL 所有规则全部通过时 matched 为 true，任一不通过则为 false 并记录首个失败原因
|
||||
- **WHEN** 目标同时配置状态、duration、元数据和内容 expectations
|
||||
- **THEN** 系统 SHALL 所有 expectations 全部通过时 matched 为 true，任一不通过则为 false 并记录首个失败原因
|
||||
|
||||
#### Scenario: cmd 默认 exitCode
|
||||
- **WHEN** cmd target 未配置 `expect.exitCode`
|
||||
- **THEN** 系统 SHALL 按默认 `exitCode: [0]` 校验命令退出码
|
||||
- **THEN** 系统 SHALL 在 Resolved cmd expect 中物化默认 `exitCode: [0]` 并按该语义校验命令退出码
|
||||
|
||||
#### Scenario: 校验 cmd stdout
|
||||
- **WHEN** cmd target 配置了有序 `expect.stdout` 规则数组
|
||||
- **THEN** 系统 SHALL 按数组顺序执行 stdout 规则，任一失败立即记录 failure 并停止后续规则
|
||||
- **WHEN** cmd target 配置了有序 `expect.stdout` ContentExpectations 数组
|
||||
- **THEN** 系统 SHALL 按数组顺序执行 stdout expectations，任一失败立即记录 failure 并停止后续 expectation
|
||||
|
||||
### Requirement: Body 校验按需解析
|
||||
系统 SHALL 仅在 HTTP target 配置了 body 校验，且 status、headers 阶段均通过，并且进入 body 前未确定 duration 已失败时才读取并解析响应体，避免不必要的读取和解析开销。HTTP target 未配置 body 校验时，系统 SHALL NOT 读取 response body。
|
||||
系统 SHALL 仅在 HTTP target 配置了 body 校验，且 status、headers 阶段均通过，并且进入 body 前未确定 `expect.durationMs` 已失败时才读取并解析响应体，避免不必要的读取和解析开销。HTTP target 未配置 body 校验时，系统 SHALL NOT 读取 response body。仅当 Resolved `durationMs` 包含上界 matcher 且当前已耗时已经使其不可能通过时，系统 MAY 在读取 body 前返回 duration failure；其他 duration matcher SHALL 在完整执行耗时可用后校验。
|
||||
|
||||
#### Scenario: status 失败时不读取 body
|
||||
- **WHEN** HTTP target 的 status 阶段不匹配
|
||||
@@ -186,17 +186,17 @@
|
||||
- **WHEN** HTTP target 的 status 阶段匹配但 headers 阶段不匹配
|
||||
- **THEN** 系统 SHALL 立即返回 matched=false，且 MUST NOT 读取 response body
|
||||
|
||||
#### Scenario: 进入 body 前 duration 已失败时不读取 body
|
||||
- **WHEN** HTTP target 已配置 `expect.maxDurationMs`，且进入 body 读取前的已耗时已经超过阈值
|
||||
#### Scenario: 进入 body 前 durationMs 上界已失败时不读取 body
|
||||
- **WHEN** HTTP target 已配置 `expect.durationMs` 上界 matcher，且进入 body 读取前的已耗时已经使该 matcher 不可能通过
|
||||
- **THEN** 系统 SHALL 返回 duration failure，且 MUST NOT 读取 response body
|
||||
|
||||
#### Scenario: 仅配置 contains 时不解析 JSON
|
||||
- **WHEN** HTTP target 仅配置 body contains 规则而未配置 json/css/xpath 规则
|
||||
- **WHEN** HTTP target 仅配置 body contains expectation 而未配置 json/css/xpath expectation
|
||||
- **THEN** 系统 SHALL 不执行 JSON.parse 或 HTML/XML 解析
|
||||
|
||||
#### Scenario: 配置 json 时解析 JSON 失败
|
||||
- **WHEN** HTTP target 配置了 body json 规则但响应体不是合法 JSON
|
||||
- **THEN** 系统 SHALL 判定 matched 为 false，并记录 json 规则对应的 failure.path
|
||||
- **WHEN** HTTP target 配置了 body json expectation 但响应体不是合法 JSON
|
||||
- **THEN** 系统 SHALL 判定 matched 为 false，并记录 json expectation 对应的 failure.path
|
||||
|
||||
### Requirement: HTTP 运行期错误归属
|
||||
HTTP checker SHALL 将运行期失败归属到实际失败阶段。请求、网络、TLS 和 timeout 错误 SHALL 记录为 request 阶段错误；body 超限、响应体解码失败、响应内容解析失败 SHALL 记录为 body 阶段错误；expect 不匹配 SHALL 记录为对应 mismatch 阶段。
|
||||
|
||||
@@ -86,27 +86,27 @@
|
||||
- **THEN** observation.banner SHALL 保存截断后的 banner 摘要，API detail SHALL 展示截断后的 banner 摘要，避免 UI 展示过长文本
|
||||
|
||||
### Requirement: tcp expect 校验
|
||||
系统 SHALL 支持 tcp 专属 expect，包括 `connected`、`banner` 和 `durationMs`，并按 connected、banner、durationMs 的阶段顺序快速失败。`connected` SHALL 保持布尔状态语义，未配置时默认 `true`。`banner` MUST 使用共享 `ContentRules` 数组，并仅在 `tcp.readBanner: true` 时允许配置。`durationMs` SHALL 使用共享 `ValueMatcher` 校验包含连接和 banner 读取在内的完整执行耗时。
|
||||
系统 SHALL 支持 tcp 专属 expect，包括 `connected`、`banner` 和 `durationMs`，并按 connected、banner、durationMs 的阶段顺序快速失败。`connected` SHALL 保持布尔状态语义，未配置时在 Resolved expect 中默认 `true`。`banner` MUST 使用共享 `RawContentExpectations` 数组输入并在运行期使用 `ContentExpectations`，且仅在 `tcp.readBanner: true` 时允许配置。`durationMs` SHALL 使用共享 `RawValueExpectation` 输入并在运行期使用 `ValueExpectation` 校验包含连接和 banner 读取在内的完整执行耗时。
|
||||
|
||||
#### Scenario: 默认 connected 成功语义
|
||||
- **WHEN** tcp target 未显式配置 `expect.connected`
|
||||
- **THEN** 系统 SHALL 使用默认 `expect.connected: true` 进行校验
|
||||
- **THEN** 系统 SHALL 在 Resolved tcp expect 中使用默认 `connected: true` 进行校验
|
||||
|
||||
#### Scenario: durationMs 校验
|
||||
- **WHEN** tcp target 配置 `expect.durationMs: {lte: 100}`，且完整执行耗时超过 100ms
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 phase 为 `duration`
|
||||
|
||||
#### Scenario: banner ContentRules 校验通过
|
||||
#### Scenario: banner ContentExpectations 校验通过
|
||||
- **WHEN** tcp target 配置 `readBanner: true`、`expect.banner: [{contains: "ESMTP"}]`，且实际 banner 包含 `ESMTP`
|
||||
- **THEN** 系统 SHALL 判定 banner 阶段通过
|
||||
|
||||
#### Scenario: banner regex 校验失败
|
||||
- **WHEN** tcp target 配置 `readBanner: true`、`expect.banner: [{regex: "^SSH-2\\.0"}]`，且实际 banner 不匹配该正则
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `banner`，path 指向失败的 banner 规则
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `banner`，path 指向失败的 banner expectation
|
||||
|
||||
#### Scenario: banner 多规则快速失败
|
||||
- **WHEN** tcp target 配置两条 banner 规则且第一条失败
|
||||
- **THEN** 系统 SHALL 返回第一条失败规则的 failure，并 MUST NOT 执行第二条规则
|
||||
- **WHEN** tcp target 配置两条 banner expectation 且第一条失败
|
||||
- **THEN** 系统 SHALL 返回第一条失败 expectation 的 failure，并 MUST NOT 执行第二条 expectation
|
||||
|
||||
#### Scenario: expect.banner 未开启 readBanner
|
||||
- **WHEN** tcp target 配置 `expect.banner`，但 `tcp.readBanner` 未配置为 `true`
|
||||
|
||||
@@ -133,13 +133,13 @@
|
||||
- **THEN** 系统 SHALL 以配置错误退出，提示 maxResponseBytes 格式错误
|
||||
|
||||
### Requirement: udp expect 校验
|
||||
系统 SHALL 支持 udp 专属 expect，包括 `responded`、`response`、`responseSize`、`sourceHost`、`sourcePort` 和 `durationMs`，并按 responded、responseSize、response、sourceHost、sourcePort、durationMs 的阶段顺序快速失败。`responded` SHALL 保持布尔状态语义，未配置时默认 `true`。`response` MUST 使用共享 `ContentRules` 数组，并作用于按 `udp.responseEncoding` 转换后的响应文本。`responseSize`、`sourceHost`、`sourcePort` 和 `durationMs` SHALL 使用共享 `ValueMatcher`。
|
||||
系统 SHALL 支持 udp 专属 expect，包括 `responded`、`response`、`responseSize`、`sourceHost`、`sourcePort` 和 `durationMs`，并按 responded、responseSize、response、sourceHost、sourcePort、durationMs 的阶段顺序快速失败。`responded` SHALL 保持布尔状态语义，未配置时在 Resolved expect 中默认 `true`。`response` MUST 使用共享 `RawContentExpectations` 数组输入并在运行期使用 `ContentExpectations`，且作用于按 `udp.responseEncoding` 转换后的响应文本。`responseSize`、`sourceHost`、`sourcePort` 和 `durationMs` SHALL 使用共享 `RawValueExpectation` 输入并在运行期使用 `ValueExpectation`。
|
||||
|
||||
#### Scenario: 默认 responded 成功语义
|
||||
- **WHEN** udp target 未显式配置 `expect.responded`
|
||||
- **THEN** 系统 SHALL 使用默认 `expect.responded: true` 进行校验
|
||||
- **THEN** 系统 SHALL 在 Resolved udp expect 中使用默认 `responded: true` 进行校验
|
||||
|
||||
#### Scenario: response ContentRules 校验通过
|
||||
#### Scenario: response ContentExpectations 校验通过
|
||||
- **WHEN** udp target 配置 `expect.response: [{ contains: "PONG" }]`，且按 `responseEncoding` 转换后的响应文本包含 `PONG`
|
||||
- **THEN** 系统 SHALL 判定 response 阶段通过
|
||||
|
||||
@@ -147,13 +147,13 @@
|
||||
- **WHEN** udp target 收到文本响应 `{"status":"ok"}` 且配置 `expect.response: [{json: {path: "$.status", equals: "ok"}}]`
|
||||
- **THEN** 系统 SHALL 判定 response 阶段通过
|
||||
|
||||
#### Scenario: response ContentRules 校验失败
|
||||
#### Scenario: response ContentExpectations 校验失败
|
||||
- **WHEN** udp target 配置 `expect.response: [{ contains: "PONG" }]`，但按 `responseEncoding` 转换后的响应文本不包含 `PONG`
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `response`，path 指向失败的 response 规则
|
||||
- **THEN** 系统 SHALL 返回 `matched=false`，failure 的 kind 为 `mismatch`，phase 为 `response`，path 指向失败的 response expectation
|
||||
|
||||
#### Scenario: responseEncoding 为 hex
|
||||
- **WHEN** udp target 配置 `udp.responseEncoding: "hex"` 且收到字节内容 `PONG`
|
||||
- **THEN** 系统 SHALL 将响应转换为小写 hex 字符串 `504f4e47` 后执行 `expect.response` 规则
|
||||
- **THEN** 系统 SHALL 将响应转换为小写 hex 字符串 `504f4e47` 后执行 `expect.response` expectation
|
||||
|
||||
#### Scenario: responseSize matcher 校验通过
|
||||
- **WHEN** udp target 配置 `expect.responseSize: { gte: 4 }`，且实际响应为 4 字节
|
||||
|
||||
@@ -590,32 +590,6 @@
|
||||
},
|
||||
"headers": {
|
||||
"additionalProperties": {
|
||||
"anyOf": [
|
||||
{
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"type": "number"
|
||||
},
|
||||
{
|
||||
"type": "boolean"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
},
|
||||
{
|
||||
"items": {},
|
||||
"type": "array"
|
||||
},
|
||||
{
|
||||
"additionalProperties": {},
|
||||
"type": "object"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"anyOf": [
|
||||
{
|
||||
"anyOf": [
|
||||
@@ -689,8 +663,6 @@
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"type": "object"
|
||||
},
|
||||
@@ -1901,32 +1873,6 @@
|
||||
"type": "array",
|
||||
"items": {
|
||||
"additionalProperties": {
|
||||
"anyOf": [
|
||||
{
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"type": "number"
|
||||
},
|
||||
{
|
||||
"type": "boolean"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
},
|
||||
{
|
||||
"items": {},
|
||||
"type": "array"
|
||||
},
|
||||
{
|
||||
"additionalProperties": {},
|
||||
"type": "object"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"anyOf": [
|
||||
{
|
||||
"anyOf": [
|
||||
@@ -2000,8 +1946,6 @@
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"type": "object"
|
||||
}
|
||||
@@ -3702,32 +3646,6 @@
|
||||
},
|
||||
"headers": {
|
||||
"additionalProperties": {
|
||||
"anyOf": [
|
||||
{
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"type": "number"
|
||||
},
|
||||
{
|
||||
"type": "boolean"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
},
|
||||
{
|
||||
"items": {},
|
||||
"type": "array"
|
||||
},
|
||||
{
|
||||
"additionalProperties": {},
|
||||
"type": "object"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"anyOf": [
|
||||
{
|
||||
"anyOf": [
|
||||
@@ -3801,8 +3719,6 @@
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"type": "object"
|
||||
},
|
||||
@@ -4625,7 +4541,7 @@
|
||||
"$id": "https://dial.local/probe-config.schema.json",
|
||||
"$schema": "http://json-schema.org/draft-07/schema#",
|
||||
"definitions": {
|
||||
"ContentRules": {
|
||||
"ContentExpectations": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"additionalProperties": false,
|
||||
@@ -4868,10 +4784,37 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"KeyValueExpect": {
|
||||
"KeyedExpectations": {
|
||||
"additionalProperties": {
|
||||
"anyOf": [
|
||||
{
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"type": "number"
|
||||
},
|
||||
{
|
||||
"type": "boolean"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"additionalProperties": false,
|
||||
"minProperties": 1,
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"contains": {
|
||||
"type": "string"
|
||||
},
|
||||
"empty": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"equals": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string"
|
||||
@@ -4895,7 +4838,31 @@
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"exists": {
|
||||
"type": "boolean"
|
||||
},
|
||||
"gt": {
|
||||
"type": "number"
|
||||
},
|
||||
"gte": {
|
||||
"type": "number"
|
||||
},
|
||||
"lt": {
|
||||
"type": "number"
|
||||
},
|
||||
"lte": {
|
||||
"type": "number"
|
||||
},
|
||||
"regex": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"type": "object"
|
||||
},
|
||||
"ValueExpectation": {
|
||||
"anyOf": [
|
||||
{
|
||||
"anyOf": [
|
||||
@@ -4969,30 +4936,8 @@
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
"type": "object"
|
||||
},
|
||||
"ValueMatcher": {
|
||||
"anyOf": [
|
||||
{
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"type": "number"
|
||||
},
|
||||
{
|
||||
"type": "boolean"
|
||||
},
|
||||
{
|
||||
"type": "null"
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"additionalProperties": false,
|
||||
"minProperties": 1,
|
||||
"type": "object",
|
||||
@@ -5047,7 +4992,5 @@
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,53 +1,67 @@
|
||||
import { DOMParser } from "@xmldom/xmldom";
|
||||
import * as cheerio from "cheerio";
|
||||
import { isPlainObject } from "es-toolkit";
|
||||
import * as xpath from "xpath";
|
||||
|
||||
import type { CheckFailure } from "../types";
|
||||
import type {
|
||||
ContentCssRule,
|
||||
ContentJsonRule,
|
||||
ContentRule,
|
||||
ContentRules,
|
||||
ContentXpathRule,
|
||||
ExpectResult,
|
||||
ContentCssExpectation,
|
||||
ContentExpectation,
|
||||
ContentExpectations,
|
||||
ContentJsonExpectation,
|
||||
ContentValueExpectation,
|
||||
ContentXpathExpectation,
|
||||
ExpectationResult,
|
||||
RawContentCssExpectation,
|
||||
RawContentExpectation,
|
||||
RawContentExpectations,
|
||||
RawContentJsonExpectation,
|
||||
RawContentXpathExpectation,
|
||||
ValueExpectation,
|
||||
ValueMatcher,
|
||||
} from "./types";
|
||||
|
||||
import { errorFailure, mismatchFailure } from "./failure";
|
||||
import { applyMatcher, evaluateJsonPath } from "./matcher";
|
||||
import { CONTENT_EXTRACTOR_KEY_SET, MATCHER_KEY_SET } from "./keys";
|
||||
import { applyValueMatcher, displayValueExpectation, evaluateJsonPath } from "./value";
|
||||
|
||||
type ParsedJsonResult = { error: string; ok: false } | { ok: true; value: unknown };
|
||||
|
||||
export function checkContentRules(
|
||||
export function checkContentExpectations(
|
||||
source: unknown,
|
||||
rules: ContentRules | undefined,
|
||||
expectations: ContentExpectations | undefined,
|
||||
options: { path?: string; phase: CheckFailure["phase"] },
|
||||
): ExpectResult {
|
||||
if (!rules || rules.length === 0) return { failure: null, matched: true };
|
||||
): ExpectationResult {
|
||||
if (!expectations || expectations.length === 0) return { failure: null, matched: true };
|
||||
|
||||
const basePath = options.path ?? options.phase;
|
||||
let parsedJson: ParsedJsonResult | undefined;
|
||||
|
||||
for (let i = 0; i < rules.length; i++) {
|
||||
const rule = rules[i]!;
|
||||
if ("json" in rule && parsedJson === undefined) {
|
||||
for (let i = 0; i < expectations.length; i++) {
|
||||
const expectation = expectations[i]!;
|
||||
if (expectation.kind === "json" && parsedJson === undefined) {
|
||||
parsedJson = parseJsonSource(source);
|
||||
}
|
||||
|
||||
const result = checkSingleContentRule(source, rule, `${basePath}[${i}]`, options.phase, parsedJson);
|
||||
const result = checkSingleContentExpectation(source, expectation, `${basePath}[${i}]`, options.phase, parsedJson);
|
||||
if (!result.matched) return result;
|
||||
}
|
||||
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
function checkCssRule(
|
||||
export function resolveContentExpectations(raw: RawContentExpectations | undefined): ContentExpectations | undefined {
|
||||
if (raw === undefined) return undefined;
|
||||
return raw.map((entry) => resolveContentExpectation(entry));
|
||||
}
|
||||
|
||||
function checkCssExpectation(
|
||||
source: unknown,
|
||||
rule: ContentCssRule,
|
||||
rulePath: string,
|
||||
expectation: ContentCssExpectation,
|
||||
expectationPath: string,
|
||||
phase: CheckFailure["phase"],
|
||||
): ExpectResult {
|
||||
const { attr, selector, ...matcher } = rule;
|
||||
const fullPath = `${rulePath}.css(${selector}${attr ? `@${attr}` : ""})`;
|
||||
): ExpectationResult {
|
||||
const fullPath = `${expectationPath}.css(${expectation.selector}${expectation.attr ? `@${expectation.attr}` : ""})`;
|
||||
|
||||
let $: cheerio.CheerioAPI;
|
||||
try {
|
||||
@@ -56,13 +70,18 @@ function checkCssRule(
|
||||
return { failure: errorFailure(phase, fullPath, "failed to parse HTML"), matched: false };
|
||||
}
|
||||
|
||||
const el = $(selector).first();
|
||||
const actual = el.length === 0 ? undefined : attr ? el.attr(attr) : el.text();
|
||||
const effectiveMatcher = Object.keys(matcher).length === 0 ? { exists: true } : matcher;
|
||||
const el = $(expectation.selector).first();
|
||||
const actual = el.length === 0 ? undefined : expectation.attr ? el.attr(expectation.attr) : el.text();
|
||||
|
||||
if (!applyMatcher(actual, effectiveMatcher)) {
|
||||
if (!applyValueMatcher(actual, expectation.matcher)) {
|
||||
return {
|
||||
failure: mismatchFailure(phase, fullPath, effectiveMatcher, actual, `css selector ${selector} mismatch`),
|
||||
failure: mismatchFailure(
|
||||
phase,
|
||||
fullPath,
|
||||
displayValueExpectation(expectation.matcher),
|
||||
actual,
|
||||
`css selector ${expectation.selector} mismatch`,
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
@@ -70,24 +89,28 @@ function checkCssRule(
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
function checkJsonRule(
|
||||
rule: ContentJsonRule,
|
||||
rulePath: string,
|
||||
function checkJsonExpectation(
|
||||
expectation: ContentJsonExpectation,
|
||||
expectationPath: string,
|
||||
phase: CheckFailure["phase"],
|
||||
parsedJson?: ParsedJsonResult,
|
||||
): ExpectResult {
|
||||
const { path, ...matcher } = rule;
|
||||
const fullPath = `${rulePath}.json(${path})`;
|
||||
): ExpectationResult {
|
||||
const fullPath = `${expectationPath}.json(${expectation.path})`;
|
||||
|
||||
if (!parsedJson?.ok) {
|
||||
return { failure: errorFailure(phase, fullPath, parsedJson?.error ?? "content is not valid JSON"), matched: false };
|
||||
}
|
||||
|
||||
const actual = evaluateJsonPath(parsedJson.value, path);
|
||||
const effectiveMatcher = Object.keys(matcher).length === 0 ? { exists: true } : matcher;
|
||||
if (!applyMatcher(actual, effectiveMatcher)) {
|
||||
const actual = evaluateJsonPath(parsedJson.value, expectation.path);
|
||||
if (!applyValueMatcher(actual, expectation.matcher)) {
|
||||
return {
|
||||
failure: mismatchFailure(phase, fullPath, effectiveMatcher, actual, `json path ${path} mismatch`),
|
||||
failure: mismatchFailure(
|
||||
phase,
|
||||
fullPath,
|
||||
displayValueExpectation(expectation.matcher),
|
||||
actual,
|
||||
`json path ${expectation.path} mismatch`,
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
@@ -95,34 +118,53 @@ function checkJsonRule(
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
function checkSingleContentRule(
|
||||
function checkSingleContentExpectation(
|
||||
source: unknown,
|
||||
rule: ContentRule,
|
||||
rulePath: string,
|
||||
expectation: ContentExpectation,
|
||||
expectationPath: string,
|
||||
phase: CheckFailure["phase"],
|
||||
parsedJson?: ParsedJsonResult,
|
||||
): ExpectResult {
|
||||
if ("json" in rule) return checkJsonRule(rule.json, rulePath, phase, parsedJson);
|
||||
if ("css" in rule) return checkCssRule(source, rule.css, rulePath, phase);
|
||||
if ("xpath" in rule) return checkXpathRule(source, rule.xpath, rulePath, phase);
|
||||
): ExpectationResult {
|
||||
switch (expectation.kind) {
|
||||
case "css":
|
||||
return checkCssExpectation(source, expectation, expectationPath, phase);
|
||||
case "json":
|
||||
return checkJsonExpectation(expectation, expectationPath, phase, parsedJson);
|
||||
case "value":
|
||||
return checkValueContentExpectation(source, expectation, expectationPath, phase);
|
||||
case "xpath":
|
||||
return checkXpathExpectation(source, expectation, expectationPath, phase);
|
||||
}
|
||||
}
|
||||
|
||||
if (!applyMatcher(source, rule, { stringifyNonString: true })) {
|
||||
function checkValueContentExpectation(
|
||||
source: unknown,
|
||||
expectation: ContentValueExpectation,
|
||||
expectationPath: string,
|
||||
phase: CheckFailure["phase"],
|
||||
): ExpectationResult {
|
||||
if (!applyValueMatcher(source, expectation.matcher, { stringifyNonString: true })) {
|
||||
return {
|
||||
failure: mismatchFailure(phase, rulePath, rule, source, `${phase} rule mismatch`),
|
||||
failure: mismatchFailure(
|
||||
phase,
|
||||
expectationPath,
|
||||
displayValueExpectation(expectation.matcher),
|
||||
source,
|
||||
`${phase} expectation mismatch`,
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
function checkXpathRule(
|
||||
function checkXpathExpectation(
|
||||
source: unknown,
|
||||
rule: ContentXpathRule,
|
||||
rulePath: string,
|
||||
expectation: ContentXpathExpectation,
|
||||
expectationPath: string,
|
||||
phase: CheckFailure["phase"],
|
||||
): ExpectResult {
|
||||
const { path, ...matcher } = rule;
|
||||
const fullPath = `${rulePath}.xpath(${path})`;
|
||||
): ExpectationResult {
|
||||
const fullPath = `${expectationPath}.xpath(${expectation.path})`;
|
||||
|
||||
let doc: ReturnType<DOMParser["parseFromString"]>;
|
||||
try {
|
||||
@@ -131,13 +173,18 @@ function checkXpathRule(
|
||||
return { failure: errorFailure(phase, fullPath, "failed to parse XML/HTML"), matched: false };
|
||||
}
|
||||
|
||||
const result = xpath.select(path, doc as unknown as Node);
|
||||
const result = xpath.select(expectation.path, doc as unknown as Node);
|
||||
const actual = xpathValue(result);
|
||||
const effectiveMatcher = Object.keys(matcher).length === 0 ? { exists: true } : matcher;
|
||||
|
||||
if (!applyMatcher(actual, effectiveMatcher)) {
|
||||
if (!applyValueMatcher(actual, expectation.matcher)) {
|
||||
return {
|
||||
failure: mismatchFailure(phase, fullPath, effectiveMatcher, actual, `xpath ${path} mismatch`),
|
||||
failure: mismatchFailure(
|
||||
phase,
|
||||
fullPath,
|
||||
displayValueExpectation(expectation.matcher),
|
||||
actual,
|
||||
`xpath ${expectation.path} mismatch`,
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
@@ -154,6 +201,28 @@ function contentText(source: unknown): string {
|
||||
return JSON.stringify(source) ?? "";
|
||||
}
|
||||
|
||||
function extractDirectMatcher(raw: Record<string, unknown>): ValueMatcher {
|
||||
const matcher: ValueMatcher = {};
|
||||
for (const [key, value] of Object.entries(raw)) {
|
||||
if (MATCHER_KEY_SET.has(key) && value !== undefined) {
|
||||
(matcher as Record<string, unknown>)[key] = value;
|
||||
}
|
||||
}
|
||||
return matcher;
|
||||
}
|
||||
|
||||
function extractExtractorMatcher(raw: Record<string, unknown>, ownFields: ReadonlySet<string>): ValueExpectation {
|
||||
const matcher: ValueMatcher = {};
|
||||
for (const [key, value] of Object.entries(raw)) {
|
||||
if (ownFields.has(key)) continue;
|
||||
if (MATCHER_KEY_SET.has(key) && value !== undefined) {
|
||||
(matcher as Record<string, unknown>)[key] = value;
|
||||
}
|
||||
}
|
||||
if (Object.keys(matcher).length === 0) return { exists: true };
|
||||
return matcher;
|
||||
}
|
||||
|
||||
function parseJsonSource(source: unknown): ParsedJsonResult {
|
||||
if (typeof source !== "string") return { ok: true, value: source };
|
||||
try {
|
||||
@@ -163,6 +232,44 @@ function parseJsonSource(source: unknown): ParsedJsonResult {
|
||||
}
|
||||
}
|
||||
|
||||
function resolveContentExpectation(raw: RawContentExpectation): ContentExpectation {
|
||||
if (!isPlainObject(raw)) {
|
||||
return { kind: "value", matcher: { equals: raw } };
|
||||
}
|
||||
const record = raw as Record<string, unknown>;
|
||||
|
||||
if (CONTENT_EXTRACTOR_KEY_SET.has("json") && isPlainObject(record["json"])) {
|
||||
const json = record["json"] as RawContentJsonExpectation;
|
||||
return {
|
||||
kind: "json",
|
||||
matcher: extractExtractorMatcher(json as unknown as Record<string, unknown>, new Set(["path"])),
|
||||
path: json.path,
|
||||
};
|
||||
}
|
||||
|
||||
if (isPlainObject(record["css"])) {
|
||||
const css = record["css"] as RawContentCssExpectation;
|
||||
const expectation: ContentCssExpectation = {
|
||||
kind: "css",
|
||||
matcher: extractExtractorMatcher(css as unknown as Record<string, unknown>, new Set(["attr", "selector"])),
|
||||
selector: css.selector,
|
||||
};
|
||||
if (css.attr !== undefined) expectation.attr = css.attr;
|
||||
return expectation;
|
||||
}
|
||||
|
||||
if (isPlainObject(record["xpath"])) {
|
||||
const xpathExpectation = record["xpath"] as RawContentXpathExpectation;
|
||||
return {
|
||||
kind: "xpath",
|
||||
matcher: extractExtractorMatcher(xpathExpectation as unknown as Record<string, unknown>, new Set(["path"])),
|
||||
path: xpathExpectation.path,
|
||||
};
|
||||
}
|
||||
|
||||
return { kind: "value", matcher: extractDirectMatcher(record) };
|
||||
}
|
||||
|
||||
function xpathValue(result: unknown): unknown {
|
||||
if (!Array.isArray(result)) return result;
|
||||
if (result.length === 0) return undefined;
|
||||
|
||||
14
src/server/checker/expect/headers.ts
Normal file
14
src/server/checker/expect/headers.ts
Normal file
@@ -0,0 +1,14 @@
|
||||
import type { ExpectationResult, KeyedExpectations } from "./types";
|
||||
|
||||
import { checkKeyedExpectations } from "./keyed";
|
||||
|
||||
export function checkHeaderExpectations(
|
||||
headers: Record<string, unknown>,
|
||||
expectations?: KeyedExpectations,
|
||||
): ExpectationResult {
|
||||
return checkKeyedExpectations(headers, expectations, {
|
||||
normalizeKey: (key) => key.toLowerCase(),
|
||||
path: "headers",
|
||||
phase: "headers",
|
||||
});
|
||||
}
|
||||
@@ -1,32 +0,0 @@
|
||||
import type { CheckFailure } from "../types";
|
||||
import type { ExpectResult, KeyValueExpect } from "./types";
|
||||
|
||||
import { mismatchFailure } from "./failure";
|
||||
import { checkExpectValue } from "./matcher";
|
||||
|
||||
export function checkKeyValueExpect(
|
||||
actual: Record<string, unknown>,
|
||||
expected: KeyValueExpect | undefined,
|
||||
options: { normalizeKey?: (key: string) => string; path?: string; phase: CheckFailure["phase"] },
|
||||
): ExpectResult {
|
||||
if (!expected) return { failure: null, matched: true };
|
||||
|
||||
const normalizeKey = options.normalizeKey ?? ((key: string) => key);
|
||||
const basePath = options.path ?? options.phase;
|
||||
const actualMap = new Map<string, unknown>();
|
||||
for (const [key, value] of Object.entries(actual)) {
|
||||
actualMap.set(normalizeKey(key), value);
|
||||
}
|
||||
|
||||
for (const [key, expectedValue] of Object.entries(expected)) {
|
||||
const actualValue = actualMap.get(normalizeKey(key));
|
||||
if (!checkExpectValue(actualValue, expectedValue)) {
|
||||
return {
|
||||
failure: mismatchFailure(options.phase, `${basePath}.${key}`, expectedValue, actualValue, `${key} mismatch`),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
46
src/server/checker/expect/keyed.ts
Normal file
46
src/server/checker/expect/keyed.ts
Normal file
@@ -0,0 +1,46 @@
|
||||
import type { CheckFailure } from "../types";
|
||||
import type { ExpectationResult, KeyedExpectations, RawKeyedExpectations } from "./types";
|
||||
|
||||
import { mismatchFailure } from "./failure";
|
||||
import { applyValueMatcher, displayValueExpectation, resolveValueExpectation } from "./value";
|
||||
|
||||
export function checkKeyedExpectations(
|
||||
actual: Record<string, unknown>,
|
||||
expectations: KeyedExpectations | undefined,
|
||||
options: { normalizeKey?: (key: string) => string; path?: string; phase: CheckFailure["phase"] },
|
||||
): ExpectationResult {
|
||||
if (!expectations || expectations.length === 0) return { failure: null, matched: true };
|
||||
|
||||
const normalizeKey = options.normalizeKey ?? ((key: string) => key);
|
||||
const basePath = options.path ?? options.phase;
|
||||
const actualMap = new Map<string, unknown>();
|
||||
for (const [key, value] of Object.entries(actual)) {
|
||||
actualMap.set(normalizeKey(key), value);
|
||||
}
|
||||
|
||||
for (const expectation of expectations) {
|
||||
const actualValue = actualMap.get(normalizeKey(expectation.key));
|
||||
if (!applyValueMatcher(actualValue, expectation.matcher)) {
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
options.phase,
|
||||
`${basePath}.${expectation.key}`,
|
||||
displayValueExpectation(expectation.matcher),
|
||||
actualValue,
|
||||
`${expectation.key} mismatch`,
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
export function resolveKeyedExpectations(raw: RawKeyedExpectations | undefined): KeyedExpectations | undefined {
|
||||
if (raw === undefined) return undefined;
|
||||
return Object.entries(raw).map(([key, value]) => ({
|
||||
key,
|
||||
matcher: resolveValueExpectation(value),
|
||||
}));
|
||||
}
|
||||
7
src/server/checker/expect/keys.ts
Normal file
7
src/server/checker/expect/keys.ts
Normal file
@@ -0,0 +1,7 @@
|
||||
export const MatcherKeys = ["contains", "empty", "equals", "exists", "gt", "gte", "lt", "lte", "regex"] as const;
|
||||
|
||||
export const MATCHER_KEY_SET: ReadonlySet<string> = new Set<string>(MatcherKeys);
|
||||
|
||||
export const ContentExtractorKeys = ["css", "json", "xpath"] as const;
|
||||
|
||||
export const CONTENT_EXTRACTOR_KEY_SET: ReadonlySet<string> = new Set<string>(ContentExtractorKeys);
|
||||
@@ -1,22 +0,0 @@
|
||||
import type { ValueMatcherPrimitive } from "./types";
|
||||
|
||||
import { isValueMatcherObject } from "./matcher";
|
||||
|
||||
export function isValueMatcherPrimitive(value: unknown): value is ValueMatcherPrimitive {
|
||||
return value === null || typeof value === "string" || typeof value === "number" || typeof value === "boolean";
|
||||
}
|
||||
|
||||
export function normalizeExpectMatchers(expect: Record<string, unknown>, keys: string[]): void {
|
||||
for (const key of keys) {
|
||||
if (key in expect) {
|
||||
expect[key] = normalizeValueMatcher(expect[key]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export function normalizeValueMatcher(value: unknown): unknown {
|
||||
if (value === undefined) return undefined;
|
||||
if (isValueMatcherObject(value)) return value;
|
||||
if (isValueMatcherPrimitive(value)) return { equals: value };
|
||||
return value;
|
||||
}
|
||||
27
src/server/checker/expect/status.ts
Normal file
27
src/server/checker/expect/status.ts
Normal file
@@ -0,0 +1,27 @@
|
||||
import { isNumber } from "es-toolkit";
|
||||
|
||||
import type { ExpectationResult } from "./types";
|
||||
|
||||
import { mismatchFailure } from "./failure";
|
||||
|
||||
export function checkStatusCode(statusCode: number, allowed: Array<number | string>): ExpectationResult {
|
||||
const matched = allowed.some((pattern) => {
|
||||
if (isNumber(pattern)) return statusCode === pattern;
|
||||
const base = parseInt(pattern[0]!, 10) * 100;
|
||||
return statusCode >= base && statusCode < base + 100;
|
||||
});
|
||||
|
||||
if (!matched) {
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
"status",
|
||||
"status",
|
||||
allowed,
|
||||
statusCode,
|
||||
`status ${statusCode} not in [${allowed.join(", ")}]`,
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
@@ -1,32 +1,75 @@
|
||||
import type { CheckFailure, JsonValue } from "../types";
|
||||
|
||||
export interface ContentCssRule extends ValueMatcher {
|
||||
export interface ContentCssExpectation {
|
||||
attr?: string;
|
||||
kind: "css";
|
||||
matcher: ValueExpectation;
|
||||
selector: string;
|
||||
}
|
||||
|
||||
export interface ContentJsonRule extends ValueMatcher {
|
||||
export type ContentExpectation =
|
||||
| ContentCssExpectation
|
||||
| ContentJsonExpectation
|
||||
| ContentValueExpectation
|
||||
| ContentXpathExpectation;
|
||||
|
||||
export type ContentExpectations = ContentExpectation[];
|
||||
|
||||
export interface ContentJsonExpectation {
|
||||
kind: "json";
|
||||
matcher: ValueExpectation;
|
||||
path: string;
|
||||
}
|
||||
|
||||
export type ContentRule =
|
||||
| ValueMatcher
|
||||
| { css: ContentCssRule }
|
||||
| { json: ContentJsonRule }
|
||||
| { xpath: ContentXpathRule };
|
||||
export interface ContentValueExpectation {
|
||||
kind: "value";
|
||||
matcher: ValueExpectation;
|
||||
}
|
||||
|
||||
export type ContentRules = ContentRule[];
|
||||
|
||||
export interface ContentXpathRule extends ValueMatcher {
|
||||
export interface ContentXpathExpectation {
|
||||
kind: "xpath";
|
||||
matcher: ValueExpectation;
|
||||
path: string;
|
||||
}
|
||||
|
||||
export interface ExpectResult {
|
||||
export interface ExpectationResult {
|
||||
failure: CheckFailure | null;
|
||||
matched: boolean;
|
||||
}
|
||||
|
||||
export type KeyValueExpect = Record<string, JsonValue | ValueMatcher>;
|
||||
export interface KeyedExpectation {
|
||||
key: string;
|
||||
matcher: ValueExpectation;
|
||||
}
|
||||
|
||||
export type KeyedExpectations = KeyedExpectation[];
|
||||
|
||||
export interface RawContentCssExpectation extends ValueMatcher {
|
||||
attr?: string;
|
||||
selector: string;
|
||||
}
|
||||
|
||||
export type RawContentExpectation =
|
||||
| ValueMatcher
|
||||
| { css: RawContentCssExpectation }
|
||||
| { json: RawContentJsonExpectation }
|
||||
| { xpath: RawContentXpathExpectation };
|
||||
|
||||
export type RawContentExpectations = RawContentExpectation[];
|
||||
|
||||
export interface RawContentJsonExpectation extends ValueMatcher {
|
||||
path: string;
|
||||
}
|
||||
|
||||
export interface RawContentXpathExpectation extends ValueMatcher {
|
||||
path: string;
|
||||
}
|
||||
|
||||
export type RawKeyedExpectations = Record<string, RawValueExpectation>;
|
||||
|
||||
export type RawValueExpectation = ValueMatcher | ValueMatcherPrimitive;
|
||||
|
||||
export type ValueExpectation = ValueMatcher;
|
||||
|
||||
export interface ValueMatcher {
|
||||
contains?: string;
|
||||
@@ -40,6 +83,4 @@ export interface ValueMatcher {
|
||||
regex?: string;
|
||||
}
|
||||
|
||||
export type ValueMatcherInput = ValueMatcher | ValueMatcherPrimitive;
|
||||
|
||||
export type ValueMatcherPrimitive = boolean | null | number | string;
|
||||
|
||||
@@ -6,14 +6,9 @@ import type { ConfigValidationIssue } from "../schema/issues";
|
||||
import type { JsonValue } from "../types";
|
||||
|
||||
import { issue, joinPath } from "../schema/issues";
|
||||
import { isValueMatcherPrimitive } from "./normalize";
|
||||
import { CONTENT_EXTRACTOR_KEY_SET, MATCHER_KEY_SET } from "./keys";
|
||||
import { isUnsafeRegex } from "./redos";
|
||||
|
||||
export const MatcherKeys = ["contains", "empty", "equals", "exists", "gt", "gte", "lt", "lte", "regex"] as const;
|
||||
|
||||
const MATCHER_KEY_SET = new Set<string>(MatcherKeys);
|
||||
const EXTRACTOR_KEYS = ["css", "json", "xpath"] as const;
|
||||
const EXTRACTOR_KEY_SET = new Set<string>(EXTRACTOR_KEYS);
|
||||
import { isValueMatcherPrimitive } from "./value";
|
||||
|
||||
export function isJsonValue(value: unknown): value is JsonValue {
|
||||
if (value === null) return true;
|
||||
@@ -28,45 +23,64 @@ export function isPlainRecord(value: unknown): value is Record<string, unknown>
|
||||
return isPlainObject(value);
|
||||
}
|
||||
|
||||
export function validateContentRules(rules: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!Array.isArray(rules)) return [issue("invalid-type", path, "必须为数组", targetName)];
|
||||
return rules.flatMap((rule, index) => validateContentRule(rule, `${path}[${index}]`, targetName));
|
||||
}
|
||||
|
||||
export function validateJsonPath(path: string, rulePath: string, targetName?: string): ConfigValidationIssue[] {
|
||||
export function validateJsonPath(path: string, expectationPath: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!path.startsWith("$.") || path.length <= 2) {
|
||||
return [issue("invalid-jsonpath", joinPath(rulePath, "path"), '必须为以 "$." 开头的有效 JSONPath', targetName)];
|
||||
return [
|
||||
issue("invalid-jsonpath", joinPath(expectationPath, "path"), '必须为以 "$." 开头的有效 JSONPath', targetName),
|
||||
];
|
||||
}
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const segments = path.slice(2).split(".");
|
||||
for (const seg of segments) {
|
||||
if (seg === "") {
|
||||
issues.push(issue("invalid-jsonpath", joinPath(rulePath, "path"), "包含空段", targetName));
|
||||
issues.push(issue("invalid-jsonpath", joinPath(expectationPath, "path"), "包含空段", targetName));
|
||||
}
|
||||
const bracketMatch = /^(.+?)\[(\d+)\]$/.exec(seg);
|
||||
if (bracketMatch?.[1]!.trim() === "") {
|
||||
issues.push(issue("invalid-jsonpath", joinPath(rulePath, "path"), "数组访问缺少属性名", targetName));
|
||||
issues.push(issue("invalid-jsonpath", joinPath(expectationPath, "path"), "数组访问缺少属性名", targetName));
|
||||
}
|
||||
}
|
||||
return issues;
|
||||
}
|
||||
|
||||
export function validateKeyValueExpect(value: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
export function validateRawContentExpectations(
|
||||
expectations: unknown,
|
||||
path: string,
|
||||
targetName?: string,
|
||||
): ConfigValidationIssue[] {
|
||||
if (!Array.isArray(expectations)) return [issue("invalid-type", path, "必须为数组", targetName)];
|
||||
return expectations.flatMap((entry, index) => validateRawContentExpectation(entry, `${path}[${index}]`, targetName));
|
||||
}
|
||||
|
||||
export function validateRawKeyedExpectations(
|
||||
value: unknown,
|
||||
path: string,
|
||||
targetName?: string,
|
||||
options?: { caseInsensitive?: boolean },
|
||||
): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(value)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
if (options?.caseInsensitive) {
|
||||
const seen = new Map<string, string>();
|
||||
for (const key of Object.keys(value)) {
|
||||
const lower = key.toLowerCase();
|
||||
const prev = seen.get(lower);
|
||||
if (prev !== undefined) {
|
||||
issues.push(issue("duplicate-key", joinPath(path, key), `与 "${prev}" 大小写归一化后重复`, targetName));
|
||||
} else {
|
||||
seen.set(lower, key);
|
||||
}
|
||||
}
|
||||
}
|
||||
for (const [key, item] of Object.entries(value)) {
|
||||
const itemPath = joinPath(path, key);
|
||||
if (isPlainRecord(item)) {
|
||||
issues.push(...validateValueMatcher(item, itemPath, targetName));
|
||||
} else if (!isJsonValue(item)) {
|
||||
issues.push(issue("invalid-type", itemPath, "必须为 JSON value 或 matcher 对象", targetName));
|
||||
}
|
||||
issues.push(...validateRawValueExpectation(item, itemPath, targetName));
|
||||
}
|
||||
return issues;
|
||||
}
|
||||
|
||||
export function validateValueMatcher(
|
||||
export function validateRawValueExpectation(
|
||||
matcher: unknown,
|
||||
path: string,
|
||||
targetName?: string,
|
||||
@@ -74,6 +88,16 @@ export function validateValueMatcher(
|
||||
): ConfigValidationIssue[] {
|
||||
const requireAtLeastOne = options.requireAtLeastOne ?? true;
|
||||
if (isValueMatcherPrimitive(matcher)) return [];
|
||||
if (Array.isArray(matcher)) {
|
||||
return [
|
||||
issue(
|
||||
"invalid-type",
|
||||
path,
|
||||
"必须为 primitive 原始值或 matcher 对象；如需数组 equals 匹配应写成 {equals: [...]}",
|
||||
targetName,
|
||||
),
|
||||
];
|
||||
}
|
||||
if (!isPlainRecord(matcher))
|
||||
return [issue("invalid-type", path, "必须为 primitive 原始值或 matcher 对象", targetName)];
|
||||
|
||||
@@ -100,82 +124,46 @@ export function validateValueMatcher(
|
||||
return issues;
|
||||
}
|
||||
|
||||
function validateContentRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const extractors = Object.keys(rule).filter((key) => EXTRACTOR_KEY_SET.has(key));
|
||||
const directMatchers = Object.keys(rule).filter((key) => MATCHER_KEY_SET.has(key));
|
||||
|
||||
for (const key of Object.keys(rule)) {
|
||||
if (!MATCHER_KEY_SET.has(key) && !EXTRACTOR_KEY_SET.has(key)) {
|
||||
issues.push(issue("unknown-field", joinPath(path, key), "是未知字段", targetName));
|
||||
}
|
||||
}
|
||||
|
||||
if (extractors.length > 1) {
|
||||
issues.push(issue("multiple-content-rules", path, "一条规则不能同时包含多个 extractor", targetName));
|
||||
}
|
||||
if (extractors.length === 1 && directMatchers.length > 0) {
|
||||
issues.push(issue("invalid-content-rule", path, "直接 matcher 不能与 extractor 混用", targetName));
|
||||
}
|
||||
if (issues.length > 0) return issues;
|
||||
|
||||
if (extractors.length === 0) return validateValueMatcher(rule, path, targetName);
|
||||
|
||||
const extractor = extractors[0]!;
|
||||
switch (extractor) {
|
||||
case "css":
|
||||
return validateCssRule(rule["css"], joinPath(path, "css"), targetName);
|
||||
case "json":
|
||||
return validateJsonRule(rule["json"], joinPath(path, "json"), targetName);
|
||||
case "xpath":
|
||||
return validateXpathRule(rule["xpath"], joinPath(path, "xpath"), targetName);
|
||||
}
|
||||
|
||||
return [];
|
||||
}
|
||||
|
||||
function validateCssRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
function validateCssExpectation(expectation: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(expectation)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
|
||||
if (!isString(rule["selector"]) || rule["selector"].trim() === "") {
|
||||
if (!isString(expectation["selector"]) || expectation["selector"].trim() === "") {
|
||||
issues.push(issue("invalid-type", joinPath(path, "selector"), "必须为非空字符串", targetName));
|
||||
}
|
||||
if ("attr" in rule && !isString(rule["attr"])) {
|
||||
if ("attr" in expectation && !isString(expectation["attr"])) {
|
||||
issues.push(issue("invalid-type", joinPath(path, "attr"), "必须为字符串", targetName));
|
||||
}
|
||||
issues.push(...validateExtractorMatcher(rule, new Set(["attr", "selector"]), path, targetName));
|
||||
issues.push(...validateExtractorMatcher(expectation, new Set(["attr", "selector"]), path, targetName));
|
||||
return issues;
|
||||
}
|
||||
|
||||
function validateExtractorMatcher(
|
||||
rule: Record<string, unknown>,
|
||||
expectation: Record<string, unknown>,
|
||||
allowedFields: Set<string>,
|
||||
path: string,
|
||||
targetName?: string,
|
||||
): ConfigValidationIssue[] {
|
||||
const matcher: Record<string, unknown> = {};
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
for (const [key, value] of Object.entries(rule)) {
|
||||
for (const [key, value] of Object.entries(expectation)) {
|
||||
if (allowedFields.has(key)) continue;
|
||||
matcher[key] = value;
|
||||
}
|
||||
issues.push(...validateValueMatcher(matcher, path, targetName, { requireAtLeastOne: false }));
|
||||
issues.push(...validateRawValueExpectation(matcher, path, targetName, { requireAtLeastOne: false }));
|
||||
return issues;
|
||||
}
|
||||
|
||||
function validateJsonRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
function validateJsonExpectation(expectation: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(expectation)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
|
||||
if (!isString(rule["path"])) {
|
||||
if (!isString(expectation["path"])) {
|
||||
issues.push(issue("invalid-type", joinPath(path, "path"), "必须为字符串", targetName));
|
||||
} else {
|
||||
issues.push(...validateJsonPath(rule["path"], path, targetName));
|
||||
issues.push(...validateJsonPath(expectation["path"], path, targetName));
|
||||
}
|
||||
issues.push(...validateExtractorMatcher(rule, new Set(["path"]), path, targetName));
|
||||
issues.push(...validateExtractorMatcher(expectation, new Set(["path"]), path, targetName));
|
||||
return issues;
|
||||
}
|
||||
|
||||
@@ -208,20 +196,62 @@ function validateMatcherValue(key: string, value: unknown, path: string, targetN
|
||||
}
|
||||
}
|
||||
|
||||
function validateXpathRule(rule: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(rule)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
function validateRawContentExpectation(
|
||||
expectation: unknown,
|
||||
path: string,
|
||||
targetName?: string,
|
||||
): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(expectation)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const extractors = Object.keys(expectation).filter((key) => CONTENT_EXTRACTOR_KEY_SET.has(key));
|
||||
const directMatchers = Object.keys(expectation).filter((key) => MATCHER_KEY_SET.has(key));
|
||||
|
||||
for (const key of Object.keys(expectation)) {
|
||||
if (!MATCHER_KEY_SET.has(key) && !CONTENT_EXTRACTOR_KEY_SET.has(key)) {
|
||||
issues.push(issue("unknown-field", joinPath(path, key), "是未知字段", targetName));
|
||||
}
|
||||
}
|
||||
|
||||
if (extractors.length > 1) {
|
||||
issues.push(
|
||||
issue("multiple-content-expectations", path, "一条 expectation 不能同时包含多个 extractor", targetName),
|
||||
);
|
||||
}
|
||||
if (extractors.length === 1 && directMatchers.length > 0) {
|
||||
issues.push(issue("invalid-content-expectation", path, "直接 matcher 不能与 extractor 混用", targetName));
|
||||
}
|
||||
if (issues.length > 0) return issues;
|
||||
|
||||
if (extractors.length === 0) return validateRawValueExpectation(expectation, path, targetName);
|
||||
|
||||
const extractor = extractors[0]!;
|
||||
switch (extractor) {
|
||||
case "css":
|
||||
return validateCssExpectation(expectation["css"], joinPath(path, "css"), targetName);
|
||||
case "json":
|
||||
return validateJsonExpectation(expectation["json"], joinPath(path, "json"), targetName);
|
||||
case "xpath":
|
||||
return validateXpathExpectation(expectation["xpath"], joinPath(path, "xpath"), targetName);
|
||||
}
|
||||
|
||||
return [];
|
||||
}
|
||||
|
||||
function validateXpathExpectation(expectation: unknown, path: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!isPlainRecord(expectation)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
|
||||
if (!isString(rule["path"]) || rule["path"].trim() === "") {
|
||||
if (!isString(expectation["path"]) || expectation["path"].trim() === "") {
|
||||
issues.push(issue("invalid-type", joinPath(path, "path"), "必须为非空字符串", targetName));
|
||||
} else {
|
||||
try {
|
||||
const doc = new DOMParser().parseFromString("<x/>", "text/xml");
|
||||
xpath.select(rule["path"], doc as unknown as Node);
|
||||
xpath.select(expectation["path"], doc as unknown as Node);
|
||||
} catch {
|
||||
issues.push(issue("invalid-xpath", joinPath(path, "path"), "xpath 不合法", targetName));
|
||||
}
|
||||
}
|
||||
issues.push(...validateExtractorMatcher(rule, new Set(["path"]), path, targetName));
|
||||
issues.push(...validateExtractorMatcher(expectation, new Set(["path"]), path, targetName));
|
||||
return issues;
|
||||
}
|
||||
@@ -1,15 +1,12 @@
|
||||
import { isEmptyObject, isEqual, isNil, isPlainObject } from "es-toolkit";
|
||||
|
||||
import type { CheckFailure, JsonValue } from "../types";
|
||||
import type { ExpectResult, ValueMatcher, ValueMatcherInput } from "./types";
|
||||
import type { CheckFailure } from "../types";
|
||||
import type { ExpectationResult, RawValueExpectation, ValueExpectation, ValueMatcher } from "./types";
|
||||
|
||||
import { mismatchFailure } from "./failure";
|
||||
import { MATCHER_KEY_SET } from "./keys";
|
||||
|
||||
export const MatcherKeys = ["contains", "empty", "equals", "exists", "gt", "gte", "lt", "lte", "regex"] as const;
|
||||
|
||||
const MATCHER_KEY_SET = new Set<string>(MatcherKeys);
|
||||
|
||||
export function applyMatcher(
|
||||
export function applyValueMatcher(
|
||||
actual: unknown,
|
||||
matcher: ValueMatcher,
|
||||
options: { stringifyNonString?: boolean } = {},
|
||||
@@ -57,28 +54,20 @@ export function applyMatcher(
|
||||
return true;
|
||||
}
|
||||
|
||||
export function checkExpectValue(actual: unknown, expected: JsonValue | ValueMatcher): boolean {
|
||||
if (isValueMatcherObject(expected)) {
|
||||
return applyMatcher(actual, expected);
|
||||
}
|
||||
return applyMatcher(actual, { equals: expected });
|
||||
}
|
||||
|
||||
export function checkValueMatcher(
|
||||
export function checkValueExpectation(
|
||||
actual: unknown,
|
||||
matcher: undefined | ValueMatcherInput,
|
||||
expectation: undefined | ValueExpectation,
|
||||
options: { message?: string; path: string; phase: CheckFailure["phase"]; stringifyNonString?: boolean },
|
||||
): ExpectResult {
|
||||
if (matcher === undefined) return { failure: null, matched: true };
|
||||
const normalized = isValueMatcherObject(matcher) ? matcher : { equals: matcher };
|
||||
if (applyMatcher(actual, normalized, { stringifyNonString: options.stringifyNonString })) {
|
||||
): ExpectationResult {
|
||||
if (expectation === undefined) return { failure: null, matched: true };
|
||||
if (applyValueMatcher(actual, expectation, { stringifyNonString: options.stringifyNonString })) {
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
options.phase,
|
||||
options.path,
|
||||
normalized,
|
||||
displayValueExpectation(expectation),
|
||||
actual,
|
||||
options.message ?? `${options.path} mismatch`,
|
||||
),
|
||||
@@ -86,6 +75,12 @@ export function checkValueMatcher(
|
||||
};
|
||||
}
|
||||
|
||||
export function displayValueExpectation(expectation: ValueExpectation): unknown {
|
||||
const entries = Object.entries(expectation).filter(([, value]) => value !== undefined);
|
||||
if (entries.length === 1 && entries[0]?.[0] === "equals") return entries[0][1];
|
||||
return expectation;
|
||||
}
|
||||
|
||||
export function evaluateJsonPath(json: unknown, path: string): unknown {
|
||||
if (!path.startsWith("$.")) return undefined;
|
||||
|
||||
@@ -113,6 +108,18 @@ export function isValueMatcherObject(value: unknown): value is ValueMatcher {
|
||||
return isPlainObject(value) && Object.keys(value).some((key) => MATCHER_KEY_SET.has(key));
|
||||
}
|
||||
|
||||
export function isValueMatcherPrimitive(value: unknown): value is boolean | null | number | string {
|
||||
return value === null || typeof value === "string" || typeof value === "number" || typeof value === "boolean";
|
||||
}
|
||||
|
||||
export function resolveValueExpectation(raw: RawValueExpectation): ValueExpectation;
|
||||
export function resolveValueExpectation(raw: RawValueExpectation | undefined): undefined | ValueExpectation;
|
||||
export function resolveValueExpectation(raw: RawValueExpectation | undefined): undefined | ValueExpectation {
|
||||
if (raw === undefined) return undefined;
|
||||
if (isValueMatcherObject(raw)) return raw;
|
||||
return { equals: raw };
|
||||
}
|
||||
|
||||
function compareNumber(
|
||||
actual: unknown,
|
||||
expected: number,
|
||||
@@ -3,11 +3,16 @@ import { resolve } from "node:path";
|
||||
|
||||
import type { CheckResult, RawTargetConfig } from "../../types";
|
||||
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
|
||||
import type { CommandExpectConfig, CommandTargetConfig, ResolvedCommandTarget } from "./types";
|
||||
import type {
|
||||
CommandTargetConfig,
|
||||
RawCommandExpectConfig,
|
||||
ResolvedCommandExpectConfig,
|
||||
ResolvedCommandTarget,
|
||||
} from "./types";
|
||||
|
||||
import { checkContentRules } from "../../expect/content";
|
||||
import { checkContentExpectations, resolveContentExpectations } from "../../expect/content";
|
||||
import { errorFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { checkValueExpectation, resolveValueExpectation } from "../../expect/value";
|
||||
import { parseSize } from "../../utils";
|
||||
import { checkExitCode } from "./expect";
|
||||
import { commandCheckerSchemas } from "./schema";
|
||||
@@ -138,7 +143,7 @@ export class CommandChecker implements CheckerDefinition<ResolvedCommandTarget>
|
||||
};
|
||||
}
|
||||
|
||||
const durationResult = checkValueMatcher(durationMs, t.expect?.durationMs, {
|
||||
const durationResult = checkValueExpectation(durationMs, t.expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
@@ -156,7 +161,10 @@ export class CommandChecker implements CheckerDefinition<ResolvedCommandTarget>
|
||||
}
|
||||
|
||||
if (t.expect?.stdout && t.expect.stdout.length > 0) {
|
||||
const stdoutResult = checkContentRules(outputResult.stdout, t.expect.stdout, { path: "stdout", phase: "stdout" });
|
||||
const stdoutResult = checkContentExpectations(outputResult.stdout, t.expect.stdout, {
|
||||
path: "stdout",
|
||||
phase: "stdout",
|
||||
});
|
||||
if (!stdoutResult.matched) {
|
||||
return {
|
||||
detail: null,
|
||||
@@ -171,7 +179,10 @@ export class CommandChecker implements CheckerDefinition<ResolvedCommandTarget>
|
||||
}
|
||||
|
||||
if (t.expect?.stderr && t.expect.stderr.length > 0) {
|
||||
const stderrResult = checkContentRules(outputResult.stderr, t.expect.stderr, { path: "stderr", phase: "stderr" });
|
||||
const stderrResult = checkContentExpectations(outputResult.stderr, t.expect.stderr, {
|
||||
path: "stderr",
|
||||
phase: "stderr",
|
||||
});
|
||||
if (!stderrResult.matched) {
|
||||
return {
|
||||
detail: null,
|
||||
@@ -207,6 +218,16 @@ export class CommandChecker implements CheckerDefinition<ResolvedCommandTarget>
|
||||
|
||||
const env = { ...process.env, ...(t.cmd.env ?? {}) } as Record<string, string>;
|
||||
|
||||
const rawExpect = target.expect as RawCommandExpectConfig | undefined;
|
||||
const resolvedExpect: ResolvedCommandExpectConfig = rawExpect
|
||||
? {
|
||||
durationMs: resolveValueExpectation(rawExpect.durationMs),
|
||||
exitCode: rawExpect.exitCode ?? [0],
|
||||
stderr: resolveContentExpectations(rawExpect.stderr),
|
||||
stdout: resolveContentExpectations(rawExpect.stdout),
|
||||
}
|
||||
: { exitCode: [0] };
|
||||
|
||||
return {
|
||||
cmd: {
|
||||
args: t.cmd.args ?? [],
|
||||
@@ -216,11 +237,12 @@ export class CommandChecker implements CheckerDefinition<ResolvedCommandTarget>
|
||||
maxOutputBytes,
|
||||
},
|
||||
description: null,
|
||||
expect: target.expect as CommandExpectConfig | undefined,
|
||||
expect: resolvedExpect,
|
||||
group: target.group ?? "default",
|
||||
id: t.id,
|
||||
intervalMs: context.defaultIntervalMs,
|
||||
name: t.name ?? null,
|
||||
rawExpect,
|
||||
timeoutMs: context.defaultTimeoutMs,
|
||||
type: "cmd",
|
||||
} satisfies ResolvedCommandTarget;
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import type { ExpectResult } from "../../expect/types";
|
||||
import type { ExpectationResult } from "../../expect/types";
|
||||
|
||||
import { mismatchFailure } from "../../expect/failure";
|
||||
|
||||
export function checkExitCode(exitCode: number, allowed: number[]): ExpectResult {
|
||||
export function checkExitCode(exitCode: number, allowed: number[]): ExpectationResult {
|
||||
if (!allowed.includes(exitCode)) {
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
|
||||
@@ -3,8 +3,8 @@ import { Type } from "@sinclair/typebox";
|
||||
import type { CheckerSchemas } from "../types";
|
||||
|
||||
import {
|
||||
createContentRulesSchema,
|
||||
createValueMatcherSchema,
|
||||
createRawContentExpectationsSchema,
|
||||
createRawValueExpectationSchema,
|
||||
sizeSchema,
|
||||
stringMapSchema,
|
||||
} from "../../schema/fragments";
|
||||
@@ -29,10 +29,10 @@ export const commandCheckerSchemas: CheckerSchemas = {
|
||||
),
|
||||
expect: Type.Object(
|
||||
{
|
||||
durationMs: Type.Optional(createValueMatcherSchema()),
|
||||
durationMs: Type.Optional(createRawValueExpectationSchema()),
|
||||
exitCode: Type.Optional(Type.Array(Type.Integer())),
|
||||
stderr: Type.Optional(createContentRulesSchema()),
|
||||
stdout: Type.Optional(createContentRulesSchema()),
|
||||
stderr: Type.Optional(createRawContentExpectationsSchema()),
|
||||
stdout: Type.Optional(createRawContentExpectationsSchema()),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
|
||||
@@ -1,4 +1,9 @@
|
||||
import type { ContentRules, ValueMatcherInput } from "../../expect/types";
|
||||
import type {
|
||||
ContentExpectations,
|
||||
RawContentExpectations,
|
||||
RawValueExpectation,
|
||||
ValueExpectation,
|
||||
} from "../../expect/types";
|
||||
import type { ResolvedTargetBase } from "../../types";
|
||||
|
||||
export interface CommandDefaultsConfig {
|
||||
@@ -6,13 +11,6 @@ export interface CommandDefaultsConfig {
|
||||
maxOutputBytes?: string;
|
||||
}
|
||||
|
||||
export interface CommandExpectConfig {
|
||||
durationMs?: ValueMatcherInput;
|
||||
exitCode?: number[];
|
||||
stderr?: ContentRules;
|
||||
stdout?: ContentRules;
|
||||
}
|
||||
|
||||
export interface CommandTargetConfig {
|
||||
args?: string[];
|
||||
cwd?: string;
|
||||
@@ -21,6 +19,13 @@ export interface CommandTargetConfig {
|
||||
maxOutputBytes?: string;
|
||||
}
|
||||
|
||||
export interface RawCommandExpectConfig {
|
||||
durationMs?: RawValueExpectation;
|
||||
exitCode?: number[];
|
||||
stderr?: RawContentExpectations;
|
||||
stdout?: RawContentExpectations;
|
||||
}
|
||||
|
||||
export interface ResolvedCommandConfig {
|
||||
args: string[];
|
||||
cwd: string;
|
||||
@@ -29,12 +34,20 @@ export interface ResolvedCommandConfig {
|
||||
maxOutputBytes: number;
|
||||
}
|
||||
|
||||
export interface ResolvedCommandExpectConfig {
|
||||
durationMs?: ValueExpectation;
|
||||
exitCode: number[];
|
||||
stderr?: ContentExpectations;
|
||||
stdout?: ContentExpectations;
|
||||
}
|
||||
|
||||
export interface ResolvedCommandTarget extends ResolvedTargetBase {
|
||||
cmd: ResolvedCommandConfig;
|
||||
expect?: CommandExpectConfig;
|
||||
expect?: ResolvedCommandExpectConfig;
|
||||
group: string;
|
||||
intervalMs: number;
|
||||
name: null | string;
|
||||
rawExpect?: RawCommandExpectConfig;
|
||||
timeoutMs: number;
|
||||
type: "cmd";
|
||||
}
|
||||
|
||||
@@ -1,17 +1,16 @@
|
||||
import { isNumber, isPlainObject, isString } from "es-toolkit";
|
||||
import { isNumber, isString } from "es-toolkit";
|
||||
|
||||
import type { ConfigValidationIssue } from "../../schema/issues";
|
||||
import type { CheckerValidationInput } from "../types";
|
||||
|
||||
import { normalizeExpectMatchers } from "../../expect/normalize";
|
||||
import { validateContentRules, validateValueMatcher } from "../../expect/validate-matcher";
|
||||
import { isPlainRecord, validateRawContentExpectations, validateRawValueExpectation } from "../../expect/validate";
|
||||
import { issue, joinPath } from "../../schema/issues";
|
||||
import { parseSize } from "../../utils";
|
||||
|
||||
export function validateCommandConfig(input: CheckerValidationInput): ConfigValidationIssue[] {
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const defaults =
|
||||
isPlainObject(input.defaults) && isPlainObject(input.defaults["cmd"]) ? input.defaults["cmd"] : undefined;
|
||||
isPlainRecord(input.defaults) && isPlainRecord(input.defaults["cmd"]) ? input.defaults["cmd"] : undefined;
|
||||
|
||||
if (isSizeInput(defaults?.["maxOutputBytes"])) {
|
||||
issues.push(...validateSizeValue(defaults["maxOutputBytes"], "defaults.cmd.maxOutputBytes"));
|
||||
@@ -19,7 +18,7 @@ export function validateCommandConfig(input: CheckerValidationInput): ConfigVali
|
||||
|
||||
for (let i = 0; i < input.targets.length; i++) {
|
||||
const target = input.targets[i] as unknown;
|
||||
if (!isPlainObject(target)) continue;
|
||||
if (!isPlainRecord(target)) continue;
|
||||
if (target["type"] !== "cmd") continue;
|
||||
issues.push(...validateCommandTarget(target, `targets[${i}]`));
|
||||
}
|
||||
@@ -39,20 +38,18 @@ function isSizeInput(value: unknown): value is number | string {
|
||||
function validateCommandExpect(target: Record<string, unknown>, path: string): ConfigValidationIssue[] {
|
||||
const targetName = getTargetName(target);
|
||||
const expect = target["expect"];
|
||||
if (expect === undefined || expect === null || !isPlainObject(expect)) return [];
|
||||
if (expect === undefined || expect === null || !isPlainRecord(expect)) return [];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const expectPath = joinPath(path, "expect");
|
||||
|
||||
normalizeExpectMatchers(expect, ["durationMs"]);
|
||||
|
||||
if (expect["stdout"] !== undefined) {
|
||||
issues.push(...validateContentRules(expect["stdout"], joinPath(expectPath, "stdout"), targetName));
|
||||
issues.push(...validateRawContentExpectations(expect["stdout"], joinPath(expectPath, "stdout"), targetName));
|
||||
}
|
||||
if (expect["stderr"] !== undefined) {
|
||||
issues.push(...validateContentRules(expect["stderr"], joinPath(expectPath, "stderr"), targetName));
|
||||
issues.push(...validateRawContentExpectations(expect["stderr"], joinPath(expectPath, "stderr"), targetName));
|
||||
}
|
||||
if (expect["durationMs"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
issues.push(...validateRawValueExpectation(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
}
|
||||
return issues;
|
||||
}
|
||||
@@ -61,7 +58,7 @@ function validateCommandTarget(target: Record<string, unknown>, path: string): C
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const targetName = getTargetName(target);
|
||||
const cmd = target["cmd"];
|
||||
if (!isPlainObject(cmd)) {
|
||||
if (!isPlainRecord(cmd)) {
|
||||
issues.push(issue("required", joinPath(path, "cmd"), "缺少 cmd.exec 字段", targetName));
|
||||
issues.push(...validateCommandExpect(target, path));
|
||||
return issues;
|
||||
|
||||
@@ -3,11 +3,12 @@ import { isError } from "es-toolkit";
|
||||
|
||||
import type { CheckResult, RawTargetConfig } from "../../types";
|
||||
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
|
||||
import type { DbExpectConfig, DbTargetConfig, ResolvedDbTarget } from "./types";
|
||||
import type { DbTargetConfig, RawDbExpectConfig, ResolvedDbExpectConfig, ResolvedDbTarget } from "./types";
|
||||
|
||||
import { checkContentRules } from "../../expect/content";
|
||||
import { checkContentExpectations, resolveContentExpectations } from "../../expect/content";
|
||||
import { errorFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { resolveKeyedExpectations } from "../../expect/keyed";
|
||||
import { checkValueExpectation, resolveValueExpectation } from "../../expect/value";
|
||||
import { checkRowCount, checkRows } from "./expect";
|
||||
import { dbCheckerSchemas } from "./schema";
|
||||
import { validateDbConfig } from "./validate";
|
||||
@@ -77,7 +78,7 @@ export class DbChecker implements CheckerDefinition<ResolvedDbTarget> {
|
||||
rowCount: null,
|
||||
rowsPreview: null,
|
||||
};
|
||||
const durationResult = checkValueMatcher(durationMs, t.expect?.durationMs, {
|
||||
const durationResult = checkValueExpectation(durationMs, t.expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
@@ -138,7 +139,7 @@ export class DbChecker implements CheckerDefinition<ResolvedDbTarget> {
|
||||
};
|
||||
}
|
||||
|
||||
const durationResult = checkValueMatcher(durationMs, t.expect?.durationMs, {
|
||||
const durationResult = checkValueExpectation(durationMs, t.expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
@@ -186,7 +187,10 @@ export class DbChecker implements CheckerDefinition<ResolvedDbTarget> {
|
||||
}
|
||||
|
||||
if (t.expect?.result && t.expect.result.length > 0) {
|
||||
const resultCheck = checkContentRules({ rowCount, rows }, t.expect.result, { path: "result", phase: "result" });
|
||||
const resultCheck = checkContentExpectations({ rowCount, rows }, t.expect.result, {
|
||||
path: "result",
|
||||
phase: "result",
|
||||
});
|
||||
if (!resultCheck.matched) {
|
||||
return {
|
||||
detail: null,
|
||||
@@ -223,17 +227,28 @@ export class DbChecker implements CheckerDefinition<ResolvedDbTarget> {
|
||||
resolve(target: RawTargetConfig, context: ResolveContext): ResolvedDbTarget {
|
||||
const t = target as RawTargetConfig & { db: DbTargetConfig; type: "db" };
|
||||
|
||||
const rawExpect = target.expect as RawDbExpectConfig | undefined;
|
||||
const resolvedExpect: ResolvedDbExpectConfig | undefined = rawExpect
|
||||
? {
|
||||
durationMs: resolveValueExpectation(rawExpect.durationMs),
|
||||
result: resolveContentExpectations(rawExpect.result),
|
||||
rowCount: resolveValueExpectation(rawExpect.rowCount),
|
||||
rows: rawExpect.rows?.map((r) => resolveKeyedExpectations(r)!),
|
||||
}
|
||||
: undefined;
|
||||
|
||||
return {
|
||||
db: {
|
||||
query: t.db.query,
|
||||
url: t.db.url,
|
||||
},
|
||||
description: null,
|
||||
expect: target.expect as DbExpectConfig | undefined,
|
||||
expect: resolvedExpect,
|
||||
group: target.group ?? "default",
|
||||
id: t.id,
|
||||
intervalMs: context.defaultIntervalMs,
|
||||
name: t.name ?? null,
|
||||
rawExpect,
|
||||
timeoutMs: context.defaultTimeoutMs,
|
||||
type: "db",
|
||||
} satisfies ResolvedDbTarget;
|
||||
|
||||
@@ -1,20 +1,20 @@
|
||||
import { isPlainObject } from "es-toolkit";
|
||||
|
||||
import type { ExpectResult, KeyValueExpect, ValueMatcherInput } from "../../expect/types";
|
||||
import type { ExpectationResult, KeyedExpectations, ValueExpectation } from "../../expect/types";
|
||||
|
||||
import { mismatchFailure } from "../../expect/failure";
|
||||
import { checkKeyValueExpect } from "../../expect/key-value";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { checkKeyedExpectations } from "../../expect/keyed";
|
||||
import { checkValueExpectation } from "../../expect/value";
|
||||
|
||||
export function checkRowCount(actual: number, matcher: ValueMatcherInput): ExpectResult {
|
||||
return checkValueMatcher(actual, matcher, {
|
||||
export function checkRowCount(actual: number, matcher: ValueExpectation): ExpectationResult {
|
||||
return checkValueExpectation(actual, matcher, {
|
||||
message: `rowCount ${actual} 不满足条件`,
|
||||
path: "rowCount",
|
||||
phase: "rowCount",
|
||||
});
|
||||
}
|
||||
|
||||
export function checkRows(rows: unknown, rules: KeyValueExpect[]): ExpectResult {
|
||||
export function checkRows(rows: unknown, rules: KeyedExpectations[]): ExpectationResult {
|
||||
if (!Array.isArray(rows)) {
|
||||
return {
|
||||
failure: mismatchFailure("row", "rows", rules, rows, "查询结果不是数组"),
|
||||
@@ -39,7 +39,7 @@ export function checkRows(rows: unknown, rules: KeyValueExpect[]): ExpectResult
|
||||
};
|
||||
}
|
||||
|
||||
const result = checkKeyValueExpect(row, rule, { path: `rows[${i}]`, phase: "row" });
|
||||
const result = checkKeyedExpectations(row, rule, { path: `rows[${i}]`, phase: "row" });
|
||||
if (!result.matched) return result;
|
||||
}
|
||||
|
||||
|
||||
@@ -2,7 +2,11 @@ import { Type } from "@sinclair/typebox";
|
||||
|
||||
import type { CheckerSchemas } from "../types";
|
||||
|
||||
import { createContentRulesSchema, createKeyValueExpectSchema, createValueMatcherSchema } from "../../schema/fragments";
|
||||
import {
|
||||
createRawContentExpectationsSchema,
|
||||
createRawKeyedExpectationsSchema,
|
||||
createRawValueExpectationSchema,
|
||||
} from "../../schema/fragments";
|
||||
|
||||
export const dbCheckerSchemas: CheckerSchemas = {
|
||||
config: Type.Object(
|
||||
@@ -19,10 +23,10 @@ export const dbCheckerSchemas: CheckerSchemas = {
|
||||
defaults: Type.Object({}, { additionalProperties: false }),
|
||||
expect: Type.Object(
|
||||
{
|
||||
durationMs: Type.Optional(createValueMatcherSchema()),
|
||||
result: Type.Optional(createContentRulesSchema()),
|
||||
rowCount: Type.Optional(createValueMatcherSchema()),
|
||||
rows: Type.Optional(Type.Array(createKeyValueExpectSchema())),
|
||||
durationMs: Type.Optional(createRawValueExpectationSchema()),
|
||||
result: Type.Optional(createRawContentExpectationsSchema()),
|
||||
rowCount: Type.Optional(createRawValueExpectationSchema()),
|
||||
rows: Type.Optional(Type.Array(createRawKeyedExpectationsSchema())),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
|
||||
@@ -1,29 +1,44 @@
|
||||
import type { ContentRules, KeyValueExpect, ValueMatcherInput } from "../../expect/types";
|
||||
import type {
|
||||
ContentExpectations,
|
||||
KeyedExpectations,
|
||||
RawContentExpectations,
|
||||
RawKeyedExpectations,
|
||||
RawValueExpectation,
|
||||
ValueExpectation,
|
||||
} from "../../expect/types";
|
||||
import type { ResolvedTargetBase } from "../../types";
|
||||
|
||||
export interface DbExpectConfig {
|
||||
durationMs?: ValueMatcherInput;
|
||||
result?: ContentRules;
|
||||
rowCount?: ValueMatcherInput;
|
||||
rows?: KeyValueExpect[];
|
||||
}
|
||||
|
||||
export interface DbTargetConfig {
|
||||
query?: string;
|
||||
url: string;
|
||||
}
|
||||
|
||||
export interface RawDbExpectConfig {
|
||||
durationMs?: RawValueExpectation;
|
||||
result?: RawContentExpectations;
|
||||
rowCount?: RawValueExpectation;
|
||||
rows?: RawKeyedExpectations[];
|
||||
}
|
||||
|
||||
export interface ResolvedDbConfig {
|
||||
query?: string;
|
||||
url: string;
|
||||
}
|
||||
|
||||
export interface ResolvedDbExpectConfig {
|
||||
durationMs?: ValueExpectation;
|
||||
result?: ContentExpectations;
|
||||
rowCount?: ValueExpectation;
|
||||
rows?: KeyedExpectations[];
|
||||
}
|
||||
|
||||
export interface ResolvedDbTarget extends ResolvedTargetBase {
|
||||
db: ResolvedDbConfig;
|
||||
expect?: DbExpectConfig;
|
||||
expect?: ResolvedDbExpectConfig;
|
||||
group: string;
|
||||
intervalMs: number;
|
||||
name: null | string;
|
||||
rawExpect?: RawDbExpectConfig;
|
||||
timeoutMs: number;
|
||||
type: "db";
|
||||
}
|
||||
|
||||
@@ -1,10 +1,14 @@
|
||||
import { isPlainObject, isString } from "es-toolkit";
|
||||
import { isString } from "es-toolkit";
|
||||
|
||||
import type { ConfigValidationIssue } from "../../schema/issues";
|
||||
import type { CheckerValidationInput } from "../types";
|
||||
|
||||
import { normalizeExpectMatchers } from "../../expect/normalize";
|
||||
import { validateContentRules, validateKeyValueExpect, validateValueMatcher } from "../../expect/validate-matcher";
|
||||
import {
|
||||
isPlainRecord,
|
||||
validateRawContentExpectations,
|
||||
validateRawKeyedExpectations,
|
||||
validateRawValueExpectation,
|
||||
} from "../../expect/validate";
|
||||
import { issue, joinPath } from "../../schema/issues";
|
||||
|
||||
export function validateDbConfig(input: CheckerValidationInput): ConfigValidationIssue[] {
|
||||
@@ -12,7 +16,7 @@ export function validateDbConfig(input: CheckerValidationInput): ConfigValidatio
|
||||
|
||||
for (let i = 0; i < input.targets.length; i++) {
|
||||
const target = input.targets[i] as unknown;
|
||||
if (!isPlainObject(target)) continue;
|
||||
if (!isPlainRecord(target)) continue;
|
||||
if (target["type"] !== "db") continue;
|
||||
issues.push(...validateDbTarget(target, `targets[${i}]`));
|
||||
}
|
||||
@@ -24,11 +28,11 @@ function collectRowExpects(rows: unknown[], path: string, targetName?: string):
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
for (let i = 0; i < rows.length; i++) {
|
||||
const row = rows[i]!;
|
||||
if (!isPlainObject(row)) {
|
||||
if (!isPlainRecord(row)) {
|
||||
issues.push(issue("invalid-type", `${path}[${i}]`, "必须为对象", targetName));
|
||||
continue;
|
||||
}
|
||||
issues.push(...validateKeyValueExpect(row, `${path}[${i}]`, targetName));
|
||||
issues.push(...validateRawKeyedExpectations(row, `${path}[${i}]`, targetName));
|
||||
}
|
||||
return issues;
|
||||
}
|
||||
@@ -41,18 +45,16 @@ function getTargetName(target: Record<string, unknown>): string | undefined {
|
||||
function validateDbExpect(target: Record<string, unknown>, path: string): ConfigValidationIssue[] {
|
||||
const targetName = getTargetName(target);
|
||||
const expect = target["expect"];
|
||||
if (expect === undefined || expect === null || !isPlainObject(expect)) return [];
|
||||
if (expect === undefined || expect === null || !isPlainRecord(expect)) return [];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const expectPath = joinPath(path, "expect");
|
||||
|
||||
normalizeExpectMatchers(expect, ["durationMs", "rowCount"]);
|
||||
|
||||
if (expect["durationMs"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
issues.push(...validateRawValueExpectation(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
}
|
||||
|
||||
if (expect["rowCount"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["rowCount"], joinPath(expectPath, "rowCount"), targetName));
|
||||
issues.push(...validateRawValueExpectation(expect["rowCount"], joinPath(expectPath, "rowCount"), targetName));
|
||||
}
|
||||
|
||||
if (expect["rows"] !== undefined) {
|
||||
@@ -64,10 +66,9 @@ function validateDbExpect(target: Record<string, unknown>, path: string): Config
|
||||
}
|
||||
|
||||
if (expect["result"] !== undefined) {
|
||||
issues.push(...validateContentRules(expect["result"], joinPath(expectPath, "result"), targetName));
|
||||
issues.push(...validateRawContentExpectations(expect["result"], joinPath(expectPath, "result"), targetName));
|
||||
}
|
||||
|
||||
// 检查未知字段
|
||||
const allowedKeys = new Set(["durationMs", "result", "rowCount", "rows"]);
|
||||
for (const key of Object.keys(expect)) {
|
||||
if (!allowedKeys.has(key)) {
|
||||
@@ -83,18 +84,16 @@ function validateDbTarget(target: Record<string, unknown>, path: string): Config
|
||||
const targetName = getTargetName(target);
|
||||
const db = target["db"];
|
||||
|
||||
if (!isPlainObject(db)) {
|
||||
if (!isPlainRecord(db)) {
|
||||
issues.push(issue("required", joinPath(path, "db"), "缺少 db.url 字段", targetName));
|
||||
issues.push(...validateDbExpect(target, path));
|
||||
return issues;
|
||||
}
|
||||
|
||||
// url 必填
|
||||
if (!isString(db["url"]) || db["url"].trim() === "") {
|
||||
issues.push(issue("required", joinPath(joinPath(path, "db"), "url"), "缺少 db.url 字段", targetName));
|
||||
}
|
||||
|
||||
// query 可选但不能为空字符串
|
||||
if (db["query"] !== undefined) {
|
||||
if (!isString(db["query"])) {
|
||||
issues.push(issue("invalid-type", joinPath(joinPath(path, "db"), "query"), "必须为字符串", targetName));
|
||||
@@ -110,7 +109,6 @@ function validateDbTarget(target: Record<string, unknown>, path: string): Config
|
||||
}
|
||||
}
|
||||
|
||||
// 检查未知字段
|
||||
const allowedDbKeys = new Set(["query", "url"]);
|
||||
for (const key of Object.keys(db)) {
|
||||
if (!allowedDbKeys.has(key)) {
|
||||
|
||||
@@ -2,18 +2,19 @@ import { isError } from "es-toolkit";
|
||||
|
||||
import type { CheckResult, RawTargetConfig } from "../../types";
|
||||
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
|
||||
import type { HttpExpectConfig, HttpTargetConfig, ResolvedHttpTarget } from "./types";
|
||||
import type { HttpTargetConfig, RawHttpExpectConfig, ResolvedHttpExpectConfig, ResolvedHttpTarget } from "./types";
|
||||
|
||||
import { checkContentRules } from "../../expect/content";
|
||||
import { checkContentExpectations, resolveContentExpectations } from "../../expect/content";
|
||||
import { errorFailure, mismatchFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher, isValueMatcherObject } from "../../expect/matcher";
|
||||
import { checkHeaderExpectations } from "../../expect/headers";
|
||||
import { resolveKeyedExpectations } from "../../expect/keyed";
|
||||
import { checkStatusCode } from "../../expect/status";
|
||||
import { checkValueExpectation, displayValueExpectation, resolveValueExpectation } from "../../expect/value";
|
||||
import { parseSize } from "../../utils";
|
||||
import { checkHeaders, checkStatus } from "./expect";
|
||||
import { httpCheckerSchemas } from "./schema";
|
||||
import { validateHttpConfig } from "./validate";
|
||||
|
||||
const CHARSET_RE = /charset="?([^";\s]+)"?/i;
|
||||
const BODY_PREVIEW_BYTES = 1024;
|
||||
const REDIRECT_STATUSES = new Set([301, 302, 303, 307, 308]);
|
||||
const SENSITIVE_HEADERS = new Set(["authorization", "cookie"]);
|
||||
|
||||
@@ -46,27 +47,11 @@ export class HttpChecker implements CheckerDefinition<ResolvedHttpTarget> {
|
||||
|
||||
const statusCode = response.status;
|
||||
const responseHeaders = truncateHeaders(Object.fromEntries(response.headers));
|
||||
const hasBodyRules = !!(expect?.body && expect.body.length > 0);
|
||||
const bodyReadResult = await readBodyStream(
|
||||
response,
|
||||
hasBodyRules ? t.http.maxBodyBytes : BODY_PREVIEW_BYTES,
|
||||
!hasBodyRules,
|
||||
);
|
||||
let bodyPreview: null | string = null;
|
||||
let bodyText: null | string = null;
|
||||
let bodyDecodeFailure: CheckResult["failure"] = null;
|
||||
const hasBodyExpectations = !!(expect?.body && expect.body.length > 0);
|
||||
|
||||
if (bodyReadResult.data.byteLength > 0) {
|
||||
const decodeResult = decodeBody(bodyReadResult.data, response.headers);
|
||||
if (decodeResult.ok) {
|
||||
bodyText = decodeResult.text;
|
||||
bodyPreview = truncateBodyPreview(decodeResult.text);
|
||||
} else {
|
||||
bodyDecodeFailure = decodeResult.failure;
|
||||
}
|
||||
}
|
||||
|
||||
const statusResult = checkStatus(statusCode, expect?.status ?? [200]);
|
||||
const statusResult = checkStatusCode(statusCode, expect?.status ?? [200]);
|
||||
if (!statusResult.matched) {
|
||||
return makeResult(
|
||||
t,
|
||||
@@ -79,7 +64,7 @@ export class HttpChecker implements CheckerDefinition<ResolvedHttpTarget> {
|
||||
);
|
||||
}
|
||||
|
||||
const headersResult = checkHeaders(Object.fromEntries(response.headers), expect?.headers);
|
||||
const headersResult = checkHeaderExpectations(Object.fromEntries(response.headers), expect?.headers);
|
||||
if (!headersResult.matched) {
|
||||
return makeResult(
|
||||
t,
|
||||
@@ -92,7 +77,7 @@ export class HttpChecker implements CheckerDefinition<ResolvedHttpTarget> {
|
||||
);
|
||||
}
|
||||
|
||||
const earlyTimeout = hasBodyRules ? checkEarlyTimeout(start, expect?.durationMs) : null;
|
||||
const earlyTimeout = hasBodyExpectations ? checkEarlyTimeout(start, expect?.durationMs) : null;
|
||||
if (earlyTimeout) {
|
||||
return makeResult(
|
||||
t,
|
||||
@@ -105,6 +90,20 @@ export class HttpChecker implements CheckerDefinition<ResolvedHttpTarget> {
|
||||
);
|
||||
}
|
||||
|
||||
if (hasBodyExpectations) {
|
||||
const bodyReadResult = await readBodyStream(response, t.http.maxBodyBytes);
|
||||
let bodyDecodeFailure: CheckResult["failure"] = null;
|
||||
|
||||
if (bodyReadResult.data.byteLength > 0) {
|
||||
const decodeResult = decodeBody(bodyReadResult.data, response.headers);
|
||||
if (decodeResult.ok) {
|
||||
bodyText = decodeResult.text;
|
||||
bodyPreview = truncateBodyPreview(decodeResult.text);
|
||||
} else {
|
||||
bodyDecodeFailure = decodeResult.failure;
|
||||
}
|
||||
}
|
||||
|
||||
if (!bodyReadResult.ok) {
|
||||
return makeResult(
|
||||
t,
|
||||
@@ -129,8 +128,7 @@ export class HttpChecker implements CheckerDefinition<ResolvedHttpTarget> {
|
||||
);
|
||||
}
|
||||
|
||||
if (hasBodyRules) {
|
||||
const bodyResult = checkContentRules(bodyText ?? "", expect.body, { path: "body", phase: "body" });
|
||||
const bodyResult = checkContentExpectations(bodyText ?? "", expect.body, { path: "body", phase: "body" });
|
||||
if (!bodyResult.matched) {
|
||||
return makeResult(
|
||||
t,
|
||||
@@ -145,7 +143,7 @@ export class HttpChecker implements CheckerDefinition<ResolvedHttpTarget> {
|
||||
}
|
||||
|
||||
const durationMs = Math.round(performance.now() - start);
|
||||
const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
const durationResult = checkValueExpectation(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
@@ -184,9 +182,19 @@ export class HttpChecker implements CheckerDefinition<ResolvedHttpTarget> {
|
||||
const method = t.http.method ?? "GET";
|
||||
const maxBodyBytes = parseSize(t.http.maxBodyBytes ?? httpDefaults?.maxBodyBytes ?? "100MB");
|
||||
|
||||
const rawExpect = target.expect as RawHttpExpectConfig | undefined;
|
||||
const resolvedExpect: ResolvedHttpExpectConfig = rawExpect
|
||||
? {
|
||||
body: resolveContentExpectations(rawExpect.body),
|
||||
durationMs: resolveValueExpectation(rawExpect.durationMs),
|
||||
headers: resolveKeyedExpectations(rawExpect.headers),
|
||||
status: rawExpect.status ?? [200],
|
||||
}
|
||||
: { status: [200] };
|
||||
|
||||
return {
|
||||
description: null,
|
||||
expect: target.expect as HttpExpectConfig | undefined,
|
||||
expect: resolvedExpect,
|
||||
group: target.group ?? "default",
|
||||
http: {
|
||||
body: t.http.body,
|
||||
@@ -200,6 +208,7 @@ export class HttpChecker implements CheckerDefinition<ResolvedHttpTarget> {
|
||||
id: t.id,
|
||||
intervalMs: context.defaultIntervalMs,
|
||||
name: t.name ?? null,
|
||||
rawExpect,
|
||||
timeoutMs: context.defaultTimeoutMs,
|
||||
type: "http",
|
||||
} satisfies ResolvedHttpTarget;
|
||||
@@ -277,20 +286,16 @@ function buildRedirectInit(init: RequestInit, statusCode: number, fromUrl: strin
|
||||
|
||||
function checkEarlyTimeout(
|
||||
start: number,
|
||||
durationMatcher: HttpExpectConfig["durationMs"] | undefined,
|
||||
durationMatcher: ResolvedHttpExpectConfig["durationMs"] | undefined,
|
||||
): null | { elapsed: number; failure: CheckResult["failure"] } {
|
||||
if (!isValueMatcherObject(durationMatcher)) return null;
|
||||
const limit = Math.min(
|
||||
durationMatcher.lte ?? Number.POSITIVE_INFINITY,
|
||||
durationMatcher.lt ?? Number.POSITIVE_INFINITY,
|
||||
);
|
||||
if (!Number.isFinite(limit)) return null;
|
||||
|
||||
if (!durationMatcher) return null;
|
||||
const elapsed = performance.now() - start;
|
||||
if (durationMatcher.lt !== undefined ? elapsed < limit : elapsed <= limit) return null;
|
||||
const lteFailed = durationMatcher.lte !== undefined && elapsed > durationMatcher.lte;
|
||||
const ltFailed = durationMatcher.lt !== undefined && elapsed >= durationMatcher.lt;
|
||||
if (!lteFailed && !ltFailed) return null;
|
||||
|
||||
const durationMs = Math.round(elapsed);
|
||||
const durationResult = checkValueMatcher(durationMs, durationMatcher, {
|
||||
const durationResult = checkValueExpectation(durationMs, durationMatcher, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
@@ -299,7 +304,13 @@ function checkEarlyTimeout(
|
||||
elapsed,
|
||||
failure:
|
||||
durationResult.failure ??
|
||||
mismatchFailure("duration", "durationMs", durationMatcher, durationMs, "durationMs mismatch"),
|
||||
mismatchFailure(
|
||||
"duration",
|
||||
"durationMs",
|
||||
displayValueExpectation(durationMatcher),
|
||||
durationMs,
|
||||
"durationMs mismatch",
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -1,36 +0,0 @@
|
||||
import { isNumber } from "es-toolkit";
|
||||
|
||||
import type { ExpectResult, KeyValueExpect } from "../../expect/types";
|
||||
|
||||
import { mismatchFailure } from "../../expect/failure";
|
||||
import { checkKeyValueExpect } from "../../expect/key-value";
|
||||
|
||||
export function checkHeaders(headers: Record<string, string>, headerExpects?: KeyValueExpect): ExpectResult {
|
||||
return checkKeyValueExpect(headers, headerExpects, {
|
||||
normalizeKey: (key) => key.toLowerCase(),
|
||||
path: "headers",
|
||||
phase: "headers",
|
||||
});
|
||||
}
|
||||
|
||||
export function checkStatus(statusCode: number, allowed: Array<number | string>): ExpectResult {
|
||||
const matched = allowed.some((pattern) => {
|
||||
if (isNumber(pattern)) return statusCode === pattern;
|
||||
const base = parseInt(pattern[0]!, 10) * 100;
|
||||
return statusCode >= base && statusCode < base + 100;
|
||||
});
|
||||
|
||||
if (!matched) {
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
"status",
|
||||
"status",
|
||||
allowed,
|
||||
statusCode,
|
||||
`status ${statusCode} not in [${allowed.join(", ")}]`,
|
||||
),
|
||||
matched: false,
|
||||
};
|
||||
}
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
@@ -3,9 +3,9 @@ import { Type } from "@sinclair/typebox";
|
||||
import type { CheckerSchemas } from "../types";
|
||||
|
||||
import {
|
||||
createContentRulesSchema,
|
||||
createKeyValueExpectSchema,
|
||||
createValueMatcherSchema,
|
||||
createRawContentExpectationsSchema,
|
||||
createRawKeyedExpectationsSchema,
|
||||
createRawValueExpectationSchema,
|
||||
httpMethodSchema,
|
||||
sizeSchema,
|
||||
statusCodePatternSchema,
|
||||
@@ -34,9 +34,9 @@ export const httpCheckerSchemas: CheckerSchemas = {
|
||||
),
|
||||
expect: Type.Object(
|
||||
{
|
||||
body: Type.Optional(createContentRulesSchema()),
|
||||
durationMs: Type.Optional(createValueMatcherSchema()),
|
||||
headers: Type.Optional(createKeyValueExpectSchema()),
|
||||
body: Type.Optional(createRawContentExpectationsSchema()),
|
||||
durationMs: Type.Optional(createRawValueExpectationSchema()),
|
||||
headers: Type.Optional(createRawKeyedExpectationsSchema()),
|
||||
status: Type.Optional(Type.Array(statusCodePatternSchema)),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
|
||||
@@ -1,4 +1,11 @@
|
||||
import type { ContentRules, KeyValueExpect, ValueMatcherInput } from "../../expect/types";
|
||||
import type {
|
||||
ContentExpectations,
|
||||
KeyedExpectations,
|
||||
RawContentExpectations,
|
||||
RawKeyedExpectations,
|
||||
RawValueExpectation,
|
||||
ValueExpectation,
|
||||
} from "../../expect/types";
|
||||
import type { ResolvedTargetBase } from "../../types";
|
||||
|
||||
export interface HttpDefaultsConfig {
|
||||
@@ -7,13 +14,6 @@ export interface HttpDefaultsConfig {
|
||||
method?: string;
|
||||
}
|
||||
|
||||
export interface HttpExpectConfig {
|
||||
body?: ContentRules;
|
||||
durationMs?: ValueMatcherInput;
|
||||
headers?: KeyValueExpect;
|
||||
status?: Array<number | string>;
|
||||
}
|
||||
|
||||
export interface HttpTargetConfig {
|
||||
body?: string;
|
||||
headers?: Record<string, string>;
|
||||
@@ -24,6 +24,13 @@ export interface HttpTargetConfig {
|
||||
url: string;
|
||||
}
|
||||
|
||||
export interface RawHttpExpectConfig {
|
||||
body?: RawContentExpectations;
|
||||
durationMs?: RawValueExpectation;
|
||||
headers?: RawKeyedExpectations;
|
||||
status?: Array<number | string>;
|
||||
}
|
||||
|
||||
export interface ResolvedHttpConfig {
|
||||
body?: string;
|
||||
headers: Record<string, string>;
|
||||
@@ -34,12 +41,20 @@ export interface ResolvedHttpConfig {
|
||||
url: string;
|
||||
}
|
||||
|
||||
export interface ResolvedHttpExpectConfig {
|
||||
body?: ContentExpectations;
|
||||
durationMs?: ValueExpectation;
|
||||
headers?: KeyedExpectations;
|
||||
status: Array<number | string>;
|
||||
}
|
||||
|
||||
export interface ResolvedHttpTarget extends ResolvedTargetBase {
|
||||
expect?: HttpExpectConfig;
|
||||
expect?: ResolvedHttpExpectConfig;
|
||||
group: string;
|
||||
http: ResolvedHttpConfig;
|
||||
intervalMs: number;
|
||||
name: null | string;
|
||||
rawExpect?: RawHttpExpectConfig;
|
||||
timeoutMs: number;
|
||||
type: "http";
|
||||
}
|
||||
|
||||
@@ -3,13 +3,12 @@ import { isNumber, isString } from "es-toolkit";
|
||||
import type { ConfigValidationIssue } from "../../schema/issues";
|
||||
import type { CheckerValidationInput } from "../types";
|
||||
|
||||
import { normalizeExpectMatchers } from "../../expect/normalize";
|
||||
import {
|
||||
isPlainRecord,
|
||||
validateContentRules,
|
||||
validateKeyValueExpect,
|
||||
validateValueMatcher,
|
||||
} from "../../expect/validate-matcher";
|
||||
validateRawContentExpectations,
|
||||
validateRawKeyedExpectations,
|
||||
validateRawValueExpectation,
|
||||
} from "../../expect/validate";
|
||||
import { issue, joinPath } from "../../schema/issues";
|
||||
import { parseSize } from "../../utils";
|
||||
|
||||
@@ -34,24 +33,6 @@ export function validateHttpConfig(input: CheckerValidationInput): ConfigValidat
|
||||
return issues;
|
||||
}
|
||||
|
||||
export function validateJsonPath(path: string, rulePath: string, targetName?: string): ConfigValidationIssue[] {
|
||||
if (!path.startsWith("$.") || path.length <= 2) {
|
||||
return [issue("invalid-jsonpath", joinPath(rulePath, "path"), '必须为以 "$." 开头的有效 JSONPath', targetName)];
|
||||
}
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const segments = path.slice(2).split(".");
|
||||
for (const seg of segments) {
|
||||
if (seg === "") {
|
||||
issues.push(issue("invalid-jsonpath", joinPath(rulePath, "path"), "包含空段", targetName));
|
||||
}
|
||||
const bracketMatch = /^(.+?)\[(\d+)\]$/.exec(seg);
|
||||
if (bracketMatch?.[1]!.trim() === "") {
|
||||
issues.push(issue("invalid-jsonpath", joinPath(rulePath, "path"), "数组访问缺少属性名", targetName));
|
||||
}
|
||||
}
|
||||
return issues;
|
||||
}
|
||||
|
||||
function getTargetName(target: Record<string, unknown>): string | undefined {
|
||||
if (isString(target["name"])) return target["name"];
|
||||
return isString(target["id"]) ? target["id"] : undefined;
|
||||
@@ -68,14 +49,16 @@ function validateHttpExpect(target: Record<string, unknown>, path: string): Conf
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const expectPath = joinPath(path, "expect");
|
||||
|
||||
normalizeExpectMatchers(expect, ["durationMs"]);
|
||||
|
||||
if (isPlainRecord(expect["headers"])) {
|
||||
issues.push(...validateKeyValueExpect(expect["headers"], joinPath(expectPath, "headers"), targetName));
|
||||
issues.push(
|
||||
...validateRawKeyedExpectations(expect["headers"], joinPath(expectPath, "headers"), targetName, {
|
||||
caseInsensitive: true,
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
if (expect["body"] !== undefined) {
|
||||
issues.push(...validateContentRules(expect["body"], joinPath(expectPath, "body"), targetName));
|
||||
issues.push(...validateRawContentExpectations(expect["body"], joinPath(expectPath, "body"), targetName));
|
||||
}
|
||||
|
||||
if (Array.isArray(expect["status"])) {
|
||||
@@ -83,7 +66,7 @@ function validateHttpExpect(target: Record<string, unknown>, path: string): Conf
|
||||
}
|
||||
|
||||
if (expect["durationMs"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
issues.push(...validateRawValueExpectation(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
}
|
||||
|
||||
return issues;
|
||||
|
||||
@@ -2,10 +2,16 @@ import { isError } from "es-toolkit";
|
||||
|
||||
import type { CheckResult, RawTargetConfig } from "../../types";
|
||||
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
|
||||
import type { PingExpectConfig, PingStats, PingTargetConfig, ResolvedPingTarget } from "./types";
|
||||
import type {
|
||||
PingStats,
|
||||
PingTargetConfig,
|
||||
RawIcmpExpectConfig,
|
||||
ResolvedIcmpExpectConfig,
|
||||
ResolvedPingTarget,
|
||||
} from "./types";
|
||||
|
||||
import { errorFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { checkValueExpectation, resolveValueExpectation } from "../../expect/value";
|
||||
import { buildPingCommand } from "./command";
|
||||
import { checkAlive, checkAvgLatency, checkMaxLatency, checkPacketLoss } from "./expect";
|
||||
import { parsePingOutput } from "./parse";
|
||||
@@ -155,9 +161,21 @@ export class IcmpChecker implements CheckerDefinition<ResolvedPingTarget> {
|
||||
|
||||
resolve(target: RawTargetConfig, context: ResolveContext): ResolvedPingTarget {
|
||||
const t = target as RawTargetConfig & { icmp: PingTargetConfig; type: "icmp" };
|
||||
|
||||
const rawExpect = target.expect as RawIcmpExpectConfig | undefined;
|
||||
const resolvedExpect: ResolvedIcmpExpectConfig = rawExpect
|
||||
? {
|
||||
alive: rawExpect.alive ?? true,
|
||||
avgLatencyMs: resolveValueExpectation(rawExpect.avgLatencyMs),
|
||||
durationMs: resolveValueExpectation(rawExpect.durationMs),
|
||||
maxLatencyMs: resolveValueExpectation(rawExpect.maxLatencyMs),
|
||||
packetLossPercent: resolveValueExpectation(rawExpect.packetLossPercent),
|
||||
}
|
||||
: { alive: true };
|
||||
|
||||
return {
|
||||
description: null,
|
||||
expect: target.expect as PingExpectConfig | undefined,
|
||||
expect: resolvedExpect,
|
||||
group: target.group ?? "default",
|
||||
icmp: {
|
||||
count: t.icmp.count ?? DEFAULT_COUNT,
|
||||
@@ -167,6 +185,7 @@ export class IcmpChecker implements CheckerDefinition<ResolvedPingTarget> {
|
||||
id: t.id,
|
||||
intervalMs: context.defaultIntervalMs,
|
||||
name: t.name ?? null,
|
||||
rawExpect,
|
||||
timeoutMs: context.defaultTimeoutMs,
|
||||
type: "icmp",
|
||||
} satisfies ResolvedPingTarget;
|
||||
@@ -184,7 +203,7 @@ export class IcmpChecker implements CheckerDefinition<ResolvedPingTarget> {
|
||||
}
|
||||
}
|
||||
|
||||
function checkStats(stats: PingStats, expect: PingExpectConfig | undefined, durationMs: number) {
|
||||
function checkStats(stats: PingStats, expect: ResolvedIcmpExpectConfig | undefined, durationMs: number) {
|
||||
const aliveResult = checkAlive(stats.alive, expect?.alive ?? true);
|
||||
if (!aliveResult.matched) return aliveResult;
|
||||
const packetLossResult = checkPacketLoss(stats.packetLoss, expect?.packetLossPercent);
|
||||
@@ -193,7 +212,7 @@ function checkStats(stats: PingStats, expect: PingExpectConfig | undefined, dura
|
||||
if (!avgLatencyResult.matched) return avgLatencyResult;
|
||||
const maxLatencyResult = checkMaxLatency(stats.maxLatencyMs, expect?.maxLatencyMs);
|
||||
if (!maxLatencyResult.matched) return maxLatencyResult;
|
||||
return checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
return checkValueExpectation(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
import type { ExpectResult, ValueMatcherInput } from "../../expect/types";
|
||||
import type { ExpectationResult, ValueExpectation } from "../../expect/types";
|
||||
|
||||
import { mismatchFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { checkValueExpectation } from "../../expect/value";
|
||||
|
||||
export function checkAlive(actual: boolean, expected: boolean): ExpectResult {
|
||||
export function checkAlive(actual: boolean, expected: boolean): ExpectationResult {
|
||||
if (actual === expected) return { failure: null, matched: true };
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
@@ -17,24 +17,24 @@ export function checkAlive(actual: boolean, expected: boolean): ExpectResult {
|
||||
};
|
||||
}
|
||||
|
||||
export function checkAvgLatency(actual: null | number, matcher: undefined | ValueMatcherInput): ExpectResult {
|
||||
return checkValueMatcher(actual, matcher, {
|
||||
export function checkAvgLatency(actual: null | number, matcher: undefined | ValueExpectation): ExpectationResult {
|
||||
return checkValueExpectation(actual, matcher, {
|
||||
message: "平均延迟不满足条件",
|
||||
path: "avgLatencyMs",
|
||||
phase: "avgLatency",
|
||||
});
|
||||
}
|
||||
|
||||
export function checkMaxLatency(actual: null | number, matcher: undefined | ValueMatcherInput): ExpectResult {
|
||||
return checkValueMatcher(actual, matcher, {
|
||||
export function checkMaxLatency(actual: null | number, matcher: undefined | ValueExpectation): ExpectationResult {
|
||||
return checkValueExpectation(actual, matcher, {
|
||||
message: "最大延迟不满足条件",
|
||||
path: "maxLatencyMs",
|
||||
phase: "maxLatency",
|
||||
});
|
||||
}
|
||||
|
||||
export function checkPacketLoss(actual: number, matcher: undefined | ValueMatcherInput): ExpectResult {
|
||||
return checkValueMatcher(actual, matcher, {
|
||||
export function checkPacketLoss(actual: number, matcher: undefined | ValueExpectation): ExpectationResult {
|
||||
return checkValueExpectation(actual, matcher, {
|
||||
message: "丢包率不满足条件",
|
||||
path: "packetLossPercent",
|
||||
phase: "packetLoss",
|
||||
|
||||
@@ -2,7 +2,7 @@ import { Type } from "@sinclair/typebox";
|
||||
|
||||
import type { CheckerSchemas } from "../types";
|
||||
|
||||
import { createValueMatcherSchema } from "../../schema/fragments";
|
||||
import { createRawValueExpectationSchema } from "../../schema/fragments";
|
||||
|
||||
export const icmpCheckerSchemas: CheckerSchemas = {
|
||||
config: Type.Object(
|
||||
@@ -17,10 +17,10 @@ export const icmpCheckerSchemas: CheckerSchemas = {
|
||||
expect: Type.Object(
|
||||
{
|
||||
alive: Type.Optional(Type.Boolean()),
|
||||
avgLatencyMs: Type.Optional(createValueMatcherSchema()),
|
||||
durationMs: Type.Optional(createValueMatcherSchema()),
|
||||
maxLatencyMs: Type.Optional(createValueMatcherSchema()),
|
||||
packetLossPercent: Type.Optional(createValueMatcherSchema()),
|
||||
avgLatencyMs: Type.Optional(createRawValueExpectationSchema()),
|
||||
durationMs: Type.Optional(createRawValueExpectationSchema()),
|
||||
maxLatencyMs: Type.Optional(createRawValueExpectationSchema()),
|
||||
packetLossPercent: Type.Optional(createRawValueExpectationSchema()),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
|
||||
@@ -1,14 +1,6 @@
|
||||
import type { ValueMatcherInput } from "../../expect/types";
|
||||
import type { RawValueExpectation, ValueExpectation } from "../../expect/types";
|
||||
import type { ResolvedTargetBase } from "../../types";
|
||||
|
||||
export interface PingExpectConfig {
|
||||
alive?: boolean;
|
||||
avgLatencyMs?: ValueMatcherInput;
|
||||
durationMs?: ValueMatcherInput;
|
||||
maxLatencyMs?: ValueMatcherInput;
|
||||
packetLossPercent?: ValueMatcherInput;
|
||||
}
|
||||
|
||||
export interface PingStats {
|
||||
alive: boolean;
|
||||
avgLatencyMs: null | number;
|
||||
@@ -25,6 +17,22 @@ export interface PingTargetConfig {
|
||||
packetSize?: number;
|
||||
}
|
||||
|
||||
export interface RawIcmpExpectConfig {
|
||||
alive?: boolean;
|
||||
avgLatencyMs?: RawValueExpectation;
|
||||
durationMs?: RawValueExpectation;
|
||||
maxLatencyMs?: RawValueExpectation;
|
||||
packetLossPercent?: RawValueExpectation;
|
||||
}
|
||||
|
||||
export interface ResolvedIcmpExpectConfig {
|
||||
alive: boolean;
|
||||
avgLatencyMs?: ValueExpectation;
|
||||
durationMs?: ValueExpectation;
|
||||
maxLatencyMs?: ValueExpectation;
|
||||
packetLossPercent?: ValueExpectation;
|
||||
}
|
||||
|
||||
export interface ResolvedPingConfig {
|
||||
count: number;
|
||||
host: string;
|
||||
@@ -32,11 +40,12 @@ export interface ResolvedPingConfig {
|
||||
}
|
||||
|
||||
export interface ResolvedPingTarget extends ResolvedTargetBase {
|
||||
expect?: PingExpectConfig;
|
||||
expect?: ResolvedIcmpExpectConfig;
|
||||
group: string;
|
||||
icmp: ResolvedPingConfig;
|
||||
intervalMs: number;
|
||||
name: null | string;
|
||||
rawExpect?: RawIcmpExpectConfig;
|
||||
timeoutMs: number;
|
||||
type: "icmp";
|
||||
}
|
||||
|
||||
@@ -1,10 +1,9 @@
|
||||
import { isNumber, isPlainObject, isString } from "es-toolkit";
|
||||
import { isNumber, isString } from "es-toolkit";
|
||||
|
||||
import type { ConfigValidationIssue } from "../../schema/issues";
|
||||
import type { CheckerValidationInput } from "../types";
|
||||
|
||||
import { normalizeExpectMatchers } from "../../expect/normalize";
|
||||
import { validateValueMatcher } from "../../expect/validate-matcher";
|
||||
import { isPlainRecord, validateRawValueExpectation } from "../../expect/validate";
|
||||
import { issue, joinPath } from "../../schema/issues";
|
||||
|
||||
export function validatePingConfig(input: CheckerValidationInput): ConfigValidationIssue[] {
|
||||
@@ -13,10 +12,10 @@ export function validatePingConfig(input: CheckerValidationInput): ConfigValidat
|
||||
const defaults = input.defaults["icmp"];
|
||||
if (defaults !== undefined && defaults !== null) {
|
||||
const targetName = "defaults.icmp";
|
||||
if (!isPlainObject(defaults)) {
|
||||
if (!isPlainRecord(defaults)) {
|
||||
issues.push(issue("invalid-type", "defaults.icmp", "必须为对象", targetName));
|
||||
} else {
|
||||
const icmpDefaults = defaults as Record<string, unknown>;
|
||||
const icmpDefaults = defaults;
|
||||
for (const key of Object.keys(icmpDefaults)) {
|
||||
issues.push(issue("unknown-field", joinPath("defaults.icmp", key), "是未知字段", targetName));
|
||||
}
|
||||
@@ -25,8 +24,8 @@ export function validatePingConfig(input: CheckerValidationInput): ConfigValidat
|
||||
|
||||
for (let i = 0; i < input.targets.length; i++) {
|
||||
const target = input.targets[i] as unknown;
|
||||
if (!isPlainObject(target)) continue;
|
||||
const targetRecord = target as Record<string, unknown>;
|
||||
if (!isPlainRecord(target)) continue;
|
||||
const targetRecord = target;
|
||||
if (targetRecord["type"] !== "icmp") continue;
|
||||
issues.push(...validatePingTarget(targetRecord, `targets[${i}]`));
|
||||
}
|
||||
@@ -41,20 +40,18 @@ function getTargetName(target: Record<string, unknown>): string | undefined {
|
||||
|
||||
function validatePingExpect(target: Record<string, unknown>, path: string): ConfigValidationIssue[] {
|
||||
const rawExpect = target["expect"];
|
||||
if (rawExpect === undefined || rawExpect === null || !isPlainObject(rawExpect)) return [];
|
||||
const expect = rawExpect as Record<string, unknown>;
|
||||
if (rawExpect === undefined || rawExpect === null || !isPlainRecord(rawExpect)) return [];
|
||||
const expect = rawExpect;
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const targetName = getTargetName(target);
|
||||
const expectPath = joinPath(path, "expect");
|
||||
|
||||
normalizeExpectMatchers(expect, ["packetLossPercent", "avgLatencyMs", "maxLatencyMs", "durationMs"]);
|
||||
|
||||
if (expect["alive"] !== undefined && typeof expect["alive"] !== "boolean") {
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, "alive"), "必须为布尔值", targetName));
|
||||
}
|
||||
for (const key of ["packetLossPercent", "avgLatencyMs", "maxLatencyMs", "durationMs"]) {
|
||||
if (expect[key] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect[key], joinPath(expectPath, key), targetName));
|
||||
issues.push(...validateRawValueExpectation(expect[key], joinPath(expectPath, key), targetName));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -73,12 +70,12 @@ function validatePingTarget(target: Record<string, unknown>, path: string): Conf
|
||||
const targetName = getTargetName(target);
|
||||
const rawIcmp = target["icmp"];
|
||||
|
||||
if (!isPlainObject(rawIcmp)) {
|
||||
if (!isPlainRecord(rawIcmp)) {
|
||||
issues.push(issue("required", joinPath(path, "icmp"), "缺少 icmp 配置分组", targetName));
|
||||
issues.push(...validatePingExpect(target, path));
|
||||
return issues;
|
||||
}
|
||||
const icmp = rawIcmp as Record<string, unknown>;
|
||||
const icmp = rawIcmp;
|
||||
|
||||
if (!isString(icmp["host"]) || icmp["host"].trim() === "") {
|
||||
issues.push(issue("required", joinPath(joinPath(path, "icmp"), "host"), "缺少 icmp.host 字段", targetName));
|
||||
|
||||
@@ -5,10 +5,12 @@ import { isError } from "es-toolkit";
|
||||
|
||||
import type { CheckResult, RawTargetConfig } from "../../types";
|
||||
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
|
||||
import type { LlmExpectConfig, LlmTargetConfig, ResolvedLlmTarget } from "./types";
|
||||
import type { LlmTargetConfig, RawLlmExpectConfig, ResolvedLlmExpectConfig, ResolvedLlmTarget } from "./types";
|
||||
|
||||
import { resolveContentExpectations } from "../../expect/content";
|
||||
import { errorFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { resolveKeyedExpectations } from "../../expect/keyed";
|
||||
import { checkValueExpectation, resolveValueExpectation } from "../../expect/value";
|
||||
import { runExpects } from "./expect";
|
||||
import {
|
||||
buildObservationFromApiCallError,
|
||||
@@ -93,7 +95,7 @@ export class LlmChecker implements CheckerDefinition<ResolvedLlmTarget> {
|
||||
};
|
||||
}
|
||||
|
||||
const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
const durationResult = checkValueExpectation(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
@@ -171,14 +173,40 @@ export class LlmChecker implements CheckerDefinition<ResolvedLlmTarget> {
|
||||
url: t.llm.url,
|
||||
};
|
||||
|
||||
const rawExpect = target.expect as RawLlmExpectConfig | undefined;
|
||||
const resolvedExpect: ResolvedLlmExpectConfig = rawExpect
|
||||
? {
|
||||
durationMs: resolveValueExpectation(rawExpect.durationMs),
|
||||
finishReason: resolveValueExpectation(rawExpect.finishReason),
|
||||
headers: resolveKeyedExpectations(rawExpect.headers),
|
||||
output: resolveContentExpectations(rawExpect.output),
|
||||
rawFinishReason: resolveValueExpectation(rawExpect.rawFinishReason),
|
||||
status: rawExpect.status ?? [200],
|
||||
stream: rawExpect.stream
|
||||
? {
|
||||
completed: rawExpect.stream.completed ?? true,
|
||||
firstTokenMs: resolveValueExpectation(rawExpect.stream.firstTokenMs),
|
||||
}
|
||||
: undefined,
|
||||
usage: rawExpect.usage
|
||||
? {
|
||||
inputTokens: resolveValueExpectation(rawExpect.usage.inputTokens),
|
||||
outputTokens: resolveValueExpectation(rawExpect.usage.outputTokens),
|
||||
totalTokens: resolveValueExpectation(rawExpect.usage.totalTokens),
|
||||
}
|
||||
: undefined,
|
||||
}
|
||||
: { status: [200] };
|
||||
|
||||
return {
|
||||
description: (target.description as null | string) ?? null,
|
||||
expect: target.expect as LlmExpectConfig | undefined,
|
||||
expect: resolvedExpect,
|
||||
group: target.group ?? "default",
|
||||
id: t.id,
|
||||
intervalMs: context.defaultIntervalMs,
|
||||
llm: resolvedConfig,
|
||||
name: (target.name as null | string) ?? null,
|
||||
rawExpect,
|
||||
timeoutMs: context.defaultTimeoutMs,
|
||||
type: "llm",
|
||||
} satisfies ResolvedLlmTarget;
|
||||
@@ -210,7 +238,7 @@ export class LlmChecker implements CheckerDefinition<ResolvedLlmTarget> {
|
||||
t: ResolvedLlmTarget,
|
||||
model: ReturnType<typeof createProviderModel>["model"],
|
||||
httpMeta: null | { headers: Record<string, string>; status: number; statusText: string },
|
||||
expect: LlmExpectConfig | undefined,
|
||||
expect: ResolvedLlmExpectConfig | undefined,
|
||||
ctx: CheckerContext,
|
||||
timestamp: string,
|
||||
start: number,
|
||||
@@ -254,7 +282,7 @@ export class LlmChecker implements CheckerDefinition<ResolvedLlmTarget> {
|
||||
);
|
||||
|
||||
const durationMs = Math.round(performance.now() - start);
|
||||
const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
const durationResult = checkValueExpectation(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
@@ -277,7 +305,7 @@ export class LlmChecker implements CheckerDefinition<ResolvedLlmTarget> {
|
||||
t: ResolvedLlmTarget,
|
||||
model: ReturnType<typeof createProviderModel>["model"],
|
||||
httpMeta: null | { headers: Record<string, string>; status: number; statusText: string },
|
||||
expect: LlmExpectConfig | undefined,
|
||||
expect: ResolvedLlmExpectConfig | undefined,
|
||||
ctx: CheckerContext,
|
||||
timestamp: string,
|
||||
start: number,
|
||||
@@ -301,7 +329,7 @@ export class LlmChecker implements CheckerDefinition<ResolvedLlmTarget> {
|
||||
);
|
||||
|
||||
const durationMs = Math.round(performance.now() - start);
|
||||
const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
const durationResult = checkValueExpectation(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
|
||||
@@ -1,15 +1,19 @@
|
||||
import type { ExpectResult } from "../../expect/types";
|
||||
import type { LlmCheckObservation, LlmExpectConfig, LlmUsageExpect } from "./types";
|
||||
import type { ExpectationResult } from "../../expect/types";
|
||||
import type { LlmCheckObservation, ResolvedLlmExpectConfig, ResolvedLlmUsageExpect } from "./types";
|
||||
|
||||
import { checkContentRules } from "../../expect/content";
|
||||
import { checkContentExpectations } from "../../expect/content";
|
||||
import { mismatchFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { checkHeaders, checkStatus } from "../http/expect";
|
||||
import { checkHeaderExpectations } from "../../expect/headers";
|
||||
import { checkStatusCode } from "../../expect/status";
|
||||
import { checkValueExpectation } from "../../expect/value";
|
||||
|
||||
export function checkStreamExpect(observation: LlmCheckObservation, expect: LlmExpectConfig): ExpectResult {
|
||||
export function checkStreamExpect(
|
||||
observation: LlmCheckObservation,
|
||||
expect: ResolvedLlmExpectConfig,
|
||||
): ExpectationResult {
|
||||
if (!observation.stream || !expect.stream) return { failure: null, matched: true };
|
||||
|
||||
const expectedCompleted = expect.stream.completed ?? true;
|
||||
const expectedCompleted = expect.stream.completed;
|
||||
if (observation.stream.completed !== expectedCompleted) {
|
||||
return {
|
||||
failure: mismatchFailure(
|
||||
@@ -24,7 +28,7 @@ export function checkStreamExpect(observation: LlmCheckObservation, expect: LlmE
|
||||
}
|
||||
|
||||
if (expect.stream.firstTokenMs && observation.stream.firstTokenMs !== null) {
|
||||
return checkValueMatcher(observation.stream.firstTokenMs, expect.stream.firstTokenMs, {
|
||||
return checkValueExpectation(observation.stream.firstTokenMs, expect.stream.firstTokenMs, {
|
||||
message: "stream.firstTokenMs mismatch",
|
||||
path: "stream.firstTokenMs",
|
||||
phase: "stream",
|
||||
@@ -45,20 +49,23 @@ export function checkStreamExpect(observation: LlmCheckObservation, expect: LlmE
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
export function runExpects(observation: LlmCheckObservation, expect: LlmExpectConfig | undefined): ExpectResult {
|
||||
export function runExpects(
|
||||
observation: LlmCheckObservation,
|
||||
expect: ResolvedLlmExpectConfig | undefined,
|
||||
): ExpectationResult {
|
||||
if (!expect) {
|
||||
const defaultStatus = checkStatus(observation.http?.status ?? 0, [200]);
|
||||
const defaultStatus = checkStatusCode(observation.http?.status ?? 0, [200]);
|
||||
if (!defaultStatus.matched) return defaultStatus;
|
||||
return { failure: null, matched: true };
|
||||
}
|
||||
|
||||
const http = observation.http;
|
||||
|
||||
const statusResult = checkStatus(http?.status ?? 0, expect.status ?? [200]);
|
||||
const statusResult = checkStatusCode(http?.status ?? 0, expect.status);
|
||||
if (!statusResult.matched) return statusResult;
|
||||
|
||||
if (http && expect.headers) {
|
||||
const headersResult = checkHeaders(http.headers, expect.headers);
|
||||
const headersResult = checkHeaderExpectations(http.headers, expect.headers);
|
||||
if (!headersResult.matched) return headersResult;
|
||||
}
|
||||
|
||||
@@ -67,11 +74,14 @@ export function runExpects(observation: LlmCheckObservation, expect: LlmExpectCo
|
||||
if (!streamResult.matched) return streamResult;
|
||||
}
|
||||
|
||||
const outputResult = checkContentRules(observation.outputText, expect.output, { path: "output", phase: "output" });
|
||||
const outputResult = checkContentExpectations(observation.outputText, expect.output, {
|
||||
path: "output",
|
||||
phase: "output",
|
||||
});
|
||||
if (!outputResult.matched) return outputResult;
|
||||
|
||||
if (expect.finishReason !== undefined) {
|
||||
const result = checkValueMatcher(observation.finishReason, expect.finishReason, {
|
||||
const result = checkValueExpectation(observation.finishReason, expect.finishReason, {
|
||||
message: "finishReason mismatch",
|
||||
path: "finishReason",
|
||||
phase: "finishReason",
|
||||
@@ -80,7 +90,7 @@ export function runExpects(observation: LlmCheckObservation, expect: LlmExpectCo
|
||||
}
|
||||
|
||||
if (expect.rawFinishReason !== undefined) {
|
||||
const result = checkValueMatcher(observation.rawFinishReason, expect.rawFinishReason, {
|
||||
const result = checkValueExpectation(observation.rawFinishReason, expect.rawFinishReason, {
|
||||
message: "rawFinishReason mismatch",
|
||||
path: "rawFinishReason",
|
||||
phase: "rawFinishReason",
|
||||
@@ -98,10 +108,10 @@ export function runExpects(observation: LlmCheckObservation, expect: LlmExpectCo
|
||||
|
||||
function checkUsageExpect(
|
||||
usage: { inputTokens: number; outputTokens: number; totalTokens: number },
|
||||
expectUsage: LlmUsageExpect,
|
||||
): ExpectResult {
|
||||
expectUsage: ResolvedLlmUsageExpect,
|
||||
): ExpectationResult {
|
||||
if (expectUsage.inputTokens !== undefined) {
|
||||
const result = checkValueMatcher(usage.inputTokens, expectUsage.inputTokens, {
|
||||
const result = checkValueExpectation(usage.inputTokens, expectUsage.inputTokens, {
|
||||
message: "usage.inputTokens mismatch",
|
||||
path: "usage.inputTokens",
|
||||
phase: "usage",
|
||||
@@ -109,7 +119,7 @@ function checkUsageExpect(
|
||||
if (!result.matched) return result;
|
||||
}
|
||||
if (expectUsage.outputTokens !== undefined) {
|
||||
const result = checkValueMatcher(usage.outputTokens, expectUsage.outputTokens, {
|
||||
const result = checkValueExpectation(usage.outputTokens, expectUsage.outputTokens, {
|
||||
message: "usage.outputTokens mismatch",
|
||||
path: "usage.outputTokens",
|
||||
phase: "usage",
|
||||
@@ -117,7 +127,7 @@ function checkUsageExpect(
|
||||
if (!result.matched) return result;
|
||||
}
|
||||
if (expectUsage.totalTokens !== undefined) {
|
||||
const result = checkValueMatcher(usage.totalTokens, expectUsage.totalTokens, {
|
||||
const result = checkValueExpectation(usage.totalTokens, expectUsage.totalTokens, {
|
||||
message: "usage.totalTokens mismatch",
|
||||
path: "usage.totalTokens",
|
||||
phase: "usage",
|
||||
|
||||
@@ -3,9 +3,9 @@ import { Type } from "@sinclair/typebox";
|
||||
import type { CheckerSchemas } from "../types";
|
||||
|
||||
import {
|
||||
createContentRulesSchema,
|
||||
createKeyValueExpectSchema,
|
||||
createValueMatcherSchema,
|
||||
createRawContentExpectationsSchema,
|
||||
createRawKeyedExpectationsSchema,
|
||||
createRawValueExpectationSchema,
|
||||
statusCodePatternSchema,
|
||||
stringMapSchema,
|
||||
} from "../../schema/fragments";
|
||||
@@ -55,17 +55,17 @@ export const llmCheckerSchemas: CheckerSchemas = {
|
||||
),
|
||||
expect: Type.Object(
|
||||
{
|
||||
durationMs: Type.Optional(createValueMatcherSchema()),
|
||||
finishReason: Type.Optional(createValueMatcherSchema()),
|
||||
headers: Type.Optional(createKeyValueExpectSchema()),
|
||||
output: Type.Optional(createContentRulesSchema()),
|
||||
rawFinishReason: Type.Optional(createValueMatcherSchema()),
|
||||
durationMs: Type.Optional(createRawValueExpectationSchema()),
|
||||
finishReason: Type.Optional(createRawValueExpectationSchema()),
|
||||
headers: Type.Optional(createRawKeyedExpectationsSchema()),
|
||||
output: Type.Optional(createRawContentExpectationsSchema()),
|
||||
rawFinishReason: Type.Optional(createRawValueExpectationSchema()),
|
||||
status: Type.Optional(Type.Array(statusCodePatternSchema)),
|
||||
stream: Type.Optional(
|
||||
Type.Object(
|
||||
{
|
||||
completed: Type.Optional(Type.Boolean()),
|
||||
firstTokenMs: Type.Optional(createValueMatcherSchema()),
|
||||
firstTokenMs: Type.Optional(createRawValueExpectationSchema()),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
@@ -73,9 +73,9 @@ export const llmCheckerSchemas: CheckerSchemas = {
|
||||
usage: Type.Optional(
|
||||
Type.Object(
|
||||
{
|
||||
inputTokens: Type.Optional(createValueMatcherSchema()),
|
||||
outputTokens: Type.Optional(createValueMatcherSchema()),
|
||||
totalTokens: Type.Optional(createValueMatcherSchema()),
|
||||
inputTokens: Type.Optional(createRawValueExpectationSchema()),
|
||||
outputTokens: Type.Optional(createRawValueExpectationSchema()),
|
||||
totalTokens: Type.Optional(createRawValueExpectationSchema()),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
|
||||
@@ -1,6 +1,13 @@
|
||||
import type { JSONObject } from "@ai-sdk/provider";
|
||||
|
||||
import type { ContentRules, KeyValueExpect, ValueMatcherInput } from "../../expect/types";
|
||||
import type {
|
||||
ContentExpectations,
|
||||
KeyedExpectations,
|
||||
RawContentExpectations,
|
||||
RawKeyedExpectations,
|
||||
RawValueExpectation,
|
||||
ValueExpectation,
|
||||
} from "../../expect/types";
|
||||
import type { ResolvedTargetBase } from "../../types";
|
||||
|
||||
export interface LlmCheckObservation {
|
||||
@@ -24,16 +31,6 @@ export interface LlmDefaultsConfig {
|
||||
providerOptions?: Record<string, JSONObject>;
|
||||
}
|
||||
|
||||
export interface LlmExpectConfig {
|
||||
durationMs?: ValueMatcherInput;
|
||||
finishReason?: ValueMatcherInput;
|
||||
headers?: KeyValueExpect;
|
||||
output?: ContentRules;
|
||||
rawFinishReason?: ValueMatcherInput;
|
||||
status?: Array<number | string>;
|
||||
stream?: LlmStreamExpect;
|
||||
usage?: LlmUsageExpect;
|
||||
}
|
||||
export interface LlmHttpMetadata {
|
||||
headers: Record<string, string>;
|
||||
status: number;
|
||||
@@ -80,11 +77,6 @@ export interface LlmOptions {
|
||||
|
||||
export type LlmProvider = "anthropic" | "openai" | "openai-responses";
|
||||
|
||||
export interface LlmStreamExpect {
|
||||
completed?: boolean;
|
||||
firstTokenMs?: ValueMatcherInput;
|
||||
}
|
||||
|
||||
export interface LlmStreamObservation {
|
||||
completed: boolean;
|
||||
firstTokenMs: null | number;
|
||||
@@ -104,18 +96,34 @@ export interface LlmTargetConfig {
|
||||
url: string;
|
||||
}
|
||||
|
||||
export interface LlmUsageExpect {
|
||||
inputTokens?: ValueMatcherInput;
|
||||
outputTokens?: ValueMatcherInput;
|
||||
totalTokens?: ValueMatcherInput;
|
||||
}
|
||||
|
||||
export interface LlmUsageObservation {
|
||||
inputTokens: number;
|
||||
outputTokens: number;
|
||||
totalTokens: number;
|
||||
}
|
||||
|
||||
export interface RawLlmExpectConfig {
|
||||
durationMs?: RawValueExpectation;
|
||||
finishReason?: RawValueExpectation;
|
||||
headers?: RawKeyedExpectations;
|
||||
output?: RawContentExpectations;
|
||||
rawFinishReason?: RawValueExpectation;
|
||||
status?: Array<number | string>;
|
||||
stream?: RawLlmStreamExpect;
|
||||
usage?: RawLlmUsageExpect;
|
||||
}
|
||||
|
||||
export interface RawLlmStreamExpect {
|
||||
completed?: boolean;
|
||||
firstTokenMs?: RawValueExpectation;
|
||||
}
|
||||
|
||||
export interface RawLlmUsageExpect {
|
||||
inputTokens?: RawValueExpectation;
|
||||
outputTokens?: RawValueExpectation;
|
||||
totalTokens?: RawValueExpectation;
|
||||
}
|
||||
|
||||
export interface ResolvedLlmConfig {
|
||||
authToken?: string;
|
||||
headers: Record<string, string>;
|
||||
@@ -130,12 +138,35 @@ export interface ResolvedLlmConfig {
|
||||
url: string;
|
||||
}
|
||||
|
||||
export interface ResolvedLlmExpectConfig {
|
||||
durationMs?: ValueExpectation;
|
||||
finishReason?: ValueExpectation;
|
||||
headers?: KeyedExpectations;
|
||||
output?: ContentExpectations;
|
||||
rawFinishReason?: ValueExpectation;
|
||||
status: Array<number | string>;
|
||||
stream?: ResolvedLlmStreamExpect;
|
||||
usage?: ResolvedLlmUsageExpect;
|
||||
}
|
||||
|
||||
export interface ResolvedLlmStreamExpect {
|
||||
completed: boolean;
|
||||
firstTokenMs?: ValueExpectation;
|
||||
}
|
||||
|
||||
export interface ResolvedLlmTarget extends ResolvedTargetBase {
|
||||
expect?: LlmExpectConfig;
|
||||
expect?: ResolvedLlmExpectConfig;
|
||||
group: string;
|
||||
intervalMs: number;
|
||||
llm: ResolvedLlmConfig;
|
||||
name: null | string;
|
||||
rawExpect?: RawLlmExpectConfig;
|
||||
timeoutMs: number;
|
||||
type: "llm";
|
||||
}
|
||||
|
||||
export interface ResolvedLlmUsageExpect {
|
||||
inputTokens?: ValueExpectation;
|
||||
outputTokens?: ValueExpectation;
|
||||
totalTokens?: ValueExpectation;
|
||||
}
|
||||
|
||||
@@ -3,13 +3,12 @@ import { isBoolean, isNumber, isString } from "es-toolkit";
|
||||
import type { ConfigValidationIssue } from "../../schema/issues";
|
||||
import type { CheckerValidationInput } from "../types";
|
||||
|
||||
import { normalizeExpectMatchers } from "../../expect/normalize";
|
||||
import {
|
||||
isPlainRecord,
|
||||
validateContentRules,
|
||||
validateKeyValueExpect,
|
||||
validateValueMatcher,
|
||||
} from "../../expect/validate-matcher";
|
||||
validateRawContentExpectations,
|
||||
validateRawKeyedExpectations,
|
||||
validateRawValueExpectation,
|
||||
} from "../../expect/validate";
|
||||
import { issue, joinPath } from "../../schema/issues";
|
||||
|
||||
const ALLOWED_MODES = new Set(["http", "stream"]);
|
||||
@@ -73,23 +72,27 @@ function validateLlmExpect(
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const expectPath = joinPath(path, "expect");
|
||||
|
||||
normalizeExpectMatchers(expect, ["durationMs", "finishReason", "rawFinishReason"]);
|
||||
|
||||
if (Array.isArray(expect["status"])) {
|
||||
issues.push(...validateStatusValues(expect["status"], joinPath(expectPath, "status"), targetName));
|
||||
}
|
||||
if (expect["headers"] !== undefined) {
|
||||
issues.push(...validateKeyValueExpect(expect["headers"], joinPath(expectPath, "headers"), targetName));
|
||||
issues.push(
|
||||
...validateRawKeyedExpectations(expect["headers"], joinPath(expectPath, "headers"), targetName, {
|
||||
caseInsensitive: true,
|
||||
}),
|
||||
);
|
||||
}
|
||||
if (expect["output"] !== undefined) {
|
||||
issues.push(...validateContentRules(expect["output"], joinPath(expectPath, "output"), targetName));
|
||||
issues.push(...validateRawContentExpectations(expect["output"], joinPath(expectPath, "output"), targetName));
|
||||
}
|
||||
if (expect["finishReason"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["finishReason"], joinPath(expectPath, "finishReason"), targetName));
|
||||
issues.push(
|
||||
...validateRawValueExpectation(expect["finishReason"], joinPath(expectPath, "finishReason"), targetName),
|
||||
);
|
||||
}
|
||||
if (expect["rawFinishReason"] !== undefined) {
|
||||
issues.push(
|
||||
...validateValueMatcher(expect["rawFinishReason"], joinPath(expectPath, "rawFinishReason"), targetName),
|
||||
...validateRawValueExpectation(expect["rawFinishReason"], joinPath(expectPath, "rawFinishReason"), targetName),
|
||||
);
|
||||
}
|
||||
if (expect["usage"] !== undefined) {
|
||||
@@ -105,7 +108,7 @@ function validateLlmExpect(
|
||||
}
|
||||
}
|
||||
if (expect["durationMs"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
issues.push(...validateRawValueExpectation(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
}
|
||||
|
||||
const allowedKeys = new Set([
|
||||
@@ -289,13 +292,11 @@ function validateStreamExpect(stream: unknown, path: string, targetName?: string
|
||||
if (!isPlainRecord(stream)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
|
||||
normalizeExpectMatchers(stream, ["firstTokenMs"]);
|
||||
|
||||
if (stream["completed"] !== undefined && !isBoolean(stream["completed"])) {
|
||||
issues.push(issue("invalid-type", joinPath(path, "completed"), "必须为布尔值", targetName));
|
||||
}
|
||||
if (stream["firstTokenMs"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(stream["firstTokenMs"], joinPath(path, "firstTokenMs"), targetName));
|
||||
issues.push(...validateRawValueExpectation(stream["firstTokenMs"], joinPath(path, "firstTokenMs"), targetName));
|
||||
}
|
||||
|
||||
const allowedKeys = new Set(["completed", "firstTokenMs"]);
|
||||
@@ -321,11 +322,9 @@ function validateUsageExpect(usage: unknown, path: string, targetName?: string):
|
||||
if (!isPlainRecord(usage)) return [issue("invalid-type", path, "必须为对象", targetName)];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
|
||||
normalizeExpectMatchers(usage, ["inputTokens", "outputTokens", "totalTokens"]);
|
||||
|
||||
for (const key of ["inputTokens", "outputTokens", "totalTokens"]) {
|
||||
if (usage[key] !== undefined) {
|
||||
issues.push(...validateValueMatcher(usage[key], joinPath(path, key), targetName));
|
||||
issues.push(...validateRawValueExpectation(usage[key], joinPath(path, key), targetName));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -2,10 +2,11 @@ import { isError } from "es-toolkit";
|
||||
|
||||
import type { CheckResult, RawTargetConfig } from "../../types";
|
||||
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
|
||||
import type { ResolvedTcpTarget, TcpExpectConfig, TcpTargetConfig } from "./types";
|
||||
import type { RawTcpExpectConfig, ResolvedTcpExpectConfig, ResolvedTcpTarget, TcpTargetConfig } from "./types";
|
||||
|
||||
import { resolveContentExpectations } from "../../expect/content";
|
||||
import { errorFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { checkValueExpectation, resolveValueExpectation } from "../../expect/value";
|
||||
import { parseSize } from "../../utils";
|
||||
import { checkBanner, checkConnected } from "./expect";
|
||||
import { tcpCheckerSchemas } from "./schema";
|
||||
@@ -159,7 +160,7 @@ export class TcpChecker implements CheckerDefinition<ResolvedTcpTarget> {
|
||||
}
|
||||
|
||||
const durationMs = Math.round(performance.now() - start);
|
||||
const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
const durationResult = checkValueExpectation(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
@@ -212,13 +213,23 @@ export class TcpChecker implements CheckerDefinition<ResolvedTcpTarget> {
|
||||
const maxBannerBytes = parseSize(t.tcp.maxBannerBytes ?? tcpDefaults?.maxBannerBytes ?? DEFAULT_MAX_BANNER_BYTES);
|
||||
const bannerReadTimeout = t.tcp.bannerReadTimeout ?? tcpDefaults?.bannerReadTimeout ?? DEFAULT_BANNER_READ_TIMEOUT;
|
||||
|
||||
const rawExpect = target.expect as RawTcpExpectConfig | undefined;
|
||||
const resolvedExpect: ResolvedTcpExpectConfig = rawExpect
|
||||
? {
|
||||
banner: resolveContentExpectations(rawExpect.banner),
|
||||
connected: rawExpect.connected ?? true,
|
||||
durationMs: resolveValueExpectation(rawExpect.durationMs),
|
||||
}
|
||||
: { connected: true };
|
||||
|
||||
return {
|
||||
description: null,
|
||||
expect: target.expect as TcpExpectConfig | undefined,
|
||||
expect: resolvedExpect,
|
||||
group: target.group ?? "default",
|
||||
id: t.id,
|
||||
intervalMs: context.defaultIntervalMs,
|
||||
name: t.name ?? null,
|
||||
rawExpect,
|
||||
tcp: {
|
||||
bannerReadTimeout,
|
||||
host: t.tcp.host,
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
import type { ContentRules, ExpectResult } from "../../expect/types";
|
||||
import type { ContentExpectations, ExpectationResult } from "../../expect/types";
|
||||
|
||||
import { checkContentRules } from "../../expect/content";
|
||||
import { checkContentExpectations } from "../../expect/content";
|
||||
import { mismatchFailure } from "../../expect/failure";
|
||||
|
||||
export function checkBanner(banner: string, rules: ContentRules): ExpectResult {
|
||||
return checkContentRules(banner, rules, { path: "banner", phase: "banner" });
|
||||
export function checkBanner(banner: string, expectations: ContentExpectations): ExpectationResult {
|
||||
return checkContentExpectations(banner, expectations, { path: "banner", phase: "banner" });
|
||||
}
|
||||
|
||||
export function checkConnected(connected: boolean, expected: boolean): ExpectResult {
|
||||
export function checkConnected(connected: boolean, expected: boolean): ExpectationResult {
|
||||
if (connected === expected) return { failure: null, matched: true };
|
||||
if (!connected && expected) {
|
||||
return {
|
||||
|
||||
@@ -2,7 +2,11 @@ import { Type } from "@sinclair/typebox";
|
||||
|
||||
import type { CheckerSchemas } from "../types";
|
||||
|
||||
import { createContentRulesSchema, createValueMatcherSchema, sizeSchema } from "../../schema/fragments";
|
||||
import {
|
||||
createRawContentExpectationsSchema,
|
||||
createRawValueExpectationSchema,
|
||||
sizeSchema,
|
||||
} from "../../schema/fragments";
|
||||
|
||||
export const tcpCheckerSchemas: CheckerSchemas = {
|
||||
config: Type.Object(
|
||||
@@ -24,9 +28,9 @@ export const tcpCheckerSchemas: CheckerSchemas = {
|
||||
),
|
||||
expect: Type.Object(
|
||||
{
|
||||
banner: Type.Optional(createContentRulesSchema()),
|
||||
banner: Type.Optional(createRawContentExpectationsSchema()),
|
||||
connected: Type.Optional(Type.Boolean()),
|
||||
durationMs: Type.Optional(createValueMatcherSchema()),
|
||||
durationMs: Type.Optional(createRawValueExpectationSchema()),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
|
||||
@@ -1,6 +1,17 @@
|
||||
import type { ContentRules, ValueMatcherInput } from "../../expect/types";
|
||||
import type {
|
||||
ContentExpectations,
|
||||
RawContentExpectations,
|
||||
RawValueExpectation,
|
||||
ValueExpectation,
|
||||
} from "../../expect/types";
|
||||
import type { ResolvedTargetBase } from "../../types";
|
||||
|
||||
export interface RawTcpExpectConfig {
|
||||
banner?: RawContentExpectations;
|
||||
connected?: boolean;
|
||||
durationMs?: RawValueExpectation;
|
||||
}
|
||||
|
||||
export interface ResolvedTcpConfig {
|
||||
bannerReadTimeout: number;
|
||||
host: string;
|
||||
@@ -9,11 +20,18 @@ export interface ResolvedTcpConfig {
|
||||
readBanner: boolean;
|
||||
}
|
||||
|
||||
export interface ResolvedTcpExpectConfig {
|
||||
banner?: ContentExpectations;
|
||||
connected: boolean;
|
||||
durationMs?: ValueExpectation;
|
||||
}
|
||||
|
||||
export interface ResolvedTcpTarget extends ResolvedTargetBase {
|
||||
expect?: TcpExpectConfig;
|
||||
expect?: ResolvedTcpExpectConfig;
|
||||
group: string;
|
||||
intervalMs: number;
|
||||
name: null | string;
|
||||
rawExpect?: RawTcpExpectConfig;
|
||||
tcp: ResolvedTcpConfig;
|
||||
timeoutMs: number;
|
||||
type: "tcp";
|
||||
@@ -24,12 +42,6 @@ export interface TcpDefaultsConfig {
|
||||
maxBannerBytes?: number | string;
|
||||
}
|
||||
|
||||
export interface TcpExpectConfig {
|
||||
banner?: ContentRules;
|
||||
connected?: boolean;
|
||||
durationMs?: ValueMatcherInput;
|
||||
}
|
||||
|
||||
export interface TcpTargetConfig {
|
||||
bannerReadTimeout?: number;
|
||||
host: string;
|
||||
|
||||
@@ -1,10 +1,9 @@
|
||||
import { isNumber, isPlainObject, isString } from "es-toolkit";
|
||||
import { isNumber, isString } from "es-toolkit";
|
||||
|
||||
import type { ConfigValidationIssue } from "../../schema/issues";
|
||||
import type { CheckerValidationInput } from "../types";
|
||||
|
||||
import { normalizeExpectMatchers } from "../../expect/normalize";
|
||||
import { validateContentRules, validateValueMatcher } from "../../expect/validate-matcher";
|
||||
import { isPlainRecord, validateRawContentExpectations, validateRawValueExpectation } from "../../expect/validate";
|
||||
import { issue, joinPath } from "../../schema/issues";
|
||||
|
||||
export function validateTcpConfig(input: CheckerValidationInput): ConfigValidationIssue[] {
|
||||
@@ -14,7 +13,7 @@ export function validateTcpConfig(input: CheckerValidationInput): ConfigValidati
|
||||
|
||||
for (let i = 0; i < input.targets.length; i++) {
|
||||
const target = input.targets[i] as unknown;
|
||||
if (!isPlainObject(target)) continue;
|
||||
if (!isPlainRecord(target)) continue;
|
||||
if (target["type"] !== "tcp") continue;
|
||||
issues.push(...validateTcpTarget(target, `targets[${i}]`));
|
||||
}
|
||||
@@ -34,7 +33,7 @@ function isNonNegativeFiniteNumber(value: unknown): boolean {
|
||||
function validateTcpDefaults(input: CheckerValidationInput): ConfigValidationIssue[] {
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const defaults = input.defaults["tcp"];
|
||||
if (defaults === undefined || defaults === null || !isPlainObject(defaults)) return issues;
|
||||
if (defaults === undefined || defaults === null || !isPlainRecord(defaults)) return issues;
|
||||
|
||||
const targetName = "defaults.tcp";
|
||||
|
||||
@@ -72,18 +71,16 @@ function validateTcpExpect(
|
||||
): ConfigValidationIssue[] {
|
||||
const targetName = getTargetName(target);
|
||||
const expect = target["expect"];
|
||||
if (expect === undefined || expect === null || !isPlainObject(expect)) return [];
|
||||
if (expect === undefined || expect === null || !isPlainRecord(expect)) return [];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const expectPath = joinPath(path, "expect");
|
||||
|
||||
normalizeExpectMatchers(expect, ["durationMs"]);
|
||||
|
||||
if (expect["connected"] !== undefined && typeof expect["connected"] !== "boolean") {
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, "connected"), "必须为布尔值", targetName));
|
||||
}
|
||||
|
||||
if (expect["durationMs"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
issues.push(...validateRawValueExpectation(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
}
|
||||
|
||||
if (expect["banner"] !== undefined) {
|
||||
@@ -92,7 +89,7 @@ function validateTcpExpect(
|
||||
issue("invalid-value", joinPath(expectPath, "banner"), "banner 断言需要启用 tcp.readBanner", targetName),
|
||||
);
|
||||
} else {
|
||||
issues.push(...validateContentRules(expect["banner"], joinPath(expectPath, "banner"), targetName));
|
||||
issues.push(...validateRawContentExpectations(expect["banner"], joinPath(expectPath, "banner"), targetName));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -111,7 +108,7 @@ function validateTcpTarget(target: Record<string, unknown>, path: string): Confi
|
||||
const targetName = getTargetName(target);
|
||||
const tcp = target["tcp"];
|
||||
|
||||
if (!isPlainObject(tcp)) {
|
||||
if (!isPlainRecord(tcp)) {
|
||||
issues.push(issue("required", joinPath(path, "tcp"), "缺少 tcp 配置分组", targetName));
|
||||
issues.push(...validateTcpExpect(target, path, false));
|
||||
return issues;
|
||||
|
||||
@@ -2,10 +2,17 @@ import { isError } from "es-toolkit";
|
||||
|
||||
import type { CheckResult, RawTargetConfig } from "../../types";
|
||||
import type { CheckerContext, CheckerDefinition, CheckerValidationInput, ResolveContext } from "../types";
|
||||
import type { ResolvedUdpTarget, UdpDefaultsConfig, UdpExpectConfig, UdpTargetConfig } from "./types";
|
||||
import type {
|
||||
RawUdpExpectConfig,
|
||||
ResolvedUdpExpectConfig,
|
||||
ResolvedUdpTarget,
|
||||
UdpDefaultsConfig,
|
||||
UdpTargetConfig,
|
||||
} from "./types";
|
||||
|
||||
import { resolveContentExpectations } from "../../expect/content";
|
||||
import { errorFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { checkValueExpectation, resolveValueExpectation } from "../../expect/value";
|
||||
import { parseSize } from "../../utils";
|
||||
import { decodePayload, encodeResponse } from "./encoding";
|
||||
import { checkResponded, checkResponseSize, checkResponseText, checkSourceHost, checkSourcePort } from "./expect";
|
||||
@@ -111,7 +118,7 @@ export class UdpChecker implements CheckerDefinition<ResolvedUdpTarget> {
|
||||
};
|
||||
}
|
||||
|
||||
const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
const durationResult = checkValueExpectation(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
@@ -255,7 +262,7 @@ export class UdpChecker implements CheckerDefinition<ResolvedUdpTarget> {
|
||||
}
|
||||
}
|
||||
|
||||
const durationResult = checkValueMatcher(durationMs, expect?.durationMs, {
|
||||
const durationResult = checkValueExpectation(durationMs, expect?.durationMs, {
|
||||
message: "durationMs mismatch",
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
@@ -305,13 +312,26 @@ export class UdpChecker implements CheckerDefinition<ResolvedUdpTarget> {
|
||||
t.udp.maxResponseBytes ?? udpDefaults?.maxResponseBytes ?? DEFAULT_MAX_RESPONSE_BYTES,
|
||||
);
|
||||
|
||||
const rawExpect = target.expect as RawUdpExpectConfig | undefined;
|
||||
const resolvedExpect: ResolvedUdpExpectConfig = rawExpect
|
||||
? {
|
||||
durationMs: resolveValueExpectation(rawExpect.durationMs),
|
||||
responded: rawExpect.responded ?? true,
|
||||
response: resolveContentExpectations(rawExpect.response),
|
||||
responseSize: resolveValueExpectation(rawExpect.responseSize),
|
||||
sourceHost: resolveValueExpectation(rawExpect.sourceHost),
|
||||
sourcePort: resolveValueExpectation(rawExpect.sourcePort),
|
||||
}
|
||||
: { responded: true };
|
||||
|
||||
return {
|
||||
description: null,
|
||||
expect: target.expect as UdpExpectConfig | undefined,
|
||||
expect: resolvedExpect,
|
||||
group: target.group ?? "default",
|
||||
id: t.id,
|
||||
intervalMs: context.defaultIntervalMs,
|
||||
name: t.name ?? null,
|
||||
rawExpect,
|
||||
timeoutMs: context.defaultTimeoutMs,
|
||||
type: "udp",
|
||||
udp: {
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
import type { ContentRules, ExpectResult, ValueMatcherInput } from "../../expect/types";
|
||||
import type { ContentExpectations, ExpectationResult, ValueExpectation } from "../../expect/types";
|
||||
|
||||
import { checkContentRules } from "../../expect/content";
|
||||
import { checkContentExpectations } from "../../expect/content";
|
||||
import { mismatchFailure } from "../../expect/failure";
|
||||
import { checkValueMatcher } from "../../expect/matcher";
|
||||
import { checkValueExpectation } from "../../expect/value";
|
||||
|
||||
export function checkResponded(responded: boolean, expected: boolean): ExpectResult {
|
||||
export function checkResponded(responded: boolean, expected: boolean): ExpectationResult {
|
||||
if (responded === expected) return { failure: null, matched: true };
|
||||
if (!responded && expected) {
|
||||
return {
|
||||
@@ -18,28 +18,28 @@ export function checkResponded(responded: boolean, expected: boolean): ExpectRes
|
||||
};
|
||||
}
|
||||
|
||||
export function checkResponseSize(size: number, matcher: ValueMatcherInput): ExpectResult {
|
||||
return checkValueMatcher(size, matcher, {
|
||||
export function checkResponseSize(size: number, matcher: ValueExpectation): ExpectationResult {
|
||||
return checkValueExpectation(size, matcher, {
|
||||
message: "响应大小不满足条件",
|
||||
path: "responseSize",
|
||||
phase: "responseSize",
|
||||
});
|
||||
}
|
||||
|
||||
export function checkResponseText(text: string, rules: ContentRules): ExpectResult {
|
||||
return checkContentRules(text, rules, { path: "response", phase: "response" });
|
||||
export function checkResponseText(text: string, expectations: ContentExpectations): ExpectationResult {
|
||||
return checkContentExpectations(text, expectations, { path: "response", phase: "response" });
|
||||
}
|
||||
|
||||
export function checkSourceHost(actual: string, matcher: ValueMatcherInput): ExpectResult {
|
||||
return checkValueMatcher(actual, matcher, {
|
||||
export function checkSourceHost(actual: string, matcher: ValueExpectation): ExpectationResult {
|
||||
return checkValueExpectation(actual, matcher, {
|
||||
message: "响应来源地址不满足条件",
|
||||
path: "sourceHost",
|
||||
phase: "sourceHost",
|
||||
});
|
||||
}
|
||||
|
||||
export function checkSourcePort(actual: number, matcher: ValueMatcherInput): ExpectResult {
|
||||
return checkValueMatcher(actual, matcher, {
|
||||
export function checkSourcePort(actual: number, matcher: ValueExpectation): ExpectationResult {
|
||||
return checkValueExpectation(actual, matcher, {
|
||||
message: "响应来源端口不满足条件",
|
||||
path: "sourcePort",
|
||||
phase: "sourcePort",
|
||||
|
||||
@@ -2,7 +2,11 @@ import { Type } from "@sinclair/typebox";
|
||||
|
||||
import type { CheckerSchemas } from "../types";
|
||||
|
||||
import { createContentRulesSchema, createValueMatcherSchema, sizeSchema } from "../../schema/fragments";
|
||||
import {
|
||||
createRawContentExpectationsSchema,
|
||||
createRawValueExpectationSchema,
|
||||
sizeSchema,
|
||||
} from "../../schema/fragments";
|
||||
|
||||
export const udpCheckerSchemas: CheckerSchemas = {
|
||||
config: Type.Object(
|
||||
@@ -26,12 +30,12 @@ export const udpCheckerSchemas: CheckerSchemas = {
|
||||
),
|
||||
expect: Type.Object(
|
||||
{
|
||||
durationMs: Type.Optional(createValueMatcherSchema()),
|
||||
durationMs: Type.Optional(createRawValueExpectationSchema()),
|
||||
responded: Type.Optional(Type.Boolean()),
|
||||
response: Type.Optional(createContentRulesSchema()),
|
||||
responseSize: Type.Optional(createValueMatcherSchema()),
|
||||
sourceHost: Type.Optional(createValueMatcherSchema()),
|
||||
sourcePort: Type.Optional(createValueMatcherSchema()),
|
||||
response: Type.Optional(createRawContentExpectationsSchema()),
|
||||
responseSize: Type.Optional(createRawValueExpectationSchema()),
|
||||
sourceHost: Type.Optional(createRawValueExpectationSchema()),
|
||||
sourcePort: Type.Optional(createRawValueExpectationSchema()),
|
||||
},
|
||||
{ additionalProperties: false },
|
||||
),
|
||||
|
||||
@@ -1,6 +1,20 @@
|
||||
import type { ContentRules, ValueMatcherInput } from "../../expect/types";
|
||||
import type {
|
||||
ContentExpectations,
|
||||
RawContentExpectations,
|
||||
RawValueExpectation,
|
||||
ValueExpectation,
|
||||
} from "../../expect/types";
|
||||
import type { ResolvedTargetBase } from "../../types";
|
||||
|
||||
export interface RawUdpExpectConfig {
|
||||
durationMs?: RawValueExpectation;
|
||||
responded?: boolean;
|
||||
response?: RawContentExpectations;
|
||||
responseSize?: RawValueExpectation;
|
||||
sourceHost?: RawValueExpectation;
|
||||
sourcePort?: RawValueExpectation;
|
||||
}
|
||||
|
||||
export interface ResolvedUdpConfig {
|
||||
encoding: UdpEncoding;
|
||||
host: string;
|
||||
@@ -10,11 +24,21 @@ export interface ResolvedUdpConfig {
|
||||
responseEncoding: UdpEncoding;
|
||||
}
|
||||
|
||||
export interface ResolvedUdpExpectConfig {
|
||||
durationMs?: ValueExpectation;
|
||||
responded: boolean;
|
||||
response?: ContentExpectations;
|
||||
responseSize?: ValueExpectation;
|
||||
sourceHost?: ValueExpectation;
|
||||
sourcePort?: ValueExpectation;
|
||||
}
|
||||
|
||||
export interface ResolvedUdpTarget extends ResolvedTargetBase {
|
||||
expect?: UdpExpectConfig;
|
||||
expect?: ResolvedUdpExpectConfig;
|
||||
group: string;
|
||||
intervalMs: number;
|
||||
name: null | string;
|
||||
rawExpect?: RawUdpExpectConfig;
|
||||
timeoutMs: number;
|
||||
type: "udp";
|
||||
udp: ResolvedUdpConfig;
|
||||
@@ -28,15 +52,6 @@ export interface UdpDefaultsConfig {
|
||||
|
||||
export type UdpEncoding = "base64" | "hex" | "text";
|
||||
|
||||
export interface UdpExpectConfig {
|
||||
durationMs?: ValueMatcherInput;
|
||||
responded?: boolean;
|
||||
response?: ContentRules;
|
||||
responseSize?: ValueMatcherInput;
|
||||
sourceHost?: ValueMatcherInput;
|
||||
sourcePort?: ValueMatcherInput;
|
||||
}
|
||||
|
||||
export interface UdpTargetConfig {
|
||||
encoding?: UdpEncoding;
|
||||
host: string;
|
||||
|
||||
@@ -1,10 +1,9 @@
|
||||
import { isNumber, isPlainObject, isString } from "es-toolkit";
|
||||
import { isNumber, isString } from "es-toolkit";
|
||||
|
||||
import type { ConfigValidationIssue } from "../../schema/issues";
|
||||
import type { CheckerValidationInput } from "../types";
|
||||
|
||||
import { normalizeExpectMatchers } from "../../expect/normalize";
|
||||
import { validateContentRules, validateValueMatcher } from "../../expect/validate-matcher";
|
||||
import { isPlainRecord, validateRawContentExpectations, validateRawValueExpectation } from "../../expect/validate";
|
||||
import { issue, joinPath } from "../../schema/issues";
|
||||
|
||||
const VALID_ENCODINGS = new Set(["base64", "hex", "text"]);
|
||||
@@ -16,7 +15,7 @@ export function validateUdpConfig(input: CheckerValidationInput): ConfigValidati
|
||||
|
||||
for (let i = 0; i < input.targets.length; i++) {
|
||||
const target = input.targets[i] as unknown;
|
||||
if (!isPlainObject(target)) continue;
|
||||
if (!isPlainRecord(target)) continue;
|
||||
if (target["type"] !== "udp") continue;
|
||||
issues.push(...validateUdpTarget(target, `targets[${i}]`));
|
||||
}
|
||||
@@ -48,7 +47,7 @@ function validateSize(value: unknown, path: string, targetName: string | undefin
|
||||
function validateUdpDefaults(input: CheckerValidationInput): ConfigValidationIssue[] {
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const defaults = input.defaults["udp"];
|
||||
if (defaults === undefined || defaults === null || !isPlainObject(defaults)) return issues;
|
||||
if (defaults === undefined || defaults === null || !isPlainRecord(defaults)) return issues;
|
||||
|
||||
const targetName = "defaults.udp";
|
||||
|
||||
@@ -71,35 +70,35 @@ function validateUdpDefaults(input: CheckerValidationInput): ConfigValidationIss
|
||||
function validateUdpExpect(target: Record<string, unknown>, path: string): ConfigValidationIssue[] {
|
||||
const targetName = getTargetName(target);
|
||||
const expect = target["expect"];
|
||||
if (expect === undefined || expect === null || !isPlainObject(expect)) return [];
|
||||
if (expect === undefined || expect === null || !isPlainRecord(expect)) return [];
|
||||
const issues: ConfigValidationIssue[] = [];
|
||||
const expectPath = joinPath(path, "expect");
|
||||
const responded: unknown = expect["responded"];
|
||||
|
||||
normalizeExpectMatchers(expect, ["durationMs", "responseSize", "sourceHost", "sourcePort"]);
|
||||
|
||||
if (responded !== undefined && typeof responded !== "boolean") {
|
||||
issues.push(issue("invalid-type", joinPath(expectPath, "responded"), "必须为布尔值", targetName));
|
||||
}
|
||||
|
||||
if (expect["durationMs"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
issues.push(...validateRawValueExpectation(expect["durationMs"], joinPath(expectPath, "durationMs"), targetName));
|
||||
}
|
||||
|
||||
if (expect["response"] !== undefined) {
|
||||
issues.push(...validateContentRules(expect["response"], joinPath(expectPath, "response"), targetName));
|
||||
issues.push(...validateRawContentExpectations(expect["response"], joinPath(expectPath, "response"), targetName));
|
||||
}
|
||||
|
||||
if (expect["responseSize"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["responseSize"], joinPath(expectPath, "responseSize"), targetName));
|
||||
issues.push(
|
||||
...validateRawValueExpectation(expect["responseSize"], joinPath(expectPath, "responseSize"), targetName),
|
||||
);
|
||||
}
|
||||
|
||||
if (expect["sourceHost"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["sourceHost"], joinPath(expectPath, "sourceHost"), targetName));
|
||||
issues.push(...validateRawValueExpectation(expect["sourceHost"], joinPath(expectPath, "sourceHost"), targetName));
|
||||
}
|
||||
|
||||
if (expect["sourcePort"] !== undefined) {
|
||||
issues.push(...validateValueMatcher(expect["sourcePort"], joinPath(expectPath, "sourcePort"), targetName));
|
||||
issues.push(...validateRawValueExpectation(expect["sourcePort"], joinPath(expectPath, "sourcePort"), targetName));
|
||||
}
|
||||
|
||||
const respondedFalse = responded === false;
|
||||
@@ -141,7 +140,7 @@ function validateUdpTarget(target: Record<string, unknown>, path: string): Confi
|
||||
const targetName = getTargetName(target);
|
||||
const udp = target["udp"];
|
||||
|
||||
if (!isPlainObject(udp)) {
|
||||
if (!isPlainRecord(udp)) {
|
||||
issues.push(issue("required", joinPath(path, "udp"), "缺少 udp 配置分组", targetName));
|
||||
issues.push(...validateUdpExpect(target, path));
|
||||
return issues;
|
||||
|
||||
@@ -5,9 +5,10 @@ import { Type } from "@sinclair/typebox";
|
||||
import type { CheckerDefinition } from "../runner/types";
|
||||
|
||||
import {
|
||||
createContentRulesSchema,
|
||||
createKeyValueExpectSchema,
|
||||
createValueMatcherSchema,
|
||||
createRawContentExpectationsSchema,
|
||||
createRawKeyedExpectationsSchema,
|
||||
createRawValueExpectationSchema,
|
||||
createValueMatcherObjectSchema,
|
||||
durationSchema,
|
||||
variableValueSchema,
|
||||
} from "./fragments";
|
||||
@@ -18,9 +19,10 @@ export function createExternalProbeConfigSchema(checkers: CheckerDefinition[]):
|
||||
$id: "https://dial.local/probe-config.schema.json",
|
||||
$schema: "http://json-schema.org/draft-07/schema#",
|
||||
definitions: {
|
||||
ContentRules: cloneSchema(createContentRulesSchema()),
|
||||
KeyValueExpect: cloneSchema(createKeyValueExpectSchema()),
|
||||
ValueMatcher: cloneSchema(createValueMatcherSchema()),
|
||||
ContentExpectations: cloneSchema(createRawContentExpectationsSchema()),
|
||||
KeyedExpectations: cloneSchema(createRawKeyedExpectationsSchema()),
|
||||
ValueExpectation: cloneSchema(createRawValueExpectationSchema()),
|
||||
ValueMatcher: cloneSchema(createValueMatcherObjectSchema()),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -6,8 +6,6 @@ import type { JsonValue } from "./types";
|
||||
|
||||
export const HTTP_METHODS = ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"] as const;
|
||||
|
||||
export const MatcherKeys = ["contains", "empty", "equals", "exists", "gt", "gte", "lt", "lte", "regex"] as const;
|
||||
|
||||
export const durationSchema = Type.String();
|
||||
|
||||
export const httpMethodSchema = Type.Union(
|
||||
@@ -43,7 +41,7 @@ export const stringMapSchema = Type.Unsafe<Record<string, string>>({
|
||||
type: "object",
|
||||
});
|
||||
|
||||
export function createContentRulesSchema(): TSchema {
|
||||
export function createRawContentExpectationsSchema(): TSchema {
|
||||
return Type.Array(
|
||||
Type.Object(
|
||||
{
|
||||
@@ -66,21 +64,23 @@ export function createContentRulesSchema(): TSchema {
|
||||
);
|
||||
}
|
||||
|
||||
export function createKeyValueExpectSchema(): TSchema {
|
||||
export function createRawKeyedExpectationsSchema(): TSchema {
|
||||
return Type.Unsafe<Record<string, unknown>>({
|
||||
additionalProperties: {
|
||||
anyOf: [jsonValueSchema, createValueMatcherSchema()],
|
||||
},
|
||||
additionalProperties: createRawValueExpectationSchema(),
|
||||
type: "object",
|
||||
});
|
||||
}
|
||||
|
||||
export function createValueMatcherSchema(): TSchema {
|
||||
export function createRawValueExpectationSchema(): TSchema {
|
||||
return Type.Unsafe({
|
||||
anyOf: [primitiveValueSchema, Type.Object(matcherProperties(), { additionalProperties: false, minProperties: 1 })],
|
||||
});
|
||||
}
|
||||
|
||||
export function createValueMatcherObjectSchema(): TSchema {
|
||||
return Type.Object(matcherProperties(), { additionalProperties: false, minProperties: 1 });
|
||||
}
|
||||
|
||||
export function matcherProperties(): Record<string, TSchema> {
|
||||
return {
|
||||
contains: Type.Optional(Type.String()),
|
||||
|
||||
@@ -352,7 +352,7 @@ export class ProbeStore {
|
||||
const serialized = checkerRegistry.get(t.type).serialize(t);
|
||||
const target = serialized.target;
|
||||
const config = serialized.config;
|
||||
const expect = t.expect ? JSON.stringify(t.expect) : null;
|
||||
const expect = t.rawExpect ? JSON.stringify(t.rawExpect) : null;
|
||||
|
||||
if (existingIds.has(t.id)) {
|
||||
updateStmt.run(t.name, t.description, type, target, config, t.intervalMs, t.timeoutMs, expect, t.group, t.id);
|
||||
|
||||
@@ -45,6 +45,7 @@ export interface ResolvedTargetBase {
|
||||
id: string;
|
||||
intervalMs: number;
|
||||
name: null | string;
|
||||
rawExpect?: unknown;
|
||||
timeoutMs: number;
|
||||
type: string;
|
||||
}
|
||||
|
||||
@@ -84,6 +84,33 @@ describe("config contract", () => {
|
||||
expect(validate(target({ equals: { status: "ok" } }))).toBe(true);
|
||||
});
|
||||
|
||||
test("导出 schema 拒绝 KeyedExpectations 的数组和对象简写", () => {
|
||||
const ajv = new Ajv({
|
||||
allErrors: true,
|
||||
coerceTypes: false,
|
||||
removeAdditional: false,
|
||||
strict: true,
|
||||
useDefaults: false,
|
||||
});
|
||||
const validate = ajv.compile(createProbeConfigJsonSchema(createDefaultCheckerRegistry()));
|
||||
const target = (headerValue: unknown) => ({
|
||||
targets: [
|
||||
{
|
||||
expect: { headers: { "x-test": headerValue } },
|
||||
http: { url: "https://example.com" },
|
||||
id: "api",
|
||||
type: "http",
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
expect(validate(target("ok"))).toBe(true);
|
||||
expect(validate(target({ contains: "ok" }))).toBe(true);
|
||||
expect(validate(target(["ok"]))).toBe(false);
|
||||
expect(validate(target({ nested: "ok" }))).toBe(false);
|
||||
expect(validate(target({ equals: { nested: "ok" } }))).toBe(true);
|
||||
});
|
||||
|
||||
test("Ajv 错误转换为中文结构化 issue", () => {
|
||||
const result = validateProbeConfigContract(
|
||||
{
|
||||
|
||||
@@ -3,14 +3,13 @@ import { mkdir, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
|
||||
import type { ValueMatcher } from "../../../src/server/checker/expect/types";
|
||||
import type { ResolvedCommandTarget } from "../../../src/server/checker/runner/cmd/types";
|
||||
import type { ResolvedHttpTarget } from "../../../src/server/checker/runner/http/types";
|
||||
import type { ResolvedPingTarget } from "../../../src/server/checker/runner/icmp/types";
|
||||
import type { ResolvedTcpTarget } from "../../../src/server/checker/runner/tcp/types";
|
||||
|
||||
import { loadConfig, parseDuration } from "../../../src/server/checker/config-loader";
|
||||
import { checkValueMatcher } from "../../../src/server/checker/expect/matcher";
|
||||
import { checkValueExpectation } from "../../../src/server/checker/expect/value";
|
||||
import { checkerRegistry } from "../../../src/server/checker/runner";
|
||||
import { CommandChecker } from "../../../src/server/checker/runner/cmd/execute";
|
||||
import { HttpChecker } from "../../../src/server/checker/runner/http/execute";
|
||||
@@ -290,7 +289,7 @@ targets:
|
||||
expect(config.targets[0]!.name).toBeNull();
|
||||
});
|
||||
|
||||
test("ValueMatcher primitive 简写在加载时归一化后可运行期匹配", async () => {
|
||||
test("ValueMatcher primitive 简写在 resolve 后可运行期匹配", async () => {
|
||||
const configPath = join(tempDir, "matcher-shorthand.yaml");
|
||||
await writeFile(
|
||||
configPath,
|
||||
@@ -309,7 +308,7 @@ targets:
|
||||
|
||||
expect(target.expect?.durationMs).toEqual({ equals: 123 });
|
||||
expect(
|
||||
checkValueMatcher(123, target.expect?.durationMs as ValueMatcher, {
|
||||
checkValueExpectation(123, target.expect?.durationMs, {
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
}).matched,
|
||||
@@ -860,11 +859,19 @@ targets:
|
||||
const config = await loadConfig(configPath);
|
||||
const t = config.targets[0]!;
|
||||
if (t.type === "http") {
|
||||
expect(t.expect).toEqual({
|
||||
expect(t.rawExpect).toEqual({
|
||||
body: [{ contains: "ok" }, { json: { equals: "ok", path: "$.status" } }],
|
||||
durationMs: { lte: 3000 },
|
||||
status: [200, 201],
|
||||
});
|
||||
expect(t.expect).toEqual({
|
||||
body: [
|
||||
{ kind: "value", matcher: { contains: "ok" } },
|
||||
{ kind: "json", matcher: { equals: "ok" }, path: "$.status" },
|
||||
],
|
||||
durationMs: { lte: 3000 },
|
||||
status: [200, 201],
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
@@ -893,12 +900,21 @@ targets:
|
||||
const config = await loadConfig(configPath);
|
||||
const t = config.targets[0]!;
|
||||
if (t.type === "cmd") {
|
||||
expect(t.expect).toEqual({
|
||||
expect(t.rawExpect).toEqual({
|
||||
durationMs: { lte: 5000 },
|
||||
exitCode: [0, 2],
|
||||
stderr: [{ empty: true }],
|
||||
stdout: [{ contains: "ok" }, { regex: "done" }],
|
||||
});
|
||||
expect(t.expect).toEqual({
|
||||
durationMs: { lte: 5000 },
|
||||
exitCode: [0, 2],
|
||||
stderr: [{ kind: "value", matcher: { empty: true } }],
|
||||
stdout: [
|
||||
{ kind: "value", matcher: { contains: "ok" } },
|
||||
{ kind: "value", matcher: { regex: "done" } },
|
||||
],
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
@@ -1917,7 +1933,7 @@ targets:
|
||||
const config = await loadConfig(configPath);
|
||||
const t = config.targets[0]! as ResolvedTcpTarget;
|
||||
expect(t.tcp.readBanner).toBe(true);
|
||||
expect(t.expect?.banner).toEqual([{ contains: "ESMTP" }]);
|
||||
expect(t.expect?.banner).toEqual([{ kind: "value", matcher: { contains: "ESMTP" } }]);
|
||||
});
|
||||
|
||||
test("tcp expect.banner 未开启 readBanner 抛出错误", async () => {
|
||||
|
||||
@@ -1,31 +0,0 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import { normalizeExpectMatchers, normalizeValueMatcher } from "../../../../src/server/checker/expect/normalize";
|
||||
|
||||
describe("normalizeValueMatcher", () => {
|
||||
test("normalizes primitive values to equals matcher", () => {
|
||||
expect(normalizeValueMatcher("stop")).toEqual({ equals: "stop" });
|
||||
expect(normalizeValueMatcher(1)).toEqual({ equals: 1 });
|
||||
expect(normalizeValueMatcher(true)).toEqual({ equals: true });
|
||||
expect(normalizeValueMatcher(null)).toEqual({ equals: null });
|
||||
});
|
||||
|
||||
test("leaves undefined, matcher objects, arrays, and plain objects unchanged", () => {
|
||||
const matcher = { lte: 5000 };
|
||||
const array = [1, 2];
|
||||
const object = { foo: "bar" };
|
||||
|
||||
expect(normalizeValueMatcher(undefined)).toBeUndefined();
|
||||
expect(normalizeValueMatcher(matcher)).toBe(matcher);
|
||||
expect(normalizeValueMatcher(array)).toBe(array);
|
||||
expect(normalizeValueMatcher(object)).toBe(object);
|
||||
});
|
||||
|
||||
test("normalizes only selected expect keys", () => {
|
||||
const expectConfig: Record<string, unknown> = { durationMs: 100, responded: true };
|
||||
|
||||
normalizeExpectMatchers(expectConfig, ["durationMs"]);
|
||||
|
||||
expect(expectConfig).toEqual({ durationMs: { equals: 100 }, responded: true });
|
||||
});
|
||||
});
|
||||
@@ -1,7 +1,7 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import type { ResolvedCommandTarget } from "../../../../../src/server/checker/runner/cmd/types";
|
||||
import type { CheckerContext } from "../../../../../src/server/checker/runner/types";
|
||||
import type { CheckerContext, ResolveContext } from "../../../../../src/server/checker/runner/types";
|
||||
|
||||
import { CommandChecker } from "../../../../../src/server/checker/runner/cmd/execute";
|
||||
|
||||
@@ -17,6 +17,10 @@ function makeCtx(timeoutMs = 5000): CheckerContext {
|
||||
return { signal: controller.signal };
|
||||
}
|
||||
|
||||
function makeResolveContext(): ResolveContext {
|
||||
return { configDir: process.cwd(), defaultIntervalMs: 30000, defaults: {}, defaultTimeoutMs: 10000 };
|
||||
}
|
||||
|
||||
function makeTarget(
|
||||
cmd: Partial<ResolvedCommandTarget["cmd"]>,
|
||||
overrides?: Partial<ResolvedCommandTarget>,
|
||||
@@ -94,7 +98,7 @@ describe("CommandChecker", () => {
|
||||
const result = await checker.execute(
|
||||
makeTarget(
|
||||
{ args: ["-e", "console.log('hello')"], exec: "bun" },
|
||||
{ expect: { stdout: [{ contains: "hello" }] } },
|
||||
{ expect: { exitCode: [0], stdout: [{ kind: "value", matcher: { contains: "hello" } }] } },
|
||||
),
|
||||
makeCtx(),
|
||||
);
|
||||
@@ -105,7 +109,7 @@ describe("CommandChecker", () => {
|
||||
const result = await checker.execute(
|
||||
makeTarget(
|
||||
{ args: ["-e", "console.log('hello')"], exec: "bun" },
|
||||
{ expect: { stdout: [{ contains: "nonexistent" }] } },
|
||||
{ expect: { exitCode: [0], stdout: [{ kind: "value", matcher: { contains: "nonexistent" } }] } },
|
||||
),
|
||||
makeCtx(),
|
||||
);
|
||||
@@ -117,7 +121,7 @@ describe("CommandChecker", () => {
|
||||
const result = await checker.execute(
|
||||
makeTarget(
|
||||
{ args: ["-e", "process.stderr.write('error\\n')"], exec: "bun" },
|
||||
{ expect: { stderr: [{ contains: "error" }] } },
|
||||
{ expect: { exitCode: [0], stderr: [{ kind: "value", matcher: { contains: "error" } }] } },
|
||||
),
|
||||
makeCtx(),
|
||||
);
|
||||
@@ -142,7 +146,10 @@ describe("CommandChecker", () => {
|
||||
|
||||
test("不使用 shell，通配符不被展开", async () => {
|
||||
const result = await checker.execute(
|
||||
makeTarget({ args: ["-e", "console.log('*')"], exec: "bun" }, { expect: { stdout: [{ contains: "*" }] } }),
|
||||
makeTarget(
|
||||
{ args: ["-e", "console.log('*')"], exec: "bun" },
|
||||
{ expect: { exitCode: [0], stdout: [{ kind: "value", matcher: { contains: "*" } }] } },
|
||||
),
|
||||
makeCtx(),
|
||||
);
|
||||
expect(result.matched).toBe(true);
|
||||
@@ -156,7 +163,7 @@ describe("CommandChecker", () => {
|
||||
env: { ...processEnv, DIAL_TEST_ENV: "resolved-env" },
|
||||
exec: "bun",
|
||||
},
|
||||
{ expect: { stdout: [{ contains: "resolved-env" }] } },
|
||||
{ expect: { exitCode: [0], stdout: [{ kind: "value", matcher: { contains: "resolved-env" } }] } },
|
||||
),
|
||||
makeCtx(),
|
||||
);
|
||||
@@ -172,4 +179,11 @@ describe("CommandChecker", () => {
|
||||
expect(config.exec).toBe("bun");
|
||||
expect(config.args).toEqual(["-e", "console.log('hello')"]);
|
||||
});
|
||||
|
||||
test("resolve 未配置 expect 时物化默认 exitCode", () => {
|
||||
const result = checker.resolve({ cmd: { exec: "true" }, id: "test", type: "cmd" }, makeResolveContext());
|
||||
|
||||
expect(result.rawExpect).toBeUndefined();
|
||||
expect(result.expect).toEqual({ exitCode: [0] });
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,8 +1,15 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import type { ResolvedDbTarget } from "../../../../../src/server/checker/runner/db/types";
|
||||
import type {
|
||||
RawDbExpectConfig,
|
||||
ResolvedDbExpectConfig,
|
||||
ResolvedDbTarget,
|
||||
} from "../../../../../src/server/checker/runner/db/types";
|
||||
import type { CheckerContext } from "../../../../../src/server/checker/runner/types";
|
||||
|
||||
import { resolveContentExpectations } from "../../../../../src/server/checker/expect/content";
|
||||
import { resolveKeyedExpectations } from "../../../../../src/server/checker/expect/keyed";
|
||||
import { resolveValueExpectation } from "../../../../../src/server/checker/expect/value";
|
||||
import { DbChecker } from "../../../../../src/server/checker/runner/db/execute";
|
||||
|
||||
const checker = new DbChecker();
|
||||
@@ -13,20 +20,31 @@ function makeCtx(timeoutMs = 5000): CheckerContext {
|
||||
return { signal: controller.signal };
|
||||
}
|
||||
|
||||
function makeTarget(db: Partial<ResolvedDbTarget["db"]>, overrides?: Partial<ResolvedDbTarget>): ResolvedDbTarget {
|
||||
function makeTarget(db: Partial<ResolvedDbTarget["db"]>, overrides?: { expect?: RawDbExpectConfig }): ResolvedDbTarget {
|
||||
const raw = overrides?.expect;
|
||||
const resolvedExpect: ResolvedDbExpectConfig | undefined = raw
|
||||
? {
|
||||
durationMs: resolveValueExpectation(raw.durationMs),
|
||||
result: resolveContentExpectations(raw.result),
|
||||
rowCount: resolveValueExpectation(raw.rowCount),
|
||||
rows: raw.rows?.map((row) => resolveKeyedExpectations(row) ?? []),
|
||||
}
|
||||
: undefined;
|
||||
|
||||
return {
|
||||
db: {
|
||||
url: "sqlite://:memory:",
|
||||
...db,
|
||||
},
|
||||
description: null,
|
||||
expect: resolvedExpect,
|
||||
group: "default",
|
||||
id: "test-db",
|
||||
intervalMs: 60000,
|
||||
name: "test-db",
|
||||
rawExpect: raw,
|
||||
timeoutMs: 5000,
|
||||
type: "db",
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -1,7 +1,14 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import type { KeyedExpectations, RawValueExpectation } from "../../../../../src/server/checker/expect/types";
|
||||
|
||||
import { resolveValueExpectation } from "../../../../../src/server/checker/expect/value";
|
||||
import { checkRowCount, checkRows } from "../../../../../src/server/checker/runner/db/expect";
|
||||
|
||||
function row(record: Record<string, RawValueExpectation>): KeyedExpectations {
|
||||
return Object.entries(record).map(([key, value]) => ({ key, matcher: resolveValueExpectation(value) }));
|
||||
}
|
||||
|
||||
describe("checkRowCount", () => {
|
||||
test("空数组通过 rowCount gte 0", () => {
|
||||
const result = checkRowCount(0, { gte: 0 });
|
||||
@@ -39,7 +46,7 @@ describe("checkRowCount", () => {
|
||||
|
||||
describe("checkRows", () => {
|
||||
test("非数组返回失败", () => {
|
||||
const result = checkRows(null, [{ col: 1 }]);
|
||||
const result = checkRows(null, [row({ col: 1 })]);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.phase).toBe("row");
|
||||
expect(result.failure!.path).toBe("rows");
|
||||
@@ -51,17 +58,17 @@ describe("checkRows", () => {
|
||||
});
|
||||
|
||||
test("单行单列匹配（字面量）", () => {
|
||||
const result = checkRows([{ col: "value" }], [{ col: "value" }]);
|
||||
const result = checkRows([{ col: "value" }], [row({ col: "value" })]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("单行单列匹配（operator）", () => {
|
||||
const result = checkRows([{ col: 100 }], [{ col: { gte: 50 } }]);
|
||||
const result = checkRows([{ col: 100 }], [row({ col: { gte: 50 } })]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("单行单列不匹配", () => {
|
||||
const result = checkRows([{ col: 10 }], [{ col: { gte: 50 } }]);
|
||||
const result = checkRows([{ col: 10 }], [row({ col: { gte: 50 } })]);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.phase).toBe("row");
|
||||
expect(result.failure!.path).toBe("rows[0].col");
|
||||
@@ -73,62 +80,61 @@ describe("checkRows", () => {
|
||||
{ id: 1, name: "Alice" },
|
||||
{ id: 2, name: "Bob" },
|
||||
],
|
||||
[{ id: { gte: 1 } }, { name: "Bob" }],
|
||||
[row({ id: { gte: 1 } }), row({ name: "Bob" })],
|
||||
);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("多行中有一行不匹配", () => {
|
||||
const result = checkRows([{ col: 1 }, { col: 2 }], [{ col: { gte: 2 } }, { col: { gte: 3 } }]);
|
||||
const result = checkRows([{ col: 1 }, { col: 2 }], [row({ col: { gte: 2 } }), row({ col: { gte: 3 } })]);
|
||||
expect(result.matched).toBe(false);
|
||||
// 第一行 { col: 1 } 不满足 { gte: 2 }，所以失败在第一行
|
||||
expect(result.failure!.path).toBe("rows[0].col");
|
||||
});
|
||||
|
||||
test("结果行数不足", () => {
|
||||
const result = checkRows([{ col: 1 }], [{ col: 1 }, { col: 2 }]);
|
||||
const result = checkRows([{ col: 1 }], [row({ col: 1 }), row({ col: 2 })]);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.message).toContain("行数不足");
|
||||
});
|
||||
|
||||
test("只检查声明的列", () => {
|
||||
const result = checkRows([{ col: 1, other: "ignored" }], [{ col: { gte: 0 } }]);
|
||||
const result = checkRows([{ col: 1, other: "ignored" }], [row({ col: { gte: 0 } })]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("行不是对象返回失败", () => {
|
||||
const result = checkRows(["not-an-object"] as unknown[], [{ col: 1 }]);
|
||||
const result = checkRows(["not-an-object"] as unknown[], [row({ col: 1 })]);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.path).toBe("rows[0]");
|
||||
});
|
||||
|
||||
test("列不存在视为 undefined", () => {
|
||||
const result = checkRows([{}], [{ col: { exists: false } }]);
|
||||
const result = checkRows([{}], [row({ col: { exists: false } })]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("列存在且值为 null", () => {
|
||||
const result = checkRows([{ col: null }], [{ col: { empty: true } }]);
|
||||
const result = checkRows([{ col: null }], [row({ col: { empty: true } })]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("contains 匹配字符串", () => {
|
||||
const result = checkRows([{ text: "hello world" }], [{ text: { contains: "hello" } }]);
|
||||
const result = checkRows([{ text: "hello world" }], [row({ text: { contains: "hello" } })]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("regex 正则匹配", () => {
|
||||
const result = checkRows([{ code: "ABC-123" }], [{ code: { regex: "^ABC-" } }]);
|
||||
const result = checkRows([{ code: "ABC-123" }], [row({ code: { regex: "^ABC-" } })]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("多个断言同时满足", () => {
|
||||
const result = checkRows([{ val: 50 }], [{ val: { gte: 10, lte: 100 } }]);
|
||||
const result = checkRows([{ val: 50 }], [row({ val: { gte: 10, lte: 100 } })]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("多个断言中有一个不满足", () => {
|
||||
const result = checkRows([{ val: 50 }], [{ val: { gte: 10, lte: 30 } }]);
|
||||
const result = checkRows([{ val: 50 }], [row({ val: { gte: 10, lte: 30 } })]);
|
||||
expect(result.matched).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,6 +1,14 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import { checkHeaders, checkStatus } from "../../../../../src/server/checker/runner/http/expect";
|
||||
import type { RawKeyedExpectations } from "../../../../../src/server/checker/expect/types";
|
||||
|
||||
import { checkHeaderExpectations } from "../../../../../src/server/checker/expect/headers";
|
||||
import { resolveKeyedExpectations } from "../../../../../src/server/checker/expect/keyed";
|
||||
import { checkStatusCode } from "../../../../../src/server/checker/expect/status";
|
||||
|
||||
function checkHeaders(headers: Record<string, unknown>, raw?: RawKeyedExpectations) {
|
||||
return checkHeaderExpectations(headers, resolveKeyedExpectations(raw));
|
||||
}
|
||||
|
||||
describe("checkHeaders", () => {
|
||||
test("未配置 headers expect 时匹配成功", () => {
|
||||
@@ -46,15 +54,15 @@ describe("checkHeaders", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("checkStatus 范围匹配", () => {
|
||||
test("无 expect 配置时默认 status [200] 可由调用方使用 checkStatus 表达", () => {
|
||||
const result = checkStatus(200, [200]);
|
||||
describe("checkStatusCode 范围匹配", () => {
|
||||
test("无 expect 配置时默认 status [200] 可由调用方使用 checkStatusCode 表达", () => {
|
||||
const result = checkStatusCode(200, [200]);
|
||||
expect(result.matched).toBe(true);
|
||||
expect(result.failure).toBeNull();
|
||||
});
|
||||
|
||||
test("status 不匹配返回 phase=status 的失败", () => {
|
||||
const result = checkStatus(503, [200]);
|
||||
const result = checkStatusCode(503, [200]);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.phase).toBe("status");
|
||||
expect(result.failure!.expected).toEqual([200]);
|
||||
@@ -62,47 +70,47 @@ describe("checkStatus 范围匹配", () => {
|
||||
});
|
||||
|
||||
test("2xx 范围匹配 200", () => {
|
||||
expect(checkStatus(200, ["2xx"]).matched).toBe(true);
|
||||
expect(checkStatusCode(200, ["2xx"]).matched).toBe(true);
|
||||
});
|
||||
|
||||
test("2xx 范围匹配 204", () => {
|
||||
expect(checkStatus(204, ["2xx"]).matched).toBe(true);
|
||||
expect(checkStatusCode(204, ["2xx"]).matched).toBe(true);
|
||||
});
|
||||
|
||||
test("2xx 范围不匹配 301", () => {
|
||||
expect(checkStatus(301, ["2xx"]).matched).toBe(false);
|
||||
expect(checkStatusCode(301, ["2xx"]).matched).toBe(false);
|
||||
});
|
||||
|
||||
test("5xx 范围匹配 503", () => {
|
||||
expect(checkStatus(503, ["5xx"]).matched).toBe(true);
|
||||
expect(checkStatusCode(503, ["5xx"]).matched).toBe(true);
|
||||
});
|
||||
|
||||
test("混合精确值与范围模式命中精确值", () => {
|
||||
expect(checkStatus(301, ["2xx", 301]).matched).toBe(true);
|
||||
expect(checkStatusCode(301, ["2xx", 301]).matched).toBe(true);
|
||||
});
|
||||
|
||||
test("混合精确值与范围模式命中范围", () => {
|
||||
expect(checkStatus(204, ["2xx", 301]).matched).toBe(true);
|
||||
expect(checkStatusCode(204, ["2xx", 301]).matched).toBe(true);
|
||||
});
|
||||
|
||||
test("混合模式都不匹配", () => {
|
||||
expect(checkStatus(404, ["2xx", 301]).matched).toBe(false);
|
||||
expect(checkStatusCode(404, ["2xx", 301]).matched).toBe(false);
|
||||
});
|
||||
|
||||
test("纯精确值仍正常工作", () => {
|
||||
expect(checkStatus(200, [200, 201]).matched).toBe(true);
|
||||
expect(checkStatus(404, [200, 201]).matched).toBe(false);
|
||||
expect(checkStatusCode(200, [200, 201]).matched).toBe(true);
|
||||
expect(checkStatusCode(404, [200, 201]).matched).toBe(false);
|
||||
});
|
||||
|
||||
test("1xx 范围匹配 101", () => {
|
||||
expect(checkStatus(101, ["1xx"]).matched).toBe(true);
|
||||
expect(checkStatusCode(101, ["1xx"]).matched).toBe(true);
|
||||
});
|
||||
|
||||
test("3xx 范围匹配 301", () => {
|
||||
expect(checkStatus(301, ["3xx"]).matched).toBe(true);
|
||||
expect(checkStatusCode(301, ["3xx"]).matched).toBe(true);
|
||||
});
|
||||
|
||||
test("4xx 范围匹配 404", () => {
|
||||
expect(checkStatus(404, ["4xx"]).matched).toBe(true);
|
||||
expect(checkStatusCode(404, ["4xx"]).matched).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,13 +1,22 @@
|
||||
import { afterAll, beforeAll, describe, expect, test } from "bun:test";
|
||||
|
||||
import type { ResolvedHttpTarget } from "../../../../../src/server/checker/runner/http/types";
|
||||
import type {
|
||||
RawHttpExpectConfig,
|
||||
ResolvedHttpExpectConfig,
|
||||
ResolvedHttpTarget,
|
||||
} from "../../../../../src/server/checker/runner/http/types";
|
||||
import type { CheckerContext, ResolveContext } from "../../../../../src/server/checker/runner/types";
|
||||
|
||||
import { resolveContentExpectations } from "../../../../../src/server/checker/expect/content";
|
||||
import { resolveKeyedExpectations } from "../../../../../src/server/checker/expect/keyed";
|
||||
import { checkStatusCode } from "../../../../../src/server/checker/expect/status";
|
||||
import { resolveValueExpectation } from "../../../../../src/server/checker/expect/value";
|
||||
import { HttpChecker } from "../../../../../src/server/checker/runner/http/execute";
|
||||
import { checkStatus } from "../../../../../src/server/checker/runner/http/expect";
|
||||
import { formatConfigIssues } from "../../../../../src/server/checker/schema/issues";
|
||||
|
||||
const checker = new HttpChecker();
|
||||
const SLOW_BODY_DELAY_MS = 1000;
|
||||
const FAST_RESPONSE_LIMIT_MS = 500;
|
||||
|
||||
function validateHttpTarget(target: unknown): string {
|
||||
return formatConfigIssues(checker.validate({ defaults: {}, targets: [target as never] }));
|
||||
@@ -145,7 +154,7 @@ describe("HttpChecker", () => {
|
||||
|
||||
function makeTarget(overrides: {
|
||||
body?: string;
|
||||
expect?: Record<string, unknown>;
|
||||
expect?: RawHttpExpectConfig;
|
||||
headers?: Record<string, string>;
|
||||
ignoreSSL?: boolean;
|
||||
maxBodyBytes?: number;
|
||||
@@ -154,9 +163,19 @@ describe("HttpChecker", () => {
|
||||
timeoutMs?: number;
|
||||
url?: string;
|
||||
}): ResolvedHttpTarget {
|
||||
const raw = overrides.expect;
|
||||
const resolvedExpect: ResolvedHttpExpectConfig | undefined = raw
|
||||
? {
|
||||
body: resolveContentExpectations(raw.body),
|
||||
durationMs: resolveValueExpectation(raw.durationMs),
|
||||
headers: resolveKeyedExpectations(raw.headers),
|
||||
status: raw.status ?? [200],
|
||||
}
|
||||
: undefined;
|
||||
|
||||
return {
|
||||
description: null,
|
||||
expect: overrides.expect,
|
||||
expect: resolvedExpect,
|
||||
group: "default",
|
||||
http: {
|
||||
body: overrides.body,
|
||||
@@ -170,6 +189,7 @@ describe("HttpChecker", () => {
|
||||
id: "test-http",
|
||||
intervalMs: 60000,
|
||||
name: "test-http",
|
||||
rawExpect: raw,
|
||||
timeoutMs: overrides.timeoutMs ?? 5000,
|
||||
type: "http",
|
||||
};
|
||||
@@ -181,6 +201,29 @@ describe("HttpChecker", () => {
|
||||
return { signal: controller.signal };
|
||||
}
|
||||
|
||||
function startSlowBodyServer(init: ResponseInit = {}) {
|
||||
return Bun.serve({
|
||||
fetch() {
|
||||
let sentFirstChunk = false;
|
||||
return new Response(
|
||||
new ReadableStream<Uint8Array>({
|
||||
async pull(controller) {
|
||||
if (!sentFirstChunk) {
|
||||
sentFirstChunk = true;
|
||||
controller.enqueue(new TextEncoder().encode("slow body"));
|
||||
return;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, SLOW_BODY_DELAY_MS));
|
||||
controller.close();
|
||||
},
|
||||
}),
|
||||
init,
|
||||
);
|
||||
},
|
||||
port: 0,
|
||||
});
|
||||
}
|
||||
|
||||
test("成功请求 200", async () => {
|
||||
const result = await checker.execute(makeTarget({ url: `${baseUrl}/ok` }), makeCtx());
|
||||
expect(result.matched).toBe(true);
|
||||
@@ -192,7 +235,7 @@ describe("HttpChecker", () => {
|
||||
test("404 不匹配默认 status [200]", async () => {
|
||||
const result = await checker.execute(makeTarget({ url: `${baseUrl}/notfound` }), makeCtx());
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.observation).toMatchObject({ bodyPreview: "not found", statusCode: 404 });
|
||||
expect(result.observation).toMatchObject({ bodyPreview: null, statusCode: 404 });
|
||||
expect(result.failure!.phase).toBe("status");
|
||||
});
|
||||
|
||||
@@ -218,7 +261,7 @@ describe("HttpChecker", () => {
|
||||
makeCtx(),
|
||||
);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.observation).toMatchObject({ bodyPreview: "hello world", statusCode: 200 });
|
||||
expect(result.observation).toMatchObject({ bodyPreview: null, statusCode: 200 });
|
||||
expect(result.failure!.phase).toBe("headers");
|
||||
});
|
||||
|
||||
@@ -279,12 +322,43 @@ describe("HttpChecker", () => {
|
||||
});
|
||||
|
||||
test("快速失败：status 失败时不读取 body", async () => {
|
||||
const slowBodyServer = startSlowBodyServer({ status: 404 });
|
||||
try {
|
||||
const start = performance.now();
|
||||
const result = await checker.execute(
|
||||
makeTarget({ expect: { body: [{ contains: "something" }], status: [200] }, url: `${baseUrl}/notfound` }),
|
||||
makeTarget({
|
||||
expect: { body: [{ contains: "something" }], status: [200] },
|
||||
url: `http://localhost:${slowBodyServer.port}/`,
|
||||
}),
|
||||
makeCtx(),
|
||||
);
|
||||
expect(performance.now() - start).toBeLessThan(FAST_RESPONSE_LIMIT_MS);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.phase).toBe("status");
|
||||
expect(result.observation?.["bodyPreview"]).toBeNull();
|
||||
} finally {
|
||||
void slowBodyServer.stop();
|
||||
}
|
||||
});
|
||||
|
||||
test("快速失败：headers 失败时不读取 body", async () => {
|
||||
const slowBodyServer = startSlowBodyServer({ headers: { "x-custom": "actual" }, status: 200 });
|
||||
try {
|
||||
const start = performance.now();
|
||||
const result = await checker.execute(
|
||||
makeTarget({
|
||||
expect: { body: [{ contains: "something" }], headers: { "x-custom": "expected" }, status: [200] },
|
||||
url: `http://localhost:${slowBodyServer.port}/`,
|
||||
}),
|
||||
makeCtx(),
|
||||
);
|
||||
expect(performance.now() - start).toBeLessThan(FAST_RESPONSE_LIMIT_MS);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.phase).toBe("headers");
|
||||
expect(result.observation?.["bodyPreview"]).toBeNull();
|
||||
} finally {
|
||||
void slowBodyServer.stop();
|
||||
}
|
||||
});
|
||||
|
||||
test("status 通过但 body 失败", async () => {
|
||||
@@ -511,12 +585,40 @@ describe("HttpChecker", () => {
|
||||
}
|
||||
});
|
||||
|
||||
test("无 body rules 时不读取 body", async () => {
|
||||
test("无 body expectations 时不读取 body", async () => {
|
||||
const slowBodyServer = startSlowBodyServer({ status: 200 });
|
||||
try {
|
||||
const start = performance.now();
|
||||
const result = await checker.execute(
|
||||
makeTarget({ expect: { status: [200] }, maxBodyBytes: 1, url: `${baseUrl}/large` }),
|
||||
makeTarget({ expect: { status: [200] }, maxBodyBytes: 1, url: `http://localhost:${slowBodyServer.port}/` }),
|
||||
makeCtx(),
|
||||
);
|
||||
expect(performance.now() - start).toBeLessThan(FAST_RESPONSE_LIMIT_MS);
|
||||
expect(result.matched).toBe(true);
|
||||
expect(result.observation?.["bodyPreview"]).toBeNull();
|
||||
} finally {
|
||||
void slowBodyServer.stop();
|
||||
}
|
||||
});
|
||||
|
||||
test("进入 body 前 durationMs 上界已失败时不读取 body", async () => {
|
||||
const slowBodyServer = startSlowBodyServer({ status: 200 });
|
||||
try {
|
||||
const start = performance.now();
|
||||
const result = await checker.execute(
|
||||
makeTarget({
|
||||
expect: { body: [{ contains: "slow" }], durationMs: { lte: 0 }, status: [200] },
|
||||
url: `http://localhost:${slowBodyServer.port}/`,
|
||||
}),
|
||||
makeCtx(),
|
||||
);
|
||||
expect(performance.now() - start).toBeLessThan(FAST_RESPONSE_LIMIT_MS);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.phase).toBe("duration");
|
||||
expect(result.observation?.["bodyPreview"]).toBeNull();
|
||||
} finally {
|
||||
void slowBodyServer.stop();
|
||||
}
|
||||
});
|
||||
|
||||
test("body 失败优先于 duration 检查", async () => {
|
||||
@@ -598,7 +700,7 @@ describe("HttpChecker", () => {
|
||||
expect(result.observation).toMatchObject({ statusCode: 200 });
|
||||
});
|
||||
|
||||
test("混合 body rules 集成检查", async () => {
|
||||
test("混合 body expectations 集成检查", async () => {
|
||||
const result = await checker.execute(
|
||||
makeTarget({
|
||||
expect: {
|
||||
@@ -649,7 +751,7 @@ describe("HttpChecker", () => {
|
||||
});
|
||||
|
||||
test("1xx 范围模式匹配 101", () => {
|
||||
const r = checkStatus(101, ["1xx"]);
|
||||
const r = checkStatusCode(101, ["1xx"]);
|
||||
expect(r.matched).toBe(true);
|
||||
});
|
||||
|
||||
@@ -868,6 +970,16 @@ describe("HttpChecker.resolve", () => {
|
||||
expect(result.http.ignoreSSL).toBe(false);
|
||||
});
|
||||
|
||||
test("未配置 expect 时在 Resolved 模型物化默认 status", () => {
|
||||
const result = checker.resolve(
|
||||
{ http: { url: "https://example.com" }, id: "test", name: "test", type: "http" },
|
||||
makeResolveContext(),
|
||||
);
|
||||
|
||||
expect(result.rawExpect).toBeUndefined();
|
||||
expect(result.expect).toEqual({ status: [200] });
|
||||
});
|
||||
|
||||
test("maxRedirects 默认值为 0", () => {
|
||||
const result = checker.resolve(
|
||||
{ http: { url: "https://example.com" }, id: "test", name: "test", type: "http" },
|
||||
|
||||
@@ -84,7 +84,10 @@ rtt min/avg/max/mdev = 1.234/2.345/3.456/0.567 ms`);
|
||||
test("packetLoss 断言失败", async () => {
|
||||
mockSpawn(`3 packets transmitted, 2 received, 33% packet loss, time 2003ms
|
||||
rtt min/avg/max/mdev = 1.234/156.000/340.000/0.567 ms`);
|
||||
const result = await checker.execute(makeTarget({ expect: { packetLossPercent: { lte: 10 } } }), makeCtx());
|
||||
const result = await checker.execute(
|
||||
makeTarget({ expect: { alive: true, packetLossPercent: { lte: 10 } } }),
|
||||
makeCtx(),
|
||||
);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("packetLoss");
|
||||
expect(result.observation).toMatchObject({ alive: true, maxLatencyMs: 340 });
|
||||
@@ -125,6 +128,8 @@ describe("IcmpChecker resolve", () => {
|
||||
);
|
||||
expect(target.icmp).toEqual({ count: 3, host: "10.0.0.1", packetSize: 56 });
|
||||
expect(target.group).toBe("default");
|
||||
expect(target.rawExpect).toBeUndefined();
|
||||
expect(target.expect).toEqual({ alive: true });
|
||||
});
|
||||
|
||||
test("serialize 返回摘要和配置", () => {
|
||||
|
||||
@@ -1,8 +1,15 @@
|
||||
import { afterAll, beforeAll, describe, expect, test } from "bun:test";
|
||||
|
||||
import type { ResolvedLlmTarget } from "../../../../../src/server/checker/runner/llm/types";
|
||||
import type {
|
||||
RawLlmExpectConfig,
|
||||
ResolvedLlmExpectConfig,
|
||||
ResolvedLlmTarget,
|
||||
} from "../../../../../src/server/checker/runner/llm/types";
|
||||
import type { CheckerContext } from "../../../../../src/server/checker/runner/types";
|
||||
|
||||
import { resolveContentExpectations } from "../../../../../src/server/checker/expect/content";
|
||||
import { resolveKeyedExpectations } from "../../../../../src/server/checker/expect/keyed";
|
||||
import { resolveValueExpectation } from "../../../../../src/server/checker/expect/value";
|
||||
import { LlmChecker } from "../../../../../src/server/checker/runner/llm/execute";
|
||||
|
||||
const MOCK_PORT = 18456;
|
||||
@@ -14,13 +21,34 @@ function makeCtx(timeoutMs = 10000): CheckerContext {
|
||||
return { signal: controller.signal };
|
||||
}
|
||||
|
||||
function makeTarget(
|
||||
overrides?: Partial<ResolvedLlmTarget["llm"]>,
|
||||
expectOverrides?: Partial<ResolvedLlmTarget["expect"]>,
|
||||
): ResolvedLlmTarget {
|
||||
function makeTarget(overrides?: Partial<ResolvedLlmTarget["llm"]>, rawExpect?: RawLlmExpectConfig): ResolvedLlmTarget {
|
||||
const resolvedExpect: ResolvedLlmExpectConfig | undefined = rawExpect
|
||||
? {
|
||||
durationMs: resolveValueExpectation(rawExpect.durationMs),
|
||||
finishReason: resolveValueExpectation(rawExpect.finishReason),
|
||||
headers: resolveKeyedExpectations(rawExpect.headers),
|
||||
output: resolveContentExpectations(rawExpect.output),
|
||||
rawFinishReason: resolveValueExpectation(rawExpect.rawFinishReason),
|
||||
status: rawExpect.status ?? [200],
|
||||
stream: rawExpect.stream
|
||||
? {
|
||||
completed: rawExpect.stream.completed ?? true,
|
||||
firstTokenMs: resolveValueExpectation(rawExpect.stream.firstTokenMs),
|
||||
}
|
||||
: undefined,
|
||||
usage: rawExpect.usage
|
||||
? {
|
||||
inputTokens: resolveValueExpectation(rawExpect.usage.inputTokens),
|
||||
outputTokens: resolveValueExpectation(rawExpect.usage.outputTokens),
|
||||
totalTokens: resolveValueExpectation(rawExpect.usage.totalTokens),
|
||||
}
|
||||
: undefined,
|
||||
}
|
||||
: undefined;
|
||||
|
||||
return {
|
||||
description: null,
|
||||
expect: expectOverrides,
|
||||
expect: resolvedExpect,
|
||||
group: "default",
|
||||
id: "test-llm",
|
||||
intervalMs: 30000,
|
||||
@@ -38,6 +66,7 @@ function makeTarget(
|
||||
...overrides,
|
||||
},
|
||||
name: null,
|
||||
rawExpect,
|
||||
timeoutMs: 10000,
|
||||
type: "llm",
|
||||
};
|
||||
|
||||
@@ -1,12 +1,32 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import type { LlmCheckObservation } from "../../../../../src/server/checker/runner/llm/types";
|
||||
import type {
|
||||
RawContentExpectations,
|
||||
RawKeyedExpectations,
|
||||
RawValueExpectation,
|
||||
} from "../../../../../src/server/checker/expect/types";
|
||||
import type { LlmCheckObservation, ResolvedLlmExpectConfig } from "../../../../../src/server/checker/runner/llm/types";
|
||||
|
||||
import { checkContentRules } from "../../../../../src/server/checker/expect/content";
|
||||
import { checkContentExpectations, resolveContentExpectations } from "../../../../../src/server/checker/expect/content";
|
||||
import { resolveKeyedExpectations } from "../../../../../src/server/checker/expect/keyed";
|
||||
import { resolveValueExpectation } from "../../../../../src/server/checker/expect/value";
|
||||
import { runExpects } from "../../../../../src/server/checker/runner/llm/expect";
|
||||
|
||||
function checkOutputRules(outputText: null | string, rules: Parameters<typeof checkContentRules>[1]) {
|
||||
return checkContentRules(outputText, rules, { path: "output", phase: "output" });
|
||||
interface RawLlmExpectInput {
|
||||
finishReason?: RawValueExpectation;
|
||||
headers?: RawKeyedExpectations;
|
||||
output?: RawContentExpectations;
|
||||
rawFinishReason?: RawValueExpectation;
|
||||
status?: Array<number | string>;
|
||||
stream?: { completed?: boolean; firstTokenMs?: RawValueExpectation };
|
||||
usage?: { inputTokens?: RawValueExpectation; outputTokens?: RawValueExpectation; totalTokens?: RawValueExpectation };
|
||||
}
|
||||
|
||||
function checkOutputRules(outputText: null | string, rawRules: RawContentExpectations | undefined) {
|
||||
return checkContentExpectations(outputText, resolveContentExpectations(rawRules), {
|
||||
path: "output",
|
||||
phase: "output",
|
||||
});
|
||||
}
|
||||
|
||||
function makeObservation(overrides?: Partial<LlmCheckObservation>): LlmCheckObservation {
|
||||
@@ -25,7 +45,31 @@ function makeObservation(overrides?: Partial<LlmCheckObservation>): LlmCheckObse
|
||||
};
|
||||
}
|
||||
|
||||
describe("LLM output rules", () => {
|
||||
function resolveLlmExpect(raw: RawLlmExpectInput | undefined): ResolvedLlmExpectConfig | undefined {
|
||||
if (raw === undefined) return undefined;
|
||||
return {
|
||||
finishReason: resolveValueExpectation(raw.finishReason),
|
||||
headers: resolveKeyedExpectations(raw.headers),
|
||||
output: resolveContentExpectations(raw.output),
|
||||
rawFinishReason: resolveValueExpectation(raw.rawFinishReason),
|
||||
status: raw.status ?? [200],
|
||||
stream: raw.stream
|
||||
? {
|
||||
completed: raw.stream.completed ?? true,
|
||||
firstTokenMs: resolveValueExpectation(raw.stream.firstTokenMs),
|
||||
}
|
||||
: undefined,
|
||||
usage: raw.usage
|
||||
? {
|
||||
inputTokens: resolveValueExpectation(raw.usage.inputTokens),
|
||||
outputTokens: resolveValueExpectation(raw.usage.outputTokens),
|
||||
totalTokens: resolveValueExpectation(raw.usage.totalTokens),
|
||||
}
|
||||
: undefined,
|
||||
};
|
||||
}
|
||||
|
||||
describe("LLM output expectations", () => {
|
||||
test("equals 严格匹配", () => {
|
||||
expect(checkOutputRules("OK", [{ equals: "OK" }]).matched).toBe(true);
|
||||
expect(checkOutputRules("OK\n", [{ equals: "OK" }]).matched).toBe(false);
|
||||
@@ -66,7 +110,7 @@ describe("LLM output rules", () => {
|
||||
expect(result.failure?.phase).toBe("output");
|
||||
});
|
||||
|
||||
test("undefined rules 返回通过", () => {
|
||||
test("undefined expectations 返回通过", () => {
|
||||
expect(checkOutputRules("anything", undefined).matched).toBe(true);
|
||||
expect(checkOutputRules(null, undefined).matched).toBe(true);
|
||||
});
|
||||
@@ -75,11 +119,14 @@ describe("LLM output rules", () => {
|
||||
describe("LLM runExpects", () => {
|
||||
test("全部 expect 通过", () => {
|
||||
const observation = makeObservation();
|
||||
const result = runExpects(observation, {
|
||||
const result = runExpects(
|
||||
observation,
|
||||
resolveLlmExpect({
|
||||
finishReason: { equals: "stop" },
|
||||
output: [{ contains: "OK" }],
|
||||
status: [200],
|
||||
});
|
||||
}),
|
||||
);
|
||||
expect(result.matched).toBe(true);
|
||||
expect(result.failure).toBeNull();
|
||||
});
|
||||
@@ -92,35 +139,35 @@ describe("LLM runExpects", () => {
|
||||
|
||||
test("status 不匹配失败", () => {
|
||||
const observation = makeObservation();
|
||||
const result = runExpects(observation, { status: [404] });
|
||||
const result = runExpects(observation, resolveLlmExpect({ status: [404] }));
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("status");
|
||||
});
|
||||
|
||||
test("finishReason 不匹配失败", () => {
|
||||
const observation = makeObservation();
|
||||
const result = runExpects(observation, { finishReason: { equals: "length" } });
|
||||
const result = runExpects(observation, resolveLlmExpect({ finishReason: { equals: "length" } }));
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("finishReason");
|
||||
});
|
||||
|
||||
test("rawFinishReason 不匹配失败", () => {
|
||||
const observation = makeObservation();
|
||||
const result = runExpects(observation, { rawFinishReason: { equals: "end_turn" } });
|
||||
const result = runExpects(observation, resolveLlmExpect({ rawFinishReason: { equals: "end_turn" } }));
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("rawFinishReason");
|
||||
});
|
||||
|
||||
test("usage 不匹配失败", () => {
|
||||
const observation = makeObservation();
|
||||
const result = runExpects(observation, { usage: { totalTokens: { gte: 100 } } });
|
||||
const result = runExpects(observation, resolveLlmExpect({ usage: { totalTokens: { gte: 100 } } }));
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("usage");
|
||||
});
|
||||
|
||||
test("usage 匹配通过", () => {
|
||||
const observation = makeObservation();
|
||||
const result = runExpects(observation, { usage: { totalTokens: { lte: 20 } } });
|
||||
const result = runExpects(observation, resolveLlmExpect({ usage: { totalTokens: { lte: 20 } } }));
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
@@ -129,9 +176,12 @@ describe("LLM runExpects", () => {
|
||||
mode: "stream",
|
||||
stream: { completed: true, firstTokenMs: 500 },
|
||||
});
|
||||
const result = runExpects(observation, {
|
||||
const result = runExpects(
|
||||
observation,
|
||||
resolveLlmExpect({
|
||||
stream: { completed: true },
|
||||
});
|
||||
}),
|
||||
);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
@@ -140,9 +190,12 @@ describe("LLM runExpects", () => {
|
||||
mode: "stream",
|
||||
stream: { completed: true, firstTokenMs: 500 },
|
||||
});
|
||||
const result = runExpects(observation, {
|
||||
const result = runExpects(
|
||||
observation,
|
||||
resolveLlmExpect({
|
||||
stream: { firstTokenMs: { lte: 1000 } },
|
||||
});
|
||||
}),
|
||||
);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
@@ -151,9 +204,12 @@ describe("LLM runExpects", () => {
|
||||
mode: "stream",
|
||||
stream: { completed: true, firstTokenMs: null },
|
||||
});
|
||||
const result = runExpects(observation, {
|
||||
const result = runExpects(
|
||||
observation,
|
||||
resolveLlmExpect({
|
||||
stream: { firstTokenMs: { lte: 1000 } },
|
||||
});
|
||||
}),
|
||||
);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("stream");
|
||||
});
|
||||
@@ -162,9 +218,12 @@ describe("LLM runExpects", () => {
|
||||
const observation = makeObservation({
|
||||
http: { headers: { "content-type": "application/json" }, status: 200, statusText: "OK" },
|
||||
});
|
||||
const result = runExpects(observation, {
|
||||
const result = runExpects(
|
||||
observation,
|
||||
resolveLlmExpect({
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
}),
|
||||
);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
@@ -172,19 +231,25 @@ describe("LLM runExpects", () => {
|
||||
const observation = makeObservation({
|
||||
http: { headers: { "content-type": "text/plain" }, status: 200, statusText: "OK" },
|
||||
});
|
||||
const result = runExpects(observation, {
|
||||
const result = runExpects(
|
||||
observation,
|
||||
resolveLlmExpect({
|
||||
headers: { "content-type": "application/json" },
|
||||
});
|
||||
}),
|
||||
);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("headers");
|
||||
});
|
||||
|
||||
test("首个 expect 失败立即返回", () => {
|
||||
const observation = makeObservation();
|
||||
const result = runExpects(observation, {
|
||||
const result = runExpects(
|
||||
observation,
|
||||
resolveLlmExpect({
|
||||
output: [{ contains: "OK" }],
|
||||
status: [404],
|
||||
});
|
||||
}),
|
||||
);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.phase).toBe("status");
|
||||
});
|
||||
@@ -196,7 +261,7 @@ describe("LLM runExpects", () => {
|
||||
outputText: null,
|
||||
usage: null,
|
||||
});
|
||||
const result = runExpects(observation, { status: [401] });
|
||||
const result = runExpects(observation, resolveLlmExpect({ status: [401] }));
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -214,7 +214,7 @@ describe("LlmChecker validate", () => {
|
||||
defaults: {},
|
||||
targets: [makeRawTarget({ expect: { output: [{ contains: "y", json: { path: "$.status" } }] } })],
|
||||
});
|
||||
expect(issues.some((i) => i.code === "invalid-content-rule")).toBe(true);
|
||||
expect(issues.some((i) => i.code === "invalid-content-expectation")).toBe(true);
|
||||
});
|
||||
|
||||
test("expect.output regex ReDoS 报错", () => {
|
||||
@@ -284,6 +284,44 @@ describe("LlmChecker resolve", () => {
|
||||
expect(resolved.group).toBe("default");
|
||||
expect(resolved.intervalMs).toBe(30000);
|
||||
expect(resolved.timeoutMs).toBe(10000);
|
||||
expect(resolved.rawExpect).toBeUndefined();
|
||||
expect(resolved.expect).toEqual({ status: [200] });
|
||||
});
|
||||
|
||||
test("stream mode 未配置 expect.stream 时不物化 completed", () => {
|
||||
const raw = makeRawTarget({
|
||||
expect: { output: [{ contains: "OK" }] },
|
||||
llm: {
|
||||
mode: "stream",
|
||||
model: "gpt-4o-mini",
|
||||
prompt: "Say OK",
|
||||
provider: "openai",
|
||||
url: "https://api.openai.com/v1",
|
||||
},
|
||||
});
|
||||
|
||||
const resolved = asLlm(checker.resolve(raw, makeResolveContext()));
|
||||
|
||||
expect(resolved.rawExpect).toEqual({ output: [{ contains: "OK" }] });
|
||||
expect(resolved.expect?.stream).toBeUndefined();
|
||||
});
|
||||
|
||||
test("配置 expect.stream 但省略 completed 时默认 true", () => {
|
||||
const raw = makeRawTarget({
|
||||
expect: { stream: { firstTokenMs: 100 } },
|
||||
llm: {
|
||||
mode: "stream",
|
||||
model: "gpt-4o-mini",
|
||||
prompt: "Say OK",
|
||||
provider: "openai",
|
||||
url: "https://api.openai.com/v1",
|
||||
},
|
||||
});
|
||||
|
||||
const resolved = asLlm(checker.resolve(raw, makeResolveContext()));
|
||||
|
||||
expect(resolved.rawExpect).toEqual({ stream: { firstTokenMs: 100 } });
|
||||
expect(resolved.expect?.stream).toEqual({ completed: true, firstTokenMs: { equals: 100 } });
|
||||
});
|
||||
|
||||
test("defaults.llm 与 target.llm 浅合并", () => {
|
||||
|
||||
@@ -1,12 +1,15 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import { checkContentRules } from "../../../../../src/server/checker/expect/content";
|
||||
import type { RawContentExpectations } from "../../../../../src/server/checker/expect/types";
|
||||
|
||||
function checkBodyExpect(body: string, rules?: Parameters<typeof checkContentRules>[1]) {
|
||||
return checkContentRules(body, rules, { path: "body", phase: "body" });
|
||||
import { checkContentExpectations, resolveContentExpectations } from "../../../../../src/server/checker/expect/content";
|
||||
|
||||
function checkBodyExpect(body: string, rawExpectations?: RawContentExpectations) {
|
||||
const resolved = resolveContentExpectations(rawExpectations);
|
||||
return checkContentExpectations(body, resolved, { path: "body", phase: "body" });
|
||||
}
|
||||
|
||||
describe("checkBodyExpect (BodyRule[])", () => {
|
||||
describe("checkBodyExpect (ContentExpectations)", () => {
|
||||
test("无规则返回匹配成功", () => {
|
||||
const r = checkBodyExpect("anything");
|
||||
expect(r.matched).toBe(true);
|
||||
@@ -19,6 +22,22 @@ describe("checkBodyExpect (BodyRule[])", () => {
|
||||
expect(r.failure).toBeNull();
|
||||
});
|
||||
|
||||
test("resolve 输出显式 kind union 并物化 extractor 默认 exists", () => {
|
||||
expect(
|
||||
resolveContentExpectations([
|
||||
{ contains: "ok" },
|
||||
{ json: { path: "$.status" } },
|
||||
{ css: { attr: "content", selector: "meta[name=status]" } },
|
||||
{ xpath: { path: "/root/status" } },
|
||||
]),
|
||||
).toEqual([
|
||||
{ kind: "value", matcher: { contains: "ok" } },
|
||||
{ kind: "json", matcher: { exists: true }, path: "$.status" },
|
||||
{ attr: "content", kind: "css", matcher: { exists: true }, selector: "meta[name=status]" },
|
||||
{ kind: "xpath", matcher: { exists: true }, path: "/root/status" },
|
||||
]);
|
||||
});
|
||||
|
||||
test("contains 规则匹配成功", () => {
|
||||
const r = checkBodyExpect("hello world", [{ contains: "hello" }]);
|
||||
expect(r.matched).toBe(true);
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import type { CheckerValidationInput } from "../../../../../src/server/checker/runner/types";
|
||||
|
||||
import { validateDbConfig } from "../../../../../src/server/checker/runner/db/validate";
|
||||
import { validateHttpConfig } from "../../../../../src/server/checker/runner/http/validate";
|
||||
import { validateLlmConfig } from "../../../../../src/server/checker/runner/llm/validate";
|
||||
|
||||
function input(target: Record<string, unknown>): CheckerValidationInput {
|
||||
return { defaults: {}, targets: [target as CheckerValidationInput["targets"][number]] };
|
||||
}
|
||||
|
||||
describe("HTTP/LLM headers reject case-insensitive duplicate keys", () => {
|
||||
test("HTTP headers 大小写不同的重复 key 报错", () => {
|
||||
const target = {
|
||||
expect: { headers: { "Content-Type": "application/json", "content-type": "text/plain" } },
|
||||
http: { url: "https://example.com" },
|
||||
id: "dup",
|
||||
type: "http",
|
||||
};
|
||||
|
||||
const issues = validateHttpConfig(input(target));
|
||||
expect(issues.some((i) => i.code === "duplicate-key" && i.path.includes("headers"))).toBe(true);
|
||||
});
|
||||
|
||||
test("LLM headers 大小写不同的重复 key 报错", () => {
|
||||
const target = {
|
||||
expect: { headers: { "X-Trace": "a", "x-trace": "b" } },
|
||||
id: "dup",
|
||||
llm: {
|
||||
mode: "stream",
|
||||
model: "test-model",
|
||||
prompt: "hello",
|
||||
provider: "openai",
|
||||
url: "https://example.com/v1/chat/completions",
|
||||
},
|
||||
type: "llm",
|
||||
};
|
||||
|
||||
const issues = validateLlmConfig(input(target));
|
||||
expect(issues.some((i) => i.code === "duplicate-key" && i.path.includes("headers"))).toBe(true);
|
||||
});
|
||||
|
||||
test("HTTP headers 不同 key 不触发 duplicate-key", () => {
|
||||
const target = {
|
||||
expect: { headers: { Accept: "application/json", "Content-Type": "application/json" } },
|
||||
http: { url: "https://example.com" },
|
||||
id: "ok",
|
||||
type: "http",
|
||||
};
|
||||
|
||||
expect(validateHttpConfig(input(target)).some((i) => i.code === "duplicate-key")).toBe(false);
|
||||
});
|
||||
|
||||
test("DB rows 保留大小写敏感不触发 duplicate-key", () => {
|
||||
const target = {
|
||||
db: { query: "SELECT 1", url: "sqlite://:memory:" },
|
||||
expect: { rows: [{ Name: "a", name: "b" }] },
|
||||
id: "dup-rows",
|
||||
type: "db",
|
||||
};
|
||||
|
||||
expect(validateDbConfig(input(target)).some((i) => i.code === "duplicate-key")).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -1,9 +1,9 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import { checkValueMatcher } from "../../../../../src/server/checker/expect/matcher";
|
||||
import { checkValueExpectation } from "../../../../../src/server/checker/expect/value";
|
||||
|
||||
function checkDuration(durationMs: number, maxDurationMs?: number) {
|
||||
return checkValueMatcher(durationMs, maxDurationMs === undefined ? undefined : { lte: maxDurationMs }, {
|
||||
return checkValueExpectation(durationMs, maxDurationMs === undefined ? undefined : { lte: maxDurationMs }, {
|
||||
path: "durationMs",
|
||||
phase: "duration",
|
||||
});
|
||||
|
||||
34
tests/server/checker/runner/shared/keyed.test.ts
Normal file
34
tests/server/checker/runner/shared/keyed.test.ts
Normal file
@@ -0,0 +1,34 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import { checkKeyedExpectations, resolveKeyedExpectations } from "../../../../../src/server/checker/expect/keyed";
|
||||
|
||||
describe("KeyedExpectations", () => {
|
||||
test("resolve 将 Raw Record 转为保持顺序的有序数组", () => {
|
||||
expect(resolveKeyedExpectations({ Count: { gte: 1 }, Name: "alice" })).toEqual([
|
||||
{ key: "Count", matcher: { gte: 1 } },
|
||||
{ key: "Name", matcher: { equals: "alice" } },
|
||||
]);
|
||||
});
|
||||
|
||||
test("failure.path 包含基础路径和原始 key", () => {
|
||||
const result = checkKeyedExpectations(
|
||||
{ "content-type": "text/plain" },
|
||||
resolveKeyedExpectations({ "Content-Type": { contains: "json" } }),
|
||||
{ normalizeKey: (key) => key.toLowerCase(), path: "headers", phase: "headers" },
|
||||
);
|
||||
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.path).toBe("headers.Content-Type");
|
||||
});
|
||||
|
||||
test("DB rows 默认保持大小写敏感匹配", () => {
|
||||
const expectations = resolveKeyedExpectations({ Name: "alice", name: "bob" });
|
||||
|
||||
expect(
|
||||
checkKeyedExpectations({ Name: "alice", name: "bob" }, expectations, { path: "rows[0]", phase: "rows" }).matched,
|
||||
).toBe(true);
|
||||
expect(
|
||||
checkKeyedExpectations({ Name: "bob", name: "alice" }, expectations, { path: "rows[0]", phase: "rows" }).matched,
|
||||
).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -1,6 +1,12 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import { applyMatcher, checkExpectValue, evaluateJsonPath } from "../../../../../src/server/checker/expect/matcher";
|
||||
import {
|
||||
applyValueMatcher,
|
||||
checkValueExpectation,
|
||||
displayValueExpectation,
|
||||
evaluateJsonPath,
|
||||
resolveValueExpectation,
|
||||
} from "../../../../../src/server/checker/expect/value";
|
||||
|
||||
describe("evaluateJsonPath", () => {
|
||||
const obj = {
|
||||
@@ -55,97 +61,115 @@ describe("evaluateJsonPath", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("applyMatcher", () => {
|
||||
describe("applyValueMatcher", () => {
|
||||
test("equals 操作符", () => {
|
||||
expect(applyMatcher("ok", { equals: "ok" })).toBe(true);
|
||||
expect(applyMatcher("ok", { equals: "error" })).toBe(false);
|
||||
expect(applyMatcher(42, { equals: 42 })).toBe(true);
|
||||
expect(applyMatcher(42, { equals: 41 })).toBe(false);
|
||||
expect(applyMatcher(null, { equals: null })).toBe(true);
|
||||
expect(applyMatcher(true, { equals: true })).toBe(true);
|
||||
expect(applyValueMatcher("ok", { equals: "ok" })).toBe(true);
|
||||
expect(applyValueMatcher("ok", { equals: "error" })).toBe(false);
|
||||
expect(applyValueMatcher(42, { equals: 42 })).toBe(true);
|
||||
expect(applyValueMatcher(42, { equals: 41 })).toBe(false);
|
||||
expect(applyValueMatcher(null, { equals: null })).toBe(true);
|
||||
expect(applyValueMatcher(true, { equals: true })).toBe(true);
|
||||
});
|
||||
|
||||
test("equals 支持 JSON 对象和数组", () => {
|
||||
expect(applyMatcher({ status: "ok" }, { equals: { status: "ok" } })).toBe(true);
|
||||
expect(applyMatcher({ status: "ok" }, { equals: { status: "fail" } })).toBe(false);
|
||||
expect(applyMatcher(["a", "b"], { equals: ["a", "b"] })).toBe(true);
|
||||
expect(applyMatcher(["a", "b"], { equals: ["b", "a"] })).toBe(false);
|
||||
expect(applyValueMatcher({ status: "ok" }, { equals: { status: "ok" } })).toBe(true);
|
||||
expect(applyValueMatcher({ status: "ok" }, { equals: { status: "fail" } })).toBe(false);
|
||||
expect(applyValueMatcher(["a", "b"], { equals: ["a", "b"] })).toBe(true);
|
||||
expect(applyValueMatcher(["a", "b"], { equals: ["b", "a"] })).toBe(false);
|
||||
});
|
||||
|
||||
test("contains 操作符", () => {
|
||||
expect(applyMatcher("hello world", { contains: "hello" })).toBe(true);
|
||||
expect(applyMatcher("hello world", { contains: "missing" })).toBe(false);
|
||||
expect(applyMatcher(12345, { contains: "23" })).toBe(true);
|
||||
expect(applyValueMatcher("hello world", { contains: "hello" })).toBe(true);
|
||||
expect(applyValueMatcher("hello world", { contains: "missing" })).toBe(false);
|
||||
expect(applyValueMatcher(12345, { contains: "23" })).toBe(true);
|
||||
});
|
||||
|
||||
test("regex matcher", () => {
|
||||
expect(applyMatcher("v2.1.0", { regex: "\\d+\\.\\d+\\.\\d+" })).toBe(true);
|
||||
expect(applyMatcher("v2.1", { regex: "\\d+\\.\\d+\\.\\d+" })).toBe(false);
|
||||
expect(applyMatcher("abc123", { regex: "^\\w+\\d+$" })).toBe(true);
|
||||
expect(applyValueMatcher("v2.1.0", { regex: "\\d+\\.\\d+\\.\\d+" })).toBe(true);
|
||||
expect(applyValueMatcher("v2.1", { regex: "\\d+\\.\\d+\\.\\d+" })).toBe(false);
|
||||
expect(applyValueMatcher("abc123", { regex: "^\\w+\\d+$" })).toBe(true);
|
||||
});
|
||||
|
||||
test("empty 操作符", () => {
|
||||
expect(applyMatcher("", { empty: true })).toBe(true);
|
||||
expect(applyMatcher(null, { empty: true })).toBe(true);
|
||||
expect(applyMatcher(undefined, { empty: true })).toBe(true);
|
||||
expect(applyMatcher([], { empty: true })).toBe(true);
|
||||
expect(applyMatcher({}, { empty: true })).toBe(true);
|
||||
expect(applyMatcher("ok", { empty: true })).toBe(false);
|
||||
expect(applyMatcher(0, { empty: true })).toBe(false);
|
||||
expect(applyMatcher(false, { empty: true })).toBe(false);
|
||||
expect(applyMatcher([1, 2], { empty: false })).toBe(true);
|
||||
expect(applyMatcher([], { empty: false })).toBe(false);
|
||||
expect(applyValueMatcher("", { empty: true })).toBe(true);
|
||||
expect(applyValueMatcher(null, { empty: true })).toBe(true);
|
||||
expect(applyValueMatcher(undefined, { empty: true })).toBe(true);
|
||||
expect(applyValueMatcher([], { empty: true })).toBe(true);
|
||||
expect(applyValueMatcher({}, { empty: true })).toBe(true);
|
||||
expect(applyValueMatcher("ok", { empty: true })).toBe(false);
|
||||
expect(applyValueMatcher(0, { empty: true })).toBe(false);
|
||||
expect(applyValueMatcher(false, { empty: true })).toBe(false);
|
||||
expect(applyValueMatcher([1, 2], { empty: false })).toBe(true);
|
||||
expect(applyValueMatcher([], { empty: false })).toBe(false);
|
||||
});
|
||||
|
||||
test("exists 操作符", () => {
|
||||
expect(applyMatcher("ok", { exists: true })).toBe(true);
|
||||
expect(applyMatcher(null, { exists: true })).toBe(true);
|
||||
expect(applyMatcher(undefined, { exists: true })).toBe(false);
|
||||
expect(applyMatcher(undefined, { exists: false })).toBe(true);
|
||||
expect(applyMatcher("ok", { exists: false })).toBe(false);
|
||||
expect(applyValueMatcher("ok", { exists: true })).toBe(true);
|
||||
expect(applyValueMatcher(null, { exists: true })).toBe(true);
|
||||
expect(applyValueMatcher(undefined, { exists: true })).toBe(false);
|
||||
expect(applyValueMatcher(undefined, { exists: false })).toBe(true);
|
||||
expect(applyValueMatcher("ok", { exists: false })).toBe(false);
|
||||
});
|
||||
|
||||
test("gte 操作符", () => {
|
||||
expect(applyMatcher(10, { gte: 5 })).toBe(true);
|
||||
expect(applyMatcher(5, { gte: 5 })).toBe(true);
|
||||
expect(applyMatcher(3, { gte: 5 })).toBe(false);
|
||||
expect(applyMatcher("10", { gte: 5 })).toBe(true);
|
||||
expect(applyValueMatcher(10, { gte: 5 })).toBe(true);
|
||||
expect(applyValueMatcher(5, { gte: 5 })).toBe(true);
|
||||
expect(applyValueMatcher(3, { gte: 5 })).toBe(false);
|
||||
expect(applyValueMatcher("10", { gte: 5 })).toBe(true);
|
||||
});
|
||||
|
||||
test("lte 操作符", () => {
|
||||
expect(applyMatcher(3, { lte: 5 })).toBe(true);
|
||||
expect(applyMatcher(5, { lte: 5 })).toBe(true);
|
||||
expect(applyMatcher(10, { lte: 5 })).toBe(false);
|
||||
expect(applyValueMatcher(3, { lte: 5 })).toBe(true);
|
||||
expect(applyValueMatcher(5, { lte: 5 })).toBe(true);
|
||||
expect(applyValueMatcher(10, { lte: 5 })).toBe(false);
|
||||
});
|
||||
|
||||
test("gt 操作符", () => {
|
||||
expect(applyMatcher(10, { gt: 5 })).toBe(true);
|
||||
expect(applyMatcher(5, { gt: 5 })).toBe(false);
|
||||
expect(applyValueMatcher(10, { gt: 5 })).toBe(true);
|
||||
expect(applyValueMatcher(5, { gt: 5 })).toBe(false);
|
||||
});
|
||||
|
||||
test("lt 操作符", () => {
|
||||
expect(applyMatcher(3, { lt: 5 })).toBe(true);
|
||||
expect(applyMatcher(5, { lt: 5 })).toBe(false);
|
||||
expect(applyValueMatcher(3, { lt: 5 })).toBe(true);
|
||||
expect(applyValueMatcher(5, { lt: 5 })).toBe(false);
|
||||
});
|
||||
|
||||
test("多操作符 AND 组合", () => {
|
||||
expect(applyMatcher(7, { gte: 5, lte: 10 })).toBe(true);
|
||||
expect(applyMatcher(3, { gte: 5, lte: 10 })).toBe(false);
|
||||
expect(applyMatcher(15, { gte: 5, lte: 10 })).toBe(false);
|
||||
expect(applyValueMatcher(7, { gte: 5, lte: 10 })).toBe(true);
|
||||
expect(applyValueMatcher(3, { gte: 5, lte: 10 })).toBe(false);
|
||||
expect(applyValueMatcher(15, { gte: 5, lte: 10 })).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("checkExpectValue", () => {
|
||||
test("原始值直接比较", () => {
|
||||
expect(checkExpectValue("ok", "ok")).toBe(true);
|
||||
expect(checkExpectValue("ok", "error")).toBe(false);
|
||||
expect(checkExpectValue(42, 42)).toBe(true);
|
||||
expect(checkExpectValue(null, null)).toBe(true);
|
||||
describe("resolveValueExpectation", () => {
|
||||
test("原始值解析为 equals matcher", () => {
|
||||
expect(resolveValueExpectation("ok")).toEqual({ equals: "ok" });
|
||||
expect(resolveValueExpectation(42)).toEqual({ equals: 42 });
|
||||
expect(resolveValueExpectation(null)).toEqual({ equals: null });
|
||||
expect(resolveValueExpectation(true)).toEqual({ equals: true });
|
||||
});
|
||||
|
||||
test("对象作为操作符", () => {
|
||||
expect(checkExpectValue(42, { gte: 10 })).toBe(true);
|
||||
expect(checkExpectValue(42, { gte: 100 })).toBe(false);
|
||||
expect(checkExpectValue("hello", { contains: "ell" })).toBe(true);
|
||||
test("对象 matcher 原样保留", () => {
|
||||
const matcher = { gte: 10 };
|
||||
expect(resolveValueExpectation(matcher)).toBe(matcher);
|
||||
expect(resolveValueExpectation({ contains: "ell" })).toEqual({ contains: "ell" });
|
||||
});
|
||||
|
||||
test("undefined 返回 undefined", () => {
|
||||
expect(resolveValueExpectation(undefined)).toBeUndefined();
|
||||
});
|
||||
|
||||
test("failure expected 使用用户可读的 equals 值", () => {
|
||||
const result = checkValueExpectation("actual", resolveValueExpectation("expected"), {
|
||||
path: "finishReason",
|
||||
phase: "finishReason",
|
||||
});
|
||||
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure?.expected).toBe("expected");
|
||||
});
|
||||
|
||||
test("displayValueExpectation 保留多 matcher 对象", () => {
|
||||
expect(displayValueExpectation({ contains: "ok", regex: "^ok$" })).toEqual({ contains: "ok", regex: "^ok$" });
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,9 +1,12 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import { checkContentRules } from "../../../../../src/server/checker/expect/content";
|
||||
import type { RawContentExpectations } from "../../../../../src/server/checker/expect/types";
|
||||
|
||||
function checkTextRules(text: string, rules: Parameters<typeof checkContentRules>[1], phase: string) {
|
||||
return checkContentRules(text, rules, { path: phase, phase });
|
||||
import { checkContentExpectations, resolveContentExpectations } from "../../../../../src/server/checker/expect/content";
|
||||
|
||||
function checkTextRules(text: string, rawRules: RawContentExpectations, phase: string) {
|
||||
const resolved = resolveContentExpectations(rawRules);
|
||||
return checkContentExpectations(text, resolved, { path: phase, phase });
|
||||
}
|
||||
|
||||
describe("checkTextRules", () => {
|
||||
|
||||
@@ -15,7 +15,7 @@ function input(target: Record<string, unknown>): CheckerValidationInput {
|
||||
}
|
||||
|
||||
describe("ValueMatcher primitive shorthand in checker validators", () => {
|
||||
test("normalizes shorthand for all checker ValueMatcher fields", () => {
|
||||
test("accepts shorthand for all checker ValueMatcher fields", () => {
|
||||
const targets = [
|
||||
{
|
||||
expect: { durationMs: 100 },
|
||||
@@ -82,9 +82,10 @@ describe("ValueMatcher primitive shorthand in checker validators", () => {
|
||||
|
||||
for (const target of targets) {
|
||||
const { validate, ...config } = target;
|
||||
const original = structuredClone(config);
|
||||
|
||||
expect(validate(input(config))).toHaveLength(0);
|
||||
expect((config.expect as Record<string, unknown>)["durationMs"]).toEqual({ equals: 100 });
|
||||
expect(config).toEqual(original);
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
@@ -1,8 +1,14 @@
|
||||
import { afterAll, beforeAll, describe, expect, test } from "bun:test";
|
||||
|
||||
import type { ResolvedTcpTarget } from "../../../../../src/server/checker/runner/tcp/types";
|
||||
import type {
|
||||
RawTcpExpectConfig,
|
||||
ResolvedTcpExpectConfig,
|
||||
ResolvedTcpTarget,
|
||||
} from "../../../../../src/server/checker/runner/tcp/types";
|
||||
import type { CheckerContext } from "../../../../../src/server/checker/runner/types";
|
||||
|
||||
import { resolveContentExpectations } from "../../../../../src/server/checker/expect/content";
|
||||
import { resolveValueExpectation } from "../../../../../src/server/checker/expect/value";
|
||||
import { TcpChecker } from "../../../../../src/server/checker/runner/tcp/execute";
|
||||
|
||||
const checker = new TcpChecker();
|
||||
@@ -20,13 +26,27 @@ function makeCtx(timeoutMs = 5000): CheckerContext {
|
||||
return { signal: controller.signal };
|
||||
}
|
||||
|
||||
function makeTarget(tcp: Partial<ResolvedTcpTarget["tcp"]>, overrides?: Partial<ResolvedTcpTarget>): ResolvedTcpTarget {
|
||||
function makeTarget(
|
||||
tcp: Partial<ResolvedTcpTarget["tcp"]>,
|
||||
overrides?: { expect?: RawTcpExpectConfig },
|
||||
): ResolvedTcpTarget {
|
||||
const raw = overrides?.expect;
|
||||
const resolvedExpect: ResolvedTcpExpectConfig | undefined = raw
|
||||
? {
|
||||
banner: resolveContentExpectations(raw.banner),
|
||||
connected: raw.connected ?? true,
|
||||
durationMs: resolveValueExpectation(raw.durationMs),
|
||||
}
|
||||
: undefined;
|
||||
|
||||
return {
|
||||
description: null,
|
||||
expect: resolvedExpect,
|
||||
group: "default",
|
||||
id: "test-tcp",
|
||||
intervalMs: 60000,
|
||||
name: "test-tcp",
|
||||
rawExpect: raw,
|
||||
tcp: {
|
||||
bannerReadTimeout: 2000,
|
||||
host: "127.0.0.1",
|
||||
@@ -37,7 +57,6 @@ function makeTarget(tcp: Partial<ResolvedTcpTarget["tcp"]>, overrides?: Partial<
|
||||
},
|
||||
timeoutMs: 5000,
|
||||
type: "tcp",
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -295,6 +314,8 @@ describe("TcpChecker resolve", () => {
|
||||
expect(target.name).toBeNull();
|
||||
expect(target.intervalMs).toBe(30000);
|
||||
expect(target.timeoutMs).toBe(10000);
|
||||
expect(target.rawExpect).toBeUndefined();
|
||||
expect(target.expect).toEqual({ connected: true });
|
||||
});
|
||||
|
||||
test("bannerReadTimeout 和 maxBannerBytes 支持 per-target 覆盖", () => {
|
||||
@@ -356,7 +377,12 @@ describe("TcpChecker resolve", () => {
|
||||
},
|
||||
{ configDir: "/tmp", defaultIntervalMs: 30000, defaults: {}, defaultTimeoutMs: 10000 },
|
||||
);
|
||||
expect(target.expect).toEqual({ banner: [{ contains: "ESMTP" }], connected: false, durationMs: { lte: 5000 } });
|
||||
expect(target.expect).toEqual({
|
||||
banner: [{ kind: "value", matcher: { contains: "ESMTP" } }],
|
||||
connected: false,
|
||||
durationMs: { lte: 5000 },
|
||||
});
|
||||
expect(target.rawExpect).toEqual({ banner: [{ contains: "ESMTP" }], connected: false, durationMs: { lte: 5000 } });
|
||||
});
|
||||
|
||||
test("name 和 group 解析", () => {
|
||||
|
||||
@@ -1,7 +1,13 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
|
||||
import type { ContentExpectations } from "../../../../../src/server/checker/expect/types";
|
||||
|
||||
import { checkBanner, checkConnected } from "../../../../../src/server/checker/runner/tcp/expect";
|
||||
|
||||
function value(matcher: ContentExpectations[number]["matcher"]): ContentExpectations[number] {
|
||||
return { kind: "value", matcher };
|
||||
}
|
||||
|
||||
describe("checkConnected", () => {
|
||||
test("connected=true 期望 true 匹配", () => {
|
||||
const result = checkConnected(true, true);
|
||||
@@ -32,34 +38,34 @@ describe("checkConnected", () => {
|
||||
|
||||
describe("checkBanner", () => {
|
||||
test("contains 匹配", () => {
|
||||
const result = checkBanner("220 smtp.example.com ESMTP", [{ contains: "ESMTP" }]);
|
||||
const result = checkBanner("220 smtp.example.com ESMTP", [value({ contains: "ESMTP" })]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("contains 不匹配", () => {
|
||||
const result = checkBanner("220 smtp.example.com ESMTP", [{ contains: "POSTFIX" }]);
|
||||
const result = checkBanner("220 smtp.example.com ESMTP", [value({ contains: "POSTFIX" })]);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.kind).toBe("mismatch");
|
||||
expect(result.failure!.phase).toBe("banner");
|
||||
});
|
||||
|
||||
test("regex 正则匹配", () => {
|
||||
const result = checkBanner("220 smtp.example.com ESMTP", [{ regex: "^220" }]);
|
||||
const result = checkBanner("220 smtp.example.com ESMTP", [value({ regex: "^220" })]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("空 banner 与 contains 空字符串", () => {
|
||||
const result = checkBanner("", [{ contains: "" }]);
|
||||
const result = checkBanner("", [value({ contains: "" })]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("多 operator 同时匹配", () => {
|
||||
const result = checkBanner("220 ESMTP", [{ contains: "ESMTP", regex: "^220" }]);
|
||||
const result = checkBanner("220 ESMTP", [value({ contains: "ESMTP", regex: "^220" })]);
|
||||
expect(result.matched).toBe(true);
|
||||
});
|
||||
|
||||
test("多 operator 部分不匹配", () => {
|
||||
const result = checkBanner("220 ESMTP", [{ contains: "ESMTP", regex: "^250" }]);
|
||||
const result = checkBanner("220 ESMTP", [value({ contains: "ESMTP", regex: "^250" })]);
|
||||
expect(result.matched).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,7 +1,13 @@
|
||||
import { describe, expect, it } from "bun:test";
|
||||
|
||||
import type { ResolvedUdpTarget, UdpExpectConfig } from "../../../../../src/server/checker/runner/udp/types";
|
||||
import type {
|
||||
RawUdpExpectConfig,
|
||||
ResolvedUdpExpectConfig,
|
||||
ResolvedUdpTarget,
|
||||
} from "../../../../../src/server/checker/runner/udp/types";
|
||||
|
||||
import { resolveContentExpectations } from "../../../../../src/server/checker/expect/content";
|
||||
import { resolveValueExpectation } from "../../../../../src/server/checker/expect/value";
|
||||
import { UdpChecker } from "../../../../../src/server/checker/runner/udp/execute";
|
||||
|
||||
async function createEchoServer(): Promise<{ close: () => void; port: number }> {
|
||||
@@ -27,14 +33,26 @@ function makeSignal(timeoutMs: number): { cleanup: () => void; signal: AbortSign
|
||||
return { cleanup: () => clearTimeout(timer), signal: controller.signal };
|
||||
}
|
||||
|
||||
function makeTarget(overrides: Partial<ResolvedUdpTarget["udp"]> = {}, expect?: UdpExpectConfig): ResolvedUdpTarget {
|
||||
function makeTarget(overrides: Partial<ResolvedUdpTarget["udp"]> = {}, raw?: RawUdpExpectConfig): ResolvedUdpTarget {
|
||||
const resolvedExpect: ResolvedUdpExpectConfig | undefined = raw
|
||||
? {
|
||||
durationMs: resolveValueExpectation(raw.durationMs),
|
||||
responded: raw.responded ?? true,
|
||||
response: resolveContentExpectations(raw.response),
|
||||
responseSize: resolveValueExpectation(raw.responseSize),
|
||||
sourceHost: resolveValueExpectation(raw.sourceHost),
|
||||
sourcePort: resolveValueExpectation(raw.sourcePort),
|
||||
}
|
||||
: undefined;
|
||||
|
||||
return {
|
||||
description: null,
|
||||
expect,
|
||||
expect: resolvedExpect,
|
||||
group: "default",
|
||||
id: "test-udp",
|
||||
intervalMs: 30000,
|
||||
name: null,
|
||||
rawExpect: raw,
|
||||
timeoutMs: 10000,
|
||||
type: "udp",
|
||||
udp: {
|
||||
@@ -322,6 +340,8 @@ describe("UdpChecker resolve", () => {
|
||||
expect(target.udp.encoding).toBe("text");
|
||||
expect(target.udp.responseEncoding).toBe("text");
|
||||
expect(target.udp.maxResponseBytes).toBe(4096);
|
||||
expect(target.rawExpect).toBeUndefined();
|
||||
expect(target.expect).toEqual({ responded: true });
|
||||
});
|
||||
|
||||
it("should use defaults.udp for missing fields", () => {
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
import { describe, expect, it } from "bun:test";
|
||||
|
||||
import type { ContentExpectations } from "../../../../../src/server/checker/expect/types";
|
||||
|
||||
import {
|
||||
checkResponded,
|
||||
checkResponseSize,
|
||||
@@ -8,6 +10,10 @@ import {
|
||||
checkSourcePort,
|
||||
} from "../../../../../src/server/checker/runner/udp/expect";
|
||||
|
||||
function value(matcher: ContentExpectations[number]["matcher"]): ContentExpectations[number] {
|
||||
return { kind: "value", matcher };
|
||||
}
|
||||
|
||||
describe("checkResponded", () => {
|
||||
it("responded=true 期望 true → 匹配", () => {
|
||||
const result = checkResponded(true, true);
|
||||
@@ -63,26 +69,26 @@ describe("checkResponseSize", () => {
|
||||
|
||||
describe("checkResponseText", () => {
|
||||
it("单条 contains 匹配", () => {
|
||||
const result = checkResponseText("PONG", [{ contains: "PONG" }]);
|
||||
const result = checkResponseText("PONG", [value({ contains: "PONG" })]);
|
||||
expect(result.matched).toBe(true);
|
||||
expect(result.failure).toBeNull();
|
||||
});
|
||||
|
||||
it("单条 contains 不匹配，phase=response", () => {
|
||||
const result = checkResponseText("PING", [{ contains: "PONG" }]);
|
||||
const result = checkResponseText("PING", [value({ contains: "PONG" })]);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.kind).toBe("mismatch");
|
||||
expect(result.failure!.phase).toBe("response");
|
||||
});
|
||||
|
||||
it("多条规则全部匹配", () => {
|
||||
const result = checkResponseText("hello world", [{ contains: "hello" }, { regex: "^hello" }]);
|
||||
const result = checkResponseText("hello world", [value({ contains: "hello" }), value({ regex: "^hello" })]);
|
||||
expect(result.matched).toBe(true);
|
||||
expect(result.failure).toBeNull();
|
||||
});
|
||||
|
||||
it("多条规则第二条失败 → 不匹配", () => {
|
||||
const result = checkResponseText("hello world", [{ contains: "hello" }, { regex: "^world" }]);
|
||||
const result = checkResponseText("hello world", [value({ contains: "hello" }), value({ regex: "^world" })]);
|
||||
expect(result.matched).toBe(false);
|
||||
expect(result.failure!.phase).toBe("response");
|
||||
expect(result.failure!.path).toBe("response[1]");
|
||||
|
||||
@@ -26,7 +26,7 @@ beforeAll(() => {
|
||||
|
||||
const httpTarget: ResolvedHttpTarget = {
|
||||
description: null,
|
||||
expect: { durationMs: { lte: 3000 }, status: [200] },
|
||||
expect: { body: [{ kind: "value", matcher: { contains: "ok" } }], durationMs: { equals: 3000 }, status: [200] },
|
||||
group: "default",
|
||||
http: {
|
||||
headers: { Accept: "application/json" },
|
||||
@@ -39,6 +39,7 @@ const httpTarget: ResolvedHttpTarget = {
|
||||
id: "test-http",
|
||||
intervalMs: 30000,
|
||||
name: "test-http",
|
||||
rawExpect: { body: [{ contains: "ok" }], durationMs: 3000 },
|
||||
timeoutMs: 10000,
|
||||
type: "http",
|
||||
};
|
||||
@@ -106,7 +107,7 @@ describe("ProbeStore", () => {
|
||||
expect(config.maxRedirects).toBe(0);
|
||||
expect(t.interval_ms).toBe(30000);
|
||||
expect(t.timeout_ms).toBe(10000);
|
||||
expect(JSON.parse(t.expect!)).toEqual({ durationMs: { lte: 3000 }, status: [200] });
|
||||
expect(JSON.parse(t.expect!)).toEqual({ body: [{ contains: "ok" }], durationMs: 3000 });
|
||||
});
|
||||
|
||||
test("cmd target 字段正确", () => {
|
||||
|
||||
@@ -103,7 +103,7 @@ describe("createTargetTableColumns", () => {
|
||||
test("延迟列超过 9999ms 时显示上限文案", () => {
|
||||
const latencyColumn = getColumn(createTargetTableColumns(["http"]), "latestCheck.durationMs");
|
||||
const renderCell = latencyColumn.cell as (params: PrimaryTableCellParams<TargetStatus>) => {
|
||||
props: { children: string; className: string };
|
||||
props: { children: string[]; className: string };
|
||||
};
|
||||
const element = renderCell({
|
||||
col: latencyColumn,
|
||||
@@ -133,7 +133,7 @@ describe("createTargetTableColumns", () => {
|
||||
test("延迟列正常值包含 ms 单位", () => {
|
||||
const latencyColumn = getColumn(createTargetTableColumns(["http"]), "latestCheck.durationMs");
|
||||
const renderCell = latencyColumn.cell as (params: PrimaryTableCellParams<TargetStatus>) => {
|
||||
props: { children: string; className: string };
|
||||
props: { children: string[]; className: string };
|
||||
};
|
||||
const element = renderCell({
|
||||
col: latencyColumn,
|
||||
|
||||
Reference in New Issue
Block a user