agent-evaluation 8

OMK — Observe. Measure. Know. Evidence-backed knowledge changes for AI applications.

Host-side Cordis control plugin for aeval: one-shot config injector (experiment variables, gateway-lease budgets, fork lineage, bundle descriptor), with a standalone status mode, a Web status chip, and a self-check command.

DeepSeek Harness plugin and Skill for Harbor Candidate and Historical Session evaluation workflows.

Agent evaluation platform: benchmark YAML, headless run orchestration, trace-based metrics, and run reports

LLM-as-a-Verifier for dsh with the Best-of-N conversation mode built in: rank N candidates with a fine-grained verifier (expected grade over the logprob distribution), on demand via the verify tool or automatically on every turn of a Best-of-N session. In

Replay real DeepSeek Harness turns against Standard, Minimal, Anchored, or plugin candidates with frozen request-surface evidence

Retrospective agent-session evaluation for DeepSeek Harness: deterministic grade cards from persisted session logs, cross-session regression diffs, era comparison — no benchmark runs, no LLM judges.

Paired experiments and promotion gates for DSH plugins.