dsh-answer-review-gate
Verified@yadsh/dsh-answer-review-gate · v0.2.9 · MIT
Independent answer review gate for DeepSeek Harness agent responses
Install
dsh plugin add @yadsh/dsh-answer-review-gate Confirm the layer applied with dsh --profile default --dump-config — see the install guide.
Source
Published to npm without a public repository. Inspect the package contents before installing.
Tags
Readme
@yadsh/dsh-answer-review-gate
Independent answer review gate for DeepSeek Harness agents: before a candidate final answer is allowed to complete its turn, an independent reviewer checks it and can send it back for correction. The review is a lifecycle requirement enforced at the turn boundary — not a line in the system prompt.
What it does
User → primary agent → research / tools / background subagents
→ candidate answer
→ answer-review-gate → reviewer
├─ PASS → the turn may finish
└─ REVISE → findings are steered back into the primary,
which produces a corrected candidate that is
reviewed again (bounded rounds)
- Interim orchestration turns are never reviewed: while the session's own
background delegations (continuable subagents, one-shot background jobs) are
still pending, a turn close is treated as a pause, not a final answer. The
decision is pure runtime state — the plugin records delegated child ids from
the structured delegation tool result and clears them when the runtime's
subagent-settlednotice arrives. No text heuristics ("waiting for…") participate in this decision. - A review PASS applies to the exact candidate content (SHA-256 over the normalized text). Any later edit invalidates the PASS and the new candidate is reviewed again; the receipt follows the user request, so an unchanged candidate is never reviewed twice.
- Review rounds are bounded per user turn (
maxReviewRounds, default 3). The budget belongs to the user request the candidate answers — the agent turns and revisions that request takes share one budget, and no agent turn hands out a fresh one. - A revision steer admits exactly one visible artifact: the corrected answer.
Findings are handed back as a delimited
<review_notes>block, and the steer asks for no account of the review — a demand to "state the disproof" of a rejected objection was resolved literally on a live stand, and the user read the primary's argument with its reviewer. A draft that opens by disputing the review (Опровержение вывода ревизора: …) is that leak: in a turn that has already been reviewed the gate demands the answer's shape once, and that draft is not handed to a reviewer, so it cannot be certified as verified. The honest bounds: a committed message cannot be edited at this seam, so a second such draft goes through the ordinary review path, and a request the user waived is settled before the guard runs. - The reviewer itself is exempt: it runs as a subagent child and subagents are never gated, so the reviewer cannot recursively review itself.
Explicit one-request waiver
When deployment policy permits it, a user can send one request without the automatic reviewer:
/no-review <request>
The command may carry the same image or file attachments as an ordinary
request. It immediately shows a notice that the answer will not be
independently verified, then submits <request> as the real user message.
The waiver applies to that request only.
This is a structured command contract, not natural-language detection. The
successful command/done.sourceEventSeq must point to the durable
agent/inbox/spliced event that admitted the exact user-message id, and the
same lifecycle must start with a human-issued command/run for no-review.
Phrases such as “review is not needed”, including quoted or negated mentions,
never bypass the gate. A failed command, unrelated message, or old command
lifecycle does not bypass it either.
Reviewer backends
Both backends receive the latest human request's original durable image blocks,
including requests with no text. The Host applies the reviewer's selected model
capabilities and image limits; the gate does not replace images with text-only
placeholders. A reviewer that cannot inspect an attachment reports its own
verification gap without inferring that the primary agent could not see it.
Image review through domain-expert requires a matching domain-experts build
with the host-only reviewExpert entry. Older builds report
expert-image-input-unavailable through the configured failure policy rather
than silently reviewing only the text.
reviewer.backend: domain-expert (default)
Runs the reviewer domain configured in dsh-domain-experts
by stable domain id (reviewer.domain, default answer-reviewer), so the
reviewer persona, policy, tools and model are tuned in the domain-experts UI,
not here. The verdict is derived from the expert's own structured protocol:
findings and conflicts are the review objections; a run with no findings and no
conflicts passes. The domain-experts test entry owns its lifecycle: a running
child cannot be interrupted through this face. The gate checks cancellation
before and after each attempt, refuses a late verdict, and starts no recovery
attempt after cancellation.
reviewer.backend: subagent
Starts a native reviewer child (reviewer.provider, default spawn) with a
fresh context: it receives only the user request and the candidate answer, an
optional persona (reviewer.persona), a read-only tool allow-list
(reviewer.allowedTools; unset means the read-only set, an explicitly empty
list means no tools) and an optional model route
(reviewer.route / reviewer.model / reviewer.reasoningEffort). The verdict
must be delivered by the native structured_output tool; prose and fenced
JSON are not captured verdicts. For correlated-failure reduction, route the
reviewer to a different provider or model than the primary agent.
Capture recovery and current review status
If a reviewer ends without native structured output and without a diagnostic,
the gate makes one fresh, isolated attempt on the same immutable candidate.
The retry explicitly requests structured_output. Real provider errors,
refusals and cancellation are not retried. Two missing captures follow the
configured failure policy; reviewer prose is never promoted to PASS.
The gate records a standard producer notice containing the answer sequence, content hash, human-request sequence and current status. QA Surface renders a compact current-status badge in QA and native chat. A later result replaces the badge for that request, and applies only to its recorded answer version. Replay preserves the status; earlier model-authored messages remain history. Waived, failed and cancelled reviews are visibly different from a verified answer.
Failure policy (failMode)
What happens when the review could not complete — reviewer failure, malformed verdict, or rounds exhausted:
open— allow the answer; audit the failure.warn(default) — allow the answer, but steer an instruction requiring the primary to explicitly qualify that independent verification did not complete.closed— steer a demand to revise the answer or end with an explicit unverified-answer notice; the answer is never presented as verified.
A reviewer failure is never converted into a PASS. At most one failure
steer is issued per user turn, so the loop cannot livelock. Note the honest bound:
a plugin cannot hard-block a turn at this seam — closed mode forces a
revision-or-disclaim step rather than silently dropping the answer.
Configuration
enabled: true
reviewer:
backend: domain-expert
domain: answer-reviewer
# backend: subagent
# provider: spawn
# route: ""
# model: ""
# reasoningEffort: ""
# persona: ""
# allowedTools: [] # unset means the read-only set; [] means no tools
maxReviewRounds: 3
failMode: warn
trackBackgroundDelegations: true
minCandidateChars: 80
excludedAgents: []
waiver:
enabled: true
allowedInClosedMode: false
audit:
enabled: true
maxEntries: 500
| Field | Meaning |
|---|---|
enabled |
Master switch; false registers no listeners at all. |
reviewer.backend |
domain-expert or subagent. |
reviewer.domain |
Reviewer domain id in dsh-domain-experts (stable id, not a display name). |
reviewer.provider |
Subagent provider name for the subagent backend. |
reviewer.route / reviewer.model / reviewer.reasoningEffort |
Reviewer route overrides for the subagent backend. |
reviewer.persona |
Persona instruction for the subagent reviewer. |
reviewer.allowedTools |
Read-only tool allow-list. Unset means the read-only set (read, read_image, glob, grep, docs_read, docs_search); an explicitly empty list means the reviewer works without tools. Destructive and writing names are removed whatever the list says. |
maxReviewRounds |
Review/revision rounds per user turn before the failure policy applies (1–10). |
failMode |
open / warn / closed, see above. |
trackBackgroundDelegations |
Suppress review while the session's background work is pending. |
minCandidateChars |
Candidates shorter than this skip review. |
excludedAgents |
Session-id substrings that are never reviewed (reviewer children are exempt structurally). |
waiver.enabled |
Register and honor /no-review <request> for exactly one request. |
waiver.allowedInClosedMode |
Permit the command while failMode: closed; default false. |
audit.enabled / audit.maxEntries |
In-memory audit ring of decisions (metadata only — never prompt or response text). |
Requirements
- DeepSeek Harness >=0.1.7-rc.2 <0.2.0
- Node.js ^22.19.0 or >=24.0.0
Installation
dsh plugin --profile <profile> add @yadsh/dsh-answer-review-gate
The --profile flag is required.
Compatibility
- DeepSeek Harness >=0.1.7-rc.2 <0.2.0 (see
compatibility.json). - Uses the
commandshost service plus theagent/turn-stopping,tools/resultandagent/inbox/insertedlifecycle seams — no DSH core changes.
Security model
- The reviewer runs in a fresh context and receives only the review material (user request, candidate answer). It does not inherit the primary agent's hidden reasoning.
- The
domain-expertbackend's tool access is whatever the configured domain grants; configure the reviewer domain read-only. Thesubagentbackend's tools arereviewer.allowedToolsminus the namessrc/reviewer-tools.tsexcludes: an unset list gives the reviewer the read-only set (reads and searches), and a listedfile_delete,write,edit,apply_patch,str_replace_editor,bash,shell,run_code,lsp,dsh_lightrag_deleteor aterminal_/job_tool is dropped before the child is started. A reviewer that reaches for a deletion stops the turn it belongs to: on a QA stand the call parked above the parent's composer for an approval a delegated child can never receive, so the boundary is held both by the allow-list and by the surface, which refuses a delegated call outright. - Reviewer output is untrusted model output: it is parsed and validated, a malformed verdict is a reviewer failure handled by the failure policy, and the primary remains responsible for checking reviewer evidence rather than blindly obeying it.
- Audit records contain decision metadata only (session, turn, candidate
hash, rounds, durations, verdict or structured waiver reason) — no prompt or
response text.
/no-reviewalso setsrecordInput: false, so its lifecycle record does not duplicate the request text; the normal user message remains the sole authoritative copy.
Scope of Phase 1
Shipped: turn-stopping gate, both reviewer backends, candidate hashing, REVISE-via-steer, round limit, recursion protection, background-delegation suppression, structured one-request waiver, in-memory audit, tests. Deferred to later phases: durable /replayable review state, evidence bundle, full metrics export, and a UI badge.
See SPEC.md for the product contract.
Development
pnpm build
pnpm lint
pnpm typecheck
pnpm test
License
MIT