dsh-privacy-gate
Verified@shypie_privacy/dsh-privacy-gate Β· v0.3.0 Β· MIT Β· Web UI
ShyPie Filter: local attachment extraction/OCR, structural filtering and a fail-closed egress guard for DeepSeek Harness
Install
dsh plugin add @shypie_privacy/dsh-privacy-gate Confirm the layer applied with dsh --profile default --dump-config β see the install guide.
Source
Tags
Creators
Readme
ShyPie Filter β @shypie_privacy/dsh-privacy-gate
A DeepSeek Harness Host bundle that keeps PII on this machine. It masks at
ingestion, because DSH's outbound request is a deep-frozen fold of the session
log and cannot be rewritten on the way out β see
../../docs/research/dsh-hooks.md Β§4.
It also asserts the final request at llm/stream, before the provider adapter (and Magpie) can
see it: anything un-maskable or uninspectable fails the turn.
What it does
| Gate | Seam | Behaviour |
|---|---|---|
| 1 β tool ingress | tools/post-execute (waterfall) |
Tool-result text is kept local by default (textPolicy: filter) or masked (textPolicy: mask). In filter mode the text is only scanned for unmaskable BLOB/media; in mask mode a success result is sanitized through value so the logged meta is re-derived from masked data. Binary payloads (BLOB) and image/file content are blocked, never forwarded. |
| 2 β local extraction | privacy_extract, privacy_ocr, privacy_scan |
PDF/Office/image text is produced by the local CLI (packages/local-extract/cli.js) and returned by the explicit tool. Tool results still follow textPolicy; a blocked conversion (NO_LOCAL_OCR, NOT_A_DOCUMENT, β¦) is a tool error, never content. |
| 3 β user ingress | agent/pre-step (waterfall) |
User text is kept local by default (textPolicy: filter) or masked (mask). An attached image is read by the local OCR engine; an attached PDF/Office file is read by the local extractor. The block is replaced by text before the message is logged (imageParts: 'ocr' / fileParts: 'extract'); when it cannot be read the whole step is rejected. Audio/video and blocked file parts are rejected. |
| Guards | ctx.tools.guard() |
Denies read_image outright, and denies read of binary/document extensions outside allowPaths. |
| Round trip | privacy_rehydrate |
Resolves placeholders inside the tool, writes the plaintext to a local artifact, and returns only { path, replaced, kinds }. |
| Extra (defensive) | tools/ptc-dispatch-log |
Masks the durable log copy of run_code sub-dispatches. |
| 4 β egress | llm/stream (waterfall, prepended) |
Inspects the exact final request before adapter dispatch. Rejects image/file/audio/video blocks, unknown block types, BLOB, provider/model outside the allowlists, and over-budget requests. In textPolicy: filter it allows plaintext PII through the default Magpie-only provider, for Magpie to mask; mask also rejects PII. Returns a terminal error before next(), so no HTTP request reaches Magpie. |
Files
index.js apply()/Config/inject β the bundle entry point
lib/config.js pure config resolution (frozen, unknown keys fail)
lib/paths.js pure path predicates used by the guard
lib/redact.js pure content/message/value masking over the pii-core API
lib/gate.js the gates as pure decision shapers + installGates()
lib/ingest.js reads prompt image/PDF attachments and replaces them with local text
lib/egress.js final llm/stream egress guard (fail-closed before the adapter)
lib/tools.js the four ToolDefinitions (raw register() form)
lib/extract.js local-extract CLI adapter (spawn, no shell, no network)
lib/keyfile.js placeholder HMAC key (mode 0600)
lib/audit.js bounded journal: kind/length/count/path_hash/ts only
scripts/vendor-core.mjs copies packages/pii-core/index.js β vendor/pii-core.js
scripts/activation-check.mjs import + Config + apply() smoke check
vendor/pii-core.js generated β never edit
Install (not done by this repository)
# 1. vendor the redactor into the bundle
node scripts/vendor-core.mjs
# 2. from the Harness, install the bundle by absolute package directory
plugin_manager { action: "install_bundle", target: "<abs>/plugins/dsh-privacy-gate" }
The bundle declares dsh.bundle.patch; its patch inserts one Host row
(privacy-gate). It declares @deepseek-ai/schemastery and @deepseek-ai/dsh-settings as peer
dependencies on purpose: the host resolves those from the running installation, and without the
declaration a fresh profile cannot resolve @deepseek-ai/schemastery from a linked checkout at
all β the entry then fails to import. Do not hand-edit the profile's
package.json/cordis.patch.yml, and do not copy vendor/ state between
installs β run the script.
From npm the same bundle installs by name β
dsh plugin --profile <name> add @shypie_privacy/dsh-privacy-gate β with the vendor/
copy already inside the package. engines.dsh is >=0.2.0 <0.3.0
(docs/publish.md).
Config (row config; overrides replace the whole block)
allowPaths: [] # path prefixes where a document may be read/extracted ([] means none)
detectors: [] # pii-core kinds; empty = all
personWords: [] # allowWords for PERSON detection
extractCli: '' # defaults to the sibling checkout, then the installed local-extract
audit: { enabled: false, path: '' } # JSONL, kind/length/count/path_hash/ts only
placeholderKey: { file: '' } # defaults to ~/.dsh/privacy/placeholder.key, created 0600
blockNonTextResults: true # block tool results containing image/file blocks
artifactDir: '' # defaults to ~/.dsh/privacy/artifacts (privacy_rehydrate target)
toolTimeoutMs: 120000
maxExtractChars: 200000
egressAllowProviders: ['magpie*'] # filter mode defaults to the Magpie gateway only
egressAllowModels: [] # [] = every model
egressMaxChars: 2000000 # reject instead of passing an uninspectable request
egressMaxNodes: 200000
scope: session # session (default) | system β see below
sessionsPath: '' # defaults to ~/.dsh/privacy/sessions.json
ocrEngine: local # local | vision | auto (vision is configuration-only today)
ocrLang: '' # optional BCP-47-ish hint passed to the extractor (--lang)
imageParts: ocr # ocr (default) | block β what an attached image becomes
fileParts: extract # extract (default) | block β PDF/Office attachments
textPolicy: filter # filter (default: raw local text) | mask (redact locally)
There is no failClosed switch: the gate always fails closed.
Attached images and PDFs are read locally, not forwarded
An image reaches the agent as an image content block and a PDF/Office file as a file block, both
referencing an opaque content-addressed object (ctx.attachments). The gate reads the bytes, runs the
local extractor (ocr for images, auto for files; Windows OCR / pdftotext on this machine), and
substitutes one text block. In the default filter mode that text stays raw in the local
transcript; in mask mode it is masked with pii-core. Example (filter):
[privacy-gate: local OCR of an attached image/jpeg "scan.jpg", 113623 bytes, engine winrt-ocr]
Rechnung DE89370400440532013000 β¦
Why agent/pre-step and not the egress guard: the loop appends the decision's messages to the
session log (dsh-agent-loop turn()), so converting there keeps the pixels out of the transcript
and out of every later request. After that seam the request is a frozen fold of the log and can only
be rejected, never rewritten.
Fail-closed cases (the step is rejected, nothing is sent): no attachment service, an attachment that
fails verification, an extractor that is missing or fails, OCR that returns nothing, a BLOB-shaped
result, and imageParts: 'block' / fileParts: 'block'. Audio/video and unknown file parts have no
local reader and are rejected.
Trade-off: the transcript keeps the OCR text, not the image. That is the point of the gate β if you need the model to see a picture, send it to a model you have declared local.
Scope β per session by default
scope: session (the default) binds the gate to sessions instead of the whole profile. On first
activation the bundle writes ~/.dsh/privacy/sessions.json and records, once, the sessions that
already exist under <DSH_HOME>/sessions: those are stored as false and are never gated, so
history written before the bundle existed cannot turn every later turn into
PRIVACY_EGRESS_BLOCKED. Sessions first seen after activation are gated; a later setEnabled in
that record wins over the snapshot.
scope: system is the old behaviour: every session, including ones that predate the bundle. Use it
only after quarantining old logs β the first request that folds unmaskable history is rejected.
The record holds session ids and booleans only β no content, no paths, no per-session timestamps β and is written atomically with mode 0600. A missing or corrupt record re-snapshots the legacy sessions, which can only widen the ungated set; it never narrows it silently.
Session ids have two spellings. The harness names a session's directory after its SessionId, so
the directory name is the id: current profiles use session-<uuid>, older ones a bare <uuid>.
The snapshot stores the name as found and every lookup tries both spellings, so one session cannot
hold two verdicts. Record version: 1 stripped the prefix and therefore never matched a lookup β
legacy sessions were gated despite being recorded as off β so a version-1 record is discarded and
re-snapshotted on first use (per-session toggles from that record are lost, and the snapshot can only
make a session ungated, never gated).
| seam | session identity |
|---|---|
tools/post-execute, ctx.tools.guard |
exec.agent.id (SessionId) |
agent/pre-step |
payload.agent.id (Agent.id is the session id) |
llm/stream |
request.sessionId; absent on compaction and session-title calls, which scope: session leaves alone |
settings route / privacy_session |
whatever id the client or the tool passes; both spellings resolve to the same row |
Settings page β own route, not a settings namespace
lib/client.js is the browser half: it registers a ShyPie Filter section
(settings.section, id privacy-gate, order 70) and renders scope, allowPaths, extractCli,
ocrEngine, visionProvider/visionModel and allowAgentDisable.
It does not use a settings namespace. In DSH 0.2.1-rc.5 the host api-proxy serves only the
namespaces in WEB_SETTINGS_NAMESPACES (@deepseek-ai/dsh-host-apiproxy/lib/types/api-proxy.js), so
a third-party namespace answers settings-not-exposed for both reads and writes. This is the same
wall dsh-client-ui-settings-plugins documents as "a plugin distributed outside this repository
cannot surface its own configuration here without a change in packages/host/apiproxy".
The working pattern β used by Plugin Hub (dsh-plugin) in this profile β is a plugin-owned
same-origin HTTP route, and that is what this bundle now does:
| Piece | Where |
|---|---|
GET /shypie-privacy/settings β { value, revision, defaults } |
lib/http.js |
POST /shypie-privacy/settings with a field patch β applied live, then persisted |
lib/http.js |
Store: ~/.dsh/privacy/settings.json ({ version, revision, value }, mode 0600, atomic) |
loadSettings / saveSettings |
Mounting: ctx.inject(['webServer'], (host) => host.effect(() => registerSettingsRoute(...))) |
index.js |
Two details matter for safety and for boot:
- The route is a security boundary. It binds loopback only, rejects a non-loopback
Host(plain HTTP and DNS rebinding), requires a matchingOriginwhen the browser sends one, accepts a fixed field allowlist (sessionsPathand friends stay host-only), validates the merged value throughresolveConfigbefore storing it, and answers400without persisting when validation fails. - Nothing new is injected at boot. The entry injects
toolsonly;webServeris requested in a child scope, and the client entry injectsslotsonly. The earlier attempt injectedsettingsScope, which left the web-boot entry pending β DSH treats a pending entry as a failed start.
The renderer evaluates the client file through the same window.__ModuleLoader__.load({ id, factory })
wrapper the core client packages use. test/client.test.mjs checks the export shape and the
registration; test/http.test.mjs covers origin checks, the field allowlist, revision bumps and the
rejection paths without a browser.
| Field | Meaning |
|---|---|
scope, allowPaths, extractCli, detectors, personWords, blockNonTextResults |
as in the row config |
auditEnabled, auditPath |
the audit journal |
egressAllowProviders, egressAllowModels |
egress allowlists |
ocrEngine |
local (default), vision or auto β a preference, not a call |
visionProvider, visionModel |
the provider/model a future vision fallback would use; no image is sent anywhere today |
Saved values layer over the row config and are re-applied on the next start; a value that no longer validates is ignored with a warning and the patch applies.
Egress guard β fail closed before Magpie
DSH's llm/stream waterfall wraps adapter dispatch (dsh-llm streamWithRegistration). This
bundle registers a prepended, global listener that inspects the final
{provider, model, messages, tools} request and, before calling next(), returns a terminal error
stream. DSH's agent loop turns that into its own LlmError with code PRIVACY_EGRESS_BLOCKED.
No provider adapter is called, so Magpie never sees a blocked request and Magpie's 250 ms timeout
cannot drop the verdict. The guard does not throw out of the hook, which keeps the turn's error
structured.
The guard rejects:
image,file,audioorvideocontent blocks;- unknown content-block types;
BLOBin text, reasoning, tool-call arguments or tool-result content (always);- PII in those fields only when
textPolicy: 'mask'; infiltermode Magpie masks it outbound; - a provider/model outside
egressAllowProviders/egressAllowModels; - a request over
egressMaxCharsoregressMaxNodes(fail closed, never degrade).
It does not mask or rewrite: the request is frozen, so rejection is the only action. Tool
declarations (request.tools, request.toolHistory) are schema, not conversation content, and are
skipped. The existing ingestion gates still run first; this is the final assertion that they left
nothing un-maskable.
PII policy. Default filter mode keeps PII in the local transcript and lets Magpie's Privacy
masking handle it outbound. The egress provider allowlist defaults to magpie*, so a direct
provider cannot receive raw text by accident. mask mode restores local placeholder masking and the
egress guard also rejects surviving PII. Filter mode is a deliberate usability/security trade: the
local log is readable, so protect the machine and keep Magpie masking enabled.
Incremental scan. DSH request messages are frozen and reused across steps. The guard caches the message objects it has successfully inspected and scans only new messages; the first request pays the full cost, later steps do not rescan the history. On a 5,000-block synthetic request the first pass was ~107 ms and later requests with one new message ~0.1 ms in Node 24 with the real redactor.
failClosed is not a setting β failing closed is the only mode: if vendor/pii-core.js cannot load,
the key file is unusable, a redactor throws, or a document path is outside allowPaths, the gates
block tool results, reject prompt steps, and the four tools throw. Nothing falls through.
PRIVACY_EGRESS_BLOCKED in a session that predates the gate
The request is a fold of the append-only session log, so one message the gate cannot mask blocks
every later turn in that session β including a message written before the bundle was installed.
This is the documented "start a new session after installing" rule, hardened into a hard stop. The
error names the reason and says what to do (β¦ (PII); an earlier message in this session still holds content the gate cannot mask β start a new session (the log is append-only)).
There is no override, by design. Recovering means leaving the poisoned history behind:
- start a new session (not a fork β a fork reseeds the old events);
- if the old log holds plaintext, delete or quarantine it rather than keeping it around:
%USERPROFILE%\.dsh\sessions\<project>\<session-id>\session.v4.jsonl.zstd; - a session whose plaintext came from before the gate was installed cannot be repaired in place; the gate never rewrites existing events.
If the rejection reason is not PII/BLOB, the hint on the error names the actual remedy
(MEDIA_* β the attachment is already logged, attach it in a new session, PROVIDER/MODEL β
allowlist, LIMIT_* β raise the budget).
Troubleshooting the extractor
privacy_extract / privacy_ocr / privacy_scan shell out to packages/local-extract/cli.js. When
the row sets no extractCli, the default is resolved in this order:
- the sibling checkout (
<bundle>/../../packages/local-extract/cli.js) β folder installs; - the installed
@shypie_privacy/local-extractpackage β npm installs (0.2.1+declares it as a dependency and the package exports./cli.js); - nothing β the tools then fail with "extractCli is not configured and @shypie_privacy/local-extract is not installed" instead of spawning a path that cannot exist.
Two other settings decide whether a document is readable at all: allowPaths ([] denies
every path) and maxExtractChars. Set them in the row config and remember that a row config
replaces the whole block, so restate the other keys:
- id: privacy-gate
config:
allowPaths: ['C:\Users\<you>\Documents\privacy-work']
extractCli: 'C:\Users\<you>\Documents\ShyPie\packages\local-extract\cli.js'
detectors: []
personWords: []
blockNonTextResults: true
audit: { enabled: true, path: 'C:\Users\<you>\.dsh\privacy\audit.jsonl' }
placeholderKey: { file: 'C:\Users\<you>\.dsh\privacy\placeholder.key' }
Tools
privacy_extract { path, mode?: pdf|doc|auto, max_chars?, pages?, lang? }
privacy_ocr { path, max_chars?, pages?, lang? }
privacy_scan { text }
privacy_rehydrate { text, artifact_path } -> { path, replaced, kinds }
privacy_session { action: status|enable|disable } -> { scope, enabled, explicit, sessionId }
The first four return masked text or throw. privacy_rehydrate never returns a value,
only a path and counts (the interface contract round-trip rule).
privacy_session is the per-session switch: status is always allowed, enable opts the current
session in, and disable is refused unless the deployment sets allowAgentDisable: true β a
prompt-injected instruction must not be able to turn the gate off by itself. It acts on
exec.agent.id, the same SessionId the gates read, and records only an id and a boolean.
Tests
node --test # 154 tests, no network, synthetic fixtures only
node scripts/vendor-core.mjs --check # vendor copy is current
node scripts/activation-check.mjs # needs @deepseek-ai/schemastery resolvable
test/integration-core.test.mjs runs the gates and tools against the real
vendored pii-core; the other suites use a fake redactor with the same
contract shape.
Residual exposure (read this before claiming "no PII leaves the machine")
- Raw prompt text stays in the local session log.
agent/pre-stepmasks theuser/messageevent and every request derived from it, but the prompt was already durably logged byagent/inbox/splicedbefore the step boundary, and there is no pre-append hook. Research note Β§5b proves it does not re-enter any request on fresh/resume/fork/replay, so it is local-only β but it is local plaintext in that session's JSONL. - History logged before the gate was active is replayed as-is. The gate does not rewrite existing events.
- A tool result's
metaon an error result cannot be rewritten bytools/post-execute(only a successful result can be re-derived throughvalue). A third-party tool that puts plaintext into an errormetakeeps it in the local log. read_imageis denied, not converted. A tool that wants to look at a file cannot be served by the prompt-side OCR (which only sees the user's attachments); useprivacy_ocrfor text.- An OCR'd attachment loses its layout. The model gets the extracted text (masked), not the picture: tables, stamps and handwriting may be misread, and nothing re-attaches the image later. Send the document to a model declared local if the bytes themselves matter.
privacy_rehydrateis session-scoped.restore()only resolves values masked by that same redactor instance; cross-restart restore would needpii-core's encrypted vault, which this bundle does not wire up.