Capability matrix, part 2

Downshift by Tiago Vilas Boas. Updated . Source on GitHub

9. Context optimization

Capability Status Where Evidence Limits Docs
Native compressor (off, observe, safe): compacts an all-ok go test, a clean git status, repeated logs; preserves failures, non-zero exit, search hits, file lists, git log Implemented internal/compressor, downshift context compress compressor_test.go (TestCompress_GoTest_FailPreservesRaw), benchmark_test.go Transforms bytes only in a shell pipe with safe --exit 0; the hook cannot replace tool output on any harness; prompts never compressed docs/design/context-optimization.md, skills/cost-reducer/references/compress-output.md
Context sensor (observe-only byte counts per session hash and format; context-sensor.json, context-compactions.jsonl) Implemented, not validated everywhere internal/sensor, Claude Code PostToolUse (recordSensorToolObservation) sensor_test.go, TestPostToolUse_CompressExitFollowsNativeFlag, TestPostHooks_PostToolUseBoundedWhenSensorLockHeld (500 ms lock) Claude Code PostToolUse only; tokens unavailable; context-sensor.json never pruned (audit) docs/design/context-optimization.md
downshift context benchmark (four scenarios printed not measured; byte table from four checked-in fixtures) Implemented (bytes only) internal/benchmark/context_benchmark.go, fixtures/ compressor_benchmark_test.go Not tokens, not dollars, no cost or accuracy README, docs/design/context-optimization.md
Headroom companion: detected on PATH, recorded by context enable, doctor row, wrap hints per harness Implemented internal/contextopt/providers/headroom, cmd/context.go context_companion_test.go, headroom_test.go Never installed, launched or wrapped; metrics unavailable; Antigravity and KiroCrew not in Headroom's list (2026-10-17); a companion's savings are not routing savings docs/guides/companions.md
Gateway-prefixed ids (OpenRouter rule: first provider/ segment stripped; ids outside the catalog stay unknown) Implemented, not validated everywhere internal/catalog/catalog.go catalog_test.go prefix-normalisation tests Not verified against a live gateway (2026-10-17) docs/reference/session-models.md
Context compression in the request path Not supported — — A hook cannot change what the model sees; that is a proxy's job docs/guides/when-to-use.md
Downshift Code Intelligence (DCI): Git discovery, multi-language structural symbol extraction (Go AST, TypeScript/JavaScript, Python, Rust, PHP, Java, C#), outgoing calls & package imports, reverse call-graph index (Blast Radius / InvertedIndex.Callers), Production-First BM25 ranking & Call-Graph Co-Reference boost (+2.0), conversational chatter abstention, structural context packaging (default 3,500 bytes UTF-8, configurable 512–16,384 via --budget) with prompt-injection defense delimiters, multi-harness hooks (claude-code, cursor, opencode, codex, generic), and downshift dci impact Implemented (v0.4) internal/codeintel, cmd/downshift/dci.go, cmd/downshift/dci_hook.go, hooks/hooks.json dci_test.go, dci_stats_test.go, internal/codeintel/*_test.go, benchmark 2026-10-19 (cold=295ms, warm=18µs, p95=5.64ms, pack=568-2801B, impact callers index) Cold build <= 2500ms, warm retrieval p95 <= 300ms, 2000 files / 512 KiB cap, 1 MiB hook stdin cap; callers capped at 32 per symbol, calls capped at 16, imports capped at 32; zero prompts or source code stored to disk docs/design/code-intelligence.md, docs/evidence/dci-mvp-v01.md, README.md

10. Project policy

Capability Status Where Evidence Limits Docs
.downshift/policy.json tier floor (min_tier, min_tier_by_class, min_tier_by_risk; nearest file within 8 levels; 64 KB; invalid file ignored) Implemented internal/policyfile, core/tier_floor.go (TierFloorHook), hookctx.ProjectDir policyfile_test.go, TestHookE2E_ProjectFloorThroughSharedRunner, TestRunKiroCrewHook_ProjectFloorApplies; counted as policy_floor Raises only; passes the session allowlist and quota gates; project dir is vendor-documented for Claude Code only (others from third-party reports or the process cwd); cannot force a model where rewrites are not honored; try and benchmark never read it docs/guides/config.md#project-tier-floor-optional, skills/cost-reducer/references/project-policy.md

11. Learning and shadow (all shadow or opt-in; none changes a spawn by default)

Capability Status Where Evidence Limits Docs
Adapt tier memory (adapt-memory.json, per <class>:<feature> hit/miss sets) recorded as AdaptWould Shadow (Experimental) internal/adapt, internal/routeadapt, downshift eval-adapt TestApply_ShadowDoesNotChangeTier; benchmark/report.md 2026-10-15: shared-shape memory 37/40 = classifier (0.0 pp), per-task upper bound 40/40 Promotion blocked on an independent outcome suite and a cost per completed task (GAP-SUITE); count-based variant 29/40 LOO, not promotable docs/reference/beta-exit.md, docs/design/engineering-loop.md
Naive-Bayes second opinion: upshift (DOWNSHIFT_NB_UPSHIFT=1) and TRIVIAL downshift (DOWNSHIFT_NB_DOWNSHIFT=1) Shadow by default, opt-in core/nb_upshift.go, core/nb_downshift.go, internal/nbtier nb_downshift_test.go (TestNBDownshift_R1HoldsWithFlagOn), nb_upshift_test.go Margin frozen on seed and holdout 2026-10-03; even when on, R1 holds an NB downshift with no baseline docs/design/router-generalization.md, docs/reference/stats-export.md
Capability scores (Epoch AI ECI, Artificial Analysis via OpenRouter): local cache, daily detached refresh from hooks, 6 h backoff; score-based pick recorded as score_would Shadow by default; live with DOWNSHIFT_SCORE_ROUTING=1 internal/scoreindex, core/score_select.go, cmd/scorerefresh.go, downshift models scores TestScoreTier_ScoreRoutingNeverBelowDecisionTier (audit), scorerefresh_test.go The only network call on the hook path (opt out DOWNSHIFT_SCORES=off); abstains on COMPLEX, any risk, any deterministic lift or an uncovered candidate (composer-2.5, flash_lite, pro) docs/reference/scores.md, matrix "Public score coverage"
Routing v2 softmax candidate (DOWNSHIFT_SHADOW_WEIGHTS, downshift train, benchmark --compare, shadow-report) Experimental internal/routingv2/* shadow_test.go, train_test.go Never changes hook output; trains only on reviewed required_tier labels docs/design/capability-router-v2.md, docs/design/classifier-shadow.md
Multilingual E5 shadow/fuse and external MiniLM embed Removed 2026-10-17 — Historical evaluation: e5-int8-evaluation-2026-10-17.md Not in the binary. Old logs may still show E5_* or GRAPH_ESCALATION CHANGELOG
Learned complexity scorer for EN and PT-BR Planned — ROADMAP "Next" Needs about 2,400 labelled prompts ROADMAP.md

12. Benchmarks

Capability Status Where Evidence Limits Docs
Classifier regression nets: seed 200 and holdout 300 (burned: tuned against, holdout 100.0%) Implemented (maintainer) downshift benchmark <file> [--report] [--gate], internal/benchmark benchmark/report.md 2026-10-17: seed 73.5% (67.0–79.1%); baseline-freeze-2026-10-17.md names the retired 69.0% (be1054ac) versus the current classifier (236c057 onward, measured at 2c8f3f4); gates FRONTIER→SMALL = 0, FRONTIER→MID ≤ 0.70, tier accuracy ≥ 0.60 (gates_test.go) Datasets live in private downshift-labs; gates protect the burned net, not unseen traffic benchmark/README.md, docs/design/router-generalization.md
Eval-only splits: fresh 100 and heldout2 120 (blind), blind-vitrine 60 Implemented (maintainer) same fresh 50.0% (40.0–60.0%) and heldout2 40.0% (31.7–49.2%) at 5a888e7; 53.0% / 41.7% at 3067340; blind-vitrine 46.7% (audit); confident and wrong: 2 of 37, 6 of 28, 8 of 23 Classifier only (no router, guardrails or quota); no new blind set of 200+ (P4 open) benchmark/report.md, audit
Executable outcome suite, 40 Go tasks, one fixed model per tier (2026-10-03) Implemented (maintainer) downshift eval-outcome --verify|--report, internal/outcome frontier claude-opus-5.5 40/40, mid deepseek-v4-flash 31/40, small qwen3-coder-30b 27/40; TRIVIAL+SIMPLE 20/20, 17/20, 16/20; classifier routing 37/40 at the picked tier (recomputed 2026-10-15) No token, retry or USD fields (run.json, issue #100); small minus frontier is −20 pp on TRIVIAL+SIMPLE; independent 6-task held-out smoke only (GAP-SUITE) benchmark/report.md, docs/internal/gap-tasks.md
Routed versus control A/B on real and mechanical tasks (2026-10-15) Implemented once (one operator) private runner in downshift-labs; DOWNSHIFT_NO_ROUTE=1 control arm b2 7/8 vs 7/8, $0.65 vs $0.77, 0 downshifts; b5 12/12 vs 12/12, $0.59 vs $0.67 (−12%), 19 of 56 (34%) rewritten down, mech-rename −33%; b6 8/12 vs 9/12, $0.75 vs $0.92, 3 of 34 (9%) Three runs per cell; Claude Code only; checks verify structure, not quality; natural-use saving not shown A/B, docs/guides/evidence.md
Context benchmark (four fixtures, bytes) Implemented downshift context benchmark compressor_benchmark_test.go Bytes; scenarios A–D not measured docs/design/context-optimization.md
Multilingual M3 frozen set (170 rows) Implemented (maintainer) labs CHANGELOG: unsafe confident recommendations 0 Private; aggregate only CHANGELOG

13. Packaging

Capability Status Where Evidence Limits Docs
Claude Code plugin: .claude-plugin/plugin.json, marketplace.json, hooks/hooks.json (four hooks and one mod module), commands/status.md, skills, agents/mechanical-worker.md Implemented repository root TestPluginHooksMatchSetupTarget; doctor warns when plugin and settings hooks both run Claude Code only; binary still required; plugin does not set opusplan, plan mode, You should know or the statusline; plugin detection reads the user settings file only docs/guides/install.md#plugin-install-claude-code, hooks/README.md
Bundled mod cache-tax (hooks/cache-tax/, MIT, vendored v2.2.1): priced cold-send guard, /keepwarm, /cache-tax, band above the prompt Implemented (third-party, pinned) hooks/cache-tax/register.ts, loaded by hooks/hooks.json modules upstream suite vendored, claude plugin test . 56 pass on 2.1.302; claude plugin validate . lists its calls; doctor warns when cache-tax@claude-code-mods is also enabled Claude Code 2.1.287+ only (mods); arrives with the plugin that setup installs, never written into a settings file; the warming ping spends usage and is armed by the user or after a detected cold write; prices are the mod's own list table, not the catalog; owner decision 2026-10-17 hooks/cache-tax/README.md
Sibling mod savvy-progress (mods/savvy-progress/, MIT, vendored v1.2.8, plugin savvy-progress@downshift): progress bar above the prompt, /agents-info panel of subagents Implemented (third-party, pinned) mods/savvy-progress/hooks/register.tsx, own hooks.json upstream suite vendored, claude plugin test mods/savvy-progress; claude plugin validate mods/savvy-progress; #349 adds coverage that the crew badge matches panel-visible entries Claude Code 2.1.287+ only (mods); separate plugin because a plugin loads one hooks module; setup installs it with claude plugin install (--no-plugin skips); crew badge counts running agents, finished agents still visible in the panel, and planned tasks; hidden completed agents no longer inflate it mods/savvy-progress/README.md
Sibling mod filetree (mods/filetree/, MIT, vendored v0.2.26, plugin filetree@downshift): file tree pane highlighting what Claude reads, writes and commits Implemented (third-party, pinned) mods/filetree/hooks/, own hooks.json upstream suite vendored, claude plugin test mods/filetree; claude plugin validate mods/filetree Claude Code 2.1.287+ only (mods); separate plugin; setup installs it with claude plugin install (--no-plugin skips); owner decision 2026-10-17 mods/filetree/README.md
Sibling mod downshift-dock (mods/downshift-dock/, Apache-2.0, original, plugin downshift-dock@downshift): band above the prompt with Files and Agents buttons that run /filetree and /agents-info open Implemented (unit-tested against the mock engine; real UI unverified) mods/downshift-dock/hooks/register.tsx, own hooks.json claude plugin test mods/downshift-dock 6 pass; claude plugin validate mods/downshift-dock Claude Code 2.1.287+ only (mods); depends on the filetree and savvy-progress commands existing, shows a button only when $.command.list() reports it; a press is queued until the session is idle; installed by setup (--no-plugin skips) mods/downshift-dock/README.md
downshift setup (per-harness targets as data, diff, backup, --dry-run, --yes, --undo, --no-session-policy) Implemented cmd/setup.go, setup_targets.go setup_test.go KiroCrew has dedicated setup that generates and registers its wrapper; Codex still needs config.toml flags and /hooks trust docs/guides/install.md
install.sh with sha256 verification against the release checksums.txt Implemented (Unreleased) install.sh install_test.go DOWNSHIFT_SKIP_CHECKSUM=1 opts out loudly docs/guides/install.md
Example subagent with no pinned model Implemented agents/mechanical-worker.md plugin loads it as downshift:mechanical-worker Claude Code agent format; the rule (omit model) applies everywhere docs/guides/subagents.md
Always-loaded memory file capped at 200 lines; doc names lowercase kebab-case and relative links checked Implemented as a rule AGENTS.md (CLAUDE.md points to it); internal/docscheck TestMarkdownNames, TestRelativeLinksResolve (no test enforces the 200-line cap) Contributor skill downshift-layering stays out of the plugin docs/design/architecture.md
Claude Code mod downshift-router (standalone plugin, own manifest, not loaded by the root plugin): routes the subagent model from the in-process agent.spawn event by running downshift claude-code over the stdin contract and returning next({ ...e, model }); applies spawn-time reasoning_effort on turn.step via next({ ...e, effort }); off unless userConfig.enabled or DOWNSHIFT_MOD_ROUTER=1 Experimental mods/downshift-router/.claude-plugin/plugin.json, hooks/hooks.json (modules), hooks/register.ts; Claude Code adapter parent_model, mod-only updatedInput.reasoning_effort, runner entry, SubagentStop inline usage; core.ClaudeCodeCaps.HookEntries hooks/register.test.ts (spawn, turn.step, turn.complete); Go: parent_model_test.go, mod_effort_test.go, entry_e2e_test.go, TestSubagentStop_PricesFromInlineUsage, entry_test.go, TestDoctorDuplicateEntry Claude Code 2.1.287+ only; no drawing under claude -p; phase 2b done (honored observation for mod-written models still open); running both paths still routes every spawn twice (doctor warns) docs/design/mod-router.md, mods/downshift-router/README.md

Contradictions and overclaims found

Each line: file:line, the sentence (abridged where long), and what the evidence supports.

  1. README.md:253 "event-driven Context Sensor to govern context token volume". RESOLVED (2026-10-17): README.md:325 now says the sensor "observes context volume (byte counts, never a rewrite)"; no "govern" on the Context Sensor (only GOVERNANCE in the project links).
  2. Semantic boost is the in-process FNV hash (hashembed.go). External MiniLM and E5 were removed 2026-10-17. See docs/design/minilm-semantic.md.
  3. docs/pt/README.md:23 "decide em menos de 2 milissegundos". Holds for short prompts (0.10 ms) only; the audit measured 0.23 s on a typical 1 MiB prompt and 275 ms with the catalog resolver after the prefilter (hardening-audit-2026-10-15.md performance note). Unqualified claim.
  4. docs/guides/install.md:573 "API (direct) | ✅ Yes | ✅ Yes". No evidence file covers an API-key session; the 2026-10-06 write-up records "Plan tier: not recorded".
  5. docs/guides/install.md:585 "On usage-based Pro/Ultra plans where subagent model selection is expanded, routing works." RESOLVED (2026-10-17): install table and prose say rewrite is emitted and child model is unconfirmed (P1.6); matrix row stays Unconfirmed.
  6. docs/guides/install.md:349 "inspects the prompt inside Subagents[0].Prompt ... rewrites Subagents[0].Model". RESOLVED (2026-10-17): install describes the last Subagents entry with a non-empty Prompt, flash_lite / flash, and the undocumented overwrite envelope.
  7. Graphify integration doc removed 2026-10-17; graph escalation is no longer part of the product.
  8. docs/reference/session-models.md:57-63 stale 2026-09-27 field table. RESOLVED (2026-10-17): table rechecked 2026-10-17 with optional session_id, cwd, transcript_path, tool_use_id, permission_mode, workspace_roots; usage via SubagentStop transcript, not async PostToolUse tokens.
  9. docs/pt/README.md:26 "Métricas em dólares reais dependem do reporte de tokens retornado pelo harness no hook posterior (PostToolUse)". On Claude Code the PostToolUse payload of an async subagent carries no tokens (posttooluse.go header, beta-exit.md P3 note); usage comes from the SubagentStop transcript.
  10. skills/downshift-audit/SKILL.md:12 "[doctor covers] whether a rewrite was honored". RESOLVED (2026-10-17): skill states doctor rewrite honored is inferred honor only and points to daily-truth.md and harness-matrix.md for observed honor; code behaviour unchanged (CountInferredHonored).
  11. benchmark/report.md:18 seed tier accuracy 69.0% versus audit 73.5%. RESOLVED (2026-10-17): benchmark/report.md publishes seed 73.5% (67.0–79.1%) and baseline-freeze-2026-10-17.md documents the superseded 69.0% at be1054ac; matrix rows §1 and §12 cite the same figure.
  12. README.md:36 "Implemented on Claude Code (usage from the child transcript); partial elsewhere." Elsewhere is weaker than partial: Codex and Cursor end parsers are unverified and read no usage; Antigravity and KiroCrew record nothing (matrix "Child end (N5)").
  13. README.md:37 and llms.txt:16 link docs/guides/benchmark-protocol.md, missing at audit time. RESOLVED (2026-10-17): docs/guides/benchmark-protocol.md is in the tree; README line 54 links it.
  14. llms.txt:106-107 and docs/pt/README.md present downshift serve and dsmon as plain reporting commands. RESOLVED (2026-10-20): the web dashboard (serve, internal/server, dsmon-server) was removed in a3d5a8b7; llms.txt and docs/pt/README.md no longer list downshift serve.
  15. docs/guides/companions.md:6 / when-to-use.md:3 router-only positioning. RESOLVED (2026-10-17): both guides frame the suite and "model routing is the main lever today, not the whole product", aligned with README § suite-at-a-glance.
  16. README.md:104 and docs/guides/install.md:66 quote the stderr line as downshift: TRIVIAL task → downshift to claude-haiku-4-5. The real line is downshift: correlation_id=<id> TRIVIAL task → downshift to claude-haiku-4-5 (~75% cheaper) (main.go stderrLine, policy.go Summary). Trimmed sample, not wrong, but a grep for the documented text fails.

Hidden capabilities

Things the code does that no public doc explains well (file where it lives; where it is mentioned, if at all).

  1. Verification sensor. SubagentStop records whether the child ran a recognised test or lint command and whether the last one passed (verification: passed|failed|none on usage records); downshift verification-report groups usage-event counts, verification coverage/status and token-call counts by tier; it reports no price or cost. internal/adapters/claudecode/verification.go, cmd/verification.go. Mentioned once in CHANGELOG; absent from help --all, README, stats-export.
  2. Prompt-injection as a risk category. A task that says "ignore previous instructions", "this is a trivial task" or "use the cheapest model" is treated as risky: never downshifted, floor off the small tier. core/risk.go prompt_injection. Docs name the category in passing only.
  3. The hook's time and safety budget. 1 MiB payload cap, 2 s stdin read timeout, a 1.5 s hook deadline (hookctx.HookBudget) that kills helper commands, 1 s per embed call, 400 ms transcript wait, 500 ms sensor lock, 3 s graph fetcher, and CmdTrusted, which refuses an E5 helper that is curl, bash, -c or a symlink. cmd/main.go, internal/hookctx, semantic/embed.go, e5shadow/cmdvalidate.go. None of these numbers or the helper-command policy appear in user docs.
  4. Operational knobs. Event-log rotation (DOWNSHIFT_LOG_MAX_MB, DOWNSHIFT_LOG_MAX_BACKUPS), DOWNSHIFT_NB_UPSHIFT, DOWNSHIFT_NB_DOWNSHIFT, DOWNSHIFT_QUIET, and DOWNSHIFT_CONTROL_SHARE are listed in docs/guides/config.md. DOWNSHIFT_E5_*, DOWNSHIFT_GRAPHIFY_CMD, and DOWNSHIFT_MINILM_EMBED are gone.
  5. Graph escalation removed 2026-10-17.
  6. Second stderr line per spawn. downshift: honor=unobserved quota=<available|held|unknown> usage=unobserved is printed for every decision on every harness; only docs/guides/daily-truth.md describes it.
  7. Antigravity multi-child rule. The decision, the task text and the written model all follow the last subagent that has a Prompt; earlier children are not routed. antigravity.go decidedSubagent, TestMultiChild_*. Only the hardening audit and CHANGELOG mention it.
  8. Decision-intelligence shadow record and in-process extras. Every decision event carries a decision_intelligence advisory block (Apply: false); internal/lifecycleobserver compiles but has no CLI or hook path (internal/orchestration and internal/routecache were removed on 2026-10-19). Design docs exist; user docs do not say they are inert.
  9. Agent-type baseline attribution. stats --detail prints "Baseline by agent type" so an unknown-baseline rate can be pinned on Explore, Plan or custom. Only stats-export.md describes it.
  10. Refuted-rewrite honesty. When Claude Code reports a different resolvedModel, the child token record links to the model that ran; the rewrite is refuted and is not counted as applied. Current reports do not emit a dollar-savings field (TestAggregate_RefutedRewriteSavesNothing). Explained in the audit and CHANGELOG, not in the stats docs a user reads.

More docs