Computer use in Claude Code: fewer tokens for every desktop read

Why computer use burns tokens

An agent that drives a desktop app works in a loop: read the window, act on one element, read the window again to check what happened. Every read lands in the context, and most of it is the same as the read before.

Reading the accessibility tree as JSON is already far smaller than an image of the screen. The JSON still repeats every key name, quote mark and the snapshot id inside every element ref, and the verify read repeats the whole window to show that one button changed.

Step one: the compact view

Downshift rewrites agent-desktop snapshot and agent-desktop find through downshift compact and prints one line per element, indented by depth. Refs are written eN under a legend, the action list most elements share is written once, and states, bounds, native ids and classes get short forms. Any field without a short form is written as JSON, so nothing is dropped. A test decodes the view back to the same JSON.

agent-desktop snapshot (envelope 2.6) app="Chat" complete=true ref_count=175 snapshot_id="s9x0lmz4"
refs: eN means @s9x0lmz4:eN
actions: an eN line without actions= has Click; actions=- means none
window "general - Acme - Chat"
  group "Sidebar"
    tree "Channels"
      e5 treeitem "# general" [selected] .sidebar-item
  group "Channel"
    e174 textfield "Message #general" actions=Click,SetFocus .editor ph="Message #general"
    e175 button "Send now" [disabled]

Step two: only what changed

The tool numbers elements the same way while the window does not change; only the snapshot id is new, and older refs stay valid. So a line that reads the same in both views is the same element with the same ref. A repeat read of the same window, within 10 minutes, comes back as the changed lines:

delta from snapshot s9x0lmz4 (same window, read 4s ago): 2 of 212 lines changed.
@@ line 211
     e174 textfield "Message #general" actions=Click,SetFocus .editor ph="Message #general"
-    e175 button "Send now" [disabled]
+    e175 button "Send now"
[downshift compact: compacted 26839 to 668 bytes; full output: downshift retrieve d3bb124d09d8]

The delta is sent only when both reads are complete, the raw read is kept locally so downshift retrieve can bring it back whole, and the delta is under 60% of the view. Otherwise the full view goes out.

How much smaller

Measured with the downshift binary on synthetic reads built to the tool's output schema, not on a live desktop. Bytes include the marker line.

The tool's own -i, --compact and --skeleton flags shrink the tree first; Downshift stacks on top. Every read is counted in downshift report, downshift stats and the session panel, with bytes read back through retrieve subtracted.

Turn it on

In Claude Code with the Downshift hook there is nothing to do: the read is rewritten for you. In any other harness, or a script, the agent runs the read through the compactor, and both steps apply:

downshift compact -- agent-desktop snapshot --app Files -i

Set DOWNSHIFT_DESKTOP_DELTA=0 to send every read whole. The MCP form of the tool is not compacted, because Downshift does not see MCP results.

Limits

• Agents that share a session, or direct calls from one directory, share the delta scope. An agent can get a delta against a read it never saw; the delta says so and points to downshift retrieve.
• A new element early in the tree renumbers the refs after it, and the full view is sent instead.
• Every figure here is from synthetic reads. Run the same task with DOWNSHIFT_EXEC=0 and with the hook on, and compare downshift report.

Frequently asked questions

Does Downshift take screenshots or drive the desktop?

No. Downshift does not install or run a desktop driver. It only shortens what agent-desktop prints when the agent reads the screen.

Is anything lost in the compact view?

No field is dropped. A test decodes the view back to the same JSON, and the raw read stays one downshift retrieve away.

Does it work outside Claude Code?

Yes. Claude Code rewrites the read automatically. In other harnesses the agent runs downshift compact -- agent-desktop snapshot ..., and the delta is scoped to the working directory.

How much does it save?

On synthetic reads, 53–60% on a first read, about 97% on a repeat read after one action, and 84.9% over a read, act, verify loop of three reads. It has not been measured on a live desktop yet.

Related guides

• Long test and build output in Claude Code: why it burns context and how to cut it
• Claude Code token optimization, ranked by evidence
• Do Claude Code subagents burn your usage limit faster?

Sources

1. Downshift guidemeasured
   docs/guides/agent-desktop.md
   how the view and the delta work, the conditions and the limits
2. Downshift fixturessynthetic
   internal/compressor/testdata/corpus
   the reads behind the table, built to the tool's output schema; measured with the downshift binary on 2026-11-09
3. Round-trip testmeasured
   agent_desktop_roundtrip_test.go
   decodes the compact view back to the same JSON