Real factory example · captured NeoDash results

Adding runner token usage to observability

I want to add details about token usage on the runners to the observability system. What subsystems are affected? When was this last modified, and why?

01 · CODING AGENTasks
02 · GRAPH AGENTplans
03 · CYPHERretrieves
04 · GRAPH AGENTchecks
05 · CODING AGENTacts
Graph agent plan

Split the question into claims.

  1. Locate the code that reads runner usage.
  2. Find its reviewed subsystem membership.
  3. Trace production callers and static test candidates.
  4. Read the surviving-line history and look for rationale.
  5. State gaps before handing the result back.
Field evidence

Establish what writes the field.

ownerpathstart lineend linerolebasis
record_usage#1yesod/src/yesod/run_ledger.py264268sql_writeestablished
ownerpathstart linebasis
LedgerPrimitiveTests#1.test_record_usage_writes_token_columns#1yesod/tests/test_run_ledger.py424lexical

The exact-commit AST inspection identifies an established SQL write and a static test reference. Its selected-file flow was truncated and never claims a complete writer inventory; the database connection's global identity also remains unestablished.

Cypher 1

Locate the implementation.

MATCH (f:Function {
  repository_id: $repository, module_name: $module, qualified_name: $target
})
MATCH (f)-[:SUPPORTED_BY]->(span:SourceSpan)
OPTIONAL MATCH (sub:Subsystem)-[:CONTAINS]->(m:Module {
  repository_id: $repository, module_name: $module
})
RETURN f.qualified_name AS function, f.path AS path,
       f.start_line AS start_line, f.end_line AS end_line,
       collect(DISTINCT sub.name) AS subsystems,
       collect(DISTINCT span.locator) AS evidence,
       f.id AS source_node
functionpathstart lineend linesubsystems
parse_usage_from_logyesod/src/yesod/dispatcher/usage.py119153factory, dispatcher

This establishes static location and reviewed taxonomy membership. It does not establish the complete observability data path.

Cypher 2

Ask when those surviving lines entered the code.

MATCH (f:Function {
  repository_id: $repository, module_name: $module, qualified_name: $target
})
MATCH (commit:Commit)-[touch:TOUCHES]->(f)
WHERE touch.join = 'git-blame-current-line'
OPTIONAL MATCH (commit)-[:TOUCHES]->(note:Note)
RETURN commit.object_id AS commit, commit.committed_at AS committed_at,
       note.natural_key AS note_id, note.title AS note_title,
       note.status AS recorded_status, touch.surviving_line_count AS surviving_lines,
       commit.evidence_json AS commit_evidence, note.evidence_json AS note_evidence
ORDER BY committed_at DESC, commit, note_id LIMIT $limit
commitcommitted atsurviving lines
b6ad1ffe3597e05be49d80ea95079fbc399cddd62026-06-11T01:55:49+00:0035

No note was directly linked to this function's contributing commit. The date describes surviving blamed lines, not every later edit or a deployment.

Follow-up

Trace callers, tests and adjacent history.

MATCH (f:Function {
  repository_id: $repository, module_name: $module, qualified_name: $target
})
MATCH (caller)-[call:CALLS]->(f)
RETURN caller.module_name AS module, caller.qualified_name AS caller,
       caller.path AS path, caller.start_line AS line,
       call.evidence_json AS evidence
ORDER BY module, caller LIMIT $limit
callerpathline
Runner._record_run_usageyesod/src/yesod/runner.py7006
Runner._sample_input_tokensyesod/src/yesod/runner.py7419
Show 7 static test candidates
selectorbasiscertainty
yesod/tests/test_run_usage.py::test_parse_log_binary_garbage_yields_nonestatic_astinferred
yesod/tests/test_run_usage.py::test_parse_log_empty_file_yields_nonestatic_astinferred
yesod/tests/test_run_usage.py::test_parse_log_finds_result_among_stream_linesstatic_astinferred
yesod/tests/test_run_usage.py::test_parse_log_huge_log_still_finds_tail_resultstatic_astinferred
yesod/tests/test_run_usage.py::test_parse_log_last_result_winsstatic_astinferred
yesod/tests/test_run_usage.py::test_parse_log_missing_file_yields_nonestatic_astinferred
yesod/tests/test_run_usage.py::test_parse_log_plain_text_yields_nonestatic_astinferred
MATCH (f:Function {
  repository_id: $repository, module_name: $module, qualified_name: $target
})
MATCH (caller)-[:CALLS]->(f)
WHERE caller.path STARTS WITH 'yesod/src/'
MATCH (commit:Commit)-[touch:TOUCHES]->(caller)
WHERE touch.join = 'git-blame-current-line'
MATCH (commit)-[:TOUCHES]->(note:Note)
RETURN caller.qualified_name AS caller, commit.object_id AS commit,
       commit.committed_at AS committed_at, note.natural_key AS note_id,
       note.title AS note_title, note.evidence_json AS evidence
ORDER BY committed_at DESC, commit, caller, note_id LIMIT $limit
callercommitcommitted at
Runner._sample_input_tokens1d80c73dc7a0d273245ad20bf1a5255c30b9c7f42026-06-03T14:13:56-07:00

The follow-up found adjacent planning context but no direct rationale. Its private note metadata is omitted, and the graph agent does not promote it into an explanation of the parser.

Graph agent → coding agent

A bounded answer

yesod.run_ledger.record_usage writes runs.tokens_in in an SQL update at lines 264-268. Start tracing the input at yesod.dispatcher.usage.parse_usage_from_log. The reviewed graph places that parser in the factory and dispatcher subsystems. Production callers in yesod.runner record run usage and sample input tokens; the graph also returns 7 inferred test candidates.

The parser's surviving implementation lines trace to b6ad1ffe3597e05be49d80ea95079fbc399cddd6, committed 2026-06-11T01:55:49+00:00. Its commit subject records the purpose as capturing Claude-lane token usage into the runs ledger. This is a recorded change description, not a deeper design rationale or deployment receipt.

Next read-only checks: trace every observability exporter and downstream consumer; inspect the listed tests; establish the connection's database identity; check runtime and deployment receipts. The current selected-file field analysis is incomplete.

Evidence receipt

Factory capture
977ac3375c295661b5eca70e9aa5fd6b8235be6fd7f27ca4801a3952e17b3194
Graph source digest
9aa1d439fbd7c1ab6a986decb739b40df86564684942c94824ed75a96080e6f4
Compared source commit
c4e46b70399608adee30dfba367cef1fd44819b0
Captured
2026-09-14T07:19:55.844475+00:00
Field-flow report
a09c567bda00521656fe3ee6967feb1d5715a611091081cc9b19f5716b596c1f
Field-test report
dc9f0572f097071245987e71a348cf8e6337ff3e363d992fcd894aaf876870a6
Gallery revision
fa38b95ab200e4df31b72dd9d55108c5bab63db5
Execution label
protected NeoDash query plus exact-commit AST inspection; reviewed captured result shown here
Remote model calls
0