Bookkeeping with your agent team

This is the first real use case for a Coder + Confidential Agent Zero pair: keeping the books for a self-employed person or small business. It exercises every mechanism from the prerequisites — profiles, isolation, A2A, and secret management.

Tax preparation is not covered here. This playbook gets collection + bookkeeping working; tax forms are a later phase layered on top of a clean ledger.

Prerequisites (all three, in order)

1. Give Agent Zero a coding-capable model
2. Configure profiles for roles and tasks
3. Set up a second, confidential instance

The invariant that governs everything here

No monetary value ever passes through a model's tokens. Amounts move file → code → ledger. A model may read data to decide (a category, an account mapping, "these two rows are a candidate match") but never to transcribe a number.

This is not paranoia — it's the only way a language model can be safely involved in bookkeeping at all. Every design decision below follows from it.

The tool stack (all open source)

Function	Tool	Note
Ledger engine	Beancount (subprocess)	Plain-text double-entry; per-posting cost supports per-wallet basis
Ledger web UI	Fava	Runs alongside Beancount
Crypto data collection	rotki (subprocess)	AGPL; process boundary, not linked
Invoice/receipt OCR	invoice2data	MIT; template-based extraction
Exchange/market data	ccxt + first-class CSV import	Provider-agnostic; a vendor's free tier is not infrastructure
Dedup / matching	deterministic Python scorer (start here)	Add ML later only if it earns its place

Milestone 1 — boring path, zero AI in the money flow

The very first goal has no model touching any number:

bank CSV ──► typed facts ──► Beancount ledger ──► register / balance sheet
exchange CSV ─┘

1. Export a bank statement or exchange history as CSV.
2. A deterministic Python script (written by the Coder, reviewed before it runs) parses columns → typed transaction records.
3. Those records become Beancount entries via another deterministic step.
4. Run bean-report for a register / balance sheet / income statement.

If this works cleanly with no model in the loop, every later feature inherits the same guarantee. This is your baseline.

Milestone 2 — ingestion hardening (AI enters safely)

Once the boring path works, AI helps with the messy parts — but always under the invariant:

• Receipts / emails → typed facts: the Confidential agent's local model reads a receipt image and outputs a categorical JSON record (merchant_category, date, "this is an expense"), never a transcribed dollar amount. The amount comes from OCR (invoice2data) or CSV fields — code, not tokens.
• Schema-constrained output: the local model's response is validated against a grammar/schema; invalid = REJECTED, never a guessed repair-retry.
• Computer Use for portal downloads: when a bank has no CSV export, the Collector agent (inside the confidential container) logs in and downloads files. Screenshots go only to the local model; they never cross the A2A boundary.

Milestone 3 — reconciliation is a diff, not a merge

When two sources disagree (your exchange says $100, the 1099-DA form says $98):

Both sides keep their figures. Differences must be explained, never averaged.

This deletes the scariest destructive operation from bookkeeping automation. The agent flags discrepancies; a human resolves them.

How A2A fits in (the bridge between your two containers)

The Confidential container owns all data and the ledger. When an ingestion parser breaks (a bank changed its CSV columns), it calls the Coder:

a2a_chat(agent_url="§§secret(CODER_A2A_URL)", message="This CSV parser chokes on field X",
         attachments=["/dead-drop/redacted_sample.csv"])

• Outbound: only a scrubbed fixture — synthetic rows shaped like the real file but containing no real account numbers, names, or amounts.
• Inbound code: comes back as reply text or via your git repo; a script lands it in a staging directory. The Auditor reviews it for exfil/supply-chain risk before it runs inside the confidential container.

No shared filesystem between containers. No raw data crossing the wire. The invariant holds across containers exactly as inside one.

Audit trail (cross-cutting)

Every user interaction and every agent action is logged to an append-only, hash-chained JSONL file — who/when, the decision made, which rule/tool fired, input file hashes. Corrections are appended, never overwritten. The log loads into a Jupyter notebook for review:

# quick audit replay
import pandas as pd
events = pd.read_json("audit.jsonl", lines=True)
events[events["action"] == "categorize"].tail(20)   # last 20 categorizations

This is how mistakes get found and corrected — the same epistemics as printing which rule fired and what facts it consumed, not interrogating a model about its reasoning.

For the agents — who does what

What this page is for: it tells each agent its concrete responsibilities for bookkeeping. Every action is either a premade script (deterministic, in the confidential workspace toolkit/) or created through the Coder-prepares protocol; nothing relies on a model producing a number.

Actor	Responsibilities	How
Confidential agent	categorize transactions; trigger ingest; flag reconciliation diffs; log every decision	Categorical JSON decisions only. Run toolkit/redact_fixture.py before any A2A handoff; toolkit/audit_log.py append after each decision. Full task list: confidential instance playbook → task catalog.
Coder	write/fix the deterministic CSV→ledger ingest, parsers, and importers	Receives a redacted fixture + a description of column shape, never real rows. Returns code as reply text or via git.
Human (owner)	approve inbound code before it runs; resolve any figure discrepancy	Reconciliation is a diff — you explain differences; no averaging, no silent overwrite.

Scope & references

• Tax preparation lives in its own use case layered on a clean ledger (working papers over Beancount with an MIT cross-check oracle; IRS e-file only as an owner-wired integration). It is not part of this page.
• At-rest encryption (gocryptfs) requires host-granted /dev/fuse + SYS_ADMIN — a host/orchestration capability, described in the confidential instance playbook.

Related

• Set up a coding-capable model (prerequisite)
• Configure profiles for roles and tasks
• Set up the confidential instance
• Rok's introduction to local-first AI: https://snap.stanford.edu/localai/