← Back to all projects
In Progress Created 2026-08-04

RBD Company Brain — Architecture Plan

Class: PREP — Research → Plan → Opus critique → /grill-me → Execute

Objective

Build a "Company Brain" at Riley Blake Designs: an accurate, permissioned, multi-employee AI agent that answers business questions over RBD's real data. First deliverable is a sales-intelligence agent. This is the first real-world productization of the Outpost pattern (solo Mission Control → multi-employee deployment).

Requirements (Cole's stipulations, Day 1)

  1. Data accuracy — data flows cleanly and correctly from all sources; every source vetted before it serves an employee.
  2. Employee interface — a surface employees actually use; must handle ~concurrent load (7–8+ simultaneous is trivial on Claude API).
  3. Harness — real agent infrastructure, not a toy.
  4. Robust permissioning — role-scoped data access enforced at the tool layer, never in the prompt.

Locked Decisions (Q1–Q4)

Q1 — Beachhead: Sales-Intelligence Agent

Not just rep-scoped lookups — the broader sales intelligence surface. Must handle both:

  • Rep-scoped: "What did Customer X buy in the last 60 days?" / "Which of my customers haven't reordered Confetti in 90 days?"
  • Aggregate / role-gated: "How many customers bought into the Sept 2026 collection?" / "Adoption rate on new Confetti colors?" / "Top 20 SKUs by unit velocity" (dollars gated to design leadership + up)
  • Boundary: reps see own book of business in $, aggregate views WITHOUT $; design leadership sees aggregate WITH $, no rep-specific breakdown; sales managers see their team; exec sees all.
  • Chosen because it has the largest user base (adoption metric is meaningful) AND forces the permissioning pattern from day one.
  • Origin trigger: design president asked today "how many customers bought into the collection?"

Q2 — Interface: Web App (primary) + Email Fallback

  • RBD tooling reality: Google Workspace + NetSuite + email. No Slack, no Teams.
  • Primary: custom web app at e.g. brain.rileyblake.com, Google SSO login, chat UI, bookmarkable.
  • Fallback: email to e.g. brain@rileyblake.com, inbound-triggered, reply-only.
  • Google Chat bot was considered (every employee has it via Workspace) — keep as a possible alt/add-on; web app chosen for control over UX + permissioning surface.
  • Email safety design (REQUIRED — carries a documented exception to Cole's "never send emails" rule; scoped to inbound-triggered internal-only reply):
  • Send LINKS, not data — email body = "your report is ready: [link]"; actual numbers render behind SSO in the web app. Eliminates wrong-recipient leak, forwarding leak, Gmail-index sprawl.
  • Sender allowlist: @rileyblake.com only; drop others silently (no bounce).
  • Auto-reply detection: drop Auto-Submitted: auto-generated / X-Autoreply (kills reply loops).
  • Rate limit per sender.
  • Prompt-injection guardrail on inbound body (untrusted input) before it hits the LLM.
  • Reply-only, never CC/BCC, never proactively email, never agent-chosen recipients (routing = deterministic code).
  • Audit-log every inbound query + outbound reply separately from Gmail.
  • NO scheduled/proactive outbound in v1 — reactive only.
  • Prereq status: DMARC/DKIM/SPF already configured on rileyblake.com ✅ (confirmed 2026-08-04).

Q3 — Backend: 100% Claude API, No Self-Host

  • No compliance/legal requirement forcing on-prem. RBD is a private manufacturer; customer names + SKU + revenue are not regulated data; Anthropic DPA (no training, no retention) covers it.
  • Self-host was evaluated and rejected: ~10× complexity, ~5× cost, ~15–25% quality drop. Only revisit if a customer contract or Bret draws a hard "data can't leave network" line.
  • Cost sizing: usage assumption ~50 users × 2–3 queries/day (Cole: "2 or 3 requests a day AT MOST") = ~2,750 queries/mo.
  • Sonnet 4.6 primary + ~20% Opus 4.8 + guardrails + retrieval + hosting = ~$500–1,500/mo all-in.
  • Escalation trigger: if usage → 10 queries/user/day, revisit at ~$3–5K/mo.
  • Budget conversation with Bret is a plan prereq; design for $1,500/mo working assumption.

Q4 — Harness: Claude Agent SDK + thin FastAPI orchestrator

  • Decision: Claude Agent SDK (Anthropic-native; renamed from Claude Code SDK early 2026) as the harness, wrapped by a ~thin FastAPI orchestrator that: (1) resolves SSO user → rep_id/role, (2) picks the per-role tool bundle, (3) logs every call to a Postgres audit table.

  • Why: tool-first design IS the permissioning primitive (agent = model + tools; LLM can't get data its tools don't return); hierarchical subagents (2026) for per-role scoping; fallback model chains (Sonnet→Opus) fit the cost profile; simple auditable loop; Python-native (matches mission-control/WarehouseAPI).

Harness landscape research (2026) — evaluated, ruled out

  • OpenClaw (Steinberger, 250K stars/60 days) — personal 24/7 assistant framework. Wrong tool: single-user, nothing solves multi-tenant permissioning.
  • Hermes Agent (NousResearch, 188K stars/4 mo, MIT) — genuinely aligned philosophy but 6mo old, model-agnostic (unused), multi-platform surface (unwanted). Don't adopt as harness. Steal patterns instead (below).
  • LangGraph 1.0 — right for stateful workflow agents (Klarna/Uber/LinkedIn); wrong for Q&A. Revisit for a future workflow-agent tier.
  • CrewAI — prototype-grade role multi-agent; overkill.
  • MS Agent Framework — .NET shops; not us.
  • Anthropic Managed Agents — check current pricing during plan write-up; could replace self-hosted VPS line if cheaper.

Hermes patterns to PORT into the Claude Agent SDK build (the real takeaway)

Hermes = Cole's Alpine setup formalized (Soul=CLAUDE.md, Memory=MEMORY.md, Skills=.claude/skills, Crons=LaunchAgents, Self-Improvement="update lessons.md after corrections"). Independent convergence on the 188K-star architecture = strong validation + an Outpost pitch asset. Five mechanisms worth stealing:

Hermes mechanism RBD implementation
Progressive disclosure (3-tier skill loading): L0 skills_list() {name,desc,category} ~3k tok → L1 skill_view(name) full → L2 skill_view(name,path) ref file Tool/skill menu at ~3k tokens; full defs loaded on demand → keeps agent cheap at 40+ tools. Implements Cole's Instruction-Budget rule. Build into orchestrator day one.
Write-approval gate: self-edits staged to pending/, survive restart, human /approve Brain-steward approval queue for any agent-proposed skill/memory change. Enforcement layer for "PREP only, human activates."
patch vs edit (patch preferred, token-efficient) Standard Edit discipline (already Cole's habit).
Grade-and-prune schedule (skills tracked for reliability, pruned on schedule) Scheduled "which skills/lessons earned their keep" audit → feeds the Constraint Audit backlog item. The piece lessons.md currently lacks.
Lineage session compression: close SQLite session row → child session seeded by summary → rotate ID → parent/child lineage + session_search tool; head/tail token-protected Postgres parent/child session rows + session_search tool → employees get real conversation history ("what did I ask last Tuesday?"). Cleaner than ever-growing context.

Do NOT steal: the whole framework; ~/.hermes/ flat-file storage (RBD is server-side multi-tenant → Postgres, role-scoped); autonomous cron self-editing (must be gated for customer-data agent).

Stack shape

[Web app UI]  ─┐
[Email inbound]─┼─→ [FastAPI orchestrator]
                    ├─→ [Google SSO → rep_id/role resolver]
                    ├─→ [Tool bundle picker (per role)]
                    ├─→ [Guardrail (Lakera) on inbound]
                    ├─→ [Claude Agent SDK w/ scoped tools]
                    │       ├─→ get_my_customers(rep_id)
                    │       ├─→ get_my_customer_orders(rep_id, customer_id)
                    │       ├─→ get_top_skus_anon(collection, mode='rank_only')  # NO $ column returned
                    │       └─→ ... more tool wrappers ...
                    └─→ [Audit log → Postgres]

OPEN — still to design

Q5 — Permissioning (DESIGN LOCKED 2026-08-04 — dial values pending Cole's org confirmation)

Vetting = confirm, not define (Cole 2026-08-04): the 4-dial model + role presets stand as the working design; Cole does a sanity pass against the real org and adjusts specific dial values. NOT a working session with dept heads. Q5 is closed for architecture purposes. Core rule: permissions enforced at the tool/retrieval layer, NEVER in the LLM prompt. Never hand the model data it shouldn't see; the tool physically doesn't return the restricted column (e.g. dollars). The LLM can't leak what it never received.

Source of truth (DECIDED 2026-08-04): we BUILD our own roles table. NetSuite employee list exists but "isn't thorough enough" — use it only to SEED (names/emails/rep_id; sales reps come free since employee_name is the rb_warehouse rep join key). Enrich by hand. ~30–50 rows, Postgres, maintained by the brain steward (Cole day 1). Login: Google SSO → email → roles-table lookup → load dials → orchestrator exposes only permitted tools. No row = no access (fail closed).

Permission model — 4 orthogonal dials, not 8 bespoke bundles. Insight from Cole: "similar to sales reps except admin levels" → roles are presets across the same dials, not custom logic per role.

  • ① Row scope: own / team / all / none — whose customers & orders?
  • ② Dollar visibility: own-only / full / none — can they see $, and whose?
  • ③ Aggregate access: rank-only / full / none — company-wide views (top SKUs, adoption) with or without $?
  • ④ Data domains: {sales, inventory, finance, product/design} — which subjects on the menu?

Dollar-suppression Cole cared about falls out of ②+③ automatically: rep asks "top Confetti SKUs" → get_top_skus(mode='rank-only') (their aggregate dial) → rank + %-of-total, NO $ column in result set. Same tool, different dial.

Role presets (PROPOSED 2026-08-04 — pending Cole vetting w/ org): | Role | ① Row | ② Dollars | ③ Aggregates | ④ Domains | |---|---|---|---|---| | Sales Rep | own | own-only | rank-only | sales | | Sales Manager | team | team | rank-only | sales | | Customer Care (NEW) | all (read) | none | rank-only | sales, inventory | | Designer | none | none | rank-only | product | | Design Leadership | none | full | full ($) | product, sales | | Finance (Ken) | all | full | full ($) | finance, sales | | Warehouse (Brett) | none | none | full (units) | inventory | | Exec | all | full | full ($) | all | Customer Care ≈ sales rep with row scope widened to all + dollars off. inventory access = guess (order-status/stock lookups?), flag for vetting.

Open sub-questions for Cole:

  • Sales Manager: whole-team dollars, or own $ + team rank-only? (currently: team)
  • Finance: needs row-scoped customer lookup, or aggregates only? (currently: all + full)
  • Confirm fail-closed default (assumed yes).
  • "Vetting" = confirm roles w/ org, or working session w/ dept heads to define each role's dials? If latter → plan prereq.

Q6 — Data accuracy (stipulation #1) — SCOPED TO BEACHHEAD (IN PROGRESS)

Scoping win: the sales-intel beachhead touches ONE source — rb_warehouse (NetSuite mirror). All sales/invoices/inventory flow through it (Cole 2026-08-05: "might not be the only but it is for now"). Do NOT vet Shopify/inventory-snapshots/financials-JSON/Drive for phase 1 — they come online with later agents. Freshness: rb_warehouse mirror is nightly (Cole). Agent MUST state as-of ("as of last night's sync") on any time-bounded query.

Accuracy = tool-layer rules (mostly already solved in memory playbooks). Each gotcha → a rule baked into the wrapper so the agent physically cannot emit a wrong number: | Gotcha | Trap | Wrapper rule | |---|---|---| | AR double-count | netsuite_transaction_items double-counts revenue via 12000 AR lines | filter expense_account_name LIKE '4%' on revenue | | Unapproved payments | inflate/distort totals | exclude per reconciliation playbook | | Tariffs | pollute line-item aggregates | handle per playbook | | Credits / net_amount | returns/credits must net | use net_amount, not gross | | is_closed / phantom lines | ghost quantities | filter phantom lines | | Pre-sale "hole" | qty − shipped − backordered = real pre-sold gap | documented formula, not raw qty | | Source lag | Tableau/NetSuite/P&L differ by timing | reconcile to NetSuite/P&L; state as-of | | Noise item lines | ShipItem/Discount/Subtotal/Payment/Markup/Group/Description pollute qty & revenue | filter item_type IN ('InvtPart','NonInvtPart') on every line-item query (Cole's Tableau filter, 2026-08-05) |

⚠️ BIGGEST GOTCHA (Cole 2026-08-05): Invoice vs SalesOrd + combined statuses.

  • "The biggest issues are invoice vs SalesOrd. There are lots of statuses that get combined for invoices."
  • Not a cleaning detail — a semantic design decision. "What did Customer X buy in last 60 days?" is ambiguous: Sales Orders (ordered/committed, incl. pre-sales) vs Invoices (shipped/billed/realized). RBD does heavy pre-sale (anticipated_release_date) → the two numbers diverge massively (e.g. $50K Jan SO for a Sept collection = real "buy" to a rep, $0 invoiced for 8 mo).
  • CRITICAL UX INSIGHT (Cole 2026-08-05): reps DON'T clarify booked vs invoiced — "it can be confusing" even human-to-human. → The confusion IS the opportunity. Agent must NOT guess a lens; it must be the thing that makes the distinction clear every time. Turns the #1 data landmine into the agent's headline value-add.

  • DESIGN LOCKED — dual-lens answer, led by booked, plain-English labels:

    Customer X — last 60 days (as of last night's sync) • Booked (ordered): $47,200 — incl. $30K pre-sold (Sept collection, not yet shipped) • Invoiced (shipped & billed): $17,100

  • Lead with booked (SalesOrd) = rep's commission-relevant instinct ("what I sold").

  • Always show invoiced (CustInvc) beside it so nobody must know to ask.
  • Collapse to one line only when booked == invoiced (fully shipped, no pre-sale).
  • Plain-English gloss ALWAYS: "Booked (ordered)" / "Invoiced (shipped & billed)" — never raw NetSuite jargon.
  • Apply revenue-sign rule (refund types negate net_amount) + AR-double-count filter underneath both numbers. See [[reference_rbd_revenue_sign_txn_types]].
  • Glossary/disambiguation layer baked into agent system prompt: booked=SalesOrd=ordered; invoiced=CustInvc=shipped/billed; pre-sold=ordered-but-unshipped. Speaks reps' language, precise underneath. Cheap, high clarity payoff.
  • OPEN — Phase-0 read-only query task (not blocking design): enumerate distinct rb_warehouse [type] values + exact invoice status values that "get combined" (which to include/exclude). Seed from Cole's Tableau revenue-sign formula (refund types: CashRfnd/CustCred/RtnAuth/CustRfnd/CardRfnd → negate net_amount). Formula "still might change a little."

Golden-set (accuracy gate): ~10 real questions w/ hand-verified answers → nothing ships until agent passes all 10. Cole to co-produce. This is the "validate its own work" lever made concrete.

Later-phase sources (deferred): WarehouseAPI per-store MySQL (Shopify truth); Shopify APIs; financials JSON/GL; netsuite_inventory_snapshots (BROKEN since 2026-05-24, fix before inventory agent); Google Drive/Sheets.

PART 2 — FULL ARCHITECTURE (drafted 2026-08-05, data-validated)

2.1 System components (concrete stack)

                    ┌─────────────────────────────────────────────┐
  Google SSO ─────► │  FastAPI orchestrator  (small VPS / rb infra)│
  (employee email)  │  1. authn: Google OAuth → email              │
                    │  2. identity: email → netsuite_employees row │
  Web app  ────────►│     → {ns_employee_id, is_sales_rep, role}   │
  (brain.rileyblake)│  3. role → 4-dial preset → tool bundle       │
  Email fallback ──►│  4. Claude Agent SDK loop w/ scoped tools    │
  (brain@, reply)   │  5. guardrail (Lakera) on inbound            │
                    │  6. audit log every turn → Postgres          │
                    └───────────────┬─────────────────────────────┘
                                    │  scoped tool calls only
                    ┌───────────────▼─────────────────────────────┐
                    │  DATA PLANE — permissioned tool wrappers     │
                    │  (SQL templates w/ canonical filters baked)  │
                    │   → rb_warehouse (read-only, nightly mirror) │
                    └──────────────────────────────────────────────┘
  • Harness: Claude Agent SDK (Python). Model routing: Sonnet 4.6 default → Opus 4.8 fallback for complex synthesis (SDK fallback chains). Haiku 4.5 for cheap classification (intent, glossary).
  • Identity DB: netsuite_employees (email, ns_employee_id, is_sales_rep, is_inactive) = authoritative. Cached nightly into a local roles table (Postgres) enriched with non-rep roles + the 4 dials.
  • Audit/session store: Postgres — audit_log (turn-level) + sessions (parent/child lineage, per Hermes pattern) + roles.
  • Hosting: small VPS or existing rb infra. Concurrency (7-8+ simultaneous) is trivial on Claude API.

2.2 Data plane — the tool wrappers (grounded in VALIDATED filters)

Every wrapper is a parameterized SQL template with the canonical recipe baked in, row-filtered by caller identity. The LLM calls the tool; it never writes SQL. Canonical base clause (all live-verified 2026-08-05):

-- revenue lens
WHERE ti.expense_account_name LIKE '4%' AND ti.expense_account_name NOT LIKE '40600%'  -- 4xxxx revenue, drop 12000 AR + 40600 tariff
  AND ti.item_type IN ('InvtPart','NonInvtPart')     -- real product; drops $764M GL/noise
  AND ti.is_closed <> 1                               -- is_closed=1 = CANCELLED
-- date: COALESCE(NULLIF(t.tran_date,'1970-01-01'), t.created_date)   -- epoch/null guard
-- sign: SUM(CASE WHEN t.type IN ('CashRfnd','CustCred','RtnAuth','CustRfnd','CardRfnd') THEN -ti.net_amount ELSE ti.net_amount END)
-- booked = t.type='SalesOrd' ; invoiced = t.type IN ('CustInvc','CashSale')

Rep attribution — TWO filters (5.4% of orders conflict; see data-dictionary "REP ATTRIBUTION"):

  • customer-level c.sales_rep_id = me (my accounts) vs transaction-level t.employee_id = me (my orders). Default for "my sales" = transaction-level (Cole's steer). Join t.entity_id = c.ns_customer_id.

Core tool set (Phase 1): | Tool | Row filter | Returns | Dial gate | |---|---|---|---| | get_my_accounts(rep_id) | c.sales_rep_id = caller | assigned customer list + booked/invoiced totals | Row scope=own (customer-level) | | get_my_sales(rep_id, period) | t.employee_id = caller | orders I wrote, dual-lens booked/invoiced | Row scope=own (txn-level) | | get_customer_orders(rep_id, customer) | caller owns account OR wrote order (per Cole's a/b/c pick) | X's orders, each labeled by writing rep, dual-lens, as-of | Row scope, $ own-only | | get_customer_reorder_gaps(rep_id, collection?) | caller's accounts | who hasn't reordered in N days | Row scope=own | | get_top_skus(collection, mode) | none | rank + %-of-total; mode='rank_only' omits $ column entirely | Aggregate dial: rank-only vs full | | get_collection_adoption(collection) | none | # customers bought in, % of active base | Aggregate dial |

  • Dollar suppression is physical: mode='rank_only' builds a SELECT with NO net_amount column. Reps' agent only ever gets the rank-only tool. LLM cannot leak a column it never received.
  • Rep-lens explicit: agent always states whether a figure is "orders you wrote" vs "your accounts" — never blends (like booked/invoiced).
  • RtnAuth: include both RtnAuth + CustCred as negatives (distinct events, per Cole — not a double-count); 📌 confirm-later note only, not blocking.

2.3 Identity & permission implementation

  1. Google OAuth → verified email.
  2. SELECT ns_employee_id, is_sales_rep, is_inactive FROM netsuite_employees WHERE email=?. Inactive/missing → fail closed ("not provisioned, contact admin").
  3. roles table maps email → role → 4 dials (Row/Dollars/Aggregate/Domains). Seeded: is_sales_rep=1 AND is_inactive=0 → Sales Rep preset; non-rep roles hand-enriched.
  4. Orchestrator loads the dial preset → exposes only that bundle's tools to the Agent SDK instance.
  5. Every tool call re-checks caller identity server-side (defense in depth; never trust the LLM to pass the right rep_id — orchestrator injects it).

2.4 Interface layer

  • Web app (primary): Google SSO, chat UI, session history via sessions table. brain.rileyblake.com.
  • Email fallback: inbound brain@ → allowlist @rileyblake.com → drop auto-replies → guardrail → same orchestrator → reply with LINK to web app, never data in body. Reactive only, no proactive/scheduled outbound v1. (DMARC/DKIM/SPF ✅.)

2.5 Observability & the accuracy gate

  • Audit log: every turn = {email, ns_employee_id, role, question, tools+args, response, tokens, latency, as-of}. Separate from Gmail; tamper-evident.
  • Langfuse for traces; computational sensors before inferential (SQL runs + row-count sanity before LLM-as-judge).
  • Golden set (SHIP GATE): ~10-15 real questions with Cole-verified answers (reconciled to NetSuite/P&L). Agent must pass ALL before any rep sees it. This is the "validate its own work" lever. Re-run on every wrapper change.

2.6 Rollout (sequenced, with acceptance criteria)

  • Phase 0 — Foundations. Build roles/audit/sessions tables; SSO + email→employee resolver (fail-closed); the 5 core tool wrappers w/ canonical filters; golden set authored + passing. Accept: golden set 100% green; a fake rep identity cannot fetch another rep's rows (permission test).
  • Phase 1 — Rep-scoped pilot. Web app, ~3 friendly reps, rep-scoped tools only (own customers, dual-lens, reorder gaps). Accept: 3 reps use it unprompted for 2 wks; zero cross-rep leak; answers reconcile.
  • Phase 2 — Role-gated aggregates. Add get_top_skus/adoption with dial-based $ suppression; onboard design leadership + a sales manager. Accept: rep gets rank-only (no $), design leadership gets full $, verified by audit log.
  • Phase 3 — Broaden. Email fallback live; roll to full rep roster + Customer Care; add Finance/Warehouse domains later. Accept: >40% weekly-active among onboarded (else kill/rethink surface).
  • Kill criteria: <40% weekly-active at month 3 = UI/adoption failure → rethink surface before adding scope.

2.7 Open items before build

  • Rep row-scope default (Cole) — a/b/c: on own account, see another rep's orders? (b union+labeled recommended).
  • ~~RtnAuth rule~~ ✅ RESOLVED (include both; confirm-later note only).
  • NonInvtPart meaning (Cole, minor).
  • Non-rep role dials — confirm design/finance/warehouse/exec/customer-care presets against real org.
  • Hosting decision — VPS vs existing rb infra; who owns the OAuth app registration on RBD Workspace.
  • Golden-set authoring — co-produce ~12 Q+verified-A with Cole.
  • Anthropic Managed Agents pricing — verify vs self-hosted VPS (could simplify ops).

Next Steps

  1. [x] Q1–Q6 decided + data-validated (2026-08-05)
  2. [x] Full architecture drafted (Part 2)
  3. [ ] Cole: RtnAuth rule + non-rep role dials + hosting/OAuth owner
  4. [ ] Co-author golden set (~12 Q&A, reconciled)
  5. [ ] Verify Anthropic Managed Agents pricing; Company Brain benchmarks (Glean/Coda/Copilot)
  6. [ ] Opus plan critique → /grill-me → then Phase 0 build
Source: ~/ai-projects/mission-control/plans/rbd-company-brain-architecture.md