Skip to content

email-to-brain.md

Latest commit

 

History

History
218 lines (182 loc) · 9.86 KB

email-to-brain.md

File metadata and controls

218 lines (182 loc) · 9.86 KB
Raw
id email-to-brain
name Email-to-Brain
version 1.0.0
description Gmail threads flow into brain pages via the native google source kind. The open-loop engine turns them into "who is waiting on you".
category sense
requires
credential-gateway
secrets
name description where
GOOGLE_CLIENT_ID
Google OAuth2 client ID (Option A — native connector; env intake works, `--client-json` is preferred)
https://console.cloud.google.com/auth/clients — create a Desktop app OAuth client
name description where
GOOGLE_CLIENT_SECRET
Google OAuth2 client secret (Option A)
https://console.cloud.google.com/auth/clients — same client, click Download JSON
name description where
CLAWVISOR_URL
ClawVisor gateway URL (Option B — alternative hosted gateway)
https://clawvisor.com — create an agent, activate Gmail service
name description where
CLAWVISOR_AGENT_TOKEN
ClawVisor agent token (Option B)
https://clawvisor.com — agent settings, copy the agent token
health_checks
type argv label
command
gbrain
google
status
--json
Google connector
type label checks
any_of
Auth provider
type name label
env_exists
GOOGLE_CLIENT_ID
Google OAuth
type url label
http
$CLAWVISOR_URL/health
ClawVisor
setup_time 20 min
cost_estimate $0 for APIs (LLM commitment extraction has a kill switch: loops.extraction_enabled)

Email-to-Brain: Gmail Messages That Update Your Brain

Emails arrive. Brain pages get smarter. The native google connector ingests your Gmail threads into the brain, resolves senders against your contacts, and runs the open-loop engine on top: who is waiting on you, what you promised, and the context needed to respond.

IMPORTANT: Instructions for the Agent

You are the installer — but you no longer write a collector script. Earlier versions of this recipe had you build a deterministic Node.js collector (pagination, deduplication, Gmail link generation, noise filtering). All of that is now IMPLEMENTED in gbrain's google source kind — see docs/guides/google-connect.md (setup + every error and its fix) and docs/guides/open-loops.md (how detection works). Do NOT re-implement it, and do not pull emails via raw API calls: the connector handles pagination, dedup, deep links, and noise filtering correctly, resumably, and under test.

The core pattern still holds: code for data, LLMs for judgment. The code half now ships in gbrain. Your job shifts to:

  1. Run the setup (gbrain google setup) and relay every fenced [SHOW USER] ... [/SHOW USER] block to the user verbatim — paraphrasing loses load-bearing detail like "Desktop app, NOT Web application".
  2. Operate the daily triage with gbrain waiting per skills/google-loops/SKILL.md (loop closing, muting, context pulls).
  3. Make the judgment calls the connector deliberately leaves to you: which entities deserve enrichment, what the digest means for today.

Architecture

Gmail Account(s)
  ↓ (BYO OAuth via gbrain google connect; tokens in ~/.gbrain/credentials.json)
gbrain google source kind (deterministic, resumable, newest-first backfill)
  ↓ Materializes markdown in the source's MANAGED DIR (not the brain repo):
  ├── emails/{YYYY}/{MM}/...md    (type: email — one page per thread, deep links baked in)
  └── people/...md                (type: person — from Contacts, aliases for sender resolution)
  ↓ standard import pipeline (chunks, embeds, aliases, links)
Open-loop engine (per sync)
  ├── deterministic thread detector (unanswered_inbound / unanswered_outbound, free)
  └── LLM commitment extractor (last 30 days, capped; kill switch loops.extraction_enabled)
  ↓
gbrain waiting  →  the daily digest: ranked people waiting on you
Agent judgment calls: enrichment, prioritization, drafting with brain context

Opinionated Defaults (now implemented in the connector)

These rules used to be specified here for agent-authored collectors. They are now code (src/core/google/google-render.ts + loop-detect.ts) — listed so you know what the connector does, not so you re-build it:

  • Noise filtering (deterministic): noreply/no-reply/notifications@/ calendar-notification/mailer-daemon/postmaster/donotreply senders never open loops. List mail (List-Unsubscribe) is excluded too.
  • Gmail deep links are generated by CODE, never by the LLM — every thread page carries an account-correct authuser deep link, so links are always right.
  • Sent mail is ingested as the negative filter. Without it, "awaiting response" lies about threads you already replied to. Your own replies close loops automatically (closed_by: reply_detected).
  • Precision rules are pinned by a labeled fixture corpus (test/google-loop-detect.test.ts): CC-only delivery, FYI/forwards without a question, self-threads, and muted senders/threads never open loops.

Prerequisites

  1. GBrain installed and configured (gbrain doctor passes)
  2. Google access via one of:
    • Native connector (recommended): run the credential-gateway recipe, Option A
    • ClawVisor (alternative hosted gateway): credential-gateway Option B
    • Your harness's own Gmail connector, if it ships one — follow your harness's docs; this recipe covers the native path

Setup Flow

Step 1: Run the One-Command Setup

gbrain google setup --json

This walks the whole chain idempotently: credential intake (prints the GCP checklist when nothing is on file) → consent → source registration → a first budgeted sync (newest mail first — the deep backfill resumes automatically on later syncs) → the first gbrain waiting digest. Re-running is always safe.

The pieces, if you want them separately:

gbrain google connect
gbrain sources add gmail-you --kind google --account you@example.com \
  [--services gmail,calendar,contacts] [--history-days N]
gbrain sync --source gmail-you
gbrain waiting

Multiple accounts: repeat with --account work@example.com; each account is its own source with independent cursors.

Relay [SHOW USER] blocks verbatim. The whole setup is exactly two user interactions: (1) the GCP checklist + the client JSON hand-back, (2) one consent click. If you're asking a third question, re-read the block you skipped. Every command supports --json and emits { ok, status, next_action, error } — errors carry { code, problem, cause, fix, doc_url }; show the user problem + fix.

Step 2: Verify

gbrain google status --json     # per-account refresh probe
gbrain waiting                  # the first digest (refuses on stale data)

Step 3: Continuous Sync

Google sources are ordinary gbrain sources: gbrain sync --source <id>, gbrain sync --all, and autopilot all pick them up. A bare un-targeted gbrain sync (repo mode) does not — target it or use --all. No cron script of its own; the connector logs funnel events to ~/.gbrain/integrations/google/heartbeat.jsonl automatically.

Step 4: Daily Triage (YOUR job — see skills/google-loops/SKILL.md)

gbrain waiting --json        # ranked people waiting on you
gbrain loops done <id>       # user handled it
gbrain loops mute sender <email>   # never track this sender again
  • waiting REFUSES on stale data (no successful sync in 24h) and names the exact fix. Run the sync, then retry — only use --stale-ok when the user explicitly accepts stale results.
  • Present each loop with: counterparty, what's owed, the evidence quote, the Gmail deep link, and the due date. The counterparty's entity card carries the context to respond.
  • Enrichment stays your judgment call: notable senders get brain pages, timeline entries, and back-links per skills/enrich/SKILL.md.

What the Agent Should Test After Setup

  1. Connectivity: gbrain google status --json shows refresh_probe: "ok".
  2. Ingestion: gbrain search "<a recent real subject>" returns the thread page with a working Gmail deep link (opens the correct account).
  3. Loop detection: gbrain waiting groups real unanswered threads and excludes noise senders and list mail.
  4. Self-close: reply to a waiting thread, gbrain sync --source <id>, verify the loop closed itself (closed_by: reply_detected).

Cost Estimate

Component Monthly Cost
Gmail API (your own OAuth client) $0 (within free quota)
Deterministic thread detector $0 (zero LLM, always on)
LLM commitment extraction small — last 30 days only, ≤50 threads/sweep; off switch: gbrain config set loops.extraction_enabled false

Tell the user once during setup that commitment extraction sends recent email text to the configured chat provider, and name the off switch.

Troubleshooting

Every failure the connector can hit maps to a typed error code with the fix attached — docs/guides/google-connect.md#troubleshooting is the canonical table. The three users actually hit:

  • "Google hasn't verified this app" during consent → expected; it's the user's own app: Advanced → Continue.
  • access_denied_test_user → add yourself under Audience → Test users.
  • invalid_grant_testing_expiry (everything silently stopped ~day 7) → publish the consent screen to Production, then gbrain google connect --reauth <email>. gbrain doctor's google_oauth check warns once an account goes 5+ days without a successful refresh (an actively-syncing account gets no pre-warning — publish to Production).