AT Search

A decentralized search and discovery layer for AT Protocol objects, built as a Kademlia DHT overlay that routes queries to indexers without replacing AT Protocol's native identity, repo, or PDS model.

Monorepo layout and tooling

This repo is a single workspace: the root package.json lists "workspaces": ["packages/*"]. pnpm is the supported tool for installs and scripts (pnpm install, pnpm run …); the lockfile is pnpm-lock.yaml. Run installs from the repository root so workspace:* links resolve.

Packages

Directory	Package name	Purpose
`packages/common`	`@atsearch/common`	Shared types, descriptor derivation, Ed25519 signing. `pnpm --filter @atsearch/common run build` runs `tsc` to `dist/`. `pnpm --filter @atsearch/common run test` runs Jest.
`packages/indexer`	`@atsearch/indexer`	SQLite index, lexicon registry (runtime schema resolution + collection triage), DHT provider, `GET /pointers/:key`, `GET /record`, `GET /lexicons`. Depends on `@atsearch/common` via `workspace:`. `build`* → `tsc`; `dev` → `tsx watch`; `test` → Jest; `seed` → `tsx src/seed.ts`. The compiled app is run with `node dist/index.js` in production and in `scripts/run-demo.sh` because better-sqlite3 is built for Node’s native ABI.
`packages/query-node`	`@atsearch/query-node`	Search API (`/search`, `/resolve`, `/interactions`, `/lexicons`), Slingshot/Constellation/XRPC hydration, ranking. Depends on `@atsearch/common` via `workspace:`. `build`* / `dev` same pattern as indexer.
`packages/demo-client`	`@atsearch/demo-client`	SvelteKit UI. No `workspace:` dependency on `@atsearch/common`; it only calls the query node over HTTP. `vite build`* / `vite dev`.

Root scripts

Defined in the root package.json and meant to be run from the repo root:

Command	Effect
`pnpm install`	Installs all workspace dependencies and links `workspace:*` packages.
`pnpm run build`	`pnpm -r run build`: runs `build` in every package that defines it.
`pnpm run dev`	Runs each package’s `dev` script in parallel (useful less often than running one package’s dev server).
`pnpm run test`	Runs `test` in each package that defines it (@atsearch/common and @atsearch/indexer).
`pnpm run lint`	`tsc --noEmit` / `svelte-check` per package where configured.
`pnpm run seed`	`pnpm --filter @atsearch/indexer run seed`: writes seed data into `ATSEARCH_DB_PATH` (default `./data/indexer.db`).
`pnpm run demo`	`bash scripts/run-demo.sh`: builds packages, seeds, then starts indexer (Node), query node (Node), and the demo (pnpm + Vite). See that script for ports and env.

Work on one package: cd packages/<name> && pnpm run dev, or from root pnpm --filter @atsearch/<name> run <script> (e.g. pnpm --filter @atsearch/query-node run dev).

Architecture

┌───────────────────────────────────────────────────────────────────┐
│  Publisher (any AT Proto app)                                     │
│  Writes records (any lexicon) into their AT Proto repo           │
└─────────────────────────┬─────────────────────────────────────────┘
                          │ Jetstream / firehose / polling
                          ▼
┌───────────────────────────────────────────────────────────────────┐
│  Indexer Node                                                     │
│  • Triages collections via the lexicon registry (resolves        │
│    schemas at runtime; drops text-free lexicons like likes)      │
│  • Normalises records: adapter → extraction plan → heuristics    │
│  • Derives descriptor keys (type, tag, token, geo, lang)         │
│  • Stores (uri, cid, descriptors) in SQLite                      │
│  • Advertises as DHT provider for each descriptor key            │
│  • Serves GET /pointers/:descriptorKey (signed PointerRecords)   │
└────────────┬──────────────────────────────┬──────────────────────┘
             │ DHT advertise                │ HTTP /pointers/:key
             ▼                              ▼
┌───────────────────┐           ┌───────────────────────────────────┐
│  Kademlia DHT     │           │  Query Node                       │
│  (libp2p kad-dht) │◄─────────►│  • Accepts GET /search?q=…        │
│  descriptorKey    │   find    │  • Derives descriptor keys        │
│    → providers    │  providers│  • Queries DHT for providers      │
└───────────────────┘           │  • Fetches pointer records        │
                                │  • Hydrates uri+cid via Slingshot → XRPC fallback → indexer cache │
                                │  • Constellation for likes/replies │
                                │  • Verifies CID on PDS            │
                                │  • Ranks and returns results      │
                                └────────────────┬──────────────────┘
                                                 │ JSON results
                                                 ▼
                                ┌────────────────────────────────────┐
                                │  Demo Client (SvelteKit + Vite)    │
                                │  • Search input                    │
                                │  • Results, pagination, interactions │
                                │  • Verification status             │
                                │  • Matched descriptors             │
                                └────────────────────────────────────┘

Identity model

Why CID alone is insufficient

A CID (Content Identifier) is an immutable hash of a record's content at a specific point in time. Using a CID alone as an identity is incorrect for three reasons:

No location. A CID tells you nothing about where to fetch the record. You need the AT URI to resolve the DID to a PDS and fetch the record.
No ownership. The same content could be published by multiple different accounts under different URIs. Records with identical content (same CID) from different repositories are distinct objects.
No update path. When a record is updated, the CID changes. The AT URI remains stable. Treating CID as identity breaks identity across updates.

Strong reference: `uri + cid`

Every result in AT Search is represented as a strong reference:

uri: at://did:plc:abc/com.example.thing/123
cid: bafyreiXXXXXX

The URI (at://did/collection/rkey) is the durable identity. It tells you who published the record and where.
The CID is the immutable content hash for that specific version of the record.
Together uri + cid uniquely identifies a specific version of a specific record.

The current serving location (PDS endpoint) is derived at query time by resolving the DID; it is not stored as part of identity and can change without invalidating the reference.

Key invariants

Scenario	Behavior
Same CID, different URIs	Never deduplicated; these are different objects
Same URI, new CID	Treated as a new version; both uri+cid pairs are indexed
PDS migration	Identity unchanged; re-resolved from DID at fetch time
Stale DHT entry	Ignored if pointer `expiresAt` is in the past
Duplicate providers	Merged; candidates deduplicated by `uri + cid`

DHT vs AT Protocol roles

Concern	DHT layer	AT Protocol
Record storage	❌ never	✅ authoritative
Record identity	❌ never	✅ AT URI
Content truth	❌ never	✅ CID from PDS
Discovery hints	✅ descriptor → provider	❌
Pointer records	✅ (hint only, verified)	❌
DID resolution	❌	✅ PLC directory / did:web

The DHT is purely a discovery and hinting layer. Every query ultimately verifies results against the live AT Protocol PDS. No record content lives in the DHT.

Descriptor derivation

The indexer normalises several collections (see packages/common / packages/indexer ingest). Descriptor derivation runs a three-tier ladder for each record: a hand-written adapter if one exists, otherwise a compiled extraction plan from a resolved lexicon schema (ATSEARCH_LEXICON_MODE=auto), otherwise heuristic field-name guessing. From a com.example.thing record, descriptors include:

Descriptor type	Example	Source
`type:…`	`type:com.example.thing`	Static
`tag:…`	`tag:mutual-aid`	`record.tags`
`token:…`	`token:fridge`	Tokenized title + description
`geo:…`	`geo:c2`, `geo:c2b2`, `geo:c2b2n`	Geohash prefixes at lengths 2, 4, full

Rules: lowercase, stopwords removed, deduped.

Each descriptor key is hashed as sha256("atsearch:v1:" + key) before being used as a DHT CID.

Cross-lexicon indexing (lexicon registry)

AT Search indexes the whole ATmosphere (any app, any lexicon) without per-app code. Lexicons are self-describing at runtime: an NSID resolves via a DNS TXT record (_lexicon.{authority} → DID) to a com.atproto.lexicon.schema record in the authority's repo. The indexer uses that to teach itself how to index unknown collections.

Normalisation ladder (packages/common/src/normalize.ts): each record goes through the first tier that applies:

Hand-written adapter: rich extraction for ~10 popular lexicons (Bluesky, WhiteWind, Frontpage, Linkat, AT Functions).
Compiled extraction plan: for unknown collections, the resolved lexicon schema is compiled into a plan (packages/common/src/lexicon/plan.ts): which fields are title/body text, tags, dates, languages, URLs, geo. Classification is schema-structure-first (string formats like at-uri/did/datetime are never indexed as text; const/enum fields are skipped; knownValues become tags), field names second.
Heuristic fallback: field-name probing for collections with no published schema.

Triage (packages/indexer/src/lexiconRegistry.ts): before ingesting, the consumer asks the registry to decide(nsid). Collections whose schema has no searchable text (likes, follows, reposts) compile to a non-indexable plan and are dropped at a map lookup, which is what makes subscribing to the entire firehose (ATSEARCH_LEXICON_MODE=auto, the default) affordable. Decisions are cached in SQLite (lexicons table); failed resolutions are negative-cached with 1h → 6h → 24h → 7d backoff; successful ones refresh weekly (schemas evolve). Malformed collection names are rejected by syntactic NSID validation before any network work, and background resolutions run through a small semaphore (6 in-flight, bounded FIFO overflow) so a burst of novel NSIDs can't amplify into unbounded DNS/HTTP fan-out.

Both GET /lexicons endpoints (indexer and query node) expose what the registry has learned: each known NSID with its status (adapter / plan / no-text / unresolvable) and the schema's description.

Query syntax

Query	Behavior
`community fridge`	Free-text search across all indexed lexicons
`type:at.functions.metadata`	List every indexed record of that lexicon (alias: `collection:`)
`resize image type:at.functions.metadata`	Free-text search restricted to one lexicon

The type: term may appear anywhere in the query and is case-insensitive.

Backfill: rebuilding the index from repos

Jetstream and the firehose carry live commits only. A record written before an index existed is invisible to it forever, which makes a streamed index one-way: lose it and the history is gone with it.

Backfill reads repos directly (describeRepo → listRecords) so the index is reproducible from the network at any time. Records go through the same triage and extraction plans as live ingestion, so a backfilled record is indistinguishable from a streamed one.

# Specific repos
pnpm --filter @atsearch/indexer run backfill -- --dids did:plc:abc,did:plc:def

# Everything a relay knows about (paginated; start small)
pnpm --filter @atsearch/indexer run backfill -- --from-relay https://bsky.network --limit 500

# One lexicon across a list of repos
pnpm --filter @atsearch/indexer run backfill -- --dids-file dids.txt --collections at.functions.metadata

Flag	Purpose
`--dids` / `--dids-file` / `--from-relay`	where the DIDs come from (`--limit` bounds relay enumeration)
`--collections`	restrict to specific NSIDs; default is every collection the repo declares
`--concurrency`	repos crawled in parallel (default 4)
`--max-per-repo`	stop after N records per repo
`--checkpoint` / `--restart`	completed DIDs are checkpointed so re-runs resume; `--restart` ignores the checkpoint

Run it twice for unknown lexicons. The first record of a collection the registry has never seen schedules schema resolution and is dropped while that is in flight; a second pass indexes it with the compiled plan. The run reports skipped collections so you can see when this applies.

Pointer record structure

Indexers serve signed pointer records over HTTP at GET /pointers/:descriptorKey.

interface PointerRecord {
  version: 1
  descriptorKey: string        // e.g. "tag:food"
  ref: {
    uri: string                // at://did/collection/rkey
    cid: string                // immutable content hash
  }
  providerPeerId: string       // libp2p peer ID of this indexer
  providerDid?: string         // optional AT Proto DID of this indexer
  indexedAt: string            // ISO timestamp
  expiresAt: string            // ISO timestamp (TTL: 24h default)
  signature: string            // Ed25519 signature (base64), covers all other fields
}

The signature covers all fields except signature itself, serialized as canonically sorted JSON. This allows query nodes to verify that pointer records have not been tampered with in transit.

Ranking

Results are scored as follows:

Condition	Score change
All query tokens match record	+5
Per matching token	+1
Per matching tag	+2
Geohash match	+2 per level
CID verified against PDS	+1
Expired pointer	-3
Fetch failure	-2
CID mismatch (stale pointer)	-2

Results are returned sorted descending by score.

Setup

Prerequisites

Node.js 22+
pnpm 10+ (or corepack enable)

Install

pnpm install

Build

pnpm run build

Test

pnpm run test

Try it from the command line

The public instance is at https://atsearch.network/api. No key, no account, no SDK: it is a plain JSON API, so curl is enough.

Search across every indexed app at once. Free text matches any lexicon the index understands:

curl -s "https://atsearch.network/api/search?q=gnocchi+nutmeg" | jq '.results[0] | {type: .record."$type", title: .record.title, score}'

List everything in one lexicon. A bare type: filter returns that collection, whoever published it and wherever their PDS lives:

curl -s "https://atsearch.network/api/search?q=type:app.rocksky.scrobble" | jq -r '.results[] | .record.title'

Search inside one lexicon. Combine free text with the filter to narrow to a single app:

curl -s "https://atsearch.network/api/search?q=closer+type:app.rocksky.scrobble" | jq -r '.results[] | .record.title'

Ask why a result ranked where it did. Every result carries the components that add up to its score:

curl -s "https://atsearch.network/api/search?q=gnocchi+nutmeg" | jq -r '.results[0].scoreBreakdown[] | "\(.points)\t\(.label)"'

5	every search term matched
3	"gnocchi" in the title
1	"nutmeg" in the text
1	verified against the PDS

See what the index has taught itself. Every lexicon it has resolved, with whether it found searchable text in it:

curl -s "https://atsearch.network/api/lexicons" | jq -r '.lexicons[] | select(.status=="plan") | .nsid' | head -20

Each result is a strong reference (ref.uri + ref.cid) plus the normalised record, score, and scoreBreakdown. Nothing is served from the index alone: records are fetched and CID-verified against their live PDS before they are returned.

Running the demo

Quick start (local mode, no AT Proto credentials needed)

# 1. Install dependencies
pnpm install

# 2. Build all packages
pnpm run build

# 3. Seed the database with synthetic records
pnpm run seed

# 4. Start the full stack
pnpm run demo

scripts/run-demo.sh uses pnpm for installs, builds, seeding, the query node, and the demo client. The indexer process is started with node because better-sqlite3 is a native addon built for Node’s ABI. Set NODE_BIN if node is not on your PATH.

The script checks that indexer HTTP, indexer DHT, query HTTP, query DHT, and Vite ports are free before starting (so you do not accidentally hit an old query node and get 404 on /interactions). Override ports if needed, for example:

INDEXER_PORT=3011 QUERY_PORT=3012 DEMO_PORT=5180 pnpm run demo

This will start:

Indexer on http://localhost:3001 (or INDEXER_PORT)
Query node on http://localhost:3002 (or QUERY_PORT)
Demo client on http://localhost:5173 (or DEMO_PORT; the script prints the real URL)

Open the printed URL and try queries like fridge, vancouver, mutual-aid, or a lexicon filter like type:at.functions.metadata (see Query syntax above). For a Bluesky / federated profile not present in your local SQLite index, enter a handle (user.bsky.social) or DID (did:plc:…) alone on the search line; the query node resolves it via Slingshot / public XRPC and returns app.bsky.actor.profile/self as a hit.

Deploy publicly (single VM, Docker Compose)

This repo includes a production-ready docker-compose.yml that runs the full stack on one host:

indexer: HTTP :3001, libp2p DHT :8001
query-node: HTTP :3002, libp2p DHT :8002
web: Caddy on :80/:443 serving the demo and proxying /api/* → query-node

Prereqs

Install Docker + Compose plugin
Point a DNS name at the server (A/AAAA record)
Allow inbound 80/tcp + 443/tcp
Optional: allow inbound 8001/tcp + 8002/tcp (if you want DHT reachability from other machines)

Configure environment

On the server (repo root), create .env with at least:

ATSEARCH_PUBLIC_HOSTNAME=atsearch.yourdomain.com

Recommended:

USE_MICROCOSM=true
MICROCOSM_SLINGSHOT_BASE_URL=https://slingshot.microcosm.blue
MICROCOSM_CONSTELLATION_BASE_URL=https://constellation.microcosm.blue
FALLBACK_ATPROTO_XRPC_BASE_URL=https://public.api.bsky.app
APP_USER_AGENT='at-search-demo/0.1 (public; +https://atsearch.yourdomain.com)'

# Optional: stable pointer signatures across restarts
# ATSEARCH_NODE_KEY=ed25519_private_key_hex

Launch

docker compose up -d --build

Then:

UI: https://atsearch.yourdomain.com
API (proxied): https://atsearch.yourdomain.com/api/health

Notes

The demo UI defaults to calling window.location.origin + "/api"; Caddy reverse-proxies that to the query node.
SQLite persists in the atsearch_data Docker volume.
If you do not want DHT ports exposed publicly, remove the 8001:8001 / 8002:8002 mappings from docker-compose.yml.
If you run another app on the same host and want it to call the search API, bind the query node to localhost only and call it at http://127.0.0.1:<port>. (The provided docker-compose.yml binds it to 127.0.0.1:13002.)

Manual start

# Terminal 1: indexer
cd packages/indexer
ATSEARCH_LIBP2P_LISTEN_HOST=127.0.0.1 \
ATSEARCH_DB_PATH=../../data/indexer.db ATSEARCH_HTTP_PORT=3001 pnpm run dev

# Terminal 2: query node
cd packages/query-node
USE_MICROCOSM=true \
MICROCOSM_SLINGSHOT_BASE_URL=https://slingshot.microcosm.blue \
MICROCOSM_CONSTELLATION_BASE_URL=https://constellation.microcosm.blue \
FALLBACK_ATPROTO_XRPC_BASE_URL=https://public.api.bsky.app \
APP_USER_AGENT='at-search-demo/0.1 (manual)' \
ATSEARCH_LIBP2P_LISTEN_HOST=127.0.0.1 \
ATSEARCH_HTTP_PORT=3002 ATSEARCH_INDEXER_URLS=http://localhost:3001 pnpm run dev

# Terminal 3: demo client
cd packages/demo-client
# The dev server proxies /api → query-node (see vite.config.ts), so you don't need VITE_QUERY_API_URL.
# If you changed the query node port, set VITE_QUERY_PROXY_TARGET accordingly.
# Example: VITE_QUERY_PROXY_TARGET=http://127.0.0.1:3012 pnpm run dev -- --port 5180
pnpm run dev -- --port 5173

With AT Proto credentials (poll mode)

ATSEARCH_MODE=poll \
ATSEARCH_PDS_URL=https://bsky.social \
ATSEARCH_POLL_DIDS=did:plc:yourDid \
ATSEARCH_POLL_COLLECTIONS=com.example.thing \
ATSEARCH_DB_PATH=./data/indexer.db \
ATSEARCH_HTTP_PORT=3001 \
  pnpm --filter @atsearch/indexer run dev

Environment variables

Indexer

Variable	Default	Description
`ATSEARCH_MODE`	`local`	`local` (seed script only), `poll` (listRecords for known DIDs), `jetstream` (public Jetstream websocket), or `firehose` (raw `com.atproto.sync.subscribeRepos` via `@atproto/sync`). Live modes batch their writes and should set `ATSEARCH_RETENTION_DAYS`.
`ATSEARCH_PDS_URL`	`https://bsky.social`	PDS base URL
`ATSEARCH_HANDLE`	-	AT handle (for auth; poll-related tooling if used)
`ATSEARCH_PASSWORD`	-	App password
`ATSEARCH_DHT_BOOTSTRAP`	-	Comma-separated bootstrap multiaddrs
`ATSEARCH_DHT_PORT`	`8001`	libp2p TCP listen port
`ATSEARCH_LIBP2P_LISTEN_HOST`	`127.0.0.1`	IPv4 for libp2p TCP bind (`0.0.0.0` can fail on some hosts with `ERR_NO_VALID_ADDRESSES`)
`ATSEARCH_HTTP_PORT`	`3001`	HTTP server port
`ATSEARCH_DB_PATH`	`./data/indexer.db`	SQLite database path
`ATSEARCH_NODE_KEY`	-	Ed25519 private key hex (persisted signing key)
`ATSEARCH_LEXICON_MODE`	`auto`	`auto`: resolve lexicon schemas at runtime (`_lexicon` DNS → `com.atproto.lexicon.schema`), compile extraction plans, triage collections (text-free ones like likes/follows are dropped). `curated`: fixed collection lists, no resolution.
`ATSEARCH_LEXICON_ALLOWLIST`	-	Comma-separated NSIDs or `prefix.*`. When set, only these collections are ingested; schema-less ones fall back to heuristic extraction. Only applies when `ATSEARCH_LEXICON_MODE=auto`.
`ATSEARCH_LEXICON_DENYLIST`	-	Comma-separated NSIDs or `prefix.*`. Always dropped. Only applies when `ATSEARCH_LEXICON_MODE=auto`.
`ATSEARCH_RETENTION_DAYS`	`0` (compose: `7`)	Delete records indexed more than this many days ago, hourly. `0` disables. A full-firehose subscription grows without bound, so set this on any long-running deployment. Freed pages are reused by SQLite; the file does not shrink.
`ATSEARCH_INGEST_BATCH`	`500`	Records buffered before an index write transaction commits (live modes only).
`ATSEARCH_INGEST_FLUSH_MS`	`2000`	Max time a buffered record waits before commit. Larger batches mean less disk I/O and a larger loss window on crash.
`ATSEARCH_LOG_INGEST`	-	`1` logs every indexed record. Off by default: at firehose volume this alone can fill a disk. A per-minute count is logged instead.

Query node

Variable	Default	Description
`ATSEARCH_DHT_BOOTSTRAP`	-	Comma-separated bootstrap multiaddrs
`ATSEARCH_DHT_PORT`	`8002`	libp2p TCP listen port
`ATSEARCH_LIBP2P_LISTEN_HOST`	`127.0.0.1`	Same as indexer; libp2p bind address
`ATSEARCH_HTTP_PORT`	`3002`	HTTP server port
`ATSEARCH_INDEXER_URLS`	`http://localhost:3001`	Comma-separated indexer base URLs
`USE_MICROCOSM`	`true` when set; if unset, defaults to on when `MICROCOSM_SLINGSHOT_BASE_URL` is non-empty	Prefer Slingshot for record/identity reads
`MICROCOSM_SLINGSHOT_BASE_URL`	-	e.g. `https://slingshot.microcosm.blue`
`MICROCOSM_CONSTELLATION_BASE_URL`	-	e.g. `https://constellation.microcosm.blue` (for `GET /interactions` / backlinks)
`FALLBACK_ATPROTO_XRPC_BASE_URL`	`https://public.api.bsky.app`	App View / public XRPC when Slingshot does not serve a method
`APP_USER_AGENT`	`at-search-demo/0.1 (local dev)`	User-Agent on outbound Microcosm + XRPC requests

Why Microcosm?

Slingshot is an edge cache for public com.atproto.* reads (records, handles). The query node tries it first, then falls back to direct XRPC and finally the indexer’s SQLite cache (seed / poll data).

Constellation indexes backlinks from firehose-seen records. The demo uses it for likes and replies on Bluesky posts (GET /interactions?subjectUri=…), not for full-text search.

Demo client

Variable	Default	Description
`VITE_QUERY_API_URL`	(unset)	Optional override for query node base URL. If unset, the demo calls `window.location.origin + "/api"` (and in dev, Vite proxies `/api` to the local query node).

Seed data

The seed script (pnpm run seed) inserts 12 synthetic records covering:

Vancouver community fridge (2 records: original + updated version)
East Van Tool Library (overlapping tags and geo)
Seattle Capitol Hill Fridge (overlapping token fridge)
Toronto Kensington food pantry (overlapping tag food)
London Hackney Bike Repair (different geography)
Berlin Kreuzberg Free Shop (different geography)
Duplicate content record: identical to the Vancouver fridge but different URI (demonstrates CID-alone deduplication is wrong)
Updated Vancouver fridge: same URI as original but new CID (demonstrates versioning)
Stale Community Garden (old createdAt, exercises stale pointer path)
Auckland Grey Lynn Repair Cafe
Sydney Newtown Tool Library (overlapping tag tools)
Vancouver Seed Library (overlapping geo)

Limitations

Collections without published lexicon schemas are invisible in auto mode (unless allowlisted, in which case they fall back to heuristic extraction). This is a deliberate precision trade-off that also nudges apps toward publishing their lexicons.
DHT → HTTP peer resolution not implemented. In production, DHT findProviders returns multiaddrs that need to be resolved to HTTP endpoints. The prototype uses configured ATSEARCH_INDEXER_URLS directly (DHT discovery supplements but does not replace this in the current implementation).
No DHT record persistence. libp2p kad-dht provider records expire; real indexers would need periodic re-advertisement.
CID computation. AT Protocol CIDs are DAG-CBOR encoded. We cannot recompute them from JSON. Instead, we compare against the CID returned by the PDS via com.atproto.repo.getRecord. This is correct but means verification requires a live PDS.
No authentication for write operations. The indexer's pointer signing keys are ephemeral by default. In production, keys should be persisted and associated with a verified AT Proto DID.
SQLite is single-node. For production, a distributed store or at least replicated SQLite would be needed.
No rate limiting on the query node or indexer HTTP endpoints.

Future work

DHT multiaddr → HTTP endpoint resolution (removes need for out-of-band indexer URL configuration)
DID-anchored indexer identity (indexer signs with DID key)
Geohash range queries (expand geo prefix search)
Server-side search pagination (the demo UI paginates 10 results per page client-side)
Persistent DHT routing tables
HTTPS + TLS for indexer HTTP endpoints
Query node result caching
Prometheus metrics for query latency, DHT lookup time, verification success rate

License

The index itself is not a proprietary asset: every record it holds is public data published to AT Protocol repos, and backfill rebuilds the whole thing from those repos. Run your own indexer and you get the same index.

Name		Name	Last commit message	Last commit date
.pnpm-store/v10/projects		.pnpm-store/v10/projects
demo		demo
deploy		deploy
docs/superpowers		docs/superpowers
lexicons		lexicons
packages		packages
scripts		scripts
.env.example		.env.example
.gitignore		.gitignore
.npmrc		.npmrc
Dockerfile.demo-client		Dockerfile.demo-client
Dockerfile.indexer		Dockerfile.indexer
Dockerfile.query-node		Dockerfile.query-node
Dockerfile.relay		Dockerfile.relay
Dockerfile.web-render		Dockerfile.web-render
LICENSE		LICENSE
README.md		README.md
docker-compose.yml		docker-compose.yml
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
render.yaml		render.yaml
tsconfig.json		tsconfig.json

Latest commit

History

Folders and files

Repository files navigation

AT Search

Monorepo layout and tooling

Packages

Root scripts

Architecture

Identity model

Why CID alone is insufficient

Strong reference: uri + cid

Key invariants

DHT vs AT Protocol roles

Descriptor derivation

Cross-lexicon indexing (lexicon registry)

Query syntax

Backfill: rebuilding the index from repos

Pointer record structure

Ranking

Setup

Prerequisites

Install

Build

Test

Try it from the command line

Running the demo

Quick start (local mode, no AT Proto credentials needed)

Deploy publicly (single VM, Docker Compose)

Prereqs

Configure environment

Launch

Notes

Manual start

With AT Proto credentials (poll mode)

Environment variables

Indexer

Query node

Why Microcosm?

Demo client

Seed data

Limitations

Future work

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Strong reference: `uri + cid`