grounding-mcp

Go CLI + MCP server for grounding AI reasoning in verifiable sources. Runs locally over stdio, or as a remote Streamable-HTTP server with a public tier and a bearer-token full tier.

Architecture

grounding
├── cmd/                    # Cobra commands (CLI + MCP tool registration)
│   ├── root.go             # Global flags (--json), version stamp
│   ├── serve.go            # MCP server: stdio or http; tier filter; access log
│   ├── landing.go          # GET / on the http transport (tier-aware, txt.btr.mt styling)
│   ├── public-tier-call-limit.go  # Per-caller tool-call allowance for the public tier
│   ├── authorship-for-agents.go   # /llms.txt, JSON-LD and the "About the author" blurb
│   ├── health.go           # check_backends
│   ├── search.go           # fetch_context
│   ├── verify.go           # check_citation
│   ├── graph.go            # citation_graph
│   ├── download.go         # download_document (full tier)
│   ├── search-downloadable-books.go  # search_downloadable_books (full tier)
│   └── schema.go           # Dump tool definitions as JSON
│
├── internal/
│   ├── auth/tier.go        # TierFull / TierPublic from the bearer token
│   ├── config/config.go    # Environment variables
│   ├── backends/           # API clients
│   │   ├── types.go        # Interfaces: SearchBackend, CitationBackend, WikipediaBackend
│   │   ├── http.go         # Shared client: rate limiters, 429 retry/cooldown, User-Agent
│   │   ├── cache.go        # Server-only GET cache (10 min)
│   │   ├── download.go     # Capped, containment-checked file transfer
│   │   ├── text.go         # Text normalisation
│   │   ├── crossref.go, openalex.go, semantic_scholar.go, europepmc.go
│   │   ├── openlibrary.go, opencitations.go, unpaywall.go
│   │   ├── wikipedia.go, exa.go, consensus.go, anna.go
│   │
│   └── tools/              # Business logic (shared by CLI + MCP)
│       ├── tools.go        # Tool registry (ToolDef, Register, All, RequiresAuth)
│       ├── check_backends.go
│       ├── check_citation.go
│       ├── citation_graph.go
│       ├── fetch_context.go
│       └── relevance.go    # Post-retrieval scoring and filtering
│
└── main.go

Each command file in cmd/ defines the CLI command, registers the MCP tool in init(), and implements handlers that call the shared logic in internal/tools, so CLI and MCP behave identically.

Backends

Backend	Used for	Key	Tier
OpenAlex	search, verification, citation graph fallback, DOI batches	OPENALEX_API_KEY (free; 10× the daily budget)	public
Crossref	search, DOI verification	none (GROUNDING_EMAIL for the polite pool)	public
Europe PMC	search, verification; abstracts and OA full text	none	public
Semantic Scholar	search, citation graph (influence markers, intents)	SEMANTIC_SCHOLAR_API_KEY (optional)	public
OpenCitations	citation graph, third fallback	OPENCITATIONS_TOKEN (optional)	public
Wikipedia	general knowledge	none (GROUNDING_EMAIL in the User-Agent)	public
OpenLibrary	book verification	none	public
Unpaywall	legal open-access copy for a DOI (download)	GROUNDING_EMAIL	full
Anna's Archive	book search, document download; opt-in backends: ["anna"]	ANNAS_SECRET_KEY	full
Consensus	study-level search; opt-in backends: ["consensus"] (~1¢/query)	CONSENSUS_APP_KEY	full
Exa	web pages with text; opt-in source_type: "web" (metered)	EXA_API_KEY	full

Search results are deduplicated across backends, filtered by relevance (query-term overlap in title and abstract) and re-ranked by a composite of relevance, abstract presence and citation count.

The shared HTTP client keeps one rate limiter per backend for the life of the process, retries a 429 once and then benches the backend (honouring Retry-After, capped at an hour), retunes Crossref from its x-rate-limit-* headers, and sends the contact email in the User-Agent — which is how Crossref's polite pool and Wikimedia's identified-client tier are granted. The server also caches GET responses for ten minutes.

Tools

Tool	Tier
fetch_context	public
check_citation	public
citation_graph	public
check_backends	public
download_document	full
search_downloadable_books	full

Full-tier tools are absent from tools/list and the landing page for public callers. Full-tier sessions (and the CLI) also see longer descriptions on fetch_context and check_backends that name the opt-in backends and what each costs; public descriptions never mention them. fetch_context diagnostics carry cost_usd (Exa, per call) or cost_note (Consensus) so a caller can see what it spent. check_backends does not probe Consensus or Exa unless force is true, since their health checks are billed queries.

download_document tries Unpaywall for a legal open-access copy first, then Anna's Archive. When ANNAS_DOWNLOAD_PATH is unset it returns a download_url rather than writing a file — the shape a remote caller needs.

Access tiers

Condition	Tier
stdio transport	full
http, correct Authorization: Bearer <token> or ?token=<token>	full
http, no or wrong token	public
http, GROUNDING_AUTH_TOKEN unset	refuses to start

Public-tier tool calls are limited per client address (a burst of 40, then one every 90 seconds), because every public caller shares one OpenAlex budget and the contact email's standing upstream. A caller over the limit gets a tool error that tells the model to explain the wait. Full-tier calls are not limited. The client address is the rightmost X-Forwarded-For entry, which is the one Caddy writes.

Every tool carries MCP annotations: a human title and readOnlyHint (true for all but download_document), so clients can run calls in parallel without confirmation prompts. The landing page shows each tool's SummaryForPeople; the model-facing descriptions and parameters sit behind a collapsed "Under the hood" section.

Environment variables

Variable	Purpose
GROUNDING_EMAIL	Contact email: Crossref polite pool, Wikimedia UA policy, Unpaywall (required by it)
GROUNDING_AUTH_TOKEN	Bearer token for the full tier over http (required for http)
OPENALEX_API_KEY	Free key; without it OpenAlex allows ~100 list queries/day per IP
SEMANTIC_SCHOLAR_API_KEY	Optional; 1 req/s per key instead of the shared pool
OPENCITATIONS_TOKEN	Optional; leave unset rather than guess (a wrong token is a hard 403)
EXA_API_KEY	Enables the Exa web backend (metered)
CONSENSUS_APP_KEY	Enables Consensus (metered)
ANNAS_SECRET_KEY	Anna's Archive membership; registers the download tools
ANNAS_BASE_URL	Override the AA domain (they rotate)
ANNAS_DOMAIN_INDEX	AA download server index (0 is unreachable from some networks; try 1)
ANNAS_DOWNLOAD_PATH	Save downloads here; unset = return URLs only

The binary does not read .env; export the variables (or use an EnvironmentFile= in systemd).

Building

go build -o grounding .
# with a version stamp (shows in /health and the User-Agent):
go build -ldflags "-X btr.mt/grounding/cmd.version=$(git describe --tags --always)" -o grounding .

Testing

go test -short ./...   # offline, httptest only
go test ./...          # also runs live smoke tests against the real APIs

Running

./grounding serve                                  # stdio (full tier)
GROUNDING_AUTH_TOKEN=… ./grounding serve --transport http --addr localhost:9471

Register locally:

claude mcp add --scope user grounding -- /path/to/grounding serve

Use a remote server at full tier:

claude mcp add --transport http --header "Authorization: Bearer $GROUNDING_TOKEN" grounding https://grounding.btr.mt/mcp

claude.ai connectors cannot set headers; use https://grounding.btr.mt/mcp?token=… there.