Repository navigation
feat(mcp): add optional trust verification middleware for MCP servers - #23758
Open
eddyflores100-lang wants to merge 1 commit into
Open
eddyflores100-lang wants to merge 1 commit into
eddyflores100-lang wants to merge 1 commit into
Conversation
Adds a lightweight, pluggable trust checker that LlamaIndex agents can
use to verify an MCP server's reliability before connecting.
The module provides:
- TrustVerdict: dataclass with score (0-1), is_trusted, reason
- TrustChecker: configurable gate with min_score, fail-open/fail-closed
modes, and a pluggable lookup function
- filter_trusted(): convenience method to filter a list of server URLs
The default implementation queries a public trust API (Dominion
Observatory, free tier, 50 queries/day, no key needed), but users can
pass their own lookup callable for local or air-gapped verification.
Usage:
from llama_index.tools.mcp.trust import TrustChecker
checker = TrustChecker(min_score=0.5)
verdict = await checker.check("https://example.com/mcp/sse")
if not verdict.is_trusted:
raise ValueError(f"Server not trusted: {verdict.reason}")
Includes 6 tests covering: trusted pass, untrusted fail, unknown
fail-closed, unknown fail-open, filter_trusted, and score normalization.
Closes run-llama#21768.
Author
|
Adding visibility — the trust middleware is opt-in (gated behind |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Add an optional, pluggable trust verification module (
trust.py) to the MCP tools integration, so LlamaIndex agents can check an MCP server's reliability before connecting.Why
Issue #21768 asks for a pre-call trust gate. Currently, LlamaIndex agents using MCP tool specs have no way to verify server reliability before calling — a compromised or unreliable server could return bad data or fail silently.
The discussion on the issue explored several approaches (Dominion Observatory API, local Python scripts, MarketNow). This PR provides a neutral, pluggable implementation that doesn't lock into any specific trust provider:
lookupcallable for local, cached, or air-gapped verificationallow_unknown=True) or fail-closed (allow_unknown=False) for servers with unknown scoresHow
Added two files:
llama_index/tools/mcp/trust.py— the module:TrustVerdict: dataclass withscore,is_trusted,reason,total_callsTrustChecker: configurable gate withmin_score,allow_unknown, and pluggablelookupfilter_trusted(urls): convenience method to filter a list of server URLs, keeping only trusted ones_default_trust_lookup: queries the public Dominion Observatory API (50 queries/day, no key)tests/test_trust.py— 6 tests:allow_unknown=Truefilter_trustedkeeps only good serversUsage
Design choices
lookupparameter lets users substitute any trust provider (local, cached, air-gapped). The default uses a public API but nothing in the module depends on it.httpxwhich is already a dependency of the MCP client.Closes #21768.