Exploring AI / RAG-Powered Search and Documentation Assistant for Kubernetes.io

[debjotyms]

Exploring AI / RAG-Powered Search and Documentation Assistant for Kubernetes.io

What is this about?

Many modern documentation sites now use AI-powered search and chat assistants (based on RAG, or Retrieval-Augmented Generation) to help developers find answers quickly. These tools index the official docs and let users ask questions in plain language instead of searching by keyword.

Right now, kubernetes.io uses Pagefind for keyword search. It is fast and works well, but it has limits. For example, it can be hard to find deeply nested concepts, compare API fields across versions, or understand how multiple resources work together. These are common pain points for both beginners and experienced users.

A previous proposal to add llms.txt for LLM-friendly indexing (#51544) was closed without active maintainer sponsorship. This discussion aims to explore the broader question: should kubernetes.io offer some form of AI-powered documentation help? And if so, what are the options, risks, and constraints?

---

What other CNCF projects are already doing

The CNCF has an official partnership with Kapa.ai to give all hosted projects a RAG-based documentation assistant at zero cost. Projects can request it through the CNCF Service Desk.

Several graduated and incubating CNCF projects have already adopted it:

CNCF Project	Status	AI Doc Assistant
Prometheus	Graduated	Kapa.ai ("Ask AI" on prometheus.io)
Envoy Gateway	Graduated (subproject)	Kapa.ai on gateway.envoyproxy.io
OpenTelemetry	Incubating	Kapa.ai (thousands of queries per month)
OpenFGA	Incubating	Kapa.ai embedded in docs
Vitess	Graduated	Building a RAG FAQ bot through LFX/GSoC mentorship
Jaeger	Graduated	Building an AI assistant (Phase 2 in 2026)
Kubernetes	Graduated	Not adopted yet

CNCF itself also uses an AI assistant at ask.cncf.io for governance and contributor questions.

What other open-source projects are doing

Project	Tool	Open Source?	Notes
Docker	"Ask Gordon" / Kapa.ai	Commercial	Evolved from chatbot into MCP-powered agent
Supabase	Custom (pgvector + OpenAI)	Yes	Fully open-source (blog post)
Astro	Kapa.ai + "Houston"	Hybrid	Also provides an official MCP server for IDE agents
MongoDB	Atlas Vector Search	Partially (mongodb/chatbot)	Full-stack TypeScript RAG framework
Cloudflare	Workers AI + Vectorize	Proprietary infra	Uses open models like Llama 3 and Mistral on the edge
Stripe	Custom + MCP Server	Proprietary	MCP server for IDE agents + web AI search
HashiCorp / Terraform	Official MCP Server	Open source	MCP-first approach, no web chatbot widget
LangChain	"Chat LangChain"	Open source	Showcases their own RAG framework

An important trend: Many projects are now moving away from web chatbot widgets and toward MCP (Model Context Protocol) servers and llms.txt standards instead. This lets developers query docs directly from their IDE (Cursor, Claude Code, etc.) without needing a browser widget at all.

Community-built Kubernetes RAG tools like K8sGPT (CNCF Sandbox), k8s-docs-assistant-rag, k8sAI, and Kubernetes-Documentation-Bot show that there is real demand for grounded Kubernetes AI assistants.

---

Challenges and constraints

1. CKA / CKAD / CKS exam integrity

This is probably the biggest concern.

Linux Foundation certification exams (CKA, CKAD, CKS) allow candidates to open kubernetes.io/docs/ inside the PSI Secure Browser during the exam. The exam handbooks explicitly ban AI tools with zero-tolerance enforcement:

"Giving, receiving, or obtaining assistance by improper means (e.g., using AI or ChatGPT) during the examination constitutes Exam Misconduct."

Violations can lead to exam termination, certification revocation, and even a lifetime ban.

If an AI chatbot is embedded directly on kubernetes.io/docs/, candidates could use it to generate complete YAML manifests or troubleshooting answers. This would be a serious policy conflict.

Possible solutions:

• Put the AI assistant on a separate subdomain (for example, ask.kubernetes.io or ai.kubernetes.io). The PSI Secure Browser uses URL-based allowlisting, so it can allow kubernetes.io/docs/ while blocking ask.kubernetes.io at the same time.
• Only provide AI through MCP servers or IDE integrations. These are already blocked in the exam environment (which uses a clean VSCodium setup), so there would be no conflict with the web docs at all.

2. Cost (the Bing to Pagefind precedent)

Kubernetes previously used Bing Custom Search, which cost CNCF thousands of dollars per month in API fees. SIG Docs migrated to Pagefind specifically to remove that recurring cost. Any new AI solution will face the same scrutiny.

Good news: the CNCF Kapa.ai partnership is free for all projects. This directly addresses the cost concern. An MCP server approach also avoids per-query API costs because inference happens on the developer's machine. Self-hosted open-source options (like vLLM or Ollama running Llama, Mistral, or Gemma) could use CNCF community infrastructure grants.

3. CNCF vendor neutrality and third-party content policy

Kubernetes documentation has a Third-Party Content Policy that requires vendor-neutral references. Options that work within this policy:

• Use the official CNCF Kapa.ai partnership (this is a foundation-level partnership, not a vendor endorsement by Kubernetes itself)
• Use open-weight models through standard protocols (MCP, OpenAI-compatible APIs via vLLM)
• Build a pluggable backend with no single-vendor lock-in

4. Kubernetes version drift and deprecations

Kubernetes APIs change frequently across releases. Old APIs get removed (for example, PodSecurityPolicy was removed in v1.25). The AI system must know which version the user is looking at so it does not generate outdated or broken YAML. Pagefind already handles versioned content. An AI layer must respect this too.

5. Multi-language (i18n) support

Kubernetes docs are translated into 15+ languages. The assistant should work across these languages and return content in the user's selected locale. The ongoing AI-assisted localization work (#54075) could complement this.

6. Trust and "AI slop" concerns

SIG Docs already has strict rules about AI-generated contributions. Any AI assistant must clearly show source citations, link back to the official docs, and include appropriate disclaimers to maintain trust in the documentation.

---

A possible step-by-step plan

Instead of trying to do everything at once, here is a phased approach:

Step 1: Machine readability (low effort)
Revive the llms.txt proposal (the closed #51544). Adding /llms.txt and /llms-full.txt to kubernetes.io lets external AI tools, IDE agents, and coding assistants read the docs more accurately. This is a small change with big impact and no policy risk.

Step 2: Official MCP server (medium effort)
Provide an official Kubernetes Docs MCP server, following what Docker, Terraform, Astro, and MongoDB have already done. This lets developers query versioned K8s docs directly from their IDE. It avoids any web widget and has no conflict with exam policies.

Step 3: Evaluate the CNCF Kapa.ai partnership (low cost)
Since the CNCF partnership covers all costs, evaluate onboarding Kubernetes through the CNCF Service Desk. Prometheus, Envoy Gateway, and OpenTelemetry have already done this successfully. A pilot could start in the #sig-docs Slack channel before any website changes.

Step 4: Dedicated experimental assistant (higher effort)
If Step 3 shows strong demand, build a standalone assistant on a separate subdomain (ask.kubernetes.io) that is explicitly excluded from exam allowlists.

---

Questions for SIG Docs and the community

1. What is SIG Docs' position on interactive AI search vs. focusing only on structured standards like llms.txt?
2. Prometheus, Envoy Gateway, and OpenTelemetry have already adopted the CNCF Kapa.ai partnership. Has this been evaluated for Kubernetes? If so, what were the concerns?
3. What approach would work best for CKA/CKAD/CKS exam compatibility? Subdomain separation? MCP-only (no web widget)? Something else?
4. Would SIG Docs consider an LFX mentorship project to prototype a version-aware MCP server or RAG indexer for Kubernetes docs? (Vitess and Jaeger have both done similar mentorships.)
5. Should the closed llms.txt proposal (#51544) be revisited as a low-risk first step?

I would love to hear thoughts from maintainers and the community. Thank you for reading!

[divya-mohan0209]

Hi @debjotyms, thanks for the suggestion.

👋 SIG Docs co-chair here.

• First, I'd like to point you to our Generative AI policy in Kubernetes, which applies to all contributions, including any issues/discussion items you submit. As confirmed in your Slack message, you've used AI to draft & verify data points. Please make this explicit by editing the description.
• Regarding the proposal itself, please note that we have already explored and evaluated the CNCF partnership with Kapa.ai. While the suggestions for an alternate domain/Slack-based chatbot are excellent, the challenge isn't just deployment. I'm sure through LFX mentorship cycles, we will be able to deploy or develop any number of tools. However, post deployment, we also require people to take ownership of the specific area & continue developing a community around it. Second, for the Slack-based bot option, the security allowances required by the Kapa bot were found to extend beyond what we were comfortable sharing as a collective.
• Regarding our stance on interactive AI search vs structured standards, IIUC, interactive AI search will involve the deployment of another tool/set of dependencies. This, in turn, will require more maintainers or overworking our existing pool. llms.txt, on the other hand, might not be great given the massive scale, frequent updates, and diverse user base of the docs. IMO, a single file would be too large for an LLM's context window. Additionally, since we release a new version every 4 months and are required, per policy, to keep the last three versions updated, it may also struggle with version branching.

Hope this helps. Tagging @reylejano @natalisucks @katcosgrove @dipesh-rawat @salaxander here for their opinions.

[natalisucks]

Yes, to echo @divya-mohan0209 here, we have already had extensive chats with the Kapa.ai team, and also with SIG Contributor Experience (as they would own any kind of integration of a Slack bot) and the consensus is that we are unable to make use of the chat bot option, all while the current SIG Docs reviewers and approvers do not wish to maintain a separate docs instance for the purposes of AI search that wouldn't clash with the needs of LFX examinations team. Link here specifically to the Slack bot thread we had with SIG ContribEx and SIG K8s Infra members: https://kubernetes.slack.com/archives/C1TU9EB9S/p1776163892561899

[reylejano]

As stated, SIG Docs co-chairs had several chats and also met in-person with the Kapa.ai team.
@divya-mohan0209 pointed out the Kubernetes projects' Generative AI policy
I agree with all of @natalisucks' and @divya-mohan0209 's statements

In addition to "current SIG Docs reviewers and approvers do not wish to maintain a separate docs instance for the purposes of AI search that wouldn't clash with the needs of LFX examinations team", current SIG Docs reviewers and approvers do not wish to be "support" or "point-of-contact" for LFX exam takers during exams if the solution is to build the docs site locally which has been discussed as an alternative.

[Antonkapa]

Hi folks,

Jumping in from the kapa.ai team, thanks for flagging this @debjotyms !

As the SIG docs co-chairs mentioned it was deemed not feasible to deploy kapa in Slack or on the documentation. On your point regarding the MCP server:

Step 2: Official MCP server (medium effort)
Provide an official Kubernetes Docs MCP server, following what Docker, Terraform, Astro, and MongoDB have already done. This lets developers query versioned K8s docs directly from their IDE. It avoids any web widget and has no conflict with exam policies.

This is something that is included in the CNCF partnership with kapa, and utilized by OpenTelemetry, OpenFGA, EnvoyGateway etc. Leaving the documentation here for the SIGs: https://docs.kapa.ai/retrieval/hosted-mcp-server if interesting to explore this. The TLDR is essentially: Let coding agents query the Kubernetes knowledge-base, to ensure the information is always up to date.

[lmktfy]

I feel like coding agents will learn within 5 years how to clone from Git and read the docs from something like a local preview. If they do learn that then no special effort ought to be needed from SIG Docs.