3.4 RAG Engine, Agent Search & Context Caching
1. Key Concept Definitions: The Grounding & Context Caching Stack
Section titled “1. Key Concept Definitions: The Grounding & Context Caching Stack”On the Gemini Enterprise Agent Platform, Google Cloud provides four complementary mechanisms to ground agent reasoning in enterprise and live knowledge:
1. Agent Search (Turnkey + ACLs)
The managed enterprise search engine with turnkey connectors (SharePoint, Jira, Drive, Confluence, GCS), automatic parsing, and built-in Document-Level Access Control Lists (ACLs) enforced at query time.
2. RAG Engine (Custom Pipeline)
The developer-controlled retrieval pipeline offering granular programmatic control over
chunk_size, chunk_overlap, custom embedding models, vector_distance_threshold, and semantic reranking.3. Gemini Context Caching
Caches a large, static prompt prefix (32k+ tokens of PDFs, videos, or codebases) in Gemini memory via
CachedContent (with a configurable ttl) or ADK ContextCacheConfig—slashing input token costs and TTFT latency.4. Web Grounding & Citations
Connects agents to live public facts via Google Search grounding (or Enterprise Web Grounding) and returns
groundingMetadata with verifiable source URIs and inline citation mappings.2. Interactive Decision Boundary: Agent Search vs. RAG Engine vs. Context Caching
Section titled “2. Interactive Decision Boundary: Agent Search vs. RAG Engine vs. Context Caching”Switch between the 3 Grounding & Caching Modes below and step through the 4 stages to compare Agent Search, RAG Engine, and Gemini Context Caching:
3. Master Decision Matrix: Agent Search vs. RAG Engine vs. Context Caching
Section titled “3. Master Decision Matrix: Agent Search vs. RAG Engine vs. Context Caching”Use this side-by-side parameter and capability matrix to select the right knowledge-grounding architecture:
| Architectural Dimension | Agent Search | RAG Engine | Gemini Context Caching |
|---|---|---|---|
| Core Purpose | Turnkey enterprise search & grounding across SaaS and document repositories. | Developer-controlled RAG pipeline with custom chunking, embeddings, and reranking. | Caching a large, static prompt prefix (PDFs, videos, codebases) in Gemini memory. |
| Key Configuration Parameters | data_store_id, Enterprise Connectors, Document ACL enforcement. |
rag_resources, chunk_size, chunk_overlap, similarity_top_k, vector_distance_threshold, reranker. |
CachedContent (contents, system_instruction, ttl) or ADK ContextCacheConfig. |
| Data Size & Change Frequency | Thousands to millions of documents; updates continuously via connectors. | Thousands to millions of custom-chunked documents; programmatic corpus updates. | Single large static payload (32k+ tokens up to 2M tokens) reused across many requests. |
| Document-Level Permissions (ACLs) | Built-in out of the box (filters search results by logged-in user identity). | Requires custom metadata filtering implementation. | Shared across all callers referencing the same CachedContent resource. |
| Primary Architectural Trigger | “Fastest setup,” “SharePoint/Drive/Jira connectors,” “enforce user document ACLs.” | “Tune chunk size/overlap,” “custom embedding model,” “custom semantic reranking.” | “Same 500-page manual or video queried 1,000 times/hour,” “reduce TTFT and input token cost.” |
4. Real-World Deep Dive: Retrieval Troubleshooting & Caching Mechanics
Section titled “4. Real-World Deep Dive: Retrieval Troubleshooting & Caching Mechanics”| Architect Question | How It Works Under the Hood in Production |
|---|---|
| “What if our RAG agent retrieves passages that cut off legal exceptions or financial tables mid-sentence?” | Tune chunk_size, chunk_overlap & Layout Parsing: Fixed-size chunks that are too small (or have zero overlap) split semantic units apart. Increase chunk_size and chunk_overlap inside ChunkingConfig (and enable layout-aware document parsing so tables and multi-paragraph rules remain intact)! |
“Why does pure vector search (similarity_top_k=20) sometimes return boilerplate paragraphs that confuse the LLM—and how do we fix it?” |
Add a Two-Stage Semantic Reranker: Dense vector embeddings measure broad semantic similarity, not exact question-answering relevance. Adding a Semantic Reranker (Ranking API) re-scores the top-20 candidate chunks against the user’s exact query and passes only the top-3 highest-precision passages to Gemini (paired with a strict vector_distance_threshold)! |
“When should we choose Gemini Context Caching (CachedContent) instead of chunking a document into RAG Engine?” |
Full-Document Synthesis vs. needle-in-a-haystack lookup: If an agent needs to cross-reference every section of the exact same 400-page static rulebook (or a 1-hour video / large codebase) across hundreds of turns, chunked RAG will miss global cross-references. Uploading the entire 400-page document into Gemini Context Caching keeps all tokens in model memory with a configurable ttl, cutting input token costs and latency! |