A Smarter Way to Search Workflow Libraries
Users needed fast access to automation workflows for both conceptual discovery and deterministic ranking. Pure semantic search was unreliable for sorting and counting. We designed a hybrid retrieval architecture that routes different intents to different systems.
Business Impact
Time reclaimed from manual searching across scattered tools
Business Impact
Executive Outcomes
Teams get answers in seconds instead of hunting across tools
Every response grounded in verified source data
Replaced 5+ separate search and lookup tools
Time savings across the team repay the build cost within months
The Challenge
“Discovering and evaluating automation workflows required navigating multiple data sources. Semantic search worked well for conceptual queries like "workflows about email automation" but failed on deterministic requests like "show the 10 newest workflows" or "count workflows in this category."”
Semantic search returned irrelevant results for deterministic queries like 'show the 10 newest workflows'
Counting and ranking queries produced inconsistent answers because vector similarity cannot count or sort
Technical deep-dive requests returned hallucinated download links and incomplete code context
No way to validate whether the AI was actually calling the right tool for each query type
Users had to manually search across multiple data sources to find and evaluate workflow templates
The Transformation
What changed after we built the system
Semantic search returned irrelevant results for deterministic queries like 'show the 10 newest workflows'
Deterministic queries route to Google Sheets for exact sorting, ranking, and counting operations
Counting and ranking queries produced inconsistent answers because vector similarity cannot count or sort
Conceptual discovery queries use Qdrant vector search across summary and technical collections
Technical deep-dive requests returned hallucinated download links and incomplete code context
Technical requests fetch raw workflow JSON from Google Drive with round-tripped source URLs that prevent hallucination
No way to validate whether the AI was actually calling the right tool for each query type
100% benchmark score validates tool-calling accuracy across multiple model providers and configurations
Users had to manually search across multiple data sources to find and evaluate workflow templates
A single chat interface searches all three retrieval systems with automatic intent-based routing
Why hybrid retrieval outperforms pure semantic search
Vector search is excellent at understanding intent. Ask 'workflows about automating customer emails' and it returns relevant results even if the exact words do not appear in the workflow titles.
But ask 'show me the 10 newest workflows sorted by date' and semantic search falls apart. There is no embedding that captures recency or count. The vector space has no concept of 'newest' or 'ten.'
Routing these queries to Google Sheets gives exact answers. A spreadsheet can sort by date and return exactly 10 rows. The AI decides which tool to use based on query intent, and the model benchmark proves it makes the right choice 100% of the time across every tested provider.
How We Built It
Technical architecture for the curious
Frontend
Streaming chat with rich markdown rendering, code-block copy support, and clear loading states.
Semantic Retrieval
Conceptual queries hit summary and technical vector collections, merging results for relevance.
Structured Retrieval
Deterministic queries route to spreadsheet backend. No vector search ambiguity for exact operations.
Technical Retrieval
Deep-dive requests fetch actual workflow JSON. Source URLs round-trip through tool outputs to prevent hallucination.
Engineering Decisions
Tradeoffs we made and why
Three specialized retrieval tools instead of one general-purpose search
Benefit
Each query type gets the retrieval system that handles it best, with no compromises
Cost
System prompt must be carefully tuned to route queries to the correct tool every time
Strict multi-step behavior caps on tool calling
Benefit
Prevents runaway tool-calling loops that burn tokens without producing useful results
Cost
Complex queries requiring 4+ steps get truncated before reaching a complete answer
Source URL round-tripping through tool outputs
Benefit
AI can only cite URLs that actually exist in tool responses, preventing hallucinated links entirely
Cost
Tool output payloads are larger, increasing token usage per response
Qdrant over pgvector for embedding storage
Benefit
Dedicated vector database with built-in collection management, filtering, and efficient similarity search
Cost
Additional infrastructure to deploy and manage compared to an embedded PostgreSQL solution
Certain client names, proprietary workflows, screenshots, and internal assets referenced in this case study are protected under a non-disclosure agreement and have been anonymized or omitted to comply with our confidentiality obligations.
Need an AI assistant that actually gets it right?
Book a free 30-minute call. We will assess your search and retrieval needs, identify where AI keeps hallucinating, and design a system that gives accurate answers every time.
30 minutes with Apurva. Not a sales call.
Book Your Free Audit