Hybrid Search at Scale: Powering AI Context

• • The model isn’t the bottleneck — the data layer is. Teams spend roughly 80% of their time getting data into a usable state, leaving only a fraction for building the AI applications that drive business value.
  • Most enterprises reason over 30% or less of their data. The rest is fragmented across object stores, file systems, SaaS apps and file shares — effectively invisible to AI.
  • Accurate retrieval requires hybrid search, not just vector search. Semantic search captures meaning, keyword search captures exact terms like error codes and SKUs and most real-world queries need both, weighted through hybrid ranking.
  • GPU acceleration transforms ingestion from days to hours — or minutes. NVIDIA cuVS integration delivers up to 12x faster vector indexing, keeping data fresh and freeing CPUs to serve faster lookups.
  • DIY stacks work — until they don’t. The challenge isn’t picking components; it’s owning the integration burden between storage, compute, networking and the vector database forever.
  • Governance and control remain non-negotiable. On-prem infrastructure with field-level security, role-based access control and full audit trails keeps sensitive and regulated data inside your perimeter.
  • Design for agents, not humans. A single agentic AI request can fan out into dozens of sub-queries, demanding search infrastructure built for an entirely different scale and speed.

Every enterprise AI conversation I have starts in roughly the same place. Budgets are up. Timelines are compressed. Leadership wants agents in production, not another proof of concept that impresses in a demo and quietly dies before it ever touches a real workload. And yet, when teams try to move from experimentation to production, they hit a wall that has almost nothing to do with the model they picked.

Here’s the uncomfortable truth I’ve watched play out again and again: the model is rarely the bottleneck. What breaks is everything underneath it.

The real problem lives beneath the model

Enterprise data is fragmented by nature. It sits in object stores, file systems, databases, SaaS applications, file shares and laptops scattered across the business. None of those systems were designed to be queried by an AI model. You have PDFs, logs, emails, contracts and spreadsheets — and none of them are in the shape a model can reason over.

To make that data usable, you have to build a pipeline: ingest the data, chunk it, embed it, index it and keep it fresh as the underlying sources change. Every one of those steps is an engineering project in its own right. String them together and you get something that works exactly once — and then turns fragile the moment anything changes. And something always changes.

The result is a statistic that should bother every AI leader: teams routinely spend around 80% of their time just getting data into a usable state, and only the remaining sliver building the agentic application they set out to build. Meanwhile, most organizations are reasoning over 30% or less of their data. The rest is effectively invisible to AI. That’s the gap between where your data lives and where your AI needs it to be — and closing it is the whole game.

Retrieval is where AI succeeds or fails

If you strip an agent down to its essentials, it needs three things from the data layer: fast ingestion, accurate retrieval and freshness. Remove the retrieval layer entirely and an agent will either hallucinate or tell you its knowledge stopped at some training cutoff. Neither is acceptable in production.

But “accurate retrieval” is more subtle than it sounds. It’s a common misconception that you can dump everything into a vector store and let the model figure it out. In practice, that gives you hallucinations, irrelevant context and answers that are confidently wrong. What agents need is precision, not just recall.

That precision requires more than one kind of search. Semantic (vector) search captures meaning — it understands that “a golden retriever” and “a dog” can be the same thing. Keyword search captures exact terms and identifiers: error codes, SKUs, product names, the things a supply-chain query lives or dies on. Most real questions need both, weighted correctly through hybrid ranking. And because enterprise data never stops changing, the stack has to continuously fold in updates so the agent always reasons over the most current sources.

Why do-it-yourself doesn’t scale

Plenty of teams try to assemble this themselves — stitching together storage, compute, networking and a vector database and hoping it holds. I want to be clear: the DIY path isn’t wrong. It just doesn’t scale past a certain point.

The hard part was never picking the components. It’s owning the integration burden between them — forever. Storage, compute, networking and the vector database have to be sized, tuned and validated together. If your GPU is sitting idle waiting on data because storage can’t keep up, you’ve wasted your most expensive investment. If the network is the bottleneck, indexing throughput collapses. These aren’t hypotheticals; they surface in the simplest proofs of concept, fast.

The Dell AI Data Platform exists to shift that burden off your team and onto the engineering teams at Dell, Elastic and NVIDIA. It’s a validated, pre-integrated stack where storage, compute, networking and the vector database are designed to work together out of the box. Crucially, it doesn’t take away choice — you still bring your own models, your own data sources and build your own pipelines. What we remove is the need to reinvent the plumbing every single time.

A conversation worth having

I recently sat down with Ramya, VP of Product Marketing for Search and AI at Elastic, and Manas, a Technical Product Manager at NVIDIA, along with Omdia analyst Stephen Catanzano, to talk through exactly this problem. What struck me in that discussion is how much the three of us — coming from very different vantage points — converged on the same conclusion: search is at the center of AI success, and infrastructure is what quietly determines whether you get there.

That conversation shaped how I describe the two engines at the heart of this platform.

The data search engine, powered by Elastic

We embed Elastic as the data search engine in the platform, and the reason comes straight from the fragmentation problem. Enterprise data doesn’t arrive in one shape. Most approaches force you to choose between semantic and keyword search, or to run two parallel systems and merge the results yourself. Elastic unifies BM25 keyword search, vector search and hybrid search into a single engine with a single index.

That unification matters more than it might sound. You don’t move your data. You don’t maintain and pay for dual infrastructure. There are no seams in your relevance pipeline. At petabyte scale, that’s the difference between AI that surfaces the right answer and AI that surfaces something merely similar.

The acceleration engine: NVIDIA cuVS

The second engine is GPU acceleration. NVIDIA integrated its cuVS (CUDA Vector Search) library into Elasticsearch to accelerate vector index builds on the GPU — delivering up to 12x faster indexing in recent releases.

What that unlocks is practical. Getting terabytes to petabytes of PDFs and spreadsheets into a searchable state used to take hours, days or weeks. GPU indexing compresses that to hours, sometimes minutes. It keeps data fresh by ingesting daily updates almost as fast as they arrive. And counterintuitively, it improves search latency — because offloading ingestion to the GPU frees the CPU to serve lookups faster. Faster ingestion, faster retrieval, more accurate results, all at once.

Control stays with you

For the IT leaders I talk to, the last question is always governance. On-prem infrastructure remains the right answer for sensitive data, regulated industries and sovereign data requirements. With this platform, you run serious AI workloads on your own infrastructure, inside your own perimeter, with your own security controls — no routing data through a third-party cloud to get GPU acceleration or vector search. Because the stack is fully known and under your control, it’s far easier to audit and clear with security and compliance teams than a DIY assembly of open source. Elastic adds field-level security, role-based access control and complete audit trails, so you apply the same governance to AI apps that you already apply everywhere else.

Where this is heading

Search is becoming adaptive, multimodal, multilingual and real-time. And agentic AI changes the math entirely: a single agent request can fan out into 10, 20, or 50 sub-queries. You’re no longer designing for search done by humans — you’re designing for search done by agents.

So, my message to leaders is simple: don’t let infrastructure be the reason your AI strategy stalls. The models are ready, the use cases are real and the ROI is proven. Start with a validated foundation, get your data flowing and spend your team’s energy on the business problem — not the plumbing.

To learn more, visit the Dell AI Data Platform to explore how unified, GPU-accelerated data search turns your unstructured data into AI context.