Building an AI Feature? Start With the Right Architecture.

A team I talked to last quarter spent two months fine-tuning a model to answer questions about their own product docs.

The docs changed weekly.

Every change meant the fine-tune was stale, and they were back to square one. A retrieval layer would have solved the whole thing in about a week, and it would have stayed fresh on its own.

This is the most common AI mistake we see right now. Not picking the wrong model. Picking the wrong architecture before anyone wrote a line of evaluation code.

The Take: Start With RAG, Fine-Tune Only When You've Proven You Need It

If your AI feature answers questions over data that changes (product docs, support tickets, customer records, a knowledge base), use Retrieval-Augmented Generation. RAG leaves the model alone and feeds it the right context at the moment of the question. Use RAG when your data changes frequently, you need citations, or you have diverse query types. Use fine-tuning for consistent style or format, or domain-specific reasoning. Most production systems use RAG first.

Fine-tuning has its place. It's just not the place most founders think it is. It's slower to update, harder to audit, and it costs more to keep current. For most enterprise knowledge tasks, a well-designed RAG pipeline with solid chunking, good embeddings, and a hybrid retrieval layer will outperform a fine-tuned model, especially as information evolves.

So here are the five rules we'd give any technical founder building an AI feature in 2026.

Rule 1: Default to RAG, Because Freshness is the Whole Game

The reason teams reach for fine-tuning is that it feels more serious. More "real AI." The real failure mode is teams defaulting to fine-tuning because it feels more AI-native.

But the honest question isn't "which is more impressive." It's "how often does my data change, and who owns it?"

If the answer is "weekly" or "it's in a database someone updates," retrieval wins. You re-index when the data changes and the answers update instantly. No retraining run. No drift.

This isn't a fringe opinion. According to the Menlo Ventures 2024 State of Generative AI in the Enterprise report, 51 percent of enterprise AI deployments use RAG in production. The market has already voted.

Rule 2: Hybrid Retrieval, Not Vector-Only

Here's where a lot of first builds quietly fail.

A founder wires up vector search, it demos beautifully on five test queries, and then real users ask things the embeddings just don't catch. Product codes. Exact names. Acronyms. Vector similarity is bad at exact-match recall, and that's exactly what users type.

The fix is hybrid retrieval: combine semantic vector search with old-fashioned keyword search (BM25). Hybrid search combines vector similarity with keyword search (BM25) in a single query, improving recall for most RAG workloads.

This is one of those changes that costs a day of engineering and saves you a month of "why didn't it find that obvious result" bug reports. If you're shipping AI features as part of a product, this is the line we'd draw in the sand.

Rule 3: Reranking is The Cheapest Quality Win You'll Find

If you only do one thing to improve answer quality after launch, add a reranker.

Most teams obsess over picking the perfect embedding model. That's the wrong lever. Most RAG quality wins in 2025 to 2026 came from better reranking, not better embedding. A cross-encoder reranker often improves quality by 15 to 35 percent with minimal engineering.

A reranker is a second pass. Your retrieval grabs the top 20 candidates, the reranker reads each one against the actual question and reorders them so the best context lands in front of the model. A reranker is non-optional.

Fifteen to thirty-five percent better answers for a few hours of work. That's the best return in the whole pipeline.

Rule 4: You Probably Don't Need a Specialist Vector Database

There's a startup pitch waiting for you the moment you say "RAG." A shiny dedicated vector database, usually with a per-query bill that grows with you.

For most teams, you don't need it. If you're already on Postgres (and on our stack, with Supabase, you are), pgvector covers it. If you already run PostgreSQL and have under 50 to 100 million vectors, with the pgvectorscale extension, performance is competitive with dedicated databases at moderate scale.

That's a lot of headroom. Most products will never see 50 million vectors. Keeping retrieval in the same database as the rest of your data means one fewer system to secure, back up, and pay for.

When does the calculus change? Beyond 100 million vectors, purpose-built databases like Milvus or Pinecone are better suited. And if multi-tenant isolation is a hard compliance requirement, there are specialist tools built for that. Weaviate wins when hybrid search and multi-tenant isolation are primary requirements. But that's the exception, not the starting point.

Rule 5: Measure Retrieval Quality Separately from Answer Quality

When a RAG system gives a bad answer, founders blame the model. Usually, it's not the model. It's that the right context never made it into the prompt.

So you have to measure the two halves separately. The standard way to do that in 2026 is the RAGAS framework, which scores faithfulness, answer relevancy, context precision, and context recall. Low context precision means fix retrieval; low faithfulness means fix prompts.

That one distinction saves you from throwing money at a bigger model when your real problem is that your chunking is wrong.

When Our Take is Wrong

Plain talk: RAG isn't always the answer.

Fine-tuning genuinely wins when you need a consistent voice or output format every time, or when the domain is stable and you're running huge volumes where per-query cost matters more than freshness. Fine-tuning is best for stable domains and high-volume or low-latency tasks; it improves task-specific accuracy and formatting.

And the honest answer at real scale is often "both." The pattern that wins at scale is hybrid. Above one million queries per month on a stable narrow domain, fine-tuning the generator on the retrieval distribution while keeping RAG for freshness beats either standalone approach on both cost and quality.

But you earn the right to that complexity by shipping RAG first, measuring it, and finding the specific failure that fine-tuning fixes. Not by starting there.

Your pre-build checklist

Before you commit to an AI architecture, answer these with your team:

1. How often does our underlying data change? (Weekly or faster means RAG.)

2. Are we set up for hybrid retrieval, or are we shipping vector-only by accident?

3. Is there a reranker in the plan, or did we skip it?

4. Are we on Postgres already, and have we ruled out pgvector before paying for a specialist DB?

5. Do we have a way to score retrieval quality separately from answer quality?

6. What specific, measured failure would justify fine-tuning later?

If you can't answer number five, you're flying blind, and that's the rule that catches most teams after launch.

We build AI features into web and mobile products on exactly this pattern: Postgres and pgvector via Supabase, hybrid retrieval, reranking, the Claude API for generation, and real evaluation before anything ships. Senior engineers only, and you talk to the people writing the code. If you've got an AI feature on the roadmap, or a RAG prototype that demos well but falls over on real questions, send us what you've built and book a 20-minute scoping call. We'll tell you honestly whether retrieval or fine-tuning fits, and what it'll actually cost to get it surviving real users. Our AI work starts from that same first principle.

P.S. The "we'll fine-tune it" plan is the one that quietly eats two months and ships stale. If a team is telling you fine-tuning is the obvious first move for a knowledge feature, ask them how they'll keep it fresh when your data changes next Tuesday. The answer tells you a lot.