DTH / Mola, in depth

A production AI assistant that helps dental practices find and evaluate technology vendors — LLM orchestration, hybrid retrieval, an editorial data moat, evals and safeguards, deployed on AWS. The reframe: vendor selection is a trust problem, not a search problem.

The reframe everything turned on

Vendor selection isn’t a search problem. It’s a trust problem. Practices weren’t short on options — they were drowning in 500+ of them with no one credible to match them. That single reframe set the product, the data strategy, and the safeguards.

The problem

Dental practices were drowning in 500+ vendor options with no trusted matcher. Every “directory” was a list; none answered the real question — which vendor is right for a practice like mine, and can I trust the answer?

• Choice overload, zero credible guidance
• High-stakes, expensive, sticky purchases
• Trust gap, not an information gap

What shipped

A live AI assistant in production, held to a no-fabrication bar by its eval suite — credible enough that the founder raised capital on the platform.

• Live at search.dentaltechhub.com
• Answers grounded in the vendor corpus, enforced by evals
• Investor-credible — raised on it

What I learned

In a trust product, your safeguards are your feature. Accuracy isn’t a backend concern you bolt on — it’s the thing the user is actually buying. So the eval suite and safeguards got first-class roadmap status, not “later.”

• Trust must be engineered, then measured
• An eval baseline is a product asset
• The data moat is the durable advantage

How it was built — decisions & trade-offs

1. Build the editorial data moat first

Curate 500+ vendors as structured, editorial-grade data. In a trust product the corpus is the product — no clever model rescues a thin dataset.

Trade-off: Slow manual editorial work up front instead of a fast scrape. Chose durability and credibility over speed-to-demo — the moat competitors can’t quickly copy.

2. Replace the state machine with an LLM conductor

The first design was a hand-built state machine — brittle, every new path meant new branches. I rearchitected to an LLM conductor that orchestrates retrieval and tools dynamically.

Trade-off: Gave up the determinism of explicit states for flexibility and far less maintenance — and bought control back with tight safeguards and evals.

3. Make accuracy retrievable: hybrid retrieval

Pair semantic search (Chroma) with keyword search (BM25) and fuse rankings with RRF. Semantic alone misses exact product names; keyword alone misses intent.

Trade-off: More moving parts vs. a single vector store. Worth it — grounded retrieval is what makes the no-fabrication bar achievable.

4. Treat safeguards as a feature, not a filter

A 7-module safeguards pipeline backed by 79 tests and an eval suite. “It didn’t make something up” is the headline feature — so it gets tested like one.

Trade-off: Engineering time with no visible feature. But it’s the difference between a demo and something a founder can raise money on.

5. Engineer latency and cost to production-grade

Drove time-to-first-content from 38s → 15s → 2–3s (streaming + retrieval/orchestration tuning) while holding $0.03–0.05 per turn.

Trade-off: Constant tension between quality, speed, and token cost. Set an explicit per-turn budget and optimized within it.

6. Keep the moat alive: corpus-refresh pipeline

A repeatable refresh pipeline (on AWS Elastic Beanstalk + RDS) keeps the 500+ vendor corpus current. A stale data moat stops being a moat.

Trade-off: Ongoing operational cost vs. a one-time dataset. Treated freshness as a product commitment — trust decays with every outdated fact.

The stack, in one line each

• LLM-conductor orchestration
• Hybrid retrieval · Chroma + BM25 + RRF
• 500+ vendor editorial moat
• 7-module safeguards · 79 tests
• Eval suite · no-fabrication bar
• AWS Elastic Beanstalk + RDS