HomeRun
A self-initiated AI shopping assistant for a real Indian construction-materials store.
The thesis
HomeRun (home-run.co) is a Shopify store selling construction materials in India — cement, tiles, waterproofing, wiring, plywood, sanitaryware. The buyer's real problem isn't browsing; it's translation. A homeowner knows 'my bathroom is leaking' but not that the fix is a surface primer, an acrylic waterproof coating at the right coverage, and a membrane. A contractor tiling a 2BHK needs a full bill-of-materials with bulk-priced bags, not a search box. I built, unprompted, a working AI assistant that closes that gap and pitched it as a candidate recommendation. The insight driving it: retrieval, not the LLM, is the product. A plain-language project gets parsed into structured intent, answered from a governed catalog of 326 real scraped products with 1,363 variants, and turned into either a homeowner's quantity calculator or a contractor's staged BOM — then routed to HomeRun's actual cart via Shopify deep-links, so the demo ends in a real, buyable basket. The bet that makes it credible is discipline: the LLM never writes a price or a SKU, every retrieval change is gated by a standing eval suite, and retrieval wins have to be proven on blind, freshly-authored held-out cases before they ship — the same rigor a real search team lives by.
What it does
Plain-language project to the right SKUs
A shopper describes a project the way they'd say it out loud, and the assistant returns the specific products that project actually needs, each with a short 'why'. It asks at most one clarifying question when the request is vague, rather than dumping a search page. The heavy lifting is a query-understanding layer that turns messy phrasing into structured intent before retrieval runs.
Two personas, two experiences
The same catalog is presented very differently for a homeowner versus a contractor, and the segment toggle drives both product ranking and tone. Homeowners get plain-language-to-trade translation, a friendlier voice, and a quantity calculator that writes its result back into the cart. Contractors get trade language, a staged bill-of-materials that owns the screen, bulk pricing, and a per-line swap drawer with A-vs-B spec comparison.
Real, buyable checkout via Shopify deep-links
Because HomeRun's cart is cookie-bound and can't be written to directly, the assistant keeps its own accurate cart and hands off through Shopify's native multi-line deep-link, landing the exact items and quantities in the store's real cart. The catalog was rebuilt from the live store so every product carries a real numeric variant ID, and sampled deep-links were verified to resolve — the demo ends in an actual purchasable basket, not a mockup.
Grounded answers that refuse to bluff
The language model never writes a price, SKU, or product title — those come only from the governed catalog — so it structurally cannot hallucinate commerce facts. For items the store genuinely doesn't stock, the system returns an honest 'no results' instead of confidently recommending the wrong thing. Every recommendation can also explain itself through a 'Why this?' disclosure backed by the real product spec sheet.
Photo or paste a materials list
As an alternative to chatting, a user can paste a text list or snap a photo of a handwritten materials list. Vision OCR reads it and each line is matched to a SKU, sorted into confident, ambiguous, and no-match buckets — never silently substituted. It turns a contractor's scribbled list into a curated cart on the same real checkout.
In focus
RETRIEVAL_V3 — query understanding, not keyword match
The deterministic spine the LLM only ever adds to
A data-driven query-understanding pipeline, default-on behind a one-flag rollback (RETRIEVAL_V3=0), that turns messy phrasing into the right SKUs without ever betting the experience on the model.
- →Ontology spine + compositional query parse → structured-hybrid retrieval (governed-field filter + dense bge embeddings + BM25 lexical, fused by Reciprocal Rank Fusion) → a CPU cross-encoder re-ranker (ms-marco-MiniLM-L-6-v2) → a calibrated answerability gate that abstains to 'no_results' when the catalog lacks the requested head noun.
- →Deterministic-spine-first: the spine is fully load-bearing and 429-proof; an optional free-model slot-extractor tail only fires on low spine-confidence or cache hits. The LLM only ever adds, never gates, and silently falls back to the spine on error/timeout.
- →A safety-net floor makes the structured filter non-destructive, so a mis-parse can never empty the grid — and guarantees the new pipeline is never worse than the old.
- →28 of 29 held-out cases answered on the spine in ~0.1s; the LLM path engaged only once (19–31s).
Eval-first, with blind held-out discipline
How 'good' got defined before anything shipped
Every RAG or catalog change is gated like a test — and retrieval wins have to survive cases authored without seeing the fix.
- →A standing relevance harness runs 14 declarative golden cases (6 contractor / 8 homeowner) on the grounded LLM-off path and exits non-zero on any regression — currently 14/14 on both the V3 default and the v2 rollback.
- →Before declaring a win, fresh cases are authored blind — colloquial phrasings, misspellings ('geyzer', 'wall puty'), Hindi trade terms ('kabza', 'saria'), and honest not-stocked probes. This caught RETRIEVAL_V3's first overfit and gated the flag-flip.
- →On the blind fresh set, V3 scored 23/29 (79.3%) vs v2's 19/29 (65.5%) with zero floor violations, and cut the confident-wrong rate on genuinely not-stocked items from 57% to 43% — tracked as still-open work, not hidden.
A look inside
Architecture & decisions
- →Three-tier stack: React + Vite client (Vercel static) → Express dev-proxy → Flask backend (server.py) hosting the RAG pipeline over a ChromaDB vector index; in production the browser calls the Flask API on a Hugging Face Space directly.
- →Retrieval is a two-call flow the UI depends on: POST /api/search renders the segment-ranked product grid instantly from retrieval (never blocks on the LLM), then POST /api/answer swaps in an LLM one-liner asynchronously — with a templated fallback so the experience survives an LLM 429 or outage.
- →RETRIEVAL_V3 query-understanding pipeline (default-on, one-flag rollback): ontology spine + compositional query parse → structured-hybrid retrieval (governed-field filter + dense bge embeddings + BM25, fused by RRF) → CPU cross-encoder re-ranker (ms-marco-MiniLM-L-6-v2) → a calibrated answerability gate that abstains to 'no_results' when the head noun isn't in the catalog → an optional free-model slot-extractor tail that only fires on low spine-confidence or cache hits.
- →Embeddings are local and free (BAAI/bge-small-en-v1.5, 384-dim) run at index and query time; chat/vision synthesis is OpenRouter free-tier models (Llama-3.3-70B primary + Qwen/Gemma fallback chain) — the only secret is OPENROUTER_API_KEY, and the system degrades gracefully without it.
- →10 selection 'archetypes' (BAGGED_DRY, LIQUID_CAN, COIL_RUN, PANEL_SHEET, MODULAR_FITMENT, CUT_LENGTH_SIZE, DISCRETE_UNIT, SPEC_DEVICE, ASSEMBLY_KIT, FINISH_VARIANT) classify all 326 SKUs and drive a per-archetype selection card and quantity engine, instead of one generic card + one area×coverage rule.
- →Assisted checkout via Shopify's native multi-line deep-link (home-run.co/cart/<variant_id>:<qty>,…): the app can't read HomeRun's cookie-bound cart, so it keeps local variant-keyed cart state (localStorage-persisted, accumulate + qty controls) and hands off to the store's real cart/checkout.
- →Alternative 'photo/paste a list' entry (POST /api/parse_list): a text list or photo runs through a free vision-model OCR, each line is matched to a SKU via the same retrieval + answerability gate, bucketed as confident/ambiguous/no-match (never silently substituted), into a curated cart on the same deep-link checkout.
Engineering highlights
- →Governed catalog as a hard gate: catalog_schema.py + validate_catalog.py schema-validate data/products.json and abort classification on any HARD violation (driven HARD violations 106 → 0; 13 soft warnings remain), emitting a machine + human catalog-health report — so bad data can't silently reach retrieval.
- →Standing relevance eval harness: 14 declarative golden cases (6 contractor / 8 homeowner) run the grounded LLM-off path deterministically and exit non-zero on any regression, gating every RAG/catalog change like a test. Currently 14/14 on both the V3 default and the v2 rollback.
- →Blind held-out discipline: before declaring a retrieval win, fresh cases are authored without seeing the fix (colloquial phrasings, misspellings, Hindi trade terms, honest not-stocked probes). This caught RETRIEVAL_V3's first overfit and gated the flag-flip; the fresh set showed V3 23/29 vs v2 19/29 with zero floor violations.
- →The LLM is walled off from ground truth: a permanent, code-verified no-regression rule guarantees the model never populates a product field — every price, SKU, and title comes only from the catalog, so synthesis can't hallucinate commerce data.
- →Honest abstention over confident-wrong: the answerability gate returns 'no results' for gibberish, off-topic queries, and real-sounding-but-uncatalogued asks (curtain rods, wallpaper, AAC blocks), measurably cutting the confident-wrong rate on not-stocked items from 57% to 43% — flagged as still-open work rather than hidden.
- →Real bulk-pricing economics: 25 real bulk tiers (all BAGGED_DRY goods) merged from source after a fabricated 'fabricator' pricing path was removed (246 → 25 honest tiers), powering bulk-aware BOM quantities, a live 'add N more to unlock' nudge, and a cart-level savings bar grounded in each line's real tier.
- →A full flow audit (both personas, 3 lenses) produced 29 findings (5 critical / 7 high / 10 medium / 7 low) triaged into Tier A/B/C; Tier A trust fixes and the Tier B over-leading-routing fix shipped, each re-verified against the eval suite before close.
- →Catalog rebuilt from scratch off the live Shopify storefront JSON API (scraper/scrape.mjs): 326 products / 1,363 variants / 147 collections with 100% numeric variant_id and image coverage, sitemap-cross-checked (delta 0), and 20/20 sampled deep-links resolved live — the data foundation that made real checkout possible.
Why this matters
HomeRun is the case study that shows I can do a real product-org job end to end. I did competitive/UX recon on a live store, wrote a canonical PRODUCT spec with a frozen API contract and a permanent no-regression gate, ran a whole team of specialist sub-agents (PM, backend, frontend, designer, QA, researcher, doc-syncer) against it, and — critically — refused to let an AI feature ship on vibes. I defined 'good' as a data-driven eval suite, insisted retrieval gains be proven on blind held-out cases (which caught my own V3 retrieval overfit and gated the release behind a safety-net floor guaranteeing the new pipeline is never worse than the old), and made honesty a product rule: for genuinely un-stocked items the assistant abstains with an honest 'no results' rather than confidently recommend the wrong thing. I scoped ruthlessly — ship the commerce loop first with zero LLM dependency, defer GST-invoice export and telemetry — and named my own blind spots on the record: post-deep-link conversion attribution, the residual confident-wrong rate, and catalog gaps that are merchandising decisions rather than bugs. That combination — domain translation, eval-first rigor, intellectual honesty, and shippable scoping — is exactly the judgment an AI PM is hired for.
Want the full story?
Ask the portfolio chat, or reach out directly.