Case study
Three constraints, and what
they forced
Marker is a consumer iOS app with a live LLM feature, built solo. The interesting parts are not the screens — they are the three constraints written down before any code, and the architecture that fell out of taking them literally. No metered APIs. AI cost near zero per user. Nothing the model says is trusted.
Flat infrastructure cost
No metered map, places or geocoding API, ever. The basemap is one static file; pins ship with the app and cluster on the device.
AI cost near zero per user
Descriptions and embeddings are one-time batch jobs. One live model call exists, it is subscriber-gated, and its worst case is bounded by a fair-use cap.
Grounding, not trust
The model may choose among rows we retrieved. It may not supply facts. The server discards anything it did not itself hand over.
The grounding contract
supabase/functions/plan-tripThe trip planner takes a free-text brief — "5 days in Scottsdale in March, 4 rounds, mid budget, prefer walkable" — and returns a day-by-day itinerary. The failure everyone expects from that feature is a confidently described golf course that does not exist.
It is prevented structurally rather than by prompting. Retrieval happens before the model is called and validation happens after, so the model's only power is to choose and to explain.
The same contract applies on every turn, not just the first. Multi-turn is where this normally rots, because the model starts treating its own earlier output as ground truth. So the candidate block is rebuilt from the database each turn rather than from the previous turn's text, the prior itinerary goes back explicitly labelled as the model's own output and not as a source of facts, and the model returns a complete itinerary every time — never a diff, which would invite it to reference something it never saw.
When a trip is impossible the planner declines rather than inventing a way to satisfy it. Ask for Wisconsin in January and it says so, and offers the playable window. That window is computed in code from the candidates' stored season data — never the model's opinion, because a fluent, plausible, unsourced month range is exactly the class of claim the whole design exists to refuse.
What the evals caught
Two findings from development · both fixedNeither of these ever reached a user. Both were caught during development by the eval suite that gates each milestone, and both were fixed before the build moved on. They are written up here because the defects are not the point — what caught them is. A test suite is only worth anything once it finds something its author did not already know, and these are the two times this one did.
Every grounding check passed on a plan for the wrong city
Region resolution averaged latitude and longitude over every city matching a wildcard. The mean of several real places is a place where none of them are.
| Asked for | Planned around | Off by |
|---|---|---|
| Austin | Longview, East Texas | 368 km |
| Pinehurst | Tri-Cities, Tennessee | 275 km |
| Denver | eastern Kansas | 856 km |
| Portland | Minnesota | 2,141 km |
The expensive half was the prompts that passed. Phoenix planned around Eagar. Monterey around Paso Robles. New York around Ramsey, New Jersey. Every grounding check went green — real IDs, no invented facts — because nothing ever asked whether the courses were near the region requested.
The fix was to cluster before averaging: real coordinates, grouped by state, median of the dominant group. The durable fix was a new eval check whose expected centre is an independent constant — where the place actually is on Earth — rather than re-resolving through the system under test. An oracle derived from the thing it tests agrees with it by construction, and would have "expected" Longview and passed the Austin plan.
"The picker alone did not fix that, contrary to what I claimed"
The new geographic check immediately found something the old one had concealed. "Scotsdale" is not a misspelling. It is a real village in Jefferson County, Missouri where the directory holds exactly one course — so an exact-match rule sent the traveler 1,988 km from the 33-course Scottsdale, Arizona they meant. The previous lexical check stayed silent because the scheduled course's city string matched the typed region exactly.
Rather than invent a typo-distance heuristic with no measured data behind it, the answer was to stop making anyone type a city name blind: the region field became a picker over the offline index. That shipped — and it did not work. The picker matched by prefix and substring, and a misspelling is not a substring of the correct spelling, so "Scotsdale" still offered only the Missouri village. Running it on the simulator is what showed the difference between the claim and the behaviour.
The real fix adds a fuzzy fallback that runs only when the exact pass left room, and only surfaces a candidate more prominent than what was already found — which is what separates the two directions of the pair. The typo surfaces the bigger place; typing "Scottsdale" correctly does not dredge up the obscure one. It stays inside the frame budget by construction: first-letter buckets cut ~7,700 cities to a few hundred, a length window filters again, and the distance loop bails early.
Cost as an architectural constraint
Two moving parts in steady state"No metered APIs" is a design input, not a preference. It rules out hosted maps, geocoding and places search at the start, which forces every one of them to be solved with something static.
The basemap is a single PMTiles file on object storage. Course pins ship inside the app bundle and are clustered and filtered on the device — the same technique the demo on the overview page uses to draw 12,640 courses in a browser with no tile server. Every course photo is a public-domain USGS aerial, fetched once by the pipeline and served flat-cost with immutable caching.
The same logic bounds the AI. Descriptions and embeddings are offline batch jobs that run on a laptop, so they cost nothing per user and nothing per session. The one live call is subscriber-gated, and the turn is claimed before the model runs rather than after — so the counter bounds spend rather than counting successes. That ordering matters: generate-then-increment lets fifty concurrent refinements all read the same count, all call the model, and all but one fail, which is fifty billable calls for one tick of the meter.
Steady-state operational surface: a Postgres database and one static file host.
Isolation and data honesty
Every user table has row-level security, and the policies live in the migration beside the table they protect rather than in a separate security pass. SQL isolation tests assert that one account cannot read another's rows. Entitlements are writable only by the server-side billing webhook — never by the app — so a compromised client cannot grant itself a subscription.
A separate rule governs what the product is allowed to say. Displayed facts come only from explicit source tags. Signals that were derived rather than sourced — hole counts estimated from mapped geometry, public or private inferred from a name pattern — are allowed to drive map filter chips, where being approximately right is useful, and are forbidden from appearing as facts on a course page, where being approximately right is a lie. The data pipeline matched a hand-labelled ground-truth set 50 out of 50.
One engine, many niches
Lint-enforcedGolf is skin #1 of a domain-agnostic engine. The engine owns reusable behaviour — maps, discovery, lists, profiles, planning — and is forbidden from containing golf vocabulary. The skin supplies vocabulary, theme, attribute schema and curated seeds. Database naming is generic too: places, place_logs, lists, never course or round.
This is the kind of rule that quietly rots, so it is not a convention — it is a lint that runs in the standard verify step and fails the build. Adding ski resorts or surf breaks means a new skin package and an ETL adapter, not a fork.
apps/mobile the engine app (Expo) — no niche vocabulary, lint-enforced packages/core skin contract + domain types packages/skins/golf vocabulary, theme, attribute schema, curated lists tooling/etl extract → transform → enrich → load tooling/eval end-to-end grounding + community evals supabase/ 17 migrations, RLS isolation tests, edge functions infra/tile-worker edge-cached tile / photo / privacy worker docs/ architecture, decision records, data-quality reports
Stack
App
- Expo / React Native
- TypeScript (strict)
- expo-router
- MapLibre + PMTiles
Backend
- Supabase Postgres
- PostGIS + pgvector
- Edge Functions (Deno)
- Cloudflare Workers + R2
AI
- Claude Haiku (planner)
- Batch API (descriptions)
- Local embeddings
- zod at every boundary
Data
- OpenStreetMap + Overture
- Wikidata enrichment
- USGS NAIP aerials
- Elevation, wind, season
I wrote the architecture plan and the decision records first, then directed a team of Claude sub-agents — mobile, backend, data, AI, release and a reviewer — against them, using security reviews and evals as the gates between milestones. The commit history is written to be read: messages explain why a change was made, including the cases where an earlier fix turned out not to work.