The product decisions behind a one-person AI content business: who it serves, where the quality bar sits, what I killed, and the honest numbers.
Topic
Llm Evals
Projects
Writing
- Hybrid LLM Evaluation: Let Code Count, Let Models JudgeWhich parts of 'good' to give code and which to an LLM judge: per-dimension weights, hard caps, and publish bands as product decisions.
- LLM Evals for Product Managers: A Field GuideA practical guide to LLM evaluation for PMs — decision-first design, golden sets, judge calibration, hybrid scoring, and regression protection.
- The Day My LLM Judge Drifted — and How I Caught ItMy LLM judge quietly inflated scores and re-ranked my configs. The before/after data, the fix, and what a PM should require before trusting a judge.