doug
DashboardScoreboardQueueDocsGitHubAbout
Menu
DashboardScoreboardQueueDocsGitHubAbout
Sign in

Getting started

  • Introduction
  • Quickstart

Concepts

  • Risk routing
  • Defect labels
  • The cleared band
  • What Doug gets wrong

Reference

  • CLI · doug-backtest
  • The report

Coming up

  • MCP · Pattern Garden
  • REST API

Meta

  • Changelog
View as llms.txt

Concepts

Risk routing

Available
Doug’s core claim is a capture curve: reading only the top-scored N% of PRs (the “budget”), what share of defect-inducing PRs land inside that band?

The capture curve is measured on the backtest CLI: scoring there is deterministic, and the holdout half learns hotspot path segments from the repo’s own past defects. Live GitHub App reviews use static hotspot segments plus, when enabled, an LLM diff-reader (ADR-0004) — not the rolling-window learner, and not a model-free hot path.

  • capture@budget — headline routing metric: % of defect PRs caught at a given flag rate
  • AUC — budget-independent summary across all flag rates
  • size-only baseline — every report shows it; if Doug can’t beat “biggest diff first,” you should not pay for Doug
Per-rule precision (sentry backtest, train half)
rule                    fired   hit   lift
hotspot_path              581    19  2.34x
refactor_pure_mod         332     9  1.94x
deletion_leaning          214     4  0.82x  # dead — kept honest

# every rule ships with its receipts;
# rules that don't pay get reported, not hidden
PreviousQuickstartNext Defect labels