Research Methodology

What we measure today

Template Mode validates local-business landing pages paired with Meta ads. Our primary signal is sustained ad spend: how long an advertiser keeps ads active and how many creative variants they run — not conversion rate or ROAS (which Meta Ad Library does not expose).

• Ad longevity — days since earliest active ad start date in Ad Library
• Variant count — distinct active creatives (collation groups)
• Structural differentials — page features that appear ≥20pp more often in winners vs controls (precision ≥80%)
• Revalidation — batch Meta check; dead ads get tombstone notes and Grade C cap

Website template grades (A / B / C)

Script-computed in compute_grades.py — no manual judgment.

• Grade A — hygiene gates pass · ads ≥180d · ≥2 variants · ≥2 Δ≥20pp differentiators · dossier complete · ads still active
• Grade B — hygiene gates pass · dossier complete · ads 90–179d or organic top-3 review velocity
• Grade C — incomplete dossier · failed capture · dead ads · thin longevity · or other gate failures

Ad template grades

• Grade A — ad longevity ≥180d and ≥2 variants (multi-variant sustained creative)
• Grade B — ad longevity 90–179d
• Grade C — under 90d (not shown in ad template gallery)

Freshness policy

• Stage 2 ad validation: July 2026 authoritative pass (strict domain/name matching)
• Gallery revalidation: batch Meta check on publish; tombstone when no active ads match
• PNG captures: static snapshots; blocked sites retain analysis data with capture-failed badge
• Hypothesis-tier fields (review velocity, some detector flags) labeled in dossier Q5 footnotes

Claims we cannot make yet

• **Conversion rates (% of visitors who book)** — No analytics on validated samples; NiceJob "10% lift" is Tier C marketing only.
• **Embedded OpenTable/Resy widget requirement** — Split detection shows **0%** embed rate across independents (`verified_claims_v3.md`); prior 92%/67% Toronto/NYC gaps were **detector artifacts** (`booking_split_report.md`).
• **Review block frequency for restaurants** — Detector precision **50%**; do not claim "most restaurants show star ratings on-page."
• **Popmenu as market-proven in NYC/Toronto** — **0** fingerprint hits in 177-domain scan; national scale claims are Tier C only.
• **Platform template = winner template** — Toast/BentoBox ship **20–30%** header phone vs **70–83%** winners; platform convergence scores ≠ winner mimicry.
• **Exact booking vendor market share (Tock vs Calendly)** — Legacy substring detection retired; vendor tags unreliable.
• **Revenue or LTV impact of Template Mode builds** — No shipped POC A/B tests yet (Stage 6).
• **Organic SERP rank causation** — Gate 2 organic URLs collected as **candidates**, not ranked winners with longevity proof (except where paired with ≥90d Meta ads).

First-party pilot roadmap (Stage 09+)

POC sites (Stage 07) enable A/B tests on live traffic. Until pilot data exists, we do not publish conversion benchmarks. NiceJob marketing lift claims remain Tier C only.