The short version
Cost per qualified lead down 38%, with monthly spend down 33%.
Key decision
Replace call volume with transcript-verified leads as the bidding signal. Put risk policy and rollback around automated changes.
Verification
38 controlled experiments, an agency benchmark, anomaly baselines, and API mutation verification.
Google Ads auction8 accounts · GAQL
Phone callstracking numbers · transcripts
Claude classification19,968 calls · ~$0.03 each
Attribution enginecall → campaign → keyword · 124K rows
Offline conversions4,449 uploaded to Google
Bidding signalverified leads, not call volume
nightlyscheduled · unattended
Architecture model. Scroll horizontally on smaller screens →
Problem
I inherited campaigns where only 5% of ad-driven phone calls were qualified leads. Google's bidding was optimizing on call volume, which meant it was optimizing on noise. The truth about each call lived in its transcript, not in the ad platform, so no off-the-shelf tool could close the loop. The engine was built to replace the bidding signal with verified outcomes and then to act on that signal every night without me.
Results
Cost per qualified lead↓ 38%
Indexed outcomes · baseline = 100 · not a time series
Architecture
One nightly cycle, scheduled by launchd with missed-run recovery, in this order:
nightly optimization cycle
SnapshotPull campaign and keyword stateread
ClassifyTranscript → qualified-lead signalevaluate
AttributeCall → campaign → keywordjoin
ConvertVerified lead → offline conversionverify
Grade + mineScore performance and find wastepropose
QueueAutomatic low risk; approval above itgate
Verify + reportCheck live state and flag anomaliesaudit
Dry-run default · verified responses · recorded rollback state
- Snapshot performanceCampaign and keyword metrics pulled across all 8 accounts via GAQL.
- Classify every new callTranscripts pulled from the call-tracking API and classified by the Claude API as qualified lead, applicant, existing client, or cannot service, with a cache to hold cost near $0.03 per call.
- Attribute leads to keywordsA custom attribution engine joins each call to the campaign and keyword that produced it. 124K attribution rows to date.
- Upload offline conversionsQualified leads are uploaded to Google Ads as call conversions, with an alert if the pipeline stalls. 4,449 uploaded. Along the way I found that click-conversion uploads do not attribute for call ads, a quiet API behavior that would have invalidated the whole loop.
- Grade campaigns and keywordsEvery campaign and keyword graded A through F against qualified-lead truth, not against Google's reported conversions.
- Mine waste patternsCannot-service transcripts are mined for recurring waste and converted into negative keywords, with location-aware compliance exceptions.
- Generate and execute the action queueBudget-aware, deduplicated, risk-tiered. Low-risk actions execute automatically. Medium and high wait for approval.
- Detect anomalies and failuresPer-campaign z-score deviation against 30-day baselines, zero-spend detection, bid-strategy phase checks, and self-monitoring that alerts when the engine itself is broken.
- ReportA prioritized email with a triaged subject line, from critical down to all clear with the count of optimizations applied.
Controls that let it spend real money
- Dry run by default. Every mutation class runs in dry-run mode unless explicitly confirmed.
- Risk tiers. Actions are classified as high, medium, or low risk. Only low risk auto-executes, under budget caps and protected-keyword whitelists.
- Paused-by-default creation. New campaigns pass a single validation gateway, are created paused, receive baseline negatives, and are verified against live API state before registration.
- Rollback on every mutation class. Full state tracking with orphan detection.
- Never assume success. After a self-audit found a dedup bug that had silently discarded approved optimizations and conversion uploads that had failed without error, every API mutation is now verified against its response, and failures have a permanent taxonomy.
- Statistically honest scoring. A failing keyword grade requires a minimum observation window, enough clicks, and longer-term confirmation, because low-volume keywords that converted infrequently were being falsely flagged.
- Bid-strategy phases require two signals. Moving from capped-CPC launch to maximize-conversions to target-CPA requires both Google-reported conversions and verified qualified leads, so the engine cannot optimize toward junk calls.
Evaluation
- 38 controlled A/B experiments in a custom framework, with a knowledge base that stops failed ideas from being re-tested.
- A head-to-head benchmark against an external agency across 4 locations, using audit scripts that compare the engine's performance to the agency's on equal footing.
- z-score anomaly detection against 30-day baselines on every campaign.
- Live-CPL reports always query the source APIs. Financials are never reported from cached snapshots.
Numbers
| Cost per qualified lead | down 38% |
|---|---|
| Monthly spend over the same period | down 33% |
| Weekly cost per acquisition | $107 against a $150 target |
| Estimated wasted spend eliminated | $30–50K per year |
| Call transcripts AI-classified | 19,968 |
| Offline conversions uploaded to Google | 4,449 |
| Attribution rows | 124K |
| Controlled A/B experiments | 38 |
| Codebase | 59,000 lines of Python |
| CLI surface | 115 commands |
| LLM cost per call | ~$0.03 |
Built on the same data
Call coaching, reporting, and distribution
- Call-scoring and coaching product. An eight-rule rubric applied by the Claude API to every qualified call at about $0.03 per call, a director review workflow with transcript evidence and a dispute path, red-flag escalation, and a cross-office roll-up. Requirements gathered from the operating client, pilot location moved on their input, live training delivered to directors across 7 locations.
- Multi-tenant attribution dashboard. Franchise directors close the loop from lead to revenue. Own page.
- Executive reporting. Automated weekly reports with Plotly charts and Excel workbooks, delivered as Outlook drafts through Microsoft Graph for human review. 18+ weekly executive reports to the C-suite.
- Distribution. An enterprise software vendor building for the same sector, after a full year of monthly product reviews in which I resolved their Google Ads API blockers, is preparing to distribute the optimization engine and the call-scoring product to its customers.