Scoring & Feedback Loop
A self-improving scoring system where human decisions feed back into future runs. Works for any workflow that assigns a score to an entity and learns from outcomes — leads, candidates, content, accounts, vendors.
How It Works
Each scoring run follows a four-step loop. The entity changes (a lead, a candidate, a content draft, an account); the loop is identical.
The feedback loop has two paths: a write path that records decisions, and a read path that loads historical context before each scoring run. Together, they create compound learning — every execution makes the next one more accurate without manual recalibration.
Use Cases
Lead Scoring
Score inbound leads by fit and intent. Sales reps mark each as qualified or disqualified. The system learns which signals (job title, company size, tech stack, engagement pattern) actually predict conversion — and adjusts weights automatically.
Candidate Screening
Score applicants against a role. Recruiters shortlist or pass. The AI learns which profile characteristics your team actually advances to interview — not just keyword matches, but the patterns reviewers implicitly prefer.
Content Quality
Score AI-generated drafts before publish. Editors approve, request revision, or reject. The system learns your team's style preferences, tone boundaries, and what triggers a rewrite — without you writing a style guide.
Customer Health
Score accounts for churn risk or expansion potential. CSMs flag at-risk accounts or mark health predictions as accurate. The system calibrates on your actual churn events and learns which early signals matter for your customer profile.
Other common uses: vendor/supplier qualification, investor-startup matching, support ticket prioritization, SEO content scoring.
Write Path
After each scoring run, the workflow writes decision edges to the Knowledge Graph. Edge types adapt to your domain — below are the generic types and examples for each use case:
| Edge Type | Meaning | Example metadata |
|---|---|---|
| SCORED | AI assigned a score to this entity | score, rationale, model, timestamp |
| APPROVED | Human moved entity forward (qualified lead, shortlisted candidate, approved draft) | reviewer, notes, timestamp |
| REJECTED | Human passed / disqualified (lost deal, pass candidate, rejected content) | reason, reviewer, timestamp |
| CORRECTION | Human overrides AI score with a different value | original_score, corrected_score, reason |
| OUTCOME | Final real-world result recorded (closed deal, hired, churned, published) | result, value, date |
Read Path
Before each scoring run, the AI step runs parallel queries against the Knowledge Graph to load context:
Scoring History
Previous scores for the same entity. The AI sees how scores changed across runs and why — useful when re-scoring a lead after new activity or a candidate who reapplied.
Human Decisions
APPROVED and REJECTED outcomes from reviewers. The AI learns which scores and rationales led to approvals versus passes — building an implicit model of your team's actual criteria.
Calibration Data
Statistical benchmarks: average scores by category, approval rates by score band, typical ranges for your dataset. Prevents score inflation and keeps outputs comparable across runs.
Pattern Insights
AI-generated observations from previous runs. Examples: "high-scoring leads from LinkedIn outreach convert at 2x the rate of inbound form leads", "candidates from agency backgrounds get rejected at shortlist stage regardless of score."
Outcome Benchmarks
Final outcomes (closed deal, hired, churned, published) compared against initial scores. The AI measures its own prediction accuracy and adjusts confidence accordingly.
Compact Profiles
The Summarize step is what keeps prompts bounded. Raw scoring logs are never injected into future runs. Instead, the system builds a compact profile per entity — a structured summary carrying only what matters for the next decision.
Lead Scoring — Jane Smith, Run 1
Title: Head of Marketing Company: Acme Corp (180 employees, SaaS) Industry: B2B SaaS LinkedIn: Active (3 posts/month)
Score
65 — “ICP match on size and industry. No prior contact data.”
Lead Scoring — Jane Smith, Run 5
Title: Head of Marketing Company: Acme Corp (180 employees, SaaS) Industry: B2B SaaS — Compact profile (4 prior runs) — 2 emails sent. Opened 2 of 2. No reply. Colleague (CTO, Mark T.) was qualified last quarter — meeting booked. Rep note: "Likely decision-influencer, not buyer. Needs CTO intro first."
Score
71 — “Re-engagement candidate. Warm path via CTO colleague. Route to account-based sequence.”
Candidate Screening — Alex Rivera, Run 1
Title: Senior Engineer, 6 yrs exp Stack: Python, TypeScript Last role: Series B startup Education: CS degree
Score
72 — “Strong technical fit. No signal on culture or team dynamics.”
Candidate Screening — Alex Rivera, Run 5
Title: Senior Engineer, 6 yrs exp Stack: Python, TypeScript Last role: Series B startup — Compact profile (4 prior runs) — Shortlisted: 1 previous cycle (rejected at offer stage — compensation mismatch). Interviewer note: "Strong systems design. Weak on product ownership." Similar profile (agency background) passed at shortlist 0/3 times.
Score
68 — “Technical fit confirmed but prior rejection pattern and product ownership gap lower confidence. Verify comp range before advancing.”
Compact profiles are built from stored decisions — not replayed logs. Prompt size stays bounded regardless of how many prior runs exist.
Accuracy Over Time
Typical progression for a lead scoring workflow processing 500+ leads per month:
Run 1: 61% accuracy (cold start, no history) Run 3: 70% accuracy (basic pattern learning) Run 5: 77% accuracy (incorporating rep feedback) Run 8: 83% accuracy (calibration adjustments) Run 12: 88% accuracy (compound learning)
The curve is similar across use cases. Starting accuracy varies by data quality; the slope depends on feedback volume and how consistently reviewers record decisions.
Workspace-Scoped Memory
All feedback data is scoped to the workspace:
- •Institutional knowledge persists across team member changes — a new rep or recruiter benefits from all prior decisions immediately
- •Multiple workflows sharing the same KG contribute to shared learning — lead scoring and deal scoring inform each other
- •Human corrections are first-class data — overrides are stored and weighted, not discarded
Next Steps
- KG API — Query scoring edges and history programmatically
- AI Steps — Configure the AI reasoning step that uses feedback loop context
- Human-in-the-Loop — Set up approval gates that feed decisions back into scoring
- KG Overview — Full architecture of the Knowledge Graph