Scorer Demo Not used when you paste a chat

Scorer Demo With Repair Rules

Score one extraction against a gold event you supply: precision, recall, F1, strict and lenient accuracy, schema adherence and safety, plus what the hand-written repair rules change.

🍱 View Kitchen Sheet
What this page is and is not

Scorer demo. Needs gold labels. Starts from the offline rules (or an injected fake flaw), applies hand-written repair rules written for these four sentences, and is not used when you paste a chat. The adversarial suite is expected to fail.

No model is called from this page. A high score here means the repair rules recognise the sentence they were written for, not that extraction is accurate on messages they have not seen.

🧪 Configure Benchmark & Evaluation Threshold

Tests handling of negated cancellations (e.g. 'cancel mat karna, 2 hi chahiye')

90.0%
60% (Lenient) 90% (Industry Standard) 100% (Strict)

The injected flaw is a hand-built wrong event, not a model output. It exists to show the scorer reacting to a known mistake.

Score Against This Gold Event PASSED
100.0% / 90.0% target
Initial: 100.0%  (Δ +0.0%)
Micro F1 Score Balanced
1.00 / 1.00
Precision: 1.00 • Recall: 1.00
Adherence Match Strict
100% strict match
Lenient match: 100%
Attempts Spent 1 Attempts
1 scoring attempts
Safety Score: 100% • Schema: 100%

🔄 Scoring & Repair-Rule Trajectory

Step-by-step audit of the score, the field-level mismatches, and what each repair rule changed.

This Sentence Reached Your Threshold
1 Attempt 1: PASSED TARGET
Score: 100.0% (F1: 1.00, Strict: 100%)
🛠️ Repair Rules Applied To This Sentence:
  • Benchmark passed target threshold. No further repairs required.

🔍 Event Inspection Against The Gold Event

Start Point (rules or injected flaw) Score: 100.0% 
[
  {
    "action": "new",
    "applied_to_plate_count": false,
    "audit_flags": [],
    "confidence": 0.9,
    "customer": "Ravi",
    "date": "tomorrow",
    "is_late": false,
    "items": [
      "thali"
    ],
    "line_index": 0,
    "meal": "lunch",
    "mods": [],
    "needs_review": false,
    "note": null,
    "pause_until": null,
    "payment_amount": null,
    "payment_note": null,
    "qty": 2,
    "raw_text": "kal lunch cancel mat karna, 2 hi chahiye please",
    "secondary_customer": null,
    "secondary_qty": 1,
    "timestamp": null
  }
]
After The Repair Rules Score: 100.0% 
[
  {
    "action": "new",
    "applied_to_plate_count": false,
    "audit_flags": [],
    "confidence": 0.9,
    "customer": "Ravi",
    "date": "tomorrow",
    "is_late": false,
    "items": [
      "thali"
    ],
    "line_index": 0,
    "meal": "lunch",
    "mods": [],
    "needs_review": false,
    "note": null,
    "pause_until": null,
    "payment_amount": null,
    "payment_note": null,
    "qty": 2,
    "raw_text": "kal lunch cancel mat karna, 2 hi chahiye please",
    "secondary_customer": null,
    "secondary_qty": 1,
    "timestamp": null
  }
]
Gold Event You Supplied 100% Target 
[
  {
    "action": "new",
    "confidence": 1.0,
    "customer": "Ravi",
    "date": "tomorrow",
    "items": [
      "thali"
    ],
    "meal": "lunch",
    "mods": [],
    "payment_amount": null,
    "payment_note": null,
    "qty": 2,
    "secondary_customer": null
  }
]