BreezyCorp Handbook

Appearance

Auto-suggest returned UNCLASSIFIED for too many rows ​

Auto-suggest ran but the confidence distribution is dominated by UNCLASSIFIED (or LOW).

What this means ​

Claude could not place a meaningful number of TB rows against the SFRS taxonomy. The model is rarely "wrong" on a healthy TB; this points at an input problem.

Causes ​

  1. Sign normalisation flipped the wrong way. The detector treated a "credit-positive" export as "debit-positive"; the LLM sees revenue and expenses with inverted signs and refuses to commit.
  2. Account descriptions are missing. The TB has codes but no descriptions; the LLM has only the code to go on. SFRS keys are description-driven.
  3. The TB is in a different language. The model handles English; descriptions in Chinese / Tamil / Malay degrade match quality. Not blocking, but degrades.
  4. The TB is highly client-specific (custom internal account names). The model has no priors. Consider building per-client vendor synonyms in the Phase 4 backlog.
  5. The worker is using the mock adapter (ai-claude mock), which returns nominal data only.

Diagnose ​

sql
SELECT mapping_confidence, count(*)
  FROM fs_trial_balance_items
  WHERE engagement_id = '<engagementId>'
  GROUP BY mapping_confidence;

A healthy distribution: > 70% HIGH, 15–25% MEDIUM, < 5% LOW. Anything > 20% UNCLASSIFIED is a red flag.

sql
SELECT account_code, account_description
  FROM fs_trial_balance_items
  WHERE engagement_id = '<engagementId>' AND taxonomy_key IS NULL
  LIMIT 20;
-- check if descriptions are present and meaningful

Fix ​

  • Sign-flip suspected. Re-upload the TB with the convention corrected at source. The engagement returns to DRAFT.
  • Missing descriptions. Ask the client for a TB export that includes account descriptions.
  • Worker on mock adapter. Check pnpm dev worker logs for the adapter line. Real adapter requires ANTHROPIC_API_KEY env var.
  • Genuine client-specific TB. Manually override the UNCLASSIFIED rows; the override is preserved on subsequent auto-suggest re-runs. Consider raising a Phase 4 per-client synonym overlay if the same client recurs.

When in doubt ​

Re-run auto-suggest with the TB sample of one well-behaved engagement (e.g. AURORA seed). A run that lands > 70% HIGH there confirms the worker + adapter are healthy and the problem is the input.

Internal use only — BreezyCorp