94.42% on BANKING77 Official Test Split — New Strong 2nd Place with Lightweight Embedding + Rerank (no 7B LLM)

94.42% Accuracy on Banking77 Official Test Split

BANKING77-77 is deceptively hard: 77 fine-grained banking intents, noisy real-world queries, and significant class overlap.

I’m excited to share that I just hit 94.42% accuracy on the official PolyAI test split using a pure lightweight embedding + example reranking system built inside Seed AutoArch framework.

Key numbers:

Official test accuracy: 94.42%

Macro-F1: 0.9441

Inference: ~225 ms / ~68 MiB

Improvement: +0.59pp over the widely-cited 93.83% baseline

This puts the result in clear 2nd place on the public leaderboard, only 0.52pp behind the current absolute SOTA (94.94%).

No large language models, no 7B+ parameter monsters

just efficient embedding + rerank magic.

Results, and demo coming very soon on HF Space

Happy to answer questions about the high-level approach

#BANKING77 #IntentClassification #EfficientAI #SLM