devwithjev
— reading now— views
Submit a build

Jev Spam and Intent Classification

VerySmallWoods tested Jev on SMS spam detection and Banking77 intent classification using 662 examples from two Kaggle datasets. Jev achieved 96.5% accuracy on SMS and 81.6% on Banking77.

View on X
VerySmallWoods@verysmallwoods𝕏
Jev 到底准不准？我拿两个 Kaggle 数据集测了 662 次，结果有点出人意料。 我用 kaggle 上的数据集： - SMS Spam Collection：5,574 条英文短信，抽取 200 条，当 smoke test - Banking77：13,000 条真实的银行 App 提问、77 个意图选项，抽取其中 462 条 一共做了 662 次 Jev 调用。同样的数据和问题再交给 Claude Haiku 4.5 做对照。同时约定 Haiku 同样输出 confidence 值。 测出来的结果如下： - 准确率：短信场景 Jev 96.5% / Haiku 93.5%；77 选 1 场景 Jev 81.6% / Haiku 80.1%，微调过的 BERT-Large 93.7% - 概率校准：报 50% 到 85% 时基本都准；报 90% 以上偏乐观，报 100% 的时候，
Sep 21, 2026X postsView on X
The author compared Jev with Claude Haiku 4.5 on the same data and questions, asking both models to provide confidence values. Haiku scored 93.5% on SMS and 80.1% on Banking77; fine-tuned BERT-Large scored 93.7% on Banking77. The author reported that Jev’s confidence estimates were broadly accurate from 50% to 85%, but optimistic above 90%.

Also filed under Triage & routing

  • Jev-Powered Self-Healing for Pistachio

    Pistachio uses Jev to triage logged errors and categorize them as GitHub issues as part of its self-healing error and bug-handling workflow.

  • Identifies Canadian Federal Tax Documents

    Emile Riberdy built a demo using Jev in Avalanche to identify Canadian federal tax documents. It takes ~1 second and costs less than 1 cent per document.

  • How to Build Things with Jev & OpenJevs

    How to Build Things with Jev & OpenJevs In this video, we build a model router using both the API-based original Jev and also using Semif. ‍ Github: code will be up in ...

  • Jev is HERE. How to use it

    Jev is HERE. How to use it In this episode, I talk with Ryan Vogel about Jev, a new type of AI built for classification. Ryan shows how Jev takes an input plus ...