JEval Metric Discovery and Extraction

JEval is an open-source tool that discovers and recommends metrics, then displays them across various viewers. It includes a metric-extraction library and CLI, and has been converted into a skill for AI agents.

View on X

HOPE | Engineer.@rlaope𝕏

Here is a useful tip for anyone using JEV. Once you extract probability estimates using JEV, you need to perform calibration. But what should be the criteria for this calibration, and how accurate are JEV's predicted probabilities in reality? More importantly, how should we adjust our business logic based on the values generated by JEV? JEval is an open-source tool that helps you discover metrics, recommends them, and displays them across various viewers. It also provides a metric-extraction library and a CLI interface. I've even converted it into a skill so it can be fed into AI agents. JE

Sep 22, 2026X postsView on X

The author advises calibrating probability estimates extracted using JEV. They raise questions about calibration criteria, prediction accuracy, and how to adjust business logic based on JEV’s outputs.

Also filed under Tools & apps

Jev: a "System One" model that only makes decisions, $0.042 per million tokens with output free. 40-second explainer of why it matters for automation
Jev: a "System One" model that only makes decisions, $0.042 per million tokens with output free. 40-second explainer of why it matters for automation Jev launched on Sep 15 from TypeSafe AI (founder Diogo Almeida, ex-OpenAI, credited on ChatGPT/InstructGPT). It does not generate text; it returns typed decisions (Choice / Score / true-false) with confidence. Price is $0.042 per million input tokens, output free. In its first week it became the fastest-adopted model on Vercel's AI Gateway, TypeSafe paused signups on Sep 22, Browser Use released jev-ultrafast (a web agent where Jev picks each action; Google Flights search in 7.1 s), and jaredpalmer/kev is an Apache-2.0 clone built on Qwen3.5 you can run yourself. My take in the video: the expensive part of automation was never the writing, i
Untitled
There's a new AI model that refuses to talk. While everyone debates what a "decision model" is even for, builders already shipped the answer. 7 things people built with Jev in 9 days. Number 5 broke my brain 👇
Untitled
I loved this idea so much that it deserves a share ❤️ I recently asked people to share what they’ve built with JEV, and then this guy 👤 shows up with an entire website dedicated to different JEV use cases! That’s honestly pretty cool. If you’re curious about JEV and want to see how it can actually be applied, definitely check this out. Huge shoutout to Animesh Mishra (@thatcasualvc) for the wonderful share 🙌 This is exactly the kind of stuff I love seeing from builders. 🚀
I built a Jev-based model router for Pi for automatic model selection
I built a Jev-based model router for Pi for automatic model selection When Jev came out, one of my first thoughts was: could something this fast and cheap pick which model should handle an agent’s next turn? So I built plugin for Pi that uses Jev to choose the model and thinking effort. The idea is to send routine work to cheaper models and reserve the expensive ones for harder tasks. In Pi, the model mappings are configurable, and you can pin a model when you disagree with the router. I’ve been using them for a few days, and so far they’ve been working well for me. In my own evaluation, routed usage came to $141 at list prices versus an estimated $166 for Opus 5.5 on every request - about 15% less. That’s one developer’s data, though. The baseline is a pricing comparison, not a separate