Jev Scores High-Volume Telemetry
Jev filters 40 billion data points every 15 minutes, scoring results and deciding what the system investigates next.
View on X time-24ms
Alex Stone@a1exstone𝕏
JEV FILTERS 40 BILLION DATA POINTS EVERY 15 MINUTES. AN LLM ON EVERY RESULT COULDN'T DO IT a research team pushes that volume through their pipeline. the first version put an LLM on every result. it worked, and it was unusable: millions of explanations nobody read, latency stacking at every stage, inference bills growing with the dataset. so they moved the LLM out of the hot path and put a judge in it. jev doesn't explain the data. it scores it and decides what the system investigates next. this is the telemetry from that redesign. p95 latency before and after, stage by stage: planner -24ms, r
The first version put an LLM on every result, producing millions of explanations nobody read while latency and inference bills grew with the dataset. The team moved the LLM out of the hot path and put a judge in its place.
Also filed under Triage & routing
- How to Build Things with Jev & OpenJevs
How to Build Things with Jev & OpenJevs In this video, we build a model router using both the API-based original Jev and also using Semif. Github: code will be up in ...
- Jev is HERE. How to use it
Jev is HERE. How to use it In this episode, I talk with Ryan Vogel about Jev, a new type of AI built for classification. Ryan shows how Jev takes an input plus ...
- Jev-Shaped Classification Wrapper
jevper is a Jev-shaped classification wrapper over OpenAI-compatible clients that returns probabilities and confidence.
- Local Jev Protocol Multi-Upstream Router
jev-switch is a local multi-upstream router for the Jev protocol with a Rust backend and React interface.