devwithjevJev Grades Code-Taste Tasks
Yuhan Luo ran a toy pipeline comparing Jev and Codex Luna as graders on FrontierCode-style code-taste tasks.
Yuhan Luo@_yuhanluo๐
ran a toy pipeline comparing jev vs codex luna as graders on FrontierCode style "code taste" tasks: decomposed text-based rubrics into structured answers (e.g. are all changes necessary? do all changes consistently use the abstractions required by the task?) -> run both models in parallel to compare accuracy, latency, cost jev got similar accuracy at <5% cost and <3% latency of luna (tiny sample.
The pipeline converted text-based rubrics into structured questions about whether changes were necessary and consistently used the required abstractions, then ran both models in parallel. In a tiny sample, Jev had similar accuracy to Luna.
Also filed under Research & data
- Visualizes French Wikipedia Elites
The project is a data visualization of French elites on Wikipedia. Its crawl is ongoing, with new biographies arriving.
- GraphRAG with Swappable Laya and Jev Models
This project is an agentic GraphRAG pipeline with swappable local Laya or cloud Jev decision models.
- Jev Probability Calibration Experiments
The repository presents reproducible experiments on Jev probability calibration, uncertainty, and forecast preservation.
- Benchmarks Jev for Malicious Skill Detection
jev-skillbench benchmarks Jev as a malicious agent-skill detector on MalSkillBench with verify-and-escalate evaluation.