←── back to feed
/topics/decision-models-and-jev-benchmarking

Decision models and Jev benchmarking

4 items●2 sources●updated 35d ago●trend 0

Researchers evaluated Jev, a decision model from TypeSafe, on network traffic classification using packet metadata, achieving 28.42% accuracy with 40 labeled examples on the CESNET-QUICEXT-25 dataset. Jev costs $0.042 per million input tokens with 70–500ms latency, but benchmarks show it underperforms compared to Random Forest (69.95% accuracy) and competing decision models like GLiDE and GLiNER2.5-Decide.

  • Jev evaluated on 52,000 network flow records across 26 collection weeks using first ten packets' sizes, directions, inter-packet times
  • TypeSafe Jev pricing: $0.042 per million input tokens; response latency 70–500 milliseconds
  • With 40 labeled examples, Jev accuracy improved from 9.80% to 28.42% on ten application labels
  • Random Forest and Extra Trees trained on 8,000 records achieved 69.95% and higher accuracy respectively
  • Competitors include Fastino GLiDE, GLiNER2.5-Decide, and four open-source decision models
[HN]hacker news2
Decision models like Jev don't beat LLM-as-a-judge or traditional classifiers
HN: LLM · tomncooper · ▲28 · 36d
Clef: Open-weight decision models, and new RL fine-tuning platform
HN: fine-tuning · jasondavies · ▲543 · 37d
[BLG]blog/rss2
A First Glance at Jev for Network Traffic Classification: Accuracy, Processing Time, and Cost
arXiv cs.LG · Shenghe Xu, Lifan Mei · 36d
Decision AI Models Explained: TypeSafe Jev vs Fastino GLiDE, GLiNER2.5-Decide and Open-Source Competitors
MarkTechPost · Asif Razzaq · 36d