Ollama now supports decision models, based on TypeSafe’s Jev API for fast, typed decisions:
- No additional costs
- Lower latency when run locally
- Three new decision models available today via Ollama
This new API is available as of Ollama 0.35 by using the new /v1/systemone endpoint. Send text as state with a set of named questions, and a model running on your machine answers them all in one request. This is great for tasks that require fast decisions, such as ticket triage, model routing, and content or safety moderation.
Three new decision models are available to run via Ollama:
- nimble: open-source 9B parameter decision model developed by Bespoke Labs
- tev1: an experimental 4B decision model from Together AI
- tev1:0.8b: an experimental 0.8B decision model from Together AI
More decision models are coming soon, including models served by Ollama’s cloud.
See the official website.
