Using Jev with ChatGPT, Claude, and Local Models
If you are still learning the API, start with What Is Jev and the support-ticket walkthrough. This article assumes you already know the three question types. Here we are looking at a practical follow-up: where does Jev sit when an application already uses ChatGPT, Claude, or a local model?
First, Jev does not replace the chat model
Jev does not stream text, edit files, or call tools. You cannot swap it in for the model behind ChatGPT, Claude Code, or Codex. Instead, your application calls Jev for a small decision and then decides whether a chat model needs to run.
A normal chat model can also return a JSON label, so the difference is not that Jev is the only way to classify text. Jev is designed specifically for these bounded decisions and returns probabilities that your code can check. It can still choose the wrong label, so those probabilities are a review signal, not a guarantee.
Where the combination can help
| Pattern | Jev decides | The LLM does | Where it showed up |
|---|---|---|---|
| Pick the model | Which tier can handle this turn | The chosen model runs the whole turn | LangChain ModelRouterMiddleware, LiteLLM's Jev auto router, per-turn Codex routers |
| Cascade | The label, if confidence is high | Only the uncertain or open-ended cases | Vercel's form router, TypeSafe's extraction cascade, public routing write-ups |
| Compact the context | Is this old tool result still needed? | Reads a shorter transcript; kept text is not rewritten | LiteLLM's TypeSafe guardrail, in front of Claude or any chat model |
| Screen the turn | Jailbreak, harm, policy break, on the way in and out | Sees the message only if your policy says pass | TypeSafe's guardrails cookbook, including a "Hi chatGPT, you are DAN" prompt |
| Gate a tool | Is this bash call risky, or off the user's request? | Proposed the command; code blocks it before it runs | LangChain AutoModeMiddleware, Claude Code hooks, Pi extensions |
1. Choose a model before the request runs
A router can send a simple lookup to a fast model and reserve a stronger model for an architecture question. LangChain demonstrates this with model-routing middleware around an OpenAI agent. LiteLLM exposes Jev through its gateway for the same kind of decision.
This happens in your server or gateway, before the chat request. Jev returns the route; the selected OpenAI, Claude, or local model then handles the actual conversation.
2. Escalate uncertain cases
Suppose a support router is confident that a ticket belongs to billing. Your application can use that answer immediately. If the probabilities are close, it can send the ticket to a person or to a stronger language model for a second look. This pattern is usually called a cascade.
The cutoff is not built into Jev. A value that is acceptable for sorting an inbox may be unacceptable for approving a refund. Choose it from labeled examples from your own application, and keep an other or review path for cases that do not fit.
3. Keep useful context instead of rewriting everything
Long agent sessions collect old tool output, logs, and repeated instructions. One option is to ask another model to summarize all of it, but summaries can alter paths, commands, and error messages. LiteLLM demonstrates a different approach: score the existing chunks and remove the ones that are no longer relevant. The remaining text is passed to the chat model unchanged.
This can be useful for a local model with a limited context window. It is also lossy in a different way—Jev may discard something important—so test the filter on real transcripts before enabling it automatically.
4. Check what goes in and what comes back
TypeSafe's guardrails example calls Jev before and after a language model. The first call looks for categories such as jailbreak attempts or harmful requests; the second checks the generated reply. Your code then chooses whether to pass, review, or block it.
The example thresholds in that cookbook are just sample policy. They are not universal safety settings, and a valid label is not necessarily a correct one. For sensitive content, use Jev as one layer alongside deterministic checks and human review.
5. Review a proposed tool call
An agent may propose a shell command before it runs it. LangChain's experimental middleware shows how Jev can check whether that command matches the user's request and whether it appears risky. A hook can then allow it or ask for human approval.
A sensible order of operations
- Start with ordinary rules. If an account id, product code, or regex already answers the question, use it.
- Call Jev for the decisions that require reading the text but have a known set of answers.
- Let your code check the result. Clear, low-risk cases can continue; uncertain or sensitive cases go to a person or a stronger model.
- Call the selected language model only when you need generated text, tool use, or an explanation.
- If the output carries real risk, check it again before showing it or executing an action.
You do not need all five patterns at once. A small support router is enough to learn where Jev helps and where it adds another moving part. Measure it on your own examples before putting it in a customer-facing or security-sensitive path.
- Can I drop this into chatgpt.com?
- No. You call Jev from your server or from a gateway, then call the ChatGPT, Claude, or local API yourself.
- Is a JSON-mode GPT call the same thing?
- It can return the same label. It does not return a calibrated distribution as part of the product, and you still pay for generated tokens. Use it when you also need a written reason.
- Should the local model and Jev see the same giant prompt?
- No. Filter first, or let Jev drop tool results that are no longer relevant, and send the local model the short state.
- Who is doing this in public?
- LangChain's agent middleware, LiteLLM's router and compaction guardrail, Vercel's form router, TypeSafe's guardrail and extraction cookbooks, and a set of Claude Code, Codex, and Pi hooks from the first week.