Skip to content
All articles

When a local model fails, the route should stay explicit

Review the model destination and shared context before changing from a local route to a remote service.

Ouros editorial team

When a local model stops responding, choosing a remote model is a new decision. It can change where your code goes and which credentials are used. A failure message should lead to a visible choice, not an assumption that every retry has the same privacy boundary.

Separate the editor from the model destination

A local editor window tells you where you type. It does not tell you where a model runs. A model request can include the selected text, nearby code, attached files or other context. Review the destination and the context together.

Mandarin's maintained configuration describes a local provider and a separately chosen model address. This is a settings reference, not a shell script, a complete launch procedure or a promise that an arbitrary settings file will be loaded:

MANDARIN_AGENT_PROVIDER=local
MANDARIN_LOCAL_URL=http://127.0.0.1:8080
MANDARIN_LOCAL_MODEL=SERVER_REPORTED_MODEL_ID

SERVER_REPORTED_MODEL_ID means the identifier supplied by the model server, not a model name to guess. A loopback address points at the same computer from the application's point of view. It does not establish what that server does next. A local proxy could still forward requests elsewhere.

Use a route-and-failure matrix

ChoiceWhat to confirm before a requestWhat to do after failure
Local model serverAddress, model identifier, server behavior and selected contextPreserve the draft; check the server and configuration without switching destinations
Remote model serviceProvider, account access, credentials and permission to share the contextRead the error; decide whether to retry that same destination
Change from local to remoteEverything above, plus the change in where the text is sentMake the change explicit before sending another request

For example, a developer wants help with a confidential parser. The local server is unavailable. The useful next step is to retain the question and inspect the local configuration. Re-sending the same question through a remote account is not a neutral repair. It needs a separate sharing decision, even if the remote account is already signed in.

Keep context review independent of connection repair

A connection failure can distract you from the contents of the request. Before trying again, check which files are attached and whether the task needs them. A short public sample may answer the question without exposing the project. Remove credentials and private records; do not rely on a model to recognize and withhold them after transmission.

OWASP's prompt-injection guidance adds another reason to keep control of the request. Instructions found in source files or retrieved pages should not select a new provider, expand the context or authorize an upload. They are task material, not permission from you.

When reporting a failure, retain the selected destination, the action and the non-secret error. Leave out keys and private file contents. Ask what a retry will do before repeating it. The practical goal is a route you can explain: what text leaves, where it goes and who approved that choice.

Sources

Put it to use