Configure the Router

• Configure the Router
  • Start with the defaults
  • Starting and stopping
  • Where configuration lives
  • Server
  • Storage
  • Authentication
  • Runtime limits
  • Rate limiting
  • Logging
  • OpenTelemetry
  • CORS
  • Retention
  • Providers
    • Generic OpenAI-compatible endpoints
  • Local models with Ollama
  • A complete example
  • Validation

smista-router is the routing and orchestration service the CLI talks to. This page covers how the router process itself runs — the server binding, storage, authentication, runtime limits and logging. It is separate from the routing policy (which model handles a task); that lives in Configure Routing and the CLI.

Start with the defaults

Most local users do not need to create router.toml. Without the file, the router uses safe local defaults: it binds to 127.0.0.1:7331, stores data in an embedded database under the global smista directory, and disables CORS and telemetry.

Create router.toml when you need to enable Ollama, change a runtime limit, use remote storage, or connect to a custom provider endpoint:

smista config init router

Starting and stopping

smista.ai ships as a single binary, so the router runs inside the smista process rather than as a separate program. Start a local router with the CLI:

smista start

By default this daemonizes: the router starts in the background and the command returns. It records its process id in a pidfile, and refuses to start a second router on top of one that is already running. Stop it again with:

smista stop

Pass --foreground to run the router in the current process instead, which is what a service manager (systemd, launchd) wants:

smista start --foreground

The pidfile defaults to a per-user location under the runtime directory. Set a different path with --pidfile <path> on both start and stop, or with the SMISTA_ROUTER_PIDFILE environment variable.

Check that the router is ready with:

smista status

Where configuration lives

Use smista config path router to print the location on your system. Pass --config <path> to smista start when you want to use another file.

The router configuration is stored in the global smista directory, but smista config commands access it with the router argument:

smista config path router
smista config edit router
smista config show router
smista config check router

The global argument selects the global CLI configuration (config.toml), not the router configuration (router.toml).

An invalid router configuration prevents the router from starting and reports which field to fix. Run smista config check router before starting to check the file directly.

Server

[router]
host = "127.0.0.1"
port = 7331

The router is meant to be reached locally. Binding it to a public interface in local mode is flagged as unsafe by validation.

The [router] table accepts:

Storage

Where users, sessions, tokens, traces and execution metadata are persisted. smista.ai uses SurrealDB, which supports both an embedded local database and a remote server for future deployments.

[router.storage]
engine = "surrealdb"
mode = "embedded"
path = ".smista/db"
namespace = "smista"
database = "local"

The [router.storage] table accepts:

Note

username and password authenticate against a remote SurrealDB server and are only used in remote mode. The password is treated as a secret: it is never logged and is never written back out when configuration is serialized.

Authentication

Controls API-key bootstrap and the lifetime of short-lived session tokens.

[router.auth]
token_ttl_seconds = 86400
api_key_version = "01"
local_bootstrap_enabled = true

local_bootstrap_enabled lets the router mint a user and API key locally without a SaaS account. It must be disabled in remote/SaaS mode.

The [router.auth] table accepts:

Runtime limits

Protect the router from oversized requests, runaway tools, excessive context and hanging provider calls.

[router.limits]
max_request_body_bytes = 10485760
max_context_bytes = 5242880
max_concurrent_requests = 8
request_timeout_ms = 120000
provider_timeout_ms = 180000
tool_timeout_ms = 60000

The [router.limits] table accepts:

Rate limiting

Caps how fast a single client can hit the router, so a runaway script or a misbehaving client cannot overwhelm it. Limiting is applied per client IP address using a token bucket: each client may send a burst of up to burst_size requests, and its allowance refills by one request every period_ms milliseconds. Requests over the limit get a 429 Too Many Requests response.

Enabled by default with limits sized for local use — roughly 100 requests per second per client with room for a burst of 200. Raise the limits if you drive the router hard from scripts, or set enabled = false to turn it off entirely.

[router.rate_limit]
enabled = true
period_ms = 10
burst_size = 200
trust_proxy_headers = false

The [router.rate_limit] table accepts:

The sustained rate is one request every period_ms milliseconds — so the default of 10 allows about 100 requests per second per client. When rate limiting is enabled, both period_ms and burst_size must be greater than zero; validation rejects a zero value.

By default each client is identified by the source address of its connection. When the router runs behind a reverse proxy, every request appears to come from the proxy, so they would all share one limit. Set trust_proxy_headers = true to identify clients by the X-Forwarded-For, X-Real-IP or Forwarded header the proxy sets instead.

Warning

Only enable trust_proxy_headers when a trusted reverse proxy sits in front of the router and sets these headers itself. With clients connecting directly, anyone can put any value in the header and hand themselves a fresh limit on every request, which defeats rate limiting entirely.

Logging

[router.logging]
level = "info"
format = "compact"
redact_secrets = true

Keep redact_secrets = true. API keys, provider credentials and auth tokens are never written to logs or traces.

The [router.logging] table accepts:

OpenTelemetry

The router can export its traces to an OpenTelemetry collector you already run, so you can watch timings and errors in your own dashboards. This is layered on top of the existing logging and changes nothing about how the router behaves or what it decides. It is disabled by default, and when disabled the router exports nothing and does no extra work.

Only span and trace metadata is exported. Secrets are never recorded on traces, so API keys, provider credentials and auth tokens are never sent to the collector.

Turn it on and point it at your collector:

[router.opentelemetry]
enabled = true
endpoint = "http://localhost:4317"
protocol = "grpc"
service_name = "smista-router"
sample_ratio = 1.0

The [router.opentelemetry] table accepts:

You can also set the most common options when starting the router, without editing the file. A value given on the command line wins over the file:

smista start \
  --otel \
  --otel-endpoint http://localhost:4317 \
  --otel-protocol grpc \
  --otel-service-name smista-router \
  --otel-sample-ratio 1.0

--otel turns export on and --no-otel turns it off, each overriding the file. The remaining flags map one to one to the table above.

CORS

Disabled by default. Only needed for browser-based clients or a future web dashboard.

[router.cors]
enabled = true
allowed_origins = ["https://app.smista.ai"]

Warning

Never enable CORS with unrestricted origins in production.

The [router.cors] table accepts:

Retention

[router.retention]
trace_retention_days = 90
session_retention_days = 365
archived_session_retention_days = 30
cleanup_interval_seconds = 3600

The [router.retention] table accepts:

Providers

The router discovers built-in providers without endpoint configuration. OpenAI, Anthropic, and Gemini always use their official endpoints.

Use [router.providers] for named OpenAI-compatible services, such as a local vLLM server or a hosted gateway. Configure a local Ollama connection under [router.ollama] instead.

[router.providers."openai-compat:my-vllm"]
base_url = "http://localhost:8000/v1"
local = true
display_name = "My vLLM"

Each [router.providers.<id>] table accepts:

Generic OpenAI-compatible endpoints

You can point the router at any number of OpenAI-compatible endpoints — a local vLLM or LM Studio server, llama.cpp, or a hosted gateway. Each one is a named instance: you choose a name, and routing rules address it as openai-compat:<name>/<model> (for example openai-compat:my-vllm/llama-3.1-70b).

Configure each instance under its full identity as the table key. The name is quoted because it contains a colon. Each model is its own [[...models]] table:

[router.providers."openai-compat:my-vllm"]
base_url = "http://localhost:8000/v1"
local = true
display_name = "My vLLM"

[[router.providers."openai-compat:my-vllm".models]]
name = "llama-3.1-70b"
max_context_tokens = 131072
max_output_tokens = 8192

[router.providers."openai-compat:lmstudio"]
base_url = "http://localhost:1234/v1"

[[router.providers."openai-compat:lmstudio".models]]
name = "qwen2.5-coder-7b"
max_context_tokens = 32768
input_cost_per_million_tokens = "0.0"
output_cost_per_million_tokens = "0.0"

These endpoints expose no catalog of model facts, so routing reads each model’s facts from the [[...models]] tables you declare. The credential, if the endpoint needs one, is set on the CLI side — see the openai-compat:<name> provider in Configure Routing and the CLI.

Each [[router.providers.<id>.models]] table accepts:

The capabilities table defaults to streaming, tools, json_output, system_prompt and memory enabled, with images and reasoning disabled. Override any of them per model:

[[router.providers."openai-compat:my-vllm".models]]
name = "llava-1.6"
max_context_tokens = 32768

[router.providers."openai-compat:my-vllm".models.capabilities]
images = true

Note

The built-in providers (OpenAI, Anthropic, Gemini) and Ollama discover their model facts at runtime and ignore the models tables. Only generic openai-compat:<name> endpoints, which publish no catalog, read facts from here.

Note

Ollama’s endpoint and model discovery are configured separately, under [router.ollama] — see the next section.

Local models with Ollama

The router connects to Ollama to run local models. Connection and discovery are configured here under [router.ollama]; which tasks actually use an Ollama model is decided by your routing policy in Configure Routing and the CLI. The two must stay consistent — see Use Local Models with Ollama for the full setup.

[router.ollama]
enabled = true
base_url = "http://127.0.0.1:11434"

The [router.ollama] table accepts:

A complete example

The example below shows every main runtime section. Most local users only need the sections that differ from the defaults.

[router]
host = "127.0.0.1"
port = 7331

[router.storage]
engine = "surrealdb"
mode = "embedded"
path = ".smista/db"
namespace = "smista"
database = "local"

[router.auth]
token_ttl_seconds = 86400
api_key_version = "01"
local_bootstrap_enabled = true

[router.limits]
max_request_body_bytes = 10485760
max_context_bytes = 5242880
max_concurrent_requests = 8
request_timeout_ms = 120000
provider_timeout_ms = 180000
tool_timeout_ms = 60000

[router.rate_limit]
enabled = true
period_ms = 10
burst_size = 200
trust_proxy_headers = false

[router.logging]
level = "info"
format = "compact"
redact_secrets = true

[router.opentelemetry]
enabled = false
endpoint = "http://localhost:4317"
protocol = "grpc"
service_name = "smista-router"
sample_ratio = 1.0

[router.cors]
enabled = false
allowed_origins = []

[router.retention]
trace_retention_days = 90
session_retention_days = 365
archived_session_retention_days = 30
cleanup_interval_seconds = 3600

[router.ollama]
enabled = false
base_url = "http://127.0.0.1:11434"

Validation

Router configuration is validated at startup. Validation rejects an invalid host or port, an unsafe public binding in local mode, missing or unsupported storage configuration, local bootstrap enabled in remote mode, invalid timeouts or size limits, a zero rate-limit period or burst while rate limiting is enabled, unsafe CORS, and invalid OpenTelemetry settings. A failure prevents startup and explains the offending field.

Validation also emits non-blocking warnings, such as setting a base_url on a built-in API provider that ignores it. Warnings are surfaced but do not prevent startup.