Run the Guardrails Server

Prerequisites

Install the server extra, which adds FastAPI and uvicorn:

pip install --extra-index-url https://downloads.mend.io/guardrails/ 'mend-guardrails[server]'

You also need a Mend licence key. Set it as an environment variable:

export MEND_KEY="your-mend-key"

Loading policies

The server supports two policy sources. Choose the one that fits your deployment.

Mend Platform API

Pass --policy-source api to have the server fetch its guardrail policy directly from the Mend Platform on startup. No local policy files are needed — the same policy that the SDK loads in online mode is applied to every request.

In this mode guardrails.config_id in the request body is accepted but ignored, and GET /v1/guardrails/configs returns an empty list.

mend-guardrails-server \
  --policy-source api \
  --name "production-proxy" \
  --host 0.0.0.0 \
  --port 8000

Local Policies

Create a directory that contains your policy files. Each file becomes an independently addressable guardrail configuration.

policies/
├── default.json       # config_id: "default"
├── strict.json        # config_id: "strict"
└── permissive.json    # config_id: "permissive"

A minimal policy file uses the same JSON format as the inline config dict accepted by MendGuardrailsClient:

{
  "version": 1,
  "pre_flight": {
    "version": 1,
    "guardrails": [
      {
        "name": "PII",
        "config": { "entities": ["EMAIL_ADDRESS", "PHONE_NUMBER"], "block": false }
      }
    ]
  },
  "input": {
    "version": 1,
    "guardrails": [
      { "name": "PromptInjection", "config": { "confidence_threshold": 0.5 } },
      { "name": "HarmfulContent" }
    ]
  },
  "output": {
    "version": 1,
    "guardrails": [
      { "name": "HarmfulContent" }
    ]
  }
}

Alternatively, you can use subdirectories — a subdirectory named strict that contains a policy.json file is equivalent to strict.json at the top level:

policies/
└── strict/
    └── policy.json    # config_id: "strict"

Start the server

Guardrails API server can be started using different methods:

CLI

mend-guardrails-server \
  --name "production-proxy" \
  --host 0.0.0.0 \
  --port 8000

CLI command for loading local policies:

mend-guardrails-server \
  --policy-dir ./policies \
  --default-config default \
  --name "production-proxy" \
  --host 0.0.0.0 \
  --port 8000

Docker

A minimal Dockerfile for the server:

FROM python:3.12-slim

WORKDIR /app

RUN pip install --extra-index-url https://downloads.mend.io/guardrails/ \
    "mend-guardrails[server]"

EXPOSE 8000

CMD ["mend-guardrails-server", "--host", "0.0.0.0", "--port", "8000"]

Dockerfile for loading local policies (local mode)

FROM python:3.12-slim

WORKDIR /app

RUN pip install --extra-index-url https://downloads.mend.io/guardrails/ \
    "mend-guardrails[server]"

COPY policies/ ./policies/

ENV MEND_GUARDRAILS_POLICY_DIR=/app/policies
ENV MEND_GUARDRAILS_DEFAULT_CONFIG_ID=default

EXPOSE 8000

CMD ["mend-guardrails-server", "--host", "0.0.0.0", "--port", "8000"]

Build and run:

docker build -t mend-guardrails-server .

docker run \
  -e MEND_KEY="$MEND_KEY" \
  -e OPENAI_API_KEY="$OPENAI_API_KEY" \
  -e MEND_GUARDRAILS_INSTANCE_NAME="production-proxy" \
  -p 8000:8000 \
  mend-guardrails-server

Uvicorn directly

MEND_GUARDRAILS_POLICY_DIR=./policies \
MEND_GUARDRAILS_DEFAULT_CONFIG_ID=default \
MEND_GUARDRAILS_INSTANCE_NAME="production-proxy" \
uvicorn mendguardrails.server.api:app --host 0.0.0.0 --port 8000

All CLI options

Option	Default	Description
`--host`	`0.0.0.0`	Network interface to bind.
`--port`	`8000`	TCP port to listen on.
`--policy-source {local,api}`	`local`	Policy source. `local` reads files from `MEND_GUARDRAILS_POLICY_DIR`. `api` fetches from the Mend Platform. Sets `MEND_GUARDRAILS_POLICY_SOURCE`.
`--policy-dir PATH`	`./policies`	Directory containing policy files. Sets `MEND_GUARDRAILS_POLICY_DIR`. Only used when `--policy-source=local`.
`--default-config ID`	(none)	Policy ID used when a request omits `guardrails.config_id`. Sets `MEND_GUARDRAILS_DEFAULT_CONFIG_ID`. Only used when `--policy-source=local`.
`--models-config PATH`	(none)	Path to a JSON file mapping model names to upstream provider settings. Sets `MEND_GUARDRAILS_MODELS_CONFIG`. See Multi-model configuration.
`--name INSTANCE_NAME`	(none)	Human-readable name for this server process, shown in the Mend dashboard. Sets `MEND_GUARDRAILS_INSTANCE_NAME`.
`--reload`	`false`	Enable uvicorn hot-reload. Development only.
`--log-level`	`info`	Uvicorn log level: `debug`, `info`, `warning`, `error`.

Configuring the upstream provider

Upstream provider configuration — choosing a provider (OpenAI, Azure OpenAI, OpenAI-compatible proxies, local Ollama), forwarding HTTP headers and query parameters, and multi-model routing (including per-entry model overrides) — has moved to its own page:

➡️ Configure the upstream provider

Instance registration

On startup, the server registers itself with the Mend platform once, unless MEND_GUARDRAILS_OFFLINE is true — the same way an SDK client does when you pass name= to MendGuardrailsAsyncOpenAI.

Use --name (or MEND_GUARDRAILS_INSTANCE_NAME) to give the server a recognisable name in the Mend dashboard:

mend-guardrails-server --name "eu-west-proxy" --policy-dir ./policies

If --name is omitted the server registers as "MendGuardrailsServer". Registration is a best-effort call — a network failure only logs a warning and does not prevent the server from starting.

Environment variables

All configuration can be supplied via environment variables, making the server container-friendly without any CLI flags.

Variable	Required	Description
`MEND_KEY`	Yes	Mend licence key.
`MEND_GUARDRAILS_OFFLINE`	No	Set to `true` to skip remote policy fetch and instance registration.
`MEND_GUARDRAILS_POLICY_SOURCE`	No	`local` (default) or `api`. Controls where guardrail policies are loaded from.
`MEND_GUARDRAILS_POLICY_DIR`	No	Directory containing policy files. Defaults to `./policies`. Only used when `MEND_GUARDRAILS_POLICY_SOURCE=local`.
`MEND_GUARDRAILS_DEFAULT_CONFIG_ID`	No	Default policy ID when a request omits `guardrails.config_id`. Only used when `MEND_GUARDRAILS_POLICY_SOURCE=local`.
`MEND_GUARDRAILS_MODELS_CONFIG`	No	Path to a JSON file with per-model upstream provider settings. See Multi-model configuration.
`MEND_GUARDRAILS_INSTANCE_NAME`	No	Human-readable name for this server process, shown in the Mend dashboard. Set via `--name`.
`MEND_GUARDRAILS_UPSTREAM_HEADERS`	No	JSON object of header name→value pairs added to every upstream request (e.g. `{"x-tenant":"acme"}`).
`MEND_GUARDRAILS_UPSTREAM_QUERY`	No	JSON object of query param→value pairs added to every upstream request (e.g. `{"api-version":"2025-01-01"}`).
`MEND_GUARDRAILS_FORWARD_HEADERS`	No	Comma-separated header names to forward from the incoming request to upstream (e.g. `x-my-header,x-tenant-id`).
`MEND_GUARDRAILS_FORWARD_QUERY_PARAMS`	No	Comma-separated query param names to forward from the incoming request URL to upstream (e.g. `api-version,region`).
`OPENAI_API_KEY`	Yes (for OpenAI upstream)	API key sent to the upstream LLM provider. Can alternatively be forwarded per-request from the calling client's `Authorization` header.
`OPENAI_BASE_URL`	No	URL of the upstream LLM provider the server proxies to. Defaults to `https://api.openai.com/v1`. Set this on the server — not to be confused with the `base_url` clients set to point at this server.
`AZURE_OPENAI_ENDPOINT`	No	Overrides `azure_endpoint` for any model with `provider=azure` in the models config file.
`AZURE_OPENAI_API_KEY`	No	Overrides `api_key` for any model with `provider=azure`. Falls back to `OPENAI_API_KEY`.
`AZURE_OPENAI_DEPLOYMENT`	No	Overrides `azure_deployment` for any model with `provider=azure`.
`AZURE_OPENAI_API_VERSION`	No	Overrides `api_version` for any model with `provider=azure`.

API endpoints

`GET /health`

Returns {"status": "ok"} with HTTP 200. Use this for liveness probes.

curl http://localhost:8000/health

`GET /v1/guardrails/configs`

Lists the guardrail policy IDs available on the server (local policy-source mode only). Returns an empty list in api policy-source mode.

curl http://localhost:8000/v1/guardrails/configs

[{"id": "default"}, {"id": "strict"}]

`GET /v1/models`

Returns the models defined in the models config file in OpenAI-compatible format. Returns an empty data list when no MEND_GUARDRAILS_MODELS_CONFIG is set — no error is raised.

The response is compatible with the OpenAI Python SDK's client.models.list(). The internal default-fallback entry ("default": true) is excluded from the list; only named entries are returned.

curl http://localhost:8000/v1/models

{
  "object": "list",
  "data": [
    {"id": "gpt-4o",      "object": "model", "created": 0, "owned_by": "mend-guardrails"},
    {"id": "gpt-4o-mini", "object": "model", "created": 0, "owned_by": "mend-guardrails"}
  ]
}

`POST /v1/chat/completions`

The guardrail-protected Chat Completions endpoint. See Chat completions for request/response details.

`POST /v1/responses`

The guardrail-protected Responses API endpoint. Accepts the standard OpenAI POST /v1/responses body with an optional guardrails block (same as Chat Completions). Returns an OpenAI-compatible Response object. Supports instructions, tools, and multi-turn via previous_response_id.

All upstream configuration — model routing, static header/query injection, per-request forwarding — works identically to Chat Completions.

See Responses API for the full request/response reference and examples.

Verify the server is running

Once the server starts you should see output similar to:

INFO:     Started server process [12345]
INFO:     Waiting for application startup.
INFO:     mendguardrails.server.api - Server instance registered with Mend platform (name='production-proxy', guardrail_id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

The registration line only appears when MEND_KEY is set and MEND_GUARDRAILS_OFFLINE is not true. If registration is skipped or fails, a DEBUG or WARNING log is emitted instead and the server starts normally.

Check the health endpoint:

curl http://localhost:8000/health

{"status": "ok"}

List your loaded policies:

curl http://localhost:8000/v1/guardrails/configs

[
  {"id": "default"},
  {"id": "strict"},
  {"id": "permissive"}
]