Skip to content

Guardrails API Server

The Mend Guardrails SDK ships with a built-in HTTP server that exposes an OpenAI-compatible REST API — the same /v1/chat/completions surface your application already calls — but with Mend guardrails applied to every request.

Client app                Guardrails Server              Upstream LLM
(OpenAI SDK)  ──POST──►  pre_flight + input rails  ──►  (OpenAI / Azure / …)
              ◄──JSON──  output rails applied        ◄──

The Chat Completions (POST /v1/chat/completions) and Responses (POST /v1/responses) APIs proxy to an upstream LLM. The Guard endpoints (POST /v1/guard/input and POST /v1/guard/output) validate arbitrary text with no upstream LLM required — designed for custom REST API agents.

Why use the server instead of the SDK?

Scenario Recommended approach
Your app is already calling OpenAI directly SDK (drop-in client, zero infra)
You want a language-agnostic guardrail layer Server (any HTTP client works)
You run multiple services, different stacks Server (one central guardrail proxy)
You can't modify application code Server (transparent HTTP proxy)
You need per-team policy isolation Server (multiple named policies)
Your agents use a custom (non-OpenAI) REST API Server — /v1/guard/input + /v1/guard/output

How it works

  1. The server registers with the Mend platform on startup. A single connection event is sent so the proxy appears in the Mend dashboard alongside SDK clients. Use --name (or MEND_GUARDRAILS_INSTANCE_NAME) to give it a recognisable label.

  2. Policies are loaded according to the configured policy source. The default (local) loads each *.json / *.yaml file in the policy directory as a named configuration. Setting --policy-source api fetches the guardrail policy directly from the Mend Platform — no local files needed.

  3. Clients are cached. The first request for a given config_id pays the model warm-up cost once. All subsequent requests reuse the cached client with zero additional overhead. In api mode a single shared client is created on the first request.

  4. The guardrail pipeline runs on every request. Pre-flight and input guardrails are checked before the upstream LLM is called. If either stage blocks the request the server returns HTTP 400 immediately — the upstream is never called. Output guardrails run on the LLM's response before it is returned to the caller.

  5. The response is standard OpenAI JSON. Any existing code that handles a ChatCompletion object continues to work without modification.

Server deployment options

Method Command
Console script mend-guardrails-server --policy-dir ./policies --name my-proxy
Uvicorn directly uvicorn mendguardrails.server.api:app --port 8000
Docker docker run … -e MEND_KEY=… -e MEND_GUARDRAILS_INSTANCE_NAME=my-proxy mend-guardrails-server

Next steps

  • Run the Server — installation, policy sources, environment variables, and Docker
  • Configure the Upstream Provider — OpenAI, Azure OpenAI, proxies, Ollama, header forwarding, and multi-model routing
  • Chat Completions — request format, enforcement responses, and examples
  • Responses API — Responses API request format, multi-turn, and examples
  • Streaming — SSE streaming, enforcement behaviour, and examples for both endpoints
  • Guard API — validate arbitrary text for custom REST API agents (no upstream LLM required)
  • List Configurations — discover and inspect loaded policies