LLM gateway: capability-based model routing.

Modality axis

Reasoning axis

Domain axis

Deployment axis

Example 1 — billing-coding-suggest

Suggest ICD-10/EBM/GOZ billing codes. Input must be anonymized — the AnonymizedPrompt type enforces this at compile time. Output goes to an approval queue before reaching the billing system.

import { createApp, AnonymizedPrompt } from '@polaris/llm-gateway'
import type { AnonymizedInput, ModelConfig, LlmProvider, GatewayConfig } from '@polaris/llm-gateway'

// 1. Configure models (provided by ops/platform team)
const models: ModelConfig[] = [
  {
    id: 'anthropic/claude-sonnet-4-5',
    provider: 'anthropic',
    modelName: 'claude-sonnet-4-5',
    capabilities: ['text', 'reasoning', 'medicalCoding', 'germanLanguage', 'medicalGermanLanguage', 'jsonMode'],
  },
]

// 2. Wire in your LLM provider (LiteLLM, Anthropic SDK, etc.)
const llmProvider: LlmProvider = async (model, request) => {
  // Forward to your actual provider implementation
  throw new Error('Replace with real provider implementation')
}

// 3. Create the gateway app
const config: GatewayConfig = {
  models,
  intentCatalogPath: './packages/llm-gateway/intent-catalog.yaml',
}
const app = createApp({ config, llmProvider })

// 4. Build an anonymized prompt (type-safe — no raw strings accepted)
const anonymizedInput: AnonymizedInput = {
  anonymizedText: 'Patient klagt über anhaltende Rückenschmerzen seit 3 Wochen, keine Ausstrahlung.',
}
const prompt = AnonymizedPrompt.from(anonymizedInput)

// 5. POST to the chat completions endpoint
const response = await app.fetch(
  new Request('http://localhost/v1/chat/completions', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      messages: [{ role: 'user', content: prompt.text }],
      gateway: {
        requires: ['text', 'medicalCoding', 'germanLanguage'],
        prefers: ['reasoning', 'medicalGermanLanguage'],
        intent: 'billing-coding-suggest',
        pii: 'anonymized',
      },
    }),
  }),
)
const result = await response.json()
// result.choices[0].message.content — JSON billing code suggestions
// Must be routed to physician approval before billing submission

Example 2 — scribe-stt-summarize

Transcribe and summarize spoken consultation notes. Audio must be processed locally or via a DSGVO-compliant processor. Output goes to an approval queue before any chart write.

// scribe-stt-summarize: audio transcription + summarization
// Note: audioIn capability requires the llmProvider to handle audio content.
// The prompt text here represents the transcribed audio content.

const app = createApp({
  config: {
    models: [
      {
        id: 'anthropic/claude-opus-4-5',
        provider: 'anthropic',
        modelName: 'claude-opus-4-5',
        capabilities: ['text', 'audioIn', 'germanLanguage', 'medicalGermanLanguage', 'streaming'],
      },
    ],
    intentCatalogPath: './packages/llm-gateway/intent-catalog.yaml',
  },
  llmProvider,
})

const transcribedNote = AnonymizedPrompt.from({
  anonymizedText: 'Anamnese: Keine Vorerkrankungen. Aktuell Husten seit 5 Tagen, leichtes Fieber 37.8°C.',
})

const response = await app.fetch(
  new Request('http://localhost/v1/chat/completions', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      messages: [
        { role: 'system', content: 'Erstelle eine strukturierte Dokumentation aus den folgenden Konsultationsnotizen.' },
        { role: 'user', content: transcribedNote.text },
      ],
      gateway: {
        requires: ['text', 'audioIn', 'germanLanguage'],
        prefers: ['medicalGermanLanguage'],
        intent: 'scribe-stt-summarize',
        pii: 'anonymized',
      },
    }),
  }),
)
// response includes X-Approval-Required: true header — must go to approval queue
const approvalRequired = response.headers.get('X-Approval-Required') === 'true'