Model Access

Choose which AI models the agent may use, route them to Anthropic, OpenAI, Azure or AWS, keep provider keys off the sandbox, and cap token spend

Model access decides which AI models the agent may use and how much it may spend. By default the agent can use no model at all. Each request the agent makes is checked on the computer (is this model approved? is there budget left? must the user approve?) and only then sent to the provider. The key that pays for the request is added outside the sandbox, so the agent never holds it.

Model access is configured in the proxies.llm section of policy.

What does the agent see?

The agent sees an ordinary model service with placeholder keys. Inside the sandbox, the standard settings that AI tools read point at Levee:

The placeholder keys are useless outside the sandbox and change every session. The real provider key never enters the sandbox, so there is nothing for the agent to leak.

Levee offers the agent two protocols:

GET /v1/models returns the models the policy allows, so tools that list models only offer approved ones.

There is no translation between protocols. A request in the Anthropic protocol can only go to a provider that speaks it, and the same holds for OpenAI:

GitHub Copilot CLI talks to GitHub's own service; it is governed by Web Access rules for GitHub's sites, not by this section.

Section shape

{
  "proxies": {
    "llm": {
      "default": "deny",
      "budgets": {
        "perProjectDailyTokens": 2000000,
        "perProjectSessionTokens": 500000,
        "perProjectMaxConcurrentRequests": 4
      },
      "providers": {
        "anthropic": {}
      },
      "models": [
        {
          "name": "claude-sonnet-4-5",
          "alias": "claude-sonnet-4-5-20250929",
          "provider": "anthropic",
          "upstreamModel": "claude-sonnet-4-5-20250929"
        }
      ],
      "promptFilters": ["project-codename-x"]
    }
  }
}

If the section is missing or invalid, the agent cannot call any model. When you publish an invalid section to a running project, it keeps its previous model rules and the rejection is recorded.

Models

Only the models you list can be used. Each entry in models approves one model:

Tools send the model name they are configured with. If a tool requests a dated or suffixed model ID (for example claude-sonnet-4-5-20250929), add it as name or alias. The audit trail shows the exact name requested when a model is refused.

Approval on first use

With "promptOnFirstUse": true, the first request to that model in a project pauses while the Levee app asks the user. The request is held open until the user answers. If the user approves, the model can be used for the rest of the project run. If the user denies, or does not answer within 5 minutes, the request fails with User denied access to this model. and later requests for that model in the same run fail with User previously denied this model. Stopping and starting the project resets the decision.

Providers

A provider is the company or cloud service that runs the model. List the ones you use under providers; each has its own settings below.

Anthropic

"providers": { "anthropic": {} }

No settings. Requests go to Anthropic's public API. The key is read from LEVEE_KEY_ANTHROPIC on the computer.

OpenAI

"providers": { "openai": {} }

No settings. The key is read from LEVEE_KEY_OPENAI.

Azure OpenAI

"providers": {
  "azure-openai": {
    "endpoint": "https://contoso-ai.openai.azure.com",
    "apiVersion": "2024-10-21"
  }
},
"models": [
  { "name": "gpt-4o", "provider": "azure-openai", "upstreamModel": "gpt-4o", "deployment": "gpt4o-prod" }
]

• endpoint must be https, on the default port, with no user name or password in it, on an Azure OpenAI host (*.openai.azure.com, *.openai.azure.us for Azure Government, or *.openai.azure.cn for Azure operated by 21Vianet).
• apiVersion is required and has the form YYYY-MM-DD or YYYY-MM-DD-preview.
• Every model needs a deployment.
• The key is read from LEVEE_KEY_AZURE_OPENAI.

Azure AI Foundry

"providers": {
  "azure-foundry": { "endpoint": "https://contoso.services.ai.azure.com" }
}

• endpoint must be https on an Azure AI host (*.services.ai.azure.com, *.inference.ai.azure.com or *.models.ai.azure.com).
• Uses the OpenAI protocol.
• The key is read from LEVEE_KEY_AZURE_FOUNDRY.

AWS Bedrock (Claude models)

"providers": {
  "bedrock": { "region": "eu-central-1" }
},
"models": [
  { "name": "claude-sonnet-4-5", "provider": "bedrock", "upstreamModel": "eu.anthropic.claude-sonnet-4-5-20250929-v1:0" }
]

• region is required, for example us-east-1 or eu-central-1.
• Uses the Anthropic protocol, so it serves Claude Code.
• Credentials are read from LEVEE_KEY_BEDROCK_ACCESS (access key ID) and LEVEE_KEY_BEDROCK_SECRET (secret access key), plus LEVEE_KEY_BEDROCK_SESSION_TOKEN when you use temporary credentials.
• upstreamModel is the Bedrock model ID or inference profile ID.

Vediram LLM Gateway

If your organization uses the Vediram LLM Gateway, the agent's model calls are billed and governed centrally, and no provider key is stored on the computer at all. Requests are authenticated with the Vediram identity of the user signed in to the Levee app. Configure one provider entry per protocol, using the gateway address Vediram gives you:

"providers": {
  "vediram-anthropic": {
    "type": "compatible-endpoint",
    "baseUrl": "https://<your-gateway-address>",
    "protocol": "anthropic",
    "authMode": "vediram-user"
  },
  "vediram-openai": {
    "type": "compatible-endpoint",
    "baseUrl": "https://<your-gateway-address>",
    "protocol": "openai",
    "authMode": "vediram-user"
  }
},
"models": [
  { "name": "claude-sonnet-4-5", "provider": "vediram-anthropic", "upstreamModel": "claude-sonnet-4-5" },
  { "name": "gpt-4o", "provider": "vediram-openai", "upstreamModel": "gpt-4o" }
]

Requirements:

• The user must be signed in to Vediram Cloud in the Levee app, and the app must be running. If nobody is signed in, model requests are refused.
• Run the Levee app normally, not "as administrator". An elevated app is not accepted as a source of the user's identity.
• Which models each user may use on the gateway, and their gateway budgets, are managed in the Vediram Portal under AI agents > LLM Catalogs and LLM Assignments. The models list in the device policy must also include them.

Provider keys

Provider keys are stored on each computer as machine-level environment variables, never in policy.

A policy section that contains an apiKey for a provider is rejected as a whole with the message inline 'apiKey' is not supported. This keeps keys out of files that are copied, synced and backed up.

Instead, set the key as a machine-level environment variable on the computer. The Levee service passes it to its model access component outside the sandbox, never to the sandbox.

From an elevated PowerShell 7 prompt:

$key = Read-Host -Prompt 'Anthropic API key' -MaskInput
[Environment]::SetEnvironmentVariable('LEVEE_KEY_ANTHROPIC', $key, 'Machine')
Remove-Variable key
Restart-Computer

Restart the computer after setting or changing a key. Windows services read machine-level environment variables when the system starts, so restarting only the Levee service does not pick up a new value.

To deploy keys at scale, use your endpoint management tool (for example an Intune remediation script or a Configuration Manager script running as SYSTEM) to set the same machine-level variables.

Security considerations:

• Machine-level environment variables are readable by any account that can sign in to the computer. Use keys dedicated to Levee, restrict them at the provider (spending limits, allowed models, allowed IP ranges where supported), and rotate them on your normal schedule.
• For shared or high-risk computers, prefer the Vediram LLM Gateway, which needs no key on the device.
• If a key is missing, requests to that provider fail with No credential configured for provider '<name>'.

Budgets

Budgets cap how many tokens (the units model providers bill by) each project may use per day and per run. When a budget is reached, further requests are refused until it resets. Budgets are enforced on the computer before a request is sent.

Both token budgets are required and must be positive. A budget of 0 refuses every request.

How budgets are counted:

• When a request arrives, Levee reserves an estimate based on the request size. If the reservation would exceed a budget, the request is refused with session token budget exceeded. or daily token budget exceeded. (HTTP 429, so SDKs treat it as a rate limit).
• When the provider reports actual usage, the reservation is replaced by the real input and output token count.
• Budgets are tracked on the computer for the lifetime of the project run. Stopping and starting a project starts a new session budget; the daily count also starts again after the model access component restarts.

Treat per-project budgets as a guard against runaway loops. For organization-wide spend control, combine them with limits at the provider or use the Vediram LLM Gateway's budgets.

Prompt filters

Prompt filters block requests that contain a term you list, such as a client name or a confidential project name. They catch accidents; they are not a security boundary.

promptFilters is a list of terms. If any term appears in a request (case-insensitive, after Unicode normalization and removal of invisible characters), the request is refused with Prompt contains a disallowed term. The term itself is never written to the audit trail or logs; the audit records only an anonymous marker.

Prompt filters are defense in depth, not a security boundary. A determined agent can rephrase, encode or split text so that a literal term no longer matches. Use them to catch accidents (for example a project code name pasted into a prompt), not to protect secrets. Keep secrets out of the sandbox instead.

Limits

• Request bodies are limited to 4 MB (Request body exceeds 4 MB limit.).
• Levee does not retry failed provider calls; the agent's SDK already does.
• Streaming responses are passed through as they arrive.
• Prompts and responses are not written to the audit trail. The audit records which model was used, token counts, latency and outcome.

Errors the agent can receive

When Levee refuses a model request, the agent receives a normal error from its model service. Errors use the native error format of the protocol the agent called, so SDKs handle them normally.

See Model errors for fixes.

Audit

See Audit Event Reference.