Enterprise LLM Gateway

Enterprise teams can now route LLM API calls through a ReclaimLLM gateway instead of giving every developer direct access to provider credentials.

Admins configure provider credentials once at the organization level, create gateway keys for individual teams, and control which model providers each team can use. Calls made through the gateway are logged as proxy sessions with org and team attribution, so enterprise admins and team leads can review usage from the same ReclaimLLM session and analytics surfaces they already use.

How it works

The gateway exposes an OpenAI-compatible proxy path scoped by enterprise slug. A team uses its ReclaimLLM gateway key as the bearer token, then calls the gateway instead of calling the upstream provider directly.

At a workflow level:

1. An enterprise admin opens the LLM Gateway page for an organization.
2. The admin stores provider credentials for supported providers.
3. The admin creates a team-scoped gateway key.
4. The team uses that key in API calls to the ReclaimLLM gateway.
5. The gateway validates the key, checks provider and model access, forwards the request to the right upstream provider, and logs the call under the organization and team.

Example shape:

curl https://reclaimllm.com/acme/v1/chat/completions \
  -H "Authorization: Bearer rclm-gw-your-team-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [
      { "role": "user", "content": "Summarize this incident report." }
    ]
  }'

The gateway is designed around team-level policy. A backend team might be allowed to use OpenAI and Anthropic, while another team might be restricted to Gemini or a narrower set of models. The key is not a provider key. It is a ReclaimLLM gateway key that authorizes access according to the org’s configured policy.

Provider credentials stay server-side. Developers and services using the gateway do not receive the upstream provider secret. The proxy resolves gateway keys through backend-owned policy logic, then uses the configured provider credential only for the outbound provider request.

What Gets Logged

Gateway calls are stored as proxy session records. They include the request route, response status, model, streaming flag, duration, token counts when available, org id, team id, and gateway key metadata.

This matters for enterprise review because proxy calls sit in the same general reporting model as other captured LLM activity. Admins and team leads can inspect LLM usage by organization and team without asking each developer to run a local capture proxy or manually share provider logs.

The first version focuses on useful operational metadata and attribution. It is not meant to replace provider billing dashboards, and provider-specific response shapes can vary. We are watching edge cases around streaming behavior, provider token reporting, and compatibility across newer provider APIs.

Why We Built It

The local ReclaimLLM proxy is useful for individual capture, but it is not enough for enterprise governance. It runs on a developer machine, depends on local provider keys, and cannot act as a central access-control point for an organization.

Enterprise customers need a different model:

• Admins should own provider credentials.
• Teams should receive scoped keys, not raw provider keys.
• Provider and model access should be explicit.
• LLM calls should be attributed to the right organization and team.
• Logging failures should not automatically break a successful provider response.

We chose a separate hosted gateway service instead of extending the local proxy. That keeps local capture and enterprise API access as separate concerns. It also lets the backend remain the owner of credential storage, key creation, access policy, and org/team attribution.

The gateway uses LiteLLM for provider routing and OpenAI-compatible request handling. That avoids maintaining custom adapters for every provider in the first version, while still leaving room to revisit the decision if latency, compatibility, or reliability requirements change.

Current Limitations

This is an initial enterprise gateway implementation.

Provider credentials are configured at the organization level. Team-owned provider credentials and team-specific billing separation are not part of this version.

Gateway reliability is now part of the customer’s LLM traffic path. The proxy is designed so logging failures do not block completed provider calls, but durable retry and stricter audit guarantees are areas we expect to harden as usage grows.

Provider support currently starts with the core providers requested by enterprise users. We expect the provider list and model-policy controls to expand based on real usage.

Feedback is especially useful around model naming, provider-specific compatibility, streaming behavior, and how teams want gateway logs to appear in enterprise review workflows.