LiteLLM on Kubernetes without Helm, with Kapitan

LiteLLM 1.108.0, a proxy that serves one OpenAI-compatible API in front of Ollama, vLLM and hosted model providers, with models from the inventory, rendered by the kadet-hub kubernetes generator 1.14.0 from LiteLLM’s own Helm chart. The option postgresql adds virtual keys.

• Latest version: 1.0.0, published 2026-11-07
• Source chart: oci://ghcr.io/berriai/litellm-helm:1.108.0@sha256:97d9ad3aa28be90cd5f5b9fd47cc72bd76a6054111337bacfd537ee2dd2de2dd
• Proof level: deploys, compared with the chart by the kadet-hub/recipes CI
• renders passed in the release run of 2026-11-07 with check 0.18.0, kind v0.33.0
• Output capabilities: none
• Inventory backends: omegaconf
• Generators: kubernetes 1.14.0 sha256:787129e1eaed6b9bc868fe4c4aec58d40de6a89fc3cc08424a4cb5822b855080
• Image security: grade D 17 findings, one per advisory and package, in the 1 image of version 1.0.0: CRITICAL 0, HIGH 2, MEDIUM 12, LOW 3, UNKNOWN 0. Scanned 2026-11-07 11:47 UTC. Full report
  The grade rates the upstream images the recipe pins, not the recipe: ghcr.io/berriai/litellm.

What it deploys

• recipes.litellm (init.yml)
• recipes.litellm.litellm (litellm.yml): litellm

Required parameters

applications.litellm.master_key is the proxy’s master key. Every request without a virtual key sends it as Authorization: Bearer <key>, and it is the only credential that can create virtual keys. LiteLLM’s documentation writes keys with the prefix sk-. Set it as a ref with the backend of your choice. The recipe puts it into the Secret litellm-masterkey, key masterkey.

applications.litellm.model_list is the proxy’s model_list, as LiteLLM’s configuration describes it: the name each model has in the API and, in litellm_params, the provider and model LiteLLM routes it to. The Include snippet routes smollm to the recipe ollama in the same namespace.

Without one of them the compile stops with Interpolation key '<key>' not found.

Include

A target includes the recipe and its options as classes and can override anything the recipe sets:

classes:
  - recipes.litellm
parameters:
  namespace: litellm
  kapitan:
    vars:
      target: litellm
  applications:
    litellm:
      master_key: <ref>
      model_list:
        - model_name: smollm
          litellm_params:
            model: ollama_chat/smollm:135m
            api_base: http://ollama:11434

Fetching it

The quickstart goes through these steps on a kind cluster with the uptime-kuma recipe, from installing Kapitan to the running app.

A separate fetch inventory pins the recipe by version and digest. This is fetch/targets/recipes.yml for version 1.0.0:

parameters:
  kapitan:
    dependencies:
      - type: oci
        source: ghcr.io/kadet-hub/recipes/litellm:1.0.0@sha256:1c0b0e14f7d460b4b61be712bd342dd409a82c5cce62760bc2847da82e6a68c3
        output_path: inventory/classes/recipes/litellm

Version 1.0.0 includes postgresql 1.1.1. The consumer workflow adds the included recipes at the digests the index lists. Kapitan's --fetch reads no index, so for a local compile pull them by digest first:

tmp=$(mktemp -d)
oras pull -o "$tmp" ghcr.io/kadet-hub/recipes/postgresql@sha256:1e8465058c256cc86995dc8d6f1eb2f095df244fadcaf756935599f4d9ead6a9
mkdir -p inventory/classes/recipes/postgresql
tar -xf "$tmp/postgresql.tar" -C inventory/classes/recipes/postgresql --no-same-owner

The first command fetches the recipe into inventory/classes/recipes/litellm, the second the generators and compiles:

kapitan compile --inventory-path fetch --fetch --output-path .
kapitan compile --fetch

The first command exits with status 1 because the fetch target compiles nothing. Kapitan's --fetch checks neither the digest nor the attestation. This verifies the attestation of the recipe and of the recipes it includes:

gh attestation verify \
  oci://ghcr.io/kadet-hub/recipes/litellm@sha256:1c0b0e14f7d460b4b61be712bd342dd409a82c5cce62760bc2847da82e6a68c3 \
  --owner kadet-hub \
  --cert-identity https://github.com/kadet-hub/registry/.github/workflows/release.yml@refs/heads/main

gh attestation verify \
  oci://ghcr.io/kadet-hub/recipes/postgresql@sha256:1e8465058c256cc86995dc8d6f1eb2f095df244fadcaf756935599f4d9ead6a9 \
  --owner kadet-hub \
  --cert-identity https://github.com/kadet-hub/registry/.github/workflows/release.yml@refs/heads/main

The attested index must also list each digest under its recipe with the pinned version, and not as yanked. Verifying by hand shows how to read it.

Settings

A provider’s key goes into the container’s environment from a Secret of the target, such as components.litellm.env.OPENAI_API_KEY with a secretKeyRef, and the model names it as api_key: os.environ/OPENAI_API_KEY.

Further sections of config.yaml, such as litellm_settings or router_settings, are keys of applications.litellm.config. The recipe writes the mapping with Kapitan’s resolver yaml, with sorted keys.

The pod annotation checksum/config is the chart’s hash for the values in tests/values.yaml. It does not change with applications.litellm.config, and LiteLLM reads config.yaml only at start, so after changing the models restart the pod with kubectl rollout restart deployment/litellm.

Any list set in the target is appended to the recipe’s list. To replace one, set the key to ~ in a class included after the recipe and the new list in the target.

Left out

The chart’s tests run only inside helm test. tests/smoke checks more on the installed recipe. They are not rendered: Pod/litellm-test-connection, Pod/litellm-env-test.

The chart’s own PostgreSQL and Redis, both Bitnami subcharts, and its migrations Job are not part of the recipe (DEC-236).

Proof

check litellm --level deploys with tests/values.yaml and tests/target.yml, on kind for each Kubernetes minor CI tests. tests/target.yml includes the option postgresql, and tests/values.yaml turns off the chart’s database and migrations Job, gives DATABASE_URL from the Secret of the recipe postgresql, and sets the master key and the model list of the target. Its one model, smoke, answers with LiteLLM’s mock_response, so the proof calls no provider. The four objects differ in two ways, both of classes check accepts. The Deployment and the Service select on name, the chart on app.kubernetes.io/name and instance, and both select the same pods. The pod sets automountServiceAccountToken: true, which the chart leaves unset while the ServiceAccount default allows the token. The chart binds no role.

tests/smoke requires a healthy proxy with its database connected from /health/readiness through the Service. With the master key it requires the model healthy from /health, only smoke from /v1/models, and the mock’s reply from /v1/chat/completions. It gets 401 from both without a key. It creates a virtual key, gets the reply with it, gets 401 with a wrong key, and waits for spend on the virtual key in /key/info. After the pod was replaced, the virtual key still gets the reply.

Versions

• 1.0.0, 2026-11-07: sha256:1c0b0e14f7d460b4b61be712bd342dd409a82c5cce62760bc2847da82e6a68c3

The recipe's files are in kadet-hub/recipes. All recipes