kadet-hub logokadet-hub

Ollama on Kubernetes without Helm, with Kapitan

Ollama 0.33.2 on the CPU, which runs language models behind an HTTP API on port 11434, rendered by the kadet-hub kubernetes generator 1.14.0 from the ollama chart 1.79.0 that Open WebUI’s chart includes.

What it deploys

Required parameters

No parameter has to be set.

Include

A target includes the recipe and its options as classes and can override anything the recipe sets:

classes:
  - recipes.ollama
parameters:
  namespace: ollama
  kapitan:
    vars:
      target: ollama

Fetching it

The quickstart goes through these steps on a kind cluster with the uptime-kuma recipe, from installing Kapitan to the running app.

A separate fetch inventory pins the recipe by version and digest. This is fetch/targets/recipes.yml for version 1.0.1:

parameters:
  kapitan:
    dependencies:
      - type: oci
        source: ghcr.io/kadet-hub/recipes/ollama:1.0.1@sha256:7d1081b5c348b8c6e119af4a972b91a5543e0fad1f92d494c8a86bd0026d7db2
        output_path: inventory/classes/recipes/ollama

The first command fetches the recipe into inventory/classes/recipes/ollama, the second the generators and compiles:

kapitan compile --inventory-path fetch --fetch --output-path .
kapitan compile --fetch

The first command exits with status 1 because the fetch target compiles nothing. Kapitan's --fetch checks neither the digest nor the attestation. This verifies the attestation of the recipe:

gh attestation verify \
  oci://ghcr.io/kadet-hub/recipes/ollama@sha256:7d1081b5c348b8c6e119af4a972b91a5543e0fad1f92d494c8a86bd0026d7db2 \
  --owner kadet-hub \
  --cert-identity https://github.com/kadet-hub/registry/.github/workflows/release.yml@refs/heads/main

The attested index must also list each digest under its recipe with the pinned version, and not as yanked. Verifying by hand shows how to read it.

Settings

<c> stands for components.ollama.

Chart value Inventory
image.tag <c>.image, tag and digest
persistentVolume.enabled <c>.volumes.ollama-data as persistentVolumeClaim and <c>.persistent_volume_claims.<claim>.storage
extraEnv <c>.env, a mapping of name to value, such as OLLAMA_KEEP_ALIVE
resources <c>.resources
ollama.gpu.enabled <c>.resources.limits with nvidia.com/gpu, not tested
service.type <c>.service.type
nodeSelector, tolerations, affinity <c>.node_selector, <c>.tolerations, <c>.affinity
podSecurityContext, securityContext <c>.workload_security_context, <c>.security_context
terminationGracePeriodSeconds <c>.grace_period

GPUs are not tested, because kind has none. The chart’s model pulls at start (ollama.models.pull), Knative service, Ingress, HTTPRoute, Gateway and autoscaler are off by default and not part of the recipe. The chart’s Helm test Pod/ollama-test-connection is left out.

Any list set in the target is appended to the recipe’s list. To replace one, set the key to ~ in a class included after the recipe and the new list in the target.

Proof

check ollama --level deploys with tests/values.yaml and tests/target.yml, on kind for each Kubernetes minor CI tests. tests/values.yaml keeps the chart’s defaults and names the image’s registry host. The three objects differ in two ways. The Deployment and the Service select on name, the chart on app.kubernetes.io/name and instance, and both select the same pods. The pod sets automountServiceAccountToken: true, which the chart leaves unset while its ServiceAccount allows the token. The chart binds no role.

tests/smoke requires Ollama is running from / through the Service and the image tag’s version from /api/version. It pulls smollm:135m, a 92 MB model, checks the digest of its manifest in /api/tags and requires a reply from /api/generate on the CPU. It prints how long the pull and the generation took.

Used in stacks

Versions

The recipe's files are in kadet-hub/recipes. All recipes