kadet-hub logokadet-hub

Immich on Kubernetes without Helm, with Kapitan

Immich v3.3.5, the photo and video management server, with its machine learning service on the CPU and a library claim, rendered by the kadet-hub kubernetes generator 1.14.0 from Immich’s Helm chart 0.13.4. The database is the included recipe postgresql with pgvector, the queue valkey.

What it deploys

Required parameters

applications.valkey.password, which the included recipe valkey requires, is the password of Valkey’s user default, with which Immich connects to its job queue. Set it as a ref with the backend of your choice. Without it the compile stops with Interpolation key 'applications.valkey.password' not found. Immich and postgresql require no parameter. The first user who signs up through the web interface or /api/auth/admin-sign-up becomes the admin, so sign up before the server is reachable by others.

Include

A target includes the recipe and its options as classes and can override anything the recipe sets:

classes:
  - recipes.immich
parameters:
  namespace: immich
  kapitan:
    vars:
      target: immich
  applications:
    valkey:
      password: <ref>

Fetching it

The quickstart goes through these steps on a kind cluster with the uptime-kuma recipe, from installing Kapitan to the running app.

A separate fetch inventory pins the recipe by version and digest. This is fetch/targets/recipes.yml for version 1.0.1:

parameters:
  kapitan:
    dependencies:
      - type: oci
        source: ghcr.io/kadet-hub/recipes/immich:1.0.1@sha256:f306966e147c2a3966fe7e5bdf4ca3749566fa9b311c942b47b8c93b592400f7
        output_path: inventory/classes/recipes/immich

Version 1.0.1 includes postgresql 1.1.1 and valkey 1.0.3. The consumer workflow adds the included recipes at the digests the index lists. Kapitan's --fetch reads no index, so for a local compile pull them by digest first:

tmp=$(mktemp -d)
oras pull -o "$tmp" ghcr.io/kadet-hub/recipes/postgresql@sha256:1e8465058c256cc86995dc8d6f1eb2f095df244fadcaf756935599f4d9ead6a9
mkdir -p inventory/classes/recipes/postgresql
tar -xf "$tmp/postgresql.tar" -C inventory/classes/recipes/postgresql --no-same-owner
oras pull -o "$tmp" ghcr.io/kadet-hub/recipes/valkey@sha256:a50d43cf9080b58040cf84909b0cd7497392709a23f5759e985bd912a9137ce5
mkdir -p inventory/classes/recipes/valkey
tar -xf "$tmp/valkey.tar" -C inventory/classes/recipes/valkey --no-same-owner

The first command fetches the recipe into inventory/classes/recipes/immich, the second the generators and compiles:

kapitan compile --inventory-path fetch --fetch --output-path .
kapitan compile --fetch

The first command exits with status 1 because the fetch target compiles nothing. Kapitan's --fetch checks neither the digest nor the attestation. This verifies the attestation of the recipe and of the recipes it includes:

gh attestation verify \
  oci://ghcr.io/kadet-hub/recipes/immich@sha256:f306966e147c2a3966fe7e5bdf4ca3749566fa9b311c942b47b8c93b592400f7 \
  --owner kadet-hub \
  --cert-identity https://github.com/kadet-hub/registry/.github/workflows/release.yml@refs/heads/main

gh attestation verify \
  oci://ghcr.io/kadet-hub/recipes/postgresql@sha256:1e8465058c256cc86995dc8d6f1eb2f095df244fadcaf756935599f4d9ead6a9 \
  --owner kadet-hub \
  --cert-identity https://github.com/kadet-hub/registry/.github/workflows/release.yml@refs/heads/main

gh attestation verify \
  oci://ghcr.io/kadet-hub/recipes/valkey@sha256:a50d43cf9080b58040cf84909b0cd7497392709a23f5759e985bd912a9137ce5 \
  --owner kadet-hub \
  --cert-identity https://github.com/kadet-hub/registry/.github/workflows/release.yml@refs/heads/main

The attested index must also list each digest under its recipe with the pinned version, and not as yanked. Verifying by hand shows how to read it.

Settings

<s> stands for components.immich-server, <m> for components.immich-machine-learning and <d> for applications.immich.component_defaults, which both components take.

Chart value Inventory
controllers.main.containers.main.image.tag <s>.image and <m>.image, tag and digest
the library claim of immich.persistence.library.existingClaim, 10Gi applications.immich.library_storage
server.controllers.main.containers.main.env <s>.env
machine-learning.controllers.main.containers.main.env <m>.env
machine-learning.persistence.cache, an emptyDir <m>.volumes.cache
controllers.main.containers.main.resources <s>.resources, <m>.resources
defaultPodOptions.nodeSelector, .tolerations, .affinity <d>.node_selector, <d>.tolerations, <d>.affinity

DB_VECTOR_EXTENSION=pgvector keeps Immich on pgvector, which the option pgvector of postgresql creates. DB_SSL_MODE=require encrypts the connection to PostgreSQL without verifying CloudNativePG’s certificate. Immich connects as the database owner app, which is no superuser. The options of postgresql create vector and earthdistance, which only a superuser may create, when CloudNativePG creates the Cluster. Immich’s nightly database backup runs pg_dump as app and works. A restore through Immich replays the dump as app, whose DROP EXTENSION lines for vector and earthdistance need a superuser, so expect it to fail. It is not tested. Moving to VectorChord later needs an image with VectorChord, vchord.so in shared_preload_libraries and CREATE EXTENSION vchord CASCADE as superuser, which postgresql does not render. Immich then rebuilds its vector indexes clip_index and face_index.

Machine learning downloads its models into /cache when a job first needs them, again on every new pod: CLIP and face models from Hugging Face, OCR models from ModelScope, about 2.2 GB together, at the unpinned main revision. A claim for /cache keeps them. GPU images and hardware acceleration are not tested.

The chart’s Ingress, its ServiceMonitors behind immich.metrics.enabled and the configuration file behind immich.configuration are off by default and not part of the recipe.

Any list set in the target is appended to the recipe’s list. To replace one, set the key to ~ in a class included after the recipe and the new list in the target.

Proof

check immich --level deploys with tests/values.yaml and tests/target.yml, on kind for each Kubernetes minor CI tests, with cnpg installed first. The objects of postgresql and valkey are left out of the comparison and applied with the recipe. The chart expects the library claim to exist, so tests/values.yaml creates immich-library through the library’s rawResources, and check compares it with the recipe’s claim. The Deployments and Services select on name, the chart on app.kubernetes.io/name, instance and controller, and both select the same pods. The ServiceAccounts set automountServiceAccountToken: true, the API default the chart leaves unset. Containers carry an empty securityContext and are named after their component. tests/target.yml runs one PostgreSQL instance.

tests/smoke gets pong from /api/server/ping and from machine learning’s /ping through their Services and requires that /api/server/version reports the Deployment’s image tag. It signs up the admin, checks that a wrong password gets 401, logs in, uploads a JPEG through /api/assets and reads back its details with its SHA-1, the original byte for byte and the thumbnail, which a job queued in Valkey creates. Every connection of app to PostgreSQL must use TLS. It does not wait for the machine learning jobs.

Versions

The recipe's files are in kadet-hub/recipes. All recipes