kadet-hub logokadet-hub

Rocket.Chat on Kubernetes without Helm, with Kapitan

Rocket.Chat 8.9.1, team chat, as one process with its first admin from parameters and registration closed, rendered by the kadet-hub kubernetes generator 1.14.0 from Rocket.Chat’s Helm chart 7.1.0, on the included recipe mongodb as a replica set on MongoDB 9.0.

What it deploys

Required parameters

applications.rocketchat.host is the host name users reach Rocket.Chat at. The recipe sets ROOT_URL to https://<host>, as the chart’s host does, and Rocket.Chat builds its links from it.

applications.rocketchat.admin_email and applications.rocketchat.admin_password are the address and password of the user admin, which Rocket.Chat creates on its first start.

applications.mongodb.password, which the included recipe mongodb requires, is the password of MongoDB’s user root.

applications.mongodb.users.rocketchat.value is the password of MongoDB’s user rocketchat, with which Rocket.Chat connects. It is part of the MongoDB URL in the Secret rocketchat-rocketchat, so use letters and digits only.

Set the three passwords as refs with the backend of your choice. Without one of the parameters the compile stops with Interpolation key '<key>' not found.

Include

A target includes the recipe and its options as classes and can override anything the recipe sets:

classes:
  - recipes.rocketchat
parameters:
  namespace: rocketchat
  kapitan:
    vars:
      target: rocketchat
  applications:
    rocketchat:
      host: chat.example.com
      admin_email: admin@example.com
      admin_password: <ref>
    mongodb:
      password: <ref>
      users:
        rocketchat:
          value: <ref>

Fetching it

The quickstart goes through these steps on a kind cluster with the uptime-kuma recipe, from installing Kapitan to the running app.

A separate fetch inventory pins the recipe by version and digest. This is fetch/targets/recipes.yml for version 1.0.0:

parameters:
  kapitan:
    dependencies:
      - type: oci
        source: ghcr.io/kadet-hub/recipes/rocketchat:1.0.0@sha256:ced1a03e2ee846534233616760796b1ccc737fc675a1863666ef2004704da865
        output_path: inventory/classes/recipes/rocketchat

Version 1.0.0 includes mongodb 1.1.0. The consumer workflow adds the included recipes at the digests the index lists. Kapitan's --fetch reads no index, so for a local compile pull them by digest first:

tmp=$(mktemp -d)
oras pull -o "$tmp" ghcr.io/kadet-hub/recipes/mongodb@sha256:bed34997d85a978e3ce5ebfd2f87284a69c38a29bb64c80e81c9b3d701eea525
mkdir -p inventory/classes/recipes/mongodb
tar -xf "$tmp/mongodb.tar" -C inventory/classes/recipes/mongodb --no-same-owner

The first command fetches the recipe into inventory/classes/recipes/rocketchat, the second the generators and compiles:

kapitan compile --inventory-path fetch --fetch --output-path .
kapitan compile --fetch

The first command exits with status 1 because the fetch target compiles nothing. Kapitan's --fetch checks neither the digest nor the attestation. This verifies the attestation of the recipe and of the recipes it includes:

gh attestation verify \
  oci://ghcr.io/kadet-hub/recipes/rocketchat@sha256:ced1a03e2ee846534233616760796b1ccc737fc675a1863666ef2004704da865 \
  --owner kadet-hub \
  --cert-identity https://github.com/kadet-hub/registry/.github/workflows/release.yml@refs/heads/main

gh attestation verify \
  oci://ghcr.io/kadet-hub/recipes/mongodb@sha256:bed34997d85a978e3ce5ebfd2f87284a69c38a29bb64c80e81c9b3d701eea525 \
  --owner kadet-hub \
  --cert-identity https://github.com/kadet-hub/registry/.github/workflows/release.yml@refs/heads/main

The attested index must also list each digest under its recipe with the pinned version, and not as yanked. Verifying by hand shows how to read it.

Settings

<c> stands for components.rocketchat-rocketchat.

Chart value Inventory
image.tag <c>.image, tag and digest
host applications.rocketchat.host
extraEnv <c>.env, such as OVERWRITE_SETTING_<setting>
persistence.enabled, .size <c>.volumes.rocket-data, a persistentVolumeClaim in place of the emptyDir, with the claim in <c>.persistent_volume_claims
prometheusScraping.port, 9100 <c>.env.OVERWRITE_SETTING_Prometheus_Port, <c>.ports.metrics.service_port and the Service annotation prometheus.io/port
service.type <c>.service.type
resources <c>.resources
nodeSelector, tolerations, affinity <c>.node_selector, <c>.tolerations, <c>.affinity

The pod template carries the chart’s annotation checksum/secret as the chart renders it with tests/values.yaml. It does not follow a change of the Secret.

Any list set in the target is appended to the recipe’s list. To replace one, set the key to ~ in a class included after the recipe and the new list in the target.

Left out

The chart’s Bitnami MongoDB subchart gives way to mongodb, as mongodb.enabled: false with externalMongodbUrl in tests/values.yaml. The chart’s default microservices.enabled: true runs Rocket.Chat as several services with NATS, which the chart’s values.yaml marks as available to Enterprise users only, so tests/values.yaml turns both off. The chart’s own tests/monolith-values.yaml runs without microservices as well. The PodMonitor renders only where the cluster serves monitoring.coreos.com/v1, which check does not install. Ingress, Matrix federation with Synapse, SMTP, the registration token and the license are off by default and not part of the recipe. More than one replica, for which the chart adds a PodDisruptionBudget, is not tested. The chart renders no hook and no Role.

Proof

check rocketchat --level deploys with tests/values.yaml and tests/target.yml, on kind for each Kubernetes minor CI tests. The objects of mongodb are left out of the comparison and applied with the recipe. The Deployment and the Service select on name, the chart on app.kubernetes.io/name and instance, and both select the same pods. The pod and the ServiceAccount set automountServiceAccountToken: true, the API default the chart leaves unset. The container is named after the Deployment, and the Service’s metrics port targets the port by its name metrics where the chart gives 9100. The Secret rocketchat is only in the recipe, accepted as a Referenced Secret: the chart’s container reads its key admin-password.

tests/smoke reaches Rocket.Chat through the Service rocketchat-rocketchat. The admin logs in and has the role admin, creates a member, and the two read each other’s message in the channel general through the REST API. Rocket.Chat’s statistics report MongoDB 9.0, the replica set rs0 has its member as primary, Rocket.Chat logged no error for its change stream, and a change stream on Rocket.Chat’s messages delivers the admin’s message. Registration is refused with error-user-registration-disabled. After the MongoDB and the Rocket.Chat pod are deleted, the member is primary again and the admin reads both messages.

Every connection from Rocket.Chat’s pod that authenticated in MongoDB did so as rocketchat, which $currentOp shows. The change stream on the messages runs as that user. listDatabases names only rocketchat to it, and it gets Unauthorized for a read of local.oplog.rs, a write in admin, serverStatus and createUser. Rocket.Chat’s statistics report MongoDB 9.0 with the storage engine unknown. After the MongoDB pod is deleted, the user still has readWrite on rocketchat as its only role.

Versions

The recipe's files are in kadet-hub/recipes. All recipes