Install on Kubernetes

What you will doCopy Anchor Link

Install a production server using the published Helm chart. Prepare PostgreSQL and a persistent S3-compatible bucket separately. Chart installation success and session data restoration success require separate checks.

Before you beginCopy Anchor Link

Follow the deployment package contract to select charts and packages. For non-AWS endpoints, align config.aws.endpointUrl and the credential allowlist and verify the protection contract. An actual AWS installation has not been verified by execution.

The chart does not create PostgreSQL, object storage, or Secret values. It also does not create worker Pods: the scheduler creates them per session. Published images are indexes containing linux/amd64 and linux/arm64, so nodes pull their architecture.

The chart does not create a Git server. Workers must reach repository URLs in the settings document through the egress proxy. The kind test Git server is not a production repository.

Examples below use werft for both namespace and release. Before running them, confirm kubectl config current-context and KUBECONFIG select this installation's cluster. For other installations, replace every -n, --namespace, release name, deploy/werft-*, and svc/werft-* with the actual installation fullname. If the release name does not contain werft, fullname is <release>-werft.

Check the checkpoint bucket and workspace storage contract first. The backend requires the installation id to be a lowercase DNS label of at most 32 characters, even if the chart accepts more. Prepare access before remote bootstrap.

StepsCopy Anchor Link

1. Select an exact chart versionCopy Anchor Link

VERSION='<published-chart-version>'
helm show chart oci://ghcr.io/jeongjaesoon/werft --version "$VERSION"
helm show values oci://ghcr.io/jeongjaesoon/werft --version "$VERSION" | sed -n '/^images:/,/^$/p'

Check for all three image digest values and worker.runtime under images. These are included in the chart. Do not repeat them in installation-specific values: doing so keeps overriding subsequent chart values.

2. Namespace and SecretCopy Anchor Link

Create the Secret from a directory containing one file per key (0700, filenames are key names). This keeps values out of command lines and shell history.

kubectl create namespace werft
kubectl -n werft create secret generic werft --from-file=<secret-dir>

Keep the Secret directory at 0700 and files readable only by operators. The default API replica count is 2. With multiple replicas, add the same BOOTSTRAP_TOKEN (32–256 characters) to the Secret and include it in secrets.catalogEnv. For console model keys, use ["BOOTSTRAP_TOKEN"]; for environment keys, include the needed names too. Every listed key must exist in the Secret.

3. Installation-specific valuesCopy Anchor Link

Keep installation-specific settings in one values-<installation>.yaml file. At minimum, include the checkpoint bucket and how workers reach it.

installationId: werft-main
secrets:
  catalogEnv: ["BOOTSTRAP_TOKEN"]
config:
  checkpoints:
    bucket: <bucket>
  egress:
    credentialAllowlist: "api.anthropic.com:443,registry.npmjs.org:443,rubygems.org:443,proxy.golang.org:443,sum.golang.org:443,github.com:443,<bucket>.s3.<region>.amazonaws.com:443"

credentialAllowlist replaces the entire default. Restate all default entries (providers and package registries), then add repository hosts (github.com:443 and any other Git hosts used) and the bucket host.

config.aws.region defaults to ap-northeast-1. Change it for a bucket in another region. Connections, repositories, MCP servers, and limits are not in values: add them through the web console after installation. The alternative bun run settings:k8s import requires a checkout matching the installed version and host Bun/kubectl. See helm show values for defaults and descriptions.

config.egress.credentialAllowlist replaces all defaults, so when adding GitHub and checkpoint bucket hosts, include every required provider and package registry again. API and proxy read the same two credential lists.

Set a unique installationId in the same file. Retain the published chart's images.* digests. Proceed after checking the database, S3, Secret, and CNI are ready.

4. Install and roll outCopy Anchor Link

helm upgrade --install werft oci://ghcr.io/jeongjaesoon/werft --version "$VERSION" \
  --namespace werft -f values-<installation>.yaml --timeout 10m

The migrate Job upgrades the schema first. Once it finishes, api, egress proxy, reconciler, scheduler, and slack are applied together. If the first installation fails due to image paths or pull permissions, correct the values and rerun the same command.

kubectl -n werft get job werft-migrate
for d in api egress-proxy reconciler scheduler; do
  kubectl -n werft rollout status deploy/werft-$d --timeout=600s
done

When the release name does not contain werft, resource names become <release>-werft-….

After a failed first installation, fix values and rerun the same command. If Deployments already exist and you changed the worker image, upgrade-check rules apply. With no sessions, remove the Kubernetes installation and start again.

VerifyCopy Anchor Link

kubectl -n werft port-forward svc/werft-api 13000:3000 &
PF=$!
trap 'kill "$PF" 2>/dev/null || true' EXIT INT TERM
READY=0
for attempt in $(seq 1 30); do
  if curl --max-time 2 -fsS -o /dev/null http://127.0.0.1:13000/readyz; then READY=1; break; fi
  sleep 1
done
if [ "$READY" != 1 ]; then
  kill "$PF" 2>/dev/null || true
  trap - EXIT INT TERM
  false
else
  curl --max-time 10 -fsS -w '\n%{http_code}\n' http://127.0.0.1:13000/readyz
  curl --max-time 10 -fsS -o /dev/null -w '%{http_code}\n' http://127.0.0.1:13000/docs
  kill "$PF" 2>/dev/null || true
  trap - EXIT INT TERM
fi

Check the ready response body and 200, and docs 200. If not ready within 30 attempts, clean up the port-forward PID started by this shell and record failure. Also run access checks through the actual user path. These checks establish empty-installation readiness, not worker execution or checkpoint restoration.

TroubleshootingCopy Anchor Link

On the first day, immediately after installation, after 1 hour, and before the end of the workday, check kubectl -n werft get job,deploy,pods,pvc, api/scheduler/reconciler logs, each --health, disk space, inodes, and PVC space. Check backup failure alerts, checkpoint GC schedules, key revocation contacts, and provider cost limits. If release or namespace differ from examples, adapt every command to this installation.

A CNI without NetworkPolicy enforcement provides no worker isolation. Use Pod and Job logs to distinguish missing Secret keys, invalid installationId, S3 protection errors, and image pull failures. A fixture chart render does not prove a published chart installation.

Roll backCopy Anchor Link

Do not automatically roll back to the previous revision. After a failed first installation, inspect this namespace's resources and follow removal. Preserve PVCs and uncollected checkpoints in the bucket.

Next stepsCopy Anchor Link

Access → First owner and keys → Model connections. For version changes, follow Upgrade Kubernetes.

Previous URLsCopy Anchor Link