Upgrading a cluster
Upgrade your Stackland cluster version using the TargetInstallationState Kubernetes resource. Perform a rolling update with incremental load evacuation and node reboot.
Getting started
-
Make sure you have admin access to the cluster.
-
Check the current version of your cluster:
kubectl get targetinstallationstate main -o jsonpath='{.status.currentVersion}' -
Find out which versions you can upgrade to. The available versions vary based on whether your cluster has internet access:
- Cluster with internet access: Available versions are pulled automatically from the Stackland container registry.
- Air-gapped cluster: Available versions are limited to the images manually imported into your local registry.
Selecting the release channel
The release channel determines which Stackland versions are available for the upgrade. The default channel is stable.
Available channels:
stable: Stable releases for use in production environments. This channel is available to all customers by default.alpha: Channel with early releases for testing new features. You can access it upon request.
To switch channels, update the PlatformConfig resource:
kubectl edit platformconfig main
Indicate your preferred channel in the specification:
spec:
releaseChannel: "stable" # or "alpha"
Before upgrading to 27.x
Stackland 26.2 introduced a new Contour-based Ingress controller deprecating the legacy implementation based on ingress-nginx. Stackland 27 will completely remove ingress-nginx from the distribution, and this serves as prior notice of these plans. In Stackland 26 releases, ingress-nginx remains available for custom workloads, but you must migrate to the new controller before upgrading to Stackland 27.
In most cases, custom Ingress manifests with the stackland-default class will remain operational, but the new controller does not support nginx.ingress.kubernetes.io/* annotations, which are specific to ingress-nginx. Make sure to remove or replace them. For more information, see Checking custom Ingress resources.
Also note that operator action is required: the upgrade to version >= 27.0.0 will fail to initialize unless you explicitly disable the legacy Ingress controller.
Checking custom Ingress resources
If your cluster contains custom Ingress resources:
-
Locate custom
Ingressresources:kubectl get ingress -A -
Check that they specify
spec.ingressClassName: stackland-defaultor have no such field. In the latter case, the default class is used. -
Make sure resource annotations do not contain nginx-specific
nginx.ingress.kubernetes.io/*keys. If you find any such annotations, remove or replace them with the new Ingress controller alternatives, or the rules will fail to apply after the upgrade.
Testing the application before the upgrade
While your cluster is still on version 26.x, you can proactively verify that your custom Ingress is compatible with the new Ingress controller.
Warning
When changing spec.ingressClassName to stackland-system, remember that stackland-system has its own external IP address that does not match the legacy Ingress controller IP address. If the DNS zone is delegated to the platform, the domain record will update automatically to point to the new LoadBalancer. However, during the standard DNS propagation delay, some client traffic will still be routed to the legacy Ingress controller IP address. A similar delay occurs after you change the class back to stackland-default.
Perform this test within a maintenance window or on a non-critical host.
-
In a temporary copy of the manifest, replace
spec.ingressClassName: stackland-defaultwithspec.ingressClassName: stackland-systemand apply the manifest:kubectl apply -f <file_name>.yaml -
Check application availability. There are two ways to do this:
-
Direct IP verification without DNS. This method works independently of DNS propagation or your application domain's DNS zone hosting.
-
Get the external IP of the LoadBalancer serving the
stackland-systemclass:kubectl get svc -A \ -l app.kubernetes.io/name=contour,app.kubernetes.io/component=envoyIf the selector does not work for your setup, manually locate the
LoadBalancerservice within your Contour namespace. -
Send a request to the application using the retrieved IP:
curl --resolve <domain>:443:<Envoy_external_IP> https://<domain>/
-
-
Domain-based verification. You can only use this method if your DNS zone is delegated: the domain record will automatically migrate to the
stackland-systemLoadBalancer. Keep in mind there is a DNS propagation delay. If you manage a custom DNS, the domain will continue to point to the legacy LoadBalancer, i.e., you will get a false negative result. Usecurl --resolveinstead.
-
-
After verification, change the
spec.ingressClassNamevalue back tostackland-defaultor remove this field.
You can skip this step on version 27.x: the new Ingress controller serves the stackland-default class by default.
Warning
stackland-system: Service class. Do not use it for production Ingress resources: this is a temporary class designed exclusively for compatibility tests prior to the upgrade.
Disabling the legacy Ingress implementation
-
Disable the legacy Ingress implementation within the
IngressConfigresource:kubectl patch ingressconfig main --type=merge \ -p '{"spec":{"settings":{"nginx":{"enabled":false}}}}'
Disabling ingress-nginx automatically restarts existing Ingress controllers on top of Contour.
-
Wait until the platform migrates to a single Ingress Controller. Check the current state:
kubectl get ingressconfig main -o jsonpath='{.status.ingress.observedIngress}'
If the cluster has no internet access, you must first pull the new version images to your local registry using the CLI.
-
In the left-hand menu, select Settings.
-
In the submenu, select Upgrade.
-
Click the Upgrade link to open the cluster upgrade page.
-
On the Cluster update page, check the Current upgrade status:
- Target version: Version to upgrade to.
- Phase: Upgrade stage.
- Message: Current state message.
-
Under Available updates, specify the version to upgrade to.
-
Click Start update.
The CLI steps depend on whether your cluster has internet access.
Cluster with internet access
For clusters with internet access, new version images are automatically pulled from the Stackland container registry.
Setting up an upgrade
Wait for the new release to appear in the AvailableReleases resource. To check for available releases, run this command:
kubectl get availablereleases main -o yaml
Here is an example of the command output:
apiVersion: stackland.yandex.cloud/v1alpha1
kind: AvailableReleases
metadata:
name: main
status:
releases:
- version: "26.1.0"
ready: true
- version: "26.1.1"
ready: true
As soon as the list displays the target version with the ready: true status, you can start the upgrade.
Starting the upgrade
-
Create the manifest file, e.g., using the
touch upgrade.yamlcommand. -
Open the file and paste the configuration:
apiVersion: stackland.yandex.cloud/v1alpha1 kind: TargetInstallationState metadata: name: main spec: targetVersion: "<version>" installationTimeout: "2h"Where:
targetVersion: Target version for the upgrade. Specify the value fromavailablereleases[main].status.releases[<preferred_release>].version.installationTimeout: Upgrade timeout.
-
Apply the manifest:
kubectl apply -f upgrade.yaml
Air-gapped cluster
If the cluster has no internet access, you must first pull the new version images to your local registry.
Downloading SLADM
On an internet-connected machine, download the latest sladm version in the same way as during its initial installation.
Pulling images on an internet-connected machine
On an internet-connected machine, run this command:
sladm pull --image-bundle full
Where:
--image-bundle: Image bundle type,fullto get all images.
Note
To upgrade images of separately licensed components, such as SpeechSense, run the command again with --image-bundle speechsense. For more information, see Uploading SpeechSense images.
Transferring artifacts into the internal infrastructure
Transfer the following to a machine with access to the local cluster registry:
sladmbinary filerelease.yaml<release_name>-ocifolder
Pushing images to the local registry
On a machine with cluster access, run this command:
sladm push --local-registry --kubeconfig=<kubeconfig_path> --image-bundle-folder <directory_name>-oci
Where:
--local-registry: Indicates the use of the cluster's local registry.--kubeconfig: Path to thekubeconfigfile for access to the cluster.--image-bundle-folder: Path to the folder with images.
Verifying release availability
After pushing the images, wait until the new release appears in the AvailableReleases resource.
kubectl get availablereleases main -o yaml
As soon as the list displays the target version with the ready: true status, you can start the upgrade.
Starting the upgrade
-
Create the manifest file, e.g., using the
touch upgrade.yamlcommand. -
Open the file and paste the configuration:
apiVersion: stackland.yandex.cloud/v1alpha1 kind: TargetInstallationState metadata: name: main spec: targetVersion: "<version>" installationTimeout: "2h"Where:
targetVersion: Target version for the upgrade. Specify the value fromavailablereleases[main].status.releases[<preferred_release>].version.installationTimeout: Upgrade timeout.
-
Apply the manifest:
kubectl apply -f upgrade.yaml
Checking the upgrade status
After applying the manifest, you can monitor the upgrade status:
kubectl get targetinstallationstate main
Here is an example of the command output:
NAME TARGET VERSION CURRENT VERSION PHASE MESSAGE AGE
main 26.1.1 26.1.0 Running Running upgrade to version 26.1.1 5m
Getting detailed information
Run this command:
kubectl describe targetinstallationstate main
The status field displays:
currentVersion: Currently installed version.phase: Upgrade stage:Pending: Upgrade awaiting start.Running: Upgrade in progress.Completed: Upgrade completed.Failed: Upgrade failed.
message: Current state message.jobName: Name of the Kubernetes upgrade job.lastUpdateTime: Status last update time.
Viewing upgrade logs
To read logs of the upgrade job, run this command:
kubectl logs -n stackland-install job/<job_name>
You can get the job name from the status.jobName field in the TargetInstallationState resource.