Changelog on Documentation

Changelog for Kubernetes 1.34

Mon, 01 Jan 0001 00:00:00 +0000

Versions

The deployed Kubernetes patch version varies based on when your cluster is deployed or upgraded. We strive to use the latest versions available.

Current release leverages Kubernetes 1.34. Official release blogpost found here with corresponding official changelog.

Optional addons

ingress-nginx is provided with version v1.14.3
Certmanager is provided with version v1.18.2

Major changes

The 4k, 8k, 16k, and v1-dynamic-40 storage classes are removed in this version. Existing volumes will not be affected. The ability to create legacy volumes will be removed. Please migrate manifests that specify storage classes to the storageclasses prefixed with v2-, which have been available since Kubernetes 1.26 and have been the default since 2024-06-28 made public in the announcement.

Noteworthy changes in upcoming versions

Announcement of changes in future versions.

Changelog for Kubernetes 1.33

Mon, 01 Jan 0001 00:00:00 +0000

Versions

The deployed Kubernetes patch version varies based on when your cluster is deployed or upgraded. We strive to use the latest versions available.

Current release leverages Kubernetes 1.33. Official release blogpost found here with corresponding official changelog.

Optional addons

ingress-nginx is provided with version v1.14.3
cert-manager is provided with version v1.18.2

Major changes

Base Ubuntu image upgraded from 22.04 to 24.04.

Kubelet configurations provided by Elastx, not configurable by client

NodeDrainVolume and NodeDrainTimeout: 5 -> 15min

Changelog for Kubernetes 1.32

Mon, 01 Jan 0001 00:00:00 +0000

Versions

The deployed Kubernetes patch version varies based on when your cluster is deployed or upgraded. We strive to use the latest versions available.

Current release leverages Kubernetes 1.32. Official release blogpost found here with corresponding official changelog.

Optional addons

ingress-nginx is provided with version v1.12.1
cert-manager is provided with version v1.16.3

Major changes

We have announced the deprecation of legacy storageClasses in v1.32. This is postponed to v1.34.
Flow control flowcontrol.apiserver.k8s.io/v1beta3 will be removed. The replacement flowcontrol.apiserver.k8s.io/v1 was implemented in Kubernetes 1.29

Changelog for Kubernetes 1.31

Mon, 01 Jan 0001 00:00:00 +0000

Versions

The deployed Kubernetes patch version varies based on when your cluster is deployed or upgraded. We strive to use the latest versions available.

Current release leverages Kubernetes 1.31. Official release blogpost found here with corresponding official changelog.

Major changes

In case there are major changes that impacts Elastx Kubernetes cluster deployments they will be listed here.

Noteworthy API changes in coming version Kubernetes 1.32

Flow control flowcontrol.apiserver.k8s.io/v1beta3 will be removed. The replacement flowcontrol.apiserver.k8s.io/v1 was implemented in Kubernetes 1.29

Changelog for Kubernetes 1.30

Mon, 01 Jan 0001 00:00:00 +0000

Versions

The deployed Kubernetes version varies based on when your cluster is deployed. We try deploying cluster using the latest patch release of Kubernetes.

Current release is Kubernetes 1.30.1

Major changes

New default storageclass v2-1k
New clusters will only have v2 storage classes available.
nodelocaldns will be removed for all clusters where it’s still deployed. This change affects only clusters created prior to Kubernetes 1.26, as the feature was deprecated in that version.
Clusters created before Kubernetes 1.26 will have their public domains removed. In Kubernetes 1.26, we migrated to using a LoadBalancer and its IP instead. If you are using an old kubeconfig with an active domain, please fetch a new one.

APIs removed in Kubernetes 1.32

More details can be found in Kubernetes official documentation.

Changelog for Kubernetes 1.29

Mon, 01 Jan 0001 00:00:00 +0000

Versions

The deployed Kubernetes version varies based on when your cluster is deployed. We try deploying cluster using the latest patch release of Kubernetes.

Current release is Kubernetes 1.29.1

Major changes

Flow control flowcontrol.apiserver.k8s.io/v1beta2. The replacement flowcontrol.apiserver.k8s.io/v1beta3 was implemented in Kubernetes 1.26

APIs removed in Kubernetes 1.32

More details can be found in Kubernetes official documentation.

Flow control flowcontrol.apiserver.k8s.io/v1beta3. The replacement flowcontrol.apiserver.k8s.io/v1 was implemented in Kubernetes 1.29

Other noteworthy deprecations

Kubernetes beta topology labels on nodes are deprecated and will be removed in a future release. You can follow the list below to see which labels are being replaced:

Changelog for Kubernetes 1.28

Mon, 01 Jan 0001 00:00:00 +0000

Versions

The deployed Kubernetes version varies based on when your cluster is deployed. We try deploying cluster using the latest patch release of Kubernetes.

Current release is Kubernetes 1.28.6

Major changes

No major changes

APIs removed in Kubernetes 1.29

More details can be found in Kubernetes official documentation.

Flow control flowcontrol.apiserver.k8s.io/v1beta2. The replacement flowcontrol.apiserver.k8s.io/v1beta3 was implemented in Kubernetes 1.26

Other noteworthy deprecations

Kubernetes beta topology labels on nodes are deprecated and will be removed in a future release. You can follow the list below to see which labels are being replaced:

Changelog for Kubernetes 1.27

Mon, 01 Jan 0001 00:00:00 +0000

Versions

The deployed Kubernetes version varies based on when your cluster is deployed. We try deploying cluster using the latest patch release of Kubernetes.

Current release is Kubernetes 1.27.10

Major changes

Removed API CSIStorageCapacity storage.k8s.io/v1beta1. The replacement storage.k8s.io/v1 was implemented in Kubernetes 1.24

APIs removed in Kubernetes 1.29

More details can be found in Kubernetes official documentation.

Flow control flowcontrol.apiserver.k8s.io/v1beta2. The replacement flowcontrol.apiserver.k8s.io/v1beta3 was implemented in Kubernetes 1.26

Other noteworthy deprecations

Kubernetes beta topology labels on nodes are deprecated and will be removed in a future release. You can follow the list below to see which labels are being replaced:

Changelog for Kubernetes 1.26

Mon, 01 Jan 0001 00:00:00 +0000

Versions

The deployed Kubernetes version varies based on when your cluster is deployed. We try deploying cluster using the latest patch release of Kubernetes.

Current release is Kubernetes 1.26.13

Major changes

Added support for node autoscaling
Removed API Flow control resources flowcontrol.apiserver.k8s.io/v1beta1. The replacement flowcontrol.apiserver.k8s.io/v1beta2 was implemented in Kubernetes 1.23
Removed API HorizontalPodAutoscaler autoscaling/v2beta2. The replacement autoscaling/v2 was introduced in Kubernetes 1.23
We no longer deploy NodeLocal DNSCache for new clusters

Deprecations

Note that all deprecations will be removed in a future Kubernetes release. This does not mean you need to perform any changes right now. However, we recommend you to start migrating your applications in order to avoid issues in future releases.

Changelog for Kubernetes 1.25

Mon, 01 Jan 0001 00:00:00 +0000

Versions

Kubernetes 1.25.6
Nginx-ingress: 1.4.0
Certmanager: 1.11.0

Major changes

Pod Security Policies has been removed.
CronJob API batch/v1beta1 has been removed and is replaced with batch/v1 that was implemented in Kubernetes 1.21
EndpointSlice API discovery.k8s.io/v1beta1 has been removed and is replaced with discovery.k8s.io/v1 that was implemented in Kubernetes 1.21
Event API events.k8s.io/v1beta1 has been removed and is replaced with events.k8s.io/v1 that was implemented in Kubernetes 1.19
PodDisruptionBudget API policy/v1beta1 has been removed and is replaced with policy/v1 that was implemented in Kubernetes 1.21
RuntimeClass API node.k8s.io/v1beta1 has been removed and is replaced with node.k8s.io/v1 that was implemented in Kubernetes 1.20

Deprecations

Note that all deprecations will be removed in a future Kubernetes release. This does not mean you need to perform any changes right now. However, we recommend you to start migrating your applications in order to avoid issues in future releases.

Changelog for Kubernetes 1.24

Mon, 01 Jan 0001 00:00:00 +0000

Versions

Kubernetes 1.24.6
Nginx-ingress: 1.4.0
Certmanager: 1.10.0

Major changes

The node-role.kubernetes.io/master= label is removed from all control plane nodes, instead use the node-role.kubernetes.io/control-plane= label.
The taint node-role.kubernetes.io/control-plane:NoSchedule has been added to all control plane nodes.

Deprecations

Note that all deprecations will be removed in a future Kubernetes release. This does not mean you need to perform any changes right now. However, we recommend you to start migrating your applications in order to avoid issues in future releases.

Changelog for Kubernetes 1.23

Mon, 01 Jan 0001 00:00:00 +0000

Versions

Kubernetes 1.23.7
Nginx-ingress: 1.3.0
Certmanager: 1.9.1

Major changes

A new storage class v1-dynamic-40 is introduced and set as the default storage class. All information about this storage class can be found here.
Worker and control plane nodes now use v1-c2-m8-d80 as their default flavor. You can find a complete list of all available flavors here.
All nodes will be migrated to the updated flavors during the upgrade. All new flavors will have the same specification however the flavor ID will be changed. This affects customers that use the node.kubernetes.io/instance-type label that can be located on nodes.
Control plane nodes will have their disk migrated from the deprecated 4k storage class to v1-dynamic-40.
Starting from Kubernetes 1.23 we will require 3 control plane (masters) nodes.

Flavor mapping

Old flavor	New flavor
v1-standard-2	v1-c2-m8-d80
v1-standard-4	v1-c4-m16-d160
v1-standard-8	v1-c8-m32-d320
v1-dedicated-8	d1-c8-m58-d800
v2-dedicated-8	d2-c8-m120-d1.6k

Changes affecting new clusters:

The storage class 4k will no longer be set-up due to it being deprecated in Openstack. The full announcement can be found here.

What happened to the metrics/monitoring node?

Previously when creating new or upgrading clusters to Kubernetes 1.23 we added an extra node that handled monitoring. This node is no longer needed and all services have been converted to run inside the Kubernetes cluster. This means that clusters being upgraded or created from now on won’t get an extra node added. Clusters that currently have the monitoring node will be migrated to the new setup within the upcoming weeks (The change is non-service affecting).

Changelog for Kubernetes 1.22

Mon, 01 Jan 0001 00:00:00 +0000

Versions

Kubernetes 1.22.8
Nginx-ingress: 1.1.1
Certmanager: 1.6.3

Major changes

When our ingress is installed we set it as the default ingress, meaning it will be used unless a custom ingress class is used/specified
Clusters are now running containerd instead of docker. This should not affect your workload at all
We reserve 5% RAM on all nodes making it easier to calculate how much is left for your workload
All components deployed by Elastx have tolerations for NoSchedule taints by default
Certmanager cert-manager.io/v1alpha2, cert-manager.io/v1alpha3, cert-manager.io/v1beta1, acme.cert-manager.io/v1alpha2, acme.cert-manager.io/v1alpha3 and acme.cert-manager.io/v1beta1 APIs are no longer served. All existing resources will be converted automatically to cert-manager.io/v1 and acme.cert-manager.io/v1, however you will still need to update your local manifests
Several old APIs are no longer served. A complete list can be found in Kubernetes documentation

Changes affecting new clusters:

All new clusters will have the cluster domain cluster.local by default
The encrypted *-enc storage-classes (4k-enc, 8k-enc and 16k-enc) are no longer available to new clusters since they are deprecated for removal in Openstack. Do not worry, all our other storage classes (4k, 8k, 16k and future classes) are now encrypted by default. Read our full announcement here

Deprecations

Note that all deprecations will be removed in a future Kubernetes release. This does not mean you need to perform any changes right now. However, we recommend you to start migrating your applications in order to avoid issues in future releases.

Changelog for Kubernetes 1.21

Mon, 01 Jan 0001 00:00:00 +0000

Versions

Kubernetes 1.21.5
Nginx-ingress: 1.0.1
Certmanager: 1.5.3

Major changes

Load Balancers are by default allowed to talk to all tcp ports on worker nodes.

New Kubernetes features:

The ability to create immutable secrets and configmaps.
Cronjobs are now stable and the new API batch/v1 is implemented.
Graceful node shutdown, when shutting worker nodes this is detected by Kubernetes and pods will be evicted.

Deprecations

Note that all deprecations will be removed in a future Kubernetes release, this does not mean you need to perform any changes now however we recommend you to start migrating your applications to avoid issues in future releases.

Changelog for Kubernetes 1.20

Mon, 01 Jan 0001 00:00:00 +0000

Versions

Kubernetes 1.20.7
Nginx-ingress: 0.46.0
Certmanager: 1.3.1

Major changes

RBAC api rbac.authorization.k8s.io/v1alpha1 has been removed. Instead use the replacement rbac.authorization.k8s.io/v1.
We no longer supports new clusters being created with pod security policy enabled. Instead we recommend using OPA Gatekeeper, in case you have any questions regarding this contact our support and we will help you guys out.
The built-in Cinder Volume Provider has gone from deprecated to disabled. Any volumes that are still using it will have to be migrated, see Known Issues.

Deprecations

Ingress api extensions/v1beta1 will be removed in kubernetes 1.22.
Kubernetes beta lables on nodes are deplricated and will be removed in a future release, follow the below list to se what lable replaces the old one:
- beta.kubernetes.io/instance-type -> node.kubernetes.io/instance-type
- beta.kubernetes.io/arch -> kubernetes.io/arch
- beta.kubernetes.io/os -> kubernetes.io/os
- failure-domain.beta.kubernetes.io/region -> topology.kubernetes.io/region
- failure-domain.beta.kubernetes.io/zone -> topology.kubernetes.io/zone
Certmanager api v1alpha2, v1alpha3 and v1beta1 will be removed in a future release. We strongly recommend that you upgrade to the new v1 api.
RBAC api rbac.authorization.k8s.io/v1beta1 will be removed in an upcoming release. The apis are replaced with rbac.authorization.k8s.io/v1.
Pod Security Policies will be removed in Kubernetes 1.25 in all clusters having the feature enabled. Instead we recommend OPA Gatekeeper.

Is downtime expected

The upgrade drains (moving all workload from) one node at the time, patches that node and brings it back in the cluster. First after all deployments and statefulsets are running again we will continue on with the next node.

Changelog for Kubernetes 1.19

Mon, 01 Jan 0001 00:00:00 +0000

Versions

Kubernetes 1.19.7
Nginx-ingress: 0.43.0
Certmanager: 1.2.0

Major changes

New security groups are implemented where you can store all youre firewall rules. The new security groups will be persistent between upgrades and called CLUSTERNAME-k8s-worker-customer and CLUSTERNAME-k8s-master-customer (CLUSTERNAME will be replaced with actual cluster name). With this change we will remove our previous default firewall rules that allowed public traffic to the Kubernetes cluster, this includes the following services:
- Master API (port 6443)
- Ingress (port 80 & 443)
- Nodeports (ports 30000 to 32676)
If you currently have any of the mentioned ports open you either need to add them to the new security groups (created during the upgrade) or mention this during the planning discussion and we will assist you with this. Please be aware that any rules added to the new security groups is not managed by us and you are responsible for them being up to date.

Deprecations

Ingress api extensions/v1beta1 will be removed in kubernetes 1.22
RBAC api rbac.authorization.k8s.io/v1alpha1 and rbac.authorization.k8s.io/v1beta1 will be removed in kubernetes 1.20. The apis are replaced with rbac.authorization.k8s.io/v1.
The node label beta.kubernetes.io/instance-type will be rmeoved in an uppcomig release. Use node.kubernetes.io/instance-type instead.
Certmanager api v1alpha2, v1alpha3 and v1beta1 will be removed in a future release. We strongly recommend that you upgrade to the new v1 api

Is downtime expected

The upgrade drains (moving all workload from) one node at the time, patches that node and brings in back in the cluster. First after all deployments and statefulsets are running again we will continue on with the next node.

Changelog for Kubernetes 1.18

Mon, 01 Jan 0001 00:00:00 +0000

Versions

Kubernetes 1.18.9
Nginx-ingress: 0.40.0
Certmanager: 1.0.3

Major changes

Moved the tcp-services configmap used by our ingress controller to the default namespace.

Deprecations

Ingress api extensions/v1beta1 will be removed in kubernetes 1.22
RBAC api rbac.authorization.k8s.io/v1alpha1 and rbac.authorization.k8s.io/v1beta1 will be removed in kubernetes 1.20. The apis are replaced with rbac.authorization.k8s.io/v1.
The node label beta.kubernetes.io/instance-type will be rmeoved in an uppcomig release. Use node.kubernetes.io/instance-type instead.
Certmanager api v1alpha2, v1alpha3 and v1beta1 will be removed in a future release. We strongly recommend that you upgrade to the new v1 api
Accessing the Kubernetes dashboard over the Kubernetes API. This feature will not be added to new clusters however if your cluster already has this available it will continue working until Kubernetes 1.19

Removals

Some older deprecated metrics, more information regarding this can be found in the official Kubernetes changelog: Link to Kubernetes changelog

Is downtime expected

For this upgrade we expect a shorter downtime on the ingress. The downtime on the ingress should be no longer than 5 minutes and hopefully even under 1 minute in length.

Changelog for Kubernetes 1.17

Mon, 01 Jan 0001 00:00:00 +0000

Versions

Kubernetes 1.17.9
Nginx-ingress: 0.32.0
Certmanager: 0.15.0

Major changes

We can now combine nodes with multiple different flavors within one cluster
Fixed a bug where some external network connections got stuck (MTU missmatch, calico)
Enabled calicos metric endpoint
New and improved monitoring system
Ingress does only support serving http over port 80 and https over port 443
Cert-manager using new APIs: Cert-manager info

Deprecations

Ingress api extensions/v1beta1 will be removed in kubernetes 1.22
RBAC api rbac.authorization.k8s.io/v1alpha1 and rbac.authorization.k8s.io/v1beta1 will be removed in kubernetes 1.20. The apis are replaced with rbac.authorization.k8s.io/v1.
The node label beta.kubernetes.io/instance-type will be rmeoved in an uppcomig release. Use node.kubernetes.io/instance-type instead.

Removals

Custom ingress ports

We no longer supports using custom ingress ports. From 1.17 http traffic will be received on port 80 and https on port 443