docs(ske): warn that load balancers block cluster deletion - #1647
Open
FabianHardt wants to merge 1 commit into
Open
docs(ske): warn that load balancers block cluster deletion#1647FabianHardt wants to merge 1 commit into
FabianHardt wants to merge 1 commit into
Conversation
A Service of type LoadBalancer makes the cloud controller create a load balancer in the project. It belongs to no Terraform state, so destroy removes the cluster and nobody removes the load balancer - the shoot then waits for it, and the cluster stays in STATE_DELETING until this resource times out after 90 minutes. Nothing points at the cause today. The state is indistinguishable from "slow", and the error after the timeout is "Cluster deletion waiting: ..." without a hint. In our case that cost an hour before we found the load balancer still sitting in STATUS_READY. This adds a note to the resource description and names the likely cause in the timeout error, including the command to check for leftovers. It does not change behaviour - the underlying issue is tracked in stackitcloud#1646, where the service side would either clean the load balancers up or expose what the deletion is waiting for. Signed-off-by: Fabian Hardt <fabian.hardt@opitz-consulting.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documentation and error message only — no behaviour change. Companion to #1646.
Problem
A
Serviceof typeLoadBalancermakes the cloud controller create a loadbalancer in the project. It belongs to no Terraform state, so
terraform destroyremoves the cluster and nobody removes the load balancer. The shootwaits for it, and the cluster stays in
STATE_DELETINGuntilDeleteClusterWaitHandlergives up after 90 minutes.Nothing points at the cause.
STATE_DELETINGis indistinguishable from "slow",and the error after the timeout is:
In our case that cost about an hour before we found the load balancer still in
STATUS_READYnext to a cluster that would not go away.Change
A note on the
stackit_ske_clusterresource description, rendered as awarning callout:
The timeout error now names the likely cause and the command to check:
docs/resources/ske_cluster.mdwas regenerated withscripts/tfplugindocs.sh;only that file changed.
Deliberately not doing
Deleting the load balancers from the provider.
stackit_ske_clusternevercreated them, and working out which ones belong to a given cluster would mean
deleting resources the provider does not manage — in a project with a second
cluster or a manually created load balancer that would be actively harmful.
The real fix belongs on the service side and is filed as #1646: either the
shoot cleans up what it caused, or the cluster status says what it is waiting
for. This PR only shortens the road to that knowledge in the meantime.
Checks