Repository navigation
Conversation
When a Cluster refers to an ObjectStore that does not exist yet, the pre-reconcile hook asked for a requeue without a delay. The operator maps that to a stopped reconciliation with RequeueAfter 0, which controller-runtime does not requeue, so the Cluster never got any instance until something changed it. This happens when both objects are applied together and the Cluster lands first, as with Helm or a GitOps tool. Set RequeueAfter to 10 seconds in that case, and add a unit test for the Pre hook and an e2e test that creates the Cluster before its ObjectStore. Closes cloudnative-pg#1236 Signed-off-by: Wan Nur Shafiqah <[email protected]>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1236
When a Cluster refers to an ObjectStore that does not exist yet, the pre-reconcile hook returns
BEHAVIOR_REQUEUEwith noRequeueAfter. The operator turns that intoStopReconciliation: truewithctrl.Result{RequeueAfter: 0}, which controller-runtime does not requeue, so the Cluster stays without instances until something changes it.This happens whenever both objects are applied together and the Cluster lands first, for example from a Helm chart (unknown kinds are sorted alphabetically) or a GitOps tool applying one directory.
Change
The hook now sets
RequeueAfterto 10 seconds when the ObjectStore is not found. It is the onlyBEHAVIOR_REQUEUEpath in the plugin.Tests
internal/cnpgi/operator/reconciler_test.go):Prereturns a requeue with a non-zero delay when the ObjectStore is missing, continues once it exists, and continues for kinds other than Cluster. The first test fails with the delay set back to 0.test/e2e/internal/tests/objectstoreorder): creates the Cluster, checks that no instance appears for 30 s, creates the ObjectStore, then waits for the Cluster to become ready without touching it and checks that its generation is unchanged.I ran the new e2e spec on k3s v1.35 with CloudNativePG 1.30.1, against the released plugin and against an image built from this branch:
With v0.16.0 the operator logs
Pre-reconcile hook stopped the reconciliation loopwith"Requeue":false,"RequeueAfter":0and nothing follows, as described in the issue.golangci-lintv2.15.0 reports 0 issues and the unit tests pass. I have not run the full e2e suite locally.