Skip to content

Avoid ws-manager-mk2 metrics duplication #18625

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 4, 2023
Merged

Avoid ws-manager-mk2 metrics duplication #18625

merged 1 commit into from
Sep 4, 2023

Conversation

aledbf
Copy link
Member

@aledbf aledbf commented Aug 30, 2023

Description

Only the leader replica maintains the workspace metrics.

Summary generated by Copilot

🤖 Generated by Copilot at 2e1c129

Improve leader election and workspace reconciler logic in ws-manager-mk2. Delay the reconciler initialization until the leader is elected and run it in a separate goroutine.

Related Issue(s)

Fixes #

How to test

Documentation

Preview status

Gitpod was successfully deployed to your preview environment.

Build Options

Build
  • /werft with-werft
    Run the build with werft instead of GHA
  • leeway-no-cache
  • /werft no-test
    Run Leeway with --dont-test
Publish
  • /werft publish-to-npm
  • /werft publish-to-jb-marketplace
Installer
  • analytics=segment
  • with-dedicated-emulation
  • workspace-feature-flags
    Add desired feature flags to the end of the line above, space separated
Preview Environment / Integration Tests
  • /werft with-local-preview
    If enabled this will build install/preview
  • /werft with-preview
  • /werft with-large-vm
  • /werft with-gce-vm
    If enabled this will create the environment on GCE infra
  • with-integration-tests=workspace
    Valid options are all, workspace, webapp, ide, jetbrains, vscode, ssh. If enabled, with-preview and with-large-vm will be enabled.
  • with-monitoring

/hold

Copy link
Member

@easyCZ easyCZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would be the cost/implication to extend the metrics to include leader=true/false instead of this change? It would allow us to observe the non-master as well as the master, and avoids surprises for future changes.

@aledbf
Copy link
Member Author

aledbf commented Aug 31, 2023

What would be the cost/implication to extend the metrics to include leader=true/false instead of this change? It would allow us to observe the non-master as well as the master, and avoids surprises for future changes.

Only the leader process reconciles objects. Not sure that change makes sense.

That said, that will introduce an undesired behavior; the leader will set leader=true, and the N standby replicas leader=false. When a new leader is elected (previously a standby replica), it will contain both leader true and false metrics.

@aledbf aledbf requested a review from easyCZ September 1, 2023 15:09
@easyCZ
Copy link
Member

easyCZ commented Sep 4, 2023

hat said, that will introduce an undesired behavior; the leader will set leader=true, and the N standby replicas leader=false. When a new leader is elected (previously a standby replica), it will contain both leader true and false metrics.

Yep, it would contain the metrics again. But dashboards/alerts could use leader=true to extract the relevant ones. I guess the main question I'm asking is whether you may need the non-leader replica metrics, or not.

@aledbf
Copy link
Member Author

aledbf commented Sep 4, 2023

Yep, it would contain the metrics again. But dashboards/alerts could use leader=true to extract the relevant ones. I guess the main question I'm asking is whether you may need the non-leader replica metrics, or not.

We already have non-leader replica metrics that make sense for the generic parts (provided by controller-runtime)

@roboquat roboquat merged commit 887a8b5 into main Sep 4, 2023
@roboquat roboquat deleted the aledbf/metrics branch September 4, 2023 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants