Skip to content

[wip] SpiceDB reconnect behavior #18570

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 24, 2023
Merged

[wip] SpiceDB reconnect behavior #18570

merged 1 commit into from
Aug 24, 2023

Conversation

geropl
Copy link
Member

@geropl geropl commented Aug 22, 2023

Description

Before, server waited at least 120s before establishing a gRPC connection to SpiceDB again.
This PR:

  • upgrades the SpiceDB client to 0.12.1 which exposes gRPC Client- and ChannelOptions
  • "manually" reconnects on-request if the channel is not "READY" (translates to: not subchannel is ready) (source)
    • we do sth similar for ws-manager, for instance (source)
  • adds a deadline of 8s to each call against SpiceDB to ensure we are not starving
  • make the ReadinessProbe on the pods more responsive: before, if initialization was too slow, we would only check readiness each 30s
  • also, it adds ClientCallMetrics as we have them for other grpc clients: f3fe53e

ℹ️ SpiceDB config is required now, and server fails to start if there is no config in the env vars.

Summary generated by Copilot

🤖 Generated by Copilot at 57c33a4

Add performance logging to SpiceDB client calls in spicedb-authorizer.ts. This helps to measure and optimize the latency of authorization checks and updates.

Related Issue(s)

Fixes EXP-379

How to test

  • open a workspace on this PR
  • signup, and prepare to create a new workspace
  • kubectl delete pod spicedb-...
  • hit "create workspace"
    • the first 1-2 tries should return "not found" errors ✔️
    • after ~20s the workspace starts with some delay ✔️

Documentation

Preview status

Gitpod was successfully deployed to your preview environment.

Build Options

Build
  • /werft with-werft
    Run the build with werft instead of GHA
  • leeway-no-cache
  • /werft no-test
    Run Leeway with --dont-test
Publish
  • /werft publish-to-npm
  • /werft publish-to-jb-marketplace
Installer
  • analytics=segment
  • with-dedicated-emulation
  • workspace-feature-flags
    Add desired feature flags to the end of the line above, space separated
Preview Environment / Integration Tests
  • /werft with-local-preview
    If enabled this will build install/preview
  • /werft with-preview
  • /werft with-large-vm
  • /werft with-gce-vm
    If enabled this will create the environment on GCE infra
  • with-integration-tests=all
    Valid options are all, workspace, webapp, ide, jetbrains, vscode, ssh. If enabled, with-preview and with-large-vm will be enabled.
  • with-monitoring

/hold

@socket-security
Copy link

socket-security bot commented Aug 22, 2023

New and updated dependencies detected. Learn more about Socket for GitHub ↗︎

Packages Version New capabilities Transitives Size Publisher
@grpc/grpc-js 1.6.7 None +0 1.46 MB murgatroid99
@authzed/authzed-node 0.10.0...0.12.1 None +0/-2 2.51 MB authzednpm

@geropl geropl force-pushed the gpl/fga-reconnect branch 2 times, most recently from 261329a to 6ed8284 Compare August 24, 2023 08:38
@roboquat roboquat added size/XL and removed size/L labels Aug 24, 2023
@geropl geropl force-pushed the gpl/fga-reconnect branch from 8ef2955 to f3fe53e Compare August 24, 2023 09:58
@@ -26,10 +28,6 @@ export class SpiceDBAuthorizer {
userId: string;
},
): Promise<boolean> {
if (!this.client) {
return true;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was really confusing in testing 🙈
And as we now require SpiceDB in every environment, we should remove it alltogether.

.toDynamicValue((ctx) => {
const config = spiceDBConfigFromEnv();
if (!config) {
throw new Error("[spicedb] Missing configuration expected in env vars!");
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We error now if the configuration is incomplete.

@geropl geropl force-pushed the gpl/fga-reconnect branch from f3fe53e to 6a91d0c Compare August 24, 2023 11:54
@geropl geropl marked this pull request as ready for review August 24, 2023 12:05
@geropl geropl requested a review from a team as a code owner August 24, 2023 12:05
@geropl geropl marked this pull request as draft August 24, 2023 12:05
@geropl
Copy link
Member Author

geropl commented Aug 24, 2023

Rebasing the PR 🙄

@geropl geropl force-pushed the gpl/fga-reconnect branch from 6a91d0c to 4f0751f Compare August 24, 2023 12:10
@roboquat roboquat added size/L and removed size/XL labels Aug 24, 2023

export function createInitializingAuthorizer(spiceDbAuthorizer: SpiceDBAuthorizer): Authorizer {
const target = new Authorizer(spiceDbAuthorizer);
const initialized = (async () => {
await target.addInstallationAdminRole(BUILTIN_INSTLLATION_ADMIN_USER_ID);
await target.addUser(BUILTIN_INSTLLATION_ADMIN_USER_ID);
})();
})().catch((err) => log.error("Failed to initialize authorizer", err));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this would hide the error and only log. Is that intentional?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoops, good catch.

const timer = spicedbClientLatency.startTimer();
let error: Error | undefined;
try {
const existing = await this.client.readRelationships(v1.ReadRelationshipsRequest.create(req));
const client = this.clientProvider.getClient();
Copy link
Member

@svenefftinge svenefftinge Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could extract this into

private get client() {
   return this.clientProvider.getClient();
}

@geropl geropl marked this pull request as ready for review August 24, 2023 13:02
@geropl
Copy link
Member Author

geropl commented Aug 24, 2023

/unhold

@roboquat roboquat merged commit a8a67cc into main Aug 24, 2023
@roboquat roboquat deleted the gpl/fga-reconnect branch August 24, 2023 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants