[server] Introduce RequestContext

[geropl]

Description

Context:

• Project: https://linear.app/gitpod/project/fga-compatible-api-tokens-a27f6bdb53d2
• Project Spec: https://www.notion.so/gitpod/FGA-compatible-API-Tokens-Project-Spec-7d47f66fab4c471da6429a90dfbd45e7

Introduces RequestContext as primary method to pass request-local information between different pieces of code, incl. for authorization and request-local caching. This PR makes sure all code that calls some ...Service "request-handling" logic in the end is wrapped inside a runWithRequestContext call:

• websocket connections
• HTTP requests to various express apps
• gRPC services
  This also means that whenever we execute code with other permissions, we need to call runWithSubjectId (in the future). See the call sites in this PR for examples.

To access the request based context, the unit exports a number of functions prefixed by ctx..., e.g.:

• ctxUserId
• ctxCheckAborted
  The idea for those is to be used sparingly, and almost exclusively in "leaf" code (authorizer, logging, tracing, etc.).

To pass authentication information independent of the User/userId string, a new SubjectId type is introduced. Currently there is only one SubjectIdKind ("user"), which serializes to: user_<id>.
This is expected to be extended by a kind apitokenv0_<id> in subsequent PRs.

Follow-up work

• add metrics to measure that we added RequestContexts everywhere
• switch authorization from "first argument (userId)" to ctx.subjectId. Not passing that will be an "access denied" error.
• In consequence this means that in future PRs, we are going to remove the first argument userId: string that we are currently passing into ...Services and down to the Authorizer. For now, in order to allow to split up work, and not interfere too much with other API-related work, we still pass that argument in the new top-level code (gRPC servicve implementations) using ctxUserId.

I will send out additional information about the future steps summarized here, and the expected changes to our code (patterns) today.

Summary generated by Copilot

🤖 Generated by Copilot at c3bde8b

This pull request introduces a new way of identifying users and organizations in the analytics system using subject ids, which are more reliable and consistent than user ids. It also refactors the request context mechanism to use subject ids and provide more information about the requests and the users. It updates various components and files to use the new request context and subject id features.

Related Issue(s)

Fixes EXP-915

How to test

For this PR, we need to make sure we don't break existing things.
Most interesting for review:

• Authorizer
• SpiceDB authorizer
• run all tests (CI)
• all changes the new gRPC code!

For testing:

• check that logging still works as expected (and carries the additional metadata)
• verify the changes to

Documentation

Preview status

Gitpod was successfully deployed to your preview environment.

• 🏷️ Name - gpl-request-context
• 🔗 URL - gpl-request-context.preview.gitpod-dev.com/workspaces.
• 📚 Documentation - See our internal documentation for information on how to interact with your preview environment.
• 📦 Version - gpl-request-context-gha.19488
• 🗒️ Logs - GCP Logs Explorer

Build Options

Build

• /werft with-werft
  Run the build with werft instead of GHA
• leeway-no-cache
• /werft no-test
  Run Leeway with --dont-test

Publish

• /werft publish-to-npm
• /werft publish-to-jb-marketplace

Installer

• analytics=log
• with-dedicated-emulation
• workspace-feature-flags
  Add desired feature flags to the end of the line above, space separated

Preview Environment / Integration Tests

• /werft with-local-preview
  If enabled this will build install/preview
• /werft with-preview
• /werft with-large-vm
• /werft with-gce-vm
  If enabled this will create the environment on GCE infra
• with-integration-tests=all
  Valid options are all, workspace, webapp, ide, jetbrains, vscode, ssh. If enabled, with-preview and with-large-vm will be enabled.
• with-monitoring

/hold

[akosyakov]

I skimmed through the code it looks reasonable. cc @svenefftinge to have a look as well

[geropl]

it is addressed now, right? Or misunderstand something.

I think we call auth only once before creating a generator

Yes, I pushed an attempt to fix it, by making auth an explicit, separate step. 👍

[geropl]

@svenefftinge @akosyakov I addressed all the things mentioned above/we discussed. The current state should be complete now and deployed.

@AlexTugarev It would be great if you could review of the AuthProviderService changes.

@AlexTugarev @mustard-mh For reading the description, and getting acquainted with the changes to APIs (e.g. not using HandlerContext, but ctxUserId - but only until we drop the need for that).

Please have a close look at the description for what this PR does and NOT does.

[mustard-mh]

AlexTugarev mustard-mh For reading the description, and getting acquainted with the changes to APIs (e.g. not using HandlerContext, but ctxUserId - but only until we drop the need for that). [ctx]

cc @jeanp413 since you have onboard grpc migration too

[akosyakov]

@geropl Could you rebase please? 🙏

[geropl]

@geropl Could you rebase please? 🙏

@akosyakov Done (again 🏃‍♂️ 😉 ). Build is still running.

[akosyakov]

@geropl I've enabled debugging logging gpctl debug log server but logs look different I would expect to see requestId and so on but getting
{"component":"server","severity":"DEBUG","time":"2023-11-15T15:08:00.065Z","message":"public api: done","payload":{"grpc_code":"Canceled"}}

[akosyakov]

Interesting I never see [redis] Succesfully handled update (debug level), but maybe i do something wrong.

[geropl]

Yeah, just found a bug in the redis handler. Pushing a fix... 🙄

[geropl]

I've enabled debugging logging gpctl debug log server but logs look different I would expect to see requestId and so on but getting

Uh, I think log-context.ts is not imported anywhere anymore 😆

[geropl]

@akosyakov The fixes for the above are pushed and deployed.

[akosyakov]

I could not test analytics, the rest looks good.

Please test analytics if it is alright for all kind of calls, please unhold.

/hold

[geropl]

Tested analytics and it works. ✔️

/unhold

[geropl]

/unhold

[AlexTugarev]

@geropl, just realized, this actually throws if unauthenticated.

Need to address it, as it breaks Login page with gRPC enabled.

[AlexTugarev]

#19094

[geropl]

@AlexTugarev Yes, this one throws. If we need one that does not throw, we should name it ctxTryUserId, following the pattern in request-context.ts 👍