[public-api] Migrate envvarService #19067

jeanp413 · 2023-11-14T06:57:52Z

Description

Summary generated by Copilot

`🤖 Generated by Copilot at 2d8b9cb`

This pull request adds a new public API service for managing user environment variables for workspaces. It defines the service and its messages using Protobuf, generates client and server code for gRPC and Connect in Go and TypeScript, and implements the service handler and client in the server and dashboard components. It also refactors the dashboard component for environment variables to use the new public API service.

Related Issue(s)

Fixes #

How to test

It should be able to CRUD env in org, configuration and user level
Start workspace in org, envs in org + user should be in workspace
Start workspace in org + configuration, envs in org + configuration + user should be in workspace
Create a new org, there should only have user envs in workspace

Documentation

Preview status

Gitpod was successfully deployed to your preview environment.

🏷️ Name - jp-practical-guppy
🔗 URL - jp-practical-guppy.preview.gitpod-dev.com/workspaces.
📚 Documentation - See our internal documentation for information on how to interact with your preview environment.
📦 Version - jp-practical-guppy-gha.20044
🗒️ Logs - GCP Logs Explorer

Build Options

Build

/werft with-werft
Run the build with werft instead of GHA
leeway-no-cache
/werft no-test
Run Leeway with --dont-test

Publish

/werft publish-to-npm
/werft publish-to-jb-marketplace

Installer

analytics=segment
with-dedicated-emulation
workspace-feature-flags
Add desired feature flags to the end of the line above, space separated

Preview Environment / Integration Tests

/werft with-local-preview
If enabled this will build install/preview
/werft with-preview
/werft with-large-vm
/werft with-gce-vm
If enabled this will create the environment on GCE infra
with-integration-tests=all
Valid options are all, workspace, webapp, ide, jetbrains, vscode, ssh. If enabled, with-preview and with-large-vm will be enabled.
with-monitoring

/hold

components/public-api/gitpod/v1/envvar.proto

components/server/src/api/envvar-service-api.ts

components/public-api/gitpod/v1/envvar.proto

jeanp413 · 2023-11-16T14:43:26Z

@akosyakov could you review again 🙏

akosyakov · 2023-11-16T14:44:39Z

components/public-api/gitpod/v1/envvar.proto

+message ResolveWorkspaceEnvironmentVariablesRequest { string workspace_id = 1; }
+
+message ResolveWorkspaceEnvironmentVariablesResponse {
+  message EnvironmentVariable {


generally looks good, not sure about it, maybe it is find to have it on global level?

akosyakov · 2023-11-16T14:45:29Z

@jeanp413 it looks ok, you can consider moving proto file and generated classes in another PR, to make this PR small, I will approve it to unblock.

jeanp413 · 2023-11-16T20:14:13Z

@akosyakov moved the proto files to another PR #19085

jeanp413 · 2023-11-17T06:15:30Z

/gh run recreate-vm

Comment triggered a workflow run

Started workflow run: 6900579887

recreate_vm: true

jeanp413 · 2023-11-17T06:48:23Z

Tested and it's working

akosyakov · 2023-11-17T08:21:18Z

@jeanp413 open for a review?

akosyakov · 2023-11-17T09:31:07Z

components/server/src/api/envvar-service-api.ts

+            repositoryPattern: "",
+        };
+
+        await this.envVarService.deleteUserEnvVar(context.user.id, context.user.id, variable);


why is not enough to pass id?

it's needed for old api resource guard validation

yeah, ignore my noise on gRPC implementation here, when we get rid of JSON-RPC we can simplify, don't want to block with it

Already updated a couple 😅

akosyakov

I left comment on gRPC implementation. I think at least we should fix error propagation, logic looks unnecessary complicated, but it seems because of underlying application service design. I wonder whether we can improve it, if it is too much work let's go with what we have for now.

components/dashboard/src/service/json-rpc-envvar-client.ts

components/dashboard/src/data/projects/set-project-env-var-mutation.ts

components/dashboard/src/service/json-rpc-envvar-client.ts

components/server/src/api/envvar-service-api.ts

mustard-mh · 2023-11-20T13:35:53Z

Add some nit changes cd19ba3

❌ Failed to update user env, checked that list API has respond with id. Going to update code

components/gitpod-db/src/typeorm/user-db-impl.ts

mustard-mh · 2023-11-20T14:00:31Z

I tried to create workspace with another ORG, it seems new workspace still includes previous ORG envs (the same repo)

jeanp413 · 2023-11-20T14:06:39Z

org env? you mean user env or project env?

mustard-mh · 2023-11-20T14:08:25Z

org env? you mean user env or project env?

I mean org project env.

Seems something wrong in our code.

Project 0c893085-6a88-4718-b5e9-4fddc52b5ae3 below to org d319b06f-3a73-4595-a341-ab63ff0fd65c. But I can access with a workspace with org e6cfe015-326a-40d7-95a2-319f1994e051 (which has it's context with the same projectId like previous one)

mysql> select * from d_b_workspace where id='mustardmh-templatesvelt-5vcqsl66if7' \G;
*************************** 1. row ***************************
                   id: mustardmh-templatesvelt-5vcqsl66if7
         creationTime: 2023-11-20T13:58:03.696Z
              ownerId: f22cb39c-b108-4651-9212-c3bca4d391b6
           contextURL: var=value,var2=value2/https://github.com/mustard-mh/template-sveltejs
          description: mustard-mh/template-sveltejs - master
              context: *****
               config: *****
          imageSource: {"baseImageResolved":"docker.io/gitpod/workspace-full:latest"}
    imageNameResolved: eu.gcr.io/gitpod-core-dev/build/workspace-images:d1ce6d3571f4a5aca988999226fcac22f3b72326ab2becf9b0e80f561ed8395a
             archived: 0
            shareable: 0
        _lastModified: 2023-11-20 13:58:16.835081
              deleted: 0
                 type: regular
baseImageNameResolved: docker.io/gitpod/workspace-full@sha256:850314e6963b47c4e5d3ca18abe6345b435fa3423a735e7d94f93badf4502700
          softDeleted: NULL
               pinned: 0
      softDeletedTime: 
   contentDeletedTime: 
    basedOnPrebuildId: 
    basedOnSnapshotId: 
            projectId: 0c893085-6a88-4718-b5e9-4fddc52b5ae3
             cloneURL: https://github.com/mustard-mh/template-sveltejs.git
       organizationId: e6cfe015-326a-40d7-95a2-319f1994e051
1 row in set (0.28 sec)

ERROR: 
No query specified

mysql> select * from d_b_project_env_var \G;
*************************** 1. row ***************************
           id: 4c70516b-a522-4afe-91c9-7524c53bb5e6
    projectId: 0c893085-6a88-4718-b5e9-4fddc52b5ae3
         name: ORG_VAR_1
        value: *****
     censored: 1
 creationTime: 2023-11-20T13:26:27.858Z
      deleted: 0
_lastModified: 2023-11-20 13:26:27.860595
*************************** 2. row ***************************
           id: 9a6c52fa-d4d6-4259-9048-047894861b04
    projectId: 0c893085-6a88-4718-b5e9-4fddc52b5ae3
         name: ORG_VAR_2_VISIABLE
        value: *****
     censored: 0
 creationTime: 2023-11-20T13:27:08.348Z
      deleted: 0
_lastModified: 2023-11-20 13:27:08.352197
2 rows in set (0.48 sec)

ERROR: 
No query specified

mysql> select * from d_b_project \G;
*************************** 1. row ***************************
               id: 0c893085-6a88-4718-b5e9-4fddc52b5ae3
             name: template-sveltejs
         cloneUrl: https://github.com/mustard-mh/template-sveltejs.git
           teamId: d319b06f-3a73-4595-a341-ab63ff0fd65c
appInstallationId: 
     creationTime: 2023-11-20T13:26:04.970Z
    _lastModified: 2023-11-20 13:26:04.973654
    markedDeleted: 0
           userId: NULL
             slug: NULL
         settings: *****
1 row in set (0.42 sec)

ERROR: 
No query specified

mysql>

mustard-mh · 2023-11-20T14:11:24Z

But ☝️ not relates to this PR's changes, tested all good.

mustard-mh · 2023-11-20T14:12:53Z

Let's wait preview env to validate updateUserEnv fixing. Once it's ok, we are good to go

jeanp413 · 2023-11-20T14:13:09Z

But ☝️ not relates to this PR's changes

😆

components/dashboard/src/service/json-rpc-envvar-client.ts

mustard-mh · 2023-11-20T14:20:06Z

components/server/src/workspace/gitpod-server-impl.ts

@@ -1830,11 +1830,11 @@ export class GitpodServerImpl implements GitpodServerWithTracing, Disposable {
        const user = await this.checkAndBlockUser("setEnvVar");
        const userEnvVars = await this.envVarService.listUserEnvVars(user.id, user.id);
        if (userEnvVars.find((v) => v.name == variable.name && v.repositoryPattern == variable.repositoryPattern)) {
-            return this.envVarService.updateUserEnvVar(user.id, user.id, variable, (envvar: UserEnvVar) => {
+            await this.envVarService.updateUserEnvVar(user.id, user.id, variable, (envvar: UserEnvVar) => {


If we update userEnv with value and scope changed, server will call addUserEnvVar even we have envId passed.

Not blocker as it's just like before 😌

yes, intended

mustard-mh

Tested all good.

Bugs [1][2] that not relates to this PR will create follow-up issues ✅

roboquat added do-not-merge/hold do-not-merge/work-in-progress size/XXL labels Nov 14, 2023

akosyakov reviewed Nov 14, 2023

View reviewed changes

components/public-api/gitpod/v1/envvar.proto Outdated Show resolved Hide resolved

akosyakov reviewed Nov 14, 2023

View reviewed changes

components/public-api/gitpod/v1/envvar.proto Outdated Show resolved Hide resolved

akosyakov reviewed Nov 14, 2023

View reviewed changes

components/public-api/gitpod/v1/envvar.proto Outdated Show resolved Hide resolved

akosyakov reviewed Nov 14, 2023

View reviewed changes

components/public-api/gitpod/v1/envvar.proto Outdated Show resolved Hide resolved

akosyakov reviewed Nov 14, 2023

View reviewed changes

components/public-api/gitpod/v1/envvar.proto Outdated Show resolved Hide resolved

akosyakov reviewed Nov 14, 2023

View reviewed changes

components/public-api/gitpod/v1/envvar.proto Outdated Show resolved Hide resolved

jeanp413 force-pushed the jp/practical-guppy branch from be9c16d to a667faa Compare November 14, 2023 16:00

jeanp413 commented Nov 15, 2023

View reviewed changes

components/server/src/api/envvar-service-api.ts Outdated Show resolved Hide resolved

jeanp413 commented Nov 15, 2023

View reviewed changes

components/server/src/api/envvar-service-api.ts Outdated Show resolved Hide resolved

akosyakov reviewed Nov 15, 2023

View reviewed changes

components/public-api/gitpod/v1/envvar.proto Outdated Show resolved Hide resolved

akosyakov reviewed Nov 15, 2023

View reviewed changes

components/public-api/gitpod/v1/envvar.proto Outdated Show resolved Hide resolved

akosyakov reviewed Nov 15, 2023

View reviewed changes

components/public-api/gitpod/v1/envvar.proto Outdated Show resolved Hide resolved

akosyakov reviewed Nov 15, 2023

View reviewed changes

components/public-api/gitpod/v1/envvar.proto Outdated Show resolved Hide resolved

akosyakov reviewed Nov 15, 2023

View reviewed changes

components/public-api/gitpod/v1/envvar.proto Outdated Show resolved Hide resolved

components/public-api/gitpod/v1/envvar.proto Outdated Show resolved Hide resolved

svenefftinge reviewed Nov 15, 2023

View reviewed changes

components/public-api/gitpod/v1/envvar.proto Outdated Show resolved Hide resolved

svenefftinge reviewed Nov 15, 2023

View reviewed changes

components/public-api/gitpod/v1/envvar.proto Outdated Show resolved Hide resolved

akosyakov reviewed Nov 16, 2023

View reviewed changes

jeanp413 mentioned this pull request Nov 16, 2023

[public-api] Add envvar proto file #19085

Merged

15 tasks

jeanp413 marked this pull request as ready for review November 17, 2023 08:51

jeanp413 requested a review from a team as a code owner November 17, 2023 08:51

roboquat removed the do-not-merge/work-in-progress label Nov 17, 2023

akosyakov reviewed Nov 17, 2023

View reviewed changes

components/dashboard/src/service/json-rpc-envvar-client.ts Outdated Show resolved Hide resolved

akosyakov reviewed Nov 17, 2023

View reviewed changes

components/dashboard/src/data/projects/set-project-env-var-mutation.ts Show resolved Hide resolved

mustard-mh self-requested a review November 17, 2023 09:46

mustard-mh reviewed Nov 17, 2023

View reviewed changes

components/dashboard/src/service/json-rpc-envvar-client.ts Show resolved Hide resolved

mustard-mh reviewed Nov 17, 2023

View reviewed changes

components/server/src/api/envvar-service-api.ts Outdated Show resolved Hide resolved

jeanp413 force-pushed the jp/practical-guppy branch from 3e2ec68 to 81d1654 Compare November 20, 2023 12:30

jeanp413 commented Nov 20, 2023

View reviewed changes

components/gitpod-db/src/typeorm/user-db-impl.ts Show resolved Hide resolved

mustard-mh reviewed Nov 20, 2023

View reviewed changes

components/dashboard/src/service/json-rpc-envvar-client.ts Outdated Show resolved Hide resolved

mustard-mh reviewed Nov 20, 2023

View reviewed changes

mustard-mh approved these changes Nov 20, 2023

View reviewed changes

jeanp413 and others added 7 commits November 20, 2023 14:55

Migrate envvarService

c54ca42

update

5e49165

fix tests

7e918f8

nit

64fcf2c

Fix

7dcf827

Fix

316853b

💄

47317a8

jeanp413 force-pushed the jp/practical-guppy branch from eae9783 to 47317a8 Compare November 20, 2023 15:05

jeanp413 removed the do-not-merge/hold label Nov 20, 2023

roboquat merged commit 4ac04ee into main Nov 20, 2023

roboquat deleted the jp/practical-guppy branch November 20, 2023 15:27

[public-api] Migrate envvarService #19067

[public-api] Migrate envvarService #19067

Uh oh!

Conversation

jeanp413 commented Nov 14, 2023 • edited by mustard-mh Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

🤖 Generated by Copilot at 2d8b9cb

Related Issue(s)

How to test

Documentation

Preview status

Build Options

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jeanp413 commented Nov 16, 2023

Uh oh!

akosyakov Nov 16, 2023

Choose a reason for hiding this comment

Uh oh!

akosyakov commented Nov 16, 2023

Uh oh!

jeanp413 commented Nov 16, 2023

Uh oh!

jeanp413 commented Nov 17, 2023 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comment triggered a workflow run

Uh oh!

jeanp413 commented Nov 17, 2023

Uh oh!

akosyakov commented Nov 17, 2023

Uh oh!

akosyakov Nov 17, 2023

Choose a reason for hiding this comment

Uh oh!

jeanp413 Nov 17, 2023

Choose a reason for hiding this comment

Uh oh!

akosyakov Nov 17, 2023

Choose a reason for hiding this comment

Uh oh!

jeanp413 Nov 17, 2023

Choose a reason for hiding this comment

Uh oh!

akosyakov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mustard-mh commented Nov 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

mustard-mh commented Nov 20, 2023

Uh oh!

jeanp413 commented Nov 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mustard-mh commented Nov 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mustard-mh commented Nov 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

jeanp413 commented Nov 14, 2023 •

edited by mustard-mh

Loading

`🤖 Generated by Copilot at 2d8b9cb`

jeanp413 commented Nov 17, 2023 •

edited by github-actions bot

Loading

mustard-mh commented Nov 20, 2023 •

edited

Loading

jeanp413 commented Nov 20, 2023 •

edited

Loading

mustard-mh commented Nov 20, 2023 •

edited

Loading

mustard-mh commented Nov 20, 2023 •

edited

Loading

jeanp413 Nov 20, 2023 •

edited

Loading

mustard-mh left a comment •

edited

Loading