Skip to content

Commit 86080b8

Browse files
author
Saketram Durbha
authored
refactor Cloud SQL samples (#3952)
* refactor Cloud SQL PostgreSQL & MySQL samples * Rename environment variable DB_SOCKET_PATH to DB_SOCKET_DIR for better description of its contents * Move region tags and refactor code so as to give users a better sample experience * minor code style and performance updates for Cloud SQL PostgreSQL and MySQL samples * make necessary environment variables such as "DB_USER" required * add comments to show examples for some input arguments * rewrite small piece of code to be more condensed and concise * refactor Cloud SQL SQL Server samples * use environment varialbe for proxy connection hostname over hardcoding it * make all environment variables required * include instructions in README for running locally on Windows * make code structure consistent with PostgreSQL and MySQL samples * fix Cloud SQL MySQL & PostgreSQL READMEs to remove unneecessary instructions * replace references in Cloud SQL samples to Key Management Service to Secret Manager * fix typo in Cloud SQL Postgres sample and refactor SQL Server sample for consistency across Cloud SQL samples
1 parent 15fede4 commit 86080b8

File tree

11 files changed

+210
-165
lines changed

11 files changed

+210
-165
lines changed

‎cloud-sql/mysql/sqlalchemy/README.md

Lines changed: 11 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -17,17 +17,6 @@ name.
1717
[instructions](https://cloud.google.com/sql/docs/mysql/connect-external-app#4_if_required_by_your_authentication_method_create_a_service_account).
1818
Download a JSON key to use to authenticate your connection.
1919

20-
1. Use the information noted in the previous steps:
21-
```bash
22-
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service/account/key.json
23-
export CLOUD_SQL_CONNECTION_NAME='<MY-PROJECT>:<INSTANCE-REGION>:<INSTANCE-NAME>'
24-
export DB_USER='my-db-user'
25-
export DB_PASS='my-db-pass'
26-
export DB_NAME='my_db'
27-
```
28-
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
29-
secure solution such as [Cloud KMS](https://cloud.google.com/kms/) to help keep secrets safe.
30-
3120
## Running locally
3221

3322
To run this application locally, download and install the `cloud_sql_proxy` by
@@ -52,6 +41,9 @@ export DB_USER='<DB_USER_NAME>'
5241
export DB_PASS='<DB_PASSWORD>'
5342
export DB_NAME='<DB_NAME>'
5443
```
44+
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
45+
secure solution such as [Secret Manager](https://cloud.google.com/secret-manager/docs/overview) to
46+
help keep secrets safe.
5547

5648
Then use this command to launch the proxy in the background:
5749
```bash
@@ -67,6 +59,9 @@ $env:DB_USER="<DB_USER_NAME>"
6759
$env:DB_PASS="<DB_PASSWORD>"
6860
$env:DB_NAME="<DB_NAME>"
6961
```
62+
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
63+
secure solution such as [Secret Manager](https://cloud.google.com/secret-manager/docs/overview) to
64+
help keep secrets safe.
7065

7166
Then use this command to launch the proxy in a separate PowerShell session:
7267
```powershell
@@ -87,7 +82,7 @@ sudo chown -R $USER /cloudsql
8782

8883
You'll also need to initialize an environment variable containing the directory you just created:
8984
```bash
90-
export DB_SOCKET_PATH=/path/to/the/new/directory
85+
export DB_SOCKET_DIR=/path/to/the/new/directory
9186
```
9287

9388
Use these terminal commands to initialize other environment variables as well:
@@ -98,10 +93,13 @@ export DB_USER='<DB_USER_NAME>'
9893
export DB_PASS='<DB_PASSWORD>'
9994
export DB_NAME='<DB_NAME>'
10095
```
96+
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
97+
secure solution such as [Secret Manager](https://cloud.google.com/secret-manager/docs/overview) to
98+
help keep secrets safe.
10199

102100
Then use this command to launch the proxy in the background:
103101
```bash
104-
./cloud_sql_proxy -dir=$DB_SOCKET_PATH --instances=$INSTANCE_CONNECTION_NAME --credential_file=$GOOGLE_APPLICATION_CREDENTIALS &
102+
./cloud_sql_proxy -dir=$DB_SOCKET_DIR --instances=$INSTANCE_CONNECTION_NAME --credential_file=$GOOGLE_APPLICATION_CREDENTIALS &
105103
```
106104

107105
### Testing the application

‎cloud-sql/mysql/sqlalchemy/app.flexible.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,8 @@ beta_settings:
2929
cloud_sql_instances: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>=tcp:<PORT>
3030

3131
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
32-
# something like https://cloud.google.com/kms/ to help keep secrets secret.
32+
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
33+
# secrets secret.
3334
env_variables:
3435
CLOUD_SQL_CONNECTION_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>
3536
DB_USER: my-db-user

‎cloud-sql/mysql/sqlalchemy/app.standard.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,8 @@
1515
runtime: python37
1616

1717
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
18-
# something like https://cloud.google.com/kms/ to help keep secrets secret.
18+
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
19+
# secrets secret.
1920
env_variables:
2021
CLOUD_SQL_CONNECTION_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>
2122
DB_USER: my-db-user

‎cloud-sql/mysql/sqlalchemy/main.py

Lines changed: 42 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -20,13 +20,6 @@
2020
import sqlalchemy
2121

2222

23-
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
24-
# something like https://cloud.google.com/kms/ to help keep secrets secret.
25-
db_user = os.environ.get("DB_USER")
26-
db_pass = os.environ.get("DB_PASS")
27-
db_name = os.environ.get("DB_NAME")
28-
cloud_sql_connection_name = os.environ.get("CLOUD_SQL_CONNECTION_NAME")
29-
3023
app = Flask(__name__)
3124

3225
logger = logging.getLogger()
@@ -68,59 +61,74 @@ def init_connection_engine():
6861

6962
def init_tcp_connection_engine(db_config):
7063
# [START cloud_sql_mysql_sqlalchemy_create_tcp]
71-
db_socket_addr = os.environ.get("DB_HOST").split(":")
72-
73-
# Extract host and port from socket address
74-
db_host = db_socket_addr[0]
75-
db_port = int(db_socket_addr[1])
76-
77-
return sqlalchemy.create_engine(
64+
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
65+
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
66+
# secrets secret.
67+
db_user = os.environ["DB_USER"]
68+
db_pass = os.environ["DB_PASS"]
69+
db_name = os.environ["DB_NAME"]
70+
db_host = os.environ["DB_HOST"]
71+
72+
# Extract host and port from db_host
73+
host_args = db_host.split(":")
74+
db_hostname, db_port = host_args[0], int(host_args[1])
75+
76+
pool = sqlalchemy.create_engine(
7877
# Equivalent URL:
7978
# mysql+pymysql://<db_user>:<db_pass>@<db_host>:<db_port>/<db_name>
8079
sqlalchemy.engine.url.URL(
8180
drivername="mysql+pymysql",
82-
username=db_user,
83-
password=db_pass,
84-
host=db_host,
85-
port=db_port,
86-
database=db_name,
81+
username=db_user, # e.g. "my-database-user"
82+
password=db_pass, # e.g. "my-database-password"
83+
host=db_hostname, # e.g. "127.0.0.1"
84+
port=db_port, # e.g. 3306
85+
database=db_name, # e.g. "my-database-name"
8786
),
8887
# ... Specify additional properties here.
89-
# [START_EXCLUDE]
88+
# [END cloud_sql_mysql_sqlalchemy_create_tcp]
9089
**db_config
91-
# [END_EXCLUDE]
90+
# [START cloud_sql_mysql_sqlalchemy_create_tcp]
9291
)
9392
# [END cloud_sql_mysql_sqlalchemy_create_tcp]
9493

94+
return pool
95+
9596

9697
def init_unix_connection_engine(db_config):
9798
# [START cloud_sql_mysql_sqlalchemy_create_socket]
98-
if os.environ.get("DB_SOCKET_PATH"):
99-
socket_path = os.environ.get("DB_SOCKET_PATH")
100-
else:
101-
socket_path = "/cloudsql"
102-
103-
return sqlalchemy.create_engine(
99+
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
100+
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
101+
# secrets secret.
102+
db_user = os.environ["DB_USER"]
103+
db_pass = os.environ["DB_PASS"]
104+
db_name = os.environ["DB_NAME"]
105+
db_socket_dir = os.environ.get("DB_SOCKET_DIR", "/cloudsql")
106+
cloud_sql_connection_name = os.environ["CLOUD_SQL_CONNECTION_NAME"]
107+
108+
pool = sqlalchemy.create_engine(
104109
# Equivalent URL:
105110
# mysql+pymysql://<db_user>:<db_pass>@/<db_name>?unix_socket=<socket_path>/<cloud_sql_instance_name>
106111
sqlalchemy.engine.url.URL(
107112
drivername="mysql+pymysql",
108-
username=db_user,
109-
password=db_pass,
110-
database=db_name,
113+
username=db_user, # e.g. "my-database-user"
114+
password=db_pass, # e.g. "my-database-password"
115+
database=db_name, # e.g. "my-database-name"
111116
query={
112117
"unix_socket": "{}/{}".format(
113-
socket_path,
114-
cloud_sql_connection_name)
118+
db_socket_dir, # e.g. "/cloudsql"
119+
cloud_sql_connection_name) # i.e "<PROJECT-NAME>:<INSTANCE-REGION>:<INSTANCE-NAME>"
115120
}
116121
),
117122
# ... Specify additional properties here.
118-
# [START_EXCLUDE]
123+
124+
# [END cloud_sql_mysql_sqlalchemy_create_socket]
119125
**db_config
120-
# [END_EXCLUDE]
126+
# [START cloud_sql_mysql_sqlalchemy_create_socket]
121127
)
122128
# [END cloud_sql_mysql_sqlalchemy_create_socket]
123129

130+
return pool
131+
124132

125133
# The SQLAlchemy engine will help manage interactions, including automatically
126134
# managing a pool of connections to your database

‎cloud-sql/postgres/sqlalchemy/README.md

Lines changed: 11 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -17,17 +17,6 @@ name.
1717
[instructions](https://cloud.google.com/sql/docs/postgres/connect-external-app#4_if_required_by_your_authentication_method_create_a_service_account).
1818
Download a JSON key to use to authenticate your connection.
1919

20-
1. Use the information noted in the previous steps:
21-
```bash
22-
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service/account/key.json
23-
export CLOUD_SQL_CONNECTION_NAME='<MY-PROJECT>:<INSTANCE-REGION>:<INSTANCE-NAME>'
24-
export DB_USER='my-db-user'
25-
export DB_PASS='my-db-pass'
26-
export DB_NAME='my_db'
27-
```
28-
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
29-
secure solution such as [Cloud KMS](https://cloud.google.com/kms/) to help keep secrets safe.
30-
3120
## Running locally
3221

3322
To run this application locally, download and install the `cloud_sql_proxy` by
@@ -51,6 +40,9 @@ export DB_USER='<DB_USER_NAME>'
5140
export DB_PASS='<DB_PASSWORD>'
5241
export DB_NAME='<DB_NAME>'
5342
```
43+
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
44+
secure solution such as [Secret Manager](https://cloud.google.com/secret-manager/docs/overview) to
45+
help keep secrets safe.
5446

5547
Then use this command to launch the proxy in the background:
5648
```bash
@@ -66,6 +58,9 @@ $env:DB_USER="<DB_USER_NAME>"
6658
$env:DB_PASS="<DB_PASSWORD>"
6759
$env:DB_NAME="<DB_NAME>"
6860
```
61+
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
62+
secure solution such as [Secret Manager](https://cloud.google.com/secret-manager/docs/overview) to
63+
help keep secrets safe.
6964

7065
Then use this command to launch the proxy in a separate PowerShell session:
7166
```powershell
@@ -85,7 +80,7 @@ sudo chown -R $USER /path/to/the/new/directory
8580

8681
You'll also need to initialize an environment variable containing the directory you just created:
8782
```bash
88-
export DB_SOCKET_PATH=/path/to/the/new/directory
83+
export DB_SOCKET_DIR=/path/to/the/new/directory
8984
```
9085
9186
Use these terminal commands to initialize other environment variables as well:
@@ -96,10 +91,13 @@ export DB_USER='<DB_USER_NAME>'
9691
export DB_PASS='<DB_PASSWORD>'
9792
export DB_NAME='<DB_NAME>'
9893
```
94+
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
95+
secure solution such as [Secret Manager](https://cloud.google.com/secret-manager/docs/overview) to
96+
help keep secrets safe.
9997
10098
Then use this command to launch the proxy in the background:
10199
```bash
102-
./cloud_sql_proxy -dir=$DB_SOCKET_PATH --instances=$INSTANCE_CONNECTION_NAME --credential_file=$GOOGLE_APPLICATION_CREDENTIALS &
100+
./cloud_sql_proxy -dir=$DB_SOCKET_DIR --instances=$INSTANCE_CONNECTION_NAME --credential_file=$GOOGLE_APPLICATION_CREDENTIALS &
103101
```
104102
105103
### Testing the application

‎cloud-sql/postgres/sqlalchemy/app.flexible.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,8 @@ beta_settings:
2929
cloud_sql_instances: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>=tcp:<PORT>
3030

3131
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
32-
# something like https://cloud.google.com/kms/ to help keep secrets secret.
32+
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
33+
# secrets secret.
3334
env_variables:
3435
CLOUD_SQL_CONNECTION_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>
3536
DB_USER: my-db-user

‎cloud-sql/postgres/sqlalchemy/app.standard.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,8 @@
1515
runtime: python37
1616

1717
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
18-
# something like https://cloud.google.com/kms/ to help keep secrets secret.
18+
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
19+
# secrets secret.
1920
env_variables:
2021
CLOUD_SQL_CONNECTION_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>
2122
DB_USER: my-db-user

‎cloud-sql/postgres/sqlalchemy/main.py

Lines changed: 41 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -20,13 +20,6 @@
2020
import sqlalchemy
2121

2222

23-
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
24-
# something like https://cloud.google.com/kms/ to help keep secrets secret.
25-
db_user = os.environ.get("DB_USER")
26-
db_pass = os.environ.get("DB_PASS")
27-
db_name = os.environ.get("DB_NAME")
28-
cloud_sql_connection_name = os.environ.get("CLOUD_SQL_CONNECTION_NAME")
29-
3023
app = Flask(__name__)
3124

3225
logger = logging.getLogger()
@@ -71,60 +64,74 @@ def init_connection_engine():
7164

7265
def init_tcp_connection_engine(db_config):
7366
# [START cloud_sql_postgres_sqlalchemy_create_tcp]
74-
db_socket_addr = os.environ.get("DB_HOST").split(":")
75-
76-
# Extract host and port from socket address
77-
db_host = db_socket_addr[0]
78-
db_port = int(db_socket_addr[1])
79-
80-
return sqlalchemy.create_engine(
67+
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
68+
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
69+
# secrets secret.
70+
db_user = os.environ["DB_USER"]
71+
db_pass = os.environ["DB_PASS"]
72+
db_name = os.environ["DB_NAME"]
73+
db_host = os.environ["DB_HOST"]
74+
75+
# Extract host and port from db_host
76+
host_args = db_host.split(":")
77+
db_hostname, db_port = host_args[0], int(host_args[1])
78+
79+
pool = sqlalchemy.create_engine(
8180
# Equivalent URL:
8281
# postgres+pg8000://<db_user>:<db_pass>@<db_host>:<db_port>/<db_name>
8382
sqlalchemy.engine.url.URL(
8483
drivername="postgres+pg8000",
85-
username=db_user,
86-
password=db_pass,
87-
host=db_host,
88-
port=db_port,
89-
database=db_name
84+
username=db_user, # e.g. "my-database-user"
85+
password=db_pass, # e.g. "my-database-password"
86+
host=db_hostname, # e.g. "127.0.0.1"
87+
port=db_port, # e.g. 5432
88+
database=db_name # e.g. "my-database-name"
9089
),
9190
# ... Specify additional properties here.
92-
# [START_EXCLUDE]
91+
# [END cloud_sql_postgres_sqlalchemy_create_tcp]
9392
**db_config
94-
# [END_EXCLUDE]
93+
# [START cloud_sql_postgres_sqlalchemy_create_tcp]
9594
)
9695
# [END cloud_sql_postgres_sqlalchemy_create_tcp]
9796

97+
return pool
98+
9899

99100
def init_unix_connection_engine(db_config):
100101
# [START cloud_sql_postgres_sqlalchemy_create_socket]
101-
if os.environ.get("DB_SOCKET_PATH"):
102-
socket_path = os.environ.get("DB_SOCKET_PATH")
103-
else:
104-
socket_path = "/cloudsql"
105-
106-
return sqlalchemy.create_engine(
102+
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
103+
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
104+
# secrets secret.
105+
db_user = os.environ["DB_USER"]
106+
db_pass = os.environ["DB_PASS"]
107+
db_name = os.environ["DB_NAME"]
108+
db_socket_dir = os.environ.get("DB_SOCKET_DIR", "/cloudsql")
109+
cloud_sql_connection_name = os.environ["CLOUD_SQL_CONNECTION_NAME"]
110+
111+
pool = sqlalchemy.create_engine(
107112
# Equivalent URL:
108113
# postgres+pg8000://<db_user>:<db_pass>@/<db_name>
109114
# ?unix_sock=<socket_path>/<cloud_sql_instance_name>/.s.PGSQL.5432
110115
sqlalchemy.engine.url.URL(
111116
drivername="postgres+pg8000",
112-
username=db_user,
113-
password=db_pass,
114-
database=db_name,
117+
username=db_user, # e.g. "my-database-user"
118+
password=db_pass, # e.g. "my-database-password"
119+
database=db_name, # e.g. "my-database-name"
115120
query={
116121
"unix_sock": "{}/{}/.s.PGSQL.5432".format(
117-
socket_path,
118-
cloud_sql_connection_name)
122+
db_socket_dir, # e.g. "/cloudsql"
123+
cloud_sql_connection_name) # i.e "<PROJECT-NAME>:<INSTANCE-REGION>:<INSTANCE-NAME>"
119124
}
120125
),
121126
# ... Specify additional properties here.
122-
# [START_EXCLUDE]
127+
# [END cloud_sql_postgres_sqlalchemy_create_socket]
123128
**db_config
124-
# [END_EXCLUDE]
129+
# [START cloud_sql_postgres_sqlalchemy_create_socket]
125130
)
126131
# [END cloud_sql_postgres_sqlalchemy_create_socket]
127132

133+
return pool
134+
128135

129136
# The SQLAlchemy engine will help manage interactions, including automatically
130137
# managing a pool of connections to your database

0 commit comments

Comments
 (0)