Skip to content

refactor Cloud SQL samples #3952

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 5, 2020
Merged

refactor Cloud SQL samples #3952

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
e5aa603
refactor Cloud SQL PostgreSQL & MySQL samples
Jun 3, 2020
b4bbea0
minor code style and performance updates for Cloud SQL PostgreSQL and…
Jun 3, 2020
9fa6f6e
refactor Cloud SQL SQL Server samples
Jun 4, 2020
17298c5
fix Cloud SQL MySQL & PostgreSQL READMEs to remove unneecessary instr…
Jun 4, 2020
cf283cf
replace references in Cloud SQL samples to Key Management Service to …
Jun 5, 2020
734c833
fix typo in Cloud SQL Postgres sample and refactor SQL Server sample …
Jun 5, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 11 additions & 13 deletions cloud-sql/mysql/sqlalchemy/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,6 @@ name.
[instructions](https://cloud.google.com/sql/docs/mysql/connect-external-app#4_if_required_by_your_authentication_method_create_a_service_account).
Download a JSON key to use to authenticate your connection.

1. Use the information noted in the previous steps:
```bash
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service/account/key.json
export CLOUD_SQL_CONNECTION_NAME='<MY-PROJECT>:<INSTANCE-REGION>:<INSTANCE-NAME>'
export DB_USER='my-db-user'
export DB_PASS='my-db-pass'
export DB_NAME='my_db'
```
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
secure solution such as [Cloud KMS](https://cloud.google.com/kms/) to help keep secrets safe.

## Running locally

To run this application locally, download and install the `cloud_sql_proxy` by
Expand All @@ -52,6 +41,9 @@ export DB_USER='<DB_USER_NAME>'
export DB_PASS='<DB_PASSWORD>'
export DB_NAME='<DB_NAME>'
```
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
secure solution such as [Secret Manager](https://cloud.google.com/secret-manager/docs/overview) to
help keep secrets safe.

Then use this command to launch the proxy in the background:
```bash
Expand All @@ -67,6 +59,9 @@ $env:DB_USER="<DB_USER_NAME>"
$env:DB_PASS="<DB_PASSWORD>"
$env:DB_NAME="<DB_NAME>"
```
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
secure solution such as [Secret Manager](https://cloud.google.com/secret-manager/docs/overview) to
help keep secrets safe.

Then use this command to launch the proxy in a separate PowerShell session:
```powershell
Expand All @@ -87,7 +82,7 @@ sudo chown -R $USER /cloudsql

You'll also need to initialize an environment variable containing the directory you just created:
```bash
export DB_SOCKET_PATH=/path/to/the/new/directory
export DB_SOCKET_DIR=/path/to/the/new/directory
```

Use these terminal commands to initialize other environment variables as well:
Expand All @@ -98,10 +93,13 @@ export DB_USER='<DB_USER_NAME>'
export DB_PASS='<DB_PASSWORD>'
export DB_NAME='<DB_NAME>'
```
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
secure solution such as [Secret Manager](https://cloud.google.com/secret-manager/docs/overview) to
help keep secrets safe.

Then use this command to launch the proxy in the background:
```bash
./cloud_sql_proxy -dir=$DB_SOCKET_PATH --instances=$INSTANCE_CONNECTION_NAME --credential_file=$GOOGLE_APPLICATION_CREDENTIALS &
./cloud_sql_proxy -dir=$DB_SOCKET_DIR --instances=$INSTANCE_CONNECTION_NAME --credential_file=$GOOGLE_APPLICATION_CREDENTIALS &
```

### Testing the application
Expand Down
3 changes: 2 additions & 1 deletion cloud-sql/mysql/sqlalchemy/app.flexible.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ beta_settings:
cloud_sql_instances: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>=tcp:<PORT>

# Remember - storing secrets in plaintext is potentially unsafe. Consider using
# something like https://cloud.google.com/kms/ to help keep secrets secret.
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
# secrets secret.
env_variables:
CLOUD_SQL_CONNECTION_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>
DB_USER: my-db-user
Expand Down
3 changes: 2 additions & 1 deletion cloud-sql/mysql/sqlalchemy/app.standard.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@
runtime: python37

# Remember - storing secrets in plaintext is potentially unsafe. Consider using
# something like https://cloud.google.com/kms/ to help keep secrets secret.
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
# secrets secret.
env_variables:
CLOUD_SQL_CONNECTION_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>
DB_USER: my-db-user
Expand Down
76 changes: 42 additions & 34 deletions cloud-sql/mysql/sqlalchemy/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,6 @@
import sqlalchemy


# Remember - storing secrets in plaintext is potentially unsafe. Consider using
# something like https://cloud.google.com/kms/ to help keep secrets secret.
db_user = os.environ.get("DB_USER")
db_pass = os.environ.get("DB_PASS")
db_name = os.environ.get("DB_NAME")
cloud_sql_connection_name = os.environ.get("CLOUD_SQL_CONNECTION_NAME")

app = Flask(__name__)

logger = logging.getLogger()
Expand Down Expand Up @@ -68,59 +61,74 @@ def init_connection_engine():

def init_tcp_connection_engine(db_config):
# [START cloud_sql_mysql_sqlalchemy_create_tcp]
db_socket_addr = os.environ.get("DB_HOST").split(":")

# Extract host and port from socket address
db_host = db_socket_addr[0]
db_port = int(db_socket_addr[1])

return sqlalchemy.create_engine(
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
# secrets secret.
db_user = os.environ["DB_USER"]
db_pass = os.environ["DB_PASS"]
db_name = os.environ["DB_NAME"]
db_host = os.environ["DB_HOST"]

# Extract host and port from db_host
host_args = db_host.split(":")
db_hostname, db_port = host_args[0], int(host_args[1])

pool = sqlalchemy.create_engine(
# Equivalent URL:
# mysql+pymysql://<db_user>:<db_pass>@<db_host>:<db_port>/<db_name>
sqlalchemy.engine.url.URL(
drivername="mysql+pymysql",
username=db_user,
password=db_pass,
host=db_host,
port=db_port,
database=db_name,
username=db_user, # e.g. "my-database-user"
password=db_pass, # e.g. "my-database-password"
host=db_hostname, # e.g. "127.0.0.1"
port=db_port, # e.g. 3306
database=db_name, # e.g. "my-database-name"
),
# ... Specify additional properties here.
# [START_EXCLUDE]
# [END cloud_sql_mysql_sqlalchemy_create_tcp]
**db_config
# [END_EXCLUDE]
# [START cloud_sql_mysql_sqlalchemy_create_tcp]
)
# [END cloud_sql_mysql_sqlalchemy_create_tcp]

return pool


def init_unix_connection_engine(db_config):
# [START cloud_sql_mysql_sqlalchemy_create_socket]
if os.environ.get("DB_SOCKET_PATH"):
socket_path = os.environ.get("DB_SOCKET_PATH")
else:
socket_path = "/cloudsql"

return sqlalchemy.create_engine(
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
# secrets secret.
db_user = os.environ["DB_USER"]
db_pass = os.environ["DB_PASS"]
db_name = os.environ["DB_NAME"]
db_socket_dir = os.environ.get("DB_SOCKET_DIR", "/cloudsql")
cloud_sql_connection_name = os.environ["CLOUD_SQL_CONNECTION_NAME"]

pool = sqlalchemy.create_engine(
# Equivalent URL:
# mysql+pymysql://<db_user>:<db_pass>@/<db_name>?unix_socket=<socket_path>/<cloud_sql_instance_name>
sqlalchemy.engine.url.URL(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add init examples next to these values?

username=db_user, # e.g. "my-database-user"

drivername="mysql+pymysql",
username=db_user,
password=db_pass,
database=db_name,
username=db_user, # e.g. "my-database-user"
password=db_pass, # e.g. "my-database-password"
database=db_name, # e.g. "my-database-name"
query={
"unix_socket": "{}/{}".format(
socket_path,
cloud_sql_connection_name)
db_socket_dir, # e.g. "/cloudsql"
cloud_sql_connection_name) # i.e "<PROJECT-NAME>:<INSTANCE-REGION>:<INSTANCE-NAME>"
}
),
# ... Specify additional properties here.
# [START_EXCLUDE]

# [END cloud_sql_mysql_sqlalchemy_create_socket]
**db_config
# [END_EXCLUDE]
# [START cloud_sql_mysql_sqlalchemy_create_socket]
)
# [END cloud_sql_mysql_sqlalchemy_create_socket]

return pool


# The SQLAlchemy engine will help manage interactions, including automatically
# managing a pool of connections to your database
Expand Down
24 changes: 11 additions & 13 deletions cloud-sql/postgres/sqlalchemy/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,6 @@ name.
[instructions](https://cloud.google.com/sql/docs/postgres/connect-external-app#4_if_required_by_your_authentication_method_create_a_service_account).
Download a JSON key to use to authenticate your connection.

1. Use the information noted in the previous steps:
```bash
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service/account/key.json
export CLOUD_SQL_CONNECTION_NAME='<MY-PROJECT>:<INSTANCE-REGION>:<INSTANCE-NAME>'
export DB_USER='my-db-user'
export DB_PASS='my-db-pass'
export DB_NAME='my_db'
```
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
secure solution such as [Cloud KMS](https://cloud.google.com/kms/) to help keep secrets safe.

## Running locally

To run this application locally, download and install the `cloud_sql_proxy` by
Expand All @@ -51,6 +40,9 @@ export DB_USER='<DB_USER_NAME>'
export DB_PASS='<DB_PASSWORD>'
export DB_NAME='<DB_NAME>'
```
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
secure solution such as [Secret Manager](https://cloud.google.com/secret-manager/docs/overview) to
help keep secrets safe.

Then use this command to launch the proxy in the background:
```bash
Expand All @@ -66,6 +58,9 @@ $env:DB_USER="<DB_USER_NAME>"
$env:DB_PASS="<DB_PASSWORD>"
$env:DB_NAME="<DB_NAME>"
```
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
secure solution such as [Secret Manager](https://cloud.google.com/secret-manager/docs/overview) to
help keep secrets safe.

Then use this command to launch the proxy in a separate PowerShell session:
```powershell
Expand All @@ -85,7 +80,7 @@ sudo chown -R $USER /path/to/the/new/directory

You'll also need to initialize an environment variable containing the directory you just created:
```bash
export DB_SOCKET_PATH=/path/to/the/new/directory
export DB_SOCKET_DIR=/path/to/the/new/directory
```

Use these terminal commands to initialize other environment variables as well:
Expand All @@ -96,10 +91,13 @@ export DB_USER='<DB_USER_NAME>'
export DB_PASS='<DB_PASSWORD>'
export DB_NAME='<DB_NAME>'
```
Note: Saving credentials in environment variables is convenient, but not secure - consider a more
secure solution such as [Secret Manager](https://cloud.google.com/secret-manager/docs/overview) to
help keep secrets safe.

Then use this command to launch the proxy in the background:
```bash
./cloud_sql_proxy -dir=$DB_SOCKET_PATH --instances=$INSTANCE_CONNECTION_NAME --credential_file=$GOOGLE_APPLICATION_CREDENTIALS &
./cloud_sql_proxy -dir=$DB_SOCKET_DIR --instances=$INSTANCE_CONNECTION_NAME --credential_file=$GOOGLE_APPLICATION_CREDENTIALS &
```

### Testing the application
Expand Down
3 changes: 2 additions & 1 deletion cloud-sql/postgres/sqlalchemy/app.flexible.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ beta_settings:
cloud_sql_instances: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>=tcp:<PORT>

# Remember - storing secrets in plaintext is potentially unsafe. Consider using
# something like https://cloud.google.com/kms/ to help keep secrets secret.
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
# secrets secret.
env_variables:
CLOUD_SQL_CONNECTION_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>
DB_USER: my-db-user
Expand Down
3 changes: 2 additions & 1 deletion cloud-sql/postgres/sqlalchemy/app.standard.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@
runtime: python37

# Remember - storing secrets in plaintext is potentially unsafe. Consider using
# something like https://cloud.google.com/kms/ to help keep secrets secret.
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
# secrets secret.
env_variables:
CLOUD_SQL_CONNECTION_NAME: <MY-PROJECT>:<INSTANCE-REGION>:<MY-DATABASE>
DB_USER: my-db-user
Expand Down
75 changes: 41 additions & 34 deletions cloud-sql/postgres/sqlalchemy/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,6 @@
import sqlalchemy


# Remember - storing secrets in plaintext is potentially unsafe. Consider using
# something like https://cloud.google.com/kms/ to help keep secrets secret.
db_user = os.environ.get("DB_USER")
db_pass = os.environ.get("DB_PASS")
db_name = os.environ.get("DB_NAME")
cloud_sql_connection_name = os.environ.get("CLOUD_SQL_CONNECTION_NAME")

app = Flask(__name__)

logger = logging.getLogger()
Expand Down Expand Up @@ -71,60 +64,74 @@ def init_connection_engine():

def init_tcp_connection_engine(db_config):
# [START cloud_sql_postgres_sqlalchemy_create_tcp]
db_socket_addr = os.environ.get("DB_HOST").split(":")

# Extract host and port from socket address
db_host = db_socket_addr[0]
db_port = int(db_socket_addr[1])

return sqlalchemy.create_engine(
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
# secrets secret.
db_user = os.environ["DB_USER"]
db_pass = os.environ["DB_PASS"]
db_name = os.environ["DB_NAME"]
db_host = os.environ["DB_HOST"]

# Extract host and port from db_host
host_args = db_host.split(":")
db_hostname, db_port = host_args[0], int(host_args[1])

pool = sqlalchemy.create_engine(
# Equivalent URL:
# postgres+pg8000://<db_user>:<db_pass>@<db_host>:<db_port>/<db_name>
sqlalchemy.engine.url.URL(
drivername="postgres+pg8000",
username=db_user,
password=db_pass,
host=db_host,
port=db_port,
database=db_name
username=db_user, # e.g. "my-database-user"
password=db_pass, # e.g. "my-database-password"
host=db_hostname, # e.g. "127.0.0.1"
port=db_port, # e.g. 5432
database=db_name # e.g. "my-database-name"
),
# ... Specify additional properties here.
# [START_EXCLUDE]
# [END cloud_sql_postgres_sqlalchemy_create_tcp]
**db_config
# [END_EXCLUDE]
# [START cloud_sql_postgres_sqlalchemy_create_tcp]
)
# [END cloud_sql_postgres_sqlalchemy_create_tcp]

return pool


def init_unix_connection_engine(db_config):
# [START cloud_sql_postgres_sqlalchemy_create_socket]
if os.environ.get("DB_SOCKET_PATH"):
socket_path = os.environ.get("DB_SOCKET_PATH")
else:
socket_path = "/cloudsql"

return sqlalchemy.create_engine(
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
# secrets secret.
db_user = os.environ["DB_USER"]
db_pass = os.environ["DB_PASS"]
db_name = os.environ["DB_NAME"]
db_socket_dir = os.environ.get("DB_SOCKET_DIR", "/cloudsql")
cloud_sql_connection_name = os.environ["CLOUD_SQL_CONNECTION_NAME"]

pool = sqlalchemy.create_engine(
# Equivalent URL:
# postgres+pg8000://<db_user>:<db_pass>@/<db_name>
# ?unix_sock=<socket_path>/<cloud_sql_instance_name>/.s.PGSQL.5432
sqlalchemy.engine.url.URL(
drivername="postgres+pg8000",
username=db_user,
password=db_pass,
database=db_name,
username=db_user, # e.g. "my-database-user"
password=db_pass, # e.g. "my-database-password"
database=db_name, # e.g. "my-database-name"
query={
"unix_sock": "{}/{}/.s.PGSQL.5432".format(
socket_path,
cloud_sql_connection_name)
db_socket_dir, # e.g. "/cloudsql"
cloud_sql_connection_name) # i.e "<PROJECT-NAME>:<INSTANCE-REGION>:<INSTANCE-NAME>"
}
),
# ... Specify additional properties here.
# [START_EXCLUDE]
# [END cloud_sql_postgres_sqlalchemy_create_socket]
**db_config
# [END_EXCLUDE]
# [START cloud_sql_postgres_sqlalchemy_create_socket]
)
# [END cloud_sql_postgres_sqlalchemy_create_socket]

return pool


# The SQLAlchemy engine will help manage interactions, including automatically
# managing a pool of connections to your database
Expand Down
Loading