Skip to content

DOC-5279 started redesign of secrets pages #1634

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -416,11 +416,32 @@ find the preparation guides for the databases that RDI supports in the
[Prepare source databases]({{< relref "/integrate/redis-data-integration/data-pipelines/prepare-dbs" >}})
section.

## Provide authentication secrets

You must provide authentication secrets for your source and target databases
before deploying a pipeline. You can supply the secrets to RDI directly
or for K8s deployments, you can also use an external secret provider, such as
[Vault](https://developer.hashicorp.com/vault) or
[AWS Secrets Manager](https://aws.amazon.com/secrets-manager/).

See [Set secrets]({{< relref "/integrate/redis-data-integration/data-pipelines/set-secrets" >}}) and
[Using an external secret provider]({{< relref "/integrate/redis-data-integration/data-pipelines/secret-providers" >}})
for more information.


## Deploy a pipeline

When your configuration is ready, you must deploy it to start using the pipeline. See
[Deploy a pipeline]({{< relref "/integrate/redis-data-integration/data-pipelines/deploy" >}})
to learn how to do this.
When you have created your configuration, including the [jobs]({{< relref "/integrate/redis-data-integration/data-pipelines/data-pipelines#job-files" >}}), you are
ready to deploy. Use [Redis Insight]({{< relref "/develop/tools/insight/rdi-connector" >}})
to configure and deploy pipelines for both VM and K8s installations.

For VM installations, you can also use the
[`redis-di deploy`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-deploy" >}})
command to deploy a pipeline:

```bash
redis-di deploy --dir <path to pipeline folder>
```

## Pipeline lifecycle

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
Title: Using an external secret provider
alwaysopen: false
categories:
- docs
- integrate
- rs
- rdi
description: |
Configure RDI to obtain authentication secrets for your source and target databases
from an external provider.
group: di
linkTitle: External secret providers
summary: Redis Data Integration keeps Redis in sync with the primary database in near
real time.
type: integration
weight: 3
---

For K8s deployments, you can use an external secret provider, such as
[Vault](https://developer.hashicorp.com/vault) or
[AWS Secrets Manager](https://aws.amazon.com/secrets-manager/) to provide
the authentication secrets for your source and target databases.
See the sections below to learn how to do this. If you prefer to set the secrets for RDI manually, see
[Set secrets]({{< relref "/integrate/redis-data-integration/data-pipelines/set-secrets" >}})
for more information.

## Configure an external provider

### Vault

### AWS Secret Manager

## Secret rotation

*Secret rotation* is a technique where secrets are changed automatically
by the provider according to a schedule.
RDI versions 1.10.0 and above let you configure the pipeline to
restart the appropriate K8s pods automatically whenever a secret rotates in
the external provider that you have configured.
Original file line number Diff line number Diff line change
@@ -1,32 +1,34 @@
---
Title: Deploy a pipeline
aliases: /integrate/redis-data-integration/ingest/data-pipelines/data-type-handling/
Title: Set secrets
aliases:
- /integrate/redis-data-integration/ingest/data-pipelines/data-type-handling/
- /integrate/redis-data-integration/data-pipelines/deploy/
alwaysopen: false
categories:
- docs
- integrate
- rs
- rdi
description: Learn how to deploy an RDI pipeline
description: Set authentication secrets for your source and target databases.
group: di
linkTitle: Deploy
linkTitle: Set secrets
summary: Redis Data Integration keeps Redis in sync with the primary database in near
real time.
type: integration
weight: 2
---

The sections below explain how to deploy a pipeline after you have created the required
[configuration]({{< relref "/integrate/redis-data-integration/data-pipelines/data-pipelines" >}}).

## Set secrets

Before you deploy your pipeline, you must set the authentication secrets for the
Before you
[deploy]({{< relref "/integrate/redis-data-integration/data-pipelines/data-pipelines#deploy-a-pipeline" >}})
your pipeline, you must set the authentication secrets for the
source and target databases. Each secret has a name that you can pass to the
[`redis-di set-secret`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-set-secret" >}})
command (VM deployment) or the `rdi-secret.sh` script (K8s deployment) to set the secret value.
You can then refer to these secrets in the `config.yaml` file using the syntax "`${SECRET_NAME}`"
(the sample [config.yaml file]({{< relref "/integrate/redis-data-integration/data-pipelines/data-pipelines#the-configyaml-file" >}}) shows these secrets in use).
command (VM deployment) or the `rdi-secret.sh` script (K8s deployment) to set the secret value.
For K8s, you can also configure RDI to obtain the secrets from an
[external provider]({{< relref "/integrate/redis-data-integration/data-pipelines/secret-providers" >}}).

The `config.yaml` file accesses these secrets with the syntax "`${SECRET_NAME}`"
(the sample [config.yaml file]({{< relref "/integrate/redis-data-integration/data-pipelines/data-pipelines#the-configyaml-file" >}}) shows the secrets in use).

The table below lists all valid secret names. Note that the
username and password are required for the source and target, but the other
Expand Down Expand Up @@ -249,17 +251,3 @@ kubectl create secret generic target-db-ssl --namespace=rdi \
```

Note that the certificate paths contained in the secrets `SOURCE_DB_CACERT`, `SOURCE_DB_CERT`, and `SOURCE_DB_KEY` (for the source database) and `TARGET_DB_CACERT`, `TARGET_DB_CERT`, and `TARGET_DB_KEY` (for the target database) are internal to RDI, so you *must* use the values shown in the example above. You should only change the certificate paths when you create the `source-db-ssl` and `target-db-ssl` secrets.

## Deploy a pipeline

When you have created your configuration, including the [jobs]({{< relref "/integrate/redis-data-integration/data-pipelines/data-pipelines#job-files" >}}), you are
ready to deploy. Use [Redis Insight]({{< relref "/develop/tools/insight/rdi-connector" >}})
to configure and deploy pipelines for both VM and K8s installations.

For VM installations, you can also use the
[`redis-di deploy`]({{< relref "/integrate/redis-data-integration/reference/cli/redis-di-deploy" >}})
command to deploy a pipeline:

```bash
redis-di deploy --dir <path to pipeline folder>
```