Skip to content

Commit 40af1da

Browse files
authored
(DOCSP-36688) Added Atlas Stream Processing content. (#78)
* (DOCSP-36688) Added Atlas Stream Processing content. * (DOCSP-36688): Incorporated Helder's feedback. * (DOCSP-36688): Small tweak. * (DOCSP-36688): Incorporated feedback. * (DOCSP-36688): Added include and small tweaks. * (DOCSP-36688): H2 headings
1 parent f5a8f09 commit 40af1da

8 files changed

+641
-10
lines changed

source/ak8so-stream-processing.txt

Lines changed: 196 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,196 @@
1+
.. _ak8so-stream-processing:
2+
3+
==============================
4+
Manage {+atlas-sp+}
5+
==============================
6+
7+
.. default-domain:: mongodb
8+
9+
.. contents:: On this page
10+
:local:
11+
:backlinks: none
12+
:depth: 1
13+
:class: singlecol
14+
15+
You can use |ak8so| to manage stream processing instances and
16+
connections for {+atlas-sp+}. {+atlas-sp+} enables you to process
17+
streams of complex data using the same :manual:`MongoDB Query API
18+
</query-api>` that {+service+} databases use. {+atlas-sp+} allows you
19+
to do the following tasks:
20+
21+
- Build :ref:`aggregation pipelines <stream-aggregation>` to
22+
continuously operate on streaming data without the delays inherent in
23+
batch processing.
24+
- Perform continuous :ref:`schema validation
25+
<streams-agg-pipeline-validate>` to check that messages are properly
26+
formed, detect message corruption, and detect late-arriving data.
27+
- Continuously publish results to
28+
{+service+} collections or {+kafka+} clusters, ensuring up-to-date
29+
views and analysis of data.
30+
31+
{+atlas-sp+} components belong directly to |service| projects and
32+
operate independent of |service| {+clusters+}. To learn more, see
33+
:ref:`what-is-atlas-sp`.
34+
35+
.. _ak8so-spi-connections:
36+
37+
Stream Processing Instances and Connections
38+
-------------------------------------------
39+
40+
{+atlas-sp+} instances provide the context for all of your operations
41+
on streaming data. You can configure a sample connection, a connection
42+
to an {+service+} :manual:`change stream <changeStreams>`, or a
43+
connection to an {+kafka+} system. Then you can add the connection to
44+
the Connection Registry for your stream processing instance. To learn
45+
more, see :ref:`manage-spi`.
46+
47+
Prerequisites
48+
-------------
49+
50+
To use |ak8so| to manage stream processing instances and connections,
51+
you must:
52+
53+
- Have an |service| user with the :authrole:`Project Owner` or
54+
the :authrole:`Project Stream Processing Owner` role to manage a
55+
Connection Registry and {+SPI+}. To learn more about the differences
56+
between the two roles, see :ref:`Project Roles <project-roles>`.
57+
58+
- Have a running |k8s| cluster with |ak8so| :ref:`deployed
59+
<ak8so-quick-start-ref>`.
60+
61+
- Ensure your |ipaddr| address is in the organization's |api|
62+
:ref:`access list <access-list>`.
63+
64+
Procedure
65+
---------
66+
67+
To use |ak8so| to manage a stream processing instance and its
68+
connections, do the following steps:
69+
70+
.. procedure::
71+
:style: normal
72+
73+
.. step:: Configure the :ref:`atlasstreaminstance-custom-resource`.
74+
75+
**Example:**
76+
77+
.. code-block::
78+
79+
cat <<EOF | kubectl apply -f -
80+
apiVersion: atlas.mongodb.com/v1
81+
kind: AtlasStreamInstance
82+
metadata:
83+
name: my-stream-instance
84+
spec:
85+
name: my-stream-instance
86+
clusterConfig:
87+
provider: AWS
88+
region: VIRGINIA_USA
89+
tier: SP30
90+
EOF
91+
92+
To learn more about the available parameters, see the
93+
:ref:`atlasstreaminstance-custom-resource`.
94+
95+
.. include:: /includes/fact-ak8so-atlas-sp-provider-region.rst
96+
97+
.. step:: Configure the :ref:`atlasstreamconnection-custom-resource`.
98+
99+
You can configure a sample connection, a connection to an
100+
{+service+} :manual:`change stream <changeStreams>`, or a
101+
connection to an {+kafka+} system.
102+
103+
**Examples:**
104+
105+
.. tabs::
106+
107+
.. tab:: Sample
108+
:tabid: sample
109+
110+
.. code-block:: sh
111+
112+
apiVersion: atlas.mongodb.com/v1
113+
kind: AtlasStreamConnection
114+
metadata:
115+
name: my-stream-connection
116+
spec:
117+
name: sample_stream_solar
118+
type: Sample
119+
120+
.. include:: /includes/fact-ak8so-atlas-sp-sample.rst
121+
122+
.. tab:: Change Stream
123+
:tabid: cluster
124+
125+
.. code-block:: sh
126+
127+
apiVersion: atlas.mongodb.com/v1
128+
kind: AtlasStreamConnection
129+
metadata:
130+
name: my-stream-connection
131+
spec:
132+
name: my-stream-connection
133+
type: Cluster
134+
clusterConfig:
135+
name: my-cluster
136+
role:
137+
name: my-db-role
138+
type: CUSTOM
139+
140+
.. tab:: Kafka
141+
:tabid: kafka
142+
143+
.. code-block:: sh
144+
apiVersion: atlas.mongodb.com/v1
145+
kind: AtlasStreamConnection
146+
metadata:
147+
name: my-stream-connection
148+
spec:
149+
name: my-stream-connection
150+
type: Kafka
151+
kafkaConfig:
152+
bootstrapServers: "comma,separated,list,of,server,addresses"
153+
authentication:
154+
mechanism: SCRAM-512
155+
credentials:
156+
name: ref-to-creds-secret
157+
namespace: default
158+
security:
159+
protocol: SSL
160+
certificate:
161+
name: ref-to-certificate-secret
162+
namespace: default
163+
164+
To learn more about the available parameters, see the
165+
:ref:`atlasstreamconnection-custom-resource`.
166+
167+
.. step:: Add your connections to the Connection Registry in the :ref:`atlasstreaminstance-custom-resource`.
168+
169+
**Example:**
170+
171+
.. code-block:: sh
172+
173+
cat <<EOF | kubectl apply -f -
174+
apiVersion: atlas.mongodb.com/v1
175+
kind: AtlasStreamInstance
176+
metadata:
177+
name: my-stream-instance
178+
spec:
179+
name: my-stream-instance
180+
clusterConfig:
181+
provider: AWS
182+
region: VIRGINIA_USA
183+
tier: SP30
184+
connectionRegistry:
185+
- name: ref-my-connection-1
186+
namespace: my-namespace1
187+
- name: ref-my-connection-2
188+
namespace: my-namespace2
189+
- name: ref-my-connection-3
190+
namespace: my-namespace1
191+
EOF
192+
193+
.. include:: /includes/fact-ak8so-atlas-sp-provider-region.rst
194+
195+
To learn more about the available parameters, see the
196+
:ref:`atlasstreaminstance-custom-resource`.

source/atlasdeployment-custom-resource.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1078,7 +1078,7 @@ examples, and the |api| documentation.
10781078
To learn more, see :ref:`faq-tls-1.0-deprecation`.
10791079

10801080
This option corresponds to the
1081-
:setting:`net.ssl.disabledProtocols` |mongod|
1081+
``net.ssl.disabledProtocols`` |mongod|
10821082
configuration file option.
10831083

10841084
.. setting:: spec.processArgs.noTableScan

0 commit comments

Comments
 (0)