You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Oct 28, 2024. It is now read-only.
Copy file name to clipboardExpand all lines: proposals/20201026-creating-control-plane-components.md
+13-13Lines changed: 13 additions & 13 deletions
Original file line number
Diff line number
Diff line change
@@ -53,13 +53,13 @@ The followings are terms that will be used in the proposal. Whether including th
53
53
54
54
***NestedControllerManager(NKCM)** - The kube-control-manager which belongs to the control plane of the nested cluster.
55
55
56
-
***Component Controller** - Operators that create components of the NCP, including the Etcd controller, KAS controller and KCM controller.
56
+
***Component Controller** - Operators that create the NCP components, including the Etcd controller, KAS controller and KCM controller.
57
57
58
58
To better illustrate the creation process, we define two Roles with different responsibilities:
59
59
60
-
***Cluster Admin** - Responsible for creating the underlying super cluster and in-charge of creating Nested clusters.
60
+
***Cluster Admin** - Responsible for creating the underlying super cluster, deploying component controllers, and in-charge of creating Nested clusters.
61
61
62
-
***End User** - Represents a nested cluster user, these users have limited access to the super cluster.
62
+
***End User** - Represents a nested cluster user. These users have limited access to the super cluster.
63
63
64
64
65
65
## Summary
@@ -73,7 +73,7 @@ CAPN aims at providing control plane level isolation while sharing physical reso
73
73
### Goals
74
74
75
75
- Define the CRD that represents each control plane component. The CRD needs to meet two requirements:
76
-
* Portable - the CRD should hold information that are required by different component controllers, e.g., [etcd-operator](https://github.com/coreos/etcd-operator), [etcd-cluster-operator](https://github.com/improbable-eng/etcd-cluster-operator/blob/f84abc6561735814debd67d45bb62d2d2ed8cf4a/api/v1alpha1/etcdcluster_types.go#L31-L47)
76
+
* Portable - the CRD should hold information that is required by different component controllers, e.g., [etcdadm](https://github.com/kubernetes-sigs/etcdadm), [etcd-operator](https://github.com/coreos/etcd-operator), and[etcd-cluster-operator](https://github.com/improbable-eng/etcd-cluster-operator/blob/f84abc6561735814debd67d45bb62d2d2ed8cf4a/api/v1alpha1/etcdcluster_types.go#L31-L47)
77
77
* Customizable - the CRD should allow end-users to customize each component, i.e., specify the image, component version, and command-line options.
78
78
79
79
- Define a standard process of creating control plane components for NCP.
@@ -136,7 +136,7 @@ type PatchSpec struct {
136
136
137
137
#### Create prerequisites
138
138
139
-
We assume that the APIServer, ContollerManager, Etcd and the NCP CR are located in the same namespace. To create an NCP, we need to first create APIserver CR, ControllerManager CR, Etcd CR, NCP CR, and a namespace that holds all the CRs, then the component controller can cooperate with each other to create components for the NCP.
139
+
We assume that the APIServer, ContollerManager, Etcd, and the NCP CR are located in the same namespace. To create an NCP, we need first to create NestedAPIserver CR, NestedControllerManager CR, NestedEtcd CR, NCP CR, and a namespace that holds all the CRs, then the component controller can cooperate to create components for the NCP.
140
140
141
141
As there exist dependencies between components, i.e., KAS cannot run without Etcd, KCM cannot work without KAS, when creating NCP components, component controllers will need to get information and status of other CRs. To achieve this, we will add three `ObjectReference` to the `NestedControlPlaneSpec` with each `ObjectReference` points to a component.
142
142
@@ -155,15 +155,15 @@ type NestedControlPlaneSpec struct {
155
155
}
156
156
```
157
157
158
-
After applying the NCP CR, the NCP controller will find the three associated components and set their `metav1.OwnerReference` as the NCP CR. Then, the component controller can find other CRs through the owner NCP, when creating the component workload.
158
+
After applying the NCP CR, the NCP controller will find the three associated components and set their `metav1.OwnerReference` as the NCP CR. Then, the component controller can find other CRs through the owner NCP, when creating the corresponding component workload.
159
159
160
160
### Creation
161
161
162
-
End-users can create component CRs manually and apply them to the cluster with an NCP to create the resources. In the future, we might introduce the `Template` CR, which will handle the creation of the component CRs in it's controller. We assume that at any given time, there will be only component controller for each component, and it is cluster administrator's responsibility to setup the proper component controllers.
162
+
End-users can create component CRs manually and apply them to the cluster with an NCP to create the resources. In the future, we might introduce the `Template` CR, which will handle the creation of the component CRs in it's controller. We assume that there will be only one component controller for each component at any given time, and it is the cluster administrator's responsibility to set up the proper component controllers.
163
163
164
164
#### In-tree
165
165
166
-
The component controller will create the component under the in-tree mode, which will create the component using the default manifests. The readiness and liveness probe will be used, and we will mark each component as ready only when the corresponding workload is ready. As the KAS cannot work without available Etcd and the KCM cannot run without KAS, the three components need to be created by their respective controllers in the order of Etcd, KAS, and KCM. Creation order is maintained using cross resource statuschecks to "wait" until the dependencies are provisioned before booting. We will host sets of default templates in this repository. Users can specify which set of templates they intend to use by specifying the corresponding `version` or `channel` in the embedded `CommonSpec` in the component's CR.
166
+
The component controller will create the component under the in-tree mode, which will create the component using the default manifests. The readiness and liveness probe will be used, and we will mark each component as ready only when the corresponding workload is ready. As the KAS cannot work without available Etcd and the KCM cannot run without KAS, the three components need to be created by their respective controllers in the order of Etcd, KAS, and KCM. Creation order is maintained using cross resource status, which checks and wait until the dependencies are provisioned. We will host sets of default templates in this repository. Users can specify which set of templates they intend to use by specifying the corresponding `version` or `channel` in the embedded `CommonSpec` in the component's CR.
167
167
168
168
Each component's controller will generate necessary certificates for the component and store them to the [secret resources](https://cluster-api.sigs.k8s.io/tasks/certs/using-custom-certificates.html) defined by CAPI. Also, The KAS controller will store the content of the kubeconfig file in a secret named `[clustername]-kubeconfig`.
169
169
@@ -183,21 +183,21 @@ The creating process will include six steps:
183
183
184
184
#### Using out-of-tree provisioners
185
185
186
-
If users intend to use an external component controller, they can disable any or all of the component controllers within the CAPN Manager. For example, if you wanted to use the [etcd-cluster-operator](https://github.com/improbable-eng/etcd-cluster-operator)this requires the [EtcdCluster](https://github.com/improbable-eng/etcd-cluster-operator/blob/master/api/v1alpha1/etcdcluster_types.go) CR. To support this you would implement a custom controller that listens for`NestedEtcd` resource and will create the necessary CRs for that implementation and update the required status fields on `NestedEtcd` to allow dependent services to be provisioned. This can be done using the [kubebuilder-declarative-pattern](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern) like is done for in-tree component controllers.
186
+
If users intend to use an external controller to create the NCP component, they may need to implement a new component controller that can interact with the component CR and the external controller to create the component. For example, if the user wanted to use the [etcd-cluster-operator](https://github.com/improbable-eng/etcd-cluster-operator)that requires the [EtcdCluster](https://github.com/improbable-eng/etcd-cluster-operator/blob/master/api/v1alpha1/etcdcluster_types.go) CR. They need to implement a custom controller that watches the`NestedEtcd` resource, creates the necessary CRs for that implementation, and updates the required status fields on `NestedEtcd` to allow dependent services to be provisioned. This can be done using the [kubebuilder-declarative-pattern](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern) like is done for in-tree component controllers.
187
187
188
188

189
189
190
-
In the following example, we assume that the user intend to use Etcd-cluster-operator(ECO) as the Etcd controller. The creating process will include seven steps:
190
+
In the following example, we assume that the user intends to use Etcd-cluster-operator(ECO) as the Etcd controller. The creating process will include seven steps:
191
191
192
192
1. The cluster administrator deletes the in-tree Etcd controller and deploys the custom Etcd controller (ECO controller).
193
193
194
194
2. The user generates all CRs and apply them.
195
195
196
-
3.As the user chooses to use the etcd-cluster-operator(ECO), the ECO controller will create the EtcdCluster CR.
196
+
3.The ECOcontroller creates the EtcdCluster CR.
197
197
198
198
4. The ECO creates the Etcd workload.
199
199
200
-
5. At the meantime, the ECO controller keeps watching the EtcdCluster CR, stores the Etcd CA into the `secret/[cluster-name]-etcd`, and update the `Etcd` CR accordingly.
200
+
5. At the meantime, the ECO controller keeps watching the EtcdCluster CR, stores the Etcd CA into the `secret/[cluster-name]-etcd`, and updates the `Etcd` CR accordingly.
201
201
202
202
6. The KAS controller creates the KAS service, generates certificates, creates the KAS workload, stores certificates into `secret/[cluster-name]-ca`, creates the kubeconfig and stores it into the `secret/[cluster-name]-kubeconfig`
203
203
@@ -345,7 +345,7 @@ There is no need to worry about malicious users to manipulate other users' resou
345
345
A malicious user can still skew the system by creating a massive amount of resources.
346
346
To avoid this, we need to enhance the syncer component; however, this topic is
347
347
beyond this proposal's scope. The proposed mechanism will not lead to any
0 commit comments