Skip to content
This repository was archived by the owner on Oct 28, 2024. It is now read-only.

Commit 2255fe6

Browse files
edit proposal; update figure
1 parent 5a9dce7 commit 2255fe6

File tree

3 files changed

+20
-59
lines changed

3 files changed

+20
-59
lines changed

proposals/20201026-creating-control-plane-components.md

Lines changed: 20 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -155,61 +155,19 @@ type NestedControlPlaneSpec struct {
155155
}
156156
```
157157

158-
After applying the NCP CR, the NCP controller will find the three associated components and set their `metav1.OwnerReference` as the NCP CR. Then, when creating the component workload, the component controller can find other CRs through the owner NCP.
158+
After applying the NCP CR, the NCP controller will find the three associated components and set their `metav1.OwnerReference` as the NCP CR. Then, the component controller can find other CRs through the owner NCP, when creating the component workload.
159159

160160
### Creation
161-
End-users can create component CRs manually and apply them to the cluster with an NCP to create the resources. In the future, we might introduce the `Template` CR, which will handle the creation of the component CRs in it's controller. If the end-user intend to use out-of-tree controllers, they can specify them in the NCP spec, we define three new types:
162161

163-
```go
164-
type NestedKASProvider string
165-
166-
type NestedEtcdProvider string
167-
168-
type NestedKCMProvider string
169-
170-
const (
171-
NativeKASProvider NestedKASProvider = "NativeKASProvider"
172-
173-
NativeEtcdProvider NestedEtcdProvider = "NativeEtcdProvider"
174-
EtcdClusterOperator NestedEtcdProvider = "EtcdClusterOperator"
175-
EtcdOperator NestedEtcdProvider = "EtcdOperator"
176-
177-
NativeKCMProvider NestedKCMProvider = "NativeKCMProvider"
178-
)
179-
```
162+
End-users can create component CRs manually and apply them to the cluster with an NCP to create the resources. In the future, we might introduce the `Template` CR, which will handle the creation of the component CRs in it's controller. We assume that at any given time, there will be only component controller for each component, and it is cluster administrator's responsibility to setup the proper component controllers.
180163

181-
and adds three new fields into the NCP spec:
164+
#### In-tree
182165

183-
```go
184-
type NestedControlPlaneSpec struct {
185-
// other fields ...
186-
187-
// EtcdProvider specifies which EtcdProvider will be used to create the Etcd
188-
// +optional
189-
EtcdProvider NestedEtcdProvider `json:"etcdProvider,omitempty"`
190-
191-
// ApiServerProvider specifies which KASProvider will be used to create the kube-apiserver
192-
// +optional
193-
ApiServerProvider NestedKASProvider `json:"apiServerProvider,omitempty"`
194-
195-
// ControllerManagerProvider specifies which KCMProvider will be used to create the kube-controller-manager
196-
// +optional
197-
ControllerManagerProvider NestedKCMProvider `json:"controllerManagerProvider,omitempty"`
198-
}
199-
```
200-
201-
202-
#### Native way
203-
204-
If no component provider is specified, the component controller will create the component under the native mode, which will create the component using the default manifests. The readiness and liveness probe will be used, and we will mark each component as ready only when the corresponding workload is ready. As the KAS cannot work without available Etcd and the KCM cannot run without KAS, the three components need to be created by their respective controllers in the order of Etcd, KAS, and KCM. Creation order is maintained using cross resource status checks to "wait" until the dependencies are provisioned before booting.
205-
206-
Except using `OwnerReference` and `CAPI.Status.Selector`, another option of sharing status across components are adding initContainer to each component. The component controllers will then create the component workload without polling the status of other CRs while having the initContainer to check and wait for other components to be ready.
207-
208-
We will host sets of default templates in this repositor. Users can specify which set of templates they intend to use by specifying the corresponding `version` or `channel` in the embedded `CommonSpec` in the component's CR.
166+
The component controller will create the component under the in-tree mode, which will create the component using the default manifests. The readiness and liveness probe will be used, and we will mark each component as ready only when the corresponding workload is ready. As the KAS cannot work without available Etcd and the KCM cannot run without KAS, the three components need to be created by their respective controllers in the order of Etcd, KAS, and KCM. Creation order is maintained using cross resource status checks to "wait" until the dependencies are provisioned before booting. We will host sets of default templates in this repository. Users can specify which set of templates they intend to use by specifying the corresponding `version` or `channel` in the embedded `CommonSpec` in the component's CR.
209167

210168
Each component's controller will generate necessary certificates for the component and store them to the [secret resources](https://cluster-api.sigs.k8s.io/tasks/certs/using-custom-certificates.html) defined by CAPI. Also, The KAS controller will store the content of the kubeconfig file in a secret named `[clustername]-kubeconfig`.
211169

212-
![Control Plane Creating Process](nativeway.png)
170+
![Control Plane Creating Process](in-tree.png)
213171

214172
The creating process will include six steps:
215173

@@ -225,24 +183,27 @@ The creating process will include six steps:
225183

226184
#### Using out-of-tree provisioners
227185

228-
If users intend to use an external component controller, they can disable any or all of the component controllers within the CAPN Manager. For example, if you wanted to use the [etcd-cluster-operator](https://github.com/improbable-eng/etcd-cluster-operator) this requires the [EtcdCluster](https://github.com/improbable-eng/etcd-cluster-operator/blob/master/api/v1alpha1/etcdcluster_types.go) CR. To support this you would implement a custom controller that listens for `Etcd` resource and will create the necessary CRs for that implementation and update the required status fields on `Etcd` to allow dependent services to be provisioned. This can be done using the [kubebuilder-declarative-pattern](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern) like is done for in-tree component controllers.
186+
If users intend to use an external component controller, they can disable any or all of the component controllers within the CAPN Manager. For example, if you wanted to use the [etcd-cluster-operator](https://github.com/improbable-eng/etcd-cluster-operator) this requires the [EtcdCluster](https://github.com/improbable-eng/etcd-cluster-operator/blob/master/api/v1alpha1/etcdcluster_types.go) CR. To support this you would implement a custom controller that listens for `NestedEtcd` resource and will create the necessary CRs for that implementation and update the required status fields on `NestedEtcd` to allow dependent services to be provisioned. This can be done using the [kubebuilder-declarative-pattern](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern) like is done for in-tree component controllers.
229187

230-
In the following example, we assume that the user intend to use Etcd-cluster-operator(ECO) as the Etcd controller, and an ECO controller has been deployed on the super cluster. The creating process will include seven steps:
231188
![Creating a Control Plane using out-of-tree provisioners](out-of-tree.png)
232189

233-
1. The user generates all CRs and apply them.
190+
In the following example, we assume that the user intend to use Etcd-cluster-operator(ECO) as the Etcd controller. The creating process will include seven steps:
191+
192+
1. The cluster administrator deletes the in-tree Etcd controller and deploys the custom Etcd controller (ECO controller).
193+
194+
2. The user generates all CRs and apply them.
234195

235-
2. As the user chooses to use the etcd-cluster-operator(ECO), the ECO controller will create the EtcdCluster CR
196+
3. As the user chooses to use the etcd-cluster-operator(ECO), the ECO controller will create the EtcdCluster CR.
236197

237-
3. The ECO creates the Etcd workload.
198+
4. The ECO creates the Etcd workload.
238199

239-
4. At the meantime, the ECO controller keeps watching the EtcdCluster CR, stores the Etcd CA into the `secret/[cluster-name]-etcd`, and update the `Etcd` CR accordingly.
200+
5. At the meantime, the ECO controller keeps watching the EtcdCluster CR, stores the Etcd CA into the `secret/[cluster-name]-etcd`, and update the `Etcd` CR accordingly.
240201

241-
5. The KAS controller creates the KAS service, generates certificates, creates the KAS workload, stores certificates into `secret/[cluster-name]-ca`, creates the kubeconfig and stores it into the `secret/[cluster-name]-kubeconfig`
202+
6. The KAS controller creates the KAS service, generates certificates, creates the KAS workload, stores certificates into `secret/[cluster-name]-ca`, creates the kubeconfig and stores it into the `secret/[cluster-name]-kubeconfig`
242203

243-
6. The KCM controller generates the KCM kubeconfig and creates the KCM workload.
204+
7. The KCM controller generates the KCM kubeconfig and creates the KCM workload.
244205

245-
7. Once all the three components are ready, the NCP controller marks the NCP CR as ready.
206+
8. Once all the three components are ready, the NCP controller marks the NCP CR as ready.
246207

247208
### Control Plane Custom Resources
248209

@@ -273,7 +234,7 @@ type NestedEtcdStatus struct {
273234
Addresses []NestedEtcdAddress `json:"addresses,omitempty"`
274235

275236
// CommonStatus allows addons status monitoring
276-
CommonStatus addonv1alpha1. CommonStatus `json:",inline"`
237+
addonv1alpha1. CommonStatus `json:",inline"`
277238
}
278239

279240
// EtcdAddress defines the observed addresses for etcd
@@ -326,7 +287,7 @@ type NestedAPIServerStatus struct {
326287
APIServerService *corev1.ObjectReference `json:"apiserverService,omitempty"`
327288

328289
// CommonStatus allows addons status monitoring
329-
CommonStatus addonv1alpha1. CommonStatus `json:",inline"`
290+
addonv1alpha1. CommonStatus `json:",inline"`
330291
}
331292

332293
// NestedAPIServer is the Schema for the APIServers API
@@ -361,7 +322,7 @@ type NestedControllerManagerStatus struct {
361322
ProviderObject *corev1.ObjectReference `json:referralObject,omitempty`
362323

363324
// CommonStatus allows addons status monitoring
364-
CommonStatus addonv1alpha1. CommonStatus `json:",inline"`
325+
addonv1alpha1. CommonStatus `json:",inline"`
365326
}
366327

367328
// NestedControllerManager is the Schema for the ControllerManagers API
File renamed without changes.

proposals/out-of-tree.png

4.13 KB
Loading

0 commit comments

Comments
 (0)