-
Notifications
You must be signed in to change notification settings - Fork 83
NooBaa in Open Shift
This section will deploy Open Shift (OpenShift) on AWS. Use funcs.io domain
Pull secret from here: https://cloud.openshift.com/clusters/install git clone https://github.com/openshift/installer
At the end of the installation, you will see something like the following
NFO Creating cluster... *******************************************************************************************************************************************************
INFO Waiting up to 30m0s for the Kubernetes API...
INFO API v1.11.0+8868a98a7b up
INFO Waiting up to 30m0s for the bootstrap-complete event...
ERROR: logging before flag.Parse: E0204 23:56:33.391618 27781 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=3, ErrCode=NO_ERROR, debug=""
WARNING RetryWatcher - getting event failed! Re-creating the watcher. Last RV: 148
INFO Destroying the bootstrap resources...
INFO Waiting up to 10m0s for the openshift-console route to be created...
INFO Install complete!
INFO Run 'export KUBECONFIG=/Users/erantamir/workspace/openshift/auth/kubeconfig' to manage the cluster with 'oc', the OpenShift CLI.
INFO The cluster is ready when 'oc login -u kubeadmin -p XXXXX-XXXXX-XXXXX-XXXXX' succeeds (wait a few minutes).
INFO Access the OpenShift web-console here: https://console-openshift-console.apps.xxxx.funcs.io
INFO Login to the console with user: kubeadmin, password: XXXXX-XXXXX-XXXXX-XXXXX
oc login https://xxxx-api.funcs.io:6443
This section setup open shift, so it can pull private images from Azure Login to Azure account, navigate to the registry repository and pull the name and access key Run the following command in order to get docker login noobaaimages.azurecr.io -u noobaaimages -p nIxPLIZsmERRfBs=77fms6bsC1VbroEe
Create bash file **azure_registry.sh **with the following content (reference https://docs.microsoft.com/en-us/azure/container-registry/container-registry-auth-aks)
#!/bin/bash
ACR_NAME=noobaaimages
SERVICE_PRINCIPAL_NAME=acr-service-principal
# Populate the ACR login server and resource id.
ACR_LOGIN_SERVER=$(az acr show --name $ACR_NAME --query loginServer --output tsv)
ACR_REGISTRY_ID=$(az acr show --name $ACR_NAME --query id --output tsv)
# Create acrpull role assignment with a scope of the ACR resource.
SP_PASSWD=$(az ad sp create-for-rbac --name $SERVICE_PRINCIPAL_NAME --role acrpull --scopes $ACR_REGISTRY_ID --query password --output tsv)
# Get the service principal client id.
CLIENT_ID=$(az ad sp show --id http://$SERVICE_PRINCIPAL_NAME --query appId --output tsv)
# Output used when creating Kubernetes secret.
echo "Service principal ID: $CLIENT_ID"
echo "Service principal password: $SP_PASSWD"
Run the bash file.
chmod 777 azure_registry.sh;./azure_registry.sh
Expect an output similar to the following:
Service principal ID: bd91bb5c-1815-4a88-b9f7-955f51b5a143
Service principal password: 08e92de0-e97a-4618-a558-448c0439d298
Run the following commands in order to create a secret that will be used by NooBaa YAML
kubectl create secret docker-registry acr-auth --docker-server noobaaimages.azurecr.io --docker-username bd91bb5c-1815-4a88-b9f7-955f51b5a143 --docker-password 08e92de0-e97a-4618-a558-448c0439d298 --docker-email [email protected]
kubectl patch serviceaccount default -p '{"imagePullSecrets": [{"name": “acr-auth”}]}'
Update the yaml and add the following lines under the image annotation ` image: noobaaimages.azurecr.io/noobaa/nbserver:<version tag>
secrets:
- name: default-token-h5864
imagePullSecrets:
- name: acr-auth
`
oc adm policy add-scc-to-group privileged system:authenticated
oc create -f noobaa_statefuleset.yaml ; sleep 15;oc get pod;oc get service
oc delete -f noobaa_statefuleset.yaml ; sleep 15; oc delete pvc/datadir-noobaa-0 ; oc delete pvc/logdir-noobaa-0;oc get pv
oc apply -f noobaa_statefuleset.yaml
oc exec -it noobaa-0 /bin/bash
copy the yaml and perform the commands from the addon itself
https://github.com/minishift/minishift-addons/tree/master/add-ons/prometheus on default project.
oc new-app -f https://raw.githubusercontent.com/ConSol/springboot-monitoring-example/master/templates/grafana.yaml -p NAMESPACE=grafana oc policy add-role-to-user view system:serviceaccount:grafana:grafana-ocp -n prometheus
Create data source with prometheus with token oc sa get-token prometheus -n prometheus
**Skip authenticated connection by running the following command and take the endpoints and use for data source: ** (better use dns, follow this article - https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/)
oc describe service prometheus -n default

git clone https://github.com/rook/rook.git
cd cluster/examples/kubernetes/ceph/
Edit operator.yaml and change ROOK_HOSTPATH_REQUIRES_PRIVILEGED to true
Edit operator.yaml and uncomment the two lines starts with name: FLEXVOLUME_DIR_PATH, set the value to /etc/kubernetes/kubelet-plugins/volume/exec
Edit cluster.yaml and change dataDirHostPath to /data/rook
Edit object.yaml and change the gateway port from 80 to 8081 oc create -f scc.yaml
kubectl create -f operator.yaml
kubectl create -f cluster.yaml
kubectl create -f object.yaml
kubectl create -f object-user.yaml
kubectl -n rook-ceph describe secret rook-ceph-object-user-my-store-my-user
kubectl create -f rgw-external.yaml # probably redundant
echo "AccessKey:";kubectl -n rook-ceph get secret rook-ceph-object-user-my-store-my-user -o yaml | grep AccessKey | awk '{print }' | base64 --decode;printf "\n";echo "secret:";kubectl -n rook-ceph get secret rook-ceph-object-user-my-store-my-user -o yaml | grep SecretKey | awk '{print }' | base64 --decode;printf "\n"
Add route on top of rook-ceph-rgw-my-store. Name it, Keep path / and select the port
You should be able to navigate to http://<your name>-rook-ceph.apps..funcs.io/ and get 403. Now you can connect NooBaa to this url and the access and secret key.
https://access.redhat.com/documentation/en-us/red_hat_quay/2.9/html-single/deploy_red_hat_quay_on_openshift/index before running the lb yaml, edit the secret according to this article https://access.redhat.com/solutions/3533201
oc new-app -e POSTGRESQL_USER=quayuser -e POSTGRESQL_PASSWORD=quaypass -e POSTGRESQL_DATABASE=quaydb registry.access.redhat.com/rhscl/postgresql-96-rhel7
oc exec -it postgresql-96-rhel7-1-9vltm /bin/bash
echo "SELECT * FROM pg_available_extensions" | /opt/rh/rh-postgresql96/root/usr/bin/psql
echo "CREATE EXTENSION pg_trgm" | /opt/rh/rh-postgresql96/root/usr/bin/psql
echo "SELECT * FROM pg_extension" | /opt/rh/rh-postgresql96/root/usr/bin/psql
echo "ALTER USER quayuser WITH SUPERUSER;" | /opt/rh/rh-postgresql96/root/usr/bin/psql
Grab the postgresql-94-rhel7 cluster ip with:
oc get services -n quay-enterprise
Configure quay to use NooBaa https://noobaa.desk.com/customer/portal/articles/2970047-quay
Turn on debug for Quay - https://access.redhat.com/solutions/3663691
If you need to cleanup, but failed, use this one https://github.com/ctron/kill-kube-ns/blob/master/kill-kube-ns