Add Helm overrides for env vars, volumes and volume mounts

[RedbackThomson]

Issue #, if available: aws-controllers-k8s/community#919

Description of changes:
Adds additional Helm chart value overrides for deployment environment variables, volumes and volume mounts.

When deploying the s3-controller with the following values.yaml overrides:

deployment:
  extraVolumes:
    - name: test-volume
      secret:
        secretName: test-secret
  extraVolumeMounts:
    - name: test-volume-mount
      mountPath: /root/
  extraEnvVars: 
    - name: SOMEVAR
      value: somevalue
    - name: PASSWORD
      valueFrom:
        secretKeyRef:
          name: mysecret
          key: password
          optional: false

helm template produces the following output:

# Source: s3-chart/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: release-name-s3-chart
  namespace: default
  labels:
    app.kubernetes.io/name: s3-chart
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/version: "1.0.4"
    k8s-app: s3-chart
    helm.sh/chart: s3-chart-1.0.4
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: s3-chart
      app.kubernetes.io/instance: release-name
  template:
    metadata:
      labels:
        app.kubernetes.io/name: s3-chart
        app.kubernetes.io/instance: release-name
        app.kubernetes.io/managed-by: Helm
        k8s-app: s3-chart
    spec:
      serviceAccountName: ack-s3-controller
      containers:
      - command:
        - ./bin/controller
        args:
        - --aws-region
        - "$(AWS_REGION)"
        - --aws-endpoint-url
        - "$(AWS_ENDPOINT_URL)"
        - --enable-development-logging
        - "$(ACK_ENABLE_DEVELOPMENT_LOGGING)"
        - --log-level
        - "$(ACK_LOG_LEVEL)"
        - --resource-tags
        - "$(ACK_RESOURCE_TAGS)"
        - --watch-namespace
        - "$(ACK_WATCH_NAMESPACE)"
        - --deletion-policy
        - "$(DELETION_POLICY)"
        image: public.ecr.aws/aws-controllers-k8s/s3-controller:1.0.4
        imagePullPolicy: IfNotPresent
        name: controller
        ports:
          - name: http
            containerPort: 8080
        resources:
          limits:
            cpu: 100m
            memory: 128Mi
          requests:
            cpu: 50m
            memory: 64Mi
        env:
        - name: ACK_SYSTEM_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: AWS_REGION
          value:
        - name: AWS_ENDPOINT_URL
          value: ""
        - name: ACK_WATCH_NAMESPACE
          value:
        - name: DELETION_POLICY
          value: delete
        - name: ACK_ENABLE_DEVELOPMENT_LOGGING
          value: "false"
        - name: ACK_LOG_LEVEL
          value: "info"
        - name: ACK_RESOURCE_TAGS
          value: "services.k8s.aws/controller-version=%CONTROLLER_SERVICE%-%CONTROLLER_VERSION%,services.k8s.aws/namespace=%K8S_NAMESPACE%"
        - name: SOMEVAR
          value: somevalue
        - name: PASSWORD
          valueFrom:
            secretKeyRef:
              key: password
              name: mysecret
              optional: false
        volumeMounts:
            - mountPath: /root/
              name: test-volume-mount
        securityContext:
          allowPrivilegeEscalation: false
          privileged: false
          runAsNonRoot: true
          capabilities:
            drop:
              - ALL
      securityContext:
        seccompProfile:
          type: RuntimeDefault
      terminationGracePeriodSeconds: 10
      nodeSelector:
        kubernetes.io/os: linux
      hostIPC: false
      hostNetwork: false
      hostPID: false
      volumes:
        - name: test-volume
          secret:
            secretName: test-secret

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[jaypipes]

instead of extraXXX can we call these variables additionalXXX?

Other than that, I dig it :)

[RedbackThomson]

instead of extraXXX can we call these variables additionalXXX?

Modeled these from the prometheus operators Helm chart, which is one of the most extensive charts I've seen before. https://github.com/prometheus-community/helm-charts/tree/main/charts/prometheus

[jaypipes]

Fair enough @RedbackThomson. Would be awesome to include the example commented out values like in the Prometheus charts, too.

[a-hilaly]

Good stuff, thank you!
/lgtm

[ack-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: A-Hilaly, jaypipes, RedbackThomson

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [A-Hilaly,RedbackThomson,jaypipes]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment