monitor secret resources ony if necessary

[kishorj]

Issue

Fixes: #2159

Description

The controller until this fix required access to all secret resources in the cluster. The secret resources were used for OIDC feature for the clientID and clientSecret stored in the secret resource. The controller setup a watch during initialization in order to reconcile the affected ingress group and and depending on the number of secrets in the cluster, there is a potential for slower initialization and increased resource usage.

The secret access is not required when OIDC feature is not in use. Furthermore, we can restrict the controller to specific secrets required for the OIDC feature. This PR contains the following changes:

• removes the default watch for all secret resources in the cluster
• create watch for specific secret resource if required for OIDC feature
• do not configure rbac permissions for secrets by default
• provide helm chart configuration options for specific secret resources

Tests

• helm install does not create any permissions for secrets resource by default
• controller works without secret permissions; is able to reconcile ingress/service resources
• if oidc is configured, reconcile fails without rbac configuration
• reconcile works eventually if rbac gets configured
• any change to the secret resource causes affected ingress to be reconciled
• if rbac permission is revoked, controller is not able to reconcile the affected ingresses until the permission is restored

Checklist

• Added tests that cover your change (if possible)
• Added/modified documentation as required (such as the README.md, or the docs directory)
• Manually tested
• Made sure the title of the PR is a good description that can go into the release notes

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

• Backfilled missing tests for code in same general area 🎉
• Refactored something and made the world a better place 🌟

[k8s-ci-robot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[codecov-commenter]

Codecov Report

Merging #2550 (8b196a7) into main (7baec17) will increase coverage by 0.17%.
The diff coverage is 71.42%.

@@            Coverage Diff             @@
##             main    #2550      +/-   ##
==========================================
+ Coverage   53.52%   53.70%   +0.17%     
==========================================
  Files         142      144       +2     
  Lines        8039     8121      +82     
==========================================
+ Hits         4303     4361      +58     
- Misses       3414     3438      +24     
  Partials      322      322              

Impacted Files	Coverage Δ
pkg/config/runtime_config.go	0.00% <0.00%> (ø)
pkg/k8s/secrets_store.go	17.85% <17.85%> (ø)
pkg/ingress/model_build_actions.go	54.90% <100.00%> (+0.29%)	⬆️
pkg/ingress/model_builder.go	64.50% <100.00%> (ø)
pkg/k8s/secrets_manager.go	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7baec17...8b196a7. Read the comment docs.

[M00nF1sh]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kishorj, M00nF1sh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [M00nF1sh,kishorj]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment