Skip to content

WIP: Catalog Source weighting for dependency resolution #1681

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 40 commits into from
Closed

WIP: Catalog Source weighting for dependency resolution #1681

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
ef888a2
(feat): Add sat resolver for version dependencies
kevinrizza Apr 20, 2020
be63c21
Extract dependencies info from bundle and add to Operator object
dinhxuanvu Apr 20, 2020
180cd1d
Strip plural from APISets in resolver cache
kevinrizza Apr 20, 2020
be5dd33
Fix unit tests and add GVK dependencies extraction
dinhxuanvu Apr 20, 2020
dfcdbca
Adapt resolver unit tests to run with both implementations.
benluddy Jul 8, 2020
0da2b35
use new solver interface in satresolver
ecordell Jul 10, 2020
5cb0eee
support semver ranges
ecordell Jul 10, 2020
7e801b2
memoize bundle walking
ecordell Jul 10, 2020
2981128
Decouple filter predicates from the operator cache itself.
benluddy Jul 10, 2020
0d3121b
don't mark virtual package nodes visited
ecordell Jul 10, 2020
ed45229
chore(deps): bump operator-registry
ecordell Jul 12, 2020
c21d3e8
resolver can resolve updates
ecordell Jul 12, 2020
ac8b60c
use cardinality-based AtMost constraint
ecordell Jul 12, 2020
7c321c6
move csvname filter to a predicate when finding bundles in the cache
ecordell Jul 12, 2020
9acfb8f
search for operators in catalog order
ecordell Jul 12, 2020
afe28d7
test updating to a package with skiprange that is not head
ecordell Jul 14, 2020
b18909d
split old/new resolver and tests
ecordell Jul 14, 2020
d533b62
pin etcd and grpc
ecordell Jul 15, 2020
8868dea
support startingCSV
ecordell Jul 15, 2020
73842c7
support skip list in bundles
ecordell Jul 15, 2020
db02e7a
refactor dependency walking to avoid recursing
ecordell Jul 15, 2020
fb40664
treat dependencies generically
ecordell Jul 15, 2020
53ed8f8
rename solve package
ecordell Jul 16, 2020
d9f11df
renaming / refactoring
ecordell Jul 16, 2020
a66dc41
ensure cache is filled with content before marking it as not a miss
ecordell Jul 22, 2020
48fbb82
fix verify
ecordell Jul 17, 2020
399feaa
wip: fixing resolver tests
ecordell Jul 17, 2020
6141359
fix nested dependency resolution
ecordell Jul 18, 2020
a2f42b0
fix resolver bugs, fix cache race, and re-enable tests
ecordell Jul 18, 2020
bb1c762
prefer the same catalog for dependencies
ecordell Jul 18, 2020
cd2a49f
enable new resolver by default
ecordell Jul 18, 2020
eb138c8
fix sorting of inputs for dependencies
ecordell Jul 20, 2020
0a4b036
expire cache when grpc connection is READY
ecordell Jul 20, 2020
ad4a104
deduplicate input installables
ecordell Jul 22, 2020
c60c8dc
Bump operator-registry dependency to v1.13.3.
benluddy Jul 22, 2020
b949331
Regenerate registry client test fake.
benluddy Jul 22, 2020
0d29da8
Try to detect kubebuilder installation path if not explicitly set.
benluddy Jul 22, 2020
d0f18e7
Prefer bundles from the default channel in resolution.
benluddy Jul 22, 2020
fcf5a51
Fix check for whether or not a sub was added during resolution.
benluddy Jul 22, 2020
21852b2
Added catalog weighting
harishsurf Jul 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
11 changes: 6 additions & 5 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ LOCAL_NAMESPACE := "olm"
export GO111MODULE=on
CONTROLLER_GEN := go run $(MOD_FLAGS) ./vendor/sigs.k8s.io/controller-tools/cmd/controller-gen
YQ_INTERNAL := go run $(MOD_FLAGS) ./vendor/github.com/mikefarah/yq/v2/
KUBEBUILDER_ASSETS ?= /usr/local/kubebuilder/bin
KUBEBUILDER_ASSETS := $(or $(KUBEBUILDER_ASSETS),$(dir $(shell command -v kubebuilder)))
export KUBEBUILDER_ASSETS

# ART builds are performed in dist-git, with content (but not commits) copied
# from the source repo. Thus at build time if your code is inspecting the local
Expand All @@ -43,7 +44,7 @@ all: test build
test: clean cover.out

unit: kubebuilder
KUBEBUILDER_ASSETS=$(KUBEBUILDER_ASSETS) go test $(MOD_FLAGS) $(SPECIFIC_UNIT_TEST) -v -race -count=1 ./pkg/...
KUBEBUILDER_ASSETS=$(KUBEBUILDER_ASSETS) go test $(MOD_FLAGS) $(SPECIFIC_UNIT_TEST) -tags "json1" -v -race -count=1 ./pkg/...

# Ensure kubebuilder is installed before continuing
KUBEBUILDER_ASSETS_ERR := not detected in $(KUBEBUILDER_ASSETS), to override the assets path set the KUBEBUILDER_ASSETS environment variable, for install instructions see https://book.kubebuilder.io/quick-start.html
Expand All @@ -61,7 +62,7 @@ endif
schema-check:

cover.out: schema-check
go test $(MOD_FLAGS) -v -race -coverprofile=cover.out -covermode=atomic \
go test $(MOD_FLAGS) -tags "json1" -v -race -coverprofile=cover.out -covermode=atomic \
-coverpkg ./pkg/controller/... ./pkg/...

coverage: cover.out
Expand Down Expand Up @@ -101,7 +102,7 @@ bin/cpb:

$(CMDS): version_flags=-ldflags "-X $(PKG)/pkg/version.GitCommit=$(GIT_COMMIT) -X $(PKG)/pkg/version.OLMVersion=`cat OLM_VERSION`"
$(CMDS):
$(arch_flags) go $(build_cmd) $(MOD_FLAGS) $(version_flags) -o bin/$(shell basename $@) $@
$(arch_flags) go $(build_cmd) $(MOD_FLAGS) $(version_flags) -tags "json1" -o bin/$(shell basename $@) $@

build: clean $(CMDS)

Expand Down Expand Up @@ -255,4 +256,4 @@ run-local: build-linux build-wait build-util-linux
mkdir -p build/resources
. ./scripts/package_release.sh 1.0.0 build/resources doc/install/local-values.yaml
. ./scripts/install_local.sh $(LOCAL_NAMESPACE) build/resources
rm -rf build
rm -rf build
11 changes: 10 additions & 1 deletion cmd/catalog/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ const (
defaultConfigMapServerImage = "quay.io/operatorframework/configmap-operator-registry:latest"
defaultUtilImage = "quay.io/operator-framework/olm:latest"
defaultOperatorName = ""
resolverV2EnableEnvVarName = "RESOLVER_V2_ENABLE"
)

// config flags defined globally so that they appear on the test binary as well
Expand Down Expand Up @@ -68,6 +69,9 @@ var (

profiling = flag.Bool(
"profiling", false, "serve profiling data (on port 8080)")

resolverV2Enable = flag.Bool(
resolverV2EnableEnvVarName, true, "using resolver V2")
)

func init() {
Expand Down Expand Up @@ -102,6 +106,11 @@ func main() {
*catalogNamespace = catalogNamespaceEnvVarValue
}

if resolverV2EnableEnvVarValue := os.Getenv(resolverV2EnableEnvVarName); resolverV2EnableEnvVarValue != "" {
logger.Infof("%s environment variable is set. Updating resolverV2Enable to be true", resolverV2EnableEnvVarName)
*resolverV2Enable = true
}

var useTLS bool
if *tlsCertPath != "" && *tlsKeyPath == "" || *tlsCertPath == "" && *tlsKeyPath != "" {
logger.Warn("both --tls-key and --tls-crt must be provided for TLS to be enabled, falling back to non-https")
Expand Down Expand Up @@ -172,7 +181,7 @@ func main() {
}

// Create a new instance of the operator.
op, err := catalog.NewOperator(ctx, *kubeConfigPath, utilclock.RealClock{}, logger, *wakeupInterval, *configmapServerImage, *utilImage, *catalogNamespace)
op, err := catalog.NewOperator(ctx, *kubeConfigPath, utilclock.RealClock{}, logger, *wakeupInterval, *configmapServerImage, *utilImage, *catalogNamespace, *resolverV2Enable)
if err != nil {
log.Panicf("error configuring operator: %s", err.Error())
}
Expand Down
15 changes: 10 additions & 5 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ require (
github.com/go-logr/logr v0.1.0
github.com/go-openapi/spec v0.19.4
github.com/golang/mock v1.3.1
github.com/google/go-cmp v0.4.0
github.com/google/go-cmp v0.5.0
github.com/irifrance/gini v1.0.1
github.com/maxbrunsfeld/counterfeiter/v6 v6.2.2
github.com/mikefarah/yq/v2 v2.4.1
Expand All @@ -23,8 +23,8 @@ require (
github.com/openshift/api v0.0.0-20200331152225-585af27e34fd
github.com/openshift/client-go v0.0.0-20200326155132-2a6cd50aedd0
github.com/operator-framework/api v0.3.7
github.com/operator-framework/operator-registry v1.12.2
github.com/otiai10/copy v1.0.2
github.com/operator-framework/operator-registry v1.13.3
github.com/otiai10/copy v1.2.0
github.com/pkg/errors v0.9.1
github.com/prometheus/client_golang v1.2.1
github.com/prometheus/client_model v0.2.0
Expand All @@ -35,7 +35,7 @@ require (
github.com/stretchr/testify v1.5.1
github.com/yalp/jsonpath v0.0.0-20180802001716-5cc68e5049a0
golang.org/x/time v0.0.0-20191024005414-555d28b269f0
google.golang.org/grpc v1.27.0
google.golang.org/grpc v1.30.0
gopkg.in/yaml.v2 v2.2.8
helm.sh/helm/v3 v3.1.2
k8s.io/api v0.18.2
Expand All @@ -48,6 +48,7 @@ require (
k8s.io/klog v1.0.0
k8s.io/kube-aggregator v0.18.0
k8s.io/kube-openapi v0.0.0-20200121204235-bf4fb3bd569c
rsc.io/letsencrypt v0.0.3 // indirect
sigs.k8s.io/controller-runtime v0.6.0
sigs.k8s.io/controller-tools v0.3.0
sigs.k8s.io/kind v0.7.0
Expand All @@ -60,7 +61,11 @@ replace (
github.com/openshift/client-go => github.com/openshift/client-go v0.0.0-20200326155132-2a6cd50aedd0 // release-4.5

github.com/prometheus/client_golang => github.com/prometheus/client_golang v0.9.3-0.20190127221311-3c4408c8b829
google.golang.org/grpc => google.golang.org/grpc v1.26.0 // https://github.com/etcd-io/etcd/issues/11563

// pinned because latest etcd does not yet work with the latest grpc version (1.30.0)
go.etcd.io/etcd => go.etcd.io/etcd v0.5.0-alpha.5.0.20200520232829-54ba9589114f
google.golang.org/grpc => google.golang.org/grpc v1.27.0
google.golang.org/grpc/examples => google.golang.org/grpc/examples v0.0.0-20200709232328-d8193ee9cc3e

// pinned because no tag supports 1.18 yet
helm.sh/helm/v3 => helm.sh/helm/v3 v3.1.0-rc.1.0.20200416205415-853ba2de16a0
Expand Down
Loading