Description

The goal of this issue is to make the SDP installable from other repos than oci.stackable.tech.

It is currently very hard (and sometimes impossible) to customize the registry/repository used by operators to construct the final product image name.
In order to fix that, the issue is tackled on a few different levels:

1. The CI pipelines must become the authoritative source of truth for our artifacts and where they are published. Currently, our operator and product container images as well as Helm Charts are only published to oci.stackable.tech. Artifacts available on quay.io/stackable are only mirrored from oci.stackable.tech and don't include Helm Charts. Going forward, we want to publish all artifacts directly to both registries via our CI pipelines.
2. Our Helm Charts carry information about their origin (where they are installed from). These pieces of data can then be used during deployment time to provide operators with a default registry used for product image selection/resolving.
3. Our operators provide a way to set the default image registry/repository used as a fallback during product image selection. Users must still be able to completely override the product image used if automatic selection is not desired.

Tasks

• Add Helm values merge mechanism
• Add registry-specific values file via operator-templating
• Add new CLI argument + env var to operators to set image registry/repository
• Adjust Helm templates to provide image registry/repository values to Deployment/DaemonSets via env var
• Roll out templating changes (Rollout, Search)
• Roll out fixed publish-helm-chart action (Rollout, Search)
• Adjust workflows in operator-templating to also publish to quay.io
• Roll out templating changes (blocked by Helm >= 3.18.x fails to push to quay.io helm/helm#32144)
• Publish product container images to quay.io: ci: Publish to quay.io docker-images#1492
• Adjust product image selection mechanism to use provided image registry/repository (Feature PR, Release PR)
• Rollout changes to all operators
  • feat: Support dynamic image repositories airflow-operator#784
  • commons-operator
  • feat: Support dynamic image repositories druid-operator#818
  • feat: Support dynamic image repositories hbase-operator#752
  • feat: Support dynamic image repositories hdfs-operator#778
  • feat: Support dynamic image repositories hive-operator#702
  • feat: Support dynamic image repositories kafka-operator#961
  • listener-operator
  • feat: Support dynamic image repositories nifi-operator#928
  • feat: Support dynamic image repositories opa-operator#830
  • feat: Support dynamic image repositories opensearch-operator#141
  • secret-operator
  • feat: Support dynamic image repositories spark-k8s-operator#684
  • feat: Support dynamic image repositories superset-operator#726
  • feat: Support dynamic image repositories trino-operator#884
  • feat: Support dynamic image repositories zookeeper-operator#1028
• Separate operator and product image repositories in Helm values.yaml (Feature PR, Rollout, Search)
• Core operators don't need to separate between operator and product image repository, because they are not managing products (Feature PR, Rollout, Search)
• Core operators also reference sig-storage images, which need to be handled in a different way

Acceptance criteria

1. Helm charts are available on both oci.stackable.tech as well as quay.io.
2. Helm charts on oci.stackable.tech reference images on that repo. Same for quay.io.
3. Operators are explicitly configured by the package managers (Helm and OLM currently) with a repo for product images.
4. Custom resources should not be affected, meaning that users which just specify spec.image.productVersion without providing a completely override should just work as before.

Relevant decisions

• https://github.com/stackabletech/decisions/issues/83
• https://github.com/stackabletech/decisions/issues/84
• https://github.com/stackabletech/decisions/issues/85