Informer pools

docs/content/en/docs/documentation/eventing.md

@@ -346,4 +346,70 @@ for [primary resources](https://github.com/operator-framework/java-operator-sdk/
 
 See
 also [CaffeineBoundedItemStores](https://github.com/operator-framework/java-operator-sdk/blob/main/caffeine-bounded-cache-support/src/main/java/io/javaoperatorsdk/operator/processing/event/source/cache/CaffeineBoundedItemStores.java)
-for more details.
+for more details.
+
+### Sharing Informers Between Controllers (Informer Pool)
+
+{{% alert title="Experimental" color="warning" %}}
+Informer pooling is marked `@Experimental`: the feature itself is production ready, but its
+configuration API may still change in a non-backwards-compatible way.
+{{% /alert %}}
+
+By default JOSDK maintains an *informer pool* so that informers are **shared** across controllers
+and event sources. When several `InformerEventSource`s (whether belonging to different controllers,
+or dynamically registered at runtime) watch the same resource type with an equivalent configuration,
+they are all backed by a single underlying `SharedIndexInformer` instead of one informer each. This
+reduces memory usage and the number of watch connections opened against the API server — which
+matters in operators where many controllers watch the same secondary resource type (for example
+`ConfigMap` or `Secret`).
+
+Two event sources share an informer when their effective informer configuration matches on all of:
+
+- the `KubernetesClient` they watch through, compared by instance: normally every event source
+  resolves the operator's own client, but an event source watching another cluster brings its own
+  (see [multi-cluster](#informereventsource-multi-cluster-support)). Two separate client instances
+  never share an informer, not even when they connect to the same API server — they may differ in
+  credentials, impersonation or TLS material, and the informer keeps using the client it was created
+  from,
+- the resource type (or the group/version/kind for generic resources),
+- the watched namespace,
+- the label, field and shard selectors,
+- the configured [item store](#bounded-caches-for-informers).
+
+The `informerListLimit` is intentionally *not* part of this identity: if two otherwise-equivalent
+event sources request a different list limit, the existing informer is reused (a warning is logged
+and the first-configured limit is kept). Indexers are also not part of the identity: they are
+registered on the shared informer under a name qualified with the controller and event source that
+added them, so index names are private to an event source and cannot collide with those of another
+one. You keep looking indexes up by the name you registered, and the indexers of an event source are
+removed from the shared informer when it stops using it.
+
+The pool is reference-counted: the shared informer is created on first use and only stopped once the
+last event source using it is de-registered (or its controller stops). Dynamically registering an
+event source for a resource that is already backed by a running informer reuses that informer, and
+the initial state already in its cache is replayed to the newly added handler.
+
+#### Selecting the pooling strategy
+
+The strategy is provided by the
+[`InformerPool`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/pool/InformerPool.java)
+configured on the `ConfigurationService`. Two implementations are available:
+
+- [`DefaultInformerPool`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/pool/DefaultInformerPool.java)
+  (the default): shares informers as described above.
+- [`NonSharingInformerPool`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/pool/NonSharingInformerPool.java):
+  never shares informers, creating a dedicated informer for every event source. Use this to opt out
+  of pooling and restore the pre-pooling behavior.
+
+You can override the strategy through the `ConfigurationService`:
+
+```java
+Operator operator = new Operator(overrider ->
+    overrider.withInformerPool(new NonSharingInformerPool()));
+```
+
+A custom strategy has to extend
+[`AbstractInformerPool`](https://github.com/operator-framework/java-operator-sdk/blob/main/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/informer/pool/AbstractInformerPool.java),
+which is what `withInformerPool` accepts: it already creates the informers from an
+`InformerClassifier` and starts them, leaving the subclass to decide only whether and how they are
+shared. `InformerPool` itself is just the narrower contract that the event sources consume.

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/AbstractConfigurationService.java

@@ -24,6 +24,9 @@
 import io.fabric8.kubernetes.client.KubernetesClient;
 import io.javaoperatorsdk.operator.ReconcilerUtilsInternal;
 import io.javaoperatorsdk.operator.api.reconciler.Reconciler;
+import io.javaoperatorsdk.operator.processing.event.source.informer.pool.AbstractInformerPool;
+import io.javaoperatorsdk.operator.processing.event.source.informer.pool.DefaultInformerPool;
+import io.javaoperatorsdk.operator.processing.event.source.informer.pool.InformerPool;
 
 /**
  * An abstract implementation of {@link ConfigurationService} meant to ease custom implementations
@@ -35,6 +38,7 @@ public class AbstractConfigurationService implements ConfigurationService {
   private KubernetesClient client;
   private Cloner cloner;
   private ExecutorServiceManager executorServiceManager;
+  private AbstractInformerPool informerPool;
 
   protected AbstractConfigurationService(Version version) {
     this(version, null);
@@ -190,4 +194,16 @@ public ExecutorServiceManager getExecutorServiceManager() {
     }
     return executorServiceManager;
   }
+
+  @Override
+  public synchronized InformerPool informerPool() {
+    // cached so that all controllers backed by this ConfigurationService share the same pool and
+    // can therefore share the underlying informers; synchronized so concurrent first-access from
+    // multiple controllers cannot create (and share out) more than one pool instance
+    if (informerPool == null) {
+      informerPool = new DefaultInformerPool();
+      informerPool.setConfigurationService(this);
+    }
+    return informerPool;
+  }
 }

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ConfigurationService.java

@@ -36,13 +36,15 @@
 import io.fabric8.kubernetes.client.utils.KubernetesSerialization;
 import io.javaoperatorsdk.operator.api.monitoring.Metrics;
 import io.javaoperatorsdk.operator.api.reconciler.Context;
+import io.javaoperatorsdk.operator.api.reconciler.Experimental;
 import io.javaoperatorsdk.operator.api.reconciler.Reconciler;
 import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResourceFactory;
 import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependent;
 import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResource;
 import io.javaoperatorsdk.operator.processing.dependent.kubernetes.KubernetesDependentResourceConfig;
 import io.javaoperatorsdk.operator.processing.dependent.workflow.ManagedWorkflowFactory;
 import io.javaoperatorsdk.operator.processing.event.source.controller.ControllerEventSource;
+import io.javaoperatorsdk.operator.processing.event.source.informer.pool.InformerPool;
 
 /** An interface from which to retrieve configuration information. */
 public interface ConfigurationService {
@@ -476,4 +478,23 @@ default boolean useSSAToPatchPrimaryResource() {
   default boolean cloneSecondaryResourcesWhenGettingFromCache() {
     return false;
   }
+
+  /**
+   * The informer pool used to create and (when using the default, sharing pool) share the informers
+   * backing the event sources of all controllers managed by this {@code ConfigurationService}.
+   *
+   * <p><strong>Implementations must return the same instance on every call.</strong> The pool is
+   * effectively a per-{@code ConfigurationService} singleton: controllers share informers only if
+   * they resolve the same pool, and reference counting / informer shutdown are only correct if
+   * {@code getInformer} and {@code releaseInformer} operate on that same instance. This is
+   * intentionally not a {@code default} method, since a {@code default} could not cache the result
+   * and would hand out a fresh (unshared) pool on each call; {@link AbstractConfigurationService}
+   * provides a cached implementation backed by the default sharing pool.
+   *
+   * @return the informer pool for this configuration service
+   */
+  @Experimental(
+      "Only the configuration API around informer pooling could still change in a"
+          + " non-backwards-compatible way, the pooling itself is prod ready.")
+  InformerPool informerPool();
 }

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/ConfigurationServiceOverrider.java

@@ -28,7 +28,10 @@
 import io.fabric8.kubernetes.client.KubernetesClient;
 import io.javaoperatorsdk.operator.Operator;
 import io.javaoperatorsdk.operator.api.monitoring.Metrics;
+import io.javaoperatorsdk.operator.api.reconciler.Experimental;
 import io.javaoperatorsdk.operator.api.reconciler.dependent.DependentResourceFactory;
+import io.javaoperatorsdk.operator.processing.event.source.informer.pool.AbstractInformerPool;
+import io.javaoperatorsdk.operator.processing.event.source.informer.pool.InformerPool;
 
 @SuppressWarnings({"unused", "UnusedReturnValue"})
 public class ConfigurationServiceOverrider {
@@ -53,6 +56,7 @@ public class ConfigurationServiceOverrider {
   private Set<Class<? extends HasMetadata>> defaultNonSSAResource;
   private Boolean useSSAToPatchPrimaryResource;
   private Boolean cloneSecondaryResourcesWhenGettingFromCache;
+  private InformerPool informerPool;
 
   @SuppressWarnings("rawtypes")
   private DependentResourceFactory dependentResourceFactory;
@@ -176,6 +180,21 @@ public ConfigurationServiceOverrider withCloneSecondaryResourcesWhenGettingFromC
     return this;
   }
 
+  /**
+   * Overrides the informer pool strategy used to create/share the informers backing the event
+   * sources. When not set, the default (informer-sharing) pool is used.
+   *
+   * <p>Custom strategies extend {@link AbstractInformerPool}, which already takes care of creating
+   * and starting the informers.
+   */
+  @Experimental(
+      "Only the configuration API around informer pooling could still change in a"
+          + " non-backwards-compatible way, the pooling itself is prod ready.")
+  public ConfigurationServiceOverrider withInformerPool(AbstractInformerPool informerPool) {
+    this.informerPool = informerPool;
+    return this;
+  }
+
   public ConfigurationService build() {
     return new BaseConfigurationService(original.getVersion(), cloner, client) {
       @Override
@@ -309,6 +328,15 @@ public boolean cloneSecondaryResourcesWhenGettingFromCache() {
             cloneSecondaryResourcesWhenGettingFromCache,
             ConfigurationService::cloneSecondaryResourcesWhenGettingFromCache);
       }
+
+      @Override
+      public InformerPool informerPool() {
+        if (informerPool == null) {
+          return super.informerPool();
+        }
+        informerPool.setConfigurationService(this);
+        return informerPool;
+      }
     };
   }
 }

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/informer/FieldSelector.java

@@ -17,6 +17,7 @@
 
 import java.util.Arrays;
 import java.util.List;
+import java.util.Objects;
 
 public class FieldSelector {
   private final List<Field> fields;
@@ -38,4 +39,21 @@ public Field(String path, String value) {
       this(path, value, false);
     }
   }
+
+  @Override
+  public boolean equals(Object o) {
+    if (o == null || getClass() != o.getClass()) return false;
+    FieldSelector that = (FieldSelector) o;
+    return Objects.equals(fields, that.fields);
+  }
+
+  @Override
+  public int hashCode() {
+    return Objects.hashCode(fields);
+  }
+
+  @Override
+  public String toString() {
+    return "FieldSelector{" + "fields=" + fields + '}';
+  }
 }

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/informer/InformerConfiguration.java

@@ -30,6 +30,7 @@
 import io.javaoperatorsdk.operator.api.config.ControllerConfiguration;
 import io.javaoperatorsdk.operator.api.config.Utils;
 import io.javaoperatorsdk.operator.api.reconciler.Constants;
+import io.javaoperatorsdk.operator.processing.GroupVersionKind;
 import io.javaoperatorsdk.operator.processing.event.source.cache.BoundedItemStore;
 import io.javaoperatorsdk.operator.processing.event.source.filter.GenericFilter;
 import io.javaoperatorsdk.operator.processing.event.source.filter.OnAddFilter;
@@ -42,6 +43,7 @@
 public class InformerConfiguration<R extends HasMetadata> {
   private final Builder builder = new Builder();
   private final Class<R> resourceClass;
+  private final GroupVersionKind resourceGroupVersionKind;
   private final String resourceTypeName;
   private String name;
   private Set<String> namespaces;
@@ -59,6 +61,7 @@ public class InformerConfiguration<R extends HasMetadata> {
 
   protected InformerConfiguration(
       Class<R> resourceClass,
+      GroupVersionKind resourceGroupVersionKind,
       String name,
       Set<String> namespaces,
       boolean followControllerNamespaceChanges,
@@ -74,7 +77,7 @@ protected InformerConfiguration(
       Boolean comparableResourceVersions,
       // TODO for removal in major release
       Duration ghostResourceCacheCheckInterval) {
-    this(resourceClass);
+    this(resourceClass, resourceGroupVersionKind);
     this.name = name;
     this.namespaces = namespaces;
     this.followControllerNamespaceChanges = followControllerNamespaceChanges;
@@ -90,9 +93,14 @@ protected InformerConfiguration(
     this.comparableResourceVersions = comparableResourceVersions;
   }
 
-  private InformerConfiguration(Class<R> resourceClass) {
+  private InformerConfiguration(Class<R> resourceClass, GroupVersionKind resourceGroupVersionKind) {
     this.resourceClass = resourceClass;
+    this.resourceGroupVersionKind = resourceGroupVersionKind;
     this.resourceTypeName =
+        // note the direction: this is true for GenericKubernetesResource, but also when the
+        // resource
+        // class is a supertype of it - i.e. a plain HasMetadata, for which no type name can be
+        // resolved from @Group/@Version annotations
         resourceClass.isAssignableFrom(GenericKubernetesResource.class)
             // in general this is irrelevant now for secondary resources it is used just by
             // controller
@@ -101,17 +109,24 @@ private InformerConfiguration(Class<R> resourceClass) {
             : ReconcilerUtilsInternal.getResourceTypeName(resourceClass);
   }
 
+  @SuppressWarnings({"rawtypes", "unchecked"})
+  public static <R extends HasMetadata> InformerConfiguration<R>.Builder builder(
+      Class<R> resourceClass, GroupVersionKind groupVersionKind) {
+    return new InformerConfiguration(resourceClass, groupVersionKind).builder;
+  }
+
   @SuppressWarnings({"rawtypes", "unchecked"})
   public static <R extends HasMetadata> InformerConfiguration<R>.Builder builder(
       Class<R> resourceClass) {
-    return new InformerConfiguration(resourceClass).builder;
+    return new InformerConfiguration(resourceClass, null).builder;
   }
 
   @SuppressWarnings({"rawtypes", "unchecked"})
   public static <R extends HasMetadata> InformerConfiguration<R>.Builder builder(
       InformerConfiguration<R> original) {
     return new InformerConfiguration(
             original.resourceClass,
+            original.resourceGroupVersionKind,
             original.name,
             original.namespaces,
             original.followControllerNamespaceChanges,
@@ -305,6 +320,10 @@ public Long getInformerListLimit() {
     return informerListLimit;
   }
 
+  public GroupVersionKind getResourceGroupVersionKind() {
+    return resourceGroupVersionKind;
+  }
+
   public FieldSelector getFieldSelector() {
     return fieldSelector;
   }
@@ -500,10 +519,20 @@ public Builder withInformerListLimit(Long informerListLimit) {
     }
 
     public Builder withFieldSelector(FieldSelector fieldSelector) {
-      InformerConfiguration.this.fieldSelector = fieldSelector;
+      // an empty selector filters nothing, so it must not be distinguishable from having none at
+      // all: the informer pool keys on the field selector, and the annotation path always builds
+      // one (@Informer#fieldSelector defaults to {}) where the programmatic path leaves it null,
+      // which would otherwise stop the two from sharing an informer
+      InformerConfiguration.this.fieldSelector = isEmpty(fieldSelector) ? null : fieldSelector;
       return this;
     }
 
+    private static boolean isEmpty(FieldSelector fieldSelector) {
+      return fieldSelector == null
+          || fieldSelector.getFields() == null
+          || fieldSelector.getFields().isEmpty();
+    }
+
     public Builder withComparableResourceVersions(boolean comparableResourceVersions) {
       InformerConfiguration.this.comparableResourceVersions = comparableResourceVersions;
       return this;

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/config/informer/InformerEventSourceConfiguration.java

@@ -76,6 +76,7 @@ default boolean followControllerNamespaceChanges() {
 
   <P extends HasMetadata> PrimaryToSecondaryMapper<P> getPrimaryToSecondaryMapper();
 
+  // todo deprecate
   Optional<GroupVersionKind> getGroupVersionKind();
 
   default String name() {
@@ -167,7 +168,7 @@ private Builder(
       this.resourceClass = resourceClass;
       this.groupVersionKind = groupVersionKind;
       this.primaryResourceClass = primaryResourceClass;
-      this.config = InformerConfiguration.builder(resourceClass);
+      this.config = InformerConfiguration.builder(resourceClass, groupVersionKind);
     }
 
     public Builder<R> withName(String name) {

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/GroupVersionKind.java

@@ -136,4 +136,8 @@ public int hashCode() {
   public String toString() {
     return toGVKString();
   }
+
+  public String getApiVersion() {
+    return apiVersion;
+  }
 }

operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/dependent/kubernetes/GroupVersionKindPlural.java

@@ -53,10 +53,11 @@ protected GroupVersionKindPlural(GroupVersionKind gvk, String plural) {
 
   @Override
   protected boolean specificEquals(GroupVersionKind that) {
-    if (plural == null) {
-      return true;
-    }
-    return that instanceof GroupVersionKindPlural gvkp && gvkp.plural.equals(plural);
+    // a GroupVersionKind that is not plural-aware carries no plural form, which is the same as an
+    // unspecified one: that keeps this consistent with hashCode(), which only mixes the plural in
+    // when it is present
+    final var thatPlural = that instanceof GroupVersionKindPlural gvkp ? gvkp.plural : null;
+    return Objects.equals(plural, thatPlural);
   }
 
   @Override