Pool Provisioning¶

Overview¶

The NextcloudPool CRD enables fast tenant onboarding by maintaining a pool of pre-provisioned, unassigned NextcloudInstances. When a tenant creates a Nextcloud resource, the operator assigns an existing instance from the pool (~30s) instead of creating one from scratch (~2min).

Architecture¶

┌────────────────────────────────────────────────────────────┐
│ Cluster-Scoped                                             │
│                                                            │
│  ┌──────────────────────┐  ┌──────────────────────┐       │
│  │ NextcloudPool        │  │ NextcloudPool        │       │
│  │ production           │  │ development          │       │
│  │                      │  │                      │       │
│  │ spec:                │  │ spec:                │       │
│  │   replicas: 5        │  │   replicas: 2        │       │
│  │   profile: prod      │  │   profile: dev       │       │
│  │                      │  │                      │       │
│  │ status:              │  │ status:              │       │
│  │   ready: 5           │  │   ready: 2           │       │
│  │   assigned: 0        │  │   assigned: 0        │       │
│  └──────────────────────┘  └──────────────────────┘       │
└────────────────────────────────────────────────────────────┘
         │                            │
         │ manages                    │ manages
         ▼                            ▼
┌────────────────────────────────────────────────────────────┐
│ Instance Namespaces                                        │
│                                                            │
│  Pool instances (unassigned):                              │
│  ┌──────────────────────┐  ┌──────────────────────┐       │
│  │ NextcloudInstance    │  │ NextcloudInstance    │       │
│  │ nc-brave-lake-g7h8i9 │  │ nc-quiet-hill-j0k1l2│       │
│  │ labels:              │  │ labels:              │       │
│  │   assigned: "false"  │  │   assigned: "false"  │       │
│  └──────────────────────┘  └──────────────────────┘       │
│                                                            │
│  Assigned instances:                                       │
│  ┌──────────────────────┐                                  │
│  │ NextcloudInstance    │                                  │
│  │ nc-happy-sun-a1b2c3  │                                  │
│  │ labels:              │                                  │
│  │   assigned: "true"   │                                  │
│  │   tenant: acme-corp  │                                  │
│  └──────────────────────┘                                  │
└────────────────────────────────────────────────────────────┘

How It Works¶

Pool-Based Assignment Flow¶

1. Tenant creates Nextcloud CR
   └─> kubectl apply -f nextcloud.yaml

2. Operator checks poolSelector
   └─> spec.poolSelector.matchLabels: {pool: "production"}

3. Operator finds matching unassigned instance
   └─> Labels: {assigned: "false", pool: "production"}
   └─> Found: nc-brave-lake-g7h8i9

4. Operator assigns instance
   ├─> Updates NextcloudInstance labels (assigned: "true")
   ├─> Copies Nextcloud.spec to NextcloudInstance.spec
   └─> Updates Nextcloud.status.instanceRef

5. NextcloudInstance reconciles with new spec
   └─> Updates HelmRelease, Secrets, etc.

6. Pool operator detects assignment
   └─> Creates new unassigned instance to maintain pool size

Fresh Instance Creation (No Pool)¶

If no poolSelector is specified or no matching instances are available:

1. Operator creates fresh NextcloudInstance
2. Copies entire spec from Nextcloud
3. NextcloudInstance creates all resources (~2min)

Creating a Pool¶

apiVersion: k8s.bnerd.com/v1alpha1
kind: NextcloudPool
metadata:
  name: production
spec:
  replicas: 5
  instanceNamespacePattern: "*"
  template:
    metadata:
      labels:
        pool: production
    spec:
      profile: production
      database:
        managed: true
        type: postgresql
  lifecycle:
    recreateOnProfileChange: true
    maxUnassignedAge: "168h"    # 7 days
    reclaimPolicy: Delete

Using a Pool¶

Create a Nextcloud resource with a poolSelector:

apiVersion: k8s.bnerd.com/v1alpha1
kind: Nextcloud
metadata:
  name: tenant-cloud
  namespace: tenant-acme
spec:
  poolSelector:
    matchLabels:
      pool: production
  ingress:
    host: cloud.acme-corp.example.com
    tls:
      enabled: true
  # Additional spec fields are applied to the assigned instance

Pool Status¶

kubectl get nextcloudpools

# Detailed pool status
kubectl describe nextcloudpool production

The pool status shows instance counts:

status:
  phase: Ready
  replicas: 5
  ready: 5
  unassigned: 4
  assigned: 1
  instances:
    - name: nc-brave-lake-g7h8i9
      phase: Ready
      assigned: false
    - name: nc-happy-sun-a1b2c3
      phase: Ready
      assigned: true

Status Phases¶

NextcloudPool: Pending -> Scaling -> Ready | Failed

Nextcloud (logical): Pending -> Assigning -> Configuring -> Ready | Failed

Instance Naming¶

Pool instances use a human-readable naming convention:

Format: nc-{adjective}-{noun}-{random}

Examples: nc-happy-sun-a1b2c3, nc-calm-moon-d4e5f6, nc-brave-lake-g7h8i9

This provides memorable names for support conversations while avoiding collisions.

Lifecycle Policies¶

Policy	Default	Description
`recreateOnProfileChange`	`false`	Recreate unassigned instances when the pool profile changes
`maxUnassignedAge`	`168h`	Maximum time an instance can stay unassigned before cleanup (e.g., `168h`)
`reclaimPolicy`	`Delete`	What happens to the instance's live data when the assigned `Nextcloud` is deleted: `Delete` destroys all data; `Retain` preserves the S3 bucket, managed PostgreSQL, PVCs, and owned namespace for manual reclamation. When the pool itself is deleted, the policy is stamped onto each still-assigned instance (see Decommissioning a pool); any remaining policy-resolution failure fails safe to `Delete`. See Deletion → reclaimPolicy: Retain.

Decommissioning a Pool¶

A pool can be deleted while assigned instances still carry its k8s.bnerd.com/pool label — the typical migration pattern: drain the pool to zero spares (replicas: 0), point new provisioning at a different pool, and leave the already-running instances labelled with their original pool.

Deleting the pool then:

deletes only unassigned spares — assigned instances are never touched by pool deletion;
stamps the pool's effective reclaimPolicy onto every still-assigned instance as the k8s.bnerd.com/reclaim-policy annotation (Retain or Delete). Once the pool CR is gone, this stamp is what the operator honours when one of those instances is eventually deleted — so a Retain guarantee survives pool decommissioning;
emits a Warning event (AssignedInstancesRemain) on the pool recording how many assigned instances still referenced it.

After the pool is gone, assigned instances keep reconciling normally — upgrades, scaling, backups, and status are driven entirely by the instance spec; the dangling pool label is only read for metrics and for reclaim-policy resolution (which now uses the stamp).

Runbook: decommission a legacy pool¶

Drain — scale the pool to zero spares and wait for the reconciler to remove them:

kubectl patch ncp <legacy-pool> --type=merge -p '{"spec":{"replicas":0}}'
kubectl get ncp <legacy-pool>    # UNASSIGNED → 0

Check the reclaim policy you are about to freeze onto the assigned instances:
```
kubectl get ncp <legacy-pool> -o jsonpath='{.spec.lifecycle.reclaimPolicy}'
```
If this is not the policy the instances should keep, fix it before deleting the pool — or relabel the instances to a current pool instead (step 4).

Delete the pool:

kubectl delete ncp <legacy-pool>

Verify the stamp landed on the assigned instances:

kubectl get nci -A -l k8s.bnerd.com/pool=<legacy-pool> \
  -o custom-columns='NS:.metadata.namespace,NAME:.metadata.name,RECLAIM:.metadata.annotations.k8s\.bnerd\.com/reclaim-policy'

Optional — relabel instances to a current pool instead of (or after) deleting the legacy pool. Relabelling is supported: the operator writes the pool label only at creation/assignment time and nothing reverts a manual change. A live pool named by the label is always authoritative over the stamp:
```
kubectl label nci <name> -n <ns> k8s.bnerd.com/pool=<current-pool> --overwrite
```
The instance then appears in the new pool's status.assigned count and inherits the new pool's reclaimPolicy. It is never treated as a claimable spare (assigned=true excludes it from spare selection).

Status Synchronization¶

The operator keeps the Nextcloud and NextcloudInstance status in sync:

Nextcloud.spec ──────────────► NextcloudInstance.spec
                 (propagation)

Nextcloud.status ◄──────────── NextcloudInstance.status
                    (sync back)

Fields synced back: phase, helmRelease, url, version, conditions

Drift Reconciliation¶

If a NextcloudInstance is modified directly, the operator detects drift during the 30-second timer reconciliation and overwrites the instance spec with the Nextcloud spec. Manual changes to assigned instances are not preserved.

Troubleshooting¶

Pool reports healthy spares but won't warm replacements (Terminating instances)¶

Symptom (pre-0.19.3): kubectl get ncp shows UNASSIGNED >= 1, yet tenant onboarding fails to find a spare, or a freshly-assigned tenant loops on a (422) Unprocessable Entity error. The pool appears full but is effectively empty.

Cause: A pool instance held open by a finalizer (state Terminating, with a metadata.deletionTimestamp) is still listed and still reports status.phase=Ready. Before 0.19.3 the pool counted such a dying instance as a live unassigned spare, so it never warmed a replacement, and the matcher could hand the dying instance to a tenant.

How to spot it: list pool instances and check for a deletion timestamp:

kubectl get nci -A -l k8s.bnerd.com/pool=<pool-name> \
  -o custom-columns=NAME:.metadata.name,ASSIGNED:.metadata.labels.k8s\\.bnerd\\.com/assigned,PHASE:.status.phase,DELETING:.metadata.deletionTimestamp

A row with ASSIGNED=false, PHASE=Ready, and a non-empty DELETING column is the phantom spare. If it is stuck Terminating, inspect its finalizers (kubectl get nci <name> -n <ns> -o jsonpath='{.metadata.finalizers}') and the underlying teardown (HelmRelease, S3, DB) to find what is blocking deletion.

Fix: As of 0.19.3 the pool self-corrects — Terminating instances are excluded from all pool accounting, so the shortage becomes truthful and the pool warms a replacement on the next 30-second reconcile tick. The matcher also skips Terminating instances, so a dying spare is never assigned to a tenant. No manual intervention is needed to recover pool capacity once the operator is upgraded; resolving whatever wedges the instance's deletion is still worthwhile to reclaim its resources.