Skip to content

feat: bump component images, re-vendor CRDs/RBAC/config - #44

Merged
mbelur merged 1 commit into
mainfrom
bump-image-version
Nov 3, 2026
Merged

mbelur merged 1 commit into
mainfrom
bump-image-version

Conversation

@mbelur

@mbelur mbelur commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

The image-tag bumps (knative v1.24.0, kserve v0.21.0, jobset v0.13.0, kfp 2.17.3, hub v0.3.18, tensorflow/serving 2.21.0, seaweedfs 4.51, ...) moved the running containers ahead of the chart's vendored CRDs/config/RBAC. Re-vendor those manifests to the target versions (per revendor-crds-plan.md) and fix the regressions surfaced on a live RKE2 deploy.

Re-vendor:

  • jobset: regenerate CRD from v0.13.0 (new status fields); ClusterRole synced to v0.13.0 (persistentvolumeclaims, scheduling.k8s.io podgroups/workloads, events.k8s.io); add namespaced manager-secrets Role/RoleBinding; livenessProbe initialDelaySeconds 15->30.
  • kserve: replace InferenceService CRD with v0.21 (was controller-gen v0.4.0, ~80 fields pruned); resync cluster/servingruntimes CRDs; add modelscope:// to ClusterStorageContainer; add protocolVersions v2 and bump xgb/lgb model-format versions on the serving runtimes.
  • knative-serving: full re-vendor of all 12 CRDs to 1.24.0 (adds ephemeral volume field + modern PodSpec schema); config-features domainmapping/ephemeral keys; Chart.yaml appVersion 1.22.0->1.24.0.
  • knative-eventing: full re-vendor of the 11 vendored CRDs to 1.24.0; controller ClusterRole endpoints->discovery.k8s.io/endpointslices; config-features delivery-backoff-max; config-logging klog-verbosity.

Deploy fixes found on-cluster:

  • kubeVersion floor >=1.31.0-0 -> >=1.35.0-0: knative v1.24.0 binaries hard-gate on k8s 1.35, so fail fast at install instead of crash-looping. README(.gotmpl) and upgrade-notes updated.
  • seaweedfs: run the restricted non-root profile (uid/gid 1000, privileged false, seccompProfile RuntimeDefault, fsGroupChangePolicy OnRootMismatch) matching the upstream community-distribution manifest, since the repackaged 4.51 image's root chown/su-exec wrapper crash-looped under drop-ALL. Add WEED_MASTER_VOLUME_GROWTH_COPY_1=1 and a configurable -volume.max (seaweedfs.volumeMax, default 30) so per-bucket collections no longer exhaust the single volume server (S3 PutObject InternalError).
  • ml-pipeline-visualizationserver: switch wget exec probes to python (the 2.17.3 image no longer ships wget).
  • istio-ingressgateway-require-jwt: scope the DENY to the gateway HTTP/HTTPS listener ports (HTTP tracks .Values.gateway.port.number, HTTPS 443) to clear the Istio "deny all TCP" validation warning without failing open.

Subchart version bumps (with umbrella dependency pins + Chart.lock): trainer 0.2.0->0.2.1, kserve 0.2.4->0.2.5, knative-serving 0.2.3->0.2.4, knative-eventing 0.2.1->0.2.2, kubeflow-pipelines 0.2.6->0.2.7, kubeflow-istio-resources 0.2.2->0.2.3.
Umbrella Chart version bump

@mbelur
mbelur requested a review from gunamata November 3, 2026 03:27
@mbelur
mbelur marked this pull request as draft November 3, 2026 03:28
@mbelur
mbelur force-pushed the bump-image-version branch from be7c63d to 4f08456 Compare November 3, 2026 03:35
The image-tag bumps (knative v1.24.0, kserve v0.21.0, jobset v0.13.0, kfp 2.17.3,
hub v0.3.18, tensorflow/serving 2.21.0, seaweedfs 4.51, ...) moved the running
containers ahead of the chart's vendored CRDs/config/RBAC. Re-vendor those
manifests to the target versions (per revendor-crds-plan.md) and fix the
regressions surfaced on a live RKE2 deploy.

Re-vendor:
- jobset: regenerate CRD from v0.13.0 (new status fields); ClusterRole synced to
  v0.13.0 (persistentvolumeclaims, scheduling.k8s.io podgroups/workloads,
  events.k8s.io); add namespaced manager-secrets Role/RoleBinding; livenessProbe
  initialDelaySeconds 15->30.
- kserve: replace InferenceService CRD with v0.21 (was controller-gen v0.4.0,
  ~80 fields pruned); resync cluster/servingruntimes CRDs; add modelscope:// to
  ClusterStorageContainer; add protocolVersions v2 and bump xgb/lgb model-format
  versions on the serving runtimes.
- knative-serving: full re-vendor of all 12 CRDs to 1.24.0 (adds ephemeral volume
  field + modern PodSpec schema); config-features domainmapping/ephemeral keys;
  Chart.yaml appVersion 1.22.0->1.24.0.
- knative-eventing: full re-vendor of the 11 vendored CRDs to 1.24.0; controller
  ClusterRole endpoints->discovery.k8s.io/endpointslices; config-features
  delivery-backoff-max; config-logging klog-verbosity.

Deploy fixes found on-cluster:
- kubeVersion floor >=1.31.0-0 -> >=1.35.0-0: knative v1.24.0 binaries hard-gate
  on k8s 1.35, so fail fast at install instead of crash-looping. README(.gotmpl)
  and upgrade-notes updated.
- seaweedfs: run the restricted non-root profile (uid/gid 1000, privileged false,
  seccompProfile RuntimeDefault, fsGroupChangePolicy OnRootMismatch) matching the
  upstream community-distribution manifest, since the repackaged 4.51 image's root
  chown/su-exec wrapper crash-looped under drop-ALL. Add
  WEED_MASTER_VOLUME_GROWTH_COPY_1=1 and a configurable -volume.max
  (seaweedfs.volumeMax, default 30) so per-bucket collections no longer exhaust
  the single volume server (S3 PutObject InternalError).
- ml-pipeline-visualizationserver: switch wget exec probes to python (the 2.17.3
  image no longer ships wget).
- istio-ingressgateway-require-jwt: scope the DENY to the gateway HTTP/HTTPS
  listener ports (HTTP tracks .Values.gateway.port.number, HTTPS 443) to clear the
  Istio "deny all TCP" validation warning without failing open.

Subchart version bumps (with umbrella dependency pins + Chart.lock):
trainer 0.2.0->0.2.1, kserve 0.2.4->0.2.5, knative-serving 0.2.3->0.2.4,
knative-eventing 0.2.1->0.2.2, kubeflow-pipelines 0.2.6->0.2.7,
kubeflow-istio-resources 0.2.2->0.2.3.
Umbrella Chart version bump
@mbelur
mbelur force-pushed the bump-image-version branch from 4f08456 to cc2e3e0 Compare November 3, 2026 18:23
@mbelur
mbelur marked this pull request as ready for review November 3, 2026 21:31

@gunamata gunamata left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mbelur
mbelur merged commit 480c09e into main Nov 3, 2026
1 check passed
@mbelur
mbelur deleted the bump-image-version branch November 3, 2026 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants