GitOps/spark-cluster/base/pvc.yaml at 5d08613cd7ccc307e0080f87ea054e8781ab083b - GitOps - Gitea

NowChess/GitOps

Files

T

Janis Eccarius ab95b4f101 fix(spark): replace shared spark-events PVC with emptyDir to fix Multi-Attach error

local-path provisioner is RWO-only; attaching a single PVC to master, two
workers, history-server, and streaming pod on separate nodes causes
Multi-Attach errors. Each pod now gets its own ephemeral emptyDir for
/spark-events. Event logging confs removed from CronJobs and streaming
Deployment (logs are pod-local and do not persist across restarts).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-06-16 20:33:48 +02:00

6 lines

363 B

YAML

Executable File

Raw Blame History

 # No cluster-level PVCs needed.
 # Analytics output and streaming checkpoint PVCs live in spark-analytics/base/pvc.yaml.
 # spark-events PVC removed: local-path provisioner is RWO-only; sharing a single PVC
 # across master, workers, and history-server on different nodes causes Multi-Attach errors.
 # Event logs now use emptyDir (pod-local, ephemeral) on each pod.