local-path PVCs are created root-owned. The apache/spark image runs as
spark (UID 185). fsGroup: 185 makes K8s chown the mounted volume to that
group on attach, fixing the mkdir IOException on /spark-checkpoint.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
local-path provisioner is RWO-only; attaching a single PVC to master, two
workers, history-server, and streaming pod on separate nodes causes
Multi-Attach errors. Each pod now gets its own ephemeral emptyDir for
/spark-events. Event logging confs removed from CronJobs and streaming
Deployment (logs are pod-local and do not persist across restarts).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Wire five Spark jobs (OpeningBook, PlayerStats, PlayerClustering,
PlayerGraph, LiveDashboard) to Kubernetes via native Spark-on-K8s mode.
- spark-analytics/base: ServiceAccount, RBAC Role/RoleBinding (driver
executor pod management), two PVCs (10Gi output, 2Gi checkpoint),
four CronJobs for batch analytics, one Deployment for the Structured
Streaming live dashboard
- spark-analytics/eu-central-1-prod: namespace overlay pinning image
to 0.1.0 with strategic merge patches that update both the container
image field and the spark.kubernetes.container.image conf arg
- eu-central-1/argo-apps/spark-analytics: ArgoCD Application pointing
at the prod overlay with automated prune+selfHeal sync
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Janis Eccarius