NowChess/GitOps
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(spark): replace shared spark-events PVC with emptyDir to fix Multi-Attach error

Browse Source 
local-path provisioner is RWO-only; attaching a single PVC to master, two
workers, history-server, and streaming pod on separate nodes causes
Multi-Attach errors. Each pod now gets its own ephemeral emptyDir for
/spark-events. Event logging confs removed from CronJobs and streaming
Deployment (logs are pod-local and do not persist across restarts).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Janis Eccarius
2026-06-16 20:33:48 +02:00
parent e20bedbff5
commit ab95b4f101
7 changed files with 8 additions and 89 deletions
Download Patch File Download Diff File Expand all files Collapse all files
spark-analytics/eu-central-1-prod/kustomization.yaml
-20
View File
@@ -31,10 +31,6 @@ patches:
- spark.executor.memory=1g
- --conf
- spark.driver.memory=512m
- --conf
- spark.eventLog.enabled=true
- --conf
- spark.eventLog.dir=/spark-events
- --class
- de.nowchess.analytics.OpeningBookJob
- local:///app/analytics.jar
@@ -65,10 +61,6 @@ patches:
- spark.executor.memory=1g
- --conf
- spark.driver.memory=512m
- --conf
- spark.eventLog.enabled=true
- --conf
- spark.eventLog.dir=/spark-events
- --class
- de.nowchess.analytics.PlayerStatsJob
- local:///app/analytics.jar
@@ -98,10 +90,6 @@ patches:
- spark.executor.memory=1g
- --conf
- spark.driver.memory=512m
- --conf
- spark.eventLog.enabled=true
- --conf
- spark.eventLog.dir=/spark-events
- --class
- de.nowchess.analytics.PlayerClusteringJob
- local:///app/analytics.jar
@@ -132,10 +120,6 @@ patches:
- spark.executor.memory=1g
- --conf
- spark.driver.memory=512m
- --conf
- spark.eventLog.enabled=true
- --conf
- spark.eventLog.dir=/spark-events
- --class
- de.nowchess.analytics.PlayerGraphJob
- local:///app/analytics.jar
@@ -163,10 +147,6 @@ patches:
- spark.executor.memory=512m
- --conf
- spark.driver.memory=512m
- --conf
- spark.eventLog.enabled=true
- --conf
- spark.eventLog.dir=/spark-events
- --class
- de.nowchess.analytics.LiveDashboardJob
- local:///app/analytics.jar