fix(spark): lower memory requests to fix scheduling on resource-constrained nodes

Scheduler uses requests (not limits) for placement. Previous 1Gi requests
per pod caused Insufficient memory even with plenty of physical RAM.
New requests: master 256Mi, workers 256Mi, history-server 128Mi.
Limits unchanged. SPARK_WORKER_MEMORY reduced to 512m to match worker budget.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Janis Eccarius
2026-06-16 20:02:02 +02:00
parent 25f230fd18
commit 710e2ec308
3 changed files with 6 additions and 6 deletions
+1 -1
View File
@@ -31,7 +31,7 @@ spec:
resources: resources:
requests: requests:
cpu: 100m cpu: 100m
memory: 512Mi memory: 128Mi
limits: limits:
cpu: 500m cpu: 500m
memory: 1Gi memory: 1Gi
+2 -2
View File
@@ -29,8 +29,8 @@ spec:
containerPort: 8080 containerPort: 8080
resources: resources:
requests: requests:
cpu: 500m cpu: 200m
memory: 1Gi memory: 256Mi
limits: limits:
cpu: "1" cpu: "1"
memory: 2Gi memory: 2Gi
+3 -3
View File
@@ -27,14 +27,14 @@ spec:
- name: SPARK_WORKER_CORES - name: SPARK_WORKER_CORES
value: "1" value: "1"
- name: SPARK_WORKER_MEMORY - name: SPARK_WORKER_MEMORY
value: "1g" value: "512m"
ports: ports:
- name: webui - name: webui
containerPort: 8081 containerPort: 8081
resources: resources:
requests: requests:
cpu: 500m cpu: 200m
memory: 1Gi memory: 256Mi
limits: limits:
cpu: "1" cpu: "1"
memory: 2Gi memory: 2Gi