feat(spark): deploy persistent Spark cluster with Web UI
- Add spark-cluster/base with Bitnami Spark chart 9.2.10 (master + 2 workers), spark-events PVC (RWO/local-path), and Ingress resources for spark.nowchess.janis-eccarius.de (master UI) and spark-history.nowchess.janis-eccarius.de (history placeholder) - Add spark-cluster/eu-central-1-prod overlay and ArgoCD Application (automated prune + selfHeal) - Migrate all spark-analytics CronJobs and streaming Deployment from K8s-native mode (k8s://...) to Spark standalone (spark://spark-master:7077); drop kubernetes-specific --conf flags; add eventLog confs; mount spark-events PVC on all job pods Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Executable
+46
@@ -0,0 +1,46 @@
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: spark-master-ingress
|
||||
labels:
|
||||
app.kubernetes.io/name: spark-cluster
|
||||
app.kubernetes.io/part-of: nowchess
|
||||
spec:
|
||||
ingressClassName: traefik
|
||||
rules:
|
||||
- host: spark.nowchess.janis-eccarius.de
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: spark-master-svc
|
||||
port:
|
||||
number: 80
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: spark-history-ingress
|
||||
labels:
|
||||
app.kubernetes.io/name: spark-cluster
|
||||
app.kubernetes.io/part-of: nowchess
|
||||
annotations:
|
||||
# NOTE: The Bitnami spark chart 9.2.10 does not include a standalone history
|
||||
# server deployment or service. This Ingress is a placeholder. To activate
|
||||
# history server access, deploy a separate spark-history-server Deployment
|
||||
# and Service named spark-history-server-svc on port 18080.
|
||||
spec:
|
||||
ingressClassName: traefik
|
||||
rules:
|
||||
- host: spark-history.nowchess.janis-eccarius.de
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: spark-history-server-svc
|
||||
port:
|
||||
number: 18080
|
||||
Reference in New Issue
Block a user