feat(spark): deploy persistent Spark cluster with Web UI

- Add spark-cluster/base with Bitnami Spark chart 9.2.10 (master +
  2 workers), spark-events PVC (RWO/local-path), and Ingress resources
  for spark.nowchess.janis-eccarius.de (master UI) and
  spark-history.nowchess.janis-eccarius.de (history placeholder)
- Add spark-cluster/eu-central-1-prod overlay and ArgoCD Application
  (automated prune + selfHeal)
- Migrate all spark-analytics CronJobs and streaming Deployment from
  K8s-native mode (k8s://...) to Spark standalone (spark://spark-master:7077);
  drop kubernetes-specific --conf flags; add eventLog confs; mount
  spark-events PVC on all job pods

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Janis Eccarius
2026-06-15 22:41:33 +02:00
parent a3eb172916
commit eefa6e728a
57 changed files with 5864 additions and 117 deletions
+18
View File
@@ -0,0 +1,18 @@
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: spark-cluster
namespace: argocd
spec:
project: default
destination:
namespace: nowchess
server: https://kubernetes.default.svc
sources:
- repoURL: git@git.janis-eccarius.de:NowChess/Gitops.git
path: ./spark-cluster/eu-central-1-prod
targetRevision: main
syncPolicy:
automated:
prune: true
selfHeal: true