Commit Graph

3 Commits

Author SHA1 Message Date
Janis Eccarius e08e8313ff fix(spark): use nginx ingress class instead of traefik
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 20:42:14 +02:00
Janis Eccarius d962dfda22 feat(spark): replace Bitnami chart with Apache Spark manifests
Bitnami Spark chart requires a Broadcom commercial license. Replace with
plain Deployments using apache/spark:3.5.4-scala2.13-java17-ubuntu
(Apache 2.0, same image used by the analytics jobs):
- spark-master: Deployment + ClusterIP Service (ports 7077/8080)
- spark-worker: Deployment, 2 replicas, connects to spark-master:7077
- spark-history-server: updated to apache/spark image
Ingress backend corrected to spark-master:8080.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 19:48:51 +02:00
Janis Eccarius eefa6e728a feat(spark): deploy persistent Spark cluster with Web UI
- Add spark-cluster/base with Bitnami Spark chart 9.2.10 (master +
  2 workers), spark-events PVC (RWO/local-path), and Ingress resources
  for spark.nowchess.janis-eccarius.de (master UI) and
  spark-history.nowchess.janis-eccarius.de (history placeholder)
- Add spark-cluster/eu-central-1-prod overlay and ArgoCD Application
  (automated prune + selfHeal)
- Migrate all spark-analytics CronJobs and streaming Deployment from
  K8s-native mode (k8s://...) to Spark standalone (spark://spark-master:7077);
  drop kubernetes-specific --conf flags; add eventLog confs; mount
  spark-events PVC on all job pods

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 22:41:33 +02:00