Commit Graph

8 Commits

Author SHA1 Message Date
Janis 9bb8f119b5 fix(spark-analytics): stream NIM AI responses to kill webview 504s
The /explain endpoint blocked on a synchronous NIM call (llama-3.3-70b,
30-60s+) while nginx's default 60s proxy-read-timeout fired first,
surfacing every AI request as a 504 Gateway Timeout.

Switch the NIM proxy to streaming (stream:true) and relay its SSE to the
browser via a new _emit() helper; the client now reads resp.body
incrementally and renders tokens as they arrive. With chunks flowing,
nginx's read-timeout resets per chunk so the 60s wall is gone, and the
frozen spinner is replaced by live text.

Ingress: disable proxy-buffering (else nginx buffers the whole response,
defeating streaming) and set proxy-read-timeout to 120s as the
inter-chunk gap budget. Server urlopen timeout 60->110s.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 09:22:31 +02:00
Janis Eccarius f64b838030 feat(spark-analytics): add AccuracyBlunder, ClockPressure, SmurfAnomaly jobs and webview datasets
Adds three new CronJobs (spark-accuracy-blunder, spark-clock-pressure,
spark-smurf-anomaly) and registers their six output tables in the
analytics webview (accuracy_by_rating, blunder_outcome, clock_by_rating,
scramble_outcome, smurf_anomaly, flagged_smurfs).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 00:18:22 +02:00
Janis Eccarius c1c81f2507 chore(staging): bump analytics image to 0.7.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 16:55:52 +02:00
Janis Eccarius 42cb4efc05 fix(staging): add NOWCHESS_PGN_CACHE_DIR to all analytics CronJobs
Points to /spark-output/.pgn-cache so the decompressed Lichess PGN file
is written to the shared PVC, accessible by both driver and executor pods.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 16:31:20 +02:00
Janis Eccarius ca78173a56 feat(spark-analytics): switch webview to PostgreSQL, always write analytics to DB
- Rewrite webview serve.py to query PostgreSQL analytics tables via psycopg2
  instead of reading CSV files from PVC; PVC mount removed from Deployment
- Remove isPgnMode guard from all 4 original jobs so results always write to DB
  regardless of whether input comes from game_records or a Lichess PGN dump
- Add JDBC write-back to all 7 new analytics jobs (game_length_distribution,
  game_length_by_result, color_advantage, elo_distribution, time_control_stats,
  hourly_activity, weekly_activity, rating_mismatch, termination_stats)
- Add analytics_component_sizes JDBC write to PlayerGraphJob
- Add NOWCHESS_PGN_PATH staging patches for the 7 new CronJobs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:35:32 +02:00
Janis Eccarius a190d50a09 feat(spark-analytics): add 7 CronJobs, results webview, and LiChess staging demo
Adds CronJobs and headless driver Services for GameLengthJob, ColorAdvantageJob,
EloDistributionJob, TimeControlJob, DailyActivityJob, RatingMismatchJob, and
TerminationStatsJob. All suspended by default; scheduled Saturday mornings.

Adds spark-analytics-webview: a stdlib Python HTTP server (ConfigMap + Deployment
+ Service) that reads CSV output from the spark-analytics-output PVC and renders
13 datasets as browsable HTML tables. Disabled (replicas=0) in base.

Staging overlay enables the webview (replicas=1), exposes it via Ingress at
spark-results-st.nowchess.janis-eccarius.de, and sets NOWCHESS_PGN_PATH on all
CronJobs to the Lichess Jan-2013 rated standard PGN dump for demo runs without
a NowChess database.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:09:49 +02:00
Janis Eccarius 2bc911ab81 fix(lf): Line Endings fixed 2026-06-21 15:09:47 +02:00
Janis 06e70a5e8a feat(spark): add staging overlay, scale down prod
- spark-cluster/eu-central-1-staging: namespace nowchess-staging, ingress
  patched to spark-st.* / spark-history-st.* with TLS + cert-manager
- spark-analytics/eu-central-1-staging: namespace nowchess-staging, image
  pinned to 0.2.0, all CronJob + live-dashboard patches carried over
- eu-central-1/argo-apps: spark-cluster-staging + spark-analytics-staging
  ArgoCD apps pointing at new staging overlays
- spark-cluster/eu-central-1-prod: scale spark-master and spark-worker to 0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 08:48:31 +02:00