Commit Graph

7 Commits

Author SHA1 Message Date
Janis Eccarius 283c6728d4 fix(spark-analytics): batch index COUNT queries in one connection, suppress BrokenPipeError
13 separate DB connections on index page caused readiness probe timeout.
Now all counts share one connection. BrokenPipeError on probe disconnect silenced.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:52:34 +02:00
Janis Eccarius 609a222211 fix(spark-analytics): robust JDBC URL parsing, log resolved DB host on startup
Handle URLs without explicit port, show DB_HOST:PORT/DB_NAME in error messages
and startup log to simplify connection debugging.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:47:48 +02:00
Janis Eccarius bdc3efc878 fix(spark-analytics): install psycopg2 to /tmp/pkg, set PYTHONPATH so it is importable
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:44:09 +02:00
Janis Eccarius 6765c5162d fix(spark-analytics): set HOME=/tmp so pip can write under non-root user 65534
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:42:50 +02:00
Janis Eccarius 4395415985 fix(spark-analytics): add --no-cache-dir to pip install for nonexistent home dir
User 65534 has no home directory; pip cache write to /nonexistent fails. Skip cache.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:41:41 +02:00
Janis Eccarius ca78173a56 feat(spark-analytics): switch webview to PostgreSQL, always write analytics to DB
- Rewrite webview serve.py to query PostgreSQL analytics tables via psycopg2
  instead of reading CSV files from PVC; PVC mount removed from Deployment
- Remove isPgnMode guard from all 4 original jobs so results always write to DB
  regardless of whether input comes from game_records or a Lichess PGN dump
- Add JDBC write-back to all 7 new analytics jobs (game_length_distribution,
  game_length_by_result, color_advantage, elo_distribution, time_control_stats,
  hourly_activity, weekly_activity, rating_mismatch, termination_stats)
- Add analytics_component_sizes JDBC write to PlayerGraphJob
- Add NOWCHESS_PGN_PATH staging patches for the 7 new CronJobs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:35:32 +02:00
Janis Eccarius a190d50a09 feat(spark-analytics): add 7 CronJobs, results webview, and LiChess staging demo
Adds CronJobs and headless driver Services for GameLengthJob, ColorAdvantageJob,
EloDistributionJob, TimeControlJob, DailyActivityJob, RatingMismatchJob, and
TerminationStatsJob. All suspended by default; scheduled Saturday mornings.

Adds spark-analytics-webview: a stdlib Python HTTP server (ConfigMap + Deployment
+ Service) that reads CSV output from the spark-analytics-output PVC and renders
13 datasets as browsable HTML tables. Disabled (replicas=0) in base.

Staging overlay enables the webview (replicas=1), exposes it via Ingress at
spark-results-st.nowchess.janis-eccarius.de, and sets NOWCHESS_PGN_PATH on all
CronJobs to the Lichess Jan-2013 rated standard PGN dump for demo runs without
a NowChess database.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:09:49 +02:00