Commit Graph

629 Commits

Author SHA1 Message Date
svc.kargo 4b1df75d7a chore(kargo): update image tag official-bots to 0.23.0-x86_v2 on htwg-1-prod (#443) 2026-06-21 22:24:22 +02:00
svc.kargo 4ab804f3bd chore(kargo): update image tag official-bots to 0.23.0 on eu-central-1-prod (#442) 2026-06-21 22:24:14 +02:00
svc.kargo df44bd176e chore(kargo): update image tag tournament to 0.5.0 on staging (#441) 2026-06-21 22:24:11 +02:00
svc.kargo 7a60f2865c chore(kargo): update image tag official-bots to 0.23.0 on staging (#440) 2026-06-21 22:20:03 +02:00
svc.kargo f823e06971 chore(kargo): update image tag frontend to 0.5.0 on eu-central-1-prod (#439) 2026-06-21 22:15:44 +02:00
svc.kargo 654e97957f chore(kargo): update image tag frontend to 0.5.0-x86_v2 on htwg-1-prod (#438) 2026-06-21 22:15:43 +02:00
svc.kargo 7d2b4774c7 chore(kargo): update image tag frontend to 0.5.0 on staging (#437) 2026-06-21 22:11:49 +02:00
Janis Eccarius 6abed9bb6e feat(nowchess): add TOURNAMENT_EXTERNAL_SERVERS env var to tournament rollout
htwg-1-prod seeds 141.37.123.132:8086 (primary bridge server) on startup
so the registry is populated without manual API calls after pod restarts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 21:34:05 +02:00
Janis Eccarius a84f62f972 fix(nowchess): correct tournament service port to 8088 and SELF_URL
- TOURNAMENT_SERVICE_URL was 8086 but K8s service and app both use 8088
- TOURNAMENT_SELF_URL on htwg-1 should be base ingress URL (port 80),
  not a direct service port — ingress routes /api/tournament to tournament service

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 21:31:16 +02:00
Janis Eccarius 3e6a7be274 fix(htwg-1): correct TOURNAMENT_SELF_URL to htwg-1 own IP
Was pointing at third-party server (141.37.123.132). Should be htwg-1's
own tournament service IP so remote clusters can proxy mutations back here.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 21:29:40 +02:00
Janis Eccarius b82a0919e9 feat(nowchess): wire TOURNAMENT_SERVICE_URL and TOURNAMENT_SELF_URL
- Official-bots now targets local tournament service (TOURNAMENT_SERVICE_URL)
  instead of calling remote cluster directly
- TOURNAMENT_SELF_URL tells each cluster's tournament and official-bots
  services their own public URL for cross-cluster replication
- htwg-1-prod: remove TOURNAMENT_SERVER_URL, add both new vars with correct
  values (tournament active service + node IP for self-announcement)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 21:23:58 +02:00
Janis Eccarius 521a39fa04 fix(official-bots): point ACCOUNT_SERVICE_URL to correct K8s service name
Service is named nowchess-account-active, not nowchess-account. DNS resolution
failed on startup causing official bots to never sync and appear unavailable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 20:59:39 +02:00
Janis Eccarius f2aeea1b40 fix(kustomization): update API and WebSocket URLs for frontend environment 2026-06-21 20:49:52 +02:00
Janis Eccarius 587ea19e0e fix(spark-analytics): increase NIM timeout and trim max_tokens to avoid read timeouts
70B model at 30-50 tok/s was exceeding the 30s urllib timeout for larger
outputs. Raise server timeout to 60s, add browser AbortController at 65s
so the UI fails gracefully. Trim max_tokens (row: 300, dataset: 512) to
keep generation well inside the window.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 17:58:29 +02:00
Janis Eccarius 1ab9d60dd2 fix(spark-analytics): use ThreadingHTTPServer to prevent probe starvation
Single-threaded HTTPServer blocked readiness probes during long NIM API
calls (~30s), causing Kubernetes to flip the pod unhealthy until the
request completed. ThreadingHTTPServer handles each request in its own
thread so probes are never starved.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 17:53:31 +02:00
Janis Eccarius ac51ad09a4 feat(spark-analytics): per-row AI analysis with queue and rate-limit handling
- Add  button on each row (first 200) — clicking analyses that specific
  row with a chess-expert prompt tailored to the entry type (opening,
  player, etc.)
- Cache results per row so already-analysed rows skip re-requests
- Serialise all NIM requests through a promise queue to avoid bursting
- Pass 429 + Retry-After through from NIM to the browser; JS surfaces
  "Rate limited — retry in Ns" error and re-enables the button for retry
- Dataset-level "Analyse top 10 rows" now sends a per-row numbered prompt
  asking for chess-specific analysis of each entry rather than a vague
  dataset summary
- Store first 200 rows in window.DATASET.rows for client-side row access

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 17:48:10 +02:00
Janis Eccarius 7f11de365c fix(spark-analytics): proxy NIM API calls through server to avoid CORS
Browser cannot call integrate.api.nvidia.com directly — no CORS headers.
Add POST /explain endpoint: browser sends key + dataset, server calls NIM
with urllib, returns JSON. Key is never logged or stored server-side.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 17:36:04 +02:00
Janis Eccarius 60467d8ff6 fix(spark-analytics): escape \\n in JS userPrompt strings
Python triple-quoted string interprets \n as literal newlines,
producing invalid JS string literals and a SyntaxError that
killed the entire script block.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 17:25:20 +02:00
Janis Eccarius 9074c3fbd0 fix(spark-analytics): replace multiline JS template literals with concatenation
Multi-line template literals spanning unindented continuation lines break the
YAML block scalar indentation, causing kustomize to fail with MalformedYAMLError.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 17:19:00 +02:00
Janis Eccarius 7aa3bd3b12 fix: update CORS_ORIGINS to new IP address 2026-06-21 17:16:59 +02:00
Janis Eccarius 252c543fed feat(spark-analytics): add per-user NVIDIA NIM AI explanations to webview
Each dataset page now has an "Explain this dataset" button that calls the
NVIDIA NIM chat completions API (meta/llama-3.3-70b-instruct) directly from
the browser with headers + first 10 rows as context. API key is entered via
a settings modal and stored in localStorage only — never sent to the server.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 17:16:26 +02:00
Janis Eccarius c1c81f2507 chore(staging): bump analytics image to 0.7.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 16:55:52 +02:00
Janis Eccarius 42cb4efc05 fix(staging): add NOWCHESS_PGN_CACHE_DIR to all analytics CronJobs
Points to /spark-output/.pgn-cache so the decompressed Lichess PGN file
is written to the shared PVC, accessible by both driver and executor pods.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 16:31:20 +02:00
svc.kargo ce56b7cc8b chore(kargo): update image tag official-bots to 0.22.0-x86_v2 on htwg-1-prod (#436) 2026-06-21 16:30:58 +02:00
svc.kargo 433227e132 chore(kargo): update image tag official-bots to 0.22.0 on staging (#435) 2026-06-21 16:22:26 +02:00
svc.kargo e1e501766b chore(kargo): update image tag frontend to 0.4.4-x86_v2 on htwg-1-prod (#434) 2026-06-21 16:22:25 +02:00
svc.kargo 5423f0000a chore(kargo): update image tag frontend to 0.4.4 on staging (#433) 2026-06-21 16:12:22 +02:00
Janis Eccarius f2d13efcd7 fix: co-locate analytics driver pods with spark-worker for RWO PVC access
do-block-storage only supports ReadWriteOnce. Executor tasks failed with
`Mkdirs failed` because the executor ran on the spark-worker node which
had no mount of the spark-analytics-output PVC.

- Mount spark-analytics-output PVC on spark-worker pod at /spark-output
- Add podAffinity to all 11 analytics CronJob driver pods to schedule on
  the same node as spark-worker, ensuring the RWO PVC is accessible by
  both driver and executor

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 16:06:51 +02:00
Janis Eccarius 283c6728d4 fix(spark-analytics): batch index COUNT queries in one connection, suppress BrokenPipeError
13 separate DB connections on index page caused readiness probe timeout.
Now all counts share one connection. BrokenPipeError on probe disconnect silenced.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:52:34 +02:00
Janis Eccarius 609a222211 fix(spark-analytics): robust JDBC URL parsing, log resolved DB host on startup
Handle URLs without explicit port, show DB_HOST:PORT/DB_NAME in error messages
and startup log to simplify connection debugging.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:47:48 +02:00
Janis Eccarius bdc3efc878 fix(spark-analytics): install psycopg2 to /tmp/pkg, set PYTHONPATH so it is importable
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:44:09 +02:00
Janis Eccarius 6765c5162d fix(spark-analytics): set HOME=/tmp so pip can write under non-root user 65534
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:42:50 +02:00
Janis Eccarius 4395415985 fix(spark-analytics): add --no-cache-dir to pip install for nonexistent home dir
User 65534 has no home directory; pip cache write to /nonexistent fails. Skip cache.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:41:41 +02:00
Janis Eccarius ca78173a56 feat(spark-analytics): switch webview to PostgreSQL, always write analytics to DB
- Rewrite webview serve.py to query PostgreSQL analytics tables via psycopg2
  instead of reading CSV files from PVC; PVC mount removed from Deployment
- Remove isPgnMode guard from all 4 original jobs so results always write to DB
  regardless of whether input comes from game_records or a Lichess PGN dump
- Add JDBC write-back to all 7 new analytics jobs (game_length_distribution,
  game_length_by_result, color_advantage, elo_distribution, time_control_stats,
  hourly_activity, weekly_activity, rating_mismatch, termination_stats)
- Add analytics_component_sizes JDBC write to PlayerGraphJob
- Add NOWCHESS_PGN_PATH staging patches for the 7 new CronJobs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:35:32 +02:00
Janis Eccarius 4379c0e867 fix(secret): update INTERNAL_SECRET value in ncs-internal-secret 2026-06-21 15:09:50 +02:00
Janis Eccarius a190d50a09 feat(spark-analytics): add 7 CronJobs, results webview, and LiChess staging demo
Adds CronJobs and headless driver Services for GameLengthJob, ColorAdvantageJob,
EloDistributionJob, TimeControlJob, DailyActivityJob, RatingMismatchJob, and
TerminationStatsJob. All suspended by default; scheduled Saturday mornings.

Adds spark-analytics-webview: a stdlib Python HTTP server (ConfigMap + Deployment
+ Service) that reads CSV output from the spark-analytics-output PVC and renders
13 datasets as browsable HTML tables. Disabled (replicas=0) in base.

Staging overlay enables the webview (replicas=1), exposes it via Ingress at
spark-results-st.nowchess.janis-eccarius.de, and sets NOWCHESS_PGN_PATH on all
CronJobs to the Lichess Jan-2013 rated standard PGN dump for demo runs without
a NowChess database.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:09:49 +02:00
Janis Eccarius 2bc911ab81 fix(lf): Line Endings fixed 2026-06-21 15:09:47 +02:00
svc.kargo 49e9d60bb0 chore(kargo): update image tag official-bots to 0.21.0-x86_v2 on htwg-1-prod (#432) 2026-06-21 13:50:51 +02:00
svc.kargo e164e6d1a2 chore(kargo): update image tag official-bots to 0.21.0 on staging (#430) 2026-06-21 13:46:54 +02:00
svc.kargo bde04c0944 chore(kargo): update image tag tournament to 0.4.0-x86_v2 on htwg-1-prod (#431) 2026-06-21 13:46:53 +02:00
svc.kargo ea4728b111 chore(kargo): update image tag core to 0.53.0-x86_v2 on htwg-1-prod (#428) 2026-06-21 13:46:46 +02:00
svc.kargo 72e623bbf5 chore(kargo): update image tag tournament to 0.4.0 on staging (#427) 2026-06-21 13:42:52 +02:00
svc.kargo 79a2968950 chore(kargo): update image tag core to 0.53.0 on staging (#426) 2026-06-21 13:42:22 +02:00
svc.kargo 45284b6e36 chore(kargo): update image tag frontend to 0.4.3-x86_v2 on htwg-1-prod (#425) 2026-06-21 12:36:28 +02:00
lq64 0030998279 fix: add WebSocket timeout annotations and use-regex to ingresses (#419)
Add proxy-read-timeout and proxy-send-timeout (3600s) so nginx does not
drop long-lived WebSocket connections at the default 60s idle timeout.

Add use-regex: "true" to staging to match production — without it, the
ImplementationSpecific path for the game WebSocket
(/api/board/game/[A-Za-z0-9]{8}/ws) is treated as a literal string
instead of a regex and never matches real game IDs.

---------

Co-authored-by: LQ63 <lkhermann@web.de>
Reviewed-on: #419
Co-authored-by: Leon Hermann <lq@blackhole.local>
Co-committed-by: Leon Hermann <lq@blackhole.local>
2026-06-21 12:35:28 +02:00
svc.kargo fef2bb728b chore(kargo): update image tag ws to 0.17.0-x86_v2 on htwg-1-prod (#423)
chore(kargo): update image tag ws to 0.17.0-x86_v2 on htwg-1-prod

[View in Kargo UI](https://kargo.nowchess.janis-eccarius.de/project/nowchess-kargo/stage/htwg-1-prod)

---------

Co-authored-by: Kargo <no-reply@kargo.io>
Reviewed-on: #423
Co-authored-by: svc.kargo <kargo@blackhole.local>
Co-committed-by: svc.kargo <kargo@blackhole.local>
2026-06-21 12:35:21 +02:00
svc.kargo 3822f2a77f chore(kargo): update image tag frontend to 0.4.3 on eu-central-1-prod (#424) 2026-06-21 11:13:22 +02:00
svc.kargo 11ede2b94c chore(kargo): update image tag ws to 0.17.0 on eu-central-1-prod (#422) 2026-06-21 11:12:42 +02:00
svc.kargo 4916dc590f chore(kargo): update image tag frontend to 0.4.3 on staging (#421) 2026-06-21 11:08:31 +02:00
Janis Eccarius c39661828b fix: wire analysis service to ingress across all clusters
Add /api/analysis → nowchess-analysis-active:8087 path rule to
eu-central-1-staging, eu-central-1-prod, and htwg-1-prod ingresses.
Rule placed before the /api catch-all so nginx routes correctly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 10:32:01 +02:00