Each expandable row now has a text input + Ask button alongside the
auto-analysis. Type any question about that row, press Ask or Enter, and
the answer streams in below via the existing /explain SSE path.
Server: row mode accepts an optional "question" field — when set it uses
a grounded Q&A prompt instead of the default analysis. Adds askRow() JS
handler, row-ask markup, and supporting CSS.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The SEP comment contained "\n\n", which Python's triple-quoted JS block
turned into real newline characters in the emitted script. That produced
an inline-<script> SyntaxError ("string literal contains an unescaped
line break"), so every handler went undefined and clicking the analyse
buttons did nothing (ReferenceError: analyzeRow is not defined).
Replace the comment text with plain words — no backslash escapes. Verified
by exec'ing serve.py (psycopg2 stubbed) to capture the Python-evaluated
JS and parsing it with node --check.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The /explain endpoint blocked on a synchronous NIM call (llama-3.3-70b,
30-60s+) while nginx's default 60s proxy-read-timeout fired first,
surfacing every AI request as a 504 Gateway Timeout.
Switch the NIM proxy to streaming (stream:true) and relay its SSE to the
browser via a new _emit() helper; the client now reads resp.body
incrementally and renders tokens as they arrive. With chunks flowing,
nginx's read-timeout resets per chunk so the 60s wall is gone, and the
frozen spinner is replaced by live text.
Ingress: disable proxy-buffering (else nginx buffers the whole response,
defeating streaming) and set proxy-read-timeout to 120s as the
inter-chunk gap budget. Server urlopen timeout 60->110s.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds three new CronJobs (spark-accuracy-blunder, spark-clock-pressure,
spark-smurf-anomaly) and registers their six output tables in the
analytics webview (accuracy_by_rating, blunder_outcome, clock_by_rating,
scramble_outcome, smurf_anomaly, flagged_smurfs).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
70B model at 30-50 tok/s was exceeding the 30s urllib timeout for larger
outputs. Raise server timeout to 60s, add browser AbortController at 65s
so the UI fails gracefully. Trim max_tokens (row: 300, dataset: 512) to
keep generation well inside the window.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Single-threaded HTTPServer blocked readiness probes during long NIM API
calls (~30s), causing Kubernetes to flip the pod unhealthy until the
request completed. ThreadingHTTPServer handles each request in its own
thread so probes are never starved.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add ⚡ button on each row (first 200) — clicking analyses that specific
row with a chess-expert prompt tailored to the entry type (opening,
player, etc.)
- Cache results per row so already-analysed rows skip re-requests
- Serialise all NIM requests through a promise queue to avoid bursting
- Pass 429 + Retry-After through from NIM to the browser; JS surfaces
"Rate limited — retry in Ns" error and re-enables the button for retry
- Dataset-level "Analyse top 10 rows" now sends a per-row numbered prompt
asking for chess-specific analysis of each entry rather than a vague
dataset summary
- Store first 200 rows in window.DATASET.rows for client-side row access
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Browser cannot call integrate.api.nvidia.com directly — no CORS headers.
Add POST /explain endpoint: browser sends key + dataset, server calls NIM
with urllib, returns JSON. Key is never logged or stored server-side.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Python triple-quoted string interprets \n as literal newlines,
producing invalid JS string literals and a SyntaxError that
killed the entire script block.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each dataset page now has an "Explain this dataset" button that calls the
NVIDIA NIM chat completions API (meta/llama-3.3-70b-instruct) directly from
the browser with headers + first 10 rows as context. API key is entered via
a settings modal and stored in localStorage only — never sent to the server.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
do-block-storage only supports ReadWriteOnce. Executor tasks failed with
`Mkdirs failed` because the executor ran on the spark-worker node which
had no mount of the spark-analytics-output PVC.
- Mount spark-analytics-output PVC on spark-worker pod at /spark-output
- Add podAffinity to all 11 analytics CronJob driver pods to schedule on
the same node as spark-worker, ensuring the RWO PVC is accessible by
both driver and executor
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
13 separate DB connections on index page caused readiness probe timeout.
Now all counts share one connection. BrokenPipeError on probe disconnect silenced.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Handle URLs without explicit port, show DB_HOST:PORT/DB_NAME in error messages
and startup log to simplify connection debugging.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Rewrite webview serve.py to query PostgreSQL analytics tables via psycopg2
instead of reading CSV files from PVC; PVC mount removed from Deployment
- Remove isPgnMode guard from all 4 original jobs so results always write to DB
regardless of whether input comes from game_records or a Lichess PGN dump
- Add JDBC write-back to all 7 new analytics jobs (game_length_distribution,
game_length_by_result, color_advantage, elo_distribution, time_control_stats,
hourly_activity, weekly_activity, rating_mismatch, termination_stats)
- Add analytics_component_sizes JDBC write to PlayerGraphJob
- Add NOWCHESS_PGN_PATH staging patches for the 7 new CronJobs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds CronJobs and headless driver Services for GameLengthJob, ColorAdvantageJob,
EloDistributionJob, TimeControlJob, DailyActivityJob, RatingMismatchJob, and
TerminationStatsJob. All suspended by default; scheduled Saturday mornings.
Adds spark-analytics-webview: a stdlib Python HTTP server (ConfigMap + Deployment
+ Service) that reads CSV output from the spark-analytics-output PVC and renders
13 datasets as browsable HTML tables. Disabled (replicas=0) in base.
Staging overlay enables the webview (replicas=1), exposes it via Ingress at
spark-results-st.nowchess.janis-eccarius.de, and sets NOWCHESS_PGN_PATH on all
CronJobs to the Lichess Jan-2013 rated standard PGN dump for demo runs without
a NowChess database.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Rolling update deadlocked: new pod couldn't attach the RWO checkpoint
PVC while the old pod still held it, so ContainerCreating stalled for
39+ minutes. Recreate terminates the old pod first.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Spark executors crashed immediately (exit 1) because the driver
advertised its pod name as the RPC hostname, which is not DNS-resolvable
in-cluster. Added headless Service `spark-live-dashboard-driver` and
pinned spark.driver.host/port + spark.blockManager.port so workers can
connect back to the driver. Stops the JVM fork storm that saturated all
three nodes at ~100% CPU.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Nodes have only ~60-210Mi free. Requests are scheduler hints only —
pods burst to 1Gi limit once placed. 64Mi fits on all three nodes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Nodes are saturated on requested resources. Driver pods (streaming +
cronjobs) were requesting 500m CPU + 768Mi each. Limits unchanged at
1 CPU / 1Gi — pods can still burst when resources are free.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
local-path PVCs are created root-owned. The apache/spark image runs as
spark (UID 185). fsGroup: 185 makes K8s chown the mounted volume to that
group on attach, fixing the mkdir IOException on /spark-checkpoint.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
local-path provisioner is RWO-only; attaching a single PVC to master, two
workers, history-server, and streaming pod on separate nodes causes
Multi-Attach errors. Each pod now gets its own ephemeral emptyDir for
/spark-events. Event logging confs removed from CronJobs and streaming
Deployment (logs are pod-local and do not persist across restarts).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Wire five Spark jobs (OpeningBook, PlayerStats, PlayerClustering,
PlayerGraph, LiveDashboard) to Kubernetes via native Spark-on-K8s mode.
- spark-analytics/base: ServiceAccount, RBAC Role/RoleBinding (driver
executor pod management), two PVCs (10Gi output, 2Gi checkpoint),
four CronJobs for batch analytics, one Deployment for the Structured
Streaming live dashboard
- spark-analytics/eu-central-1-prod: namespace overlay pinning image
to 0.1.0 with strategic merge patches that update both the container
image field and the spark.kubernetes.container.image conf arg
- eu-central-1/argo-apps/spark-analytics: ArgoCD Application pointing
at the prod overlay with automated prune+selfHeal sync
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Janis
Janis Eccarius