Add proxy-read-timeout and proxy-send-timeout (3600s) so nginx does not
drop long-lived WebSocket connections at the default 60s idle timeout.
Add use-regex: "true" to staging to match production — without it, the
ImplementationSpecific path for the game WebSocket
(/api/board/game/[A-Za-z0-9]{8}/ws) is treated as a literal string
instead of a regex and never matches real game IDs.
---------
Co-authored-by: LQ63 <lkhermann@web.de>
Reviewed-on: #419
Co-authored-by: Leon Hermann <lq@blackhole.local>
Co-committed-by: Leon Hermann <lq@blackhole.local>
Add /api/analysis → nowchess-analysis-active:8087 path rule to
eu-central-1-staging, eu-central-1-prod, and htwg-1-prod ingresses.
Rule placed before the /api catch-all so nginx routes correctly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The merged rollout set JWT_PUBLIC_KEY_PATH, but the running image's
config never references it, so mp.jwt.verify.publickey.location stayed
null and every token failed with "Verification key is unresolvable".
Replace it with the MicroProfile properties the runtime reads directly
(MP_JWT_VERIFY_PUBLICKEY_LOCATION, MP_JWT_VERIFY_ISSUER), pointing at the
already-mounted ncs-jwt-keys public key. No image rebuild needed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reviewed-on: #416
The official-bots rollout never mounted the ncs-jwt-keys secret nor set
JWT_PUBLIC_KEY_PATH, so the service fell back to the image-baked
public.pem, which does not match the deployed signing key. Every token
failed RS256 verification -> 401.
Mount ncs-jwt-keys at /secrets/jwt and point JWT_PUBLIC_KEY_PATH there,
mirroring account/core/tournament. Verify-only, no private key.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reviewed-on: #413
/api/bots/official/join-tournament returned 404 because no ingress
rule matched the prefix; requests fell through to the /api catch-all
and hit nowchess-core (8080), which has no such endpoint.
Add /api/bots Prefix rule -> nowchess-official-bots-active:8088 in
all three environments (staging, eu-central-1-prod, htwg-1-prod).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reviewed-on: #403
With SPARK_WORKER_CORES=1 and 2 workers the cluster has 2 total cores.
LiveDashboard holds both — batch jobs wait indefinitely. 4 cores per
worker gives 8 total, enough to run streaming + batch concurrently.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bitnami-labs GitHub Pages repo (bitnami-labs.github.io/sealed-secrets)
returns 404 — chart moved to charts.bitnami.com/bitnami. Updates chart
version from 2.16.1 to 2.5.19 (controller 0.27.1 → 0.31.0). Controller
is backward-compatible with existing sealed secrets.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Cluster stable. Re-enabling automated sync on spark-analytics and
spark-cluster. Driver DNS fix and Recreate strategy already in main,
so next sync deploys the corrected manifests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CPU saturation caused an unclean AOF write, corrupting the tail of
appendonly.aof.41.incr.aof at offset 32653224. Redis refuses to start.
Setting aof-load-corrupt-tail-max-size to 88 (value Redis reported) so
it truncates and loads the intact data instead of crash-looping.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Cluster saturated at 100% CPU due to Spark executor crash loop.
Disabling automated sync on spark-analytics and spark-cluster to prevent
further changes until cluster is stable.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Rolling update deadlocked: new pod couldn't attach the RWO checkpoint
PVC while the old pod still held it, so ContainerCreating stalled for
39+ minutes. Recreate terminates the old pod first.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Spark executors crashed immediately (exit 1) because the driver
advertised its pod name as the RPC hostname, which is not DNS-resolvable
in-cluster. Added headless Service `spark-live-dashboard-driver` and
pinned spark.driver.host/port + spark.blockManager.port so workers can
connect back to the driver. Stops the JVM fork storm that saturated all
three nodes at ~100% CPU.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Nodes have only ~60-210Mi free. Requests are scheduler hints only —
pods burst to 1Gi limit once placed. 64Mi fits on all three nodes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
svc.kargo
lq64
Janis Eccarius
Janis