Commit Graph

312 Commits

Author SHA1 Message Date
Janis Eccarius f43d1930d8 fix(official-bots): make botToken optional, fall back to env, fix 502 status
Build & Test (NowChessSystems) TeamCity build finished
botToken in JoinTournamentRequest is now Option[String]. When absent the
service resolves it from TOURNAMENT_BOT_TOKEN env var so official-bot
join requests no longer need a token in the body.

Response status on join failure changed from BAD_GATEWAY (502) to
BAD_REQUEST (400).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:40:09 +02:00
Janis Eccarius da0e6d1ee2 feat(analytics): always write results to PostgreSQL regardless of input source
Build & Test (NowChessSystems) TeamCity build failed
Remove isPgnMode JDBC guard from all 4 original jobs so staging (Lichess PGN mode)
and production (game_records JDBC mode) both persist analytics results to the DB.

Add JDBC write-back to all 7 new jobs:
- GameLengthJob → analytics_game_length_distribution + analytics_game_length_by_result
- ColorAdvantageJob → analytics_color_advantage
- EloDistributionJob → analytics_elo_distribution
- TimeControlJob → analytics_time_control_stats
- DailyActivityJob → analytics_hourly_activity + analytics_weekly_activity
- RatingMismatchJob → analytics_rating_mismatch
- TerminationStatsJob → analytics_termination_stats

Add analytics_component_sizes JDBC write to PlayerGraphJob.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:36:07 +02:00
TeamCity a6c600d6ce ci: bump version with Build-131 analytics-0.5.0 2026-06-21 13:28:40 +00:00
Janis Eccarius 8e17c14dff feat(analytics): add 7 new Spark analytics jobs and extend GameSource
Build & Test (NowChessSystems) TeamCity build finished
Adds GameLengthJob, ColorAdvantageJob, EloDistributionJob, TimeControlJob,
DailyActivityJob, RatingMismatchJob, and TerminationStatsJob bringing total
batch pipelines to 11 (+ 1 streaming).

Extends GameSource with loadExtended() / fromLichessPgnExtended() extracting
WhiteElo, BlackElo, TimeControl, UTCDate, UTCTime, Termination, ECO from PGN
headers; JDBC path returns nulls for extended columns, keeping all existing
jobs unaffected.

PlayerStatsJob gains a CSV output alongside the existing Parquet write so
the analytics webview can display player statistics without pyarrow.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 15:03:07 +02:00
TeamCity a91ba5da9a ci: bump version with Build-130 analytics-0.4.0 core-0.53.0 official-bots-0.21.0 2026-06-21 11:34:38 +00:00
Janis Eccarius f079f42d28 ci: skip spotless and scalafix checks in native build
Build & Test (NowChessSystems) TeamCity build finished
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 12:43:21 +02:00
Janis Eccarius be941ff414 style: apply spotless formatting
Build & Test (NowChessSystems) TeamCity build finished
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 12:39:44 +02:00
TeamCity 36a0b26c65 ci: bump version with Build-128 api-0.18.0 core-0.52.0 tournament-0.4.0 2026-06-21 09:39:54 +00:00
Janis Eccarius f2d79e4952 fix: wrap server list response in ExternalTournamentServerList
Build & Test (NowChessSystems) TeamCity build finished
Frontend expects `{ servers: [...] }` but endpoint returned a plain array.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 11:15:39 +02:00
Janis Eccarius fba324a5b0 feat(game): add GET /{gameId}/fen-history endpoint
Build & Test (NowChessSystems) TeamCity build finished
Returns a FEN string for every ply of the game (initial position + one
per move) by replaying moves via ruleSet.applyMove and exporting each
GameContext to FEN. Enables full per-move engine analysis from clients
without requiring a chess library on their side.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-21 10:54:35 +02:00
TeamCity 7bf91b2280 ci: bump version with Build-126 analytics-0.3.0 official-bots-0.20.0 ws-0.17.0 2026-06-19 10:28:49 +00:00
lq64 343e2bdd10 fix: NCS-122 authenticate WebSocket connections via first-message auth (#73)
Build & Test (NowChessSystems) TeamCity build finished
Replace header-based auth (not possible with browser WebSocket API) with a
first-message auth protocol: client sends {"type":"auth","token":"<JWT>"}
as the first text frame; server validates and proceeds or closes the connection.

Both GameWebSocketResource and UserWebSocketResource now hold incoming
connections in a pendingAuth set until the auth frame arrives, preventing
any game or event messages from being processed before identity is established.

Also removes the broken Bearer-prefix handling that caused header-based auth
to silently fail even for non-browser clients.

---------

Co-authored-by: LQ63 <lkhermann@web.de>
Reviewed-on: #73
Co-authored-by: Leon Hermann <lq@blackhole.local>
Co-committed-by: Leon Hermann <lq@blackhole.local>
2026-06-17 10:42:52 +02:00
Janis 751a58b606 feat(official-bots): park expert bot on tournament server at startup (#76)
Build & Test (NowChessSystems) TeamCity build was queued
Reviewed-on: #76
2026-06-17 10:42:42 +02:00
Janis 30295a4bb9 feat(official-bots): park expert bot on tournament server at startup (#75)
Build & Test (NowChessSystems) TeamCity build failed
Park the expert bot on the configured tournament server (default
http://141.37.123.132:8086) on startup, reusing a fixed
TOURNAMENT_BOT_TOKEN when present instead of minting a new identity.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Reviewed-on: #75
2026-06-17 10:35:36 +02:00
TeamCity f44d3ee376 ci: bump version with Build-125 account-0.25.0 official-bots-0.19.0 2026-06-17 07:24:13 +00:00
lq64 688d30e2b1 fix: enable official bots to connect to external tournament server (#71)
Build & Test (NowChessSystems) TeamCity build finished
Two bugs prevented official bots from joining the external tournament-server:

1. JWT claim mismatch — bot tokens lacked the `isBot: true` claim the
   tournament server requires. Added the claim to generateBotToken() in
   AccountService, which covers both user-owned bots and official bots.

2. Broken join flow — TournamentBotGamePlayer.joinTournament() called
   registerBot() which hit POST /api/auth/register on the tournament server,
   an endpoint that does not exist. Removed registerBot() and updated
   JoinTournamentRequest to accept a botToken field so the caller supplies
   the pre-existing NowChessSystems token directly.

---------

Co-authored-by: LQ63 <lkhermann@web.de>
Reviewed-on: #71
Co-authored-by: Leon Hermann <lq@blackhole.local>
Co-committed-by: Leon Hermann <lq@blackhole.local>
2026-06-17 09:10:13 +02:00
Janis 98c64fc0d5 fix(official-bots): configure JWT verification (#72)
The official-bots service enabled smallrye-jwt but never set
mp.jwt.verify.publickey.location or issuer, so it could not validate
any incoming token and rejected every authenticated request with 401.

Add the verify public key (issuer nowchess) mirroring tournament/core,
and ship keys/public.pem from the shared keypair.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Reviewed-on: #72
2026-06-17 09:10:01 +02:00
TeamCity 9e800ecb59 ci: bump version with Build-124 analytics-0.2.0 official-bots-0.18.0 tournament-0.3.0 2026-06-16 19:41:52 +00:00
Janis 39f1657e1d feat(analytics): add Spark batch analytics module (#70)
Build & Test (NowChessSystems) TeamCity build finished
Co-authored-by: Janis Eccarius <eccariusjanis@gmail.com>
Reviewed-on: #70
2026-06-16 20:38:14 +02:00
Janis Eccarius 2fd85dbadb chore(analytics): merge remote main — keep analytics module additions
Build & Test (NowChessSystems) TeamCity build failed
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 20:10:56 +02:00
Janis Eccarius 46af1154de fix(analytics): upgrade Spark to 4.0.3 — 3.5.x has no official Docker image
apache/spark:3.5.4-scala2.13-java17-ubuntu does not exist on Docker Hub.
Oldest available scala2.13 image is 4.0.3. Bump compileOnly deps and
Dockerfile base image to match.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 20:08:29 +02:00
TeamCity 85cb9b2e7a ci: bump version with Build-123 analysis-0.2.0 2026-06-15 20:52:53 +00:00
Janis Eccarius 0e0ea4c989 feat(analytics): add PostgreSQL JDBC write-back to all four batch jobs
Each batch job now writes its results to a Postgres table in addition to
the existing Parquet/CSV output. OpeningBookJob → analytics_opening_stats,
PlayerStatsJob → analytics_player_stats, PlayerClusteringJob →
analytics_player_clusters + analytics_cluster_archetypes, PlayerGraphJob
→ analytics_player_graph. MLlib Vector columns are excluded from the JDBC
write by reusing the already-selected scalar DataFrame in
PlayerClusteringJob.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 22:35:30 +02:00
Janis Eccarius 95215b6a42 feat(analytics): add Dockerfile, CI workflow, and stable jar name for K8s deployment
- Pin jar output to analytics.jar (no version suffix) so Dockerfile COPY is stable
- Add Dockerfile based on apache/spark:3.5.4-scala2.13-java17-ubuntu
- Add versions.env (0.1.0) matching GitOps overlay image tag
- Add analytics-image.yml CI workflow following native-image.yml conventions

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 22:30:31 +02:00
Janis b1c9e962e7 fix(version): Wrong version file
Build & Test (NowChessSystems) TeamCity build finished
2026-06-15 22:26:27 +02:00
Janis Eccarius e1d80b9331 feat(analytics): add Structured Streaming, MLlib clustering, GraphX jobs
Three new Spark jobs demonstrating complementary Spark pillars:

LiveDashboardJob (Structured Streaming):
- Simulates NowChess game-over event stream via rate source
- Watermarking (45 s late-data tolerance)
- Tumbling 1-min windows → append-mode Parquet output
- Sliding 5-min/1-min windows → update-mode console output
- Checkpointing for exactly-once fault tolerance
- Production wiring comments show Kafka / spark-redis swap-in

PlayerClusteringJob (MLlib):
- Derives 4 player features from game_records via JDBC
- VectorAssembler + StandardScaler + KMeans inside a Pipeline
- ClusteringEvaluator (silhouette score) to measure quality
- Per-cluster archetype averages show what each tier represents

PlayerGraphJob (GraphX):
- Builds directed player graph (vertices=players, edges=games)
- PageRank — identifies most influential/active players
- ConnectedComponents — finds isolated player communities
- Bridges GraphX RDD results back to DataFrames via explicit schema
  (avoids spark.implicits._ which breaks Scala 3 → Spark 2.13 interop)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 22:15:24 +02:00
Janis Eccarius 259b3bbb24 feat(analytics): add Spark batch analytics module
New standalone modules:analytics submodule with two Spark jobs:

- OpeningBookJob: reads game_records.pgn, extracts first N plies using
  pure Catalyst SQL expressions (no UDFs), aggregates win/draw/loss rates
  per opening sequence, writes Parquet + CSV top-1000 summary.

- PlayerStatsJob: unions each game into a player-centric view, aggregates
  total_games/wins/losses/draws/avg_move_count/win_rate per player_id,
  writes Parquet.

Module uses Scala 3 calling spark-sql_2.13 via JVM binary compatibility
(DataFrame API only; no spark.implicits._ / typed Datasets). Spark is
compileOnly; the fat jar bundles only scala3-library + postgresql driver.
Submit via spark-submit; see build.gradle.kts header for invocation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 21:58:05 +02:00
Janis 0bdf72bddc feat(analysis): scaffold chess analysis microservice (NCS-71) NCI-10 (#69)
Build & Test (NowChessSystems) TeamCity build finished
NCS-95 NCS-96 NCS-97 NCI-10

---------

Co-authored-by: Janis Eccarius <eccariusjanis@gmail.com>
Reviewed-on: #69
2026-06-15 21:40:24 +02:00
TeamCity 0a5a216032 ci: bump version with Build-121 tournament-0.2.0 2026-06-10 09:57:45 +00:00
TeamCity 4be32afe13 ci: bump version with Build-120
Build & Test (NowChessSystems) TeamCity build finished
core-0.51.0
2026-06-10 09:47:42 +00:00
Janis 1aee39c1ad feat(reflection): add GameWritebackEventDto to native reflection configuration
Build & Test (NowChessSystems) TeamCity build finished
2026-06-10 11:19:21 +02:00
Janis e31825021c feat(reflection): add native reflection configuration for tournament classes
Build & Test (NowChessSystems) TeamCity build failed
fix(ws): improve WebSocket connection cleanup on close
chore(stream): simplify group name generation in GameCreationStreamClient
2026-06-10 09:51:52 +02:00
TeamCity e6df9d7b2a ci: bump version with Build-119 account-0.24.0 ws-0.16.0 2026-06-10 07:28:23 +00:00
Janis 65bc6a7599 feat(reflection): add native reflection configuration for tournament classes
Build & Test (NowChessSystems) TeamCity build finished
fix(ws): improve WebSocket connection cleanup on close
chore(stream): simplify group name generation in GameCreationStreamClient
2026-06-10 09:00:24 +02:00
Janis a50884a11b fix(tournament): replace scala.util.Random singleton with UUID for native image
Build & Test (NowChessSystems) TeamCity build failed
GraalVM native image fails when scala.util.Random companion object (a static
singleton with cached seed) is reachable from the image heap. UUID.randomUUID()
is always runtime-initialized and safe.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-10 08:48:55 +02:00
TeamCity 60f4e87579 ci: bump version with Build-118 account-0.23.0 api-0.17.0 bot-platform-0.12.0 core-0.50.0 official-bots-0.17.0 tournament-0.1.0 ws-0.15.0 2026-06-09 22:12:20 +00:00
shosho996 145f467648 feat: NCS-121 pipeline for tournament (#68)
Build & Test (NowChessSystems) TeamCity build finished
Image for Tournment

---------

Co-authored-by: Lala, Shahd <Shahd.Lala@sybit.de>
Reviewed-on: #68
2026-06-09 23:50:51 +02:00
Janis db9d153391 feat(official-bots): consume GameOver stream for bot cleanup (#67)
Build & Test (NowChessSystems) TeamCity build finished
Add consumer group official-bots-game-over on {prefix}:game-over stream.
Track pub/sub subscribers per gameId in gameWatches map. On GameOver event,
unsubscribe from the game s2c channel and remove from watch map.
XACK after cleanup; DLQ after maxRetries failures.

Closes NCS-103
https://knockoutwhist.youtrack.cloud/issue/NCS-103

Reviewed-on: #67
2026-06-09 21:49:42 +02:00
Janis 55f102cbaa feat(ws): migrate challenge notifications to Redis Streams (#66)
Replace pub/sub publish in EventPublisher with XADD to user event stream.
UserWebSocketResource subscribes via XREADGROUP consumer group (per-connection
group, '$' offset). DLQ after maxRetries=3 on delivery failure. Poll loop
uses connection identity to prevent thread leak on reconnect.

Closes NCS-104
https://knockoutwhist.youtrack.cloud/issue/NCS-104

Reviewed-on: #66
2026-06-09 21:49:21 +02:00
Janis d66b6fa471 chore(account): remove dead CoreGameClient REST trait (#65)
Move CoreCreateGameRequest, CorePlayerInfo, CoreTimeControl to CoreGameDtos.
Delete CoreGameClient trait (replaced by GameCreationStreamClient) and
CoreGameResponse (unused after stream migration). Remove from reflection config.

Closes NCS-105
https://knockoutwhist.youtrack.cloud/issue/NCS-105

Reviewed-on: #65
2026-06-09 21:49:05 +02:00
Janis 676e4110c0 feat(core): publish GameOver event to Redis Streams (#64)
Add GameOver to EventType enum and GameOverPayload DTO.
GameRedisPublisher publishes to {prefix}:game-over stream (MAXLEN ~1000)
on game completion. NativeReflectionConfig updated for core module.

Closes NCS-102
https://knockoutwhist.youtrack.cloud/issue/NCS-102

Reviewed-on: #64
2026-06-09 21:48:41 +02:00
Janis 0ad2e10999 feat(bot-platform): migrate BotRegistry to Redis Streams consumer group (#63)
Replace pub/sub subscribe with XREADGROUP on bot game-start stream.
Remove dual-write from EventPublisher.publishGameStart.
Consumer group: bot-platform-consumer, XACK after forwarding.
Poll loop uses emitter identity to prevent thread leak on re-registration.
Group created with '$' offset — no historical replay on first connect.

Closes NCS-101
https://knockoutwhist.youtrack.cloud/issue/NCS-101

Reviewed-on: #63
2026-06-09 21:48:21 +02:00
TeamCity 225c2285b7 ci: bump version with Build-116 account-0.22.0 core-0.49.0 official-bots-0.16.0 2026-06-09 13:28:34 +00:00
lq64 c5661de4a0 feat: NCS-82 add Swiss-system tournament module (#55)
Build & Test (NowChessSystems) TeamCity build finished
## Summary

  - Implements the full tournament lifecycle (create, join, withdraw, start,
    round progression, finish) as a standalone Quarkus module
  - All 11 endpoints from the OpenAPI spec (`docs/tournament-openapi.yaml`) are covered
  - Swiss pairing algorithm with Buchholz tiebreak and bye support
  - Per-bot NDJSON event stream with targeted `gameStart` events carrying
    the correct `color` field
  - Game results ingested via Redis writeback stream (`GameResultStreamListener`)

  ## Known gaps (deferred)

  - `GET /results` `nb` param defaults to 100 instead of all
  - `PairingDto` exposes an internal `id` field not in the spec
  - `GameExport.moves` emits PGN instead of UCI (upstream `GameWritebackEventDto`
    does not carry UCI moves)
  - `Pairing.white` can be `null` for bye rounds (spec has no bye concept)

  ## Test plan

  - [x] 23 `TournamentResourceTest` integration tests (H2, mocked core client) — all pass
  - [x] 5 `SwissPairingServiceTest` unit tests — all pass
  - [x] Redis listener excluded in test/dev profiles; no Docker required to run tests

---------

Co-authored-by: LQ63 <lkhermann@web.de>
Co-authored-by: Lala, Shahd <Shahd.Lala@sybit.de>
Reviewed-on: #55
2026-06-09 15:09:53 +02:00
TeamCity 3b6c5297f6 ci: bump version with Build-115 account-0.21.0 api-0.16.0 core-0.48.0 official-bots-0.15.0 2026-06-09 08:54:49 +00:00
Janis a24924c230 feat(events): migrate game-creation and bot flows to Redis Streams NCS-89 (#62)
Build & Test (NowChessSystems) TeamCity build finished
Replace synchronous account→core game-creation HTTP call and plain
pub/sub bot game-start events with Redis Streams using consumer groups,
XACK, retry, and a Dead Letter Queue for at-least-once delivery and
observability.

- account: GameCreationStreamClient publishes game-creation requests and
  correlates responses via a per-instance consumer group (NCS-91)
- core: GameCreationStreamListener consumes requests, calls
  GameCreationService, publishes response events, retries, and routes
  exhausted/unparseable events to the DLQ (NCS-91, NCS-93, NCS-94)
- official-bots: bot game-start events migrated from pub/sub to Streams
  with consumer group, XACK, retry, and DLQ (NCS-92)
- account EventPublisher dual-writes to the stream and legacy pub/sub
  channel for backward compatibility
- all flows use the typed EventEnvelope (eventId/type/payload/timestamp/
  correlationId) with DLQ error context (eventType, error, attempt)
- register new DTOs and EventEnvelope/EventType for native reflection

Closes NCS-91, NCS-92, NCS-93, NCS-94

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Janis Eccarius <eccariusjanis@gmail.com>
Reviewed-on: #62
2026-06-09 10:31:32 +02:00
Janis Eccarius 11826356c1 docs: add automated defect creation workflow guide 2026-06-06 15:31:33 +02:00
TeamCity e18b1df744 ci: bump version with Build-114
Build & Test (NowChessSystems) TeamCity build failed
account-0.20.0 api-0.15.0
2026-06-05 10:28:26 +00:00
Janis 595c172900 feat(api): define shared EventEnvelope and EventType for Redis EventBus (#61)
Build & Test (NowChessSystems) TeamCity build finished
- Add EventEnvelope case class (eventId, type, payload, timestamp, correlationId)
- Add EventType enum with all known event types
- Update account EventPublisher to use EventEnvelope instead of raw string interpolation
- Add EventEnvelope/EventType to account NativeReflectionConfig
- Add Jackson Scala and JSR310 modules to api dependencies
- Add api module dependency to account module
- Add NativeReflectionConfig rule to CLAUDE.md

Closes NCS-90
https://knockoutwhist.youtrack.cloud/issue/NCS-90

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Reviewed-on: #61
2026-06-05 12:11:14 +02:00
Janis cf225826c0 chore(commands): re-validate after review findings in fix/implement workflows
Add Step 5b to fix-defect and implement-feature commands: after applying
review comments, rerun compile, test, spotlessScalaApply, and lint, then
re-spawn reviewer to confirm findings are resolved.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 11:17:54 +02:00