Compare commits

..

37 Commits

Author SHA1 Message Date
TeamCity 07c0c8adc3 ci: bump version with Build-162 2026-06-30 23:07:08 +00:00
Janis 6e37a7d209 fix(bot): drop game-stream play, poll with low delay
Build & Test (NowChessSystems) TeamCity build finished
The native JAX-RS client buffers the NDJSON game stream, so the read
blocked forever and the bot never moved. Remove streamGameLoop and play
purely via polling at a 150ms interval; clock-aware budgets and the
server-clock read are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 00:56:32 +02:00
TeamCity eeae4f01b4 ci: bump version with Build-161 2026-06-30 20:29:13 +00:00
Janis 45b5719d63 feat(bot): clock-aware time management and stream-driven tournament play
Build & Test (NowChessSystems) TeamCity build finished
Tournament bots flagged in 5+3 classical: budgets were fixed and
clock-blind, HybridBot's veto re-search double-spent (up to 4s/move),
and the game loop polled every 1s, burning our clock waiting on the
opponent.

- Bot is now a trait taking a TimeControl (remaining + increment);
  apply(ctx) defaults to Unlimited so local/self-play/tests keep their
  fixed budgets.
- TimeControl.budget derives a per-move budget from the real clock with
  an overhead reserve, a panic mode under 20s, and a hard ceiling, so a
  bot can no longer flag from thinking.
- HybridBot splits one budget across main (0.7) and veto (0.3) searches
  instead of running two full searches.
- TournamentBotGamePlayer reads the server clock (seconds -> ms) and
  plays stream-driven via GET /game/{id}/stream (NDJSON, heartbeat-kept),
  so the opponent's move arrives instantly; polling stays as a fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 22:12:55 +02:00
Janis b2683a7f5a feat(bot): implement bot-vs-bot harness for NNUE evaluation 2026-06-30 22:12:53 +02:00
Janis 3437dab49b feat(bot): add Lazy SMP parallel search for the NNUE bot
Adds optional multithreaded search behind a thread count that defaults to
1, so the live bot's play is unchanged until explicitly configured.

- ParallelSearch runs N AlphaBetaSearch workers over one shared,
  already-lock-protected TranspositionTable. Each worker has its own NNUE
  evaluator (independent accumulator) and ordering state; helpers only
  deepen the shared TT, the main worker's move is returned.
- AlphaBetaSearch gains bestMoveWithTimeSharedTt: the coordinator clears
  the shared TT once before launching workers, so helpers must not clear.
- EvaluationNNUE.freshEvaluator builds independent evaluators sharing the
  immutable weights (one per thread); the singleton still backs the
  default single-instance path.
- NNUEBot uses ParallelSearch with NNUE_SEARCH_THREADS (default 1).

numThreads <= 1 takes the single-worker clearing path, identical to the
previous sequential search. Strength can be validated by self-play
(threads N vs 1) before promoting the default.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 22:12:52 +02:00
Janis b72e8ec017 refactor(bot): split NNUE into shared weights and per-thread evaluator
Prerequisite for parallel search. NNUE held all state on one instance:
the immutable transposed L1 weight matrix alongside the mutable
accumulator stack, scratch buffers and eval cache. That made concurrent
eval calls corrupt shared buffers.

Extract the read-only parameters into NNUEWeights (heavy to build, safe
to share). NNUE now owns only per-instance mutable buffers and references
the shared weights, so many evaluators can run in parallel over one weight
matrix without duplicating it. Single-instance behaviour is unchanged —
EvaluationNNUE still uses one evaluator, so play is identical.

Also applies scalafmt alignment to the MopUp files.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 22:12:50 +02:00
TeamCity 7136803c7e ci: bump version with Build-160 2026-06-30 10:21:06 +00:00
Janis faf7eb38ea fix(bot): seed search with game history, add contempt and NNUE mop-up
Build & Test (NowChessSystems) TeamCity build finished
Repetition: alpha-beta seeded the repetition map with only the root
position, so search was blind to positions already reached in the real
game and would happily shuffle into draws when ahead. Reconstruct the
full game-history position hashes by replaying moves and seed the search
state with them; treat a twofold occurrence at non-root nodes as a draw.

Contempt: draws are now scored CONTEMPT (25cp) away from zero, signed by
ply parity, so the bot avoids dead-equal repetitions instead of settling.

Endgame: pure NNUE lacks mating knowledge and stalls KX-vs-K conversions.
Add a MopUp correction (edge-driving + king-proximity) applied only in
lone-king endgames with sufficient mating material; zero elsewhere so
middlegame NNUE output is untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 12:04:27 +02:00
TeamCity 344bed6935 ci: bump version with Build-159 2026-06-29 17:34:14 +00:00
Janis Eccarius 4938560014 fix(bot): include quiet promotions in quiescence search
Build & Test (NowChessSystems) TeamCity build finished
Quiescence tactical filter only flagged capture-promotions, so a quiet
queening on an empty back-rank square was treated as non-tactical and
skipped at the search horizon. A bot could therefore miss a winning
promotion sitting exactly at the horizon and play another move. All bots
(Classical/NNUE/Hybrid) share AlphaBetaSearch and were affected.

Treat every promotion as tactical so quiescence always expands it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 19:18:31 +02:00
TeamCity 4a397eed7f ci: bump version with Build-158 2026-06-24 20:41:25 +00:00
Janis Eccarius 9d656624d8 fix(official-bots): stream NNUE features as sparse indices to stop host OOM
Build & Test (NowChessSystems) TeamCity build finished
Densifying the 98304-dim HalfKP vector per item filled host RAM and crashed the
Colab runtime even at small batch sizes. The dataset now yields only the ~64
active feature indices; a custom collate carries (row, col) pairs and the
training loop scatters them into a dense [B, INPUT_SIZE] tensor on the GPU. Host
RAM stays tiny; GPU holds one dense batch transiently.

- NNUEDataset.__getitem__ returns indices via new fen_to_indices.
- fen_to_features now derives from fen_to_indices (kept for external callers).
- _collate_sparse builds row/col index batches; loaders use it.
- train/val loops scatter to a GPU dense batch; loss weighting uses batch size.
- Notebook: BATCH_SIZE 4096 -> 8192 (host no longer the limit; GPU is).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 22:28:53 +02:00
Janis Eccarius e2b4342f60 fix(official-bots): prevent Colab OOM in NNUE training
Build & Test (NowChessSystems) TeamCity build finished
Dense 98304-dim HalfKP features at batch_size=16384 cost ~6.4 GB/batch on the
host; with 8 hardcoded DataLoader workers and prefetch this OOM-killed the Colab
runtime.

- train.py: adaptive DataLoader workers (min(4, cpu_count), Colab free tier = 2),
  overridable via NNUE_LOADER_WORKERS; persistent_workers only when > 0.
- NNUETraining.ipynb: lower BATCH_SIZE 16384 -> 4096 with a memory-cost note.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 22:18:18 +02:00
TeamCity 9d56446c65 ci: bump version with Build-156 2026-06-24 20:17:17 +00:00
Janis Eccarius 1c80abdb8a feat(official-bots): standalone self-play + one-shot dataset builder for NNUE training
Build & Test (NowChessSystems) TeamCity build finished
Add an easy local data pipeline feeding GPU training on Colab.

- SelfPlayMain: standalone NNUEBot self-play (no microservices) writing FENs
  for labeling; randomised openings for game diversity, sequential due to the
  shared EvaluationNNUE accumulator. Exposed via the `selfPlay` Gradle task and
  selfplay.sh.
- NNUEBot: optional fixedMoveTimeMs so self-play runs fast (default unchanged).
- NbaiLoader: honor `-Dnnue.weights=<path>` to load weights from a file before
  falling back to the bundled resource.
- build_dataset.py / dataset.sh: one command builds the entire dataset
  (Lichess eval-DB backbone + self-play + tactical + random filler), dedups,
  balances the eval histogram, writes append-only zstd shards + manifest, and
  rclone-pushes to Drive.
- train.py: NNUEDataset reads a directory of .jsonl.zst shards (streaming) in
  addition to a single file.
- NNUETraining.ipynb: clone to ephemeral /content, sync shards from Drive
  (cache-aware), train on the shards dir; removed Colab generation/upload steps.
- Concept + implementation plan docs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 22:04:22 +02:00
TeamCity c8cbcdca3b ci: bump version with Build-155 2026-06-24 18:21:11 +00:00
Janis e4fee85134 feat(ncs-110): feed NNUE root-move scores into search move ordering (#83)
Build & Test (NowChessSystems) TeamCity build finished
Pre-evaluated NNUE scores from NNUEBot.batchEvaluateRoot are now passed
as root hints into AlphaBetaSearch, improving move ordering at ply 0 before
the TT is populated. Hints are threaded immutably through SearchParams to
satisfy the no-var constraint.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Janis Eccarius <eccariusjanis@gmail.com>
Reviewed-on: #83
2026-06-24 20:09:28 +02:00
TeamCity b4709b4a33 ci: bump version with Build-154 2026-06-24 17:55:44 +00:00
Janis Eccarius 9f9140cb58 fix: modified training pipeline
Build & Test (NowChessSystems) TeamCity build finished
2026-06-24 19:37:26 +02:00
Janis fa10852bc9 feat(official-bots): add Google Colab notebook for NNUE training (NCS-111) (#81)
Build & Test (NowChessSystems) TeamCity build finished
Adds python/NNUETraining.ipynb with five sections:
- Setup: mount Drive, clone/update repo, install deps + Stockfish
- Data: Option A (generate + label) or Option B (upload existing labeled.jsonl)
- Train: standard epoch loop or burst mode (recommended for Colab free tier)
- Export: convert best .pt checkpoint to .nbai via export.py
- Download: pull .nbai and .pt to local machine via files.download

Checkpoints and datasets are persisted to Google Drive so training
survives session disconnects and can be resumed automatically.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Janis Eccarius <eccariusjanis@gmail.com>
Reviewed-on: #81
2026-06-24 19:33:24 +02:00
Janis 44f376f032 feat(official-bots): implement king-relative (HalfKP) encoding in NNUE (NCS-109) (#80)
Co-authored-by: Janis Eccarius <eccariusjanis@gmail.com>
Reviewed-on: #80
2026-06-24 19:33:12 +02:00
TeamCity 7372867a82 ci: bump version with Build-152 2026-06-23 22:30:53 +00:00
Janis Eccarius c3e7b82ae8 feat(analytics): add accuracy and blunder analysis job for Lichess data
Build & Test (NowChessSystems) TeamCity build finished
2026-06-24 00:21:40 +02:00
TeamCity e88b081947 ci: bump version with Build-151 2026-06-23 21:54:06 +00:00
Janis Eccarius 1b30c3be39 fix(official-bots): use ThreadLocalRandom in PolyglotBook for native image
Build & Test (NowChessSystems) TeamCity build finished
A stored java.util.Random field is reachable from BotController's static
openingBook, so GraalVM baked it into the image heap and aborted the
native build (Random in image heap has a cached seed). Use
ThreadLocalRandom.current() at call time instead — no stored instance,
nothing in the image heap, still thread-safe.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 23:42:15 +02:00
Janis Eccarius f8ca95af3c refactor(official-bots): use java.util.Random in PolyglotBook
Build & Test (NowChessSystems) TeamCity build finished
scala.util.Random delegates to a shared global java.util.Random, a
contention point across concurrent bot games. Use a per-book
java.util.Random instance instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 23:34:38 +02:00
TeamCity 4a50db0721 ci: bump version with Build-150 2026-06-23 21:27:19 +00:00
Janis Eccarius 260db25803 feat(official-bots): activate opening book in expert bot (native-safe)
Build & Test (NowChessSystems) TeamCity build finished
Load the Polyglot opening book as a classpath resource and wire it into
the expert HybridBot. Previously the bot supported Option[PolyglotBook]
but BotController passed None, so the book was never used.

PolyglotBook.fromResource reads via getResourceAsStream so the book is
embedded in the GraalVM native image instead of read from the filesystem
(FileInputStream) — no file needs mounting into the pod. The filesystem
apply(path) factory is kept for tests. Moved codekiddy.bin into
resources as opening_book.bin. Dropped the per-probe debug println.

NCS-43

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 23:17:52 +02:00
TeamCity 80e1cc258b ci: bump version with Build-149 2026-06-23 21:08:35 +00:00
Janis bfc46723e6 fix(official-bots): derive tournament game color from game endpoint (#79)
Build & Test (NowChessSystems) TeamCity build finished
Tournament-server reports wrong color in pairings (everyone white), so
auto-joined games could play with an inverted color and never move on
their real turn. The game endpoint white/black ids are correct, so the
poll loop now derives our color from it, falling back to the passed-in
color. Both stream and auto-join entry paths are now immune to the bug.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Janis Eccarius <eccariusjanis@gmail.com>
Reviewed-on: #79
2026-06-23 22:58:09 +02:00
TeamCity bace029a8a ci: bump version with Build-148 2026-06-23 20:31:27 +00:00
Janis Eccarius 7664042193 fix(tournament): mirror bot join onto native twin
Build & Test (NowChessSystems) TeamCity build finished
The UI reads participant/standings fields from the native-server twin
(nativeOverlay), but bot join only wrote the NowChess participant list,
so bots never appeared in replicated/native-published tournaments. On
join, register the bot on the native server by name and join the twin
as that bot. Also run this for the AlreadyJoined case so bots stuck in
the NowChess list (but missing on native) get reconciled, and return
200 instead of 409 for it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 22:20:56 +02:00
TeamCity a604b4ad42 ci: bump version with Build-147 2026-06-23 19:33:36 +00:00
Janis Eccarius fdf4c94811 fix(official-bots): resolve per-difficulty bot token on tournament join
Build & Test (NowChessSystems) TeamCity build finished
joinTournament only ever had a token for the startup difficulty
(default medium); other difficulties fell back to the single shared
TOURNAMENT_BOT_TOKEN, which our tournament server rejects (401),
surfacing as 400 "Failed to join tournament" in the UI. Resolve and
cache a token for the requested difficulty instead.

Prefer the account-service token over anonymous register in
resolveToken so the bot joins as its canonical identity rather than a
throwaway account (medium joined but never appeared as a participant).

Add NativeReflectionConfig for JoinTournamentRequest/Response so the
success path serializes in native image instead of returning an empty
200 body.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 21:23:41 +02:00
TeamCity d9f30f0bfe ci: bump version with Build-146 2026-06-23 18:45:05 +00:00
Janis 1f4e9c8498 fix(tournament): sync native-server participants and route start (#78)
Build & Test (NowChessSystems) TeamCity build finished
Bots joining a published tournament directly on the native server were not
reflected in NowChess (0 players) and the tournament could not be started,
because create() kept a local copy plus a separate native copy whose id was
discarded — leaving the two records disconnected.

- Capture the native tournament id: createNative/publishNative now return the
  id instead of Boolean; persist it on Tournament.nativeTournamentId.
- Reverse-sync on read: get()/list() overlay nbPlayers/standing/status/round/
  winner from the native twin (with a fullName backfill for tournaments created
  before the id was captured).
- start(): proxy to the native twin (director token via authFor) so the native
  participants are used; mirror the started status locally.
- Skip the native server in the replicate loop (it has no /replicate endpoint),
  removing the per-create "Failed to replicate" warning.
- Isolate native integration in tournament unit tests (native-server-url no
  longer defaults to the live server).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Janis Eccarius <eccariusjanis@gmail.com>
Reviewed-on: #78
2026-06-23 20:34:30 +02:00
48 changed files with 3555 additions and 327 deletions
+17
View File
@@ -81,3 +81,20 @@
* **analytics:** upgrade Spark to 4.0.3 — 3.5.x has no official Docker image ([46af115](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/46af1154de34a8596cb6cb28c6fad7aba90f597c)) * **analytics:** upgrade Spark to 4.0.3 — 3.5.x has no official Docker image ([46af115](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/46af1154de34a8596cb6cb28c6fad7aba90f597c))
* **analytics:** write decompressed PGN to shared PVC path for executor access ([a268a9a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a268a9acb7ba190c76e996ccf3ea3bd00e5cec92)) * **analytics:** write decompressed PGN to shared PVC path for executor access ([a268a9a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a268a9acb7ba190c76e996ccf3ea3bd00e5cec92))
## (2026-06-23)
### Features
* **analytics:** add 7 new Spark analytics jobs and extend GameSource ([8e17c14](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/8e17c14dff740cd115011dfbf17de35083b8fe46))
* **analytics:** add accuracy and blunder analysis job for Lichess data ([c3e7b82](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c3e7b82ae806adf5713ce4d267c1155e73a40ff5))
* **analytics:** add Dockerfile, CI workflow, and stable jar name for K8s deployment ([95215b6](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/95215b6a420fd526df1aa395f9b087556c8ad03b))
* **analytics:** add PostgreSQL JDBC write-back to all four batch jobs ([0e0ea4c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/0e0ea4c9893c6efed52e633e55d05ab3ed004502))
* **analytics:** add Spark batch analytics module ([259b3bb](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/259b3bbb24c0f23326269b93f4b3c84012f727cd))
* **analytics:** add Structured Streaming, MLlib clustering, GraphX jobs ([e1d80b9](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/e1d80b9331666feea191b1fd08aa762f3581c918))
* **analytics:** always write results to PostgreSQL regardless of input source ([da0e6d1](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/da0e6d1ee2d391ecb6291396f82471eb51b1b25e))
* **official-bots:** park expert bot on tournament server at startup ([#76](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/76)) ([751a58b](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/751a58b6061f7434115e229a7661894c76768bc2))
### Bug Fixes
* **analytics:** upgrade Spark to 4.0.3 — 3.5.x has no official Docker image ([46af115](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/46af1154de34a8596cb6cb28c6fad7aba90f597c))
* **analytics:** write decompressed PGN to shared PVC path for executor access ([a268a9a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a268a9acb7ba190c76e996ccf3ea3bd00e5cec92))
@@ -0,0 +1,191 @@
package de.nowchess.analytics
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.expressions.Window
import org.apache.spark.sql.functions as F
/** Per-move accuracy & blunder analysis mined from Lichess `[%eval ...]` move annotations.
*
* Unlike the flat single-`groupBy` summaries (opening rates, colour advantage), this job reconstructs the *quality of
* every move* from the engine evaluations Lichess embeds in the movetext (`{ [%eval 0.24] }`, mate scores `[%eval
* #-3]`) and turns them into the same accuracy signals lichess.com surfaces: average centipawn loss (ACPL), and counts
* of inaccuracies / mistakes / blunders.
*
* Pipeline (all Spark SQL string/array functions + window funcs — no UDFs, Catalyst-friendly):
* 1. Keep only games carrying `[%eval` comments.
* 2. `regexp_extract_all` pulls every eval in ply order; mate scores collapse to ±10 pawns, normal evals are clamped
* to ±10 so a single huge swing cannot dominate the mean. All evals are White-POV pawns.
* 3. `posexplode` → one row per ply; a per-game window `lag` gives the eval *before* the move.
* 4. Centipawn loss for the side that moved = how much the eval moved against them (white wants it up, black down),
* floored at 0 and scaled to centipawns.
* 5. Roll up to (game, side): ACPL + inaccuracy(≥50cp) / mistake(≥100cp) / blunder(≥200cp) counts, tagged with that
* side's Elo and whether they won.
*
* Outputs (Parquet + CSV + JDBC):
* - `accuracy_by_rating` — ACPL, avg blunders/mistakes/inaccuracies per game and win-rate, per Elo band. Shows how
* move quality scales with rating.
* - `blunder_outcome` — win-rate bucketed by number of blunders in the game. Quantifies "one blunder costs you the
* game".
*
* Requires the eval-annotated Lichess dump (`NOWCHESS_PGN_PATH` → an evals dump); JDBC games carry no per-move evals.
*/
object AccuracyBlunderJob:
def main(args: Array[String]): Unit =
val jdbcUrl = sys.env.getOrElse("NOWCHESS_JDBC_URL", "jdbc:postgresql://localhost:5432/nowchess")
val dbUser = sys.env.getOrElse("NOWCHESS_DB_USER", "nowchess")
val dbPass = sys.env.getOrElse("NOWCHESS_DB_PASS", "nowchess")
val outputDir = if args.length > 0 then args(0) else "/tmp/nowchess-accuracy"
val spark = SparkSession
.builder()
.appName("NowChess Accuracy & Blunders")
.getOrCreate()
run(spark, jdbcUrl, dbUser, dbPass, outputDir)
spark.stop()
def run(spark: SparkSession, jdbcUrl: String, dbUser: String, dbPass: String, outputDir: String): Unit =
val games = GameSource
.loadExtended(spark, jdbcUrl, dbUser, dbPass)
.select("pgn", "result", "white_elo", "black_elo")
.filter(F.col("result").isNotNull.and(F.col("pgn").contains("[%eval")))
.withColumn("game_id", F.monotonically_increasing_id())
// White-POV pawn evals in ply order; mate → ±10, normal evals clamped to ±10.
val evalStrs = F.expr("""regexp_extract_all(pgn, '\\[%eval ([^\\]]+)\\]', 1)""")
val evalCps = F.expr(
"transform(eval_strs, x -> CASE " +
"WHEN x LIKE '#-%' THEN -10.0 " +
"WHEN x LIKE '#%' THEN 10.0 " +
"ELSE greatest(-10.0, least(10.0, cast(x as double))) END)",
)
val withEvals = games
.withColumn("eval_strs", evalStrs)
.withColumn("eval_cp", evalCps)
.filter(F.size(F.col("eval_cp")) >= 2)
val plies = withEvals.select(
F.col("game_id"),
F.col("result"),
F.col("white_elo"),
F.col("black_elo"),
F.posexplode(F.col("eval_cp")).as(Seq("ply", "eval_after")),
)
val byGame = Window.partitionBy("game_id").orderBy("ply")
val mover = F.when(F.col("ply") % 2 === 0, "white").otherwise("black")
val evalBefore = F.coalesce(F.lag("eval_after", 1).over(byGame), F.lit(0.15))
val cpl = F.greatest(
F.lit(0.0),
F.when(F.col("mover") === "white", evalBefore - F.col("eval_after"))
.otherwise(F.col("eval_after") - evalBefore),
) * 100
val moves = plies
.withColumn("mover", mover)
.withColumn("cpl", cpl)
val perSide = moves
.groupBy("game_id", "mover", "result", "white_elo", "black_elo")
.agg(
F.round(F.avg("cpl"), 1).as("acpl"),
F.sum(F.when(F.col("cpl") >= 200, 1).otherwise(0)).as("blunders"),
F.sum(F.when(F.col("cpl") >= 100 && F.col("cpl") < 200, 1).otherwise(0)).as("mistakes"),
F.sum(F.when(F.col("cpl") >= 50 && F.col("cpl") < 100, 1).otherwise(0)).as("inaccuracies"),
)
.withColumn(
"self_elo",
F.when(F.col("mover") === "white", F.col("white_elo")).otherwise(F.col("black_elo")),
)
.withColumn("won", F.when(F.col("mover") === F.col("result"), 1).otherwise(0))
writeAccuracyByRating(perSide, jdbcUrl, dbUser, dbPass, outputDir)
writeBlunderOutcome(perSide, jdbcUrl, dbUser, dbPass, outputDir)
private def writeAccuracyByRating(
perSide: org.apache.spark.sql.DataFrame,
jdbcUrl: String,
dbUser: String,
dbPass: String,
outputDir: String,
): Unit =
val elo = F.col("self_elo")
val band = F
.when(elo < 1200, "<1200")
.when(elo < 1500, "12001499")
.when(elo < 1800, "15001799")
.when(elo < 2100, "18002099")
.otherwise("2100+")
val bandOrder = F
.when(elo < 1200, 1)
.when(elo < 1500, 2)
.when(elo < 1800, 3)
.when(elo < 2100, 4)
.otherwise(5)
val stats = perSide
.filter(elo.isNotNull)
.withColumn("band", band)
.withColumn("band_order", bandOrder)
.groupBy("band", "band_order")
.agg(
F.count("*").as("player_games"),
F.round(F.avg("acpl"), 1).as("avg_acpl"),
F.round(F.avg("blunders"), 2).as("avg_blunders"),
F.round(F.avg("mistakes"), 2).as("avg_mistakes"),
F.round(F.avg("inaccuracies"), 2).as("avg_inaccuracies"),
F.round(F.avg("won"), 3).as("win_rate"),
)
.orderBy(F.asc("band_order"))
.drop("band_order")
write(stats, outputDir, "accuracy_by_rating", jdbcUrl, dbUser, dbPass, "analytics_accuracy_by_rating")
private def writeBlunderOutcome(
perSide: org.apache.spark.sql.DataFrame,
jdbcUrl: String,
dbUser: String,
dbPass: String,
outputDir: String,
): Unit =
val b = F.col("blunders")
val bucket = F.when(b === 0, "0").when(b === 1, "1").when(b === 2, "2").otherwise("3+")
val order = F.when(b === 0, 0).when(b === 1, 1).when(b === 2, 2).otherwise(3)
val stats = perSide
.withColumn("blunder_bucket", bucket)
.withColumn("bucket_order", order)
.groupBy("blunder_bucket", "bucket_order")
.agg(
F.count("*").as("player_games"),
F.round(F.avg("won"), 3).as("win_rate"),
F.round(F.avg("acpl"), 1).as("avg_acpl"),
)
.orderBy(F.asc("bucket_order"))
.drop("bucket_order")
write(stats, outputDir, "blunder_outcome", jdbcUrl, dbUser, dbPass, "analytics_blunder_outcome")
private def write(
df: org.apache.spark.sql.DataFrame,
outputDir: String,
name: String,
jdbcUrl: String,
dbUser: String,
dbPass: String,
table: String,
): Unit =
df.write.mode("overwrite").parquet(s"$outputDir/$name")
df.write.mode("overwrite").option("header", "true").csv(s"$outputDir/${name}_csv")
if !GameSource.isPgnMode then
df.write
.mode("overwrite")
.format("jdbc")
.option("url", jdbcUrl)
.option("dbtable", table)
.option("user", dbUser)
.option("password", dbPass)
.option("driver", "org.postgresql.Driver")
.save()
@@ -0,0 +1,199 @@
package de.nowchess.analytics
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.expressions.Window
import org.apache.spark.sql.functions as F
/** Time-management & clock-pressure analysis mined from Lichess `[%clk ...]` move annotations.
*
* Lichess records each player's remaining clock after every move (`{ [%clk 0:02:31] }`). This job reconstructs
* per-move thinking time and remaining-time from those stamps to answer questions the existing time-control summary
* cannot: how long do players actually think, how often do they fall into time scrambles (<10 s left), how often do
* they flag (lose on time), and does burning the clock correlate with winning?
*
* Pipeline (Spark SQL string/array funcs + window funcs — no UDFs):
* 1. `regexp_extract_all` pulls every `h:mm:ss` clock in ply order, converted to seconds.
* 2. `posexplode` → one row per ply; even plies are White's clock, odd plies Black's.
* 3. A per-(game,side) window `lag` gives the same side's previous clock; the difference is that move's thinking time.
* Remaining clock <10 s marks a time-scramble move.
* 4. Roll up to (game, side): avg move time, scramble fraction, min clock, Elo, win flag, and whether the side lost on
* time (`Termination "Time forfeit"`).
*
* Outputs (Parquet + CSV + JDBC):
* - `clock_by_rating` — avg move time, scramble fraction, flag-loss rate and win-rate per Elo band.
* - `scramble_outcome` — win-rate bucketed by how much of the game was played in time-scramble. Quantifies the cost of
* time trouble.
*
* Requires a clock-annotated Lichess dump (`NOWCHESS_PGN_PATH`).
*/
object ClockPressureJob:
def main(args: Array[String]): Unit =
val jdbcUrl = sys.env.getOrElse("NOWCHESS_JDBC_URL", "jdbc:postgresql://localhost:5432/nowchess")
val dbUser = sys.env.getOrElse("NOWCHESS_DB_USER", "nowchess")
val dbPass = sys.env.getOrElse("NOWCHESS_DB_PASS", "nowchess")
val outputDir = if args.length > 0 then args(0) else "/tmp/nowchess-clock-pressure"
val spark = SparkSession
.builder()
.appName("NowChess Clock Pressure")
.getOrCreate()
run(spark, jdbcUrl, dbUser, dbPass, outputDir)
spark.stop()
def run(spark: SparkSession, jdbcUrl: String, dbUser: String, dbPass: String, outputDir: String): Unit =
val games = GameSource
.loadExtended(spark, jdbcUrl, dbUser, dbPass)
.select("pgn", "result", "white_elo", "black_elo", "termination")
.filter(F.col("result").isNotNull.and(F.col("pgn").contains("[%clk")))
.withColumn("game_id", F.monotonically_increasing_id())
val clkStrs = F.expr("""regexp_extract_all(pgn, '\\[%clk ([^\\]]+)\\]', 1)""")
// "h:mm:ss" → seconds.
val clkSecs = F.expr(
"transform(clk_strs, x -> " +
"cast(split(x, ':')[0] as double) * 3600 + " +
"cast(split(x, ':')[1] as double) * 60 + " +
"cast(split(x, ':')[2] as double))",
)
val withClk = games
.withColumn("clk_strs", clkStrs)
.withColumn("clk_sec", clkSecs)
.filter(F.size(F.col("clk_sec")) >= 4)
val plies = withClk.select(
F.col("game_id"),
F.col("result"),
F.col("white_elo"),
F.col("black_elo"),
F.col("termination"),
F.posexplode(F.col("clk_sec")).as(Seq("ply", "clk_after")),
)
val mover = F.when(F.col("ply") % 2 === 0, "white").otherwise("black")
val bySide = Window.partitionBy("game_id", "mover").orderBy("ply")
val moveTime = F.lag("clk_after", 1).over(bySide) - F.col("clk_after")
val moves = plies
.withColumn("mover", mover)
.withColumn("move_time", moveTime)
val perSide = moves
.groupBy("game_id", "mover", "result", "white_elo", "black_elo", "termination")
.agg(
F.round(F.avg("move_time"), 1).as("avg_move_time"),
F.count("*").as("moves"),
F.round(F.min("clk_after"), 1).as("min_clk"),
F.sum(F.when(F.col("clk_after") < 10, 1).otherwise(0)).as("scramble_moves"),
)
.withColumn("scramble_fraction", F.round(F.col("scramble_moves") / F.col("moves"), 3))
.withColumn(
"self_elo",
F.when(F.col("mover") === "white", F.col("white_elo")).otherwise(F.col("black_elo")),
)
.withColumn("won", F.when(F.col("mover") === F.col("result"), 1).otherwise(0))
.withColumn(
"flag_loss",
F.when(
F.coalesce(F.col("termination"), F.lit("")).contains("Time forfeit") && F.col("won") === 0,
1,
).otherwise(0),
)
writeClockByRating(perSide, jdbcUrl, dbUser, dbPass, outputDir)
writeScrambleOutcome(perSide, jdbcUrl, dbUser, dbPass, outputDir)
private def writeClockByRating(
perSide: org.apache.spark.sql.DataFrame,
jdbcUrl: String,
dbUser: String,
dbPass: String,
outputDir: String,
): Unit =
val elo = F.col("self_elo")
val band = F
.when(elo < 1200, "<1200")
.when(elo < 1500, "12001499")
.when(elo < 1800, "15001799")
.when(elo < 2100, "18002099")
.otherwise("2100+")
val bandOrder = F
.when(elo < 1200, 1)
.when(elo < 1500, 2)
.when(elo < 1800, 3)
.when(elo < 2100, 4)
.otherwise(5)
val stats = perSide
.filter(elo.isNotNull)
.withColumn("band", band)
.withColumn("band_order", bandOrder)
.groupBy("band", "band_order")
.agg(
F.count("*").as("player_games"),
F.round(F.avg("avg_move_time"), 1).as("avg_move_time_s"),
F.round(F.avg("scramble_fraction"), 3).as("avg_scramble_fraction"),
F.round(F.avg("flag_loss"), 3).as("flag_loss_rate"),
F.round(F.avg("won"), 3).as("win_rate"),
)
.orderBy(F.asc("band_order"))
.drop("band_order")
write(stats, outputDir, "clock_by_rating", jdbcUrl, dbUser, dbPass, "analytics_clock_by_rating")
private def writeScrambleOutcome(
perSide: org.apache.spark.sql.DataFrame,
jdbcUrl: String,
dbUser: String,
dbPass: String,
outputDir: String,
): Unit =
val sf = F.col("scramble_fraction")
val bucket = F
.when(sf === 0, "none")
.when(sf < 0.05, "<5%")
.when(sf < 0.20, "520%")
.otherwise(">20%")
val order = F
.when(sf === 0, 0)
.when(sf < 0.05, 1)
.when(sf < 0.20, 2)
.otherwise(3)
val stats = perSide
.withColumn("scramble_bucket", bucket)
.withColumn("bucket_order", order)
.groupBy("scramble_bucket", "bucket_order")
.agg(
F.count("*").as("player_games"),
F.round(F.avg("won"), 3).as("win_rate"),
F.round(F.avg("flag_loss"), 3).as("flag_loss_rate"),
)
.orderBy(F.asc("bucket_order"))
.drop("bucket_order")
write(stats, outputDir, "scramble_outcome", jdbcUrl, dbUser, dbPass, "analytics_scramble_outcome")
private def write(
df: org.apache.spark.sql.DataFrame,
outputDir: String,
name: String,
jdbcUrl: String,
dbUser: String,
dbPass: String,
table: String,
): Unit =
df.write.mode("overwrite").parquet(s"$outputDir/$name")
df.write.mode("overwrite").option("header", "true").csv(s"$outputDir/${name}_csv")
if !GameSource.isPgnMode then
df.write
.mode("overwrite")
.format("jdbc")
.option("url", jdbcUrl)
.option("dbtable", table)
.option("user", dbUser)
.option("password", dbPass)
.option("driver", "org.postgresql.Driver")
.save()
@@ -0,0 +1,154 @@
package de.nowchess.analytics
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.expressions.Window
import org.apache.spark.sql.functions as F
/** Smurf / sandbagging anomaly detection via population z-scores.
*
* Smurfs (strong players on fresh accounts) and sandbaggers leave a statistical signature: a win-rate, an upset-rate
* (beating higher-rated opponents) and a self-Elo climb that sit far above the population norm. This job builds those
* three features per player, standardises each against the whole player base, and flags the players whose combined
* deviation is extreme.
*
* Features per player (from each game's own/opponent Elo):
* - win_rate — fraction of decisive results won
* - upset_rate — wins vs higher-rated opponents / games vs higher-rated opponents
* - elo_climb — max self-Elo min self-Elo across their games (rapid rating gain)
*
* Standardisation uses a single unbounded window (`Window.partitionBy()`), i.e. mean/stddev over every qualifying
* player, so z = (x μ) / σ. The composite anomaly score sums the three z-scores. No UDFs — pure SQL aggregates +
* window functions, so Catalyst plans the whole job.
*
* Outputs (Parquet + CSV + JDBC):
* - `anomaly_scores` — every qualifying player with features, z-scores and composite, ranked most-anomalous first.
* - `flagged_smurfs` — the suspicious subset (high composite, or the classic high-winrate / few-games / steep-climb
* profile).
*
* Meaningful only when Elo is present (Lichess dump); requires `minGames` (arg 1, default 15) to avoid small-sample
* noise.
*/
object SmurfAnomalyJob:
def main(args: Array[String]): Unit =
val jdbcUrl = sys.env.getOrElse("NOWCHESS_JDBC_URL", "jdbc:postgresql://localhost:5432/nowchess")
val dbUser = sys.env.getOrElse("NOWCHESS_DB_USER", "nowchess")
val dbPass = sys.env.getOrElse("NOWCHESS_DB_PASS", "nowchess")
val outputDir = if args.length > 0 then args(0) else "/tmp/nowchess-smurf-anomaly"
val minGames = if args.length > 1 then args(1).toInt else 15
val spark = SparkSession
.builder()
.appName("NowChess Smurf Anomaly Detection")
.getOrCreate()
run(spark, jdbcUrl, dbUser, dbPass, outputDir, minGames)
spark.stop()
def run(
spark: SparkSession,
jdbcUrl: String,
dbUser: String,
dbPass: String,
outputDir: String,
minGames: Int,
): Unit =
val games = GameSource
.loadExtended(spark, jdbcUrl, dbUser, dbPass)
.select("white_id", "black_id", "result", "white_elo", "black_elo")
.filter(F.col("result").isNotNull)
val asWhite = games.select(
F.col("white_id").as("player_id"),
F.col("white_elo").as("self_elo"),
F.col("black_elo").as("opp_elo"),
F.when(F.col("result") === "white", 1).otherwise(0).as("won"),
)
val asBlack = games.select(
F.col("black_id").as("player_id"),
F.col("black_elo").as("self_elo"),
F.col("white_elo").as("opp_elo"),
F.when(F.col("result") === "black", 1).otherwise(0).as("won"),
)
val playerGames = asWhite
.union(asBlack)
.filter(F.col("self_elo").isNotNull.and(F.col("opp_elo").isNotNull))
val higher = F.col("opp_elo") > F.col("self_elo")
val features = playerGames
.groupBy("player_id")
.agg(
F.count("*").as("total_games"),
F.round(F.avg("won"), 3).as("win_rate"),
F.round(F.avg("self_elo"), 0).as("avg_self_elo"),
(F.max("self_elo") - F.min("self_elo")).as("elo_climb"),
F.sum(F.when(higher, 1).otherwise(0)).as("vs_higher"),
F.sum(F.when(higher && F.col("won") === 1, 1).otherwise(0)).as("upsets"),
)
.filter(F.col("total_games") >= minGames)
.withColumn("upset_rate", F.round(F.col("upsets") / F.greatest(F.col("vs_higher"), F.lit(1)), 3))
val all = Window.partitionBy()
def z(col: String): org.apache.spark.sql.Column =
val mean = F.avg(col).over(all)
val std = F.stddev(col).over(all)
F.round((F.col(col) - mean) / F.when(std === 0 || std.isNull, F.lit(1.0)).otherwise(std), 2)
val scored = features
.withColumn("z_win_rate", z("win_rate"))
.withColumn("z_upset_rate", z("upset_rate"))
.withColumn("z_elo_climb", z("elo_climb"))
.withColumn(
"anomaly_score",
F.round(F.col("z_win_rate") + F.col("z_upset_rate") + F.col("z_elo_climb"), 2),
)
.withColumn(
"flagged",
(F.col("anomaly_score") >= 4.0)
.or(F.col("win_rate") >= 0.8 && F.col("total_games") < 50 && F.col("elo_climb") >= 300),
)
val ordered = scored
.select(
"player_id",
"total_games",
"win_rate",
"avg_self_elo",
"elo_climb",
"upset_rate",
"z_win_rate",
"z_upset_rate",
"z_elo_climb",
"anomaly_score",
"flagged",
)
.orderBy(F.desc("anomaly_score"))
write(ordered, outputDir, "anomaly_scores", jdbcUrl, dbUser, dbPass, "analytics_smurf_anomaly")
val flagged = ordered.filter(F.col("flagged") === true)
write(flagged, outputDir, "flagged_smurfs", jdbcUrl, dbUser, dbPass, "analytics_flagged_smurfs")
private def write(
df: org.apache.spark.sql.DataFrame,
outputDir: String,
name: String,
jdbcUrl: String,
dbUser: String,
dbPass: String,
table: String,
): Unit =
df.write.mode("overwrite").parquet(s"$outputDir/$name")
df.write.mode("overwrite").option("header", "true").csv(s"$outputDir/${name}_csv")
if !GameSource.isPgnMode then
df.write
.mode("overwrite")
.format("jdbc")
.option("url", jdbcUrl)
.option("dbtable", table)
.option("user", dbUser)
.option("password", dbPass)
.option("driver", "org.postgresql.Driver")
.save()
+1 -1
View File
@@ -1,3 +1,3 @@
MAJOR=0 MAJOR=0
MINOR=7 MINOR=8
PATCH=0 PATCH=0
+643
View File
@@ -797,3 +797,646 @@
### Reverts ### Reverts
* Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656)) * Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
## (2026-06-23)
### Features
* add initialization metrics for various services ([d438e97](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d438e97f32bdde0bfc63c1b4a8cc810cdd093166))
* add OpenTelemetry trace configuration with parentbased sampler ([3904d5a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3904d5ad8ad4930ddee65287a7bfab785a6148f5))
* **analytics:** add Spark batch analytics module ([#70](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/70)) ([39f1657](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39f1657e1db6e84889af338c43be8cb5c03c3ec3))
* **config:** update application.yml for PostgreSQL and remove staging/production configurations ([2404e61](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/2404e6164c3b50ffccbea5238d636060d6abe4d6))
* **config:** update application.yml for staging and production environments ([6113432](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6113432a14c476a3a0dfc0d449e17d023697f2ba))
* configure logging and add OpenTelemetry support ([#49](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/49)) ([d57c488](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d57c4886612d1d92da0e1b79209fc83e6ef537a1))
* **docker:** add .dockerignore and .gitignore files for build exclusions ([c987d8e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c987d8e258c0e6c4cfbdaa8381c64c410d7a2b83))
* **docker:** add Dockerfiles for building Quarkus application in native and JVM modes ([3f2d2bb](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3f2d2bb4c97fa8cddba66e1da4427c54236dfeed))
* **docker:** add Dockerfiles for Quarkus application in JVM and native modes ([34b9933](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/34b993304670cf2aa62cd2f6460cee7b9864b08e))
* **events:** migrate game-creation and bot flows to Redis Streams NCS-89 ([#62](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/62)) ([a24924c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a24924c23057db3d700a75dbc4333557789cd991))
* NCS-78 Add Traceability to the Applications ([#46](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/46)) ([649566e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/649566eb3fcf38f91c8896a739f74ea318af312d))
* NCS-78 Add Traceability to the Applications ([#47](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/47)) ([87dfc6c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/87dfc6c2bcce7f7d58fc641bd8d468a2e584c108))
* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
* **official-bots:** consume GameOver stream for bot cleanup ([#67](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/67)) ([db9d153](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/db9d1533912f4b41c4d1ca80ccffdde5d23d6ff6))
* **official-bots:** make HybridBot veto actionable and use it for expert ([1df29cf](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1df29cf3a6e21af3f396b2b7a6da67d978f941ae))
* **official-bots:** park expert bot on tournament server at startup ([#75](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/75)) ([30295a4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/30295a4bb95855ee8261c92278bb9ebc80ee12ee))
* **official-bots:** resolve tournament bot token from Redis and account service ([386ddc5](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/386ddc5c19f8f893b16c6422aa5393b54c872e45))
* **tournament:** auto-join external tournaments and publish created ones ([#77](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/77)) ([9978b7e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9978b7ea78eb658a225a461b9cd339386c0c14f3))
* **tournament:** federate tournaments across clusters with DB replication ([5b000a6](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/5b000a6e5f04ea6770d1c7ab6bfdaded77a99172))
* **tournament:** seed external server registry from env var on startup ([845dc9c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/845dc9c2935c8bc1be42541dfaf31c9a861d3272))
* true-microservices ([#40](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/40)) ([5909242](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/590924254e8a2754de661a57a03e43f89ceb6299))
### Bug Fixes
* enable official bots to connect to external tournament server ([#71](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/71)) ([688d30e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/688d30e2b10026923372be5fca3c63eaaee2de2a))
* **official-bots:** configure JWT verification ([#72](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/72)) ([98c64fc](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/98c64fc0d56dc542beb31c75f4b9056d91de03cd))
* **official-bots:** correct parkOn path from /api/bots to /api/account/bots ([1be9949](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1be9949c0b5c6a1db535696620d77735050d6c93))
* **official-bots:** discover tournament games by polling, not just the stream ([10113fd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/10113fd0579b614d15870798d933bc9c495d2049))
* **official-bots:** make botToken optional, fall back to env, fix 502 status ([f43d193](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f43d1930d80670d810c57b54eaa3789854fa082c))
* **official-bots:** NCS-70-auto-register official bots with account service ([#59](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/59)) ([7117a93](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/7117a93376272094d0b1a6abf2121254ce396684))
* **official-bots:** park on external tournament servers using correct endpoint and token ([3188241](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/31882417377468b41bbe3ff94506aa4928024450))
* **official-bots:** play games by polling state instead of NDJSON stream ([bfb15c7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfb15c7299bd471d5e064a577ed10af98e2ea90a))
* **official-bots:** play only own tournament games with correct color ([4651bb7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/4651bb796f07a21bd013d9521b2dfe2e1078cebb))
* **official-bots:** prioritize Redis token over stale env var in joinTournament ([83dd2d4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/83dd2d4335ca48eb3e5aa234a75367574276ba63))
* **official-bots:** register with tournament server directly to get correct token ([64b5d55](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/64b5d5567f110c2fe152558c7de275a1e0b30e21))
* **official-bots:** resolve per-difficulty bot token on tournament join ([fdf4c94](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fdf4c94811d086996447bb4657fac1d9bd6e5a93))
* **official-bots:** resume tournaments already joined after restart ([285b73e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/285b73efbd6dd98cec410ade9eead9881d693a8f))
* **official-bots:** sync bots before token fetch on first startup after DB wipe ([b0ddb27](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b0ddb274d23bca8b1b3f691ce0d643f33e0b54cd))
### Reverts
* Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
## (2026-06-23)
### Features
* add initialization metrics for various services ([d438e97](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d438e97f32bdde0bfc63c1b4a8cc810cdd093166))
* add OpenTelemetry trace configuration with parentbased sampler ([3904d5a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3904d5ad8ad4930ddee65287a7bfab785a6148f5))
* **analytics:** add Spark batch analytics module ([#70](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/70)) ([39f1657](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39f1657e1db6e84889af338c43be8cb5c03c3ec3))
* **config:** update application.yml for PostgreSQL and remove staging/production configurations ([2404e61](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/2404e6164c3b50ffccbea5238d636060d6abe4d6))
* **config:** update application.yml for staging and production environments ([6113432](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6113432a14c476a3a0dfc0d449e17d023697f2ba))
* configure logging and add OpenTelemetry support ([#49](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/49)) ([d57c488](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d57c4886612d1d92da0e1b79209fc83e6ef537a1))
* **docker:** add .dockerignore and .gitignore files for build exclusions ([c987d8e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c987d8e258c0e6c4cfbdaa8381c64c410d7a2b83))
* **docker:** add Dockerfiles for building Quarkus application in native and JVM modes ([3f2d2bb](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3f2d2bb4c97fa8cddba66e1da4427c54236dfeed))
* **docker:** add Dockerfiles for Quarkus application in JVM and native modes ([34b9933](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/34b993304670cf2aa62cd2f6460cee7b9864b08e))
* **events:** migrate game-creation and bot flows to Redis Streams NCS-89 ([#62](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/62)) ([a24924c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a24924c23057db3d700a75dbc4333557789cd991))
* NCS-78 Add Traceability to the Applications ([#46](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/46)) ([649566e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/649566eb3fcf38f91c8896a739f74ea318af312d))
* NCS-78 Add Traceability to the Applications ([#47](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/47)) ([87dfc6c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/87dfc6c2bcce7f7d58fc641bd8d468a2e584c108))
* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
* **official-bots:** consume GameOver stream for bot cleanup ([#67](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/67)) ([db9d153](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/db9d1533912f4b41c4d1ca80ccffdde5d23d6ff6))
* **official-bots:** make HybridBot veto actionable and use it for expert ([1df29cf](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1df29cf3a6e21af3f396b2b7a6da67d978f941ae))
* **official-bots:** park expert bot on tournament server at startup ([#75](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/75)) ([30295a4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/30295a4bb95855ee8261c92278bb9ebc80ee12ee))
* **official-bots:** resolve tournament bot token from Redis and account service ([386ddc5](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/386ddc5c19f8f893b16c6422aa5393b54c872e45))
* **tournament:** auto-join external tournaments and publish created ones ([#77](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/77)) ([9978b7e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9978b7ea78eb658a225a461b9cd339386c0c14f3))
* **tournament:** federate tournaments across clusters with DB replication ([5b000a6](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/5b000a6e5f04ea6770d1c7ab6bfdaded77a99172))
* **tournament:** seed external server registry from env var on startup ([845dc9c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/845dc9c2935c8bc1be42541dfaf31c9a861d3272))
* true-microservices ([#40](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/40)) ([5909242](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/590924254e8a2754de661a57a03e43f89ceb6299))
### Bug Fixes
* enable official bots to connect to external tournament server ([#71](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/71)) ([688d30e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/688d30e2b10026923372be5fca3c63eaaee2de2a))
* **official-bots:** configure JWT verification ([#72](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/72)) ([98c64fc](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/98c64fc0d56dc542beb31c75f4b9056d91de03cd))
* **official-bots:** correct parkOn path from /api/bots to /api/account/bots ([1be9949](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1be9949c0b5c6a1db535696620d77735050d6c93))
* **official-bots:** derive tournament game color from game endpoint ([#79](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/79)) ([bfc4672](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfc46723e615bb9b65f7f9bba5f53877c4f079a7))
* **official-bots:** discover tournament games by polling, not just the stream ([10113fd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/10113fd0579b614d15870798d933bc9c495d2049))
* **official-bots:** make botToken optional, fall back to env, fix 502 status ([f43d193](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f43d1930d80670d810c57b54eaa3789854fa082c))
* **official-bots:** NCS-70-auto-register official bots with account service ([#59](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/59)) ([7117a93](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/7117a93376272094d0b1a6abf2121254ce396684))
* **official-bots:** park on external tournament servers using correct endpoint and token ([3188241](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/31882417377468b41bbe3ff94506aa4928024450))
* **official-bots:** play games by polling state instead of NDJSON stream ([bfb15c7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfb15c7299bd471d5e064a577ed10af98e2ea90a))
* **official-bots:** play only own tournament games with correct color ([4651bb7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/4651bb796f07a21bd013d9521b2dfe2e1078cebb))
* **official-bots:** prioritize Redis token over stale env var in joinTournament ([83dd2d4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/83dd2d4335ca48eb3e5aa234a75367574276ba63))
* **official-bots:** register with tournament server directly to get correct token ([64b5d55](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/64b5d5567f110c2fe152558c7de275a1e0b30e21))
* **official-bots:** resolve per-difficulty bot token on tournament join ([fdf4c94](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fdf4c94811d086996447bb4657fac1d9bd6e5a93))
* **official-bots:** resume tournaments already joined after restart ([285b73e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/285b73efbd6dd98cec410ade9eead9881d693a8f))
* **official-bots:** sync bots before token fetch on first startup after DB wipe ([b0ddb27](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b0ddb274d23bca8b1b3f691ce0d643f33e0b54cd))
### Reverts
* Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
## (2026-06-23)
### Features
* add initialization metrics for various services ([d438e97](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d438e97f32bdde0bfc63c1b4a8cc810cdd093166))
* add OpenTelemetry trace configuration with parentbased sampler ([3904d5a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3904d5ad8ad4930ddee65287a7bfab785a6148f5))
* **analytics:** add Spark batch analytics module ([#70](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/70)) ([39f1657](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39f1657e1db6e84889af338c43be8cb5c03c3ec3))
* **config:** update application.yml for PostgreSQL and remove staging/production configurations ([2404e61](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/2404e6164c3b50ffccbea5238d636060d6abe4d6))
* **config:** update application.yml for staging and production environments ([6113432](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6113432a14c476a3a0dfc0d449e17d023697f2ba))
* configure logging and add OpenTelemetry support ([#49](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/49)) ([d57c488](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d57c4886612d1d92da0e1b79209fc83e6ef537a1))
* **docker:** add .dockerignore and .gitignore files for build exclusions ([c987d8e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c987d8e258c0e6c4cfbdaa8381c64c410d7a2b83))
* **docker:** add Dockerfiles for building Quarkus application in native and JVM modes ([3f2d2bb](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3f2d2bb4c97fa8cddba66e1da4427c54236dfeed))
* **docker:** add Dockerfiles for Quarkus application in JVM and native modes ([34b9933](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/34b993304670cf2aa62cd2f6460cee7b9864b08e))
* **events:** migrate game-creation and bot flows to Redis Streams NCS-89 ([#62](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/62)) ([a24924c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a24924c23057db3d700a75dbc4333557789cd991))
* NCS-78 Add Traceability to the Applications ([#46](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/46)) ([649566e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/649566eb3fcf38f91c8896a739f74ea318af312d))
* NCS-78 Add Traceability to the Applications ([#47](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/47)) ([87dfc6c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/87dfc6c2bcce7f7d58fc641bd8d468a2e584c108))
* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
* **official-bots:** activate opening book in expert bot (native-safe) ([260db25](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/260db25803ec55ce99e55782791eabdc190dfed4))
* **official-bots:** consume GameOver stream for bot cleanup ([#67](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/67)) ([db9d153](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/db9d1533912f4b41c4d1ca80ccffdde5d23d6ff6))
* **official-bots:** make HybridBot veto actionable and use it for expert ([1df29cf](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1df29cf3a6e21af3f396b2b7a6da67d978f941ae))
* **official-bots:** park expert bot on tournament server at startup ([#75](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/75)) ([30295a4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/30295a4bb95855ee8261c92278bb9ebc80ee12ee))
* **official-bots:** resolve tournament bot token from Redis and account service ([386ddc5](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/386ddc5c19f8f893b16c6422aa5393b54c872e45))
* **tournament:** auto-join external tournaments and publish created ones ([#77](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/77)) ([9978b7e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9978b7ea78eb658a225a461b9cd339386c0c14f3))
* **tournament:** federate tournaments across clusters with DB replication ([5b000a6](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/5b000a6e5f04ea6770d1c7ab6bfdaded77a99172))
* **tournament:** seed external server registry from env var on startup ([845dc9c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/845dc9c2935c8bc1be42541dfaf31c9a861d3272))
* true-microservices ([#40](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/40)) ([5909242](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/590924254e8a2754de661a57a03e43f89ceb6299))
### Bug Fixes
* enable official bots to connect to external tournament server ([#71](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/71)) ([688d30e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/688d30e2b10026923372be5fca3c63eaaee2de2a))
* **official-bots:** configure JWT verification ([#72](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/72)) ([98c64fc](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/98c64fc0d56dc542beb31c75f4b9056d91de03cd))
* **official-bots:** correct parkOn path from /api/bots to /api/account/bots ([1be9949](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1be9949c0b5c6a1db535696620d77735050d6c93))
* **official-bots:** derive tournament game color from game endpoint ([#79](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/79)) ([bfc4672](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfc46723e615bb9b65f7f9bba5f53877c4f079a7))
* **official-bots:** discover tournament games by polling, not just the stream ([10113fd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/10113fd0579b614d15870798d933bc9c495d2049))
* **official-bots:** make botToken optional, fall back to env, fix 502 status ([f43d193](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f43d1930d80670d810c57b54eaa3789854fa082c))
* **official-bots:** NCS-70-auto-register official bots with account service ([#59](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/59)) ([7117a93](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/7117a93376272094d0b1a6abf2121254ce396684))
* **official-bots:** park on external tournament servers using correct endpoint and token ([3188241](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/31882417377468b41bbe3ff94506aa4928024450))
* **official-bots:** play games by polling state instead of NDJSON stream ([bfb15c7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfb15c7299bd471d5e064a577ed10af98e2ea90a))
* **official-bots:** play only own tournament games with correct color ([4651bb7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/4651bb796f07a21bd013d9521b2dfe2e1078cebb))
* **official-bots:** prioritize Redis token over stale env var in joinTournament ([83dd2d4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/83dd2d4335ca48eb3e5aa234a75367574276ba63))
* **official-bots:** register with tournament server directly to get correct token ([64b5d55](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/64b5d5567f110c2fe152558c7de275a1e0b30e21))
* **official-bots:** resolve per-difficulty bot token on tournament join ([fdf4c94](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fdf4c94811d086996447bb4657fac1d9bd6e5a93))
* **official-bots:** resume tournaments already joined after restart ([285b73e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/285b73efbd6dd98cec410ade9eead9881d693a8f))
* **official-bots:** sync bots before token fetch on first startup after DB wipe ([b0ddb27](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b0ddb274d23bca8b1b3f691ce0d643f33e0b54cd))
### Reverts
* Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
## (2026-06-23)
### Features
* add initialization metrics for various services ([d438e97](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d438e97f32bdde0bfc63c1b4a8cc810cdd093166))
* add OpenTelemetry trace configuration with parentbased sampler ([3904d5a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3904d5ad8ad4930ddee65287a7bfab785a6148f5))
* **analytics:** add Spark batch analytics module ([#70](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/70)) ([39f1657](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39f1657e1db6e84889af338c43be8cb5c03c3ec3))
* **config:** update application.yml for PostgreSQL and remove staging/production configurations ([2404e61](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/2404e6164c3b50ffccbea5238d636060d6abe4d6))
* **config:** update application.yml for staging and production environments ([6113432](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6113432a14c476a3a0dfc0d449e17d023697f2ba))
* configure logging and add OpenTelemetry support ([#49](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/49)) ([d57c488](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d57c4886612d1d92da0e1b79209fc83e6ef537a1))
* **docker:** add .dockerignore and .gitignore files for build exclusions ([c987d8e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c987d8e258c0e6c4cfbdaa8381c64c410d7a2b83))
* **docker:** add Dockerfiles for building Quarkus application in native and JVM modes ([3f2d2bb](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3f2d2bb4c97fa8cddba66e1da4427c54236dfeed))
* **docker:** add Dockerfiles for Quarkus application in JVM and native modes ([34b9933](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/34b993304670cf2aa62cd2f6460cee7b9864b08e))
* **events:** migrate game-creation and bot flows to Redis Streams NCS-89 ([#62](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/62)) ([a24924c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a24924c23057db3d700a75dbc4333557789cd991))
* NCS-78 Add Traceability to the Applications ([#46](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/46)) ([649566e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/649566eb3fcf38f91c8896a739f74ea318af312d))
* NCS-78 Add Traceability to the Applications ([#47](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/47)) ([87dfc6c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/87dfc6c2bcce7f7d58fc641bd8d468a2e584c108))
* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
* **official-bots:** activate opening book in expert bot (native-safe) ([260db25](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/260db25803ec55ce99e55782791eabdc190dfed4))
* **official-bots:** consume GameOver stream for bot cleanup ([#67](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/67)) ([db9d153](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/db9d1533912f4b41c4d1ca80ccffdde5d23d6ff6))
* **official-bots:** make HybridBot veto actionable and use it for expert ([1df29cf](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1df29cf3a6e21af3f396b2b7a6da67d978f941ae))
* **official-bots:** park expert bot on tournament server at startup ([#75](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/75)) ([30295a4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/30295a4bb95855ee8261c92278bb9ebc80ee12ee))
* **official-bots:** resolve tournament bot token from Redis and account service ([386ddc5](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/386ddc5c19f8f893b16c6422aa5393b54c872e45))
* **tournament:** auto-join external tournaments and publish created ones ([#77](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/77)) ([9978b7e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9978b7ea78eb658a225a461b9cd339386c0c14f3))
* **tournament:** federate tournaments across clusters with DB replication ([5b000a6](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/5b000a6e5f04ea6770d1c7ab6bfdaded77a99172))
* **tournament:** seed external server registry from env var on startup ([845dc9c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/845dc9c2935c8bc1be42541dfaf31c9a861d3272))
* true-microservices ([#40](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/40)) ([5909242](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/590924254e8a2754de661a57a03e43f89ceb6299))
### Bug Fixes
* enable official bots to connect to external tournament server ([#71](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/71)) ([688d30e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/688d30e2b10026923372be5fca3c63eaaee2de2a))
* **official-bots:** configure JWT verification ([#72](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/72)) ([98c64fc](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/98c64fc0d56dc542beb31c75f4b9056d91de03cd))
* **official-bots:** correct parkOn path from /api/bots to /api/account/bots ([1be9949](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1be9949c0b5c6a1db535696620d77735050d6c93))
* **official-bots:** derive tournament game color from game endpoint ([#79](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/79)) ([bfc4672](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfc46723e615bb9b65f7f9bba5f53877c4f079a7))
* **official-bots:** discover tournament games by polling, not just the stream ([10113fd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/10113fd0579b614d15870798d933bc9c495d2049))
* **official-bots:** make botToken optional, fall back to env, fix 502 status ([f43d193](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f43d1930d80670d810c57b54eaa3789854fa082c))
* **official-bots:** NCS-70-auto-register official bots with account service ([#59](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/59)) ([7117a93](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/7117a93376272094d0b1a6abf2121254ce396684))
* **official-bots:** park on external tournament servers using correct endpoint and token ([3188241](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/31882417377468b41bbe3ff94506aa4928024450))
* **official-bots:** play games by polling state instead of NDJSON stream ([bfb15c7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfb15c7299bd471d5e064a577ed10af98e2ea90a))
* **official-bots:** play only own tournament games with correct color ([4651bb7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/4651bb796f07a21bd013d9521b2dfe2e1078cebb))
* **official-bots:** prioritize Redis token over stale env var in joinTournament ([83dd2d4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/83dd2d4335ca48eb3e5aa234a75367574276ba63))
* **official-bots:** register with tournament server directly to get correct token ([64b5d55](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/64b5d5567f110c2fe152558c7de275a1e0b30e21))
* **official-bots:** resolve per-difficulty bot token on tournament join ([fdf4c94](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fdf4c94811d086996447bb4657fac1d9bd6e5a93))
* **official-bots:** resume tournaments already joined after restart ([285b73e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/285b73efbd6dd98cec410ade9eead9881d693a8f))
* **official-bots:** sync bots before token fetch on first startup after DB wipe ([b0ddb27](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b0ddb274d23bca8b1b3f691ce0d643f33e0b54cd))
* **official-bots:** use ThreadLocalRandom in PolyglotBook for native image ([1b30c3b](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1b30c3be393d25712c8743d3d9057207f8bbb67c))
### Reverts
* Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
## (2026-06-24)
### Features
* add initialization metrics for various services ([d438e97](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d438e97f32bdde0bfc63c1b4a8cc810cdd093166))
* add OpenTelemetry trace configuration with parentbased sampler ([3904d5a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3904d5ad8ad4930ddee65287a7bfab785a6148f5))
* **analytics:** add Spark batch analytics module ([#70](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/70)) ([39f1657](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39f1657e1db6e84889af338c43be8cb5c03c3ec3))
* **config:** update application.yml for PostgreSQL and remove staging/production configurations ([2404e61](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/2404e6164c3b50ffccbea5238d636060d6abe4d6))
* **config:** update application.yml for staging and production environments ([6113432](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6113432a14c476a3a0dfc0d449e17d023697f2ba))
* configure logging and add OpenTelemetry support ([#49](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/49)) ([d57c488](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d57c4886612d1d92da0e1b79209fc83e6ef537a1))
* **docker:** add .dockerignore and .gitignore files for build exclusions ([c987d8e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c987d8e258c0e6c4cfbdaa8381c64c410d7a2b83))
* **docker:** add Dockerfiles for building Quarkus application in native and JVM modes ([3f2d2bb](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3f2d2bb4c97fa8cddba66e1da4427c54236dfeed))
* **docker:** add Dockerfiles for Quarkus application in JVM and native modes ([34b9933](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/34b993304670cf2aa62cd2f6460cee7b9864b08e))
* **events:** migrate game-creation and bot flows to Redis Streams NCS-89 ([#62](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/62)) ([a24924c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a24924c23057db3d700a75dbc4333557789cd991))
* NCS-78 Add Traceability to the Applications ([#46](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/46)) ([649566e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/649566eb3fcf38f91c8896a739f74ea318af312d))
* NCS-78 Add Traceability to the Applications ([#47](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/47)) ([87dfc6c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/87dfc6c2bcce7f7d58fc641bd8d468a2e584c108))
* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
* **official-bots:** activate opening book in expert bot (native-safe) ([260db25](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/260db25803ec55ce99e55782791eabdc190dfed4))
* **official-bots:** add Google Colab notebook for NNUE training (NCS-111) ([#81](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/81)) ([fa10852](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fa10852bc98451d4068ec6fb9e7a486b5e53ef5c))
* **official-bots:** consume GameOver stream for bot cleanup ([#67](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/67)) ([db9d153](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/db9d1533912f4b41c4d1ca80ccffdde5d23d6ff6))
* **official-bots:** implement king-relative (HalfKP) encoding in NNUE (NCS-109) ([#80](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/80)) ([44f376f](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/44f376f03221f086b898741436e13c93fd314dd1))
* **official-bots:** make HybridBot veto actionable and use it for expert ([1df29cf](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1df29cf3a6e21af3f396b2b7a6da67d978f941ae))
* **official-bots:** park expert bot on tournament server at startup ([#75](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/75)) ([30295a4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/30295a4bb95855ee8261c92278bb9ebc80ee12ee))
* **official-bots:** resolve tournament bot token from Redis and account service ([386ddc5](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/386ddc5c19f8f893b16c6422aa5393b54c872e45))
* **tournament:** auto-join external tournaments and publish created ones ([#77](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/77)) ([9978b7e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9978b7ea78eb658a225a461b9cd339386c0c14f3))
* **tournament:** federate tournaments across clusters with DB replication ([5b000a6](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/5b000a6e5f04ea6770d1c7ab6bfdaded77a99172))
* **tournament:** seed external server registry from env var on startup ([845dc9c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/845dc9c2935c8bc1be42541dfaf31c9a861d3272))
* true-microservices ([#40](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/40)) ([5909242](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/590924254e8a2754de661a57a03e43f89ceb6299))
### Bug Fixes
* enable official bots to connect to external tournament server ([#71](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/71)) ([688d30e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/688d30e2b10026923372be5fca3c63eaaee2de2a))
* modified training pipeline ([9f9140c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9f9140cb585345cd244a1dfee1a06e51a5f7f7a8))
* **official-bots:** configure JWT verification ([#72](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/72)) ([98c64fc](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/98c64fc0d56dc542beb31c75f4b9056d91de03cd))
* **official-bots:** correct parkOn path from /api/bots to /api/account/bots ([1be9949](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1be9949c0b5c6a1db535696620d77735050d6c93))
* **official-bots:** derive tournament game color from game endpoint ([#79](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/79)) ([bfc4672](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfc46723e615bb9b65f7f9bba5f53877c4f079a7))
* **official-bots:** discover tournament games by polling, not just the stream ([10113fd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/10113fd0579b614d15870798d933bc9c495d2049))
* **official-bots:** make botToken optional, fall back to env, fix 502 status ([f43d193](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f43d1930d80670d810c57b54eaa3789854fa082c))
* **official-bots:** NCS-70-auto-register official bots with account service ([#59](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/59)) ([7117a93](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/7117a93376272094d0b1a6abf2121254ce396684))
* **official-bots:** park on external tournament servers using correct endpoint and token ([3188241](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/31882417377468b41bbe3ff94506aa4928024450))
* **official-bots:** play games by polling state instead of NDJSON stream ([bfb15c7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfb15c7299bd471d5e064a577ed10af98e2ea90a))
* **official-bots:** play only own tournament games with correct color ([4651bb7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/4651bb796f07a21bd013d9521b2dfe2e1078cebb))
* **official-bots:** prioritize Redis token over stale env var in joinTournament ([83dd2d4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/83dd2d4335ca48eb3e5aa234a75367574276ba63))
* **official-bots:** register with tournament server directly to get correct token ([64b5d55](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/64b5d5567f110c2fe152558c7de275a1e0b30e21))
* **official-bots:** resolve per-difficulty bot token on tournament join ([fdf4c94](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fdf4c94811d086996447bb4657fac1d9bd6e5a93))
* **official-bots:** resume tournaments already joined after restart ([285b73e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/285b73efbd6dd98cec410ade9eead9881d693a8f))
* **official-bots:** sync bots before token fetch on first startup after DB wipe ([b0ddb27](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b0ddb274d23bca8b1b3f691ce0d643f33e0b54cd))
* **official-bots:** use ThreadLocalRandom in PolyglotBook for native image ([1b30c3b](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1b30c3be393d25712c8743d3d9057207f8bbb67c))
### Reverts
* Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
## (2026-06-24)
### Features
* add initialization metrics for various services ([d438e97](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d438e97f32bdde0bfc63c1b4a8cc810cdd093166))
* add OpenTelemetry trace configuration with parentbased sampler ([3904d5a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3904d5ad8ad4930ddee65287a7bfab785a6148f5))
* **analytics:** add Spark batch analytics module ([#70](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/70)) ([39f1657](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39f1657e1db6e84889af338c43be8cb5c03c3ec3))
* **config:** update application.yml for PostgreSQL and remove staging/production configurations ([2404e61](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/2404e6164c3b50ffccbea5238d636060d6abe4d6))
* **config:** update application.yml for staging and production environments ([6113432](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6113432a14c476a3a0dfc0d449e17d023697f2ba))
* configure logging and add OpenTelemetry support ([#49](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/49)) ([d57c488](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d57c4886612d1d92da0e1b79209fc83e6ef537a1))
* **docker:** add .dockerignore and .gitignore files for build exclusions ([c987d8e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c987d8e258c0e6c4cfbdaa8381c64c410d7a2b83))
* **docker:** add Dockerfiles for building Quarkus application in native and JVM modes ([3f2d2bb](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3f2d2bb4c97fa8cddba66e1da4427c54236dfeed))
* **docker:** add Dockerfiles for Quarkus application in JVM and native modes ([34b9933](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/34b993304670cf2aa62cd2f6460cee7b9864b08e))
* **events:** migrate game-creation and bot flows to Redis Streams NCS-89 ([#62](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/62)) ([a24924c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a24924c23057db3d700a75dbc4333557789cd991))
* **ncs-110:** feed NNUE root-move scores into search move ordering ([#83](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/83)) ([e4fee85](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/e4fee8513430093d46957970618935e99591519f))
* NCS-78 Add Traceability to the Applications ([#46](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/46)) ([649566e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/649566eb3fcf38f91c8896a739f74ea318af312d))
* NCS-78 Add Traceability to the Applications ([#47](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/47)) ([87dfc6c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/87dfc6c2bcce7f7d58fc641bd8d468a2e584c108))
* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
* **official-bots:** activate opening book in expert bot (native-safe) ([260db25](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/260db25803ec55ce99e55782791eabdc190dfed4))
* **official-bots:** add Google Colab notebook for NNUE training (NCS-111) ([#81](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/81)) ([fa10852](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fa10852bc98451d4068ec6fb9e7a486b5e53ef5c))
* **official-bots:** consume GameOver stream for bot cleanup ([#67](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/67)) ([db9d153](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/db9d1533912f4b41c4d1ca80ccffdde5d23d6ff6))
* **official-bots:** implement king-relative (HalfKP) encoding in NNUE (NCS-109) ([#80](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/80)) ([44f376f](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/44f376f03221f086b898741436e13c93fd314dd1))
* **official-bots:** make HybridBot veto actionable and use it for expert ([1df29cf](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1df29cf3a6e21af3f396b2b7a6da67d978f941ae))
* **official-bots:** park expert bot on tournament server at startup ([#75](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/75)) ([30295a4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/30295a4bb95855ee8261c92278bb9ebc80ee12ee))
* **official-bots:** resolve tournament bot token from Redis and account service ([386ddc5](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/386ddc5c19f8f893b16c6422aa5393b54c872e45))
* **tournament:** auto-join external tournaments and publish created ones ([#77](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/77)) ([9978b7e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9978b7ea78eb658a225a461b9cd339386c0c14f3))
* **tournament:** federate tournaments across clusters with DB replication ([5b000a6](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/5b000a6e5f04ea6770d1c7ab6bfdaded77a99172))
* **tournament:** seed external server registry from env var on startup ([845dc9c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/845dc9c2935c8bc1be42541dfaf31c9a861d3272))
* true-microservices ([#40](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/40)) ([5909242](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/590924254e8a2754de661a57a03e43f89ceb6299))
### Bug Fixes
* enable official bots to connect to external tournament server ([#71](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/71)) ([688d30e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/688d30e2b10026923372be5fca3c63eaaee2de2a))
* modified training pipeline ([9f9140c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9f9140cb585345cd244a1dfee1a06e51a5f7f7a8))
* **official-bots:** configure JWT verification ([#72](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/72)) ([98c64fc](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/98c64fc0d56dc542beb31c75f4b9056d91de03cd))
* **official-bots:** correct parkOn path from /api/bots to /api/account/bots ([1be9949](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1be9949c0b5c6a1db535696620d77735050d6c93))
* **official-bots:** derive tournament game color from game endpoint ([#79](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/79)) ([bfc4672](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfc46723e615bb9b65f7f9bba5f53877c4f079a7))
* **official-bots:** discover tournament games by polling, not just the stream ([10113fd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/10113fd0579b614d15870798d933bc9c495d2049))
* **official-bots:** make botToken optional, fall back to env, fix 502 status ([f43d193](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f43d1930d80670d810c57b54eaa3789854fa082c))
* **official-bots:** NCS-70-auto-register official bots with account service ([#59](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/59)) ([7117a93](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/7117a93376272094d0b1a6abf2121254ce396684))
* **official-bots:** park on external tournament servers using correct endpoint and token ([3188241](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/31882417377468b41bbe3ff94506aa4928024450))
* **official-bots:** play games by polling state instead of NDJSON stream ([bfb15c7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfb15c7299bd471d5e064a577ed10af98e2ea90a))
* **official-bots:** play only own tournament games with correct color ([4651bb7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/4651bb796f07a21bd013d9521b2dfe2e1078cebb))
* **official-bots:** prioritize Redis token over stale env var in joinTournament ([83dd2d4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/83dd2d4335ca48eb3e5aa234a75367574276ba63))
* **official-bots:** register with tournament server directly to get correct token ([64b5d55](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/64b5d5567f110c2fe152558c7de275a1e0b30e21))
* **official-bots:** resolve per-difficulty bot token on tournament join ([fdf4c94](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fdf4c94811d086996447bb4657fac1d9bd6e5a93))
* **official-bots:** resume tournaments already joined after restart ([285b73e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/285b73efbd6dd98cec410ade9eead9881d693a8f))
* **official-bots:** sync bots before token fetch on first startup after DB wipe ([b0ddb27](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b0ddb274d23bca8b1b3f691ce0d643f33e0b54cd))
* **official-bots:** use ThreadLocalRandom in PolyglotBook for native image ([1b30c3b](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1b30c3be393d25712c8743d3d9057207f8bbb67c))
### Reverts
* Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
## (2026-06-24)
### Features
* add initialization metrics for various services ([d438e97](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d438e97f32bdde0bfc63c1b4a8cc810cdd093166))
* add OpenTelemetry trace configuration with parentbased sampler ([3904d5a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3904d5ad8ad4930ddee65287a7bfab785a6148f5))
* **analytics:** add Spark batch analytics module ([#70](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/70)) ([39f1657](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39f1657e1db6e84889af338c43be8cb5c03c3ec3))
* **config:** update application.yml for PostgreSQL and remove staging/production configurations ([2404e61](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/2404e6164c3b50ffccbea5238d636060d6abe4d6))
* **config:** update application.yml for staging and production environments ([6113432](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6113432a14c476a3a0dfc0d449e17d023697f2ba))
* configure logging and add OpenTelemetry support ([#49](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/49)) ([d57c488](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d57c4886612d1d92da0e1b79209fc83e6ef537a1))
* **docker:** add .dockerignore and .gitignore files for build exclusions ([c987d8e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c987d8e258c0e6c4cfbdaa8381c64c410d7a2b83))
* **docker:** add Dockerfiles for building Quarkus application in native and JVM modes ([3f2d2bb](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3f2d2bb4c97fa8cddba66e1da4427c54236dfeed))
* **docker:** add Dockerfiles for Quarkus application in JVM and native modes ([34b9933](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/34b993304670cf2aa62cd2f6460cee7b9864b08e))
* **events:** migrate game-creation and bot flows to Redis Streams NCS-89 ([#62](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/62)) ([a24924c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a24924c23057db3d700a75dbc4333557789cd991))
* **ncs-110:** feed NNUE root-move scores into search move ordering ([#83](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/83)) ([e4fee85](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/e4fee8513430093d46957970618935e99591519f))
* NCS-78 Add Traceability to the Applications ([#46](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/46)) ([649566e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/649566eb3fcf38f91c8896a739f74ea318af312d))
* NCS-78 Add Traceability to the Applications ([#47](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/47)) ([87dfc6c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/87dfc6c2bcce7f7d58fc641bd8d468a2e584c108))
* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
* **official-bots:** activate opening book in expert bot (native-safe) ([260db25](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/260db25803ec55ce99e55782791eabdc190dfed4))
* **official-bots:** add Google Colab notebook for NNUE training (NCS-111) ([#81](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/81)) ([fa10852](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fa10852bc98451d4068ec6fb9e7a486b5e53ef5c))
* **official-bots:** consume GameOver stream for bot cleanup ([#67](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/67)) ([db9d153](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/db9d1533912f4b41c4d1ca80ccffdde5d23d6ff6))
* **official-bots:** implement king-relative (HalfKP) encoding in NNUE (NCS-109) ([#80](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/80)) ([44f376f](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/44f376f03221f086b898741436e13c93fd314dd1))
* **official-bots:** make HybridBot veto actionable and use it for expert ([1df29cf](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1df29cf3a6e21af3f396b2b7a6da67d978f941ae))
* **official-bots:** park expert bot on tournament server at startup ([#75](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/75)) ([30295a4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/30295a4bb95855ee8261c92278bb9ebc80ee12ee))
* **official-bots:** resolve tournament bot token from Redis and account service ([386ddc5](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/386ddc5c19f8f893b16c6422aa5393b54c872e45))
* **official-bots:** standalone self-play + one-shot dataset builder for NNUE training ([1c80abd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1c80abdb8a45814d642d43c633cde81ce7374c4f))
* **tournament:** auto-join external tournaments and publish created ones ([#77](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/77)) ([9978b7e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9978b7ea78eb658a225a461b9cd339386c0c14f3))
* **tournament:** federate tournaments across clusters with DB replication ([5b000a6](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/5b000a6e5f04ea6770d1c7ab6bfdaded77a99172))
* **tournament:** seed external server registry from env var on startup ([845dc9c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/845dc9c2935c8bc1be42541dfaf31c9a861d3272))
* true-microservices ([#40](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/40)) ([5909242](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/590924254e8a2754de661a57a03e43f89ceb6299))
### Bug Fixes
* enable official bots to connect to external tournament server ([#71](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/71)) ([688d30e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/688d30e2b10026923372be5fca3c63eaaee2de2a))
* modified training pipeline ([9f9140c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9f9140cb585345cd244a1dfee1a06e51a5f7f7a8))
* **official-bots:** configure JWT verification ([#72](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/72)) ([98c64fc](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/98c64fc0d56dc542beb31c75f4b9056d91de03cd))
* **official-bots:** correct parkOn path from /api/bots to /api/account/bots ([1be9949](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1be9949c0b5c6a1db535696620d77735050d6c93))
* **official-bots:** derive tournament game color from game endpoint ([#79](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/79)) ([bfc4672](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfc46723e615bb9b65f7f9bba5f53877c4f079a7))
* **official-bots:** discover tournament games by polling, not just the stream ([10113fd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/10113fd0579b614d15870798d933bc9c495d2049))
* **official-bots:** make botToken optional, fall back to env, fix 502 status ([f43d193](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f43d1930d80670d810c57b54eaa3789854fa082c))
* **official-bots:** NCS-70-auto-register official bots with account service ([#59](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/59)) ([7117a93](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/7117a93376272094d0b1a6abf2121254ce396684))
* **official-bots:** park on external tournament servers using correct endpoint and token ([3188241](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/31882417377468b41bbe3ff94506aa4928024450))
* **official-bots:** play games by polling state instead of NDJSON stream ([bfb15c7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfb15c7299bd471d5e064a577ed10af98e2ea90a))
* **official-bots:** play only own tournament games with correct color ([4651bb7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/4651bb796f07a21bd013d9521b2dfe2e1078cebb))
* **official-bots:** prioritize Redis token over stale env var in joinTournament ([83dd2d4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/83dd2d4335ca48eb3e5aa234a75367574276ba63))
* **official-bots:** register with tournament server directly to get correct token ([64b5d55](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/64b5d5567f110c2fe152558c7de275a1e0b30e21))
* **official-bots:** resolve per-difficulty bot token on tournament join ([fdf4c94](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fdf4c94811d086996447bb4657fac1d9bd6e5a93))
* **official-bots:** resume tournaments already joined after restart ([285b73e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/285b73efbd6dd98cec410ade9eead9881d693a8f))
* **official-bots:** sync bots before token fetch on first startup after DB wipe ([b0ddb27](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b0ddb274d23bca8b1b3f691ce0d643f33e0b54cd))
* **official-bots:** use ThreadLocalRandom in PolyglotBook for native image ([1b30c3b](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1b30c3be393d25712c8743d3d9057207f8bbb67c))
### Reverts
* Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
## (2026-06-24)
### Features
* add initialization metrics for various services ([d438e97](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d438e97f32bdde0bfc63c1b4a8cc810cdd093166))
* add OpenTelemetry trace configuration with parentbased sampler ([3904d5a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3904d5ad8ad4930ddee65287a7bfab785a6148f5))
* **analytics:** add Spark batch analytics module ([#70](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/70)) ([39f1657](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39f1657e1db6e84889af338c43be8cb5c03c3ec3))
* **config:** update application.yml for PostgreSQL and remove staging/production configurations ([2404e61](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/2404e6164c3b50ffccbea5238d636060d6abe4d6))
* **config:** update application.yml for staging and production environments ([6113432](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6113432a14c476a3a0dfc0d449e17d023697f2ba))
* configure logging and add OpenTelemetry support ([#49](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/49)) ([d57c488](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d57c4886612d1d92da0e1b79209fc83e6ef537a1))
* **docker:** add .dockerignore and .gitignore files for build exclusions ([c987d8e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c987d8e258c0e6c4cfbdaa8381c64c410d7a2b83))
* **docker:** add Dockerfiles for building Quarkus application in native and JVM modes ([3f2d2bb](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3f2d2bb4c97fa8cddba66e1da4427c54236dfeed))
* **docker:** add Dockerfiles for Quarkus application in JVM and native modes ([34b9933](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/34b993304670cf2aa62cd2f6460cee7b9864b08e))
* **events:** migrate game-creation and bot flows to Redis Streams NCS-89 ([#62](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/62)) ([a24924c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a24924c23057db3d700a75dbc4333557789cd991))
* **ncs-110:** feed NNUE root-move scores into search move ordering ([#83](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/83)) ([e4fee85](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/e4fee8513430093d46957970618935e99591519f))
* NCS-78 Add Traceability to the Applications ([#46](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/46)) ([649566e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/649566eb3fcf38f91c8896a739f74ea318af312d))
* NCS-78 Add Traceability to the Applications ([#47](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/47)) ([87dfc6c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/87dfc6c2bcce7f7d58fc641bd8d468a2e584c108))
* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
* **official-bots:** activate opening book in expert bot (native-safe) ([260db25](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/260db25803ec55ce99e55782791eabdc190dfed4))
* **official-bots:** add Google Colab notebook for NNUE training (NCS-111) ([#81](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/81)) ([fa10852](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fa10852bc98451d4068ec6fb9e7a486b5e53ef5c))
* **official-bots:** consume GameOver stream for bot cleanup ([#67](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/67)) ([db9d153](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/db9d1533912f4b41c4d1ca80ccffdde5d23d6ff6))
* **official-bots:** implement king-relative (HalfKP) encoding in NNUE (NCS-109) ([#80](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/80)) ([44f376f](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/44f376f03221f086b898741436e13c93fd314dd1))
* **official-bots:** make HybridBot veto actionable and use it for expert ([1df29cf](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1df29cf3a6e21af3f396b2b7a6da67d978f941ae))
* **official-bots:** park expert bot on tournament server at startup ([#75](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/75)) ([30295a4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/30295a4bb95855ee8261c92278bb9ebc80ee12ee))
* **official-bots:** resolve tournament bot token from Redis and account service ([386ddc5](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/386ddc5c19f8f893b16c6422aa5393b54c872e45))
* **official-bots:** standalone self-play + one-shot dataset builder for NNUE training ([1c80abd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1c80abdb8a45814d642d43c633cde81ce7374c4f))
* **tournament:** auto-join external tournaments and publish created ones ([#77](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/77)) ([9978b7e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9978b7ea78eb658a225a461b9cd339386c0c14f3))
* **tournament:** federate tournaments across clusters with DB replication ([5b000a6](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/5b000a6e5f04ea6770d1c7ab6bfdaded77a99172))
* **tournament:** seed external server registry from env var on startup ([845dc9c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/845dc9c2935c8bc1be42541dfaf31c9a861d3272))
* true-microservices ([#40](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/40)) ([5909242](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/590924254e8a2754de661a57a03e43f89ceb6299))
### Bug Fixes
* enable official bots to connect to external tournament server ([#71](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/71)) ([688d30e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/688d30e2b10026923372be5fca3c63eaaee2de2a))
* modified training pipeline ([9f9140c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9f9140cb585345cd244a1dfee1a06e51a5f7f7a8))
* **official-bots:** configure JWT verification ([#72](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/72)) ([98c64fc](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/98c64fc0d56dc542beb31c75f4b9056d91de03cd))
* **official-bots:** correct parkOn path from /api/bots to /api/account/bots ([1be9949](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1be9949c0b5c6a1db535696620d77735050d6c93))
* **official-bots:** derive tournament game color from game endpoint ([#79](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/79)) ([bfc4672](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfc46723e615bb9b65f7f9bba5f53877c4f079a7))
* **official-bots:** discover tournament games by polling, not just the stream ([10113fd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/10113fd0579b614d15870798d933bc9c495d2049))
* **official-bots:** make botToken optional, fall back to env, fix 502 status ([f43d193](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f43d1930d80670d810c57b54eaa3789854fa082c))
* **official-bots:** NCS-70-auto-register official bots with account service ([#59](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/59)) ([7117a93](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/7117a93376272094d0b1a6abf2121254ce396684))
* **official-bots:** park on external tournament servers using correct endpoint and token ([3188241](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/31882417377468b41bbe3ff94506aa4928024450))
* **official-bots:** play games by polling state instead of NDJSON stream ([bfb15c7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfb15c7299bd471d5e064a577ed10af98e2ea90a))
* **official-bots:** play only own tournament games with correct color ([4651bb7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/4651bb796f07a21bd013d9521b2dfe2e1078cebb))
* **official-bots:** prevent Colab OOM in NNUE training ([e2b4342](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/e2b4342f602215b5e8de6fccafc4105525a1ddd1))
* **official-bots:** prioritize Redis token over stale env var in joinTournament ([83dd2d4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/83dd2d4335ca48eb3e5aa234a75367574276ba63))
* **official-bots:** register with tournament server directly to get correct token ([64b5d55](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/64b5d5567f110c2fe152558c7de275a1e0b30e21))
* **official-bots:** resolve per-difficulty bot token on tournament join ([fdf4c94](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fdf4c94811d086996447bb4657fac1d9bd6e5a93))
* **official-bots:** resume tournaments already joined after restart ([285b73e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/285b73efbd6dd98cec410ade9eead9881d693a8f))
* **official-bots:** stream NNUE features as sparse indices to stop host OOM ([9d65662](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9d656624d85889f55746faa5704578e248f9b088))
* **official-bots:** sync bots before token fetch on first startup after DB wipe ([b0ddb27](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b0ddb274d23bca8b1b3f691ce0d643f33e0b54cd))
* **official-bots:** use ThreadLocalRandom in PolyglotBook for native image ([1b30c3b](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1b30c3be393d25712c8743d3d9057207f8bbb67c))
### Reverts
* Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
## (2026-06-29)
### Features
* add initialization metrics for various services ([d438e97](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d438e97f32bdde0bfc63c1b4a8cc810cdd093166))
* add OpenTelemetry trace configuration with parentbased sampler ([3904d5a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3904d5ad8ad4930ddee65287a7bfab785a6148f5))
* **analytics:** add Spark batch analytics module ([#70](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/70)) ([39f1657](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39f1657e1db6e84889af338c43be8cb5c03c3ec3))
* **config:** update application.yml for PostgreSQL and remove staging/production configurations ([2404e61](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/2404e6164c3b50ffccbea5238d636060d6abe4d6))
* **config:** update application.yml for staging and production environments ([6113432](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6113432a14c476a3a0dfc0d449e17d023697f2ba))
* configure logging and add OpenTelemetry support ([#49](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/49)) ([d57c488](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d57c4886612d1d92da0e1b79209fc83e6ef537a1))
* **docker:** add .dockerignore and .gitignore files for build exclusions ([c987d8e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c987d8e258c0e6c4cfbdaa8381c64c410d7a2b83))
* **docker:** add Dockerfiles for building Quarkus application in native and JVM modes ([3f2d2bb](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3f2d2bb4c97fa8cddba66e1da4427c54236dfeed))
* **docker:** add Dockerfiles for Quarkus application in JVM and native modes ([34b9933](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/34b993304670cf2aa62cd2f6460cee7b9864b08e))
* **events:** migrate game-creation and bot flows to Redis Streams NCS-89 ([#62](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/62)) ([a24924c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a24924c23057db3d700a75dbc4333557789cd991))
* **ncs-110:** feed NNUE root-move scores into search move ordering ([#83](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/83)) ([e4fee85](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/e4fee8513430093d46957970618935e99591519f))
* NCS-78 Add Traceability to the Applications ([#46](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/46)) ([649566e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/649566eb3fcf38f91c8896a739f74ea318af312d))
* NCS-78 Add Traceability to the Applications ([#47](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/47)) ([87dfc6c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/87dfc6c2bcce7f7d58fc641bd8d468a2e584c108))
* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
* **official-bots:** activate opening book in expert bot (native-safe) ([260db25](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/260db25803ec55ce99e55782791eabdc190dfed4))
* **official-bots:** add Google Colab notebook for NNUE training (NCS-111) ([#81](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/81)) ([fa10852](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fa10852bc98451d4068ec6fb9e7a486b5e53ef5c))
* **official-bots:** consume GameOver stream for bot cleanup ([#67](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/67)) ([db9d153](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/db9d1533912f4b41c4d1ca80ccffdde5d23d6ff6))
* **official-bots:** implement king-relative (HalfKP) encoding in NNUE (NCS-109) ([#80](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/80)) ([44f376f](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/44f376f03221f086b898741436e13c93fd314dd1))
* **official-bots:** make HybridBot veto actionable and use it for expert ([1df29cf](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1df29cf3a6e21af3f396b2b7a6da67d978f941ae))
* **official-bots:** park expert bot on tournament server at startup ([#75](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/75)) ([30295a4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/30295a4bb95855ee8261c92278bb9ebc80ee12ee))
* **official-bots:** resolve tournament bot token from Redis and account service ([386ddc5](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/386ddc5c19f8f893b16c6422aa5393b54c872e45))
* **official-bots:** standalone self-play + one-shot dataset builder for NNUE training ([1c80abd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1c80abdb8a45814d642d43c633cde81ce7374c4f))
* **tournament:** auto-join external tournaments and publish created ones ([#77](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/77)) ([9978b7e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9978b7ea78eb658a225a461b9cd339386c0c14f3))
* **tournament:** federate tournaments across clusters with DB replication ([5b000a6](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/5b000a6e5f04ea6770d1c7ab6bfdaded77a99172))
* **tournament:** seed external server registry from env var on startup ([845dc9c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/845dc9c2935c8bc1be42541dfaf31c9a861d3272))
* true-microservices ([#40](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/40)) ([5909242](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/590924254e8a2754de661a57a03e43f89ceb6299))
### Bug Fixes
* **bot:** include quiet promotions in quiescence search ([4938560](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/49385600147021cd29f00a8eecc6be7ba8470717))
* enable official bots to connect to external tournament server ([#71](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/71)) ([688d30e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/688d30e2b10026923372be5fca3c63eaaee2de2a))
* modified training pipeline ([9f9140c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9f9140cb585345cd244a1dfee1a06e51a5f7f7a8))
* **official-bots:** configure JWT verification ([#72](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/72)) ([98c64fc](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/98c64fc0d56dc542beb31c75f4b9056d91de03cd))
* **official-bots:** correct parkOn path from /api/bots to /api/account/bots ([1be9949](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1be9949c0b5c6a1db535696620d77735050d6c93))
* **official-bots:** derive tournament game color from game endpoint ([#79](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/79)) ([bfc4672](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfc46723e615bb9b65f7f9bba5f53877c4f079a7))
* **official-bots:** discover tournament games by polling, not just the stream ([10113fd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/10113fd0579b614d15870798d933bc9c495d2049))
* **official-bots:** make botToken optional, fall back to env, fix 502 status ([f43d193](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f43d1930d80670d810c57b54eaa3789854fa082c))
* **official-bots:** NCS-70-auto-register official bots with account service ([#59](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/59)) ([7117a93](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/7117a93376272094d0b1a6abf2121254ce396684))
* **official-bots:** park on external tournament servers using correct endpoint and token ([3188241](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/31882417377468b41bbe3ff94506aa4928024450))
* **official-bots:** play games by polling state instead of NDJSON stream ([bfb15c7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfb15c7299bd471d5e064a577ed10af98e2ea90a))
* **official-bots:** play only own tournament games with correct color ([4651bb7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/4651bb796f07a21bd013d9521b2dfe2e1078cebb))
* **official-bots:** prevent Colab OOM in NNUE training ([e2b4342](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/e2b4342f602215b5e8de6fccafc4105525a1ddd1))
* **official-bots:** prioritize Redis token over stale env var in joinTournament ([83dd2d4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/83dd2d4335ca48eb3e5aa234a75367574276ba63))
* **official-bots:** register with tournament server directly to get correct token ([64b5d55](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/64b5d5567f110c2fe152558c7de275a1e0b30e21))
* **official-bots:** resolve per-difficulty bot token on tournament join ([fdf4c94](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fdf4c94811d086996447bb4657fac1d9bd6e5a93))
* **official-bots:** resume tournaments already joined after restart ([285b73e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/285b73efbd6dd98cec410ade9eead9881d693a8f))
* **official-bots:** stream NNUE features as sparse indices to stop host OOM ([9d65662](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9d656624d85889f55746faa5704578e248f9b088))
* **official-bots:** sync bots before token fetch on first startup after DB wipe ([b0ddb27](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b0ddb274d23bca8b1b3f691ce0d643f33e0b54cd))
* **official-bots:** use ThreadLocalRandom in PolyglotBook for native image ([1b30c3b](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1b30c3be393d25712c8743d3d9057207f8bbb67c))
### Reverts
* Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
## (2026-06-30)
### Features
* add initialization metrics for various services ([d438e97](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d438e97f32bdde0bfc63c1b4a8cc810cdd093166))
* add OpenTelemetry trace configuration with parentbased sampler ([3904d5a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3904d5ad8ad4930ddee65287a7bfab785a6148f5))
* **analytics:** add Spark batch analytics module ([#70](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/70)) ([39f1657](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39f1657e1db6e84889af338c43be8cb5c03c3ec3))
* **config:** update application.yml for PostgreSQL and remove staging/production configurations ([2404e61](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/2404e6164c3b50ffccbea5238d636060d6abe4d6))
* **config:** update application.yml for staging and production environments ([6113432](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6113432a14c476a3a0dfc0d449e17d023697f2ba))
* configure logging and add OpenTelemetry support ([#49](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/49)) ([d57c488](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d57c4886612d1d92da0e1b79209fc83e6ef537a1))
* **docker:** add .dockerignore and .gitignore files for build exclusions ([c987d8e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c987d8e258c0e6c4cfbdaa8381c64c410d7a2b83))
* **docker:** add Dockerfiles for building Quarkus application in native and JVM modes ([3f2d2bb](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3f2d2bb4c97fa8cddba66e1da4427c54236dfeed))
* **docker:** add Dockerfiles for Quarkus application in JVM and native modes ([34b9933](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/34b993304670cf2aa62cd2f6460cee7b9864b08e))
* **events:** migrate game-creation and bot flows to Redis Streams NCS-89 ([#62](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/62)) ([a24924c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a24924c23057db3d700a75dbc4333557789cd991))
* **ncs-110:** feed NNUE root-move scores into search move ordering ([#83](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/83)) ([e4fee85](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/e4fee8513430093d46957970618935e99591519f))
* NCS-78 Add Traceability to the Applications ([#46](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/46)) ([649566e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/649566eb3fcf38f91c8896a739f74ea318af312d))
* NCS-78 Add Traceability to the Applications ([#47](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/47)) ([87dfc6c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/87dfc6c2bcce7f7d58fc641bd8d468a2e584c108))
* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
* **official-bots:** activate opening book in expert bot (native-safe) ([260db25](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/260db25803ec55ce99e55782791eabdc190dfed4))
* **official-bots:** add Google Colab notebook for NNUE training (NCS-111) ([#81](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/81)) ([fa10852](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fa10852bc98451d4068ec6fb9e7a486b5e53ef5c))
* **official-bots:** consume GameOver stream for bot cleanup ([#67](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/67)) ([db9d153](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/db9d1533912f4b41c4d1ca80ccffdde5d23d6ff6))
* **official-bots:** implement king-relative (HalfKP) encoding in NNUE (NCS-109) ([#80](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/80)) ([44f376f](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/44f376f03221f086b898741436e13c93fd314dd1))
* **official-bots:** make HybridBot veto actionable and use it for expert ([1df29cf](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1df29cf3a6e21af3f396b2b7a6da67d978f941ae))
* **official-bots:** park expert bot on tournament server at startup ([#75](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/75)) ([30295a4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/30295a4bb95855ee8261c92278bb9ebc80ee12ee))
* **official-bots:** resolve tournament bot token from Redis and account service ([386ddc5](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/386ddc5c19f8f893b16c6422aa5393b54c872e45))
* **official-bots:** standalone self-play + one-shot dataset builder for NNUE training ([1c80abd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1c80abdb8a45814d642d43c633cde81ce7374c4f))
* **tournament:** auto-join external tournaments and publish created ones ([#77](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/77)) ([9978b7e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9978b7ea78eb658a225a461b9cd339386c0c14f3))
* **tournament:** federate tournaments across clusters with DB replication ([5b000a6](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/5b000a6e5f04ea6770d1c7ab6bfdaded77a99172))
* **tournament:** seed external server registry from env var on startup ([845dc9c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/845dc9c2935c8bc1be42541dfaf31c9a861d3272))
* true-microservices ([#40](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/40)) ([5909242](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/590924254e8a2754de661a57a03e43f89ceb6299))
### Bug Fixes
* **bot:** include quiet promotions in quiescence search ([4938560](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/49385600147021cd29f00a8eecc6be7ba8470717))
* **bot:** seed search with game history, add contempt and NNUE mop-up ([faf7eb3](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/faf7eb38ea7a0d3bc41ae4c2ef9a5195822f390c))
* enable official bots to connect to external tournament server ([#71](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/71)) ([688d30e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/688d30e2b10026923372be5fca3c63eaaee2de2a))
* modified training pipeline ([9f9140c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9f9140cb585345cd244a1dfee1a06e51a5f7f7a8))
* **official-bots:** configure JWT verification ([#72](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/72)) ([98c64fc](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/98c64fc0d56dc542beb31c75f4b9056d91de03cd))
* **official-bots:** correct parkOn path from /api/bots to /api/account/bots ([1be9949](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1be9949c0b5c6a1db535696620d77735050d6c93))
* **official-bots:** derive tournament game color from game endpoint ([#79](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/79)) ([bfc4672](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfc46723e615bb9b65f7f9bba5f53877c4f079a7))
* **official-bots:** discover tournament games by polling, not just the stream ([10113fd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/10113fd0579b614d15870798d933bc9c495d2049))
* **official-bots:** make botToken optional, fall back to env, fix 502 status ([f43d193](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f43d1930d80670d810c57b54eaa3789854fa082c))
* **official-bots:** NCS-70-auto-register official bots with account service ([#59](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/59)) ([7117a93](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/7117a93376272094d0b1a6abf2121254ce396684))
* **official-bots:** park on external tournament servers using correct endpoint and token ([3188241](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/31882417377468b41bbe3ff94506aa4928024450))
* **official-bots:** play games by polling state instead of NDJSON stream ([bfb15c7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfb15c7299bd471d5e064a577ed10af98e2ea90a))
* **official-bots:** play only own tournament games with correct color ([4651bb7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/4651bb796f07a21bd013d9521b2dfe2e1078cebb))
* **official-bots:** prevent Colab OOM in NNUE training ([e2b4342](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/e2b4342f602215b5e8de6fccafc4105525a1ddd1))
* **official-bots:** prioritize Redis token over stale env var in joinTournament ([83dd2d4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/83dd2d4335ca48eb3e5aa234a75367574276ba63))
* **official-bots:** register with tournament server directly to get correct token ([64b5d55](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/64b5d5567f110c2fe152558c7de275a1e0b30e21))
* **official-bots:** resolve per-difficulty bot token on tournament join ([fdf4c94](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fdf4c94811d086996447bb4657fac1d9bd6e5a93))
* **official-bots:** resume tournaments already joined after restart ([285b73e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/285b73efbd6dd98cec410ade9eead9881d693a8f))
* **official-bots:** stream NNUE features as sparse indices to stop host OOM ([9d65662](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9d656624d85889f55746faa5704578e248f9b088))
* **official-bots:** sync bots before token fetch on first startup after DB wipe ([b0ddb27](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b0ddb274d23bca8b1b3f691ce0d643f33e0b54cd))
* **official-bots:** use ThreadLocalRandom in PolyglotBook for native image ([1b30c3b](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1b30c3be393d25712c8743d3d9057207f8bbb67c))
### Reverts
* Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
## (2026-06-30)
### Features
* add initialization metrics for various services ([d438e97](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d438e97f32bdde0bfc63c1b4a8cc810cdd093166))
* add OpenTelemetry trace configuration with parentbased sampler ([3904d5a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3904d5ad8ad4930ddee65287a7bfab785a6148f5))
* **analytics:** add Spark batch analytics module ([#70](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/70)) ([39f1657](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39f1657e1db6e84889af338c43be8cb5c03c3ec3))
* **bot:** add Lazy SMP parallel search for the NNUE bot ([3437dab](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3437dab49b2cc3f7b7e726febb07ad759e878079))
* **bot:** clock-aware time management and stream-driven tournament play ([45b5719](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/45b5719d630c9480398f51510cc598d2d6ce39db))
* **bot:** implement bot-vs-bot harness for NNUE evaluation ([b2683a7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b2683a7f5ab6f3e41883736ac5eeaaee2e1ea5c1))
* **config:** update application.yml for PostgreSQL and remove staging/production configurations ([2404e61](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/2404e6164c3b50ffccbea5238d636060d6abe4d6))
* **config:** update application.yml for staging and production environments ([6113432](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6113432a14c476a3a0dfc0d449e17d023697f2ba))
* configure logging and add OpenTelemetry support ([#49](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/49)) ([d57c488](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d57c4886612d1d92da0e1b79209fc83e6ef537a1))
* **docker:** add .dockerignore and .gitignore files for build exclusions ([c987d8e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c987d8e258c0e6c4cfbdaa8381c64c410d7a2b83))
* **docker:** add Dockerfiles for building Quarkus application in native and JVM modes ([3f2d2bb](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3f2d2bb4c97fa8cddba66e1da4427c54236dfeed))
* **docker:** add Dockerfiles for Quarkus application in JVM and native modes ([34b9933](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/34b993304670cf2aa62cd2f6460cee7b9864b08e))
* **events:** migrate game-creation and bot flows to Redis Streams NCS-89 ([#62](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/62)) ([a24924c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a24924c23057db3d700a75dbc4333557789cd991))
* **ncs-110:** feed NNUE root-move scores into search move ordering ([#83](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/83)) ([e4fee85](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/e4fee8513430093d46957970618935e99591519f))
* NCS-78 Add Traceability to the Applications ([#46](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/46)) ([649566e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/649566eb3fcf38f91c8896a739f74ea318af312d))
* NCS-78 Add Traceability to the Applications ([#47](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/47)) ([87dfc6c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/87dfc6c2bcce7f7d58fc641bd8d468a2e584c108))
* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
* **official-bots:** activate opening book in expert bot (native-safe) ([260db25](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/260db25803ec55ce99e55782791eabdc190dfed4))
* **official-bots:** add Google Colab notebook for NNUE training (NCS-111) ([#81](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/81)) ([fa10852](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fa10852bc98451d4068ec6fb9e7a486b5e53ef5c))
* **official-bots:** consume GameOver stream for bot cleanup ([#67](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/67)) ([db9d153](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/db9d1533912f4b41c4d1ca80ccffdde5d23d6ff6))
* **official-bots:** implement king-relative (HalfKP) encoding in NNUE (NCS-109) ([#80](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/80)) ([44f376f](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/44f376f03221f086b898741436e13c93fd314dd1))
* **official-bots:** make HybridBot veto actionable and use it for expert ([1df29cf](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1df29cf3a6e21af3f396b2b7a6da67d978f941ae))
* **official-bots:** park expert bot on tournament server at startup ([#75](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/75)) ([30295a4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/30295a4bb95855ee8261c92278bb9ebc80ee12ee))
* **official-bots:** resolve tournament bot token from Redis and account service ([386ddc5](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/386ddc5c19f8f893b16c6422aa5393b54c872e45))
* **official-bots:** standalone self-play + one-shot dataset builder for NNUE training ([1c80abd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1c80abdb8a45814d642d43c633cde81ce7374c4f))
* **tournament:** auto-join external tournaments and publish created ones ([#77](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/77)) ([9978b7e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9978b7ea78eb658a225a461b9cd339386c0c14f3))
* **tournament:** federate tournaments across clusters with DB replication ([5b000a6](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/5b000a6e5f04ea6770d1c7ab6bfdaded77a99172))
* **tournament:** seed external server registry from env var on startup ([845dc9c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/845dc9c2935c8bc1be42541dfaf31c9a861d3272))
* true-microservices ([#40](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/40)) ([5909242](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/590924254e8a2754de661a57a03e43f89ceb6299))
### Bug Fixes
* **bot:** include quiet promotions in quiescence search ([4938560](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/49385600147021cd29f00a8eecc6be7ba8470717))
* **bot:** seed search with game history, add contempt and NNUE mop-up ([faf7eb3](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/faf7eb38ea7a0d3bc41ae4c2ef9a5195822f390c))
* enable official bots to connect to external tournament server ([#71](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/71)) ([688d30e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/688d30e2b10026923372be5fca3c63eaaee2de2a))
* modified training pipeline ([9f9140c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9f9140cb585345cd244a1dfee1a06e51a5f7f7a8))
* **official-bots:** configure JWT verification ([#72](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/72)) ([98c64fc](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/98c64fc0d56dc542beb31c75f4b9056d91de03cd))
* **official-bots:** correct parkOn path from /api/bots to /api/account/bots ([1be9949](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1be9949c0b5c6a1db535696620d77735050d6c93))
* **official-bots:** derive tournament game color from game endpoint ([#79](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/79)) ([bfc4672](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfc46723e615bb9b65f7f9bba5f53877c4f079a7))
* **official-bots:** discover tournament games by polling, not just the stream ([10113fd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/10113fd0579b614d15870798d933bc9c495d2049))
* **official-bots:** make botToken optional, fall back to env, fix 502 status ([f43d193](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f43d1930d80670d810c57b54eaa3789854fa082c))
* **official-bots:** NCS-70-auto-register official bots with account service ([#59](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/59)) ([7117a93](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/7117a93376272094d0b1a6abf2121254ce396684))
* **official-bots:** park on external tournament servers using correct endpoint and token ([3188241](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/31882417377468b41bbe3ff94506aa4928024450))
* **official-bots:** play games by polling state instead of NDJSON stream ([bfb15c7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfb15c7299bd471d5e064a577ed10af98e2ea90a))
* **official-bots:** play only own tournament games with correct color ([4651bb7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/4651bb796f07a21bd013d9521b2dfe2e1078cebb))
* **official-bots:** prevent Colab OOM in NNUE training ([e2b4342](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/e2b4342f602215b5e8de6fccafc4105525a1ddd1))
* **official-bots:** prioritize Redis token over stale env var in joinTournament ([83dd2d4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/83dd2d4335ca48eb3e5aa234a75367574276ba63))
* **official-bots:** register with tournament server directly to get correct token ([64b5d55](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/64b5d5567f110c2fe152558c7de275a1e0b30e21))
* **official-bots:** resolve per-difficulty bot token on tournament join ([fdf4c94](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fdf4c94811d086996447bb4657fac1d9bd6e5a93))
* **official-bots:** resume tournaments already joined after restart ([285b73e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/285b73efbd6dd98cec410ade9eead9881d693a8f))
* **official-bots:** stream NNUE features as sparse indices to stop host OOM ([9d65662](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9d656624d85889f55746faa5704578e248f9b088))
* **official-bots:** sync bots before token fetch on first startup after DB wipe ([b0ddb27](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b0ddb274d23bca8b1b3f691ce0d643f33e0b54cd))
* **official-bots:** use ThreadLocalRandom in PolyglotBook for native image ([1b30c3b](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1b30c3be393d25712c8743d3d9057207f8bbb67c))
### Reverts
* Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
## (2026-06-30)
### Features
* add initialization metrics for various services ([d438e97](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d438e97f32bdde0bfc63c1b4a8cc810cdd093166))
* add OpenTelemetry trace configuration with parentbased sampler ([3904d5a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3904d5ad8ad4930ddee65287a7bfab785a6148f5))
* **analytics:** add Spark batch analytics module ([#70](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/70)) ([39f1657](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39f1657e1db6e84889af338c43be8cb5c03c3ec3))
* **bot:** add Lazy SMP parallel search for the NNUE bot ([3437dab](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3437dab49b2cc3f7b7e726febb07ad759e878079))
* **bot:** clock-aware time management and stream-driven tournament play ([45b5719](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/45b5719d630c9480398f51510cc598d2d6ce39db))
* **bot:** implement bot-vs-bot harness for NNUE evaluation ([b2683a7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b2683a7f5ab6f3e41883736ac5eeaaee2e1ea5c1))
* **config:** update application.yml for PostgreSQL and remove staging/production configurations ([2404e61](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/2404e6164c3b50ffccbea5238d636060d6abe4d6))
* **config:** update application.yml for staging and production environments ([6113432](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6113432a14c476a3a0dfc0d449e17d023697f2ba))
* configure logging and add OpenTelemetry support ([#49](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/49)) ([d57c488](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d57c4886612d1d92da0e1b79209fc83e6ef537a1))
* **docker:** add .dockerignore and .gitignore files for build exclusions ([c987d8e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c987d8e258c0e6c4cfbdaa8381c64c410d7a2b83))
* **docker:** add Dockerfiles for building Quarkus application in native and JVM modes ([3f2d2bb](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3f2d2bb4c97fa8cddba66e1da4427c54236dfeed))
* **docker:** add Dockerfiles for Quarkus application in JVM and native modes ([34b9933](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/34b993304670cf2aa62cd2f6460cee7b9864b08e))
* **events:** migrate game-creation and bot flows to Redis Streams NCS-89 ([#62](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/62)) ([a24924c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a24924c23057db3d700a75dbc4333557789cd991))
* **ncs-110:** feed NNUE root-move scores into search move ordering ([#83](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/83)) ([e4fee85](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/e4fee8513430093d46957970618935e99591519f))
* NCS-78 Add Traceability to the Applications ([#46](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/46)) ([649566e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/649566eb3fcf38f91c8896a739f74ea318af312d))
* NCS-78 Add Traceability to the Applications ([#47](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/47)) ([87dfc6c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/87dfc6c2bcce7f7d58fc641bd8d468a2e584c108))
* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
* **official-bots:** activate opening book in expert bot (native-safe) ([260db25](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/260db25803ec55ce99e55782791eabdc190dfed4))
* **official-bots:** add Google Colab notebook for NNUE training (NCS-111) ([#81](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/81)) ([fa10852](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fa10852bc98451d4068ec6fb9e7a486b5e53ef5c))
* **official-bots:** consume GameOver stream for bot cleanup ([#67](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/67)) ([db9d153](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/db9d1533912f4b41c4d1ca80ccffdde5d23d6ff6))
* **official-bots:** implement king-relative (HalfKP) encoding in NNUE (NCS-109) ([#80](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/80)) ([44f376f](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/44f376f03221f086b898741436e13c93fd314dd1))
* **official-bots:** make HybridBot veto actionable and use it for expert ([1df29cf](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1df29cf3a6e21af3f396b2b7a6da67d978f941ae))
* **official-bots:** park expert bot on tournament server at startup ([#75](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/75)) ([30295a4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/30295a4bb95855ee8261c92278bb9ebc80ee12ee))
* **official-bots:** resolve tournament bot token from Redis and account service ([386ddc5](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/386ddc5c19f8f893b16c6422aa5393b54c872e45))
* **official-bots:** standalone self-play + one-shot dataset builder for NNUE training ([1c80abd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1c80abdb8a45814d642d43c633cde81ce7374c4f))
* **tournament:** auto-join external tournaments and publish created ones ([#77](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/77)) ([9978b7e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9978b7ea78eb658a225a461b9cd339386c0c14f3))
* **tournament:** federate tournaments across clusters with DB replication ([5b000a6](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/5b000a6e5f04ea6770d1c7ab6bfdaded77a99172))
* **tournament:** seed external server registry from env var on startup ([845dc9c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/845dc9c2935c8bc1be42541dfaf31c9a861d3272))
* true-microservices ([#40](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/40)) ([5909242](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/590924254e8a2754de661a57a03e43f89ceb6299))
### Bug Fixes
* **bot:** drop game-stream play, poll with low delay ([6e37a7d](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6e37a7d20992b7d27c5ffebb12184916ed9d3120))
* **bot:** include quiet promotions in quiescence search ([4938560](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/49385600147021cd29f00a8eecc6be7ba8470717))
* **bot:** seed search with game history, add contempt and NNUE mop-up ([faf7eb3](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/faf7eb38ea7a0d3bc41ae4c2ef9a5195822f390c))
* enable official bots to connect to external tournament server ([#71](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/71)) ([688d30e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/688d30e2b10026923372be5fca3c63eaaee2de2a))
* modified training pipeline ([9f9140c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9f9140cb585345cd244a1dfee1a06e51a5f7f7a8))
* **official-bots:** configure JWT verification ([#72](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/72)) ([98c64fc](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/98c64fc0d56dc542beb31c75f4b9056d91de03cd))
* **official-bots:** correct parkOn path from /api/bots to /api/account/bots ([1be9949](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1be9949c0b5c6a1db535696620d77735050d6c93))
* **official-bots:** derive tournament game color from game endpoint ([#79](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/79)) ([bfc4672](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfc46723e615bb9b65f7f9bba5f53877c4f079a7))
* **official-bots:** discover tournament games by polling, not just the stream ([10113fd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/10113fd0579b614d15870798d933bc9c495d2049))
* **official-bots:** make botToken optional, fall back to env, fix 502 status ([f43d193](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f43d1930d80670d810c57b54eaa3789854fa082c))
* **official-bots:** NCS-70-auto-register official bots with account service ([#59](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/59)) ([7117a93](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/7117a93376272094d0b1a6abf2121254ce396684))
* **official-bots:** park on external tournament servers using correct endpoint and token ([3188241](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/31882417377468b41bbe3ff94506aa4928024450))
* **official-bots:** play games by polling state instead of NDJSON stream ([bfb15c7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/bfb15c7299bd471d5e064a577ed10af98e2ea90a))
* **official-bots:** play only own tournament games with correct color ([4651bb7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/4651bb796f07a21bd013d9521b2dfe2e1078cebb))
* **official-bots:** prevent Colab OOM in NNUE training ([e2b4342](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/e2b4342f602215b5e8de6fccafc4105525a1ddd1))
* **official-bots:** prioritize Redis token over stale env var in joinTournament ([83dd2d4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/83dd2d4335ca48eb3e5aa234a75367574276ba63))
* **official-bots:** register with tournament server directly to get correct token ([64b5d55](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/64b5d5567f110c2fe152558c7de275a1e0b30e21))
* **official-bots:** resolve per-difficulty bot token on tournament join ([fdf4c94](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/fdf4c94811d086996447bb4657fac1d9bd6e5a93))
* **official-bots:** resume tournaments already joined after restart ([285b73e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/285b73efbd6dd98cec410ade9eead9881d693a8f))
* **official-bots:** stream NNUE features as sparse indices to stop host OOM ([9d65662](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9d656624d85889f55746faa5704578e248f9b088))
* **official-bots:** sync bots before token fetch on first startup after DB wipe ([b0ddb27](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b0ddb274d23bca8b1b3f691ce0d643f33e0b54cd))
* **official-bots:** use ThreadLocalRandom in PolyglotBook for native image ([1b30c3b](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1b30c3be393d25712c8743d3d9057207f8bbb67c))
### Reverts
* Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
+8
View File
@@ -47,6 +47,14 @@ tasks.withType<JavaCompile> {
options.compilerArgs.add("-parameters") options.compilerArgs.add("-parameters")
} }
tasks.register<JavaExec>("selfPlay") {
group = "nnue"
description = "Run standalone NNUEBot self-play and write FENs for labeling."
mainClass.set("de.nowchess.bot.selfplay.SelfPlayMain")
classpath = sourceSets["main"].runtimeClasspath
args((project.findProperty("spArgs")?.toString() ?: "").split(" ").filter { it.isNotBlank() })
}
dependencies { dependencies {
compileOnly("org.scala-lang:scala3-compiler_3") { compileOnly("org.scala-lang:scala3-compiler_3") {
@@ -0,0 +1,212 @@
# Concept: NNUE Training Data — Quality, Scale, and Transfer to Colab
Local generation + labeling is **not** a constraint (Ryzen 9800X3D / RTX 5070 / 32 GB).
So the design splits cleanly:
- **Data plane = local box.** Generate, label, shard, publish. Cheap, fast, no limits.
- **Train plane = Colab.** Pull a dataset version, GPU-train, export `.nbai`.
Colab never runs Stockfish and never sees a browser upload. Three problems below:
**(1) good data, (2) growing it over time, (3) getting it there easily** — (3) is the priority.
---
## 1. Generating *good* training sets
### The current weak spot
`generate.py` plays **fully random games** (`random.choice(legal_moves)`). Random play
produces positions that never occur in real games — material chaos, nonsense pawn
structures. An NNUE trained on that learns to evaluate a distribution it will never
face. Fine as filler, wrong as the backbone.
### What a good NNUE dataset needs
1. **Realistic position distribution.** Positions should resemble what the bot actually
reaches in search — from real games and engine play, not coin-flip moves.
2. **Phase coverage.** Openings, middlegames, endgames all represented. Endgames are
under-sampled by random play and matter most for precise eval.
3. **Eval balance.** Real game data is dominated by near-equal positions. If 80% of
labels sit in `[-0.5, +0.5]`, the net learns "everything is roughly equal." Resample
to flatten the eval histogram (cap per-bucket counts).
4. **Accurate labels.** Deeper Stockfish = better target. Locally you can afford
depth 1620. Or skip labeling entirely with the Lichess eval DB (below).
5. **Clean positions.** Dedup by FEN; drop terminal/checkmate/stalemate; the side to
move should not already be in check unless intended; tag the game phase.
### Recommended source mix (per dataset version)
| Source | Role | How | Weight |
|---|---|---|---|
| **Lichess eval DB** | Backbone | `lichess_importer.py` — millions of FENs **pre-labeled** by deep Stockfish, real human positions, correct sign convention | 5070% |
| **Engine self-play** | Bot's own distribution | NNUEBot (or vs Stockfish) plays games; sample positions; label with local Stockfish | 2040% |
| **Tactical puzzles** | Sharp/critical positions | `tactical_positions_extractor.py` (Lichess puzzle DB) | 515% |
| **Random play** | Cheap diversity filler | existing `generate.py`, capped low | ≤10% |
The backbone is real, pre-labeled data — so labeling cost is near zero and quality is
high. Self-play is the part that adapts data to *your* bot. Random play stays only as
a thin diversity sprinkle.
### Self-play flywheel (the quality engine over time)
The strongest lever: **net N generates the games that train net N+1.**
```
net_vN ──play self-play games──► sample positions ──label (Stockfish)──►
▲ │
└──────────────── train on (backbone + new self-play) ◄─────────────────┘
net_v(N+1)
```
Each generation, the bot reaches positions closer to its real playing distribution,
labels them with a stronger-than-bot oracle (Stockfish), and learns the gap. Standard
modern NNUE practice. Keep the Lichess backbone mixed in every round so the net does
not overfit to its own blind spots.
---
## 2. Scaling datasets over time — append-only shards
Do **not** maintain one growing `labeled.jsonl` and re-copy it. Make a dataset an
**immutable set of shards plus a manifest**:
```
datasets/
shards/
lichess_000001.jsonl.zst # ~50100k positions each, ~510 MB compressed
lichess_000002.jsonl.zst
selfplay_v7_000001.jsonl.zst
tactical_000001.jsonl.zst
...
manifest.json
```
`manifest.json`:
```json
{
"dataset_version": 7,
"created": "2026-06-24T...",
"total_positions": 4200000,
"scale": 300.0,
"shards": [
{"file": "lichess_000001.jsonl.zst", "positions": 100000,
"sha256": "...", "source": "lichess_eval", "stockfish_depth": 0},
{"file": "selfplay_v7_000001.jsonl.zst", "positions": 80000,
"sha256": "...", "source": "selfplay", "net": "v7", "stockfish_depth": 18}
]
}
```
Properties this buys:
- **Growth = add shards.** Generate a new batch, label it, write one new shard, append
one manifest entry. Never touch existing shards. O(new data), not O(total).
- **Provenance.** Each shard records source + net + depth. You can later down-weight or
drop a bad batch by editing the manifest, no relabeling.
- **Dedup across shards** by FEN hash at build time; record dropped counts in metadata.
- **Reproducible mixes.** A "dataset version" is just a manifest selecting shards +
per-source sampling weights. Cheap to define many mixes over the same shard pool.
- **Resumable, cache-friendly transfer** (next section) — the whole reason for shards.
`dataset.py`'s existing `ds_vN` + `metadata.json` scheme generalizes to this directly:
the dataset dir holds `shards/` + `manifest.json` instead of one `labeled.jsonl`.
---
## 3. Getting data to Colab easily ← top priority
Shards make this trivial: **incremental sync, never a full re-upload.**
### Recommended: rclone → Google Drive, read from mounted Drive
Colab mounts Drive natively, so the cheapest path is to make Drive the shard store and
sync into it with `rclone` (only uploads new/changed shards):
```bash
# Local, after building shards:
rclone copy datasets/ gdrive:NowChess/datasets --progress
# ^ uploads only shards Drive doesn't have yet. Adding 80k positions = one small file.
```
Colab side, one cell:
```python
SRC = '/content/drive/MyDrive/NowChess/datasets' # mounted, no download
import json, shutil, pathlib
manifest = json.load(open(f'{SRC}/manifest.json'))
local = pathlib.Path('/content/datasets'); local.mkdir(exist_ok=True)
for sh in manifest['shards']: # copy Drive→local SSD (fast seq read)
dst = local / sh['file']
if not dst.exists(): # cache: only copy missing shards
shutil.copy(f"{SRC}/shards/{sh['file']}", dst)
```
Why this wins on "easy":
- **No browser upload, ever.** One `rclone copy` from your PC.
- **Incremental both directions.** Add a shard locally → next `rclone copy` ships only
that shard. Colab copies only shards it doesn't already have on `/content`.
- **Zero new infra.** Drive is already mounted in the notebook.
### Alternative: Gitea release per dataset version (if Drive quota hurts)
You self-host `git.janis-eccarius.de`. Tag `ds_v7`, attach shards + `manifest.json` as
release assets. Colab reads the manifest, then parallel-`wget` only the shards it lacks
(checksum-verified). Versioned, immutable, no Drive quota, token-gated. Slightly more
wiring than rclone→Drive.
Pick rclone→Drive for minimum friction; Gitea releases if you want hard versioning and
to keep Drive small.
### Notebook changes either way
- Clone repo to **ephemeral `/content`** (fast), not Drive. Persist only datasets +
checkpoints.
- Drop Option A (no Colab generation) and Option B (no browser upload). One "sync
dataset version" cell instead.
- Train reads shards via a streaming `.jsonl.zst` loader (apply per-source sampling
weights + eval-bucket balancing here). Keep burst-train + Drive checkpoints + `.nbai`
export.
---
## Resulting workflow
```
LOCAL (9800X3D / RTX5070) COLAB (GPU)
───────────────────────── ───────────
import Lichess eval DB ─┐
self-play with net_vN ─┼─► label ─► dedup ─► write new shard(s) ─► manifest++
tactical / random ─┘ │
rclone copy ────────┘
datasets/ → Drive
│ (only new shards move)
sync version → copy missing shards → train (GPU)
export .nbai
place in src/main/resources/, rebuild native image
```
## Build order
1. **Shard format + manifest** in `dataset.py`: write/read `shards/*.jsonl.zst` +
`manifest.json`; dedup-across-shards on build; provenance per shard.
2. **Streaming `.zst` dataloader** in `train.py`: read shards, apply per-source weights
and eval-bucket balancing.
3. **Self-play generator** in `src/`: NNUEBot/Stockfish self-play → positions → local
Stockfish label → new shard. This is the scaling engine.
4. **`dataset_sync.py`**: `push` (rclone→Drive or Gitea upload) / `pull` (cache-aware).
5. **Notebook rewrite**: ephemeral clone, single sync cell, weighted streaming loader.
6. Wire `lichess_importer.py` as the backbone shard source.
## Open decisions
- **Transfer backend** — rclone→Drive (easiest, recommended) vs Gitea releases (hard
versioning).
- **Self-play opponent** — NNUEBot vs itself (own distribution) vs vs-Stockfish
(stronger, more decisive games). Likely a mix.
- **Backbone/self-play ratio** — start ~60/30/10 (lichess/selfplay/tactical), tune by
measured strength.
- **Shard size** — 50k vs 100k positions/shard (transfer granularity vs file count).
@@ -0,0 +1,180 @@
# Implementation Plan: Two One-Liner Tools (self-play + dataset)
Goal: **two tools, two start scripts, minimal params.**
```
./selfplay.sh # bot plays games against itself, writes selfplay FENs (Scala, standalone)
./dataset.sh # builds the ENTIRE training dataset + rclone push to Drive (Python, one script)
```
Both default-everything. Optional first positional arg only when you want to override
the one number that matters.
---
## Tool 1 — `selfplay.sh` (standalone bot, no microservices)
### Why it can be standalone
`Bot` is just `GameContext => Option[Move]` (`Bot.scala`). `NNUEBot.apply` needs only
`DefaultRules` (rule module) + `EvaluationNNUE` (loads the bundled `.nbai`). No Quarkus,
no coordinator/account/ws. The bot module already depends on `api, rule, io`, and `io`
has `FenExporter` + `GameContext.initial` exists. So a plain JVM `main` can run games
with zero service wiring.
### New file: `SelfPlayMain.scala`
`modules/official-bots/src/main/scala/de/nowchess/bot/selfplay/SelfPlayMain.scala`
Loop per game:
1. Start from `GameContext.initial`.
2. **Opening diversity** — play `R` random legal plies (default 8). Without this,
NNUEBot vs itself is deterministic → the *same game every time*. Random openings are
what make the games diverse. (Optional later: seed from polyglot book instead.)
3. Then both sides = `NNUEBot(difficulty)`. Apply moves via `DefaultRules.applyMove`.
4. Stop on `isCheckmate / isStalemate / isInsufficientMaterial / isFiftyMoveRule /
isThreefoldRepetition`, or ply cap (default 200).
5. Emit one **FEN per ply** (via `FenExporter`), skipping positions where side-to-move
is in check and terminal positions — same filter philosophy the labeler wants.
6. Append FENs to the output file (one per line) — exactly the format `label.py` reads.
Config = a small `case class` with defaults; read from env/args. Defaults:
`games=2000`, `randomOpeningPlies=8`, `maxPlies=200`, `out=python/data/selfplay.txt`,
`threads = availableProcessors`. Parallelize games across threads (each game is
independent; bot is pure).
Output is **FENs only** — labeling happens in Tool 2 with Stockfish. Keeps the bot tool
single-responsibility and fast.
### Gradle: a plain run task (not Quarkus)
Add to `modules/official-bots/build.gradle.kts`:
```kotlin
tasks.register<JavaExec>("selfPlay") {
group = "nnue"
mainClass.set("de.nowchess.bot.selfplay.SelfPlayMain")
classpath = sourceSets["main"].runtimeClasspath
args(project.findProperty("spArgs")?.toString()?.split(" ") ?: emptyList())
}
```
### `selfplay.sh` (repo `python/` dir)
```bash
#!/usr/bin/env bash
set -euo pipefail
GAMES="${1:-2000}"
cd "$(dirname "$0")/../../.." # repo root
./gradlew -q :official-bots:selfPlay -PspArgs="--games $GAMES --out modules/official-bots/python/data/selfplay.txt"
echo "Self-play FENs -> modules/official-bots/python/data/selfplay.txt"
```
Usage:
```bash
./selfplay.sh # 2000 games, bundled net
./selfplay.sh 8000 # more games
```
---
## Tool 2 — `dataset.sh` → `build_dataset.py` (builds EVERYTHING)
One Python script that produces a complete, sharded, pushed dataset. No TUI, no
multi-step menus. It runs the whole data plane end to end:
```
lichess eval DB ─┐
selfplay.txt ─┼─► label (local Stockfish, skip already-labeled) ─► dedup ─►
tactical ─┤ eval-bucket
random filler ─┘ balance ─►
write shards/*.jsonl.zst + manifest.json ─► rclone push
```
### New file: `build_dataset.py` (top-level `python/`)
Reuses existing modules — orchestrates, doesn't reinvent:
- **Backbone:** `lichess_importer.py` — download + sample N pre-labeled positions from
the Lichess eval DB (no Stockfish cost).
- **Self-play:** read `data/selfplay.txt` FENs → `label.py` with local Stockfish
(depth 18, all cores — your box eats this).
- **Tactical:** `tactical_positions_extractor.py` → `label.py`.
- **Random filler:** `generate.py` (small cap) → `label.py`.
- **Merge:** dedup by FEN across all sources; **eval-bucket balancing** (cap positions
per eval bin so near-equal positions don't dominate).
- **Shard + manifest:** split into `shards/*.jsonl.zst` (~100k positions each) + write
`manifest.json` (positions, sha256, source, net, depth per shard). Append-only:
existing shards untouched, new run adds shards + entries (the scaling story from the
concept).
- **Push:** `rclone copy datasets/ gdrive:NowChess/datasets` — ships only new shards.
### One config block, sane defaults
Top of the script — the *only* thing you ever touch:
```python
LICHESS_POSITIONS = 2_000_000 # backbone
USE_SELFPLAY = True # reads data/selfplay.txt if present
TACTICAL_PUZZLES = 200_000
RANDOM_FILLER = 100_000
STOCKFISH_DEPTH = 18
RCLONE_REMOTE = "gdrive:NowChess/datasets"
```
Everything else (paths, workers=all cores, shard size, balancing bins) is internal.
### `dataset.sh`
```bash
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")"
python build_dataset.py "$@"
```
Usage:
```bash
./dataset.sh # full dataset (lichess + selfplay + tactical + filler) -> Drive
```
That single command: downloads backbone, labels self-play/tactical/filler, dedups,
balances, shards, and rclone-pushes to Drive. Colab then syncs (concept doc §3).
---
## End-to-end loop (the flywheel)
```
./selfplay.sh # bot generates games with the current net
./dataset.sh # fold them into a new dataset version, push to Drive
# (Colab) sync + train -> export nnue_weights.nbai
# drop .nbai into modules/official-bots/src/main/resources/, rebuild
./selfplay.sh # next net plays stronger, better games... repeat
```
---
## Build order
1. `SelfPlayMain.scala` — standalone game loop, random openings, parallel games, FEN out.
2. `selfPlay` Gradle `JavaExec` task + `selfplay.sh`.
3. `build_dataset.py` — orchestrate existing importer/label/tactical/generate into
shards + manifest; rclone push.
4. `dataset.sh`.
5. Shard/manifest read support in `dataset.py` + zstd streaming loader in `train.py`
(consumed on Colab).
6. Notebook: single "sync dataset version" cell, ephemeral `/content` clone.
## Decisions to confirm
- **Self-play opponent:** NNUEBot vs itself + random openings (planned). Add vs-Stockfish
later if more decisive games wanted.
- **Self-play net source:** use the `.nbai` bundled in `resources` (simplest), or accept
a `--weights path`? Plan = bundled by default.
- **rclone remote name:** confirm `gdrive` is your configured rclone remote, and the
target folder `NowChess/datasets`.
- **Stockfish path on your box:** `$STOCKFISH_PATH` or `/usr/games/stockfish`?
@@ -0,0 +1,190 @@
{
"nbformat": 4,
"nbformat_minor": 5,
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.10.0"
},
"colab": {
"provenance": [],
"gpuType": "T4"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": "# NNUE Training Pipeline\n\nGPU training on Colab. Data is built **locally** (`./dataset.sh` → sharded, pushed to\nDrive via rclone); this notebook only **syncs shards → trains → exports `.nbai`**.\nNo generation, no Stockfish labeling, no browser uploads here.\n\n**Runtime:** GPU (T4 or better). Runtime → Change runtime type → T4 GPU.\n\n**Persistence:** Datasets and checkpoints live on Google Drive, so training resumes\nafter a session timeout. The repo is cloned to ephemeral `/content` for speed.",
"id": "intro-md"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"## ⚙️ 1 — Setup"
],
"id": "setup-md"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Mount Google Drive for checkpoint persistence\n",
"from google.colab import drive\n",
"drive.mount('/content/drive')"
],
"id": "mount-drive"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "import os\n\n# ── Configure these paths once ───────────────────────────────────────────────\nREPO_URL = 'https://git.janis-eccarius.de/NowChess/NowChessSystems.git'\nDRIVE_ROOT = '/content/drive/MyDrive/NowChess' # datasets + weights persist here\nREPO_DIR = '/content/NowChessSystems' # ephemeral, fast local clone\nPYTHON_DIR = f'{REPO_DIR}/modules/official-bots/python'\n# ─────────────────────────────────────────────────────────────────────────────\n\nos.makedirs(DRIVE_ROOT, exist_ok=True)\n\n# Clone to ephemeral /content (NOT Drive) — fast checkout, no Drive bloat.\nif not os.path.isdir(REPO_DIR):\n !git clone --depth=1 \"{REPO_URL}\" \"{REPO_DIR}\"\n print('Repo cloned to /content.')\nelse:\n !git -C \"{REPO_DIR}\" pull --ff-only\n print('Repo updated.')",
"id": "clone-repo"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "# Install Python dependencies. No Stockfish — labeling happens on the local box,\n# this notebook only trains on already-labeled shards.\n!pip install -q chess tqdm rich zstandard\n\nimport sys\nsys.path.insert(0, f'{PYTHON_DIR}/src')\nsys.path.insert(0, PYTHON_DIR)\nprint('Python path configured.')",
"id": "install-deps"
},
{
"cell_type": "markdown",
"metadata": {},
"source": "---\n## 🗄️ 2 — Data\n\nDatasets are built **locally** (`./dataset.sh`) and pushed to Drive with rclone as\ncompressed shards under `MyDrive/NowChess/datasets/`. Here we just sync those shards\nto the fast local disk — no generation, no labeling, no browser uploads.\n\nThe cell reads `manifest.json` and copies only shards not already cached on `/content`.",
"id": "data-md"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "import json, shutil\nfrom pathlib import Path\n\n# Source: shards synced from the local box via `rclone copy datasets/ gdrive:NowChess/datasets`\nDRIVE_DATASETS = Path(DRIVE_ROOT) / 'datasets'\nLOCAL_DATASETS = Path('/content/datasets')\n(LOCAL_DATASETS / 'shards').mkdir(parents=True, exist_ok=True)\n\nmanifest = json.load(open(DRIVE_DATASETS / 'manifest.json'))\nprint(f\"Dataset v{manifest['dataset_version']}: \"\n f\"{manifest['total_positions']:,} positions across {len(manifest['shards'])} shards\")\n\ncopied = 0\nfor sh in manifest['shards']:\n dst = LOCAL_DATASETS / 'shards' / sh['file']\n if not dst.exists(): # cache: only copy shards we don't already have\n shutil.copy(DRIVE_DATASETS / 'shards' / sh['file'], dst)\n copied += 1\nshutil.copy(DRIVE_DATASETS / 'manifest.json', LOCAL_DATASETS / 'manifest.json')\n\nDATA_PATH = str(LOCAL_DATASETS) # train_nnue / burst_train read this dir of shards directly\nprint(f\"Synced {copied} new shard(s). Dataset ready at {DATA_PATH}\")",
"id": "data-paths"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"## 🏋️ 3 — Train\n",
"\n",
"Standard training runs a fixed number of epochs. \n",
"**Burst mode** is better for Colab: it repeatedly restarts from the best checkpoint within a time budget, surviving session disconnects gracefully."
],
"id": "train-md"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "from train import train_nnue, burst_train, DEFAULT_HIDDEN_SIZES\n\nWEIGHTS_DIR = Path(DRIVE_ROOT) / 'weights'\nWEIGHTS_DIR.mkdir(parents=True, exist_ok=True)\nOUTPUT_FILE = str(WEIGHTS_DIR / 'nnue_weights.pt')\n\n# ── Training hyperparameters ──────────────────────────────────────────────────\nHIDDEN_SIZES = DEFAULT_HIDDEN_SIZES\n# Features are streamed as sparse indices and densified on the GPU per batch, so\n# host RAM is no longer the limit — GPU memory is. A dense batch is\n# batch_size * 98304 * 4 bytes on the GPU (~3.2 GB at 8192 on a 16 GB T4).\nBATCH_SIZE = 8192\nEPOCHS = 100\nEARLY_STOPPING = 10 # None to disable\nSUBSAMPLE_RATIO = 1.0\n\n# Resume from latest checkpoint if one exists\ncheckpoints = sorted(WEIGHTS_DIR.glob('nnue_weights_v*.pt'))\nCHECKPOINT = str(checkpoints[-1]) if checkpoints else None\nif CHECKPOINT:\n print(f'Resuming from checkpoint: {CHECKPOINT}')\nelse:\n print('Starting training from scratch.')",
"id": "train-config"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "# ── Standard training ─────────────────────────────────────────────────────────\n# Use this when you have a reliable long-running session.\n\ntrain_nnue(\n data_file=DATA_PATH,\n output_file=OUTPUT_FILE,\n epochs=EPOCHS,\n batch_size=BATCH_SIZE,\n checkpoint=CHECKPOINT,\n use_versioning=True,\n early_stopping_patience=EARLY_STOPPING,\n subsample_ratio=SUBSAMPLE_RATIO,\n hidden_sizes=HIDDEN_SIZES,\n)",
"id": "standard-train"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": "# ── Burst training (recommended for Colab free tier) ─────────────────────────\n# Restarts from the global best each time early stopping fires.\n# Set BURST_MINUTES to slightly less than the Colab session limit (~70 min).\n\nBURST_MINUTES = 70\nEPOCHS_PER_SEASON = 30\nBURST_PATIENCE = 8\n\nburst_train(\n data_file=DATA_PATH,\n output_file=OUTPUT_FILE,\n duration_minutes=BURST_MINUTES,\n epochs_per_season=EPOCHS_PER_SEASON,\n early_stopping_patience=BURST_PATIENCE,\n batch_size=BATCH_SIZE,\n initial_checkpoint=CHECKPOINT,\n use_versioning=True,\n subsample_ratio=SUBSAMPLE_RATIO,\n hidden_sizes=HIDDEN_SIZES,\n)",
"id": "burst-train"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"## 📦 4 — Export\n",
"\n",
"Convert the best `.pt` checkpoint to the `.nbai` binary format read by `NbaiLoader` in Scala."
],
"id": "export-md"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from export import export_to_nbai\n",
"\n",
"NBAI_FILE = Path(DRIVE_ROOT) / 'nnue_weights.nbai'\n",
"\n",
"# Pick the latest versioned checkpoint\n",
"checkpoints = sorted(WEIGHTS_DIR.glob('nnue_weights_v*.pt'))\n",
"if not checkpoints:\n",
" raise FileNotFoundError('No checkpoints found in ' + str(WEIGHTS_DIR))\n",
"\n",
"latest = checkpoints[-1]\n",
"print(f'Exporting {latest.name} → {NBAI_FILE.name}')\n",
"\n",
"export_to_nbai(\n",
" weights_file=str(latest),\n",
" output_file=str(NBAI_FILE),\n",
" trained_by='colab',\n",
")\n",
"print('Export complete.')"
],
"id": "export-cell"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"## ⬇️ 5 — Download\n",
"\n",
"Download the `.nbai` weights file and the latest `.pt` checkpoint to your local machine.\n",
"\n",
"Place `nnue_weights.nbai` in `modules/official-bots/src/main/resources/` and rebuild the native image."
],
"id": "download-md"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from google.colab import files\n",
"\n",
"if NBAI_FILE.exists():\n",
" files.download(str(NBAI_FILE))\n",
" print(f'Downloading {NBAI_FILE.name}')\n",
"else:\n",
" print('No .nbai file found — run the Export cell first.')\n",
"\n",
"checkpoints = sorted(WEIGHTS_DIR.glob('nnue_weights_v*.pt'))\n",
"if checkpoints:\n",
" latest = checkpoints[-1]\n",
" files.download(str(latest))\n",
" print(f'Downloading checkpoint {latest.name}')\n",
"else:\n",
" print('No .pt checkpoint found.')"
],
"id": "download-cell"
}
]
}
@@ -0,0 +1,281 @@
#!/usr/bin/env python3
"""Build the ENTIRE NNUE training dataset with one command.
Orchestrates the existing source modules (Lichess eval DB, self-play, tactical puzzles,
random filler), labels what needs labeling with local Stockfish, deduplicates, balances
the eval distribution, writes append-only compressed shards + a manifest, and pushes to
Google Drive with rclone.
./dataset.sh # build everything + push
./dataset.sh --no-push # build only
./dataset.sh --no-lichess # skip the (large) Lichess backbone
Tune the CONFIG block below — that is the only thing you normally touch.
"""
import argparse
import hashlib
import json
import os
import random
import subprocess
import sys
import urllib.request
from datetime import datetime, timezone
from pathlib import Path
import zstandard as zstd
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE / "src"))
from generate import play_random_game_and_collect_positions
from label import label_positions_with_stockfish
from lichess_importer import import_lichess_evals
from tactical_positions_extractor import download_and_extract_puzzle_db, extract_tactical_only
# ── CONFIG — the only knobs you normally touch ───────────────────────────────
LICHESS_POSITIONS = 2_000_000 # backbone positions from the Lichess eval DB
USE_SELFPLAY = True # label data/selfplay.txt if present
TACTICAL_PUZZLES = 200_000 # tactical positions from the Lichess puzzle DB
RANDOM_FILLER = 100_000 # cheap random-play positions
STOCKFISH_DEPTH = 14 # local labeling depth (selfplay/tactical/random)
RCLONE_REMOTE = "gdrive:NowChess/datasets"
# ─────────────────────────────────────────────────────────────────────────────
LABEL_BATCH = 64 # positions per Stockfish task (small = smooth progress + load balance)
SHARD_SIZE = 100_000 # positions per shard
BALANCE_BINS = 64 # eval histogram bins over [-1, 1]
BALANCE_FACTOR = 2.0 # cap each bin at FACTOR x the uniform bin size
LICHESS_EVAL_URL = "https://database.lichess.org/lichess_db_eval.jsonl.zst"
STOCKFISH_PATH = os.environ.get("STOCKFISH_PATH", "/usr/games/stockfish")
WORKERS = os.cpu_count() or 4
DATA_DIR = HERE / "data"
WORK_DIR = HERE / "data" / "_build"
DATASETS_DIR = HERE / "datasets"
SHARDS_DIR = DATASETS_DIR / "shards"
MANIFEST = DATASETS_DIR / "manifest.json"
LICHESS_DB = HERE / "trainingdata" / "lichess_db_eval.jsonl.zst"
def label(fens_file: Path, out: Path) -> int:
"""Label a FEN file with local Stockfish. Returns positions written."""
if not fens_file.exists():
return 0
label_positions_with_stockfish(
str(fens_file), str(out), STOCKFISH_PATH,
batch_size=LABEL_BATCH, depth=STOCKFISH_DEPTH, num_workers=WORKERS,
)
return count_lines(out)
def count_lines(path: Path) -> int:
if not path.exists():
return 0
with open(path) as f:
return sum(1 for _ in f)
def source_lichess(out: Path) -> int:
if not LICHESS_DB.exists():
print(f"Downloading Lichess eval DB → {LICHESS_DB} (large, one-time)...")
LICHESS_DB.parent.mkdir(parents=True, exist_ok=True)
urllib.request.urlretrieve(LICHESS_EVAL_URL, LICHESS_DB)
return import_lichess_evals(str(LICHESS_DB), str(out), max_positions=LICHESS_POSITIONS)
def source_selfplay(out: Path) -> int:
return label(DATA_DIR / "selfplay.txt", out)
def source_tactical(out: Path) -> int:
puzzle_csv = download_and_extract_puzzle_db(output_dir=str(HERE / "tactical_data"))
if puzzle_csv is None:
return 0
fens = WORK_DIR / "tactical_fens.txt"
extract_tactical_only(str(puzzle_csv), str(fens), max_puzzles=TACTICAL_PUZZLES)
return label(fens, out)
def source_random(out: Path) -> int:
fens = WORK_DIR / "random_fens.txt"
play_random_game_and_collect_positions(
str(fens), total_positions=RANDOM_FILLER, num_workers=WORKERS,
)
return label(fens, out)
def build_sources(args) -> dict[str, Path]:
"""Run each enabled source into its own labeled jsonl. Returns {name: path}."""
WORK_DIR.mkdir(parents=True, exist_ok=True)
plan = [
("lichess", args.lichess, source_lichess),
("selfplay", args.selfplay, source_selfplay),
("tactical", args.tactical, source_tactical),
("random", args.random, source_random),
]
outputs: dict[str, Path] = {}
for name, enabled, fn in plan:
if not enabled:
continue
out = WORK_DIR / f"{name}_labeled.jsonl"
out.unlink(missing_ok=True)
print(f"\n=== Source: {name} ===")
written = fn(out)
print(f"{name}: {written:,} labeled positions")
if written:
outputs[name] = out
return outputs
def existing_fens() -> set[str]:
"""FENs already present in the dataset, so growth stays deduplicated."""
seen: set[str] = set()
if not MANIFEST.exists():
return seen
manifest = json.loads(MANIFEST.read_text())
for shard in manifest.get("shards", []):
for rec in read_shard(SHARDS_DIR / shard["file"]):
seen.add(rec["fen"])
return seen
def read_shard(path: Path):
dctx = zstd.ZstdDecompressor()
with open(path, "rb") as fh, dctx.stream_reader(fh) as reader:
for line in iter_text(reader):
yield json.loads(line)
def iter_text(reader):
import io
yield from io.TextIOWrapper(reader, encoding="utf-8")
def merge_dedup(outputs: dict[str, Path], skip: set[str]):
"""Merge all source jsonl, drop dupes (within batch + vs existing dataset)."""
seen = set(skip)
records, per_source = [], {}
for name, path in outputs.items():
kept = 0
with open(path) as f:
for line in f:
rec = json.loads(line)
fen = rec["fen"]
if fen in seen:
continue
seen.add(fen)
rec["source"] = name
records.append(rec)
kept += 1
per_source[name] = kept
return records, per_source
def balance(records: list) -> list:
"""Flatten the eval histogram: cap each bin at FACTOR x the uniform bin size."""
if not records:
return records
cap = max(1, int(BALANCE_FACTOR * len(records) / BALANCE_BINS))
bins: dict[int, int] = {}
kept = []
random.shuffle(records)
for rec in records:
b = min(BALANCE_BINS - 1, int((rec["eval"] + 1.0) / 2.0 * BALANCE_BINS))
if bins.get(b, 0) < cap:
bins[b] = bins.get(b, 0) + 1
kept.append(rec)
return kept
def sha256(path: Path) -> str:
h = hashlib.sha256()
with open(path, "rb") as f:
for chunk in iter(lambda: f.read(1 << 20), b""):
h.update(chunk)
return h.hexdigest()
def write_shards(records: list, build_id: str) -> list[dict]:
SHARDS_DIR.mkdir(parents=True, exist_ok=True)
cctx = zstd.ZstdCompressor(level=10)
entries = []
for i in range(0, len(records), SHARD_SIZE):
chunk = records[i : i + SHARD_SIZE]
name = f"{build_id}_{i // SHARD_SIZE:05d}.jsonl.zst"
path = SHARDS_DIR / name
with open(path, "wb") as fh, cctx.stream_writer(fh) as w:
for rec in chunk:
w.write((json.dumps(rec) + "\n").encode("utf-8"))
entries.append({"file": name, "positions": len(chunk),
"sha256": sha256(path), "build_id": build_id})
print(f" wrote {name} ({len(chunk):,} positions)")
return entries
def update_manifest(new_shards: list[dict], build: dict) -> None:
manifest = json.loads(MANIFEST.read_text()) if MANIFEST.exists() else {
"dataset_version": 0, "scale": 300.0, "builds": [], "shards": [],
}
manifest["dataset_version"] += 1
manifest["created"] = build["created"]
manifest["builds"].append(build)
manifest["shards"].extend(new_shards)
manifest["total_positions"] = sum(s["positions"] for s in manifest["shards"])
MANIFEST.write_text(json.dumps(manifest, indent=2))
print(f"\nDataset version {manifest['dataset_version']}: "
f"{manifest['total_positions']:,} total positions across {len(manifest['shards'])} shards")
def push() -> None:
if not subprocess.run(["which", "rclone"], capture_output=True).stdout:
print("rclone not found — skipping push.")
return
print(f"\nPushing {DATASETS_DIR}{RCLONE_REMOTE} ...")
subprocess.run(["rclone", "copy", str(DATASETS_DIR), RCLONE_REMOTE, "--progress"], check=True)
def parse_args():
p = argparse.ArgumentParser(description="Build the entire NNUE dataset.")
for name in ("lichess", "selfplay", "tactical", "random", "push"):
p.add_argument(f"--no-{name}", dest=name, action="store_false")
p.add_argument("--push-only", action="store_true", help="Push the existing dataset, build nothing.")
return p.parse_args()
def main() -> None:
args = parse_args()
if args.push_only:
push()
return
build_id = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
outputs = build_sources(args)
if not outputs:
print("No sources produced data — nothing to build.")
return
print("\n=== Merge / dedup / balance ===")
records, per_source = merge_dedup(outputs, existing_fens())
print(f"merged unique (new): {len(records):,}")
records = balance(records)
print(f"after balancing: {len(records):,}")
new_shards = write_shards(records, build_id)
update_manifest(new_shards, {
"build_id": build_id,
"created": datetime.now(timezone.utc).isoformat(),
"stockfish_depth": STOCKFISH_DEPTH,
"sources": per_source,
"kept_after_balance": len(records),
})
if args.push:
push()
print("\nDone.")
if __name__ == "__main__":
main()
+17
View File
@@ -0,0 +1,17 @@
#!/usr/bin/env bash
# Build the ENTIRE NNUE training dataset + push to Drive. One command.
#
# ./dataset.sh # build everything + rclone push
# ./dataset.sh --no-push # build only
# ./dataset.sh --no-lichess # skip the large Lichess backbone
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
cd "$SCRIPT_DIR"
PY="python3"
if [[ -x "$SCRIPT_DIR/.venv/bin/python" ]]; then
PY="$SCRIPT_DIR/.venv/bin/python"
fi
exec "$PY" build_dataset.py "$@"
+23
View File
@@ -0,0 +1,23 @@
#!/usr/bin/env bash
# Standalone bot self-play -> FENs for labeling. No microservices.
#
# ./selfplay.sh # 500 games with the bundled net
# ./selfplay.sh 2000 # more games
# ./selfplay.sh 2000 path.nbai # play with a specific net
set -euo pipefail
GAMES="${1:-500}"
WEIGHTS="${2:-}"
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
OUT="$SCRIPT_DIR/data/selfplay.txt"
cd "$REPO_ROOT"
SP_ARGS="--games $GAMES --out $OUT"
if [[ -n "$WEIGHTS" ]]; then
SP_ARGS="$SP_ARGS --weights $WEIGHTS"
fi
./gradlew -q :modules:official-bots:selfPlay -PspArgs="$SP_ARGS"
echo "Self-play FENs -> $OUT"
+119 -34
View File
@@ -13,6 +13,38 @@ import chess
from datetime import datetime, timedelta from datetime import datetime, timedelta
import re import re
import numpy as np import numpy as np
import os
# DataLoader workers: cap to the machine's CPUs (Colab free tier = 2). Too many
# workers each fork the dataset and OOM-kill the runtime.
LOADER_WORKERS = int(os.environ.get("NNUE_LOADER_WORKERS", min(4, os.cpu_count() or 2)))
def _shard_files(data_file):
"""Resolve a data path to a list of shard files. Accepts a single .jsonl/.jsonl.zst
file, or a directory (searched recursively for shards, e.g. a synced datasets/ dir)."""
p = Path(data_file)
if p.is_dir():
shards = sorted(p.rglob("*.jsonl.zst")) or sorted(p.rglob("*.jsonl"))
if not shards:
raise FileNotFoundError(f"No .jsonl/.jsonl.zst shards found under {p}")
print(f"Loading {len(shards)} shard(s) from {p}")
return shards
return [p]
def _iter_dataset_lines(data_file):
"""Yield text lines from every shard, transparently decompressing .zst shards."""
import io
for shard in _shard_files(data_file):
if str(shard).endswith(".zst"):
import zstandard as zstd
with open(shard, "rb") as fh, zstd.ZstdDecompressor().stream_reader(fh) as reader:
yield from io.TextIOWrapper(reader, encoding="utf-8")
else:
with open(shard, "r") as fh:
yield from fh
class NNUEDataset(Dataset): class NNUEDataset(Dataset):
"""Dataset of chess positions with evaluations.""" """Dataset of chess positions with evaluations."""
@@ -23,8 +55,7 @@ class NNUEDataset(Dataset):
self.evals_raw = [] self.evals_raw = []
self.is_normalized = None self.is_normalized = None
with open(data_file, 'r') as f: for line in _iter_dataset_lines(data_file):
for line in f:
try: try:
data = json.loads(line) data = json.loads(line)
fen = data['fen'] fen = data['fen']
@@ -51,7 +82,15 @@ class NNUEDataset(Dataset):
def __getitem__(self, idx): def __getitem__(self, idx):
fen = self.positions[idx] fen = self.positions[idx]
eval_val = self.evals[idx] eval_val = self.evals[idx]
features = fen_to_features(fen) # Return only the active feature indices (~64), not a dense 98304-dim vector.
# The training loop scatters these into a dense batch on the GPU, keeping host
# RAM tiny. Densifying per-item here OOM-kills the runtime.
indices = fen_to_indices(fen)
# Board is flipped for Black-to-move in fen_to_indices; negate eval
# so the label still means "good for the side shown as White after flip"
if ' b ' in fen:
eval_val = -eval_val
# Use evaluation as-is if normalized, otherwise apply sigmoid scaling # Use evaluation as-is if normalized, otherwise apply sigmoid scaling
if self.is_normalized: if self.is_normalized:
@@ -59,40 +98,82 @@ class NNUEDataset(Dataset):
else: else:
target = torch.sigmoid(torch.tensor(eval_val / 400.0, dtype=torch.float32)) target = torch.sigmoid(torch.tensor(eval_val / 400.0, dtype=torch.float32))
return features, target return indices, target
def fen_to_features(fen): # King-relative (HalfKP) encoding: two perspectives, one per side's king.
"""Convert FEN to 768-dimensional binary feature vector.""" # Each piece is encoded as: kingSq * 768 + pieceIdx * 64 + sq
# Piece type to index: pawn=0, knight=1, bishop=2, rook=3, queen=4, king=5 # White perspective uses white king square; black perspective uses black king square.
piece_to_idx = {'p': 0, 'n': 1, 'b': 2, 'r': 3, 'q': 4, 'k': 5, # Total input dimension = 2 × 64 × 12 × 64 = 98304.
'P': 6, 'N': 7, 'B': 8, 'R': 9, 'Q': 10, 'K': 11} _HALF_SIZE = 64 * 12 * 64 # 49152 features per perspective
INPUT_SIZE = _HALF_SIZE * 2 # 98304
features = torch.zeros(768, dtype=torch.float32) _PIECE_TO_IDX = {
'p': 0, 'n': 1, 'b': 2, 'r': 3, 'q': 4, 'k': 5,
'P': 6, 'N': 7, 'B': 8, 'R': 9, 'Q': 10, 'K': 11,
}
def fen_to_indices(fen):
"""Active king-relative (HalfKP) feature indices for a FEN (~64 entries).
For Black-to-move positions the board is mirrored (ranks flipped, colours
swapped) so the network always sees the position from the side-to-move's
perspective. The caller is responsible for negating the eval label to match.
"""
indices = []
try: try:
board = chess.Board(fen) board = chess.Board(fen)
# Perspective flip: present all positions as if White is to move
if board.turn == chess.BLACK:
board = board.mirror()
wk = board.king(chess.WHITE)
bk = board.king(chess.BLACK)
if wk is None or bk is None:
return torch.zeros(0, dtype=torch.long)
for sq in chess.SQUARES:
piece = board.piece_at(sq)
if piece is None:
continue
pidx = _PIECE_TO_IDX[piece.symbol()]
# White-king perspective (indices 0 .. _HALF_SIZE-1)
indices.append(wk * 768 + pidx * 64 + sq)
# Black-king perspective (indices _HALF_SIZE .. INPUT_SIZE-1)
indices.append(_HALF_SIZE + bk * 768 + pidx * 64 + sq)
except Exception:
return torch.zeros(0, dtype=torch.long)
return torch.tensor(indices, dtype=torch.long)
# 12 piece types × 64 squares = 768
for square in chess.SQUARES:
piece = board.piece_at(square)
if piece is not None:
piece_char = piece.symbol()
if piece_char in piece_to_idx:
piece_idx = piece_to_idx[piece_char]
feature_idx = piece_idx * 64 + square
features[feature_idx] = 1.0
except:
pass
def fen_to_features(fen):
"""Dense 98304-dim HalfKP vector. Kept for external callers; training uses the
sparse indices + GPU scatter path instead (see _collate_sparse)."""
features = torch.zeros(INPUT_SIZE, dtype=torch.float32)
features[fen_to_indices(fen)] = 1.0
return features return features
DEFAULT_HIDDEN_SIZES = [1536, 1024, 512, 256]
def _collate_sparse(batch):
"""Collate (indices, target) items into (row_idx, col_idx, batch_size), targets.
Row/col index pairs address the active features of a dense [B, INPUT_SIZE] tensor
that the training loop allocates on the GPU — so the host only ever holds the
sparse indices, never a dense batch."""
idx_list, targets = zip(*batch)
rows = torch.cat([torch.full((idx.numel(),), i, dtype=torch.long)
for i, idx in enumerate(idx_list)])
cols = torch.cat(idx_list)
return (rows, cols, len(idx_list)), torch.stack(targets)
# Smaller hidden layers are appropriate: the L1 input is very sparse (~64 active
# features out of 98304) so the L1 itself is cheap to update incrementally; the
# larger capacity comes from the wider perspective encoding, not deeper layers.
DEFAULT_HIDDEN_SIZES = [512, 256, 128]
class NNUE(nn.Module): class NNUE(nn.Module):
"""NNUE neural network with configurable hidden layers. """NNUE neural network with configurable hidden layers.
Architecture: 768 → hidden_sizes[0] → ... → hidden_sizes[-1] → 1 Architecture: INPUT_SIZE → hidden_sizes[0] → ... → hidden_sizes[-1] → 1
Layer attributes follow the naming l1, l2, ..., lN so export.py can Layer attributes follow the naming l1, l2, ..., lN so export.py can
infer the architecture directly from the state_dict. infer the architecture directly from the state_dict.
""" """
@@ -102,7 +183,7 @@ class NNUE(nn.Module):
if hidden_sizes is None: if hidden_sizes is None:
hidden_sizes = DEFAULT_HIDDEN_SIZES hidden_sizes = DEFAULT_HIDDEN_SIZES
self.hidden_sizes = list(hidden_sizes) self.hidden_sizes = list(hidden_sizes)
sizes = [768] + self.hidden_sizes + [1] sizes = [INPUT_SIZE] + self.hidden_sizes + [1]
num_hidden = len(self.hidden_sizes) num_hidden = len(self.hidden_sizes)
for i in range(num_hidden): for i in range(num_hidden):
@@ -204,17 +285,19 @@ def _setup_training(data_file, batch_size, subsample_ratio):
train_dataset, train_dataset,
batch_size=batch_size, batch_size=batch_size,
sampler=train_sampler, sampler=train_sampler,
num_workers=8, num_workers=LOADER_WORKERS,
pin_memory=True, pin_memory=True,
persistent_workers=True persistent_workers=LOADER_WORKERS > 0,
collate_fn=_collate_sparse,
) )
val_loader = DataLoader( val_loader = DataLoader(
val_dataset, val_dataset,
batch_size=batch_size, batch_size=batch_size,
shuffle=False, shuffle=False,
num_workers=8, num_workers=LOADER_WORKERS,
pin_memory=True, pin_memory=True,
persistent_workers=True persistent_workers=LOADER_WORKERS > 0,
collate_fn=_collate_sparse,
) )
return device, dataset, train_dataset, val_dataset, train_loader, val_loader, num_positions return device, dataset, train_dataset, val_dataset, train_loader, val_loader, num_positions
@@ -252,8 +335,9 @@ def _run_training_season(
model.train() model.train()
train_loss = 0.0 train_loss = 0.0
with tqdm(total=len(train_loader), desc=f"Epoch {epoch_display}/{total_epochs} - Train") as pbar: with tqdm(total=len(train_loader), desc=f"Epoch {epoch_display}/{total_epochs} - Train") as pbar:
for batch_features, batch_targets in train_loader: for (rows, cols, bsz), batch_targets in train_loader:
batch_features = batch_features.to(device) batch_features = torch.zeros(bsz, INPUT_SIZE, device=device)
batch_features[rows.to(device), cols.to(device)] = 1.0
batch_targets = batch_targets.to(device).unsqueeze(1) batch_targets = batch_targets.to(device).unsqueeze(1)
optimizer.zero_grad() optimizer.zero_grad()
@@ -266,7 +350,7 @@ def _run_training_season(
scaler.step(optimizer) scaler.step(optimizer)
scaler.update() scaler.update()
train_loss += loss.item() * batch_features.size(0) train_loss += loss.item() * bsz
pbar.update(1) pbar.update(1)
train_loss /= len(train_dataset) train_loss /= len(train_dataset)
@@ -276,14 +360,15 @@ def _run_training_season(
val_loss = 0.0 val_loss = 0.0
with torch.no_grad(): with torch.no_grad():
with tqdm(total=len(val_loader), desc=f"Epoch {epoch_display}/{total_epochs} - Val") as pbar: with tqdm(total=len(val_loader), desc=f"Epoch {epoch_display}/{total_epochs} - Val") as pbar:
for batch_features, batch_targets in val_loader: for (rows, cols, bsz), batch_targets in val_loader:
batch_features = batch_features.to(device) batch_features = torch.zeros(bsz, INPUT_SIZE, device=device)
batch_features[rows.to(device), cols.to(device)] = 1.0
batch_targets = batch_targets.to(device).unsqueeze(1) batch_targets = batch_targets.to(device).unsqueeze(1)
with torch.amp.autocast('cuda' if torch.cuda.is_available() else 'cpu'): with torch.amp.autocast('cuda' if torch.cuda.is_available() else 'cpu'):
outputs = model(batch_features) outputs = model(batch_features)
loss = criterion(outputs, batch_targets) loss = criterion(outputs, batch_targets)
val_loss += loss.item() * batch_features.size(0) val_loss += loss.item() * bsz
pbar.update(1) pbar.update(1)
val_loss /= len(val_dataset) val_loss /= len(val_dataset)
@@ -3,4 +3,33 @@ package de.nowchess.bot
import de.nowchess.api.game.GameContext import de.nowchess.api.game.GameContext
import de.nowchess.api.move.Move import de.nowchess.api.move.Move
type Bot = GameContext => Option[Move] /** Remaining wall-clock for the side to move and the Fischer increment, both in milliseconds. [[TimeControl.Unlimited]]
* is the sentinel for callers without a real clock (local play, self-play, tests): bots then fall back to their own
* fixed budgets.
*/
final case class TimeControl(remainingMs: Long, incrementMs: Long):
def isClocked: Boolean = remainingMs >= 0L
def budgetMs: Long = TimeControl.budget(remainingMs, incrementMs)
object TimeControl:
val Unlimited: TimeControl = TimeControl(-1L, 0L)
private val OverheadMs = 1500L
private val PanicMs = 20000L
private val MaxBudget = 8000L
private val PanicCap = 2500L
private val FloorMs = 50L
def budget(remainingMs: Long, incrementMs: Long): Long =
val usable = math.max(0L, remainingMs - OverheadMs)
if usable <= 0L then FloorMs
else if remainingMs < PanicMs then clamp(usable / 15 + incrementMs / 2, PanicCap)
else clamp(usable / 30 + incrementMs * 4 / 5, MaxBudget)
private def clamp(value: Long, ceiling: Long): Long =
math.max(FloorMs, math.min(value, ceiling))
trait Bot:
def move(context: GameContext, time: TimeControl): Option[Move]
def apply(context: GameContext): Option[Move] = move(context, TimeControl.Unlimited)
def apply(context: GameContext, time: TimeControl): Option[Move] = move(context, time)
@@ -1,17 +1,20 @@
package de.nowchess.bot package de.nowchess.bot
import de.nowchess.bot.bots.{ClassicalBot, HybridBot} import de.nowchess.bot.bots.{ClassicalBot, HybridBot}
import de.nowchess.bot.util.PolyglotBook
import jakarta.enterprise.context.ApplicationScoped import jakarta.enterprise.context.ApplicationScoped
import org.jboss.logging.Logger import org.jboss.logging.Logger
object BotController: object BotController:
private val log = Logger.getLogger(classOf[BotController]) private val log = Logger.getLogger(classOf[BotController])
private val openingBook = PolyglotBook.fromResource("/opening_book.bin")
private val bots: Map[String, Bot] = Map( private val bots: Map[String, Bot] = Map(
"easy" -> ClassicalBot(BotDifficulty.Easy), "easy" -> ClassicalBot(BotDifficulty.Easy),
"medium" -> ClassicalBot(BotDifficulty.Medium), "medium" -> ClassicalBot(BotDifficulty.Medium),
"hard" -> ClassicalBot(BotDifficulty.Hard), "hard" -> ClassicalBot(BotDifficulty.Hard),
"expert" -> HybridBot(BotDifficulty.Expert, vetoReporter = log.debug(_)), "expert" -> HybridBot(BotDifficulty.Expert, vetoReporter = log.debug(_), book = Some(openingBook)),
) )
def getBot(name: String): Option[Bot] = bots.get(name.toLowerCase) def getBot(name: String): Option[Bot] = bots.get(name.toLowerCase)
@@ -1,25 +1,28 @@
package de.nowchess.bot.bots package de.nowchess.bot.bots
import de.nowchess.bot.Bot
import de.nowchess.api.game.GameContext import de.nowchess.api.game.GameContext
import de.nowchess.api.move.Move
import de.nowchess.api.rules.RuleSet import de.nowchess.api.rules.RuleSet
import de.nowchess.bot.bots.classic.EvaluationClassic import de.nowchess.bot.bots.classic.EvaluationClassic
import de.nowchess.bot.logic.AlphaBetaSearch import de.nowchess.bot.logic.AlphaBetaSearch
import de.nowchess.bot.util.PolyglotBook import de.nowchess.bot.util.PolyglotBook
import de.nowchess.bot.{BotDifficulty, BotMoveRepetition} import de.nowchess.bot.{Bot, BotDifficulty, BotMoveRepetition, TimeControl}
import de.nowchess.rules.sets.DefaultRules import de.nowchess.rules.sets.DefaultRules
object ClassicalBot: object ClassicalBot:
private val defaultBudgetMs = 1000L
def apply( def apply(
difficulty: BotDifficulty, difficulty: BotDifficulty,
rules: RuleSet = DefaultRules, rules: RuleSet = DefaultRules,
book: Option[PolyglotBook] = None, book: Option[PolyglotBook] = None,
): Bot = ): Bot =
val search = AlphaBetaSearch(rules, weights = EvaluationClassic) val search = AlphaBetaSearch(rules, weights = EvaluationClassic)
val timeBudgetMs = 1000L new Bot:
context => def move(context: GameContext, time: TimeControl): Option[Move] =
val budget = if time.isClocked then time.budgetMs else defaultBudgetMs
val blockedMoves = BotMoveRepetition.blockedMoves(context) val blockedMoves = BotMoveRepetition.blockedMoves(context)
book book
.flatMap(_.probe(context)) .flatMap(_.probe(context))
.filterNot(blockedMoves.contains) .filterNot(blockedMoves.contains)
.orElse(search.bestMoveWithTime(context, timeBudgetMs, blockedMoves)) .orElse(search.bestMoveWithTime(context, budget, blockedMoves))
@@ -7,12 +7,20 @@ import de.nowchess.api.rules.RuleSet
import de.nowchess.bot.ai.Evaluation import de.nowchess.bot.ai.Evaluation
import de.nowchess.bot.bots.classic.EvaluationClassic import de.nowchess.bot.bots.classic.EvaluationClassic
import de.nowchess.bot.bots.nnue.EvaluationNNUE import de.nowchess.bot.bots.nnue.EvaluationNNUE
import de.nowchess.bot.logic.{AlphaBetaSearch, TranspositionTable} import de.nowchess.bot.logic.{ParallelSearch, TranspositionTable}
import de.nowchess.bot.util.PolyglotBook import de.nowchess.bot.util.PolyglotBook
import de.nowchess.bot.{BotDifficulty, BotMoveRepetition, Config} import de.nowchess.bot.{Bot, BotDifficulty, BotMoveRepetition, Config, TimeControl}
import de.nowchess.rules.sets.DefaultRules import de.nowchess.rules.sets.DefaultRules
object HybridBot: object HybridBot:
private def defaultThreads: Int =
sys.env.get("NNUE_SEARCH_THREADS").flatMap(_.toIntOption).filter(_ >= 1).getOrElse(1)
// The veto re-search must share the move's budget, not double it: give the main search the bulk and
// reserve a slice for the at-most-one veto re-search so a vetoed move never costs two full budgets.
private val MainSearchShare = 0.7
def apply( def apply(
difficulty: BotDifficulty, difficulty: BotDifficulty,
rules: RuleSet = DefaultRules, rules: RuleSet = DefaultRules,
@@ -20,28 +28,34 @@ object HybridBot:
nnueEvaluation: Evaluation = EvaluationNNUE, nnueEvaluation: Evaluation = EvaluationNNUE,
classicalEvaluation: Evaluation = EvaluationClassic, classicalEvaluation: Evaluation = EvaluationClassic,
vetoReporter: String => Unit = println(_), vetoReporter: String => Unit = println(_),
searchThreads: Int = defaultThreads,
): Bot = ): Bot =
val search = AlphaBetaSearch(rules, TranspositionTable(), classicalEvaluation) // Use ParallelSearch to enable multi-threaded (SMP) search similar to NNUEBot
context => val search = ParallelSearch(rules, TranspositionTable(), () => classicalEvaluation, searchThreads)
new Bot:
def move(context: GameContext, time: TimeControl): Option[Move] =
val totalBudget = if time.isClocked then time.budgetMs else Config.TIME_LIMIT_MS
val mainBudget = math.max(1L, (totalBudget * MainSearchShare).toLong)
val vetoBudget = math.max(1L, totalBudget - mainBudget)
val blockedMoves = BotMoveRepetition.blockedMoves(context) val blockedMoves = BotMoveRepetition.blockedMoves(context)
def nnueScore(move: Move): Int = nnueEvaluation.evaluate(rules.applyMove(context)(move)) def nnueScore(m: Move): Int = nnueEvaluation.evaluate(rules.applyMove(context)(m))
def classicalScore(move: Move): Int = classicalEvaluation.evaluate(rules.applyMove(context)(move)) def classicalScore(m: Move): Int = classicalEvaluation.evaluate(rules.applyMove(context)(m))
def refine(move: Move): Move = def refine(m: Move): Move =
val moveNnue = nnueScore(move) val moveNnue = nnueScore(m)
if (classicalScore(move) - moveNnue).abs <= Config.VETO_THRESHOLD then move if (classicalScore(m) - moveNnue).abs <= Config.VETO_THRESHOLD then m
else else
search search
.bestMoveWithTime(context, Config.TIME_LIMIT_MS, blockedMoves + move) .bestMoveWithTime(context, vetoBudget, blockedMoves + m)
.filterNot(blockedMoves.contains) .filterNot(blockedMoves.contains)
.filter(alt => nnueScore(alt) < moveNnue) .filter(alt => nnueScore(alt) < moveNnue)
.map { alt => .map { alt =>
vetoReporter(f"[Veto] ${move.from}->${move.to} replaced by ${alt.from}->${alt.to} — NNUE prefers it") vetoReporter(f"[Veto] ${m.from}->${m.to} replaced by ${alt.from}->${alt.to} — NNUE prefers it")
alt alt
} }
.getOrElse(move) .getOrElse(m)
book.flatMap(_.probe(context)).filterNot(blockedMoves.contains).orElse { book.flatMap(_.probe(context)).filterNot(blockedMoves.contains).orElse {
search.bestMoveWithTime(context, Config.TIME_LIMIT_MS, blockedMoves).map(refine) search.bestMoveWithTime(context, mainBudget, blockedMoves).map(refine)
} }
@@ -1,23 +1,28 @@
package de.nowchess.bot.bots package de.nowchess.bot.bots
import de.nowchess.bot.Bot
import de.nowchess.api.game.GameContext import de.nowchess.api.game.GameContext
import de.nowchess.api.move.Move import de.nowchess.api.move.Move
import de.nowchess.api.rules.RuleSet import de.nowchess.api.rules.RuleSet
import de.nowchess.bot.bots.nnue.EvaluationNNUE import de.nowchess.bot.bots.nnue.EvaluationNNUE
import de.nowchess.bot.logic.AlphaBetaSearch import de.nowchess.bot.logic.{ParallelSearch, TranspositionTable}
import de.nowchess.bot.util.{PolyglotBook, ZobristHash} import de.nowchess.bot.util.{PolyglotBook, ZobristHash}
import de.nowchess.bot.{BotDifficulty, BotMoveRepetition} import de.nowchess.bot.{Bot, BotDifficulty, BotMoveRepetition, TimeControl}
import de.nowchess.rules.sets.DefaultRules import de.nowchess.rules.sets.DefaultRules
object NNUEBot: object NNUEBot:
private def defaultThreads: Int =
sys.env.get("NNUE_SEARCH_THREADS").flatMap(_.toIntOption).filter(_ >= 1).getOrElse(1)
def apply( def apply(
difficulty: BotDifficulty, difficulty: BotDifficulty,
rules: RuleSet = DefaultRules, rules: RuleSet = DefaultRules,
book: Option[PolyglotBook] = None, book: Option[PolyglotBook] = None,
fixedMoveTimeMs: Option[Long] = None,
searchThreads: Int = defaultThreads,
): Bot = ): Bot =
val search = AlphaBetaSearch(rules, weights = EvaluationNNUE) val search = ParallelSearch(rules, TranspositionTable(), () => EvaluationNNUE.freshEvaluator(), searchThreads)
context => new Bot:
def move(context: GameContext, time: TimeControl): Option[Move] =
val blockedMoves = BotMoveRepetition.blockedMoves(context) val blockedMoves = BotMoveRepetition.blockedMoves(context)
book book
.flatMap(_.probe(context)) .flatMap(_.probe(context))
@@ -28,7 +33,8 @@ object NNUEBot:
else else
val scored = batchEvaluateRoot(rules, context, moves) val scored = batchEvaluateRoot(rules, context, moves)
val bestMove = scored.maxBy(_._2)._1 val bestMove = scored.maxBy(_._2)._1
search.bestMoveWithTime(context, allocateTime(scored), blockedMoves).orElse(Some(bestMove)) val budget = fixedMoveTimeMs.getOrElse(if time.isClocked then time.budgetMs else allocateTime(scored))
search.bestMoveWithTime(context, budget, blockedMoves, scored.toMap).orElse(Some(bestMove))
} }
private def batchEvaluateRoot(rules: RuleSet, context: GameContext, moves: List[Move]): List[(Move, Int)] = private def batchEvaluateRoot(rules: RuleSet, context: GameContext, moves: List[Move]): List[(Move, Int)] =
@@ -4,15 +4,17 @@ import de.nowchess.api.game.GameContext
import de.nowchess.api.move.Move import de.nowchess.api.move.Move
import de.nowchess.bot.ai.Evaluation import de.nowchess.bot.ai.Evaluation
object EvaluationNNUE extends Evaluation: /** One independent NNUE evaluator: wraps its own [[NNUE]] (own accumulator stack, scratch buffers and eval cache) plus
* the endgame mop-up correction. Independent instances may run concurrently as long as they share only the read-only
private val nnue = NNUE(NbaiLoader.loadDefault()) * [[NNUEWeights]].
*/
final class NNUEEvaluator(nnue: NNUE) extends Evaluation:
val CHECKMATE_SCORE: Int = 10_000_000 val CHECKMATE_SCORE: Int = 10_000_000
val DRAW_SCORE: Int = 0 val DRAW_SCORE: Int = 0
/** Full-board evaluate — used as fallback and by non-search callers. */ /** Full-board evaluate — used as fallback and by non-search callers. */
def evaluate(context: GameContext): Int = nnue.evaluate(context) def evaluate(context: GameContext): Int = nnue.evaluate(context) + MopUp.score(context)
// ── Accumulator hooks (incremental L1) ─────────────────────────────────── // ── Accumulator hooks (incremental L1) ───────────────────────────────────
@@ -23,9 +25,34 @@ object EvaluationNNUE extends Evaluation:
nnue.copyAccumulator(parentPly, childPly) nnue.copyAccumulator(parentPly, childPly)
override def pushAccumulator(childPly: Int, move: Move, parent: GameContext, child: GameContext): Unit = override def pushAccumulator(childPly: Int, move: Move, parent: GameContext, child: GameContext): Unit =
// Use incremental updates, but recompute from scratch every 10 plies to prevent accumulation errors // Recompute every 10 plies to prevent floating-point drift; king moves always recompute internally
if childPly % 10 == 0 then nnue.recomputeAccumulator(childPly, child.board) if childPly % 10 == 0 then nnue.recomputeAccumulator(childPly, child.board)
else nnue.pushAccumulator(childPly, move, parent.board) else nnue.pushAccumulator(childPly, move, parent.board, child.board)
override def evaluateAccumulator(ply: Int, context: GameContext, hash: Long): Int = override def evaluateAccumulator(ply: Int, context: GameContext, hash: Long): Int =
nnue.evaluateAtPlyWithValidation(ply, context.turn, hash, context.board) nnue.evaluateAtPlyWithValidation(ply, context.turn, hash, context.board) + MopUp.score(context)
/** Default singleton evaluator plus a factory for independent per-thread evaluators that share the loaded weights. */
object EvaluationNNUE extends Evaluation:
private val weights = NNUEWeights(NbaiLoader.loadDefault())
private val default = NNUEEvaluator(NNUE(weights))
/** Build a fresh evaluator backed by its own [[NNUE]] but sharing the immutable [[weights]] — one per search thread.
*/
def freshEvaluator(): Evaluation = NNUEEvaluator(NNUE(weights))
val CHECKMATE_SCORE: Int = default.CHECKMATE_SCORE
val DRAW_SCORE: Int = default.DRAW_SCORE
def evaluate(context: GameContext): Int = default.evaluate(context)
override def initAccumulator(context: GameContext): Unit = default.initAccumulator(context)
override def copyAccumulator(parentPly: Int, childPly: Int): Unit = default.copyAccumulator(parentPly, childPly)
override def pushAccumulator(childPly: Int, move: Move, parent: GameContext, child: GameContext): Unit =
default.pushAccumulator(childPly, move, parent, child)
override def evaluateAccumulator(ply: Int, context: GameContext, hash: Long): Int =
default.evaluateAccumulator(ply, context, hash)
@@ -0,0 +1,61 @@
package de.nowchess.bot.bots.nnue
import de.nowchess.api.board.{Color, PieceType, Square}
import de.nowchess.api.game.GameContext
/** Endgame "mop-up" correction for the NNUE evaluation.
*
* Pure NNUE lacks explicit mating knowledge, so KX-vs-K conversions stall. When one side is reduced to a lone king and
* the other holds sufficient mating material, this term rewards driving the bare king to the edge and walking the
* winning king in. Returns a value from the side-to-move perspective (positive = good for side to move). Zero in any
* position that is not a lone-king endgame, so middlegame NNUE output is untouched.
*/
object MopUp:
private val EDGE_WEIGHT = 10
private val PROXIMITY_WEIGHT = 4
private val MIN_WINNER_VALUE = 400
def score(context: GameContext): Int =
loneKingColor(context) match
case None => 0
case Some(loser) =>
val winner = loser.opposite
if winnerValue(context, winner) < MIN_WINNER_VALUE then 0
else mopUp(context, winner, loser) * (if context.turn == winner then 1 else -1)
private def mopUp(context: GameContext, winner: Color, loser: Color): Int =
(for
loserKing <- context.kingSquare(loser)
winnerKing <- context.kingSquare(winner)
yield EDGE_WEIGHT * centerDistance(loserKing) +
PROXIMITY_WEIGHT * (14 - kingDistance(winnerKing, loserKing))).getOrElse(0)
private def loneKingColor(context: GameContext): Option[Color] =
val nonKing = context.board.pieces.values.filter(_.pieceType != PieceType.King)
val whiteHasOther = nonKing.exists(_.color == Color.White)
val blackHasOther = nonKing.exists(_.color == Color.Black)
if whiteHasOther == blackHasOther then None
else if whiteHasOther then Some(Color.Black)
else Some(Color.White)
private def winnerValue(context: GameContext, winner: Color): Int =
context.board.pieces.values.foldLeft(0) { (sum, piece) =>
if piece.color != winner then sum
else
sum + (piece.pieceType match
case PieceType.Queen => 900
case PieceType.Rook => 500
case PieceType.Bishop => 330
case PieceType.Knight => 320
case _ => 0
)
}
private def centerDistance(sq: Square): Int =
val fileDist = math.max(3 - sq.file.ordinal, sq.file.ordinal - 4)
val rankDist = math.max(3 - sq.rank.ordinal, sq.rank.ordinal - 4)
fileDist + rankDist
private def kingDistance(a: Square, b: Square): Int =
(a.file.ordinal - b.file.ordinal).abs + (a.rank.ordinal - b.rank.ordinal).abs
@@ -1,29 +1,28 @@
package de.nowchess.bot.bots.nnue package de.nowchess.bot.bots.nnue
import de.nowchess.api.board.{Board, Color, File, Piece, PieceType, Square} import de.nowchess.api.board.{Board, Color, Piece, PieceType, Square}
import de.nowchess.api.game.GameContext import de.nowchess.api.game.GameContext
import de.nowchess.api.move.{Move, MoveType, PromotionPiece} import de.nowchess.api.move.{Move, MoveType, PromotionPiece}
class NNUE(model: NbaiModel): object NNUE:
def apply(model: NbaiModel): NNUE = new NNUE(NNUEWeights(model))
def apply(weights: NNUEWeights): NNUE = new NNUE(weights)
private val featureSize = model.layers(0).inputSize /** Per-thread NNUE evaluator: owns the mutable accumulator stack, scratch buffers and eval cache, while sharing the
private val accSize = model.layers(0).outputSize * read-only [[NNUEWeights]]. Construct one instance per search thread (cheap only buffer allocation); they may all
private val validateAccum = sys.env.contains("NNUE_VALIDATE") // Enable with NNUE_VALIDATE=1 * share a single weights instance.
*/
class NNUE(weights: NNUEWeights):
// Column-major L1 weights for cache-friendly sparse & incremental updates. import weights.{accSize, l1WeightsT, model, HALF_SIZE}
// l1WeightsT(featureIdx * accSize + outputIdx) = l1Weights(outputIdx * featureSize + featureIdx)
private val l1WeightsT: Array[Float] = private val validateAccum = sys.env.contains("NNUE_VALIDATE")
val w = model.weights(0).weights
val t = new Array[Float](featureSize * accSize)
for j <- 0 until featureSize; i <- 0 until accSize do t(j * accSize + i) = w(i * featureSize + j)
t
// ── Accumulator stack ──────────────────────────────────────────────────── // ── Accumulator stack ────────────────────────────────────────────────────
private val MAX_PLY = 128 private val MAX_PLY = 128
private val l1Stack: Array[Array[Float]] = Array.fill(MAX_PLY + 1)(new Array[Float](accSize)) private val l1Stack: Array[Array[Float]] = Array.fill(MAX_PLY + 1)(new Array[Float](accSize))
// Shared evaluation buffers: index i holds the output of layers(i) (all except the scalar output layer).
private val evalBuffers: Array[Array[Float]] = model.layers.init.map(l => new Array[Float](l.outputSize)) private val evalBuffers: Array[Array[Float]] = model.layers.init.map(l => new Array[Float](l.outputSize))
// ── Eval cache ─────────────────────────────────────────────────────────── // ── Eval cache ───────────────────────────────────────────────────────────
@@ -36,9 +35,29 @@ class NNUE(model: NbaiModel):
private def squareNum(sq: Square): Int = sq.rank.ordinal * 8 + sq.file.ordinal private def squareNum(sq: Square): Int = sq.rank.ordinal * 8 + sq.file.ordinal
private def featureIndex(piece: Piece, sqNum: Int): Int = // Mirror square vertically (rank 0 rank 7) for the perspective flip
val colorOffset = if piece.color == Color.White then 6 else 0 private def flipSqNum(sqNum: Int): Int = (7 - sqNum / 8) * 8 + sqNum % 8
(colorOffset + piece.pieceType.ordinal) * 64 + sqNum
private def pieceIdx(piece: Piece): Int =
if piece.color == Color.White then 6 + piece.pieceType.ordinal else piece.pieceType.ordinal
// White-king perspective: index in [0, HALF_SIZE)
private def featureIdxWhite(piece: Piece, sqNum: Int, wkSq: Int): Int =
wkSq * 768 + pieceIdx(piece) * 64 + sqNum
// Black-king perspective: index in [HALF_SIZE, featureSize)
private def featureIdxBlack(piece: Piece, sqNum: Int, bkSq: Int): Int =
HALF_SIZE + bkSq * 768 + pieceIdx(piece) * 64 + sqNum
private def wkSqOf(board: Board): Int =
board.pieces
.collectFirst { case (sq, p) if p.pieceType == PieceType.King && p.color == Color.White => squareNum(sq) }
.getOrElse(0)
private def bkSqOf(board: Board): Int =
board.pieces
.collectFirst { case (sq, p) if p.pieceType == PieceType.King && p.color == Color.Black => squareNum(sq) }
.getOrElse(0)
private def addColumn(l1Pre: Array[Float], featureIdx: Int): Unit = private def addColumn(l1Pre: Array[Float], featureIdx: Int): Unit =
val offset = featureIdx * accSize val offset = featureIdx * accSize
@@ -48,92 +67,96 @@ class NNUE(model: NbaiModel):
val offset = featureIdx * accSize val offset = featureIdx * accSize
for i <- 0 until accSize do l1Pre(i) -= l1WeightsT(offset + i) for i <- 0 until accSize do l1Pre(i) -= l1WeightsT(offset + i)
private def addPiece(l1: Array[Float], piece: Piece, sqNum: Int, wkSq: Int, bkSq: Int): Unit =
addColumn(l1, featureIdxWhite(piece, sqNum, wkSq))
addColumn(l1, featureIdxBlack(piece, sqNum, bkSq))
private def removePiece(l1: Array[Float], piece: Piece, sqNum: Int, wkSq: Int, bkSq: Int): Unit =
subtractColumn(l1, featureIdxWhite(piece, sqNum, wkSq))
subtractColumn(l1, featureIdxBlack(piece, sqNum, bkSq))
// ── Accumulator init ───────────────────────────────────────────────────── // ── Accumulator init ─────────────────────────────────────────────────────
def initAccumulator(board: Board): Unit = def initAccumulator(board: Board): Unit =
val wkSq = wkSqOf(board)
val bkSq = bkSqOf(board)
System.arraycopy(model.weights(0).bias, 0, l1Stack(0), 0, accSize) System.arraycopy(model.weights(0).bias, 0, l1Stack(0), 0, accSize)
for (sq, piece) <- board.pieces do addColumn(l1Stack(0), featureIndex(piece, squareNum(sq))) for (sq, piece) <- board.pieces do addPiece(l1Stack(0), piece, squareNum(sq), wkSq, bkSq)
// ── Accumulator push (incremental updates) ─────────────────────────────── // ── Accumulator push (incremental updates) ───────────────────────────────
def pushAccumulator(childPly: Int, move: Move, board: Board): Unit = def pushAccumulator(childPly: Int, move: Move, parentBoard: Board, childBoard: Board): Unit =
System.arraycopy(l1Stack(childPly - 1), 0, l1Stack(childPly), 0, accSize) System.arraycopy(l1Stack(childPly - 1), 0, l1Stack(childPly), 0, accSize)
val l1 = l1Stack(childPly) if isKingMove(move, parentBoard) then recomputeAccumulatorInto(l1Stack(childPly), childBoard)
move.moveType match else applyNonKingDelta(l1Stack(childPly), move, parentBoard)
case MoveType.Normal(_) => applyNormalDelta(l1, move, board)
case MoveType.EnPassant => applyEnPassantDelta(l1, move, board) private def isKingMove(move: Move, board: Board): Boolean =
case MoveType.CastleKingside | MoveType.CastleQueenside => applyCastleDelta(l1, move, board) move.moveType == MoveType.CastleKingside ||
case MoveType.Promotion(p) => applyPromotionDelta(l1, move, p, board) move.moveType == MoveType.CastleQueenside ||
board.pieceAt(move.from).exists(_.pieceType == PieceType.King)
def copyAccumulator(parentPly: Int, childPly: Int): Unit = def copyAccumulator(parentPly: Int, childPly: Int): Unit =
System.arraycopy(l1Stack(parentPly), 0, l1Stack(childPly), 0, accSize) System.arraycopy(l1Stack(parentPly), 0, l1Stack(childPly), 0, accSize)
def recomputeAccumulator(ply: Int, board: Board): Unit = def recomputeAccumulator(ply: Int, board: Board): Unit =
System.arraycopy(model.weights(0).bias, 0, l1Stack(ply), 0, accSize) recomputeAccumulatorInto(l1Stack(ply), board)
for (sq, piece) <- board.pieces do addColumn(l1Stack(ply), featureIndex(piece, squareNum(sq)))
private def recomputeAccumulatorInto(l1: Array[Float], board: Board): Unit =
val wkSq = wkSqOf(board)
val bkSq = bkSqOf(board)
System.arraycopy(model.weights(0).bias, 0, l1, 0, accSize)
for (sq, piece) <- board.pieces do addPiece(l1, piece, squareNum(sq), wkSq, bkSq)
def validateAccumulator(ply: Int, board: Board): Boolean = def validateAccumulator(ply: Int, board: Board): Boolean =
// Compute what L1 should be from scratch val expected = new Array[Float](accSize)
val expectedL1 = new Array[Float](accSize) val wkSq = wkSqOf(board)
System.arraycopy(model.weights(0).bias, 0, expectedL1, 0, accSize) val bkSq = bkSqOf(board)
for (sq, piece) <- board.pieces do addColumn(expectedL1, featureIndex(piece, squareNum(sq))) System.arraycopy(model.weights(0).bias, 0, expected, 0, accSize)
for (sq, piece) <- board.pieces do addPiece(expected, piece, squareNum(sq), wkSq, bkSq)
// Compare with actual L1
val actual = l1Stack(ply) val actual = l1Stack(ply)
val maxError = (0 until accSize).forall(i => math.abs(actual(i) - expected(i)) < 0.001f)
(0 until accSize).foldLeft(0f) { (currentMax, i) =>
val error = math.abs(actual(i) - expectedL1(i))
math.max(currentMax, error)
}
maxError < 0.001f // Allow small floating-point errors // ── Non-king incremental deltas ──────────────────────────────────────────
private def applyNormalDelta(l1: Array[Float], move: Move, board: Board): Unit = private def applyNonKingDelta(l1: Array[Float], move: Move, board: Board): Unit =
// Extract source and destination square indices early val wkSq = wkSqOf(board)
val bkSq = bkSqOf(board)
move.moveType match
case MoveType.Normal(_) => applyNormalDelta(l1, move, board, wkSq, bkSq)
case MoveType.EnPassant => applyEnPassantDelta(l1, move, board, wkSq, bkSq)
case MoveType.Promotion(p) => applyPromotionDelta(l1, move, p, board, wkSq, bkSq)
case _ => () // king moves handled before this point
private def applyNormalDelta(l1: Array[Float], move: Move, board: Board, wkSq: Int, bkSq: Int): Unit =
board.pieceAt(move.from).foreach { mover =>
val fromNum = squareNum(move.from) val fromNum = squareNum(move.from)
val toNum = squareNum(move.to) val toNum = squareNum(move.to)
removePiece(l1, mover, fromNum, wkSq, bkSq)
// Get the moving piece board.pieceAt(move.to).foreach(cap => removePiece(l1, cap, toNum, wkSq, bkSq))
board.pieceAt(move.from).foreach { mover => addPiece(l1, mover, toNum, wkSq, bkSq)
subtractColumn(l1, featureIndex(mover, fromNum))
// If there's a capture, subtract the captured piece
board.pieceAt(move.to).foreach { cap =>
subtractColumn(l1, featureIndex(cap, toNum))
} }
// Add the piece to its new location private def applyEnPassantDelta(l1: Array[Float], move: Move, board: Board, wkSq: Int, bkSq: Int): Unit =
addColumn(l1, featureIndex(mover, toNum))
}
private def applyEnPassantDelta(l1: Array[Float], move: Move, board: Board): Unit =
board.pieceAt(move.from).foreach { pawn => board.pieceAt(move.from).foreach { pawn =>
val capturedSq = Square(move.to.file, move.from.rank) val capturedSq = Square(move.to.file, move.from.rank)
subtractColumn(l1, featureIndex(pawn, squareNum(move.from))) removePiece(l1, pawn, squareNum(move.from), wkSq, bkSq)
board.pieceAt(capturedSq).foreach(cap => subtractColumn(l1, featureIndex(cap, squareNum(capturedSq)))) board.pieceAt(capturedSq).foreach(cap => removePiece(l1, cap, squareNum(capturedSq), wkSq, bkSq))
addColumn(l1, featureIndex(pawn, squareNum(move.to))) addPiece(l1, pawn, squareNum(move.to), wkSq, bkSq)
} }
private def applyCastleDelta(l1: Array[Float], move: Move, board: Board): Unit = private def applyPromotionDelta(
board.pieceAt(move.from).foreach { king => l1: Array[Float],
val rank = move.from.rank move: Move,
val kingside = move.moveType == MoveType.CastleKingside promo: PromotionPiece,
val (rookFrom, rookTo) = board: Board,
if kingside then (Square(File.H, rank), Square(File.F, rank)) wkSq: Int,
else (Square(File.A, rank), Square(File.D, rank)) bkSq: Int,
val rook = Piece(king.color, PieceType.Rook) ): Unit =
subtractColumn(l1, featureIndex(king, squareNum(move.from)))
addColumn(l1, featureIndex(king, squareNum(move.to)))
subtractColumn(l1, featureIndex(rook, squareNum(rookFrom)))
addColumn(l1, featureIndex(rook, squareNum(rookTo)))
}
private def applyPromotionDelta(l1: Array[Float], move: Move, promo: PromotionPiece, board: Board): Unit =
board.pieceAt(move.from).foreach { pawn => board.pieceAt(move.from).foreach { pawn =>
val toNum = squareNum(move.to) val toNum = squareNum(move.to)
subtractColumn(l1, featureIndex(pawn, squareNum(move.from))) removePiece(l1, pawn, squareNum(move.from), wkSq, bkSq)
board.pieceAt(move.to).foreach(cap => subtractColumn(l1, featureIndex(cap, toNum))) board.pieceAt(move.to).foreach(cap => removePiece(l1, cap, toNum, wkSq, bkSq))
addColumn(l1, featureIndex(Piece(pawn.color, promotedType(promo)), toNum)) addPiece(l1, Piece(pawn.color, promotedType(promo)), toNum, wkSq, bkSq)
} }
private def promotedType(promo: PromotionPiece): PieceType = promo match private def promotedType(promo: PromotionPiece): PieceType = promo match
@@ -154,7 +177,6 @@ class NNUE(model: NbaiModel):
score score
def evaluateAtPlyWithValidation(ply: Int, turn: Color, hash: Long, board: Board): Int = def evaluateAtPlyWithValidation(ply: Int, turn: Color, hash: Long, board: Board): Int =
// For debugging: validate that incremental accumulator matches recomputation
if validateAccum && ply > 0 && ply % 10 != 0 then if validateAccum && ply > 0 && ply % 10 != 0 then
val isValid = validateAccumulator(ply, board) val isValid = validateAccumulator(ply, board)
if !isValid then System.err.println(s"WARNING: NNUE accumulator diverged at ply $ply") if !isValid then System.err.println(s"WARNING: NNUE accumulator diverged at ply $ply")
@@ -206,9 +228,23 @@ class NNUE(model: NbaiModel):
private val legacyL1 = new Array[Float](accSize) private val legacyL1 = new Array[Float](accSize)
def evaluate(context: GameContext): Int = def evaluate(context: GameContext): Int =
// Match training: for Black-to-move positions, mirror the board (ranks flipped,
// colours swapped) so the model always sees from the side-to-move's perspective.
// The scoreFromOutput negation then converts back to White's absolute perspective.
val (wkSq, bkSq, pieces, turn) =
if context.turn == Color.Black then
val wk = flipSqNum(bkSqOf(context.board)) // flipped Black king new "White" king
val bk = flipSqNum(wkSqOf(context.board)) // flipped White king new "Black" king
val flipped = context.board.pieces.map { case (sq, p) =>
(sq, Piece(p.color.opposite, p.pieceType))
}
(wk, bk, flipped, Color.Black) // pass Black so scoreFromOutput negates the result
else (wkSqOf(context.board), bkSqOf(context.board), context.board.pieces, context.turn)
System.arraycopy(model.weights(0).bias, 0, legacyL1, 0, accSize) System.arraycopy(model.weights(0).bias, 0, legacyL1, 0, accSize)
for (sq, piece) <- context.board.pieces do addColumn(legacyL1, featureIndex(piece, squareNum(sq))) for (sq, piece) <- pieces do
runL2toOutput(legacyL1, context.turn) val sqNum = if turn == Color.Black then flipSqNum(squareNum(sq)) else squareNum(sq)
addPiece(legacyL1, piece, sqNum, wkSq, bkSq)
runL2toOutput(legacyL1, turn)
def benchmark(): Unit = def benchmark(): Unit =
val context = GameContext.initial val context = GameContext.initial
@@ -0,0 +1,21 @@
package de.nowchess.bot.bots.nnue
/** Immutable, shareable NNUE parameters.
*
* Heavy to build (transposes the L1 weight matrix once, ~98304 × accSize floats) but read-only thereafter, so a single
* instance is safely shared across many per-thread [[NNUE]] evaluators. Holds no accumulator or scratch state those
* live on each [[NNUE]] instance which is what makes parallel search (independent evaluators sharing these weights)
* possible without duplicating the weight matrix.
*/
class NNUEWeights(val model: NbaiModel):
val HALF_SIZE: Int = 49152 // 64 king-squares × 12 piece-types × 64 piece-squares
val featureSize: Int = model.layers(0).inputSize // 98304 (= HALF_SIZE * 2) for king-relative
val accSize: Int = model.layers(0).outputSize
// Column-major L1 weights: l1WeightsT(featureIdx * accSize + outputIdx)
val l1WeightsT: Array[Float] =
val w = model.weights(0).weights
val t = new Array[Float](featureSize * accSize)
for j <- 0 until featureSize; i <- 0 until accSize do t(j * accSize + i) = w(i * featureSize + j)
t
@@ -1,6 +1,7 @@
package de.nowchess.bot.bots.nnue package de.nowchess.bot.bots.nnue
import java.io.InputStream import java.io.InputStream
import java.nio.file.{Files, Path}
import java.nio.{ByteBuffer, ByteOrder} import java.nio.{ByteBuffer, ByteOrder}
import java.nio.charset.StandardCharsets import java.nio.charset.StandardCharsets
@@ -17,13 +18,28 @@ object NbaiLoader:
val weights = descs.map(_ => readLayerWeights(buf)) val weights = descs.map(_ => readLayerWeights(buf))
NbaiModel(metadata, descs, weights) NbaiModel(metadata, descs, weights)
/** Tries /nnue_weights.nbai on the classpath; falls back to migrating /nnue_weights.bin. */ /** Loads weights from the `nnue.weights` system property if it points at a readable file; otherwise tries
* /nnue_weights.nbai on the classpath, falling back to migrating /nnue_weights.bin.
*/
def loadDefault(): NbaiModel = def loadDefault(): NbaiModel =
overrideModel().getOrElse {
Option(getClass.getResourceAsStream("/nnue_weights.nbai")) match Option(getClass.getResourceAsStream("/nnue_weights.nbai")) match
case Some(s) => case Some(s) =>
try load(s) try load(s)
finally s.close() finally s.close()
case None => NbaiMigrator.migrateFromBin() case None => NbaiMigrator.migrateFromBin()
}
private def overrideModel(): Option[NbaiModel] =
sys.props
.get("nnue.weights")
.map(Path.of(_))
.filter(Files.isRegularFile(_))
.map { path =>
val s = Files.newInputStream(path)
try load(s)
finally s.close()
}
private def checkHeader(buf: ByteBuffer): Unit = private def checkHeader(buf: ByteBuffer): Unit =
val magic = buf.getInt() val magic = buf.getInt()
@@ -0,0 +1,12 @@
package de.nowchess.bot.config
import de.nowchess.bot.resource.{JoinTournamentRequest, JoinTournamentResponse}
import io.quarkus.runtime.annotations.RegisterForReflection
@RegisterForReflection(
targets = Array(
classOf[JoinTournamentRequest],
classOf[JoinTournamentResponse],
),
)
class NativeReflectionConfig
@@ -1,6 +1,6 @@
package de.nowchess.bot.logic package de.nowchess.bot.logic
import de.nowchess.api.board.PieceType import de.nowchess.api.board.{CastlingRights, PieceType}
import de.nowchess.api.game.GameContext import de.nowchess.api.game.GameContext
import de.nowchess.api.move.{Move, MoveType} import de.nowchess.api.move.{Move, MoveType}
import de.nowchess.bot.ai.Evaluation import de.nowchess.bot.ai.Evaluation
@@ -26,12 +26,15 @@ final class AlphaBetaSearch(
private val TIME_CHECK_FREQUENCY = 1000 private val TIME_CHECK_FREQUENCY = 1000
private val FUTILITY_MARGIN = 100 private val FUTILITY_MARGIN = 100
private val CHECK_EXTENSION = 1 private val CHECK_EXTENSION = 1
private val CONTEMPT = 25
private val timeStartMs = AtomicLong(0L) private val timeStartMs = AtomicLong(0L)
private val timeLimitMs = AtomicLong(0L) private val timeLimitMs = AtomicLong(0L)
private val nodeCount = AtomicInteger(0) private val nodeCount = AtomicInteger(0)
private val ordering = MoveOrdering.OrderingContext() private val ordering = MoveOrdering.OrderingContext()
def lastNodeCount: Int = nodeCount.get()
private final case class QuiescenceNode( private final case class QuiescenceNode(
context: GameContext, context: GameContext,
ply: Int, ply: Int,
@@ -47,6 +50,17 @@ final class AlphaBetaSearch(
bestMove(context, maxDepth, Set.empty) bestMove(context, maxDepth, Set.empty)
def bestMove(context: GameContext, maxDepth: Int, excludedRootMoves: Set[Move]): Option[Move] = def bestMove(context: GameContext, maxDepth: Int, excludedRootMoves: Set[Move]): Option[Move] =
doDepthSearch(context, maxDepth, excludedRootMoves, Map.empty)
def bestMove(context: GameContext, maxDepth: Int, excludedRootMoves: Set[Move], hints: Map[Move, Int]): Option[Move] =
doDepthSearch(context, maxDepth, excludedRootMoves, hints)
private def doDepthSearch(
context: GameContext,
maxDepth: Int,
excludedRootMoves: Set[Move],
hints: Map[Move, Int],
): Option[Move] =
tt.clear() tt.clear()
ordering.clear() ordering.clear()
weights.initAccumulator(context) weights.initAccumulator(context)
@@ -54,6 +68,7 @@ final class AlphaBetaSearch(
timeLimitMs.set(Long.MaxValue / 4) timeLimitMs.set(Long.MaxValue / 4)
nodeCount.set(0) nodeCount.set(0)
val rootHash = ZobristHash.hash(context) val rootHash = ZobristHash.hash(context)
val history = historyCounts(context)
(1 to maxDepth) (1 to maxDepth)
.foldLeft((None: Option[Move], 0)) { case ((bestSoFar, prevScore), depth) => .foldLeft((None: Option[Move], 0)) { case ((bestSoFar, prevScore), depth) =>
val (alpha, beta) = val (alpha, beta) =
@@ -65,7 +80,9 @@ final class AlphaBetaSearch(
beta, beta,
ASPIRATION_DELTA, ASPIRATION_DELTA,
rootHash, rootHash,
history,
excludedRootMoves, excludedRootMoves,
hints,
) )
(move.orElse(bestSoFar), score) (move.orElse(bestSoFar), score)
} }
@@ -78,13 +95,42 @@ final class AlphaBetaSearch(
bestMoveWithTime(context, timeBudgetMs, Set.empty) bestMoveWithTime(context, timeBudgetMs, Set.empty)
def bestMoveWithTime(context: GameContext, timeBudgetMs: Long, excludedRootMoves: Set[Move]): Option[Move] = def bestMoveWithTime(context: GameContext, timeBudgetMs: Long, excludedRootMoves: Set[Move]): Option[Move] =
tt.clear() doTimedSearch(context, timeBudgetMs, excludedRootMoves, Map.empty, clearTt = true)
def bestMoveWithTime(
context: GameContext,
timeBudgetMs: Long,
excludedRootMoves: Set[Move],
hints: Map[Move, Int],
): Option[Move] =
doTimedSearch(context, timeBudgetMs, excludedRootMoves, hints, clearTt = true)
/** Timed search over a transposition table that is shared with other workers (Lazy SMP): the caller is responsible
* for clearing it once before launching all workers, so this worker must not clear it.
*/
def bestMoveWithTimeSharedTt(
context: GameContext,
timeBudgetMs: Long,
excludedRootMoves: Set[Move],
hints: Map[Move, Int],
): Option[Move] =
doTimedSearch(context, timeBudgetMs, excludedRootMoves, hints, clearTt = false)
private def doTimedSearch(
context: GameContext,
timeBudgetMs: Long,
excludedRootMoves: Set[Move],
hints: Map[Move, Int],
clearTt: Boolean,
): Option[Move] =
if clearTt then tt.clear()
ordering.clear() ordering.clear()
weights.initAccumulator(context) weights.initAccumulator(context)
timeStartMs.set(System.currentTimeMillis) timeStartMs.set(System.currentTimeMillis)
timeLimitMs.set(timeBudgetMs) timeLimitMs.set(timeBudgetMs)
nodeCount.set(0) nodeCount.set(0)
val rootHash = ZobristHash.hash(context) val rootHash = ZobristHash.hash(context)
val history = historyCounts(context)
@scala.annotation.tailrec @scala.annotation.tailrec
def loop(bestSoFar: Option[Move], prevScore: Int, depth: Int, lastDepth: Int): (Option[Move], Int) = def loop(bestSoFar: Option[Move], prevScore: Int, depth: Int, lastDepth: Int): (Option[Move], Int) =
@@ -99,7 +145,9 @@ final class AlphaBetaSearch(
beta, beta,
ASPIRATION_DELTA, ASPIRATION_DELTA,
rootHash, rootHash,
history,
excludedRootMoves, excludedRootMoves,
hints,
) )
loop(move.orElse(bestSoFar), score, depth + 1, depth) loop(move.orElse(bestSoFar), score, depth + 1, depth)
@@ -123,15 +171,19 @@ final class AlphaBetaSearch(
beta: Int, beta: Int,
initialWindow: Int, initialWindow: Int,
rootHash: Long, rootHash: Long,
history: Map[Long, Int],
excludedRootMoves: Set[Move], excludedRootMoves: Set[Move],
hints: Map[Move, Int],
): (Int, Option[Move]) = ): (Int, Option[Move]) =
val state = SearchState(rootHash, Map(rootHash -> 1)) val state = SearchState(rootHash, history)
@scala.annotation.tailrec @scala.annotation.tailrec
def loop(currentAlpha: Int, currentBeta: Int, delta: Int, attempt: Int): (Int, Option[Move]) = def loop(currentAlpha: Int, currentBeta: Int, delta: Int, attempt: Int): (Int, Option[Move]) =
if attempt >= 3 || attempt >= depth then search(context, depth, 0, Window(-INF, INF), state, excludedRootMoves) if attempt >= 3 || attempt >= depth then
search(context, depth, 0, Window(-INF, INF), state, excludedRootMoves, hints)
else else
val (score, move) = search(context, depth, 0, Window(currentAlpha, currentBeta), state, excludedRootMoves) val (score, move) =
search(context, depth, 0, Window(currentAlpha, currentBeta), state, excludedRootMoves, hints)
if score > currentAlpha && score < currentBeta then (score, move) if score > currentAlpha && score < currentBeta then (score, move)
else if score <= currentAlpha then else if score <= currentAlpha then
loop(score - delta, currentBeta, math.min(delta * 2, ASPIRATION_DELTA_MAX), attempt + 1) loop(score - delta, currentBeta, math.min(delta * 2, ASPIRATION_DELTA_MAX), attempt + 1)
@@ -139,6 +191,32 @@ final class AlphaBetaSearch(
loop(alpha, beta, initialWindow, 0) loop(alpha, beta, initialWindow, 0)
private def drawScore(ply: Int): Int =
if ply % 2 == 0 then weights.DRAW_SCORE - CONTEMPT else weights.DRAW_SCORE + CONTEMPT
private def historyCounts(context: GameContext): Map[Long, Int] =
val initialTurn = if context.moves.size % 2 == 0 then context.turn else context.turn.opposite
val root = GameContext(
board = context.initialBoard,
turn = initialTurn,
castlingRights = CastlingRights.Initial,
enPassantSquare = None,
halfMoveClock = 0,
moves = List.empty,
)
val rootHash = ZobristHash.hash(root)
context.moves
.foldLeft((root, rootHash, List(rootHash))) { case ((cur, curHash, acc), move) =>
val next = rules.applyMove(cur)(move)
val nextHash = ZobristHash.nextHash(cur, curHash, move, next)
(next, nextHash, nextHash :: acc)
}
._3
.groupBy(identity)
.view
.mapValues(_.size)
.toMap
private def hasNonPawnMaterial(context: GameContext): Boolean = private def hasNonPawnMaterial(context: GameContext): Boolean =
context.board.pieces.values.exists { piece => context.board.pieces.values.exists { piece =>
piece.color == context.turn && piece.color == context.turn &&
@@ -156,12 +234,14 @@ final class AlphaBetaSearch(
beta: Int, beta: Int,
state: SearchState, state: SearchState,
excludedRootMoves: Set[Move], excludedRootMoves: Set[Move],
hints: Map[Move, Int],
): Option[Int] = ): Option[Int] =
val nullCtx = nullMoveContext(context) val nullCtx = nullMoveContext(context)
val nullState = state.advance(ZobristHash.hash(nullCtx)) val nullState = state.advance(ZobristHash.hash(nullCtx))
val reductionDepth = math.max(0, depth - 1 - NULL_MOVE_R) val reductionDepth = math.max(0, depth - 1 - NULL_MOVE_R)
weights.copyAccumulator(ply, ply + 1) weights.copyAccumulator(ply, ply + 1)
val (score, _) = search(nullCtx, reductionDepth, ply + 1, Window(-beta, -beta + 1), nullState, excludedRootMoves) val (score, _) =
search(nullCtx, reductionDepth, ply + 1, Window(-beta, -beta + 1), nullState, excludedRootMoves, hints)
if -score >= beta then Some(beta) else None if -score >= beta then Some(beta) else None
/** Negamax alpha-beta search returning (score, best move). */ /** Negamax alpha-beta search returning (score, best move). */
@@ -172,8 +252,9 @@ final class AlphaBetaSearch(
window: Window, window: Window,
state: SearchState, state: SearchState,
excludedRootMoves: Set[Move], excludedRootMoves: Set[Move],
hints: Map[Move, Int],
): (Int, Option[Move]) = ): (Int, Option[Move]) =
val params = SearchParams(context, depth, ply, window, state, excludedRootMoves) val params = SearchParams(context, depth, ply, window, state, excludedRootMoves, hints)
searchNode(params) searchNode(params)
private def searchNode(params: SearchParams): (Int, Option[Move]) = private def searchNode(params: SearchParams): (Int, Option[Move]) =
@@ -189,7 +270,8 @@ final class AlphaBetaSearch(
): Option[(Int, Option[Move])] = ): Option[(Int, Option[Move])] =
if count % TIME_CHECK_FREQUENCY == 0 && isOutOfTime then if count % TIME_CHECK_FREQUENCY == 0 && isOutOfTime then
Some((weights.evaluateAccumulator(params.ply, params.context, params.state.hash), None)) Some((weights.evaluateAccumulator(params.ply, params.context, params.state.hash), None))
else if params.state.repetitions.getOrElse(params.state.hash, 0) >= 3 then Some((weights.DRAW_SCORE, None)) else if params.ply > 0 && params.state.repetitions.getOrElse(params.state.hash, 0) >= 2 then
Some((drawScore(params.ply), None))
else ttCutoff(params) else ttCutoff(params)
private def ttCutoff(params: SearchParams): Option[(Int, Option[Move])] = private def ttCutoff(params: SearchParams): Option[(Int, Option[Move])] =
@@ -211,12 +293,12 @@ final class AlphaBetaSearch(
if legalMoves.isEmpty then if legalMoves.isEmpty then
Some( Some(
( (
if rules.isCheckmate(params.context) then -(weights.CHECKMATE_SCORE - params.ply) else weights.DRAW_SCORE, if rules.isCheckmate(params.context) then -(weights.CHECKMATE_SCORE - params.ply) else drawScore(params.ply),
None, None,
), ),
) )
else if rules.isInsufficientMaterial(params.context) || rules.isFiftyMoveRule(params.context) then else if rules.isInsufficientMaterial(params.context) || rules.isFiftyMoveRule(params.context) then
Some((weights.DRAW_SCORE, None)) Some((drawScore(params.ply), None))
else if params.depth == 0 then else if params.depth == 0 then
Some((quiescence(params.context, params.ply, params.window.alpha, params.window.beta, params.state.hash), None)) Some((quiescence(params.context, params.ply, params.window.alpha, params.window.beta, params.state.hash), None))
else None else None
@@ -235,13 +317,14 @@ final class AlphaBetaSearch(
params.window.beta, params.window.beta,
params.state, params.state,
params.excludedRootMoves, params.excludedRootMoves,
params.rootHints,
), ),
) )
.flatten .flatten
nullResult.map((_, None)).getOrElse { nullResult.map((_, None)).getOrElse {
val ttBest = tt.probe(params.state.hash).flatMap(_.bestMove) val ttBest = tt.probe(params.state.hash).flatMap(_.bestMove)
val ordered = MoveOrdering.sort(params.context, legalMoves, ttBest, params.ply, ordering) val ordered = MoveOrdering.sort(params.context, legalMoves, ttBest, params.ply, ordering, params.rootHints)
searchSequential( searchSequential(
params.context, params.context,
params.depth, params.depth,
@@ -250,6 +333,7 @@ final class AlphaBetaSearch(
ordered, ordered,
params.state, params.state,
params.excludedRootMoves, params.excludedRootMoves,
params.rootHints,
) )
} }
@@ -280,6 +364,7 @@ final class AlphaBetaSearch(
Window(-a - 1, -a), Window(-a - 1, -a),
childState, childState,
params.excludedRootMoves, params.excludedRootMoves,
params.rootHints,
) )
val s = -rs val s = -rs
if s > a then if s > a then
@@ -290,6 +375,7 @@ final class AlphaBetaSearch(
Window(betaNeg, -a), Window(betaNeg, -a),
childState, childState,
params.excludedRootMoves, params.excludedRootMoves,
params.rootHints,
) )
-fs -fs
else s else s
@@ -301,6 +387,7 @@ final class AlphaBetaSearch(
Window(betaNeg, -a), Window(betaNeg, -a),
childState, childState,
params.excludedRootMoves, params.excludedRootMoves,
params.rootHints,
) )
-rs -rs
@@ -364,8 +451,9 @@ final class AlphaBetaSearch(
ordered: List[Move], ordered: List[Move],
state: SearchState, state: SearchState,
excludedRootMoves: Set[Move], excludedRootMoves: Set[Move],
rootHints: Map[Move, Int] = Map.empty,
): (Int, Option[Move]) = ): (Int, Option[Move]) =
val params = SearchParams(context, depth, ply, window, state, excludedRootMoves) val params = SearchParams(context, depth, ply, window, state, excludedRootMoves, rootHints)
val (bestMove, bestScore, cutoff) = searchLoop(0, 0, LoopAcc(None, -INF, window.alpha), params, ordered) val (bestMove, bestScore, cutoff) = searchLoop(0, 0, LoopAcc(None, -INF, window.alpha), params, ordered)
val flag = val flag =
if cutoff then TTFlag.Lower if cutoff then TTFlag.Lower
@@ -425,5 +513,5 @@ final class AlphaBetaSearch(
private def isCapture(context: GameContext, move: Move): Boolean = move.moveType match private def isCapture(context: GameContext, move: Move): Boolean = move.moveType match
case MoveType.Normal(true) => true case MoveType.Normal(true) => true
case MoveType.EnPassant => true case MoveType.EnPassant => true
case MoveType.Promotion(_) => context.board.pieceAt(move.to).exists(_.color != context.turn) case MoveType.Promotion(_) => true
case _ => false case _ => false
@@ -38,8 +38,10 @@ object MoveOrdering:
ttBestMove: Option[Move], ttBestMove: Option[Move],
ply: Int = 0, ply: Int = 0,
ordering: OrderingContext = new OrderingContext(), ordering: OrderingContext = new OrderingContext(),
rootHints: Map[Move, Int] = Map.empty,
): Int = ): Int =
if ttBestMove.exists(m => m.from == move.from && m.to == move.to) then Int.MaxValue if ttBestMove.exists(m => m.from == move.from && m.to == move.to) then Int.MaxValue
else if ply == 0 && rootHints.nonEmpty then rootHints.getOrElse(move, Int.MinValue / 2)
else else
move.moveType match move.moveType match
case MoveType.Promotion(PromotionPiece.Queen) => case MoveType.Promotion(PromotionPiece.Queen) =>
@@ -56,8 +58,9 @@ object MoveOrdering:
ttBestMove: Option[Move], ttBestMove: Option[Move],
ply: Int = 0, ply: Int = 0,
ordering: OrderingContext = new OrderingContext(), ordering: OrderingContext = new OrderingContext(),
rootHints: Map[Move, Int] = Map.empty,
): List[Move] = ): List[Move] =
moves.sortBy(m => -score(context, m, ttBestMove, ply, ordering)) moves.sortBy(m => -score(context, m, ttBestMove, ply, ordering, rootHints))
private def scoreQuietMove(move: Move, ply: Int, ordering: OrderingContext): Int = private def scoreQuietMove(move: Move, ply: Int, ordering: OrderingContext): Int =
val isKiller = ordering.getKillerMoves(ply).exists(k => k.from == move.from && k.to == move.to) val isKiller = ordering.getKillerMoves(ply).exists(k => k.from == move.from && k.to == move.to)
@@ -0,0 +1,56 @@
package de.nowchess.bot.logic
import de.nowchess.api.game.GameContext
import de.nowchess.api.move.Move
import de.nowchess.api.rules.RuleSet
import de.nowchess.bot.ai.Evaluation
import de.nowchess.rules.sets.DefaultRules
import java.util.concurrent.{Callable, Executors}
import scala.jdk.CollectionConverters.*
/** Lazy SMP search coordinator.
*
* Runs `numThreads` independent [[AlphaBetaSearch]] workers over one shared transposition table for the same time
* budget. Every worker has its own evaluator (independent NNUE accumulator) and move-ordering state, but they share
* the thread-safe TT, so faster-progressing threads deepen entries the others reuse. Only the main worker's move is
* returned; helpers exist purely to enrich the shared TT.
*
* `numThreads <= 1` runs a single worker via the ordinary clearing entry point, byte-identical to sequential
* [[AlphaBetaSearch]].
*/
final class ParallelSearch(
rules: RuleSet = DefaultRules,
tt: TranspositionTable = TranspositionTable(),
evalFactory: () => Evaluation,
numThreads: Int = 1,
):
private val threadCount = math.max(1, numThreads)
private val workers = Vector.fill(threadCount)(AlphaBetaSearch(rules, tt, evalFactory()))
def bestMoveWithTime(
context: GameContext,
timeBudgetMs: Long,
excludedRootMoves: Set[Move] = Set.empty,
hints: Map[Move, Int] = Map.empty,
): Option[Move] =
if threadCount == 1 then workers.head.bestMoveWithTime(context, timeBudgetMs, excludedRootMoves, hints)
else runParallel(context, timeBudgetMs, excludedRootMoves, hints)
private def runParallel(
context: GameContext,
timeBudgetMs: Long,
excludedRootMoves: Set[Move],
hints: Map[Move, Int],
): Option[Move] =
tt.clear()
val pool = Executors.newFixedThreadPool(threadCount)
try
val tasks = workers.map { worker =>
new Callable[Option[Move]]:
def call(): Option[Move] =
worker.bestMoveWithTimeSharedTt(context, timeBudgetMs, excludedRootMoves, hints)
}
pool.invokeAll(tasks.asJava).get(0).get()
finally pool.shutdownNow()
@@ -14,6 +14,7 @@ final case class SearchParams(
window: Window, window: Window,
state: SearchState, state: SearchState,
excludedRootMoves: Set[Move], excludedRootMoves: Set[Move],
rootHints: Map[Move, Int] = Map.empty,
) )
final case class SearchState(hash: Long, repetitions: Map[Long, Int]): final case class SearchState(hash: Long, repetitions: Map[Long, Int]):
@@ -0,0 +1,180 @@
package de.nowchess.bot.selfplay
import de.nowchess.api.game.GameContext
import de.nowchess.api.move.Move
import de.nowchess.api.rules.RuleSet
import de.nowchess.api.board.Color
import de.nowchess.bot.{Bot, BotDifficulty}
import de.nowchess.bot.bots.{HybridBot, NNUEBot}
import de.nowchess.io.fen.FenExporter
import de.nowchess.rules.sets.DefaultRules
import java.io.{BufferedWriter, FileWriter}
import java.nio.file.{Files, Path}
import scala.collection.mutable
import scala.util.Random
enum GameResult:
case Bot1Wins
case Bot2Wins
case Draw
/** Standalone bot-vs-bot harness. Runs two NNUEBots against each other from randomised openings and writes the visited
* positions as one FEN per line. Each bot can use different difficulty levels and weight files.
*
* Games run sequentially. Bots alternate: bot1 plays white, then bot1 plays black, alternating per game to ensure fair
* evaluation.
*/
object BotVsBotMain:
private case class Config(
games: Int = 1,
out: String = "modules/official-bots/python/data/botvbot.txt",
weights1: Option[String] = None,
weights2: Option[String] = None,
difficulty1: String = "Hard",
difficulty2: String = "Hard",
moveTimeMs: Long = 1000L,
randomPlies: Int = 8,
maxPlies: Int = 200,
seed: Long = System.nanoTime(),
)
def main(args: Array[String]): Unit =
val config = parse(args.toList, Config())
val rules = DefaultRules
val difficulty1 = parseDifficulty(config.difficulty1)
val difficulty2 = parseDifficulty(config.difficulty2)
config.weights1.foreach(System.setProperty("nnue.weights", _))
val bot1 = HybridBot(difficulty1, rules, searchThreads = 6)
config.weights2.foreach(System.setProperty("nnue.weights", _))
val bot2 = HybridBot(difficulty2, rules)
val rng = new Random(config.seed)
val seen = mutable.HashSet.empty[String]
var bot1Wins = 0
var bot2Wins = 0
var draws = 0
Files.createDirectories(Path.of(config.out).toAbsolutePath.getParent)
val writer = new BufferedWriter(new FileWriter(config.out))
try
var game = 0
while game < config.games do
val bot1AsWhite = game % 2 == 0
val result = playGame(rules, bot1, bot2, rng, config, seen, writer, bot1AsWhite)
game += 1
result match
case Some(GameResult.Bot1Wins) =>
bot1Wins += 1
println(s"Game $game: Bot1 wins${if bot1AsWhite then " (as white)" else " (as black)"}")
case Some(GameResult.Bot2Wins) =>
bot2Wins += 1
println(s"Game $game: Bot2 wins${if !bot1AsWhite then " (as white)" else " (as black)"}")
case Some(GameResult.Draw) =>
draws += 1
println(s"Game $game: Draw")
case None => ()
if game % 25 == 0 then
writer.flush()
println(
s" Progress: $game/${config.games} | Positions: ${seen.size} | Bot1: $bot1Wins, Bot2: $bot2Wins, Draws: $draws\n",
)
finally writer.close()
println(s"\nFinal Statistics:")
println(s" Bot1 wins: $bot1Wins")
println(s" Bot2 wins: $bot2Wins")
println(s" Draws: $draws")
println(s" Positions: ${seen.size} -> ${config.out}")
private def playGame(
rules: RuleSet,
bot1: Bot,
bot2: Bot,
rng: Random,
config: Config,
seen: mutable.HashSet[String],
writer: BufferedWriter,
bot1AsWhite: Boolean,
): Option[GameResult] =
randomOpening(rules, rng, config.randomPlies, GameContext.initial) match
case None => None
case Some(start) =>
var ctx = start
var plies = config.randomPlies
var live = true
while live && plies < config.maxPlies do
if isTerminal(rules, ctx) then live = false
else
val currentBot =
if (plies - config.randomPlies) % 2 == 0 then if bot1AsWhite then bot1 else bot2
else if bot1AsWhite then bot2
else bot1
currentBot(ctx) match
case None => live = false
case Some(move) =>
ctx = rules.applyMove(ctx)(move)
plies += 1
record(rules, ctx, seen, writer)
determineWinner(rules, ctx, bot1AsWhite)
private def determineWinner(rules: RuleSet, ctx: GameContext, bot1AsWhite: Boolean): Option[GameResult] =
val legalMoves = rules.allLegalMoves(ctx)
if legalMoves.nonEmpty then Some(GameResult.Draw)
else if rules.isCheck(ctx) then
if ctx.turn == (if bot1AsWhite then Color.Black else Color.White) then Some(GameResult.Bot1Wins)
else Some(GameResult.Bot2Wins)
else if rules.isFiftyMoveRule(ctx) || rules.isThreefoldRepetition(ctx) || rules.isInsufficientMaterial(ctx) then
Some(GameResult.Draw)
else Some(GameResult.Draw)
private def randomOpening(rules: RuleSet, rng: Random, plies: Int, start: GameContext): Option[GameContext] =
var ctx = start
var i = 0
while i < plies do
val legal = rules.allLegalMoves(ctx)
if legal.isEmpty then return None
ctx = rules.applyMove(ctx)(legal(rng.nextInt(legal.size)))
i += 1
Some(ctx)
private def record(rules: RuleSet, ctx: GameContext, seen: mutable.HashSet[String], writer: BufferedWriter): Unit =
if !rules.isCheck(ctx) && !isTerminal(rules, ctx) then
val fen = FenExporter.gameContextToFen(ctx)
if seen.add(fen) then
writer.write(fen)
writer.newLine()
private def isTerminal(rules: RuleSet, ctx: GameContext): Boolean =
rules.allLegalMoves(ctx).isEmpty ||
rules.isInsufficientMaterial(ctx) ||
rules.isFiftyMoveRule(ctx) ||
rules.isThreefoldRepetition(ctx)
private def parse(args: List[String], acc: Config): Config = args match
case "--games" :: v :: rest => parse(rest, acc.copy(games = v.toInt))
case "--out" :: v :: rest => parse(rest, acc.copy(out = v))
case "--weights1" :: v :: rest => parse(rest, acc.copy(weights1 = Some(v)))
case "--weights2" :: v :: rest => parse(rest, acc.copy(weights2 = Some(v)))
case "--difficulty1" :: v :: rest => parse(rest, acc.copy(difficulty1 = v))
case "--difficulty2" :: v :: rest => parse(rest, acc.copy(difficulty2 = v))
case "--move-ms" :: v :: rest => parse(rest, acc.copy(moveTimeMs = v.toLong))
case "--random-plies" :: v :: rest => parse(rest, acc.copy(randomPlies = v.toInt))
case "--max-plies" :: v :: rest => parse(rest, acc.copy(maxPlies = v.toInt))
case "--seed" :: v :: rest => parse(rest, acc.copy(seed = v.toLong))
case Nil => acc
case unknown :: rest => println(s"Ignoring unknown arg: $unknown"); parse(rest, acc)
private def parseDifficulty(name: String): BotDifficulty = name match
case "Easy" => BotDifficulty.Easy
case "Medium" => BotDifficulty.Medium
case "Expert" => BotDifficulty.Expert
case _ => BotDifficulty.Hard
@@ -0,0 +1,112 @@
package de.nowchess.bot.selfplay
import de.nowchess.api.game.GameContext
import de.nowchess.api.move.Move
import de.nowchess.api.rules.RuleSet
import de.nowchess.bot.{Bot, BotDifficulty}
import de.nowchess.bot.bots.NNUEBot
import de.nowchess.io.fen.FenExporter
import de.nowchess.rules.sets.DefaultRules
import java.io.{BufferedWriter, FileWriter}
import java.nio.file.{Files, Path}
import scala.collection.mutable
import scala.util.Random
/** Standalone self-play harness. Runs NNUEBot against itself from randomised openings and writes the visited positions
* as one FEN per line the input format expected by the Python labeler. No microservices.
*
* Games run sequentially because EvaluationNNUE holds a shared accumulator; the small per-move time budget keeps
* throughput high. Stockfish relabels every position later, so shallow self-play search is sufficient.
*/
object SelfPlayMain:
private case class Config(
games: Int = 500,
out: String = "modules/official-bots/python/data/selfplay.txt",
weights: Option[String] = None,
moveTimeMs: Long = 50L,
randomPlies: Int = 8,
maxPlies: Int = 200,
seed: Long = System.nanoTime(),
)
def main(args: Array[String]): Unit =
val config = parse(args.toList, Config())
config.weights.foreach(System.setProperty("nnue.weights", _))
val rules = DefaultRules
val bot = NNUEBot(BotDifficulty.Hard, rules, fixedMoveTimeMs = Some(config.moveTimeMs))
val rng = new Random(config.seed)
val seen = mutable.HashSet.empty[String]
Files.createDirectories(Path.of(config.out).toAbsolutePath.getParent)
val writer = new BufferedWriter(new FileWriter(config.out))
try
var game = 0
while game < config.games do
playGame(rules, bot, rng, config, seen, writer)
game += 1
if game % 25 == 0 then
writer.flush()
println(s"games=$game/${config.games} positions=${seen.size}")
finally writer.close()
println(s"Done. ${seen.size} unique positions -> ${config.out}")
private def playGame(
rules: RuleSet,
bot: Bot,
rng: Random,
config: Config,
seen: mutable.HashSet[String],
writer: BufferedWriter,
): Unit =
randomOpening(rules, rng, config.randomPlies, GameContext.initial) match
case None => ()
case Some(start) =>
var ctx = start
var plies = config.randomPlies
var live = true
while live && plies < config.maxPlies do
if isTerminal(rules, ctx) then live = false
else
bot(ctx) match
case None => live = false
case Some(move) =>
ctx = rules.applyMove(ctx)(move)
plies += 1
record(rules, ctx, seen, writer)
private def randomOpening(rules: RuleSet, rng: Random, plies: Int, start: GameContext): Option[GameContext] =
var ctx = start
var i = 0
while i < plies do
val legal = rules.allLegalMoves(ctx)
if legal.isEmpty then return None
ctx = rules.applyMove(ctx)(legal(rng.nextInt(legal.size)))
i += 1
Some(ctx)
private def record(rules: RuleSet, ctx: GameContext, seen: mutable.HashSet[String], writer: BufferedWriter): Unit =
if !rules.isCheck(ctx) && !isTerminal(rules, ctx) then
val fen = FenExporter.gameContextToFen(ctx)
if seen.add(fen) then
writer.write(fen)
writer.newLine()
private def isTerminal(rules: RuleSet, ctx: GameContext): Boolean =
rules.allLegalMoves(ctx).isEmpty ||
rules.isInsufficientMaterial(ctx) ||
rules.isFiftyMoveRule(ctx) ||
rules.isThreefoldRepetition(ctx)
private def parse(args: List[String], acc: Config): Config = args match
case "--games" :: v :: rest => parse(rest, acc.copy(games = v.toInt))
case "--out" :: v :: rest => parse(rest, acc.copy(out = v))
case "--weights" :: v :: rest => parse(rest, acc.copy(weights = Some(v)))
case "--move-ms" :: v :: rest => parse(rest, acc.copy(moveTimeMs = v.toLong))
case "--random-plies" :: v :: rest => parse(rest, acc.copy(randomPlies = v.toInt))
case "--max-plies" :: v :: rest => parse(rest, acc.copy(maxPlies = v.toInt))
case "--seed" :: v :: rest => parse(rest, acc.copy(seed = v.toLong))
case Nil => acc
case unknown :: rest => println(s"Ignoring unknown arg: $unknown"); parse(rest, acc)
@@ -2,7 +2,7 @@ package de.nowchess.bot.service
import com.fasterxml.jackson.databind.{JsonNode, ObjectMapper} import com.fasterxml.jackson.databind.{JsonNode, ObjectMapper}
import de.nowchess.api.move.{Move, MoveType, PromotionPiece} import de.nowchess.api.move.{Move, MoveType, PromotionPiece}
import de.nowchess.bot.{Bot, BotController} import de.nowchess.bot.{Bot, BotController, TimeControl}
import de.nowchess.bot.client.AccountServiceClient import de.nowchess.bot.client.AccountServiceClient
import de.nowchess.bot.config.RedisConfig import de.nowchess.bot.config.RedisConfig
import de.nowchess.io.fen.FenParser import de.nowchess.io.fen.FenParser
@@ -42,6 +42,8 @@ class TournamentBotGamePlayer:
private val hardestDifficulty = "expert" private val hardestDifficulty = "expert"
private val autoJoinIntervalMs = 15000L private val autoJoinIntervalMs = 15000L
// Detect the opponent's move fast: every poll spent waiting runs our clock without us thinking.
private val pollIntervalMs = 150L
private val gameTerminalStatuses = private val gameTerminalStatuses =
Set("checkmate", "stalemate", "draw", "resigned", "timeout", "aborted", "finished") Set("checkmate", "stalemate", "draw", "resigned", "timeout", "aborted", "finished")
@@ -119,7 +121,9 @@ class TournamentBotGamePlayer:
yield result yield result
private def findBotGame(pairings: JsonNode, botId: String): Option[(String, String)] = private def findBotGame(pairings: JsonNode, botId: String): Option[(String, String)] =
pairings.elements().asScala pairings
.elements()
.asScala
.flatMap { p => .flatMap { p =>
val whiteId = p.path("white").path("id").asText() val whiteId = p.path("white").path("id").asText()
val blackId = p.path("black").path("id").asText() val blackId = p.path("black").path("id").asText()
@@ -129,7 +133,9 @@ class TournamentBotGamePlayer:
.nextOption() .nextOption()
private def activeMatch(matches: JsonNode): Option[String] = private def activeMatch(matches: JsonNode): Option[String] =
matches.elements().asScala matches
.elements()
.asScala
.find(m => m.path("gameId").asText().nonEmpty && !(m.has("outcome") && !m.path("outcome").isNull)) .find(m => m.path("gameId").asText().nonEmpty && !(m.has("outcome") && !m.path("outcome").isNull))
.map(_.path("gameId").asText()) .map(_.path("gameId").asText())
@@ -151,9 +157,12 @@ class TournamentBotGamePlayer:
private def openTournaments(): List[String] = private def openTournaments(): List[String] =
Try { Try {
val response = client.target(autoJoinServerUrl) val response = client
.path("api").path("tournament") .target(autoJoinServerUrl)
.request(MediaType.APPLICATION_JSON).get() .path("api")
.path("tournament")
.request(MediaType.APPLICATION_JSON)
.get()
if response.getStatus == 200 then if response.getStatus == 200 then
val node = objectMapper.readTree(response.readEntity(classOf[String])) val node = objectMapper.readTree(response.readEntity(classOf[String]))
response.close() response.close()
@@ -164,8 +173,8 @@ class TournamentBotGamePlayer:
private def resolveToken(difficulty: String): Option[String] = private def resolveToken(difficulty: String): Option[String] =
val name = botName(difficulty) val name = botName(difficulty)
val redisKey = s"${redisConfig.prefix}:tournament-bot:token:$name" val redisKey = s"${redisConfig.prefix}:tournament-bot:token:$name"
registerWithServer(tournamentServiceUrl, name) fetchTokenFromAccountService(name)
.orElse(fetchTokenFromAccountService(name)) .orElse(registerWithServer(tournamentServiceUrl, name))
.map { token => .map { token =>
redis.value(classOf[String]).set(redisKey, token) redis.value(classOf[String]).set(redisKey, token)
log.infof("Refreshed bot token for %s — stored in Redis", name) log.infof("Refreshed bot token for %s — stored in Redis", name)
@@ -187,8 +196,11 @@ class TournamentBotGamePlayer:
private def registerWithServer(serverUrl: String, name: String): Option[String] = private def registerWithServer(serverUrl: String, name: String): Option[String] =
Try { Try {
val body = s"""{"name":"${name.replace("\"", "\\\"")}","isBot":true}""" val body = s"""{"name":"${name.replace("\"", "\\\"")}","isBot":true}"""
val response = client.target(serverUrl) val response = client
.path("api").path("auth").path("register") .target(serverUrl)
.path("api")
.path("auth")
.path("register")
.request(MediaType.APPLICATION_JSON) .request(MediaType.APPLICATION_JSON)
.post(Entity.entity(body, MediaType.APPLICATION_JSON)) .post(Entity.entity(body, MediaType.APPLICATION_JSON))
val status = response.getStatus val status = response.getStatus
@@ -201,11 +213,11 @@ class TournamentBotGamePlayer:
log.warnf("Register %s on %s returned status %d: %s", name, serverUrl, status, errBody) log.warnf("Register %s on %s returned status %d: %s", name, serverUrl, status, errBody)
response.close() response.close()
None None
}.recover { case ex => log.warnf(ex, "Register %s on %s failed", name, serverUrl); None } }.recover { case ex => log.warnf(ex, "Register %s on %s failed", name, serverUrl); None }.toOption.flatten
.toOption.flatten
private def fetchTokenFromAccountService(name: String): Option[String] = private def fetchTokenFromAccountService(name: String): Option[String] =
Try(accountServiceClient.getBotToken(name).token).toOption.filter(_.nonEmpty) Try(accountServiceClient.getBotToken(name).token).toOption
.filter(_.nonEmpty)
.orElse { .orElse {
Try { Try {
val allNames = BotController.listBots.map(botName) val allNames = BotController.listBots.map(botName)
@@ -231,9 +243,13 @@ class TournamentBotGamePlayer:
private def fetchRemoteServers(): List[String] = private def fetchRemoteServers(): List[String] =
Try { Try {
val response = client.target(tournamentServiceUrl) val response = client
.path("api").path("tournament").path("servers") .target(tournamentServiceUrl)
.request(MediaType.APPLICATION_JSON).get() .path("api")
.path("tournament")
.path("servers")
.request(MediaType.APPLICATION_JSON)
.get()
if response.getStatus == 200 then if response.getStatus == 200 then
val node = objectMapper.readTree(response.readEntity(classOf[String])) val node = objectMapper.readTree(response.readEntity(classOf[String]))
response.close() response.close()
@@ -244,7 +260,11 @@ class TournamentBotGamePlayer:
private def parkOnAccountService(serverUrl: String, difficulty: String, token: String): Unit = private def parkOnAccountService(serverUrl: String, difficulty: String, token: String): Unit =
Try { Try {
val body = s"""{"name":"${botName(difficulty)}"}""" val body = s"""{"name":"${botName(difficulty)}"}"""
val response = client.target(serverUrl).path("api").path("account").path("bots") val response = client
.target(serverUrl)
.path("api")
.path("account")
.path("bots")
.request(MediaType.APPLICATION_JSON) .request(MediaType.APPLICATION_JSON)
.header("Authorization", s"Bearer $token") .header("Authorization", s"Bearer $token")
.post(Entity.entity(body, MediaType.APPLICATION_JSON)) .post(Entity.entity(body, MediaType.APPLICATION_JSON))
@@ -258,7 +278,10 @@ class TournamentBotGamePlayer:
private def parkOnTournamentServer(serverUrl: String, name: String, token: String): Unit = private def parkOnTournamentServer(serverUrl: String, name: String, token: String): Unit =
Try { Try {
val body = s"""{"name":"${name.replace("\"", "\\\"")}"}""" val body = s"""{"name":"${name.replace("\"", "\\\"")}"}"""
val response = client.target(serverUrl).path("api").path("bots") val response = client
.target(serverUrl)
.path("api")
.path("bots")
.request(MediaType.APPLICATION_JSON) .request(MediaType.APPLICATION_JSON)
.header("Authorization", s"Bearer $token") .header("Authorization", s"Bearer $token")
.post(Entity.entity(body, MediaType.APPLICATION_JSON)) .post(Entity.entity(body, MediaType.APPLICATION_JSON))
@@ -277,9 +300,10 @@ class TournamentBotGamePlayer:
difficulty: String, difficulty: String,
): Either[String, String] = ): Either[String, String] =
val redisKey = s"${redisConfig.prefix}:tournament-bot:token:${botName(difficulty)}" val redisKey = s"${redisConfig.prefix}:tournament-bot:token:${botName(difficulty)}"
val resolvedToken = botToken.filter(_.nonEmpty) val resolvedToken = botToken
.filter(_.nonEmpty)
.orElse(Option(redis.value(classOf[String]).get(redisKey)).filter(_.nonEmpty)) .orElse(Option(redis.value(classOf[String]).get(redisKey)).filter(_.nonEmpty))
.orElse(System.getenv().asScala.get("TOURNAMENT_BOT_TOKEN").filter(_.nonEmpty)) .orElse(resolveToken(difficulty))
resolvedToken match resolvedToken match
case None => Left("No bot token provided and TOURNAMENT_BOT_TOKEN not configured") case None => Left("No bot token provided and TOURNAMENT_BOT_TOKEN not configured")
case Some(token) => case Some(token) =>
@@ -336,8 +360,7 @@ class TournamentBotGamePlayer:
log.infof("Listening to tournament %s event stream", cfg.tournamentId) log.infof("Listening to tournament %s event stream", cfg.tournamentId)
forEachLine(response.readEntity(classOf[InputStream])): line => forEachLine(response.readEntity(classOf[InputStream])): line =>
parse(line).foreach: node => parse(line).foreach: node =>
if node.path("type").asText() == "gameStart" then if node.path("type").asText() == "gameStart" then onGameStart(cfg, node.path("gameId").asText())
onGameStart(cfg, node.path("gameId").asText())
private def onGameStart(cfg: TournamentBotConfig, gameId: String): Unit = private def onGameStart(cfg: TournamentBotConfig, gameId: String): Unit =
if gameId.isEmpty then () if gameId.isEmpty then ()
@@ -393,20 +416,40 @@ class TournamentBotGamePlayer:
if gameTerminalStatuses.contains(status) then if gameTerminalStatuses.contains(status) then
log.infof("Game %s ended — status=%s", gameId, status); done = true log.infof("Game %s ended — status=%s", gameId, status); done = true
else else
// TEMP: tournament-server reports wrong color in pairings (everyone white).
// The game endpoint white/black ids are correct, so derive our color from it.
val whiteId = node.path("white").path("id").asText()
val blackId = node.path("black").path("id").asText()
val myColor =
if whiteId == cfg.botId then "white"
else if blackId == cfg.botId then "black"
else color
val turn = node.path("turn").asText() val turn = node.path("turn").asText()
val fen = node.path("fen").asText() val fen = node.path("fen").asText()
if turn == color && status == "ongoing" && fen.nonEmpty && fen != lastFen then if turn == myColor && status == "ongoing" && fen.nonEmpty && fen != lastFen then
lastFen = fen lastFen = fen
log.infof("Our turn in game %s — computing move (fen=%s)", gameId, fen) val time = readTimeControl(node, myColor)
computeUci(cfg, fen) match log.infof("Our turn in game %s — computing move (fen=%s, budget=%dms)", gameId, fen, time.budgetMs)
computeUci(cfg, fen, time) match
case None => log.warnf("No move found for game %s (fen=%s)", gameId, fen) case None => log.warnf("No move found for game %s (fen=%s)", gameId, fen)
case Some(uci) => submitMove(cfg, gameId, uci) case Some(uci) => submitMove(cfg, gameId, uci)
sleep(1000) sleep(pollIntervalMs)
private def computeUci(cfg: TournamentBotConfig, fen: String): Option[String] = // Server clock is reported in seconds; convert to a millisecond TimeControl so the engine can
// size its move budget against the real clock instead of a fixed guess.
private def readTimeControl(node: JsonNode, myColor: String): TimeControl =
val clock = node.path("clock")
if clock.isMissingNode || clock.isNull then TimeControl.Unlimited
else
val field = if myColor == "white" then "whiteTime" else "blackTime"
val remainingMs = (clock.path(field).asDouble(0.0) * 1000.0).toLong
val incrementMs = (clock.path("increment").asDouble(0.0) * 1000.0).toLong
TimeControl(remainingMs, incrementMs)
private def computeUci(cfg: TournamentBotConfig, fen: String, time: TimeControl): Option[String] =
FenParser.parseFen(fen) match FenParser.parseFen(fen) match
case Left(err) => log.warnf("FEN parse failed: %s (%s)", fen, err.toString); None case Left(err) => log.warnf("FEN parse failed: %s (%s)", fen, err.toString); None
case Right(context) => engine(cfg).apply(context).map(toUci) case Right(context) => engine(cfg).move(context, time).map(toUci)
private def submitMove(cfg: TournamentBotConfig, gameId: String, uci: String): Unit = private def submitMove(cfg: TournamentBotConfig, gameId: String, uci: String): Unit =
Try { Try {
@@ -4,9 +4,9 @@ import de.nowchess.api.board.*
import de.nowchess.api.game.GameContext import de.nowchess.api.game.GameContext
import de.nowchess.api.move.{Move, MoveType, PromotionPiece} import de.nowchess.api.move.{Move, MoveType, PromotionPiece}
import java.io.{DataInputStream, FileInputStream} import java.io.{DataInputStream, FileInputStream, InputStream}
import java.util.concurrent.ThreadLocalRandom
import scala.collection.mutable import scala.collection.mutable
import scala.util.Random
/** Reads a Polyglot opening book (.bin file) and probes it for moves. /** Reads a Polyglot opening book (.bin file) and probes it for moves.
* *
@@ -16,24 +16,11 @@ import scala.util.Random
* - weight: 2 bytes (Short) move weight (higher = preferred) * - weight: 2 bytes (Short) move weight (higher = preferred)
* - learn: 4 bytes (Int) learning data (unused) * - learn: 4 bytes (Int) learning data (unused)
*/ */
final class PolyglotBook(path: String): final class PolyglotBook private (entries: Map[Long, Vector[BookEntry]]):
private val entries: Map[Long, Vector[BookEntry]] =
try {
val r = loadBookFile(path)
println(s"Book loaded successfully. ${r.size} entries found.")
r
} catch
case e: Exception =>
println(s"Error loading book: $e")
// Gracefully fail: return empty map if book cannot be loaded
// This allows the bot to work even if the book file is missing
scala.collection.immutable.Map.empty
/** Probe the book for a move in the given position. Returns a weighted random move, or None if not in book. */ /** Probe the book for a move in the given position. Returns a weighted random move, or None if not in book. */
def probe(context: GameContext): Option[Move] = def probe(context: GameContext): Option[Move] =
val hash = PolyglotHash.hash(context) val hash = PolyglotHash.hash(context)
println(f"0x$hash%016X")
entries.get(hash).flatMap { bookEntries => entries.get(hash).flatMap { bookEntries =>
if bookEntries.isEmpty then None if bookEntries.isEmpty then None
else else
@@ -41,24 +28,6 @@ final class PolyglotBook(path: String):
decodeMove(entry.move, context) decodeMove(entry.move, context)
} }
private def loadBookFile(path: String): Map[Long, Vector[BookEntry]] =
val input = DataInputStream(FileInputStream(path))
try
val result = mutable.Map[Long, Vector[BookEntry]]()
while input.available() > 0 do
val key = input.readLong()
val move = input.readShort()
val weight = input.readShort()
input.readInt() // learning data (unused)
val entry = BookEntry(key, move, weight)
result.updateWith(key) {
case Some(entries) => Some(entries :+ entry)
case None => Some(Vector(entry))
}
result.toMap
finally input.close()
/** Decode a packed Polyglot move short into an Option[Move]. /** Decode a packed Polyglot move short into an Option[Move].
* *
* Bit layout of the move Short: * Bit layout of the move Short:
@@ -124,7 +93,7 @@ final class PolyglotBook(path: String):
if entries.length == 1 then entries.head if entries.length == 1 then entries.head
else else
val totalWeight = entries.map(_.weight).sum val totalWeight = entries.map(_.weight).sum
val pick = Random.nextInt(totalWeight.max(1)) // NOSONAR val pick = ThreadLocalRandom.current().nextInt(totalWeight.max(1)) // NOSONAR
@scala.annotation.tailrec @scala.annotation.tailrec
def select(remaining: Int, idx: Int): BookEntry = def select(remaining: Int, idx: Int): BookEntry =
@@ -134,4 +103,48 @@ final class PolyglotBook(path: String):
select(pick, 0) select(pick, 0)
object PolyglotBook:
/** Load a book from a filesystem path. Fails gracefully to an empty book. */
def apply(path: String): PolyglotBook =
safeLoad(s"file $path")(FileInputStream(path))
/** Load a book from a classpath resource (native-image safe: the resource is embedded in the binary, so no file must
* be mounted into the pod).
*/
def fromResource(name: String): PolyglotBook =
Option(getClass.getResourceAsStream(name)) match
case Some(stream) => safeLoad(s"resource $name")(stream)
case None =>
println(s"Error loading book: resource $name not found on classpath")
new PolyglotBook(Map.empty)
private def safeLoad(source: String)(stream: => InputStream): PolyglotBook =
try
val entries = parse(stream)
println(s"Book loaded successfully from $source. ${entries.size} entries found.")
new PolyglotBook(entries)
catch
case e: Exception =>
println(s"Error loading book from $source: $e")
new PolyglotBook(Map.empty)
private def parse(stream: InputStream): Map[Long, Vector[BookEntry]] =
val input = DataInputStream(stream)
try
val result = mutable.Map[Long, Vector[BookEntry]]()
while input.available() > 0 do
val key = input.readLong()
val move = input.readShort()
val weight = input.readShort()
input.readInt() // learning data (unused)
val entry = BookEntry(key, move, weight)
result.updateWith(key) {
case Some(entries) => Some(entries :+ entry)
case None => Some(Vector(entry))
}
result.toMap
finally input.close()
private case class BookEntry(key: Long, move: Short, weight: Int) private case class BookEntry(key: Long, move: Short, weight: Int)
@@ -253,6 +253,30 @@ class AlphaBetaSearchTest extends AnyFunSuite with Matchers:
val search = AlphaBetaSearch(promoCaptureRules, weights = EvaluationClassic) val search = AlphaBetaSearch(promoCaptureRules, weights = EvaluationClassic)
search.bestMove(ctx, maxDepth = 1) should be(Some(promoCapture)) search.bestMove(ctx, maxDepth = 1) should be(Some(promoCapture))
test("quiet promotion is treated as tactical in quiescence"):
// Pawn pushes to an empty back-rank square (no capture). Must still be searched in
// quiescence so a bot does not skip queening at the search horizon.
val quietPromo = Move(Square(File.E, Rank.R7), Square(File.E, Rank.R8), MoveType.Promotion(PromotionPiece.Queen))
val board = Board(
Map(
Square(File.E, Rank.R7) -> Piece.WhitePawn,
),
)
val ctx = GameContext.initial.withBoard(board).withTurn(Color.White)
val quietPromoRules = new RuleSet:
def candidateMoves(context: GameContext)(square: Square): List[Move] = Nil
def legalMoves(context: GameContext)(square: Square): List[Move] = Nil
def allLegalMoves(context: GameContext): List[Move] = List(quietPromo)
def isCheck(context: GameContext): Boolean = false
def isCheckmate(context: GameContext): Boolean = false
def isStalemate(context: GameContext): Boolean = false
def isInsufficientMaterial(context: GameContext): Boolean = false
def isFiftyMoveRule(context: GameContext): Boolean = false
def isThreefoldRepetition(context: GameContext): Boolean = false
def applyMove(context: GameContext)(move: Move): GameContext = context
val search = AlphaBetaSearch(quietPromoRules, weights = EvaluationClassic)
search.bestMove(ctx, maxDepth = 1) should be(Some(quietPromo))
test("draw when isInsufficientMaterial with legal moves present"): test("draw when isInsufficientMaterial with legal moves present"):
val legalMove = Move(Square(File.E, Rank.R2), Square(File.E, Rank.R4), MoveType.Normal()) val legalMove = Move(Square(File.E, Rank.R2), Square(File.E, Rank.R4), MoveType.Normal())
val drawRules = new RuleSet: val drawRules = new RuleSet:
@@ -312,6 +336,24 @@ class AlphaBetaSearchTest extends AnyFunSuite with Matchers:
val search = AlphaBetaSearch(qRules, weights = ZeroEval) val search = AlphaBetaSearch(qRules, weights = ZeroEval)
search.bestMove(GameContext.initial, maxDepth = 1) should be(Some(rootMove)) search.bestMove(GameContext.initial, maxDepth = 1) should be(Some(rootMove))
test("bestMove with root hints returns a valid move without regression"):
val context = GameContext.initial
val legalMoves = DefaultRules.allLegalMoves(context)
val hints = legalMoves.zipWithIndex.map { case (m, i) => m -> (legalMoves.length - i) }.toMap
val withHints = AlphaBetaSearch(DefaultRules, weights = EvaluationClassic)
.bestMove(context, maxDepth = 2, Set.empty, hints)
withHints should not be None
legalMoves should contain(withHints.get)
test("bestMoveWithTime with root hints returns a valid move without regression"):
val context = GameContext.initial
val legalMoves = DefaultRules.allLegalMoves(context)
val hints = legalMoves.zipWithIndex.map { case (m, i) => m -> (legalMoves.length - i) }.toMap
val withHints = AlphaBetaSearch(DefaultRules, weights = EvaluationClassic)
.bestMoveWithTime(context, 500L, Set.empty, hints)
withHints should not be None
legalMoves should contain(withHints.get)
test("quiescence depth-limit in-check branch is exercised"): test("quiescence depth-limit in-check branch is exercised"):
val rootMove = Move(Square(File.E, Rank.R2), Square(File.E, Rank.R3), MoveType.Normal()) val rootMove = Move(Square(File.E, Rank.R2), Square(File.E, Rank.R3), MoveType.Normal())
val capMove = Move(Square(File.D, Rank.R2), Square(File.D, Rank.R3), MoveType.Normal(true)) val capMove = Move(Square(File.D, Rank.R2), Square(File.D, Rank.R3), MoveType.Normal(true))
@@ -0,0 +1,34 @@
package de.nowchess.bot
import de.nowchess.api.board.{Board, Color, File, Piece, Rank, Square}
import de.nowchess.api.game.GameContext
import de.nowchess.bot.bots.nnue.MopUp
import org.scalatest.funsuite.AnyFunSuite
import org.scalatest.matchers.should.Matchers
class MopUpTest extends AnyFunSuite with Matchers:
private def ctx(turn: Color, pieces: (Square, Piece)*): GameContext =
GameContext.initial.withBoard(Board(pieces.toMap)).withTurn(turn)
private val wk = Square(File.E, Rank.R1) -> Piece.WhiteKing
private val wq = Square(File.D, Rank.R1) -> Piece.WhiteQueen
private val bkCorner = Square(File.H, Rank.R8) -> Piece.BlackKing
private val bkCenter = Square(File.D, Rank.R4) -> Piece.BlackKing
test("zero in a balanced middlegame-like position (both sides have material)"):
MopUp.score(ctx(Color.White, wk, wq, bkCorner, Square(File.A, Rank.R8) -> Piece.BlackQueen)) should be(0)
test("zero when winner lacks mating material (lone king vs king)"):
MopUp.score(ctx(Color.White, wk, bkCorner)) should be(0)
test("positive for the winning side to move in KQ vs K"):
MopUp.score(ctx(Color.White, wk, wq, bkCorner)) should be > 0
test("negative for the bare-king side to move in KQ vs K"):
MopUp.score(ctx(Color.Black, wk, wq, bkCorner)) should be < 0
test("cornered bare king scores higher than centralized bare king"):
val cornered = MopUp.score(ctx(Color.White, wk, wq, bkCorner))
val centralized = MopUp.score(ctx(Color.White, wk, wq, bkCenter))
cornered should be > centralized
@@ -217,3 +217,60 @@ class MoveOrderingTest extends AnyFunSuite with Matchers:
val castle = Move(Square(File.E, Rank.R1), Square(File.G, Rank.R1), MoveType.CastleKingside) val castle = Move(Square(File.E, Rank.R1), Square(File.G, Rank.R1), MoveType.CastleKingside)
MoveOrdering.score(context, castle, None) should be(0) MoveOrdering.score(context, castle, None) should be(0)
test("root hints override capture heuristics at ply 0"):
val board = Board(
Map(
Square(File.E, Rank.R4) -> Piece.WhiteQueen,
Square(File.E, Rank.R5) -> Piece.BlackPawn,
Square(File.D, Rank.R5) -> Piece.BlackRook,
),
)
val context = GameContext.initial.withBoard(board).withTurn(Color.White)
val quietMove = Move(Square(File.E, Rank.R4), Square(File.E, Rank.R6))
val rookCapture = Move(Square(File.E, Rank.R4), Square(File.D, Rank.R5), MoveType.Normal(true))
val hints = Map(quietMove -> 500, rookCapture -> 100)
MoveOrdering.score(context, quietMove, None, ply = 0, rootHints = hints) should equal(500)
MoveOrdering.score(context, rookCapture, None, ply = 0, rootHints = hints) should equal(100)
MoveOrdering.score(context, rookCapture, None, ply = 0, rootHints = hints) should be <
MoveOrdering.score(context, quietMove, None, ply = 0, rootHints = hints)
test("root hints ignored at ply > 0"):
val board = Board(Map(Square(File.E, Rank.R4) -> Piece.WhiteQueen, Square(File.E, Rank.R5) -> Piece.BlackPawn))
val context = GameContext.initial.withBoard(board).withTurn(Color.White)
val capture = Move(Square(File.E, Rank.R4), Square(File.E, Rank.R5), MoveType.Normal(true))
val quiet = Move(Square(File.E, Rank.R4), Square(File.D, Rank.R4))
val hints = Map(quiet -> 99999, capture -> -99999)
val captureScore = MoveOrdering.score(context, capture, None, ply = 1, rootHints = hints)
val quietScore = MoveOrdering.score(context, quiet, None, ply = 1, rootHints = hints)
captureScore should be > quietScore
test("move absent from root hints gets Int.MinValue / 2 fallback"):
val board = Board(Map(Square(File.E, Rank.R4) -> Piece.WhiteQueen))
val context = GameContext.initial.withBoard(board).withTurn(Color.White)
val move1 = Move(Square(File.E, Rank.R4), Square(File.E, Rank.R6))
val move2 = Move(Square(File.E, Rank.R4), Square(File.E, Rank.R5))
val hints = Map(move1 -> 0)
MoveOrdering.score(context, move2, None, ply = 0, rootHints = hints) should equal(Int.MinValue / 2)
test("sort uses root hints at ply 0 to reorder moves"):
val board = Board(
Map(
Square(File.E, Rank.R4) -> Piece.WhiteQueen,
Square(File.E, Rank.R5) -> Piece.BlackPawn,
Square(File.D, Rank.R5) -> Piece.BlackRook,
),
)
val context = GameContext.initial.withBoard(board).withTurn(Color.White)
val rookCapture = Move(Square(File.E, Rank.R4), Square(File.D, Rank.R5), MoveType.Normal(true))
val pawnCapture = Move(Square(File.E, Rank.R4), Square(File.E, Rank.R5), MoveType.Normal(true))
val quiet = Move(Square(File.E, Rank.R4), Square(File.E, Rank.R6))
val hints = Map(quiet -> 9999, pawnCapture -> 500, rookCapture -> 100)
val sorted = MoveOrdering.sort(context, List(rookCapture, pawnCapture, quiet), None, ply = 0, rootHints = hints)
sorted.head should equal(quiet)
sorted(1) should equal(pawnCapture)
sorted(2) should equal(rookCapture)
@@ -0,0 +1,28 @@
package de.nowchess.bot
import de.nowchess.api.game.GameContext
import de.nowchess.bot.bots.classic.EvaluationClassic
import de.nowchess.bot.logic.{ParallelSearch, TranspositionTable}
import de.nowchess.rules.sets.DefaultRules
import org.scalatest.funsuite.AnyFunSuite
import org.scalatest.matchers.should.Matchers
class ParallelSearchTest extends AnyFunSuite with Matchers:
private def search(threads: Int): ParallelSearch =
ParallelSearch(DefaultRules, TranspositionTable(), () => EvaluationClassic, threads)
test("single-threaded coordinator returns a legal move on the initial position"):
val move = search(1).bestMoveWithTime(GameContext.initial, 200L)
move should not be None
DefaultRules.allLegalMoves(GameContext.initial) should contain(move.get)
test("multi-threaded Lazy SMP returns a legal move and does not crash under concurrency"):
val parallel = search(4)
for _ <- 1 to 5 do
val move = parallel.bestMoveWithTime(GameContext.initial, 200L)
move should not be None
DefaultRules.allLegalMoves(GameContext.initial) should contain(move.get)
test("numThreads below one is clamped to a single worker"):
search(0).bestMoveWithTime(GameContext.initial, 100L) should not be None
+1 -1
View File
@@ -1,3 +1,3 @@
MAJOR=0 MAJOR=0
MINOR=32 MINOR=44
PATCH=0 PATCH=0
+43
View File
@@ -94,3 +94,46 @@
* **tournament:** use HS256 director token for native tournament-server calls ([b98bdd2](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b98bdd2a64eb6c8279bd3cfe15d70628025ef0e5)) * **tournament:** use HS256 director token for native tournament-server calls ([b98bdd2](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b98bdd2a64eb6c8279bd3cfe15d70628025ef0e5))
* **tournament:** use Optional[String] for selfUrl ConfigProperty to avoid startup failure ([28cbc2e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/28cbc2e18447aa8a04a5868889a49b555075d0c6)) * **tournament:** use Optional[String] for selfUrl ConfigProperty to avoid startup failure ([28cbc2e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/28cbc2e18447aa8a04a5868889a49b555075d0c6))
* wrap server list response in ExternalTournamentServerList ([f2d79e4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f2d79e4952aea6bde762c294eb202474b7827054)) * wrap server list response in ExternalTournamentServerList ([f2d79e4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f2d79e4952aea6bde762c294eb202474b7827054))
## (2026-06-23)
### Features
* **analytics:** add Spark batch analytics module ([#70](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/70)) ([39f1657](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39f1657e1db6e84889af338c43be8cb5c03c3ec3))
* NCS-121 pipeline for tournament ([#68](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/68)) ([145f467](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/145f4676483f92bfe6f2d9ca40e2cb4200982e87))
* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
* **reflection:** add GameWritebackEventDto to native reflection configuration ([1aee39c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1aee39c1ad286984501ac4b47da2b72d60b58a6f))
* **reflection:** add native reflection configuration for tournament classes ([65bc6a7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/65bc6a759937543df2d29905688bfa9e68d0c9d4))
* **tournament:** auto-join external tournaments and publish created ones ([#77](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/77)) ([9978b7e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9978b7ea78eb658a225a461b9cd339386c0c14f3))
* **tournament:** federate tournaments across clusters with DB replication ([5b000a6](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/5b000a6e5f04ea6770d1c7ab6bfdaded77a99172))
* **tournament:** remove dynamic server add/remove endpoints ([6d06edd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6d06edda69a50de65cd9efa27f26a4cc6b437f9d))
* **tournament:** seed external server registry from env var on startup ([845dc9c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/845dc9c2935c8bc1be42541dfaf31c9a861d3272))
### Bug Fixes
* **tournament:** replace scala.util.Random singleton with UUID for native image ([a50884a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a50884a11b1de500e74c18fd08d2d102d53cc3e9))
* **tournament:** sync native-server participants and route start ([#78](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/78)) ([1f4e9c8](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1f4e9c8498f55d95ab48758df60c7618445bf6ca))
* **tournament:** use HS256 director token for native tournament-server calls ([b98bdd2](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b98bdd2a64eb6c8279bd3cfe15d70628025ef0e5))
* **tournament:** use Optional[String] for selfUrl ConfigProperty to avoid startup failure ([28cbc2e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/28cbc2e18447aa8a04a5868889a49b555075d0c6))
* wrap server list response in ExternalTournamentServerList ([f2d79e4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f2d79e4952aea6bde762c294eb202474b7827054))
## (2026-06-23)
### Features
* **analytics:** add Spark batch analytics module ([#70](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/70)) ([39f1657](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39f1657e1db6e84889af338c43be8cb5c03c3ec3))
* NCS-121 pipeline for tournament ([#68](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/68)) ([145f467](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/145f4676483f92bfe6f2d9ca40e2cb4200982e87))
* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
* **reflection:** add GameWritebackEventDto to native reflection configuration ([1aee39c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1aee39c1ad286984501ac4b47da2b72d60b58a6f))
* **reflection:** add native reflection configuration for tournament classes ([65bc6a7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/65bc6a759937543df2d29905688bfa9e68d0c9d4))
* **tournament:** auto-join external tournaments and publish created ones ([#77](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/77)) ([9978b7e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9978b7ea78eb658a225a461b9cd339386c0c14f3))
* **tournament:** federate tournaments across clusters with DB replication ([5b000a6](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/5b000a6e5f04ea6770d1c7ab6bfdaded77a99172))
* **tournament:** remove dynamic server add/remove endpoints ([6d06edd](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6d06edda69a50de65cd9efa27f26a4cc6b437f9d))
* **tournament:** seed external server registry from env var on startup ([845dc9c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/845dc9c2935c8bc1be42541dfaf31c9a861d3272))
### Bug Fixes
* **tournament:** mirror bot join onto native twin ([7664042](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/76640421930c26a9260da002c90e2966b97a57a4))
* **tournament:** replace scala.util.Random singleton with UUID for native image ([a50884a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a50884a11b1de500e74c18fd08d2d102d53cc3e9))
* **tournament:** sync native-server participants and route start ([#78](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/78)) ([1f4e9c8](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/1f4e9c8498f55d95ab48758df60c7618445bf6ca))
* **tournament:** use HS256 director token for native tournament-server calls ([b98bdd2](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/b98bdd2a64eb6c8279bd3cfe15d70628025ef0e5))
* **tournament:** use Optional[String] for selfUrl ConfigProperty to avoid startup failure ([28cbc2e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/28cbc2e18447aa8a04a5868889a49b555075d0c6))
* wrap server list response in ExternalTournamentServerList ([f2d79e4](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/f2d79e4952aea6bde762c294eb202474b7827054))
@@ -7,7 +7,7 @@ import java.time.Instant
@Entity @Entity
@Table(name = "tournaments") @Table(name = "tournaments")
class Tournament: class Tournament:
// scalafix:off DisableSyntax.var // scalafix:off
@Id @Id
var id: String = uninitialized var id: String = uninitialized
@@ -33,4 +33,7 @@ class Tournament:
@Column(nullable = true) @Column(nullable = true)
var originServerUrl: String = null var originServerUrl: String = null
@Column(nullable = true)
var nativeTournamentId: String = null
// scalafix:on // scalafix:on
@@ -52,9 +52,9 @@ class TournamentResource:
@PermitAll @PermitAll
def list(): Response = def list(): Response =
val (created, started, finished) = tournamentService.list() val (created, started, finished) = tournamentService.list()
val internalCreated = created.map(t => objectMapper.valueToTree[JsonNode](tournamentService.toDto(t))) val internalCreated = created.map(nativeOverlay)
val internalStarted = started.map(t => objectMapper.valueToTree[JsonNode](tournamentService.toDto(t))) val internalStarted = started.map(nativeOverlay)
val internalFinished = finished.map(t => objectMapper.valueToTree[JsonNode](tournamentService.toDto(t))) val internalFinished = finished.map(nativeOverlay)
val (extCreated, extStarted, extFinished) = registry val (extCreated, extStarted, extFinished) = registry
.serverUrls() .serverUrls()
@@ -96,7 +96,7 @@ class TournamentResource:
val form = CreateTournamentForm(name, nbRounds, clockLimit, clockIncrement, rated) val form = CreateTournamentForm(name, nbRounds, clockLimit, clockIncrement, rated)
val t = tournamentService.create(userId, form) val t = tournamentService.create(userId, form)
selfUrl.ifPresent { url => selfUrl.ifPresent { url =>
registry.serverUrls().foreach { remoteUrl => registry.serverUrls().filterNot(externalClient.isNativeServer).foreach { remoteUrl =>
if !externalClient.replicateTournament(remoteUrl, toReplicateRequest(t), url) then if !externalClient.replicateTournament(remoteUrl, toReplicateRequest(t), url) then
log.warnf("Failed to replicate tournament %s to %s", t.id, remoteUrl) log.warnf("Failed to replicate tournament %s to %s", t.id, remoteUrl)
} }
@@ -115,8 +115,52 @@ class TournamentResource:
"rated" -> t.rated.toString, "rated" -> t.rated.toString,
), ),
) )
if !externalClient.publishNative(nativeServerUrl, directorName, form) then externalClient.publishNative(nativeServerUrl, directorName, form) match
log.warnf("Failed to publish tournament %s to native server %s", t.id, nativeServerUrl) case Some(nativeId) => tournamentService.setNativeTournamentId(t.id, nativeId)
case None => log.warnf("Failed to publish tournament %s to native server %s", t.id, nativeServerUrl)
// Mirror a bot join onto the native twin so it surfaces in the UI, which reads participant and
// standings fields from the native server (see nativeOverlay).
private def joinNativeTwin(id: String, botName: String): Unit =
if nativeServerUrl.nonEmpty then
tournamentService.get(id).flatMap(nativeIdFor).foreach { nativeId =>
if externalClient.joinNativeAsBot(nativeServerUrl, nativeId, botName) then
log.infof("Joined bot %s on native twin %s of tournament %s", botName, nativeId, id)
else log.warnf("Failed to join bot %s on native twin %s of tournament %s", botName, nativeId, id)
}
// Resolve the native-server twin of a local tournament. Backfills the stored id by matching
// fullName against the native list for tournaments created before the id was captured.
private def nativeIdFor(t: de.nowchess.tournament.domain.Tournament): Option[String] =
if nativeServerUrl.isEmpty then None
else
Option(t.nativeTournamentId).filter(_.nonEmpty).orElse {
val found = externalClient
.fetchList(nativeServerUrl)
.flatMap { node =>
Seq("created", "started", "finished").iterator
.flatMap(k => node.path(k).elements().asScala)
.find(_.path("fullName").asText() == t.fullName)
.map(_.path("id").asText())
.filter(_.nonEmpty)
}
found.foreach(id => tournamentService.setNativeTournamentId(t.id, id))
found
}
// Overlay live participant/standings/status fields from the native twin onto a local DTO so
// bots that joined directly on the native server are reflected in NowChess.
private def nativeOverlay(t: de.nowchess.tournament.domain.Tournament): JsonNode =
val standings = tournamentService.getStandings(t.id)
val dto = objectMapper.valueToTree[JsonNode](tournamentService.toDto(t, standings))
nativeIdFor(t).flatMap(nid => externalClient.fetch(nativeServerUrl, nid)) match
case Some(native) =>
val merged = dto.deepCopy[com.fasterxml.jackson.databind.node.ObjectNode]()
Seq("nbPlayers", "standing", "status", "round", "winner").foreach { field =>
if native.has(field) then merged.set(field, native.get(field))
}
merged
case None => dto
private def encodeForm(params: Map[String, String]): String = private def encodeForm(params: Map[String, String]): String =
params params
@@ -132,8 +176,7 @@ class TournamentResource:
def get(@PathParam("id") id: String): Response = def get(@PathParam("id") id: String): Response =
tournamentService.get(id) match tournamentService.get(id) match
case Some(t) => case Some(t) =>
val standings = tournamentService.getStandings(id) Response.ok(nativeOverlay(t)).build()
Response.ok(tournamentService.toDto(t, standings)).build()
case None => case None =>
resolveServer(id) resolveServer(id)
.flatMap(url => externalClient.fetch(url, id).map(node => Response.ok(node).build())) .flatMap(url => externalClient.fetch(url, id).map(node => Response.ok(node).build()))
@@ -178,6 +221,13 @@ class TournamentResource:
val auth = Option(headers.getHeaderString("Authorization")) val auth = Option(headers.getHeaderString("Authorization"))
val (status, body) = externalClient.proxyPost(originUrl, s"api/tournament/$id/start", auth) val (status, body) = externalClient.proxyPost(originUrl, s"api/tournament/$id/start", auth)
Response.status(status).entity(body).build() Response.status(status).entity(body).build()
case None =>
tournamentService.get(id).flatMap(nativeIdFor) match
case Some(nativeId) =>
val auth = Option(headers.getHeaderString("Authorization"))
val (status, body) = externalClient.proxyPost(nativeServerUrl, s"api/tournament/$nativeId/start", auth)
if status / 100 == 2 then tournamentService.markStatus(id, "started")
Response.status(status).entity(body).build()
case None => case None =>
tournamentService.start(id, userId) match tournamentService.start(id, userId) match
case Right(t) => Response.ok(tournamentService.toDto(t)).build() case Right(t) => Response.ok(tournamentService.toDto(t)).build()
@@ -210,7 +260,14 @@ class TournamentResource:
val botId = Option(jwt.getSubject).getOrElse("") val botId = Option(jwt.getSubject).getOrElse("")
val botName = Option(jwt.getClaim[AnyRef]("name")).map(_.toString).getOrElse(botId) val botName = Option(jwt.getClaim[AnyRef]("name")).map(_.toString).getOrElse(botId)
tournamentService.join(id, botId, botName) match tournamentService.join(id, botId, botName) match
case Right(_) => Response.ok(OkDto()).build() case Right(_) =>
joinNativeTwin(id, botName)
Response.ok(OkDto()).build()
// Already in the NowChess participant list but possibly never mirrored onto the native
// twin (where the UI reads participants from) make the native join idempotent.
case Left(TournamentError.AlreadyJoined) =>
joinNativeTwin(id, botName)
Response.ok(OkDto()).build()
case Left(error) => case Left(error) =>
error match error match
case TournamentError.NotFound(_) => case TournamentError.NotFound(_) =>
@@ -317,7 +374,8 @@ class TournamentResource:
tournamentService.get(id) match tournamentService.get(id) match
case Some(t) if Option(t.originServerUrl).isDefined => case Some(t) if Option(t.originServerUrl).isDefined =>
val auth = Option(headers.getHeaderString("Authorization")) val auth = Option(headers.getHeaderString("Authorization"))
externalClient.proxyGetStream(t.originServerUrl, s"api/tournament/$id/stream", auth) externalClient
.proxyGetStream(t.originServerUrl, s"api/tournament/$id/stream", auth)
.map { inputStream => .map { inputStream =>
Response Response
.ok(new StreamingOutput { .ok(new StreamingOutput {
@@ -334,7 +392,9 @@ class TournamentResource:
.`type`("application/x-ndjson") .`type`("application/x-ndjson")
.build() .build()
} }
.getOrElse(Response.status(Response.Status.NOT_FOUND).entity(ErrorDto(s"Tournament $id stream unavailable")).build()) .getOrElse(
Response.status(Response.Status.NOT_FOUND).entity(ErrorDto(s"Tournament $id stream unavailable")).build(),
)
case Some(_) => case Some(_) =>
val botId = Option(jwt.getSubject).getOrElse("") val botId = Option(jwt.getSubject).getOrElse("")
val queue = new java.util.concurrent.LinkedBlockingQueue[Option[String]]() val queue = new java.util.concurrent.LinkedBlockingQueue[Option[String]]()
@@ -440,7 +500,8 @@ class TournamentResource:
.getOrElse(Response.status(Response.Status.NOT_FOUND).build()) .getOrElse(Response.status(Response.Status.NOT_FOUND).build())
private def resolveServer(tournamentId: String): Option[String] = private def resolveServer(tournamentId: String): Option[String] =
tournamentService.get(tournamentId) tournamentService
.get(tournamentId)
.flatMap(t => Option(t.originServerUrl)) .flatMap(t => Option(t.originServerUrl))
.orElse(registry.findServerUrl(tournamentId)) .orElse(registry.findServerUrl(tournamentId))
.orElse { .orElse {
@@ -30,7 +30,7 @@ class ExternalTournamentClient:
// RS256 user token to it swap in the director token registered on that server. // RS256 user token to it swap in the director token registered on that server.
private def normalize(url: String): String = url.stripSuffix("/") private def normalize(url: String): String = url.stripSuffix("/")
private def isNativeServer(serverUrl: String): Boolean = def isNativeServer(serverUrl: String): Boolean =
nativeServerUrl.nonEmpty && normalize(serverUrl) == normalize(nativeServerUrl) nativeServerUrl.nonEmpty && normalize(serverUrl) == normalize(nativeServerUrl)
private def directorBearer(): Option[String] = private def directorBearer(): Option[String] =
@@ -45,24 +45,27 @@ class ExternalTournamentClient:
private def authFor(serverUrl: String, userAuth: Option[String]): Option[String] = private def authFor(serverUrl: String, userAuth: Option[String]): Option[String] =
if isNativeServer(serverUrl) then directorBearer() else userAuth if isNativeServer(serverUrl) then directorBearer() else userAuth
def publishNative(serverUrl: String, directorName: String, form: String): Boolean = def publishNative(serverUrl: String, directorName: String, form: String): Option[String] =
val token = directorToken.orElse { val token = directorToken.orElse {
val fresh = registerDirector(serverUrl, directorName) val fresh = registerDirector(serverUrl, directorName)
directorToken = fresh directorToken = fresh
fresh fresh
} }
token.exists { tok => token.flatMap { tok =>
if createNative(serverUrl, tok, form) then true createNative(serverUrl, tok, form).orElse {
else
val refreshed = registerDirector(serverUrl, directorName) val refreshed = registerDirector(serverUrl, directorName)
directorToken = refreshed directorToken = refreshed
refreshed.exists(createNative(serverUrl, _, form)) refreshed.flatMap(createNative(serverUrl, _, form))
}
} }
private def registerDirector(serverUrl: String, name: String): Option[String] = private def registerDirector(serverUrl: String, name: String): Option[String] =
registerAccount(serverUrl, name, isBot = false)
private def registerAccount(serverUrl: String, name: String, isBot: Boolean): Option[String] =
Try { Try {
val client = buildClient() val client = buildClient()
val body = s"""{"name":"${name.replace("\"", "\\\"")}","isBot":false}""" val body = s"""{"name":"${name.replace("\"", "\\\"")}","isBot":$isBot}"""
val response = client val response = client
.target(s"$serverUrl/api/auth/register") .target(s"$serverUrl/api/auth/register")
.request(MediaType.APPLICATION_JSON) .request(MediaType.APPLICATION_JSON)
@@ -76,7 +79,25 @@ class ExternalTournamentClient:
client.close() client.close()
}.getOrElse(None) }.getOrElse(None)
private def createNative(serverUrl: String, token: String, form: String): Boolean = // The tournament server holds only the director token, which cannot join as a bot. Register the
// bot on the native server by name to mint a bot token, then join the native twin as that bot.
def joinNativeAsBot(serverUrl: String, tournamentId: String, botName: String): Boolean =
registerAccount(serverUrl, botName, isBot = true).exists { token =>
Try {
val client = buildClient()
val response = client
.target(s"$serverUrl/api/tournament/$tournamentId/join")
.request(MediaType.APPLICATION_JSON)
.header("Authorization", s"Bearer $token")
.post(Entity.json(""))
try response.getStatus / 100 == 2 || response.getStatus == 409
finally
response.close()
client.close()
}.getOrElse(false)
}
private def createNative(serverUrl: String, token: String, form: String): Option[String] =
Try { Try {
val client = buildClient() val client = buildClient()
val response = client val response = client
@@ -84,11 +105,14 @@ class ExternalTournamentClient:
.request(MediaType.APPLICATION_JSON) .request(MediaType.APPLICATION_JSON)
.header("Authorization", s"Bearer $token") .header("Authorization", s"Bearer $token")
.post(Entity.entity(form, MediaType.APPLICATION_FORM_URLENCODED)) .post(Entity.entity(form, MediaType.APPLICATION_FORM_URLENCODED))
try response.getStatus / 100 == 2 try
if response.getStatus / 100 == 2 then
Option(objectMapper.readTree(response.readEntity(classOf[String])).path("id").asText()).filter(_.nonEmpty)
else None
finally finally
response.close() response.close()
client.close() client.close()
}.getOrElse(false) }.getOrElse(None)
def fetchList(serverUrl: String): Option[JsonNode] = def fetchList(serverUrl: String): Option[JsonNode] =
Try { Try {
@@ -82,6 +82,14 @@ class TournamentService:
def get(id: String): Option[Tournament] = def get(id: String): Option[Tournament] =
tournamentRepository.findOptById(id) tournamentRepository.findOptById(id)
@Transactional
def setNativeTournamentId(id: String, nativeId: String): Unit =
tournamentRepository.findOptById(id).foreach(_.nativeTournamentId = nativeId)
@Transactional
def markStatus(id: String, status: String): Unit =
tournamentRepository.findOptById(id).foreach(_.status = status)
def list(): (List[Tournament], List[Tournament], List[Tournament]) = def list(): (List[Tournament], List[Tournament], List[Tournament]) =
( (
tournamentRepository.findByStatus("created"), tournamentRepository.findByStatus("created"),
@@ -30,3 +30,7 @@ nowchess:
secret: test-secret secret: test-secret
auth: auth:
enabled: false enabled: false
tournament:
self-url: ""
external-servers: ""
native-server-url: "http://localhost:1"
+1 -1
View File
@@ -1,3 +1,3 @@
MAJOR=0 MAJOR=0
MINOR=7 MINOR=9
PATCH=0 PATCH=0