feat(analytics): feed Lichess PGN dumps into Spark batch jobs

Add GameSource: normalises game records into a shared schema and selects backend via NOWCHESS_PGN_PATH. Unset = PostgreSQL game_records (unchanged); set = a Lichess PGN dump (file or http(s) URL). - Parse Lichess PGN with Spark SQL string functions only (no UDFs). - URLs fetched once via SparkContext.addFile, distributed to executors. - .pgn.zst decompressed in-process via zstd-jni, plain .pgn redistributed. - All four batch jobs read through GameSource and skip JDBC write-back in PGN mode (Parquet/CSV output only). Enables driving the analytics demo straight from https://database.lichess.org standard dumps. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
feat(official-bots): park expert bot on tournament server at startup
2026-06-17 10:40:29 +02:00 · 2026-06-17 10:34:48 +02:00 · 2026-06-17 07:24:13 +00:00 · 2026-06-17 09:10:13 +02:00 · 2026-06-17 09:10:01 +02:00
17 changed files with 312 additions and 111 deletions
@@ -569,3 +569,44 @@

 * Revert "feat: add authentication permissions for metrics endpoints in application.yml" ([a298417](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a298417b9e4d68dc73bbf40be63d9484536e9f83))
 * Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
+##  (2026-06-17)
+
+### Features
+
+* **account:** implement token pair handling for login and refresh endpoints ([9296db8](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/9296db88b7131bbda9b9b0da65c327ef9063ee31))
+* add authentication permissions for metrics endpoints in application.yml ([04edd4d](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/04edd4d6fd8a63196c36f6d67992832febc9bebb))
+* add initialization metrics for various services ([d438e97](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d438e97f32bdde0bfc63c1b4a8cc810cdd093166))
+* add OpenTelemetry trace configuration with parentbased sampler ([3904d5a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3904d5ad8ad4930ddee65287a7bfab785a6148f5))
+* **api:** define shared EventEnvelope and EventType for Redis EventBus ([#61](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/61)) ([595c172](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/595c172900da99de367c274488c3ccbeaef55882))
+* **bot-platform:** migrate BotRegistry to Redis Streams consumer group ([#63](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/63)) ([0ad2e10](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/0ad2e10999213df6dd00f0c31a088c28a4dc0083))
+* **config:** add H2 database configuration for testing environment ([39c9e49](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39c9e492cef2515368c074da9406f95e9c0c9e64))
+* **config:** update application.yml for PostgreSQL and remove staging/production configurations ([2404e61](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/2404e6164c3b50ffccbea5238d636060d6abe4d6))
+* **config:** update application.yml for staging and production environments ([6113432](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6113432a14c476a3a0dfc0d449e17d023697f2ba))
+* configure logging and add OpenTelemetry support ([#49](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/49)) ([d57c488](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d57c4886612d1d92da0e1b79209fc83e6ef537a1))
+* **docker:** add .dockerignore and .gitignore files for build exclusions ([c987d8e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c987d8e258c0e6c4cfbdaa8381c64c410d7a2b83))
+* **docker:** add Dockerfiles for building Quarkus application in native and JVM modes ([3f2d2bb](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3f2d2bb4c97fa8cddba66e1da4427c54236dfeed))
+* **docker:** add Dockerfiles for Quarkus application in JVM and native modes ([34b9933](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/34b993304670cf2aa62cd2f6460cee7b9864b08e))
+* **events:** migrate game-creation and bot flows to Redis Streams NCS-89 ([#62](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/62)) ([a24924c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a24924c23057db3d700a75dbc4333557789cd991))
+* **logging:** add DEBUG/INFO/WARN logging across services (NCS-72) ([#41](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/41)) ([804a4bf](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/804a4bf179e3dfb19e2be4390e7e543caf5237c6))
+* NCS-78 Add Traceability to the Applications ([#46](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/46)) ([649566e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/649566eb3fcf38f91c8896a739f74ea318af312d))
+* NCS-78 Add Traceability to the Applications ([#47](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/47)) ([87dfc6c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/87dfc6c2bcce7f7d58fc641bd8d468a2e584c108))
+* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
+* **reflection:** add native reflection configuration for tournament classes ([65bc6a7](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/65bc6a759937543df2d29905688bfa9e68d0c9d4))
+* true-microservices ([#40](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/40)) ([5909242](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/590924254e8a2754de661a57a03e43f89ceb6299))
+* update application.yml with new API root paths and add Micrometer and OpenTelemetry dependencies ([72ce262](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/72ce262bc491f94297700e6002fb5d0812e2cc2a))
+* **ws:** migrate challenge notifications to Redis Streams ([#66](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/66)) ([55f102c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/55f102cbaa684be94a158b16aaa42a50b36afaf3))
+
+### Bug Fixes
+
+* **account:** configure JDBC connection pool size to prevent exhaustion under load ([29072ef](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/29072efbfb1cfa1c3b1a85b4c1a587c971d245f9))
+* **auth:** add InternalClientHeadersFactory for custom client headers management ([e279c39](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/e279c39246470156bf11e745ee72204018d4229d))
+* enable official bots to connect to external tournament server ([#71](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/71)) ([688d30e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/688d30e2b10026923372be5fca3c63eaaee2de2a))
+* NCS-84 More Verbose Logging ([#51](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/51)) ([4ad92ab](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/4ad92ab23698267f8faa59c4e18388d4a0042cca))
+* **official-bots:** NCS-70-auto-register official bots with account service ([#59](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/59)) ([7117a93](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/7117a93376272094d0b1a6abf2121254ce396684))
+* remove unused HTTP root-path configurations from application.yml ([3ed3e59](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3ed3e59ee456d54cd3d65ece4f36623e256b9736))
+* **tests:** update token path to accessToken in ChallengeResourceTest ([354db11](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/354db11972342c47a1034303c11bccfb92e60109))
+
+### Reverts
+
+* Revert "feat: add authentication permissions for metrics endpoints in application.yml" ([a298417](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a298417b9e4d68dc73bbf40be63d9484536e9f83))
+* Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
@@ -239,6 +239,7 @@ class AccountService:
      .subject(botId.toString)
      .expiresAt(Long.MaxValue)
      .claim("type", "bot")
+      .claim("isBot", true)
      .claim("name", botName)
      .sign()

@@ -1,3 +1,3 @@
 MAJOR=0
-MINOR=24
+MINOR=25
 PATCH=0
@@ -22,6 +22,9 @@
 //   NOWCHESS_JDBC_URL  (default: jdbc:postgresql://localhost:5432/nowchess)
 //   NOWCHESS_DB_USER   (default: nowchess)
 //   NOWCHESS_DB_PASS   (default: nowchess)
+//   NOWCHESS_PGN_PATH  (optional) — file or http(s) URL of a Lichess PGN dump (.pgn or .pgn.zst).
+//                      When set, all batch jobs read games from the dump instead of PostgreSQL and
+//                      skip JDBC write-back (Parquet/CSV output only). Demo data source.

 plugins {
    id("scala")
@@ -71,6 +74,10 @@ dependencies {

    // PostgreSQL JDBC driver bundled so it is available on executor classpath.
    implementation("org.postgresql:postgresql:42.7.4")
+
+    // zstd-jni: decompress Lichess .pgn.zst dumps in-process. Provided at runtime by Spark
+    // (it uses zstd-jni internally for shuffle/event-log compression), so compile-only here.
+    compileOnly("com.github.luben:zstd-jni:1.5.6-9")
 }

 application {
@@ -0,0 +1,119 @@
+package de.nowchess.analytics
+
+import org.apache.spark.SparkFiles
+import org.apache.spark.sql.DataFrame
+import org.apache.spark.sql.SparkSession
+import org.apache.spark.sql.functions as F
+
+/** Normalised game-record source for the batch jobs.
+  *
+  * Every batch job consumes the same five-column shape:
+  *   - white_id, black_id : player identifiers
+  *   - result             : one of "white", "black", "draw"
+  *   - move_count         : number of plies
+  *   - pgn                : full PGN ("[Event …]…\n\n1. e4 …"), header and movetext separated by a blank line
+  *
+  * Two backends, selected by the `NOWCHESS_PGN_PATH` environment variable:
+  *   - unset → PostgreSQL `game_records` table (production)
+  *   - set   → a Lichess PGN dump file/URL (demo). Point it at a `lichess_db_standard_rated_*.pgn[.zst]`
+  *             to drive every batch job from real Lichess games.
+  *
+  * Lichess parsing uses only Spark SQL string functions — no UDFs — so Catalyst can push predicates,
+  * matching the no-UDF approach already used in OpeningBookJob.
+  */
+object GameSource:
+
+  private val PgnPathEnv = "NOWCHESS_PGN_PATH"
+
+  /** True when a Lichess PGN dump is configured; jobs use this to skip JDBC write-back. */
+  def isPgnMode: Boolean = sys.env.contains(PgnPathEnv)
+
+  def load(spark: SparkSession, jdbcUrl: String, dbUser: String, dbPass: String): DataFrame =
+    sys.env.get(PgnPathEnv) match
+      case Some(path) => fromLichessPgn(spark, path)
+      case None       => fromJdbc(spark, jdbcUrl, dbUser, dbPass)
+
+  def fromJdbc(spark: SparkSession, jdbcUrl: String, dbUser: String, dbPass: String): DataFrame =
+    spark.read
+      .format("jdbc")
+      .option("url", jdbcUrl)
+      .option("dbtable", "game_records")
+      .option("user", dbUser)
+      .option("password", dbPass)
+      .option("driver", "org.postgresql.Driver")
+      .option("fetchsize", "10000")
+      .load()
+      .select("white_id", "black_id", "result", "move_count", "pgn")
+
+  /** Parses a Lichess PGN dump into the normalised game shape.
+    *
+    * `path` may be:
+    *   - an http(s)/ftp URL — fetched once via SparkContext.addFile and distributed to executors, then read
+    *     from the local replica (no S3/PVC needed; handy for a staging demo)
+    *   - any Hadoop-readable path (file://, hdfs://, s3a://, …)
+    *
+    * `.zst` dumps (Lichess' native format) are decompressed in-process via zstd-jni; `.gz`/`.bz2` are
+    * handled by Spark's text reader codecs.
+    *
+    * Records are split on the "[Event " tag that opens every game, so each row holds one complete game
+    * (the empty fragment before the first game is filtered out). Header tags are read with regexp_extract;
+    * the movetext (after the blank line) is cleaned of clock/eval comments and move numbers to count plies.
+    */
+  def fromLichessPgn(spark: SparkSession, path: String): DataFrame =
+    val resolved = resolvePath(spark, path)
+    val record   = F.col("value")
+
+    val resultTag = F.regexp_extract(record, "Result \"([^\"]*)\"", 1)
+    val result = F
+      .when(resultTag === "1-0", "white")
+      .when(resultTag === "0-1", "black")
+      .when(resultTag === "1/2-1/2", "draw")
+      .otherwise(F.lit(null).cast("string"))
+
+    val moveText  = F.coalesce(F.split(record, "\n\n").getItem(1), F.lit(""))
+    val noComment = F.regexp_replace(moveText, "\\{[^}]*\\}", "")
+    val noResult  = F.regexp_replace(noComment, "(1-0|0-1|1/2-1/2|\\*)", "")
+    val noNumbers = F.regexp_replace(noResult, "\\d+\\.+", " ")
+    val plies     = F.size(F.filter(F.split(F.trim(noNumbers), "\\s+"), tok => F.length(tok) > 0))
+
+    spark.read
+      .option("lineSep", "[Event ")
+      .text(resolved)
+      .filter(F.length(F.trim(record)) > 0)
+      .select(
+        F.regexp_extract(record, "White \"([^\"]*)\"", 1).as("white_id"),
+        F.regexp_extract(record, "Black \"([^\"]*)\"", 1).as("black_id"),
+        result.as("result"),
+        plies.as("move_count"),
+        F.concat(F.lit("[Event "), record).as("pgn"),
+      )
+      .filter((F.col("white_id") =!= "").and(F.col("black_id") =!= ""))
+
+  /** Turns an http(s)/ftp URL into a cluster-local path by fetching it once with SparkContext.addFile,
+    * which distributes the file to every executor. `.zst` is decompressed in-process and the plain `.pgn`
+    * is redistributed. Non-URL paths are returned unchanged.
+    */
+  private def resolvePath(spark: SparkSession, path: String): String =
+    if !path.matches("^(https?|ftp)://.*") then path
+    else
+      spark.sparkContext.addFile(path)
+      val local = SparkFiles.get(baseName(path))
+      if !local.endsWith(".zst") then "file://" + local
+      else distribute(spark, decompressZstd(local))
+
+  private def baseName(path: String): String = path.substring(path.lastIndexOf('/') + 1)
+
+  private def distribute(spark: SparkSession, localPath: String): String =
+    spark.sparkContext.addFile("file://" + localPath)
+    "file://" + SparkFiles.get(baseName(localPath))
+
+  /** Decompresses a `.zst` file to a temp `.pgn` using zstd-jni (bundled with Spark at runtime). */
+  private def decompressZstd(srcPath: String): String =
+    val out = java.io.File.createTempFile("lichess-", ".pgn")
+    out.deleteOnExit()
+    val in = com.github.luben.zstd.ZstdInputStream(
+      java.io.BufferedInputStream(java.io.FileInputStream(srcPath)),
+    )
+    try java.nio.file.Files.copy(in, out.toPath, java.nio.file.StandardCopyOption.REPLACE_EXISTING)
+    finally in.close()
+    out.getAbsolutePath
@@ -37,15 +37,8 @@ object OpeningBookJob:
      outputDir: String,
      maxPlies: Int,
  ): Unit =
-    val games = spark.read
-      .format("jdbc")
-      .option("url", jdbcUrl)
-      .option("dbtable", "game_records")
-      .option("user", dbUser)
-      .option("password", dbPass)
-      .option("driver", "org.postgresql.Driver")
-      .option("fetchsize", "10000")
-      .load()
+    val games = GameSource
+      .load(spark, jdbcUrl, dbUser, dbPass)
      .select("pgn", "result")
      .filter(F.col("result").isNotNull.and(F.col("pgn").isNotNull))

@@ -79,15 +72,16 @@ object OpeningBookJob:
      .option("header", "true")
      .csv(s"$outputDir/opening_book_top1000")

-    top1000.write
-      .mode("overwrite")
-      .format("jdbc")
-      .option("url", jdbcUrl)
-      .option("dbtable", "analytics_opening_stats")
-      .option("user", dbUser)
-      .option("password", dbPass)
-      .option("driver", "org.postgresql.Driver")
-      .save()
+    if !GameSource.isPgnMode then
+      top1000.write
+        .mode("overwrite")
+        .format("jdbc")
+        .option("url", jdbcUrl)
+        .option("dbtable", "analytics_opening_stats")
+        .option("user", dbUser)
+        .option("password", dbPass)
+        .option("driver", "org.postgresql.Driver")
+        .save()

  /** Extracts the first `maxPlies` moves from a PGN column as a space-separated string.
    *
@@ -50,15 +50,8 @@ object PlayerClusteringJob:
      outputDir: String,
      k: Int,
  ): Unit =
-    val games = spark.read
-      .format("jdbc")
-      .option("url", jdbcUrl)
-      .option("dbtable", "game_records")
-      .option("user", dbUser)
-      .option("password", dbPass)
-      .option("driver", "org.postgresql.Driver")
-      .option("fetchsize", "10000")
-      .load()
+    val games = GameSource
+      .load(spark, jdbcUrl, dbUser, dbPass)
      .select("white_id", "black_id", "result", "move_count")
      .filter(F.col("result").isNotNull)

@@ -126,25 +119,26 @@ object PlayerClusteringJob:
      .option("header", "true")
      .csv(s"$outputDir/cluster_archetypes")

-    clustersDf.write
-      .mode("overwrite")
-      .format("jdbc")
-      .option("url", jdbcUrl)
-      .option("dbtable", "analytics_player_clusters")
-      .option("user", dbUser)
-      .option("password", dbPass)
-      .option("driver", "org.postgresql.Driver")
-      .save()
+    if !GameSource.isPgnMode then
+      clustersDf.write
+        .mode("overwrite")
+        .format("jdbc")
+        .option("url", jdbcUrl)
+        .option("dbtable", "analytics_player_clusters")
+        .option("user", dbUser)
+        .option("password", dbPass)
+        .option("driver", "org.postgresql.Driver")
+        .save()

-    archetypes.write
-      .mode("overwrite")
-      .format("jdbc")
-      .option("url", jdbcUrl)
-      .option("dbtable", "analytics_cluster_archetypes")
-      .option("user", dbUser)
-      .option("password", dbPass)
-      .option("driver", "org.postgresql.Driver")
-      .save()
+      archetypes.write
+        .mode("overwrite")
+        .format("jdbc")
+        .option("url", jdbcUrl)
+        .option("dbtable", "analytics_cluster_archetypes")
+        .option("user", dbUser)
+        .option("password", dbPass)
+        .option("driver", "org.postgresql.Driver")
+        .save()

  private def buildPlayerStats(games: org.apache.spark.sql.DataFrame): org.apache.spark.sql.DataFrame =
    val asWhite = games.select(
@@ -53,15 +53,8 @@ object PlayerGraphJob:
      dbPass: String,
      outputDir: String,
  ): Unit =
-    val gamesRdd: RDD[Row] = spark.read
-      .format("jdbc")
-      .option("url", jdbcUrl)
-      .option("dbtable", "game_records")
-      .option("user", dbUser)
-      .option("password", dbPass)
-      .option("driver", "org.postgresql.Driver")
-      .option("fetchsize", "10000")
-      .load()
+    val gamesRdd: RDD[Row] = GameSource
+      .load(spark, jdbcUrl, dbUser, dbPass)
      .select("white_id", "black_id", "result")
      .filter(F.col("result").isNotNull)
      .rdd
@@ -116,15 +109,16 @@ object PlayerGraphJob:
      .mode("overwrite")
      .parquet(s"$outputDir/player_graph")

-    result.write
-      .mode("overwrite")
-      .format("jdbc")
-      .option("url", jdbcUrl)
-      .option("dbtable", "analytics_player_graph")
-      .option("user", dbUser)
-      .option("password", dbPass)
-      .option("driver", "org.postgresql.Driver")
-      .save()
+    if !GameSource.isPgnMode then
+      result.write
+        .mode("overwrite")
+        .format("jdbc")
+        .option("url", jdbcUrl)
+        .option("dbtable", "analytics_player_graph")
+        .option("user", dbUser)
+        .option("password", dbPass)
+        .option("driver", "org.postgresql.Driver")
+        .save()

    // How many players belong to each connected component?
    // A large dominant component + many singletons is the expected shape.
@@ -34,15 +34,8 @@ object PlayerStatsJob:
      dbPass: String,
      outputDir: String,
  ): Unit =
-    val games = spark.read
-      .format("jdbc")
-      .option("url", jdbcUrl)
-      .option("dbtable", "game_records")
-      .option("user", dbUser)
-      .option("password", dbPass)
-      .option("driver", "org.postgresql.Driver")
-      .option("fetchsize", "10000")
-      .load()
+    val games = GameSource
+      .load(spark, jdbcUrl, dbUser, dbPass)
      .select("white_id", "black_id", "result", "move_count")
      .filter(F.col("result").isNotNull)

@@ -84,12 +77,13 @@ object PlayerStatsJob:
      .mode("overwrite")
      .parquet(s"$outputDir/player_stats")

-    stats.write
-      .mode("overwrite")
-      .format("jdbc")
-      .option("url", jdbcUrl)
-      .option("dbtable", "analytics_player_stats")
-      .option("user", dbUser)
-      .option("password", dbPass)
-      .option("driver", "org.postgresql.Driver")
-      .save()
+    if !GameSource.isPgnMode then
+      stats.write
+        .mode("overwrite")
+        .format("jdbc")
+        .option("url", jdbcUrl)
+        .option("dbtable", "analytics_player_stats")
+        .option("user", dbUser)
+        .option("password", dbPass)
+        .option("driver", "org.postgresql.Driver")
+        .save()
@@ -281,3 +281,32 @@
 ### Reverts

 * Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
+##  (2026-06-17)
+
+### Features
+
+* add initialization metrics for various services ([d438e97](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d438e97f32bdde0bfc63c1b4a8cc810cdd093166))
+* add OpenTelemetry trace configuration with parentbased sampler ([3904d5a](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3904d5ad8ad4930ddee65287a7bfab785a6148f5))
+* **analytics:** add Spark batch analytics module ([#70](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/70)) ([39f1657](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/39f1657e1db6e84889af338c43be8cb5c03c3ec3))
+* **config:** update application.yml for PostgreSQL and remove staging/production configurations ([2404e61](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/2404e6164c3b50ffccbea5238d636060d6abe4d6))
+* **config:** update application.yml for staging and production environments ([6113432](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/6113432a14c476a3a0dfc0d449e17d023697f2ba))
+* configure logging and add OpenTelemetry support ([#49](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/49)) ([d57c488](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/d57c4886612d1d92da0e1b79209fc83e6ef537a1))
+* **docker:** add .dockerignore and .gitignore files for build exclusions ([c987d8e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c987d8e258c0e6c4cfbdaa8381c64c410d7a2b83))
+* **docker:** add Dockerfiles for building Quarkus application in native and JVM modes ([3f2d2bb](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/3f2d2bb4c97fa8cddba66e1da4427c54236dfeed))
+* **docker:** add Dockerfiles for Quarkus application in JVM and native modes ([34b9933](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/34b993304670cf2aa62cd2f6460cee7b9864b08e))
+* **events:** migrate game-creation and bot flows to Redis Streams NCS-89 ([#62](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/62)) ([a24924c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/a24924c23057db3d700a75dbc4333557789cd991))
+* NCS-78 Add Traceability to the Applications ([#46](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/46)) ([649566e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/649566eb3fcf38f91c8896a739f74ea318af312d))
+* NCS-78 Add Traceability to the Applications ([#47](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/47)) ([87dfc6c](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/87dfc6c2bcce7f7d58fc641bd8d468a2e584c108))
+* NCS-82 add Swiss-system tournament module ([#55](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/55)) ([c5661de](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/c5661de4a0ebf4b33211f5a391840dcf744656b7))
+* **official-bots:** consume GameOver stream for bot cleanup ([#67](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/67)) ([db9d153](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/db9d1533912f4b41c4d1ca80ccffdde5d23d6ff6))
+* true-microservices ([#40](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/40)) ([5909242](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/590924254e8a2754de661a57a03e43f89ceb6299))
+
+### Bug Fixes
+
+* enable official bots to connect to external tournament server ([#71](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/71)) ([688d30e](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/688d30e2b10026923372be5fca3c63eaaee2de2a))
+* **official-bots:** configure JWT verification ([#72](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/72)) ([98c64fc](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/98c64fc0d56dc542beb31c75f4b9056d91de03cd))
+* **official-bots:** NCS-70-auto-register official bots with account service ([#59](https://git.janis-eccarius.de/NowChess/NowChessSystems/issues/59)) ([7117a93](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/7117a93376272094d0b1a6abf2121254ce396684))
+
+### Reverts
+
+* Revert "refactor: update metrics paths formatting in application.yml for clarity" ([3870566](https://git.janis-eccarius.de/NowChess/NowChessSystems/commit/38705663498d5f47c40dafe2f26198589ede8656))
@@ -12,6 +12,12 @@ quarkus:
    enabled: true
  log:
    level: INFO
+  mp:
+    jwt:
+      verify:
+        publickey:
+          location: ${JWT_PUBLIC_KEY_PATH:keys/public.pem}
+        issuer: nowchess

 nowchess:
  redis:
@@ -0,0 +1,9 @@
+-----BEGIN PUBLIC KEY-----
+MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAxDsnsCAl0vQx7Vu9CLDZ
+g0SG05NgUzu9T+3DTEaHGq60T2uriO8BenwyvsF3BnDqTbKf4voohZ1DNfzdbT1J
+Fj8B62FrDmxcO+sp1/b5HUCJP6y2uSRCmzOHe5k7Pk1IEi72FgBpKXSRkFibRlVf
+634g7mgsPZAQ9PJEsv4Qvm05T9L6+Gmq6N3bMVLKRXs4RhDhaFbYH9GtUg1eI0yH
+YjGyRfqzW/nqVMstOLHt8CuPouq4p7eMzeDH3YHkxPm4GG5foCXMOd2DZrW0SCcr
+7dhFeNVWzQ2m53eOhBzNQX+v3pgjVStsePhBRt2LyGfwkNzmqDgqWsMzSHRMY+cn
+WQIDAQAB
+-----END PUBLIC KEY-----
@@ -2,6 +2,7 @@ package de.nowchess.bot.resource

 case class JoinTournamentRequest(
    tournamentId: String,
+    botToken: String,
    difficulty: String,
    serverUrl: Option[String],
 )
@@ -33,7 +33,7 @@ class TournamentJoinResource:
      difficulty,
      serverUrl,
    )
-    player.joinTournament(req.tournamentId, difficulty, serverUrl) match
+    player.joinTournament(req.tournamentId, req.botToken, difficulty, serverUrl) match
      case Right(botId) =>
        val resp = JoinTournamentResponse(botId, difficulty, "joining")
        Response.ok(resp).build()
@@ -20,7 +20,7 @@ object TournamentBotConfig:
      tournamentId <- env.get("TOURNAMENT_ID").filter(_.nonEmpty)
      token        <- env.get("TOURNAMENT_BOT_TOKEN").filter(_.nonEmpty)
      botId        <- jwtSubject(token)
-      serverUrl  = env.getOrElse("TOURNAMENT_SERVER_URL", "http://localhost:8089")
+      serverUrl  = env.getOrElse("TOURNAMENT_SERVER_URL", "http://141.37.123.132:8086")
      difficulty = env.getOrElse("TOURNAMENT_BOT_DIFFICULTY", "medium")
    yield TournamentBotConfig(serverUrl, tournamentId, token, botId, difficulty)

@@ -39,10 +39,11 @@ class TournamentBotGamePlayer:
  // scalafix:on DisableSyntax.var

  val defaultServerUrl: String =
-    System.getenv().asScala.getOrElse("TOURNAMENT_SERVER_URL", "http://localhost:8089")
+    System.getenv().asScala.getOrElse("TOURNAMENT_SERVER_URL", "http://141.37.123.132:8086")

  @PostConstruct
  def initialize(): Unit =
+    parkOnStartup()
    config match
      case None =>
        log.info("Tournament bot disabled — set TOURNAMENT_ID and TOURNAMENT_BOT_TOKEN to enable")
@@ -50,11 +51,42 @@ class TournamentBotGamePlayer:
        log.infof("Tournament bot enabled — server=%s tournament=%s bot=%s", cfg.serverUrl, cfg.tournamentId, cfg.botId)
        startAsync(cfg)

-  def joinTournament(tournamentId: String, difficulty: String, serverUrl: String): Either[String, String] =
-    registerBot(serverUrl, difficulty) match
-      case None => Left("Failed to register bot with tournament server")
-      case Some((botId, token)) =>
-        val cfg = TournamentBotConfig(serverUrl, tournamentId, token, botId, difficulty)
+  private def parkOnStartup(): Unit =
+    park(defaultServerUrl, "expert") match
+      case Some(id) => log.infof("Parked expert bot on %s as id %s", defaultServerUrl, id)
+      case None     => log.warnf("Failed to park expert bot on %s", defaultServerUrl)
+
+  private def park(serverUrl: String, difficulty: String): Option[String] =
+    System.getenv().asScala.get("TOURNAMENT_BOT_TOKEN").filter(_.nonEmpty).flatMap { token =>
+      Try {
+        val body = s"""{"name":"${botName(difficulty)}"}"""
+        val response = client
+          .target(serverUrl)
+          .path("api")
+          .path("bots")
+          .request(MediaType.APPLICATION_JSON)
+          .header("Authorization", s"Bearer $token")
+          .post(Entity.entity(body, MediaType.APPLICATION_JSON))
+        if response.getStatus == 201 || response.getStatus == 200 then
+          val id = objectMapper.readTree(response.readEntity(classOf[String])).path("id").asText()
+          response.close()
+          Option(id).filter(_.nonEmpty)
+        else { log.warnf("Parking bot %s returned status %d", botName(difficulty), response.getStatus); response.close(); None }
+      }.getOrElse(None)
+    }
+
+  private def botName(difficulty: String): String = s"NowChess ${difficulty.capitalize}"
+
+  def joinTournament(
+      tournamentId: String,
+      botToken: String,
+      difficulty: String,
+      serverUrl: String,
+  ): Either[String, String] =
+    TournamentBotConfig.jwtSubject(botToken) match
+      case None => Left("Invalid bot token — could not extract subject")
+      case Some(botId) =>
+        val cfg = TournamentBotConfig(serverUrl, tournamentId, botToken, botId, difficulty)
        if join(cfg) then
          startAsync(cfg)
          Right(botId)
@@ -65,26 +97,6 @@ class TournamentBotGamePlayer:
    thread.setDaemon(true)
    thread.start()

-  private def registerBot(serverUrl: String, difficulty: String): Option[(String, String)] =
-    Try {
-      val name = s"NowChess ${difficulty.capitalize}"
-      val body = s"""{"name":"$name","isBot":true}"""
-      val response = client
-        .target(serverUrl)
-        .path("api")
-        .path("auth")
-        .path("register")
-        .request(MediaType.APPLICATION_JSON)
-        .post(Entity.entity(body, MediaType.APPLICATION_JSON))
-      if response.getStatus == 201 then
-        val node  = objectMapper.readTree(response.readEntity(classOf[String]))
-        val id    = node.path("id").asText()
-        val token = node.path("token").asText()
-        response.close()
-        if id.nonEmpty && token.nonEmpty then Some((id, token)) else None
-      else { log.warnf("Bot registration returned status %d", response.getStatus); response.close(); None }
-    }.getOrElse(None)
-
  @PreDestroy
  def cleanup(): Unit =
    running = false
@@ -1,3 +1,3 @@
 MAJOR=0
-MINOR=18
+MINOR=19
 PATCH=0
Author	SHA1	Message	Date
Janis	6351a19b67	feat(analytics): feed Lichess PGN dumps into Spark batch jobs Build & Test (NowChessSystems) TeamCity build failed Details Add GameSource: normalises game records into a shared schema and selects backend via NOWCHESS_PGN_PATH. Unset = PostgreSQL game_records (unchanged); set = a Lichess PGN dump (file or http(s) URL). - Parse Lichess PGN with Spark SQL string functions only (no UDFs). - URLs fetched once via SparkContext.addFile, distributed to executors. - .pgn.zst decompressed in-process via zstd-jni, plain .pgn redistributed. - All four batch jobs read through GameSource and skip JDBC write-back in PGN mode (Parquet/CSV output only). Enables driving the analytics demo straight from https://database.lichess.org standard dumps. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-17 10:40:29 +02:00
Janis	d7df76b769	feat(official-bots): park expert bot on tournament server at startup Build & Test (NowChessSystems) TeamCity build was removed from queue Details Park the expert bot on the configured tournament server (default http://141.37.123.132:8086) on startup, reusing a fixed TOURNAMENT_BOT_TOKEN when present instead of minting a new identity. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-17 10:34:48 +02:00
TeamCity	f44d3ee376	ci: bump version with Build-125	2026-06-17 07:24:13 +00:00
lq64	688d30e2b1	fix: enable official bots to connect to external tournament server (#71 ) Build & Test (NowChessSystems) TeamCity build finished Details Two bugs prevented official bots from joining the external tournament-server: 1. JWT claim mismatch — bot tokens lacked the `isBot: true` claim the tournament server requires. Added the claim to generateBotToken() in AccountService, which covers both user-owned bots and official bots. 2. Broken join flow — TournamentBotGamePlayer.joinTournament() called registerBot() which hit POST /api/auth/register on the tournament server, an endpoint that does not exist. Removed registerBot() and updated JoinTournamentRequest to accept a botToken field so the caller supplies the pre-existing NowChessSystems token directly. --------- Co-authored-by: LQ63 <lkhermann@web.de> Reviewed-on: #71 Co-authored-by: Leon Hermann <lq@blackhole.local> Co-committed-by: Leon Hermann <lq@blackhole.local>	2026-06-17 09:10:13 +02:00
Janis	98c64fc0d5	fix(official-bots): configure JWT verification (#72 ) The official-bots service enabled smallrye-jwt but never set mp.jwt.verify.publickey.location or issuer, so it could not validate any incoming token and rejected every authenticated request with 401. Add the verify public key (issuer nowchess) mirroring tournament/core, and ship keys/public.pem from the shared keypair. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Reviewed-on: #72	2026-06-17 09:10:01 +02:00