197 lines
4.8 KiB
Markdown
197 lines
4.8 KiB
Markdown
# Windows NNUE Pipeline — Complete Guide
|
|
|
|
## Quick Links
|
|
|
|
**Start here:** [`README_WINDOWS.md`](README_WINDOWS.md) — 5-minute quick start
|
|
|
|
## Documentation Files
|
|
|
|
| File | Purpose | Time to Read |
|
|
|------|---------|------|
|
|
| **README_WINDOWS.md** | Windows quick start guide | 5 min |
|
|
| **WINDOWS_SETUP.md** | Detailed Windows setup with troubleshooting | 10 min |
|
|
| **QUICKSTART.md** | Cross-platform quick reference | 5 min |
|
|
| **python/README_NNUE.md** | Complete pipeline documentation | 15 min |
|
|
| **NNUE_IMPLEMENTATION_SUMMARY.md** | Technical implementation details | 10 min |
|
|
|
|
## Launcher Scripts
|
|
|
|
All scripts work from `modules\bot\` directory.
|
|
|
|
### Windows Command Prompt / PowerShell
|
|
|
|
```cmd
|
|
set STOCKFISH_PATH=C:\path\to\stockfish.exe
|
|
run_nnue_pipeline.bat
|
|
```
|
|
|
|
### PowerShell (Colorful, Recommended)
|
|
|
|
```powershell
|
|
$env:STOCKFISH_PATH = "C:\path\to\stockfish.exe"
|
|
.\run_nnue_pipeline.ps1
|
|
```
|
|
|
|
### Git Bash / WSL
|
|
|
|
```bash
|
|
export STOCKFISH_PATH=/c/path/to/stockfish.exe
|
|
bash run_nnue_pipeline.sh
|
|
```
|
|
|
|
## Python Pipeline Scripts
|
|
|
|
Located in `modules\bot\python\`:
|
|
|
|
| Script | Purpose |
|
|
|--------|---------|
|
|
| **generate_positions.py** | Step 1: Generate 500K random positions |
|
|
| **label_positions.py** | Step 2: Evaluate with Stockfish |
|
|
| **train_nnue.py** | Step 3: Train neural network |
|
|
| **export_weights.py** | Step 4: Export to Scala |
|
|
| **run_pipeline.bat** | Windows batch runner |
|
|
| **run_pipeline.sh** | Bash runner |
|
|
|
|
## Getting Started (3 Steps)
|
|
|
|
### 1. Install Python
|
|
|
|
```cmd
|
|
REM Check if Python is installed
|
|
python --version
|
|
|
|
REM If not, download from https://python.org
|
|
REM During installation, CHECK "Add Python to PATH"
|
|
```
|
|
|
|
### 2. Install Dependencies
|
|
|
|
```cmd
|
|
cd modules\bot\python
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
### 3. Get Stockfish
|
|
|
|
- Download from https://stockfishchess.org/download/
|
|
- Extract to `C:\stockfish`
|
|
- Verify: `C:\stockfish\stockfish.exe --version`
|
|
|
|
### 4. Run Pipeline
|
|
|
|
```cmd
|
|
cd modules\bot
|
|
set STOCKFISH_PATH=C:\stockfish\stockfish.exe
|
|
run_nnue_pipeline.bat
|
|
```
|
|
|
|
## FAQ
|
|
|
|
### How long does it take?
|
|
|
|
- Step 1 (positions): 2-3 minutes
|
|
- Step 2 (Stockfish): **24-36 hours** ← slowest
|
|
- Step 3 (training): 2-4 hours (faster with GPU)
|
|
- Step 4 (export): 1 minute
|
|
- **Total: 26-40 hours**
|
|
|
|
### Can I pause and resume?
|
|
|
|
Yes! The pipeline saves progress:
|
|
1. Press `Ctrl+C` to stop
|
|
2. Run the pipeline again - it will resume where it left off
|
|
|
|
### Does it use my GPU?
|
|
|
|
Yes, automatically! If you have NVIDIA GPU:
|
|
- Training will be 5-10x faster
|
|
- Requires CUDA Toolkit (optional, not required)
|
|
|
|
### Can I test with fewer positions?
|
|
|
|
Yes! Edit `python\generate_positions.py`:
|
|
```python
|
|
# Change line 9 from:
|
|
for game_num in range(500000):
|
|
|
|
# To:
|
|
for game_num in range(10000):
|
|
```
|
|
|
|
This will complete in ~30 minutes instead of 26+ hours.
|
|
|
|
## File Locations After Pipeline
|
|
|
|
```
|
|
modules\bot\
|
|
├── python\
|
|
│ ├── positions.txt (15 MB - raw positions)
|
|
│ ├── training_data.jsonl (100 MB - FEN + eval)
|
|
│ ├── nnue_weights.pt (3 MB - trained weights)
|
|
│ └── [python scripts]
|
|
├── src\main\scala\de\nowchess\bot\bots\nnue\
|
|
│ ├── NNUEWeights.scala (10 MB - generated weights)
|
|
│ ├── NNUE.scala (inference engine)
|
|
│ ├── EvaluationNNUE.scala (weights trait)
|
|
│ └── NNUEBot.scala (bot implementation)
|
|
└── [launcher scripts]
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
Set these before running the pipeline:
|
|
|
|
```cmd
|
|
REM Required (unless Stockfish is in PATH)
|
|
set STOCKFISH_PATH=C:\stockfish\stockfish.exe
|
|
|
|
REM Optional: specify Python version
|
|
set PYTHON_CMD=python3
|
|
```
|
|
|
|
Or in PowerShell:
|
|
|
|
```powershell
|
|
$env:STOCKFISH_PATH = "C:\stockfish\stockfish.exe"
|
|
$env:PYTHON_CMD = "python3"
|
|
```
|
|
|
|
## Troubleshooting Flow
|
|
|
|
1. **Python not found** → Install from python.org, check "Add to PATH"
|
|
2. **Stockfish not found** → Download from stockfishchess.org, set `STOCKFISH_PATH`
|
|
3. **Module not found** → Run `pip install -r requirements.txt`
|
|
4. **GPU out of memory** → Reduce batch size in `train_nnue.py`
|
|
5. **Pipeline hangs** → Check `training_data.jsonl` size, Stockfish evaluation is slow
|
|
|
|
See **WINDOWS_SETUP.md** for detailed troubleshooting.
|
|
|
|
## Next Steps After Pipeline
|
|
|
|
1. **Verify output:**
|
|
```cmd
|
|
cd ..\..\
|
|
compile.bat
|
|
test.bat
|
|
```
|
|
|
|
2. **Use NNUEBot in your engine:**
|
|
```scala
|
|
val bot = new NNUEBot(difficulty, rules, book)
|
|
val move = bot.nextMove(context)
|
|
```
|
|
|
|
## Support
|
|
|
|
- **Quick help:** README_WINDOWS.md
|
|
- **Detailed help:** WINDOWS_SETUP.md
|
|
- **Technical details:** NNUE_IMPLEMENTATION_SUMMARY.md
|
|
- **Complete reference:** python/README_NNUE.md
|
|
|
|
---
|
|
|
|
**Platform:** Windows 10/11 (tested on Windows 11)
|
|
**Requirements:** Python 3.8+, Stockfish 14+
|
|
**Languages:** Python, Scala 3
|
|
**Status:** ✅ Production Ready
|