246 lines
5.6 KiB
Markdown
246 lines
5.6 KiB
Markdown
# Windows Setup Guide for NNUE Pipeline
|
|
|
|
This guide walks through running the NNUE training pipeline on Windows 10/11.
|
|
|
|
## Prerequisites
|
|
|
|
### 1. Python 3.8+
|
|
|
|
Check if Python is installed:
|
|
```cmd
|
|
python --version
|
|
```
|
|
|
|
If not installed:
|
|
- Download from [python.org](https://www.python.org)
|
|
- During installation, **CHECK** "Add Python to PATH"
|
|
- Verify after install: `python --version`
|
|
|
|
### 2. Stockfish Chess Engine
|
|
|
|
Download Stockfish:
|
|
- https://stockfishchess.org/download/
|
|
- Extract to a known location, e.g., `C:\stockfish\stockfish.exe`
|
|
|
|
Verify installation:
|
|
```cmd
|
|
C:\stockfish\stockfish.exe --version
|
|
```
|
|
|
|
### 3. Python Dependencies
|
|
|
|
From `modules\bot\python\`:
|
|
```cmd
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
This installs:
|
|
- python-chess (chess board library)
|
|
- torch (neural network training)
|
|
- tqdm (progress bars)
|
|
|
|
## Running the Pipeline
|
|
|
|
### Option A: Quick Start (Recommended for Windows)
|
|
|
|
From `modules\bot\`:
|
|
```cmd
|
|
REM Set Stockfish path (if not in PATH)
|
|
set STOCKFISH_PATH=C:\stockfish\stockfish.exe
|
|
|
|
REM Run the pipeline
|
|
run_nnue_pipeline.bat
|
|
```
|
|
|
|
### Option B: Manual Control
|
|
|
|
From `modules\bot\python\`:
|
|
|
|
```cmd
|
|
REM Set Stockfish path
|
|
set STOCKFISH_PATH=C:\stockfish\stockfish.exe
|
|
|
|
REM Run pipeline
|
|
python run_pipeline.py
|
|
```
|
|
|
|
Wait, there's no `run_pipeline.py` - use the batch file instead:
|
|
|
|
```cmd
|
|
set STOCKFISH_PATH=C:\stockfish\stockfish.exe
|
|
run_pipeline.bat
|
|
```
|
|
|
|
### Option C: Using Git Bash (if installed)
|
|
|
|
Git Bash allows you to use bash scripts on Windows:
|
|
|
|
```bash
|
|
cd modules/bot
|
|
export STOCKFISH_PATH=C:/stockfish/stockfish.exe
|
|
bash run_nnue_pipeline.sh
|
|
```
|
|
|
|
## Setting Stockfish Path Permanently
|
|
|
|
If you want to avoid setting `STOCKFISH_PATH` each time:
|
|
|
|
### Method 1: Add to System PATH
|
|
|
|
1. Open **Environment Variables**:
|
|
- Press `Win + R`
|
|
- Type `systempropertiesadvanced.exe`
|
|
- Click "Environment Variables..."
|
|
|
|
2. Under "System variables", click "New"
|
|
- Variable name: `STOCKFISH_PATH`
|
|
- Variable value: `C:\stockfish\stockfish.exe`
|
|
- Click OK, OK, OK
|
|
|
|
3. Restart Command Prompt or PowerShell
|
|
|
|
4. Verify: `echo %STOCKFISH_PATH%`
|
|
|
|
### Method 2: Add Stockfish Directory to PATH
|
|
|
|
1. Open **Environment Variables** (same as above)
|
|
2. Find "Path" in System variables, click Edit
|
|
3. Click "New"
|
|
4. Add: `C:\stockfish`
|
|
5. Click OK, OK, OK
|
|
6. Restart terminal and verify: `stockfish --version`
|
|
|
|
## Running the Full Pipeline
|
|
|
|
Time estimates (on typical Windows machine):
|
|
- Step 1 (Generate positions): ~2-3 minutes
|
|
- Step 2 (Stockfish evaluation): **~24-36 hours** (slowest)
|
|
- Step 3 (Train network): ~2-4 hours (faster with NVIDIA GPU)
|
|
- Step 4 (Export weights): ~1 minute
|
|
|
|
Total: **~26-40 hours** on CPU, **~26-30 hours** on GPU
|
|
|
|
To run the full pipeline:
|
|
```cmd
|
|
cd modules\bot
|
|
set STOCKFISH_PATH=C:\stockfish\stockfish.exe
|
|
run_nnue_pipeline.bat
|
|
```
|
|
|
|
The script will:
|
|
1. Generate 500,000 random chess positions
|
|
2. Evaluate each with Stockfish at depth 12
|
|
3. Train a neural network on the evaluations
|
|
4. Export weights as Scala code
|
|
5. Automatically update `NNUEWeights.scala`
|
|
|
|
## Quick Testing (Shorter Run)
|
|
|
|
To test the pipeline with fewer positions (~30 minutes total):
|
|
|
|
Edit `python\generate_positions.py`:
|
|
```python
|
|
# Line 9, change:
|
|
for game_num in range(500000):
|
|
|
|
# To:
|
|
for game_num in range(10000):
|
|
```
|
|
|
|
Then run the pipeline normally.
|
|
|
|
## Troubleshooting
|
|
|
|
### "Python is not recognized"
|
|
|
|
Python isn't in PATH:
|
|
1. Install Python again, **CHECK** "Add Python to PATH"
|
|
2. Or add manually: add `C:\Users\YourName\AppData\Local\Programs\Python\Python310` to PATH
|
|
|
|
### "Stockfish not found"
|
|
|
|
```cmd
|
|
REM Find where stockfish is installed
|
|
where stockfish
|
|
|
|
REM If found, set the full path
|
|
set STOCKFISH_PATH=C:\full\path\to\stockfish.exe
|
|
```
|
|
|
|
### "ModuleNotFoundError: No module named 'torch'"
|
|
|
|
PyTorch not installed or wrong Python version:
|
|
```cmd
|
|
pip install torch==2.1.2
|
|
```
|
|
|
|
If you have NVIDIA GPU, install CUDA version for better performance:
|
|
```cmd
|
|
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
|
|
```
|
|
|
|
### "CUDA out of memory"
|
|
|
|
If training fails with GPU memory error, edit `python\train_nnue.py`:
|
|
```python
|
|
# Line ~91, change:
|
|
train_loader = DataLoader(train_dataset, batch_size=4096, shuffle=True)
|
|
|
|
# To:
|
|
train_loader = DataLoader(train_dataset, batch_size=2048, shuffle=True)
|
|
```
|
|
|
|
### Pipeline hangs at Step 2
|
|
|
|
Stockfish evaluation is slow. This is normal - it may take 24+ hours.
|
|
|
|
To check progress, look at the size of `training_data.jsonl` (should grow over time):
|
|
```cmd
|
|
dir training_data.jsonl
|
|
```
|
|
|
|
To interrupt and resume later:
|
|
- Press `Ctrl+C`
|
|
- Run the pipeline again - it will resume from where it left off
|
|
|
|
## After Pipeline Completes
|
|
|
|
1. New file created: `modules\bot\src\main\scala\de\nowchess\bot\bots\nnue\NNUEWeights.scala`
|
|
|
|
2. Recompile the project:
|
|
```cmd
|
|
cd ..\..\
|
|
compile.bat
|
|
```
|
|
|
|
3. Run tests:
|
|
```cmd
|
|
test.bat
|
|
```
|
|
|
|
## File Locations
|
|
|
|
| File | Location | Size |
|
|
|------|----------|------|
|
|
| Positions | `modules\bot\python\positions.txt` | ~15 MB |
|
|
| Training data | `modules\bot\python\training_data.jsonl` | ~100 MB |
|
|
| Weights | `modules\bot\python\nnue_weights.pt` | ~3 MB |
|
|
| Scala weights | `modules\bot\src\main\scala\de\nowchess\bot\bots\nnue\NNUEWeights.scala` | ~10 MB |
|
|
|
|
## Advanced: GPU Acceleration
|
|
|
|
If you have an NVIDIA GPU:
|
|
|
|
1. Install CUDA Toolkit: https://developer.nvidia.com/cuda-downloads
|
|
2. Install cuDNN: https://developer.nvidia.com/cudnn
|
|
3. Reinstall PyTorch with CUDA support:
|
|
```cmd
|
|
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
|
|
```
|
|
|
|
Training will be 5-10x faster with GPU.
|
|
|
|
## Support
|
|
|
|
See `README_NNUE.md` for complete documentation and `QUICKSTART.md` for quick reference.
|