Validator Setup Guide

Prerequisites

Required Software

• Python 3.11+
• Git
• Docker — grading runs the task's own container. No registry login is needed: every image the validator pulls is public at the time it needs it.
• Bittensor wallet with registered hotkey on the subnet

Required Accounts

• Bittensor Wallet: Registered and staked on netuid 114

System Requirements

Installation

Install Python Dependencies

# Create virtual environment
python3.11 -m venv .venv
source .venv/bin/activate

# Install subnet dependencies
pip install --upgrade pip
pip install -e .

# Install validator-specific dependencies
pip install -r reqs_tmp.txt

cd validator

Configuration

Copy the example configuration file and edit with your values:

cd validator
cp .env.example .env
nano .env  # or use your preferred editor

What the validator grades

Every run is scored the same way — did the agent's patch make the task's tests pass — but a competition draws its tasks from two places, and each has its own grading path. Both are built in; there is nothing to choose or configure.

The platform names the dataset on each validation task, and the validator routes on it.

SOMA tasks. These are not a Hugging Face dataset. Each task ships a test image holding the repository at base_commit with the task's test patch already applied, its dependencies installed, and a run_tests entrypoint. Grading is the container equivalent of the harness: pull the image, start it with no network, copy the miner's patch in, apply it, run run_tests, and check the task's FAIL_TO_PASS / PASS_TO_PASS ids against the pytest JSON report it produces. Where to run and what to run come from the image's own soma.* labels.

No registry credentials needed. Those images live in a private Docker Hub repository so a competition's hidden tasks are not published in advance. Rather than distributing a shared registry token to every validator, the platform makes the repository public for the evaluation window and private again once it closes.

That repository holds exactly the current competition's tasks: the platform mirrors the images of the tasks the competition actually references into it and removes the rest, so a tag you can pull there is a task that is being scored right now. Tags from past competitions disappear once their window has closed.

Running the Validator

Using PM2 (Recommended)

# Install PM2 (if not already installed)
npm install -g pm2

# 🚀 Start validator with auto-update watcher
cd /path/to/MCP-subnet/validator
pm2 start run_validator.sh --name mcp-validator-watch --interpreter bash -- 60

# 🔄 Auto-restart on system reboot
pm2 startup
pm2 save

# 📊 View status
pm2 status

# 🛑 Stop watcher (and validator will no longer be restarted)
pm2 stop mcp-validator-watch

# 🔁 Restart watcher
pm2 restart mcp-validator-watch

Monitoring

Check Validator Logs

# 📖 View live logs
pm2 logs mcp-validator

# 📄 View specific log file
tail -f ~/.pm2/logs/mcp-validator-out.log
tail -f ~/.pm2/logs/mcp-validator-error.log

# 🔍 Search logs for errors
grep -i error ~/.pm2/logs/mcp-validator-error.log

Key Metrics to Monitor

Healthy Indicators:

• Successful task fetches from platform
• Regular score submissions to platform
• Consistent weight setting

Warning Signs:

• LLM API errors or timeouts
• Platform connection failures
• Memory/CPU exhaustion

Troubleshooting

Common Issues

1. No Tasks Available (503 Error)

Problem: Platform returns 503 "No tasks available"

Expected behavior:

INFO: No tasks available (attempt 1), backing off to 30.0s poll interval

What to do:

• Wait for new miner submissions
• Let the validator auto-retry with backoff
• Check platform status

2. pull access denied for a task test image

Problem: The hidden-task repository is still private.

What to do:

• Ignore an isolated occurrence in the first minutes of the evaluation window
• If it persists, report it — the platform-side visibility flip may be failing, and no validator will be able to grade that competition's tasks until it is fixed

A manifest unknown for a single task is a different problem: the repository is reachable but that task's image was never copied into it. Report the instance id — the platform holds such tasks back from dispatch, so a run existing for one means something copied only partially.

3. Platform Connection Failed

Problem: Can't connect to platform API

Checklist:

• Platform URL is correct in .env
• Platform signer SS58 is correct
• Network connectivity to platform
• Platform is online and accepting requests

Success!

Your validator is now running! 🚀

Next steps:

• Monitor your validator's performance
• Join the community Discord
• Read the full documentation
• Keep your validator updated