Chess960 round-robin tournament - Top 6 engines

[Ray]

I have just finished running a 3,000 game round-robin tournament of the top 6 chess960 engines. That is 1,000 games per engine.

Conditions:
AMD Opteron 1352 2.1GHz
Shredder GUI
Time control 1 Minutes/Game + 1 Seconds/Move
100 randomly selected opening positions with switched sides
Each engine one thread
Ponder ON
256MB hash
5-men tablebases

Results

Rybka 4 x64 763.0 / 1000 Stockfish 1.7 x64 658.5 / 1000 Naum 4.2 x64 532.5 / 1000 Shredder 12 489.0 / 1000 Hiarcs 13.1 295.0 / 1000 Sjeng WC2008 x64 262.0 / 1000

BayesELO Ratings calculation:

Rank Engine ELO + - Score AvOp Games 1 Rybka 4 64-bit 3296 +18 -17 76.3% -209.8 1000 2 Stockfish 1.7 64-bit 3226 +17 -17 65.8% -125.8 1000 3 Naum 4.2 64-bit 3144 +17 -17 53.2% -27.4 1000 4 Shredder 12 3116 +16 -16 48.9% +6.2 1000 5 Hiarcs 13.1 2987 +17 -18 29.5% +161.0 1000 6 Deep Sjeng WC2008 64-bit 1CPU 2958 +18 -18 26.2% +195.8 1000

This was just a bit of fun, but also I was interested to see if Rybka 4 performed as well at chess960 under these conditions ("game in x minutes" time control with increment, and ponder on) as it did in the CCRL 40/4 repeating tests with ponder off. It seems that it does. If we take the gap betwen Rybka 4 and Stockfish 1.7:

CCRL 40/4 FRC list - gap = +73 ELO
This round-robin - gap = +70 ELO

The gap is almost identical, so my conclusion would be yes, despite the very different test conditions, Rybka 4's chess960 performance is similar.

The pgn is attached in case anyone is interested in the games.

This was something different and a bit of fun, I'll now think about another chess960 tournament of some sort.

[Ray]

Here are the statistics for the 100 opening positions:

Opening games Draws White score NNBRKBRQ 30 16.7% 71.7% RNQKBNRB 30 20.0% 66.7% NBRNBQKR 30 43.3% 65.0% RQKBBNNR 30 36.7% 65.0% RNKQRNBB 30 30.0% 65.0% BRKBQNNR 30 6.7% 63.3% NQRNBKRB 30 26.7% 63.3% RBNNBKRQ 30 20.0% 63.3% RQBNKBNR 30 16.7% 61.7% NBRKBQRN 30 23.3% 61.7% BRNKRBNQ 30 16.7% 61.7% RBBKNQRN 30 33.3% 60.0% NRKRBBNQ 30 13.3% 60.0% NQBBRKRN 30 20.0% 60.0% RQKBBNRN 30 13.3% 60.0% NBRKNRBQ 30 13.3% 60.0% RKRQNBBN 30 20.0% 60.0% NBBQRKRN 30 13.3% 60.0% QRBBKNRN 30 20.0% 60.0% RKRBNQBN 30 30.0% 58.3% BBRKNNQR 30 16.7% 58.3% RKBRNBNQ 30 23.3% 58.3% RKBQRBNN 30 23.3% 58.3% RBKQBRNN 30 36.7% 58.3% BBRKNRNQ 30 20.0% 56.7% RNQKBRNB 30 13.3% 56.7% BRKNNQRB 30 26.7% 56.7% RKNRNBBQ 30 20.0% 56.7% RKBNNBRQ 30 13.3% 56.7% NRBQKBNR 30 20.0% 56.7% NRNQBBKR 30 20.0% 56.7% NQRNKBBR 30 26.7% 56.7% BRKNRBQN 30 10.0% 55.0% NQRNKRBB 30 10.0% 55.0% NBNRQKBR 30 16.7% 55.0% RQBKRNNB 30 16.7% 55.0% QBNRKNBR 30 30.0% 55.0% BNQRKBRN 30 23.3% 55.0% BRNBQKRN 30 30.0% 55.0% BRKNNBQR 30 26.7% 53.3% QBRKNRBN 30 13.3% 53.3% NRNQKRBB 30 26.7% 53.3% BBRNQKNR 30 40.0% 53.3% RBNQBKNR 30 33.3% 53.3% NBBRNKRQ 30 13.3% 53.3% BQRKRNNB 30 26.7% 53.3% BNRKQBNR 30 20.0% 53.3% RNNBQKBR 30 20.0% 53.3% BNQRNKRB 30 10.0% 51.7% RNKBNRBQ 30 23.3% 51.7% NNQRKBBR 30 23.3% 51.7% RBKQBNRN 30 23.3% 51.7% RKNBBQRN 30 23.3% 51.7% BRKNRQNB 30 23.3% 51.7% NBRQBKNR 30 16.7% 51.7% NBBRKQRN 30 23.3% 51.7% RNBKRQNB 30 16.7% 38.3% BRKNQRNB 30 36.7% 38.3%

[Ray]

Another view of the results: