Round-Robin Tournament using SPCC UHO_2022 openings

[Ray]

I decided to try a one-off experiment, a tourmament using the mildly unbalanced 6-move UHO_2022_6mvs_+110_+119.pgn compiled by Stefan Pohl.

Testing conditions:
Format: Round robin tournament with the top 14 engines from CCRL single CPU lists
GUI: Cutechess with 500ms overstep margin, concurrency 12
Hardware: Ryzen 9 5900X,
Time Control: 90”+0.75” (which is CCRL 2'+1" on this hardware)
Book: 6-move UHO_2022_6mvs_+110_+119.pgn
60 games per pairing (30 random openings played reversed sides)
Each engine 512MB hash, 1 thread, ponder off
Adjudication: Syzygy 5-men only, otherwise the games played out until their natural conclusion
(checkmate, stalemate, insufficient material, 3-fold repetition or 50-moves rule,)

Participants:
Stockfish 20230531 (new bigger net as at date of this tournament)
Komodo Dragon 3.2
Berserk 11.1
Ethereal 14.00
Koivisto 9
RubiChess 20230410
Revenge 3.0
Rebel 16.2
Seer 2.6.0
Igel 3.4.0
Clover 4.1
SlowChess 2.9
Rofchade 3.0
Uralochka3.39d

Results in another 24 hours.

[Ray]

BayesElo ratings, default parameters:

Rank Name Elo + - games score draws 1 Stockfish 20230531 157 20 19 780 76% 47% 2 Komodo Dragon 3.2 120 19 19 780 70% 50% 3 Berserk 11.1 70 18 18 780 62% 53% 4 Ethereal 14.00 35 18 18 780 56% 54% 5 RubiChess 20230410 17 18 18 780 53% 54% 6 Koivisto 9 17 18 18 780 53% 52% 7 Revenge 3.0 -10 18 18 780 48% 54% 8 Rebel 16.2 -25 18 18 780 46% 52% 9 Seer 2.6.0 -35 18 18 780 44% 52% 10 Igel 3.4.0 -42 18 18 780 43% 54% 11 Clover 4.1 -50 18 18 780 41% 51% 12 SlowChess 2.9 -73 18 19 780 38% 48% 13 Uralochka3.39d -89 18 19 780 35% 51% 14 Rofchade 3.0 -92 19 19 780 35% 45%

[Ray]

Head to head statistics: 1) Stockfish 20230531 780 (+409,=367,-4), 76.0 % vs. : games ( +, =, -), (%) : Komodo Dragon 3.2 : 60 ( 25, 31, 4), 67.5 : Berserk 11.1 : 60 ( 27, 33, 0), 72.5 : Ethereal 14.00 : 60 ( 31, 29, 0), 75.8 : RubiChess 20230410 : 60 ( 32, 28, 0), 76.7 : Koivisto 9 : 60 ( 32, 28, 0), 76.7 : Revenge 3.0 : 60 ( 31, 29, 0), 75.8 : Rebel 16.2 : 60 ( 32, 28, 0), 76.7 : Seer 2.6.0 : 60 ( 34, 26, 0), 78.3 : Igel 3.4.0 : 60 ( 32, 28, 0), 76.7 : Clover 4.1 : 60 ( 33, 27, 0), 77.5 : SlowChess 2.9 : 60 ( 35, 25, 0), 79.2 : Uralochka3.39d : 60 ( 32, 28, 0), 76.7 : Rofchade 3.0 : 60 ( 33, 27, 0), 77.5 : 2) Komodo Dragon 3.2 780 (+355,=387,-38), 70.3 % vs. : games ( +, =, -), (%) : Stockfish 20230531 : 60 ( 4, 31, 25), 32.5 : Berserk 11.1 : 60 ( 21, 37, 2), 65.8 : Ethereal 14.00 : 60 ( 28, 31, 1), 72.5 : RubiChess 20230410 : 60 ( 26, 30, 4), 68.3 : Koivisto 9 : 60 ( 26, 32, 2), 70.0 : Revenge 3.0 : 60 ( 25, 33, 2), 69.2 : Rebel 16.2 : 60 ( 28, 31, 1), 72.5 : Seer 2.6.0 : 60 ( 29, 31, 0), 74.2 : Igel 3.4.0 : 60 ( 34, 26, 0), 78.3 : Clover 4.1 : 60 ( 31, 28, 1), 75.0 : SlowChess 2.9 : 60 ( 36, 24, 0), 80.0 : Uralochka3.39d : 60 ( 34, 26, 0), 78.3 : Rofchade 3.0 : 60 ( 33, 27, 0), 77.5 : 3) Berserk 11.1 780 (+280,=411,-89), 62.2 % vs. : games ( +, =, -), (%) : Stockfish 20230531 : 60 ( 0, 33, 27), 27.5 : Komodo Dragon 3.2 : 60 ( 2, 37, 21), 34.2 : Ethereal 14.00 : 60 ( 17, 37, 6), 59.2 : RubiChess 20230410 : 60 ( 20, 30, 10), 58.3 : Koivisto 9 : 60 ( 22, 30, 08), 61.7 : Revenge 3.0 : 60 ( 22, 34, 4), 65.0 : Rebel 16.2 : 60 ( 31, 23, 6), 70.8 : Seer 2.6.0 : 60 ( 25, 33, 2), 69.2 : Igel 3.4.0 : 60 ( 23, 36, 1), 68.3 : Clover 4.1 : 60 ( 29, 30, 1), 73.3 : SlowChess 2.9 : 60 ( 27, 32, 1), 71.7 : Uralochka3.39d : 60 ( 28, 30, 2), 71.7 : Rofchade 3.0 : 60 ( 34, 26, 0), 78.3 : 4) Ethereal 14.00 780 (+228,=420,-132), 56.2 % vs. : games ( +, =, -), (%) : Stockfish 20230531 : 60 ( 0, 29, 31), 24.2 : Komodo Dragon 3.2 : 60 ( 1, 31, 28), 27.5 : Berserk 11.1 : 60 ( 6, 37, 17), 40.8 : RubiChess 20230410 : 60 ( 15, 30, 15), 50.0 : Koivisto 9 : 60 ( 14, 43, 3), 59.2 : Revenge 3.0 : 60 ( 17, 34, 9), 56.7 : Rebel 16.2 : 60 ( 21, 32, 7), 61.7 : Seer 2.6.0 : 60 ( 23, 29, 08), 62.5 : Igel 3.4.0 : 60 ( 26, 30, 4), 68.3 : Clover 4.1 : 60 ( 23, 34, 3), 66.7 : SlowChess 2.9 : 60 ( 24, 33, 3), 67.5 : Uralochka3.39d : 60 ( 27, 31, 2), 70.8 : Rofchade 3.0 : 60 ( 31, 27, 2), 74.2 : 5) RubiChess 20230410 780 (+205,=418,-157), 53.1 % vs. : games ( +, =, -), (%) : Stockfish 20230531 : 60 ( 0, 28, 32), 23.3 : Komodo Dragon 3.2 : 60 ( 4, 30, 26), 31.7 : Berserk 11.1 : 60 ( 10, 30, 20), 41.7 : Ethereal 14.00 : 60 ( 15, 30, 15), 50.0 : Koivisto 9 : 60 ( 14, 34, 12), 51.7 : Revenge 3.0 : 60 ( 17, 32, 11), 55.0 : Rebel 16.2 : 60 ( 20, 32, 08), 60.0 : Seer 2.6.0 : 60 ( 16, 39, 5), 59.2 : Igel 3.4.0 : 60 ( 17, 39, 4), 60.8 : Clover 4.1 : 60 ( 18, 34, 08), 58.3 : SlowChess 2.9 : 60 ( 21, 32, 7), 61.7 : Uralochka3.39d : 60 ( 23, 34, 3), 66.7 : Rofchade 3.0 : 60 ( 30, 24, 6), 70.0 : 6) Koivisto 9 780 (+209,=409,-162), 53.0 % vs. : games ( +, =, -), (%) : Stockfish 20230531 : 60 ( 0, 28, 32), 23.3 : Komodo Dragon 3.2 : 60 ( 2, 32, 26), 30.0 : Berserk 11.1 : 60 ( 8, 30, 22), 38.3 : Ethereal 14.00 : 60 ( 3, 43, 14), 40.8 : RubiChess 20230410 : 60 ( 12, 34, 14), 48.3 : Revenge 3.0 : 60 ( 16, 35, 9), 55.8 : Rebel 16.2 : 60 ( 22, 27, 11), 59.2 : Seer 2.6.0 : 60 ( 23, 34, 3), 66.7 : Igel 3.4.0 : 60 ( 18, 32, 10), 56.7 : Clover 4.1 : 60 ( 28, 26, 6), 68.3 : SlowChess 2.9 : 60 ( 24, 29, 7), 64.2 : Uralochka3.39d : 60 ( 27, 28, 5), 68.3 : Rofchade 3.0 : 60 ( 26, 31, 3), 69.2 : 7) Revenge 3.0 780 (+164,=425,-191), 48.3 % vs. : games ( +, =, -), (%) : Stockfish 20230531 : 60 ( 0, 29, 31), 24.2 : Komodo Dragon 3.2 : 60 ( 2, 33, 25), 30.8 : Berserk 11.1 : 60 ( 4, 34, 22), 35.0 : Ethereal 14.00 : 60 ( 9, 34, 17), 43.3 : RubiChess 20230410 : 60 ( 11, 32, 17), 45.0 : Koivisto 9 : 60 ( 9, 35, 16), 44.2 : Rebel 16.2 : 60 ( 9, 40, 11), 48.3 : Seer 2.6.0 : 60 ( 16, 30, 14), 51.7 : Igel 3.4.0 : 60 ( 17, 37, 6), 59.2 : Clover 4.1 : 60 ( 19, 35, 6), 60.8 : SlowChess 2.9 : 60 ( 19, 30, 11), 56.7 : Uralochka3.39d : 60 ( 23, 32, 5), 65.0 : Rofchade 3.0 : 60 ( 26, 24, 10), 63.3 : 8.) Rebel 16.2 780 (+153,=409,-218), 45.8 % vs. : games ( +, =, -), (%) : Stockfish 20230531 : 60 ( 0, 28, 32), 23.3 : Komodo Dragon 3.2 : 60 ( 1, 31, 28), 27.5 : Berserk 11.1 : 60 ( 6, 23, 31), 29.2 : Ethereal 14.00 : 60 ( 7, 32, 21), 38.3 : RubiChess 20230410 : 60 ( 8, 32, 20), 40.0 : Koivisto 9 : 60 ( 11, 27, 22), 40.8 : Revenge 3.0 : 60 ( 11, 40, 9), 51.7 : Seer 2.6.0 : 60 ( 15, 33, 12), 52.5 : Igel 3.4.0 : 60 ( 11, 38, 11), 50.0 : Clover 4.1 : 60 ( 20, 30, 10), 58.3 : SlowChess 2.9 : 60 ( 22, 27, 11), 59.2 : Uralochka3.39d : 60 ( 21, 34, 5), 63.3 : Rofchade 3.0 : 60 ( 20, 34, 6), 61.7 : 9) Seer 2.6.0 780 (+141,=402,-237), 43.8 % vs. : games ( +, =, -), (%) : Stockfish 20230531 : 60 ( 0, 26, 34), 21.7 : Komodo Dragon 3.2 : 60 ( 0, 31, 29), 25.8 : Berserk 11.1 : 60 ( 2, 33, 25), 30.8 : Ethereal 14.00 : 60 ( 8, 29, 23), 37.5 : RubiChess 20230410 : 60 ( 5, 39, 16), 40.8 : Koivisto 9 : 60 ( 3, 34, 23), 33.3 : Revenge 3.0 : 60 ( 14, 30, 16), 48.3 : Rebel 16.2 : 60 ( 12, 33, 15), 47.5 : Igel 3.4.0 : 60 ( 12, 32, 16), 46.7 : Clover 4.1 : 60 ( 14, 34, 12), 51.7 : SlowChess 2.9 : 60 ( 22, 25, 13), 57.5 : Uralochka3.39d : 60 ( 23, 30, 7), 63.3 : Rofchade 3.0 : 60 ( 26, 26, 08), 65.0 : 10) Igel 3.4.0 780 (+121,=423,-236), 42.6 % vs. : games ( +, =, -), (%) : Stockfish 20230531 : 60 ( 0, 28, 32), 23.3 : Komodo Dragon 3.2 : 60 ( 0, 26, 34), 21.7 : Berserk 11.1 : 60 ( 1, 36, 23), 31.7 : Ethereal 14.00 : 60 ( 4, 30, 26), 31.7 : RubiChess 20230410 : 60 ( 4, 39, 17), 39.2 : Koivisto 9 : 60 ( 10, 32, 18), 43.3 : Revenge 3.0 : 60 ( 6, 37, 17), 40.8 : Rebel 16.2 : 60 ( 11, 38, 11), 50.0 : Seer 2.6.0 : 60 ( 16, 32, 12), 53.3 : Clover 4.1 : 60 ( 15, 33, 12), 52.5 : SlowChess 2.9 : 60 ( 18, 27, 15), 52.5 : Uralochka3.39d : 60 ( 20, 35, 5), 62.5 : Rofchade 3.0 : 60 ( 16, 30, 14), 51.7 : 11) Clover 4.1 780 (+123,=401,-256), 41.5 % vs. : games ( +, =, -), (%) : Stockfish 20230531 : 60 ( 0, 27, 33), 22.5 : Komodo Dragon 3.2 : 60 ( 1, 28, 31), 25.0 : Berserk 11.1 : 60 ( 1, 30, 29), 26.7 : Ethereal 14.00 : 60 ( 3, 34, 23), 33.3 : RubiChess 20230410 : 60 ( 8, 34, 18), 41.7 : Koivisto 9 : 60 ( 6, 26, 28), 31.7 : Revenge 3.0 : 60 ( 6, 35, 19), 39.2 : Rebel 16.2 : 60 ( 10, 30, 20), 41.7 : Seer 2.6.0 : 60 ( 12, 34, 14), 48.3 : Igel 3.4.0 : 60 ( 12, 33, 15), 47.5 : SlowChess 2.9 : 60 ( 19, 34, 7), 60.0 : Uralochka3.39d : 60 ( 22, 28, 10), 60.0 : Rofchade 3.0 : 60 ( 23, 28, 9), 61.7 : 12) SlowChess 2.9 780 (+107,=374,-299), 37.7 % vs. : games ( +, =, -), (%) : Stockfish 20230531 : 60 ( 0, 25, 35), 20.8 : Komodo Dragon 3.2 : 60 ( 0, 24, 36), 20.0 : Berserk 11.1 : 60 ( 1, 32, 27), 28.3 : Ethereal 14.00 : 60 ( 3, 33, 24), 32.5 : RubiChess 20230410 : 60 ( 7, 32, 21), 38.3 : Koivisto 9 : 60 ( 7, 29, 24), 35.8 : Revenge 3.0 : 60 ( 11, 30, 19), 43.3 : Rebel 16.2 : 60 ( 11, 27, 22), 40.8 : Seer 2.6.0 : 60 ( 13, 25, 22), 42.5 : Igel 3.4.0 : 60 ( 15, 27, 18), 47.5 : Clover 4.1 : 60 ( 7, 34, 19), 40.0 : Uralochka3.39d : 60 ( 11, 35, 14), 47.5 : Rofchade 3.0 : 60 ( 21, 21, 18), 52.5 : 13) Uralochka3.39d 780 (+74,=396,-310), 34.9 % vs. : games ( +, =, -), (%) : Stockfish 20230531 : 60 ( 0, 28, 32), 23.3 : Komodo Dragon 3.2 : 60 ( 0, 26, 34), 21.7 : Berserk 11.1 : 60 ( 2, 30, 28), 28.3 : Ethereal 14.00 : 60 ( 2, 31, 27), 29.2 : RubiChess 20230410 : 60 ( 3, 34, 23), 33.3 : Koivisto 9 : 60 ( 5, 28, 27), 31.7 : Revenge 3.0 : 60 ( 5, 32, 23), 35.0 : Rebel 16.2 : 60 ( 5, 34, 21), 36.7 : Seer 2.6.0 : 60 ( 7, 30, 23), 36.7 : Igel 3.4.0 : 60 ( 5, 35, 20), 37.5 : Clover 4.1 : 60 ( 10, 28, 22), 40.0 : SlowChess 2.9 : 60 ( 14, 35, 11), 52.5 : Rofchade 3.0 : 60 ( 16, 25, 19), 47.5 : 14) Rofchade 3.0 780 (+95,=350,-335), 34.6 % vs. : games ( +, =, -), (%) : Stockfish 20230531 : 60 ( 0, 27, 33), 22.5 : Komodo Dragon 3.2 : 60 ( 0, 27, 33), 22.5 : Berserk 11.1 : 60 ( 0, 26, 34), 21.7 : Ethereal 14.00 : 60 ( 2, 27, 31), 25.8 : RubiChess 20230410 : 60 ( 6, 24, 30), 30.0 : Koivisto 9 : 60 ( 3, 31, 26), 30.8 : Revenge 3.0 : 60 ( 10, 24, 26), 36.7 : Rebel 16.2 : 60 ( 6, 34, 20), 38.3 : Seer 2.6.0 : 60 ( 8, 26, 26), 35.0 : Igel 3.4.0 : 60 ( 14, 30, 16), 48.3 : Clover 4.1 : 60 ( 9, 28, 23), 38.3 : SlowChess 2.9 : 60 ( 18, 21, 21), 47.5 : Uralochka3.39d : 60 ( 19, 25, 16), 52.5 :

[Ray]

If you prefer Ordo ratings:

1 Stockfish 20230531 : 2494.2 592.5 780 76 2 Komodo Dragon 3.2 : 2446.5 548.5 780 70 3 Berserk 11.1 : 2384.8 485.5 780 62 4 Ethereal 14.00 : 2341.4 438.0 780 56 5 RubiChess 20230410 : 2319.9 414.0 780 53 6 Koivisto 9 : 2319.5 413.5 780 53 7 Revenge 3.0 : 2286.7 376.5 780 48 8 Rebel 16.2 : 2269.9 357.5 780 46 9 Seer 2.6.0 : 2256.0 342.0 780 44 10 Igel 3.4.0 : 2247.5 332.5 780 43 11 Clover 4.1 : 2239.4 323.5 780 41 12 SlowChess 2.9 : 2212.4 294.0 780 38 13 Uralochka3.39d : 2191.8 272.0 780 35 14 Rofchade 3.0 : 2189.9 270.0 780 35

[Graham Banks]

No real surprises.

[bastiball]

BayesElo ratings, default parameters:

Rank Name Elo + - games score oppo. draws 1 Stockfish 20230531 157 20 19 780 76% -12 47% 2 Komodo Dragon 3.2 120 19 19 780 70% -9 50% 3 Berserk 11.1 70 18 18 780 62% -5 53% 4 Ethereal 14.00 35 18 18 780 56% -3 54% 5 RubiChess 20230410 17 18 18 780 53% -1 54% 6 Koivisto 9 17 18 18 780 53% -1 52% 7 Revenge 3.0 -10 18 18 780 48% 1 54% 8 Rebel 16.2 -25 18 18 780 46% 2 52% 9 Seer 2.6.0 -35 18 18 780 44% 3 52% 10 Igel 3.4.0 -42 18 18 780 43% 3 54% 11 Clover 4.1 -50 18 18 780 41% 4 51% 12 SlowChess 2.9 -73 18 19 780 38% 6 48% 13 Uralochka3.39d -89 18 19 780 35% 7 51% 14 Rofchade 3.0 -92 19 19 780 35% 7 45%

Interesting

[Ray]

Very low draw rate as expected with unbalanced openings.

Rankings of engines are similar to the standard CCRL blitz list. Any differences could just be statistical margins of error.

Games can be downloaded here for anyone interested:

http://ccrl.chessdom.com/public/UHO2022 ... ndrobin.7z

[Graham Banks]

Very low draw rate as expected with unbalanced openings.

Indeed, but probably a lot of rubbish games.

[Ray]

No real surprises.

No, I don't see any big surprises. Yes there is a very low draw rate, but the resulting rankings and ratings of the engines seem similar, all within the error margins with a small number of games.

[Ray]

Very low draw rate as expected with unbalanced openings.

Indeed, but probably a lot of rubbish games.

I don't think so, these are the least-unbalanced in that opening set, nothing extreme, and remember they are all from human games.

Thing is with NNUE engines, the evals are no longer centipawns in many cases, so you can't judge the "fairness" of the opening from those evals anymore.

[bastiball]

Try run some tournament with a handicap

[Ray]

Try run some tournament with a handicap

I'm not really interested in that. But other ideas for something a bit different are welcome . I'm still keen on a DFRC tournament of some sort, but I need to put some work in to get a subset of those openings to work with.