Round-Robin Tournament using SPCC UHO_2022 openings
Round-Robin Tournament using SPCC UHO_2022 openings
I decided to try a one-off experiment, a tourmament using the mildly unbalanced 6-move UHO_2022_6mvs_+110_+119.pgn compiled by Stefan Pohl.
Testing conditions:
Format: Round robin tournament with the top 14 engines from CCRL single CPU lists
GUI: Cutechess with 500ms overstep margin, concurrency 12
Hardware: Ryzen 9 5900X,
Time Control: 90”+0.75” (which is CCRL 2'+1" on this hardware)
Book: 6-move UHO_2022_6mvs_+110_+119.pgn
60 games per pairing (30 random openings played reversed sides)
Each engine 512MB hash, 1 thread, ponder off
Adjudication: Syzygy 5-men only, otherwise the games played out until their natural conclusion
(checkmate, stalemate, insufficient material, 3-fold repetition or 50-moves rule,)
Participants:
Stockfish 20230531 (new bigger net as at date of this tournament)
Komodo Dragon 3.2
Berserk 11.1
Ethereal 14.00
Koivisto 9
RubiChess 20230410
Revenge 3.0
Rebel 16.2
Seer 2.6.0
Igel 3.4.0
Clover 4.1
SlowChess 2.9
Rofchade 3.0
Uralochka3.39d
Results in another 24 hours.
Testing conditions:
Format: Round robin tournament with the top 14 engines from CCRL single CPU lists
GUI: Cutechess with 500ms overstep margin, concurrency 12
Hardware: Ryzen 9 5900X,
Time Control: 90”+0.75” (which is CCRL 2'+1" on this hardware)
Book: 6-move UHO_2022_6mvs_+110_+119.pgn
60 games per pairing (30 random openings played reversed sides)
Each engine 512MB hash, 1 thread, ponder off
Adjudication: Syzygy 5-men only, otherwise the games played out until their natural conclusion
(checkmate, stalemate, insufficient material, 3-fold repetition or 50-moves rule,)
Participants:
Stockfish 20230531 (new bigger net as at date of this tournament)
Komodo Dragon 3.2
Berserk 11.1
Ethereal 14.00
Koivisto 9
RubiChess 20230410
Revenge 3.0
Rebel 16.2
Seer 2.6.0
Igel 3.4.0
Clover 4.1
SlowChess 2.9
Rofchade 3.0
Uralochka3.39d
Results in another 24 hours.
Re: Round-Robin Tournament using SPCC UHO_2022 openings
BayesElo ratings, default parameters:
Rank Name Elo + - games score draws
1 Stockfish 20230531 157 20 19 780 76% 47%
2 Komodo Dragon 3.2 120 19 19 780 70% 50%
3 Berserk 11.1 70 18 18 780 62% 53%
4 Ethereal 14.00 35 18 18 780 56% 54%
5 RubiChess 20230410 17 18 18 780 53% 54%
6 Koivisto 9 17 18 18 780 53% 52%
7 Revenge 3.0 -10 18 18 780 48% 54%
8 Rebel 16.2 -25 18 18 780 46% 52%
9 Seer 2.6.0 -35 18 18 780 44% 52%
10 Igel 3.4.0 -42 18 18 780 43% 54%
11 Clover 4.1 -50 18 18 780 41% 51%
12 SlowChess 2.9 -73 18 19 780 38% 48%
13 Uralochka3.39d -89 18 19 780 35% 51%
14 Rofchade 3.0 -92 19 19 780 35% 45%
Re: Round-Robin Tournament using SPCC UHO_2022 openings
Head to head statistics:
1) Stockfish 20230531 780 (+409,=367,-4), 76.0 %
vs. : games ( +, =, -), (%) :
Komodo Dragon 3.2 : 60 ( 25, 31, 4), 67.5 :
Berserk 11.1 : 60 ( 27, 33, 0), 72.5 :
Ethereal 14.00 : 60 ( 31, 29, 0), 75.8 :
RubiChess 20230410 : 60 ( 32, 28, 0), 76.7 :
Koivisto 9 : 60 ( 32, 28, 0), 76.7 :
Revenge 3.0 : 60 ( 31, 29, 0), 75.8 :
Rebel 16.2 : 60 ( 32, 28, 0), 76.7 :
Seer 2.6.0 : 60 ( 34, 26, 0), 78.3 :
Igel 3.4.0 : 60 ( 32, 28, 0), 76.7 :
Clover 4.1 : 60 ( 33, 27, 0), 77.5 :
SlowChess 2.9 : 60 ( 35, 25, 0), 79.2 :
Uralochka3.39d : 60 ( 32, 28, 0), 76.7 :
Rofchade 3.0 : 60 ( 33, 27, 0), 77.5 :
2) Komodo Dragon 3.2 780 (+355,=387,-38), 70.3 %
vs. : games ( +, =, -), (%) :
Stockfish 20230531 : 60 ( 4, 31, 25), 32.5 :
Berserk 11.1 : 60 ( 21, 37, 2), 65.8 :
Ethereal 14.00 : 60 ( 28, 31, 1), 72.5 :
RubiChess 20230410 : 60 ( 26, 30, 4), 68.3 :
Koivisto 9 : 60 ( 26, 32, 2), 70.0 :
Revenge 3.0 : 60 ( 25, 33, 2), 69.2 :
Rebel 16.2 : 60 ( 28, 31, 1), 72.5 :
Seer 2.6.0 : 60 ( 29, 31, 0), 74.2 :
Igel 3.4.0 : 60 ( 34, 26, 0), 78.3 :
Clover 4.1 : 60 ( 31, 28, 1), 75.0 :
SlowChess 2.9 : 60 ( 36, 24, 0), 80.0 :
Uralochka3.39d : 60 ( 34, 26, 0), 78.3 :
Rofchade 3.0 : 60 ( 33, 27, 0), 77.5 :
3) Berserk 11.1 780 (+280,=411,-89), 62.2 %
vs. : games ( +, =, -), (%) :
Stockfish 20230531 : 60 ( 0, 33, 27), 27.5 :
Komodo Dragon 3.2 : 60 ( 2, 37, 21), 34.2 :
Ethereal 14.00 : 60 ( 17, 37, 6), 59.2 :
RubiChess 20230410 : 60 ( 20, 30, 10), 58.3 :
Koivisto 9 : 60 ( 22, 30, 08), 61.7 :
Revenge 3.0 : 60 ( 22, 34, 4), 65.0 :
Rebel 16.2 : 60 ( 31, 23, 6), 70.8 :
Seer 2.6.0 : 60 ( 25, 33, 2), 69.2 :
Igel 3.4.0 : 60 ( 23, 36, 1), 68.3 :
Clover 4.1 : 60 ( 29, 30, 1), 73.3 :
SlowChess 2.9 : 60 ( 27, 32, 1), 71.7 :
Uralochka3.39d : 60 ( 28, 30, 2), 71.7 :
Rofchade 3.0 : 60 ( 34, 26, 0), 78.3 :
4) Ethereal 14.00 780 (+228,=420,-132), 56.2 %
vs. : games ( +, =, -), (%) :
Stockfish 20230531 : 60 ( 0, 29, 31), 24.2 :
Komodo Dragon 3.2 : 60 ( 1, 31, 28), 27.5 :
Berserk 11.1 : 60 ( 6, 37, 17), 40.8 :
RubiChess 20230410 : 60 ( 15, 30, 15), 50.0 :
Koivisto 9 : 60 ( 14, 43, 3), 59.2 :
Revenge 3.0 : 60 ( 17, 34, 9), 56.7 :
Rebel 16.2 : 60 ( 21, 32, 7), 61.7 :
Seer 2.6.0 : 60 ( 23, 29, 08), 62.5 :
Igel 3.4.0 : 60 ( 26, 30, 4), 68.3 :
Clover 4.1 : 60 ( 23, 34, 3), 66.7 :
SlowChess 2.9 : 60 ( 24, 33, 3), 67.5 :
Uralochka3.39d : 60 ( 27, 31, 2), 70.8 :
Rofchade 3.0 : 60 ( 31, 27, 2), 74.2 :
5) RubiChess 20230410 780 (+205,=418,-157), 53.1 %
vs. : games ( +, =, -), (%) :
Stockfish 20230531 : 60 ( 0, 28, 32), 23.3 :
Komodo Dragon 3.2 : 60 ( 4, 30, 26), 31.7 :
Berserk 11.1 : 60 ( 10, 30, 20), 41.7 :
Ethereal 14.00 : 60 ( 15, 30, 15), 50.0 :
Koivisto 9 : 60 ( 14, 34, 12), 51.7 :
Revenge 3.0 : 60 ( 17, 32, 11), 55.0 :
Rebel 16.2 : 60 ( 20, 32, 08), 60.0 :
Seer 2.6.0 : 60 ( 16, 39, 5), 59.2 :
Igel 3.4.0 : 60 ( 17, 39, 4), 60.8 :
Clover 4.1 : 60 ( 18, 34, 08), 58.3 :
SlowChess 2.9 : 60 ( 21, 32, 7), 61.7 :
Uralochka3.39d : 60 ( 23, 34, 3), 66.7 :
Rofchade 3.0 : 60 ( 30, 24, 6), 70.0 :
6) Koivisto 9 780 (+209,=409,-162), 53.0 %
vs. : games ( +, =, -), (%) :
Stockfish 20230531 : 60 ( 0, 28, 32), 23.3 :
Komodo Dragon 3.2 : 60 ( 2, 32, 26), 30.0 :
Berserk 11.1 : 60 ( 8, 30, 22), 38.3 :
Ethereal 14.00 : 60 ( 3, 43, 14), 40.8 :
RubiChess 20230410 : 60 ( 12, 34, 14), 48.3 :
Revenge 3.0 : 60 ( 16, 35, 9), 55.8 :
Rebel 16.2 : 60 ( 22, 27, 11), 59.2 :
Seer 2.6.0 : 60 ( 23, 34, 3), 66.7 :
Igel 3.4.0 : 60 ( 18, 32, 10), 56.7 :
Clover 4.1 : 60 ( 28, 26, 6), 68.3 :
SlowChess 2.9 : 60 ( 24, 29, 7), 64.2 :
Uralochka3.39d : 60 ( 27, 28, 5), 68.3 :
Rofchade 3.0 : 60 ( 26, 31, 3), 69.2 :
7) Revenge 3.0 780 (+164,=425,-191), 48.3 %
vs. : games ( +, =, -), (%) :
Stockfish 20230531 : 60 ( 0, 29, 31), 24.2 :
Komodo Dragon 3.2 : 60 ( 2, 33, 25), 30.8 :
Berserk 11.1 : 60 ( 4, 34, 22), 35.0 :
Ethereal 14.00 : 60 ( 9, 34, 17), 43.3 :
RubiChess 20230410 : 60 ( 11, 32, 17), 45.0 :
Koivisto 9 : 60 ( 9, 35, 16), 44.2 :
Rebel 16.2 : 60 ( 9, 40, 11), 48.3 :
Seer 2.6.0 : 60 ( 16, 30, 14), 51.7 :
Igel 3.4.0 : 60 ( 17, 37, 6), 59.2 :
Clover 4.1 : 60 ( 19, 35, 6), 60.8 :
SlowChess 2.9 : 60 ( 19, 30, 11), 56.7 :
Uralochka3.39d : 60 ( 23, 32, 5), 65.0 :
Rofchade 3.0 : 60 ( 26, 24, 10), 63.3 :
8.) Rebel 16.2 780 (+153,=409,-218), 45.8 %
vs. : games ( +, =, -), (%) :
Stockfish 20230531 : 60 ( 0, 28, 32), 23.3 :
Komodo Dragon 3.2 : 60 ( 1, 31, 28), 27.5 :
Berserk 11.1 : 60 ( 6, 23, 31), 29.2 :
Ethereal 14.00 : 60 ( 7, 32, 21), 38.3 :
RubiChess 20230410 : 60 ( 8, 32, 20), 40.0 :
Koivisto 9 : 60 ( 11, 27, 22), 40.8 :
Revenge 3.0 : 60 ( 11, 40, 9), 51.7 :
Seer 2.6.0 : 60 ( 15, 33, 12), 52.5 :
Igel 3.4.0 : 60 ( 11, 38, 11), 50.0 :
Clover 4.1 : 60 ( 20, 30, 10), 58.3 :
SlowChess 2.9 : 60 ( 22, 27, 11), 59.2 :
Uralochka3.39d : 60 ( 21, 34, 5), 63.3 :
Rofchade 3.0 : 60 ( 20, 34, 6), 61.7 :
9) Seer 2.6.0 780 (+141,=402,-237), 43.8 %
vs. : games ( +, =, -), (%) :
Stockfish 20230531 : 60 ( 0, 26, 34), 21.7 :
Komodo Dragon 3.2 : 60 ( 0, 31, 29), 25.8 :
Berserk 11.1 : 60 ( 2, 33, 25), 30.8 :
Ethereal 14.00 : 60 ( 8, 29, 23), 37.5 :
RubiChess 20230410 : 60 ( 5, 39, 16), 40.8 :
Koivisto 9 : 60 ( 3, 34, 23), 33.3 :
Revenge 3.0 : 60 ( 14, 30, 16), 48.3 :
Rebel 16.2 : 60 ( 12, 33, 15), 47.5 :
Igel 3.4.0 : 60 ( 12, 32, 16), 46.7 :
Clover 4.1 : 60 ( 14, 34, 12), 51.7 :
SlowChess 2.9 : 60 ( 22, 25, 13), 57.5 :
Uralochka3.39d : 60 ( 23, 30, 7), 63.3 :
Rofchade 3.0 : 60 ( 26, 26, 08), 65.0 :
10) Igel 3.4.0 780 (+121,=423,-236), 42.6 %
vs. : games ( +, =, -), (%) :
Stockfish 20230531 : 60 ( 0, 28, 32), 23.3 :
Komodo Dragon 3.2 : 60 ( 0, 26, 34), 21.7 :
Berserk 11.1 : 60 ( 1, 36, 23), 31.7 :
Ethereal 14.00 : 60 ( 4, 30, 26), 31.7 :
RubiChess 20230410 : 60 ( 4, 39, 17), 39.2 :
Koivisto 9 : 60 ( 10, 32, 18), 43.3 :
Revenge 3.0 : 60 ( 6, 37, 17), 40.8 :
Rebel 16.2 : 60 ( 11, 38, 11), 50.0 :
Seer 2.6.0 : 60 ( 16, 32, 12), 53.3 :
Clover 4.1 : 60 ( 15, 33, 12), 52.5 :
SlowChess 2.9 : 60 ( 18, 27, 15), 52.5 :
Uralochka3.39d : 60 ( 20, 35, 5), 62.5 :
Rofchade 3.0 : 60 ( 16, 30, 14), 51.7 :
11) Clover 4.1 780 (+123,=401,-256), 41.5 %
vs. : games ( +, =, -), (%) :
Stockfish 20230531 : 60 ( 0, 27, 33), 22.5 :
Komodo Dragon 3.2 : 60 ( 1, 28, 31), 25.0 :
Berserk 11.1 : 60 ( 1, 30, 29), 26.7 :
Ethereal 14.00 : 60 ( 3, 34, 23), 33.3 :
RubiChess 20230410 : 60 ( 8, 34, 18), 41.7 :
Koivisto 9 : 60 ( 6, 26, 28), 31.7 :
Revenge 3.0 : 60 ( 6, 35, 19), 39.2 :
Rebel 16.2 : 60 ( 10, 30, 20), 41.7 :
Seer 2.6.0 : 60 ( 12, 34, 14), 48.3 :
Igel 3.4.0 : 60 ( 12, 33, 15), 47.5 :
SlowChess 2.9 : 60 ( 19, 34, 7), 60.0 :
Uralochka3.39d : 60 ( 22, 28, 10), 60.0 :
Rofchade 3.0 : 60 ( 23, 28, 9), 61.7 :
12) SlowChess 2.9 780 (+107,=374,-299), 37.7 %
vs. : games ( +, =, -), (%) :
Stockfish 20230531 : 60 ( 0, 25, 35), 20.8 :
Komodo Dragon 3.2 : 60 ( 0, 24, 36), 20.0 :
Berserk 11.1 : 60 ( 1, 32, 27), 28.3 :
Ethereal 14.00 : 60 ( 3, 33, 24), 32.5 :
RubiChess 20230410 : 60 ( 7, 32, 21), 38.3 :
Koivisto 9 : 60 ( 7, 29, 24), 35.8 :
Revenge 3.0 : 60 ( 11, 30, 19), 43.3 :
Rebel 16.2 : 60 ( 11, 27, 22), 40.8 :
Seer 2.6.0 : 60 ( 13, 25, 22), 42.5 :
Igel 3.4.0 : 60 ( 15, 27, 18), 47.5 :
Clover 4.1 : 60 ( 7, 34, 19), 40.0 :
Uralochka3.39d : 60 ( 11, 35, 14), 47.5 :
Rofchade 3.0 : 60 ( 21, 21, 18), 52.5 :
13) Uralochka3.39d 780 (+74,=396,-310), 34.9 %
vs. : games ( +, =, -), (%) :
Stockfish 20230531 : 60 ( 0, 28, 32), 23.3 :
Komodo Dragon 3.2 : 60 ( 0, 26, 34), 21.7 :
Berserk 11.1 : 60 ( 2, 30, 28), 28.3 :
Ethereal 14.00 : 60 ( 2, 31, 27), 29.2 :
RubiChess 20230410 : 60 ( 3, 34, 23), 33.3 :
Koivisto 9 : 60 ( 5, 28, 27), 31.7 :
Revenge 3.0 : 60 ( 5, 32, 23), 35.0 :
Rebel 16.2 : 60 ( 5, 34, 21), 36.7 :
Seer 2.6.0 : 60 ( 7, 30, 23), 36.7 :
Igel 3.4.0 : 60 ( 5, 35, 20), 37.5 :
Clover 4.1 : 60 ( 10, 28, 22), 40.0 :
SlowChess 2.9 : 60 ( 14, 35, 11), 52.5 :
Rofchade 3.0 : 60 ( 16, 25, 19), 47.5 :
14) Rofchade 3.0 780 (+95,=350,-335), 34.6 %
vs. : games ( +, =, -), (%) :
Stockfish 20230531 : 60 ( 0, 27, 33), 22.5 :
Komodo Dragon 3.2 : 60 ( 0, 27, 33), 22.5 :
Berserk 11.1 : 60 ( 0, 26, 34), 21.7 :
Ethereal 14.00 : 60 ( 2, 27, 31), 25.8 :
RubiChess 20230410 : 60 ( 6, 24, 30), 30.0 :
Koivisto 9 : 60 ( 3, 31, 26), 30.8 :
Revenge 3.0 : 60 ( 10, 24, 26), 36.7 :
Rebel 16.2 : 60 ( 6, 34, 20), 38.3 :
Seer 2.6.0 : 60 ( 8, 26, 26), 35.0 :
Igel 3.4.0 : 60 ( 14, 30, 16), 48.3 :
Clover 4.1 : 60 ( 9, 28, 23), 38.3 :
SlowChess 2.9 : 60 ( 18, 21, 21), 47.5 :
Uralochka3.39d : 60 ( 19, 25, 16), 52.5 :
Re: Round-Robin Tournament using SPCC UHO_2022 openings
If you prefer Ordo ratings:
1 Stockfish 20230531 : 2494.2 592.5 780 76
2 Komodo Dragon 3.2 : 2446.5 548.5 780 70
3 Berserk 11.1 : 2384.8 485.5 780 62
4 Ethereal 14.00 : 2341.4 438.0 780 56
5 RubiChess 20230410 : 2319.9 414.0 780 53
6 Koivisto 9 : 2319.5 413.5 780 53
7 Revenge 3.0 : 2286.7 376.5 780 48
8 Rebel 16.2 : 2269.9 357.5 780 46
9 Seer 2.6.0 : 2256.0 342.0 780 44
10 Igel 3.4.0 : 2247.5 332.5 780 43
11 Clover 4.1 : 2239.4 323.5 780 41
12 SlowChess 2.9 : 2212.4 294.0 780 38
13 Uralochka3.39d : 2191.8 272.0 780 35
14 Rofchade 3.0 : 2189.9 270.0 780 35
- Graham Banks
- Posts: 27029
- Joined: Sun Dec 18, 2005 5:47 pm
- Sign-up code: 0
- Location: Auckland, NZ
Re: Round-Robin Tournament using SPCC UHO_2022 openings
No real surprises.
-
- Posts: 1992
- Joined: Thu Aug 05, 2021 2:35 pm
- Sign-up code: 10159
- Location: Cavite, Philippines
- Contact:
Re: Round-Robin Tournament using SPCC UHO_2022 openings
InterestingRay wrote: ↑Mon Jun 05, 2023 12:58 am BayesElo ratings, default parameters:
Rank Name Elo + - games score oppo. draws 1 Stockfish 20230531 157 20 19 780 76% -12 47% 2 Komodo Dragon 3.2 120 19 19 780 70% -9 50% 3 Berserk 11.1 70 18 18 780 62% -5 53% 4 Ethereal 14.00 35 18 18 780 56% -3 54% 5 RubiChess 20230410 17 18 18 780 53% -1 54% 6 Koivisto 9 17 18 18 780 53% -1 52% 7 Revenge 3.0 -10 18 18 780 48% 1 54% 8 Rebel 16.2 -25 18 18 780 46% 2 52% 9 Seer 2.6.0 -35 18 18 780 44% 3 52% 10 Igel 3.4.0 -42 18 18 780 43% 3 54% 11 Clover 4.1 -50 18 18 780 41% 4 51% 12 SlowChess 2.9 -73 18 19 780 38% 6 48% 13 Uralochka3.39d -89 18 19 780 35% 7 51% 14 Rofchade 3.0 -92 19 19 780 35% 7 45%
CCRL Testing Group
Re: Round-Robin Tournament using SPCC UHO_2022 openings
Very low draw rate as expected with unbalanced openings.
Rankings of engines are similar to the standard CCRL blitz list. Any differences could just be statistical margins of error.
Games can be downloaded here for anyone interested:
http://ccrl.chessdom.com/public/UHO2022 ... ndrobin.7z
Rankings of engines are similar to the standard CCRL blitz list. Any differences could just be statistical margins of error.
Games can be downloaded here for anyone interested:
http://ccrl.chessdom.com/public/UHO2022 ... ndrobin.7z
- Graham Banks
- Posts: 27029
- Joined: Sun Dec 18, 2005 5:47 pm
- Sign-up code: 0
- Location: Auckland, NZ
Re: Round-Robin Tournament using SPCC UHO_2022 openings
No, I don't see any big surprises. Yes there is a very low draw rate, but the resulting rankings and ratings of the engines seem similar, all within the error margins with a small number of games.
Re: Round-Robin Tournament using SPCC UHO_2022 openings
I don't think so, these are the least-unbalanced in that opening set, nothing extreme, and remember they are all from human games.
Thing is with NNUE engines, the evals are no longer centipawns in many cases, so you can't judge the "fairness" of the opening from those evals anymore.
-
- Posts: 1992
- Joined: Thu Aug 05, 2021 2:35 pm
- Sign-up code: 10159
- Location: Cavite, Philippines
- Contact:
Re: Round-Robin Tournament using SPCC UHO_2022 openings
Try run some tournament with a handicap
CCRL Testing Group
Re: Round-Robin Tournament using SPCC UHO_2022 openings
I'm not really interested in that. But other ideas for something a bit different are welcome . I'm still keen on a DFRC tournament of some sort, but I need to put some work in to get a subset of those openings to work with.