CCRL 40/40: Free Single-CPU "Diagonal" tournament

Questions and comments related to CCRL testing study

CCRL 40/40: Free Single-CPU "Diagonal" tournament

Postby Kirill Kryukov » Thu Aug 31, 2006 1:29 pm

Hi all! I am starting to post my results here. I am doing "diagonal" testing for free single-CPU engines at the moment. Diagonal means that I test engine pairs which have close ratings - +- 5 positions in the rating list. This results in a cross-table where diagonal going from top-left to bottom-right corner is filled with results, while bottom-left area and rop-right area remains empty.

Today's results:

Ruffian 1.0.5 - Slow Chess Blitz WV2.1: 16 - 16 (+7-7=18 )
Ruffian 1.0.5 - Naum 1.91 64-bit: 21 - 11 (+14-4=14)


Games will be available with the next update of our rating list and database here: http://computerchess.org.uk/ccrl/4040/

Any comments, suggestions, or help is welcome. :-)
Last edited by Kirill Kryukov on Fri Sep 01, 2006 3:33 pm, edited 1 time in total.
User avatar
Kirill Kryukov
Site Admin
 
Posts: 7385
Joined: Sun Dec 18, 2005 9:58 am
Location: Mishima, Japan

Postby Graham Banks » Thu Aug 31, 2006 6:38 pm

Hi Kirill,

it's good to see you posting your results here for those interested to follow! 8)

A good result there for Ruffian 1.0.5 over Naum 1.91! :shock:

Graham.
User avatar
Graham Banks
 
Posts: 18497
Joined: Sun Dec 18, 2005 5:47 pm
Location: Auckland, NZ

Postby Kirill Kryukov » Fri Sep 01, 2006 3:33 pm

Thanks for comments, Graham! :-) I'll try to remember to post here.

Good results from Ruffian 1.0.5 so far. Someone suggested that it can compete within top 10, and so far it seems correct. Good score for engine of its age.

Today another match finished here:

List 5.12 - Delfi 4.6: 21 - 11 (+14-4=14)
User avatar
Kirill Kryukov
Site Admin
 
Posts: 7385
Joined: Sun Dec 18, 2005 9:58 am
Location: Mishima, Japan

Postby Graham Banks » Fri Sep 01, 2006 7:34 pm

Kirill Kryukov wrote:Thanks for comments, Graham! :-) I'll try to remember to post here.

Good results from Ruffian 1.0.5 so far. Someone suggested that it can compete within top 10, and so far it seems correct. Good score for engine of its age.

Today another match finished here:

List 5.12 - Delfi 4.6: 21 - 11 (+14-4=14)


Delfi 4.6 is a solid engine. I'm surprised that List 512 won the match so comfortably. :o
User avatar
Graham Banks
 
Posts: 18497
Joined: Sun Dec 18, 2005 5:47 pm
Location: Auckland, NZ

Postby Kirill Kryukov » Sat Sep 02, 2006 3:12 am

Graham Banks wrote:Delfi 4.6 is a solid engine. I'm surprised that List 512 won the match so comfortably. :o

I'm surprised too - it is +66 performance for List 5.12. Also my guess is that Delfi 4.6 rating was a bit high, due to uneven opposition or statistical error.. This guess is based on Delfi 5.0 performance so far. Anyway, soon three more Delfi 4.6 matches should finish here, so it will have more reliable rating.
User avatar
Kirill Kryukov
Site Admin
 
Posts: 7385
Joined: Sun Dec 18, 2005 9:58 am
Location: Mishima, Japan

Postby Graham Banks » Sat Sep 02, 2006 5:57 am

Kirill Kryukov wrote:Anyway, soon three more Delfi 4.6 matches should finish here, so it will have more reliable rating.


Good - Charles also has Delfi 4.6 in his Goodwill II Tournament, so there's a further 42 games to come also.
User avatar
Graham Banks
 
Posts: 18497
Joined: Sun Dec 18, 2005 5:47 pm
Location: Auckland, NZ

Postby Kirill Kryukov » Sun Sep 03, 2006 6:49 am

One match finished today:

Ruffian 1.0.5 - Delfi 4.6: 17.5 - 14.5 (+9-6=17)
User avatar
Kirill Kryukov
Site Admin
 
Posts: 7385
Joined: Sun Dec 18, 2005 9:58 am
Location: Mishima, Japan

Postby Kirill Kryukov » Tue Sep 05, 2006 3:09 am

More results for Delfi 4.6:

Delfi 4.6 - Aristarch 4.50: 11.5 - 20.5 (+7-16=19)
Delfi 4.6 - Jonny 2.83 64-bit: 10 - 22 (+4-16=12)
User avatar
Kirill Kryukov
Site Admin
 
Posts: 7385
Joined: Sun Dec 18, 2005 9:58 am
Location: Mishima, Japan

Postby Kirill Kryukov » Wed Sep 06, 2006 5:40 pm

Aristarch 4.50 - Wildcat 6: 18.5 - 13.5 (+11-6=15)
User avatar
Kirill Kryukov
Site Admin
 
Posts: 7385
Joined: Sun Dec 18, 2005 9:58 am
Location: Mishima, Japan

Postby Kirill Kryukov » Sat Sep 09, 2006 5:20 am

Wildcat 6 - Jonny 2.83 64-bit: 16 - 16 (+8-8=16)
Wildcat 6 - Delfi 5.0: 17 - 15 (+11-9=12)
User avatar
Kirill Kryukov
Site Admin
 
Posts: 7385
Joined: Sun Dec 18, 2005 9:58 am
Location: Mishima, Japan

Postby Kirill Kryukov » Mon Sep 25, 2006 12:11 am

New Wildcat 6 results:

Wildcat 6 - List 5.12: 17 - 15 (+11-9=12)
Wildcat 6 - Ruffian 1.0.5: 15.5 - 16.5 (+10-11=11)
User avatar
Kirill Kryukov
Site Admin
 
Posts: 7385
Joined: Sun Dec 18, 2005 9:58 am
Location: Mishima, Japan

Postby Kirill Kryukov » Sat Sep 30, 2006 1:21 pm

New results for Ruffian 1.0.5:

Ruffian 1.0.5 - Rybka 1.0 Beta 64-bit: 7 - 25 (+1-19=12)
Ruffian 1.0.5 - Toga II 1.2.1a 32-bit: 6.5 - 25.5 (+2-21=9)
Ruffian 1.0.5 - Spike 1.2 Turin: 8.5 - 23.5 (+4-19=9)
Ruffian 1.0.5 - Glaurung 1.2.1 64-bit 1-CPU: 13 - 19 (+6-12=14)
Ruffian 1.0.5 - Scorpio 1.8 1-CPU 4-men-egbb: 10.5 - 21.5 (+5-16=11)
Ruffian 1.0.5 - Jonny 2.83 64-bit: 24 - 8 (+20-4=8)
Ruffian 1.0.5 - Delfi 5.0: 17.5 - 14.5 (+10-7=15)


(All results of Ruffian 1.0.5 in CCRL 40/40)

With these results Ruffian 1.0.5 has rating of 2722 ELO points, and shares 9-10 places in 40/40 Free Single-CPU list (shared with Jonny 2.83). So far still within top 10, quite good for engine of its age. Still good challenge for new amateur engines. :-)

Currently Ruffian 1.0.5 is only 9 points lower than Ruffian 2.1.0 (latest version) in our list. (Comparison page).
Last edited by Kirill Kryukov on Tue Oct 10, 2006 1:40 am, edited 1 time in total.
User avatar
Kirill Kryukov
Site Admin
 
Posts: 7385
Joined: Sun Dec 18, 2005 9:58 am
Location: Mishima, Japan

Postby Kirill Kryukov » Mon Oct 09, 2006 4:30 pm

Results of some more Free Single-CPU matches:

Pseudo 0.7c - Wildcat 6: 15 - 17 (+8-10=14)
Pseudo 0.7c - Aristarch 4.50: 17 - 15 (+13-11=8)
Pseudo 0.7c - Anaconda 2.0.1: 13.5 - 18.5 (+6-11=15)
Pseudo 0.7c - SOS 5.1: 17 - 15 (+10-8=14)
Pseudo 0.7c - Ruffian 1.0.5: 14 - 18 (+7-11=14)

Pharaon 3.5.1 - Jonny 2.83 64-bit: 17 - 15 (+12-10=10)
Pharaon 3.5.1 - Ruffian 1.0.5: 14 - 18 (+8-12=12)
Pharaon 3.5.1 - List 5.12: 18 - 14 (+15-11=6)
Pharaon 3.5.1 - Glaurung 1.2.1 64-bit 1-CPU: 11 - 21 (+5-15=12)
Pharaon 3.5.1 - Scorpio 1.8 1-CPU 4-men-egbb: 14 - 18 (+9-13=10)
Pharaon 3.5.1 - Delfi 5.0: 22.5 - 9.5 (+17-4=11)

Jonny 2.83 32-bit - List 5.12: 7.5 - 24.5 (+4-21=7)

Crafty 21.1 64-bit 1-CPU - Naum 1.91 64-bit: 8.5 - 23.5 (+4-19=9)
Crafty 21.1 64-bit 1-CPU - Delfi 5.0: 10 - 22 (+6-18=8)
User avatar
Kirill Kryukov
Site Admin
 
Posts: 7385
Joined: Sun Dec 18, 2005 9:58 am
Location: Mishima, Japan

Postby Graham Banks » Mon Oct 09, 2006 7:53 pm

Great stuff Kirill.

You're giving a good number of games to engines that need more games. 8)
User avatar
Graham Banks
 
Posts: 18497
Joined: Sun Dec 18, 2005 5:47 pm
Location: Auckland, NZ

Postby Kirill Kryukov » Tue Oct 10, 2006 1:41 am

Thanks Graham! Just plugging some holes in Free Single-CPU List. Still lot of work to do. :-)
User avatar
Kirill Kryukov
Site Admin
 
Posts: 7385
Joined: Sun Dec 18, 2005 9:58 am
Location: Mishima, Japan

Postby Kirill Kryukov » Sat Oct 14, 2006 5:39 am

Some more results this week:

Jonny 2.83 32-bit - Ruffian 1.0.5: 14.5 - 17.5 (+10-13=9)
Pharaon 3.5.1 - Rybka 1.0 Beta 64-bit: 5 - 27 (+2-24=6)
Pharaon 3.5.1 - Toga II 1.2.1a 32-bit: 8.5 - 23.5 (+5-20=7)
Pharaon 3.5.1 - Spike 1.2 Turin: 11.5 - 20.5 (+5-14=13)
User avatar
Kirill Kryukov
Site Admin
 
Posts: 7385
Joined: Sun Dec 18, 2005 9:58 am
Location: Mishima, Japan


Return to CCRL Public

Who is online

Users browsing this forum: No registered users and 3 guests