Suggestion for testing directions

Questions and comments related to CCRL testing study
Post Reply
megamau
Posts: 3
Joined: Sun Oct 21, 2007 12:17 pm

Suggestion for testing directions

Post by megamau »

Hello.

Let me start by saying that I love your work.
Not because it is one of the most complete and rigorous rating lists (together with CEGT and SSDF), but because of the way the data is classified and presented.
Lots of stats, everything customizable and clearly explained through colors, graphs and so on.

Because of this easiness of consulting the data and the richness of the data, it is easy to find which are the interesting directions for testing and to ask for more.
I give a couple of suggestions for the site that I think will be interesting to implement. Feel free to ignore some or all of them....I understand there is many things to do and little time, and I know probably you already though about some of this ideas and didn't implement them just for lack of time, but I think they would really be very nice.

- Testing Glaurung 2-epsilon/5 4CPU
from the difference between the 2 and 4 CPU Glaurung 1.2.1 and the current position of Glaurung 2-epsilon/5 2 CPU it seems that we may have a new champion in the "free" category

- Make default the comparison of the list with the previous one
I like especially how you can see at a glance the number of games played in the last week, without having to go to the relevant topic in this forum

- Dedicate an own tab to the "Custom comparison".
This would make this great feature more visible (it is quite hidden now) and free some space from the "Index" page which is a bit too long.

- Remove the "download" from the bottom of the "index",
This because there is already a dedicated tab "Games", and it would free some space on the bottom of the index. I would also rename this tab to "Download Games", but that is a question of tastes.
- Remove the "About" tab
It seems there is nothing under this tab.

- Improve the step 1 in the custom comparison
This is a great feature, but i think it could be improved even more.
Instead of all those classes, I would give 3 block of multiple choices, so the user could combine at his will:
a) CPU
with six choices, called All, 1cpu, 2cpu, 4cpu, 1-2 cpu, 2-4 cpu
b) Class
with 5 choices, called All, private, commercial, free, open source
c) 32-64 bit
with the obvious 3 choices (all, 32bit, 64bit)
By the way, I think the naming for class a is more clear that the current naming (used at the bottom of the rating list when showing further lists)

- Publicize more the "testing profile"
Maybe by including it by default in the "index" page. It is a great feature, but it is king of hidden now.

- Test more Rybka 2.1 64bit 2CPU
Looking at the rest of the Rybka tests, this one seems to stand out a little as the odd one out. In particular it seems stronger than it should be, given the performance of the other version. I think more test of this particular version would make the situation clearer.

- Dynamically adjust the color scale in the tables with performance
With the current fixed -120 to +120 scale, most of the cells end up in a greenish/blue zone, which is difficult to tell apart.

That is all for now. Just let me repeat, this is not a critic in any way. I greatly enjoy the site, and these are just my suggestions for improving it even further.

Maurizio
User avatar
Graham Banks
Posts: 26889
Joined: Sun Dec 18, 2005 5:47 pm
Sign-up code: 0
Location: Auckland, NZ

Re: Suggestion for testing directions

Post by Graham Banks »

Welcome to the forum Maurizio and thanks for your suggestions.
We will certainly take a look at them all. :)

Regards, Graham.
User avatar
Shaun
Posts: 6889
Joined: Sat May 13, 2006 3:24 pm
Sign-up code: 10159
Location: Brighton. UK

Re: Suggestion for testing directions

Post by Shaun »

Maurizio,

thank you for your detailed comments it is always useful to have feedback.

Shaun
User avatar
Kirill Kryukov
Site Admin
Posts: 7399
Joined: Sun Dec 18, 2005 9:58 am
Sign-up code: 0
Location: Mishima, Japan
Contact:

Re: Suggestion for testing directions

Post by Kirill Kryukov »

Thanks for comments, Maurizio! We'll look into them although you are right to note that free time is a major issue.
megamau
Posts: 3
Joined: Sun Oct 21, 2007 12:17 pm

Re: Suggestion for testing directions

Post by megamau »

Just to restate the suggestions that I think are easier to implement and will be very much appreciated by the users.

- Create a dedicated "custom comparison" Tab

- Improve the step 1 in the custom comparison
Instead of all those classes, give 3 block of multiple choices, so the user could combine at his will:
a) CPU
All, 1cpu, 2cpu, 4cpu, 1-2 cpu, 2-4 cpu
b) Class
All, private, commercial, free, open source
c) 32-64 bit
with the obvious 3 choices (all, 32bit, 64bit)
Uri Blass
Posts: 39
Joined: Tue Jun 20, 2006 12:44 pm
Sign-up code: 0

Re: Suggestion for testing directions

Post by Uri Blass »

I can suggest to test Movei progress 10 10 10 in future tournaments and not the default personality that has enough games.

All the evidence in tournaments of other people that I see support that this personality is better than the default
personality.

From the CCRL blitz
Movei 00.8.438(10 10 10) 2729 +18 −18 51.3% −9.5 32.6% 1063
95.1%
Movei 00.8.438 2703 +25 −26 44.2% +38.9 29.4% 540

Other lists at longer time control like the infinite loop also support the idea that 10 10 10 is better than the default
and progress 10 10 10 is leading WBEC when it won chesstiger2007.1 2.5:1.5 in the last match(I asked Leo to use 10 10 10 and I assume that he does it).

progress is one of the possible uci parameters of movei so it is easy to change progress if you use Movei as a uci engine.

Uri
Post Reply