"Detailed" testing settings?

[BFG]

CCRL does a pretty good job of documenting the testing settings they use, and a great job of standardizing everything. I'm trying to exactly match the CCRL testing protocol, and wondered if someone (probably Graham ) could review my settings, below, to see if I messed up anything. I'm especially concerned that some of the Book/TB settings may be wrong!


GUI: Arena 3.5

Time/Move control: 40/32, 40/10, or 40/3 (using Crafty's benchmark of 39s), repeating, ponder OFF, autoflag enabled, always check legality and 50 Moves rule, blunder check enabled (200 centipawns in 1 move)

Tourney Options: Adjudicate at -1300 centipawns or 250 moves or by Tablebases, autorotate sides, Restart Arena after 20 games

Engine controls (general): Strength 100%, normal CPU priority, Starting ELO 2200, Restart after every game, Use Arena Main Books with all engines (checkbox under Books tab of Engine details), no Arena enginebook -specified (on Books tab of Engine details), all Winboard features autodetermined

Engine controls (UCI): Resign disabled, UCI Filter in Tournament Mode and Analysis Mode enabled, Always Use UCI Filter disabled, hashtable 256MB (I'm on a dual core), Nalimov EGTB not set, EGTB cache 32MB, "Common own book setting" enabled, "Engines use their own book" disabled

Engine controls (XBoard): "Extended level command for WB engines" enabled, Max memory 256MB, Gaviota Tablebase set, Nalimov and Scorpio tablebase not set

Books: EGTB enabled (Gaviota 3-4-5 piece, compression 4), Use Arena Main Book enabled, Use Arena Engine Books disabled, Use Transpositions disabled, Arena Main Book "Perfect 2010", Use book up to halfmove 24, default minimal conditions for move selection

[Graham Banks]

I haven't used Arena for a long time, but that all looks okay to me.

Although initially we started with three lists (40/40, 40/10 and 40/3), we ditched 40/10 as it was just too difficult to maintain up to date lists for three different time controls.

[BFG]

Thanks for the response! I'm still not completely certain I have the books and EGTBs set up properly but will check the Arena documentation to be sure. Out of curiosity, what GUI do you typically use, and what makes it a better choice for you than Arena?

[Ray]

Blunder check should be disabled.

[Graham Banks]

Thanks for the response! I'm still not completely certain I have the books and EGTBs set up properly but will check the Arena documentation to be sure. Out of curiosity, what GUI do you typically use, and what makes it a better choice for you than Arena?

I use ChessGUI predominantly, but others used by testers include Arena, Winboard and Fritz.

[BFG]

I'll have to take a look at those other 3, thanks.

Ray, thanks for your input; let me know if you spot any other incorrect settings.

[Ray]

I have barely used Arena, and not V3.5 so it is difficult to comment. However, a couple of observations:

- I don't know why you wouldn't enable the Namimov path under UCI options, and

- you mention Naimiovs under Book options, I have never seen that either. What have tablebases got to do with opening books ?

[BFG]

I think I listed Namilov under Book options by mistake - probably got my books and EGTBs mixed up

The reason I don't have a Namilov or Scorpio path set is because I don't have those EGTBs. I only have the Gaviota 3-4-5s. I can certainly try to download/source the additional tables but I wasn't sure it was necessary or would make a significant difference. And I don't have enough disc space (1.2TB!) for the new 6s.

[Graham Banks]

I think I listed Namilov under Book options by mistake - probably got my books and EGTBs mixed up

The reason I don't have a Namilov or Scorpio path set is because I don't have those EGTBs. I only have the Gaviota 3-4-5s. I can certainly try to download/source the additional tables but I wasn't sure it was necessary or would make a significant difference. And I don't have enough disc space (1.2TB!) for the new 6s.

If you download the latest 4 men egbb files, make sure that any engines that use them have their parameters set to use 4 and not 5. Otherwise the engines concerned will crash.
Using the 5 piece Syzygy tablebases is fine, you don't need 6 piece.

[BFG]

I'm still learning about EGTBs, so forgive me if this is a silly question. Is there any practical use to downloading Namilov, Scorpio, etc. EGTBs when I already have a full set of Gaviotas that are accessible by all engines?

[Graham Banks]

I'm still learning about EGTBs, so forgive me if this is a silly question. Is there any practical use to downloading Namilov, Scorpio, etc. EGTBs when I already have a full set of Gaviotas that are accessible by all engines?

Not all engines use Gaviota tablebases. Some use the Nalimov tablebases, some now like Syzygy tablebases and others like egbbs.

[BFG]

Ah! Now I understand. I thought any UCI engine could use any type of EGTB. Obviously, I was wrong. Thanks.