Testing

[Dark_wizzie]

Hi,

The website mentions that the engines are tested with AMD64 X2 4600+. But tests are done by multiple people. So how do we adjust the results or testing method? As far as I know, you double the processing power, it's almost the same as doubling the time control. And if we look at Houdini vs Stockfish at 40/40 vs 40/4, at 40/4 Houdini dominates. So from what I can tell, Houdini is the king of low time control on weaker hardware but Stockfish catches up once both increase. So depending on your hardware and time control, either Houdini or beta Stockfish could be the strongest. So what about the 4CPU version of the test? Houdini 4CPU: Is this run on a custom mobo with four CPUs, or are we talking about a quad core? I think AMD64 X2 4600+ is a dual core CPU? If so, then it's two AMD64 X2 4600+ cpus?

I think in the future the CPU standard might have to change. The kilonodes from an AMD64 X2 4600+ is going to be very different from a 4670k overclocked or a 12 core Xeon. More and more years will pass until nobody really knows what kind of processing power an AMD64 X2 4600+ actually has.

When testing engines, why do sites always use 40/40, 40/4 or so, instead of something like 15 minutes + 5 second increment? Is it better to use 40/40 for testing?

[Graham Banks]

The testing conditions thread near the top explains how our computers are benchmarked.

Repeating time controls were chosen because some engines at the time didn't run with incremental time controls.

Time controls are a matter of taste.
Personally, I prefer repeating time controls due to the better consistency of quality play throughout the game.