The latest CCRL update (13th April 2024)

[Graham Banks]

The purpose of our rating lists is provide engine authors and enthusiasts with a general comparison of engine strength. We also provide other data that could be of interest.
For various reasons, there may be some engines that do not appear on our lists, therefore it is useful to look at other rating lists as well as ours.
The inclusion or exclusion of engines in our lists should not be taken as our group making a statement about their legality or status.
We test chess engines for our own enjoyment and receive no payment for doing so.

The latest CCRL Rating Lists and Statistics are available for viewing from the following links:
www.computerchess.org.uk/ccrl/4040 (40/15 and 15m+10s)
www.computerchess.org.uk/ccrl/404 (BLITZ 2m+1s)
www.computerchess.org.uk/ccrl/402.archive/ (BLITZ 40/2 Archive)
www.computerchess.org.uk/ccrl/404FRC (FRC 40/2)

Please note that the three lists are often updated separately to each other. The FRC list is only updated when a new engine or engine version is being/has been tested.

The links to the various rating lists can be found just beneath the Index page.



Our 40 moves in 15 minutes repeating and 2 minutes with 1 second increments are both adjusted to the Intel i7 4770k (3.5GHz) using the Stockfish 10 64-bit benchmark.
All of our current testing uses AVX2 compiles where available.

Be aware that in the early stages of testing, an engine's rating can often fluctuate a lot.
It is strongly advised to look at the many other rating lists available in order to get a more accurate overall picture of an engine's rating relative to others.

The LOS (likelihood of superiority) stats to the right hand side of each rating list tell you the likelihood in percentage terms of each engine being superior to the engine directly below them.

All games are available for download by engine or ECO code. The total games database in its entirety is always available.
The current ELO ratings are saved in all game databases for those engines that have 200 games or more.

Clicking on an engine name will give details as to opponents played plus homepage links where applicable.

Custom lists of engines can be selected for comparison.

An openings report page lists the number of games played by ECO codes with draw percentage and White win percentage. Clicking on a column heading will sort the list by that column.

The inclusion or exclusion of engines in our lists should not be taken as our group making a statement about their legality or status. Where there is controversy, it is usually mentioned in the engine notes.

Thanks to:
- all of our currently active testers (Graham, Gabor, Sergio, Basti, Shaun, Jorge).
- Ray for the weekly updates.
- Jay Honnold for hosting Graham's live broadcasts.
- all engine authors.

[ChessKnight]

Graham Banks

The latest CCRL update

I am highly appreciative and thankful to your efforts for regular updates.

thanks.

[Graham Banks]

Graham Banks

The latest CCRL update

I am highly appreciative and thankful to your efforts for regular updates.

thanks.

Thanks. It's a team effort to do all that we do. Producing an update report every two weeks is just one little way I can help the team.

Regards, Graham.

[Mindbreaker]

It would be nice to see a rating for Deep Fritz 11 in the 40/4 category. I love all the stats, results matrices, links to engine homepages, and version comparisons. Great job. I have enjoyed this site for years.

[Graham Banks]

It would be nice to see a rating for Deep Fritz 11 in the 40/4 category. I love all the stats, results matrices, links to engine homepages, and version comparisons. Great job. I have enjoyed this site for years.

Thanks for your kind comments.
I'm sure that the 40/4 testers will read your request. Hopefully one of them might have purchased Deep Fritz 11, but I can't guarantee it. We'll see what we can do.

[Davemt01]

Please excuse me for asking a very basic question. Perhaps it has already been answered somewhere on these forums, but if so, I have been unable to find the answer.

The question is regarding the CPU mentioned in the listings. The abbreviations used for the listing include "single-CPU", "2CPU" and "4CPU", and it's separately mentioned that the CPU in question is the Athlon 64 X2 4600+.

However, the Athlon 64 X2 4600+ is a dual core CPU. Am I correct in assuming, for example, that the top listing, which is:

Rybka 3 64-bit 4CPU

actually means:

Chess engine: Rybka, version 3, 64 bit
OS: some 64 bit OS, probably Windows Vista?
CPU Type: Athlon 64 X2 4600+
CPU Number: 4 separate CPUs, probably on some server-type motherboard? With a total of 8 cores?

By the same token, am I correct in assuming that "2CPU" means two separate Athlon 64 X2 4600+ CPUs (4 cores total)?

I would really appreciate it if I could see a description of the actual hardware, including CPUs, motherboard, amount and type of memory in particular, as well as the name and version of the operating system. Is this information available somewhere?

Thanks.

Dave M.

[Kirill Kryukov]

Please excuse me for asking a very basic question. Perhaps it has already been answered somewhere on these forums, but if so, I have been unable to find the answer.

The question is regarding the CPU mentioned in the listings. The abbreviations used for the listing include "single-CPU", "2CPU" and "4CPU", and it's separately mentioned that the CPU in question is the Athlon 64 X2 4600+.

However, the Athlon 64 X2 4600+ is a dual core CPU. Am I correct in assuming, for example, that the top listing, which is:

Rybka 3 64-bit 4CPU

actually means:

Chess engine: Rybka, version 3, 64 bit
OS: some 64 bit OS, probably Windows Vista?
CPU Type: Athlon 64 X2 4600+
CPU Number: 4 separate CPUs, probably on some server-type motherboard? With a total of 8 cores?

By the same token, am I correct in assuming that "2CPU" means two separate Athlon 64 X2 4600+ CPUs (4 cores total)?

Hi Dave,

"single-CPU", "2CPU" and "4CPU" in fact mean "single-core", "2-core" and "4-core". They are called CPUs because this list started before multi-core CPUs became common. Perhaps it is time to change the wording.

When Athlon 64 X2 4600+ is referred as a reference for 1-CPU testing, it should be taken as "one of the cores of Athlon 64 X2 4600+.

I would really appreciate it if I could see a description of the actual hardware, including CPUs, motherboard, amount and type of memory in particular, as well as the name and version of the operating system. Is this information available somewhere?

Thanks.

Dave M.

We use a broad variety of CPUs, motherboards, and other parts. It is simply not practical to try unifying the hardware. We are not that rich to buy identical machines just for chess, so we use our ordinary machines, whatever we have, for testing.

So, instead we use calibrated time control. We benchmark all of our machines, and adjust the time control so that it becomes close to what it would be on the reference machine (Athlon 64 X2 4600+).

Best,
Kirill

[Davemt01]

Hi Kirill

Thanks very much for the reply. Could you let me know what you use to benchmark the machines, and if this benchmarking software is available to me for testing?

Thanks.

Dave M.

[Kirill Kryukov]

Hi Dave,

You're welcome! For historical reasons, we use Crafty 19.17 BH (Brian Hoffman's compile) as a benchmark. You can get a copy here.

[Davemt01]

I downloaded the benchmark and was able to successfully run it on 3 different machines.

I am interested in chess engines (in a purely amateur/hobbyist way; I have no deep understanding of them), and would like at some point to help in the testing you're using to rate and compare engines. I can see that I still have some reading to do before I fully understand your methodology. I will continue to follow these forums.

Thanks again for the link to Crafty, and for answering my questions.

Dave M.

[Kirill Kryukov]

Hi again, Dave. In this thread you can find discussion about CCRL testing conditions, including details on benchmarking and time control calibration. Good luck!

[guillef]

For the CCRL Team,

I really enjoy using all the information you publish. CCRL is currently the primary source of information to know the progress of each new version of Chronos.

Thank you very much for your efforts in keeping the site updated.

Guillef.

[Graham Banks]

For the CCRL Team,

I really enjoy using all the information you publish. CCRL is currently the primary source of information to know the progress of each new version of Chronos.

Thank you very much for your efforts in keeping the site updated.

Guillef.

Hi Guillef,

Chronos 1.9.6 will pass the 200 game mark in the next update.
Thanks for your kind words.

Cheers,
Graham.

[Mindbreaker]

I am getting a very different result between Rybka 3 Human vs StockFish 1.7.1 than vs Rybka 3. Rybka 3 Human appears to be a more difficult opponent for StockFish 1.7.1 It would be nice if someone official would run a match between these two.

[Shaun]

I am getting a very different result between Rybka 3 Human vs StockFish 1.7.1 than vs Rybka 3. Rybka 3 Human appears to be a more difficult opponent for StockFish 1.7.1 It would be nice if someone official would run a match between these two.

I am running a head to head R3 v SF1.71 I will also run R3Human and R3Dynamic - Stockfish and R3 a close so lots of games are required for a conclusion one way or the other

Shaun

[Philippe]

Hi all,

It is just for a confirmation please... My processor is an Intel Core i7 940 (2.93 Ghz), I have 12 Gb RAM, I have Windows 7 64 bits. I think it is a quad core... So could you really confirm that it is better for me to buy the Deep Rybka 4?

Multi processor, multi core but... monolithic processor, so it sounds not so easy for me to be sure that the Deep Rybka 4 will be better for me than the "not Deep"

Thank you very much for you answer!

[Quitch]

I'm confused by the pure list in relation to this update. Why, if Deep Fritz 11 has a higher elo than Deep Fritz 10.1, is the latter used on the pure listing?

[Kirill Kryukov]

Hi all,

It is just for a confirmation please... My processor is an Intel Core i7 940 (2.93 Ghz), I have 12 Gb RAM, I have Windows 7 64 bits. I think it is a quad core... So could you really confirm that it is better for me to buy the Deep Rybka 4?

Multi processor, multi core but... monolithic processor, so it sounds not so easy for me to be sure that the Deep Rybka 4 will be better for me than the "not Deep"

Thank you very much for you answer!

Hi Philippe,

Deep Rybka would certainly be better than single-core Rybka on a Core i7 940, but it's more expensive too. Whether it is better for you is something only you can tell, especially with so many strong freeware engines around.

Best,
Kirill

[Kirill Kryukov]

I'm confused by the pure list in relation to this update. Why, if Deep Fritz 11 has a higher elo than Deep Fritz 10.1, is the latter used on the pure listing?

Hi Quitch,

I guess that you are talking about "Deep Fritz 11 4CPU" vs "Deep Fritz 10.1 4CPU". Most probably "Deep Fritz 11 4CPU" did not play enough games with the other participants of the pure list. At least 200 games are required.

Best,
Kirill

[Classen]

This might be the wrong place to ask, but how can I get the testers to include Ghost 2.0.1 in the 40/4 testing? It has shown some nice results in 40/40, so I'm curious to see how it performs under shorter time limits.

[Adam Hair]

This might be the wrong place to ask, but how can I get the testers to include Ghost 2.0.1 in the 40/4 testing? It has shown some nice results in 40/40, so I'm curious to see how it performs under shorter time limits.

It can happen fairly soon. I am always eager to include an engine when an author makes a request.

[Classen]

It can happen fairly soon. I am always eager to include an engine when an author makes a request.[/quote]

Ok, just to make sure: you can understand that as an official request.

I didn't see any postings related to 40/4 test matches yet, I'll be looking forward to that...

[Adam Hair]

It can happen fairly soon. I am always eager to include an engine when an author makes a request.

Ok, just to make sure: you can understand that as an official request. [/quote]

I didn't see any postings related to 40/4 test matches yet, I'll be looking forward to that...

I will be starting the gauntlet this week.

[Mindbreaker]

Maybe I am fussing a bit, but I like the live list but the links on the top direct instead to the normal list. It would make more sense that if one was already on a live list say the 40/4 list that when the 40/40 or Fischer Random list button is pushed that the live version shows up. Currently, I have to go back to the CCRL discussion board to find the live version. Alternatively, you could have both sets of buttons on each type of page.

I consider your list the best out there by quite a lot. I really appreciate your efforts and quality of presentation.

I also have another suggestion that is not quite as simple to implement. I think it would be outstanding if there was a graphical view of how rating changes for an engine relative to the others with more time / more threads. They could be combined as different ways to quantify the same thing: "Computation." There are losses in computing efficiency associated with more cores but the data points can be adjusted to appear be in appropriate locations. One only has to take the average kn/s from several games by an engine and average it and compare that to the average kn/s in games with different time controls or # of cores. On one axis would be Computation, on the other would be Relative Elo. It would mean that all the lists should have something like the same Elo average for the top 15 competitors that are on both lists. That really is the only change that would be needed to the way the current lists are made.

The beauty of such a graph is that it can project ratings for times that are inconvenient to test. Each engine fully tested would have a data point for 1-core/40/4, 2-core/40/4, 4-core/40/4, 6-core/40/4, 1-core/40/40, 2-core/40/40, 4-core/40/40, and 6-core/40/40. The points would lie either correctly spaced based on computation or on a log scale and the appropriate locations. "Computation" is average (kn/s)*(seconds/40 moves).

To make calculations easier, you initially can take averages of "Computation" among the leading engines at the various time controls and core activity levels, and just use those placements on the graph to put any future engine that is also of the same time control and thread level. That way it should be easy to generate the graph. I would also mean that you would not have to require testers to provide the average kn/s for each engine each game.

It would also be convenient if when one rolls over a particular line the name of the corresponding engine is shown.

Another nice aspect of the graph is that it can suggest where there might be statistical anomalies as a typical pattern should emerge for engines.

[Kirill Kryukov]

Thanks for suggestions. I don't have time at the moment to implement any of this, but will look into this in the future.