Stockfish +80 ELO Patch

[XulChris]

Hi:

I just noticed a patch for Stockfish222 which claims to increase ELO by as much as 80pts.[1]

The problem is that Stockfish does not take into account how many pawns are on the 7th rank when considering to move rook or queen to 7th rank. This patch fixes that issue.

I think this is really interesting stuff, so I wanted to share.

[1] https://github.com/mcostalba/Stockfish/pull/22

[Graham Banks]

Sounds exciting, but so far we've stuck to testing official releases of Stockfish.
Hopefully, the Stockfish team will be able to implement some of that strength increase in their next release.

[XulChris]

Yea, Marco's tests indicate no ELO improvement, so it could just be a big disappointment.

Speaking of official Stockfish builds, what version is this?

http://www.computerchess.org.uk/ccrl/40 ... _09_64-bit

I could not find that anywhere on the net...

[Ray]

Speaking of official Stockfish builds, what version is this?

http://www.computerchess.org.uk/ccrl/40 ... _09_64-bit

I could not find that anywhere on the net...

http://talkchess.com/forum/viewtopic.ph ... kfish+ve09

[XulChris]

Speaking of official Stockfish builds, what version is this?

http://www.computerchess.org.uk/ccrl/40 ... _09_64-bit

I could not find that anywhere on the net...

http://talkchess.com/forum/viewtopic.ph ... kfish+ve09

This is not an official release.

[Ray]

This is not an official release.

Did I say that it was ?

You said you couldn't find it anywhere on the internet, so I simply told you where to find it.

[Graham Banks]

Speaking of official Stockfish builds, what version is this?

http://www.computerchess.org.uk/ccrl/40 ... _09_64-bit

I could not find that anywhere on the net...

http://talkchess.com/forum/viewtopic.ph ... kfish+ve09

This is not an official release.

My statement was incorrect in this case.

[XulChris]

It looks like Gary Linscott just optimized the patch by Ryan Taker to use bitboards to count the pawns on a rank.[1]

Who knows what kind of ELO improvement this means now.

[1] https://github.com/RyanTaker/Project-Ro ... cf69853e3c

[XulChris]

I noticed another patch recently checked in by Marco which claims a 20% performance increase in perft.[1]

I'm not sure what perft is, I guess that is a move generator performance calculator or something, but 20% performance increase sounds pretty significant!

I also read that Houdini Chess 3.0 will be coming out soon, and the author is claiming a 50 elo increase over Houdini 2.0, so we might see some next generation engines coming out over the next few months, let's cross our fingers.

[1] https://github.com/mcostalba/Stockfish/ ... f9b0a58941

[XulChris]

I guess it's not 80 Elo as originally claimed by the author, but it is giving a 99.99% LOS over the older version in initial testing.[1]

Wins: 3390 Losses: 2972 Draws: 11323
LOS: 99.999992%
ELO: 8.213465 +- 99%: 6.746506 95%: 5.124415
Win%: 51.181792 +- 99%: 0.969791 95%: 0.736740

[1] https://github.com/mcostalba/Stockfish/pull/23

[Graham Banks]

I wonder whether Stockfish 2.3 has incorporated some of this?

[XulChris]

I wonder whether Stockfish 2.3 has incorporated some of this?

Not yet, the patch has just got to the point where the maintainers are willing to take a look at it and test it out. If the tests hold up, however, I'm sure it will make it in.

I wouldn't be surprised to see Stockfish 2.3 challenge Houdini 2.0, only time will tell. If Stockfish 2.3 does topple Houdini 2.0, I'm sure we will see a Houdini 3.0 to challenge that. The Houdini author has mentioned that a new release may come out soon.

There was also a bug reported by Ray Banks which recently got fixed.[1]

[1] https://github.com/mcostalba/Stockfish/ ... adfd84e8d6

[XulChris]

Looks like the patch made it into 2.3.1 this weekend.[1]

However, it looks like it could possibly be tweaked to provide additional Elo in the future.[2]

[1] http://blog.stockfishchess.org/
[2] https://github.com/mcostalba/Stockfish/ ... nt-1890917

[Ray]

Well, 2.3.1 chess960 testing is underway, so we will see how that turns out

Standard chess 404 has had a few 2.3 games
http://www.computerchess.org.uk/ccrl/40 ... +opponents

And it shows no real improvement over 2.2.2 so far, but the erroe margins may be masking any improvement.

I'm wondering, at chess960, if Stockfish VE_09 64-bit will in fact remain the strongest Stockfish version

[XulChris]

Well, 2.3.1 chess960 testing is underway, so we will see how that turns out

Standard chess 404 has had a few 2.3 games
http://www.computerchess.org.uk/ccrl/40 ... +opponents

And it shows no real improvement over 2.2.2 so far, but the erroe margins may be masking any improvement.

I'm wondering, at chess960, if Stockfish VE_09 64-bit will in fact remain the strongest Stockfish version

I was wondering, if an engine crashes like with that Stockfish FRC bug, does that count as a loss?

Also, if VE_09 is still stronger than 2.3, that would mean a regression took place in the past couple of months because AFAIK, VE_09 is simply a compiled snapshot of the git repository.

I guess we'll just have to wait and see the results, it's possible a regression could have been introduced between VE_09 and 2.3.

EDIT: I was actually expecting 20-25 Elo improvement with 2.3 because of the results of VE_09, I was surprised when Marco announce no Elo improvement with 2.3. How is VE_09 getting the ELO improvement then?

[Ray]

I'm wondering, at chess960, if Stockfish VE_09 64-bit will in fact remain the strongest Stockfish version

It would not get tested at all with a critical bug

[Ray]

Testing completed

[XulChris]

Yeah, I don't know if you saw my edit in my previous post. I was actually expecting 20-25 Elo from Stockfish 2.3.0 because of the VE_09 performance. I'm not sure where the VE_09 version is getting the extra Elo from.

Strange, ...and disappointing.

[Ray]

The Stockfish VE_09 64-bit is not a particularly fast compile either - Intel C++ non-PGO compile. So a JA GCC compile of it would likely be faster. Might give it another couple of Elo... But Jim only compiles what the team give him.

I may yet remove Stockfish VE_09 64-bit now that the "official" Stockfish 2.3.1 is out. The statistical error bars are such that we can't tell if the small Elo differences are real or not.