New engine: Tcheran

[jgilchrist]

Hello! It's my first time posting here - I've been working on a chess engine recently. It's recently reached a level I'm generally happy with and I'm curious to see where it would land on the CCRL rankings (both 40/15 and 2+1).

The source code is available here: https://github.com/jgilchrist/chess-engine. It's written in Rust and is buildable with a simple 'cargo build --release' from the root. If it would be better to provide binaries please let me know and I can get some uploaded.

The bot is currently registered on Lichess and is rated ~2000 ELO across Rapid, Blitz and Bullet: https://lichess.org/@/jpg-bot

Thanks,
Jonathan

[Graham Banks]

An exe would be helpful.
We all use AVX2.

[Gabor Szots]

I suggest releasing various versions under 'Releases' for easier tracking and distinction.

[jgilchrist]

I suggest releasing various versions under 'Releases' for easier tracking and distinction.

Thanks Gabor, you're right, I've been holding off on releasing v1.0 but now feels like the right time!

An exe would be helpful.
We all use AVX2.

Thanks Graham, I have uploaded a Windows binary under the v1.0 release (https://github.com/jgilchrist/chess-eng ... tag/v1.0.0), compiled with AVX2. Please let me know if you have any issues running it.

[Gabor Szots]

Hello Jonathan,

Your exe seems OK. However, when I try to compile the source using the latest Rust I get this error message:

Compiling engine v1.0.0 (E:\Sakk\compiles\Tcheran\chess-engine-1.0.0\engine)
error: git describe exited with status 128: fatal: not a git repository (or any parent up to mount point /)
--> engine\src/lib.rs:16:5
|
16 | git_describe!("--always", "--dirty=-modified")
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: this error originates in the macro `$crate::git_version` which comes from the expansion of the macro `git_describe` (in Nightly builds, run with -Z macro-backtrace for more info)

error: could not compile `engine` (lib) due to previous error

As I am not a programmer I can't resolve this issue.

[Ray]

As I am not a programmer I can't resolve this issue.

Well you can use his exe in that case.

[Gabor Szots]

As I am not a programmer I can't resolve this issue.

Well you can use his exe in that case.

Thanks for the hint Ray. It would have never occurred to me.
As I wrote, his exe is OK. But still. And it is verification to see that the provided exe and the compiled source gives the same output.

[Ray]

Thanks for the hint Ray. It would have never occurred to me.

[jgilchrist]

Hello Jonathan,

Your exe seems OK. However, when I try to compile the source using the latest Rust I get this error message:

...

As I am not a programmer I can't resolve this issue.

Appreciate you trying this! Am I right in thinking that you got the source code by downloading the zip from the release? I'm using the technique causing that error to embed version information into the .exe at build time - for example my current local build reports:

Code: Select all

id name Tcheran (v1.0-3-g123ae92)

Which tells me it was built from 3 commits after v1.0, from the exact commit "123ae92". It's been very useful when running playoffs against different versions of the engine. Unfortunately, if you try building with just the source code, rather than a fully clone of the repository, it fails because it can't find that information.

I've tweaked the approach slightly and pushed a new v1.0 tag (and deleted the old v1.0.0 one). Now, if it's unable to find information from the repository, the version will be reported as 'unknown':

Code: Select all

id name Tcheran (unknown)

I've also re-uploaded new binaries for the new v1.0 tag, which should be identical aside from the removal of the trailing .0 in the version.

Finally, to avoid any confusion, I've removed the in-repository version in Cargo.toml which was not being used. Unfortunately, this will cause the project to stop building with the current stable Rust version (1.74.0) as support for removing this property is only available in the next Rust version, 1.75.0, which is currently in beta and will be released December 28th.

Until then, it is still possible to build the project by adding just a single line to both chess/Cargo.toml and engine/Cargo.toml specifying a version, e.g.

Code: Select all

name = "chess"
version = "0.0.0"

[Gabor Szots]

Yes, I used the zip from the release.

I may have had succes by cloning the repository. But, IMO, the zip in the release should build to the same exe, to avoid confusion. Also, what if I want to revert to a previous version? Cloning the repository would not help then. I admit, that is quite a remote possibility.
I have currently very limited time to experiment but if I get a chance I will try your suggestions.

BTW, I am just running a tournament using the exe (1.0.0) you provided. With a bit of luck it may appear on this week's blitz list.

[jgilchrist]

But, IMO, the zip in the release should build to the same exe, to avoid confusion.

I agree. That said, I can't think of a good way to have this work but continue to also have the extra version information embedded when building from the repository. I'm a bit loth to give that up as it's been so useful.

Also, what if I want to revert to a previous version?

If you do have the repository cloned, you can check out any particular version by checking out the tag. The way you do that would depend on which tool you used to clone the repository - for me, if I had released a hypothetical v2.0 and then wanted to get back to v1.0 to build it I would:

Code: Select all

git checkout v1.0

And could then build the same v1.0 binary as before. I understand this may be a significant departure from your workflow though - I'll continue to try and find a better solution here.

I have currently very limited time to experiment but if I get a chance I will try your suggestions.

I'm not sure it's worth the trouble given things will start to work as soon as Rust 1.75.0 is released in a couple of weeks

BTW, I am just running a tournament using the exe (1.0.0) you provided. With a bit of luck it may appear on this week's blitz list.

That's wonderful news! Thank you.

[jgilchrist]

I think I've come up with a better solution, which will work with the current stable Rust version and no manual file editing.

I have re-released v1.0 (https://github.com/jgilchrist/chess-eng ... s/tag/v1.0) - downloading the zip + building should 'just work'.

[Gabor Szots]

I think I've come up with a better solution, which will work with the current stable Rust version and no manual file editing.

I have re-released v1.0 (https://github.com/jgilchrist/chess-eng ... s/tag/v1.0) - downloading the zip + building should 'just work'.

Indeed it does. Thank you. Is this totally equivalent with 1.0.0? Then I'm going to replace the name in my tournament to 1.0 to avoid confusion.

PS. Rust 1.74.1 and 1.76-nightly have been just released.

[Gabor Szots]

What does strategy do, does it make sense to change it and how to do it?

Is there a way to disable logging? That file has already grown to 25 MB.

[jgilchrist]

Indeed it does. Thank you. Is this totally equivalent with 1.0.0?

Yes, totally equivalent and I'll be following a vX.Y versioning scheme from now on so replacing the name in the tournament would be great - thank you.

Rust 1.74.1 and 1.76-nightly have been just released.

So they have - I've checked the release notes for 1.74.1 and don't see anything that would make it worth recompiling (but I'll use it to compile the next published version, unless 1.75 has been released by then). 1.76-nightly will be released every day, but I'm not using the nightly version to compile (although it would be interesting to try a playoff between v1.0 built with 1.74.1 and with the latest 1.76-nightly in case any code generation improvements result in an ELO increase).

What does strategy do, does it make sense to change it and how to do it?

That was inspired by https://www.youtube.com/watch?v=DpXy041BIlA - I wanted to leave the door open to trying some 'just for fun' alternative strategies in the vein of that video, but haven't explored that much yet. The default strategy is the 'serious' strategy so should definitely be the one used for tournaments.

Is there a way to disable logging? That file has already grown to 25 MB.

Ah, sorry, that's not ideal - unfortunately there is not (yet). I suppose it should also be off by default.

I've made that change in https://github.com/jgilchrist/chess-eng ... aa1a0fa446, so from the next release there'll be no default logging. For this tournament, the file can be deleted at any point without affecting the engine - or, if it'll be too much trouble, I can just submit a v1.1 with the logging switched off for participation in a future tournament.

[Gabor Szots]

For this tournament, the file can be deleted at any point without affecting the engine - or, if it'll be too much trouble, I can just submit a v1.1 with the logging switched off for participation in a future tournament.

No trouble at all, just leave it as it is then.

[jgilchrist]

I have just seen Tcheran pop up in this week's Blitz rating list - very exciting. Thanks Gabor!

A couple of questions if you have time -

When releasing a new version of the engine, I've seen some authors post a new forum topic, and others post in their original announcement topic - is there a preferred approach?

I downloaded the games that were played to get Tcheran's initial rating. The main reason (besides curiosity) was to check for any losses on time. However, the PGNs available through the site do not list - as far as I can tell - the termination reason for the game. Some of the games ended early, but I assume that adjudication is also used for the testing. Is there a way to differentiate between a loss on time vs. a loss by adjudication? Additionally, is there any info available about the adjudication settings that are used for testing? I couldn't see anything about it on the 'about' page so I imagine it is probably left up to individual testers?

[Gabor Szots]

I have just seen Tcheran pop up in this week's Blitz rating list - very exciting. Thanks Gabor!

A couple of questions if you have time -

When releasing a new version of the engine, I've seen some authors post a new forum topic, and others post in their original announcement topic - is there a preferred approach?

I downloaded the games that were played to get Tcheran's initial rating. The main reason (besides curiosity) was to check for any losses on time. However, the PGNs available through the site do not list - as far as I can tell - the termination reason for the game. Some of the games ended early, but I assume that adjudication is also used for the testing. Is there a way to differentiate between a loss on time vs. a loss by adjudication? Additionally, is there any info available about the adjudication settings that are used for testing? I couldn't see anything about it on the 'about' page so I imagine it is probably left up to individual testers?

Hi Jonathan,

I don't think there is a preferred way of announcing a new version, it's the author's choice. Some authors do not announce them at all, he who discovers there is a new versions may or may not announce it at TalkChess.
We at CCRL may announce engines for each other (not always). Also, I follow engines that I know of at their github page.

You did not find games lost on time because the processing scripts remove them (at least that is my understanding). My policy is to check my PGN's for time forfeits, then I edit the PGN so that if the side losing on time would have lost anyway I remove the "forfeits on time" or similar parts so that the games are kept and an engine cannot escape a loss by refusing to move.
If you want to have time loss information I think the only way to do it is to contact the testers themselves.

Adjudication by a GUI is usually there in a comment at the end of the game. At least for games played under Arena. Adjudication rules are, as you have guessed, up to individual testers. I myself switch off adjudication for very weak engines and I loosen the rules as engine strength increases.

[Graham Banks]

My recommendation would be to announce any new versions both here and in the Talkchess forum.

[jgilchrist]

Thanks Graham & Gabor.

I have recently published v1.1 of Tcheran which includes a variety of move generation optimisations, which I wanted to get into a separate release before v2.0.

https://github.com/jgilchrist/chess-eng ... s/tag/v1.1
https://github.com/jgilchrist/chess-eng ... ELOG.md#11

v1.1 is slightly stronger than v1.0 - the upcoming v2.0 includes more search pruning techniques and should be a much larger improvement.

Code: Select all

ELO   | 18.5 +/- 9.4
CONF  | 8+0.08 Threads=1 Hash=128MB
GAMES | N: 4096 W: 1693 L: 1475 D: 928

ELO   | 16.3 +/- 18.4
CONF  | 40+0.4 Threads=1 Hash=128MB
GAMES | N: 1024 W: 406 L: 358 D: 260

[jgilchrist]

Version 2.0 of Tcheran was released yesterday with a variety of standard search pruning techniques:

In short time controls, there's a ~244 Elo increase in self-play.

Code: Select all

ELO   | 244.1 +/- 11.4
CONF  | 8+0.08 Threads=1 Hash=128MB
GAMES | N: 4096 W: 2942 L: 460 D: 694

In longer time controls, a slightly bigger ~275 Elo increase:

Code: Select all

ELO   | 275.5 +/- 24.3
CONF  | 40+0.4 Threads=1 Hash=128MB
GAMES | N: 1024 W: 772 L: 96 D: 156

[bastiball]

Version 2.0 of Tcheran was released yesterday with a variety of standard search pruning techniques:

In short time controls, there's a ~244 Elo increase in self-play.

Code: Select all

ELO   | 244.1 +/- 11.4
CONF  | 8+0.08 Threads=1 Hash=128MB
GAMES | N: 4096 W: 2942 L: 460 D: 694

In longer time controls, a slightly bigger ~275 Elo increase:

Code: Select all

ELO   | 275.5 +/- 24.3
CONF  | 40+0.4 Threads=1 Hash=128MB
GAMES | N: 1024 W: 772 L: 96 D: 156

Wow, huge elo gain

[Gabor Szots]

Version 2.0 of Tcheran was released yesterday with a variety of standard search pruning techniques:

Noted.

[jgilchrist]

Wow, huge elo gain

A lot of low hanging fruit there since the search implementation for v1.x was almost entirely vanilla alpha-beta and its rating was mostly due a lot of time spent profiling and optimising move generation and the simple search implementation. Plenty of easy-to-implement pruning techniques were still on the table and can now take advantage of the optimisations to search speed from 1.x. And still plenty to come!

There's also a lot of low hanging fruit in the evaluation function which is what I intend to focus on for v3.0. Currently it's just material balance and piece square tables so it often loses games due to, for example, a lack of understanding of the perils of passed pawns.

[jgilchrist]

Hello all,

v2.1 of Tcheran was released today, containing two fixes resulting in Elo changes which are as follows:

STC:

Code: Select all

ELO   | 109.2 +/- 9.4
CONF  | 8+0.08 Threads=1 Hash=128MB
GAMES | N: 4096 W: 2142 L: 895 D: 1059

LTC:

Code: Select all

ELO   | 121.4 +/- 18.5
CONF  | 40+0.4 Threads=1 Hash=128MB
GAMES | N: 1024 W: 534 L: 190 D: 300

https://github.com/jgilchrist/tcheran/releases/tag/v2.1
https://github.com/jgilchrist/tcheran/b ... ELOG.md#21