6 Men TB Generator

[Martin Kreuzer]

Hallo,
this is my first post here, so please bear with me
if the answer to my question is obvious to you.
Does anybody have the source code of the version of
tbgen to create 6 men tablebases?

I have a powerful 64 bit compute available and would
like to try it.

Greetings to all, Martin Kreuzer

[jshriver]

Hallo,
this is my first post here, so please bear with me
if the answer to my question is obvious to you.
Does anybody have the source code of the version of
tbgen to create 6 men tablebases?

I have a powerful 64 bit compute available and would
like to try it.

Greetings to all, Martin Kreuzer

I've been wanting the same thing for a while now. It seems the 6man Nalimov egtb isn't available. The tbgen program found on various sites is just for 3-5men.

I read a post a long time ago, about not releasing it because the requirements are huge needing a couple gigs of ram and a 64bit CPU. While a couple years ago that might seem restrictive to the home users. It's a lot more common now adays for even a desktop to have a 64bit CPU and a couple gigs of RAM (especially game machine).

So guess we just have to wait it out. Nalimov has done a lot of great work for the chess communinty but it would have been nice to have access to the 6men generator. It would have made the process of creating them a lot quicker since multiple people and machines could have done different sets.

-Josh

[Kirill Kryukov]

Hi Martin, Josh! I also have a fast machine, and would like to contribute to generation. Nalimov 6-men bases are nearly complete, so I think we should already start thinking about 7-men bases. As soon as someone releases a generator I think several people can contribute computing resources.

[jshriver]

Hi Martin, Josh! I also have a fast machine, and would like to contribute to generation. Nalimov 6-men bases are nearly complete, so I think we should already start thinking about 7-men bases. As soon as someone releases a generator I think several people can contribute computing resources.

I'm getting an alpha sometime this month, older and somewhat slow but 64bit and going to put 4gigs of ram in it for use as a dedicated chess machine.

Then I'm hoping to get a Athlon64 (maybe Opteron money pending) this fall as a desktop machine. I also have several 1ghz+ machines here I use as part of a cluster so there's a decent amount of computation machinery here I'd just love to toss into egtb generation

Hopefully a 7men will come out. I really see a parallelized 7men egtb generator as the key for success in such a task.

Anyway 3:15am and just came home from work, time for bed
Take care everyone.

-Josh

[Arpad Rusz]

I think I have the source code of the version of tbgen to create 6 men tablebases! I downloaded from Crafty's site two years ago. I have put it on emule (tbgen.zip).

[jshriver]

I think I have the source code of the version of tbgen to create 6 men tablebases! I downloaded from Crafty's site two years ago. I have put it on emule (tbgen.zip).

What's the hash for this file?
-Josh

[Arpad Rusz]

ed2k://|file|tbgen.zip|111087|792A1D54FD96940D3937915F5F0BC320|h=253EQJBGABX4FANURRNM44D5QYEE5NT2|/

[Mister X]

Hi Arpad,

thank you for sharing the file tbgen.zip. I downloaded it today and tried to generate tbgen.exe. In file tbindex.cpp at line 4427 the file tbdecode.c is referenced, which is not in the archive. Compilation failed.

So my question is: Can you please verify, that you share what you get two years ago?

TIA, X

[Wulff]

Hi X!

If memory serves, tbdecode.c & tbdecode.h are part of the Crafty source....

Don't kill me if I remember incorrectly.

[Mister X]

Hi Wulff,

thank's for your hint. But I'm sorry to tell, that I could not find the requested file in several crafty sources either. Then I googled tbgen with no success.

Don't worry, I kill nobody

X

[Martin Kreuzer]

Dear Arpad,

thank you very much for this source code.

As for Mr. X's question about tbdecode.c:
1) There is a file called tbdecode.h in the crafty source code.
2) There is a file called tbdecode.c in the scid source code.
Both files seem to be similar. I have asked a student
to compile the tbgen souces on my 64 bit AMD dual opteron
with 24 GB memory. I hope we will be computing
6-men tablebases soon.

If anybody succeds in compiling a 32 bit windows version,
I would appreciate the executable for my 3.4 GHz Perntium IV
at home.

Greetings,
Martin Kreuzer

[guyhaw]

I keep meaning to do something with one of Rob's CRAFTY offerings - haven't managed it yet.
However, I see that in the Crafty 19.20 zipfile, tbdecode.h (but not .c) exists.
g

[Martin Kreuzer]

Hallo Guy,

indeed, the file tbdecode.h in the Crafty sources seems to contain both the header and the c code. In the file egtb.cpp of Crafty it is
included in the same way as tbdecode.c in tbindex.cpp.

There are also some hints on generating a "cleaner" version of
tbindex.cpp from egtb.cpp in the post

http://perl.guru.org/ccc/m2001.10/192968.txt

Sorry for mentioning the file tbdecode.c of the scid sources
in my previous post. It seems to be an old version.
Not even the typedefs match.

Hope this helps. Best wishes and cordial greetings,
Martin Kreuzer

[Martin Kreuzer]

Hi all,

after fiddling with the code a bit, my student and I have
been able to compile it using g++ under linux.
The file tbdecode.h from the crafty souces needs to be renamed
tbdecode.c
There were a number of further compiler error messages due to
the difference between g++ and the ms compiler Eugene used.
I am in the process of computing my first 6 men tablebase.
For 5 men bases the program works correctly.

If you are interested in the modified sources, drop me
an e-mail. (I am at Dortmund University). I have asked Eugene
permission to put the code on my web pages.

Servus,
Martin Kreuzer

[jshriver]

Hi all,

after fiddling with the code a bit, my student and I have
been able to compile it using g++ under linux.
The file tbdecode.h from the crafty souces needs to be renamed
tbdecode.c
There were a number of further compiler error messages due to
the difference between g++ and the ms compiler Eugene used.
I am in the process of computing my first 6 men tablebase.
For 5 men bases the program works correctly.

If you are interested in the modified sources, drop me
an e-mail. (I am at Dortmund University). I have asked Eugene
permission to put the code on my web pages.

Servus,
Martin Kreuzer

Wonderful news. Can the source be used to generate any 6men egtb, or even better yet 7men?

Anxious to see a gcc version. Just checked your profile and dont see an email addy. What's your website?
-Josh

[guyhaw]

I'm no expert on this but Eugene writes compilers for Microsoft and is said to sail 'very close to the wind' in his use of computer-language facilities.

John Tamplin had quite a bit of work to do to get it running on Unix. The parallel-process synchronisation is rumoured to be unreliable, though it took a 12-processor machine to discover this.

Elsewhere, re CHINOOK's endgames, EGT-errors have been created by bugs within the compilers, e.g. doing 32-bit arithmetic internally when 64-bit arithmetic was required.

So I would certainly recommend full validation of any EGTs produced in a non-Windows context.

g

[guido]

Hi all,

after fiddling with the code a bit, my student and I have
been able to compile it using g++ under linux.
The file tbdecode.h from the crafty souces needs to be renamed
tbdecode.c
There were a number of further compiler error messages due to
the difference between g++ and the ms compiler Eugene used.
I am in the process of computing my first 6 men tablebase.
For 5 men bases the program works correctly.

If you are interested in the modified sources, drop me
an e-mail. (I am at Dortmund University). I have asked Eugene
permission to put the code on my web pages.

Servus,
Martin Kreuzer

Wonderful news. Can the source be used to generate any 6men egtb, or even better yet 7men?

Anxious to see a gcc version. Just checked your profile and dont see an email addy. What's your website?
-Josh

Pls may you record execution time and memory used for each 6-men TB generated by you (and the PC you use)?
I'm the author of another TBs generator (see the thread "Semi-new egtb format") and in this time I'm generating 6-men TBs, so these data can be very ineteresting for me.

Moreover I can extract from my TBs positions with a specific result and it would be possible to check that the two generators give the same results.

Regards
Guid

[jshriver]

Pls may you record execution time and memory used for each 6-men TB generated by you (and the PC you use)?
I'm the author of another TBs generator (see the thread "Semi-new egtb format") and in this time I'm generating 6-men TBs, so these data can be very ineteresting for me.

Moreover I can extract from my TBs positions with a specific result and it would be possible to check that the two generators give the same results.

Regards
Guid

Is your generator open to the public and is source or linux binaries available?
Is there an API for engines to use the egtb's?

-Josh

[Martin Kreuzer]

Hi all,

some infos on the status of the new compilation of
Eugene Nalimovs TB generator:

1) The code can generate 6-men endgame tablebases.
3) I have updated my profile, so you can send me a request
for the code or the executables if you are interested.
(I am also easy to find via google.)
3) Over the weekend I was able to compile the code on
32 bit windows using visual c++.
4) I have already computed 2 endgames on windows:
knnknn (12 hours) and kqqkqq (25 hours).
I have a 3.4 GHz Pentium IV and 2 GB of RAM.
5) Meanwhile I have complied a new executable for windows
with all the optimizations on, and it is about 3x as fast as the
one used to compute the two endgames.
6) The linux version has computed the endgame knnnkr
in 29 hours. It is a version compiled with g++ -O2
It runs on an dual AMD Opteron with 24 GB of memory,
but uses only one of the two processors.
7) The md5sums of the two endgames computed on windows
were correct. The third one is in the process of being checked.
(I will report later.)

Best wishes to everybody,
Martin Kreuzer

[Martin Kreuzer]

Hi all,

also the endgame knnnkr computed under 64 bit linux
has a correct md5 hash value.

Greetings,
Martin Kreuzer

[jshriver]

Hi all,

also the endgame knnnkr computed under 64 bit linux
has a correct md5 hash value.

Greetings,
Martin Kreuzer

Wonderful Can you try building the remaining 13-16 files that Nalimov hasn't released?

-Josh

[jshriver]

Hi all,

also the endgame knnnkr computed under 64 bit linux
has a correct md5 hash value.

Greetings,
Martin Kreuzer

Hi I tried sending you an email threw this forum, but it seems to have been mailed to me. So not sure if you received it or not or if the email I received was just a carbon copy.

My email address is jshriver <at> gmail.com

-Josh

[Martin Kreuzer]

Wonderful Can you try building the remaining 13-16 files that Nalimov hasn't released?

-Josh

Hi Josh,

indeed this is what I have started to do. Of the 16 "missing" endgames
the easiest ones to do seem to be kqpkrn and kqpkrb.
E.g. for the first one I need to have kqqkrn, kqrkrn, kqbkrn, and
kqnkrn before I can begin. Do you have any of these?
Could you post them on olympuschess? (My university does not
allow me to use emule.)

Greetings,
Martin Kreuzer

[guido]

Pls may you record execution time and memory used for each 6-men TB generated by you (and the PC you use)?
I'm the author of another TBs generator (see the thread "Semi-new egtb format") and in this time I'm generating 6-men TBs, so these data can be very ineteresting for me.

Moreover I can extract from my TBs positions with a specific result and it would be possible to check that the two generators give the same results.

Regards
Guid

Is your generator open to the public and is source or linux binaries available?
Is there an API for engines to use the egtb's?

-Josh

My TBs generator (gafs) and other programs can be used freely but only for private use, i.e. without commercial scopes. The program is available in Linux binary format in two versions, for 32-bits and 64-bits PCs.

In order to use my TBs in a chess program I wrote a small library in the format of an object file (gafslib.o) that can be linked to the user program. The library has only two entry points (a function for initialization and another function for obtaining the information) while all the other functions and variables are private to avoid conflicts with functions
and names used by the programmer.

You can find my generator and library in the site http://users.iol.it/rigel_g/ already known to you.

I put in my site another program (gafsdim) to calculate the chess positions in function of the total number of men.
For people interested to this problem the program is present in both binary and source format.
Is someone interested to improve it with me?
Compilation requests the use of the gmp library.
(A small problem: explanations and results are at present in Italian).

In this moment I'm reviewing gafs for some errors found during the generation of 6-men TBs. In the next weeks I'll put the new version on the site even if the present version is ok for 5-men TBs with some precautions.
For gafslib I will do the same and it is my intention to add also a 32-bits Windows version to permit the use of my TBs under Windows.

Best regards
guido

[guido]

Hi all,

some infos on the status of the new compilation of
Eugene Nalimovs TB generator:

1) The code can generate 6-men endgame tablebases.
3) I have updated my profile, so you can send me a request
for the code or the executables if you are interested.
(I am also easy to find via google.)
3) Over the weekend I was able to compile the code on
32 bit windows using visual c++.
4) I have already computed 2 endgames on windows:
knnknn (12 hours) and kqqkqq (25 hours).
I have a 3.4 GHz Pentium IV and 2 GB of RAM.
5) Meanwhile I have complied a new executable for windows
with all the optimizations on, and it is about 3x as fast as the
one used to compute the two endgames.
6) The linux version has computed the endgame knnnkr
in 29 hours. It is a version compiled with g++ -O2
It runs on an dual AMD Opteron with 24 GB of memory,
but uses only one of the two processors.
7) The md5sums of the two endgames computed on windows
were correct. The third one is in the process of being checked.
(I will report later.)

Best wishes to everybody,
Martin Kreuzer

Thank you for the info.

I found these times on Athlon64 3800 and 2 GB RAM:

Ending cpu elapsed method
time time

kqqkqq 46133,35 / 63857 s 2
kqqkrr 83278,59 / 100370 s 2
kqqkbb 61392.54 / 79497 s 2
krrkrr 59497,02 / 70983 s 2
krrkrb 260143.32 / 316047 s 2
krrkbb 84555.21 / 95918 s 2
krrknn 74312.54 / 90714 s 2
kbbkbb 38042.50 / 41150 s 2
kbbknn 193439.59 / 216974 s 2
knnknn 5033.84 / 6254 s 3
kqqqqk 2040,31 / ~2045 s 1 all in RAM
krrrrk 2252.02 / 2257 s 1 all in RAM

The results have been checked verifying their congruence.

Gafs can use three different methods for computing TBs.
As you can see times are all obtained with method 2 except knnknn (method 3) and kqqqqk, krrrrk (method 1)
Method 3 (retrograde algorithm) is the fastest but uses more the disks and there is an error (that I finally found!) so I
have used normally method 2 (semi-retrograde algorithm).
Method 1 (direct algorithm) is good only for useless endings.

Best regards
guido