Kirill Kryukov wrote:jshriver wrote:It's now 2008 and even I can afford a 1-2tb every couple months. So disk space doesnt seem to be the issue anymore.
So where are the 7-men tables and has anyone have any updates?
Here is one interesting post from Rybka forum:
Victor Zakharov wrote:After reading this thread I’ve made a conclusion that most people can’t imagine how difficult is the process of creating the tablebases. So here are my 2 pence to this topic.
It is true that an experienced person can write a 7-men tablebase generator in one-two days. The only problem with such a program is that it either will work for ages (using disk all the time) or will need about 3 TB RAM (and then it will be several times quicker). 3 TB RAM is 1000 times more than a modern PC has. Fortunately, it is possible to implement slicing, i.e. splitting the tables to the parts that do not much depend on each other.
...
I agree with every word of it.
Except, as far as I know, Nalimov's generator is public.
Look here for example.
Some tricks are:
First trick - don't keep much information about each position in ram.
Don't use distance directly, instead only store what you need to compute the next iteration.
Compute only wins for white and losses for black.
When computing new black losses:
Use 1 bit/wtm position, for white win or illegal position.
Use 1 bit/wtm position, for new white win
For each black position, if there is no legal move that avoids loss and one legal move reaches a new white win, it is a new black loss.
When computing white wins:
Use 1 bit/btm position, for black losses.
Use 1 bit/btm position for white win or illegal position.
For each white position, if it is not already won and a legal move reaches a black loss, it is a new white win.
White wins are computed by or-ing the previous white wins with the new white wins.
So far, we are down to 3 bits of ram per position.
After using mirroring and the like we need <3*2^(7*6-3) bits of ram. ~= 192 GB
Second trick - partition the tables so that each part can be computed separately.
Say you construct the index by joining the positions of the pieces like this:
BBWWxyz, i.e. two black pieces,, two white and then the remaining 3.
Then, by fixing the position of the two pieces of the side whose moves you are not examining (except for possible mirroring) you reduce the size to 1/512..
There is a downside that you get to do disk reads and writes in blocks of only 64^3 bits but that can be helped so you get blocks 8^3 times larger. i.e. ~= 32 KB or 16 MB
Using the larger block size the reduction is only to 1/64, but 3 GB is still not overmuch on a pc of today.
Third trick is examining moves for bitboard size chunks at a time in order to use less cpu.
So, while plenty of disk space is needed to store the results, you don't need all that much ram to do the computation.
I discovered parts of this in discussion on this site (special credits to h.g.muller) and there are other ways to partition the computation.
With the first two tricks you should need 3GB, not 3TB of ram.
Is my explanation clear and have I made some mistakes?
I'm just anxious and feel I've and many others have been patient. 