8-man tablebases already exist?

Endgame analysis using tablebases, EGTB generation, exchange, sharing, discussions, etc..
Post Reply
jkominek
Posts: 150
Joined: Mon Dec 04, 2006 9:02 am
Sign-up code: 0
Location: Pittsburgh, PA

8-man tablebases already exist?

Post by jkominek »

It is Christmas break for me and one of my annual habits during the yearly downtime is to spend a little time searching for interesting endgame news. I may have found one. In the Y-Combinator Hacker News discussion board, while discussing Centaur chess and Leela, one commentator makes a brief aside stating that 8-man tablebases have been created. No details on the metric computed or the team or the computer employed. But if true, given that Y-Combinator is San Francisco based and is well tapped into the venture investor community, it is a fair guess that the team is based in Silicon Valley.

The information is evidently second-hand, and from a year and a half ago, so make of it what you will.
BTW: A research team has created the 8-man Tablebase, although it takes probably
100s of TB to store. This means that 8-man Tablebases will remain as Hard-drive only
for now. https://news.ycombinator.com/item?id=20774774 [Aug 23 2019]
In the intervening year and a half I'm not aware of anyone coming forward with an announcement. It would be notable news for a team seeking priority of accomplishment, so such a long period of silence is hard to square with the claim.

The only bit I can add is that I have a memory from a couple years ago of reading a similarly opaque comment from similar circles that 8-man tablebases will be created "sooner than you think". I have not been able to relocate that comment.

Any other rumored sightings of the Yeti of computer chess are welcome.

P.S. I am aware that TalkChess is where it's at these days. But I have a fondness for Kirill's EGTB discussion board. Nostalgia probably.
jkominek
Posts: 150
Joined: Mon Dec 04, 2006 9:02 am
Sign-up code: 0
Location: Pittsburgh, PA

Re: 8-man tablebases already exist?

Post by jkominek »

koistinen
Posts: 92
Joined: Fri May 02, 2008 7:59 pm
Sign-up code: 0
Location: Stockholm

Re: 8-man tablebases already exist?

Post by koistinen »

I've been thinking of how to do 8-man and beyond and a decentralized way of computing it seems interesting as storage is one of the biggest problems.
For 8-man each pawnless endgame table need only take 2-3 months (with 50 move rule, proportional with dtc and worse for dtm) given each computing participant has one large hard disk, say 10 or 18 TB and an uncapped 100 Mb internet connection.
Storage participants might contribute using less resources.
Endings with pawns can of course be done with less centralized resources.

What do you think, would this way of doing things be nice? (Having lots of people share the glory.)
jkominek
Posts: 150
Joined: Mon Dec 04, 2006 9:02 am
Sign-up code: 0
Location: Pittsburgh, PA

Re: 8-man tablebases already exist?

Post by jkominek »

I think distributed computing of large tablebases (large=8-9 men, at the time of writing) is technically possible. But this going on in real life I only see it happening if egtb generation were somehow how a by-product of time-space-proving crypto-currency mining. I am thinking in particular of Chia crypto and the current run on large capacity SSDs and hard drives. Discover a way to have tablebase generation become a way to make money in China, and the computers will surely follow.

You've been pondering endgame generation for a very long time -- author of the 2001 Koistinen algorithm sketch, after all. I bet you've noticed that when a newbie comes on board the top questions are:

1. Where can I find/download/install tablebases?
2. Why don't you just generate and save the important positions?
3. Why don't you use distributed computing like folding@home?

A few experts then respond, sometimes patiently, sometimes not.

Oh, let us not forget four,
4. Yeah, that's nice, but when are you going to do the N+1 tables.

c.f. viewtopic.php?f=6&t=7618 for the eternal answer "never". With the implied eternal words of wisdom: never say never. (Pertaining to extrapolation of Kryder's Law, we do not yet have somewhat affordable 10 TB SSDs.)

I divide the main idea into five aspects.
a) distributed computing
b) distributed storage
c) distributed probing
d) project management
e) participant motivation

For a) I think the contributors necessarily divide into two camps. There will be a small group of participants with high-capacity computers, defined as able to generate pawnless 8-man tables in isolation. Then there is a larger group of ordinary computers owning regular desktops and laptop. The work allocation manager calls upon them to compute and make available pawn slices.

Items b) and c) scare me personally. I do not know high-security programming and would be hesitant to even adopt an existing infrastructure. The thought of the system being hijacked to work as a spam bot network, or DDOS net, or storing large videos of a "certain nature", or worst of all injecting malware onto peoples computers -- that's a though I don't want weighing on my conscience. A more expensive alternative is for participants buy-in to one of the major cloud providers such Google, Microsoft, or Amazon; all the hosting is on rented computers.

d) The project won't just run itself. It takes plenty of energy to be one of the organizers.

e) And what I believe to be the biggest show-stopper: what would motivate people to participate in droves? The stockfish and lczero training networks are impressive, but that runs only among the dozens to hundreds of contributors. What takes it to the tens of thousands and beyond? Endgame crypto-mining is my answer, if it were even possible. Outside that, how would it become a fashion?
koistinen
Posts: 92
Joined: Fri May 02, 2008 7:59 pm
Sign-up code: 0
Location: Stockholm

Re: 8-man tablebases already exist?

Post by koistinen »

I think Google, Facebook, et.c. have shown us that centrally controlled computing easily turns becomes evil.
That is why I prefer decentralized computing, not just distributed. That way I don't get to lord it over people.
Besides computer chess I have also been interested in crypto techniques for quite a while.
Merkle trees and fraud proofs would help people know they distribute what they want and not something else.
Yet another shitcoin, that is a possibility but I am not making it a priority.

I am still uncertain of my ability to produce working code as I too easily get sucked into optimizing lots of special cases.
Avoiding it, the code might be 5-10 times slower because more data needs to be moved around.
The way I do it, spinning rust is fine for storage, no SSD required.
Without the optimization, very little RAM is required. If I could program it, I guess the computer on the disk drive would have enough RAM.
Post Reply