7-man EGTB Bounty Reborn - Infrastructure Discussion
Posted: Thu Apr 21, 2011 2:47 am
Infrastructure is the vital part of this project, so it deserves it's own discussion thread.
Here is a very rough idea. The infrastructure should provide support for the following tasks:
1. Generation of the tables:
Here is a very rough idea. The infrastructure should provide support for the following tasks:
1. Generation of the tables:
- Information support. Centralized database should exist to track the already constructed tables. It should be easy to see what tables have been already built, and what tables can be done next. Multi-metric support (parallel WDL and WDL50 generation).
- Logistics. It should be easy to obtain the already generated sub-endgame tables, that are necessary for building a new table.
- Distribution network. Is eMule still practical? All up-to-date solutions have to be evaluated.
- Support for tracking the availability of particular tables. (Something that I have been doing manually for 6-piece sharing project). How many copies of each table are available on network.
- Support for tracking the free space. A participant tells the system "I can host another 1 TB". The system tells the participant: "Such and such tables need more copies, please download from such and such people (priority download)."
- Cenrtalized system to probe the whole already computed dataset, even when tables are stored on thousands of computers.
- Clever load balancing. (For a dumb version, just accept a fixed N requests per minute from any host).
- Participants (those who host the tables) get some probing rate bonus (Dumb version: 10 times the limit compared to non participants).