The 16 "missing" Nalimov files
The 16 "missing" Nalimov files
I have created the 16 missing Nalimov tablebases for 33p endings by conversion from Johan de Koning's FEG data. The total compressed size is about 130GB, with kqpkrb being the largest at about 17GB, and kppkpp the smallest at about 1 GB. The original FEG data is about 90 GB, smaller partially because of larger block size (32k as opposed to 8k), and a more efficient compression algorithm.
I have run verification on a few, but not all the endings.
If no other source is forthcoming, I'd be willing to put these on a disk and send to someone in the US who has all the existing datasets and can run a complete verification before distributing more widely. Verification is reasonably fast, kqpkrb took about 1.5 days on a 3.8 Ghz machine, but the other endings will run a lot faster.
I have run verification on a few, but not all the endings.
If no other source is forthcoming, I'd be willing to put these on a disk and send to someone in the US who has all the existing datasets and can run a complete verification before distributing more widely. Verification is reasonably fast, kqpkrb took about 1.5 days on a 3.8 Ghz machine, but the other endings will run a lot faster.
-
- Posts: 489
- Joined: Sat Jan 21, 2006 10:43 am
- Sign-up code: 10159
- Location: Reading, UK
- Contact:
Neatly done ...
Fine job as ever, Marc. Excellent news: thank you.
I'd be happy to file your FEG-derived 'EN .tbs' stats alongside your FEG.lof stats if you care to leave a file on this CCRL board.
Maybe a disc on a round trip from Eiko or Renze in Europe would expedite the verification. I don't have the requisite EGTs myself.
g
I'd be happy to file your FEG-derived 'EN .tbs' stats alongside your FEG.lof stats if you care to leave a file on this CCRL board.
Maybe a disc on a round trip from Eiko or Renze in Europe would expedite the verification. I don't have the requisite EGTs myself.
g
- Kirill Kryukov
- Site Admin
- Posts: 7399
- Joined: Sun Dec 18, 2005 9:58 am
- Sign-up code: 0
- Location: Mishima, Japan
- Contact:
Excellent news, thank you very much, Marc! I'd suggest to send it to Joshua Shriver, as he lives in US (AFAIK), and he has good connection, and because I know he will share the files online. (And he also volanteered in another thread).
KCEC | EGTB Online | 3x3 Chess | 3x4 Chess | 4x4 Chess | Longest Checkmates | EGTB Test Suite | Opening Sampler | EGTB Bounty | NULP
-
- Posts: 489
- Joined: Sat Jan 21, 2006 10:43 am
- Sign-up code: 10159
- Location: Reading, UK
- Contact:
... and the MD5sums please ...
... which will help check the EGTs after whatever trip they do. g
- jshriver
- Posts: 298
- Joined: Tue Jan 31, 2006 5:59 am
- Sign-up code: 0
- Location: Toledo, OH, USA
- Contact:
Aye I live in Ohio. What are the prereq sets for verification? Regardless I'd be willing to host them.Kirill Kryukov wrote:Excellent news, thank you very much, Marc! I'd suggest to send it to Joshua Shriver, as he lives in US (AFAIK), and he has good connection, and because I know he will share the files online. (And he also volanteered in another thread).
-Josh
- jshriver
- Posts: 298
- Joined: Tue Jan 31, 2006 5:59 am
- Sign-up code: 0
- Location: Toledo, OH, USA
- Contact:
Re: The 16 "missing" Nalimov files
Does this include kppkpp? Either way congrats on the great workmbourzut wrote:I have created the 16 missing Nalimov tablebases for 33p endings by conversion from Johan de Koning's FEG data. The total compressed size is about 130GB, with kqpkrb being the largest at about 17GB, and kppkpp the smallest at about 1 GB. The original FEG data is about 90 GB, smaller partially because of larger block size (32k as opposed to 8k), and a more efficient compression algorithm.
I have run verification on a few, but not all the endings.
If no other source is forthcoming, I'd be willing to put these on a disk and send to someone in the US who has all the existing datasets and can run a complete verification before distributing more widely. Verification is reasonably fast, kqpkrb took about 1.5 days on a 3.8 Ghz machine, but the other endings will run a lot faster.
-Josh
-
- Posts: 489
- Joined: Sat Jan 21, 2006 10:43 am
- Sign-up code: 10159
- Location: Reading, UK
- Contact:
3-3p: the 'last 16 endgames' ....
The 'last 16' 3-3p endgames for which Nalimov did not publish the EGTs are:
KBNKPP, KBPKNP, KBPKPP, KNPKPP, KPPKPP, KQPKBP, KQPKNP. KQPKPP,
KQPKRB, KQPKRN, KQPKRP, KRBKPP, KRNKPP, KRPKBP, KRPKNP, KRPKPP
Marc is asking the (USA-)offload-receiver to verify these chessicly using Nalimov's EGT-verification software before sharing them on 'EGTs Online'. Therefore, any 6- and 5-man endgame these can convert to is needed as verification is not just an internal matter [and indeed, known mistakes have occurred at the conversion-point].
I agree: this is more great work from Marc, and its good to see FEG making a bigger contribution, and a contribution to the verification process.
g
KBNKPP, KBPKNP, KBPKPP, KNPKPP, KPPKPP, KQPKBP, KQPKNP. KQPKPP,
KQPKRB, KQPKRN, KQPKRP, KRBKPP, KRNKPP, KRPKBP, KRPKNP, KRPKPP
Marc is asking the (USA-)offload-receiver to verify these chessicly using Nalimov's EGT-verification software before sharing them on 'EGTs Online'. Therefore, any 6- and 5-man endgame these can convert to is needed as verification is not just an internal matter [and indeed, known mistakes have occurred at the conversion-point].
I agree: this is more great work from Marc, and its good to see FEG making a bigger contribution, and a contribution to the verification process.
g
- Kirill Kryukov
- Site Admin
- Posts: 7399
- Joined: Sun Dec 18, 2005 9:58 am
- Sign-up code: 0
- Location: Mishima, Japan
- Contact:
I can provide any of already shared 33p sets. I have all except two of them currently, the last two are downloading right now, should finish in less than a week. If Josh will do verification, I can set up an FTP to provide him with all necessary sets.
KCEC | EGTB Online | 3x3 Chess | 3x4 Chess | 4x4 Chess | Longest Checkmates | EGTB Test Suite | Opening Sampler | EGTB Bounty | NULP
Hi Mark,
Super job nice work! Listen when do you expect these 16 TB's to be verified for correctness? I have been studying the data from the TBS files and am very much interested in the results of the TBS files and not the actual tablebases. I guess until all 16 of these files are verified for correctness there is no sense in analyzing the TBS files at this point. Mark one other big question. Is there any chance if these 16 come out ok that you would be willing to generate the 5-1 tbalebases along with their TBS files?? It would be important to me for sure to support some of my research in EGTB's. I would think that except for the KPPPPK tablebase that the others should converge pretty quickly IMHO. Even the KPPPPK probably converges quickly also. Any chance that you could do them???
Thanks in advance,
Les
Super job nice work! Listen when do you expect these 16 TB's to be verified for correctness? I have been studying the data from the TBS files and am very much interested in the results of the TBS files and not the actual tablebases. I guess until all 16 of these files are verified for correctness there is no sense in analyzing the TBS files at this point. Mark one other big question. Is there any chance if these 16 come out ok that you would be willing to generate the 5-1 tbalebases along with their TBS files?? It would be important to me for sure to support some of my research in EGTB's. I would think that except for the KPPPPK tablebase that the others should converge pretty quickly IMHO. Even the KPPPPK probably converges quickly also. Any chance that you could do them???
Thanks in advance,
Les
- jshriver
- Posts: 298
- Joined: Tue Jan 31, 2006 5:59 am
- Sign-up code: 0
- Location: Toledo, OH, USA
- Contact:
Hrm if I could get the generator working under 32-bit linux I'd start generating them will add to queue.vb4 wrote:Hi Mark,
Super job nice work! Listen when do you expect these 16 TB's to be verified for correctness? I have been studying the data from the TBS files and am very much interested in the results of the TBS files and not the actual tablebases. I guess until all 16 of these files are verified for correctness there is no sense in analyzing the TBS files at this point. Mark one other big question. Is there any chance if these 16 come out ok that you would be willing to generate the 5-1 tbalebases along with their TBS files?? It would be important to me for sure to support some of my research in EGTB's. I would think that except for the KPPPPK tablebase that the others should converge pretty quickly IMHO. Even the KPPPPK probably converges quickly also. Any chance that you could do them???
Thanks in advance,
Les
-Josh
jshriver wrote:Hrm if I could get the generator working under 32-bit linux I'd start generating them will add to queue.vb4 wrote:Hi Mark,
Super job nice work! Listen when do you expect these 16 TB's to be verified for correctness? I have been studying the data from the TBS files and am very much interested in the results of the TBS files and not the actual tablebases. I guess until all 16 of these files are verified for correctness there is no sense in analyzing the TBS files at this point. Mark one other big question. Is there any chance if these 16 come out ok that you would be willing to generate the 5-1 tbalebases along with their TBS files?? It would be important to me for sure to support some of my research in EGTB's. I would think that except for the KPPPPK tablebase that the others should converge pretty quickly IMHO. Even the KPPPPK probably converges quickly also. Any chance that you could do them???
Thanks in advance,
Les
Hi Josh,
Wow that would be great !!!! I dont know much about Linux but perhaps someone out here might be able to assist you. Keeping my fingers crossed.
Thanks,
Les
-Josh
With the current version of the EGTB generator you can't create any of the 5+1 tablebases. It has to be modified. I will try to check the code during my holidays within the next three weeks. If anyone else has more information just let me know.vb4 wrote:Hi Mark,
...... Is there any chance if these 16 come out ok that you would be willing to generate the 5-1 tbalebases along with their TBS files?? It would be important to me for sure to support some of my research in EGTB's. I would think that except for the KPPPPK tablebase that the others should converge pretty quickly IMHO. Even the KPPPPK probably converges quickly also. Any chance that you could do them???
Thanks in advance,
Les
Hi Lutz,Lutz Nebe wrote:With the current version of the EGTB generator you can't create any of the 5+1 tablebases. It has to be modified. I will try to check the code during my holidays within the next three weeks. If anyone else has more information just let me know.vb4 wrote:Hi Mark,
...... Is there any chance if these 16 come out ok that you would be willing to generate the 5-1 tbalebases along with their TBS files?? It would be important to me for sure to support some of my research in EGTB's. I would think that except for the KPPPPK tablebase that the others should converge pretty quickly IMHO. Even the KPPPPK probably converges quickly also. Any chance that you could do them???
Thanks in advance,
Les
I didnt know that Eugene's code would need to be modified!! I hope it isnt too difficult to do since in the past I have heard his code is rather complicating. Please keep me posted as to your progress when you have time to look at this or if anyone else out here is knowledgeable enough to make the necessary code changes.
Thanks,
Les
Re: The 16 "missing" Nalimov files
I have decided to run verification on all the endings to minimize the chances of repeated file exchanges. All the verifications were successful. Attached are the .tbs and md5sums for the lot. I have contacted Josh if he is willing to host the data files.mbourzut wrote:I have created the 16 missing Nalimov tablebases for `3.8 Ghz machine, but the other endings will run a lot faster.
- Attachments
-
- nali_16.zip
- (53.95 KiB) Downloaded 747 times
- jshriver
- Posts: 298
- Joined: Tue Jan 31, 2006 5:59 am
- Sign-up code: 0
- Location: Toledo, OH, USA
- Contact:
Re: The 16 "missing" Nalimov files
Sounds great anxious to be part of this. I can host 30gigs from my website, and can try and host all 130 gig using emule from my home computer.mbourzut wrote:I have decided to run verification on all the endings to minimize the chances of repeated file exchanges. All the verifications were successful. Attached are the .tbs and md5sums for the lot. I have contacted Josh if he is willing to host the data files.mbourzut wrote:I have created the 16 missing Nalimov tablebases for `3.8 Ghz machine, but the other endings will run a lot faster.
After receiving the data, it will take maybe a day or two to upload to the site.
-Josh
- jshriver
- Posts: 298
- Joined: Tue Jan 31, 2006 5:59 am
- Sign-up code: 0
- Location: Toledo, OH, USA
- Contact:
In retrospect, to make things run even smoother. I can setup a hidden area on my site for people in this forum download. That way I can get the files to other key distributors before releasing it to the public.
What do you all think? This would take some strain off my server by allowing it to be more easily shared via emule, especially initially. Figure once these go public it's going to be insane traffic will be fun watching the system stats though lol.
I could also send some to TapaniS.
-Josh
What do you all think? This would take some strain off my server by allowing it to be more easily shared via emule, especially initially. Figure once these go public it's going to be insane traffic will be fun watching the system stats though lol.
I could also send some to TapaniS.
-Josh
Re: The 16 "missing" Nalimov files
Thank you so much for doing this! Now all I need are those darn 5-1 positions and I can kick some ... completing my study on the 3-4-5-6 piece egtb's. There have been some interesting findings so far which I will eventually summarize and share with everyone.mbourzut wrote:I have decided to run verification on all the endings to minimize the chances of repeated file exchanges. All the verifications were successful. Attached are the .tbs and md5sums for the lot. I have contacted Josh if he is willing to host the data files.mbourzut wrote:I have created the 16 missing Nalimov tablebases for `3.8 Ghz machine, but the other endings will run a lot faster.
Thanks,
Les
- jshriver
- Posts: 298
- Joined: Tue Jan 31, 2006 5:59 am
- Sign-up code: 0
- Location: Toledo, OH, USA
- Contact:
Re: The 16 "missing" Nalimov files
I was reading this page:mbourzut wrote:I have created the 16 missing Nalimov tablebases for 33p endings by conversion from Johan de Koning's FEG data.
http://www.aarontay.per.sg/Winboard/egt ... masteredgb
It mentions that the Koning egtb only stores wins from the white point of view. Will this effect the quality of the nalimov conversion? Will we have the information for black's side?
On a side note this quote seems promising:
"The generator also makes generating 6-man with KWWWWB possible on a 32 bit computer possible , but that requires about 300MB (which isn't a lot by todays standards) of free RAM"
I'm interpreting this as being able to generate 5-1 egtbs
-Josh
Last edited by jshriver on Thu Jul 27, 2006 3:00 am, edited 1 time in total.
Is this right ??
Hi Mark,
Listen I just took a peek inside one of your tbs files (KPPKPP) and see that it only lists the stats for wtm, win or lose but no stats for btm win or lose. I checked with the 4 piece EGTB KPKP and it has both sides to move stats?? Can you explain Mark? I would appreciate it.
Thanks,
Les
Listen I just took a peek inside one of your tbs files (KPPKPP) and see that it only lists the stats for wtm, win or lose but no stats for btm win or lose. I checked with the 4 piece EGTB KPKP and it has both sides to move stats?? Can you explain Mark? I would appreciate it.
Thanks,
Les
- Kirill Kryukov
- Site Admin
- Posts: 7399
- Joined: Sun Dec 18, 2005 9:58 am
- Sign-up code: 0
- Location: Mishima, Japan
- Contact:
Thank you, Marc! I'm looking forward into completing the online collection!
I think it's a good idea - to distribute the files to several key uploaders first, who will then share them by eMule.jshriver wrote:In retrospect, to make things run even smoother. I can setup a hidden area on my site for people in this forum download. That way I can get the files to other key distributors before releasing it to the public.
What do you all think? This would take some strain off my server by allowing it to be more easily shared via emule, especially initially. Figure once these go public it's going to be insane traffic will be fun watching the system stats though lol.
I could also send some to TapaniS.
-Josh
KCEC | EGTB Online | 3x3 Chess | 3x4 Chess | 4x4 Chess | Longest Checkmates | EGTB Test Suite | Opening Sampler | EGTB Bounty | NULP
- jshriver
- Posts: 298
- Joined: Tue Jan 31, 2006 5:59 am
- Sign-up code: 0
- Location: Toledo, OH, USA
- Contact:
Re: The 16 "missing" Nalimov files
Would your code be able to convert 5-1 FEG data to nalimov? While in another thread it was mentioned that the tbgen doesnt support 5-1.. does the code which reads the data.mbourzut wrote:I have created the 16 missing Nalimov tablebases for 33p endings by conversion from Johan de Koning's FEG data. The total compressed size is about 130GB, with kqpkrb being the largest at about 17GB, and kppkpp the smallest at about 1 GB. The original FEG data is about 90 GB, smaller partially because of larger block size (32k as opposed to 8k), and a more efficient compression algorithm.
I have run verification on a few, but not all the endings.
If no other source is forthcoming, I'd be willing to put these on a disk and send to someone in the US who has all the existing datasets and can run a complete verification before distributing more widely. Verification is reasonably fast, kqpkrb took about 1.5 days on a 3.8 Ghz machine, but the other endings will run a lot faster.
After a lot of digging I finally found feg on ubisofts ftp site. I'm generating the 5man knnnk right now and will start knnnnk when this finishes. Assuming it works
-Josh
Re: The 16 "missing" Nalimov files
jshriver wrote:I was reading this page:mbourzut wrote:I have created the 16 missing Nalimov tablebases for 33p endings by conversion from Johan de Koning's FEG data.
http://www.aarontay.per.sg/Winboard/egt ... masteredgb
It mentions that the Koning egtb only stores wins from the white point of view. Will this effect the quality of the nalimov conversion? Will we have the information for black's side?
Hi Josh,
You must have been reading my mind regarding the btm stats. Do you or anyone reading this know if we will be able to get this missing data?? It seems as though all the rest of the files I had looked at had both WTM and BTM info. Any feedback regarding this would be most appreciated.
Take care,
Les
On a side note this quote seems promising:
"The generator also makes generating 6-man with KWWWWB possible on a 32 bit computer possible , but that requires about 300MB (which isn't a lot by todays standards) of free RAM"
I'm interpreting this as being able to generate 5-1 egtbs
-Josh
Few more questions with these 16 egtb's (Mark?)
I just finished looking at all 16 egtb's that Mark was generous enough to generate and have a few questions. As mentioned in anearlier post there is no BTM data for kppkpp and hope that someone here can resolve if we can get that info. Secondly the following 3 egtb's have all the data except the last line which is broken positions for BTM (I think or atleast I hope it was just clipped off the file when it was zipped) Can someone fill me in about these 3 egtb's krnkpp - krbkpp - kbnkpp. I know it has nothing to do with the functionality but for my research it is important.
Thanks in advance,
Les
Thanks in advance,
Les
- Kirill Kryukov
- Site Admin
- Posts: 7399
- Joined: Sun Dec 18, 2005 9:58 am
- Sign-up code: 0
- Location: Mishima, Japan
- Contact:
Re: Few more questions with these 16 egtb's (Mark?)
BTM statistics for symmetrical endgames is identical to WTM statistics. Also, in my collection of TBS files there are symmetrical sets with missing BTM info (eg. KRNKRN). If you need to have separate WTM and BTM info, you can simply duplicate the WTM stats.vb4 wrote:I just finished looking at all 16 egtb's that Mark was generous enough to generate and have a few questions. As mentioned in anearlier post there is no BTM data for kppkpp and hope that someone here can resolve if we can get that info.
KCEC | EGTB Online | 3x3 Chess | 3x4 Chess | 4x4 Chess | Longest Checkmates | EGTB Test Suite | Opening Sampler | EGTB Bounty | NULP