Rollover problem

[vb4]

Some time ago I remember that there was an issue of some numbers in some of the tbs stat files actually were large enough to roll over. I seem to remember 3 if my memory is correct but we were far from completing the generation up to the 6 piece egtb's. Is anyone aware of this problem and since this did not happen for the majority of the stat files can someone provide me with a list of just those parts of a tbs stat file where the problem occured so that I may make the necessary adjustments?

Thanks,

Les

[guyhaw]

There certainly was a time when some counts in the Nalimov .tbs-creation program would go 'over the top'.
I for one detected it when using the 'redundant check' of comparing the total of win+drawn+lost+broken positions with the integer range as defined in Eugene's code. I think it first happened in some 6-man P-less endgames, and a whole 2^32 positions would be lost - usually a lot of draws.
The most I saw in 6-man P-ful .tbs stats was 3*2^32 positions being lost: anyway, Eugene fixed it and I think you will find all .tbs stats ok on this front at least.

There was another .tbs problem whose nature remained unclear, but which surfaced with KBPKN: our DTC/Z stats did not agree with Eugene's DTM stats and it turned out that the latter were wrong. They may still stubbornly persist on Rob Hyatt's site. I stopped asking about this years ago - but a check against the index-range figures will reveal the problem, and if Marc B cares to produce some 'EN' stats via FEG.lof, that would be another check on these stats.

g

[vb4]

There certainly was a time when some counts in the Nalimov .tbs-creation program would go 'over the top'.
I for one detected it when using the 'redundant check' of comparing the total of win+drawn+lost+broken positions with the integer range as defined in Eugene's code. I think it first happened in some 6-man P-less endgames, and a whole 2^32 positions would be lost - usually a lot of draws.
The most I saw in 6-man P-ful .tbs stats was 3*2^32 positions being lost: anyway, Eugene fixed it and I think you will find all .tbs stats ok on this front at least.

There was another .tbs problem whose nature remained unclear, but which surfaced with KBPKN: our DTC/Z stats did not agree with Eugene's DTM stats and it turned out that the latter were wrong. They may still stubbornly persist on Rob Hyatt's site. I stopped asking about this years ago - but a check against the index-range figures will reveal the problem, and if Marc B cares to produce some 'EN' stats via FEG.lof, that would be another check on these stats.

g

Thanks Guy,

As always your input is appreciated. Lets hope someone will be able to check the correctness of all those tbs files.

Les

[guyhaw]

I haven't been checking the .tbs files from the beginning as (a) I assumed they were ok [wrong] and (b) I didn't know the index-ranges were in the Nalimov TBGEN source-code until someone told me.
However, when John T rang Marc B's GTBGEN to produce DTC/Z and DTZ50 EGTs for 6-man pawnless endgames, I certainly started x-checking between DTC, DTM and DTZ stats then and picked up these errors - and I've run this check on all .tbs files since to show them apparently aok.
The dropping of n*2^32 positions is of course because .tbs used to use 32-bit integer arithmetic: I guess it now uses 64-bit integer arithmetic.
Can't remember how I picked up the KBPKN error: I don't think it was with a check against the FEG.lof stats, (which is possible iff there is a P on the board). I think it was before JT produced DTC and DTZ EGTs but I could be wrong.
Incidentally, there are 147,148,759 wtm wins in KBPKN and not 139,481,562, so if you see the latter, the 'public stats are still wrong.
g

[vb4]

I haven't been checking the .tbs files from the beginning as (a) I assumed they were ok [wrong] and (b) I didn't know the index-ranges were in the Nalimov TBGEN source-code until someone told me.
However, when John T rang Marc B's GTBGEN to produce DTC/Z and DTZ50 EGTs for 6-man pawnless endgames, I certainly started x-checking between DTC, DTM and DTZ stats then and picked up these errors - and I've run this check on all .tbs files since to show them apparently aok.
The dropping of n*2^32 positions is of course because .tbs used to use 32-bit integer arithmetic: I guess it now uses 64-bit integer arithmetic.
Can't remember how I picked up the KBPKN error: I don't think it was with a check against the FEG.lof stats, (which is possible iff there is a P on the board). I think it was before JT produced DTC and DTZ EGTs but I could be wrong.
Incidentally, there are 147,148,759 wtm wins in KBPKN and not 139,481,562, so if you see the latter, the 'public stats are still wrong.
g

Hi Guy,

Listen I looked at KBPKN and I do in fact show 139,481,562. Good catch Guy! Do you have the correct tbs file for that TB and if you do can you make it available?? Can you explain to me Guy how this could happen and how do we know it hasnt happened in other tbs files?

Thanks,

Les

[guyhaw]

The correct stats for the Nalimov DTM KBPKN EGT are attached.
Why did it happen? No idea, but the effect can be seen in the stats.
Counts of broken positions, wtm losses and btm wins were ok, of wtm wins and btm losses were too low, of draws were too high.
The correction came on request from Eugene and briefly appeared on Rob Hyatt's ftp site.

A principle is that data should be tested with any 'redundant' check that is available. That's what is happening with the current EN-MB x-check of EGT-file MD5sums.
A check, but not one that would have found the above error, is as previously stated: ensure that all index-positions (as advertised in the source code) are accounted for. Another test is to check the alignment of FEG.lof data with EN.tbs data, but only when at least one Pawn is present (as the counting methods are otherwise different). This error was picked up because the DTC stats didn't agree with the DTM ones.
g

[vb4]

The correct stats for the Nalimov DTM KBPKN EGT are attached.
Why did it happen? No idea, but the effect can be seen in the stats.
Counts of broken positions, wtm losses and btm wins were ok, of wtm wins and btm losses were too low, of draws were too high.
The correction came on request from Eugene and briefly appeared on Rob Hyatt's ftp site.

A principle is that data should be tested with any 'redundant' check that is available. That's what is happening with the current EN-MB x-check of EGT-file MD5sums.
A check, but not one that would have found the above error, is as previously stated: ensure that all index-positions (as advertised in the source code) are accounted for. Another test is to check the alignment of FEG.lof data with EN.tbs data, but only when at least one Pawn is present (as the counting methods are otherwise different). This error was picked up because the DTC stats didn't agree with the DTM ones.
g

Hi Guy,

Once again I thank you for your response. Listen do you think similar problems exist in any of the other tbs files Guy? I mena have they been checked out or is it something I can do with perhaps a utility? Just curious

Thanks,

Les

[guyhaw]

Actually, I am listening ...
There obviously could be errors in the programme that calculates Nalimov or FEG stats. It's a very long time ago now but I think that i've checked that:
a) all 3-to-5-man endgames have the same number of wins, losses, draws and broken in the DTC, DTM and DTZ EGTs
b) all the 6-man endgames have all their index-positions set to either win, draw, loss or broken and all are accounted for.
Maybe sometime, I'll check back to what I did.

If you would like to do something yourself, you could generate the 3-to-5-man FEG EGTs, and compare the stats with the EN stats for the P-endgames where the stats are comparable.
g