MD5sum of the entire library

[jshriver]

Anyone have the a md5sum for the entire 6man library?

-Josh

[Kirill Kryukov]

Hi Josh! Just take combined md5 files for 33, 33p, 42 and 42p and merge them together. Also check this thread.

MD5 for 3-4-5 men are available too at the project page.

[jshriver]

Hi Josh! Just take combined md5 files for 33, 33p, 42 and 42p and merge them together. Also check this thread.

MD5 for 3-4-5 men are available too at the project page.

Does the list on your page also contain the new 16 sets? Thanks for the reply grabbing them now had forgotten about the links. Will cat them together and if need be add the missing 16 later.

-Josh

[Kirill Kryukov]

Does the list on your page also contain the new 16 sets? Thanks for the reply grabbing them now had forgotten about the links. Will cat them together and if need be add the missing 16 later.

-Josh

Yes, they contain MD5 for all last 16 sets. Always welcome.

[jshriver]

Does the list on your page also contain the new 16 sets? Thanks for the reply grabbing them now had forgotten about the links. Will cat them together and if need be add the missing 16 later.

-Josh

Yes, they contain MD5 for all last 16 sets. Always welcome.

Thanks! I started working on my own Whilhelm like program, and using md5sums for initial verification. Know it might sound kinda pointless but I'm designing the code to be portable using linux as my main desktop and using the Qt tools.. hoping to make it so it can be used on Linux, OS X, and Windows.

I really like Whilhelm as I've tested it under Windows, but since I use Linux 99.99% of the time, it desperately need a similar tool.

Any comments, praises or flames are appreciated I plan to release it under the GPL.

-Josh

[Kirill Kryukov]

It's not pointless at all to make open-source software. Someone (like me) will read the source and learn something.

As a side idea.. Have you considered possibility of making a web-site for EGTB analysis? I mean server-side program that will not only query the given position, but perform some search using an engine. Search may be depth-limited or time-limited, to save server resources. Too many requests should be collected in a queue and computed gradually one by one, etc.. If I had time I would do this myself.. Still may be I'll be able to help with something. If you develop anything like this I may install it on my machine which has 900 GB now and it will be very good service for those who don't want to download the tablebase. Existing web-interfaces allow to query just one position, but it is EGTB-assisted search which is most exciting part of using tablebases.. Would be good to hear your thoughts about it.

[Tulean]

Thanks! I started working on my own Whilhelm like program...
I really like Whilhelm...

Any comments, praises or flames are appreciated

Hi Josh! Good luck with a new program!
I like Wilhelm too.
It is especially good for finding studies through its UMS algorithm.
To be more powerful (for study mining) Wilhelm needs:
1) 'Tree UMS' - to show not only one of longest lines, but all of UMS lines possible,
2) 'Advanced UMS' for wins - to spot out repetitions of moves.
For example wKh1, Rb1, bKa8, Bb4.
Wilhelm stops because of 'dual' 1.Rxb4 and 1.Ra1+
But after 1.Ra1+ Kb7 2.Rb1! Ka8! White has no progress...
Any ideas?
- Best regards

[jshriver]

Hi Josh! Good luck with a new program!
I like Wilhelm too.
It is especially good for finding studies through its UMS algorithm.
To be more powerful (for study mining) Wilhelm needs:
1) 'Tree UMS' - to show not only one of longest lines, but all of UMS lines possible,
2) 'Advanced UMS' for wins - to spot out repetitions of moves.
For example wKh1, Rb1, bKa8, Bb4.
Wilhelm stops because of 'dual' 1.Rxb4 and 1.Ra1+
But after 1.Ra1+ Kb7 2.Rb1! Ka8! White has no progress...
Any ideas?
- Best regards

I just read the Wilhelm readme and UMS does look a nice feature.
One of the goals of my project is to setup a framework for analysis and data mining. I'll post more as time permits..

Also if anyone else has some things on their wish-list, feel free to make recommendations.

-Josh

[jshriver]

As a side idea.. Have you considered possibility of making a web-site for EGTB analysis? I mean server-side program that will not only query the given position, but perform some search using an engine.

I thought of doing something like that a year or so ago, but due to the large size of the 3-6men set the cost of hosting the entire thing is well beyond my reach.

Coding wise it's a fun project, just paying out of pocket to host such a beast is to much.

From a programming point of view I was developing a lot SOAP/XML RPC applications at the time, and thought it would be neat to have a webservice anyone could bind to and do a simple:

sendFEN()
getResponce().. and voila it would send out the position and get back the win/lose/draw and move sequence.

-Josh

[guyhaw]

I'm addressing your point about a search for 'Effective UMSs', EUMSs if you like, where White's moves are all unique _after_ the removal of move-wasting moves which allow Black to force a return to the current position.
Writing-up the algorithm now. Basically, the method is to take a correct EGT, and alter the value of the current wtm position from '1-0 win' to 'draw'. Of course, this is not then consistent with the values of its successor-positions. The 'demotion' of the position is then 'reflected back', resulting in other positions being set to draws - either because they allow a return to the current position, or just lead to cycles 'above' the current position'.
The EG 165.15958 'KQRKQR' study is a case in point. About 14 positions with dual-moves, all but two just allowing Black to force repetition. All but one of these dual moves are eliminated from consideration even before 'arriving' at the position at which they become a choice.
So, a server-side application that could be invoked by the client submitting a Chess Study to examine it for dual moves would be excellent.
The algorithm can also be used hypothetically in games where (say) White again is pursuing a win but the 50-move-rule may intervene. White would want to use any 'spare moves' to encourage the opponent to concede 'depth' so that the win is not threatened by the 50-move rule. This might involve 'diving down' to greater depths, like a fish trying to lose a hook, but the issue about not decreasing the depth with each move is that useless repetition might occur.
A 'Basin of Repetition' B(P) of a position P is the set of positions from which Black can force a return to P: note that P is not in B(P). Of course, the 'Basin of Repeition' B(S) of a set S of positions (previously visited in a game) is potentially greater than the union of the Basins for the individual positions. The 'doping' or alteration of the endgame table EGT can be done cumulatively to represent a past set of positions to be avoided.
Anyway, I'm playing around with the best way to write this up now. I'm at http://www.tinyurl.com/law6k if you want more info.
g

[Arpad Rusz]

A database with new metrics - DTE - Depth To End(of the study) - can be created.

All positions with duals have DTE=0 (there the study ends).
A mate position has DTE 0, a position with uniqe mating move has DTE 1 etc.
In a stalemate position we have DTE=0 (this could be the end of a stalemate study).
If the longest UMS from a given position(win or draw) has lenght of n then DTE=n.
A losing position with black to move has DTE=n if black has a move which leads to a position with DTE=n but it hasn't a move which leads to a position with higher depth( and the similar for a WTM position).

The best is if in the whole database the Effective UMS is used, so the pseudoduals have been eliminated.

Beside the DTE the win/draw/lose information has to be coded.

In general the positions with higher DTE have bigger chances the be useful for creating a study. They now can be easily found.
The solution for a given position can be written imediately even with the possible tries i.e. the moves with only one refutation.

The new database can be well compressed, given the very big number of positions with duals(all with DTE=0), and with very rare positions with DTE above 10.

[guyhaw]

No, I'm afraid your 'DTE' idea only works if the (E)UMSs are non-intersecting trees with their roots in the lesser depths and their leaves in the greater depths.
This is unfortunately not so, as Black may have a choice of moves to give White an (Essentially) Unique reply. An example would be useful to prove this but I assume it is so. The btm position would then potentially have two different depths in your scheme.
Nice try though: some way of managing the (E)UMSs has to be found.

Another point, associated with the chessic rather than graph nature of Chess Studies, is that attitudes to Duals vary across the Community and judgements as to their demerit are often made in the context of the whole study.
The famous (Barbier-)Saavedra study would be ignored by your elimination of all positions with more than one route 'down' to win. White's K is at b4 and needs to get to c2 to prevent Rc1. Both Kb3-Kc2 and Kc3-c2 win.
The PCCC Wageningen meeting (2006) proposes to emphasise that certain kinds of dual-move should be given little to no weight. Obviously, move-wasting moves (allowing repetition) are the least significant. But, somewhat to my surprise, they also propose to discount alternative paths 'down the hill to the win' which reconverge with the mainline. I think this is justifiable in many cases, but not with the same broad brush as they discount move-wasting moves.
Thus, I would not like to be denied the delights of the Saavedra study and others. Of course, the quality of the Study is affected by the clarity and completeness of the solution's expoisition, and I would like to see composers identifying all dual moves, even if they classify and discount them on some grounds later.
The EG 165.15958 KQRKQR study by Zhuravlyov has been unjustifiably pilloried by John Roycroft with a misbegotten critique as if a computer had done all the work of discovery [as if!]. However, it is clear - a minor niggle perhaps - that the author did not identify all the 'harmless' duals, even second time round.

I thnk it is also true that an (E)UMS does not guarantee a study. An EUMS may just end up with a win but not one that is obvious to the human enjoyer of studies. Take a line from any maxDTM position: in the end, the UMS terminates and maybe the EUMS does too.

Rafael Andrist did something withWILHELM to identify UMSs (but not EUMSs) in 3-to-5-man endgames. Helmut Conrady published some examples but I don't know if a thorough review was done. He was I think able to talk about the maximum-length of a UMS in an endgame.

There is no difficulty in identifying a position in which White has an Effectively Unique Move. That has (belatedly) become clear now. Even a position like Rinck's KRNRN study (#632 in '1234 Modern End-Game Studies) might be analysed to see if White has only move-wasting dual moves. The difficulty is in doing so efficiently so that a whole endgame may be similarly treated.

g

[Arpad Rusz]

Sorry Guy, I still think it works.
Don't you think that every position has a maximal length Unique Move Sequence? That length gives us the DTE for a position. Wilhelm gives exactly that number.
Or put it this way:
The black can choose between moves:
-a black move which leads to a position with more then one good white move is unimportant (cannot be part of the solution)
-every other move is important as it leads to positions with unique white moves(DTE>0). The maximum of this depths give us the DTE of that initial BTM position.
I think only the time losing duals must be incorporated in the database, as the other kind of duals(e.g. different path for a king/knight for reaching a square, interchange of two moves) are not universally accepted. The famous Saavedra study can still be found by its key move(c8R!!), and then working backwards ignoring the minor dual.
Can you post us the Zhuravlyov study?

[guyhaw]

Arpad: you have now redefined DTE with a couple of nudges as the maximal length of a UMS, presumably starting from the given position and measured in White's moves.
That's ok, because you have introduced the word 'maximal' and referred it to the graph-concept 'UMS' rather than to the artistic concept of a 'Chess Study'.
I'm pleased to hear that that is what WILHELM gives: despite my enthusiasm for WILHELM and UMSs, I haven't found out how to relate to that function in WILHELM yet so yr advice will be welcome.
In fact, DTE could be the maximal length of an EUMS, defining a move (as we have) to be 'effectively unique' if it is unique after all move-wasting moves (allowing forced repetition) are discounted.

Attitudes to 'alternative paths' converging with the main line downstream (or even worse, not converging) vary, and this is to be expected. I'm rather surprised that the PCCC is proposing to de-emphasise them as much as it appears to be doing.
Some move-choices (e.g. line-moves, triangulations) could be accommodated by a chess-move notation that says 'make any move from the list that is legal'. I'd certainly discount these duals but don't have an algorithm for doing so.
My pgn re the Zhuravlyov study (+ ignorable stuff) and my EGT-fueled analysis of the mainline attached. I hope you enjoy it as much as I did.
g

[Tulean]

The EG 165.15958 KQRKQR study by Zhuravlyov has been unjustifiably pilloried by John Roycroft with a misbegotten critique as if a computer had done all the work of discovery [as if!]. However, it is clear - a minor niggle perhaps - that the author did not identify all the 'harmless' duals, even second time round.
g

Good to see some words about my study

Of course all necessary analysis (including duals and time-wasting) was sent to the judge of the Sukharev MT and then to J.Roycroft for the EG.
But seems like composers have no consensus on that matter.

'Repetitions' and 'time-wasting':
In win studies, a move that leads to a repetition is not a 'dual' (in terms of chess composition Codex) - not an 'effective solution', White has no progress here...
Time-wasting (if it is not a repetition) is a dual. Both alternatives are effective, but aim is the same and one of ways is longer. Not a great defect if we can get 'aimed' position in 1-2 half-moves.
But if both ways are long enough to include different 'thematic' play?

My study has only two duals of this kind:
- 23.Qh4+? Rh2 24.Qe4+ Rg2 instead of 23.Qe4+ Rg2 and
- 37.Kc7? Qa5+ 38.Kc6 (38.Kb8! Qb5+) Qa6+ 39.Kc7 Qa5+ 40.Kb7 Qb5+ 41.Ka7 Qa5+ 42.Kb8 Qb5+ instead of 37.Kb8 Qb5+

Anyway, EUMS would be excellent thing for study composing!

- Andrey

[guyhaw]

Delighted to find you at this board, and congratulations on your study.
I agree that a move that allows forced-return to a previous position is not 'effective progress' -http://www.saunalahti.fi/~stniekat/pccc/codex.htm, Article 10 - but it might be regarded as 'effective' in that the win is retained, even if it is "artificial prolongation" of the solution, the 'type c' dual mentioned.
So, one might argue that such a dual is not a dual but I think it's a fine point which is difficult to support in an international multilanguage context, and anyway - the composer has to identify these moves for what they are and discount them.
Personally, all other things being equal (which they almost always are not), I would rather see an absolutely unique move than just an 'only way to advance move'. So I would give a move-wasting move a demerit-weighting of 'epsilon', i.e. greater than zero but arbitrarily small and certainly not a factor to discount a study. How much would we lose if it were not so - lots?!
Yes - I picked up the two alternative paths at moves 23 & 37, rather like taking the long route round a rock when walking down a mountain path. Not really significant. The 'plies to reconvergence in the main line' and 'extra length' might be parameters in the consideration of demerit, but chessic rather than graph considerations are probably more significant.
There are some matters which I would like to discuss offline (as they're rather out of this board's scope) if you would like to contact me via http://www.tinyurl.com/law6k
Thanks and congratulations again: best regards - g

[Kirill Kryukov]

This is what I call a derailed thread.. Regards to everyone, interesting read anyway.

[guyhaw]

I think the original destination was essentially and is now irrelevant. You might however like to put all MD5sums into one 'Data Assurance Certificate' file for automatic verification purposes, but we've had that conversation before.
So the thread is not so much derailed as being serendipitously rerouted by passenger action through more interesting pastures.

Just one footnote on the definitions of 'DTE': the (E)UMS potentially strides across changes-of-force, and so bootstraps like other DTx metrics from a 'successor endgame' to 'prior endgames'. It is not necessarily ground to zero within an endgame.
g

[Arpad Rusz]

Andrey, what do you think of this study? (see the pgn file!)
Can it be a joined composition?

[Tulean]

Arpad, especially good to see your added logic - Ra7-a8-a7. And different, 'non-ladder' way to rzz after 8.Kd3!
Do you have EG 165? Detailed analysis here...
Please note, G.Haworth's file above has analysis on only repetitions and time-wasting, not on 'study-thematic' content.
The variation 13...Qg3+ leads to strong 'thematic' 17...Rg7 and is slightly out of 'mainstream' - to seize the 'Dolgov's fortress' Kh7+Qh6+Rg6...
Anyway, I like your work very much!

[Arpad Rusz]

Sorry Andrey, I don't have the EG165.
I have realised that Guy's file contains only the main line and I am convinced that your study contains many interesting sidelines.
I didn't analysed very hard the positions after the reciprocal zugzwang(the anticipated part of the new study).

It is very interesting how I have composed this new study. One could think that I started with your study, but in fact a study by Dolgov(Nauka i zhizn, 1984) gave me the inspiration!! Here is the story:
After Guy posted your excellent study, I realized that I don't have the KQRKQR Nalimov database, so I downloaded it. When I analyzed some studies with this material, I have found Dolgov's study. That study has a dual on the 20th move(Qd5!) which could be followed by two chameleon-echo R sacrifices. This 'nice' dual contains other possible winning ways - so it is itself dualistic!
Trying to eliminate the duals (with the help of Wilhelm) I have found the following position:
6k1/4q1r1/8/8/8/1QK5/8/7R b - -

Here the wK blockes the possible Qg3+ after Rf7. To my biggest surprize the winning line after 1...Rf7 2.Rg1+ Kh8! contained your reciprocal zugzwang!

P.S. Can you post here your study with the full analyses?

[guyhaw]

Hw/sw in a pre-term state of flux here, and FRITZ is crashing on my new 2-cylinder Tosh laptop ... but I think I can probably scan the relevant pages of EG165 into a .pdf tomorrow if that helps.

My .pgn file was the result of a focus on the dual-moves (all unimportant as it happens) in response to some questions posed by AJR in his rant about this study. I assume it was AJR who protested a much higher award to this study: AZ has every right to feel robbed.
So the study was really about the move-graph rather than about the chess. The move-graph meets all my requirements for examples to illustrate the 'Scorched Earth' algorithm.
Also, it was not clear to me from the EG165 rendition of the study what had been submitted to the tourney, and what had been provided to AJR later.
g

[Tulean]

P.S. Can you post here your study with the full analyses?

I can make pgn file with English comments in a couple of hours. But who may explain how to post it here ready for download?
-Andrey

[guyhaw]

When you create your message, you get invited to attach files to it further down the form.
Alternatively, if you'd like to post the full .pgn to me, I'll file it on the CCRL board. I will not proceed with the scanning idea.
g

[Tulean]

I attached file with two versions of Sukharev MT study in new topic about Wilhelm's UMSs.
Definitely, we need to ask Kirill to move our posts there...