Toga Returns : Special Edition ... Need Help ...

Questions and comments related to CCRL testing study
Post Reply
Tennison
Posts: 6
Joined: Fri Dec 28, 2012 2:23 pm
Sign-up code: 10159

Toga Returns : Special Edition ... Need Help ...

Post by Tennison »

Hi everybody,

I'm actually trying to realize a special edition of Toga Returns : a Sparring Partner for Human and for Engine makers ... In fact I saw some people needs engines to test their own.

My goal is to realize a "human like blunder maker" lol.
I want a realistic 1400 engine, 2000 engine, 2600 engine, ...
So I'm trying different methods. Actually I have a formula that seems pretty good.

To activate this option : simply choose level between 1400 and 2800.
[In LittleBlitzer GUI, simple add "Level=xxxx" in the engines.lbe]

Actually, in self testing, here is results (in 10ms/move games) :

Code: Select all

Games Completed = 60000 of 60000 (Avg game length = 2.538 sec)
Settings = RR/128MB/10ms per move/M 600cp for 12 moves, D 500 moves/EPD:8moves.epd(3710)
Time = 25458 sec elapsed, 0 sec remaining
 1.  Toga Returns 1.1                 	    6816.5/7500	6602-469-429  	(L: m=172 t=0 i=0 a=297)	(D: r=307 i=79 f=40 s=3 a=0)	(tpm=29.9 d=7.70 nps=0)
 2.  Sparring Partner 2800                	6681.5/7500	6441-578-481  	(L: m=235 t=0 i=1 a=342)	(D: r=334 i=88 f=56 s=3 a=0)	(tpm=30.6 d=7.36 nps=0)
 3.  Sparring Partner 2700                	6441.0/7500	6185-803-512  	(L: m=329 t=0 i=0 a=474)	(D: r=357 i=84 f=66 s=5 a=0)	(tpm=30.6 d=7.33 nps=0)
 4.  Sparring Partner 2600                	5983.0/7500	5733-1267-500  	(L: m=533 t=0 i=0 a=734)	(D: r=343 i=89 f=62 s=6 a=0)	(tpm=30.6 d=7.23 nps=0)
 5.  Sparring Partner 2500                	5531.5/7500	5307-1744-449  	(L: m=772 t=0 i=0 a=972)	(D: r=293 i=87 f=67 s=2 a=0)	(tpm=30.6 d=7.16 nps=0)
 6.  Sparring Partner 2400                	4980.5/7500	4771-2310-419  	(L: m=1105 t=0 i=0 a=1205)	(D: r=295 i=62 f=58 s=4 a=0)	(tpm=30.6 d=7.06 nps=0)
 7.  Sparring Partner 2300                	4479.5/7500	4278-2819-403  	(L: m=1561 t=0 i=0 a=1258)	(D: r=282 i=61 f=55 s=5 a=0)	(tpm=30.6 d=7.04 nps=0)
 8.  Sparring Partner 2200                	3938.0/7500	3765-3389-346  	(L: m=1959 t=0 i=0 a=1430)	(D: r=236 i=68 f=41 s=1 a=0)	(tpm=30.5 d=6.97 nps=0)
 9.  Sparring Partner 2100                	3401.0/7500	3228-3926-346  	(L: m=2534 t=0 i=0 a=1392)	(D: r=228 i=57 f=57 s=4 a=0)	(tpm=30.5 d=6.94 nps=0)
10.  Sparring Partner 2000                	2920.5/7500	2744-4403-353  	(L: m=3079 t=0 i=0 a=1324)	(D: r=248 i=56 f=45 s=4 a=0)	(tpm=30.5 d=6.92 nps=0)
11.  Sparring Partner 1900                	2459.0/7500	2305-4887-308  	(L: m=3685 t=0 i=0 a=1202)	(D: r=210 i=44 f=49 s=5 a=0)	(tpm=30.5 d=6.90 nps=0)
12.  Sparring Partner 1800                	2041.5/7500	1913-5330-257  	(L: m=4237 t=0 i=0 a=1093)	(D: r=185 i=35 f=33 s=4 a=0)	(tpm=30.5 d=6.93 nps=0)
13.  Sparring Partner 1700                	1604.0/7500	1480-5772-248  	(L: m=4793 t=0 i=0 a=979)	(D: r=168 i=41 f=38 s=1 a=0)	(tpm=30.5 d=6.92 nps=0)
14.  Sparring Partner 1600                	1186.5/7500	1088-6215-197  	(L: m=5377 t=0 i=0 a=838)	(D: r=138 i=30 f=27 s=2 a=0)	(tpm=30.6 d=6.93 nps=0)
15.  Sparring Partner 1500                	 907.0/7500	 822-6508-170  	(L: m=5842 t=0 i=0 a=666)	(D: r=124 i=28 f=16 s=2 a=0)	(tpm=30.6 d=6.94 nps=0)
16.  Sparring Partner 1400                	 629.0/7500	 572-6814-114  	(L: m=6207 t=0 i=0 a=607)	(D: r=86 i=11 f=16 s=1 a=0)	(tpm=30.6 d=6.99 nps=0)
And the ratings with bayeselo (offset set to Sparring Partner 2800 as 2800):

Code: Select all

Rank Name                    Elo    +    - games score oppo. draws 
   1 Toga Returns 1.1       2840   13   13  7542   91%  2107    6% 
   2 Sparring Partner 2800  2800   12   12  7486   89%  2110    6% 
   3 Sparring Partner 2700  2738   12   12  7486   86%  2114    7% 
   4 Sparring Partner 2600  2630   11   11  7495   80%  2121    7% 
   5 Sparring Partner 2500  2529   11   11  7500   74%  2128    6% 
   6 Sparring Partner 2400  2410   11   11  7500   66%  2136    6% 
   7 Sparring Partner 2300  2301   11   11  7499   60%  2143    5% 
   8 Sparring Partner 2200  2189   11   11  7500   53%  2151    5% 
   9 Sparring Partner 2100  2077   11   10  7500   45%  2158    5% 
  10 Sparring Partner 2000  1979   10   11  7500   39%  2165    5% 
  11 Sparring Partner 1900  1882   11   11  7500   33%  2171    4% 
  12 Sparring Partner 1800  1795   11   11  7500   27%  2177    3% 
  13 Sparring Partner 1700  1703   11   11  7500   21%  2183    3% 
  14 Sparring Partner 1600  1602   12   12  7500   16%  2190    3% 
  15 Sparring Partner 1500  1530   12   12  7500   12%  2195    2% 
  16 Sparring Partner 1400  1443   14   14  7490    8%  2201    2% 
As you can see, with this formula, ratings seems not so far from the goal.

But it's with self play and in 10ms/move.

I need testers to help me to verify all this :

- tests against other engines
- tests with longer time controls
- tests the playing style of the different "partners" : to see if realistic ones.


If you are interested in testing this, I can send you the bin file (for windows only for this time, only 1 core for now). I don't give the src now : I'll distribute this as soon as tests are goods and the formula seems the best one (realistic and well tune partners). It's why I don't put here an online version. I prefer sending this to testers accepting my point of view.

Thanks a lot.

Benoît
Post Reply