Testing on Linux

Questions and comments related to CCRL testing study
Post Reply
govert
Posts: 9
Joined: Tue Jan 13, 2009 7:42 am

Testing on Linux

Post by govert »

Would CCRL accept games played on linux?
I guess I would run Crafty 19.17 BH through wine to get the correct time settings...

/Martin
User avatar
Shaun
Posts: 6910
Joined: Sat May 13, 2006 3:24 pm
Sign-up code: 10159
Location: Brighton. UK

Re: Testing on Linux

Post by Shaun »

govert wrote:Would CCRL accept games played on linux?
I guess I would run Crafty 19.17 BH through wine to get the correct time settings...

/Martin
Hi Martin,

I am not familiar with wine - however I would expect a slow down due to an emulator but saying that I seem to remember people reporting no slow down?

The difficulty (although hopefully not impossible to overcome) would be to join games played under Windows and Unix e.g. you would need to know that the Windows and Unix compile of a number of engines performed almost identically on the same hardware.

e.g. Say (and I am just guessing here) - on a dual boot machine it could be proved that Windows and Unix Glaurung compiles performed almost identically we would then know that the Glaurung rating would be the same on either O/S - if on the same machine it could be proved that the Windows compile under wine was also as fast that would then be perfect.

Shaun
Ray
Posts: 22607
Joined: Sun Dec 18, 2005 6:33 pm
Sign-up code: 10159
Location: NZ

Re: Testing on Linux

Post by Ray »

WINE = Wine Is Not an Emulator

There should be no performance penalty with WINE at all

From WineHQ:
Windows programs running in Wine act as native programs would, running without the performance or memory usage penalties of an emulator, with a similar look and feel to other applications on your desktop.
govert
Posts: 9
Joined: Tue Jan 13, 2009 7:42 am

Re: Testing on Linux

Post by govert »

Yes I see that we would have to prove "equal behaviour" first.

Running a a chess engine windows binary with wine seems to perform at least as good as a vanilla gcc compile on linux in many cases.

How do you handle the fact that there are multiple windows compiles out there with different optimizations? Does the CCRL team choose a specific binary that all testers use?
User avatar
Shaun
Posts: 6910
Joined: Sat May 13, 2006 3:24 pm
Sign-up code: 10159
Location: Brighton. UK

Re: Testing on Linux

Post by Shaun »

govert wrote: How do you handle the fact that there are multiple windows compiles out there with different optimizations? Does the CCRL team choose a specific binary that all testers use?
Yes - typically the same compile version is used.
User avatar
Shaun
Posts: 6910
Joined: Sat May 13, 2006 3:24 pm
Sign-up code: 10159
Location: Brighton. UK

Re: Testing on Linux

Post by Shaun »

Ray wrote:WINE = Wine Is Not an Emulator

There should be no performance penalty with WINE at all

From WineHQ:
Windows programs running in Wine act as native programs would, running without the performance or memory usage penalties of an emulator, with a similar look and feel to other applications on your desktop.
Looking into Wine is on my list - I have a long list of questions and just have not looked into it.

Shaun
Ray
Posts: 22607
Joined: Sun Dec 18, 2005 6:33 pm
Sign-up code: 10159
Location: NZ

Re: Testing on Linux

Post by Ray »

govert wrote: How do you handle the fact that there are multiple windows compiles out there with different optimizations? Does the CCRL team choose a specific binary that all testers use?
99% of engines only have one compile, ignoring the 32-bit and 64-bit versions which we always show separately
Ray
Posts: 22607
Joined: Sun Dec 18, 2005 6:33 pm
Sign-up code: 10159
Location: NZ

Re: Testing on Linux

Post by Ray »

In a lot of cases the differences between the compiles is small - e.g. JA and DM compiles are probably not that much different usually. A slight speed variance isn't a problem - the way we use the Crafty bench to equalise various hardware is not exact anyway, different engines may perform slightly differently on the various architectures from AMD/Intel.

On the other hand, sometimes the difference is big. See MicroMax:

http://computerchess.org.uk/ccrl/4040/c ... +opponents

The Denis P. Mendoza Intel C compiler version is approximately 1.8 times faster than the original Cygwin/gcc build by H.G. - and you can see that resulted in an approx +35 ELO gain. So in that case, it was clearly important that the two were kept separate.
govert
Posts: 9
Joined: Tue Jan 13, 2009 7:42 am

Re: Testing on Linux

Post by govert »

btw, what does DM & JA stand for?
User avatar
Graham Banks
Posts: 26919
Joined: Sun Dec 18, 2005 5:47 pm
Sign-up code: 0
Location: Auckland, NZ

Re: Testing on Linux

Post by Graham Banks »

govert wrote:btw, what does DM & JA stand for?
Dennis Mendoza and Jim Ablett.
Post Reply