Home * Organizations * ICGA * Investigations * Rybka Controversy

Allegations have surfaced that Rybka 1.0 beta [1] and other versions of Rybka are derivatives of Fruit 2.1 and Crafty. If versions of Rybka are derived from Fruit and participated in ICGA tournaments, then Rybka has violated ICGA Tournament Rules. Fruit 2.1 [2] source code was distributed with a specific license. Part of this license reads "For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights." Allegations state that by distributing Rybka which is based on Fruit, this GNU license was violated. The ICGA rules state "Programming teams whose code is derived from or including game-playing code written by others must name all other authors, or the source of such code, in the details of their submission form.". No such notification was included in the Rybka entry forms (samples: Rybka Entry Form, ICGA Tournament, 2008 and Rybka 4 Entry Form 2010).

Timeline

Wegner

Zach Wegner analyzed the Rybka 1.0 beta exe file and noted many similarities in the two program's evaluation functions:

Letouzey

On Jan 23, 2011, Fabien Letouzey, author of Fruit, sent a letter to Tord Romstad about Strelka and Rybka [3]

Watkins

Mark Watkins also analysed Rybka 1.0 beta and details even more similarities:

Levy

On February 19, 2011 ICGA president David Levy broached on cloning issues in a ChessVibes column [4].

Programmers

In March 2011, following computer chess programmers signed an open letter to David Levy, Jaap van den Herik and the Board of the ICGA, to support the claim Rybka 1.0 beta and subsequent versions were allegedly derivatives from Fabien Letouzey’s program Fruit 2.1:

The letter was published in the private ICGA Investigations Wiki [7] and elsewhere [8] [9]. The German sites Heise online [10] and Spiegel Online broached the issue [11].

Pre-History of Rybka

2003

About March or April 2003, Rajlich starts work on Rybka (see interview by Nelson Hernandez).

2004

In January, Rybka competed in CCT6, the largest online author-based tournament (a posting of Rajlich's from time gives a "Rybka 1.3" as a version number). In April, Rybka 1.5 enters the privately-run author-based tournaments Chess War V and Le Systeme du Saison. In October(?), Rybka 1.6 enters Chess War VI and AEGT 3. Rybka 1.6 also competed in Chess War VII.

There is ample evidence that the latter two versions contained large amounts of Crafty code, in direct contravention of both the relevant Crafty license and the rules of these tournaments. Based on the last tournament pre-Rybka played in, its rating was about 2064 ELO [12].

Fruit 2.1

Fruit 2.1 was released in June 2005. For the purposes of prima facie evidence, it is necessary to establish that Rajlich obtained a copy. This can be inferred from his own words: I went through the Fruit 2.1 source code forwards and backwards and took many things.

History of Rybka 1.0 and beyond

2005

  • Rybka 1.0 Beta is released Dec 5 as a free download, and after a week the "Rybka 1" series becomes available on a subscription basis. About 13 "beta" versions of Rybka 1.0.1 were released over the next few months. Rybka 1.0.1-Beta-(x)? competed (as Rybka beta) and won first place in Paderborn (December 27 2005) [13].
  • The first commercial version, Rybka 1 end of 2005.

2006

  • On June 10 (date from RybkaChess FAQ about Rybka 2.x), the MP-only Rybka 2.0 Beta is released. The date on various executables are Apr 23 (Beta 8) and May 17 (Beta 4).
  • Rybka (under the name Rajlich) competed in the 2006 ICGA Chess Tournament [14]
  • Rybka 2.1 was released Jul 26, 2006.
  • Rybka 2.2 was released Nov 10, 2006.
  • Rybka 2.2n2 was released Dec 14, 2006. All these dates are from the RybkaChess FAQ about Rybka 2.x (though that page seems to disappear every now and then).

2007

  • Rybka 2.3 was released in late Feb 2007 (?). There was an "LK" version that had minor differences.
  • A few more "2.3.1 beta" versions also appeared around this time (again including Kaufman's latest material imbalance numbers). The Rybka 2.3 evaluation function appears to retain the numerology from Rybka 1.0 Beta. Rybka 2.3.1 starts to make a few changes (due to Kaufman?) in the numbers, though keeping the structure largely intact.
  • Rybka 2.3.2 was released Jun 13, 2007, and 2.3.2a about 5 days later.

Won ICGA Tournaments


Later Rybkas

There is still investigation to be done concerning what was in these later Rybkas.
The evaluation function in Rybka 2.3 (late Feb 2007, as above) looks to be that of Rybka 1.0 Beta (the executable contains many variants of the eval function, which seem to be either obfuscation or optimisation -- for instance, rook eval is dropped in one, so maybe if there are no rooks on the root position, this path would be taken).
Here is a raw dump of the eval, with annotations to be added [16]:

Evaluation

Here is my list of differences in eval between Rybka 2.3 (mid Feb 2007) and Rybka 1.0 Beta. As with some other Rybka versions, there are multiple "versions" of the eval function (e.g., some might ignore rooks) which might be due to some sort of optimisations (e.g. no rooks in root position).

  • The 1-3-5-10 material weighting is multiplied by 3717 rather than 3399.
  • The material imbalance adjustments are slightly perturbed. It seems that Rybka 2.2mp (Oct 31 2006) still had the same material imbalances as Rybka 1.0 Beta, but 2.2n2 (Dec 2 2006) differed (this version also is the first I can find to have 3717 rather than 3399).
  • lazy eval has some sort of "positional evaluation" incrementation (likely changed fairly early in the Rybka series).
  • The 3-centipawn tempo bonus was removed (not sure when - Rybka 2.1o had a 10-centipawn bonus, in some eval funcs).
  • PawnFile PST is now [-4,-1,0,+1,+1,0,-1,-4] rather than -3 on a/h files.
  • Drawishness from pawn files was added (already in Rybka 1.2, maybe earlier).
  • The opening/endgame interpolation is now linear.

Everything else (including numerology and computation methods) seems the same in Rybka 2.3 and Rybka 1.0 Beta.
There are minor differences in the SWAR-based popcnt method, which could be just a different choice of compiler. Similarly, part of the mobility computations with lazy-eval seems to be inlined.

I think it is safe to say that very little work was done on the Rybka eval from Dec 2005 to somewhere around late 2006 or early 2007 (when Kaufman was hired).

  • The evaluation function of Rybka 2.3.2a contains a few modifications, presumably due to Kaufman. Partially annotated version of Rybka 2.3.2a 64-bit evaluation function
  • This document aligns the evaluations of Rybka 1.0 beta and Rybka 2.3.2a, showing nearly identical evaluation features (with some minor additions and tunings to 2.3.2a):
  • Here is the "iterative deepening" part of the root search in Rybka 2.3.2a, to be compared to latter half of the function (starting with the easy move check) in Appendix A of the PDF:

Comparisons with other Engines

A detailed comparison of Rybka 1.0 and 2.3.2a versus other chess engines shows about a 1 in 10^12 or more chance of evaluation overlap occurring "randomly"


Claims and Evidence



Summary Report

The Summary Report on the Rybka Controversy prepared by the secretariat, can be found here:


This report has been sent by David Levy to Vasik Rajlich, asking him to provide comments. Panel members are reminded to keep this report private to allow Vasik Rajlich a chance to respond. It is expected sometime in the future this summary will be made public [17].

Note from the Summary Report concerning Rybka 3 Evaluation co-developer Larry Kaufman:
Note: Larry Kaufman is a Grandmaster long associated with helping programmers improve chess engines. There is absolutely no suspicion or belief on the part of the investigation panel or the ICGA that Kaufman has been in any way involved in any aspect of the copying behavior alleged of Vasik Rajlich. So far as the panel secretariat and the ICGA are concerned Kaufman’s reputation in this respect is squeaky clean

Decision concerning Rybka

The final decision on the Rybka case by the ICGA Board, June 28, 2011: Rybka disqualified and banned from World Computer Chess Championships [18] [19] [20].

Rybka Disqualified and Banned from World Computer Chess Championships

The International Computer Games Association (ICGA) has been conducting an investigation into allegations that, in the chess program Rybka, the programmer Vasik Rajlich plagiarized two other programs: Crafty and Fruit. The ICGA has considered and evaluated the evidence presented to the investigation panel and the report prepared by the panel’s Secretariat. (The report and evidence files are attached.) We would like to thank those members of the panel who contributed to this investigation and the Secretariat for the enormous amount of conscientious work they have put in to this matter.

By a unanimous 5-0 decision of executive members of the ICGA we find ourselves in agreement with the verdict of the Secretariat’s report. We are convinced that the evidence against Vasik Rajlich is both overwhelming in its volume and beyond reasonable question in its nature. Vasik Rajlich is guilty of plagiarizing the programs Crafty and Fruit, and has violated the ICGA’s tournament rules with respect to the World Computer Chess Championships in the years 2006, 2007, 2008, 2009 and 2010. Specifically, Vasik Rajlich, on all five occasions, violated Tournament Rule 2 which requires that:
Each program must be the original work of the entering developers. Programming teams whose code is derived from or including game-playing code written by others must name all other authors, or the source of such code, in their submission details. Programs which are discovered to be close derivatives of others (e.g., by playing nearly all moves the same), may be declared invalid by the Tournament Director after seeking expert advice. For this purpose a listing of all game-related code running on the system must be available on demand to the Tournament Director.

By claiming other programmers’ work as his own, and failing to comply with the above mentioned rule, Vasik Rajlich has unfairly been awarded one shared 2nd-3rd place (in 2006) and four World Computer Chess Championship titles (in 2007, 2008, 2009 and 2010). Furthermore, it seems to the ICGA that Vasik Rajlich clearly knew that he was in the wrong in doing so, since he has repeatedly denied plagiarizing the work of other programmers.

The ICGA regards Vasik Rajlich’s violation of the above mentioned rule as the most serious offence that a chess programmer and ICGA member can commit with respect to his peers and to the ICGA. During the course of the investigation and upon presentation of the Secretariat’s report Vasik Rajlich did not offer, despite repeated invitations from the ICGA to do so, any kind of defence to the allegations, or to the evidence, or to the Secretariat’s report, other than to claim in an e-mail to myself on May 13th 2011 that:
  • Rybka has does not “include game-playing code written by others”, aside from standard exceptions which wouldn’t count as ‘game-playing’.
  • The vague phrase “derived from game-playing code written by others” also does not in my view apply to Rybka.

The ICGA is of the view that such a serious offence deserves to be met with correspondingly serious sanctions against the perpetrator. In deciding on appropriate sanctions the ICGA has borne in mind the approach of the International Olympic Committee for dealing with the most serious cases of the violations of its rules.

The ICGA has therefore decided as follows:

  • Vasik Rajlich is hereby disqualified from the World Computer Chess Championships (WCCC) of 2006, 2007, 2008, 2009 and 2010.
  • The 2nd-3rd place awarded to the program called “Rajlich” in the 2006 WCCC is hereby annulled, sole 2nd place is awarded to the program Shredder, and 3rd place in that event is awarded to the program Zappa.
  • The 1st places and World Computer Chess Champion titles awarded to the program Rybka in the 2007, 2008, 2009 and 2010 WCCCs are hereby annulled, and all the other programs that competed in those events are moved up in the final tournament standings by one place. Thus the revised tournament standings and titles for those events will now be as follows.

2007


2008


2009


2010


In due course those programmers whose programs have been elevated to World Champion (or joint World Champion) status will receive from the ICGA replicas of the Shannon trophy for the appropriate years.

The plaques on the Shannon trophy that currently bear the name Rybka (for the years 2007-2010) will be removed from the trophy and new plaques will be engraved with the names of the revised winners of the title.

Similarly, the titles of World Computer Speed (Blitz) Chess Champion that were awarded to Rybka in 2009 and 2010 are hereby annulled. The revised winners of the speed chess title for those years are therefore:

Vasik Rajlich is banned for life from competing in the World Computer Chess Championship or any other event organized by or sanctioned by the ICGA.

The ICGA demands that Vasik Rajlich return to the ICGA the four replicas of the Shannon Trophy presented at the World Computer Chess Championships in 2007, 2008, 2009 and 2010, and to return to the ICGA all prize money awarded for Rybka’s performances in those events.

David Levy [President - ICGA]
June 28th 2011



Aftermath

After the ICGA decision, heated forum discussions arose. Chris Whittington [21] and Ed Schröder [22] [23] et al. criticize the ICGA investigation process, the work of the panel, and the ICGA decision. Sven Schüle [24], Miguel A. Ballicora [25] [26], Chris Whittington [27], Richard Vida [28], Mark Watkins [29] [30] and others elaborate and discuss Piece-Square Table matters [31].

CSVN Decision

In August 2011, the board of the Dutch Computer Chess Federation (CSVN) declared the most serious doubts as to the rightfulness of ICGA's decision. Therefore, they have chosen not to abide by their sanctions against Rybka [32] [33]:
Those who were in favour of the sanctions were severely questioned by (e.g.) Miguel A. Ballicora, George Speight and Søren Riis. Their opposition did make an impression on us, because these people can rely upon a vast expertise in the field of chess programming, law and mathematical logic. When finally dutchman Ed Schröder, former world computer chess champion, joined the aforementioned critics of ICGA, we no longer seemed to have a choice. [34]

Open Letter

At September 21, 2001, 15 former participants (14 program authors), Amir Ban, Don Dailey, Robert Hyatt, Gerd Isenberg, Marcel van Kervinck, Stefan Meyer-Kahlen, Fabien Letouzey, Thomas Mayer, Daniel Mehrmann, Gian-Carlo Pascutto, Richard Pijl, Ralf Schäfer, Mark Uniacke, Ben-Hur Carlos Vieira Langoni Júnior, and Harvey Williamson signed an open letter to the CSVN and its board - to protest against their decision to ignore the ICGA suggestion and to allow Rybka playing their tournaments, the Dutch Open Computer Chess Championship and the International CSVN Tournament, and declared they will no longer enter these tournaments under the current CSVN direction [35].

Different Views

In November 2011, Ed Schröder announced his new web-sites [36] [37] summarizing his alternative view of the historic ICGA - RYBKA event that led to the ban of Rybka and Vasik Rajlich. Short later, in January 2012, ChessBase, a Rybka distributer, took a stand, publishing an article series in four parts by Danish mathematician and Rybka Forum moderator Søren Riis on their site - A Gross Miscarriage of Justice in Computer Chess, also published as pdf [38].

Litmus Test?

Among other things matter of counter critic on Riis' article [39], the nasty >= 0.0 match from Rybka's integer based - with Fruit's float based time control code was declared a litmus test of the whole topic [40]. It was argued it could be a simple typo with only one additional trailing dot behind the zero. ‘0.’ already quantifies as float constant in C to force some compiler to emit the suspicious x87 instructions. Considering on how many places a chess program branches on the sign of an integer with >= 0 or < 0, it is peculiar that the typo happened exactly there, in that time control code where Fruit has ‘0.0’. However, Miguel A. Ballicora found the initial float condition was different in Rybka, not >= 0.0 but > 0.0, as published and mentioned by Ed Schröder in February 2012 [41] [42]. The different > 0.0 condition was missed by the ICGA investigation panel, and implies the one and only possible literal copied line of Fruit code was at least modified.

Rebuttal

At January 08, 2012 the ICGA responds with two rebuttal articles on Riis' A Gross Miscarriage of Justice in Computer Chess, with an introduction letter from David Levy, President of the ICGA [43] [44] [45]:
This week Dr Soren Riis posted a four part article on Chessbase.com in a somewhat belated attempt to defend Vasik Rajlich. Regular readers will recall that Rajlich's actions led the International Computer Games Association (ICGA) to find him guilty of breaking a crucial tournament rule, as a result of which the chess program Rybka was stripped of its World
Computer Chess Champion titles and Rajlich was banned for life from ICGA events.


Dr Riis gave his article a title to suggest that a miscarriage of justice has taken place. The ICGA now responds with two rebuttal articles, one by myself [46] (as President of the ICGA) concerning some of the legalistic points raised by Riis's article, and a robust detailed and thorough technical rebuttal by Mark Watkins [47].

Everyone interested in the Rybka scandal should read both of these rebuttals and then decide for themselves who and what they believe - Dr Riis or the ICGA's investigation report.

Dr David Levy
President - ICGA


FIDE

In early 2012, the FIDE Ethics Commission [48] received a complaint signed by Vasik Rajlich, co-signed by Søren Riis, Chris Whittington and Ed Schröder, against the ICGA. As announced at July 01, 2012, additional information is requested and the FIDE [49]. Secretariat will communicate the decision to the complainants. The decision was made in April 2014. Parties are not allowed to publish the judgement as a whole, but may quote from it [50] [51].

See also


Publications


Forum Posts

[55]

2011

2012

2013

2015


External Links


References

  1. ^ Rybka 1.0 beta
  2. ^ Fruit 2.1
  3. ^ Fabien's open letter to the community by Tord Romstad, CCC, January 23, 2011
  4. ^ Attack of the clones : ChessVibes by David Levy, 19 February, 2011
  5. ^ In retrospect by Rebel, Rybka Forum, July 14, 2011
  6. ^ Apology to Vas by Rebel, Rybka Forum, August 17, 2011
  7. ^ ICGA Investigations Wiki
  8. ^ Programmers write open letter about Rybka-Fruit issue : ChessVibes by Peter Doggers, March 01, 2011
  9. ^ Programmers Open Letter to ICGA on Rybka/Fruit by BB+, OpenChess Forum, March 01, 2011
  10. ^ Plagiatsvorwurf gegen Computerschach-Weltmeister, Heise online, March 01, 2011 (German)
  11. ^ Schach-Software Rybka - Programmierer vermuten Intelligenzklau by Frank Patalong, Spiegel Online, March 03, 2011 (German)
  12. ^ ChessWar VII C 40m/20' - Final standings after round 11
  13. ^ Rybka Wins Paderborn 2005 !!! - Topic Powered by Social Strata
  14. ^ 14th World Computer Chess Championship - Turin 2006 (ICGA Tournaments)
  15. ^ WCCC 2010 program (pdf)
  16. ^ History of eval changes in Rybka versions by BB+, OpenChess Forum, March 23, 2011
  17. ^ Statement from May 12, 2011, public since June 28, 2011
  18. ^ Rybka disqualified and banned from World Computer Chess Championships | ChessVibes by Peter Doggers, June 29, 2011
  19. ^ The ICGA Process by BB+, OpenChess Forum, July 01, 2011
  20. ^ ICGA Home Page
  21. ^ Minority Report Plagiarism - Quantifying Evaluation Features by Trotsky, Rybka Forum, July 09, 2011
  22. ^ In retrospect by Rebel, Rybka Forum, July 14, 2011
  23. ^ Apology to Vas by Rebel, Rybka Forum, August 17, 2011
  24. ^ Re: Crafty accused of copying Fruit PST by Sven Schüle, CCC, August 15, 2011
  25. ^ PST Calculation by michiguel, Rybka Forum, August 5, 2011
  26. ^ Take the Fruit Bishop tables... by michiguel, Rybka Forum, August 16, 2011
  27. ^ A small detail about additive constants to PSTs by Trotsky, Rybka Forum, August 13, 2011
  28. ^ About PSQ tables by Richard Vida, Rybka Forum, August 15, 2011
  29. ^ Fruit 1.0 PST by BB+, OpenChess Forum, August 13, 2011
  30. ^ PST of Fruit 2.1 and Rybka 1.0 Beta by BB+, OpenChess Forum, August 22, 2011
  31. ^ PST of Fruit 2.1 and Rybka 1.0 Beta by Ed Schröder
  32. ^ Rybka and the ICGA, CSVN Computerschaak
  33. ^ Rybka to play in Leiden? by Graham Banks, CCC, August 23, 2011
  34. ^ Apology to Vas by Rebel, Rybka Forum, August 17, 2011
  35. ^ Fwd: Open letter to the CSVN by Marcel van Kervinck, CCC, September 21, 2011, on behalf of the signees
  36. ^ Rybka vs ICGA a different view by Ed Schröder, November 09, 2011
  37. ^ Rybka evidence by Ed Schröder, November 09, 2011
  38. ^ Søren Riis (2012). A Gross Miscarriage of Justice in Computer Chess. pdf hosted by ChessBase
  39. ^ Re: Hyatt Is Gone! by BB+ OpenChess Forum, Jan 03, 2012
  40. ^ A Gross Miscarriage of Justice in Computer Chess (part three) by Søren Riis, ChessBase News, January 04, 2012 » Dots amazing: the case of the errant ‘0.0’
  41. ^ Re: Rybka 1.0 source code by Ed Schröder, CCC, February 12, 2012
  42. ^ Did Vasik really copied Fruit's time control ?
  43. ^ ICGA response to ChessBase 4-part article by Robert Hyatt, CCC, January 08, 2012
  44. ^ Controversy over Rybka's disqualification and ban (UPDATE) | ChessVibes by Peter Doggers, January 08, 2012
  45. ^ Feedback on the ICGA/Rybka disqualification scandal, ChessBase News, January 13, 2012
  46. ^ David Levy (2012). No Miscarriage of Justice - Just Biased Reporting. doc
  47. ^ Mark Watkins (2012). A Critical Aanalysis of the four parts of Riis. pdf
  48. ^ Roberto Rivello: How the FIDE Ethics Commission works, ChessBase News, May 11, 2007
  49. ^ FIDE Ethics Commission ... during the meeting held in Lausanne on 30th June – 1st July 2012, assumed the following ... case 2/2012 (pdf)
  50. ^ Initiative in Stockfish vs. Crafty by Trotsky, Rybka Forum, April 07, 2015
    Bob, 2015-04-20: FIDE has said this ...
    Trotsky, 2015-04-20: Strange. Do we have the same copy of the FIDE EC decision?
  51. ^ Complaint to FIFE Ethics Commission by Lukas Cimiotti, Rybka Forum, April 24, 2015
  52. ^ Feb 12 version: Rybka 1.0 Beta / Fruit 2.1 document by BB+, OpenChess Forum, February 12, 2011
  53. ^ Rybka evidence recapitulation by BB+, OpenChess Forum, January 03, 2012
  54. ^ A Pairwise Comparison of Chess Engine Move Selections by Adam Hair, hosted by Ed Schröder
  55. ^ some Rybka Forum posts seem to require registration
  56. ^ Voir dire from Wikipedia
  57. ^ The Rybka Investigation: A Survey Of Evidence From Both Perspectives by Adam Hair, January 2012, hosted by Ed Schröder
  58. ^ Feedback on the ICGA/Rybka disqualification scandal, ChessBase News, January 13, 2012
  59. ^ Fissure opens in chess AI scene - Boing Boing by Rob Beschizza, January 19, 2012
  60. ^ Andrew Dalke's Dairy archive
  61. ^ Vasik Rajlich responds to his accusers by Nelson Hernandez, Rybka Forum, July 12, 2011
  62. ^ Søren Riis (2012). A Gross Miscarriage of Justice in Computer Chess. pdf hosted by ChessBase
  63. ^ CB: Feedback on the ICGA/Rybka disqualification scandal by Ted Summers, CCC, January 13, 2012
  64. ^ "Fissure opens in chess AI scene" (Rob Beschizza/B by Julien Marcel, CCC, January 20, 2012

What links here?


Up one level