Home * Learning * Book Learning

Book Learning,
is a technique of unsupervised reinforcement learning motivated by basic requirements for successfully playing a sequence of games, such as avoiding losing games in the same way from certain lines from the opening book, or to recognize its own strong points and guide the engine into positions it does well [1].

Unsupervised reinforcement learning is characterized by trial and error and feeding back reinforcement signals, in this context to modify move probabilities or to add new book moves. The signals are either based on the final outcome of the game, called result-driven learning, and/or by search-driven learning, i.e. from the score and trend of first few searches after leaving the book to determine how the program likes the position [2].

Chess Engines

Crafty

Robert Hyatt on Book learning in Crafty [3]:
  • Crafty does two kinds of 'book learning'. Result learning sound just like what you are doing. If it loses a game, it won't play the last book move it played ever again. And this gets backed up several moves back as well to get it out of bad lines quickly.
  • Normal book learning uses the first ten non-book moves (search results) to determine the 'trend' of the game. This trend score is then used to adjust the book move that was played in this game...

Both are on by default...
and further [4]:
  1. Crafty remembers the evaluations for the first 10 moves out of book, after each search has been completed. It uses these evaluations to detect a "trend". IE is the evaluation good and getting better? Is it bad and getting worse? Is it good but dropping (ie it grabbed a gambit pawn and is beginning to see that it was bad) or is it bad but getting better (IE it offered a gambit, the opponent took it, and the score is going up). It factors all of that together and marks the book line as good or bad.
  2. Crafty takes the result of a game when it loses, and updates the book line so that moves tried near the end of the line simply don't get played, and alternatives near the front of the book line get tried next.
  3. There is more to it than that, and you can look at the crafty.doc file [5] to at least see what I am doing in more detail. It is very effective.

MChess

Marty Hirsch in Advances in Computer Games 9 on machine learning in MChess Pro, excerpt [6]
Some sets of heuristics are employed to decide which moves to add to the opening book, and which to delete from it. Moves are deleted when the score just out of book is not too high, and the score later in the game is worse. Moves are added when the score just out of book is not too low, and the score later in the game is satisfactory. A further pre-condition for moves that may be added to the opening book is that the increase in score has been mainly monotonic. If this condition is not met, a future game may enable better moves to be found due to the general learning. In other words, only the best moves the program is able to discover (at a given level) are added to the opening book. Different thresholds are applied for the storage and deletion of moves for White and Black.
...
Some programs now support a different type of book learning in which no moves are added to the book, but instead, unsuccessful moves are given a lower probability of selection. This approach narrows the book over the time and may optimize the results against a specific opponent, at the expense of the results against other opponents. The book learning described in this paper maintains the opening book variety and improves the book against multiple opponents.

RomiChess

As explained by Michael Sherwin, RomiChess applies Monkey see Monkey do mimicry to learn book lines [7] :
Romi remembers and incorporates winning lines regardless of which side played the moves into the opening book and can play them back instantly up to 180 ply if the stats for that line remain good.

See also


Selected Publications


Forum Posts

1998 ...

2000 ...

2010 ...


External Links


References

  1. ^ Learning Methods by Ed Schröder
  2. ^ Robert Hyatt (1999). Book Learning - a Methodology to Tune an Opening Book Automatically. ICCA Journal, Vol. 22, No. 1
  3. ^ Re: Book learning by Robert Hyatt, CCC, September 12, 1999
  4. ^ Re: questions about book learning by Robert Hyatt, CCC, January 03, 2004
  5. ^ Crafty Documentation
  6. ^ Marty Hirsch (2001). Machine Learning in MChess Professional - Advances in Computer Games 9, pp. 133-142, 135
  7. ^ Re: Positional learning by Michael Sherwin, CCC, December 18, 2010
  8. ^ Anybody tried Logistello's book learning for chess? by Rémi Coulom, CCC, December 29, 2013

What links here?


Up one Level