Komodo,
a chess engine by primary author Don Dailey (died at the age of 57 on November 22, 2013) and since October 2013, by Mark Lefler, supported by chess advisor and evaluation expert and Don's long time collaborator Larry Kaufman. In July 2016, Komodo 10.x won the Shannon Trophy.
Komodo appeared in January 2010 derived from Don's former engine Doch. It uses bitboards as internal board representation, and has a sophisticated search and a knowledge based, balanced evaluation. Komodo is a standalone chess engine supporting the UCI protocol and is available for multiple platforms and operating systems, and is therefore compatible with free and commercial UCI compliant chess graphical user interfaces and database front ends.
As of December 2011, Komodo became commercial with version 4[1], earlier versions running under Windows, Linux, Mac OS and Android are available from the Komodo download site, free for non-commercial use [2]. Next release date was May 2013, with CCT15 winner Komodo CCT[3].
Komodo 5.1 MP, released in June 2013, was a major rewrite in C++11, able to perform a parallel search to run on multiple processors [5] .
Komodo 6
In October 2013, Don Dailey announced the release of Komodo 6 and also bad news concerning the future status of Komodo due to his fatal illness of a acute form of Leukemia[6] . He introduced Mark Lefler as new member of the Komodo team [7][8] . Don Dailey died November 22, 2013 at age 57, just about the same time that Komodo pulled ahead of Stockfish by winning game 2 in the TCEC Season 5 final[9][10] .
Komodo TCEC and 7
As winner of the TCEC 5 final, Komodo TCEC was released on December 04, 2013 [11], the further improved Komodo 7 on May 21, 2014, during the TCEC Season 6 final versus Stockfish again, now supporting Syzygy Bases[12], probing code interface implementation by Jeremy Bernstein[13].
Komodo 8
Komodo 8, released on September 05, 2014, is a substantial improvement over Komodo 7a, which was already rated among the top three chess engines on almost all rating lists [14]. Komodo 8 supports Syzygy Bases and is available for Windows, Linux, Mac OS and Android, the latter immediately available for import in Chess for Android and in Chess for Glass[15]. Since mid September 2014, Komodo 8 for Windows is available as ChessBase engine, bundled with the Deep Fritz interface, Syzygy bases not included [16].
Komodo 9
Komodo 9, released on April 28, 2015, continues the progress with approximately +50 Elo over Komodo 8 due to improved search, evaluation, and time management, as well as more efficient use of multiprocessing [17]. Most noticeable is the increase in search depth reached in a given amount of time, which is due to better extension, reduction, and pruning rules [18]. Further, Komodo 9 now features Chess960 and a persistant hash to save and restore analysis.
On May 23, 2017, Komodo 11.01 was released, which corrected some minor issues of Komodo 11, which was already released by ChessBase a few days before. It is estimated about 55 Elo stronger than Komodo 10 at intermediate time controls, gains related to evaluation[20], revised quiescence search, singular extensions, LMR , and parallel search[21] . Komodo 11.2, released on July 17, 2017, was the version which won the WCCC 2017[22]. Komodo 11.3 was released on March 04, 2018 with subsequent improvements [23]. It also comes with a BMI2 compile for compatible x86-64 processors - so far not recommend for AMDRyzen due to its slow PEXT implementation [24].
Komodo had its first tournament success in 2013, when it won the CCT15 with 6½/7, and later that year the TCEC Season 5 after qualifying for the Superfinal and 48 thrilling games versus Stockfish by a margin of 25 - 23. In 2014, Komodo won the CCT16 with 5½/6. In May 2014, at TCEC Season 6, Komodo 7 qualified for the Superfinal - again versus Stockfish - both engines further improved and supporting Syzygy Bases. This time, Stockfish won the 64 game match 35½ - 28½. Again, the other way around at TCEC Season 7 - Superfinal in December 2014 - version 8 successor Komodo 1333 won from Stockfish 141214 with 33½ - 30½.
WCCC / WCSC / WCRCC 2015
At the WCCC 2015 in Leiden, Komodo 9.x won the Blitz tournament, and became runner-up behind and losing from the massively parallel and distributed Jonny which played on a cluster of 100x24 cores. After the revenge winning against Jonny at the equal hardware WCSC 2015 a few days later, Komodo was unlucky and made too many draws against apparently weaker opponents [25], to became third behind Shredder and Ginkgo. Two weeks later, Komodo won the WCRCC 2015 convincingly with 9/9!
In Fall 2015, Komodo 9.3x dominated the TCEC Season 8 and won the Superfinal again versus Stockfish, this time 100 games with +9, =89, -2, that is 53½ - 46½ [27].
In 2017, Komodo could defend it titles at the WCRCC 2017 online, and the soon later released Komodo 11.2 the ICGAWCCC 2017 over the board, again in a play-off versus Jonny. Komodo further became runner-up at the uniform hardware WCSC 2017 behind Shredder.
Komodo never looks at rook or bishop under promotions and only looks at knight under promotions when it gives check. But even for the knight case there are positions where my rule won't find the right move.
Don, Your program is kind of a beancounter that relies heavily positional upon piece square tables.
You could not be more incorrect. We have piece square tables in Komodo that are a very minor part of Komodo and get very small weights but Komodo puts a huge emphasis on getting the evaluation right with hundreds of evaluation terms that are carefully balanced to produce a find positional chess program. I personally believe that Komodo has the best evaluation function of any chess program in the world. ... Total amount of knowledge is not what is important, it's the QUALITY of the knowledge that is. Komodo tries to have the most balanced evaluation function possible, and probably much more knowledge than most programs. I don't believe in quantity but Komodo evolved to have a lot of knowledge anyway - probably because every piece of knowledge in Komodo is needed. I tried removing knowledge in Komodo to get speed and we were not able to remove ANYTHING without noticing that it weakened Komodo, at least slightly.
Komodo pushes the selectivity pretty hard and we DO get a decent nodes per second increase with Lazy evaluation but the problem is that we get a big increase in nodes. It is this way because we assume we will not make scout if the guesstimate is too low. So we miss some of the beta cutoffs, you cannot have your cake and eat it too. Komodo's positional component can vary enormously so we do take a lot of damage positionally when we use lazy margins.
As Larry mentioned we are counting nodes differently. More specifically, both versions have always counted a node when a move is made, but Komodo 3 would count nodes that were forward pruned (if the move was actually executed) and Komodo 4 does not. In Komodo 3 I figured that if any work was done (such as attack testing after making the move) it should count as a node. But in Komodo 4 we changed this because the move is technically being pruned (even if we had to check it out a bit first.) Of course if a move is searched or reduced it is counted in both versions.
It's mostly semantics, but we get more stable node counts and this works better for fixed node testing which we sometimes do.
I have several things in mind for getting serious speedups in future versions of Komodo which will bring the speed of Komodo back up to that of other programs, or at least not as far away. I believe Komodo is one of the slower programs in terms of raw search speed as we have put a really heavy focus on having a smart program and little on heavy optimization. The reasoning is the same for delaying the MP stuff too, we want to start with a really strong program before we play the optimization games.
How you approach a problem is strongly related to how you DEFINE a problem or task. The goal of MP is to make the program play as well as possible using more cores, to take advantage of the hardware as fully as possible and the only measure that really matters (at least to us) is playing strength. We ALL wish that our program played as if it they were 4x faster when playing on 4 cores - but nobody gets that kind of efficiency.
Some have implied that we search wider because we don't get quite the same increase in depth and that this is a horrible thing. The intuition is that if that if it works you could do it on 1 thread too. However I do not think that follows. The problem is that 4 threads is just not 4x better than 1 thread no matter how good your MP implementation, so it always comes down to how to best utilize what you have. Anything goes in my opinion if it gets more out of extra cores in terms of ELO.
It is well known that programs do not scale well beyond a few processors. Going from 128 to 256 processors gives you very little. One of the things Larry and I want to do is to explore how to better exploit more and more cores and hopefully what we learn will translate to better performance on few cores. It is unlikely to be minor refinements to what we already do.
I'm not breaking new ground as this has been studied for decades and lots of ideas have been tried - I'm not pretending otherwise. But we will look at new original ideas we come up with and revisit old-fashioned ideas that have been tried in the past. A lot of the old ideas which have been rejected have later come back to life, so there is no idea that is off-limits.
As far as whether searching "wider" is a bad thing and isn't real parallelism, I disagree with that. If 4 cores were 4 times better than one I probably would agree but since it isn't, we have to determine where the effort is best spent. If you have 11 people in the room and 10 are doctors and the one who isn't has a heart attack, what is the best use of the doctors time? Not all 10 doctors can give CPR even though normally that would be very best use of their talent, but surely one of them will be calling for an ambulance while another will be doing CPR and the rest will be doing whatever they can to assist, even in little ways.
So if you are getting very little benefit out of additional processors you need to figure out if there is something else they can do to be more useful. Perhaps one thing is to deal with the lines that are severely reduced in the single core program?
a chess engine by primary author Don Dailey (died at the age of 57 on November 22, 2013) and since October 2013, by Mark Lefler, supported by chess advisor and evaluation expert and Don's long time collaborator Larry Kaufman. In July 2016, Komodo 10.x won the Shannon Trophy.
Komodo appeared in January 2010 derived from Don's former engine Doch. It uses bitboards as internal board representation, and has a sophisticated search and a knowledge based, balanced evaluation. Komodo is a standalone chess engine supporting the UCI protocol and is available for multiple platforms and operating systems, and is therefore compatible with free and commercial UCI compliant chess graphical user interfaces and database front ends.
As of December 2011, Komodo became commercial with version 4 [1], earlier versions running under Windows, Linux, Mac OS and Android are available from the Komodo download site, free for non-commercial use [2]. Next release date was May 2013, with CCT15 winner Komodo CCT [3].
Table of Contents
Further Versions
Komodo 5
Komodo 5.1 MP, released in June 2013, was a major rewrite in C++11, able to perform a parallel search to run on multiple processors [5] .Komodo 6
In October 2013, Don Dailey announced the release of Komodo 6 and also bad news concerning the future status of Komodo due to his fatal illness of a acute form of Leukemia [6] . He introduced Mark Lefler as new member of the Komodo team [7] [8] . Don Dailey died November 22, 2013 at age 57, just about the same time that Komodo pulled ahead of Stockfish by winning game 2 in the TCEC Season 5 final [9] [10] .Komodo TCEC and 7
As winner of the TCEC 5 final, Komodo TCEC was released on December 04, 2013 [11], the further improved Komodo 7 on May 21, 2014, during the TCEC Season 6 final versus Stockfish again, now supporting Syzygy Bases [12], probing code interface implementation by Jeremy Bernstein [13].Komodo 8
Komodo 8, released on September 05, 2014, is a substantial improvement over Komodo 7a, which was already rated among the top three chess engines on almost all rating lists [14]. Komodo 8 supports Syzygy Bases and is available for Windows, Linux, Mac OS and Android, the latter immediately available for import in Chess for Android and in Chess for Glass [15]. Since mid September 2014, Komodo 8 for Windows is available as ChessBase engine, bundled with the Deep Fritz interface, Syzygy bases not included [16].Komodo 9
Komodo 9, released on April 28, 2015, continues the progress with approximately +50 Elo over Komodo 8 due to improved search, evaluation, and time management, as well as more efficient use of multiprocessing [17]. Most noticeable is the increase in search depth reached in a given amount of time, which is due to better extension, reduction, and pruning rules [18]. Further, Komodo 9 now features Chess960 and a persistant hash to save and restore analysis.Komodo 10
On May 23, 2016, Komodo 10 came with revised and improved king safety, pawn storm and passed pawn evaluation, as well with revised LMR, NMP, IID and history heuristic concerning search, and is about 60 Elo stronger than Komodo 9 [19]. In July 2016, Komodo 10.x won the Shannon Trophy.Komodo 11
On May 23, 2017, Komodo 11.01 was released, which corrected some minor issues of Komodo 11, which was already released by ChessBase a few days before. It is estimated about 55 Elo stronger than Komodo 10 at intermediate time controls, gains related to evaluation [20], revised quiescence search, singular extensions, LMR , and parallel search [21] . Komodo 11.2, released on July 17, 2017, was the version which won the WCCC 2017 [22]. Komodo 11.3 was released on March 04, 2018 with subsequent improvements [23]. It also comes with a BMI2 compile for compatible x86-64 processors - so far not recommend for AMD Ryzen due to its slow PEXT implementation [24].Tournament Play
Komodo played the ICT 2010 in Leiden over the board, and further the CCT12 and CCT14.CCT 15/16 and TCEC 5-7
Komodo had its first tournament success in 2013, when it won the CCT15 with 6½/7, and later that year the TCEC Season 5 after qualifying for the Superfinal and 48 thrilling games versus Stockfish by a margin of 25 - 23. In 2014, Komodo won the CCT16 with 5½/6. In May 2014, at TCEC Season 6, Komodo 7 qualified for the Superfinal - again versus Stockfish - both engines further improved and supporting Syzygy Bases. This time, Stockfish won the 64 game match 35½ - 28½. Again, the other way around at TCEC Season 7 - Superfinal in December 2014 - version 8 successor Komodo 1333 won from Stockfish 141214 with 33½ - 30½.WCCC / WCSC / WCRCC 2015
At the WCCC 2015 in Leiden, Komodo 9.x won the Blitz tournament, and became runner-up behind and losing from the massively parallel and distributed Jonny which played on a cluster of 100x24 cores. After the revenge winning against Jonny at the equal hardware WCSC 2015 a few days later, Komodo was unlucky and made too many draws against apparently weaker opponents [25], to became third behind Shredder and Ginkgo. Two weeks later, Komodo won the WCRCC 2015 convincingly with 9/9!TCEC Season 8
In Fall 2015, Komodo 9.3x dominated the TCEC Season 8 and won the Superfinal again versus Stockfish, this time 100 games with +9, =89, -2, that is 53½ - 46½ [27].WCCC / WCSC / WCRCC 2016
The 2016 ICGA tournaments in Leiden were a huge success for Komodo 10.x, which did not only won the Shannon Trophy in the open hardware World Computer Chess Championship after a nerve-racking six games play-off versus defending champion Jonny - Komodo with 48 threads on 60 core Intel i7 versus Jonny's cluster of 100x24 AMD cores, but aso the equal hardware World Chess Software Championship with 5/6 - 1½ points ahead of runner-up Jonny and Shredder. Two weeks later Komodo defended its ACCA title in rapid chess at the WCRCC 2016 with 14 wins and one draw versus runner-up Andscacs, that is 14½ out of 15.WCCC / WCSC / WCRCC 2017
In 2017, Komodo could defend it titles at the WCRCC 2017 online, and the soon later released Komodo 11.2 the ICGA WCCC 2017 over the board, again in a play-off versus Jonny. Komodo further became runner-up at the uniform hardware WCSC 2017 behind Shredder.Quotes
Promotions
Don Dailey on Minor Promotions [30] :Evaluation
Centered Point Values
In 2012, Don Dailey gave following point values in a kind of millipawn scale, tapered between opening and endgame phase with knight and bishop values constant - positional scores from mobility and piece square tables centered around zero [31] :Quality of Knowledge
Don Dailey in a reply to Vincent Diepeveen [32] :...
Total amount of knowledge is not what is important, it's the QUALITY of the knowledge that is. Komodo tries to have the most balanced evaluation function possible, and probably much more knowledge than most programs. I don't believe in quantity but Komodo evolved to have a lot of knowledge anyway - probably because every piece of knowledge in Komodo is needed. I tried removing knowledge in Komodo to get speed and we were not able to remove ANYTHING without noticing that it weakened Komodo, at least slightly.
Lazy Evaluation
Don Dailey on Selectivity and Lazy Evaluation [33] :Node Counting
Don Dailey on counting nodes in Komodo 4 [34] :It's mostly semantics, but we get more stable node counts and this works better for fixed node testing which we sometimes do.
I have several things in mind for getting serious speedups in future versions of Komodo which will bring the speed of Komodo back up to that of other programs, or at least not as far away. I believe Komodo is one of the slower programs in terms of raw search speed as we have put a really heavy focus on having a smart program and little on heavy optimization. The reasoning is the same for delaying the MP stuff too, we want to start with a really strong program before we play the optimization games.
Parallel Search
Don Dailey about aspects of parallel search in Komodo 5.1 MP [35]Some have implied that we search wider because we don't get quite the same increase in depth and that this is a horrible thing. The intuition is that if that if it works you could do it on 1 thread too. However I do not think that follows. The problem is that 4 threads is just not 4x better than 1 thread no matter how good your MP implementation, so it always comes down to how to best utilize what you have. Anything goes in my opinion if it gets more out of extra cores in terms of ELO.
It is well known that programs do not scale well beyond a few processors. Going from 128 to 256 processors gives you very little. One of the things Larry and I want to do is to explore how to better exploit more and more cores and hopefully what we learn will translate to better performance on few cores. It is unlikely to be minor refinements to what we already do.
I'm not breaking new ground as this has been studied for decades and lots of ideas have been tried - I'm not pretending otherwise. But we will look at new original ideas we come up with and revisit old-fashioned ideas that have been tried in the past. A lot of the old ideas which have been rejected have later come back to life, so there is no idea that is off-limits.
As far as whether searching "wider" is a bad thing and isn't real parallelism, I disagree with that. If 4 cores were 4 times better than one I probably would agree but since it isn't, we have to determine where the effort is best spent. If you have 11 people in the room and 10 are doctors and the one who isn't has a heart attack, what is the best use of the doctors time? Not all 10 doctors can give CPR even though normally that would be very best use of their talent, but surely one of them will be calling for an ambulance while another will be doing CPR and the rest will be doing whatever they can to assist, even in little ways.
So if you are getting very little benefit out of additional processors you need to figure out if there is something else they can do to be more useful. Perhaps one thing is to deal with the lines that are severely reduced in the single core program?
See also
Publications
Forum Posts
2010
2011
2012
Re: The scaling of Houdini 3 and Komodo 5 by Don Dailey, CCC, November 16, 2012
2013
Re: Komodo TCEC: Serious castling bug ! by Mark Lefler, CCC, December 11, 2013
2014
2015
Re: Komodo 9 Announcement by Mark Lefler, CCC, April 28, 2015
2016
2017
2018
Re: Komodo 11.3 by Mark Lefler, CCC, March 04, 2018 » AMD, BMI2 PEXT
Matches
What will the match score be for Komodo against FM Bolzoni? by Larry Kaufman, CCC, July 22, 2015
External Links
Chess Engine
Purchase
Komodo chess engine - Downloads
Interviews
Interview with Don Dailey - TCEC - Immortal Chess
Reports
Android
Rating Lists
Misc
References
What links here?
Up one Level