Neural Networks,
a series of connected neurons which communicate due to neurotransmission. The interface through which neurons interact with their neighbors consists of axon terminals connected via synapses to dendrites on other neurons. If the sum of the input signals into one neuron surpasses a certain threshold, the neuron sends an action potential at the axon hillock and transmits this electrical signal along the axon.

In 1949, Donald O. Hebb introduced his theory in The Organization of Behavior, stating that learning is about to adapt weight vectors (persistent synaptic plasticity) of the neuron pre-synaptic inputs, whose dot-product activates or controls the post-synaptic output, which is the base of Neural network learning ^{[1]}.

Already in the early 40s, Warren S. McCulloch and Walter Pitts introduced the artificial neuron as a logical element with multiple analogue inputs and a single digital output with a boolean result. The output fired "true", if the sum of the inputs exceed a threshold. In their 1943 paper A Logical Calculus of the Ideas Immanent in Nervous Activity^{[3]}, they attempted to demonstrate that a Turing machine program could be implemented in a finite network of such neurons of combinatorial logic functions of AND, OR and NOT.

The perceptron is an algorithm for supervised learning of binary classifiers. It was the first artificial neural network, introduced in 1957 by Frank Rosenblatt^{[5]}, implemented in custom hardware. In its basic form it consists of a single neuron with multiple inputs and associated weights.

Perceptron ^{[6]}

Supervised learning is applied using a set D of labeled training data with pairs of feature vectors (x) and given results as desired output (d), usually started with cleared or randomly initialized weight vector w. The output is calculated by all inputs of a sample, multiplied by its corresponding weights, passing the sum to the activation function f. The difference of desired and actual value is then immediately used modify the weights for all features using a learning rate 0.0 < α <= 1.0:

for(j=0, Σ =0.0; j < nSamples;++j){for(i=0, X = bias; i < nFeatures;++i)
X += w[i]*x[j][i];
y = f ( X );
Σ +=abs(Δ = d[j]- y);for(i=0; i < nFeatures;++i)
w[i]+= α*Δ*x[j][i];}

AI Winter

Although the perceptron initially seemed promising, it was proved that perceptrons could not be trained to recognise many classes of patterns. This led to neural network research stagnating for many years, the AI-winter, before it was recognised that a feedforward neural network with two or more layers had far greater processing power than with one layer. Single layer perceptrons are only capable of learning linearly separable patterns. In their 1969 book Perceptrons, Marvin Minsky and Seymour Papert wrote that it was impossible for these classes of network to learn the XOR function. It is often believed that they also conjectured (incorrectly) that a similar result would hold for a multilayer perceptron^{[7]}. However, this is not true, as both Minsky and Papert already knew that multilayer perceptrons were capable of producing an XOR function ^{[8]}-

Backpropagation algorithm for a 3-layer network ^{[24]}:

initialize the weights in the network (often small random values)dofor each example e in the training set
O = neural-net-output(network, e)// forward pass
T = teacher output for e
compute error (T - O) at the output units
compute delta_wh for all weights from hidden layer to output layer // backward pass
compute delta_wi for all weights from input layer to hidden layer // backward pass continued
update the weights in the network
until all examples classified correctly or stopping criterion satisfied
return the network

Deep Learning

Deep learning has been characterized as a buzzword, or a rebranding of neural networks. A deep neural network (DNN) is an ANN with multiple hidden layers of units between the input and output layers which can be discriminatively trained with the standard backpropagation algorithm. Two common issues if naively trained are overfitting and computation time.

Convolutional NNs

Convolutional neural networks form a subclass of feedforward neural networks that have special weight constraints, individual neurons are tiled in such a way that they respond to overlapping regions. Convolutional NNs are suited for deep learning and are highly suitable for parallelization on GPUs^{[25]}. They are research topic applied to the game of Go^{[26]}.

More sophisticated attempts to replace static evaluation by neural networks and perceptrons feeding in more unaffiliated feature sets like board representation and attack tables etc., where not yet that successful like in other games. Chess evaluation seems not that well suited for neural nets, but there are also aspects of too weak models and feature recognizers as addressed by Gian-Carlo Pascutto with Stoofvlees^{[28]}, huge training effort, and weak floating point performance - but there is still hope due to progress in hardware and parallelization using SIMD instructions and GPUs, and deeper and more powerful neural network structures and methods successful in other domains.

In 2016, Omid E. David, Nathan S. Netanyahu, and Lior Wolf introduced DeepChess obtaining a grandmaster-level chess playing performance using a learning method incorporating two deep neural networks, which are trained using a combination of unsupervised pretraining and supervised training. The unsupervised training extracts high level features from a given chess position, and the supervised training learns to compare two chess positions to select the more favorable one. In order to use DeepChess inside a chess program, a novel version of alpha-beta is used that does not require bounds but positions αpos and βpos^{[33]}.

Go

In 2014, two teams independently investigated whether deep convolutional neural networks could be used to directly represent and learn a move evaluation function for the game of Go. Christopher Clark and Amos Storkey trained an 8-layer convolutional neural network by supervised learning from a database of human professional games, which without any search, defeated the traditional search program Gnu Go in 86% of the games ^{[34]}^{[35]}^{[36]}^{[37]}. In their paper Move Evaluation in Go Using Deep Convolutional Neural Networks^{[38]}, Chris J. Maddison, Aja Huang, Ilya Sutskever, and David Silver report they trained a large 12-layer convolutional neural network in a similar way, to beat Gnu Go in 97% of the games, and matched the performance of a state-of-the-art Monte-Carlo Tree Search that simulates a million positions per move ^{[39]}.

Arthur E. Bryson (1961). A gradient method for optimizing multi-stage allocation processes. In Proceedings of the Harvard University Symposium on digital computers and their applications » Backpropagation

Richard Sutton (1978). Single channel theory: A neuronal theory of learning. Brain Theory Newsletter 3, No. 3/4, pp. 72-75. pdf

1980 ...

Kunihiko Fukushima (1980). Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position. Biological Cybernetics, Vol. 36 ^{[48]}

Richard Sutton, Andrew Barto (1981). Toward a modern theory of adaptive networks: Expectation and prediction. Psychological Review, Vol. 88, pp. 135-170. pdf

A. Harry Klopf (1982). The Hedonistic Neuron: A Theory of Memory, Learning, and Intelligence. Hemisphere Publishing Corporation, University of Michigan

Richard Sutton (1988). Learning to Predict by the Methods of Temporal Differences. Machine Learning, Vol. 3, No. 1, pp. 9-44. Kluwer Academic Publishers, Boston. ISSN 0885-6125.

Erach A. Irani, John P. Matts, John M. Long, James R. Slagle, POSCH group (1989). Using Artificial Neural Nets for Statistical Discovery: Observations after Using Backpropogation, Expert Systems, and Multiple-Linear Regression on Clinical Trial Data. University of Minnesota, Minneapolis, MN 55455, USA, Complex Systems 3, pdf

Sebastian Thrun, Tom Mitchell (1993). Integrating Inductive Neural Network Learning and Explanation-Based Learning. Proceedings of the 13th IJCAI, pp. 930-936. Morgan Kaufmann, San Mateo, CA, zipped ps

Sebastian Thrun (1994). Neural Network Learning in the Domain of Chess. Machines That Learn, Snowbird, Extended abstract

Christian Posthoff, S. Schawelski, Michael Schlosser (1994). Neural Network Learning in a Chess Endgame Positions. IEEE World Congress on Computational Intelligence

Kieran Greer, Piyush Ojha, David A. Bell (1997). Learning Search Heuristics from Examples: A Study in Computer Chess. Seventh Conference of the Spanish Association for Artificial Intelligence, CAEPIA’97, November, pp. 695-704.

Anna Górecka, Maciej Szmit (1999). Exchange rates prediction by ARIMA and Neural Networks Models. 47th International Atlantic Economic Conerence (Abstract: International Advances of Economic Research Vol 5 Nr 4 Nov. 1999, St Louis, MO, USA 1999), pdf

Don Beal, Martin C. Smith (2001). Temporal difference learning applied to game playing and the results of application to Shogi. Theoretical Computer Science, Volume 252, Issues 1-2, pp. 105-119

Daniel Abdi, Simon Levine, Girma T. Bitsuamlak (2009). Application of an Artificial Neural Network Model for Boundary Layer Wind Tunnel Profile Development. 11th Americas conference on wind Engineering, pdf

^ A two-layer neural network capable of calculating XOR. The numbers within the neurons represent each neuron's explicit threshold (which can be factored out so that all neurons have the same threshold, usually 1). The numbers that annotate arrows represent the weight of the inputs. This net assumes that if the threshold is not reached, zero (not -1) is output. Note that the bottom layer of inputs is not always considered a real neural network layer, Feedforward neural network from Wikipedia

^Arthur E. Bryson (1961). A gradient method for optimizing multi-stage allocation processes. In Proceedings of the Harvard University Symposium on digital computers and their applications

^Seppo Linnainmaa (1970). The representation of the cumulative rounding error of an algorithm as a Taylor expansion of the local rounding errors. Master's thesis, University of Helsinki

Home * Learning * Neural NetworksNeural Networks,a series of connected neurons which communicate due to neurotransmission. The interface through which neurons interact with their neighbors consists of axon terminals connected via synapses to dendrites on other neurons. If the sum of the input signals into one neuron surpasses a certain threshold, the neuron sends an action potential at the axon hillock and transmits this electrical signal along the axon.

In 1949, Donald O. Hebb introduced his theory in

The Organization of Behavior, stating that learning is about to adapt weight vectors (persistent synaptic plasticity) of the neuron pre-synaptic inputs, whose dot-product activates or controls the post-synaptic output, which is the base of Neural network learning^{[1]}.^{[2]}## Table of Contents

## AN

Already in the early 40s, Warren S. McCulloch and Walter Pitts introduced the artificial neuron as a logical element with multiple analogue inputs and a single digital output with a boolean result. The output fired "true", if the sum of the inputs exceed a threshold. In their 1943 paperA Logical Calculus of the Ideas Immanent in Nervous Activity^{[3]}, they attempted to demonstrate that a Turing machine program could be implemented in a finite network of such neurons of combinatorial logic functions of AND, OR and NOT.## ANNs

Artificial Neural Networks (ANNs) are a family of statistical learning devices or algorithms used in regression, and binary or multiclass classification, implemented in hardware or software inspired by their biological counterparts. The artificial neurons of one or more layers receive one or more inputs (representing dendrites), and after being weighted, sum them to produce an output (representing a neuron's axon). The sum is passed through a nonlinear function known as an activation function or transfer function. The transfer functions usually have a sigmoid shape, but they may also take the form of other non-linear functions, piecewise linear functions, or step functions^{[4]}. The weights of the inputs of each layer are tuned to minimize a cost or loss function, which is a task in mathematical optimization and machine learning.## Perceptron

^{[5]}, implemented in custom hardware. In its basic form it consists of a single neuron with multiple inputs and associated weights.^{[6]}## AI Winter

Perceptrons, Marvin Minsky and Seymour Papert wrote that it was impossible for these classes of network to learn the XOR function. It is often believed that they also conjectured (incorrectly) that a similar result would hold for a multilayer perceptron^{[7]}. However, this is not true, as both Minsky and Papert already knew that multilayer perceptrons were capable of producing an XOR function^{[8]}-^{[9]}## Backpropagation

In 1974, Paul Werbos started to end the AI winter concerning neural networks, when he first descibed the mathematical process of training multilayer perceptrons through backpropagation of errors^{[10]}, derived in the context of control theory by Henry J. Kelley in 1960^{[11]}and by Arthur E. Bryson in 1961^{[12]}using principles of dynamic programming, simplified by Stuart Dreyfus in 1961 applying the chain rule^{[13]}. It was in 1982, when Werbos applied a automatic differentiation method described in 1970 by Seppo Linnainmaa^{[14]}to neural networks in the way that is widely used today^{[15]}^{[16]}^{[17]}^{[18]}. Backpropagation is a generalization of the delta rule to multilayered feedforward networks, made possible by using the chain rule to iteratively compute gradients for each layer. Backpropagation requires that the activation function used by the artificial neurons be differentiable, which is true for the common sigmoid logistic function or its softmax generalization in multiclass classification. Along with an optimization method such as gradient descent, it calculates the gradient of a cost or loss function with respect to all the weights in the neural network. The gradient is fed to the optimization method which in turn uses it to update the weights, in an attempt to minimize the loss function, which choice depends on the learning type (supervised, unsupervised, reinforcement) and the activation function - mean squared error or cross-entropy error function are used in binary classification^{[19]}. The gradient is almost always used in a simple stochastic gradient descent algorithm. In 1983, Yurii Nesterov contributed an accelerated version of gradient descent that converges considerably faster than ordinary gradient descent^{[20]}^{[21]}^{[22]}^{[23]}.Backpropagation algorithm for a 3-layer network

^{[24]}:## Deep Learning

Deep learning has been characterized as a buzzword, or a rebranding of neural networks. A deep neural network (DNN) is an ANN with multiple hidden layers of units between the input and output layers which can be discriminatively trained with the standard backpropagation algorithm. Two common issues if naively trained are overfitting and computation time.## Convolutional NNs

Convolutional neural networks form a subclass of feedforward neural networks that have special weight constraints, individual neurons are tiled in such a way that they respond to overlapping regions. Convolutional NNs are suited for deep learning and are highly suitable for parallelization on GPUs^{[25]}. They are research topic applied to the game of Go^{[26]}.## ANNs in Games

Applications of neural networks in computer games and chess are learning of evaluation and search control. Evaluation topics include feature selection and automated tuning, search control move ordering, selectivity and time management. The perceptron looks like the ideal learning algorithm for automated evaluation tuning.## Backgammon

In the late 80s, Gerald Tesauro pioneered in applying ANNs to the game of Backgammon. His program Neurogammon won the Gold medal at the 1st Computer Olympiad 1989 - and was further improved byTD-Lambdabased Temporal Difference Learning within TD-Gammon^{[27]}. Today all strong backgammon programs rely on heavily trained neural networks.## Chess

Logistic regression as applied in Texel's Tuning Method may be interpreted as supervised learning application of the single-layer perceptron with one neuron. This is also true for reinforcement learning approaches, such as TD-Leaf in KnightCap or Meep's TreeStrap, where the evaluation consists of a weighted linear combination of features. Despite these similarities with the perceptron, these engines are not considered using ANNs - since they use manually selected chess specific feature construction concepts like material, piece square tables, pawn structure, mobility etc..More sophisticated attempts to replace static evaluation by neural networks and perceptrons feeding in more unaffiliated feature sets like board representation and attack tables etc., where not yet that successful like in other games. Chess evaluation seems not that well suited for neural nets, but there are also aspects of too weak models and feature recognizers as addressed by Gian-Carlo Pascutto with Stoofvlees

^{[28]}, huge training effort, and weak floating point performance - but there is still hope due to progress in hardware and parallelization using SIMD instructions and GPUs, and deeper and more powerful neural network structures and methods successful in other domains.## Move Ordering

Concerning move ordering - there were interesting NN proposals like the Chessmaps Heuristic by Kieran Greer et al.^{[29]}, and the Neural MoveMap Heuristic by Levente Kocsis et al.^{[30]}.## Giraffe & Zurichess

In 2015, Matthew Lai trained Giraffe's deep neural network by TD-Leaf^{[31]}. Zurichess by Alexandru Moșoi uses the TensorFlow library for automated tuning - in a two layers neural network, the second layer is responsible for a tapered eval to phase endgame and middlegame scores^{[32]}.## DeepChess

In 2016, Omid E. David, Nathan S. Netanyahu, and Lior Wolf introduced DeepChess obtaining a grandmaster-level chess playing performance using a learning method incorporating two deep neural networks, which are trained using a combination of unsupervised pretraining and supervised training. The unsupervised training extracts high level features from a given chess position, and the supervised training learns to compare two chess positions to select the more favorable one. In order to use DeepChess inside a chess program, a novel version of alpha-beta is used that does not require bounds but positions αpos and βpos^{[33]}.## Go

In 2014, two teams independently investigated whether deep convolutional neural networks could be used to directly represent and learn a move evaluation function for the game of Go. Christopher Clark and Amos Storkey trained an 8-layer convolutional neural network by supervised learning from a database of human professional games, which without any search, defeated the traditional search program Gnu Go in 86% of the games^{[34]}^{[35]}^{[36]}^{[37]}. In their paperMove Evaluation in Go Using Deep Convolutional Neural Networks^{[38]}, Chris J. Maddison, Aja Huang, Ilya Sutskever, and David Silver report they trained a large 12-layer convolutional neural network in a similar way, to beat Gnu Go in 97% of the games, and matched the performance of a state-of-the-art Monte-Carlo Tree Search that simulates a million positions per move^{[39]}.In 2015, a team affiliated with Google DeepMind around David Silver and Aja Huang, supported by Google researchers John Nham and Ilya Sutskever, build a Go playing program dubbed AlphaGo

^{[40]}, combining Monte-Carlo tree search with their 12-layer networks^{[41]}.## See also

AlphaGo

Keynote Lecture CG 2016 Conference by Aja Huang

## NN Chess Programs

## Selected Publications

## 1940 ...

1942).Some observations on the simple neuron circuit. Bulletin of Mathematical Biology, Vol. 4, No. 31943).A Logical Calculus of the Ideas Immanent in Nervous Activity. Bulletin of Mathematical Biology, Vol. 5, No. 1, pdf1949).The Organization of Behavior. Wiley & Sons## 1950 ...

1951)Representation of Events in Nerve Nets and Finite Automata. RM-704, RAND paper, pdf, reprinted inClaude Shannon, John McCarthy (eds.) (

1956).Automata Studies. Annals of Mathematics Studies, No. 341954).Neural Nets and the Brain Model Problem. Ph.D. dissertation, Princeton University1954).Simulation of Self-Organizing Systems by Digital Computer. IRE Transactions on Information Theory, Vol. 41956).Probabilistic Logic and the Synthesis of Reliable Organisms From Unreliable Components. inClaude Shannon, John McCarthy (eds.) (

1956).Automata Studies. Annals of Mathematics Studies, No. 34, pdf1956).Tests on a Cell Assembly Theory of the Action of the Brain, Using a Large Digital Computer. IRE Transactions on Information Theory, Vol. 2, No. 31957).The Perceptron - a Perceiving and Recognizing Automaton. Report 85-460-1, Cornell Aeronautical Laboratory^{[42]}## 1960 ...

1960).Gradient Theory of Optimal Flight Paths. [[http://arc.aiaa.org/loi/arsj|ARS Journal, Vol. 30, No. 10 » Backpropagation1961).A gradient method for optimizing multi-stage allocation processes. In Proceedings of the Harvard University Symposium on digital computers and their applications » Backpropagation1961).The numerical solution of variational problems. RAND paper P-2374 » Backpropagation1962).Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Spartan Books1965).Cybernetic Predicting Devices. Naukova Dumka1969).Perceptrons.^{[43]}^{[44]}## 1970 ...

1970).The representation of the cumulative rounding error of an algorithm as a Taylor expansion of the local rounding errors. Master's thesis, University of Helsinki » Backpropagation^{[45]}1971).Polynomial theory of complex systems. IEEE Transactions on Systems, Man, and Cybernetics, Vol. 1, No. 41972).Brain Function and Adaptive Systems - A Heterostatic Theory. Air Force Cambridge Research Laboratories, Special Reports, No. 133, pdf1972).Perceptrons: An Introduction to Computational Geometry. The MIT Press, 2nd edition with corrections1973).Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks. Studies in Applied Mathematics, Vol. 52, pdf1974).Classical and instrumental learning by neural networks. Progress in Theoretical Biology. Academic Press1974).Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. Ph. D. thesis, Harvard University^{[46]}^{[47]}1978).Single channel theory: A neuronal theory of learning. Brain Theory Newsletter 3, No. 3/4, pp. 72-75. pdf## 1980 ...

1980).Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position. Biological Cybernetics, Vol. 36^{[48]}1981).Toward a modern theory of adaptive networks: Expectation and prediction. Psychological Review, Vol. 88, pp. 135-170. pdf1982).Applications of advances in nonlinear sensitivity analysis. System Modeling and Optimization, Springer, pdf1982).The Hedonistic Neuron: A Theory of Memory, Learning, and Intelligence. Hemisphere Publishing Corporation, University of Michigan1985).A Learning Algorithm for Boltzmann Machines. Cognitive Science, Vol. 9, No. 1, pdf1986).Learning representations by back-propagating errors. Nature, Vol. 323, pdf19871987).A 'Neural' Network that Learns to Play Backgammon. NIPS 19871987).Supervised Learning of Probability Distributions by Neural Networks. NIPS 198719881988).Learning to Predict by the Methods of Temporal Differences. Machine Learning, Vol. 3, No. 1, pp. 9-44. Kluwer Academic Publishers, Boston. ISSN 0885-6125.1988).Neural network defeats creator in backgammon match. Technical report no. CCSR-88-6, Center for Complex Systems Research, University of Illinois at Urbana-Champaign1988).What Size Net Gives Valid Generalization?NIPS 19881988).On the capabilities of multilayer perceptrons. Complexity, Vol. 4, No. 319891989).The Perceptron Algorithm Is Fast for Non-Malicious Distributions. NIPS 19891989).A Proposal for More Powerful Learning Algorithms. Neural Computation, Vol. 1, No. 21989).Using Artificial Neural Nets for Statistical Discovery: Observations after Using Backpropogation, Expert Systems, and Multiple-Linear Regression on Clinical Trial Data. University of Minnesota, Minneapolis, MN 55455, USA, Complex Systems 3, pdf1989).A Parallel Network that Learns to Play Backgammon. Artificial Intelligence, Vol. 39, No. 31989).Random Neural Networks with Negative and Positive Signals and Product Form Solution. Neural Computation, Vol. 1, No. 4## 1990 ...

1990).Backpropagation Through Time: What It Does and How to Do It. Proceedings of the IEEE, Vol. 78, No. 10, pdf1990).Maximization of Mutual Information in a Context Sensitive Neural Network. Ph.D. thesis1990).Neural Networks. Review. in Multi Component Systems (Russian)1990).Polynomial Time Algorithms for Learning Neural Nets. COLT 199019911991).Untersuchungen zu dynamischen neuronalen Netzen. Diploma thesis, TU Munich, advisor Jürgen Schmidhuber, pdf (German)^{[49]}1991).Neural Networks as a Guide to Optimization - The Chess Middle Game Explored. ICCA Journal, Vol. 14, No. 31991).Using sequential adaptive Neuro-control for efficient Learning of Rotation and Translation Invariance. In Teuvo Kohonen, Kai Mäkisara, Olli Simula, and Jari Kangas, editors,Artificial Neural Networks. Elsevier1991).Dynamische neuronale Netze und das fundamentale raumzeitliche Lernproblem(Dynamic Neural Nets and the Fundamental Spatio-Temporal Credit Assignment Problem). Ph.D. thesis1991).Unsupervised Learning of Distributions of Binary Vectors Using 2-Layer Networks. NIPS 19911991).Neural networks that teach themselves through genetic discovery of novel examples. IEEE International Joint Conference on Neural Networks19921992).Temporal Sequence Processing in Neural Networks. Ph.D. thesis, King's College London, advisor John G. Taylor, pdf1992).A Neural Network designed to solve the N-Queens Problem. Biological Cybernetics, Vol. 66 No. 4, pdf1992).Temporal Difference Learning of Backgammon Strategy. ML 19921992).Practical Issues in Temporal Difference Learning. Machine Learning, Vol. 8, No. 3-41992).Biological metaphors and the design of modular artificial neural networks. Master’s thesis, Leiden University, pdf1992).Rprop - A Fast Adaptive Learning Algorithm. Proceedings of the International Symposium on Computer and Information Science1992).Modular Neural Networks for Learning Context-Dependent Game Strategies. Master's thesis, University of Cambridge, pdf1992).The Computational Brain. MIT Press19931993).A Neural Network performing Boolean Logic Operations. Optical Memory and Neural Networks, Vol. 2, No. 1, pdf1993).Integrating Inductive Neural Network Learning and Explanation-Based Learning. Proceedings of the 13th IJCAI, pp. 930-936. Morgan Kaufmann, San Mateo, CA, zipped ps1993).Evolving Optimal Neural Networks Using Genetic Algorithms with Occam's Razor. Complex Systems, Vol. 7, pdf1993).A direct adaptive method for faster backpropagation learning: The RPROP algorithm. IEEE International Conference On Neural Networks, pdf19941994).The Roots of Backpropagation. From Ordered Derivatives to Neural Networks and Political Forecasting. John Wiley & Sons1994).Evolving Neural Networks to focus Minimax Search. AAAI-94, pdf1994).SCAN: A Neural Model of Covert Attention. Ph.D. thesis, Maastricht University, advisor Jaap van den Herik1994).Neural Network Learning in the Domain of Chess. Machines That Learn, Snowbird, Extended abstract1994).Neural Network Learning in a Chess Endgame Positions. IEEE World Congress on Computational Intelligence1994).Temporal Difference Learning of Position Evaluation in the Game of Go. Advances in Neural Information Processing Systems 6^{[50]}1994).Efficient Neural Net α-β-Evaluators. pdf^{[51]}1994).Fast bounded smooth regression with lazy neural trees. ICNN 1994, DOI: 10.1109/ICNN.1994.3744211994).Rprop - Description and Implementation Details. Technical Report, University of Karlsruhe, pdf19951995).Artificial neural networks: an introduction to ANN theory and practice. LNCS 931, Springer1995).Learning of Position Evaluation in the Game of Othello. Master's Project, University of Massachusetts, Amherst, Massachusetts, pdf1995).Simplifying Neural Nets by Discovering Flat Minima. In Gerald Tesauro, David S. Touretzky and Todd K. Leen (eds.),Advances in Neural Information Processing Systems 7, NIPS'7, pages 529-536. MIT Press1995).Learning to Play the Game of Chess. in Gerald Tesauro, David S. Touretzky, Todd K. Leen (eds.) Advances in Neural Information Processing Systems 7, MIT Press1995).Simplifying Neural Nets by Discovering Flat Minima. In Gerald Tesauro, David S. Touretzky and Todd K. Leen (eds.),Advances in Neural Information Processing Systems 7, NIPS'7, pages 529-536. MIT Press1995).Explanation-Based Neural Network Learning - A Lifelong Learning Approach. Ph.D. thesis, University of Bonn, advisors Armin Cremers and Tom Mitchell1995).Temporal Difference Learning and TD-Gammon. Communications of the ACM Vol. 38, No. 31995).Optimization Networks. Artificial Neural Networks1995).Rough Set Reduction of Attributes and their Domains for Neural Networks. Computational Intelligence, Vol. 11, No. 21995).Applying Genetic Algorithms to Recurrent Neural Networks for Learning Network Parameters and Architecture, Masters Thesis, Case Western Reserve University1995).Forecasting with Neural networks. ICANN 19951995).TD Learning of Game Evaluation Functions with Hierarchical Neural Architectures. Master's thesis, University of Amsterdam, pdf1995, 2002).The Handbook of Brain Theory and Neural Networks. The MIT Press1995).Optimization of Entropy with Neural Networks. Ph.D. thesis, University of California, San Diego1995).Pipelined Neural Tree Learning by Error Forward-Propagation. ICNN 1995, DOI: 10.1109/ICNN.1995.488132, pdf1995).Tools for Neural Trees. Technical Report No. 681995).Tempering Backpropagation Networks: Not All Weights are Created Equal. NIPS 1995, pdf19961996).Explanation-Based Neural Network Learning: A Lifelong Learning Approach. Kluwer Academic Publishers1996).Agnostic Learning and Single Hidden Layer Neural Networks.Ph.D. thesis, Australian National University, ps1996).The Integration of A Priori Knowledge into a Go Playing Neural Network.1996).Elegance: Genetic Algorithms in Neural Reinforcement Control. Master thesis, Delft University of Technology, pdf1996).Neural Networks - A Systematic Introduction. Springer, available as pdf ebook19971997).Long short-term memory. Neural Computation, Vol. 9, No. 8, pdf^{[52]}1997).Learning Search Heuristics from Examples: A Study in Computer Chess. Seventh Conference of the Spanish Association for Artificial Intelligence, CAEPIA’97, November, pp. 695-704.1997).Learning Piece Values Using Temporal Differences. ICCA Journal, Vol. 20, No. 319981998).A Neural Network Based Search Heuristic and its Application to Computer Chess. D.Phil. Thesis, University of Ulster1998).A Neural Network Program of Tsume-Go. CG 1998^{[53]}1998).Pedagogical Method for Extraction of Symbolic Knowledge from Neural Networks. Rough Sets and Current Trends in Computing 19981998).Neural network models for a resource allocation problem. IEEE Transactions on Systems, Man, and Cybernetics, Part B 28(2)1998).Experiments in Parameter Learning Using Temporal Differences. ICCA Journal, Volume 21 No. 2, pdf1998).Chess Endgames and Neural Networks. ICCA Journal, Vol. 21, No. 41998).First Results from Using Temporal Difference Learning in Shogi. CG 19981998).Centering Neural Network Gradient Factors. Neural Networks: Tricks of the Trade1998).Fundamentals of the New Artificial Intelligence: Beyond Traditional Paradigms. 1st edition, Springer, 2nd edition 200819991999).Evolution, Neural Networks, Games, and Intelligence. Proceedings of the IEEE, September, pp. 1471-1496. CiteSeerX1999).Evolving Neural Networks to Play Checkers without Expert Knowledge. IEEE Transactions on Neural Networks, Vol. 10, No. 6, pp. 1382-1391.1999).A Pattern-Oriented Approach to Move Ordering: the Chessmaps Heuristic. ICCA Journal, Vol. 22, No. 11999).Exchange rates prediction by ARIMA and Neural Networks Models. 47th International Atlantic Economic Conerence (Abstract: International Advances of Economic Research Vol 5 Nr 4 Nov. 1999, St Louis, MO, USA 1999), pdf1999).Learning Piece-Square Values using Temporal Differences.ICCA Journal, Vol. 22, No. 41999).Neural Networks: A Comprehensive Foundation. 2nd Edition, Prentice-Hall1999).Neural Codes and Distributed Representations. MIT Press1999).Unsupervised Learning: Foundations of Neural Computation. MIT Press1999).Recurrent Sampling Models for the Helmholtz Machine. Neural Computation, Vol. 11, No. 3, pdf^{[54]}## 2000 ...

2000).Learning Time Allocation using Neural Networks. CG 20002000).Learning of Depth Two Neural Networks with Constant Fan-in at the Hidden Nodes. Electronic Colloquium on Computational Complexity, Vol. 72000).Learning to Play Chess Using Temporal Differences. Machine Learning, Vol 40, No. 3, pdf2000).Tree-Structured Neural Networks: Efficient Evaluation of Higher-Order Derivatives and Integrals. IJCNN 20002000).Chess Neighborhoods, Function Combination, and Reinforcement Learning. CG 2000, pdf2000).Automatic Generation of an Evaluation Function for Chess Endgames. ETH Zurich Supervisors: Thomas Lincke and Christoph Wirth, pdf » Endgame2000).Designing neural network architectures for pattern recognition. The Knowledge Engineering Review, Vol. 15, No. 22000).Multi-Valued and Universal Binary Neurons: Theory, Learning and Applications. Springer^{[55]}20012001).Visual Learning in Go. 6th Computer Olympiad Workshop, pdf2001).Move Ordering using Neural Networks. IEA/AIE 2001, LNCS 2070, pdf2001).Neural Networks for Structured Data. BSc-Thesis, zipped ps2001).Temporal Difference Learning Applied to a High-Performance Game-Playing Program. IJCAI 20012001).Temporal difference learning applied to game playing and the results of application to Shogi. Theoretical Computer Science, Volume 252, Issues 1-2, pp. 105-1192001).Learning to Evaluate Go Positions via Temporal Difference Methods. in Norio Baba, Lakhmi C. Jain (eds.) (2001).Computational Intelligence in Games, Studies in Fuzziness and Soft Computing. Physica-Verlag2001, 2005).Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. MIT Press20022002).The Neural MoveMap Heuristic in Chess. CG 20022002).Local Move Prediction in Go. CG 20022002).Programming backgammon using self-teaching neural nets. Artificial Intelligence Vol. 134 No. 1-22002).Temporal difference learning and the Neural MoveMap heuristic in the game of Lines of Action. in GAME-ON 2002, pdf2002).Neural Networks for the N-Queens Problem: a Review. Control and Cybernetics, Vol. 31, No. 2, pdf2002)Machine Nature: The Coming Age of Bio-Inspired Computing. McGraw-Hill, New York2002).Many-Layered Learning. Neural Computation, Vol. 14, No. 10, pdf2002).Graphical Models: Foundations of Neural Computation. MIT Press20032003).Learning Search Decisions. Ph.D thesis, Maastricht University, pdf2003).Evaluation in Go by a Neural Network using Soft Segmentation. Advances in Computer Games 10, pdf2003).Yes, Trees May Have Neurons. Computer Science in Perspective 200320042004).Learning to Play Draughts using Temporal Difference Learning with Neural Networks and Databases. Cognitive Artiﬁcial Intelligence, Utrecht University, Benelearn’042004).Learning to play chess using TD(λ)-learning with database games. Cognitive Artiﬁcial Intelligence, Utrecht University, Benelearn’042004).Evaluation of Chess Position by Modular Neural network Generated by Genetic Algorithm. EuroGP 2004^{[56]}2004).The MORPH Project in 2004. ICGA Journal, Vol. 27, No. 420062006).Neural Networks as Cybernetic Systems. 2nd and revised edition, Department of Biological Cybernetics, Bielefeld University2006).A Fast Learning Algorithm for Deep Belief Nets. Neural Computation, Vol. 18, No. 7, pdf2006).Reducing the Dimensionality of Data with Neural Networks. Science, Vol. 313, pdf20072007).Temporal Difference Learning of an Othello Evaluation Function for a Small Neural Network with Shared Weights. IEEE Symposium on Computational Intelligence and AI in Games2007).State Space Partition for Reinforcement Learning Based on Fuzzy Min-Max Neural Network. ISNN 20072007).A Brief Introduction to Neural Networks. available at http://www.dkriesel.com2007).Applying Backpropagation Networks to Anaphor Resolution. In: António Branco (Ed.),Anaphora: Analysis, Algorithms, and Applications. Selected Papers of the 6th Discourse Anaphora and Anaphor Resolution Colloquium, DAARC 2007, Lagos, Portugal20082008).Mimicking Go Experts with Convolutional Neural Networks. ICANN 2008, pdf2008).Neural Networks: A Comprehensive Foundation. 3rd Edition, https://en.wikipedia.org/wiki/Prentice_HallPrentice-Hall2008).Fundamentals of the New Artificial Intelligence: Neural, Evolutionary, Fuzzy and More. 2nd edition, Springer, 1st edition 19982008).Hypernetworks: A molecular evolutionary architecture for cognitive learning and memory. IEEE Computational Intelligence Magazine, Vol. 3, No. 3, pdf2008).Robust Neural Network Tracking Controller Using Simultaneous Perturbation Stochastic Approximation. IEEE Transactions on Neural Networks, Vol. 19, No. 5, 2003 pdf » SPSA20092009).Application of an Artificial Neural Network Model for Boundary Layer Wind Tunnel Profile Development. 11th Americas conference on wind Engineering, pdf## 2010 ...

2010).Tuning Neural Networks by Both Connectivity and Size. ITNG 201020112011).Discrete Tomography: A Neural Network Approach. Master's thesis, Leiden University, pdf2011).Discrete Tomography: A Neural Network Approach. BNAIC 2011 pdf2011).Training Neural Networks to Play Backgammon Variants Using Reinforcement Learning. Proceedings of Evogames 2011, Part I, LNCS 6624, pdf20122012).Neural-fitted TD-leaf learning for playing Othello with structured neural networks. IEEE Transactions on Neural Networks and Learning Systems, Vol. 23, No. 112012).Sequence Learning with Artificial Recurrent Neural Networks. (Aiming to become the definitive textbook on RNN.) Invited by Cambridge University Press2012).Clustered ensemble neural network for breast mass classification in digital mammography. IJCNN 20122012).Neural Networks: Tricks of the Trade. (2nd Edition) LNCS 7700, SpringerNicol N. Schraudolph (

2012).Centering Neural Network Gradient Factors.Léon Bottou (

2012).Stochastic Gradient Descent Tricks. Microsoft Research, pdfRonan Collobert, Koray Kavukcuoglu, Clément Farabet (

2012).Implementing Neural Networks Efficiently.^{[57]}20132013).On Layer-Wise Representations in Deep Neural Networks. Ph.D. Thesis, TU Berlin, advisor Klaus-Robert Müller2013).Playing Atari with Deep Reinforcement Learning. arXiv:1312.5602^{[58]}20142014).Generative Adversarial Networks. arXiv:1406.2661v1^{[59]}2014).Teaching Deep Convolutional Neural Networks to Play Go. arXiv:1412.3409^{[60]}^{[61]}2014).Move Evaluation in Go Using Deep Convolutional Neural Networks. arXiv:1412.6564v1 » Go2014).Sequence to Sequence Learning with Neural Networks. arXiv:1409.321520152015).Neural networks and deep learning. Determination Press2015).Distilling the Knowledge in a Neural Network. arXiv:1503.025312015).Giraffe: Using Deep Reinforcement Learning to Play Chess. M.Sc. thesis, Imperial College London, arXiv:1509.01549v1 » Giraffe2015).Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games. arXiv:1509.067312015).Explorations in Parallel Distributed Processing: A Handbook of Models, Programs, and Exercises. Second Edition, Contents2015).Better Computer Go Player with Neural Network and Long-term Prediction. arXiv:1511.06410^{[62]}^{[63]}» Go2015).Dissecting the Winning Solution of the HiggsML Challenge. NIPS 20142015).Convolutional Monte Carlo Rollouts in Go. arXiv:1512.033752015).Adam: A Method for Stochastic Optimization. arXiv:1412.6980v8, ICLR 2015^{[64]}2015).Human-level control through deep reinforcement learning. Nature, Vol. 5182015).Deep Learning in Neural Networks: An Overview. Neural Networks, Vol. 612015).A Critical Review of Recurrent Neural Networks for Sequence Learning. arXiv:1506.00019v42015).Predicting Moves in Chess using Convolutional Neural Networks. pdf^{[65]}^{[66]}2015).Deep Learning. Nature, Vol. 521^{[67]}20162016).Dueling Network Architectures for Deep Reinforcement Learning. arXiv:1511.065812016).Mastering the game of Go with deep neural networks and tree search. Nature, Vol. 529 » AlphaGo2016).Using Deep Convolutional Neural Networks in Monte Carlo Tree Search. CG 20162016).AlphaGo: Combining Deep Neural Networks with Tree Search. CG 2016, Keynote Lecture2016).Convolutional Monte Carlo Rollouts for Computer Go. CG 20162016).An Empirical Study on Applying Deep Reinforcement Learning to the Game 2048. CG 20162016).DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess. ICAAN 2016, Lecture Notes in Computer Science, Vol. 9887, Springer, pdf preprint » DeepChess^{[68]}^{[69]}2016).DNN-Buddies: A Deep Neural Network-Based Estimation Metric for the Jigsaw Puzzle Problem. ICAAN 2016, Lecture Notes in Computer Science, Vol. 9887, Springer^{[70]}2016).Deep Learning. MIT Press2016).Asynchronous Methods for Deep Reinforcement Learning. arXiv:1602.01783v22016).The Computational Brain, 25th Anniversary Edition. MIT Press2016).Memory-Efficient Backpropagation Through Time. arXiv:1606.034012016).Progressive Neural Networks. arXiv:1606.0467120172017).Style Transfer Generative Adversarial Networks: Learning to Play Chess Differently. arXiv:1702.06762v1^{[71]}2017).DeepStack: Expert-level artificial intelligence in heads-up no-limit poker. Science, Vol. 356, No. 6337## Blog & Forum Posts

## 1996 ...

Re: Evaluation by neural network ? by Jay Scott, CCC, November 10, 1997

^{[72]}## 2000 ...

Re: Whatever happened to Neural Network Chess programs? by Andy Walker, rgcc, March 28, 2000 » Advances in Computer Chess 1, Ron Atkin

Combining Neural Networks and Alpha-Beta by Matthias Lüscher, rgcc, April 01, 2000 » Chessterfield

^{[73]}Neural nets in backgammon by Albert Silver, CCC, April 07, 2004

## 2005 ...

^{[74]}## 2010 ...

Re: Chess program with Artificial Neural Networks (ANN)? by Gian-Carlo Pascutto, CCC, January 07, 2010 » Stoofvlees

Re: Chess program with Artificial Neural Networks (ANN)? by Gian-Carlo Pascutto, CCC, January 08, 2010

Re: Chess program with Artificial Neural Networks (ANN)? by Volker Annuss, CCC, January 08, 2010 » Hermann

## 2015 ...

2016^{[75]}Re: Deep Learning Chess Engine ? by Alexandru Mosoi, CCC, July 21, 2016 » Zurichess

Re: Deep Learning Chess Engine ? by Matthew Lai, CCC, August 04, 2016 » Giraffe

^{[76]}2017^{[77]}Re: Is AlphaGo approach unsuitable to chess? by Peter Österlund, CCC, May 31, 2017 » Texel

## External Links

BiologicalANNsTopicsNeurogrid from Wikipedia

History of the Perceptron

RNNsRestricted Boltzmann machine from Wikipedia

Activation FunctionsBackpropagationGradientBlogsThe Single Layer Perceptron

The Sigmoid Function in C#

Hidden Neurons and Feature Space

Training Neural Networks Using Back Propagation in C#

Data Mining with Artificial Neural Networks (ANN)

Neural Net in C++ Tutorial on Vimeo (also on YouTube)

^{[78]}LibrariesRepositoriesSoftwareNeural Lab from Wikipedia

SNNS from Wikipedia

CoursesPart 1: Data and Architecture, YouTube Videos

Part 2: Forward Propagation

Part 3: Gradient Descent

Part 4: Backpropagation

Part 5: Numerical Gradient Checking

Part 6: Training

Part 7: Overfitting, Testing, and Regularization

MusicMarc Ribot, Kenny Wollesen, Joey Baron, Jamie Saft, Trevor Dunn, Cyro Baptista, John Zorn

## References

1943).A Logical Calculus of the Ideas Immanent in Nervous Activity. Bulletin of Mathematical Biology, Vol. 5, No. 1, pdf1957).The Perceptron - a Perceiving and Recognizing Automaton. Report 85-460-1, Cornell Aeronautical Laboratory1974).Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. Ph. D. thesis, Harvard University1960).Gradient Theory of Optimal Flight Paths. [[http://arc.aiaa.org/loi/arsj|ARS Journal, Vol. 30, No. 101961).A gradient method for optimizing multi-stage allocation processes. In Proceedings of the Harvard University Symposium on digital computers and their applications1961).The numerical solution of variational problems. RAND paper P-23741970).The representation of the cumulative rounding error of an algorithm as a Taylor expansion of the local rounding errors. Master's thesis, University of Helsinki1982).Applications of advances in nonlinear sensitivity analysis. System Modeling and Optimization, Springer, pdf1994).The Roots of Backpropagation. From Ordered Derivatives to Neural Networks and Political Forecasting. John Wiley & Sons2008).Mimicking Go Experts with Convolutional Neural Networks. ICANN 2008, pdf1998).Reinforcement Learning: An Introduction. MIT Press, 11.1 TD-Gammon1999).A Pattern-Oriented Approach to Move Ordering: the Chessmaps Heuristic. ICCA Journal, Vol. 22, No. 12002).The Neural MoveMap Heuristic in Chess. CG 20022016).DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chess. ICAAN 2016, Lecture Notes in Computer Science, Vol. 9887, Springer, pdf preprint2014).Teaching Deep Convolutional Neural Networks to Play Go. arXiv:1412.34092014).Move Evaluation in Go Using Deep Convolutional Neural Networks. arXiv:1412.6564v12016).Mastering the game of Go with deep neural networks and tree search. Nature, Vol. 5291962).Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Spartan Books1976).Taylor expansion of the accumulated rounding error. BIT Numerical Mathematics, Vol. 16, No. 21994).The Roots of Backpropagation. From Ordered Derivatives to Neural Networks and Political Forecasting. John Wiley & Sons2015)Critique of Paper by "Deep Learning Conspiracy" (Nature 521 p 436).1994).Efficient Neural Net α-β-Evaluators. pdf2004).Evaluation of Chess Position by Modular Neural network Generated by Genetic Algorithm. EuroGP 20042017).Style Transfer Generative Adversarial Networks: Learning to Play Chess Differently. arXiv:1702.06762v1## What links here?

Up one Level