Concepts for Neural Networks: A Survey
In this work, the dynamic vision sensors DVS was used to detect the land markers by generating a sequence of events. The learning phase was conducted by repeatedly training and switching the robot from the start positions in the inner and outer lanes. Figure 4. Control architecture of feed-forward SNN.
Recommended for you
Different from the feed-forward networks, recurrent neural networks RNNs transmit their information with a directed cycle and exhibit dynamic temporal behaviors. It is worth pointing out that recurrent neural networks are recursive neural networks Wikipedia, d with a certain structure such as a linear chain. Living organisms seem to use this mechanism to process arbitrary sequences of inputs with their internal memory stored inside RNNs. In Rueckert et al. In their finite horizon planning task, the agent spatial position is controlled by nine state neurons. The context neurons produce spatiotemporal spike patterns that represent high-level goals and context information.
A Survey of Machine Learning Approaches and Techniques for Student Dropout Prediction
In this case, its average firing rate represents the target spatial position at different time step. They show that the optimal planning policy can be learned using the reward modulated update rule in a network where the state neurons follow winner-take-all WTA dynamics. Due to the probability, in each time step exactly one state neuron is active and encodes the current position of the agent. Their results demonstrated a successful planner trajectory planning task using a recurrent SNN.
Figure 5. Control architecture of recurrent-SNN. A recurrent layer of state neurons is used to control the state of the agent and receives signals from the content population, which decides the target position according to different time step. Changes in the strength of synaptic connections between neurons are thought to be the physiological basis of learning Vasilaki et al.
These changes can either be gated by neuromodulators that encode the presence of reward or inner co-activation among neurons and synapses. In control tasks presented in this section, the network is supposed to learn a function that maps some state input to a control or action output. When successfully learned, the network is able to perform simple tasks such as wall following, obstacle avoidance, target reaching, lane following, taxi behavior, or food foraging. In most cases, the network input directly comes from the robot's sensors, which range from binary sensors, e.
In other cases, the input can be pre-processed data, e. Similarly, the output can range from one-dimensional, binary behavior control to multi-dimensional continuous output values, e. Initially, solving simulated control tasks was done by manually setting network weights, e. However, this approach is limited to solving simple behavioral tasks such as wall following Wang et al.
Therefore, a variety of training methods for SNNs in control tasks has been researched and published. Instead of focusing on criteria such as field of research, biological plausibility or the specific task, this section is meant to serve as a classification of published algorithms into the basic underlying training mechanisms from a robotics and machine learning perspective. In the first part of this section, some implementations of SNN control are introduced that use some form of Hebbian-based learning.
In the second part, publications are shown that try to bridge the gap between classical reinforcement learning and spiking neural networks. Finally, some alternative methods on how to train and implement spiking neural networks are discussed.
One of the earliest theories in neuroscience explaining the adaption of synaptic efficacies in the brain during the learning process was introduced by Donald Hebb in his book The Organization of Behavior Hebb, Hebbian-based learning rule that rely on the precise timing of pre and post-synaptic spikes play a crucial part in the emergence of highly non-linear functions in SNNs. Learning based on Hebbs rule has been successfully applied to problems such as input clustering, pattern recognition, source separation, dimensionality reduction, formation of associative memories, or formation of self-organizing maps Hinton and Sejnowski, Furthermore, different biologically plausible learning rules have been used for using Spiking Neural Networks in robot control tasks.
However, as the basic underlying mechanism stays the same, training these networks can be achieved in different ways as follows see Table 1. In the table, the two-wheel vehicle means a vehicle with two active wheels.
Because of the absence of direct goals, correction functions or a knowledgeable supervisor, this kind of learning is usually categorized as unsupervised learning Hinton and Sejnowski, Learning based on STDP rule has been successfully applied to many problems such as input clustering, pattern recognition, and spatial navigation and mental exploration of the environment. Wang et al. Compared with other classical NNs, they demonstrated that SNN needs fewer neurons and is relatively simple.
Afterwards, they Wang et al. In a similar research, Arena et al. Their controller allowed the robot to learn high-level sensor features, based on a set of basic reflexes, depending on some low-level sensor inputs by continuously strengthening the association between the unconditioned stimuli contact and target sensors and conditioned stimuli distance and vision sensors. In non-spiking neural networks, many successes in recent years can be summarized as finding ways to efficiently learn from labeled data. This type of learning, where a neural network mimics a known outcome from given data is called supervised learning Hastie et al.
A variety of different neuroscientific studies has shown that this type of learning can also be found in the human brain Knudsen, , e. But despite the extensive exploration of these topics, the exact mechanisms of supervised learning in biological neurons remain unknown. Accordingly, a simple way of training SNNs for robot control tasks is by providing an external training signal that adjusts the synapses in a supervised learning setting. As shown in Figure 6 , when an external signal is induced into the network as a post-synaptic spike-train, the synapses can adjust their weights, for example, using learning rules such as STDP.
After an initial training phase, this will cause the network to mimic the training signal with satisfactory precision. Even though this approach provides a simple, straight-forward way for training networks, it is dependent on an external controller. Especially for control tasks involving high-dimensional network inputs, this may not be feasible. Figure 6.
Designing neural networks through neuroevolution
Supervised Hebbian training of a synapse: The weight of the synapse between pre and post-synaptic neurons, N pre and N post , is adjusted by the timing of the pre-synaptic spike-train s syn and external post-synaptic training signal s train. Several models have been proposed on how this might work, either by using activity templates to be reproduced Miall and Wolpert, or error signals to be minimized Kawato and Gomi, ; Montgomery et al. In the nervous system, these teaching signals might be provided by sensory feedback or other supervisory neural structures Carey et al.
One of these models that is primarily suitable for single-layer networks is called supervised Hebbian learning SHL. Based on the learing rule derived in 8, a teaching signal is used to train the postsynaptic neuron to fire at target times and to remain silent at other times. It can be expressed as.
- A Political and Economic Dictionary of Western Europe (Political and Economic Dictionaries)?
- Neural networks in manufacturing: A survey - IEEE Conference Publication;
- The Nurse Professional: Leveraging Your Education for Transition Into Practice?
- Psychological Treatment of Obsessive-Compulsive Disorder: Fundamentals And Beyond.
Carrillo et al. The spiking cerebellum model is trained by simulating the robotics arm to seven different targets repeatedly.
Designing neural networks through neuroevolution
In contrast to other STDP learning rules, only long-term depression was externally induced by a training signal, which relied on the motor error, namely the difference between the desired and actual state. In a similar experiment, Bouganis and Shanahan trained a single-layer network to control a robotic arm with 4 degrees of freedom in 3D space. The training signal was computed using an inverse kinematics model of the arm, adjusting the synaptic weights with a symmetric STDP learning rule.
More examples can be found in Table 1 with an order by descending year. Classical conditioning Wikipedia, a refers to a learning procedure in which a biologically potent stimulus e. It will result that the neutral stimulus comes to elicit a response e. In the famous experiment on classical conditioning Pavlov and Anrep, , Pavlov's dog learns to associate an unconditioned stimulus US , in this case food, and a conditioned stimulus CS , a bell, with each other. While, it is not clear how the high-level stimuli given in his experiment are processed within the brain, the same learning principle can be used for training on a neural level as well.
Figure 7. The conditioned stimulus CS firing shortly before its associated US will adjust its weights so that N post will fire even in the absence of US. Due to the Hebbian learning rule, the synaptic weight is unchanged when the other, unrelated stimulus causes N post to fire. Following this principle, bio-inspired robots can learn to associate a CS, e.