In the past few years we have seen an increasing number of projects and demonstrations at the Telluride Neuromorphic Engineering Workshops that involve multi-chip spiking systems interfaced via the address-event representation (AER). Indeed, a renewed interest in spiking neural networks is leading to the design and fabrication of an increasing number of VLSI AER devices that implement various types of networks of integrate-and-fire (I&F) neurons.^1–5 Such devices can be used in conjunction with PCI-AER boards of the type described in the article by Dante et al.⁶ In this case, they can be considered as a new generation of hardware neural-network emulators that enable researchers to carry out simulations of large networks of spiking neurons with complex dynamics in real-time, possibly solving computationally demanding tasks.

These systems use mixed-mode analog/digital circuits, exploiting the best of both worlds, and use asynchronous logic to implement event-based communication across multiple chips. They are mainly used to implement specific models of biological neural systems for basic research and scientific investigation: but with the additional aim of developing the technology and infrastructure for engineering applications along the way.

In this article we present one of these new breed of devices. We show how it can be used in conjunction with the PCI-AER board⁶ to emulate in real-time spiking neural networks with realistic synaptic dynamics, and to characterize spike-based learning algorithms in arbitrarily-configured networks of spiking neurons.

Block diagram of the aVLSI chip architecture. Small trapezoids in the array represent integrate-and-fire neurons, while squares represent inhibitory and excitatory synapses. Output spikes can be redirected to input synapses via the address-event-representation (AER) interface.

The aVLSI chip was implemented using a standard AMS 0.8μm CMOS process, and comprises a linear array of 32 low-power I&F neurons, a 2D array of 32×8 synaptic circuits, and input/output AER interfacing circuits (see Figure 1). The chip's silicon synapses can update their synaptic weights via a learning algorithm based on the recently-proposed spike-timing-dependent plasticity (STDP) mechanism. If a pre-synaptic spike arrives at the synaptic terminal before a post-synaptic spike is emitted, within a critical time window, the synaptic efficacy is increased. Conversely, if the post-synaptic spike is emitted just before the pre-synaptic spike arrives, synaptic efficacy is decreased.

The silicon synapses also comprise bistability circuits for driving the synaptic weight to one of two possible analog values (either potentiated or depressed). These circuits drive the synaptic-weight voltage with a current that is superimposed on that generated by the STDP circuits and which can be either positive or negative. If, on short time scales, the synaptic weight is increased above a set threshold by the network activity via the STDP learning mechanism, the bistability circuits generate a constant weak positive current. In the absence of activity (and hence learning) this current will drive the weight toward its potentiated state. If the STDP circuits decrease the synaptic weight below the threshold, the bistability circuits will generate a negative current that, in the absence of spiking activity, will actively drive the weight toward the analog value, encoding its depressed state.

The STDP and bistability circuits allow us to implement learning and long-term storage of the synaptic states. The use of the AER communication protocol allows us to access individual synapses of the network for providing input signals, to read from each neuron generating output signals, and most importantly to (re)-configure the neural network topology.

Network of central pattern generators (CPGs) coupled via excitatory connections.

(top) Mean probability of long-term potentiation (LTP) over all STDP synapses in the chip, as a function of pre- and post-synaptic firing rates. (bottom) Mean probability of long-term depression (LTD).

At the 2004 Neuromorphic Engineering Workshop at Telluride, CO, Francesco Tenore and Murat Sekerli used the aVLSI chip and PCI-AER board to implement central-pattern-generator (CPG)-type spiking neurons with different kinds of biologically-plausible neuron couplings. Specifically, they coupled 16 pairs of mutually-inhibiting neurons with excitatory connections (see Figure 2). They were able to change the strength of the synaptic weight, which was controlled by an external voltage reference, and vary the type of connectivity patterns in the network (via the PCI-AER board). This way, they were both able to observe different types of network behaviors and reproduce activity patterns that have been seen, in some form, in biology. This simple experiment, set-up and carried out in the last three days of the workshop, demonstrated the system's flexibility in configuring arbitrary types of networks of spiking neurons.

Another experiment started at the 2004 workshop involved the characterization of the learning properties of the STDP synapses as a function of the mean firing rates of pre- and post-synaptic neurons. This experiment (initially carried out by Massimiliano Giulioni and Patrick Degenaar) was recently extended: we computed the probabilities of inducing long-term potentiation (LTP) and long-term depression (LTD) in all of the synapses of all of the chip's neurons (see Figure 3).

In this experiment we stimulated each synapse with Poisson-distributed spike trains via the PCI-AER board, and generated post-synaptic firing rates by injecting constant currents into the neurons. To measure the probability of LTP of a synapse we first reset its weight to its low (depressed) state and then applied pre- and post-synaptic stimulation. We subsequently determined the state of the synapse by measuring the response of the post-synaptic neuron to a regular pre-synaptic spike train. We repeated this procedure 50 times and applied it to all synapses of all neurons in the array.

To measure the probability of LTD of a synapse we applied a similar experimental protocol, but first initialized the synapse to its high state. The particular shape and the position of the region where LTP/LTD occurs can be modified by varying the parameters of the bi-stability and the STDP circuits. For low values of mean post-synaptic firing rates, LTP does not occur. For high and increasing values, on the other hand, the probability of LTP varies with a bell-shape dependence on the mean pre-synaptic firing rates. The probability of LTD follows a complementary tendency and is in accordance with theoretical predictions.⁵

The STDP learning circuits allow us to implement a learning mechanism useful for real-time unsupervised learning tasks. They also provide a mechanism to set arbitrary (bistable) synaptic weights in a supervised way, by producing appropriate pre- and post-synaptic firing rates, without requiring dedicated pins or wires for each synapse in the array. The AER communication infrastructure and devices of the type presented are actively under development by the neuromorphic engineering community. The simple examples described here indicate that this technology can be used reliably in massively parallel VLSI networks of I&F neurons for real-time simulation of complex spike-based learning algorithms.

2. R. J. Vogelstein, U. Mallik and G. Cauwenberghs, Silicon spike-based synaptic array and address-event transceiver, Proc. of IEEE Int'l Symp. on Circuits and Systems, pp. 385-388, 2004.
   
   
3. F. Tenore, R. Etienne-Cummings and M. Lewis, A programmable array of silicon neurons for the control of legged locomotion, Proc. IEEE Int'l Symp. on Circuits and Systems, pp. 349-352, 2004.
   
   
4. E. Chicca, D. Badoni, V. Dante, M. D. Andreagiovanni, G. Salina, S. Fusi and P. Del Giudice, A VLSI recurrent network of integrate and fire neurons connected by plastic synapses with long term memory, IEEE Trans. Neural Net. 14 (5), pp. 1 September, 2003.
   
   
5. S. -C. Liu and R. Douglas, Temporal coding in a silicon network of integrate-and-fire neurons, IEEE Trans. on Neural Networks 15 (5), pp. 1 September, 2004.
   
   
6. G. Indiveri, E. Chicca and R. Douglas, A VLSI array of low-power spiking neurons and bistable synapses with spike timing dependent plasticity, submitted to IEEE Trans, on Neural Networks, 2004.
   
   
7. V. Dante, P. Del Giudice and A. M. Whatley, Interfacing with address events, The Neuromorphic Engineer, 2005.