A synapse is essentially a programmable wire used to connect groups of neurons together. The human brain possesses approximately 10 billion neurons, each of which has direct synaptic connections to approximately 10,000 neurons. Neuromorphic computers aimed at mimicking biological computation, and which have numbers of neurons and synapses approaching biological scale, can be modeled with supercomputers or neural hardware accelerators. However, in order for such neural computing devices to achieve a biologically plausible synaptic density, it is imperative to minimize synaptic size.

This feat is challenging because the synaptic weight of each synapse must be stored. Since digital synapse implementations require that several bits of data per synapse be memorized, analog synapses may be a superior choice. Analog synapses based on floating-gate transistors store the weight as charge that is trapped between insulating layers. The charge can be manipulated by injecting and tunneling electrons to and from the floating node.¹ Such transistors rely on proven technology and allow a relatively high density, rendering them worthwhile synaptic candidates. However, a new class of devices, known as memristive devices, may be the next leap forward in high density synapse fabrication. Memristive devices will allow for fabrication of single device synapses as crossbar arrays on top of complimentary metal-oxide-semiconductor (CMOS) circuits. As the synapses would be in the memristive layer on top of the CMOS,² the entire silicon area would be left for neurons.

Memristive devices are a class of two-terminal resistive devices with a state.³ The state (and thus resistance) of memristive devices systematically changes as a function of state, voltage, and time, enabling device resistance programming. Hewlett-Packard's TiO₂-based memristor has garnered the most attention,⁴ but memristive characteristics have been observed in a variety of materials (e.g. in metal oxides and organic substances) thereby enabling numerous fabrication techniques. A memristor's change in resistance is a consequence of oxygen ion redistribution, itself controlled by application of an electric field. Although memristive device theory can also be extended to describe memories based upon other physical phenomena (such as phase change), herein we limit our discussion to metal-oxide based devices.

Although the first reported memristors were of digital (on/off) programmability, devices with analog programmable resistance value have also recently appeared.⁵ Herein we consider a qualitative model of the analog memristor.^5,6. Let V(t) be the voltage applied across the memristor. The current I(t) is given as:

The current-voltage (I/V) curve (1) has a sinh shape, which is typical of electron tunneling.⁷ From (2) one observes that the programming sensitivity of the device model is a nonlinear function of the voltage (sinhshape).⁸ The programming threshold of these devices is typical. The sinh-shaped programming sensitivity causes small voltages over the device to leave the state virtually unaffected, whereas programming occurs at voltages exceeding the threshold.⁶ Consequently, the programming threshold assists device programming in a controllable manner. Note that, though a memristor⁹ is just a special case of a memristive device, it has become customary to refer to all memristive devices as memristors. Figure 1(a) illustrates an I/V curve of the analog memristor with transient simulation from 0 to 0.5s. The input voltage is shown in Figure 1(c). Each programming pulse renders the device more conductive. Figure 1(b) illustrates how negative programming voltages program the memristor to be less conductive. The simulation was carried out with the following parameters: a1=4e−8, b1=1.2, a2=1.25e−7, b2=1.2, c1=6e−4, d1=2, c2=6.6e−4, d2=3.8.

(a) Current-voltage (I/V) curve of an analog memristor with transient simulation from 0 to 0.5s and input voltage as shown in (c). The device becomes more conductive after each pulse. (b) Transient simulated I/V curve from 0.5 to 1s (negative input pulses). The device is programmed as less conductive.

Spike-timing-dependent plasticity (STDP) with memristors has been proposed.⁸ The idea is that an individual postsynaptic or a presynaptic spike does not induce memristor state change (the voltage stays below the programming threshold) but, when spikes occur simultaneously, their superposed voltage magnitude exceeds the threshold. This STDP scheme has been experimentally verified,⁵ with computation and adaptation divided into different synchronously applied operation phases.⁸ Since then, several groups have proposed that, given specific post- and presynaptic spike shapes, learning can take place asynchronously (without separate computing and adaptation phases).^10–13 Figure 2(a) illustrates spike shapes that realize STDP, whereas Figure 2(b) illustrates the corresponding STDP learning curve. By changing pulse shape, different STDP curves can be generated.¹³

(a) Spike shapes that feature spike-timing dependent plasticity (STDP). V^t₊and V^t_- are the positive and negative programming threshold voltages. (b) Change of memristor state as a function of spike-timing: i.e. the STDP learning curve.

Figure 3(a) illustrates a typical transistor-based synapse. The synapse transistor is directly connected to the membrane potential. Figure 3(b) illustrates how a memristive synapse is interconnected to post- and presynaptic neurons. The presynaptic neurons drive horizontal lines and the postsynaptic neurons are connected to vertical lines. Note that the current through a memristor is dependent on the voltage across it. This is in contrast to transistors in which the current is largely independent of drain-source voltage in saturation. Therefore, if memristors were directly connected to a membrane capacitor, the contribution of each presynaptic spike would depend upon membrane potential. To prevent this, the lines driven by the postsynaptic lines are tied to virtual ground unless a postsynaptic spike is present.^12,13 When the postsynaptic neuron fires, the pulse generator emits a postsynaptic spike, and the integrator is kept at reset for a refractory period T_refr.

(a) Transistor-based synapse and membrane potential. (b) Memristive synapse and pre/postsynaptic connections.

With properly selected spike shapes, memristive synapses can perform spike-timing dependent learning. When these synapses are built as crossbar arrays on top of CMOS neurons, very dense neural hardware appears feasible. However, note that the technology is in an early stage of development. For example, limits of memristive device scaling have yet to be explored. Currently, device dimensions are typically on the order of tens of nanometers, greater than the anticipated several nanometers. Another point to address is reliability issues due to device aging, device-to-device mismatch (e.g. in the programming threshold), and manufacturing imperfections (defects). Such imperfections will play a role in circuit design, but currently the severity of these issues is unknown. As memristive devices are under intensive study, fast progress in device technology is expected. We are currently working on simulation models for memristive devices, as well as searching for computing schemes that would maximally benefit from device physics. Memristive synapses are obviously becoming reality, but their real competitive advantage against mainstream technologies (such as CMOS-based floating-gate memories) has yet to be evaluated.

2. P. Hasler and J. Dugger, An analog floating-gate node for supervised learning, IEEE Trans. Circuits-I 52, pp. 834-845, 2005.
   
   
3. K. Likharev, CMOL: possible hybrid semiconductor/nanodevice circuits, Proc. Euro. Conf. Circuit Theory Design-2 2, pp. II/237, 2005.
   
   
4. L. O. Chua and S. M. Kang, Memristive devices and systems, Proc. IEEE 64 (2), pp. 209-223, 1976.
   
   
5. D. Strukov, The missing memristor found, Nature 453, pp. 80-83 May, 2008.
   
   
6. S. H. Jo, Nanoscale memristor device as synapse in neuromorphic systems, Nano Lett. 10, pp. 1, 2010.
   
   
7. E. Lehtonen, J. Poikonen, M. Laiho and W. Lu, Time-dependency of the threshold voltage in memristive devices, submitted to Int'l Symp. Circuits. Systems (ISCAS), 2011.
   
   
8. J. Yang, Memristive switching mechanism for metal/oxide/metal nanodevices, Nat. Nanotechnol. 3, pp. 429-433, 2008.
   
   
9. G. Snider, Spike-timing-dependent learning in memristive nanodevices, IEEE Int'l Symp. on Nanoscale Architecture, pp. 85-92, 2008.
   
   
10. L. O. Chua, Memristor - the missing circuit element, IEEE Trans. Circuit Theory 18 (5), pp. 507-519, 1971.
    
    
11. M. Pankaala, M. Laiho and P. Hasler, Compact floating gate learning array with STDP, Int'l Conf. Neural Networks, pp. 2, 2009. Atlanta
    
    
12. A. M. Haas, T. Datta, Pamela A. Abshire and Martin C. Peckerar, Two transistor synapse with spike timing dependent plasticity, Digital Repository at the University of Maryland. http://drum.lib.umd.edu/handle/1903/8650
    
    
13. A. Afifi, A. Ayatollahi and F. Raissi, STDP implementation using memristive nanodevice in CMOS-nano neuromorphic networks, IEICE Electronics Express 6 (3), pp. 148-153, 2009.
    
    
14. B. Linares-Barranco and T. Serrano-Gotarredona, Exploiting memristance in adaptive asynchronous spiking neuromorphic nanotechnology aystems, IEEE Conf. Nanotechnology, 2009.