I'm currently reading Ray Kurzweil's book, The Singularity is Near, which I'll properly review later. Among other things, the book talks about the supposed imminence of our being able to simulate the brain. I'm afraid I'm not convinced by his arguments. Don't get me wrong: it's not that I think it's not going to happen. It's just I really think he minimizes the engineering challenges that will have to be overcome to make it happen. I've been interested for many years in the challenge of building brain-like hardware and it's not (to say the least) a trivial problem.

One thing in particular that he mentions but glosses over, is how we will tackle the problem of connectivity. We know that the brain has of the order a trillion neural processors, neurons, linked by as many as thousand trillion synapses: so that's an average of a thousand connections each. According to California Institute of Technology professor Yaser Abu-Mostafa, this connectivity is crucial for learning in biologically-plausible neural networks, and cannot be traded off easily: in other words, you can't just have fewer local connections and a larger (or faster) network and expect to be able to compensate for the deficit.

When you consider that a handful of connections is about as much as you can expect between elements on a chip, and that getting information from one chip to another is even harder (the number of connections around the edge goes as the �2N, where N is the number of elements in the chip), you can see that this communications bottleneck could be a major obstacle.

I've been interested in clever ways to get around this problem for some time. A lot of them are optoelectronic, using light to communicate between chips and boards of neurons. There are also ways of having lots of different neurons share connections. However, one of the most interesting possibilities from my perspective has been true 3D interconnection of stacked chips: chips that have had their backs thinned and are then electronically connected to each other not just via edge connectors (as normal) but also across their surface.

I first heard about a project that Irvine Sensors was doing in this area back in the late 1990s (John Carson, Chief Technical Officer at that time, was particularly interested in building brainlike systems). Lots of companies seem to be into this now, but most seem to be doing it as a way of trying to combine different materials systems (like gallium and silicon) and so different functionalities, rather than for pure connectivity reasons. Also, success has been limited: most teams have stacked just two and three chips rather than the dozens that Irvine Sensors had hoped for when they started working on the problem ten years ago.

So our current best hope is not yet a done deal. Calculations on the back of an envelope are easy. Engineering is hard...

Figure: Irvine Sensors scheme for 3D interconnection back in 1998.