In plain words: A perceptron is built entirely from memristors—resistors that remember past inputs—using them for both the connections and the computing units, with the usual learning rules rewritten to fit. It learns exactly like a standard perceptron, and can approximate any function.
Abstract · Perceptrons from Memristors
Memristors, resistors with memory whose outputs depend on the history of their inputs, have been used with success in neuromorphic architectures, particularly as synapses and non-volatile memories. However, to the best of our knowledge, no model for a network in which both the synapses and the neurons are implemented using memristors has been proposed so far. In the present work we introduce models for single and multilayer perceptrons based exclusively on memristors. We adapt the delta rule to the memristor-based single-layer perceptron and the backpropagation algorithm to the memristor-based multilayer perceptron. Our results show that both perform as expected for perceptrons, including satisfying Minsky-Papert's theorem. As a consequence of the Universal Approximation Theorem, they also show that memristors are universal function approximators. By using memristors for both the neurons and the synapses, our models pave the way for novel memristor-based neural network architectures and algorithms. A neural network based on memristors could show advantages in terms of energy conservation and open up possibilities for other learning systems to be adapted to a memristor-based paradigm, both in the classical and quantum learning realms.
Francisco Silva, Mikel Sanz, João Seixas, Enrique Solano, Yasser Omar
arXiv:1807.04912 · cs.ET, cs.NE, quant-ph · submitted Jul 13, 2018 · updated Dec 26, 2018
abstract · pdf · html · Added new result on universality of memristors, minor changes in the introduction and algorithm, references updated
In any case, I have yet to see a conclusive, publicly explained solution to the significant system-level problems with memristor-based neural architectures, or indeed any analog neural architecture. The best claimed digital architectures are around ~250 fJ per multiply-and-accumulate (MAC) [Groq], and these generally involve 8-bit multiplication, which is extremely expensive in the analog domain thanks to the exponential scaling of power with precision levels. Even if you set aside the monstrous fabrication and device-level variance issues with memristors, DAC and ADC consume tens of pJ per sample in the realistic IP blocks that are commercially available. Although only one pair of DAC and ADC operations is required per dot product, this is still 40 fJ per MAC from DAC and ADC alone, assuming a 256x256 matrix multiplication and not taking other system-level issues into account. This limits memristors to a 5x over current digital architectures, and as nodes shrink, by the time memristors come out, this will be around a 3x. While a 3x is considerable, I don't think it justifies the moonshot-level deep tech risk that memristors will continue to represent. Many hardware companies [Tabula...] have failed attempting to reach something like a 3x in the main figure-of-merit, only to find that system-level issues get them a 1x instead. Besides, I'm sure digital architectures have more than 3x room for improvement- plenty of tricks left for digital!
I'm hoping for a breakthrough, because I am fundamentally an optimist, but memristors have been failing to deliver since 2008.
[1] http://www.cs.utah.edu/~rajeev/pubs/isca16-old.pdf [2] https://ieeexplore.ieee.org/document/7010034/