Introduction
In the past few decades, computer vision (CV) has achieved considerable progress with advanced intelligent algorithms and computing hardware. [1-3] However, most image-data-based processing algorithms require a large number of parameters that are usually stored in the memory module, which will cause frequent data transfer between memory and processor when the systems receive sensor information.[4, 5] Inspired by the in-memory computing feature in the human brain and combined with modern deep neural networks (DNN), the crossbar arrays made of emerging nonvolatile memories (eNVM) are implemented to accelerate the multiply-accumulate (MAC) operations which dominated mostly in DNN.[6-10, 37] In addition to the synaptic devices exploration, recently, as also the core components of neuromorphic computing, there was plenty of research about searching for the hardware implementation of neurons based on emerging materials and devices.[11-14] For instance, the memristive neurons, including Mott memristors,[15-19] redox memristors,[20, 21] phase-change memristors,[22]etc, all of them can emulate the leaky integrate-and-fire (LIF) function of biological neurons.
However, the device type and materials between the hardware implementation of synapses and neurons usually differ from each other, which will cause additional fabrication costs in large-scale integration and severe limits on the scalability for further applications.[15, 21] Nowadays, many researchers put forward the idea of reconfiguring device’s functions on the same hardware platform.[14, 23-25] One of the studies made use of reconfigurable synaptic and neuronal functions in the V/VOx/HfWOx/Pt memristors for spiking neural network,[23] manipulating the ion distributions in HfWOx memristors to enable devices working on different modes.
Moreover, inspired by how neurotransmitters modulate human neural networks, investigators tend to exploit moveable ions, such as H+, Li+ and O2-,[24, 26-30] to regulate the electrical properties of materials, which has made neuromorphic hardware advance a big step. For instance, by changing the local distribution of hydrogen ions, researchers have demonstrated the reconfigurable perovskite nickelate electronics for reservoir computing and incremental learning.[24]
In a complete neuromorphic system, it is also critical to pre-process external information after sensing from the outside world. Most of the information humans receive is obtained through vision, simulating the vision systems of humans is of great importance to the artificial perception system.[31] There were also many explorations about building an artificial vision system to process the data correlated with vision. However, little effort had been devoted to combining the reconfiguration ideas with energy-efficient neuromorphic vision systems, which can help reduce complexity of the system.
Inspired by ionic regulation methods and reconfiguring advantages, we propose an energy-efficient vision system based on reconfigurable ion-modulated memtransistors. With different stimuli ranges, the temporal scales of ion dynamics inside the devices can be well controlled. As for the short-term dynamics, the accumulation effect can help filter the random noises and enhance the original patterns simultaneously, which was demonstrated in the reconstruction from a set of noisy images. After that, we investigate the relationship between the channel conductance changes and stimuli amplitudes. The observed nonlinear relation can both be used for softplus-like neurons and filtering units. By changing external stimuli, long-term channel conductance modulation can also be achieved to implement weight storage. Based on the above considerations, we present an architecture for neuromorphic vision systems based on the reconfigurable ion-modulated memtransistors. In the system-level performance demonstration, an artificial neural network (ANN) was implemented to recognize the Fashion-MNIST datasets where the filtering units, synapse weights and activation neurons were all based on the ion-modulated memtransistors. Through detailed analysis and testing of the mapping strategies and noises on the network-level performances, we prove that the neuromorphic vision system can help recognize images in practice with relatively high accuracy and improved robustness.