Building applications inspired by the human eye
Prof. Khaled Salama holds a fabricated flexible photoreceptor array. Photo: KAUST
Computational power has been a significant engine in the technological, industrial and economic growth of societies worldwide. Since the advent of microprocessors, we've relied on the computing abilities of machines to perform complex calculations in a fraction of the time that the human brain would require. But the raw computational power capable of outperforming the human brain for complex calculations pales compared to a human's ability to recognize patterns and take action or, for example, identify an individual's face among a crowd.
Neuromorphic engineering or neuromorphic computing is a branch of science inspired by the human brain, and focused on building and organizing the structures of practical computer systems. By applying some of what we know about the brain's function into designing artificial neural systems, scientists hope to maximize important advances in artificial intelligence (AI) and machine learning to build better computer vision systems, autonomous robots and cars, smarter sensors, as well as many other applications.
A team of researchers in the King Abdullah University of Science and Technology (KAUST) Sensors Lab, led by Professor Khaled Nabil Salama, is focused on developing brain-mimicking neuromorphic circuits to build efficient vision sensors. In a paper published earlier this year in Nature titled "A flexible capacitive photoreceptor for the biomimetic retina," the team of scientists detail exciting research on hybrid perovskite-based flexible photoreceptors whose capacitance is tunable by light stimulation, similar to the human eye's retina.
Drop casting photosensitive polymer while fabricating Flexible photoreceptor array
Electrical engineers are used to working on tuning capacitors by changing their properties. Still, it was an interesting discovery by a student in the group to learn that capacitor properties could not only be altered by shining a light on it, but also be influenced differently based on the light's color, such as green, blue or red.
"We found that the range that it affects is the same range as the human eye, which is what we call visible range, so we started thinking about this as more than a capacitor. It looked like the eye, because the eye changes its properties when light comes in," Salama said.
The next step for the Sensors Lab scientists was to seek the expertise of University colleagues working on image characterization and brain pattern recognition to build a full system, connecting the vision and processing. Working with Professor Boon Ooi, who specializes in optical detection systems, the team's aim was to essentially connect the eye to the brain, so that when the eye sees something, the brain identifies the object.
Useful applications to help save energy
The development of smart cities is a growing priority for both wealthy and emerging nations concerned about a sustainable future. Digitization, data optimization and the energy required to power computational processing represent a central priority. Sensors are everywhere — from the smartphone cameras that people use to capture daily life to security surveillance and monitoring — and they're increasingly required and able to make decisions on the spot.
"As autonomous cars remove the need for drivers, we rely on sensors, cameras and lasers to actually monitor the vehicles' environment," Salama said. "Having a technology like this will allow the autonomous car to actually navigate faster because it cannot wait until an image is sent to a computer to analyze it and feedback info before the car breaks to avoid hitting an obstacle."
Neuromorphic computing can be extremely useful to save power by increasing procession speed and accuracy. It means building hardware that resembles the human brain more than an accurate calculator. There's a growing need for devices replicating the human retina's photoreceptors, such as sensors.
Drop casting photosensitive polymer while fabricating Flexible photoreceptor array
A good example is security cameras that continuously record pictures and video without precisely knowing what they're capturing. Continuous recording consumes a significant amount of energy and computing power. There's also the data communication required for sending this information. Designing and using a capacitor as a tuned visual sensor instead of a traditional camera saves a lot of power.
"It assesses the image of the scene in the same way the brain does. So we now have in our hands a type of security camera that mimics exactly how humans recognize things," Salama said.
The potential of the KAUST researchers' proposed sensor for neuromorphic vision applications is vast, as it expands the range from sensing to perception, and is supported by machine learning and deep neural networks.
Having such a camera on a drone, for instance, can greatly optimize the drone's flight time, which usually doesn't exceed an hour. In specific applications, such as military, the drone may be required to take some pre-defined action based on the images it captures. Those decisions can be made on the spot by the device without the need to send the information to someone prior to action.
The KAUST team now aims to go beyond their small 10x10 pixels solution to upscaling the size and resolution. The engineering work also involves exploring other types of materials that could offer better accuracy and sensitivity.