Neuromorphic Computing with Dynamic Vision System and Depth Camera.

Motivation

The point of this documentation is to describe and provide examples on how to use a depth camera and a dynamic vision sensor. In addition, the documentation introduces the use of neuromorphic hardware to compute data generated by the different cameras. The general purpose of the project is to showcase the benefits of using a dynamic vision sensor as well as the processing of events on neuromorphic hardware.

Advantages

A dynamic vision sensor generates events when there are changes in the field of view. Specifically, the camera sends ON/OFF events when the brightness rises or diminishes respectively. In addition, the sensor is able to capture changes with a high temporal fidelity and accuracy. Compared to a traditional frame-based camera where the complete scene compose a frame at a specific rate, the DVS is able to capture specific events depending on the rate of changes happening in the visual scene. This supports a more efficient computing of data and allows a considerable economy in energy consumption. In this documentation, we will focus on the Prophesee EVK4 device.

A neuromorphic hardware aims to compute information, and more specifically neural networks with a biological approach. Contrary to current hardware (CPU/GPU) where neural networks are synchronously computed, neuromorphic hardware takes example from biological neurons to compute spikes and thus drastically reduce power consumption. The spiking neural network models running on neuromorphic chips also offers a significant reduction in time execution and almost completely aligns with DVS. As we will see in the tutorials, the use of neuromorphic chips can even perform post processing filtering without sacrificing execution time. Here, we are going to use the Intel Loihi 2 as neuromorphic hardware.

Code

This repository https://github.com/rouzinho/Neuromorphic-Computing.git gathers the documentation, source code and the samples to run all the examples and tutorials. The source code can be found in the src folder.