The smart sensors group exploits principles of biological neural systems for optical sensor development and signal processing for advanced vision applications.

The smart sensors group develops and designs full-custom vision and image sensors in state-of-the-art nanometer CMOS technology, and builds embedded systems containing analogue VLSI (very large scale integration) circuits and sensors, and digital signal processing devices.

Specialized data processing algorithms and tools are developed for different sensor architectures and application demands.

Digital ASICs (application-specific integrated circuits) with asynchronous sensor data interfaces and processing cores are custom-made for compact, embedded vision systems based on biology-inspired sensors.

Several types of CMOS vision and image sensors for visible light and thermal infrared (IR) have been developed and designed by ARC`s smart sensors group:

128×128 pixel high-dynamic-range, high temporal resolution, low-power dynamic vision sensor with in-pixel analog signal processing, asynchronous data output, on-chip programmable bias generator and programmable matrix configuration. (joint design effort with ETH Zürich, T. Delbruck) [b,d]

2×256 pixel dual-line optical transient sensor with in-pixel analog signal pre-processing, on-chip precision time stamp generation and digital arbitration for high-speed vision. [c,g]

64×64 pixel transient vision sensor for the thermal infrared (IR) range with in-pixel analog signal pre-processing and asynchronous output using micro-machined bolometer IR detector technology. [a,f,h]

QVGA (304×240) ultra-wide dynamic range CMOS imager and high-temporal-resolution transient vision sensor with focal-plane data compression, sparse asynchronous readout and on-chip programmable modes-of-operation, regions-of-interest and multi-channel bias generator. [e,j,k]

The sensors are based on a unique temporal-contrast analogue pixel circuit (i,l) and are commonly characterized by:

• Wide dynamic range (robust operation under strongly varying illumination conditions)
• High temporal resolution (high-speed machine vision)
• Near complete temporal redundancy suppression (low data rate)
• Highly efficient asynchronous event-based data representation
• Low power consumption

In contrast to traditional CCD or CMOS imagers that encode image irradiance and produce constant data volume at a fixed frame rate - irrespective of scene activity - these sensors contain arrays of autonomous, self-signaling pixels which individually respond in real-time to relative changes in illumination. Because there is no pixel readout clock, no initial time quantization takes place at the focal plane.

Transient pixels report changes with low latency and high temporal resolution, imaging pixels initiate exposure cycles only if new local scene information is present. Pixels that are not stimulated by activity in the observed scene do not generate output. The result is a near complete suppression of data redundancy and the consequential reduction in output data volume. As output bandwidth is automatically dedicated to dynamic parts of the scene, this type of sensor is especially suited for applications involving motion detection and analysis.

The most effective approach to processing address-event-representation (AER) data takes advantage of the efficient coding of the visual information by directly interpreting the spatial and temporal information contained in the data stream. Customized AER data processing algorithms and tools for the different sensor architectures are developed by ARC`s smart sensors group according to application and customer demands.

In order to realize compact, embedded systems based on neuromorphic vision sensors, custom digital post-processing hardware is required. The smart sensors group develops and designs specialised digital ASICs with full-custom asynchronous sensor data interfaces that support the unique information representation of AER-based vision and image sensors. SPARC-compatible processor cores and on-chip integrated memory allow for efficient sensor data processing and straightforward implementation of proprietary AER algorithms.