Selective Change Driven Vision
Selective Change Driven (SCD) vision reduces the amount of visual data to be
processed by selectively choosing those scene events that are most relevant to
the addressed video processing application. The concept of image does not fit
well in this objective, since one single snapshot may contain a lot of data that
is not important for the processing algorithm. Also, a sequence of images is not
the best data structure for event signalling and processing. It seems that a
sequence of pixels is more appropriate in order to obtain the objectives of data
and processing power reduction from the sensor itself to the processing
hardware.
Video capturing and processing is usually based on a sequence of images,
snapshots, taken at regular intervals. Whereas this is the basis of most of
current artificial visual systems, it seems that there are almost no biological
systems that use this methodology. Artificial image processing has evolved from
single photo cameras to modern video cameras; all of them being based on a
sequence of images. Biological systems seem to have a different evolution, from
simple single photoreceptors to complicated vision systems with multiple
photoreceptors even having different roles. It seems that the concept of image,
or snapshot, is missing in most biological systems, at least in the first stages
of the visual process. The objective of our approach is not to mimic biological
systems, but to artificially reduce, by current advances in technology, the
amount of data to be processed, especially from the sensing point of view.
Sometimes current technology outperforms most biological systems, but at other
times biology demonstrates outstanding behaviour which is difficult to match. For
example, it is almost impossible to achieve the current bandwidth of the human
visual system where millions of photoreceptors work in parallel, but current
technology offers photoreceptors that can signal events at speeds several orders
of magnitude above that of biological systems. In our approach, the objective
has been to exploit all that technology can offer, looking at biological systems
for inspiration, but not as an objective.
Commonly, artificial video capturing and processing is based on a sequence of
images. One single image contains a large amount of information that can take a
long time to be processed, or even just moved from one place to another. Video,
as a sequence of images, multiplies the data involved in the processing and
requires high performance architectures for real-time (25~fps) applications. For
example, if higher image rates are required in closed-loop control applications
then the problem may become impossible to solve. Even for standard image rates,
the required hardware can be beyond the scope of embedded or low power
applications, where small size, or power consumption are the main constraints of
the system.
Current vision sensors, or cameras, send complete images at regular intervals.
It is common that only part of the image changes between two consecutive images,
especially if the image rate is very fast, and that the rest of the image
remains unchanged. It therefore follows that any video processing algorithm does
not need those parts of the image that have not changed at all, because a new
result, for a new image, can be obtained just using the past results along with
the changes introduced in the new image. We can go even further: the pixels that
did not change between two images are not necessary for the processing;
moreover, in all probability those pixels that have changed little have almost
no impact on the result, so it is possible that they are not needed either, at
least to some extent depending on the application. The proposed Selective Change
Driven Vision implements these arguments in order to reduce the amount of data
to be acquired, transmitted and therefore processed, thus reducing the hardware
required to process video and/or increasing the system’s performance (usually
both).
The chips images shown at to correspond to our realizations of two SCD sensors.
At top-right is the last SCD sensor designed with 64x64 resolution and fast
response. It is based on a continuous reading logarithmic cell. Figure at
to-left shows the layout of our first CMOS Selective
Change-Driven Vision Sensor. It has 32x32 pixels and delivers the pixel that
have changed most from the last time it was read-out. As a first prototype, it
lacks the resolution and image quality required for most real applications. We are now working on
continuously improving speed, resolution and quality of these sensors.
Experiments
There are some experiments that has been carried out with the new SCD sensor.
The papers at the end of this page include some of these experiments. In the
tracking experiment it was possible to track the electron beam
of an analog oscilloscope, rotating at 5 kHz. The following image-left shows the
set-up of this experiment. The image at middle corresponds to a moving hand
in front of the camera in SCD mode. The image at right is a picture taken with
the SCD sensor since it can work also as a normal imager.
Selected bibliography
- Fernando Pardo, Jose A. Boluda and Francisco Vegara.
Selective Change Driven Vision Sensor With Continuous-Time Logarithmic
Photoreceptor and Winner-Take-All Circuit for Pixel Selection.
IEEE Journal of Solid-State circuits. Vol. 50, No. 3, pg. 786-798. Mars
2015. DOI:
10.1109/JSSC.2014.2386899
- Francisco Vegara, Pedro Zuccarello,
Jose A. Boluda and
Fernando Pardo.
Taking Advantage
of Selective Change Driven Processing for 3D Scanning.
Sensors. ISSN:
1424-8220. Vol. 13, Issue 10. October 2013. pp: 13143-13162.
DOI: 10.3390/s131013143
- Jose A. Boluda, Pedro Zuccarello, Fernando Pardo
and Francisco Vegara. Selective Change
Driven Imaging: A Biomimetic Visual Sensing Strategy.
Sensors. ISSN:
1424-8220. 2011. Vol. 11, Issue 11. November 2011. pp: 11000-11020.
Free PDF available
in MDPI (open access). DOI:
10.3390/s111111000
- Fernando Pardo, Pedro Zuccarello, Jose A. Boluda, Francisco Vegara.
Advantages of Selective Change-Driven Vision for Resource-Limited
Systems.
IEEE
Transactions on Circuits and Systems for Video Technology, vol. 21, no.
10, pp. 1415-1423, October 2011. DOI:
10.1109/TCSVT.2011.2162761. ISSN: 1051-8215.
- P. Zuccarello, F. Pardo, A.
de la Plaza and J.A. Boluda, 32x32 winner-take-all matrix with single winner
selection,
Electronics
Letters, vol. 46, no. 5,
pp. 333-335, March 2010.
