Abstract

The aim of this project is the realization of an omnidirectional vision system for surveillance robotics inspired by biological vision. We would realize a sensor which exploits two innovative technologies. The first is that of omnidirectional catadioptric sensors, and the second is that of resolution variant CMOS sensors, also known as "retina-like" sensors. Omnidirectional catadioptric vision sensors are systems composed by a standard camera equipped with a field of view between 50° and 90° and by a rotational-symmetric convex mirror, which reflects the light gathered at 360° horizontally and at 100°-160° vertically into the camera. In CMOS "retina-like" sensors the photo-sensible elements (pixels) are not organized on the classical orthogonal grid, but on a radial grid. This organization makes the sensor composed by a high-resolution zone in the center (foveal region), and a second region with resolution decreasing toward the periphery (peripheral region). The biological world will give inspiration both for the realization of the mirror shape (which composes the catadioptric sensor), and for the algorithms which will process the images acquired by the retinal sensor.

The project will start from the work of a former European project "Omni-directional visual system" (FP5 RTD - Future and emerging technologies project No: IST-1999-29017) realized by:
- LIRA-Lab, University of Genova, Genova, Italy
- CMP, Czech Technical University, Prague, Czech Republic
- IST/ISR, Instituto Superiore Técnico, Lisboa, Portugal

The former project focalized on the integration of optic and electronic components building the sensor, and on the demonstration of the advantages of this sensor in some applications. In our projects we would continue the study of the sensor characteristics, and we would explore further improvements by exploiting a more biologically-inspired design of the whole sensor.

The possible improvements of the sensor are both in hardware and in software. Thank to the knowledge of the scientific advisor of the project we will study the best omindirectional mirror shape for the surveillance robotics application. Beside the visual task, that require an optimized design by itself, the mirror shape would also have to match the peculiarity of the retinal sensor: in this field we will further evolve the shape of the mirrors proposed in the literature.
A second part of the project, only marginally investigated in the above project, is the realization of algorithms for omnidirectional vision which exploit the retinal structure of the sensor, taking inspiration by the human and animal vision. In this second part of the project, the expertise of the second partner of the project in the analysis and in the functioning of the human retina will be very important.

The main applications of this sensor, which will be investigated, are in the field of mobile robotics, automatic surveillance and distributed vision systems. In fact, an omnidirectional catadioptric vision sensor, conjugated with a retinal-like receptor, will realize a great reduction of bit to be processed and of the bit transmitted over an intelligent sensor network in order to understand the visual input.

The topic of this proposal was partially investigated by a former European project in the years 2000-2001: "Omni-directional visual system" carried on by partners in University of Genova (Italy), Czech Technical University (Czech Republic), and Instituto Superior Técnico (Portugal).

As today, this is the only research attempt of coping retina-like sensors with omnidirectional vision. Another interesting project attempting to exploit the retinal sensor geometry for vision application is being carried on by University at Buffalo in collaboration with Amherst Systems, NASA and GRIT.

The latter of these two projects have focused mainly on the development of the sensor as integrated circuit and only now they have started to focus on software dedicated to retinal vision. On the other side, the first project showed that the fusion of the technology of catadioptric omnidirectional vision sensor with that of resolution-variant "retina-like" sensor, brings a drastic reduction of the computational burden required by the image analysis and of the amount of information to be transmitted along a net linking together multiple sensors. Despite the fact that few researchers are working on retinal sensor, this is a mature technology. The first retinal sensors, realized by G. Sandini at the end of the 80s, was built on a CCD sensor with 2000 pixels. The shift to CMOS technology have permitted an always growing integration and sensor resolution from 8000 pixels black and white, to 8000 pixels color, to end with the current 33000 pixels color sensor.

In the former European project, the design of the omnidirectional mirror has focused only on the optimization of the shape to obtain a direct reading of the 360° panoramic view from the CMOS sensor. It has not be taken into account the possibility of designing a task-dependent, ad-hoc omnidirectional mirror, which has been proved to be a common and successful approach in omnidirectional vision robotics. Four motivation can be found to design custom profile mirrors:

1. with a custom mirror is possible to implement vision systems with special imaging properties, e.g. constant resolution, single projection point;

2. The custom mirror can implement the image processing in-hardware by means of optic transforms, e.g. unwarping, optic compensation of distortions;

3. The custom mirror can be designed for a particular mobile robots, taking in consideration its geometry and task to be performed, in a way to simplify the robot behaviour programming;

4. The custom mirror can be designed to better exploit the characteristics of a peculiar image sensor.

The two innovative technologies composing an omnidirectional retina-like vision system, are becoming mature, and singularly taken they are almost ready to enter the market of computer vision and robotics.

On the algorithmic side, quite a few research has been carried on trying to exploit via software a space variant approach. This has been mainly done by transforming an orthogonal lattice into a radial one and remapping the image with this transformation, or by resampling the image with a radial lattice. These strategies have shown that a great computational power can be saved, while at the same time yielding good results in the tracking and automatic robot positioning within known environment.

In the field of computer vision there have been some attempts of biologically-like vision systems, but none of the proposed approaches have been considered a breakthrough. It has been strongly highlighted that, beside the poor knowledge of how the animal vision system works, the unsatisfactory results are due to the difference between the synthetic and the biological vision sensor. The utilization of a retina-like camera, and the development of new algorithms for this sensor could shed some more lights on robot vision tasks.