Transcript omniviews_achievements.ppt

IST-1999-29017 – OMNIVIEWS
Omnidirectional Visual System
Final Review
September 27-28, 2001
Lisbon
Agenda of the meeting
First Day
14:30
15:00
15:10
15:20
16:00
16:30
16:50
17:10
17:20
18:00
18:30
20:30
Reviewers’ private meeting
Welcome
Goal of Review
Omniviews Main Achievements
Coffee Break
Mirror design principles
Mirror design tools
Demos Introduction
Surveillance Demo
Transmission Demo
End of first Day
Dinner
Jose' Santos-Victor
Pekka Karp
Giulio Sandini
Branislav Micusik
Jose' Santos-Victor, Claudia Decco
Tomas Pajdla, Alex Bernardino
Pedro Soares, Giulio Sandini
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Agenda of the meeting
Second Day
9:00
9:40
10:10
11:10
11:40
Navigation Demo
Future Outlook and General Discussion
Reviewer’s Private Meeting (with coffee)
Preliminary Evaluation Report
End of meeting
José Santos-Victor
Giulio Sandini
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Main Facts
Open-scheme, Assessment Phase Project
Consortium:
DIST - University of Genova - Genova
CMP - Czech Technical University in Prague
VISLAB - Instituto Superior Técnico - Lisbon
Project Start and Duration:
September 1st 2000 – One year
Funding: 100 K€
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Project’s Main Objective
The main objective of the project is to integrate optical,
hardware, and software technology for the realization of a
smart visual sensor, and to demonstrate its utility in key
application areas.
In particular our intention is to design and realize a lowcost, digital camera acquiring panoramic (360°) images and
performing a useful low-level processing on the incoming
stream of images.
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Key Technologies
Retina-like visual sensor
Omnidirectional Mirrors
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Specific Objectives of
Assessment Phase



Define the optimal profile of a mirror matching a retina-like
visual sensor. Optimal in the sense that direct read-out of
panoramic images is obtained.
Demonstrate its utility in key application areas
If successful present a follow-up proposal
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Methodologies
1.
2.
3.
4.
Use the currently available SVAVISCA
camera for initial experiments
Design and simulate mirror using
SVAVISCA camera
Realize the OMNIVIEWS mirror for the
current sensor
Demonstrate the mirror in key
applications
SVAVISCA
Pixel layout
Simulated image:
SVAVISCA camera
Hyperbolic mirror
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Mirror’s design principle
1.
2.
Uniform Cylindrical
Projection
Direct read-out through
log-polar mapping
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
OMNIVIEWS Mirror
Mirror’s Profile
Experimental Set-up and test images
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Assessment Criteria
1.
2.
3.
4.
5.
6.
7.
Direct read-out of panoramic images
Frame rate
Resolution and layout of the sensor
Mirror profile and size
Lens characteristics
Camera cost
Image quality
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
AC1: Direct read-out of
panoramic images
Direct read-out from OMNIVIEWS:
About 30,000 read-out operations
Image Obtained from a conventional camera:
About 1.8 M operations required:
882,000 read-outs (30 times more)
882,000 additions
30,000 divisions
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
AC2: Frame rate
Currently the maximum read-out frequency is fixed by
the camera’s interface (PC Parallel port) limiting the
frame rate to about 12 frames/s.
More than 25 frames/s is achievable with a faster
interface (e.g. USB or PCMCIA)
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
AC3: Resolution and Layout of
the sensor
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
AC4: Mirror Profile and Size
Mirror profile and size meets the original plan (6 cm.).
Furthermore:
1) New technology for mirror realization
2) Mirror’s design tool of general utility
3) Design and realization of “mixed-mirror”
4) Overall size can be reduced
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
AC5: Lens characteristics
Standard C-Mount lenses have been used for all
experiments and demos
No difficulties in principle are envisaged for the
design of smaller size lenses (possibly including
the mirror).
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
AC6: Camera Costs
Cost of obtaining panoramic images is zero in our case
Compared to conventional solutions no extra-cost for the
mirror is required. Lower cost is possible with the new
“glass-based” technology.
The cost of the sensor is equivalent to the cost of
conventional sensors realized with the same technology
and with the same size.
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
AC7: Image quality
The project will be successful if we demonstrate that it is
possible to create virtual images by simple reading out the
pixels from the proposed sensor and to use such images in the
aimed applications ….



Topology of images meets the quality criteria
Evaluation of numerical approximations
Three demonstrations:
1) Surveillance (two parts)
2) Navigation
3) Image Transmission

Further processing experiments:
1) Localization using Agam fiducials
2) 3D reconstruction
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Additional remarks
1.
2.
3.
4.
Software mirror-design tools have been developed
New kind of mirror have been proposed extending the
original plan (i.e. the “mixed mirror”)
10 scientific papers have been published
Plans for the future are clearer.
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Future Outlook
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Objectives



Miniature Omnidirectional Camera with increased
performance
Pre-industrial prototype
Focused applications
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Miniature Camera
1. Increase the resolution (and/or reduce the size) of the
sensor using currently available (but forefront) CMOS
technology
2. Improve and integrate optical components
3. Faster Camera Interface
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Sensor’s Layout
Current CMOS 0.35 µm technology:
min pixel size 6.8 µm – 33,000 pixels (equivalent: 1060x1060)
Forefront CMOS 0.18 µm technology:
min pixel size 3.6 µm – 100,000 pixels (equivalent: 2000x2000)
In this case: about 100 K read-out operations are equivalent to
6.3 Million operations required by a “conventional” camera with
2000x2000 pixels.
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Simulations are worst than
actual images
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Mirror’s Technology
Investigate the integration of current mirror-lens assembly
from an optical design point of view (application driven)
Possibly adopt cheaper technology such as “glass coating”
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Applications



(Remote) Surveillance (e.g. traffic monitoring and emergency callbox for highways)
Endoscopes for inspection of body cavities (pipe-like).
Sewer inspection systems
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project
Consortium



Add “optic-design” expert
Add “silicon designer”
Add Industrial partners for realistic
requirements (surveillance + medical)
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme - Assessment Project