GB2423156A - Wide angle camera system with planar and non planar mirrors - Google Patents

Wide angle camera system with planar and non planar mirrors Download PDF

Info

Publication number
GB2423156A
GB2423156A GB0600818A GB0600818A GB2423156A GB 2423156 A GB2423156 A GB 2423156A GB 0600818 A GB0600818 A GB 0600818A GB 0600818 A GB0600818 A GB 0600818A GB 2423156 A GB2423156 A GB 2423156A
Authority
GB
United Kingdom
Prior art keywords
region
camera
mirror
solid angle
mirror system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
GB0600818A
Other versions
GB0600818D0 (en
GB2423156B (en
Inventor
Jens Schick
Alexander Wuerz-Wessel
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Robert Bosch GmbH
Original Assignee
Robert Bosch GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Robert Bosch GmbH filed Critical Robert Bosch GmbH
Publication of GB0600818D0 publication Critical patent/GB0600818D0/en
Publication of GB2423156A publication Critical patent/GB2423156A/en
Application granted granted Critical
Publication of GB2423156B publication Critical patent/GB2423156B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B37/00Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B17/00Systems with reflecting surfaces, with or without refracting elements
    • G02B17/02Catoptric systems, e.g. image erecting and reversing system
    • G02B17/06Catoptric systems, e.g. image erecting and reversing system using mirrors only, i.e. having only one curved mirror
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50Constructional details
    • H04N23/55Optical parts specially adapted for electronic image sensors; Mounting thereof
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/698Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/12Mirror assemblies combined with other articles, e.g. clocks
    • B60R2001/1253Mirror assemblies combined with other articles, e.g. clocks with cameras, video cameras or video screens

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)

Abstract

A camera system for observing one or more spatial regions, in particular a panoramic camera system, comprises a camera (110) with means (216) eg CCD chip for storing image information in a two-dimensional image region (134), a first mirror system (116) and a second mirror system (210). The first mirror system (116) is in such a form, and/or arranged in such a way, that light rays (124) which impinge on the first mirror system (116), which has at least one non-planar mirror surface (118), from at least one first solid angle region (126) are mapped into a first partial region (138) of the two-dimensional image region (134) of the at least one camera (110). The second mirror system (210) has at least one planar mirror region (210), and is in such a form, and/or arranged in such a way, that light rays which impinge on the second mirror system (210) from at least one second solid angle region (212) are mapped into a second partial region (218) of the two-dimensional image region (134). The second mirror system (210) is also in such a form that it prevents at least a subset of a set (132) of light rays leaving the at least one camera (110) being mapped by the first mirror system (116) into the first partial region (138). Imaging optics 114, support 122 and positioning device 222 are shown. The system may be used for interior and/or exterior observation in a motor vehicle.

Description

I
Wide-angle camera-mirror system for observation purposes
Technical field
As well as passive safety systems, active safety or assistance systems are increasingly used in automotive engineering. Such systems capture, in particular, a current state of a vehicle environment (for instance a region in front of or behind a motor vehicle) or a current state in the interior of the motor vehicle. Such systems can have, for instance, one or more cameras. A camera system which can observe a relatively large region, for instance an interior and an exterior region of a motor vehicle simultaneously, and thus specifically determine a position of individual objects within this region stereoscopically, is proposed.
Prior art
Today, several safety systems in which camera systems are used are known from the automotive field. A basic distinction must be made between observation of the interior and observation of the exterior, which are each done for different purposes.
For instance, DE 101 58 415 Al and US 2003/0098909 Al describe methods and arrangements for optical monitoring of the interior of a motor vehicle. At least one all-round view camera is used, and supplies images in curvilinear co-ordinates. The images are then transformed into cylindrical or planar co-ordinates by means of an equalisation device. These transformed images can then be subjected to electronic image analysis. In this way, for instance, persons in the vehicle interior can be recognised, and the seat position of these persons can be determined. This information can be used, for instance, to control airbag systems. Partial observation of an exterior is also possible with the disclosed system.
EP 1 375 253 A2 also concerns a method of monitoring the interior and/or exterior of a vehicle, and an arrangement with at least one all-round view camera. The captured image data is analysed, at least one region being selected by means of the analysis. This region is captured by means of a second sensor, which can be directed and has restricted spatial coverage, and the data which is captured by the second sensor is subjected to analysis.
From US 2001/0048816 Al, a camera system which has a camera and a convex mirror is known. The system can be used to capture wide-angle images. For instance, the system can be used to observe a reversing region of a motor vehicle.
Also, in US 2001/0048816 Al, a method by which the image information which is acquired by means of the convex mirror can be equalised, so that an image in rectilinear co-ordinates is generated, is described.
In EP 1 197 937 Al, an all-round monitoring system for an object in motion is described, for instance for the exterior monitoring of a motor vehicle. Among other things, an arrangement in which the all-round monitoring system has a hyperboloid- shaped mirror and a camera system which observes this hyperboloid-shaped mirror and in this way generates image information within a specified image region, is described.
In US 2003/0156187 Al, a catadioptric (single camera) sensor system in which one or more mirrors are used to generate stereoscopic images is described. Among other things, a system in which stereoscopic images are generated using two convex mirrors placed one on top of the other is described.
In WO 00/41024, a system for capturing super-wide-angle panoramic images is described. The system has two curved reflectors, the optical errors of which correct each other. Incident rays are first captured by the first reflector and directed onto the second reflector, from which they are again reflected and directed into a camera system.
In S.K. Nayar et al. "Folded Catadioptric Cameras", proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Fort Collins, June 1999, a method is described by which image information which is acquired by reflection on an almost arbitrarily curved mirror surface can be "equalised". Among other things, camera systems which have two mirrors, and in which incident light rays are first reflected by a surface of a curved mirror and then directed by a second mirror to a camera, are described. Mirror systems with hyperboloid or paraboloid mirrors are also described.
However, the systems which are known from the prior art for all-round observation have a series of disadvantages for practical use, particularly in automotive engineering. Thus simple all-round view systems such as those in described in WO 00/41024, EP 1197 937 Al, US 2001/0048816 Al or US 2003/0098909 Al make determining the position of individual objects within the captured image region possible only with difficulty. On the other hand, conventional catadioptric systems for capturing stereo images are usually associated with a high space requirement, since position determination by means of such systems depends heavily on the distance between the virtual points of view of the stereo system (base distance).
Additionally, the image region which such conventional systems capture is usually very small, and does not make all-round observation possible. A combination of multiple curved mirror systems, e.g. as in US 2003/0156187 Al, has the disadvantage, in particular, that the image region of at least one curved mirror in such systems captures the camera itself, resulting in an image region which cannot meaningfully be used ("dead region").
Presentation of the invention A camera system for observing one or more spatial regions is therefore proposed. It avoids the stated disadvantages of the prior art, and in particular is also suitable for use of interior andlor exterior observation in a motor vehicle.
A basic idea of this invention is, in particular, to use the abovementioned "dead" region in the image region of an all-round view camera system for a second mirror system. Correspondingly, a camera system for observation of one or more spatial regions is proposed. It has at least one camera with means for storing image information in a two-dimensional image region, and a first mirror system with at least one non-planar mirror surface. This first mirror system is in such a form, and/or arranged in such a way, that light rays which impinge on the first mirror system from at least one first solid angle region are mapped into a first partial region of the two-dimensional image region of the at least one camera. In principle, the first mirror system can have a course which is curved arbitrarily to a large extent, auxiliary systems (e.g. in the form of software and/or hardware) being required to equalise the image which this first mirror system reflects. These are described, for instance, in S.K. Nayar et al. "Folded Catadioptric Cameras", proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Fort Collins, June 1999. However, it has been shown to be specially advantageous if the first mirror system has a mirror surface which is curved in at least one dimension, following a hyperbolic function. In particular, for instance, the first mirror system can have a curved mirror surface following the course of a first haif- nappe of a two- nappe hyperboloid. Such hyperboloids have two hyperbolic foci: a first hyperbolic focus which is associated with the first haif-nappe on a concave side of the first half-nappe, and a second hyperbolic focus which is associated with a second half- nappe on a convex side of the first haif-nappe. Advantageously, the camera system is in such a form that the at least one camera (or imaging optics which are associated with the camera) has an optical axis and a principal point, the at least one camera being arranged in such a way that the optical axis of the camera is on a straight line connecting the two hyperbolic foci, and that the principal point coincides with the second hyperbolic focus. Such a system is also described, for instance, in EP 1 197 937 Al, Fig. 6. Stated very simply, the camera in this advantageous form acts like a pinhole camera, and the hole of the pinhole camera comes to lie in a hyperbolic focus on the convex side of the hyperboloid. In this way, light rays which impinge on the hyperboloid are suitably projected into the image region of the camera.
Alternatively, the first mirror system can also have a curved mirror surface following the course of an elliptical paraboloid. The elliptical paraboloid should have an axis of rotation, the at least one camera having an optical axis which lies on the axis of rotation of the elliptical paraboloid. In this case, it is advantageous to use a camera (with projection optics if appropriate) and/or image processing which make orthographic projection possible. With such projection, the camera "observes" an observation volume corresponding to a cuboid which is parallel to the optical axis of the camera. Objects of the same size within this observation volume appear in the two-dimensional image region of the camera with the same size.
Orthographic projections and suitable cameras are known to the person skilled in the art, and are described, for instance, in Gluckman J., Nayar S.K., Thoresz K.J.: "Real-Time Omnidirectional and Panoramic Stereo", in "Proceedings of the DARPA Image Understanding Workshop", 1998, pp. 299-303.
The camera system also has at least one second mirror system with at least one planar mirror region. This second mirror system is in such a form, and/or arranged in such a way, that light rays which impinge on the second mirror system from at least one second solid angle region are mapped into a second partial region of the two-dimensional image region. The second mirror system is also in such a form that it prevents at least a subset of a set of light rays leaving the at least one camera being mapped by the first mirror system into the first partial region. Thus this second mirror system exploits the "dead" image region described above, within which, if only the first mirror system was used, the camera would "see" its own mirror image. The second mirror system can exploit this "dead" image region partially or fully, or the second mirror system can even extend into the image region of the first mirror system.
It has been shown to be specially advantageous if the second mirror system has a single reflecting surface. It is specially advantageous if the second solid angle region, which is captured by the second mirror system, is smaller than the first solid angle region, which is captured by the first mirror system. The two solid angle regions can advantageously overlap at least partially. It is specially advantageous if the camera system also has a positioning device, which aligns and/or arranges the second mirror system in such a way that the second solid angle region can be adjusted. This form is particularly advantageous if the camera system also has an image processing system with means for full or partial equalisation of the image information which is stored in the two-dimensional image region, and additionally or alternatively, an image processing system with means for carrying out a triangulation method for determining the position of an object which is mapped in the first partial region and second partial region of the two-dimensional image region. For instance, the described camera system can be used in such a way that first a first solid angle region is mapped into a first partial region of a twodimensional image region. Subsequently or in parallel, a second solid angle region which at least partly overlaps with the first solid angle region is mapped into a second partial region of the two-dimensional image region. Thus the image data which is generated from both the first mirror system and the second mirror system is now stored in the same twodimensional image region of the camera. By means of an image recognition method (e.g. based on edge detection) which is known to the person skilled in the art, an object which is mapped in both partial regions of the two-dimensional image region, for instance a person inside the motor vehicle or a pedestrian outside a motor vehicle, is detected. By means of a triangulation method, the position and/or distance of the object relative to the camera system is calculated.
This position can, for instance, be made available to an on-board computer or directly to a driver of the motor vehicle. Alternatively or additionally, if a danger is detected, for instance a pedestrian stepping into the roadway, counter-measures can be initiated directly.
The invention can be extended, in particular, in such a way that a specially interesting solid angle region is selected (for instance automatically in the form of a specially clearly defined object, or manually by a driver), and the second mirror system is then adjusted by means of the positioning device, in such a way that the second solid angle region includes the specially interesting solid angle region fully or partially. In particular, the method can also be carried out in such a way that the second mirror system screens or scans away the first solid angle region of the first mirror system with its second solid angle region (for instance controlled by a stepping motor), thus to determine gradually, by means of a triangulation method, the position of all, some selected or at least all essential objects in the image region of the first mirror system.
Compared with the arrangements which are known from the prior art, the proposed camera system has the advantage that the "dead" image region of curved mirror systems is optimally exploited. The camera system also makes it possible to determine the position of objects in the image region, and the precision of the determined location information far exceeds the precision of known all-round view systems. Also, the size of the described camera system is so small that the described camera system for interior andlor exterior observation can be used in motor vehicles in which a small size is a decisive advantage. The system can also be used as optional equipment instead of previously known camera systems, for instance in a headlight housing of a motor vehicle.
Drawings The invention is explained in more detail below on the basis of the drawings.
Fig. I shows an all-round view camera system corresponding to the prior art, with a "dead" image region; Fig. 1 a shows image information of the camera system according to Fig. 1, stored in a two-dimensional image region; Fig. 2 shows a camera system according to the invention, using the "dead" image region for a second mirror system; Fig. 2a shows image information, stored in a two-dimensional image region of a camera system according to Fig. 2; Fig. 3 shows a schematic representation of a form of the curvature of the first mirror system in the form of a hyperboloid; Fig. 4 shows a schematic form of a curvature of the first mirror system in the form of a paraboloid; and Fig. 5 shows a schematic flowchart of a possible form of a method according to the invention for optical environment monitoring by means of a camera system according to the invention.
Variants In Fig. 1, a panoramic camera system corresponding to the prior art is shown. The camera system has a camera 110, which has a housing 112 and corresponding imaging optics 114. The camera system also has a curved mirror 116, the surface 118 of which in this embodiment is curved according to a haif-nappe of a two- nappe hyperboloid. Correspondingly, the curved mirror 116 has a focus 120, which is associated with the surface 118 and is on the concave side of the hyperboloid surface 118. The precise ray course is explained below (see Fig. 3). The curved mirror 116 is held by a support 122.
The curved mirror 116 reflects light rays 124 which impinge on the mirror surface 118 to the camera 110. The camera 110 thus "sees" the same image which a virtual observer in the focus 120 of the curved mirror 116 would perceive. The maximum solid angle region within which incident light rays 124 are reflected to the camera is identified symbolically in Fig. 1 by reference numbers 126, and is delimited by the limit rays 128. The precise position of the limit rays 128 depends on the form of the curved mirror 116 and/or support 122.
As also shown in Fig. 1, the camera 110 is also reflected in the curved mirror 116.
In this embodiment, the solid angle region which is taken up by the camera 110 is delimited by the edge rays 130, which extend from the outer edge of the housing of the imaging optics 114 of the camera. These edge rays 130 are reflected by the surface 118 of the curved mirror 116 back into the camera 110, and thus form a "dead" solid angle region 132, which essentially has no usable image information.
The camera 110 has means 216 for storing image information in a twodimensional image region, for instance a CCD chip 216. In Fig. 1 a, an illustration of the image information which is acquired using an arrangement according to Fig. 1 a corresponding to the prior art in a twodimensional image region 134 of such a CCD chip 216 is shown. As an example, the use of a camera system in the interior of a motor vehicle is shown. The limit rays 128 in Fig. 1, which delimit the maximum solid angle region 126 of the camera system, are mapped in the two- dimensional image region 134 in Fig. la as a round or elliptical viewing range limit 136, according to the form of the curved mirror 116, the imaging optics 114 and the CCD chip 216. All rays which are reflected within the maximum solid angle region 126 are mapped in the two-dimensional image region 134 within this viewing range limit 136 and in a first partial region 138. Because of the curvature of the surface 118 of the curved mirror 116, the image information is distorted in comparison with reality. As can also be seen in Fig. 1 a, the edge rays 130 in the two-dimensional image region 134 are mapped in the form of a round or elliptical dead region limit 140. In the so-called dead region 142 within this dead region limit 140, in the image region 134 image information is mapped via the camera 110 using the arrangement according to Fig. 1. This dead region 142 thus contains no useful image information, and is shown hatched in Fig. 1 a.
In Fig. 2, a camera system according to the invention with two mirror systems is shown. The camera system again has a camera 110 with imaging optics 114, and a curved mirror 116, for instance again a curved mirror 116 with a surface 118 which follows the course of a half-nappe of a twonappe hyperboloid. Accordingly, the mirror 116 again has a focus 120. In this embodiment too, in principle the camera 110, because of the edge rays 130 of the camera going out from the housing of the imaging optics 114, would map itself, so that again a "dead" solid angle region 132 would result. However, in this embodiment according to Fig. 2, a second mirror system 210, which in this embodiment has only a planar mirror, is inserted into the "dead" solid angle region 132. The mirror system 210 "blocks" at least part of the bundle of rays which goes out from the camera 110 and is delimited by the edge rays 130, and thus "eliminates" part of the "dead" solid angle region 132. Instead, the second mirror system 210 reflects rays, which are delimited by limit rays 214, and which impinge on the second mirror system 210 from a second solid angle region 212, into the camera 110.
In Fig. 2a, similarly to Fig. 1 a, image information which is stored in a two- dimensional image region 134 of a CCD chip 216 of the camera 110, and was acquired using an arrangement according to Fig. 2, is shown schematically. The image information from the maximum solid angle region 126 is again stored in the first partial region 138 within the viewing range limit 136. Again in this embodiment, the dead region 142 is delimited by the dead region limit 140, which represents a mapping of the edge rays 130 of the camera. However, within the dead region 142, image information resulting from the rays of the second solid angle region 212 and their mapping onto the CCD chip 216 is (also) now stored according to the invention. In this embodiment, in which an approximately square secrnd mirror system 210 is used, the limit rays 214, which delimit the second solid angle region 212, are mapped as a rectangular second image limit 216. In the second partial region 218 of the two-dimensional image region 134, which is within the second image limit 216, image information from the second solid angle region 212 is stored. Thus the dead region 142 of the two- dimensional image region 134 is exploited by the second mirror system 210.
Instead of the shown arrangement with an approximately square second mirror system 210, other forms of the mirror system 210 can be implemented, for instance round mirrors or mirror systems with multiple individual mirrors. For instance, the second mirror system 210 can also have an autonomous stereoscopic mirror system, which comprises at least two individual (e.g. planar) mirrors. The second mirror system 210 can also be in such a form that the dead region 142 is, for instance, completely filled by the second partial region 218. An arrangement in which the second mirror system 210 extends beyond the dead solid angle region 132 is also conceivable. In this case, the second partial region 218 in Fig. 2a extends beyond the dead region limit 140, so that the second partial region 218 also covers parts of the first partial region 138 which do not belong to the dead region 142.
As shown in Fig. 2, the second solid angle region 212, from the point of view of the camera 110, acts like the viewing range of a virtual observer in the virtual point of view 220. If the second solid angle region 212 captures a region of the maximum solid angle region 126 of the first mirror system 116, in the second partial region 218 of the two-dimensional image region 134 according to Fig. 2a image information is stored about objects which are also in the first partial region 138, which contains image information of the first mirror system 116. This is shown symbolically in Fig. 2a in the form of a head of the driver of a motor vehicle. In this case, the two mirror systems 116, 210 define a stereoscopic mirror system. The resolution of this stereoscopic mirror system is essentially determined by a distance D between the focus 120 of the first mirror system 116 and a centre 220 of the second mirror system 210. If, in the second partial region 218 and the first partial region 138 of the two-dimensional image region 134, one or more objects are identified (for instance by suitable image processing), a position of the object(s) relative to the mirror system can be calculated from the known geometry of the stereoscopic system. Here triangulation methods which are known to the person skilled in the art, and are described, for instance, in US 2003/0156187 Al, can be used.
The second mirror system 210 in the embodiment according to Fig. 2 is connected via a positioning device 222 to the first mirror system 116. This positioning device allows, for instance, rotation of the second mirror system 210 about an optical axis 224. Positioning in the form that the angle a between a central ray 226 of the second solid angle region 212 and the optical axis 224 is changed is also possible. In particular, the positioning device 222 can be in such a form that the positioning device 222 has one or more stepping motors, with which the second solid angle region 212 can be adjusted exactly. In this way, for instance, the second partial region 218 of the two-dimensional image region 134 can be adjusted to any object, manually or automatically. For instance, the position of any object can be determined.
In Figs. 3 and 4, possible curvatures of the surface 118 of the first mirror system 116 are shown, as examples and schematically. Whereas Fig. 3 describes a hyperbolic curvature, in Fig. 4 a parabolic curvature is shown.
As shown in Fig. 3, the surface 118 of the first mirror system 116 follows the course of a first half-nappe 310 of a hyperboloid. A first focus 120 is associated on its concave side with this haif-nappe 310. Similarly, for this hyperboloid a second half- nappe 312, a second focus 214 and two directrices 316 exist. Any light rays 318, 320 which reach the focus 120 are reflected in the points P1 and P2 on the surface 118 of the first haif-nappe 310 to the second focus 314. A virtual observer in the second focus 314 thus perceives the same image as a virtual observer in the first focus 120. The camera system according to the invention makes use of this geometrical peculiarity in that a camera 110 which has a principal point 322 which coincides with the second focus 314 is used. In this way, the camera 110 acts like a pinhole camera with the hole in the principal point 322.
In Fig. 4, a version of the surface 118 which follows the course of a paraboloid 410 is shown schematically. In contrast to a hyperboloid, a paraboloid 410 has only one focus 120. Light rays 412, 414 which reach a virtual observer in the focus 120 are reflected on the surface 118 of the paraboloid 410 in the points P1, P2, and after this reflection run parallel to the optical axis 224. To capture all the light rays which are reflected by the surface 118, therefore, a camera 110, the optical axis 224 of which coincides with the axis 224 of the paraboloid 410, is required. The camera 110 must also be capable of capturing all the rays which are reflected from the surface 118, i.e. the camera must have a clear diameter of at least d. Also, the camera 110 (or its imaging optics 114) must similarly map all points P1, P2 on the surface 118, which can be achieved by an orthographic camera (for instance using an telecentric lens or screen). Such a camera is described, for instance, in Joshua M. Gluckman and Shree K. Nayar: "Planar catadioptric stereo: geometry and calibration", Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Fort Collins, June 1999.
In Fig. 5, a method according to the invention for optical environment monitoring by means of a camera system according to the invention is shown. The shown method steps do not necessarily have to be carried out in the shown sequence, and additional method steps, not shown in Fig. 5, can also be carried out.
First, in method step 510, a first solid angle region 126 is mapped by means of a first mirror system 116 into a first partial region 138 of a two-dimensional image region 134. In method step 512, this image information of the first solid angle region 126, which is stored in the first partial region 138, is processed by means of a suitable image processing algorithm, for instance by means of the algorithm which is described in S.K. Nayar et al. "Folded Catadioptric Cameras", proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Fort Collins, June 1999. In particular, this equalisation 512 simplifies image recognition in the subsequent steps. In a parallel method step 514, a second solid angle region 212 which at least partly overlaps with the first solid angle region 126 is mapped into a second partial region 218 of the two- dimensional image region 134. By means of a suitable image recognition method (e.g. one which works according to the edge detection method) which is known to the person skilled in the art, in step 516 an object is detected in both partial regions 138, 218 of the two-dimensional image region 134. This object can be, for instance, a person inside the motor vehicle or a pedestrian or another motor vehicle outside a motor vehicle. However, the object can also be merely parts of such an object. For instance, certain objects for which a deliberate search can be carried out can be specified, for instance a number plate of a motor vehicle which is moving ahead. Then, in method step 518, if an object has been identified in both partial regions 138, 218, by means of a triangulation method, the position and/or distance of the object relative to the camera system is calculated.
Subsequently or in parallel, in method step 520 the solid angle region 212 of the second mirror system 210 is readjusted. For instance, this readjustment 518 can take place manually, for instance in that a user manually selects a region of special interest, or automatic adjustment can also take place. In particular, the readjustment takes place by means of apositioning device 222. In particular, this readjustment 520 can take place by the second solid angle region 212 screening away the maximum solid angle region 126 (see above). By means of this screening method, for instance, the whole maximum solid angle region 126 of the first mirror system 116 can be gradually screened away, so that objects in this solid angle region 126 are detected and their position is determined. The method can be coupled with various other method steps, for instance with appropriate warning functions, if there is a dangerously close corresponding object in the path of a motor vehicle or it is to be expected that this object moves into the path. Following certain objects, for instance by integration into a traffic jam following system, in which the gap to a motor vehicle which is moving ahead is kept constant, is also conceivable. The method can also be used, for instance, in a system which can be added optionally (e.g. as optional equipment) to previously existing camera systems in motor vehicles.

Claims (13)

  1. Claims 1. Camera system for observation of one or more spatial regions,
    with a) at least one camera (110) with means (216) for storing image information in a two-dimensional image region (134); b) at least one first mirror system (116) with at least one non-planar mirror surface (118), the first mirror system (116) being in such a form, and/or arranged in such a way, that light rays which impinge on the first mirror system (116) from at least one first solid angle region (126) are mapped into a first partial region (138) of the two-dimensional image region (134) of the at least one camera (110); and c) at least one second mirror system (210) with at least one planar mirror region (210), - the second mirror system (210) being in such a form, and/or arranged in such a way, that light rays which impinge on the second mirror system (210) from at least one second solid angle region (212) are mapped into a second partial region (218) of the two-dimensional image region (134), - the second mirror system (210) being also in such a form that it prevents at least a subset of a set (132) of light rays leaving the at least one camera (110) being mapped by the first mirror system (116) into the first partial region (138).
  2. 2. Camera system according to the preceding claim, characterized in that the first mirror system (116) has a mirror surface (118) which is curved in at least one dimension, following the course of a hyperbolic function.
  3. 3. Camera system according to the preceding claim, characterized in that the first mirror system (116) has a curved mirror surface (118) following the course of a first half-nappe (310) of a two-nappe hyperboloid, the hyperboloid having a first hyperbolic focus (120) which is associated with the first haif-nappe (310), and a second hyperbolic focus (314) which is associated with a second haif-nappe (312), the at least one camera (110) having an optical axis (224) and a principal point (322), and the at least one camera (110) being arranged in such a way that the optical axis (224) of the camera (110) is on a straight line (224) connecting the two hyperbolic foci (120, 314), and that the principal point (322) coincides with the second hyperbolic focus (314).
  4. 4. Camera system according to Claim 2, characterized in that the first mirror system (116) has a curved mirror surface (118) following the course of an elliptical paraboloid (410), the elliptical paraboloid (410) having an axis of rotation (224), the at least one camera (110) having an optical axis (224), and the at least one camera (110) being arranged in such a way that the optical axis (224) of the camera (224) is identical to the axis of rotation (224).
  5. 5. Camera system according to the preceding claim, characterized in that the camera (110) has imaging optics (114) for orthographic projection.
  6. 6. Camera system according to one of the preceding claims, with additionally: d) an image processing system (228) with means (228) for full or partial equalisation of the image information which is stored in the two-dimensional image region (134).
  7. 7. Camera system according to one of the preceding claims, with additionally: e) an image processing system (228) with means (228) for carrying out a triangulation method for determining the position of an object which is mapped in the first partial region (138) and second partial region (218) of the two-dimensional image region (134).
  8. 8. Camera system according to one of the preceding claims, characterized in that the second solid angle region (212) is smaller than the first solid angle region (126).
  9. 9. Camera system according to one of the preceding claims, characterized by a positioning device (222) to adjust the alignment and/or arrangement of the second mirror system (210) to determine the second solid angle region (212).
  10. 10. Method of optical environment monitoring by means of a camera system according to one of the preceding claims, with the following steps: a) a first solid angle region (126) is mapped into a first partial region (138) of a two-dimensional image region (134); b) a second solid angle region (212) which at least partly overlaps with the first solid angle region (126) is mapped into a second partial region (218) of the two-dimensional image region (134); c) by means of an image recognition method, an object which is mapped in both partial regions (138, 218) of the two-dimensional image region (134) is detected; and d) by means of a triangulation method, a position and/or distance of the object relative to the camera system is calculated.
  11. 11. Method according to the preceding claim, with the following additional step: e) by means of an image processing method, the image information in at least one partial region (138, 218) of the two- dimensional image region (134) is equalised.
  12. 12. Method according to one of the two preceding claims, with the following additional step: 1) a specially interesting solid angle region (212) is selected, and the second mirror system (210) is then adjusted by means of a positioning device (222), in such a way that the second solid angle region (212) includes the specially interesting solid angle region (212) fully or partially.
  13. 13. Use of a camera system according to one of the preceding claims directed at a camera system, for monitoring an interior and/or exterior of a motor vehicle.
GB0600818A 2005-02-15 2006-01-16 Wide-angle camera-mirror system for observation purposes Expired - Fee Related GB2423156B (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
DE102005006754A DE102005006754A1 (en) 2005-02-15 2005-02-15 Wide-angle camera mirror system for observation purposes

Publications (3)

Publication Number Publication Date
GB0600818D0 GB0600818D0 (en) 2006-02-22
GB2423156A true GB2423156A (en) 2006-08-16
GB2423156B GB2423156B (en) 2007-02-14

Family

ID=35998097

Family Applications (1)

Application Number Title Priority Date Filing Date
GB0600818A Expired - Fee Related GB2423156B (en) 2005-02-15 2006-01-16 Wide-angle camera-mirror system for observation purposes

Country Status (3)

Country Link
DE (1) DE102005006754A1 (en)
FR (1) FR2882159B1 (en)
GB (1) GB2423156B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2940578A1 (en) * 2008-12-23 2010-06-25 Bernard Taillade Video surveillance system, has mobile mirror generating variable circular image on sensor of camera, where circular image is represented irrespective of panoramic scene along site and azimuth position of mobile mirror
CN102555919A (en) * 2010-12-01 2012-07-11 株式会社电装 Electronic mirror
US8305425B2 (en) 2008-08-22 2012-11-06 Promos Technologies, Inc. Solid-state panoramic image capture apparatus
RU2535631C2 (en) * 2013-03-13 2014-12-20 Федеральное государственное унитарное предприятие "Российский Федеральный Ядерный Центр-Всероссийский Научно-Исследовательский Институт Технической Физики имени академика Е.И. Забабахина" (ФГУП "РФЯЦ-ВНИИТФ им. академ. Е.И. Забабахина") Method of locating object in surrounding space
US9717980B2 (en) 2011-11-08 2017-08-01 Active Silicon Limited Optical system for a roulette wheel

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102010041490A1 (en) 2010-09-27 2012-03-29 Carl Zeiss Microimaging Gmbh Optical instrument and method for optical monitoring

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1197937A1 (en) * 2000-10-11 2002-04-17 Sharp Kabushiki Kaisha Surround surveillance apparatus for mobile body

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1197937A1 (en) * 2000-10-11 2002-04-17 Sharp Kabushiki Kaisha Surround surveillance apparatus for mobile body

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8305425B2 (en) 2008-08-22 2012-11-06 Promos Technologies, Inc. Solid-state panoramic image capture apparatus
FR2940578A1 (en) * 2008-12-23 2010-06-25 Bernard Taillade Video surveillance system, has mobile mirror generating variable circular image on sensor of camera, where circular image is represented irrespective of panoramic scene along site and azimuth position of mobile mirror
CN102555919A (en) * 2010-12-01 2012-07-11 株式会社电装 Electronic mirror
CN102555919B (en) * 2010-12-01 2014-12-31 株式会社电装 Electronic mirror
US9717980B2 (en) 2011-11-08 2017-08-01 Active Silicon Limited Optical system for a roulette wheel
RU2535631C2 (en) * 2013-03-13 2014-12-20 Федеральное государственное унитарное предприятие "Российский Федеральный Ядерный Центр-Всероссийский Научно-Исследовательский Институт Технической Физики имени академика Е.И. Забабахина" (ФГУП "РФЯЦ-ВНИИТФ им. академ. Е.И. Забабахина") Method of locating object in surrounding space

Also Published As

Publication number Publication date
GB0600818D0 (en) 2006-02-22
GB2423156B (en) 2007-02-14
FR2882159A1 (en) 2006-08-18
FR2882159B1 (en) 2009-09-18
DE102005006754A1 (en) 2006-08-17

Similar Documents

Publication Publication Date Title
US6304285B1 (en) Method and apparatus for omnidirectional imaging
US10830588B2 (en) Surveying instrument for scanning an object and image acquistion of the object
US6744569B2 (en) Method and apparatus for omnidirectional three dimensional imaging
JP4620131B2 (en) Panoramic 3D adapter for optical equipment and combination of panoramic 3D adapter and optical equipment
US11015932B2 (en) Surveying instrument for scanning an object and image acquisition of the object
EP1172678A2 (en) Omnidirectional vision sensor
TW201802541A (en) 3D multi-aperture imaging devices, multi-aperture imaging device, method for providing an output signal of a 3D multi-aperture imaging device and method for capturing a total field of view
RU2000117577A (en) GENERAL DIRECTIONAL IMAGE DEVICE
GB2423156A (en) Wide angle camera system with planar and non planar mirrors
CN108470360B (en) Image device for generating depth map by using non-plane projection image and related method thereof
JP2004258266A (en) Stereoscopic adapter and distance image input device using the same
US20140152823A1 (en) Techniques to Obtain Information About Objects Around a Vehicle
CN112255880B (en) Stereo camera
US12101455B2 (en) Multi-aperture zoom digital cameras and methods of using same
EP2848995A1 (en) Multi-view image system with a single camera and a transreflective device
JP6983740B2 (en) Stereo camera system and distance measurement method
JP2006054503A (en) Image generation method and apparatus
JP7043375B2 (en) Stereo camera, in-vehicle lighting unit, and stereo camera system
US11780368B2 (en) Electronic mirror system, image display method, and moving vehicle
US20220321779A1 (en) Measuring system with panoramic image acquisition functionality
WO2021005921A1 (en) Stereo camera and light unit with integrated stereo camera
JP2010019924A (en) Photographing apparatus
HICKS et al. Catadioptric sensors for panoramic viewing
Torres et al. Omnidirectional stereoscopic vision systems for planetary exploration rovers
KR102288951B1 (en) vehicle and control method thereof

Legal Events

Date Code Title Description
PCNP Patent ceased through non-payment of renewal fee

Effective date: 20150116