MXPA00005531A - An omnidirectional imaging apparatus - Google Patents

An omnidirectional imaging apparatus

Info

Publication number
MXPA00005531A
MXPA00005531A MXPA/A/2000/005531A MXPA00005531A MXPA00005531A MX PA00005531 A MXPA00005531 A MX PA00005531A MX PA00005531 A MXPA00005531 A MX PA00005531A MX PA00005531 A MXPA00005531 A MX PA00005531A
Authority
MX
Mexico
Prior art keywords
reflector
image
paraboloid
scene
clause
Prior art date
Application number
MXPA/A/2000/005531A
Other languages
Spanish (es)
Inventor
Shree K Nayar
Original Assignee
The Trustees Of Columbia University In The City Of New York
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by The Trustees Of Columbia University In The City Of New York filed Critical The Trustees Of Columbia University In The City Of New York
Publication of MXPA00005531A publication Critical patent/MXPA00005531A/en

Links

Abstract

The invention disclosed herein is an omnidirectional imaging apparatus for sensing an image of a scene (130) from a single viewpoint. The omnidirectional imaging apparatus includes a truncated, substantially paraboloid-shaped reflector (135) positioned to orthographically reflect principal rays of electromagnetic radiation radiating from said scene, said paraboloid-shaped reflector having a focus coincident with said single viewpoint of said omnidirectional imaging apparatus, including said paraboloid-shaped reflector. The omnidirectional imaging apparatus also includes telecentric means (112, 113), optically coupled to said paraboloid-shaped reflector, for substantially filtering out principal rays of electromagnetic radiation which are not orthographically reflected by said paraboloid-shaped reflector. The omnidirectional imaging apparatus also includes one or more image sensors (111) positioned to receive said orthographically reflected principal rays of electromagnetic radiation from said paraboloid-shaped reflector, thereby sensing said image of said scene.

Description

AN OMNIDIRECTIONAL IMAGE APPARATUS BACKGROUND OF THE INVENTION 1. Field of the Invention This invention relates to the omnidirectional image perception with reference to a single point of view, and more particularly, to such image perception using a reflector of essentially paraboloid and truncated shape. 2. Discussion of the Previous Art For many applications such as teleconferencing surveillance, remote sensing, photogrammetry, model acquisition, virtual reality, computer graphics, machine vision and robotics, it is desirable that an image formation system have a large field of view to be capable of Take as much information as possible about the world around this.
Traditional imaging systems include a camera with a lens that provides a perspective projection of an image. However, a camera with very wide angle lenses has only a limited view field (for example, covering less than a half hemisphere). This limited field of view can be expanded by tilting and moving the camera over a visual field of a complete imaging system around its projection center. One of such system is described by S.E. Che "VR of Quick Time - An Image-Based Approach to Virtual Environment Navigation" SIGGRAPH Procedure 9 (8): 29-38, August (1995), The article by L. McMillan and G. Bisho "Plenóptico Modeling: An Image-Based Surrendering System "Computer Graphics: Proc from SIGGRAPH, August 1995, page 39-46, also describes a pan and tilt system. This system has two serious disadvantages though, one being the obvious disadvantages associated with a device that has critical movement ports, and the second being the significant amount of time required to make a full rotation in order to see the surrounding world. This time limitation makes such a device not suitable for real-time applications.
Another approach to increase the field of vision in an imaging system is by employing the so-called "fish eye" lens as described in EL Hal and others "Omnidirectional Vision Using Fish Eye Lenses" SPIE Volume 728 Optical Illumination and Perception _d Image for Machine Vision (1986), page 250. Since fish eye lenses have a very short focal length, the field of view can be as large as one hemisphere. The use of such lenses in a problematic imaging system, however in the sense that these are significantly larger and more complex than conventional lenses. In addition, it has been difficult to develop a fish eye lens with a fixed point of view for all points of the relevant scene. U.S. Patent No. 5,187,667 issued to Zimmerman, and U.S. Patent No. 5,359,363 issued to Kuban et al. Is also directed to the use of fish eye lenses to replace the tilt and lock mechanisms. conventional panoramic toms and therefore suffer from the same disadvantages.
Other prior art devices have used reflecting surfaces to increase the field of vision. One such device prior art is discussed by V.S. Nalw "A True Omnidirectional Observer" AT &T Bell Laboratorie Technical Memorandum, BL0115500-960115-01, January 1996. Nalw describes the use of multiple planar reflecting surfaces and conjunction with multiple-load coupling device ("CCD") cameras to obtain a 360 ° panoramic image of a 50 band. ° of a hemispheric scene. Specifically, e Nalwa, four planar mirrors are arranged in the form of a pyramid, with a camera being placed on top of each of the four planar reflecting sides, and with each chamber viewing slightly more than 90 ° by 50 ° of the hemispheric scene. This system suffers of the serious disadvantage of requiring multiple sensore to capture a 360 ° image. In addition, this system suffers from the inherent problems associated with distortion in seams when separate images are combined to provide a full 360 ° view.
The arched reflector surfaces have also been used in conjunction with the image sensors. Both Yag and others "Evaluating the Effectiveness of Map Generation by Tracking Vertical Borders in the Sequence of Omnidirectional Image Formation," and IEEE Internationa Conference on Robotics and Automation, June 1995, page 2334 and Yagi et al. "Map Based Navigation for a Mobile Theft with Omnidirectional COPIS Image Sensor", IEE Transactions on Robotics and Automation, volume II number 5, October 1995, describes a conical projection image sensor (COPIS) which uses a surface cone reflector to gather images of the surrounding environment, and process the information to guide the navigation of a mobile robot. Even when the conical projection image sensor is able to obtain 360 ° vision, it is not a true omn ± directional imaging sensor because the field of view is limited by the apex angle of the conical mirror and by Viewing angle of camera lenses. In addition, the conical projection image sensor does not have a single point of view but instead has a place of viewpoints that lie on a circle. This place of multiple views causes distortion in the captured images, which can not be deleted to obtain images in pure perspective Yamazawa et al., "Obstacle Detection with HIMROMNI Vision of Omnidirectional Image Sensor "International Conference on Robotics and Automation IEEE, October 1995, page 1062, describes an improvement attributed to the conical projection image sensor system which involves the use of a hyperboloidal reflecting surface instead of A conical surface As discussed here, the rays of light which are reflected off the hyperboloidal surface, regardless of where the origin point is, will all converge to a single point, thus allowing a perspective view Even though the use of the hyperboloidal and advantageous mirror in the sense that it allows the perception of a complete perspective image, because the light rays that constitute the reflected image converge at the reflector focal point, the placement of the sensor in relation to the the reflective surface is critical, and any disturbance will impair the quality of the image. In addition, the use of a perspective projection model inherently requires that by increasing the distance between the sensor and the mirror, the mirror cross section must be increased. Therefore, practical considerations dictate that in order to keep the mirror at a reasonable size, the mirror should be placed near the sensor. This in turn causes complications that arise with respect to the design of the image sensor optics. In addition, the image mapping perceived for usable coordinates requires a complex calibration due to the nature of the convergent image. A further disadvantage is that the relative positions of the mirror and the optics can not be changed while maintaining a single point of view. Therefore, a hyperboloidal mirror system can not take advantage of the relative movement of the mirror and the optics to adjust the system's field of view, while maintaining a unique point of view.
Before Yamazawa and others, the patent for United States of America No. 3,505,465 issued to Donald Ree also describes the use of a hyperboloidal reflector surface. Therefore, the description of Rees also suffers from the same disadvantages as that of Yamazawa and others.
The prior art devices described above fail in one of two ways. These either fail to provide a true omnidirectional imaging apparatus which is capable of perceiving a scene from a single point of view making it impossible to provide distortion-free images with the apparatus, or they provide an apparatus that requires complex calibration and implementation.
SYNTHESIS OF THE INVENTION The disadvantages of the prior art as discussed above are essentially solved by the present invention which in one aspect is an omnidirectional imaging apparatus for perceiving an image of a scene from a unique standpoint that includes a reflector shaped essentially in Paraboloid and truncated placed pair orthographically reflect the main rays of the electromagnetic radiation radiating from the scene. The paraboloid shaped reflector has a focus coincident with the single point of view of the omnidirectional imaging apparatus. The omnidirectional imaging apparatus also includes telecentric lenses, optically coupled to the paraboloid-shaped reflecto, to essentially filter out the main radiation rays. electromagnetic which are not reflected orthographically by the paraboloid reflector. The omnidirectional image forming apparatus further includes one or more image sensors positioned to receive the orthographically reflected main rays of the electromagnetic radiation from the paraboloid-shaped reflector, thereby perceiving the image of the scene.
The paraboloid-shaped reflector of the present invention can be either convex or concave. The telecentric means may include telecentric lenses, a telecentric opening or collimating lenses.
Preferably, the paraboloid-shaped reflector comprises a paraboloidal mirror having a surface which substantially obeys the equation expressed in cylindrical coordinates: h 2 h z being an axis of rotation of the surface, r being a radial coordinate and h being a constant. Since the equation represents a symmetric rotation surface, the shape of the surface is not a function of the angular coordinate f.
In a preferred embodiment of the invention, one or more sensors comprises one or more video cameras. These video cameras may employ one or more charge coupled devices or one or more charge injection devices. Alternatively, the one or more image sensors may comprise photographic film. In another preferred embodiment, at least one image sensor has a non-uniform resolution to compensate for the non-uniform resolution of the image reflected from the paraboloid-shaped reflector.
Preferably, the paraboloid-shaped reflector comprises a mirror truncated in a plane which includes the foc of the reflector of paraboloid shape and which is perpendicular to the axis passing through the focus and the vertex of the reflector is formed paraboloid.
In an exemplary embodiment, the paraboloid-shaped reflector is mounted on a fixed base and the one or more image sensors are mounted on a movable base, whereby the movement of the one or more image sensors produces a changeable field of view. Alternatively, the paraboloid-shaped reflector can be mounted on a movable base and the one or more image sensors can be mounted on a fixed base so that the movement of the paraboloid-shaped reflector produces a changing field of view. In each of these embodiments, it is also preferred that an approach lens be provided to optically couple the one or more image sensors and the reflector in a paraboloid manner.
In a further example embodiment, the one or more image sensors provide an image signal representative of the scene image. An image signal processing apparatus is coupled to one or more image sensors, which convert the image signal d of the image sensors into image signal data. The image signal processing apparatus then forms a map of the image signal data in a Cartesian coordinate system to produce a perspective image or a cylindrical coordinate system to produce a panoramic image. The image signal processing may include d interpolation means to provide interpolated image data, so that the interpolated image formation data and the image signal data are combined to form a digital image. Advantageously, the image processing apparatus may also include means for approaching a preselected part of the digital image to provide an amplified image of the preselected part from a predetermined focal distance.
In a preferred embodiment, the omnidirectional imaging apparatus comprises at least one lens optically coupled to one or more paraboloid image and reflector sensors. This coupling lens can be a lens for rapid change of plane, a lens d microscope, or a field flattening lens. Advantageously, the field leveling lens has a field curvature approximately opposite to the curvature of the reflector field. paraboloid shape. Preferably, the field flattening lens is either a plano-concave lens or a meniscus aplanatic lens.
In yet another preferred arrangement, the omnidirectional imaging apparatus is used to form an essentially spherical scene image by employing two reflectors of essentially truncated paraboloid shape positioned to orthographically reflect main radiation and electromagnetic rays radiating from two hemispheric scenes complementary The two paraboloid-shaped mirrors are placed to share a common paraboloid axis. In addition, when the two reflectors of paraboloid shape are convex, they are placed from back to back along their truncation planes, so that they share a common focus point. When the two paraboloid-shaped reflectors are concave, they are placed in such a way that their vertexes coincide.
In a further exemplary embodiment of the present invention, a plurality of beam splitters provide for dividing the main rays orthographically reflected from an electromagnetic radiation from the paraboloid-shaped reflector into a plurality of ray bundles. In this embodiment, a plurality of image sensors is required, with each image sensor positioned to receive at least a plurality of ray bundles, and so perceive a portion of the scene image.
In yet another embodiment of further example a plurality of dichroic ray splitters is provided by dividing the main rays orthographically reflected from electromagnetic radiation from the paraboloid-shaped reflector into a plurality of monochromatic principal rays of the electromagnetic radiation. As in the previous embodiment, a plurality of image sensors are required, with each image sensor positioned to receive at least one of the plural monochromatic main rays of electromagnetic radiation, and therefore to perceive at least one monochromatic image of the scene.
According to the nature of the present invention, a method for perceiving an image of a scene from a single point of view is also provided. In an example embodiment, the method includes the steps of: (a) orthographically reflecting the main electromagnetic radiation rays radiating from the scene onto a reflector of essentially truncated paraboloidal shape so that the unique point of view of the omnidirectional image formation method coincides with a focal point of the paraboloidal reflector; (b) telecentrically filtering a substance part of any principal rays of electromagnetic radiation which are not orthographically reflected by the paraboloid-shaped reflector; Y (c) perceiving the image of the scene by perceiving the orthographically reflected main rays of the electromagnetic radiation from the paraboloid-shaped reflector with one or more image sensors.
In an additional exemplary embodiment, a method for omnidirectionally perceiving images of a scene from a single point of view is provided which includes: (a) mounting a reflector essentially paraboloid and truncated on a fixed base; (b) mounting one or more image sensors on a mobile base; (c) orthographically reflecting the main electromagnetic radiation rays radiating from the scene of a reflector of essentially paraboloid form d so that the single point of view of the omnidirectional image formation method coincides with a reflector focus point of paraboloid shape; (d) telecentrically filtering a substance part of any principal rays of electromagnetic radiation which are not orthographically reflected from the paraboloid reflector; (e) moving the mobile base to a first position; (f) perceiving a first image of the scene that has a first field of view by perceiving the orthographically reflected main rays of the electromagnetic radiation from the paraboloidal reflector with one or more image sensors; (g) moving the mobile base to a second position different from the first position; Y (h) perceiving a second image of the scene having a second field of view by perceiving the orthographically reflected main rays of electromagnetic radiation from the paraboloidal reflector with one or more image sensors.
Alternatively, instead of mounting the paraboloid-shaped reflector on a fixed base and mounting the image sensors on a movable base, the paraboloid-shaped reflector can be mounted on a movable base and the image sensors can be mounted on a fixed base . Preferably, the method described above also includes the step of optically coupling the reflector of paraboloid shape and the image sensors with lenses for rapid change of plane which can be used to amplify an area of interest in the scene.
BRIEF DESCRIPTION OF THE DRAWINGS The example embodiments of the present invention will now be described in detail with reference to the accompanying drawings in which: Figure IA is a side view of an example embodiment of an omnidirectional imaging apparatus; Figure IB is a side view of an alternate embodiment in which a reflector of paraboloid shape is connected to an image sensor for transparent support; Figure 2 is an isometric view of a reflector of paraboloid shape mounted on a base plate; Figure 3 is a partially isometric view of a reflector of paraboloid shape mapped in a cylindrical coordinate system; Figure 4 is a geometric representation d an orthographic reflection from a curved reflecting surface Figure 5 is an illustration of a orthographic reflection from a paraboloid-shaped reflector essentially to an image sensor; Figure 6 illustrates how any selected part of the hemispheric scene can be viewed from a single point of view using a reflector of paraboloid shape; Figure 7 is a side view of an omnidirectional image forming apparatus with two reflectors of essentially paraboloid shape from back to back and two image sensors.
Figure 8 is a cross-sectional view of two reflectors of essentially paraboloid shape placed back to back and having a common paraboloidal axis and common approach; Figure 9 illustrates the mapping of the image data to the cylindrical coordinates to allow the production of a panoramic view, - Figure 10 is a flow diagram of an example embodiment of a method for perceiving and processing an image of a scene essentially hemispheric from a single point of view.
Figure 11 is a side view of an embodiment of an omnidirectional imaging apparatus according to the present invention, including a reflector of extended paraboloid shape.
Figure 12 is a side view of an incorporation of an omnidirectional imaging apparatus according to the present invention, the quad includes a reflector of paraboloid shape truncated in a plan that is inclined with respect to the reflector paraboloidal axis; Figure 13 is a side view of an incorporation of an omnidirectional imaging apparatus according to the present invention, including a paraboloid-shaped reflector that is larger than the imaging area of the image sensor; Figure 14 is a side view of an embodiment of an omnidirectional image forming apparatus according to the present invention including a reflector of concave paraboloid shape.Fig. 15 is a side view of an embodiment of an omnidirectional imaging apparatus according to the present invention including a lens for rapid change of optical plane by coupling a paraboloid-shaped reflector and an image sensor; Figure 16 is a partially isometric view of an incorporation of an omnidirectional image forming apparatus according to the present invention including a reflector of paraboloid shape mounted on a movable base; Figure 17A is a side view of an embodiment of an omnidirectional image forming apparatus according to the present invention including an image sensor mounted on a movable base; Figure 17B is a side view of an embodiment of an omnidirectional image forming apparatus according to the present invention including a mobile camera; Figure 17C is a side view of an embodiment of an omnidirectional image forming apparatus according to the present invention including a moving and optical camera; Figure 18 is a partially isometric view of an incorporation of an omnidirectional image forming apparatus according to the present invention, which includes an image sensor comprising four coupled charge devices placed side by side; Figure 19 is a side view of an embodiment of an omnidirectional image forming apparatus according to the present invention, which includes multiple image sensors and beam splitters.
Figure 20 is a top view of an image sensor according to an embodiment of the present invention whose sensor elements are not uniformly distributed and dimensioned; Figure 21 is a side view of an incorporation of an omnidirectional image forming apparatus according to the present invention, including a planar mirror that optically couples a paraboloid reflector and an image sensor; Fig. 22 is a side view of an embodiment of an omnidirectional image forming apparatus according to the present invention, including an optically coupled microscope objective or reflector of paraboloid shape and an image sensor.
Figure 23 is a side view of an incorporation of an omnidirectional image forming apparatus according to the present invention, which includes a lens collimator that optically couples a reflector of paraboloid shape and an image forming lens; Fig. 24A is a side view of an incorporation of an omnidirectional image forming apparatus according to the present invention which includes a plan concave planar μn lens; Figure 24B is a side view of an embodiment of an omnidirectional image forming apparatus according to the present invention which includes a convex concave field flattening lens with aplanatic sides; and Figure 25 is a side view of an incorporation of an omnidirectional image forming apparatus according to the present invention, which includes two concave paraboloidal mirrors used to form an essentially spherical view image.
DETAILED DESCRIPTION Figure IA illustrates an omnidirectional image forming apparatus 100 according to an embodiment of the present invention. A reflector of convex paraboloid shape 135, which is mounted on a base plate 140, is positioned to reflect orthographically an image of an essentially hemispherical scene 130. An image sensor 110, such as a color video camera device, is 3CCD commercially available 111 having a telecentric lens and an amplifying lens 112 and a telecentric aperture 113, is positioned to receive the orthographic reflection of the image. The telecentric lens or the aperture works to filter all light rays which are not perpendicular to the plane of the lens or aperture, for example, the backlight which is not part of the orthographic reflection of the hemispheric scene.
Although the description given here is in relation to visible light, the present invention has other application of other forms of electromagnetic radiation such as ultraviolet light or infrared light.
In an alternate example embodiment of image forming apparatus 100 according to the invention shown in Figure IB, the paraboloid-shaped reflector can be coupled to the image sensor by a transparent support 136, such as a transparent tube section.
Referring now to Figure IA, the video camera 110 generates an analog video signal representative of the orthographically reflected image which is sent through cable 150. The video signal is converted to a digit signal by the digitizer 120 which It is commercially available as an analog-to-digital NTSC video signal converter.
The digital signal is then sent through a cable 155 to a general-purpose computer 125, such as a DEC 3000/600 workstation. As will be explained in more detail below, the computer 125 is programmed to allow the user to see any desired part of the hemispheric scene, to quickly change plans on a selected part of the scene or to set the scene in any desired way.
The image sensor 110 may simply be a moving or non-moving film camera using a conventional photographic film. The image sensor 110 can also be a video camera or a camcorder 116 which provides a digital video signal output, which can be provided directly to the computer 125 without the need for a digital analog converter 120.
Figure 2 shows an isometric view of a reflector of paraboloid shape 135, which extends from the base 140 from which it is formed. The reflector 135 may comprise a plastic body of paraboloid shape coated with a thin layer 145 of a highly reflective metal such as aluminum or silver. Alternatively, the reflector 135 may comprise a polished metal body of paraboloid shape. For this latter embodiment, a metal such as stainless steel can be used.
Figure 3 illustrates in greater detail, the preferred geometry of the reflector of paraboloid shape 135, as well as the orthographic reflection of the image of the essentially hemispheric scene 130 on the image sensor 110. The reflector 13 of Figure 3 is defined in coordinates cylindrical, r, f and as essentially obeying the equation: h (1) 2 h where z is the axis of rotation, r is the radial coordinate and being a constant. The z axis coincides with the optical axis of the imaging array, and a paraboloid focus point 315 defined by equation (1) coincides with the origin of the coordinate system. The reflector 135 of FIG. 3 is truncated in a plane p which is essentially perpendicular to axis z 310 and which includes the focus point 315 of its paraboloidal surface.
All incoming rays 305 that will otherwise pass through the focus point 315 are reflected orthographically to the image sensor 110 by reflecting the paraboloidal surface. Thus, the focus point 315 is coincident with the single point of view from which the essentially hemispheric scene 130 is viewed. The image sensor 110 is positioned along the optical axis 310 of the image formation system and the The photosensitive surface of it is perpendicular to the optical axis. The use of orthographic reflection to allow the view of an essentially hemispheric scene from a single point of view is an advantageous feature of the present invention.
The orthographic reflection allows the view from a single point of view and can be demonstrated with reference to Figure 4. In Figure 4, z and r are perpendicular cylindrical coordinates for a given value of f, the angular coordinate The angle of an incoming ray 405 in relation to the axis r- * "is of f The incoming ray 405 is reflected orthographically by means of the reflecting surface 415 as a protruding ray 410.
To have a unique point of view 420 any incoming ray must satisfy: tan (?) = z / r, (2) and for the orthographic reflection, all the rays must be reflected at an angle a = tt / 2, (3) where a is the angle between the outgoing ray 410 and the axis For these two constraints to be satisfied, and for the angle of incidence to be equal to the angle of reflection, it is clear that the angle, ß, between the reflected ray 410 and the normal direction of the surface at the point of reflection, rX must be equal to: ß = o¿- ß 7T 2 T (4) which can also be expressed as tan 2ß = tano? -tan (= 2tanß (5) 1 + tancü tan (l-tan2ß Finally, the inclination of the reflecting surface 415 in the plane z? -rA at the point of reflection is s1 - dz = -tanß. (6) dr Substituting (6) and (4) in (5) yields The quadratic expression of equation (7) can be resuelt to obtain two solutions dz but to avoid dr self-occlusion by the reflecting surface, the slope of the curve in the right quadrant becomes negative (for example, the convex surface). The result is: If a = z / r, the expression given above is reduced to where h is an integration constant. Substituting z = ra e equation (9) gives equation (1). " Therefore, there is a curve which when rotated around the z axis with z ?, generates a surface that will allow the orthographic reflection of an essentially hemispheric scene from a single point of view. This curve is the parabola defined by equation (1) which has a unique view point d that is coincident with the focus 420 of the parabola In addition to providing the vision of an essentially hemispherical scene from a single point of view, the omnidirectional image forming apparatus according to the present invention allows the viewing of any part of the scene, allows the approach or rapid change of plane on a selected part, and allows the action of moving the camera of the scene, all with respect to the single point of view without requiring the reconstruction of the image or a complex quad transformation.
Figure 5 illustrates how a part of an essentially hemispheric scene is viewed by the image sensor from a single point of view. Figure 5 also illustrates how u reflector of essentially convex and truncated paraboloid shape 135 is mapped in a Cartesian coordinate system. The optical ex 502 of the image formation array is coincident with the z axis, and the focus 501 of the reflector is essentially paraboloid 135 located at the origin. The incoming rays 505 and 510 from a part of the scene 300 being viewed intersect the reflecting surface at points 515 and 520, which can be defined by their respective e and co-ordinates. The points 515 and 520 lie along imaginary radial lines 516 and 521, respectively, which originate from the point of view of the scene, for example, and focus 501 of the paraboloid-shaped reflector. Since these rays are orthographically reflected towards the image sensor 110, which has a surface at planar light perpendicular to the z axis, the projected rays will intersect the light sensitive surface at the same coordinates x and y Only the z coordinate will change. Therefore, there is a one-to-one correspondence between the coordinate x and the point d intersection with the reflector 135 of the orthographically projected beam, and the coordinate x and the point at which the ray projected orthographically intersects at the surface sensitive to the planar light of the 11Q image sensor.
In a preferred arrangement, the image sensor 11 includes a planar coupled device ("CCD") image sensor having an array of light-sensing cells. Each cell perceives the intensity of the light in its particular place in the array. . Therefore, with a one-to-one correspondence the image signals produced by the CCD cells cover a particular range of x and y coordinates in the grid e representative of the rays which are orthographically reflected from the reflecting surface 135 at points from the same range of the xy coordinates. Therefore, the mapping of the image in a Cartesian coordinate system is a simple task for persons skilled in the art.
With the one-to-one correspondence explained above in mind, Figure 6 illustrates a technique for rapid change of plane in any selected part of the scene and essentially hemispherical. The reflector 135 is placed in relation to the orthogonal axes x, y and z in the same manner as in FIG. 5. In order to rapidly change planes at a focal length f over a selected part of the central scene around a point 550, with a specified size only the image signals of the CCD cells with the same x and y coordinate range as those of the region of the reflecting surface that project the selected part of the scene are selected for an amplification and view.
More particularly, to determine the appropriate light intensity for point 570 in the selected part of the scene, the light intensity signal generated by cell CC which lies at 580 is chosen. As shown in Figure 6, a line segment drawn between point 570 and point 551 intersects reflector 135 at point 552. The light intensity at point 570 is set equal to that represented by the image signal generated by the CCD cell at 580 which is located on the x coordinate and on the grid closest to the xy coordinate of point 552. It is repeated for each CCD cell within the same xy coordinate range as the region of the reflecting surface that projects the selected part of the scene. As a result of the orthographic reflection and the one-to-one correspondence described above, a complex picture transformation or image reconstruction is not required.
A general-purpose computer 125 can easily be programmed by an artisan to carry out the above-mentioned steps to allow the viewing of any part of the hemispheric scene from a single view point, and also allows rapid change of plane. e any particular part to provide an amplified image of that part. In addition, by designating successive points along the reflector, the hemispheric scene can be panned, as if one were viewing the scene from a single point of view.
-In the incorporation discussed above, it is readily apparent that by changing one of plane in rapid form into smaller parts of the scene, the number of CCD cells that provide information to the computer 125 and reduced, and therefore the granularity of the Sight image increases. In a preferred embodiment, the information about the points in the scene that do not correspond exactly to the CCD cells are closer approximated mediant interpolation. A suitable interpolation program which can be executed on a computer 125 is included in appendix I of this description. The attached program as appendix I will give the map of the omnidirectional image perceived to an ordinary perspective image that is suitable for the display on the computer 125. The program requires the user to enter the name, the central location and the radius of the image. omnidirectional that is going to be converted. The program also requires the user to enter a name for the generated image and perspective, as well as a focal length and a size for the perspective image.
Therefore, instead of simply choosing the image signal generated by the closest CCD cell to represent parts of the image that does not correspond precisely to a CCD cell, the image for such scene parts is stimulated by the learned program based on the adequate average d image signals generated by the CCD cells which correspond to the neighboring parts of the scene. Of course, more sophisticated interpolation programs known to those skilled in the art may be used, such as those based on temporal or polynomial marriage, if departing from the scope of the invention as defined by the appended claims.
In addition to the Cartesian coordinate mapping that has been described, which produces an image in perspective, a cylindrical coordinate mapping can also be carried out to achieve a panoramic image of the scene being viewed. The cylindrical coordinate mapping will be described with reference to FIG. 9. In FIG. 9, a main beam 954 from a point 945 in a scene strikes a paraboloid-shaped reflector 93 and is reflected orthographically to an image sensor 910. The reflected beam orthographically 960 sticks on the sensor d image on the sensor element 965. To map the dot represented by the sensor element 965 in cylindrical coordinates, a truncated cylinder 970 is formed in even images surrounding the paraboloid-shaped reflector 935 and the image sensor 910 The point represented by sensor element 965 and then traced back through rays 960 and 950, and point of intersection 955 of beam 950 with truncated cylinder 970 is determined. The point 955 is then assigned the light intensity of the perception of the element 965. This same calculation is carried out for each sensor element of the image sensor 910. The resulting collection of points (with approximately assigned light intensities) located over the truncated cylinder 970 produces a panoramic image of the scene that is being viewed. This panoramic image can be seen on a display by additionally mapping the truncated cylinder to a planar surface. This mapping and easily carried out by those skilled in the art can be visualized by imagining that the cylinder is cut lengthwise and flattened. In addition, those skilled in the art will readily appreciate, the interpolation of image data as discussed above in relation to the mapping of the Cartesian coordinate can also be used with the mapping of the cylindrical coordinate.
In a preferred embodiment of the present invention, a one-third-inch CCD is used with a paraboloid-shaped mirror of 0.4-inch focal length through its focus and having a diameter of 1.6 inches. A collimating lens, such as model P32921 from EDMUND SCIENTIFIC of Barrington, New Jersey, was used with an 8.5-inch focal length image forming lens to optically attach the mirror to the CCD. _ In a further exemplary embodiment of the invention, the omnidirectional imaging apparatus includes a reflector of essentially additional paraboloid shape 735 as shown in FIG. 7. The additional reflector is positioned to orthographically project an image of an additional hemispherical stage 730 which is complementary to the hemispheric scene 130 so that together they constitute a spherical scene. An additional image sensor 710 is positioned to receive the image projected orthographically by the additional reflector 735.
An image signal representative of the orthographic reflection of the additional reflector 735 is converted to a digital signal by the converter 720 in the same manner as s described above and sent to the same general purpose computer 125 through line 725.
As shown in Figure 8, the reflectors 13 and 735 are positioned from back to back, share a rotating common ej 810, which is also the optical axis of the image forming apparatus, and a common approach 805, and is each truncated in a plane p which is essentially perpendicular to the axis of rotation 810 and which includes the foc 805.
Referring to Figure 10, there is shown a flow scheme 1000 illustrating a method for perceiving an image of a hemispherical or essentially spherical scene from a unique viewpoint according to an example embodiment of the present invention. Flow scheme 1000 shows the steps necessary to perceive the hemispheric scene from a unique point of view. The method requires the orthographically reflects the essentially hemispheric scene 1010, to perceive the orthographically reflected image 1020.
The method may further include the steps d to convert the image signal into image signal data 1030 to map the image data in an appropriate coordinate system 1040, to interpolate the image data 1060 to derive the approximate values for the missing image data. and forms a digital image 1070 from the mapped image formation data and from the interpolated image data Advantageously, the data specify a viewing direction, a focusing length and an image size 1045 and fast switching from flat to 1050 envelope a selected part of the image data can be carried out before the interpolation step Therefore, up to now, the described example embodiments have all used a "normal" paraboloid reflector. As used in the description and appended claims, the term "normal" in association with a paraboloid-shaped reflector refers to a paraboloid-shaped reflector that is truncated in a plane passing through the focal point of the paraboloid-shaped reflector and that is essentially perpendicular to the paraboloidal axis of the paraboloid-shaped reflector. As used in this description and in the appended claims, the parabolic axis of a paraboloid-d reflector is the axis passing through the vertex and the focal point of the paraboloid-shaped reflector. As s described above, using a reflector of normal paraboloid shape, one can form the image of a complete hemisphere (steradians), or by placing two such reflectors d back to back, a full sphere (27T steradians). Figures 11 to 15 show a further example of the incorporations of the omnidirectional image forming apparatus in which the paraboloid-shaped reflector can also take the form of several non-normal paraboloids Figure 11 shows an omnidirectional image forming apparatus that is capable of imagining a field of vist ("FOV") greater than the hemisphere only by using the camera 1111 and its paraboloid-shaped reflector 1135. In the embodiment of figure 11, the reflector of paraboloid shape 1135 is an extended paraboloid that is obtained by cutting or reflector with a plane that is normal to the paraboloid axis (z) but passes below the focus point 1130 of the paraboloid. Advantageously, because the paraboloid extends below its focal point, the paraboloid-shaped reflector is capable of orthographically reflecting rays from the hemisphere below its focal point. In the embodiment illustrated in Figure 11, for example, the FOV covered by the paraboloid-shaped reflector is 240 °, or 75% of its entire spher. Preferably, as shown in Figure 11, the camera 1111 and the paraboloid-shaped reflector 1135 are coupled by the optical system 1112.
Figure 12 shows an omnidirectional image forming apparatus that can be used to form an image of a FOV that is inclined with respect to the paleoboloidal ej of the reflector of paraboloid shape. The embodiment of figure 12 includes a camera 1211, the optical system 121 and a reflector of paraboloid shape 1235. The reflector of paraboloid form 1235 is truncated in a plane passing through focus of the reflector of paraboloid shape 1235 and inclined with respect to to its paraboloidal axis (z). The FOV of this reflector is therefore an inclined hemisphere, as shown by the dotted lines of Figure 12. Even though the embodiment in Figure 12 shows the truncation plane passing through focus of the paraboloid, the invention is not limited to this incorporation. The truncation plane can also pass up from the 1230 focus of the paraboloid (resulting in a FO smaller than a hemisphere) or the truncation plane can pass below the 1230 focus (thus resulting in a FOV greater than the hemisphere) .
Figure 13 shows an omnidirectional image formation apparatus that can be used to form an image of a FOV smaller than a hemisphere. The embodiment of Figure 13 includes a camera 1311 coupled to a reflector d of paraboloid shape 1335 by optical system 1312. In this embodiment, the paraboloid-shaped reflector 1335 is formed so that it is "larger" than the image forming area of the camera 1311. In this context, a paraboloid-shaped reflector is "larger" than the imagined area of a camera if the base of a normal paraboloid having the same plane as the reflector (for example having the same constant paraboloid h as defined in equation (1)) is larger than the smallest dimension of the image area of the camera. In the case of a normal paraboloid for illustrative purposes, it is clear that when such a paraboloid is larger than the image formation area of a camera, only a FOV smaller than a complete hemisphere is capable of being captured in the image forming area. of the camera because the rays orthographically reflected on the outer edges of the paraboloid will not stick on the image forming area. Advantageously, however, the image captured using the mirror Paraboloid form has a higher resolution than l of a corresponding image captured using a smaller paraboloid. As shown in Figure 13, the paraboloidal paraboloidal axis of paraboloid shape 1335 (z ') can be changed co with respect to the optical axis (z) to obtain fields of view toward the horizon. In addition, the reflector of paraboloid form 1335 n requires to be a normal paraboloid, but can be truncated according to the FOV that is going to be formed in images.
Therefore until now, all the discussed incorporations have included a reflector of a convex paraboloid shape. In Figure 14, an incorporation of an omnidirectional imaging apparatus according to the present invention is shown and includes a camera 1411, optic 1412, and a reflector of concave paraboloidal shape 1435. A reflector of concave paraboloid shape can be used in the applications where in hiding the reflector is desirable (as for example in outdoor applications where protection against weather is desirable). In the case of a reflector of concave paraboloid shape, the paraboloidal image d the scene is flipped, but the image continues to satisfy the restriction of the single point of view described previously. Thus, pure perspective images can be generated from the concave paraboloidal image, just as with the convex paraboloidal image. In the case of the concave paraboloid, however, at most a hemispherical field of view can be obtained with a single reflector. This hemispherical FOV is obtained by truncating the paraboloid with a plane passing through the focal point 1435 of the paraboloid (the plane being already normal or inclined with respect to the axis of the paraboloid (z)). When a concave paraboloid that is truncated above is the focal point it can also be used, such as a paraboloid n is desired because it causes self-obstruction of the image.
As shown in Figure 25, a greater FOV than a hemisphere can be obtained by using multiple concave paraboloid reflectors. In figure 25, reflectors of paraboloid shape 2535a and 2535b are positioned so that they share a common paraboloidal axis (z) and their vertices 2545 coincide. Together with the image sensors 2511 and 2511b, the two reflectors of paraboloid shape 2535a and 2535 are capable of forming two hemisphere images 2530a and 2530 respectively. This system can be used advantageously when the reflectors are required to be discounted for concealment or protection. A disadvantage to using the concave mirrors in this arrangement, instead of using convex mirrors in the arrangement of FIG. 7, is that the small blind spot that comprises the area between the truncated planes of the reflectors is unavoidable.
Figure 15 shows an incorporation of an omnidirectional imaging system according to the present invention with fast plane change capabilities. The omnidirectional imaging system of FIG. 1 includes a reflector of paraboloid shape 1535, a camera 1511, a lens for rapid change of plane 1512 and optical relays 1513. (As used in this specification and in the appended claims, optics relays and optical collimators are synonymous). With the 1512 fast-change plane lens set to its lowest strength, the omnidirectional imaging system provides a full hemisphere image (or greater or less than a hemisphere if the incorporations of figures 11 or 13 are used). When approaching, the plane quick change lens 151 provides a higher amplification (and therefore a higher resolution) of a smaller FOV. As the approach is made, the effective center of the projection of the 1512 plane quick change lens should remain approximately fixed to ensure that the image formation system remains telecentric. Preferably, the optical system rel 1513 is used to ensure that the lens for rapid change d 1512 remains telecentric on its complete locations.
In the embodiment of FIG. 15, the plane quick change lens 1512 can be either fixed or mobile co with respect to the paraboloid-shaped reflector 1535. If the plane rapid change lens 1512 is fixed, only the regions around the axis Paraboloidal (z) can be observed under amplification Preferably, therefore, the plan rapid change lens 1512 is equipped with some movement means, allowing the fast change lens to be placed flat on the image regions throughout. of the outer edges of reflector of paraboloid shape 1535. Of course, such means of movement must ensure that the optical axis of the rapid-change lens 1512 remains parallel to the paraboloidal ej of the paraboloid-shaped reflector 1535 at all times.
Figure 16 shows an omnidirectional image formation system that can be used to produce dynamically changing field of view of a scene. A paraboloid-shaped reflector 1635 is mounted on a movable base 1640 l which allows translation of the paraboloid-shaped reflector 1635 along the x, y, and z axes. The movable base 164 can be controlled either manually or with a computer. Using the movable base 1640, a dynamically changing field of view of a scene can be produced, for example, by a circular movement of the movable base 1640 around the optical ej (z ). Preferably, the images are imaged processed as previously described to obtain a perspective or panoramic view.
Figure 16 further shows the use of the 1612 plane quick change lens in combination with the 1640 movibl base. A fast plane change lens adds the ability to fast plane change in sections of the paraboloid reflector form 1635 brought under the view of the image formation system by the movement of the movable base 1640 Preferably, a relay lens 1613 is used to attach the plane rapid change lens and the paraboloid form reflector 1635. In addition, the plan rapid change lens 1612 preferably includes automatic manual focus control to ensure that image definition is maintained over all sections of the paraboloid-shaped reflector 1635. Alternatively, the translation of reflectance along the z-axis can also be used to adjust the focus of the reflector. an image.
Instead of moving the reflector in a paraboloid manner as the embodiment of FIG. 16, one or more parts of the camera or optical system of the image forming system may alternatively be moved to achieve the same effect as in the embodiment of FIG. 16. Figures 17A, 17B and 17 show several exemplary embodiments of such omnidirectional imaging systems. In Figure 17A, image sensor 1710 (such as a CCD) is provided with moving media; in Figure 17B, a camera 1711 is provided with moving means; and in Figure 17C both a camera 1711 and optical system 1712 are provided with movable means, to move together simultaneously. As shown in the figures, each of these components can be moved along any of the axes x, y or z to change the field of vision that is being formed in images. As in the embodiment of Figure 16, a fast plane change lens can be used to amplify the areas of interest. Advantageously, by moving the camera or the optical system instead of moving the reflector in a paraboloid manner, the viewpoint of the omnidirectional imaging system remains fixed in the space at the focus point of the paraboloid-shaped reflector.
The embodiments of Figs. 16, 17A, 17B 17C can be used to further advantage in a surveillance system. The omnidirectional imaging capability of this incorporations allows an operator to monitor a complete area of interest at a time. When the operator observes a particular region of interest within the area being monitored, the operator can then select the appropriate translational coordinates (for the movement of the camera, optical system or reflector in a paraboloid fashion) and the lens positions of the fast plane change appropriate for the view of the region of interest in greater detail.
Figure 18 shows an omnidirectional image formation system that uses multiple imaging sensors to achieve increased image resolution. The embodiment of figure 18 includes a reflector of paraboloid form 1835, video electronics 1809, four elements CCD 1810a-1810d, and the optical system of image formation 1812. In this embodiment, the four elements CC 1810a to 1810d are placed side by side in an overlapping arrangement. The embodiment of Figure 18 takes advantage in fact that commercial CCD elements are typically manufactured in standard resolutions regardless of their size. So muchBy using four commercial quarter-inch CCD elements instead of a single commercial half-inch CCD element, the resolution of an image can quadruple advantageously. Although Figure 18 shows the use of the CCD elements placed in an overlapping array, the invention described herein is not limited to the arrangement. Thus an array where multiple CC elements overlap partially can similarly be used. In addition, the multiple imaging sensors can be manufactured in a single integrated circuit with each image sensor connected to its own video circuit.
Figure 19 shows another embodiment that uses multiple image forming sensors to increase image resolution. In this case, the multiple image sensors are provided by multiple cameras 1911. The beam splitters 1916 are used to direct the separate sections of a paraboloidal image of different cameras Advantageously, therefore, each part of the paraboloid image is formed into images with a higher resolution than if the whole image were formed in images by a single camera.
In another example embodiment of the present invention, the dichroic ray splitters (not shown) can be used to divide an image into a plurality of monochromatic images, which can be perceived by a plurality of monochromatic image detectors. These monochromatic images can then be appropriately combined in a complete color image by the means of image processing well known in the art.
Figure 20 shows a flat image sensor 2010 as in the example, for example a CCD element. Using a typical planar image sensor with a paraboloid-shaped mirror, the effective resolution of a captured paraboloid image is increased more toward the outer edge of an image than at its center. For example, when a flat image sensor is used to capture an image reflected by a normal paraboloidal reflector, the resolution of the captured image increases by a factor of 4 from the center of the image to the edge. To compensate for this effect, an image sensor has perceived the 2008 elements whose sizes and placements are varied to result in a uniform resolution over the entire image. The same approach can also be used to increase the resolution in selected parts of the FOV. When specific resolution variations are difficult to incorporate, standard resolution variations, such as those provided by polar log sensors, can be used.
One or more planar mirrors may be included in an omnidirectional imaging apparatus according to the present invention for the flexibility of positioning of the optical system and the reflector. Figure 21 shows a preferred embodiment in which an omnidirectional imaging system includes a paraboloid-shaped reflector 2135, a planar mirror 2116, a relay lens 2113, an imaging lens 2112, and a 2111 camera. shown embodiment, the paraboloid-shaped reflector 213 is positioned above a surface 2140, and the flat mirror 2116, the relay lens 2113, the image forming lens 2112 and the camera 2111 are hidden under the surface 2140. The mirror Planar 2116 is placed below an aperture 2145 on surface 2140 and folds the image from the reflecto in a 90 ° paraboloid shape, thereby redirecting the image of the relay lens, to the imaging lens and to the camera. Planar is shown between the paraboloid reflector and the relay lens, the planar mirror can be placed between the relay lens and the image forming lens between the lens It's about images and the camera, as those experts in art will appreciate.
Figure 22 shows an incorporation of an omnidirectional imaging system in which the optical system between a paraboloid-shaped mirror 2235 and its image sensor 2210 comprises a low-strength inverted microscope lens 2212. In this embodiment, the reflector 2235 it is in the position normally occupied by the eye piece of the microscope and the image sensor 2210 is in the position normally occupied by a stage. The use of an inverted microscope objective is advantageous for the formation of images since commercial microscope lenses are highly corrected for aberrations.
Figure 23 shows an incorporation of an omnidirectional imaging system in which a collimator lens 2313 is positioned between a paraboloid-shaped mirror 2335 and the optical imaging system 2312. It is desirable to use a commercially available imaging lens. in many cases to save the cost and time of designing special lenses. The most commercial imaging lenses are however intended for scenes of images that are far from the lens. It's true, these are normally designed for objects that are infinitely distant from the lens. Therefore when used for imaging objects that are close to the lens, the image suffers from various types of aberrations which degrade the effective resolution of the lens. The result is a "muddy" or blurred image. In this embodiment this problem is solved by the use of a collimating lens 2313 which produces a virtual object at infinity for the optical system forming images 2312. Advantageously, therefore, the use of a collimating lent 2313 allows the use of optical lenses. commercially available imaging.
The embodiments of Figures 24A and 24 illustrate the use of field flattening lenses between an image sensor 2410 and an image forming lens 2413. Field flattening means are desirable because the reflector of paraboloid shape of the present invention has a typically small focal length of a few millimeters, is afflicted by a very strong field curvature. One method to eliminate this image formation defect is to use an image sensor with an arched surface that matches the curvature of the field. More preferably, however, a special lens, called a field flattening lens, can be introduced which has a curvature opposite to that of the reflector. Therefore, the two field curvatures cancel, and the resulting image surface is flat allowing the entire image to be in a focused focus on a planar image sensor.
Two types of preferred field flattening lenses are illustrated in Figures 24A and 24B. In Fig. 24A, a plano-concave lens 2412 is shown. The plano-concave lent 2412a is positioned as close as possible to image sensor 2410. Preferably, the plano-concave lens 2412a is placed in contact with the sensor window. of image 2417. In this position of the concave plane lens 2412 compensates for the field curvature of the reflector while s introduce only small amounts of undesirable aberrations A second type of a preferred field flattening lens, a convex concave lens 2412b, is shown in FIG. 24B. Both of the surfaces of the concave convex lens 2412b are flattened to the centering light. If a surface is aplanatic, it does not introduce a spherical aberration, or astigmatism in the ray of light; it only introduces the field curvature. The convex concave lens 2412b has a marked field flattening effect which is determined by the thickness of the lens; The thicker is the larger lens and the flattening effect of the field. In contrast to the concave plane lent 2412a of Fig. 24a, the meniscus lens 2412 of Fig. 24B is not used in contact with the image sensor. 2410.
The theory of the field flattening lens will now explain. Ideally the best focusing surface of an optical system is a plane. With the planar surface, a CCD other type of flat image sensor can match the surface of the best focus on its entire area, providing both a maximum resolution for the image. Unfortunately an optical system has a tendency to form its best images on an arched surface. Therefore, the arcuate focusing surface and the flat CCD surface can not equal their full area, and some or all of the images will not be better focused.
The curvature field of an optical system is called its Petzval curvature. Each optical element in an optical system contributes to the Petzval curvature for the system. If a surface of an optical element is refractory, its contribution Petzval to the curvature of the system is: P = 1 - n nR where n is the refractive index of the optical element and R is e radius of curvature of the surface of the optical element Clearly the Petzval contribution of a surface depends on the sign of the radius. If the surface is a mirror instead of a refracting surface, its Petzval contribution is: P = R The curvature field of an image is calculated by taking the sum of the contributions of all the reflecting and refracting surfaces and multiplying the sum by a simple constant. If this value is not zero, then the image field is arched and the problem discussed above is found (for example, the surface of the image and the surface of the image sensor will not be completely married).
Unfortunately, the curvatures of the optical surfaces can not be eliminated because they are necessary for other purposes, such as to control spherical aberration, coma and astigmatism. Because the control of these aberrations depends on the curvatures of the optical elements, if the curvature of those elements is changed, these aberrations can be adversely affected. There are two ways, however, in which the Petzval curvature of an optical system can be changed without changing the other aberration of a system. These two methods form the basis for two types of field flattening lenses described above.
The first method to change the Petzva curvature will depend on the optical characteristics of an optical surface located on the surface of an image. If an optical surface is located on the surface of an image (either an intermediate image or the final image of the optical system) then this surface will not change the spherical aberration, the coma or the astigmatism of the image. The only change will be Petzval curvatur. Therefore, the Petzval curvature of a system can be correlated by inserting a surface with an appropriate radius of curvature to the final focus of the system. This is the basis for the flat-concav field flattening lenses described above.
The second method to change the Petzval curvatur will depend on the optical characteristics of the aplanatic surfaces. Presume that an aplanatic surface, which is defined as follows: let s be the object distance for the surface and s' is the distance d image. Also, let n and n 'be the refractive indices of the materials before and after the surface, respectively (where n = 1 for air and n> 1 for glasses). If s and s' are related by s 'R (n' + n) = ns n 'n' then the surface will not introduce a spherical aberration a coma and only very small amounts of astigmatism. If a thick lens is now introduced, both of whose surface satisfies this condition, then the difference in the radii will depend on the thickness of the lens. This fact can again be used to control the Petzval curvature of the system by adjusting the thickness of the aplanatic lens. This is the basis of thickness, of the concave-convex field flattening lens discussed above.
In the preferred embodiment of the concave plan lens 2412a of FIG. 24A, the concave plane lens is composed of BK7 and has a refractive index (n) of 1.517. The radius of the arcuate surface (concave) r2 is 6. millimeters. The opposite surface which is the arched surface rx is flat and is placed in contact with the imager window 2417. The axial thickness of the lens is 1.5 millimeters and the optical diameter is 3 millimeters.
In a preferred embodiment of the flattened lent 2412b of Figure 24B, the aplastic lens is composed of acrylic plastic and has a refractive index (n) of 1494. The radius of the arcuate (convex) surface r2 is 4.78 millimeters and the radius of the arcuate (concave) surface r is 2.16 millimeters. The axial thickness of the lens 6.7 mm. The optical diameter of the arched surface r2 e of 7 millimeters and the optical diameter of the arched surface r is 2.7 millimeters.
Although the present invention has been described with reference to certain preferred embodiments, it will be known or obvious to those skilled in the art various modifications, alterations and substitutions without departing from the spirit and scope of the invention as defined by the appended claims.
APPENDIX I compute image. c #include "stdlib.h" #include "imageutil. h" #include "stdio.h" #include "math.h" / * int main (int argc, char ** argv) * / main (arg, argv) int argc; char * argv []; . { double sqrt (), atan (), sin (), cos (), acosf); not signed char * r, * g, * b; not signed char * red; not signed char * green; not signed char * blue; int xsize, ysize; int xosize, yosize; int i, j, xO, yO, xl, yl; double theta, phi; double ox, oy, oz; double px, py, pz; double qz, ay, qz; double tempx, tempy, tempz; double sx, sy, sz; double rad, mag; double xs, ys, zs; double dispx, dispy; int xcent, ycent, xnuevo, ynuevo, xpix, ypix, xpunto, ypunto; intxpixel, ypixel, indicex, index, xcentro, and center; float radio, focal; / * imprimirf ("complete initializations \ n \ n"); * / yes (argc! = 4). { printf ("arguments: xcentro, ycentro, radio \ n"); exit (0); } printf ("\ n"); xcent = atoi (argv [1]); ycent = atoi (argv [2]); radio = atof (argv [3]); printf ("omni-image: xcentro =% d ycentro =% d radio =% f \ n \ n" xcent, ycent, (float) radius); printf (inner view pixel [xnovel ynovel]: "); exploref ("% d% d ", & xnew, &new); printf (" \ n "); printf ("selected view pixel: xnew =% d and new =% d \ n \ n", xnew, andnew); imprimirf ("insert new image parameters [xpixeles ypixele focal]:"); scanf ("% d% d% f", &xpix, &ypix, &focal); printf ("\ n"); printf ("outer image: xpixels =% d andpixels =% d focal% f \ n \ n", xpix, ypix, (float) focal); PPM load ("ppm test", & amp; amp; amp; amp; amp; amp; b, & amp; size, &size); printf ("omni-image file loaded \ n \ n"); x size = xpix; y size = ypix; / * imprimirf ("put new image size, size =% d, and size% d \ n \ n", x size, and size), * / red = (char * not signed) malloc (x size * y size * size of (unsigned char)), -green = (char * unsigned) malloc (x size * and size * size of (unsigned char)); blue = (char * not signed) malloc (x size * and size * size (char not signed)); printf ("allocated memory for new image file \ n \ n"); xcentro = xcent; ycentro = ycent; xpunto = ynuevo - ycent; ypunto = xnew - xcent; tempx = (double) xpunto, -tempy = (double) and point; tempz = (radius * radius - (temp * tempx + tempy * tempy)) / (2 * radius); ox = tempx / sqrt (tempx * tempx + tempy * tempy + tempz * tempz); oy = tempy / sqrt (tempx * tempx + tempy * tempy + tempz * tempz); oz = tempz / sqrt (tempx * tempx + tempy * tempy + tempz * tempz); / * computed optical axis (z) * / tempx = -oy; tempy = ox; tempz = 0; px = tempx / sqrt (tempx * tempx + tempy * tempy + tempz * tempz); py = tempy / sqrt (tempx * tempx + tempy * tempy + tempz * tempz); pz = tempz / sqrt (tempx * tempx + tempy * tempy + tempz * tempz); / * computed horizontal axis * / tempx = py * ox - pz * oy; tempy = pz * ox - px * oz; tempz = px * oy - py * ox; qx = tempx / sqrt (tempx * tempx + tempy * tempy + tempz * tempz); qy = tempy / sqrt (tempx * tempx + tempy * tempy + tempz * tempz); qz = tempz / sqrt (tempx * tempx + tempy * tempy + tempz * tempz); / * vertical axis computed * / imprimirf ("computed perspective image box \ n \ n") / * perspective image plane to explore raster * / for (i = 0; i &ypix; i ++). { dispy = (double) i - (double) ypix / 2; for (j = 0; h <xpix; j ++). { dispx = (double) xpix / 2 - (double) j; sx = ox * focal + px * dispx + qx * dispy; sy = oy * focal + py * dispx + qy * dispy; sz = oz * focal + pz * dispx + qz * dispy; mag = sqrt (sx * sx + sy * s + sz * sz); sx = sx / mag sy = sy / mag sz = sz / mag / * vector computed in the current pixel direction * phi = atan2 (sy, sx); theta = acos (sz / sqrt (sx * sx + sy * sy + sz * sz)); / * vector converted to polar coordinates * / rad = 2 * radius * (1-cos (theta)) / (1-cos (2 * theta)); / * radio found intersection on pabola- ^ xs = rad * sin (theta) * cos (phi); ys = rad * sin (theta) * sin (phi); zs = rad * cos (theta); / * coordinates x, y, z found on paraboloid * / / * printf ("xs =% f ys =% f zs =% f \ n \ n, (float) s, (float) ys, (float) zs ); * / / * use xs, ys to read from the input image and save an output image * / / * check if the image point lies outside the satellite image * / yes (sqrt (xs * xs + ys * ys) > radio). { red or [i * xpix + j] = 255; green [i * xpix + j] = 255; blue [i * xpix + j] = 255; otherwise. { indexxx = (int) and s + xcentro; indicexy = (int) xs + ycentro; / * printf ("one pixel \ n \ n"); * / / * write the closest color value in the pixel * / red or [i + xpix + j] = r [index x * x size + index]; green [i + xpix + j] = f [index x * x size + index]; blue [i + xpix + j] = b [index x * x size + index]; printf ("computer perspective image \ n \ n"); savePPM ("fera.ppm", red, green, blue, xpix, andpix); printf ("save new image file \ n \ n"); system ("xv out .ppm &"), - free (r); free (g); free (b); free (red); free (green); free (blue); printf ("free allocated memory \ n \ n"); return 0

Claims (55)

R E I V I N D I C A C I O N S
1. An omnidirectional image forming apparatus for perceiving an image of a scene from a unique point of view, comprising: (a) a reflector of essentially truncated paraboloid shape placed to orthographically reflect the main rays of the electromagnetic radiation radiating from said scene, said reflector of paraboloid shape having a focus coincident with said single point of view of said omnidirectional image forming apparatus, including reflector of paraboloid shape; (b) telecentric means, optically coupled said reflector of paraboloid shape, to filter essentially the main rays of electromagnetic radiation which n are reflected orthographically by said reflector of paraboloid form; Y (c) one or more image sensors positioned to receive said main rays orthographically __ reflected d said electromagnetic radiation from said reflector of paraboloid form thereby perceiving said image of said scene
2. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that said reflector has a paraboloid and convex shape.
3. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that said reflector has a paraboloid and concave shape.
4. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that said paraboloid-shaped reflector comprises a paraboloidal mirror having a surface which substantially obeys the equation expressed in cylindrical coordinates: 2 h z being an axis of rotation of said surface, r being a radial coordinate and h being a constant.
5. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that one or more of said image sensors comprise one or more coupled charge devices.
6. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that said one or more image sensors comprise one or more charge injection devices.
7. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that said one or more image sensors comprise a photographic film.
8. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that said one or more image sensors comprise one or more video cameras.
9. A forming apparatus with an omnidirectional imaging, as claimed in clause 1, characterized in that one or more of said image sensors have an arcuate surface that houses a field curvature of said image.
10. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that at least one or more image sensors have a non-uniform resolution.
11. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that said one or more image sensors are positioned along an axis passing through the apex d said paraboloidal reflector and through said reflector. focus of said reflector of paraboloid shape.
12. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that it also comprises one or more planar mirrors placed between said paraboloidal reflector and said one or more image sensors, wherein said one or more planar mirrors are coupled optically said reflector of paraboloidal shape to said one or more image sensors.
13. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that said reflector of paraboloidal shape comprises a mirror truncated in a plane which includes said focus of said reflector of paraboloid shape.
14. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that said paraboloid-shaped reflector comprises a truncated mirror in a plane that is essentially perpendicular to an axis that passes through the vertex of reflector diah of paraboloid shape and through said focus reflector reflector of paraboloid shape.
15. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that said paraboloid-shaped reflector comprises a normal paraboloidal mirror.
16. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that it also comprises a transparent support which couples said reflector of paraboloid shape to said one or more image sensors to thereby maintain the relative position thereof.
17. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that it also comprises a fixed base and a moving base, characterized in that said paraboloid-shaped reflector is mounted on said fixed base and said one or more image sensors are mounted on said mobile base, whereby the movement of said one or more image sensors produces a change of the field of view.
18. An omnidirectional imaging apparatus, as claimed in clause 17, characterized in that it also comprises a lens for rapid change of plane placed between and optically coupling said one or more image sensors and said reflector of paraboloid shape
19. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that it further comprises a fixed base and a mobile base, wherein said paraboloid-shaped reflector is mounted on said mobile base and said one or more image sensors are mounted on said fixed base, whereby the movement of reflector dich of paraboloid shape produces a change of field d view.
20. An omnidirectional imaging apparatus, as claimed in clause 19, characterized in that it further comprises a fast-changing plane lens placed between and optically coupling said one or more image sensors and said reflector of paraboloid shape.
21. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that said one or more image sensors generate an image signal representative of said image of said scene, further comprising an image signal processing apparatus coupled to said image. one or more image sensors and receiving said image signal to convert said image signal and image signal data.
22. An omnidirectional imaging apparatus, as claimed in clause 21, characterized in that said image signal processor apparatus maps said image signal data into a coordinate system. Cartesian to produce an image in perspective.
23. An omnidirectional imaging apparatus, as claimed in clause 21, characterized in that said image signal processing apparatus maps said image signal data into a cylindrical coordinate system to produce a panoramic image.
24. An omnidirectional imaging apparatus, as claimed in clause 21, characterized in that said image signal processing apparatus further includes interpolation means for providing interpolated image data whereby said interpolated image data and said data of Image signal are combined to form a digital image.
25. An omnidirectional imaging apparatus, as claimed in clause 24, characterized in that said image processing apparatus further includes means for zooming into a preselected part of said digital image to thereby provide an amplified image of said preselected part from a digital image. seal distance predetermined.
26. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that said telecentric means comprise a telecentric lens.
27. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that said telecentric means comprise a telecentric opening.
28. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that it also comprises at least one lens optically coupling said one or more image sensors and said reflector of paraboloid shape.
29. An omnidirectional imaging apparatus, as claimed in clause 28 characterized by said at least one lens having a focal plan between said one or more image sensors and said at least one lens, and wherein said telecentric means they are a telecentric opening placed along said focal plane
30. An omnidirectional imaging apparatus, as claimed in clause 28, characterized in that said telecentric means comprise a collimating lens optically coupling said paraboloid-shaped reflector and said at least one lens.
31. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that it also comprises a plane rapid change lens by optically coupling said one or more image sensors and said reflector of paraboloid shape.
32. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that it further comprises a microscope objective optically coupling said one or more image sensors and reflector dich of a paraboloid shape.
33. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that it also comprises a field-leveling lens optically coupling said one or more image sensors and said paraboloid reflector, said field flattening lens having a field curvature approximately opposite to the curvature of the reflector field of paraboloid shape.
34. An omnidirectional imaging apparatus, as claimed in clause 33, characterized in that said camp leveling lens comprises a plano-concave lens which is positioned close to said one or more image sensors.
35. An omnidirectional imaging apparatus, as claimed in clause 33, characterized in that said camp leveling lens comprises a concave-convex lens having aplanatic sides.
36. An omnidirectional imaging apparatus, as claimed in clause 1, characterized in that said scene is an essential hemispheric scene and also comprises: a reflector of essentially truncated paraboloid shape positioned to orthographically reflect the main electromagnetic radiation rays radiating from an additional hemispherical scene, said reflector of additional paraboloid form has a focus coincident with a single viewpoint of said additional hemispheric scene; additional telecentric means, optically coupled to said reflector of additional paraboloid shape to substantially filter the main rays of electromagnetic radiation which are not orthographically reflected by said reflector of additional paraboloidal form; Y one or more additional image sensors placed to receive said main rays orthographically reflected from the electromagnetic radiation from said reflector of additional paraboloid form, thus perceiving said additional essentially hemispherical scene.
37. An omnidirectional imaging apparatus, as claimed in clause 36, characterized in that said additional hemispherical scene and dich hemispherical scene are essentially complementary to one another so that the combination thereof is an essentially hemispheric scene and wherein said reflector of paraboloid form and said reflector of additional paraboloidal shape or normal convex paraboloids, placed from back to back along their truncation planes and having a common paraboloidal ej and a common focal point.
38. An omnidirectional imaging apparatus, as claimed in clause 36, characterized in that said additional hemispherical scene and dich hemispherical scene are essentially complementary to each other so that the combination thereof is essentially a spherical scene and wherein said reflector of paraboloid shape and said additional paraboloid reflector are normal concave paraboloids, positioned so that their vertices coincide and this shares a common paraboloidal axis.
39. An omnidirectional imaging method for perceiving an image of a scene from a single point of view that includes the steps of: (a) orthographically reflecting the main electromagnetic radiation rays radiating from said scene onto a reflector of essentially truncated paraboloidal shape so that a single point of view of said omnidirectional image formation method coincides with the focus point of said reflector of paraboloid shape; (b) telecentrically filtering a substance part of any principal electromagnetic radiation rays which are not orthographically reflected by said paraboloid-shaped reflecto; Y (c) perceiving said principal rays orthographically reflected from electromagnetic radiation reflecting a paraboloid shape with one or more image sensors so as to perceive said image of said scene.
40. The method, as claimed in clause 39, characterized in that step (c) comprises and perceiving said image of said scene from a position along an axis passing through the vertex of said reflector in a paraboloid manner and through of the focus of said reflector d paraboloid shape.
41. The method, as claimed in clause 39, characterized in that it further comprises the step d optically coupling said reflector of paraboloid shape and dich at least one or more image sensors with one or more planar mirrors placed between said reflector Paraboloid said one or more image sensors.
42. The method, as claimed in clause 39, characterized in that it further comprises the steps d provide an image signal which is representative of said image of said scene and convert said image signal and image signal data.
43. The method, as claimed in clause 42, characterized in that it further comprises the step d mapping said image signal data in a Cartesian coordinate system to produce an image in perspective.
44. The method, as claimed in clause 42, characterized in that it further comprises the step d mapping said image signal data in a cylindrical coordinate system to produce a panoramic image.
45. The method, as claimed in clause 42, characterized in that it further comprises the steps d interpolating said image signal data to define approximate values for missing image data, and forming a digital image of said mapped image data and said interpolated image data.
46. The method, as claimed in clause 45, characterized in that it further comprises the steps d approaching a preselected part of said digital image to thereby obtain an enlarged image of said preselected part from a predetermined focal distance by interpolating said image data for define the approximate values for missing image data, and form a digital image from said mapped image data and said interpolated image data.
47. The method, as claimed in clause 39, further characterized in that said essentially hemispherical scene and further comprises the steps of: reflect orthographically said principal rays of said electromagnetic radiation radiating from an essentially hemispheric additional scene onto a reflector of essentially additional truncated paraboloid shape so that the unique point of view of said hemispherical scene adds coincides with a focal point of said reflector of paraboloid form additional; filtering telecentrically a substantial part of any principal rays of electromagnetic radiation which are not orthographically reflected by said reflecto of additional paraboloid form; Y perceiving said main rays orthographically reflected from the electromagnetic radiation from said reflector of additional paraboloid shape with one or more additional image sensors to thereby perceive said additional hemispherical stage. - -
48. A method to omnidirectionally perceive images of a scene from a single point of view, the method comprises the steps of: (a) mounting a reflector essentially paraboloid on a fixed base; (b) mounting one or more image sensors on a mobile base; (c) orthographically reflecting the main electromagnetic radiation rays radiating from said scene onto said reflector essentially paraboloid so that said unique point of view of said omnidirectional image formation method coincides with a focus point of said reflector paraboloid; (d) telecentrically filtering a substance part of any principal rays of electromagnetic radiation which are not reflected orthographically from said reflector in a paraboloidal manner. (e) moving said mobile base to a first position (f) perceiving said first image of said first scene having a first field of view by perceiving said principal rays reflected orthographically from said electromagnetic radiation from said paraboloidal reflector with said one or more image sensors; (g) moving said mobile base to a second position different from the first position; Y (h) perceiving a second image of said scene having a second field of view by perceiving said principal rays orthographically reflected from electromagnetic radiation from said reflector of paraboloidal shape with said one or more image sensors.
49. A method to omnidirectionally perceive images of a scene, as claimed in the clause 48, characterized in that it comprises the step of optically coupling said reflector of essentially paraboloid shape and said one more image sensors with a rapid change lens of plane
50. A method to omnidirectionally perceive images of a scene, as claimed in the clause 49, characterized in that it also comprises the steps of: locate an area of interest within that scene with the lens for fast close-up on a first amplification force; Y amplifying said area of interest by placing said fast-changing lens flat to a second amplifying force greater than said first amplifying force.
51. A method to omnidirectionally perceive images of a scene from a single point of view, the method comprises the steps of: (a) mounting a reflector essentially paraboloid and truncated on a movable base, - (b) mounting one or more image sensors on a fixed base; (c) orthographically reflecting the main electromagnetic radiation rays that radiate from said scene onto a reflector of essentially paraboloid shape so that a single point of view of said omnidirectional image formation method coincides with a focal point of said reflector paraboloid; (d) telecentrically filtering a substance part of any principal rays of electromagnetic radiation which are not reflected orthographically from said reflector in a paraboloidal manner. (e) moving said mobile base to a first position; (f) perceiving a first image of said scene having a first field of view by perceiving said main rays reflected orthographically from the electromagnetic radiation from said reflector of paraboloidal shape with said one or more image sensors; (g) moving said mobile base to a second position different from said first position; Y (h) perceiving a second image of said scene having a second field of view by perceiving said principal rays orthographically reflected from the electromagnetic radiation from said reflector of paraboloidal shape with said one or more image sensors.
52. A method to omnidirectionally perceive images of a scene, as claimed in the clause 51, further characterized in that it comprises the step of optically coupling said reflector of essentially paraboloidal shape said one or more image sensors with a lens of rapid change of plane.
53. A method to omnidirectionally perceive images of a scene, as claimed in the clause 52, characterized in that it also comprises the steps of: locating an area of interest within said scene with said lens for rapid change of plane set to a first amplification force; Y amplifying said area of interest by placing said lens of rapid change from plane to a second force d amplification greater than the first force of amplification.
54. An apparatus for forming omnidirectional images to perceive an image of a scene from a unique point of view, comprising: (a) a reflector of essentially truncated paraboloid shape positioned to orthographically reflect the main rays of electromagnetic radiation radiating from said scene, the paraboloid-shaped reflector has a focus coincident with said unique view of said omnidirectional image-forming apparatus, including said reflecto of paraboloid shape; (b) telecentric means, optically coupled said reflector of paraboloid shape, to filter essentially the main rays of electromagnetic radiation which n are reflected orthographically by said reflector of paraboloid form; (c) a plurality of splitters of par rays dividing said main rays orthographically reflected electromagnetic radiation in a plurality of bundles of rays each bundle of ray comprises a part of said main ray orthographically reflected electromagnetic radiation from said reflector of paraboloid shape; Y (d) a plurality of image sensors, each image sensor is positioned to receive at least one d of said plurality of monochromatic electromagnetic radiation main rays, thereby perceiving at least one monochromatic image of said scene.
55. An apparatus for forming omnidirectional images to perceive an image of a scene from a unique point of view, comprising: (a) a reflector of essentially truncated paraboloid shape positioned to orthographically reflect the main rays of electromagnetic radiation radiating from said scene, the paraboloid-shaped reflector has a focus coincident with said unique view of said omnidirectional image-forming apparatus, including said reflecto of paraboloid shape; (b) telecentric means, optically coupled said reflector of paraboloid shape, to filter essentially the main rays of electromagnetic radiation which n are reflected orthographically by said reflector of paraboloid form, - (c) a plurality of dichroic ray splitters for dividing said principal rays orthographically reflected from electromagnetic radiation into a plurality of monochromatic main rays of electromagnetic radiation; Y (d) a plurality of image sensors, each image sensor is positioned to receive at least one d of said plurality of monochromatic electromagnetic radiation main rays, thereby perceiving at least one monochromatic image of said scene. SUMMARY The invention described herein is an omnidirectional image forming apparatus for perceiving an image of a scene from a single point of view. The omnidirectional imaging device includes a paraboloid reflector that is essentially truncated and placed to orthographically reflect the main rays of electromagnetic radiation radiating from said scenesaid reflector of paraboloid shape has a focus coincident with said single viewpoint of said omnidirectional imaging apparatus including said reflector of paraboloid shape. The omnidirectional imaging apparatus also includes a telecentric means, optically coupled to a dich reflector of form. paraboloid, to filter essentially the main rays of electromagnetic radiation which n are reflected orthographically by said reflector of paraboloid form. The omnidirectional image forming apparatus also includes one or more image sensors positioned to receive said main rays orthographically reflected d said electromagnetic radiation from said paraboloid-shaped reflector, thereby perceiving said image of said scene
MXPA/A/2000/005531A 1997-12-05 2000-06-05 An omnidirectional imaging apparatus MXPA00005531A (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US08986082 1997-12-05

Publications (1)

Publication Number Publication Date
MXPA00005531A true MXPA00005531A (en) 2001-07-03

Family

ID=

Similar Documents

Publication Publication Date Title
US6118474A (en) Omnidirectional imaging apparatus
EP0897636B1 (en) Omnidirectional imaging apparatus
US8213087B2 (en) Integrated panoramic and forward optical device, system and method for omnidirectional signal processing
US6313865B1 (en) Method and apparatus for implementing a panoptic camera system
Nayar Catadioptric omnidirectional camera
US6304285B1 (en) Method and apparatus for omnidirectional imaging
US7298548B2 (en) Multi-directional viewing and imaging
US6856472B2 (en) Panoramic mirror and system for producing enhanced panoramic images
JP3485261B2 (en) System and method for electronic imaging and processing of a hemispherical field of view
US6744569B2 (en) Method and apparatus for omnidirectional three dimensional imaging
Nayar Omnidirectional vision
WO2001068540A2 (en) Imaging apparatus
JP2006011103A (en) Spherical mirror imaging apparatus
KR100482727B1 (en) Omnidirectional imaging device and method
MXPA00005531A (en) An omnidirectional imaging apparatus
US7253969B2 (en) Spherical and nearly spherical view imaging assembly
Ohte et al. A practical spherical mirror omnidirectional camera
CA2251879C (en) Omnidirectional imaging apparatus
CA2439296A1 (en) Panoramic mirror and system for producing panoramic images