WO2005106543A1

WO2005106543A1 - Panoramic mirror and imaging system using the same

Info

Publication number: WO2005106543A1
Application number: PCT/KR2005/001249
Authority: WO
Inventors: Gyeong-Il Kweon
Original assignee: Nanophotonics Ltd.
Priority date: 2004-04-30
Filing date: 2005-04-29
Publication date: 2005-11-10
Also published as: KR100491271B1

Abstract

The present invention provides a panoramic mirror and imaging system using the same, where the panoramic mirror provides desirable projection schemes such as equi-angular projection and equi-distance projection. The dead-zone at the back of the camera which is inevitable in the conventional panoramic imaging system can be eliminated using the proposed mirror profile. This invention further provides a folded panoramic imaging system where the line of sight is changed from the rear to the front of the camera.

Description

Description PANORAMIC MIRROR AND IMAGING SYSTEM USING THE SAME Technical Field

[1] The present invention generally relates to a panoramic imaging system, and more particularly to a panoramic imaging system having a panoramic mirror with an extended field of view (FOV) in order to obtain an image of a region directly behind a camera. Background Art

[2] In an exemplary panoramic imaging system, there is provided a panoramic camera for photographing various desired sceneries. Such a camera is capable of taking an image of every direction (i.e., 360°) in one photograph. In this respect, the panoramic camera is interpreted as an imaging system capable of taking the 360 degrees view from the given position. Therefore panoramic image is one that includes all the views that a person could observe from a given position as he/she turns around from the standing position. On the other hand, an omni-directional imaging system means a system that captures the view of every possible direction from the given position. Thus, an omni-directional imaging system shows the view that a person could observe from the given position by turning around and looking up and down. Mathematically speaking, the region taken by the imaging system has a solid angle of 4π steradian.

[3] There have been a lot of studies and developments of panoramic imaging systems in various fields of applications such as real estate brokerage, nature photography and video documentary, stellar astronomy and so on. Recently more vigorous studies are undertaken in various fields such as security monitoring system, virtual tour sights for real estate, advertisement for hotels and tourist resorts, or navigational aid for mobile robots and unmanned vehicles. In such imaging systems, fisheye lenses have been commonly used. For instance, using a fisheye lens having a FOV larger than 180°, it is possible to capture the whole sky and the horizon in a single image. On this reason, the fisheye lens have been also referred to as the "all-sky lens."

[4] For an example, a high-end fisheye lens by Nikon, namely, 6mm f/5.6 Fisheye- Nikkor, has a FOV of 220° . In consequence, images taken by a camera equipped with this lens can show part of a region behind the camera. In addition, when two such cameras are employed in back-to-back configuration, then each camera captures more than 2π radians of hemi- spherical image. When the two images are stitched together using a dedicated software, then a image is provided which shows the whole 4π radians (i.e., views of every possible direction from a given point). The acquired images from the panoramic imaging system can be transmitted to a undetermined number of users at geographically distant locations through a communication medium such as the Internet. The user can select his/her own desired viewpoint in the panoramic scene and then enjoy a perspectively correct planar image on the computing device by means of image processing software, which can extract a portion of a panoramic image corresponding to the user's selected viewpoint.

[5] Therefore, every user can independently perform various operations such as panning, tilting, or zooming in or out by using the said image processing software, as though he/she was actually there while the image was taken. This is advantageous in that each user can choose his/her own viewpoint within the received image even when multiple users are connected to the same Internet site. Such advantages cannot be enjoyed in the panoramic imaging system that adopts a motion camera (e.g., a pan-tilt camera).

[6] Despite the above-mentioned advantages, however, the panoramic or omni-directional imaging system using the fisheye lens is not efficient in terms of size, weight and production costs. For instance, a sizeable number of optical glass elements are required to produce a fisheye lens having a large FOV. This is because various aberrations should be minimized to offer a clear image. Therefore, high quality fisheye lenses are usually large, heavy and expensive. In this reason, fisheye lenses are not widely usually in security /surveillance area due to the high costs and not in mobile robots/unmanned vehicles due to the large size and weight.

[7] For security monitoring purposes, it will be best if the panoramic imaging system operates at the eye-level. That is because to verify the identity of a person who enters the selected area, a detailed facial image must be secured. However, the FOV of a fisheye lens is generally not much larger than 180°. Thus, the region that can be monitored by a fisheye lens is also limited to a half sphere. In this reason, to monitor the entire area with a fisheye-based imaging system, the imaging system should be installed on the ceiling with the line of sight pointing to the floor (i.e., nadir). Needless to say, panoramic imaging system operating at the eye-level can be obtained if two such fisheye lens-based cameras are used in back-to-back configuration and the two hemi- spherical images are stitched together. However, manufacturing cost will be high, and the size and the weight might be out of spec.

[8] To overcome the aforementioned deficiencies, catadioptric imaging systems simultaneously employing a mirror and a lens are actively pursued worldwide. Fig. 1 illustrates a representative catadioptric panoramic imaging system using a convex mirror. (See J.S. Chahl and M.V. Srinivasan, "Reflective surfaces for panoramic imaging, " Appl. Opt., vol. 36, no. 31, pp. 8275-8285, 1997)

[9] As illustrated in Fig. 1, a conventional panoramic imaging system 100 comprises a rotationally symmetric mirror 102 whose cross-sectional profile is nearly a hyperbola, and a camera 104 facing the mirror 102 and the optical axis of the camera coinciding with the rotational symmetry axis 106 of the mirror 102. An incident ray 110 having an elevation or a nadir angle Φ and an aribitrary azimuth angle (i.e., 360°) around the rotational symmetry axis 106 travels toward the rotational symmetry axis 106. It is then reflected at a given point P on the mirror 102 and captured as a reflected ray 108 having the zenith angle θ with respect to the optical axis 106 of the camera 104.

[10] Fig. 2 illustrates a conceptual drawing of a rural landscape obtained using the conventional panoramic imaging system 100. As shown in Fig. 2, the photographic film or the image sensor has a square or a rectangle shape, while a panoramic image 200 obtained using the panoramic imaging system 100 has an annular shape. Non-hatched region in 200 constitute the actual panoramic image. A hatched circle 220 at the center corresponds to a region behind the camera 104, which is not captured by the camera 104 because the camera body obstructs its view. The elevation angle Φ at this region is about 0°. Within the circle 220 lies the reflected image of the camera 104 by the mirror 102. Meanwhile, the hatched regions at the four corners show normal images of forward areas, where the corresponding rays have not been reflected by the panoramic mirror 102. Fig. 3 illustrates an unwarped image converted (with respect to a cutting line 210 shown in Fig. 2) from a ring-shaped panoramic image 200 through the use of an image processing software.

[11] As illustrated in Fig. 3, processed image 230 is a planar image shaped as a normal rectangle. Accordingly, one can simultaneously obtain the images of all four directions (i.e., East, West, North and South) with only one camera 104, and this feature allows it to be advantageously used in the security /surveillance field by eliminating the dead zones that is normally inevitable with the conventional security camera. Moreover, the chromatic aberration that is normally present with conventional refractive lenses can be minimized since the catadioptric panoramic imaging system 100 adopts a reflective optical element such as the mirror 102. The lens 105 installed at the camera 104 is a standard or a wide-angle lens, and consequently the users can choose among the numerous excellent quality lenses with well-corrected aberrations. Therefore, the catadioptric panoramic imaging system 100 can provide superior images with less complex structure compared to the panoramic imaging system adopting fisheye lens.

[12] Fig. 4 shows the vertical field of view of the panoramic imaging system 100 in Fig. 1. It is herein assumed that the panoramic imaging system 300 is vertically set-up like a street lamp in order to capture an outdoor scene. A camera 301 is vertically installed heading toward the sky. A panoramic mirror 302 is located inside a transparent housing 303 and relatively fixed with respect to the camera 301 by means of a connecting member 304. The housing 303 is also connected to the camera 301. To protect the panoramic mirror 302 and the camera 301 from rain, wind and dust, they can be all located within the housing 303. The housing 303 is shaped as a dome or a shape similar thereto and can be coated with antireflective material so that a ray incident to the panoramic mirror 302 can be maximally and optimally transmitted. The camera 301 and the housing 303 are set and fixed at a proper height from the ground by a supporting member 305. As denoted in Fig. 4, the field of view of the above panoramic imaging system 300 ranges from the minimum elevation angle Φ greater than 0° to the maximum elevation angle Φ smaller than 180°, wherein the difference ΔΦ(ΔΦ=Φ -Φ ) is typically between 90° and 100°. Particularly, the minimum elevation angle Φ is usually larger than 30°, and in consequence there exists a dead zone at the backside of the camera in the shape of a cone with the vertex half angle Φ . This is the shortcoming of the current panoramic monitoring system in that an intruder standing underneath the panoramic imaging system 300 cannot be detected.

[13] Fig. 5 shows a schematic diagram of the conventional panoramic mirror and the variables needed in analyzing the mirror profile. Since the left side of the panoramic mirror 102 is the mirror image of its own right side, only the right side will be explained for convenience with reference to Fig. 5.

[14] As illustrated in Fig. 5, the incident ray 110 is reflected at a selected point 404 on the mirror 102 and passes through the origin 406 of the coordinate system. At the origin 406 lies the aperture lens or the nodal point of the camera and the optical axis of the camera coincides with the rotational symmetry axis 106 of the mirror 102. The nodal point is the location of the pinhole when the camera is approximated as an ideal pinhole camera. Here, "r" denotes the distance from the origin 406 to the point 404 on the mirror 102 in the spherical coordinate system with the z-axis coinciding with the rotational symmetry axis 106; "β" denotes the zenith angle of the tangent plane T to the mirror at the point 404 with respect to the rotational symmetry axis 106; "γ" is an angle between the tangent plane T and a straight line 402, which extends from the origin 406 to the mirror point 404; and "δ" denotes the angle between the tangent plane T and the incident ray 110. Using the laws of specular reflection, the relation between the elevation angle Φ of the incident ray 110 and the zenith angle θ of the reflected ray 108 can be expressed by equation 1, as follows.

[15] MathFigure 1 φ = π-2β+θ

[16] Radian is used as a unit of angles in equation 1 and all other equations to be explained hereinafter. The following equation 2 can be inferred from the detailed Fig. 6.

[17] MathFigure 2

[18] In the aforementioned conventional panoramic imaging system 100, an optimal shape of the mirror is such that the differential increment dΦ in the elevation angle Φ of the incident ray 110 (hereinafter referred to as the "elevation angle Φ") is proportional to the differential increment in the zenith angle dθ of the reflected ray 108 (hereinafter referred to as the "zenith angle θ") as shown below.

[19] MathFigure 3

[20] In equation 3, " " means the gain of the mirror and is selected from a natural or a real number greater than 3. [21] Using equations 1 to 3, the shape of the mirror can be derived as given in equation 4. [22] MathFigure 4

[23] In equation 4, "B" is an integral constant, "A" is an angle (i.e., A=γ(θ=0)) between the rotational symmetry axis 106 and the tangent plane T at the point of intersection r (i.e., r =r(θ=0)) of the mirror 102 and the rotational symmetry axis 106. "A" is expressed by equation 5, as shown below.

[24] MathFigure 5 A = tan ^" (rdQ/dr) I _θ=0

[25] In case A=π/2 and gain =3, equation 4 can be expressed by equation 6, as shown below. [26] MathFigure 6 2 2 ^r 0 cos(2θ)

[27] Equation 6 can be readily converted to an equation in the rectangular coordinate. In the rectangular coordinate, it can be immediately recongnized as a hyperbolic equation. (See U.S. Patent No. 5,790,181)

[28] MathFigure 7 2 2 _ 2 x - y - r ₀

[29] In equation 7, "x" and "y" are r-cosθ and r-sinθ, respectively. Therefore, in the particular case that mirror gain equals 3, the mirror has a hyperbolic shape. The mirror has a complicated shape when the mirror gain is other than 3. The typical mirror profiles insofar adopted for various applications have been spheric, conic, parabolic and hyperbolic surfaces. Particularly, the hyperbolic surface is widely used as it allows a single viewpoint imaging system due to its geometrical characteristics.

[30] The conventional panoramic imaging system adopting a convex mirror, however, cannot capture images of the region behind the camera 301, where the elevation angle Φ is near 0°. This is because as illustrated in Fig. 4, the camera 301 obstructs the view of the panoramic mirror 302, which can be a serious defect for some application purposes of the panoramic imaging system 100. For example, when the panoramic imaging system 100 is installed on a high place such as the wall, flag post, anemoscope or lightening conductor in order to monitor a building and its surroundings, the region directly underneath the camera 301 are excluded from the monitoring zone, which cannot be tolerated.

[31] To overcome such a problem, a panoramic imaging system capable of taking images of the region directly underneath the camera has been proposed. As illustrated in Fig. 7, a panoramic imaging system 500 comprises a camera 504 and a mirror 502 having the angle "A" exceeding 90°. An incident ray 506 with the elevation angle Φ=0 ⁰ is reflected by the mirror 502 and captured by the camera 504. Hatched areas in Fig. 7 are non-imaged regions obstructed by the camera 504. When the system 500 is used for the purpose as illustrated in Fig. 4, the non-imaged regions are reduced to a narrow cylindrical region rather than a wide conical region, where the shape of the non-imaged regions is determined by the horizontal cross-section of the camera 301 and its supporting member 305. However, there is a problem in that the system 500 cannot maintain the gain of mirror (i.e., =dΦ/dθ) as a constant. When the gain of the mirror is not a constant, the captured image of an object appears to be severely distorted and image resolution varies as the elevation angle of the incident ray varies.

[32] A stereovision (or a stereoscopic vision) is one of the major fields in computer vision seeking to mimic the ability of a creature with binocular vision (including humans) to retrieve three-dimensional distance information. A stereovision system is mainly used either for a ranging purpose or to retrieve the three dimensional profile of an object or a scene. The three-dimensional shape measurement is basically a task for assigning distance information to all pixels corresponding to a captured object. Therefore, distance measurement (i.e., ranging) is the central technology in the stereovision system. [33] As schematically shown in Fig. 8, the most common method of embodying the stereovision system is employing two cameras 1302, 1304 with identical specifications that are laterally disposed with an interval D and pointing the same direction (i.e., optical axes of the two cameras are parallel to each other). In other words, the nodal points of the two cameras are apart from each other with an interval D and a line connecting the two nodal points is perpendicular to the optical axes of the two cameras. Depending on the purpose of the application, the interval D can be set similar to a distance between the eyes of an average human.

[34] To retrieve distance information with the stereo imaging system so configured, the object must be captured by both the left camera 1302 and the right camera 1304. Then a specific point P of the object is selected from the left and the right images. More specifically, a pixel corresponding to a particular object point P is found from the left image taken by the left camera 1302, and the corresponding pixel is found from the right image taken by the right camera 1304. Numerous technologies are employed for finding the matching pair of pixels corresponding to a given point. Once a matching pair of pixels corresponding to the point P are found, angles θ and θ between the point P and optical axes of the two cameras 1302 and 1304 are computed based on the pixel coordinates. Once we know the two angles θ 1 , θ2 and the interval D, the distance can be calculated using a simple triangulation.

[35] It is not necessarily mandatory to employ two cameras in constructing a stereovision system. For example, the screen of a single camera can be divided into left and right parts by means of a mirror or a bi-prism, thus allowing two separate images of a same object to be captured. However, the fundamental principle is the same as the method explained above.

[36] Depending on the application fields, a panoramic stereo imaging system or a panoramic rangefinder may be desirable. In the field of security /surveillance, for instance, it will be very useful if the distance information to a trespasser is available. Similarly, the panoramic stereo imaging system can be used by the military for monitoring mountain ranges, wildernesses and coasts. In such cases, the distance information to a potential invader can be of paramount importance. Transgressor who is far way from here is not really a transgressor, or at least a less threatening one. Further, the panoramic stereo imaging system can be also useful for mobile robots, unmanned vehicles and aircrafts. Such self-navigating devices should be equipped with a collision avoidance system and consequently a distance to an obstacle must be computed swiftly and precisely. The conventional stereovision system, as illustrated in Fig. 8, can detect and measure the distance to an obstacle in the front area only. Accordingly, it is impossible for the conventional system to generate a warning message or take an appropriate preventive measure against an obstacle or a mobile system approaching from the side or from the rear.

[37] The above-mentioned problems can be resolved by using stereovision systems comprising two panoramic mirrors and one or two cameras, as illustrated in Figs. 9 to 13. Then the same shortcomings associated with the panoramic imaging system 100 in Fig. 1 is found with the conventional panoramic stereovision systems. In other words, in the systems shown in Figs. 9 to 13, the camera itself obstructs the view of the panoramic mirror. In such cases, it is impossible to detect or to measure the distance to an object that is located near the nadir having the elevation angle 0°. Furthermore, there exist additional dead zones because one panoramic mirror partly obstructs the view of the other panoramic mirror.

[38] In addition to this, for the panoramic stereovision systems given in Figs. 11 to 13, the vertical FOV and the mirror gain are different for the two panoramic mirrors. This causes the realization of a panoramic stereovision system technically difficult, and a low resolution in the captured image is inevitable. Disclosure of Invention Technical Problem

[39] The object of the present invention is to provide a panoramic mirror capable of obtaining images of region directly behind the camera, while maintaining a constant resolution over the entire fields of vision. Another object of the present invention is to provide various imaging systems that incorporate and use the above. Technical Solution

[40] In order to achieve the above objects, the present invention provides a panoramic mirror comprising: a mirror surface having a rotationally symmetric profile about the z-axis in a spherical coordinate, wherein the z-axis has a zero zenith angle, the mirror profile is given as a set of spherical coordinate pairs (θ, r(θ)) where θ is the zenith angle of a first reflected ray reflected at a first point on the mirror surface and passing through the origin of the spherical coordinate, the zenith angle θ ranges from the minimum zenith ang ^σle θ 1 to the maximum zenith ang ^σle θ 2 (θ 1 ≤θ≤θ 2 ), and r(θ) is the distance from the origin to the first point and satisfies the following equation:

[41]

[42] where θ is the zenith angle of a second reflected ray reflected at a second point on the mirror surface and passing through the origin, the zenith angle θ takes a value between the minimum zenith angle θ 1 and the maximum zenith angle θ 2 (θ 1 <θ i <θ 2 ), r(θ i ) is the distance from the origin to the second point, and Φ(θ) is the zenith angle of the tangent plane T to the mirror at the first point and this is also given as a function of the zenith angle θ and [43] wherein the first reflected ray is formed by an incident ray having a zenith angle δ, the zenith angle δ being a function of the zenith angle θ, and θ, Φ and δ have a functional relation given in the following equation: [44]

[45] In order to achieve the above objects, the present invention also provides a double panoramic mirror, comprising: a first mirror surface having a rotationally symmetric profile about the z-axis in a spherical coordinate, wherein, the z-axis has zero zenith angle, the profile of the first mirror surface is given as a set of spherical coordinate pairs (θ , r (θ )), θ is the zenith angle of a first reflected ray reflected at a first point on the first mirror surface and passing through the origin of the spherical coordinate, the zenith angle θ ranges from the minimum zenith angle θ to the maximum zenith angle θ (θ <θ <θ ), and r (θ ) is the distance from the origin to the first point and satisfies 12 II I 12 I I the following equation:

[46]

[47] where θ is the zenith angle of a second reflected ray reflected at a second point on the first mirror surface and passing through the origin, the zenith angle θ Ii takes a value between the minimum zenith ang ^σle θ II and the maximum zenith ang ^σle θ 12 (θ II <θ Ii <θ 12 ), r I (θ Ii ) is the distance from the origin to the second point, and Φ I (θI ) is the zenith angle of the first tangent plane to the first mirror surface at the first point and is a function of the zenith angle θ , and [48] wherein the first reflected ray is formed by a first incident ray having a zenith angle δ , the zenith angle δ is a function of the zenith angle θ , and θ , Φ and δ have a I I I I I I functional relation given in the following equation:

[49]

Φ I 2 ;and [50] a second mirror surface having another rotationally symmetric profile about the said z-axis, wherein the profile of the second mirror surface is described with a set of spherical coordinate pairs (θ , r (θ )), θ is the zenith angle of a third reflected ray reflected at a third point on the second mirror surface and passing through the origin of the spherical coordinate, the zenith angle θ ranges from the minimum zenith angle θ 01 to the maximum zenith angle θ 02 (θ 01 <θ O <θ 02 ), and r 0 (θ 0 ) is the distance from the origin to the third point and satisfies the following equation:

[51]

[52] where θ is the zenith angle of a fourth reflected ray, the fourth reflected ray

reflected at a fourth point on the second mirror surface and passing through the origin, the zenith angle θ takes a value between the minimum zenith angle θ and the

maximum zenith angle θ is the distance from the origin to the

fourth point, Φ(θ ) is the zenith angle of the second tangent plane to the second mirror surface at the third point and is a function of the zenith angle θ , and [53] wherein the third reflected ray is formed by a second incident ray having a zenith angle δ , the zenith angle δ is a function of the zenith angles θ and θ , Φ and δ o o o o o o have the following functional relation:

[54] 6₀+ θ₀

[55] and wherein the distance r (θ ) from the origin to the first point having the maximum zenith angle θ and the distance r (θ ) from the origin to the third point 12 O 01 having the minimum zenith angle θ are the same (r (θ )= r (θ )). ^{σ σ} 01 I 12 O 01

[56] In order to achieve the above objects, the present invention also provides a folded panoramic mirror, comprising: a first mirror including a curved mirror surface having a rotationally symmetric profile, wherein the curved mirror surface extends from a first inner hoop having a radius p to a first outer hoop having a radius p , and the first mirror has a circular hole inside of the inner hoop; and

[57] a second mirror including a plane mirror surface having a ring shape, wherein the plane mirror surface faces the curved mirror surface and the dimension of the second mirror is determined by a second inner hoop having a radius p and a second outer hoop having a radius p

[58] wherein, the first mirror and the second mirror share the same rotational symmetry axis, the curved mirror surface is described with a set of spherical coordinate pairs (θ, r(θ)) where the z axis (i.e, the axis having a zenith angle θ=0) of the spherical coordinate system coinciding with the rotational symmetry axis of the mirror, θ is the zenith angle of a first reflected ray formed by an incident ray having a zenith angle δ at a first point on the curved mirror surface, the first reflected ray passes through the origin of the spherical coordinate, the zenith angle θ ranges from the minimum zenith angle θ to the maximum zenith angle θ (θ ≤θ≤θ ), and r(θ) is the distance from the origin to the first point and satisfies the following equation:

[59] sin θ +cotψ( θ )cos θ r(θ)=r( θ,-)exp d cos θ - cotψ( θ )sin θ

[60] where θ is the zenith angle of a second reflected ray reflected at a second point on the curved mirror surface and passing through the origin, and the zenith angle θ takes a value between the minimum zenith angle θ 1 and the maximum zenith angle θ 2 (θ 1 <θ i <θ 2 ), r(θ l ) is the distance from the origin to the second point, and Φ(θ) is the zenith angle of the tangent plane to the first mirror surface at the first point and is a function of the zenith angle θ and satisfies the following equation: [61]

[62] where δ(θ) is the zenith angle of the incident ray and is a function of the zenith angle θ, and the zenith angle δ(θ) of the incident ray ranges from the minimum zenith angle δ larger than zero to the maximum zenith angle δ equal to π(0<δ <δ≤δ =π),

[63] and wherein the radius p of the first inner hoop is determined as the following equation:

[64]

[65] and wherein the radius p of the first outer hoop is determined as the following equation:

[66] ρ ₂= θ ₂)sin θ

[67] and wherein the radius p of the second inner hoop is determined as the following equation: [68] p_j— z ₀tan θ

[69] where z is the distance from the origin to the plane mirror surface and satisfies the following equation:

[70] z ₀- p cot θ ₂

[71] and wherein the radius p of the second outer hoop of the planar ring mirror is identical to the radius p of the first inner hoop of the curved mirror (i.e., p =p ).

[72] In order to achieve the above objects, the present invention also provides a panoramic imaging system, comprising:

[73] a panoramic mirror including a curved mirror surface having a rotationally symmetric profile, wherein the curved mirror surface extends from the inner hoop having a radius p to the outer hoop having a radius p ; and

[74] a camera facing the panoramic mirror, whereby the curved mirror surface is within the field of view of the camera,

[75] wherein the curved mirror surface is described with a set of spherical coordinate pairs (θ, r(θ)) with the z-axis coinciding with the rotational symmetry axis of the mirror, θ is the zenith angle of a first reflected ray formed by an incident ray having a zenith angle δ at the first point on the curved mirror surface, the first reflected ray passes through the origin of the spherical coordinate, the zenith angle θ ranges from the minimum zenith angle θ to the maximum zenith angle θ (θ ≤θ≤θ ); and r(θ) is the distance from the origin to the first point and satisfies the following equation:

[76] sin θ +cotψ( θ )cos θ r(θ)=r( θ )exρ d O Θ, cos θ - cotψ( θ )sin θ

[77] where θ is the zenith angle of a second reflected ray reflected at a second point on the curved mirror surface and passing through the origin, the zenith angle θ takes a value between the minimum zenith angle θ and the maximum zenith angle θ (θ ≤θ ≤θ 1 2 1 i 2 ), r(θ l ) is the distance from the origin to the second point, Φ(θ) is the zenith angle of the tangent plane to the mirror surface at the first point and is a function of the zenith angle θ, and θ, Φ and δ have the following functional relation:

[78] 5+θ Φ [79] and wherein the camera optical axis coincides with the z-axis, and the nodal point of the camera is located at the origin. [80] In order to achieve the above objects, the present invention also provides a double panoramic imaging system, comprising: [81] a curved mirror including :

[82] a first mirror surface having a rotationally symmetric profile, wherein the first surface extends from a first inner hoop having a radius p to a first outer hoop having a radius p 12 ; and

[83] a second mirror surface having another rotationally symmetric profile, wherein the second mirror surface extends from a second inner hoop having a radius p to a second outer hoop having a radius p , and the radius p of the second inner hoop is not 02 01 smaller than the radius p 12 of the first outer hoop (p 12 ≤p 01 ); and

[84] a camera facing the mirror, whereby the first and the second mirror surfaces are within the field of view of the camera,

[85] wherein, the profile of the first mirror surface is described with a set of spherical coordinate pairs (θ , r (θ )), θ is the zenith angle of a first reflected ray reflected at a first point on the first mirror surface and passing through the origin of the spherical coordinate, the zenith angle θ ranges from the minimum zenith angle θ to the maximum zenith ang °le θ 12 (θ II ≤θ I ≤θ 12 ), and r I (θ I ) is the distance from the orig °in to the first point and satisfies the following equation:

[86]

[87] where θ is the zenith angle of a second reflected ray reflected at a second point of the first mirror surface and passing through the origin, the zenith angle θ takes a value Ii between the minimum zenith ang ^σle θ II and the maximum zenith ang ^σle θ 12 (θ II ≤θ Ii ≤θ 12 ), r I (θ Ii ) is the distance from the origin to the second point, and Φ I (θI ) is the zenith angle of the first tangent plane to the first mirror surface at the first point and is a function of the zenith angle θ , [88] and wherein the first reflected ray is formed by a first incident ray having a zenith ang °le δ I , the zenith ang °le δ I is a function of the zenith ang °le θ I and θ I , Φ I and δ I have the following functional relation:

[89]

[90] and wherein the profile of the second mirror surface is described with a set of spherical coordinate pairs (θ , r (θ )), θ is the zenith angle of a third reflected ray reflected at a third point on the second mirror surface and passing through the said origin of the said spherical coordinate, the zenith angle θ ranges from the minimum zenith angle θ to the maximum zenith angle θ (θ ≤θ ≤θ ), and r (θ ) is the ^σ 01 02 01 O 02 O 0 distance from the said origin to the third point and satisfies the following equation:

[91]

[92] where θ is the zenith angle of a fourth reflected ray reflected at a fourth point on

θ

angle of the second tangent plane to the mirror surface at the third point and is a function of the zenith angle θ , the third reflected ray is formed by a second incident ray having a zenith angle δ , the zenith angle δ is a function of the zenith angle θ , ^J o o o and θ , Φ and δ have the following functional relation: 0 0 0

[93] δ₀+ θ ₀ t o 2

[94] and wherein the camera optical axis coincides with the z-axis, and the nodal point of the camera is located at the origin. [95] In order to achieve the above objects, the present invention also provides a folded panoramic imaging system, comprising: [96] a first mirror including a curved mirror surface having a rotationally symmetric profile, wherein the curved mirror surface extends from a first inner hoop having a radius p to a first outer hoop having a radius p , and the first mirror has a circular hole inside the inner hoop; [97] a second mirror including a plane mirror surface having a ring shape, wherein the plane mirror surface faces the curved mirror surface and the dimension of the planar ring mirror is defined by a second inner hoop having a radius p and a second outer hoop having a radius p ; a camera facing the planar ring mirror, whereby the second mirror surface is within the field of view of the camera, [98] wherein, the first mirror and the second mirror share the same rotational symmetry axis, the curved mirror surface is described with a set of spherical coordinate pairs (θ, r(θ)) with the z-axis coinciding with the rotational symmetry axis, θ is the zenith angle of a first reflected ray formed by an incident ray having a zenith angle δ at a first point on the curved mirror surface, the first reflected ray passes through the origin of the spherical coordinate, the zenith angle θ ranges from the minimum zenith angle θ to the maximum zenith angle θ (θ ≤θ≤θ ), and r(θ) is the distance from the origin to the first point and satisfies the following equation:

[99]

[100] where θ is the zenith angle of a second reflected ray reflected at a second point on the curved mirror surface and passing through the origin, the zenith angle θ takes a value between the minimum zenith angle θ and the maximum zenith angle θ (θ ≤θ ≤θ 1 2 1 i ), r(θ ) is the distance from the origin to the second point, and Φ(θ) is the zenith angle 2 l of the tangent plane to the first mirror at the first point and is a function of the zenith angle θ and satisfies the following equation: [101]

[102] where δ(θ) is the zenith angle of the incident ray and is a function of the zenith angle θ and the zenith angle δ(θ) of the incident ray ranges from the minimum zenith angle δ larger than zero to the maximum zenith angle δ equal to π(0<δ ≤δ≤δ =π),

[103] and wherein the radius p of the first inner hoop is determined as the following equation:

[104] p ₁ = p( θ ₁) = r( θ ₁)sin θ ₁

[105] and wherein the radius p of the first outer hoop is determined as the following equation:

[106] ρ ₂ ⁼ θ ₂)sin θ

[107] and wherein the radius p of the second inner hoop is determined as the following equation: [108] p_j— z ₀tan θ where z is the distance from the origin to the plane mirror surface and satisfies the following equation: z ₀- p cot θ ₂ and wherein the radius p of the second outer hoop is identical to the radius p of the first inner hoop (p =p ). Brief Description of the Drawings Fig. 1 is a schematic diagram illustrating a conventional panoramic imaging system employing a convex mirror. Fig. 2 is a conceptual drawing of a rural landscape obtainable by the conventional panoramic imaging system in Fig. 1. Fig. 3 is a conceptual drawing of a planar image with perspectively normal view obtainable by unwarping the panoramic image in Fig. 2. Fig. 4 illustrates the vertical field of view of the panoramic imaging system shown in Fig. 1. Figs. 5 and 6 are the schematic diagrams showing the profile of the panoramic mirror in Fig. 1 and the proper variables needed in analyzing the profile. Fig. 7 is a schematic diagram of a panoramic imaging system capable of monitoring the region directly behind the camera. Fig. 8 is a schematic diagram illustrating the principle of the conventional stereovision system. Figs. 9 to 13 are the schematic diagrams illustrating the structure of the panoramic stereovision system in prior arts. Fig. 14 is a schematic diagram showing the profile of the panoramic mirror in accordance with the present invention and the proper variables needed in analyzing the profile. Fig. 15 is an enlarged view of a portion in Fig. 14. Fig. 16 is a schematic diagram illustrating the principle of an equi-angular projection. Fig. 17 shows a profile of an exemplary panoramic mirror providing the equiangular projection in accordance with the present invention. Fig. 18 is an enlarged view of the mirror profile in Fig. 17 near the inner rim. Fig. 19 shows the profile of the panoramic mirror in Fig. 17 fitted a 7 order power series in p using the least square error approximation. Fig. 20 is a schematic diagram of a panoramic imaging system employing the panoramic mirror shown in Fig. 17. Fig. 21 is a graph showing the projection scheme of the panoramic mirror in Fig. 17 as the functional relation between the zenith angle of the incident ray and the zenith angle of the reflected ray. Fig. 22 shows an inverting type panoramic mirror profile providing the equi-angular projection in accordance with the present invention. Fig. 23 is a schematic diagram of a panoramic imaging system employing the panoramic mirror shown in Fig. 22. Fig. 24 is a graph comparing the projection schemes of the normal and the inverting type panoramic mirrors respectively shown in Figs. 17 and 22. Fig. 25 illustrates the vertical field of view of the panoramic imaging system shown in Fig. 23. Fig. 26 is a schematic diagram illustrating the principle of the equi-distance projection. Fig. 27 shows a profile of the panoramic mirror providing the equi-distance projection in accordance with the present invention, along with the profile of the mirror fitted to a 8 order power series in p by using the least square error approximation. Fig. 28 compares the profile of the panoramic mirror shown in Fig. 27 providing the equi-distance projection with the profile of the panoramic mirror shown in Fig. 17 providing the equi-angular projection. Fig. 29 is a graph showing the projection scheme of the panoramic mirror shown in

Fig. 27. Fig. 30 shows a schematic diagram of a panoramic stereovision system comprising double panoramic mirrors in accordance with the present invention. Fig. 31 is a schematic diagram illustrating the principle of the distance measurement in the panoramic stereovision system shown in Fig. 30. Fig. 32 is a graph showing the projection scheme of the panoramic stereovision system shown in Fig. 30. Fig. 33 is a schematic diagram of another embodiment of the panoramic stereovision system in accordance with the present invention. Fig. 34 is a schematic diagram of still another embodiment of the panoramic stereovision system in accordance with the present invention. Fig. 35 shows a conceptual drawing of a folded panoramic mirror in accordance with the present invention. Fig. 36 is a schematic diagram illustrating the location and the dimension of the planar ring mirror in the folded panoramic imaging system. Fig. 37 is a schematic diagram showing how the maximal height of the planar ring mirror is decided. [144] Fig. 38 is a schematic diagram illustrating the location and the dimension of the planar ring mirror in the folded panoramic imaging system in the particular case that the zenith ang ^σle δ 1 is 180°.

[145] Fig. 39 shows a conceptual drawing of a folded panoramic imaging system including blinds for preventing unnecessary rays.

[146] Fig. 40 is a conceptual drawing of a panoramic imaging system for obtaining an additional perspectively normal image through the center hole of the planar ring mirror.

[147] Fig. 41 is a schematic diagram of a desirable image sensor for a panoramic camera which is suitable in simultaneously capturing the panoramic image and the perspectively normal image.

[148] Figs. 42 and 43 show conceptual drawings of a motion tacking imaging system comprising a camera and the base that is capable of providing pan-tilt motion in accordance with the present invention. Best Mode for Carrying Out the Invention

[149] Referring to Fig. 14, the profile of the panoramic mirror constructed in accordance with the present invention will be described.

[150] As shown in Fig. 14, the panoramic mirror 604 of the present invention has a rotationally symmetric structure. The nodal point of the camera coincides with the reference point on the rotational symmetry axis 602 (i.e., at the origin 610). Both the optical axis of the camera (not shown) and the rotational symmetry axis 602 of the panoramic mirror 604 coincide with the z-axis of the coordinate system. An incident ray 606 directed to the rotational symmetry axis 602 and reflected at a point P on the panoramic mirror 604 has a zenith angle δ. The reflected ray 608 having a zenith angle θ passes through the nodal point 610 of the camera and is captured on the image sensor (not shown). The point P on the panoramic mirror 604 can be defined by two variables in the cylindrical coordinates, namely, a perpendicular (horizontal) distance from the rotational symmetry axis 602 (hereinafter referred to as p) and a distance (height) from the origin 610 along the z-axis (hereinafter referred to as z). As such, the location of the point P is given as a coordinate pair (p, z). In this respect, the panoramic mirror profile can be defined by providing the vertical distance z=z(p) for every valid value of the horizontal distance p. Therefore, p becomes the independent variable and z becomes the dependent variable.

[151] As can be seen in Fig. 14, the zenith angle θ of the reflected ray 608, which is reflected at the point P of the panoramic mirror 604 and passing through the nodal point 610, ranges from the minimum zenith angle θ to the maximum zenith angle θ . For each zenith angle θ of a reflected ray that were reflected at a point P on the mirror surface, a zenith angle δ of the incident ray and the coordinate pair (p, z) corresponding to the point P are uniquely associated. Furthermore, the minimum zenith angle θ and the maximum zenith angle θ correspond to the distances p and p , respectively, which are the minimum and the maximum values of distance p. On the other hand, δ and z , which correspond to the minimum zenith angle θ , are not always smaller than δ and z , respectively. Rather, δ and z may be the minimum, the maximum or an intermediate value within their own range, respectively.

[152] Shown in Fig. 15 is a detailed schematic diagram of the right part of the panoramic mirror 604 with the rotational symmetry axis 602. As shown in Fig. 15, the point P on the panoramic mirror 604 can be defined in the spherical coordinates as a set of the zenith angle θ of the reflected ray 608 and the distance r from the origin 610. Similar to the case of the cylindrical coordinates, the profile of the panoramic mirror 604 can be expressed as the distance from the origin r as a function of the zenith angle θ of the reflected ray as shown by equation 8 below.

[153] MathFigure 8 r=r(θ)

[154] Also, the variables p and z in the cylindrical coordinate can be given in terms of the spherical coordinates as shown by equations 9 and 10 below. [155] MathFigure 9 z=r(θ)cos(θ)

[156] MathFigure 10 p(θ) = r(θ)sinθ

[157] The profile of the panoramic mirror 604 is determined so that an incident ray 606, which is incident on the panoramic mirror 604 with the zenith angle δ from all directions, namely, with an arbitrary azimuth angle ω ranging from 0° to 360°, to pass through the nodal point 610 of the camera as the reflected ray 608 having a zenith angle θ after the incident ray 606 is reflected on the surface of the panoramic mirror 604. The profile of the panoramic mirror 604 can also be defined with the zenith angle Φ of the tangent plane T to the mirror at an arbitrary point P on the surface of the panoramic mirror 604. For a point P on the mirror having a spherical coordinate (θ, r(θ)), the zenith angle Φ of the tangent plane T (i.e., the inclination angle of the mirror surface) can be given as a function of the zenith angle θ of the reflected ray, that is, Φ=Φ(Θ). The zenith angle Φ of the tangent plane T satisfies the following equation 11.

[158] MathFigure 11 dp tanφ dz

[159] Both z and p can be given as functions of θ using equations 9 and 10. The following equation 12 can be obtained by inverting the equation 11. It is required to invert the equation 11 because tanΦ diverges to infinity near Φ=90°.

[160] MathFigure 12 dz

dQ

[161] In order to calculate the numerator in equation 12, namely, dz/dθ, equation 13 is obtained by differentiating equation 9. [162] MathFigure 13 dz dr cosθ-r sinθ ⁼ r 'cosθ-r sinθ d§ d

[163] The prime symbol denotes derivation with respect to θ. In the same manner, in order to calculate the denominator in equation 12, namely, dp/dθ, equation 14 is obtained by differentiating equation 10.

[164] MathFigure 14 dp ^r 'sinθ+rcosθ d

[165] Equation 12 can then be expressed as equation 15 using equations 13 and 14.

[166] MathFigure 15

[167] As schematically shown in Fig. 15, reflections on a mirror follows the familiar law of specular reflections. Therefore, the zenith angle Φ of the tangent plane T can be given as a simple function of the zenith angle δ of the incident ray 606 and the zenith angle θ of the reflected ray 608 as the following equation.

[168] MathFigure 16 _ δ + θ

[169] Alternatively, the zenith angle δ of the incident ray 606 can be denoted as the following. [170] MathFigure 17 δ = 2 φ -θ [171] After a separation of variables, the equation 15 can be reduced to the equation 18.

[172] MathFigure 18 r' _ sinθ+ cotφcosθ r cosθ- cotψ sinθ

[173] By formally integrating the equation 18, the following equation 19 can be obtained.

[174] MathFigure 19

[175] In equation 19, θ' is a dummy variable, the zenith angle θ of the reflected ray wwhhoossee value is between θ and θ , denotes the lower limit of the indefinite integral, and r(θ ) is

. According to the present invention, the variables θ , θ , δ , δ , θ , p and Φ(θ) are 1 2 1 2 i i design parameters for designing the profile of the panoramic mirror 604. Practically, θ can be chosen between θ and θ , and r is given as r =p /sinθ from equation 10, and z 1 2 l i l l l can be determined using equation 9. Although the limiting values of the function Φ(θ), namely Φ =(δ +θ )/2 and Φ =(δ +θ ))/2, are determined automatically, in principle, it can have arbitrary functional forms between θ and θ . The functional form of Φ(θ) should be determined in order to give a desirable projection scheme. [176] In designing profile of the panoramic mirror 604 with which images of region directly behind the camera is to be obtained, the minimum perpendicular distance p from the rotational symmetry axis 602 should be larger than the size of the lens, the body and the maximum radius of the camera support. In other words, the NMA of the panoramic mirror 604, which is the non-imaging region on the surface of the panoramic mirror 604, should be large enough to cover the largest elements other than the mirror such as the lens, the body and the camera support (see Fig. 14). The distance r , which demarks the border between the NMA and the mirror surface of the panoramic mirror 604 is given as r =p /sinθ by using the equation 10. However, this constraint is not needed if is not necessary to take the images of the region directly behind the camera.

[177] Although the functional form of Φ(θ), and hence the mirror profile, can be given as an arbitrary function, one desirable mirror profile can be the one implementing the equi-angular projection scheme. Referring to Fig. 16, for the convenience of description, it is illustrated as if an incident ray 810 which is incident on the surface of the panoramic mirror, directly passes through the nodal point 806, while a reflected ray 808 passing through the nodal point 806 is captured on the image sensor 802 of the camera. The distance from the nodal point 806 to the image sensor 802 is approximately equal to the focal length /of the camera. A part of the reflected ray path 808 (i.e., from the reflection point on the panoramic mirror to the nodal point 806) has been omitted in Fig. 16, and the beginning point of the vector denoting the incident ray 810 is moved from the reflection point P on the panoramic mirror to the nodal point 806 of the camera. As schematically shown in Fig. 16, the mirror profile providing the equi-angular projection is one where the ratio between the differential increment of the zenith angle δ of the incident ray 810 and the differential increment of the zenith angle θ of the reflected ray 808 is a constant.

[178] MathFigure 20

[179] In equation 20, is a constant. Thus, the inclination angle Φ of the surface of the panoramic mirror 604 is given as a linear function of the zenith angle θ of the reflected ray, as shown in equation 21.

[180] MathFigure 21 φ = βθ + Ψ

[181] In equation 21, β and Ψ are other constants. By using the equi-angular projection scheme, it is possible to maintain a constant angular resolution of the images. When we see an object, the size of the object is estimated by measuring the angular extension of the object image on the retina. Therefore the equi-angular projection scheme is most similar to the way a human sees an object.

[182] Hereinafter, the ranges of the zenith angle δ of the incident ray 606 and the zenith angle θ of the reflected ray 608, which should be considered in designing the mirror profile, will be described. The zenith angle θ ranges from the minimum zenith angle θ to the maximum zenith angle θ . On the other hand, the zenith angle δ ranges from δ to δ . In cases where the surface profile is similar to that of the quasi-convex mirror shown in Fig. 20, δ 1 and δ2 become the maximum and the minimum zenith angles, re- spectively. On the other hand, in cases where the surface profile is similar to that of the quasi-concave mirror shown in Fig. 23, δ and δ become the minimum and the maximum zenith angles, respectively.

[183] First, the range of the zenith angle δ of the incident ray will be described. In accordance with the present invention, the range of the zenith angle δ is determined by considering the particular application fields of the panoramic imaging system having the panoramic mirror 604. For instance, when the panoramic imaging system of the present invention is installed on a place near the ground, regions below the panoramic imaging system will be hardly of any interest. Therefore, it suffices if the maximum value of the zenith angle δ of the incident ray is π/2. On the other hand, if the panoramic imaging system of the present invention is installed on a higher place, such as a roof of a building, then the regions below the panoramic imaging system will be of major concern. Therefore, the maximum value of the zenith angle δ of the incident ray should not be far less from π. If the maximum value of the zenith angle δ of the incident ray exceeds π, then there are some region, of which images are captured twice. Thus, this characteristic can be utilized in a field requiring stereovision. In the example given in Fig. 4, even an image of the pole supporting the panoramic imaging system can be captured. Therefore it is possible to acquire images of a snake or a worm that is sitting on the pole.

[184] For the range of the zenith angle θ of the reflected ray, the maximum value of the zenith angle θ is determined by considering the field of view (FOV) of the refractive lens in the camera of the panoramic imaging system. To be more specific, the maximum value of the zenith angle θ should be smaller than the half of the FOV of the refractive lens in order to get full circle panoramic images without clipping. For instance, 35 mm film has a rectangular shape, 36mm wide and 24mm high. In a case of using 35 mm film with a camera equipped with a standard 50mm focal length lens, the half angle of the image captured on the film is mere 19.8° horizontal, 13.5° vertical and 23.4° diagonal. Conventionally, the FOV of this standard lens is referred as 46°. Therefore, the maximum zenith angle of the reflected ray should be limited below 13.5° in order to get an undipped panoramic image in a full circle. The maximum zenith angle of the reflected ray can be enlarged by using a lens having a focal length shorter than that of the standard lens or by using a camera, of which the image sensor size is larger than that of the conventional 35 mm film.

[185] Unlike the maximum zenith angle θ , the range of the minimum zenith angle θ of the reflected ray is limited by different factors. If the focal length of the camera lens is "f" as denoted in Fig. 16, the reflected ray having the zenith angle θ forms a circle in the image sensor 802 of which the radius is "f-tanθ ". Therefore, if the minimum zenith angle θ is too small, then the number of pixels along the perimeter of a circle having that radius will be too small, and the captured image with that reflection angle will be too low in resolution because the image is represented by an insufficient number of pixels.

[186] If the zenith angle δ of the incident ray is π, then it suffices that the minimum zenith angle θ of the reflected ray is near 0°. This is because the imaged region with the incident ray having the zenith angle δ equal to π is directly behind the camera. As such, if all objects are at a relatively long distance from the camera, then the region having that angle is rather a single point. It is more like a south pole on earth with latitude angle -π/2. Thus, image of a tiny region is captured with very few pixels. On the other hand, if the zenith angle δ of the incident ray is near π/2, then the reflected ray having the minimum zenith angle θ were originated from the horizon. Therefore, if the minimum zenith angle θ is near 0°, then the widest region (a great circle including the horizon) has been captured with the least number of pixels. If the image captured by the panoramic mirror 604 in accordance with the present invention is geometrically transformed to a conventional planar image, then the unwarped image will be low in resolution. In an embodiment of the present invention, it has been assumed that the zenith angles θ and θ are 10° and 20°, respectively. However, the key arguments of the present invention are not changed even if the values of the zenith angles θ and θ are replaced with other values.

[187] Now, the size of the panoramic imaging system of the present invention will be described. If it is required to capture the images of the region directly behind the camera, the minimum value p of the perpendicular distance from the rotational symmetry axis 602 (see Fig. 15) is bounded by the size of the camera, and more precisely by the largest cross-section of the camera and its peripherals perpendicular to the optical axis. If the minimum zenith angle θ of the reflected ray is set to a very small value, then the distance r demarking the border between the NMA and the mirror surface of the panoramic mirror 604 should be very large in accordance with the equation 10. Therefore, this should be considered in designing the panoramic imaging system. Once the zenith angles δ and δ of the incident ray and the minimum and maximum zenith angles θ and θ of the reflected ray are determined, the constants in equation 21 can be obtained by equations 22 to 25.

[188] MathFigure 22 δ i+ θ i φ( θ ₁)= Φ ι = 2 ⁼ P ^θ ι^+ψ

[189] MathFigure 23 δ ₂+ θ ₂ Φ( θ ₂)= φ ₂= ^" = β θ ₂ + Ψ

[190] MathFigure 24

[191] MathFigure 25

[192] Referring to Fig. 17, the profile of the panoramic mirror with the equi-angular projection scheme in accordance with the present invention is discussed below. In Fig. 17, the profile of a panoramic mirror to the right side of the rotational symmetry axis is shown. It is assumed that p is 2.5cm, the minimum and maximum zenith angles θ and θ of the reflected ray are 10° and 20°, respectively, and the corresponding zenith angles δ and δ of the incident ray are 180° and 45°, respectively. Therefore, it is possible to get image of all directions with the panoramic mirror, except for a cone shaped region with a half-angle of 45° in the forward direction of the camera.

[193] Fig. 18 shows an enlarged view of the mirror profile given in Fig. 17 near the central rim (i.e., inner hoop). Note that the scale is different for the vertical and the horizontal axis in Fig. 18. As can be noted from Fig. 18, the panoramic mirror profile in accordance with the present invention is not a simple convex, such as parabolic or hyperbolic surfaces. Rather, the panoramic mirror has the inner rim rolled in toward the center, namely, the rotational symmetry axis as shown in Fig. 18. Shown in Fig. 19 is the profile of the panoramic mirror fitted with the least square error approximation to a 7 order power series in p. The 7 order polynomial is given as equation 26.

[194] MathFigure 26 /?(p)=0.0004 p ⁷-0.003 ⁶+0.0128 p ⁵-0.0233 p ⁴-0.0478 p +0.5662 p ²- 1.8647p+2.0663

[195] Here, h is the vertical distance to the mirror surface measured from the lowest point on the mirror (i.e., h(p)=z(p)-min(z)).

[196] Fig. 20 shows a schematic diagram of a panoramic imaging system adopting the panoramic mirror shown in Fig. 17. According to the first embodiment of the present invention, a panoramic imaging system 900 can be installed on a high place such as a ceiling, the roof of a building, a flag stand and a fence. The surface of the panoramic mirror faces the ground and a small CCD camera 904 faces the surface of the mirror. In the case the body of the small CCD camera is smaller than 5 cm in diameter, the incident ray having a zenith angle 180° is not blocked by the CCD camera 904. The incident ray is reflected on the panoramic mirror 902 and the reflected ray is captured by the CCD camera. Therefore, images of all regions below the horizon can be captured with the panoramic imaging system, except for a region having a cylindrical shape of 5cm in diameter, that is, a region directly below the camera. Further, disregarding the sky directly above the camera, images of virtually all regions above the horizon can be captured with the panoramic imaging system. Because the vertical field of view extends 45° above the horizon, images of an object can be effectively captured even if the panoramic mirror 902 is installed at an eye level unless the object is very close to the imaging system 900.

[197] Fig. 21 shows the ranges of the zenith angle δ of the incident ray (from δ to δ ), the zenith angle θ of the reflected ray (from θ to θ ) and the functional relation between the two variables (i.e., δ=δ(θ)). As shown in Fig. 21, the panoramic imaging system 900 in Fig. 20 provides an ideal equi-angular projection scheme as described by equation 21.

[198] In the case where the region of major interest is directly below the camera of the panoramic imaging system 900 and an image thereof is difficult to be taken, the panoramic mirror shown in Fig. 22, which is in accordance with the second embodiment of the present invention, is available. Variables θ and p of Fig. 22 are equal to θ and p , respectively, "θ " is a lower bound of the indefinite integral given in equation 19, p is 2.5cm, the minimum zenith angle θ of the reflected ray is 10°, the maximum zenith angle θ of the reflected ray is 20°, and δ and δ are 60° and 180°, respectively. The panoramic mirror 1000 shown in Fig. 22 differs from the panoramic mirror 902 of Fig. 20 in that δ is larger than δ , and the maximum radius p of the panoramic mirror 1000 is set to 2.5cm instead of setting the minimum radius p to 2.5 cm. For convenience of nomenclature, panoramic mirrors 902 and 1000 in Figs. 20 and 22 can be referred to as a normal type mirror and an inverting type mirror, respectively. [199] Fig. 23 shows a schematic diagram of a panoramic imaging system adopting the panoramic mirror 1000 shown in Fig. 22. As shown in Fig. 23, all incident rays including the incident ray having the zenith angle 180° are not blocked by the camera 1002 and captured on the image sensor 1004 of the camera 1002 as long as the radius of the largest part of the inverting type panoramic imaging system 1010 is smaller than p (see Fig. 15). Therefore, the size of the panoramic mirror 1000 can be reduced in the imaging system 1010 adopting the inverting type mirror. The inverting type panoramic imaging system 1010 provides another advantage in that the region directly below the camera 1002 having the zenith angle δ =180°corresponds to the maximum zenith angle θ of the incident ray, and consequently the image of the region can be captured with the largest number of pixels. Therefore, additional advantage of taking an image of a region of interest with a higher resolution is provided through the use of the inverting type panoramic imaging system 1010.

[200] Fig. 24 shows the functional relations 1006 and 1008 between the zenith angle δ of the incident ray and the zenith angle θ of the reflected ray for the normal (1006) and the inverting (1008) type panoramic imaging systems 900 and 1010 respectively given in Figs. 20 and 23. For convenience of comparison, δ for the inverting type panoramic mirror has been replaced by 45°. When using the inverting type panoramic mirror, some of the incident rays can be blocked by the outer rim of the mirror. Therefore, it is proper that δ is larger than 60°. From Fig. 24, it is apparent that both the normal and the inverting type panoramic imaging systems 900 and 1010 given in Figs. 20 and 23, respectively, provide ideal equi-angular projections scheme described by equation 21.

[201] Fig. 25 shows regions (shaded area) that can be covered by the inverting type panoramic imaging system 1010 in Fig. 23 in accordance with the present invention. Similar to the panoramic imaging system 300 shown in Fig. 4, the inverting type panoramic imaging system 1010 is assumed to have a shape of a street lamp with a view of using the imaging system for monitoring outside areas. As shown in Fig. 25, the region 1012 that can be covered by the inverting type panoramic imaging system 1010 in Fig. 23 are extended all the way to the region having a zenith angle δ of 180°. Therefore, all the regions below the horizon can be monitored by the imaging system without any dead zone, and the whole size of the imaging system can be made smaller.

[202] Fig. 26 illustrates another useful concept of projection scheme, that is, an equi- distance projection scheme adopted by the present invention. A mirror providing the equi-distance projection is one where the ratio between the differential increment of the zenith angle of the incident ray and the differential increment of the pixel distance on the image sensor is a constant. Namely, all pixels on the image sensor correspond to an equal amount of angular extension of the incident ray.

[203] The equi-distance projection scheme is considered as an ideal projection scheme to an imaging system having a flat image sensor such as a film or a CCD sensor. This is because all the objects are captured on the image sensor with the same resolution. As such, various designs are studied to embody the ideal equi-distance projection scheme for fish-eye lenses. Further, when using the equi-distance projection scheme, the zenith angle δ of the incident ray from an object can be calculated by measuring the distance (i.e., the number of the pixels) from the center of the image sensor to the particular pixels corresponding to the object. Therefore, the equi-distance projection scheme can be effectively applied to an imaging system where a rapid acquisition of an object angular coordinates is important, for example, for aerial surveillance of enemy planes, or for a panoramic rangefinder on a moving platform. When other projection scheme is employed, it is not easy to rapid extract the object angular coordinates. This is because the angular coordinates can only be obtained by calculating a complex function of the pixel coordinates or by using a look-up table. Furthermore, panoramic imaging system implicitly assumes a post-image processing after the image acquisition. Therefore, by employing the equi-distance projection scheme that allows an easy conversion of the pixel coordinates into the angular coordinates, the amount of required system resources, such as the memory, can be minimized. The image processing speed is especially important in a system that should operate in real time such as in a surveillance system. In imaging systems with the equi-distance projection scheme, the distance from the center of the image sensor to the particular pixel corresponding to the object point is proportional to the tangent of the zenith angle θ of the reflected ray. Thus, the panoramic mirror providing the equi-distance projection should satisfy the condition given in equation 27. [204] MathFigure 27 _<itanθ X/X ^{≡ β}

[205] Therefore, "tanθ" should be a linear function of the zenith angle δ as given in equation 28. [206] MathFigure 28 tanθ = βδXψ

[207] The constants β and Ψ in equation 28 can be determined as equations 29 to 32, after the ranges of the zenith angles of the incident and the reflected rays corresponding thereto are obtained.

[208] MathFigure 29 tan Θ ^ β δ i +Ψ [209] MathFigure 30 tan Θ ₂= β δ ₂+Ψ [210] MathFigure 31

[211] MathFigure 32 δ ₂tan 0 _j- δ _x tan θ ₂ Ψ= δ ₂- δ _x

[212] Further, as shown in the following equation 33, the zenith angle δ of the incident ray can be given in terms of the zenith angle θ of the reflected ray using the equation 28.

[213] MathFigure 33 tanθ-ψ β

[214] Finally, inclination angle Φ of the tangent plane to the mirror can be given as in equation 34 by using equation 16. [215] MathFigure 34 _ tanθ-ψ+βθ ^Φ 2

[216] The profile of the panoramic mirror providing the equi-distance projection can be obtained by integrating equation 19 in conjunction with the equation 34.

[217] Fig. 27 shows the profile of the panoramic mirror with the equi-distance projection scheme, which is in accordance with a third embodiment of the present invention, with identical ranges of the zenith angles of the incident and the reflected rays as the first embodiment of the present invention given in Fig. 17. The profile of the mirror has been fitted to a 8 order power series in p using the least square error method. The 8 order polynomial is expressed as equation 35.

[218] MathFigure 35 /z(p)=0.0001 p ⁸-0.0006 p ⁷+0.0034 p ⁶-0.0138 p ⁵+0.0499 p ⁴-0.1707 p +0.6538 p ²- 1.8027p+ 1.9520 [219] Fig. 28 shows the difference in profiles between the panoramic mirror 1202 in Fig. 27 providing the equi-distance projections scheme and the panoramic mirror 1201 in Fig. 17 providing the equi-angular projection scheme. As is clear from Fig. 28, the difference in profiles between the two mirrors is not significant. Needless to say, if the maximum zenith angle θ of the reflected ray is increased, the difference in profiles between the two mirrors will be larger. The same tendency can be found in the inverting type panoramic mirror 1000 of the second embodiment of the present invention described in Figs. 22 and 23. [220] Fig. 29 shows the functional relation between the zenith angle δ of the incident ray and the distance from the center of the image sensor to the particular pixel corresponding to the object point for the panoramic mirror given in Fig. 27. From Fig. 29, it can be seen that the panoramic mirror implements the ideal equi-distance projection scheme described by equation 28.

[221] As has been stated, a panoramic stereo imaging system can be used for measuring the distance to an invader in the field of security /surveillance, or for creating a map with a mobile robot, for self-navigational aid of an automobile or a helicopter, and for preventing the mobile robot from colliding with obstacles. The prior implementation of panoramic stereovision system shown in Fig. 9 through Fig. 13 had the same drawback of limited field of view and the dead zone near the back of the camera. These drawbacks can be resolved by using a panoramic mirror in accordance with the present invention.

[222] Fig. 30 shows a schematic diagram of an panoramic stereo imaging system comprising double panoramic mirrors with the equi-distance projection scheme in accordance with a fourth embodiment of the present invention. Equi-distance projection panoramic stereo imaging system 1400 adopts the double panoramic mirrors including an inner panoramic mirror 1404 having the inverting type and an outer panoramic mirror 1402 having a normal type. The ranges of the zenith angles of the inner panoramic mirror 1404 and that of the outer panoramic mirror 1402 are identical. Namely, zenith angles δ and δ of the incident ray for the inner panoramic mirror 1404 are 60° and 180°, respectively, and zenith angles δ and δ of the incident ray for the outer panoramic mirror 1402 are 180° and 60°, respectively. Subscripts "I" and "O" denote inner and outer, respectively. The limiting values of the zenith angles θ and θ of the reflected rays for the inner panoramic mirror 1404 are 10° and 15°, respectively, and the limiting values of the zenith angles θ and θ of the reflected rays for the outer panoramic mirror 1402 are 15° and 20°, respectively. When the inner panoramic mirror has the inverting type and the outer panoramic mirror has the normal type as shown in Fig. 30, the edges of the two panoramic mirrors are connected smoothly. This is advantageous in the viewpoint of fabrication and maintenance.

[223] Fig. 31 illustrates the concept of taking a stereo image through the use of the panoramic stereo imaging system 1400 shown in Fig. 30. As shown in Fig. 31, incident rays from point P or P of an object are reflected on the double panoramic mirror, and dual images are formed on the image sensor (not shown). For instance, two incident rays, L and L from one object point P , are reflected on the outer and the inner panoramic mirrors with zenith angles θ and θ , respectively, and are subsequently captured on the image sensor (not shown). By obtaining dual images in this manner, a distance from the imaging system to the object or the three dimensional shape of the object can be measured.

[224] Shown in Fig σ. 32 is the functional relation between the zenith ang σles δ . and δ ₀ of the incident rays and the tangent of the zenith angles θ and θ of the reflected rays in the equi-distance projection type panoramic imaging system employing the dual panoramic mirrors 1402 and 1404 shown in Fig. 30. More specifically, the horizontal axis corresponds to the distance from the center of the image sensor to each pixel divided by the focal length f of the camera lens. As illustrated in Fig. 32, the inverted panoramic mirror 1404 at the inner side and the normal panoramic mirror 1402 at the outer side accurately implement the equi-distance projection scheme as defined by equation 28. Referring to Figs. 31 and 32, the zenith angles δ and δ of the incident rays are obtained from the pair of pixel coordinates (f-tanθ , f-tanθ ) corresponding to the object point P that has been captured twice by the image sensor. The zenith angles δ and δ respectively correspond to the zenith angle θ of the reflected ray from the inner panoramic mirror 1404 and the zenith angle θ of the reflected ray from the outer panoramic mirror 1402. The distance to the object point P can be determined from the two zenith angles δ and δ and the the profile of the mirror r(θ).

[225] Fig. 33 shows a conceptual drawing of a panoramic stereovision system according to a fifth embodiment of the present invention. A panoramic stereovision system 1500 according to the fifth embodiment consists of two inverted equi-distance projection type panoramic mirrors 1502 and 1504 having the same ranges of the zenith angles of the incident and the reflected rays as the mirror in the fourth embodiment.

[226] Fig. 34 shows a conceptual drawing of a panoramic stereovision system in accordance with a sixth embodiment of the present invention. Similar to the panoramic stereovision system 1500 employing two inverted type equi-distance projection panoramic mirrors 1502 and 1504 of the fifth embodiment, a panoramic stereovision system 1600 also employs two normal type equi-distance projection panoramic mirrors 1602 and 1604. Although the system 1600 employing two panoramic mirrors 1602 and 1604 is not a system with a single viewpoint, it shows lesser distortion for an object in a close range compared to those employing two inverted panoramic mirrors. This is because it shows relatively small variations in the viewpoints as the zenith angles δ and δ of incident rays vary. Therefore, it is more advantageous to use two normal type panoramic mirrors in order to observe objects in close ranges. As shown in Fig. 34, two normal type panoramic mirrors 1602 and 1604 are employed to monitor areas with the corresponding zenith angles of the incident rays ranging from 45° to 180° (i.e., δ =δ 01 =180°, δ 12 =δ 02 =45°) '. The rang oe of the zenith ang ole of the reflected ray ^J s is identical to those of the panoramic stereovision systems in Figs. 30 and 33. However, if the two normal type panoramic mirrors 1602 and 1604 are adjacent to each other similar to those in Figs. 30 and 33, the effective field of view of the two mirrors will be narrower than it was originally designed for. The reason is that one mirror partly occludes the view of the other mirror. To resolve this problem, outer mirror 1602 is moved away in a radial direction as illustrated in Fig. 34. Accordingly, the double panoramic mirrors in Fig. 30 and Fig. 33 satisfy an equality given as r(θ )=r(θ ), while that of Fig. 34 satisfy an inequality given as r(θ )<r(θ ).

[227] Disregarding the several disadvantages including the larger size and the manufacturing difficulty, the panoramic stereovision system 1600 employing two normal type panoramic mirrors 1602 and 1604 has an additional advantage of enhanced distance resolution due to an effectively wider separation between the two panoramic mirrors 1602 and 1604. Using trigonometry, the distance resolution of an object far away from the stereovision system is proportional to the distance between the nodal points of two cameras or the viewpoints of two panoramic mirrors. Of course, the distance resolution of the panoramic stereovision systems of Fig. 30 and Fig. 33 can be increased using the same principle.

[228] Fig. 35 shows a conceptual drawing of a combination of mirrors used for the panoramic imaging system in accordance with the present invention, in which the system folds optical path by using a planar ring mirror. As illustrated in Fig. 35, the combined mirror 1700 consists of a planar ring mirror 1702 and a curved mirror 1706. Mirrors 1702 and 1706 are fixed to each other by a suitable means such as posts 1704 or a transparent acrylic cylinder, which is not illustrated in the drawings. The rotational symmetry axis of the combined mirror 1700 is aligned so that it is coincident with the optical axis of the camera and the combined mirror 1700 is fixed at a predetermined distance from the nodal point of the camera lens. For this purpose, various well- established camera technologies can be adopted. The curved mirror 1706 in Fig. 35 can be any one among the panoramic mirrors shown in the first to the fifth embodiments of the present invention, or more generally a panoramic mirror described by equation 19. The planar ring mirror 1702 has inner and outer hoops in a concentric circle, and has the mirror surface therebetween. Inside the inner hoop of the planar ring mirror 1702 exists a center hole or a part of the mirror that is coated in black paint to absorb lights. If necessary, a positive, a negative or a group of lenses having an appropriate power may be placed near the center hole of the planar ring mirror in order to convert the effective field of view extended by the center hole. This lens or a group of lenses are generally called as a converter. For instance, a group of lenses having a negative power that enlarges the effective field of view of the camera is called a wide-angle converter.

[229] The most important aspect of the panoramic imaging system with the folded design is to alter its major line of sight from the rear to the front of the camera. This can be very useful when the system needs to be installed on the ceiling. That is, the imaging system with the folded design vertically looks down from the ceiling to the floor and accordingly the camera and its peripheral devices can be buried inside the ceiling minimizing the protrusion from the ceiling. Therefore it will look better in appearance and easier in maintenance. Furthermore, it can be also advantageous in an anti-aircraft system as well as in the field of stellar astronomy where the imaging system is set on the ground to monitor the sky. [230] Fig. 36 shows a schematic diagram illustrating the location of the new nodal point N' of the camera and the location and the minimum sizes (p , p ) of the planar ring mirror 1802 in the combined mirror 1800 with the folded design. The combined mirror 1800 is similar in construction to that given in Fig. 35. To clearly illustrate the relations among the relevant variables, it is assumed that the minimum zenith angle θ of an equi-distance projection curved (i.e., panoramic) mirror 1804 in Fig. 36 is 30° and the maximum zenith angle θ is 40°, while δ and δ are 135° and 45° respectively. In such a panoramic imaging system, it is preferable that the height z of the planar ring mirror 1802 is lower than the height z at which all the reflected rays fall within a circle defined by the minimum radius p of the curved mirror 1804, and set the size of the planar ring mirror 1802 large enough so that all the reflected rays to be reflected again on the planar mirror at the height z . As schematically shown in Fig. 36, a planar ring mirror with the inner hoop having a radius p and the outer hoop having a radius p can be a reasonable choice for the combined mirror, and the height z from the o o original nodal point N to the mirror surface of the planar mirror 1802 is identical to the height from the mirror surface of the planar mirror 1802 to the new nodal point N'. [231] Fig. 37 illustrates the procedure for determining the maximum height of the planar ring mirror 1802 in the folded panoramic mirror 1800. As schematically shown in Fig. 37, if the zenith angle δ of the curved mirror 1804 is smaller than π, then the planar mirror 1802 need not reside within the circle having the minimum radius p of the curved mirror in order to reflect all the reflected rays in a way the reflected rays are not occluded by the curved mirror 1804. Therefore, the height z can be larger than the height z and the radius p of the outer hoop of the planar mirror becomes larger than c O the minimum radius p of the curved mirror. 1

[232] Since the most important advantage of the folded panoramic imaging system is to move the major line of sight from the rear to the front of the camera, it is desirable to set the zenith angle δ in order to get images of the forward area of the camera. Fig. 38 shows the folded panoramic imaging system with the equi-distance projection scheme. In the folded panoramic imaging system in Fig. 38, the minimum zenith angle θ of the reflected ray and the maximum zenith angle θ of the reflected ray are set to 10° and 20°, respectively. Further, the corresponding zenith angles δ and δ of the incident rays are set to 180° and 90°, respectively. As such, the minimum height z of the planar mirror 1902 measured from the nodal point N of the camera can be given as equation 36 below when the zenith ang ^σle δ 1 is 180°.

[233] MathFigure 36

[234] In addition to this, the new coordinate Z(p) of the curved mirror 1904 in the folded design is given as a simple function of the old coordinate z(p) of the curved mirror in the non-folded design.

[235] MathFigure 37 Z(p)-z₀-(z(p)-z₀)-2 z₀-z(p)

[236] Further, the radius p of the inner hoop of the planar mirror 1902 is described as equation 38 provided below. [237] MathFigure 38 p ₇— z ₀tan θ _λ

[238] The radius p of the outer hoop of the planar mirror 1902 is described as equation 39 and is identical to the minimum radius p 1 of the curved mirror 1904.

[239] MathFigure 39 p ₀- z₀tan θ ₂ ^:

[240] With the panoramic imaging system comprising the planar mirror 1902 and the curved mirror 1904, images of a wide area can be obtained except for a cylindrical region in the front of the camera having a radius p .

[241] Fig. 39 shows a conceptual drawing of the panoramic imaging system further comprising blinds B , B and B in order to block harmful rays which degrades the quality of the image. The blind B shields harmful rays intruding through the openings between the inner hoop of the curved mirror 1904 and the lens barrel 1906. The blind B shields harmful rays entering the refractive lens with a zenith angle larger than θ . The rays blocked by the blind B have not been previously reflected on the planar mirror 1902 and hence does not form a panoramic image. The blind B can be connected to the thread in the lens barrel prepared for mounting optical filters. The blind B shields harmful rays intruding through the center hole of the planar mirror 1902. The blind B need not be physically separate from the planar mirror. It can be the center of the planar mirror painted in black. Instead, if the planar mirror has the center hole, then it is possible to secure an additional view through the center hole.

[242] Fig. 40 shows a conceptual drawing of a folded panoramic imaging system in accordance with another embodiment of the present invention. Image of a very wide area can be obtained with the folded panoramic imaging system, where the cor- responding range of the incident ray angle is from 0° to 150° with respect to the optical axis of the camera. As shown in Fig. 40, the folded panoramic imaging system comprises the curved mirror 1904, the planar mirror 1902 and at least one refractive lens 1912 located near the center of the planar mirror 1902. The lens 1912 can be a wide-angle converter having a negative refractive power for enlarging the field of view extended by the center hole in the planar mirror or a tele-converter for obtaining a finer image of an object at a long distance. For example, if a wide-angle converter is located near the center of the planar mirror 1902 instead of the blind B , a FOV larger than 2Θ ₃ 1 corresponding to the radius of the inner hoop of the planar mirror 1902 can be secured.

[243] As described above, by actively using the center hole of the planar mirror 1902, a perspectively normal image can be obtained as if the image were captured with a conventional camera, as well as the panoramic image with the corresponding range of the reflected ray angle from θ to θ .

[244] In a like manner, the optical path can be folded for the case of an inverting type panoramic mirror or a double panoramic mirror of the stereo imaging system. Furthermore, the method of folding the optical path is independent from the particular projection scheme adopted by the panoramic mirror. In other words, the same principle can be used to fold the optical path of panoramic mirrors with various projection schemes including the equi-angular projection scheme.

[245] Fig. 41 shows a schematic diagram of a desirable image sensor of a camera suitable for panoramic imaging and panoramic stereo imaging in accordance with the present invention. As schematically shown in Fig. 41, a regular square shaped image sensor is most effective for panoramic imaging. The diameter of the outer circle OC of the ring shaped panoramic image is identical to the width of the regular square shaped image sensor. A perspectively normal image is obtained within the inner circle IC on the image sensor with the rays that pass through the center hole of the planar mirror or the lens located near the center hole. Then a rectangular portion of the image M having the same aspect ratio as a normal TV screen can be extracted from the central image within the inner circle IC, and then displaced on a usual video monitor. This kind of imaging system can be particularly useful for motion tracking.

[246] Figs. 42 and 43 show a conceptual drawing of a motion tracking imaging system comprising a camera with the folded panoramic design and a camera base capable of providing pan-tilt motion in accordance with the present invention. The panoramic camera 2105 is mounted on a first base 2106. The camera 2105 can perform tilt motion with respect to the first base 2106. The first base 2106 is connected to a second base 2107 with a rotational axis. The first base 2106, on which the camera 2105 is mounted, can perform pan motion with respect to the second base 2107. The pan-tilt motion mechanism shown in Figs. 42 and 43 are merely an exemplary embodiment of the present invention and various other pan-tilt motion mechanisms can be provided for the same purpose.

[247] The motion tracking imaging system 2100 adopts the combined mirror shown in Figs. 38 to 40. The double panoramic mirrors shown in Figs. 30 to 34 can be also adopted. By using the motion tracking imaging system 2100 employing a folded panoramic lens having a large FOV, most of the directions can be simultaneously monitored except for a cone shaped region behind the camera 2105. More specifically, the motion of an object can be easily monitored by comparing a sequence of time-lapse images. For example, subtracting the current image from the previous image obtained a short time ago, only the part of the image that has been changed since then can be easily isolated. Also, the direction can be easily computed by simply reading the coordinates of the pixels corresponding to the object. Using this information, the camera can be controlled to follow the moving object and detailed and perspectively normal images of the moving object can be obtained within the inner circle in the image sensor. Furthermore, due to the wide FOV of the panoramic mirror, most of the directions can be simultaneously monitored even when the camera is pointing in one direction.

[248] This motion tracking imaging system 2100 can be used in various fields. For instance, if the motion tracking imaging system 2100 is installed on a ceiling and looking down on the floor, the whole room can be monitored by analyzing the ring shaped panoramic images. Further, in case a motion of an invader is detected, the camera automatically heads toward the invader and a detailed image of the invader can be captured.

[249] This motion tracking imaging system 2100 can be alternatively used in nature photography and video documentary such as in taking images of birds and beasts. Taking a wild animal photography or video usually means lots and lots of waiting time waiting for the subject to appear. Sometimes it can be also very dangerous for the cameraman if the subjects are carnivorous animals such as tigers and bears. However, if the motion tracking imaging system is used for this purpose, the cameraman needs not be present at the site all the time. Once installed at a proper site, the imaging system will track the subject by its own. Therefore it can provide a safer and a cheaper way of producing nature photography/video. In the field of security /surveillance or nature documentary, having a motion tracking imaging system equipped with infrared CCD sensor can be advantageous. Furthermore, the motion tracking imaging system 2100 can be used in an automatic gunfire system in the military. Namely, the motion tracking imaging system 2100 can provide the coordinates of suddenly appearing highspeed enemy airplanes and missiles in real time, thus enabling us to cope with the situations appropriately. [250] Moreover, the panoramic imaging system in accordance with the present invention can be used as a PC camera, a web camera, or a network camera. The PC camera, such as the QuickCam Sphere from Logitech Corporation provides pan-tile motion with an automatic face tracking algorithm. Therefore, such a PC camera (e.g., QuickCam Sphere) can provide images of a person moving around the room always in the center of the captured image. However, the QuickCam Sphere can automatically track a person's face only after the person has been captured by the PC camera that is standing still. This is mainly because the horizontal and the vertical field of view are not wide enough to cover the entire room. However, the imaging system for automatically tracking a motion or a face in accordance with the present invention can automatically chase the target wherever the target might be in the room.

[251] Although a potential intruder is detected, the intruder can hardly be called an intruder if he or she is far away, or at least he or she is not as threatening as might have been if he/she were nearby. Since the field of view of the panoramic imaging system of the current invention is extremely large, the motion tracking imaging system might vainly track an object that is very far away. For example, if it were to take the images of wile birds, the motion tracking should not be triggered by an airplane flying high in the sky. The stereo imaging systems shown in Figs. 30, 33 and 34 can selectively track the target by calculating the distance when a motion is detected and thus operational errors can be prevented.

[252] In the foregoing descriptions of the invention, all the examples were implicitly based on the panoramic mirrors for visible light. However, unlike the refractive optical elements, the profile of the panoramic mirror does not change as the wavelength of light changes. In other words, the profile of the panoramic mirror is identical whatever the wavelength of light might be including millimeter waves, microwaves, as well as visible, near-infrared, far-infrared, ultraviolet and soft X-rays. Therefore, the same equation can be used to design various panoramic mirrors regardless of the wavelength of light, even though it may be necessary to use a different design for the refractive optical elements and a different coating material. Therefore, the present invention is by no means limited to imaging systems for the visible light.

[253] Furthermore, although CCD sensor is most widely used in the field of digital photography, the argument of the current invention is irrespective of the image sensor type. Therefore other types of sensors, such as CMOS sensor, image intensifier, roll film, sheet film, instant film, X-ray film, etc., can be used without changing any of the major arguments of the current invention. Industrial Applicability

[254] As mentioned above, images of a wide region including the region directly behind the camera can be captured with the panoramic imaging system in accordance with the present invention. The ratio between the differential increment of the angle of incident ray and the differential increment of the angle of reflected ray or the differential increment of the tangent of reflected ray angle can be maintained as a constant. Therefore, this system is particularly useful in simultaneously monitoring a wide region. The present invention may be embodied in other specific forms without departing from its essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims, rather than by the foregoing description. All changes, which come within the equivalent meaning and range of the claims, are to be embraced within their scope.

Claims

[1] A panoramic mirror, comprising: a mirror surface having a rotationally symmetric profile about the z-axis in a spherical coordinate, wherein the z-axis has zero zenith angle, the profile of the mirror surface is described with a set of coordinate pairs (θ, r(θ)) in the spherical coordinate, θ is a zenith angle of a first reflected ray reflected at a first point on the mirror surface and passing through the origin of the said spherical coordinate, the zenith angle θ ranges from the minimum zenith angle θ to the maximum zenith angle θ (θ ≤θ≤θ ), and r(θ) is a distance from the said origin to the first point and satisfies the following equation 1: Equation 1 r(θ) = r( θ _I-)exρ

_θ cosθ'- cotψ(θ')sinθ' where θ is a zenith angle of a second reflected ray reflected at a second point on the mirror surface and passing through the said origin, the zenith angle θ takes a value between the minimum zenith angle θ and the maximum zenith angle θ (θ 1 2 1 ≤θ ≤θ ), r(θ ) is the distance from the said origin to the said second point, and i 2 l Φ(θ) is a zenith angle of the tangent plane to the mirror surface at the first point and is a function of the zenith angle θ and wherein the first reflected ray is formed by an incident ray having a zenith angle δ, the zenith angle δ is a function of the zenith angle θ, and θ, Φ and δ have a function relation in accordance with the following equation 2: Equation 2

[2] The panoramic mirror of Claim 1, wherein the zenith angle Φ of the tangent plane is a function of the zenith angle θ in accordance with the following equation 3: Equation 3 φ = βθ + Ψ where β and Ψ are constants having function relations with the zenith angles δ and δ , respectively corresponding to the zenith angles θ and θ as the following equations 4 and 5: Equation 4

, and Equation 5 δ ₁ θ ₂- δ ₂ θ ₁ ψ= ( θ ₂- θ _!) and the values of the zenith angles δ 1 and δ 2 can take a value between 0 and 3π/2.

[3] The panoramic mirror of Claim 1, wherein the zenith angle Φ of the tangent plane is a function of the zenith angle θ as the following equation 6: Equation 6 _ tanθ-Ψ+βθ ^Φ 2β where β and Ψ are constants having functional relations with the zenith angles δ and δ respectively corresponding to the zenith angles θ and θ as the following equations 7 and 8: Equation 7 tan θ ₂-tan θ _x β = δ ₂- δ _x Equation 8 δ ₂tan θ _!~ δ ^an θ ₂ ψ- δ ^- δ and the values of the zenith ang ^σles δ 1 and δ 2 can take a value between 0 and 3π/2.

[4] A double panoramic mirror, comprising: a first mirror surface having a rotationally symmetric profile about a z-axis in a spherical coordinate, wherein the z-axis has zero zenith angle, the profile of the first mirror surface is described with a set of coordinate pairs (θ , r (θ )) in the spherical coordinate, θ is a zenith angle of a first reflected ray reflected at a first point on the first mirror surface and passing through the origin of the spherical coordinate, the zenith angle θ ranges from the minimum zenith angle θ to the maximum zenith ang °le θ 12 (θ II ≤θ I≤θ 12 ), and r I (θ I ) is the distance from the orig °in to the first point and satisfies the following equation 9: Equation 9

where θ is a zenith angle of a second reflected ray reflected at a second point on the first mirror surface and passing through the origin, the zenith angle θ takes a Ii value between the minimum zenith angle θ and the maximum zenith angle θ (θ II 12 ≤θ ≤θ ), r (θ ) is a distance from the origin to the second point, and Φ (θ ) is a

II Ii 12 I Ii I I zenith angle of a first tangent plane to the mirror at the first point and is a function of the zenith angle θ , and wherein the first reflected ray is formed by a first incident ray having a zenith ang °le δ I , the zenith ang °le δ I is a function of the zenith ang °le θ I , and θ I , Φ I and δ 1 have a functional relation as the following equation 10: Equation 10

^Φ ' 2

; and a second mirror surface having another rotationally symmetric profile about the said z-axis, wherein the profile of the second mirror surface is described with a set of coordinate pairs (θ , r (θ )) in the spherical coordinate, θ is a zenith angle 0 0 0 o of a third reflected ray reflected at a third point on the second mirror surface and passing through the origin of the spherical coordinate, the zenith angle θ ranges from the minimum zenith angle θ to the maximum zenith angle θ (θ <θ <θ 01 02 01 O ), and r (θ ) is a distance from the origin to the third point and satisfies the

02 O O following equation 11 : Equation 11

where θ is a zenith angle of a fourth reflected ray, the fourth reflected ray

reflected at a fourth point on the second mirror surface and passing through the origin, the zenith angle θ

and the maximum zenith angle θ is a distance from the

origin to the fourth point, Φ (θ ) is a zenith angle of a second tangent plane to the mirror at the third point and is a function of the zenith angle θ , and wherein the third reflected ray is formed by a second incident ray having a zenith angle δ , the zenith angle δ is a function of the zenith angles θ , and θ , Φ and ° o ° o ° 0 0 0 δ have a functional relation as the following equation 12: Equation 12 _ 5o+ θ_σ 2 and wherein a distance r (θ ) from the origin to the first point having the maximum zenith angle θ and a distance r (θ ) from the origin to the third point 12 O 01 having the minimum zenith angle θ are the same (r (θ )= r (θ )). 01 I 12 O 01

[5] The double panoramic mirror of Claim 4, wherein the zenith angle Φ of the first tangent plane is a function of the zenith angle θ as the following equation 13: Equation 13

where β and Ψ are constants having functional relations with the zenith angles δ and δ respectively corresponding to the zenith angles θ and θ as the following equations 14 and 15: Equation 14

, and Equation 15 ^δ/l ^θ/2^" δ/2 ^θ/l Ψ 2( θ/₂" θ/i) and the zenith angeles δ and δ takes a value between 0 and 3π/2, respectively, and wherein the zenith angle Φ of the second tangent plane is a function of the zenith angle θ as the following equation 16: Equation 16 Φ o= β ₀ θ₀+ Ψ_σ where β ¹ o and Ψ o are constants having ° functional relations with the zenith ang °les δδ 0011 aanndd δδ 0022 rreess ^rrppeeccttiivveelly ^Jy ^J ccoorrrreess ^rrpponding ° to the zenith ang °les θ 01 and θ 02 as the following equations 17 and 18: Equation 17

, and Equation 18 ψ = δ <31 θ θ2^" ^ c?2 θ < 1 2( 6 ^₂" θ ₀₁) and the zenith angles δ and δ takes a value between 0 and 3π/2. 01 02

[6] The double panoramic mirror of Claim 4, wherein the zenith angle Φ of the first tangent plane is a fuction of the zenith angle θ as the following equation 19: Equation 19

where β and Ψ are constants having functional relations with the zenith angles δ and δ respectively corresponding to the zenith angles θ and θ as the following equations 20 and 21: Equation 20 tan θ _/2- tan θ ₇₁ ^{P =} R , and Equation 21

and the zenith ang ^σles δ II and δ 12 takes a value between 0 and 3π/2, and wherein the zenith angle Φ of the second tangent plane is a function of the zenith angle θ as the following equation 12: Equation 22

where β and Ψ are constants having functional relations with the zenith angles o o δ 01 and δ 02 res ^rpectively ^J corres ^rponding ° to the zenith ang °les θ 01 and θ 02 as the following equations 23 and 24: Equation 23 tan θ ^-tan θ _0\

, and Equation 24 _ δ^tan θ^- δ^tan θ^ ° O2^" ° O1 and the zenith ang °les δ 01 and δ 02 takes a value between 0 and 3π/2.

[7] The double panoramic mirror of Claim 6, wherein the zenith angle δ takes a value between 0 and π/2, the zenith ang ^σle δ 12 is larg ^σer than δ II (δ II <δ 12 ), the zenith ang °les δ 01 and δ 12 have the same value, and the zenith ang °les δ 02 and δ II have the same value.

[8] A folded panoramic mirror, comprising: a first mirror including a curved mirror surface having a rotationally symmetric profile, wherein the curved mirror surface extends from a first inner hoop having a radius p to a first outer hoop having a radius p , and the first mirror has a circular hole inside of the inner hoop; and a second mirror including a plane mirror surface having a ring shape, wherein the plane mirror surface faces the curved mirror surface and the dimension of the planar mirror is determined by a second inner hoop having a radius p and a second outer hoop having a radius p wherein, the first mirror and the second mirror share the same rotational symmetry axis, the curved mirror surface is described with a set of spherical coordinate pairs (θ, r(θ)) where the z axis (i.e, the axis having zenith angle θ=0) of the spherical coordinate system coinciding with the rotational symmetry axis of the mirror, θ is a zenith angle of a first reflected ray formed by an incident ray having a zenith angle δ at a first point on the curved mirror surface, the first reflected ray passes through the origin of the spherical coordinate, the zenith angle θ ranges from the minimum zenith angle θ to the maximum zenith angle θ (θ ≤θ≤θ ), and r(θ) is a distance from the origin to the first point and satisfies the following equation 25: Equation 25

where θ is a zenith angle of a second reflected ray reflected at a second point on the curved mirror surface and passing through the origin, the zenith angle θ takes a value between the minimum zenith angle θ and the maximum zenith angle θ (θ ≤θ ≤θ ), r(θ ) is a distance from the origin to the second point, and Φ(θ) is a 1 i 2 l zenith angle of a tangent plane to the mirror at the first point and is a function of the zenith angle θ and satisfies the following equation 26: Equation 26 θ+δ(θ) Φ(θ) where δ(θ) is a zenith angle of the incident ray and is a function of the zenith angle θ and the zenith angle δ(θ) of the incident ray ranges from the minimum zenith angle δ larger than zero to the maximum zenith angle δ equal to π(0<δ ≤δ≤δ =π), and wherein the radius p of the first inner hoop is given as equation 27: Equation 27 P ι = p ( θ ₁)=r( θ ₁ )sin θ ₁ and wherein the radius p of the first outer hoop is given as equation 28: Equation 28 p ₂= θ ₂) sin θ ₂ and wherein the radius p of the second inner hoop is given as equation 29: Equation 29 p _j= z o tan θ _x where z is a distance from the origin to the plane mirror surface and satisfies the o following equation (30): Equation 30 z ₀= _{P l} cot θ ₂ and wherein the radius p of the second outer hoop is identical to the radius p of the first inner hoop (p = p ).

[9] The folded panoramic mirror of Claim 8, wherein the zenith angle Φ(θ) of the tangent plane is a function of the zenith angle θ as the following equation 31 : Equation 31 _ tanθ-Ψ+βθ

where β and Ψ are constants having functional relations with the maximum zenith angle δ and the minimum zenith angle δ as the following equations 32 and 33: Equation 32 tan θ ₂-tan θ _x β = δ ₂- δ _x , and Equation 33 δ ₂tan θ _x- δ _x tan θ ₂ Ψ= δ ₂- δ _x

[10] A panoramic imaging system, comprising: a panoramic mirror including a curved mirror surface having a rotationally symmetric profile, wherein the curved mirror surface extends from an inner hoop having a radius p to an outer hoop having a radius p ; and a camera facing the curved mirror surface, whereby the curved mirror surface is within the field of view of the camera, wherein the profile of the curved mirror surface is described with a set of coordinate pairs (θ, r(θ)) in a spherical coordinate having the rotational symmetry axis as z-axis having zero zenith angle, θ is a zenith angle of a first reflected ray formed by an incident ray having a zenith angle δ at a first point on the curved mirror surface, the first reflected ray passes through the origin of the spherical coordinate, the zenith angle θ ranges from the minimum zenith angle θ to the maximum zenith angle θ (θ ≤θ≤θ ); and r(θ) is a distance from the origin to the first point and satisfies the following equation 34: Equation 34

where θ is a zenith angle of a second reflected ray reflected at a second point on the curved mirror surface and passing through the origin, the zenith angle θ takes a value between the minimum zenith ang ^σle θ 1 and the maximum zenith ang ^σle θ 2 (θ 1 ≤θ i ≤θ 2 ), r(θ l ) is a distance from the origin to the second point, Φ(θ) is a zenith angle of tangent plane to the mirror at the first point and is a function of a zenith angle θ and θ, Φ and δ have a functional relation as the following equation 35: Equation 35

and wherein an optical axis of the camera coincides with the z-axis of the spherical coordinate, and the nodal point of the camera is located at the origin of the said coordinate.

[11] The panoramic imaging system of Claim 10, wherein the zenith angle Φ of the tangent plane is a function of the zenith angle θ as the following equation 36: Equation 36

where β and Ψ are constants having functional relations with the zenith angles δ and δ respectively corresponding to the zenith angles θ and θ as the following equations 37 and 38: Equation 37 tan θ ₂-tan θ _x β= — 8 ,- δ , and Equation 38 δ ₂tan θ _j- δ _x tan θ ₂ Ψ= δ ₂- δ _x and the zenith ang ^σles δ 1 and δ 2 takes a value between 0 and 3π/2.

[12] The panoramic imaging system of Claim 11, wherein the camera includes: a body having an image sensor selected from a group consisting of roll film, sheet film, instant film, X-ray film, charge-coupled device (CCD) sensor and complementary metal oxide semiconductor (CMOS) sensor; and a lens for focusing electromagnetic waves including visible rays, infrared rays, ultraviolet rays, soft X-rays, microwaves and milimeter waves.

[13] The panoramic imaging system of Claim 11, wherein the camera includes a body and a lens, and wherein the largest cross-section of the body and the lens perpendicular to the optical axis is contained within a circle having a radius p described as the following equation 39: Equation 39 p ι = r( θ ι ) sin θ _λ and wherein the zenith angle δ is π, and the zenith angle δ is smaller than π/2.

[14] The panoramic imaging system of Claim 11, wherein the camera includes a body and a lens, and wherein the largest cross-section of the body and the lens perpendicular to the optical axis is contained within a circle having a radius p described as the following equation 40, Equation 40 _{P 2} ⁼ θ ₂) sin θ ₂ and wherein the zenith angle δ is π and the zenith angle δ is smaller than π/2.

[15] A double panoramic imaging system, comprising: a curved mirror including: a first mirror surface having a rotationally symmetric profile, wherein the first mirror surface extends from a first inner hoop having a radius p to a first outer hoop having a radius p ; and a second mirror surface having another rotationally symmetric profile, wherein the second mirror surface extends from a second inner hoop having a radius p to a second outer hoo ^rp having a radius p 02 , and the radius " p01 of the second inner hoop is not smaller than the radius p of the first outer hoop (p ≤p ); and a camera facing the curved mirror surface, whereby the first and the second mirror surfaces are within the field of view of the camera, wherein the profile of the first mirror surface is described with a set of coordinate pairs (θ , r (θ )) in a spherical coordinate, θ is a zenith angle of a first reflected ray reflected at a first point on the first mirror surface and passing through the origin of the spherical coordinate, the zenith angle θ ranges from the minimum zenith angle θ to the maximum zenith angle θ (θ ≤θ ≤θ ), and r (θ ) is a II 12 II I 12 I I distance from the origin to the first point and satisfies the following equation 41: Equation 41

where θ is a zenith angle of a second reflected ray reflected at a second point on Ii the first mirror surface and passing through the origin, the zenith angle θ takes a Ii value between the minimum zenith ang ^σle θ II and the maximum zenith ang ^σle θ 12 (θ

II ≤θ I ≤θ 12 ), r I (θI ) is a distance from the orig °in to the second ^r point, and Φ I (θ I ) is a zenith angle of a first tangent plane to the mirror at the first point and is a function of the zenith angle θ , and wherein the first reflected ray is formed by a first incident ray having a zenith angle δ , the zenith angle δ is a function of the zenith angle θ , and θ , Φ I I I I I and δ have a functional relation as the following equation 42: Equation 42

and wherein the profile of the second mirror surface is described with a set of coordinate pairs (θ , r (θ )) in the spherical coordinate, θ is a zenith angle of a 0 0 0 o third reflected ray reflected at a third point on the second mirror surface and passing through the origin of the spherical coordinate, the zenith angle θ ranges from the minimum zenith angle θ to the maximum zenith angle θ (θ ≤θ ≤θ ° 01 02 01 O 02

), and r (θ ) is a distance from the origin to the third point and satisfies the following equation 43: Equation 43 ⁱ⁰ sinθ'+cot φ _σ(θ')cosθ'

a cosθ'-cot φ ₀(θ')sinθ'

where θ is a zenith angle of a fourth reflected ray reflected at a fourth point on the second mirror surface and passing through the origin of the spherical cordinate, the zenith ang ^σle θ Oi takes a value between the minimum zenith angle θ and the maximum zenith angle θ (θ ≤θ ≤θ ), r (θ ) is a distance from the

origin to the fourth point, Φ (θ ) is a zenith angle of a second tangent plane to the mirror at the third point and is a function of the zenith angle θ , the third reflected ray is formed by a second incident ray having a zenith angle δ , the zenith angle δ is a function of the zenith angle θ , and θ , Φ and δ have a o o o o o functional relation as the following equation 44: Equation 44

and wherein an optical axis of the camera coincides with the z-axis, and the nodal point of the camera is located at the origin.

[16] The double panoramic imaging system of Claim 15, wherein the zenith angle Φ of the first tangent plane is a function of the zenith angle θ as the following equation 45: Equation 45

where β and Ψ are constants having functional relations with the zenith angles δ and δ respectively corresponding to the zenith angles θ and θ as the following equations 46 and 47: Equation 46

, and Equation 47

and the zenith angles δ and δ takes a value between 0 and 3π/2, and wherein II 12 the zenith ang °le Φ O of the second tang oent r plane is a fuction of the zenith ang σle θ ₀ as the following equation 48: Equation 48 _ tan θ₀- Ψ₀+ β ₀ θ₀ Φ o 2 β o where β and Ψ are constants having functional relations with the zenith angles o o δ 01 and δ 02 res ^rpectively ^J corres ^rponding ° to the zenith ang °les θ 01 and θ 02 as the following equations 49 and 50: Equation 49 tan θ ^- tan θ _σι ^ό O2^{" ό} Ol Equation 50 _ δ^tan θ^- δ^tan θ^ ° O2~ ° O1 and the zenith angles δ and δ takes a value between 0 and 3π/2. II 12

[17] The double panoramic imaging system of Claim 16, wherein the zenith angle δ takes a value between 0 and π/2, the zenith ang ^σle δ 12 is π, the zenith ang ^σles δ 01 and δ 12 have the same value, the zenith ang ^σles δ 02 and δ II have the same values, the first outer hoop of the first mirror surface and the second inner hoop of the second mirror surface have the same radii (r i (θ 12 )=r o (θ or )).

[18] The double panoramic imaging system of Claim 15, wherein the camera includes: a body having an image sensor selected from a group consisting of roll film, sheet film, instant film, X-ray film, charge-coupled device (CCD) sensor and complementray metal oxide semiconductor (CMOS) sensor; and a lens for focusing electromagnetic waves including visible rays, infrared rays, ultraviolet rays, soft X-rays, microwaves and milimeter waves.

[19] A folded panoramic imaging system, comprising: a first mirror including a curved mirror surface having a rotationally symmetric profile, wherein the curved mirror surface extends from a first inner hoop having a radius p to a first outer hoop having a radius p , and the first mirror has a circular hole inside of the inner hoop; a second mirror including a plane mirror surface having a ring shape, wherein the plane mirror surface faces the curved mirror surface and the dimension of the planar mirror is defined with a second inner hoop having a radius p and a second outer hoop having a radius p ; and a camera facing the second mirror, whereby the plane mirror surface is within the field of view of the camera, wherein the first mirror and the second mirror shares the same rotational symmetry axis, the curved mirror surface is described with a set of coordinate pairs (θ, r(θ)) in a spherical coordinate having the rotational symmetry axis as the z-axis having zero zenith angle, θ is a zenith angle of a first reflected ray formed by an incident ray having a zenith angle δ at a first point on the curved mirror surface, the first reflected ray passes through the origin of the spherical coordinate, the zenith angle θ ranges from the minimum zenith angle θ to the maximum zenith angle θ (θ ≤θ≤θ ), and r(θ) is a distance from the origin to the first point and satisfies the following equation 51: Equation 51

where θ is a zenith angle of a second reflected ray reflected at a second point on the curved mirror surface and passing through the origin, the zenith angle θ takes a value between the minimum zenith angle θ and the maximum zenith angle θ (θ ≤θ ≤θ ), r(θ ) is a distance from the origin to the second point, and Φ(θ) is a 1 i 2 l zenith angle of a tangent plane to the mirror at the first point and is a function of the zenith angle θ and satisfies the following equation 52: Equation 52

where δ(θ) is a zenith angle of the incident ray and is a function of the zenith angle θ, and the zenith angle δ(θ) of the incident ray ranges from the minimum zenith angle δ larger than zero to the maximum zenith angle δ equal to π(0<δ ≤δ≤δ =π), and wherein the radius p of the first inner hoop is given as equation 53: Equation 53 p ^pC Θ ^ θ sin θ _! and wherein the radius p of the first outer hoop is given as equation 54: Equation 54 p ₂= θ ₂)sin θ ₂ and wherein the radius p of the second inner hoop is given as equation 55: Equation 55 p _j= z o tan θ _: where z 0 is a distance from the orig oin to the r plane mirror surface and satisfies the following equation 56: Equation 56 z ₀= p ₁ cot θ ₂ and wherein the radius p of the second outer hoop is identical to the radius p of the first inner hoop (p =p ).

[20] The folded panoramic imaging system of Claim 19, wherein the camera includes: a body having an image sensor selected from a group consisting of roll film, sheet film, instant film, X-ray film, charge-coupled device (CCD) sensor and complementary metal oxide semiconductor (CMOS) sensor; and a lens for focusing electromagnetic waves including visible rays, infrared rays, ultraviolet rays, soft X-rays, microwaves and millimeter waves.

[21] The folded panoramic imaging system of Claim 19, wherein the planar mirror has a circular hole having a radius p inside the inner hoop therof , at least one lens is located near the central hole of the planar mirror, a camera having a regular square shaped image sensor, and an optical axis of the camera coincides with the optical axis of the said lens located near the central hole and the rotational symmetry axis of the mirrors and wherein a normal image formed by the said lens and a ring shaped panoramic image formed by the folded panoramic mirror are captured on the image sensor.

[22] The folded panoramic imaging system of Claim 21, wherein the folded panoramic imaging system is mounted on camera base capable of providing pan and tilt motion, the panoramic imaging system follows a moving object by analyzing the ring shape panoramic image and the imaging system further extracts a portion of the normal image having the same aspect ratio as a TV screen.