JP2002218296A - Image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image pickup device capable of reproducing a visually natural form without complicated correcting processing and of outputting the image data of a wide field angle without the fluctuation of resolution caused by position. SOLUTION: This device is composed of a mirror face (11) on which a beam from an object is made incident and an imaging device (included in 12) where the image of that is formed is configured so that an incident angle (θ) on the horizontal plane of the light to the mirror face and one coordinate value (μ) expressing a position on the surface of an imaging device, where the image of that beam is to be formed, can have a linear relation.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus, and more particularly to an image pickup apparatus used for a moving image recording / reproducing apparatus such as a video recording system.

[0002]

2. Description of the Related Art As such an imaging apparatus, for example, an "omnidirectional vision system" described in Japanese Patent No. 02939087, a "panoramic image apparatus" described in Japanese Patent No. "Panoramic monitoring system".

[0003] These all relate to a 360-degree panoramic photographing camera using a hyperbolic dome type mirror.
In these configurations, a 360-degree (hemispheric) scene can be captured by capturing an image reflected by a dome-shaped mirror. If such a camera is installed on a ceiling or the like, the entire room can be observed, and thus the camera is suitable for surveillance and the like. In the above configuration, the captured image has a shape distorted in a circumferential shape.
The "panoramic image apparatus" described in the above publication includes a means for correcting such an image as if it was taken by a normal camera.

However, the data processing for converting a circumferentially distorted image into a plane as described above requires a large amount of calculation, and a high-speed computer is required to process a moving image in real time. Therefore, there is a problem that the scale of the device is increased and the device is expensive. Furthermore, since the magnification changes depending on the position depending on such conversion, there is a problem that the substantial resolution of the converted image differs depending on the position. In addition, for applications other than monitoring, for example, when photographing a panoramic photograph of a landscape, the 3
In many cases, it is not necessary to capture a scene around the entire 60 degrees. One ideal configuration that can be considered for capturing a panoramic image is a perspective transformation imaging system using a cylindrical imaging surface as shown in FIG. A plan view of this configuration is shown in FIG. In this case, the image of the photographing target existing at the point a (the component in the direction perpendicular to the paper surface is arbitrary) is a cylindrical imaging surface centered on O toward the optical center O (thick curve). ) Is projected onto the point b. In the imaging by such an imaging system, there is no particular correction (only the cylindrical imaging surface is developed as it is on the plane), and the localization is much smaller than that of the imaging with the normal planar imaging surface. An image with a wide field of view can be obtained without inferiority. In this case, there is no problem of a change in resolution depending on the position. However, since it is difficult to arrange photodiode elements and the like in a curved shape, it is difficult to realize such an ideal imaging system as it is as a product.

SUMMARY OF THE INVENTION The present invention solves the above-mentioned problems and provides an image pickup apparatus capable of easily realizing an image pickup system having a function equivalent to the structure shown in FIG. With the goal.

As shown in FIG. 2B, the characteristic of an imaging system using a cylindrical surface as described above is that the angle of an object projected on a reference horizontal plane is θ, and that the object is projected on the imaging surface on which the object is projected. When the position at x is x, θ and x are proportional.
In order to realize such a configuration, the invention according to claim 1 includes an optical system on which a light beam from an object is incident and an image pickup device on which the light beam forms an image. Has a linear relationship between the angle of incidence when projected on the flat surface and one coordinate value indicating the position on the imaging element surface where the light beam forms an image.

[0005] In the invention according to claim 2, the optical system comprises an optical system isotropic about the optical axis and a columnar mirror having a curved bottom.

Further, the invention according to claim 3 further comprises means for performing processing having different characteristics depending on directions on data obtained by the image pickup device.

Further, in the invention described in claim 4, the processing having different characteristics depending on the direction comprises a scaling processing having a different magnification depending on the direction.

In the invention described in claim 5, the processing having different characteristics depending on the direction further comprises a smoothing processing.

According to a sixth aspect of the present invention, the image pickup device generates image data having different resolution characteristics depending on directions.

Further, in the invention according to claim 7, the image pickup device has a structure in which light receiving elements having different sizes depending on directions are arranged.

Further, according to the invention described in claim 8, the image pickup device has a configuration in which a color separation filter in a stripe shape is provided.

According to a ninth aspect of the present invention, a half mirror is provided between the curved mirror of the optical system and the image pickup device.

[0013] According to a tenth aspect of the present invention, the curved mirror of the optical system is a mirror having a corner that generates a blind spot, and the image pickup device is disposed within the blind spot.

Therefore, by combining a planar imaging element that is easy to manufacture and an optical system in which the position of the object on the planar imaging surface is proportional to the angle of the object, the configuration shown in FIG. It is possible to realize an imaging system having various functions, to reproduce a natural shape for visual observation without complicated correction processing, and to obtain image data with a wide viewing angle without fluctuation in resolution depending on position. It becomes possible.

Specifically, according to the second aspect of the present invention,
The above-mentioned desired characteristics can be obtained with a simple configuration in which a one-dimensional curved mirror is added to a normal optical system.
Further, according to the third aspect of the present invention, it is possible to perform processing with different characteristics depending on the direction on the imaging data deformed in one direction, thereby obtaining image data having the same vertical and horizontal shape and resolution. Become. According to the fourth aspect of the present invention, it is possible to reproduce a captured image in a natural shape vertically and horizontally. Further, according to the invention described in claim 5, it is possible to make the resolutions of the imaged image data vertically and horizontally equal. According to the sixth, seventh and eighth aspects of the present invention, it is possible to obtain image data having the same vertical and horizontal shapes and resolution.

According to the ninth aspect of the present invention, it is possible to obtain an image without obscuring by a camera (including an image sensor). According to the tenth aspect of the present invention, it is possible to photograph a wide range of scenes except for an area hidden by a camera (including an image sensor).

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First, the basic principle of the present invention will be described.

According to the present invention, when an angle of a subject to be imaged from a certain reference on a horizontal plane is θ and a position on the imaging surface is x,
This is an imaging apparatus having a configuration in which θ and x are proportional to each other, and realizes such a configuration by combining a planar imaging element and a curved mirror. The operation of the present invention will be described below with reference to FIG.

Although the camera used here is provided with a normal image forming optical system, for simplification of description,
Here, a description will be given as a pinhole model.

FIG. 1 shows the entire optical system viewed from above (y-axis).

In the following description, for simplicity, it is assumed that the camera has a wide horizontal viewing angle. While this is generally desirable due to human visual characteristics, it should be understood that rotating the entire device can expand other directions. In the figure, O indicates an optical center, M of a thick line indicates a mirror, and I indicates an imaging surface.

Each of the mirrors M has a linear cross section in the y-axis (perpendicular to the plane of the paper) and is bent only in the xz direction. The imaging plane I is orthogonal to the z-axis at a distance of f from the optical center O.

A light ray reaching a point p at a distance u from the z-axis on the imaging plane is traced in reverse. That is, by extending the line connecting the point p and the optical center O to the point at which the mirror M intersects,
Then, the light ray is reflected symmetrically about the normal qq 'of the mirror M at the point q and reaches the point r. Where the rays qr and z
The angle between the axis and the axis is θ. Here, assuming that the object to be photographed is sufficiently distant, and neglecting the substantial movement of the optical center, the position u on the imaging surface and the angle θ of the light ray reaching there are θ = αu. If (1) is satisfied, an image equivalent to that obtained by the configuration in FIG. 2 can be obtained. The shape of the mirror M for realizing the characteristics of the above equation (1) will be described with reference to FIG. This figure is an enlarged view showing the vicinity of the point q in FIG. Here, the shape of the mirror M on the xz plane (xz cross-sectional shape) is represented by r (φ) in polar coordinates. Where φ is the angle from the z axis, r
(Φ) indicates the distance from the optical center O.

With this definition, the inclination of the surface of the mirror M in the radial direction is

[0024]

(Equation 1) It is represented by This corresponds to the perpendicular q at the point q of the ray Oq in the figure.
the angle ∠aqb between b and the tangent qa at the point q of the mirror M
The tangent of. ∠aqb is equal to ∠Oqq 'if rotated 90 degrees clockwise. Since qq 'is the normal of the mirror M, ∠Oqq' is equal to ∠q'qr. Also, the ray Oq
Assuming that the angle between と and the z-axis is φ, the angle ∠Oqc between qc parallel to the z-axis and the ray is also φ. Therefore, ∠aqb = (φ + θ) / 2 (3) From FIG. 3, u = ftan (φ) (4), so that the equations (1) and (4) are substituted into the equation (3). Then, if {aqb = {φ + αftan (φ)} / 2 (5), the mirror M satisfies the condition of the expression (1). Such a shape is obtained from the equation (2) by the equation

[0025]

(Equation 2) Is obtained as the solution of The solution to this equation is

[0026]

(Equation 3) It can be expressed as. Here, C is an arbitrary constant. FIG. 5 shows the shape of the mirror M as a result of numerically calculating the expression (7) (however, only the part where x> 0 is shown. The part where x <0 is symmetric with respect to the z-axis). Here, C
= 1, αf = 6. Move this mirror M in the horizontal direction ±
When combined with a camera with a viewing angle of 15 degrees, 6 tan
Since (15 °) × 360 / 2π ≒ 92 °, a scene in a range of about 184 degrees can be captured. In the prior art Japanese Patent Application Laid-Open No. 09-50547 using a hyperboloidal dome mirror, as described by the term "altitude gain", FIG.
Is disclosed in which θ is proportional to φ. On the other hand, the present invention is characterized in that the angle θ is proportional to the distance u instead of the angle φ. Here, a normal video camera has a shooting surface with an aspect ratio of 3: 4. In this case, if the horizontal viewing angle is ± 15 degrees as described above, the vertical viewing angle is tan ⁻¹ (3/4 tan (15 °)) ≒ 11.4 °.

That is, when the curved mirror of FIG. 5 is combined with an ordinary video camera, an image in a range of ± 11.4 degrees in the vertical direction and ± 92 degrees in the horizontal direction is displayed on a screen having an aspect ratio of 3: 4. This means that the image is heavily compressed in the horizontal direction. 92 / 11.4 in the horizontal direction
If the image is stretched and displayed at × 3/4 ≒ 6 times, the aspect ratio becomes approximately 11.4: 92, and the viewing angle and the length on the image match in the vertical and horizontal directions, thereby reproducing a natural image. it can. Such a horizontal extension process on digital image data can be easily realized by, for example, well-known one-dimensional interpolation. That is, while holding the value of the pixel of interest (P _i ) and the value of the immediately preceding pixel (P _i−1 ), calculation is performed by the following equation (8) for a total of six values of j = 0 to 5 for each input pixel. .

[0028]

(Equation 4) As a result, the pixel value Q of the image expanded 6 times is obtained. Since the one-dimensional decompression processing of the above equation (8) requires a small amount of calculation, it is possible to process moving image data in real time by software using a normal personal computer. FIG. 1A shows a system configuration according to the first embodiment of the present invention.

As shown in FIG.
Reference numeral 1 denotes a curved mirror M having the shape shown in FIG. Also, 1
2 is a flat CCD having a viewing angle of ± 15 degrees in the horizontal direction.
1 shows a video camera having an image sensor. FIG. 1B is a perspective view illustrating an arrangement of the mirror 11 and the camera 12.

The camera 12 is arranged so that its optical center coincides with O in FIG. As described above, the camera 12 captures an image around the camera in a range of a horizontal viewing angle of ± 92 degrees and a vertical viewing angle of ± 11.4 degrees, and outputs this as video data.

The video data output from the camera 12 is input to an A / D converter 13, converted into digital data, and then compressed by a compression processor 14 connected to a computer 15 according to a well-known MPEG compression algorithm. It is processed. The compressed data is recorded on the hard disk 16.

The MPEG compressed data recorded in this manner is restored to image data for each frame by MPEG decoder software operating on the computer 15 using a well-known MPEG restoration algorithm. The restored image data is expanded (expanded) by a factor of 6 in the horizontal direction according to the above equation (8), and then transferred to the frame memory of the computer 15 and displayed on the screen of the computer 15. In FIG. 1, the recording and reproduction and display of the moving image data are processed by one computer 15, but it is also possible to use different computers connected by a network for each function. Also, it is possible to directly reproduce the input moving image data without recording the data on the hard disk 16. In the first embodiment, the recorded moving image data is displayed 6 times in the horizontal direction. However, assuming that the original moving image data has the same resolution in the vertical and horizontal directions, the image displayed in this way has a lower horizontal resolution (band) than the vertical direction and is blurred only in the horizontal direction. turn into.

In the second embodiment of the present invention described below, in order to eliminate such unnaturalness, a low-pass filter having a function corresponding to a horizontal magnification, instead of a horizontal magnification, is used. Is applied vertically to reduce in the vertical direction.
This makes it possible to make the vertical and horizontal resolutions at the time of reproduction uniform. FIG. 6 shows an imaging apparatus 10 according to a second embodiment of the present invention.
1 is a block diagram.

Using the same camera 12 and mirror 11 as those in the first embodiment, after A / D conversion, MPEG
Before compression, the image data is reduced in the vertical direction by the reduction processing means 17. Since the reduction ratio is determined by the characteristics of the camera 12 and the mirror 11, it is 1/6 in this case.

The size of the image data after A / D conversion is 64
Assuming 0 × 480 pixels, an image having a size of 640 × 80 pixels after reduction is obtained. Here, the filtering process and the reduction process are performed simultaneously by setting the average value of six pixels in the vertical direction to the value of one pixel. In this case, when the image is displayed, the horizontal enlargement processing as in the case of the above-described first embodiment is unnecessary, and the image data subjected to the MPEG restoration processing may be displayed as it is. Next, the third aspect of the present invention
An embodiment will be described.

In the first and second embodiments, the resolution of the camera 12 (except for the curved mirror 11) is the same in the vertical and horizontal directions. Would. Therefore, in the third embodiment, this problem is reduced by changing the resolution of the camera 12 vertically and horizontally according to the characteristics of the mirror 11. This third
In the embodiment, the imaging apparatus has an imaging element having an array of rectangular light receiving cells, and uses a camera having a high resolution in the horizontal direction. The block diagram of this embodiment is the same as that of the first embodiment (FIG. 1).

The arrangement of the mirror 11 and the camera 12 is the same as that of the first and second embodiments.

The main difference from the embodiment described above lies in the image pickup device of the camera 12, and this configuration is shown in FIG.

This image pickup device is a monochrome image pickup device in which light receiving elements having an aspect ratio of 6: 1 are arranged 80 pixels vertically and 640 pixels horizontally. This means that the number of light receiving elements that are 6 times longer in the vertical direction than in the image pickup element (640 × 480 pixels) of the second embodiment is １ ／. Therefore, the image data obtained by the image pickup device is equivalent to the image data reduced in the vertical direction in the second embodiment, assuming that the data obtained by each light receiving element is one pixel.
The pixel data obtained by each light receiving element is directly subjected to A / D conversion, subjected to MPEG compression processing as image data having an aspect ratio of 1: 8, and recorded. In this case, similarly to the second embodiment, the recorded MPEG compressed data can be restored and displayed as it is. In the third embodiment, an example in which a monochrome image sensor is used for simplification of description is described. However, it is needless to say that a color image can be captured using a color separation filter or the like. Next, a fourth embodiment of the present invention will be described.

Since an ordinary image pickup device has a light receiving element close to a square, it has a 6: 1 length and width as in the third embodiment described above.
In order to use an image pickup device composed of light receiving elements having the ratio of, they must be specially prepared. on the other hand,
In the case where a color separation filter is arranged on one image pickup device to form a color image pickup device, the direction of the resolution can be given by the arrangement of the filters. In the present embodiment, the resolution is optimized using the color separation filter arrangement as described above. This fourth embodiment has the same block configuration (FIG. 6) as the second embodiment. The main difference lies in the image sensor, and the configuration is shown in FIG. In the image pickup device used in the fourth embodiment shown in FIG. 8, the order of RGBG is repeated in the vertical direction, and the same filter is continuously arranged in the horizontal direction. Has a square shape.

Among the RGB, the G filter has a spectral transmittance close to the human luminous luminance characteristic and most contributes to the resolution. For simplicity, if only light receiving elements having a G filter are extracted and considered, light receiving elements are arranged in the horizontal direction at twice the density as in the vertical direction. Therefore, it is considered that an image pickup device having such a color separation filter array can obtain an image having a resolution nearly twice as high in the horizontal direction as in the vertical direction. Therefore, the image obtained by this image sensor has the same vertical and horizontal resolutions only when it is enlarged twice in the horizontal direction. When the curved mirror 11 and the camera 12 are the same as those in the above-described first embodiment, the captured image needs to be enlarged 6 times in the horizontal direction before being displayed. Since the vertical and horizontal directions are equal when the image is enlarged twice, it is sufficient to scale the image at the same ratio in the vertical and horizontal directions. That is, by scaling the image vertically to 1/3 times and horizontally to 2 times, the aspect ratio of the entire image can be increased 6 times in the horizontal direction with the same vertical and horizontal resolution. If the total number of light receiving elements is the same as that of the third embodiment, the image obtained in the present embodiment has twice as many pixels vertically and horizontally as the third embodiment. Therefore, the imaging data can be used more effectively. Next, a fifth embodiment of the present invention will be described.

In the embodiment described so far, the camera 12 is arranged in front of the curved mirror 11 as shown in FIG. Therefore, the camera 12 is located at the center of the screen.
The image itself reflects the object in that direction. In the fifth embodiment of the present invention, the camera 12 itself is not imaged by using a half mirror. FIG. 9 is a side view showing the configuration of the mirror 11 and the camera 12 according to the fifth embodiment of the present invention. The difference from the first embodiment is that a half mirror 18 is arranged between the curved mirror 11 and the camera 12.

The mirror 11 can be seen through the half mirror 18 from the camera 12 installed upward in the figure. Because of the half mirror 18, the optical arrangement is equivalent to that of the first embodiment (as indicated by the broken line). Next, a sixth embodiment of the present invention will be described.

In the present embodiment, as in the case of the above-described fifth embodiment, a method for avoiding reflection of the camera itself is realized. Here, the shape of the mirror 11 is changed and the camera 1
This is to prevent the area where 2 exists from being captured. In the first embodiment, θ and u are configured to have a proportional relationship in the above equation (1). However, when an offset β is added to θ = αu + β (9) The shape of the mirror 11 expressed by the expression 7) is as follows.

[0045]

(Equation 5) Becomes In this case, when φ = 0 (u = 0), the slope of the curved surface M is not 0, and has a shape having a corner at the center as shown in FIG. Therefore, the light that goes almost directly above the optical center O in the figure swings right and left depending on which side of the corner of the mirror M hits, and a blind spot occurs as shown in the figure. Therefore, if the camera 12 is arranged in this area, the camera 1
It is possible to avoid photographing the image 2 itself and thereby widen the field of view.

For example, as in the first embodiment, the viewing angle of the camera 12 is ± 15 degrees, αf = 6, and β =
If it is 3 degrees, the field of view captured by the camera 12 is shifted by 3 degrees as a whole to 3 to 95 degrees (-3 to -95 degrees), and the central 6
The range is 190 degrees excluding the degree component. The embodiments of the present invention are not limited to those described above, and it goes without saying that various forms can be considered based on the basic concept of the present invention.

According to the first aspect of the present invention, an image similar to that of the cylindrical surface imaging system shown in FIG. 2 can be obtained, so that an image in a natural state for visual observation without complicated correction processing. Can be reproduced, and an image with a wide viewing angle without variation in resolution depending on the position can be taken. According to the second aspect of the present invention, since a one-dimensional curved mirror and a normal optical system are combined,
The above-described desired function can be realized with a simple configuration.

According to the third aspect of the invention, it is possible to correct the anisotropy of an image by the curved mirror.
According to the fourth aspect of the present invention, it is possible to correct anisotropy of an image by a curved mirror and generate an image having a natural shape. According to the fifth aspect of the present invention, it is possible to correct anisotropy of an image by a curved mirror and generate an image having a natural resolution characteristic. Further, according to the present invention, it is possible to correct the anisotropy of the image by the curved mirror and generate an image having a natural shape and resolution characteristics.

According to the ninth and tenth aspects of the present invention, since the camera itself does not enter the field of view, the field of view can be used effectively.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of a first embodiment of the present invention and a perspective view showing an arrangement of a mirror and a camera.

FIG. 2 is a perspective view and a plan view for explaining the function of a cylindrical imaging surface.

FIG. 3 is a diagram for explaining a basic principle of the present invention.

FIG. 4 is an enlarged view showing the vicinity of a point q in FIG. 3;

FIG. 5 is a diagram illustrating an example of a mirror shape calculated according to the basic principle of the present invention described with reference to FIGS.

FIG. 6 is a block diagram illustrating a schematic configuration of a second embodiment of the present invention.

FIG. 7 is a diagram illustrating a configuration of an image sensor of a camera used in a third embodiment of the present invention.

FIG. 8 is a diagram illustrating a configuration of an image sensor of a camera used in a fourth embodiment of the present invention.

FIG. 9 is a plan view showing an arrangement of a mirror, a camera, and a half mirror according to a fifth embodiment of the present invention.

FIG. 10 is a cross-sectional view showing a shape of a mirror used in a sixth embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1, 101 Image pickup device 11 Mirror (curved mirror) 12 Camera (including image sensor) 13 A / D converter 14 Compression processor 15 Computer 16 Hard disk 17 Reduction processing means 18 Half mirror

Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat II (Reference) H04N 9/07 H04N 9/07 A (72) Inventor Takashi Kitaguchi 1-3-6 Nakamagome, Ota-ku, Tokyo, Japan Stock Company F-term in Ricoh (reference) 2H087 KA03 TA03 TA06 5C022 AA00 AC42 AC51 5C054 AA01 EG01 GA01 GB01 HA18 5C065 AA01 BB48 CC01 DD02 EE02 EE03 GG13 HH04

Claims (10)

[Claims] An optical system on which light rays from an object are incident;
An image pickup element on which the light beam forms an image, wherein the angle of incidence of the light ray on the optical system when projected on a predetermined plane and one coordinate value indicating the position on the image pickup element surface on which the light ray forms an image An imaging device having a configuration having a linear relationship. 2. The imaging apparatus according to claim 1, wherein the optical system includes an optical system that is isotropic about an optical axis and a mirror having a cylindrical surface with a curved bottom. 3. The imaging apparatus according to claim 1, further comprising means for performing processing having different characteristics depending on directions on data obtained by said imaging element. 4. The imaging apparatus according to claim 3, wherein the processing having different characteristics depending on the direction includes a scaling processing having a different magnification depending on the direction. 5. The imaging apparatus according to claim 4, wherein the processing having different characteristics depending on the direction further comprises a smoothing processing. 6. The imaging device according to claim 1, wherein the imaging device generates image data having different resolution characteristics depending on a direction. 7. The imaging apparatus according to claim 6, wherein the imaging element has a configuration in which light receiving elements having different sizes depending on directions are arranged. 8. The imaging apparatus according to claim 6, wherein the imaging element has a stripe-shaped color separation filter. 9. The imaging apparatus according to claim 2, wherein a half mirror is provided between a curved mirror of the optical system and an imaging device. 10. The imaging apparatus according to claim 2, wherein the mirror of the optical system is a mirror having a corner that forms a blind spot, and the image pickup device is disposed within the blind spot.