JP2002034054A - Image processing system and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an image processing system equipped with an imaging device having an optical system including at least a pair of mirrors in which a high quality stereoscopic image can be obtained by correcting distortion of the image. SOLUTION: The image processing system comprises a mirror device 81 receiving light from a subject, an imaging optical system 8 having a imaging lens group and functions of zoom, autofocus, and the like, an image sensor 83 for converting a focused image into an electric signal, a digitizing/signal processing section 84 for processing an image signal from the image sensor 83, an R image memory 85 for storing digitized image data, an L image memory 86, a unit 87 for correcting the image based on the stored image data, an R image memory 88 for storing image data corrected by the correction unit 87, an L image memory 89, a section 90 for controlling each section, a section 91 for designating the content of control, a section 92 for reading out the image data stored in the R image memory 88 and the L image memory 89.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing system for correcting distortion of an image captured by an image pickup apparatus having an optical system including at least a pair of mirrors facing each other at a predetermined angle, and an image processing system. The present invention relates to a recording medium on which image information processed by the above is recorded.

[0002]

2. Description of the Related Art A conventional image photographing apparatus is configured, for example, as shown in FIG. In FIG. 8, an imaging system 3 that can capture a wide-angle image is formed by an optical unit 1 that combines two mirrors and an imaging device 2 that captures an image guided through the optical unit 1. Make up.

Reference numeral 10 denotes a reference base.
The first mirror 11 and the second mirror 12 are provided vertically on 0. For example, the first mirror 11 is fixed, and the second mirror 12 rotates around the mirror rotation axis 13 to control the angle facing the first mirror 11. The rotation can be performed via gears 16 and 17 by rotating the motor 15 in accordance with an instruction from the drive / control circuit 14.

In such a system, the image can be guided to the image pickup device 2 by rotating the second mirror 12 with respect to a wide range of objects Q _{1 to} Q _n .

[0005]

However, it is known that an image photographed by the apparatus as shown in FIG. 8 is distorted when the first mirror 11 and the second mirror 12 are not parallel. Hereinafter, distortion of an image captured by the imaging system 3 illustrated in FIG. 8 will be described.

FIGS. 9 to 11 show first and second mirrors 11 and 1.
2 shows a subject and its mirror image. FIG. 9 shows the first
When the mirror 11 and the second mirror 12 are parallel,
Body S ₀Is a mirror image S by the first mirror 11₁, The second mirror 12
Mirror image S_TwoAnd the subject S₀And mirror image S_TwoIs parallel
Therefore, the distortion as described later does not occur.

[0007] However, as shown in FIG.
If the first and second mirrors 12 are not parallel and face each other at an angle + θ, the subject S ₀ becomes a mirror image S ₁ by the first mirror 11, a mirror image S ₂ by the second mirror 12, and a mirror image S _2. will have an angle + theta with respect to the subject S ₀ is.

Further, as shown in FIG.
When the case where the second mirror 12 is opposed at an angle - [theta], the object S ₀ is the mirror image S ₁ by the first mirror 11, the mirror image S ₂ becomes the second mirror 12, mirror image S ₂ is subject S ₀ Has an angle −θ.

[0009] If the mirror image S ₂ with respect to the way the object S ₀ has a slope, distortion occurs in the captured image. For example, FIG.
In the state of 0, B _{2 of the} mirror image S ₂ is farther than A ₂ by Δd. Thus assuming the subject S ₀ example A, and a rectangle is a side opposing the B, front side A (A ₂₀ in the imaging plane when the mirror image S ₂ as shown in FIG. 12 taken with camera viewpoint P To A ₂₁ ) to a _{20 to} a ₂₁ , and the back side B (B _{20 to} B ₂₁ ) to b _{20 to} b ₂₁ , and a _{20 to} a ₂₁ and b ₂₀ to b ₂₁ , which should be the same length, b ₂₁ is a ₂₀ ~a
₂₁ is longer, and the captured image is distorted as shown in FIG. Of course, who b ₂₀ ~b ₂₁ is longer than a ₂₀ ~a ₂₁ in the state of FIG. 11.

Therefore, in the case of + θ deviation in FIG.
13 point b ₂₀ in (a), its rightful point b ₂₀ as shown in FIG. 13 (b) the point b ₂₁ ', point b _21' convert, must be corrected, also in FIG. 11 - When θ is shifted, the reverse operation is required.

Accordingly, the present invention corrects distortion of an image guided through a pair of mirrors having a predetermined angle to obtain a high-quality image, and uses this optical system to provide an effective and high-quality three-dimensional image. The task is to obtain images.

[0012]

An image processing system according to the present invention for solving the above-mentioned problems has an image having a pair of mirrors perpendicular to a predetermined reference plane and facing each other at a predetermined angle to each other. An image capturing apparatus includes a correcting unit configured to correct a distortion of a captured image (claim 1).

[0013] Further, it is perpendicular to a predetermined reference plane, and
A first pair of mirrors facing each other at a first predetermined angle, and a second pair of mirrors facing each other at a second predetermined angle; An image photographing apparatus that allows a first image guided by the second pair of mirrors and a second image guided by the second pair of mirrors to be simultaneously photographed with a predetermined parallax, wherein distortion of the photographed image is (Claim 5).

Further, it is perpendicular to a predetermined reference plane, and
An image having a pair of mirrors facing each other at a predetermined angle, and being separated by a predetermined distance between the pair of mirrors so that a subject can be directly photographed, and having a predetermined parallax guided by the pair of mirrors; Is an image photographing apparatus capable of photographing at the same time, and is provided with correction means for correcting distortion of a photographed image (claim 10).

Further, the image processing system according to the present invention can control the angle at which the pair of mirrors face each other, and the correction means includes at least one of a mirror angle, a distance to a subject, and a photographing magnification. Is used as a parameter, a correction operation is performed, and two subjects at the same horizontal position are specified in an image captured by the image capturing device, and correction is performed so that they are horizontal. It is configured to correct an image guided by a pair of mirrors based on an image obtained by directly photographing a subject in a photographed image.

Further, the recording medium of the present invention is characterized in that:
An image information processed by the image processing system according to claim 10 is recorded.

[0017]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of the image processing system of the present invention. In FIG. 1, 31 is a mirror device for receiving light from a subject, 32 is an imaging optical system having functions such as a photographing lens group, zoom, and autofocus,
33 is an image sensor that converts the formed image into an electric signal,
34, a digitizing / signal processing unit for processing an image signal from the image sensor 33; 35, an image memory for storing the processed image data; 36, a correction operation for correcting an image based on the processed image data An image memory 37 for storing image data corrected by the correction operation device 36;
Reference numeral 38 denotes a control unit that controls these units.

Reference numeral 39 denotes an operation instruction setting unit for instructing the contents of control, reference numeral 40 denotes a reading unit for reading image data stored in the image memory 37, and the mirror device 31 is equivalent to the optical unit 1 shown in FIG. It is.

As a correction method, one of or a combination of a mirror facing angle, a distance to a subject, and a photographing magnification is used, and correction is performed by the correction arithmetic unit 36 in accordance with an instruction from the control unit 38.
At this time, it is possible to input a correction condition from the outside via the operation instruction setting unit 39 to perform correction, and to automatically detect the magnification of the imaging device, the distance to the subject, and the facing angle of the mirror by a sensor (not shown). It is also possible to perform a correction operation using a predetermined formula.

Alternatively, two original subjects located at the same horizontal position may be specified by looking at the original image projected on the display device, and the entire image may be corrected so that they are horizontal.

The corrected image data is read from the image memory 37 via the reading section 40, displayed on a display device, recorded by a recording device, and transmitted by a communication device. As the recording device, any device that uses a magnetic tape, a magnetic disk, an optical disk, a semiconductor memory, or the like as a recording medium can be used.

Next, an embodiment in which the above-described invention is applied to a stereoscopic image photographing apparatus will be described. First, a stereoscopic image photographing apparatus using an optical system as shown in FIGS. 2 and 3 is being proposed by the present inventors.

The apparatus shown in FIG. 2 includes a first R mirror 51, a second R mirror 52, a first L mirror 53, and a second L mirror 5.
The second mirror 52 for R has an angle θ _{1 with} respect to the first mirror 51 for R, and the second mirror 54 for L has an angle θ ₂ with respect to the first mirror 53 for L. Note that θ ₁ may be θ ₂ .

By providing such an optical system on the front surface of the imaging device, images having parallax for the right and left eyes of the subjects M and N are formed on the imaging device 55, and an image which can be stereoscopically viewed is formed. Obtainable.

The apparatus shown in FIG. 3 comprises a first mirror 61 and a second mirror 62. The first mirror 61 and the second mirror 62
Are separated by a predetermined distance so that subjects M and N can be directly photographed. Further, the second mirror 62 is opposed to the first mirror 61 at an angle of θ, and the images of the subjects M and N through the optical system are simultaneously photographed by the image sensor 63. The directly photographed image and the image via the optical system can provide a stereoscopic image having parallax for the right eye and the left eye.

The stereoscopic viewing by the images obtained by the above two configurations is performed as follows, for example.
First, reference numerals 71 and 72 in FIG. 4 denote images having parallax for the left eye and right eye, respectively. This is alternately displayed, and a shutter provided in front of the observer is opened and closed in synchronization with the display. For example, m ₁ and n ₁ of reference numeral 71 are images of subjects M and N for the left eye, respectively, and m ₂ and n ₂ of reference numeral 72 are images of subjects M and N for the right eye, respectively.

Reference numeral 73 denotes a state where the image is switched at a high speed and displayed, and the shutter 74 is synchronized with the switching.
Is switched, and m ₁ and n ₁ are viewed with the left eye, and m ₂ and n ₂ are viewed with the right eye.

Now, a three-dimensional image can be obtained as described above, but the configuration of the optical part is the same as that described above, and distortion occurs. Therefore, the left and right images do not always match.

For example, FIG. 5A is an image obtained by photographing a parallel line using the optical unit having the configuration shown in FIG. 2, and the “parallel line” that should be parallel is determined by the angle between the pair of mirrors. It will be distorted. Therefore, if a stereoscopic image is viewed as it is, a suitable stereoscopic image can be obtained only at the center.

FIG. 5B is an image obtained by correcting these images by the correction means of the present invention and returning them to the original "parallel lines", and the upper and lower positional relations are properly adjusted.
As the correction, other parts are enlarged or reduced with reference to an arbitrary position, but when the number of pixels is insufficient due to the enlargement, interpolation is performed by calculating from pixel information of front, rear, left and right, and when reducing, at predetermined intervals. A method such as removing pixels is used.

FIG. 6A shows an image obtained by photographing a parallel line using the optical unit having the structure shown in FIG. 3. The L image directly photographed remains "parallel line". In the R image taken through the mirror, the "parallel line" which should be originally parallel is distorted by the angle at which the mirror faces.

FIG. 6 (b) shows the relationship R
This is an image that has been corrected to return to the original "parallel lines"
The upper and lower positional relationship is also adjusted appropriately. As the correction, the R image is arithmetically corrected based on the L image. When the number of pixels is insufficient due to the enlargement, the calculation and interpolation based on the front, rear, left and right pixel information are the same as those described above.

As described above, in the formation of a stereoscopic image by any configuration, a preferable stereoscopic image can be obtained by correcting the left and right images.

FIG. 7 is a block diagram showing the photographing and image processing of a stereoscopic image. In FIG. 7, reference numeral 81 denotes a mirror device for receiving light from a subject, which is described with reference to FIGS. 2 and 3. Reference numeral 82 denotes an imaging optical system having functions such as a photographing lens group, zoom, and autofocus. Reference numeral 83 denotes an image formed. , An image sensor for converting an image signal from the image sensor 83; 85, an R image memory for storing digitized image data; 86, an L image memory; Denotes a correction operation device that corrects an image based on the stored image data, 88 denotes an R image memory that stores the image data corrected by the correction operation device 87, 89 denotes an L image memory, and 90 controls these units. It is a control unit.

Reference numeral 91 denotes an operation instruction setting unit for instructing the contents of control, and 92 denotes a reading unit for reading out image data stored in the R image memory 88 and the L image memory 89.

As a correction method, one of the mirror facing angle, the distance to the subject, and the photographing magnification, or a combination thereof, is used to transfer the image data in the R image memory 85 and the L image memory 86 according to an instruction from the control unit 90. The correction is performed by the correction operation device 87 based on the correction. At this time, correction can be performed by inputting a correction condition from the outside via the operation instruction setting unit 91, and the magnification of the imaging device, the distance to the subject, and the mirror facing angle are automatically detected by a sensor (not shown). It is also possible to perform a correction operation using a predetermined formula.

Further, by looking at the original image projected on the display device, two subjects at the same horizontal position may be designated, and the entire image may be corrected so that they are horizontal.

The corrected image data is read out from the R image memory 88 and the L image memory 89 via the reading section 92, displayed on a display device, recorded by a recording device, and transmitted by a communication device. Will be As described above, any device that uses a magnetic tape, a magnetic disk, an optical disk, a semiconductor memory, or the like as a recording medium can be used as the recording device.

In the case where the optical unit shown in FIG. 3 is used, it is not necessary to correct an object which can directly photograph a subject in terms of image distortion, but it may be corrected so as to be more effective as a three-dimensional image. .

[0040]

As described above, according to the present invention, it is possible to correct a distortion of an image photographed through an optical system combining two mirrors and obtain a good image. Also, when obtaining a three-dimensional image, the left and right images can be easily matched, the distortion of the image itself can be removed, and an effective image correction can be performed as a three-dimensional image. A high-quality stereoscopic image can be obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram of an imaging system according to an embodiment of the present invention, to which an image distortion correction unit is added.

FIG. 2 is a schematic view illustrating an optical configuration of a first stereoscopic imaging apparatus having a combination of mirrors according to an embodiment of the present invention.

FIG. 3 is a schematic diagram illustrating another embodiment of the present invention and illustrating an optical configuration of a second stereoscopic imaging apparatus having a combination of mirrors.

FIG. 4 is an explanatory diagram illustrating a stereoscopic configuration of stereoscopic imaging of an image.

FIG. 5 is an explanatory diagram showing image distortion and correction in the first stereoscopic imaging apparatus of the present invention.

FIG. 6 is an explanatory diagram showing image distortion and correction in the second stereoscopic imaging apparatus of the present invention.

FIG. 7 is a block diagram of a stereoscopic photographing apparatus according to another embodiment of the present invention, to which an image distortion correcting unit is added.

FIG. 8 is a schematic configuration diagram of a conventional imaging system having an optical unit using two mirrors and an imaging device.

FIG. 9 is an explanatory diagram showing a relationship between a subject and a mirror image when the first mirror and the second mirror are parallel in the image photographing apparatus.

FIG. 10 is an explanatory diagram showing a relationship between a subject and a mirror image when the second mirror has an angle of + θ with respect to the first mirror in the image photographing apparatus.

FIG. 11 is an explanatory diagram showing a relationship between a subject and a mirror image when the second mirror has an angle of −θ with respect to the first mirror in the image photographing apparatus.

FIG. 12 is an explanatory diagram illustrating a state of image distortion.

FIG. 13 is an explanatory diagram illustrating a state of distortion correction of an image.

[Explanation of symbols]

31, 81: mirror device, 32, 82: imaging optical system, 3
3, 83: image sensor, 34, 84: digitizing / signal processing unit, 35, 37: image memory, 36, 87: correction arithmetic unit, 38, 90: control unit, 39, 91: operation instruction setting unit, 40 , 92... Readout unit, 51.
52 second mirror for R, 53 first mirror for L, 54
L second mirror, 55, 63: imaging element, 61: first mirror, 62: second mirror, 74: shutter, 85, 8
8 ... R image memory, 86, 89 ... L image memory.

 ──────────────────────────────────────────────────の Continued on the front page F term (reference) 5B057 BA02 CA08 CA12 CA16 CB08 CB12 CB16 CC01 CD01 5C061 AA03 AB06 AB08 AB21 5C076 AA23 BA01 BA06

Claims (15)

[Claims] 1. An image photographing apparatus having a pair of mirrors perpendicular to a predetermined reference plane and facing each other at a predetermined angle with respect to each other, comprising a correcting means for correcting distortion of a photographed image. An image processing system characterized by: 2. An image processing system according to claim 1, wherein an angle between the pair of mirrors is controllable. 3. The correction unit according to claim 1, wherein the correction unit performs a correction calculation using at least one of a mirror facing angle, a distance to a subject, and a shooting magnification as parameters.
An image processing system according to claim 1. 4. The image processing apparatus according to claim 1, wherein the correction unit specifies two subjects located at the same horizontal position in the image captured by the image capturing apparatus, and corrects them so that they are horizontal. Item 2. The image processing system according to Item 1. 5. A first pair of mirrors perpendicular to a predetermined reference plane and facing each other at a first predetermined angle,
A second pair of mirrors facing each other at a second predetermined angle, and a first image guided by the first pair of mirrors and a second image guided by the second pair of mirrors An image processing apparatus, comprising: an image photographing apparatus capable of simultaneously photographing two images with a predetermined parallax; and a correcting unit for correcting distortion of the photographed image. 6. The image processing system according to claim 5, wherein each of the first predetermined angle and the second predetermined angle is configured to be controllable. 7. The correction unit according to claim 5, wherein the correction unit performs a correction operation using at least one of a mirror facing angle, a distance to a subject, and a photographing magnification as parameters.
An image processing system according to claim 1. 8. The image processing apparatus according to claim 1, wherein the correction unit specifies two subjects located at the same horizontal position in the image captured by the image capturing apparatus, and corrects them so that they are horizontal. Item 6. The image processing system according to Item 5. 9. A recording medium on which image information processed by the image processing system according to claim 5 is recorded. 10. A pair of mirrors perpendicular to a predetermined reference plane and facing each other at a predetermined angle with respect to each other, wherein the pair of mirrors are separated by a predetermined distance so that a subject can be directly photographed. And an image photographing apparatus capable of simultaneously photographing an image having a predetermined parallax guided by the pair of mirrors, the image photographing apparatus including a correcting unit for correcting distortion of the photographed image. Processing system. 11. The image processing system according to claim 10, wherein an angle between the pair of mirrors is controllable. 12. The image processing system according to claim 10, wherein the correction unit performs a correction calculation using at least one of a mirror facing angle, a distance to a subject, and a shooting magnification as parameters. 13. The image capturing apparatus according to claim 13, wherein the correcting unit is configured to detect two images located at the same horizontal position in the image captured by the image capturing apparatus.
The image processing system according to claim 10, wherein two objects are designated and the objects are corrected so as to be horizontal. 14. The image processing apparatus according to claim 1, wherein the correction unit corrects an image guided by a pair of mirrors, based on an image obtained by directly capturing the subject, among the images captured by the image capturing apparatus. The image processing system according to claim 10. 15. A recording medium on which image information processed by the image processing system according to claim 10 is recorded.