JP4759789B2 - Image processing system - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention has been opposed by a predetermined angle, relates to an image processing system for correcting distortion of a captured image by the imaging apparatus having an optical system comprising at least a pair of mirrors.
[0002]
[Prior art]
A conventional image capturing apparatus is configured as shown in FIG. In FIG. 8, an imaging system 3 that can capture an image of a wide angle with an optical unit 1 that combines two mirrors and an imaging device 2 that captures an image guided through the optical unit 1. It is composed.
[0003]
Reference numeral 10 denotes a reference base. On the reference base 10, a first mirror 11 and a second mirror 12 are provided vertically. For example, the first mirror 11 is fixed, and the second mirror 12 rotates about the mirror rotation shaft 13 to control the angle facing the first mirror 11. The rotation can be performed via gears 16 and 17 by the rotation of the motor 15 in accordance with an instruction from the drive / control circuit 14.
[0004]
In such a system, the image can be guided to the imaging device 2 by rotating the second mirror 12 with respect to a wide range of subjects Q _{1 to} Q _n .
[0005]
[Problems to be solved by the invention]
However, it is known that the image captured by the apparatus as shown in FIG. 8 is distorted when the first mirror 11 and the second mirror 12 are not parallel. Hereinafter, distortion of an image captured by the imaging system 3 illustrated in FIG. 8 will be described.
[0006]
9 to 11 show the subject and its mirror image by the first and second mirrors 11 and 12. FIG. 9 shows a case where the first mirror 11 and the second mirror 12 are parallel, and the subject S ₀ becomes a mirror image S ₁ by the first mirror 11 and a mirror image S ₂ by the second mirror 12, and the subject S ₀ and the mirror image S _2. Are parallel and do not generate distortion as described later.
[0007]
However, as shown in FIG. 10, when the first mirror 11 and the second mirror 12 are not parallel but are opposed to each other with an angle + θ, the subject S ₀ is mirrored by the first mirror 11 with the mirror image S ₁ and the second mirror. 12 becomes a mirror image S ₂ , and the mirror image S ₂ has an angle + θ with respect to the subject S ₀ .
[0008]
As shown in FIG. 11, when the first mirror 11 and the second mirror 12 face each other with an angle −θ, the subject S ₀ is mirror image S ₁ by the first mirror 11 and the second mirror 12. It becomes a mirror image S ₂ , and the mirror image S ₂ has an angle −θ with respect to the subject S ₀ .
[0009]
Thus, when the mirror image S ₂ has an inclination with respect to the subject S ₀ , the captured image is distorted. For example B ₂ of mirror image S ₂ is in the state of FIG. 10 is as lies far only Δd than A _2. Thus assuming the subject S ₀ example A, and a rectangle is a side opposing the B, front side A (A ₂₀ in the imaging plane when the mirror image S ₂ as shown in FIG. 12 taken with camera viewpoint P to a ₂₁₎ also in a ₂₀ ~a _21, the back side B (B ₂₀ ~B ₂₁₎ is b ₂₀ ~b becomes _21, a ₂₀ which should be originally the same length ~a ₂₁ and b ₂₀ ~ b ₂₁ becomes longer toward a ₂₀ ~a _21, the captured image as shown in FIG. 13 (a) will be distorted. Of course, who b ₂₀ ~b ₂₁ is longer than a ₂₀ ~a ₂₁ in the state of FIG. 11.
[0010]
Therefore, in the case of the deviation of + θ in FIG. 10, the points b ₂₀ and b ₂₁ in FIG. 13A are converted and corrected to the points b ₂₀ ′ and b ₂₁ ′ that should be as shown in FIG. 13B. In addition, the reverse operation is required in the case where the shift is −θ in FIG. 11.
[0011]
Therefore, the present invention corrects distortion of an image guided through a pair of mirrors having a predetermined angle to obtain a high-quality image, and obtains an effective and high-quality stereoscopic image using this optical system. This is the issue.
[0013]
An image processing system according to a first aspect of the present invention includes a first pair of mirrors perpendicular to a predetermined reference plane and facing each other at a first predetermined angle, and a second predetermined A first pair of mirrors having a second pair of mirrors facing each other at an angle, and a second image guided from the first pair of mirrors, and a second image guided from the second pair of mirrors An image photographing device capable of simultaneously photographing two images by forming an image on a single image sensor having a predetermined parallax, and a pair of first mirrors or a pair of second mirrors. Correction means for correcting distortion of an image captured by the image capturing apparatus that occurs when the images are not parallel to each other by enlarging or reducing other portions based on an arbitrary position in the image according to the distortion It comprises. The correction means corrects the entire image so that two subjects at the same specified horizontal position are horizontal in the image captured by the image capturing apparatus.
[0014]
The image processing system according to the second aspect of the present invention includes a pair of mirrors that are perpendicular to a predetermined reference plane and that are opposed to each other at a predetermined angle, and between the pair of mirrors. by a predetermined distance apart as well as a possible shot directly by one of the imaging device to a subject, guided by the pair of mirrors, by the one of the imaging device also images having a predetermined disparity with respect to the directly photographed image The distortion of the image taken by the imaging device guided by the pair of mirrors, which is generated when the imaging device capable of simultaneously capturing and the pair of mirrors are not parallel to each other , according to the distortion And correction means for correcting by enlarging or reducing other portions with reference to an arbitrary position in the image . The correction means corrects the entire image so that two subjects at the same specified horizontal position are horizontal in the image captured by the image capturing apparatus.
[0015]
For example, in the image processing system according to the second aspect of the present invention, the opposing angles of the pair of mirrors may be controllable.
Also, for example, in the image processing system according to the second aspect of the present invention, the correction unit may perform correction calculation using at least one of a mirror facing angle, a distance to a subject, and a shooting magnification as a parameter. good.
Also, for example, in the image processing system according to the second aspect of the present invention, the image guided by the pair of mirrors is corrected based on the image obtained by directly photographing the subject among the images photographed by the image photographing device. It may be configured as follows.
[0017]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of an image processing system according to the present invention. In FIG. 1, 31 is a mirror device that receives light from a subject, 32 is a photographing lens group, an imaging optical system having functions such as zooming and autofocusing, 33 is an imaging element that converts an image formed into an electrical signal, 34 Is a digitization / signal processing unit for processing an image signal from the image sensor 33, 35 is an image memory for storing processed image data, and 36 is a correction arithmetic unit for correcting an image based on the processed image data. , 37 is an image memory for storing image data corrected by the correction arithmetic unit 36, and 38 is a control unit for controlling these units.
[0018]
39 is an operation instruction setting unit for instructing the contents of control, 40 is a reading unit for reading out image data stored in the image memory 37, and the mirror device 31 is equivalent to the optical unit 1 shown in FIG.
[0019]
As a correction method, one of the mirror facing angle, the distance to the subject, and the photographing magnification or a combination thereof is corrected by the correction calculation device 36 according to an instruction from the control unit 38. At this time, correction conditions can be inputted and corrected from the outside via the operation instruction setting unit 39, and the magnification of the imaging device, the distance to the subject, and the mirror facing angle are detected by a sensor (not shown) and automatically It is also possible to perform a correction operation using a predetermined formula.
[0020]
Alternatively, by looking at the original image displayed on the display device, two subjects at the same horizontal position may be designated, and the entire image may be corrected so that they are horizontal.
[0021]
The corrected image data is read from the image memory 37 via the reading unit 40, displayed on the display device, recorded by the recording device, and transmitted by the communication device. As the recording apparatus, any apparatus using a magnetic tape, a magnetic disk, an optical disk, a semiconductor memory, or the like as a recording medium can be used.
[0022]
Next, an embodiment in which the above-described invention is applied to a stereoscopic image capturing device will be described. First, the present inventors have proposed a stereoscopic image photographing apparatus using an optical system as shown in FIGS.
[0023]
The apparatus shown in FIG. 2 includes an R first mirror 51, an R second mirror 52, an L first mirror 53, and an L second mirror 54, and the R second mirror with respect to the R first mirror 51. Reference numeral 52 denotes an angle θ ₁ , and the second L mirror 54 has an angle θ ₂ with respect to the first L mirror 53. Note that θ ₁ = θ ₂ may be used.
[0024]
By providing such an optical system on the front surface of the imaging apparatus, images having parallaxes for the right and left eyes of the subjects M and N are formed on the imaging element 55 to obtain an image that can be viewed stereoscopically. it can.
[0025]
3 includes a first mirror 61 and a second mirror 62, and the first mirror 61 and the second mirror 62 are separated by a predetermined distance so that the subjects M and N can be directly photographed. Further, the second mirror 62 is opposed to the first mirror 61 with an angle of θ, and the images of the subjects M and N through this optical system are simultaneously captured by the image sensor 63. The directly photographed image and the image via the optical system can obtain a stereoscopically viewable image having right-eye and left-eye parallax.
[0026]
As an example, stereoscopic vision using images obtained by the two configurations described above is performed as follows. First, reference numerals 71 and 72 in FIG. 4 are images having parallax for the left eye and the right eye, respectively. This is displayed alternately, and the shutter provided in front of the observer's eyes is opened and closed in synchronization with the display. For example, m ₁ and n ₁ denoted by reference numeral 71 are images of subjects M and N for the left eye, respectively, and m ₂ and n ₂ denoted by reference numeral 72 are images of subjects M and N for the right eye, respectively.
[0027]
Reference numeral 73 denotes a state in which switching is performed at high speed, and the shutter 74 is switched in synchronization with the switching so that m ₁ and n ₁ are viewed with the left eye and m ₂ and n ₂ are viewed with the right eye. The objects M ′ and N ′ appear three-dimensional.
[0028]
Now, as described above, a stereoscopic image can be obtained. The configuration of the optical unit is the same as that described above, and distortion occurs. Therefore, the left and right images do not necessarily match.
[0029]
For example, FIG. 5A is an image obtained by photographing parallel lines using the optical unit having the configuration shown in FIG. 2, and the “parallel lines” that should be parallel to each other are distorted by the opposing angles of the pair of mirrors. Become. Accordingly, when stereoscopic viewing is performed as it is, a suitable stereoscopic image can be obtained only at the central portion.
[0030]
FIG. 5B is an image in which these images are corrected by the correcting means of the present invention and returned to the original “parallel lines”, and the vertical positional relationship is also adjusted appropriately. As correction, other parts are enlarged or reduced with an arbitrary position as a reference, but if the number of pixels is insufficient due to enlargement, it is calculated and interpolated from the front, rear, left and right pixel information, and when reduced, at a predetermined interval A method such as removing pixels is used.
[0031]
FIG. 6A is an image obtained by photographing parallel lines using the optical unit having the configuration shown in FIG. 3, and the directly captured L image remains “parallel lines”. The “parallel lines” that should be parallel to each other are distorted in the R image captured via the mirror, depending on the angle at which the mirrors face each other.
[0032]
FIG. 6B shows an image in which the R image is corrected by the correcting means of the present invention and returned to the original “parallel line”, and the vertical positional relationship is also adjusted to suitability. As the correction, the R image is calculated and corrected based on the L image. When the number of pixels becomes insufficient due to enlargement, the calculation and interpolation from the pixel information of front, rear, left and right is the same as described above.
[0033]
As described above, in creating a stereoscopic image by any configuration, a suitable stereoscopic image can be obtained by correcting the left and right images.
[0034]
FIG. 7 is a block configuration diagram of stereoscopic image capturing and image processing. 7, 81 is the mirror device described in FIGS. 2 and 3 that receives light from the subject, 82 is a photographing lens group, an imaging optical system having functions such as zoom and autofocus, and 83 is a formed image. Is an image sensor for converting image signals into electrical signals, 84 is a digitization / signal processing unit for processing image signals from the image sensor 83, 85 is an R image memory for storing digitized image data, 86 is an L image memory, 87 Is a correction arithmetic unit that corrects an image based on the stored image data, 88 is an R image memory that stores image data corrected by the correction arithmetic unit 87, 89 is an L image memory, and 90 is a control unit for these units. It is a control unit.
[0035]
91 is an operation instruction setting unit for instructing the content of control, and 92 is a reading unit for reading out image data stored in the R image memory 88 and the L image memory 89.
[0036]
As a correction method, correction is made based on the image data of the R image memory 85 and the L image memory 86 according to an instruction from the control unit 90 by one of the mirror facing angle, the distance to the subject, and the shooting magnification, or a combination thereof. Correction is performed by the arithmetic unit 87. At this time, correction conditions can be input and corrected from the outside via the operation instruction setting unit 91, and the magnification of the imaging device, the distance to the subject, and the mirror facing angle are automatically detected by a sensor (not shown). It is also possible to perform a correction operation using a predetermined formula.
[0037]
Alternatively, by looking at the original image displayed on the display device, two subjects at the same horizontal position may be designated, and the entire image may be corrected so that they are horizontal.
[0038]
The corrected image data is read from the R image memory 88 and the L image memory 89 via the reading unit 92, displayed on the display device, recorded by the recording device, and transmitted by the communication device. As described above, any recording device that uses a magnetic tape, a magnetic disk, an optical disk, a semiconductor memory, or the like as a recording medium can be used.
[0039]
When the optical unit shown in FIG. 3 is used, an object that can directly shoot a subject does not need to be corrected in terms of image distortion, but may be corrected so as to be more effective as a stereoscopic image.
[0040]
【The invention's effect】
As described above, according to the present invention, it is possible to correct distortion of an image photographed through an optical system in which two mirrors are combined, and obtain a good image. In addition, when obtaining a stereoscopic image, the left and right images can be easily matched, distortion of the image itself can be removed, and further, effective image correction can be applied as a stereoscopic image. A high-quality stereoscopic image can be obtained.
[Brief description of the drawings]
FIG. 1 is a block diagram of an image pickup system to which an image distortion correction unit is added, showing an embodiment of the present invention.
FIG. 2 is a schematic diagram showing an optical configuration of a first stereoscopic imaging apparatus that represents an embodiment of the present invention and has a combination of mirrors.
FIG. 3 is a schematic diagram illustrating an optical configuration of a second stereoscopic imaging apparatus having a combination of mirrors, illustrating another embodiment of the present invention.
FIG. 4 is an explanatory diagram showing a three-dimensional configuration of stereoscopic shooting of an image.
FIG. 5 is an explanatory diagram showing image distortion and correction of the first stereoscopic imaging apparatus of the present invention.
FIG. 6 is an explanatory diagram showing image distortion and correction of the second stereoscopic imaging apparatus of the present invention.
FIG. 7 is a block diagram of a stereoscopic imaging apparatus to which another embodiment of the present invention is added and image distortion correction means is added.
FIG. 8 is a schematic configuration diagram of an imaging system having an optical unit and an imaging apparatus using two conventional mirrors.
FIG. 9 is an explanatory diagram illustrating a relationship between a subject and a mirror image when a first mirror and a second mirror are parallel in the image capturing apparatus.
FIG. 10 is an explanatory diagram showing a relationship between a subject and a mirror image when the second mirror has an angle of + θ with respect to the first mirror in the image capturing device.
FIG. 11 is an explanatory diagram illustrating a relationship between a subject and a mirror image when the second mirror has an angle of −θ with respect to the first mirror in the image capturing device.
FIG. 12 is an explanatory diagram showing a state of image distortion.
FIG. 13 is an explanatory diagram showing a state of image distortion correction.
[Explanation of symbols]
31, 81 ... Mirror device, 32, 82 ... Imaging optical system, 33, 83 ... Imaging element, 34, 84 ... Digitization / signal processing unit, 35, 37 ... Image memory, 36, 87 ... Correction operation device, 38, 90 ... control unit, 39, 91 ... operation instruction setting unit, 40,92 ... reading unit, 51 ... first mirror for R, 52 ... second mirror for R, 53 ... first mirror for L, 54 ... first for L 2 mirrors, 55, 63 ... imaging device, 61 ... first mirror, 62 ... second mirror, 74 ... shutter, 85, 88 ... R image memory, 86, 89 ... L image memory.

Claims (9)

A first pair of mirrors that are perpendicular to a predetermined reference plane and that face each other with a first predetermined angle; and a second pair of mirrors that face each other with a second predetermined angle A first image guided by the first pair of mirrors and a second image guided by the second pair of mirrors are connected to one image sensor with a predetermined parallax. An image capturing device capable of capturing two images simultaneously by being imaged,
The distortion of the image captured by the image capturing device , which occurs when the first pair of mirrors or the second pair of mirrors are no longer parallel to each other, is determined according to the distortion. Correction means for correcting by enlarging or reducing other parts with respect to the position ,
An image processing system in which the correcting means designates two subjects at the same horizontal position among images captured by the image capturing device and corrects the entire image so that they are horizontal. The image processing system according to claim 1 , wherein each of the first predetermined angle and the second predetermined angle is configured to be controllable. The image processing system according to claim 1 , wherein the correction unit performs correction calculation using at least one of a mirror facing angle, a distance to a subject, and a photographing magnification as a parameter. A display device for displaying an image captured by the image capturing device;
The image processing system according to claim 1 , wherein the correction unit corrects the entire image so that two subjects at the same horizontal position specified in the image displayed on the display device are horizontal. It is perpendicular to the predetermined reference plane, and has a pair of mirrors which face each other a predetermined angle to each other, between the pair of mirrors directly photographing possible by a single image sensor of an object by a predetermined distance apart And an image photographing device capable of simultaneously photographing an image guided by the pair of mirrors and having a predetermined parallax with respect to the directly photographed image by the one image sensor ;
Distortion of an image taken by the image capturing device guided by the pair of mirrors, which occurs when the pair of mirrors are not parallel to each other, is based on an arbitrary position in the image according to the distortion. And correcting means for correcting by enlarging or reducing other parts ,
An image processing system in which the correcting means designates two subjects at the same horizontal position among images captured by the image capturing device and corrects the entire image so that they are horizontal. The image processing system according to claim 5 , wherein the opposing angles of the pair of mirrors are configured to be controllable. The image processing system according to claim 5 , wherein the correction unit performs correction calculation using at least one of a mirror facing angle, a distance to a subject, and a photographing magnification as a parameter. A display device for displaying an image captured by the image capturing device;
The image processing system according to claim 5 , wherein the correction unit corrects the two subjects at the same horizontal position specified in the image displayed on the display device to be horizontal. The image processing system according to claim 5 , wherein the correction unit corrects an image guided by a pair of mirrors based on an image obtained by directly photographing a subject among images photographed by the image photographing device.

Images