JP2018182470A - Stereoscopic image pickup apparatus adopting wavelength selective polarization separation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to capture a stereoscopic image of excellent image quality by securing a light amount while making an image pickup apparatus for capturing a stereoscopic image compact and light-weight.SOLUTION: In a region where light condensed by a photographing lens becomes collimated light, first polarized light and second polarized light formed by polarizing only components of a specific wavelength band respectively selected on the right half surface and the left half surface of a polarization filter is converted into a first image and a second image by an image capturing device, each of which is adjusted in image quality on the basis of a pixel pattern of a specific wavelength band component to generate binocular parallax images.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a stereoscopic imaging apparatus that stereoscopically captures an object.

  Conventionally, there is a camera provided with two lenses as a means for acquiring an optical image having binocular parallax in photographing a stereoscopic image.

  However, when capturing a stereoscopic image with two lenses, it is difficult to adjust the position of the camera according to the distance from the subject, etc. and to link the zoom, focus and iris of the lens, making the mechanism complicated. In order to contribute to cost increase, various inventions have been proposed for making it possible to capture a stereoscopic image with one lens.

  For example, by arranging a prism mirror in a region where light diffused from one point of the subject becomes parallel light, the collected light is divided in the left direction and the right direction, and then each is imaged by the imaging lens Therefore, there is an invention of acquiring an optical image having parallax (see Patent Document 1 below). According to the present invention, it is possible to acquire an optical image having parallax without loss of light quantity.

  Further, by arranging polarization filters having different polarization characteristics between the left half and the right half in a region where light diffused from one point of the subject becomes parallel light, the collected light is divided between the left half and the right half. There is an invention of acquiring an optical image having parallax by arranging polarizing plates having polarization characteristics corresponding to the different polarizations from each other in a predetermined pattern on the surface of an imaging device (see Patent Document 2 below). reference). According to the present invention, a reduction in size and weight can be realized because a new optical system is not required.

Patent No. 5531348 gazette Patent No. 539914

  However, the invention of Patent Document 1 has a problem that an increase in apparatus size and an increase in weight can not be avoided because an optical system including a prism mirror and two imaging lenses is required.

  Further, in the invention of Patent Document 2, there is a problem that the loss of light quantity depending on the transmittance of the polarizer is large, the obtained image becomes dark, or the image quality mainly on the S / N ratio, that is, the signal ratio to noise deteriorates. .

  Therefore, in the conventional stereoscopic imaging device, a large reduction in the light amount due to the reduction in size and weight can not be avoided, and it is difficult to achieve both the reduction in size and the reduction in weight to achieve bright and favorable three-dimensional imaging.

  The present invention has been made in view of such circumstances, and it is an object of the present invention to provide a three-dimensional imaging apparatus capable of performing bright and favorable three-dimensional imaging with excellent image quality as well as reduction in size and weight.

The three-dimensional imaging apparatus according to the present invention has been made to solve the above-mentioned problems,
A shooting lens for collecting light from a subject,
A stop for adjusting the amount of the collected light;
In a region where light diffused from one point of the subject becomes parallel light, it is arranged in series in parallel in close proximity to the stop, and the condensed light is selected in the right half plane and the left half plane. A polarized light filter that transmits the first polarized light and the second polarized light that are polarized only in the
A polarization half mirror that transmits the first polarized light and reflects the second polarized light among the first and second polarized lights transmitted through the polarizing filter;
A first imaging element configured to convert a first polarized light transmitted through the polarized half mirror into a first image by an electronic signal;
A first signal processing unit that performs predetermined signal processing on the first image;
A second imaging element configured to convert the second polarized light reflected by the polarized half mirror into a second image by an electronic signal;
A second signal processing unit that performs predetermined signal processing on the second image;
An image recording unit for storing the first and second images subjected to signal processing in the first and second signal processing units;
And the like.

  ADVANTAGE OF THE INVENTION According to this invention, the outstanding effect that the three-dimensional imaging of bright and favorable image quality is attained with size reduction and weight reduction can be show | played.

It is a general view of the optical system in the prior art of this invention. It is top sectional drawing of the optical system in the prior art of this invention. It is a block diagram which shows the function structure of the three-dimensional imaging device in the 1st Embodiment of this invention. FIG. 1 is an overall view of an optical system according to a first embodiment of the present invention. It is upper surface sectional drawing of the optical system in the 1st Embodiment of this invention. It is a flow chart which shows a flow of processing of image quality adjustment of an imaging device in a 1st embodiment of the present invention. It is a block diagram which shows the function structure of the three-dimensional imaging device in the 2nd Embodiment of this invention. It is a figure which shows the contrast sensitivity characteristic with respect to the brightness | luminance and chromaticity of a human (KATHY T. MULLEN, "THE CONTRAST SENSITIVITY OF HUMAN COLOR VISION TO RED-GREEN AND BLUE-YELLOW CHROMATIC GRATINGS", J. Physiol, VOl 359, 381-400. , 1985).

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The present invention is not limited to the following description, and can be appropriately changed without departing from the scope of the present invention.

  First, the configuration of the stereoscopic imaging device 100 according to the first embodiment of the present invention will be described. Here, FIG. 3 is a block diagram showing a functional configuration of the stereoscopic imaging device 100 according to the first embodiment of the present invention, and FIG. 4 is an overall view of an optical system according to the first embodiment of the present invention. FIG. 5 is a top cross-sectional view of the optical system in the first embodiment of the present invention. In these figures, the notations R, G and B respectively indicate components corresponding to the respective elements of red, green and blue which are the three primary colors of light, and the subscripts of P and S indicate polarization filters Corresponding to being for the P wave and the S wave, it represents that the polarization is the P wave component and the S wave component, respectively. The same applies below.

  As shown in FIG. 3, the stereoscopic imaging device 100 according to the first embodiment of the present invention includes an imaging unit 31, an image processing unit 34, and an image recording unit 37.

  As shown in FIGS. 4 and 5, the imaging unit 31 includes a photographing lens 2, an aperture, polarizing filters 21 and 22, a polarizing half mirror 6, a first imaging device 9, a second imaging device 10 and the like.

  The photographing lens 2 is a lens for condensing incident light 1 from a subject. The photographing lens 2 also includes a focus lens for focusing, a zoom lens for enlarging an object, and the like.

  A diaphragm is usually disposed at a position where light diffused from one point of an object becomes parallel light, and adjusts the amount of light collected. The aperture is generally configured by combining a plurality of plate-like blades.

  The polarization filters 21 and 22 are wavelength selective polarizers that provide polarization to the components of the specific wavelength band respectively selected in the right half surface 21 and the left half surface 22.

  The polarization filters 21 and 22 are disposed in series in parallel with the stop at a position where light diffused from one point of the object becomes parallel light, and the collected light is arranged on the right half surface 21 and the left half surface 22 respectively. It transmits the first polarized light 23 and the second polarized light 24 in which only the components of the selected specific wavelength band are polarized.

  FIGS. 4 and 5 show examples of wavelength selective polarizers that polarize components in the wavelength band corresponding to the element of G. In the first polarized light 23 that passes through the wavelength selective polarizer 21 for P wave with respect to the component of the wavelength band corresponding to the element of G and finally reaches the imaging device 9, only the element of G becomes the P wave component. Similarly, in the second polarized light 24 that passes through the wavelength selective polarizer 21 for S wave with respect to the wavelength band of the element of G and finally reaches the imaging device 10, only the element of G becomes an S wave component.

  The polarization filters 21 and 22 can be realized by thin film design or the like. For example, wavelength selective polarizers for wavelengths from 480 nm to 550 nm are commercially available, with 62% to 82% transmission.

  The polarization half mirror 6 is a mirror for separating a specific polarization. The polarized half mirror 6 transmits the first polarized light 23 and reflects the second polarized light 24 among the polarized lights 23 and 24 respectively transmitted through the polarizing filters 21 and 22.

  The first polarized light 23 transmitted through the polarization half mirror 6 is projected to the first imaging device 9, and the second polarized light 24 reflected from the polarization half mirror 6 is projected to the second imaging device 10. .

  The imaging elements 9 and 10 are photoelectric conversion elements that convert incident light into an image by an electronic signal. The first imaging device 9 converts the first polarization 23 transmitted through the polarization half mirror 6 into a first image by an electronic signal. Similarly, the second imaging device 10 converts the second polarized light 24 reflected by the polarizing half mirror 6 into a second image by an electronic signal. The imaging elements 9 and 10 are realized by, for example, a charge coupled device (CCD) image sensor, a complementary metal oxide semiconductor (CMOS) image sensor, or the like.

  The image processing unit 34 includes at least a first signal processing unit 35 and a second signal processing unit 36. Here, the signal processing units 34 and 35 are electronic circuits, and various programs for executing predetermined signal processing are implemented. The first signal processing unit 35 performs A / D conversion on a first image output as an electronic signal from the first imaging device 9 and performs predetermined signal processing such as demosaicing processing, interpolation processing, image quality adjustment, and the like. Output the first image. Similarly, the second signal processing unit 36 A / D converts the second image output as an electronic signal from the second imaging element 10, and performs predetermined signal processing such as demosaicing processing, interpolation processing, image quality adjustment, and the like. Output the second image on which

  Here, for image quality adjustment of an image, for example, when the parallax component of the image obtained by the imaging elements 9 and 10 is a wavelength band component corresponding to the element of G, a pixel of interest is included for each of the R and B pixel patterns. While investigating a certain range of target pixel groups in the vertical direction and the horizontal direction, sequentially calculate the sum of squared differences (SSD) between the pixel groups of the similar range of the pixel pattern of G, and The position where the value becomes the smallest, that is, the coordinate values matching the R and B pixel patterns with the G pixel pattern, respectively, is searched for, and the R and B pixel patterns are moved to that position. By this processing, it is possible to obtain a good image in which the parallax information is more reflected.

Here, the sum of squared differences (SSD) is expressed by the following equation. In this equation, T (i, j) is a template pixel pattern, I (i, j) is a target pixel pattern, i and j are coordinate values in the vertical and horizontal directions of the pixel pattern, and m and n are pixel patterns. Represents the number of pixels in the vertical and horizontal directions. The same shall apply hereinafter.

  In the pixel pattern matching in the image quality adjustment, in addition to the above-mentioned difference square sum, the sum of absolute differences (SAD: Sum of Absolute Difference) is calculated, and the pixel is found by searching for a coordinate value that minimizes the value. It is possible to match patterns. In addition, it is possible to match a pixel pattern by calculating a normalized correlation coefficient (NCC) and searching for a coordinate value at which the value becomes maximum.

Here, the difference absolute value sum and the normalized correlation coefficient are respectively expressed by the following equations.

  The image recording unit 37 includes at least a first image memory 38, a second image memory 39, a first encoding unit 40, a second encoding unit 41, and a storage unit 42.

  The image memories 38 and 39 are memories for temporarily holding an image subjected to predetermined signal processing in the signal processing units 35 and 36, respectively. The first image memory 38 temporarily holds the first image subjected to the predetermined signal processing in the first signal processing unit 35. Similarly, the second image memory 39 temporarily holds the second image subjected to the predetermined signal processing in the second signal processing unit 36.

  The encoding units 40 and 41 are each for encoding an image. The first encoding unit 40 encodes the first image stored in the first image memory 38 and outputs the encoded first image to the storage unit 42. Similarly, the second encoding unit 41 encodes the second image stored in the second image memory 39 and outputs the encoded image to the storage unit 42.

  Next, the process of image quality adjustment in the first embodiment of the present invention will be described. FIG. 6 is a flowchart showing the flow of image quality adjustment processing according to the embodiment of the present invention. Here, the notations Left (*) and Right (*) indicate that the polarized light is transmitted through the left half and the right half of the polarizing filter, respectively. In the following, as an example, the process of adjusting the image quality of the pixel pattern Left (R (i, j)) of R when the parallax component of the second image obtained by the imaging device 10 is a wavelength band component corresponding to G Will be explained. The processes of the other pixel patterns Left (B (i, j), Right (R (i, j)) and Right (B (i, j)) are the same, and thus the description thereof is omitted.

  First, in S51, a captured image is acquired. Although the second polarized light Left (R + 1 / 2Gp + B) is shown as an example here, the description is omitted because the first polarized light Right (R + 1 / 2Gs + B) is similar.

  Next, in S52, the balance of RGB of the image is gain adjusted. The RGB level adjustment can be realized by multiplying the pixel value by a coefficient.

  Next, in S53, the search for the entire image is started in the range of i = 0 to Y-1, j = 0 to X-1 (loop 1). In loop 1, first, at S54, the initial value of the variable min is set.

  Next, in S55, a search for partial areas of the image is started in the range of m = 0 to M-1, n = 0 to N-1 (loop 2).

  In loop 2, first, the sum of squared differences is calculated in S56, and then it is determined in S57 whether the value of the sum of squared differences is the minimum value, and if it is the minimum value, the process proceeds to S58, The coordinate values (m, n) of the pixel of are stored. Otherwise, the process returns to S55.

  The search for the partial area of the image (loop 2) is completed in S59, and then the updated value is stored in New_Left (R (i, j)) in order to update Left (R (i, j)) in S60. Do.

  When the search for the entire image is completed in S61, the processing of loop 1 is completed.

  According to this process, it is possible to obtain coordinate values at which the R pixel pattern and the G pixel pattern included in the image match. Therefore, by moving the pixel pattern of R included in the image to the position of this coordinate value, it is possible to obtain a good image on which the parallax information of R is more reflected.

  Therefore, according to the configuration according to the first embodiment described above, the component of the specific wavelength band respectively selected in the right half surface 21 and the left half surface 22 of the polarization filter among the three primary colors RGB is a parallax component that brings stereoscopic vision and Therefore, an image having binocular parallax is generated on the basis of the first polarized light and the second polarized light obtained by polarizing only the components of the specific wavelength band respectively selected in the right half surface 21 and the left half surface 22 of the polarizing filter, and recording can do.

  Next, the configuration of the stereoscopic imaging device 200 according to the second embodiment of the present invention will be described. Here, FIG. 6 is a block diagram showing a functional configuration of the stereoscopic imaging device 200 according to the second embodiment of the present invention.

  As shown in FIG. 6, a stereoscopic imaging apparatus 200 according to the second embodiment of the present invention includes an imaging unit 31, an image processing unit 34, an image recording unit 37, an image reproduction unit 43, and a display unit 48. Here, since the imaging unit 31, the image processing unit 34, and the image recording unit 37 are the same as those in the first embodiment described above, the description will be omitted.

  The image reproduction unit 43 reads and reproduces the image stored in the image storage unit 42. The image reproduction unit 43 includes at least a first decoding unit 44, a second decoding unit 45, a first display control unit 46, and a second display control unit 47.

  The decoding units 44 and 45 decode the image encoded by the image recording unit 37, respectively. The first decoding unit 44 decodes the first image read from the image storage unit 42 and outputs the decoded first image to the display control unit 46. Similarly, the second decoding unit 45 decodes the second image read from the image storage unit 42 and outputs the decoded second image to the display control unit 47.

  The display control units 46 and 47 perform control for displaying two images having binocular parallax on the display unit 48. Here, the first display control unit 46 performs control for displaying the first image decoded by the first decoding unit 44 on the display unit 48. Similarly, the second display control unit 47 performs control for displaying the second image decoded by the second decoding unit 45 on the display unit 48.

  The display unit 48 synchronizes and displays the first image and the second image output from the image reproduction unit 43. The display unit 48 is, for example, a liquid crystal display device, a projector device, an organic EL display device, or the like.

  Therefore, according to the configuration according to the second embodiment described above, the component of the specific wavelength band respectively selected in the right half surface 21 and the left half surface 22 of the polarization filter among the three primary colors RGB is a parallax component that brings stereoscopic vision and Therefore, an image having binocular parallax is generated on the basis of the first polarized light and the second polarized light obtained by polarizing only the components of the specific wavelength band respectively selected in the right half surface 21 and the left half surface 22 of the polarizing filter. The video can be played back and displayed.

  As described above, according to the embodiment of the present invention, the first polarized light and the second polarized light are obtained by polarizing only the components of the specific wavelength band respectively selected in the right half surface 21 and the left half surface 22 of the polarizing filter. An image having binocular parallax can be generated and recorded, and a stereoscopic video can be reproduced and displayed.

  Further, according to the embodiment of the present invention, for example, the light quantity 25 of the first polarized light 23 which transmits the right half surface 21 of the polarization filter and finally reaches the imaging device 9 is the right half surface 21 of the polarization filter. For the component of the selected specific wavelength band, the product of 1⁄2 of the area ratio of the right half surface 21 of the polarizing filter to 1⁄2 of the polarized light with respect to the light 1 incident on the condenser lens 2 It becomes 1⁄4 and 1⁄2 for components other than the component of the selected specific wavelength band. Therefore, in the case of polarizing only the component of the G wavelength band, the light quantity 25 of the first polarized light 23 which passes through the right half surface 21 of the polarizing filter and finally reaches the image sensor 9 is It becomes 5/12 of the incident light 1.

  On the other hand, in the prior art shown in FIGS. 1 and 2, the light is polarized by a general polarizer having two different polarization characteristics, for example, transmitted through the right half of the polarizing filter and finally reaching the imaging device The amount of polarized light is 1⁄4 which is the product of 1⁄2 of the area ratio of the right half of the polarizing filter and 1⁄2 of one of the polarized lights with respect to the light incident on the condenser lens.

  Therefore, according to the embodiment of the present invention, since the light quantity can be increased to 5/3 times that of the prior art, stereoscopic imaging of bright and good image quality becomes possible.

  Further, according to the embodiment of the present invention, since polarization is performed by the polarization filters 21 and 22 which polarize only the component of the specific wavelength band, it is not necessary to newly add an optical system such as an imaging lens, and the size and weight are reduced. Suitable for

  Therefore, according to the embodiment of the present invention, it is possible to realize small size and light weight as well as bright and favorable stereoscopic imaging and recording and reproduction of stereoscopic video.

  The polarization half mirror 6 in the embodiment of the present invention described above may be a laterally symmetrical polarizer that polarizes the collected light 1 in a linear direction orthogonal to each other. In addition, it may be a left-right symmetric polarizer that polarizes the condensed light 1 in rotational directions opposite to each other.

  In addition, the polarizing half mirror 6 according to the embodiment of the present invention described above is attached to the front surface of the imaging device as a polarizing member to form first polarization or second polarization for each area of the imaging device corresponding to the pixel or pixel group. One of the polarized light may be transmitted. By doing this, in the imaging devices 9 and 10, the first and second polarizations can be selected for each region corresponding to the pixel or the pixel group. Moreover, by doing in this way, the imaging elements 9 and 10 can be comprised by one element. In this case, the missing pixel information can be interpolated and combined with the surrounding pixel information. The above-described pixel group may include a pixel group arranged continuously in the horizontal direction, and may include a pixel group arranged in a rectangular shape.

  Further, the image processing unit 34 according to the embodiment of the present invention described above determines the pattern of the pixel of the component of the remaining wavelength band if the component of the selected specific wavelength band is included at a constant ratio or more. The first and second images may be generated as they are without processing.

That is, when the component of the selected specific wavelength band dominantly contributes to the luminance among the three primary colors RGB of the image, the image quality of the color misregistration which is visually inconspicuous even without the above processing. An image capable of providing sufficient stereoscopic vision can be obtained while including the deterioration of For example, ITU-R BT. According to the standard of 709, the luminance is defined as the following equation. When only the component of the wavelength band corresponding to G is polarized, the contribution rate to the luminance exceeds 71%. The proportion of the component of the selected specific wavelength band that predominantly contributes to the luminance is considered to be 50% or more, which is a sufficient contribution ratio.

  This is because human beings have high contrast sensitivity to luminance patterns in most of the spatial frequency of the image, low contrast sensitivity to chromaticity patterns (see FIG. 7), and are obtained by the imaging device. This is because the parallax due to the two images to be obtained is in principle sufficiently small as the order of blurring of the outline of the image.

  INDUSTRIAL APPLICABILITY The stereoscopic imaging device of the present invention can be widely applied to cameras that are compact and lightweight and acquire bright and favorable stereoscopic images, such as digital cameras, digital video cameras, medical cameras, security cameras, and the like.

  The present invention is not limited to the embodiments described in the specification, and including other examples and modifications can be understood by those skilled in the art to which the present invention belongs from the embodiments and ideas described herein. It is clear to you.

100, 200 Three-dimensional image pickup device 1 Incident light 2 Condensing lens 3 General polarizer (for P wave)
4 Common polarizer (for S wave)
5 Part of the optical path 6 Polarized half mirror 7 Transmits the P wave polarizer, transmits it through the polarized half mirror, and then transmits the component of polarized light that reaches the image sensor (Left) 8 Transmits the S wave polarizer and reaches the polarized half mirror Component of the polarized light that reaches the image sensor (Right) after reflection by the light source 9 Image sensor (Left)
10 Image sensor (Right)
11 light amount of polarized light reaching the image sensor (left) 12 light amount of polarized light reaching the image sensor (Right) 21 wavelength selective polarizer for P wave 22 wavelength selective polarizer for S wave 23 transmissive wavelength selective polarizer for P wave Component of polarized light that reaches the image sensor (Left) after passing through the polarized half mirror, transmits the wavelength selective polarizer for 24 S wave, reflects it by the polarized half mirror, and then reaches the component of polarized light that reaches the image sensor (Right) 25 light quantity of polarized light reaching the image sensor (Left) 26 light quantity of polarized light reaching the image sensor (Right) 31 imaging unit 32 image sensor (Left)
33 Image sensor (Right)
34 Image Processing Unit 35 Signal Processing Unit (Left)
36 Signal processing unit (Right)
37 Image recording unit 38 Storage unit (Left)
39 Storage (Right)
40 Encoding section (Left)
41 Encoding section (Right)
42 Image Recording Unit 43 Image Reproduction Unit 44 Decoding Unit (Left)
45 Decoding unit (Right)
46 Display control unit (Left)
47 Display control unit (Right)
48 Display

Claims (4)

A shooting lens for collecting light from a subject,
A stop for adjusting the amount of the collected light;
In a region where light diffused from one point of the subject becomes parallel light, it is arranged in series in parallel in close proximity to the stop, and the condensed light is selected in the right half plane and the left half plane. A polarized light filter that transmits the first polarized light and the second polarized light that are polarized only in the
A polarization half mirror that transmits the first polarized light and reflects the second polarized light among the first and second polarized lights transmitted through the polarizing filter;
A first imaging element configured to convert a first polarized light transmitted through the polarized half mirror into a first image by an electronic signal;
A first signal processing unit that performs predetermined signal processing on the first image;
A second imaging element configured to convert the second polarized light reflected by the polarized half mirror into a second image by an electronic signal;
A second signal processing unit that performs predetermined signal processing on the second image;
An image recording unit for storing the first and second images subjected to signal processing in the first and second signal processing units;
A three-dimensional imaging apparatus comprising:   The first and second signal processing units adjust the image quality of the first and second images, respectively, based on the pixel pattern of the component of the specific wavelength band selected by the polarization filter. The stereoscopic imaging device according to claim 1. Instead of the polarization half mirror,
A first polarizing member that transmits the first polarized light and does not transmit the second polarized light;
A second polarizing member that does not transmit the first polarized light but transmits the second polarized light;
The three-dimensional imaging device according to any one of claims 1 and 2, further comprising: The stereoscopic imaging device according to any one of claims 1 to 3, wherein the first and second images are stored in the image recording unit.
An image reproduction unit that controls to synchronously display the first and second images stored in the image recording unit;
A display unit for displaying the first and second images output from the image reproduction unit;
And a three-dimensional imaging device.