JP2009278577A5 - - Google Patents

Description

Color imaging device

  The present invention relates to a color imaging apparatus such as a digital camera having a color photographing function.

  2. Description of the Related Art Conventionally, in a color imaging apparatus such as a digital camera, a technique has been used in which a subject image is formed on an imaging element and subjected to photoelectric conversion to generate a color image. At this time, the imaging element is configured such that a plurality of photoelectric conversion elements are arranged in a matrix and an electric signal corresponding to the amount of received light is output for each photoelectric conversion element. In a color imaging device, it is necessary to read a plurality of color components included in imaging light.

On the other hand, in the color imaging device, which imaging element is constituted by one (hereinafter, referred to as single-chip color image pickup device) and, as the image sensor is composed of a plurality (hereinafter, referred to as multi-plate type color image pickup device) There is.

  The single-plate color imaging device includes a plurality of color filters on the front surface of the imaging device, and generates a color image by performing signal processing on the pixel signal that has passed through the color filter.

  In the single-plate color image pickup apparatus, a plurality of photoelectric conversion elements are configured as an image pickup element in a matrix shape, and for example, R (red) G (green) B (blue) are associated with the photoelectric conversion elements on the front surface. Each color filter is provided, and signal processing is performed on a single color image signal output through the color filter to generate a color image (see, for example, Patent Document 1).

  In other words, in an image output through a single-plate image sensor, each pixel is a color mosaic image having only single color information, and signal processing is performed to generate a color image. It is necessary to provide a plurality of pieces of color information such as red (R), green (G), and blue (B).

  Therefore, in color image processing using a single-plate image sensor, other color information that is deficient in each pixel based on a color mosaic image in which each pixel has color information of only one of R, G, and B components. Is interpolated using the color information of other pixels around the pixel to generate a color image in which each pixel has all the R, G, and B component color information (so-called color interpolation processing). is there).

  On the other hand, in the multi-plate color imaging device, a color separation prism for separating the imaging light into RGB color lights is arranged on the optical path, and on the imaging surface of each color light emitted through the color separation prism, An image sensor that receives R color light, an image sensor that receives G color light, and an image sensor that receives B color light are arranged, and a plurality of pixel signals output from the respective image sensors are combined. A color image having color components is generated (see, for example, Patent Document 2).

For example, in a multi-plate color imaging device, a color separation prism is associated with a wavelength of color light by a B (blue) light reflection mirror, an R (red) light reflection mirror, a G (green) light transmission filter, or the like. Thus, the imaging light is imaged on different imaging elements for each of RGB and a pixel signal is generated. Then, RGB color information in one pixel is generated by synthesizing R, G, and B pixel signals output from each image sensor.
JP 2005-286536 A JP-A-7-84177

  According to the conventional single-plate color image pickup device, a color filter is arranged for each pixel of the image pickup device, and other colors that are insufficient in each pixel to obtain a color image signal of one pixel. Since the information is interpolated using the color information of other pixels around the pixel, there is a problem that the resolution is lower than that of the double-plate type color imaging device.

  Further, according to the conventional single-plate color image pickup device, in order to obtain the same resolution as the double-plate color image pickup device, it is necessary to increase the number of pixels of the image pickup element (for example, 2 to 3 times). As the number of pixels of the imaging device increases, the imaging surface widens, and the focal length from the lens to the imaging device becomes longer even at the same angle of view, which makes it difficult to reduce the size in the optical axis direction. .

  On the other hand, according to the conventional double-plate type color image pickup device, the arrangement space of the color separation prism, the optical path length for branching for each RGB through the color separation prism, etc. are configured on the optical axis from the subject image to the image pickup device. Therefore, there is a problem that the size in the optical axis direction is increased and it is difficult to reduce the size. Further, according to the conventional double-plate type color image pickup device, the color separation prism is required to have high accuracy for guiding the image pickup light to the image pickup device, which may impair the productivity.

  Therefore, the present invention does not require a color separation prism, can maintain the same image quality as a double-plate color image pickup device, can be smaller in size in the optical axis direction than the double-plate color image pickup device, and can be reduced in cost. An object of the present invention is to provide a possible color imaging device.

In order to achieve this object, the invention according to claim 1 is characterized in that an imaging optical system that guides subject light to an imaging device and a plurality of photoelectric conversion elements are arranged in a matrix, and the imaging optical system is interposed therebetween. An image sensor that photoelectrically converts the subject light that has been imaged and outputs a pixel signal, wherein the imaging optical system and the image sensor are R (red), G (green), A plurality of B (blue) color lights are provided so as to be paired with each other, and one of the plurality of color lights is transmitted through the light receiving surfaces of the plurality of image sensors or the optical path of the imaging optical system. A plurality of imaging optical systems and imaging elements , wherein the imaging optical system and imaging element that exposes R (red) color light; and an imaging optical system that exposes G (green) color light; Image sensor, imaging optical system for exposing B (blue) color light, and image sensor Each of the plurality of imaging optical systems and the plurality of imaging elements are arranged in parallel, and are arranged so that substantially the same range of the subject is imaged on each of the plurality of imaging elements. Are configured to generate pixel signals of an R (red) image, a G (green) image, and a B (blue) image via each of the imaging elements of Detecting a parallax amount of the R (red) image and detecting a parallax amount of the B (blue) image, and canceling the parallax amount detected via the parallax amount detecting means. A parallax correction unit that corrects the parallax amount by moving the R (red) image and the B (blue) image in a parallel arrangement direction of the plurality of image sensors, and the parallax amount via the parallax correction unit. A color image is generated by synthesizing the R image and the B image with G corrected and the G image. Characterized in that it comprises a color image generation unit, a to.

The color imaging apparatus according to claim 1 includes a plurality of imaging optical systems and imaging elements so as to be paired with R (red), G (green), and B (blue) color lights. On the light receiving surface of the image sensor or on the optical path of the imaging optical system, a color filter that transmits any one of a plurality of color lights is provided, and the plurality of imaging optical systems and the imaging elements are R (red). An imaging optical system and an image sensor for exposing colored light, an imaging optical system and an image sensor for exposing G (green) color light, and an imaging optical system and an image sensor for exposing B (blue) color light And a plurality of imaging optical systems and a plurality of imaging elements are arranged in parallel, and are arranged so that substantially the same range of the subject is imaged on each of the plurality of imaging elements. , R (red) image, G (green) image, and B (blue) image pixel signals are generated. With respect to the G (green) image, the parallax amount detecting unit detects the parallax amount of the R (red) image and detects the parallax amount of the B (blue) image, and the parallax amount detecting unit. A parallax correction unit that corrects the parallax amount by moving the R (red) image and the B (blue) image in a parallel arrangement direction of the plurality of image sensors so as to cancel the detected parallax amount; A color image generation unit that generates a color image by synthesizing the R image and the B image with the parallax amount corrected via the G image, so that a conventional color separation prism is used. Compared to a double-plate imaging device, the size in the optical axis direction can be reduced and the cost can be reduced while maintaining the same image quality.

That is, according to the color imaging device according to claim 1, is provided with the plurality of imaging optical systems and the image pickup device which is to correspond to the RGB respective color lights, a plurality of subject light through the color decomposition prism The structure for branching into the colored light becomes unnecessary, and the size in the optical axis direction can be reduced and the productivity can be improved. According to the color imaging device of claim 1, with respect to the G (green) image, the parallax amount of the R (red) image and the parallax amount of the B (blue) image are detected. A parallax amount detecting unit that moves the R (red) image and the B (blue) image in a parallel arrangement direction of a plurality of image sensors so as to cancel the parallax amount detected through the parallax amount detecting unit. A parallax correction unit that corrects the amount, and a color image generation unit that generates a color image by synthesizing the R and B images and the G image whose parallax amount has been corrected through the parallax correction unit. Therefore, the color shift due to the parallax between the colored lights can be accurately suppressed, and the image quality equivalent to that of the conventional double-plate imaging device can be obtained.

Next, in the color imaging device according to claim 1, as in the invention according to claim 2, the plurality of imaging optical systems include a pupil filter that increases a depth of focus, and the pupil filter includes: A phase filter that gives a predetermined optical phase difference to a subject light incident on the image sensor according to a pupil transmission position, and is designed according to a corresponding color light for each of the plurality of imaging optical systems. As a result, the resolution and the object depth can be improved.

Next, in the color imaging device according to claim 1 or 2, as in the invention according to claim 3, the parallax amount detection unit includes an R (red) image with respect to a G (green) image, and When detecting the parallax amount of the B (blue) image, the R (red) image and the B (blue) image in the direction in which the parallax is corrected within a predetermined number of pixels with the coordinates of the target pixel as the base point And the correlation coefficient between the G image and the R image and the correlation coefficient between the G image and the B image are calculated for each image obtained by shifting, and the correlation coefficient is calculated to be the highest. Since the shift amount is detected as the parallax amount of the R image and the parallax amount of the B image, the color shift due to the parallax between the colored lights can be detected with high accuracy.

Next, the color imaging device according to any one of claims 1 to 3, as in the invention according to claim 4, the parallax correcting unit corrects the parallax amount when the parallax correction unit corrects the parallax amount. Since it is configured to change the amount of movement of the image sensor in the parallel direction according to the distance from the system to the subject, color misalignment is possible even in close-up photography (macro photography) where color misalignment is relatively likely to occur. Can be suppressed.

Next, in the color imaging device according to any one of claims 1 to 4, as in the invention according to claim 5, R (output from the plurality of imaging elements) is output to the color image generation unit. Low-frequency component extracting means for extracting low-frequency components in single-color images of red, G (green), and B (blue), and color-difference generating means for generating a color difference of a color image using the extracted low-frequency components. Are provided, the color shift in the color image can be made inconspicuous.

Next, in the color imaging device according to any one of claims 1 to 5, as in the invention according to claim 6, the color image generation unit receives the G (green) color light. Using a high-frequency component extracting means for extracting a high-frequency component in a monochromatic image output from the element and a high-frequency component extracted by the high-frequency component extracting means, at least one of edge enhancement and noise suppression of the color image is corrected. By providing the image correction unit to be performed, edge enhancement in the high frequency component can be improved and noise can be suppressed.

Next, in the color imaging device according to any one of claims 1 to 6, as in the invention according to claim 7, the color image generation unit extracts the R extracted by the low-frequency component extraction unit. (Red) G (green) B (blue) Low frequency luminance signal generating means for generating a low frequency luminance signal using low frequency components of each color light, and the low frequency generated by the low frequency luminance signal generating means There is no color misregistration in the color image by including luminance combining means for combining the luminance signal and the high frequency component of the G (green) color light generated by the high frequency component extracting means to generate a luminance signal. A high frequency component of luminance can be imparted.

Color image pickup apparatus of the present invention, the imaging optical system and the imaging element, R (red), G (green), B comprises a plurality such that the color light of each pair of (blue), a plurality of imaging elements On the light receiving surface or the optical path of the imaging optical system , each of which includes a color filter that transmits any one of the plurality of colored lights, and the plurality of imaging optical systems and the imaging device emit R (red) colored light. An imaging optical system and an imaging device for exposure, an imaging optical system and an imaging device for exposing G (green) color light, and an imaging optical system and an imaging device for exposing B (blue) color light, A plurality of imaging optical systems and a plurality of image sensors are arranged in parallel, and are arranged so that substantially the same range of the subject is imaged on each of the plurality of image sensors, and through each of the plurality of image sensors, R Configured to generate pixel signals of a (red) image, a G (green) image, and a B (blue) image, Detecting the parallax amount of the R (red) image and the parallax amount detecting means for detecting the parallax amount of the B (blue) image and the parallax amount detecting means for the G (green) image A parallax correction unit that corrects the parallax amount by moving the R (red) image and the B (blue) image in the juxtaposition direction of the plurality of imaging elements so as to cancel the parallax amount, and the parallax correction unit Since it includes a color image generation unit that generates a color image by synthesizing the R image and the B image in which the amount of parallax is corrected, and the G image, a double-plate type using a conventional color separation prism Compared to the imaging device, the size in the optical axis direction can be reduced and the cost can be reduced while maintaining the same image quality.

According to the color imaging device of claim 1, with respect to the G (green) image, the parallax amount of the R (red) image and the parallax amount of the B (blue) image are detected. A parallax amount detecting unit that moves the R (red) image and the B (blue) image in a parallel arrangement direction of a plurality of image sensors so as to cancel the parallax amount detected through the parallax amount detecting unit. A parallax correction unit that corrects the amount, and a color image generation unit that generates a color image by synthesizing the R and B images and the G image whose parallax amount has been corrected through the parallax correction unit. Therefore, the color shift due to the parallax between the colored lights can be accurately suppressed. According to the color imaging device of the first aspect, it is possible to obtain an image quality equivalent to that of a conventional double-plate imaging device provided with an imaging element for each of a plurality of color lights branched via the color separation prism. .

In the color imaging device of the present invention, the plurality of imaging optical systems are provided with a pupil filter that increases the depth of focus, and the pupil filter is predetermined according to the pupil transmission position with respect to the subject light incident on the imaging device. This is a phase filter that provides the optical phase difference, and is designed in accordance with the corresponding color light for each of the plurality of imaging optical systems, so that the resolution and depth of field can be improved.

In the color imaging device of the present invention, when the parallax amount detection unit detects the parallax amount of the R (red) image and the B (blue) image with respect to the G (green) image, the coordinate of the target pixel is detected. The R (red) image and the B (blue) image are shifted by one pixel in the direction of correcting the parallax within a range of a predetermined number of pixels as a base point, and the G image and the R image are shifted for each shifted amount. The correlation coefficient and the correlation coefficient between the G image and the B image are calculated, and the shift amount when the correlation coefficient is calculated to be the highest is detected as the parallax amount of the R image and the parallax amount of the B image. With this configuration, it is possible to accurately detect a color shift due to parallax between colored lights.

In the color imaging apparatus of the present invention, when the parallax correction unit corrects the parallax amount, the movement amount of the imaging elements in the juxtaposed direction is changed according to the distance from the imaging optical system to the subject. By being configured, color misregistration can be suppressed even in close-up photography (macro photography) where color misregistration is relatively likely to occur.

In the color imaging device of the present invention, the color image generating unit extracts a low frequency component in a single color image of R (red), G (green), and B (blue) output from a plurality of imaging elements. By providing the frequency component extracting means and the color difference generating means for generating the color difference of the color image using the extracted low frequency component, the color shift in the color image can be made inconspicuous.

In the color imaging apparatus of the present invention, the color image generation unit includes a high-frequency component extraction unit that extracts a high-frequency component in a single-color image output from an image sensor that receives G (green) color light, and a high-frequency component extraction. The image correction unit that corrects at least one of edge enhancement and noise suppression of the color image using the high frequency component extracted by the means can be provided, and the edge enhancement in the high frequency component can be improved and the noise can be improved. Can be suppressed.

In the color imaging device of the present invention, the color image generation unit uses the low frequency components of the color lights of R (red), G (green), and B (blue) extracted by the low frequency component extraction unit to generate a low frequency. A low frequency luminance signal generating means for generating a luminance signal, a low frequency luminance signal generated by the low frequency luminance signal generating means and a high frequency component of G (green) color light generated by the high frequency component extracting means; By providing the luminance synthesis means for generating the luminance signal, it is possible to give a high-frequency component having luminance without color shift to the color image.

(First embodiment)
Next, a first embodiment of the color imaging apparatus of the present invention will be described with reference to FIGS.

FIG. 1 is a diagram illustrating the configuration of a color imaging apparatus 1A according to the first embodiment of the present invention, where (a) is a configuration diagram of the entire color imaging apparatus, and (b) is Y in (a). It is a block diagram of the semiconductor substrate 5 seen from -Y. FIG. 2 is a configuration diagram of the image processing unit 200 in the color imaging apparatus according to the first embodiment.

  As shown in FIG. 1A, the color imaging device 1A includes color filters 6R and G (green) which are associated with the three primary colors of RGB and transmit only R (red) color light included in the subject light S. A color filter 6G that transmits only color light, a color filter 6B that transmits only B (blue) color light, an imaging optical system 110R that forms an image of R color light transmitted through the color filter 6R on the light receiving unit 5R, and a color filter 6G An imaging optical system 110G that forms an image of the transmitted G color light on the light receiving unit 5G, an imaging optical system 110B that forms an image of the B color light transmitted through the color filter 6B on the light receiving unit 5B, and the imaging optical systems 110R, 110G, A lens barrel 7 that supports 110B, a control unit 11 that controls exposure of the light receiving units 5R, 5G, and 5B, and the like are provided. The light receiving units 5R, 5G, and 5B generate and output an analog electric signal for each pixel according to the detected amount of received light.

In addition, the color imaging device 1A includes variable gain amplifiers PGA120R, PGA120G, which perform sensitivity adjustment and white balance adjustment by varying the gain of the analog electrical signals output from the light receiving units 5R, 5G, and 5B. PGA120B etc. and ADC130R, ADC130G, ADC130B etc. which convert the analog electric signal inputted via PGA120R, PGA120G, and PGA120B into digital image data are provided.

  In addition, the color imaging apparatus 1A includes an image processing unit 200 that generates color image data by combining image data for each RGB output from the ADC 130R, ADC 130G, ADC 130B, and the like, and color image data generated by the image processing unit 200. Is compressed by a method such as JPEG (Joint Photographic Experts Group) or MPEG (Moving Picture Experts Group), and color image data compressed by the compression unit 150 is recorded on a recording medium (for example, a flash memory). And a recording unit 160.

  The functions of the plurality of image sensors in the present invention are expressed by the semiconductor substrate 5 and the light receiving portions 5R, 5G, 5B, which will be described later. In addition, the imaging optical system and the imaging device for exposing R (red) color light in the present invention have their functions expressed by the imaging optical system 110R, the light receiving unit 5R, the semiconductor substrate 5 and the like, and (G) in the present invention. The imaging optical system and the imaging device for exposing the colored light of the imaging optical system 110G, the light receiving unit 5G, the semiconductor substrate 5 and the like exhibit their functions, and the imaging optical for exposing the B (blue) colored light in the present invention. The functions of the system and the imaging device are expressed by the imaging optical system 110B, the light receiving unit 5B, the semiconductor substrate 5, and the like.

  Specifically, the imaging optical system 110R collects the R color light that has passed through the color filter 6R and forms an image on the light receiving unit 5R, and the R color light forms an image on the light receiving unit 5R. A pupil filter 3R that expands the depth of focus is arranged along the optical axis direction.

  The imaging optical system 110G condenses the G color light transmitted through the color filter 6G and forms an image on the light receiving unit 5G, and the depth of focus when the G color light forms an image on the light receiving surface 5G. Is arranged along the optical axis direction.

  The imaging optical system 110B condenses the B color light transmitted through the color filter 6B and forms an image on the light receiving unit 5B, and the depth of focus when the B color light forms an image on the light receiving unit 5B. Is arranged along the optical axis direction.

  In addition, the imaging optical systems 110R, 110G, and 110B are arranged through the light shielding wall 10 so that color light is not mixed with each other.

  Further, the imaging optical systems 110R, 110G, and 110B are arranged so that substantially the same range of the subject is imaged on each of the light receiving portions 5R, 5G, and 5B.

  The imaging optical systems 110R, 110G, and 110B have an imaging optical system 110G and a light receiving unit 5G that correspond to the color light having the highest visibility among color lights such as R, G, and B, and are positioned at the center. The imaging optical system 110R and the light receiving unit 5R, the imaging optical system 110B, the light receiving unit 5B, and the like are arranged on both sides via the optical system 110G and the light receiving unit 5G.

  Further, in the color imaging apparatus 1A, the color filters 6R, 6G, and 6B are provided on the optical paths of the incident light of the imaging optical systems 110R, 110G, and 110B, respectively, and monochromatic light that has passed through the color filters 6R, 6G, and 6B. An image is formed on the light receiving portions 5R, 5G, and 5B via the imaging optical systems 110R, 110G, and 110B.

  In the color imaging device 1A, the imaging optical system 110R, the color filter 6R, the pupil filter 3R, and the light receiving unit 5R are paired to form a camera block that captures an image of R color light, and the imaging optical system 110G, The color filter 6G, the pupil filter 3G, and the light receiving unit 5G are paired to form a camera block that captures an image of G color light, and the imaging optical system 110B, the color filter 6B, the pupil filter 3B, and the light receiving unit 5B are paired. Thus, a camera block that captures an image of B color light is configured.

  Next, as illustrated in FIG. 1B, three light receiving portions 5 </ b> R, 5 </ b> G, and 5 </ b> B are configured on the semiconductor substrate 5.

The light receiving portions 5R, 5G, and 5B are image sensors (for example, image sensors such as CCD and CMOS) configured on the semiconductor substrate 5, and are arranged so that the long sides are close to each other in order to reduce parallax. Each pixel is composed of 1920 × 1080≈2M pixels.

  In the light receiving portions 5R, 5G, and 5B, square pixels each having a side of 1.4 μm are arranged in a matrix. Then, 1920 pixels are arranged in the L direction and 1080 pixels are arranged in the W direction, L = 2.7 mm, W = 1.5 mm, and the gap t between the light receiving portions 5R, 5G, and 5B is 0. 0.1 mm, and the pitch P of the light receiving portions 5R, 5G, 5B is 1.6 mm.

  The imaging optical systems 110R, 110G, and 110B are designed to have a focal length f = 2.1 mm. This is an angle of view equivalent to a focal length f = 28 mm in 35 mm film conversion (commonly known as Leica size conversion).

  The imaging optical systems 110R, 110G, and 110B have the same focal length, but have different wavelengths of colored light that pass through the imaging optical systems 110R, 110G, and 110B. Therefore, the curvature and thickness of the imaging lenses 2R, 2G, 2B, 4R, 4G, and 4B are Each is designed to be optimal in relation to the corresponding color light. Thereby, the imaging optical systems 110R, 110G, and 110B can obtain high imaging performance.

  For example, in a conventional single-plate image pickup device (an image pickup device using a single-plate image pickup device that outputs a color mosaic image), an image quality equivalent to that of a 3M image pickup device of 2M pixels (so-called image quality equivalent to this embodiment) 5M pixels (5M pixels = 3072 × 1728) is needed to obtain 1920 × √ (5M / 2M) ≈3072 in the L direction and 1080 × √ (5M in the W direction) / 2M)) ≈1728. In this case, if a square pixel with a side of 1.4 μm is used, the size of the light receiving unit is 4.3 mm (L) × 2.4 mm (W), and the same angle of view as in this embodiment is provided. The focal length is f = 3.3 mm.

  As a result, the size of the imaging optical system in the optical axis direction is basically proportional to the focal length, so that the imaging optical systems 110R, 110G, and 110B of this embodiment are imaging optical systems of conventional single-plate imaging devices. As compared with the above, the size in the optical axis direction is reduced to about 63% (2.1 / 3.3≈0.63), and the thickness of the color imaging device 1A in the optical axis direction can be reduced.

  Next, pupil filters 3R, 3G, and 3B are installed on the pupil planes in the imaging optical systems 110R, 110G, and 110B in order to improve resolution and subject depth. The pupil filters 3R, 3G, and 3B represent pupil transmission positions (so-called pupil filtering positions (for example, center, outer edge, right, left) that transmit light rays with respect to incident subject light) A phase filter that gives a predetermined optical phase difference in accordance with the position of the transmitted light on the filter surface, and is adapted to the wavelength range of the colored light transmitted through the imaging optical systems 110R, 110G, and 110B, and its effect Is designed for each of them.

  Conventionally, since the phase filter has wavelength dependency, it has been difficult to obtain a sufficient effect of improving the depth of field for visible light having a wide wavelength range of 400 nm to 600 nm. Since the phase filter is designed for each of the imaging optical systems 110R, 110G, and 110B according to the wavelength of the transmitted color light, a sufficient effect can be obtained.

  Next, the control unit 11 is configured with a control circuit that controls and simultaneously exposes the three light receiving units 5R, 5G, and 5B, and converts the monochrome image exposed to each light receiving unit 5R, 5G, and 5B into an analog image. Output as an electrical signal.

  The analog electric signal output from the control unit 11 is subjected to gain adjustment for white balance and sensitivity adjustment in the PGA 120R, PGA 120G, and PGA 120B. After that, the analog electric signal is converted into a digital signal (hereinafter, referred to as a digital signal) Converted to digital image data) and output to the image processing unit 200.

  Next, as shown in FIG. 2, the image processing unit 200 is configured to combine the monochrome R image, G image, and B image output from the ADC 130 R, ADC 130 G, and ADC 130 B and output a color image signal. ing. Note that the color image generation unit of the present invention has its function expressed by the image processing unit 200.

  The image processing unit 200 includes an R parallax amount detection unit 210R that detects the parallax amount of the R image with respect to the G image, a B parallax amount detection unit 210B that detects the parallax amount of the B image with respect to the G image, and the R parallax amount. R parallax correction unit 220R for correcting the B, B parallax correction unit 220B for correcting the parallax amount of B, and the like. Note that the parallax correction means in the present invention exhibits its function by the R parallax correction unit 220R, the B parallax correction unit 220B, and the like.

  The image processing unit 200 also includes a first LPF (low-pass filter) 230R that extracts a low-frequency component in the R image, a second LPF (low-pass filter) 230G that extracts a low-frequency component in the G image, A third LPF (low-pass filter) 230B that extracts a low-frequency component in the B image, an HPF (high-pass filter) 240 that extracts a high-frequency component in the G image, a first LPF 230R, a second LPF 230G, a third And a color conversion unit 250 that generates luminance and color difference (YCbCr) using the low-frequency component output from the LPF 230B.

In addition, the image processing unit 200 uses the high-frequency component extracted by the HPF 240 to perform edge enhancement and noise suppression of a color image, an edge enhancement and noise suppression unit 260, and a luminance composed of low-frequency components generated by the color conversion unit 250. Are combined with a high frequency component extracted by the HPF 240 to generate a luminance , a color correction and γ correction unit 280, and the like.

The function of the low-frequency component extracting means in the present invention is expressed by the first LPF 230R, the second LPF 230G , the third LPF 230B , and the like . Further, the function of the high frequency component extracting means of the present invention is expressed by the HPF 240. Further, the function of the image correction unit in the present invention is expressed by the edge enhancement and noise suppression unit 260. Further, the function of the luminance synthesizing means in the present invention is expressed by the synthesizing unit 270.

  The R parallax amount detection unit 210R detects the parallax amount of the R image at each pixel position (u, v) (u = 0, 1,... 1919, v = 0, 1,..., 1079) in the G image. That is, as shown in FIG. 1, since the R imaging optical system is located on the left side with respect to the G imaging optical system, the parallax amount of the R image is set to the right of the erect image (that is, v Direction). Here, v is a pixel coordinate along the parallel arrangement direction of the light receiving parts 5R, 5G, and 5B, and u is a pixel coordinate in a direction orthogonal to the v direction. In general, the image formed on the light receiving unit is an inverted image, and the parallax amount of the R image is leftward on the light receiving unit in FIG.

  The parallax amount detection processing here is the same as the processing generally referred to as stereo matching, but in this embodiment, the pitch P between the imaging optical system 110R and the imaging optical system 110G is small (1.6 mm). ), The detection range can be reduced, and arithmetic processing can be reduced.

  Further, since the optical axes of the imaging optical systems 110R, 110G, and 110B are configured in parallel, the parallax on the imaging plane is 0 (zero) when the subject distance is ∞.

  On the other hand, when the subject distance is 0.6 m, the parallax on the imaging plane is 1.6 mm × 2.1 mm (= f) /0.6 m = 5.6 μm, and between the G image and the R image, A parallax equivalent to 4 pixels occurs between the G image and the B image in terms of the number of pixels.

  Accordingly, if the shortest shooting distance of the color imaging device 1A is 0.6 m, the R image is shifted in the direction of correcting the parallax (to the left in FIG. 1) in the range of 0 to 4 pixels, and the coordinates of the pixel of interest ( In the vicinity of u, v), the deviation amount k when the correlation between the two images (the R image and the G image) becomes the highest is output as the parallax amount Dr. If the optical axes of the imaging optical systems 110R, 110G, and 110B intersect 1.2m ahead, the parallax is 0 (zero) when the subject distance is 1.2m, and the subject distance is ∞. Is 2 pixels in the negative direction of v, and 2 pixels in the positive direction of v when the subject distance is 0.6 m.

At this time, for example, the product-moment correlation coefficient is obtained using the arithmetic expression of (Expression 1), and the degree of correlation can be detected.

In (Expression 1), <X (u, v)> is the average value of X in the vicinity of (u, v) <X (u, v)> = mean _{| (} u ′, v _{′) − (u , V) | <ω} X (u ′ _{, v ′)} . Further, ω is the number of pixels corresponding to the distance from the pixel of interest to the pixel considered as the vicinity, and ω = 2 to 3 is preferable, but may be appropriately changed according to the noise level of the image.

  That is, the degree of correlation between the image R (u, v) and the image G (u ′, v ′) is expressed in the range of | (u ′, v ′) − (u, v) | <ω (the image R and It is calculated from a relational expression of (covariance of image G) / ((standard deviation of image R) × (standard deviation of image G)).

The deviation amount k when the absolute value of the correlation in (Equation 1) is maximized is set as the parallax amount Dr between the R image and the G image as in (Equation 2).

Further, in the equation (2) has been calculated amount of parallax by using a pixel value Monono correlation that the R and G, a correlation between the differential value ∂R / ∂v and ∂g / ∂v of R and G maximum The amount of deviation at this time may be the amount of parallax.

  Next, the R parallax correction unit 220R translates the R image in a direction to cancel the parallax amount Dr detected by the R parallax amount detection unit 210R, and converts the corrected image R ′ into R ′ (u, v) = R ( u, v + Dr (u, v)). Thereafter, the corrected image R ′ output from the R parallax correction unit 220R is used by the color conversion unit 250 via the first LPF 230R.

  Next, the B parallax amount detection unit 210B detects the parallax amount of the B image at each pixel position (u, v) (u = 0, 1,... 1919, v = 0, 1,... 1079) in the G image. . That is, as shown in FIG. 1, since the B imaging optical system is arranged on the right side with respect to the G imaging optical system, the parallax amount of the B image is set to the left of the erect image (that is, v Negative direction).

  In the parallax amount detection process in the B parallax amount detection unit 210B, the pitch P between the imaging optical system 110B and the imaging optical system 110G is very small (1.6 mm), as in the R parallax amount detection unit 210R. Therefore, the detection range can be reduced and the calculation processing can be reduced.

  Specifically, the B image is shifted to the right in FIG. 1 in the range of 0 to −4 pixels, and the correlation between the two images (the B image and the G image) is the highest in the vicinity of the coordinate (u, v) of the target pixel. The shift amount k when it becomes higher is output as the parallax amount Db.

At this time, for example, a product-moment correlation coefficient can be used as a correlation index as shown in (Expression 3).

Next, the B parallax correction unit 220B translates the B image in a direction to cancel the parallax amount Db detected by the B parallax amount detection unit 210B, and converts the corrected image B ′ into B ′ (u, v) = B ( u, v + Db (u, v)). Thereafter, the corrected image B ′ output from the B parallax correction unit 220B is used by the color conversion unit 250 via the third LPF 230B .

  Next, the G monochrome image output from the ADC 130G is separated into a high frequency component and a low frequency component by the second LPF 230G and the HPF 240.

  At this time, the second LPF 230G preferably has the same configuration as the first LPF 230R and the third LPF 230B. Further, HPF 240 and LPF 230G are preferably complementary (configured so as not to extract so-called common frequency components).

As such a filter, a kernel spatial filter expressed by (Expression 4) is applied to the first LPF 230R, the second LPF 230G, and the third LPF 230B, and the HPF 240 is expressed by (Expression 5). A kernel spatial filter may be applied.

  Next, the color conversion unit 250 applies a predetermined color conversion matrix to the low-frequency components output from the first LPF 230R, the second LPF 230G, the third LPF 230B, and the like, and based on the luminance and color difference signals of YCrCb. A color image is generated. In the color image signal described above, Y is the luminance, Cr is the R color difference with respect to Y, and Cb is the B color difference with respect to Y.

  On the other hand, the high-frequency component of the G image output from the HPF 240 is input to the edge enhancement and noise suppression unit 260, and the edge enhancement and noise suppression unit 260 suppresses a component regarded as noise among the high-frequency components and Amplifies components that are considered as edges.

For example, when a high frequency component is expressed as _GH , if _GH (u, v) at a certain pixel coordinate (u, v) is equal to or smaller than a predetermined value σ, it is regarded as noise, and is a value exceeding the predetermined value σ. For example, the edge component is amplified by using (Equation 6). In (Expression 6), _GH ′ is an edge component obtained by amplifying _GH . P is an edge enhancement coefficient, and generally 3 to 5 is preferable.

  Next, the synthesis unit 270 synthesizes the high-frequency component output from the edge enhancement and noise suppression unit 260 with the luminance signal Y of the color image output from the color conversion unit 250. At this time, since the color difference CrCb does not include a high frequency component, even if a correction error occurs in the R parallax correction unit 210R and the B parallax correction unit 210B, the influence of the error can be reduced, and a high-quality color image can be obtained. It is done.

Next, the color image data output from the synthesis unit 270 is subjected to various corrections such as skin color correction and γ correction in the color correction and γ correction unit 280, and then output to the compression unit 150 and compressed by the compression unit 150. Is done. The color image data compressed by the compression unit 150 is recorded in a recording medium such as a flash memory in the recording unit 160 .

  As described above, in the color imaging device 1A described in the first embodiment, the imaging optical systems 110R, 110G, and 110B and the light receiving units 5R, 5G, and 5B are arranged in parallel in association with RGB color lights. Since the light receiving units 5R, 5G, and 5B are arranged so that substantially the same range of the subject is imaged, the same image quality is maintained as compared with the conventional multi-plate imaging device using the color separation prism. However, the size in the optical axis direction can be reduced and the cost can be reduced.

  Further, in the color imaging device 1A described in the first embodiment, the light receiving units 5R, 5G, and 5B are installed on the same semiconductor substrate 5 so as to be close to each other, so that the influence of parallax can be reduced.

  In addition, the color imaging device 1A described in the first embodiment has an image quality equivalent to that of a conventional single-plate imaging device (an imaging device using a single-plate imaging device that outputs a color mosaic image). In addition, the area of each of the light receiving portions 5R, 5G, and 5B can be reduced, the focal length of each imaging optical system 110R, 110G, and 110B can be shortened, and the thickness of the color imaging device 1A in the optical axis direction can be reduced.

(Second Embodiment)
Next, a second embodiment of the color imaging device of the present invention will be described with reference to FIGS.

FIG. 3 is a diagram illustrating the configuration of the color imaging apparatus 1B according to the second embodiment of the present invention, in which (a) is a configuration diagram of the entire color imaging apparatus, and (b) is Y in (a). It is a block diagram of the semiconductor substrate 31 seen from -Y. FIG. 4 is a configuration diagram of the image processing unit 400 in the color imaging apparatus according to the first embodiment.

  Note that the color imaging device 1B in the second embodiment is basically the same in configuration as the color imaging device 1A shown in the first embodiment, and therefore, common components are assigned the same reference numerals and detailed description is given. The description will be omitted and the characteristic parts will be described below.

  As illustrated in FIG. 3A, the color imaging device 1B includes an imaging optical system 111R that is associated with the three primary colors RGB and forms an image of the subject light S on the imaging device 30 via the color filter 25R, and a color filter. An imaging optical system 111G that forms an image of the subject light S on the image sensor 30 through 25G, an imaging optical system 111B that forms an image of the subject light S on the image sensor 30 through the color filter 25B, and the subject light S. A color filter 25R that transmits only R (red) color light, a color filter 25G that transmits only G (green) color light, a color filter 25B that transmits only B (blue) color light, and the like are provided. These imaging optical systems 111R, 111G, and 111B are inverted optical systems.

  The imaging optical systems 111R, 111G, and 111B are configured by laminating a plurality of optical plates 22, 23, and 24, and spherical optical surfaces 22R and 22G are formed on the upper surfaces and boundary surfaces of the optical plates 22, 23, and 24, respectively. , 22B.

  The optical plates 22, 23, and 24 are covered with light-shielding layers 26, 27, and 28 except for the transmission portions of the imaging optical systems 111 </ b> R, 111 </ b> G, and 111 </ b> B.

As shown in FIG. 3B, the image sensor 30 is an image sensor such as a CCD or CMOS in which a plurality of photoelectric conversion elements constituting pixels are arranged in a matrix on the upper surface of a semiconductor substrate 31. Three rectangular exposure areas 30r, 30g, and 30b defined by the guard band 30n are arranged in the main scanning direction so as not to overlap.

  Then, only the R color light transmitted through the color filter 25R forms an image on the exposure area 30r, only the G color light transmitted through the color filter 25G forms an image on the exposure area 30g, and the exposure area 30b includes Only the B color light that has passed through the color filter 25B forms an image.

  The exposure areas 30r, 30g, and 30b are arranged so that the long sides are adjacent to each other in order to reduce the parallax, and further, the color light having the highest visibility among the RGB color lights is imaged. A light receiving area 30g is arranged at the center.

In addition, the imaging optical systems 111R, 111G, and 111B are arranged in parallel with their optical axes so that substantially the same range of the subject is imaged in each of the exposure areas 30r, 30g, and 30b .

  The color imaging device 1B includes an imaging optical system 111R, a color filter 25R, and a camera block that captures an image of R color light with the exposure area 30r as a pair, and includes an imaging optical system 111G, a color filter 25G, and an exposure. The area 30g is provided with a camera block that captures a G color light image, and the imaging optical system 111B, the color filter 25B, and the exposure area 30b are paired to capture a B color light image. ing.

  At this time, as in the first embodiment, the imaging optical systems 111R, 111G, 111B, etc. have the same focal length, but the wavelengths of the colored lights that pass through the optical surfaces 22R, 22G, 22B are different. These curvatures may be designed to be optimal in accordance with the transmitted color light. However, in the second embodiment, as will be described later, since the high-frequency components of the R image and the B image output from the image sensor 30 are not used in the image processing unit 400, the image is formed in the exposure area 30r and the exposure area 30b. The image-forming optical systems 111R, 111G, and 111B may be designed to be optimal for the G color light without regard to blurring of the R and B images.

  In the color image pickup apparatus 1B of the present embodiment, the color filters 25R, 25G, 25B, the plurality of optical plates 22, 23, 24, the light shielding layers 26, 27, 28, etc. are formed without gaps on the semiconductor substrate 31 on which the image pickup device 30 is formed. By stacking, alignment (position adjustment) at the time of assembly is simplified.

  Further, the color imaging device 1B of the present embodiment covers the light shielding layers 26, 27, 28, etc. on the boundaries of the optical plates 22, 23, 24 except for the optical surfaces 22R, 22G, 22B, etc. The light shielding layers 26, 27, 28, etc. act as a stop ring to absorb stray light. The light shielding layers 26, 27, and 28 are formed on the surfaces of the optical plates 22, 23, and 24, for example, by etching or the like.

  Thereby, even if there is no light shielding wall (reference numeral 10 in FIG. 1 of the first embodiment) between the imaging optical systems 111R, 111G, and 111B, light is not mixed between these camera blocks.

  Next, as shown in FIG. 3B, the exposure areas 30r, 30g, and 30b have 480 pixels in the horizontal direction (main scanning direction) and 640 pixels in the vertical direction (direction orthogonal to the main scanning direction), respectively. And Further, the guard band 30n between the exposure areas 30r, 30g, and 30b is 80 pixels, and the image sensor 30 is composed of 1600 pixels (480 + 80 + 480 + 80 + 480 = 1600) pixels in the horizontal direction and 640 pixels in the vertical direction.

Further, if the pitch between pixels to 2.2 .mu.m, the size of the imaging device, the vertical direction is 2.2 .mu.m × 640 ≒ 1.4 mm, lateral becomes 2.2μm × 1600 ≒ 3.5mm. Further, the distance (pitch) between the light receiving unit systems is 2.2 μm × (480 + 80) ≈1.2 mm. The interval between the optical axes of the imaging optical systems 111R, 111G, 111B, etc. is also set to 1.2 mm, similar to the pitch.

  On the other hand, the focal length in the imaging optical systems 111R, 111G, and 111B is f = 1.4 mm. This is an angle of view equivalent to a focal length f = 35 mm in terms of 35 mm film.

  Further, since the optical axes of the imaging optical systems 111R, 111G, and 111B are configured in parallel, when the subject distance is ∞, the parallax on the imaging plane is 0 (zero).

On the other hand, when the subject distance is 0.75 m, the parallax on the imaging plane is 1.2 mm × 1.4 mm (= f) /0.75 m≈2.2 μm, and between the G image and the R image, A 1-pixel parallax is generated between the G image and the B image in terms of the number of pixels.

  When the subject distance is 0.38 m, a parallax of 2 pixels in terms of the number of pixels is generated between the G image and the R image, and between the G image and the B image. At 19 m, a 4-pixel parallax in terms of the number of pixels occurs between the G image and the R image, and between the G image and the B image.

These parallax amounts are corrected in the image processing unit 400 described later by switching the tele / macro selector switch 170 according to the subject distance. In the color imaging device 1B of the present embodiment, the switch is set to the tele side when the subject distance is 0.4 m or more, and the switch is set to the macro side when the subject distance is around 0.3 m. It is configured to switch to. The function of the parallax amount detection means in the present invention is expressed by the tele / macro changeover switch 170 that operates according to the subject distance.

The image sensor 30 outputs an analog electric signal as one image in which the R image, the G image, and the B image received through the three exposure areas are arranged, and the analog electric signal is input to the ADC 131. Is done.

  Next, the ADC 131 converts the analog electric signal input from the image sensor 30 into digital image data and outputs the digital image data to the image processing unit 400.

  The image processing unit 400 converts the digital image data input from the ADC 131 into a color image signal and outputs the color image signal to the compression unit 150. Note that the function of the color image generation unit in the present invention is expressed by the image processing unit 400.

  The compression unit 150 performs compression such as JPEG or MPEG on the color image signal input from the image processing unit 400 and outputs the compressed signal to the output unit 161.

  The output unit 161 transmits the compressed data input from the compression unit 150 to an external device (not shown) connected to the color imaging device 1B. Examples of the external device include image / video devices such as a remote monitor display device, an image / video recording device, and an image / video processing device.

  Next, as shown in FIG. 4, the image processing unit 400 is configured to synthesize a monochrome R image, G image, and B image input from the ADC 131 and output a color image signal.

  Specifically, the image processing unit 400 extracts an R region in the input image input via the ADC 131, an R region extracting unit 410R that extracts an R region, a G region extracting unit 410G that extracts a G region, and B that extracts a B region. An area extraction unit 410B is provided.

  Further, the image processing unit 400 amplifies the R image data extracted by the R region extraction unit 410R, an R gain multiplication device 415R, and the G gain multiplication device amplifies the G image data extracted by the G region extraction unit 410G. 415G, a B gain multiplier 415B that amplifies the B image data extracted by the B region extraction unit 410B, an R parallax correction unit 420R that corrects the parallax amount of the R image with respect to the G image, and a parallax amount of the B image with respect to the G image A B parallax correction unit 420B for correction is provided. Note that the parallax correction means in the present invention has its function expressed by the R parallax correction unit 420R, the B parallax correction unit 420B, and the like.

  The image processing unit 400 also includes first LPFs (low pass filters) 430R that extract low frequency components in the R image, and second LPFs (low pass filters) 430G and B that extract low frequency components in the G image. Output from a third LPF (low-pass filter) 430B that extracts a low-frequency component in the image, HPF (high-pass filter) 440 that extracts a high-frequency component in the G image, the first LPF 430R, the second LPF 430G, and the third LPF 430B And a color conversion unit 250 that generates luminance and color difference (YCbCr) using the low frequency components for each of RGB.

In addition, the image processing unit 400 uses the high-frequency component extracted by the HPF 440 to perform edge enhancement and noise suppression of a color image, an edge enhancement and noise suppression unit 260, and a luminance composed of low-frequency components generated by the color conversion unit 250. Are combined with a high frequency component extracted by the HPF 440 to generate a luminance, a color correction and γ correction unit 280, and the like. The low-frequency component extraction means of the present invention, first LPF430R, second LPF430G, thus its function to a third LPF430B like are expressed. Further, the function of the high-frequency component extracting means of the present invention is expressed by the HPF 440.

  Specifically, the R region extraction unit 410R extracts a region corresponding to the R exposure area 30r from the input image input via the ADC 131, and outputs the image data of this region to the R gain multiplier 415R.

  The R gain multiplier 415R is implemented as a multiplier, and multiplies the R image signal input from the R region extraction unit 410R by values related to corrections such as sensitivity correction, white balance, and shading, and performs R parallax correction. To the unit 420R.

  The R parallax correction unit 420R refers to the tele / macro selector switch 170, translates the R image by 1 pixel to the opposite side in the main scanning direction in the inverted image state if the tele side, and the R image if the macro side. Is corrected so as to translate 3 pixels to the opposite side in the main scanning direction.

  That is, before the correction by the R parallax correction unit 420R, if the tele side (the subject distance is in the range of ∞ to 0.4 m), the R image captured by the R camera block is captured by the G camera block. The image is shifted by 0 to 2 pixels on the main scanning direction side in FIG. Therefore, by shifting the R image by 1 pixel in the reverse direction in the main scanning direction, the deviation of the R image from the G image is corrected within ± 1 pixel.

  On the other hand, before correction by the R parallax correction unit 420R, on the macro side (the subject distance is less than 0.4 m to 0.2 m), the R image captured by the R camera block is the G camera block. The captured G image is shifted by 2 to 4 pixels on the main scanning direction side in FIG. Therefore, the shift of the R image with respect to the G image is corrected to be within ± 1 pixel by translating the R image to the opposite side in the main scanning direction by 3 pixels.

  As a result, when the subject distance is in the range of ∞ to 0.2 m, the R parallax correction unit 420R corrects the deviation of the R image with respect to the G image so as to be always within a range of ± 1 pixel. The image is output to the first LPF 430R.

  Next, the B region extraction unit 410B extracts a region corresponding to the B exposure area 30B from the input image input via the ADC 131, and outputs the region to the B gain multiplier 415B.

  The B gain multiplier 415B, which is implemented as a multiplier, multiplies the B image signal input from the B region extraction unit 410B by values related to corrections such as sensitivity correction, white balance, and shading, and performs B parallax correction. To the unit 420B.

  The B parallax correction unit 420B refers to the tele / macro changeover switch, and translates the B image by 1 pixel in the main scanning direction in the inverted image state if the tele side, and moves the B image in the main scanning direction if the macro side. Correct to move 3 pixels in parallel.

  In other words, before the correction by the B parallax amount correction unit 420B, if it is on the tele side (the subject distance is in the range of ∞ to 0.4 mm), the R image captured by the B camera block is captured by the B camera block. The G image is shifted by 0 to 2 pixels on the opposite side in the main scanning direction in FIG. Therefore, by shifting the B image by 1 pixel in the main scanning direction, the deviation of the B image from the G image is corrected within 1 pixel.

  On the other hand, before the correction by the B parallax amount correction unit 420B, on the macro side (the subject distance is in the range of less than 0.4 m to 0.2 m), the B image captured by the B camera block is the G camera block. The G image taken in step 2 is shifted by 2 to 4 pixels on the opposite side in the main scanning direction. Therefore, by shifting the B image by 3 pixels in the main scanning direction in FIG. 3B, the deviation of the B image with respect to the G image is corrected within 1 pixel.

Accordingly, when the subject distance is ∞ to 0.2 m, the B parallax amount correction unit 420B always corrects the deviation of the B image from the G image within a range of ± 1 pixel, and the corrected B image is It is output to the third LPF 430B .

  The G region extraction unit 410G extracts a region corresponding to the G exposure area 30G from the input image input via the ADC 131, and outputs the region to the G gain multiplier 415G.

  The G gain multiplier 415G is implemented as a multiplier, and multiplies the G image signal input from the G region extraction unit 410G by values related to corrections such as sensitivity correction, white balance, and shading, and the second Output to LPF430G and HPF440B.

Next, in the R and B images output from the R parallax correction unit 420R and the B parallax correction unit 420B, the deviation from the G image is corrected within ± 1 pixel. In order to make this deviation inconspicuous. , first LPF430R, second LPF430G, depending on the third LPF430B like, low-frequency filter is applied.

That is, each image shift occurs only in the direction in which the imaging optical systems 111R, 111G, and 111B are juxtaposed. For example, the kernel spatial filter expressed by (Equation 7) is applied, and the imaging optical systems 111R, 111G, What is necessary is just to remove the high frequency of the parallel direction of 111B. The column in (Expression 7) corresponds to the parallel direction of the imaging optical system.

  Next, the color conversion unit 250 applies a predetermined color conversion matrix to the low-frequency components output from the first LPF 430R, the second LPF 430G, the third LPF 430B, and the like, and based on the YCrCb luminance and color difference signals. A color image is generated.

  Next, since the high-frequency component removed by the LPF is not included in the color image signal generated by the color conversion unit 250, the high-frequency component is generated using the HPF 440.

  When generating high-frequency components in the HPF 440, only a single color light image is used so as not to be affected by a shift caused by the plurality of imaging optical systems 111R, 111G, 111B and the like. As the color light at this time, the G image photographed by the G imaging optical system 111G having the highest visual sensitivity among the plurality of imaging optical systems 111R, 111G, 111B, etc. is optimal. .

Therefore, the HPF 440 extracts a high-frequency component from the G image input via the G region extraction unit 410G and the G gain multiplier 415G. At this time, the HPF 440 serves to compensate for the low-frequency components removed by the first LPF 430R, the second LPF 430G, and the third LPF 430B, and the components in the juxtaposed direction of the imaging optical systems 111R, 111G, and 111B, etc. The kernel spatial filter expressed in (Equation 8) may be applied so that The column in (Equation 8) corresponds to the parallel arrangement direction of the imaging optical systems 111R, 111G, and 111B.

  Next, as in the color imaging device 1A of the first embodiment, the edge enhancement and noise suppression unit 260 uses the high-frequency component input from the HPF 440 to suppress the component regarded as noise among the high-frequency components. In addition, the components regarded as the edges of the image are amplified, and then, in the synthesis unit 270, the high frequency component output from the edge enhancement and noise suppression unit 260 is converted into the luminance signal Y of the color image output from the color conversion unit 250. After that, the color correction and γ correction unit 280 performs various corrections such as skin color correction and γ correction, and outputs the result to the compression unit 150.

As described above, the color image pickup apparatus 1B according to the second embodiment, the imaging optical system 111R by simple and convenient circuit configuration, 111G, it is possible to correct the disparity between 111B, produce good color image quality it can.

  The color imaging device 1B described in the second embodiment has a plurality of camera blocks by stacking the color filters 25R, 25G, 25B, the optical plates 22, 23, 24, etc. on a single semiconductor substrate 31. Since it is configured, alignment (position adjustment) during assembly can be simplified, productivity can be improved, and cost can be reduced.

  Further, the color imaging device 1B described in the second embodiment has a thinner thickness in the optical axis direction than a conventional single-plate imaging device (an imaging device using a single-plate imaging device that outputs a color mosaic image). Can reduce the thickness of the optical plates 22, 23, 24, etc., thereby reducing the resources of the optical material and suppressing the decrease in the transmittance of the subject light that passes through the optical plates 22, 23, 24, etc. Can do.

  As mentioned above, although one Embodiment of this invention was described, this invention is not limited to the said embodiment, A various aspect can be taken.

1A and 1B are diagrams illustrating a configuration of a color imaging device according to a first embodiment of the present invention, where FIG. 1A is a configuration diagram of the entire color imaging device, and FIG. 2B is a view taken from YY in FIG. It is a block diagram of a semiconductor substrate. It is a block diagram of the image process part in the color imaging device of the 1st embodiment. 4A and 4B are diagrams illustrating a configuration of a color imaging device according to a second embodiment of the present invention, where FIG. 5A is a configuration diagram of the entire color imaging device, and FIG. 5B is a view taken from YY in FIG. It is a block diagram of a semiconductor substrate. It is a block diagram of the image process part in the color imaging device of the said 2nd Embodiment.

  1A, 1B ... Color imaging device, 2R, 2G, 2B, 4R, 4G, 4B ... Imaging lens, 3R, 3G, 3B ... Pupil filter, 5, 31 ... Semiconductor substrate, 5R, 5G, 5B ... Light receiving unit, 6R, 6G, 6B, 25R, 25G, 25B ... color filter, 7 ... lens barrel, 10 ... light shielding wall, 11 ... control section, 22, 23, 24 ... optical plate, 26-28 ... light shielding layer, 30 ... imaging device, 110R , 110G, 110B, 111R, 111G, 111B ... imaging optical system, 30r, 30g, 30b ... exposure area, 30n ... guard band, 120R, 120G, 120B ... variable gain amplifier (PGA), 130R, 130G , 130B, 131 ... A / D converter (ADC: Analog Digital Converter), 2 DESCRIPTION OF SYMBOLS 0,400 ... Image processing part, 150 ... Compression part, 160 ... Recording part, 161 ... Output part, 170 ... Tele macro switch, 210R ... R parallax amount detection part, 210B ... B parallax amount detection part, 220R, 420R ... R parallax correction unit, 220B, 420B ... B parallax correction unit, 230R, 430R ... first LPF, 230G, 430G ... second LPF, 230B, 430B ... third LPF, 240, 440 ... HPF, 250 ... Color conversion unit, 260 ... edge enhancement and noise suppression unit, 270 ... composition unit, 280 ... color correction and γ correction unit, 410R ... R region extraction unit, 410G ... G region extraction unit, 410B ... B region extraction unit, 415R ... R gain multiplier, 415G... G gain multiplier, 415B... B gain multiplier.

Claims (7)

An imaging optical system that guides subject light to the image sensor and a plurality of photoelectric conversion elements are arranged in a matrix, and the subject light imaged through the imaging optical system is photoelectrically converted to output a pixel signal. In a color imaging device comprising an element,
A plurality of the imaging optical system and the imaging device are provided so as to be paired with R (red), G (green), and B (blue) color lights, respectively, and the light receiving surfaces of the plurality of imaging devices or the connection are provided. A color filter that transmits any one of the plurality of color lights on the optical path of the image optical system;
The plurality of imaging optical systems and imaging devices are an imaging optical system and imaging device that exposes R (red) color light, an imaging optical system and imaging device that exposes G (green) color light, and B ( It is composed of an imaging optical system that exposes (blue) colored light and an image sensor,
The plurality of imaging optical systems and the plurality of imaging elements are arranged in parallel, and are arranged so that substantially the same range of the subject is imaged on each of the plurality of imaging elements. To generate pixel signals of an R (red) image, a G (green) image, and a B (blue) image,
Parallax amount detection means for detecting the parallax amount of the R (red) image and detecting the parallax amount of the B (blue) image with respect to the G (green) image;
The R (red) image and the B (blue) image are moved in the juxtaposition direction of the plurality of image sensors so as to cancel the parallax amount detected through the parallax amount detecting means, and the parallax amount is corrected. Parallax correction means for
A color image generation unit that generates a color image by combining the R image and the B image in which the amount of parallax has been corrected through the parallax correction unit and the G image;
A color imaging apparatus comprising: The plurality of imaging optical systems are provided with pupil filters that increase the depth of focus,
The pupil filter is a phase filter that gives a predetermined optical phase difference to subject light incident on the imaging element according to a pupil transmission position, and converts the color light corresponding to each of the plurality of imaging optical systems. Designed according to the
The color imaging apparatus according to claim 1. When the parallax amount detecting means detects the parallax amount of the R (red) image and the B (blue) image with respect to the G (green) image, a range of a predetermined number of pixels based on the coordinates of the target pixel Then, the R (red) image and the B (blue) image are shifted by one pixel in the direction in which the parallax is corrected, and the correlation coefficient between the G image and the R image and the G image are obtained for each of the shifted images. Calculating a correlation coefficient between the image B and the B image, and detecting a shift amount when the correlation coefficient is calculated to be the highest as a parallax amount of the R image and a parallax amount of the B image,
The color imaging apparatus according to claim 1, wherein the color imaging apparatus is a color imaging apparatus. The parallax correction unit is configured to change the amount of movement of the imaging elements in the juxtaposition direction according to the distance from the imaging optical system to the subject when correcting the parallax amount .
The color imaging device according to claim 1, wherein the color imaging device is a color imaging device. The color image generation unit includes a low-frequency component extraction unit that extracts a low-frequency component in each monochrome image of R (red), G (green), and B (blue) output from the plurality of image sensors, and the extraction A color difference generation means for generating a color difference of a color image using the low frequency component that is made,
The color imaging device according to claim 1, wherein the color imaging device is a color imaging device. In the color image generator,
High-frequency component extraction means for extracting a high-frequency component in a monochromatic image output from an image sensor that has received the G (green) color light;
An image correction unit that performs correction of at least one of edge enhancement and noise suppression of a color image using the high-frequency component extracted by the high-frequency component extraction unit;
The color imaging device according to claim 1, wherein the color imaging device is a color imaging device. In the color image generator,
Low-frequency luminance signal generating means for generating a low-frequency luminance signal using the low-frequency components of each color light of R (red) G (green) B (blue) extracted by the low-frequency component extracting means;
Luminance combining means for generating a luminance signal by combining the low frequency luminance signal generated by the low frequency luminance signal generating means and the high frequency component of G (green) color light generated by the high frequency component extracting means; Provided,
The color imaging apparatus according to claim 6.