JP2006279389A - Solid-state imaging apparatus and signal processing method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a solid-state imaging apparatus for applying signal processing to an image picked up with an imaging element provided with four-color filters and to provide a signal processing method thereof. <P>SOLUTION: When an imaging section 22 provided with the four-color filters carries out imaging to produce respective pixel data including red data, blue data, and first green data, and second green data and when a digital signal processing section 30 applies signal processing to each of the pixel data, a color reproduction region discrimination section 58 discriminates whether the characteristic emphasized by interpolation processing of the pixels is a color reproduction region or a resolution on the basis of the first and second green data of each of the pixel data to output the pixel interpolation characteristic of each pixel, and an interpolation processing section 60 applies interpolation processing to each of the pixel data by determining the color reproduction region or the resolution on the basis of the pixel interpolation characteristic of each pixel so that the solid-state imaging apparatus 10 can obtain an image with high resolution as required. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a solid-state image pickup device that performs signal processing on an image picked up by an image pickup device having a four-color filter, and a signal processing method thereof.

  2. Description of the Related Art Conventionally, there are some solid-state imaging devices that attempt to satisfy router characteristics by generating an image having a wider color reproducibility by using four color filters of an imaging element. Such an image sensor includes, for example, a second green filter having a negative characteristic of a red filter, for example, an emerald green color filter, in addition to a filter that transmits the three primary colors RGB.

  For example, in the image processing apparatus described in Patent Document 1, as a four-color filter, an R filter that transmits only red light, a B filter that transmits only blue light, and only green light in the first wavelength band are transmitted. And a G2 filter having a high correlation with the G1 filter and transmitting only green light in the second wavelength band as a minimum pixel unit, and an image sensor based on incident light transmitted through these filters Generates four types of signals, and generates RGB signals by performing signal processing based on these four types of signals.

JP2003-284084

  However, when using an image sensor with a four-color filter, as in the image processing apparatus described in Patent Document 1, the color reproducibility of the captured image is improved, but through a green filter indicating one of the three primary colors. Since the obtained signal is halved, an image with low resolution is obtained.

  The present invention eliminates the disadvantages of the prior art and provides a solid-state imaging device capable of obtaining a high-resolution luminance signal without lowering the resolution of the obtained image even when an imaging device having a four-color filter is used. An object of the present invention is to provide a signal processing method.

  According to the present invention, the first color data indicating the first color component corresponding to the pixel data of each pixel, the first color data, and the first color data corresponding to the pixel data of each pixel are obtained by color-separating the incident light, photoelectrically converting the color separated incident light Imaging means for generating second color data indicating the second color component, third color data indicating the third color component, and fourth color data indicating the fourth color component, and first, second, In the solid-state imaging device including the signal processing means for performing signal processing on the third and fourth color data, the signal processing means determines, based on the pixel data, important characteristics that are important in the interpolation processing of each pixel. A determination means for outputting a determination result, and a pixel interpolation characteristic of each pixel is obtained according to the determination result, and the first, second, third and fourth color data of each pixel are obtained based on the pixel interpolation characteristic. Interpolation processing means for performing interpolation processing is included.

  In addition, the first color data indicating the first color component corresponding to the pixel data of each pixel by color-separating the incident light, photoelectrically converting the color-separated incident light and capturing an image in the imaging unit, When the second color data indicating the second color component, the third color data indicating the third color component, and the fourth color data indicating the fourth color component are generated, the first and second The signal processing method for performing signal processing on the third and fourth color data includes a determination step of determining an important characteristic important in interpolation processing of each pixel based on the pixel data and outputting a determination result, and the determination An interpolation processing step of obtaining a pixel interpolation characteristic of each pixel according to the result and interpolating the first, second, third, and fourth color data of each pixel based on the pixel interpolation characteristic. Features.

  As described above, according to the solid-state imaging device of the present invention, based on the first and second green data of each pixel, it is determined whether the interpolation process of the pixel should focus on the color gamut or resolution, By performing an interpolation process based on the determination result, an image with higher resolution can be obtained as necessary.

  Next, an embodiment of a solid-state imaging device according to the present invention will be described in detail with reference to the accompanying drawings.

  As shown in FIG. 1, the solid-state imaging device 10 of the embodiment takes in incident light from an object scene in the optical system 12 and operates the operation unit 14 to operate the system control unit 16, the optical system driving unit 18, and the imaging. The drive unit 20 controls each part, and the imaging unit 22 having a four-color filter captures the scene image. The captured image with high color reproducibility is converted into an analog signal processing unit 24 and analog / digital ( Analog / Digital: A / D) A digital image signal is generated by the converter 26. The memory 28, the digital signal processing unit 30, the compression / decompression processing unit 32, the integration unit 34, and the recording unit are connected to the data bus 42. The digital image signal is processed by the 36 and the display unit 38. Note that portions not directly related to understanding the present invention are not shown and redundant description is avoided.

  The optical system 12 includes a lens, an aperture adjustment mechanism, a shutter mechanism, a zoom mechanism, an automatic focus (AF) adjustment mechanism, an automatic exposure (AE) adjustment mechanism, and the like, although a specific configuration is not illustrated. However, an infrared ray (IR) cut filter or an optical low pass filter (LPF) may be included.

  The optical system 12 of the present embodiment is controlled according to the drive signal 112 from the optical system drive unit 18, and the lens, the aperture adjustment mechanism, and the zoom mechanism are driven to capture a desired object scene image, and the image pickup unit This is a light incident mechanism that enters 22. In the optical system 12, the AE adjustment mechanism and the AF adjustment mechanism may be driven according to the drive signal 112. In the following description, each signal is specified by the reference number of the connecting line in which it appears.

  The operation unit 14 is a manual operation device that inputs an operator's instruction, and supplies an operation signal 102 to the system control unit 16 in accordance with an operator's manual operation state, for example, a stroke operation of a shutter button (not shown). Has the function of

  The system control unit 16 is a control function unit that controls and supervises the overall operation of the apparatus in response to the operation signal 102 supplied from the operation unit 12. For example, a central processing unit (CPU) You may have.

  The system control unit 16 of the present embodiment supplies, for example, control signals 104, 106, and 108 generated according to the operation signal 102 to the optical system driving unit 18, the imaging driving unit 20, and the analog signal processing unit 24, respectively. The generated control signal 110 is supplied to the control bus 40 to control each unit connected to the control bus 40.

  Although a specific configuration is not illustrated, the imaging unit 22 includes an imaging surface that forms one screen of a captured image, and captures an object scene image that is formed on the imaging surface via the optical system 12. It has a function of performing photoelectric conversion to an analog electric signal 116 in accordance with a drive signal 114 from 20. The imaging unit 22 of the present embodiment may be any image sensor such as a charge coupled device (CCD) or a metal oxide semiconductor (MOS).

  The imaging surface of the imaging unit 22 is formed by arranging a plurality of photosensitive units, and each photosensitive unit is provided with a color filter to color-separate incident light and photoelectrically convert the incident light into an electrical signal. It is formed including an optical sensor. In this embodiment, the plurality of photosensitive portions include a red photosensitive portion having a red filter that transmits only red light indicating one of the three primary colors, and a blue filter that transmits only blue light indicating one of the three primary colors. A blue light-sensitive portion, a first green light-sensitive portion having a first green filter that transmits only green light in the first wavelength band, and a second wavelength band that has a high correlation with the first green filter. A second green photosensitive portion having a second green filter that transmits only the light of the first green. For example, the first green filter transmits only green light indicating one of the three primary colors, and the second green filter transmits only light of the emerald green color having the negative characteristics of the red filter. Things can be used.

  The imaging unit 22 is intended to constitute one pixel based on the red photosensitive unit, the blue photosensitive unit, the first green photosensitive unit, and the second green photosensitive unit. The red photosensitive unit, the blue photosensitive unit, the first photosensitive unit, Generating red data, blue data, first green data, and second green data from each of the green photosensitive portion and the second green photosensitive portion, and obtaining one pixel data based on these color data To do. In addition, the imaging unit 22 has a plurality of red photosensitive units, a blue photosensitive unit, a first green photosensitive unit, and a second green photosensitive unit in order to obtain a plurality of pixel data, and generates a plurality of color data. What to do.

  Further, in this embodiment, although the plurality of photosensitive portions are not shown, when they are green photosensitive portions regardless of whether they are the first green photosensitive portion or the second green photosensitive portion, they are fixed pitches in the row direction and the column direction, respectively. Are arranged in a square matrix, the green photosensitive parts are arranged in a stripe pattern, and the red photosensitive part and the blue photosensitive part are arranged in a checkered pattern between them, so-called G stripe RB complete checkered pattern, row direction and Using a honeycomb arrangement in which every other position is shifted in the row direction, the green photosensitive parts are arranged in a square lattice, and further, the red photosensitive part or the blue photosensitive part is arranged diagonally across the green photosensitive part. The so-called honeycomb-type G square lattice RB complete checkered pattern arranged in a perfect checkered pattern, or a green photosensitive part is arranged in a checkered pattern, and the red photosensitive part and the blue photosensitive part are surrounded by the green photosensitive part, Each row and each column of the matrix It is arranged to include one of the red-sensitive portion and a blue-sensitive unit may be arranged in an array pattern such as Bayer (Bayer) pattern. Here, when considering whether the green photosensitive portion is the first green photosensitive portion or the second green photosensitive portion, the first green photosensitive portion corresponds to the red photosensitive portion and the second green photosensitive portion in a predetermined pixel. May be arranged so as to correspond to the blue photosensitive part, or the first green photosensitive part corresponds to the blue photosensitive part and the second green photosensitive part corresponds to the red photosensitive part. Also good.

  The analog signal processing unit 24 is controlled by the control signal 108 from the system control unit 16 and has a function of performing analog signal processing on the analog electrical signal 116 from the imaging unit 22, for example, a correlated double sampling circuit (Correlated Analog signal processing may be executed by a circuit such as Double Sampling (CDS) and a gain controlled amplifier (GCA), and as a result, an analog image signal 118 may be generated and output.

  The A / D converter 26 performs an analog / digital conversion process on the analog image signal 118 from the analog signal processing unit 24 to generate a digital image signal 120 and supplies the digital image signal 120 to each unit via the data bus 42. The A / D converter 26 according to the present embodiment supplies the generated digital image signal 120 to the integration unit 34 at the time of shooting preparation, and to the digital signal processing unit 30 at the time of main shooting, regardless of the type of shooting. Instead, it may be supplied to the memory 28 and stored.

  The memory 28 is a storage device that receives and temporarily accumulates the digital image signal 122 processed by each unit connected to the data bus 42 via the data bus 42.

  The digital signal processing unit 30 of the present embodiment is controlled by the system control unit 16 through the control bus 40 and performs digital signal processing on the digital image signal 124 input from the A / D converter 26 through the data bus 42. The resulting digital image signal 124 may be supplied to the memory 28 or the compression / decompression processor 32 via the data bus 42.

  In the present embodiment, as shown in FIG. 2, the digital signal processing unit 30 inputs a digital image signal 124, and performs an offset correction processing unit 52, a white balance (WB) correction processing unit 54, and a vertical direction simultaneously. After the processing by the conversion processing unit 56, the color gamut determining unit 58 determines whether a wide color gamut is necessary, and the interpolation processing unit 60 performs an interpolation process according to the determination result. Processing is performed by a linear matrix (LMTX) processing unit 62, a gamma (γ) correction processing unit 64, a YC conversion unit 66, and the like.

  The offset correction processing unit 52 corrects a deviation from the reference voltage level for the input digital image signal 124 and outputs the resulting digital image signal 140. The WB correction processing unit 54 Is subjected to WB correction processing, and the resulting digital image signal 142 is output.

  The vertical direction synchronization processing unit 56 synchronizes the WB-corrected digital image signal 142 for each color data, and as a result, the red data 144, the first green data 146, and the second green data 148 for each pixel. The blue data 150 is output to the color gamut determining unit 58.

  The color gamut determination unit 58 determines, in units of pixels, whether the interpolation process for a predetermined pixel should be emphasized in terms of the characteristics of the color gamut or resolution. The characteristic is determined according to the first green data 146 and the second green data 148, while the red data 144, the first green data 146, the second green data 148 and the blue data 150 are directly subjected to the interpolation processing unit. You may output to 60.

  When the color reproduction area determination unit 58 of the present embodiment determines a predetermined pixel, the comparison unit 68 determines the first green data 146 and the second green data 148 of the pixel as predetermined, as shown in FIG. Is compared with the threshold value 178, and based on the comparison result, the resolution or the color reproduction range is determined as the importance characteristic, and the determination result 152 indicating the importance characteristic of each pixel as the pixel interpolation characteristic of each pixel is output.

  For example, if the absolute value of the difference between the first green data 144 and the second green data 146 is greater than the predetermined threshold 178 as a result of the comparison by the comparison unit 68, the color reproduction area determination unit 58 determines the color reproduction area. In other cases, the determination is made as the priority characteristic, and in other cases, the resolution is determined as the priority characteristic, and a determination result 152 indicating the determined importance characteristic as the pixel interpolation characteristic of the pixel is output.

  When the comparison in the comparison unit 68 is conceptualized, as shown in FIG. 4, in the graph showing the relationship between the first green data 146 and the second green data 148, the predetermined threshold 178 is between the lines 180 and 182. It is shown in the range. When the second green data 148 with respect to the first green data 146 in a predetermined pixel is between the lines 180 and 182, the color gamut determining unit 58 determines that the important characteristic of the pixel is resolution, and Otherwise, it is determined as a color gamut.

  The interpolation processing unit 60 obtains pixel interpolation characteristics of each pixel according to the determination result 152 from the color gamut determination unit 58, and interpolates each pixel with emphasis on the color gamut or resolution indicated by the pixel interpolation characteristic. For example, the red data 144, the first green data 146, the second green data 148 and the blue data 150 in a predetermined pixel are interpolated, and the resulting red data 154 and first green data 156 are obtained. The second green data 158 and the blue data 160 are output to the LMTX processing unit 62.

  When the pixel interpolation characteristic of a predetermined pixel indicates a color gamut according to the determination result 152, the interpolation processing unit 60 according to the present embodiment uses an interpolation filter 70 for emphasizing the color gamut as shown in FIG. When the color data is interpolated and the resolution is indicated, the red data 146 and the blue data 150 are interpolated by the interpolation filter 70 used for emphasizing the color gamut, and the first green data 146 and As shown in FIG. 6, the second green data 148 is interpolated by a resolution-oriented interpolation filter 72 having a wider LPF band than the interpolation filter 70.

  As shown in FIGS. 5 and 6, the interpolation filters 70 and 72 have values 0, 1, 2, and 4, but in practice, an interpolation filter with 1/4 of this value is used. Good.

  The LMTX processing unit 62 performs matrix calculation on the four color data of each pixel to improve color reproducibility and perform color correction to generate three color data. For example, the LMTX processing unit 62 according to the present embodiment includes (3 × 4 matrix) for the red data 154, the first green data 156, the second green data 158, and the blue data 160 in a predetermined pixel. A matrix operation based on a predetermined matrix coefficient is performed to generate red data 162, green data 164, and blue data 166 and output them to the γ correction processing unit 64.

  The γ correction processing unit 64 γ corrects the red data 162, the green data 164, and the blue data 166 of each pixel according to the gradation characteristics of the imaging unit 22, and the resulting red data 168, green data 170, and blue data 172. Is output to the YC converter 66.

  The YC conversion unit 66 converts the red data 168, the green data 170, and the blue data 172 of each pixel indicated by the three primary colors RGB into luminance (Y) data 174 and color difference (Cr, Cb) data 176.

  The digital signal processing unit 30 of the present embodiment may supply the luminance data 174 and the color difference data 176 resulting from the YC conversion unit 66 to the compression / decompression processing unit 32 via the data bus 42 as the digital image signal 124.

  The compression / decompression processing unit 32 receives a digital image signal 126 from the digital signal processing unit 30 via the data bus 42, performs compression processing and supplies the digital image signal 126 to the recording unit 36 via the data bus 42, The digital image signal 126 is input via the data bus 42, and decompressed and supplied to the memory 28 via the data bus 42.

  The integration unit 34 has a function of calculating an evaluation value for AE adjustment or automatic WB adjustment. In this practical example, the integration unit 34 is based on the digital image signal 128 input from the A / D converter 26 via the data bus 42. Thus, the evaluation value may be calculated.

  The recording unit 36 has a function of recording the digital image signal 130 input from the compression / decompression processing unit 32 or the like via the data bus 42. For example, the recording unit 36 is a rotational recording of a memory card or a magneto-optical disk mounted with a semiconductor memory. An information recording medium using a package containing a body may be included, and the information recording medium may be detachable.

  The display unit 38 has a function of displaying an image based on the digital image signal 132 input from the digital signal processing unit 30 or the like via the data bus 42. For example, a liquid crystal display (LCD) panel or the like is used. It is done.

  Next, in the solid-state imaging device 10 of this embodiment, the operation in the digital signal processing unit 30 will be described with reference to the flowchart of FIG. Here, first, the digital image signal 120 that has been analog-digital converted by the A / D converter 26 by imaging by the apparatus 10 is supplied to the digital signal processing unit 30 as the digital image signal 124 via the data bus 42. The

  In the digital signal processing unit 30, the digital image signal 124 is processed by the offset correction processing unit 52, the WB correction processing unit 54, and the vertical direction synchronization processing unit 56. As a result, the red data 144 of each pixel, the first data The green data 146, the second green data 148 and the blue data 150 are supplied to the color gamut determination unit 58.

  In the color gamut determining unit 58, the importance characteristics of each pixel are determined. For example, when determination for a predetermined pixel is started (S202), first, the comparing unit 68 first compares the first green data 144 and the second green data of the pixel. The absolute value of the difference between the green data 146 is compared with a predetermined threshold value 178 (S204). As a result, when the absolute value is larger than the predetermined threshold value 178, the importance characteristic of the pixel is determined to be a color gamut and the process proceeds to step S206. Otherwise, the resolution is determined and the process proceeds to step S208.

  In step S206, the flag is set to 0 so that the pixel interpolation characteristic of the pixel determined in step S204 indicates a color gamut, and in step S208, the flag is set to 1 to indicate resolution.

  In this way, the determination of the predetermined pixel is completed (S210), and a flag indicating the pixel interpolation characteristics of all the pixels is output to the interpolation processing unit 60 as the determination result 152.

  Next, in the interpolation processing unit 60, based on the determination result 152, the red data 144, the first green data 146, the second green data 148, and the blue data 150 of each pixel are subjected to interpolation processing. At this time, if the flag indicating the pixel interpolation characteristic of the predetermined pixel indicates 0 in the determination result 152, each color data of the pixel is subjected to interpolation processing by the interpolation filter 70 that emphasizes the color gamut. On the other hand, when the flag indicating the pixel interpolation characteristic of the pixel indicates 1, the red data 144 and the blue data 150 of the pixel are interpolated by the interpolation filter 70 that emphasizes the color gamut, but the first green data 146 The second green data 148 is interpolated by the resolution-oriented interpolation filter 72.

  The red data 154, the first green data 156, the second green data 158 and the blue data 160 of each pixel subjected to interpolation processing in this way are processed by the LMTX processing unit 62 to generate color data of three colors. Thereafter, the signal is processed by the γ correction processing unit 64 and the YC conversion unit 66 to finish the signal processing.

  In this way, the digital image signal 124 subjected to the signal processing by the digital signal processing unit 30 is supplied to the memory 28 and the compression / decompression processing unit 32 via the data bus 42.

  As another embodiment, as shown in FIG. 8, the solid-state imaging device 10 determines, in image units, which characteristic of color reproduction area or resolution should be emphasized in the interpolation processing of a predetermined pixel. The color reproduction area determination unit 80 is included, and the interpolation processing unit 82 performs interpolation processing according to the determination result 310 of the color reproduction area determination unit 80 for each image.

  In this embodiment, as shown in FIG. 8, the color gamut determining unit 80 is included in the integrating unit 34 and is input from the A / D converter 26 via the data bus 42 at the time of shooting preparation or the like. A determination is made based on the digital image signal 128. The color gamut determining unit 80 may be included in the digital signal processing unit 30, or may be directly connected to the data bus 42 independently.

  The color gamut determining unit 80 sequentially determines importance characteristics suitable for a predetermined pixel for all pixels of one image, and as a result, out of the importance characteristics of each pixel, the importance characteristics suitable for the entire image are determined for all pixels. The determination result 310 indicated as the interpolation characteristic is output to the interpolation processing unit 82.

  For example, the color gamut determining unit 80 compares the absolute value of the difference between the first green data 146 and the second green data 148 in each pixel with a first threshold value, and this absolute value is greater than the first threshold value. In this case, the number of pixels, that is, the number of pixels whose important characteristic is the color gamut, and the number of pixels in other cases, that is, the number of pixels whose important characteristic is the resolution are obtained. Further, the difference between the number of pixels is compared with the second threshold value. When the difference is larger than the second threshold value, the determination result 310 indicating the color gamut as the pixel interpolation characteristics of all the pixels is displayed. In other cases, the resolution is displayed. The determination result 310 is output. This second threshold may be determined according to the difference in spectral intensity between the first green filter and the second green filter.

  When this comparison is conceptualized, as shown in FIG. 9, in the graph showing the relationship between the first green data 146 and the second green data 148, the first threshold 312 is in the range between the lines 314 and 316. Indicated. The color gamut determining unit 80 compares the difference between the number of the second green data 148 with respect to the first green data 146 between the lines 314 and 316 and the other number with the second threshold 318. In accordance with the comparison result, the pixel interpolation characteristic of the image is determined as the resolution or the color reproduction range, and the determination result 310 is output.

  The interpolation processing unit 82 interpolates each pixel with emphasis on the color gamut or resolution in accordance with the determination result 310 in units of images from the color gamut determination unit 80, and the determination result 310 is a pixel. When the color reproduction gamut is shown as the interpolation characteristic, the color reproduction gamut-oriented interpolation filter 70 is used, and when the resolution is shown, the resolution-oriented interpolation filter 72 is used, and the red data 144 of each pixel, the first Green data 146, second green data 148 and blue data 150 are respectively interpolated to generate red data 302, first green data 304, second green data 306 and blue data 308, and γ correction processing Output to part 64.

  Next, in the solid-state imaging device 10 of this embodiment, the operation in the integrating unit 34 will be described with reference to the flowchart of FIG. Here, first, the digital image signal 120 that has been analog-digital converted by the A / D converter 26 by imaging by the apparatus 10 is supplied to the integrating unit 34 as the digital image signal 128 via the data bus 42.

  In the integration unit 34, the digital image signal 124 is integrated with its color data to calculate an evaluation value for AE adjustment or automatic WB adjustment, and is also supplied to the color reproduction range determination unit 80. .

  The color gamut determining unit 58 starts determining important characteristics suitable for each image (S352). First, a counter that counts the number of pixels whose color characteristics are the color gamut and the number of pixels that are resolutions in one image, respectively. e and g are reset to 0 (S354).

  Next, importance characteristic determination is sequentially performed on all pixels in one image, and first, it is confirmed whether importance characteristic determination for all pixels is completed (S356). If there is still a pixel for which the emphasis characteristic is determined, the process proceeds to step S358. On the other hand, if the emphasis characteristic determination for all the pixels has been completed, the process proceeds to step S366.

  In step S358, the comparison value D1 is calculated from the absolute value of the difference between the first green data 146 and the second green data 148 of the pixel for which the important characteristic is determined, and for example, the first green data 146 and the second green data 146 are calculated. When the data 148 is G1 and G2, respectively, the comparison value D1 is calculated by the formula D1 = | G1-G2 |.

  Next, the process proceeds to step S360, where the comparison value D1 is compared with the first threshold value TH1, and if the comparison value D1 is greater than the first threshold value TH1, it is determined that the weighted characteristic of the pixel is the color gamut. The process proceeds to S362, and otherwise, it is determined that the resolution is set, and the process proceeds to Step S364.

  In step S362, the pixel is counted and added to the counter e of the number of pixels whose importance characteristic is the color gamut, and in step S364, the pixel is counted in the counter g of the number of pixels whose importance characteristic is the resolution. Add one.

  When the processes in steps S362 and S364 are completed, the process returns to step S356 again, and thus the determination of the importance characteristic is repeated for all the pixels.

  In step S366, the counters e and g of the number of pixels whose emphasis characteristics are the color gamut and resolution are compared with the second threshold TH2, for example, when the comparison value D2 is calculated with D2 = eg, If the comparison value D2 is greater than the second threshold TH2, it is determined that the pixel interpolation characteristics of all the pixels of the image are the color gamut, and the process proceeds to step S368. Otherwise, the resolution is determined to be the resolution. Proceed to S370.

  In step S368, the flag is set to 0 as the determination result 310 so that the pixel interpolation characteristic of the image determined in step S366 is the color gamut, and in step S370, the flag is set to 1 to indicate the resolution. .

  In this way, the determination of the importance characteristic suitable for the image is completed (S372), and the determination result 310 is output to the interpolation processing unit 80.

  Incidentally, in the interpolation processing unit 80, based on the determination result 310, the red data 144, the first green data 146, the second green data 148, and the blue data 150 of each pixel are subjected to interpolation processing. At this time, when the flag indicates 0 as the determination result 310, each color data of all pixels in one image is subjected to interpolation processing by the interpolation filter 70 that emphasizes the color reproduction range. On the other hand, when the flag of the determination result 310 indicates 1, for all pixels in one image, the red data 144 and the blue data 150 are interpolated by the interpolation filter 70 that emphasizes the color gamut, but the first green data 146 The second green data 148 is interpolated by the resolution-oriented interpolation filter 72.

  Further, the red data 302, the first green data 304, the second green data 306, and the blue data 308 of each pixel subjected to the interpolation processing in the interpolation processing unit 80 are processed by the LMTX processing unit 62 to obtain three colors. Are then processed by the γ correction processing unit 64 and the YC conversion unit 66, and the signal processing is completed.

  Further, in the solid-state imaging device 10 of the present embodiment, the importance characteristics in the interpolation processing are determined in units of pixels or images. For example, one image is divided into a plurality of blocks, and the plurality of pixels in each block are regarded as the importance characteristics. It is also possible to determine the importance characteristics for each block by determining.

It is a block diagram which shows one Example of the solid-state imaging device which concerns on this invention. It is a block diagram shown in detail about the signal processing part which the solid-state imaging device of the Example shown in FIG. 1 has. FIG. 3 is a block diagram illustrating in detail a color gamut determining unit included in the signal processing unit illustrated in FIG. 2. It is the graph which showed notionally the comparison of the 1st and 2nd green data in the color reproduction range determination part shown in FIG. FIG. 3 is an example diagram of an interpolation filter for emphasizing a color gamut used in an interpolation processing unit included in the signal processing unit shown in FIG. 2. FIG. 3 is a diagram illustrating an example of a resolution-oriented interpolation filter used in an interpolation processing unit included in the signal processing unit illustrated in FIG. 2. It is a flowchart which shows operation | movement of the signal processing part which the solid-state imaging device of the Example shown in FIG. 1 has. It is a block diagram which shows the other Example of the signal processing part which the solid-state imaging device concerning this invention has. It is the graph which showed notionally the comparison of the 1st and 2nd green data in the color reproduction range determination part which the signal processing part shown in FIG. 8 has. 10 is a flowchart illustrating an operation of a color reproduction area determination unit illustrated in FIG. 9.

Explanation of symbols

10 Solid-state imaging device
12 Optical system
14 Operation unit
16 System controller
18 Optical system drive
20 Imaging drive unit
22 Imaging unit
24 Analog signal processor
26 A / D converter
28 memory
30 Digital signal processor
32 Compression / decompression processor
34 Integration unit
36 Recording section
38 Display
40 control bus
42 Data bus

Claims (14)

The incident light is color-separated, the incident light subjected to the color separation is photoelectrically converted and imaged, and first color data indicating the first color component corresponding to the pixel data of each pixel and second color component are indicated. Imaging means for generating second color data, third color data indicating a third color component, and fourth color data indicating a fourth color component;
In a solid-state imaging device including signal processing means for performing signal processing on the first, second, third, and fourth color data,
The signal processing means is configured to determine an important characteristic to be emphasized in the interpolation processing of each pixel based on the pixel data and output a determination result;
Interpolation processing means for obtaining a pixel interpolation characteristic of each pixel according to the determination result and interpolating the first, second, third and fourth color data of each pixel based on the pixel interpolation characteristic A solid-state imaging device.   2. The solid-state imaging device according to claim 1, wherein the first color component is a red component, the second color component is a blue component, the third color component is a first green component in the first wavelength band, and a fourth color component. The color component is a second green component that transmits only green light in the second wavelength band having a high correlation with the first green component.   3. The solid-state imaging device according to claim 1, wherein the determination unit determines an absolute value of a difference between the third and fourth color data of the predetermined pixel when determining the importance characteristic of the predetermined pixel. If the absolute value is larger than the first threshold, the priority characteristic is determined as a color gamut, and in other cases, the priority characteristic is determined as resolution according to the comparison result. A solid-state imaging device.   4. The solid-state imaging device according to claim 3, wherein when the pixel interpolation characteristic of the predetermined pixel corresponding to the determination result is a color gamut, the interpolation processing unit performs the first, second, When the third and fourth color data are interpolated by a color gamut interpolation filter, and the pixel interpolation characteristic is resolution, the first and second color data of the predetermined pixel are converted to the color gamut interpolation filter. A solid-state imaging device, wherein the third and fourth color data of the predetermined pixel are interpolated by a resolution interpolation filter. 5. The solid-state imaging device according to claim 1, wherein the determination unit outputs the determination result indicating, in pixel units, the importance characteristics of each pixel as the pixel interpolation characteristics of each pixel, respectively.
The interpolation processing unit is configured to interpolate the first, second, third, and fourth color data of a predetermined pixel based on the pixel interpolation characteristic corresponding to the predetermined pixel according to the determination result. A solid-state imaging device characterized by performing interpolation processing. 5. The solid-state imaging device according to claim 1, wherein the determination unit indicates the importance characteristic suitable for the entire image among the importance characteristics of each pixel as the pixel interpolation characteristic of all pixels in units of images. Output the result,
The interpolation processing means performs interpolation processing based on pixel interpolation characteristics of all the pixels according to the determination result when interpolating the first, second, third and fourth color data of a predetermined pixel. A solid-state imaging device characterized by being applied.   The solid-state imaging device according to claim 6, wherein the determination unit uses a second threshold value as a first pixel number having the importance characteristic as a color gamut and a second pixel number having the resolution as the importance characteristic. As a result, if the value obtained by subtracting the second number of pixels from the first number of pixels is larger than the second threshold value, the pixel interpolation characteristic of all the pixels is set as a color gamut, otherwise, A solid-state imaging device characterized in that the pixel interpolation characteristic of all the pixels is defined as a resolution. First color data indicating a first color component corresponding to pixel data of each pixel, second color by performing color separation on incident light, photoelectrically converting the color-separated incident light, and capturing an image with an imaging unit When the second color data indicating the second color component, the third color data indicating the third color component, and the fourth color data indicating the fourth color component are generated, the first, second, second In a signal processing method for signal processing the third and fourth color data, the method includes:
A determination step of determining an importance characteristic to be emphasized in the interpolation processing of each pixel based on the pixel data and outputting a determination result;
An interpolation processing step of obtaining a pixel interpolation characteristic of each pixel according to the determination result, and interpolating the first, second, third, and fourth color data of each pixel based on the pixel interpolation characteristic. And a signal processing method.   9. The signal processing method according to claim 8, wherein the first color component is a red component, the second color component is a blue component, the third color component is a first green component in the first wavelength band, and a fourth color component. The signal component is a second green component that transmits only green light in the second wavelength band having a high correlation with the first green component.   10. The signal processing method according to claim 8, wherein the determination step determines an absolute value of a difference between the third and fourth color data of the predetermined pixel when determining the importance characteristic of the predetermined pixel. If the absolute value is larger than the first threshold, the priority characteristic is determined as a color gamut, and in other cases, the priority characteristic is determined as resolution according to the comparison result. And a signal processing method.   The signal processing method according to claim 10, wherein the interpolation processing step includes the first, second, and second of the predetermined pixel when the pixel interpolation characteristic of the predetermined pixel corresponding to the determination result is a color gamut. When the third and fourth color data are interpolated by a color gamut interpolation filter, and the pixel interpolation characteristic is resolution, the first and second color data of the predetermined pixel are converted to the color gamut interpolation filter. The signal processing method is characterized in that the third and fourth color data of the predetermined pixel are interpolated by a resolution interpolation filter. 12. The signal processing method according to claim 8, wherein the determination step outputs the determination result indicating, in pixel units, the importance characteristics of each pixel as the pixel interpolation characteristics of each pixel, respectively.
The interpolation processing step is based on the pixel interpolation characteristic corresponding to the predetermined pixel according to the determination result when the first, second, third and fourth color data of the predetermined pixel are interpolated. And a signal processing method characterized by performing an interpolation process. 12. The signal processing method according to claim 8, wherein the determination step indicates the importance characteristic suitable for the entire image among the importance characteristics of each pixel as the pixel interpolation characteristic of all pixels in image units. Output the result,
In the interpolation processing step, when the first, second, third and fourth color data of a predetermined pixel are subjected to interpolation processing, interpolation processing is performed based on pixel interpolation characteristics of all the pixels according to the determination result. And a signal processing method. 14. The signal processing method according to claim 13, wherein the determination step compares the first number of pixels having the important characteristic as a color gamut and the second number of pixels having the important characteristic as a resolution with a second threshold value. As a result, when the value obtained by subtracting the second number of pixels from the first number of pixels is larger than the second threshold value, the pixel interpolation characteristic of all the pixels is set as a color gamut, and in other cases, A signal processing method, wherein the pixel interpolation characteristic of all pixels is a resolution.

Images