JP4098438B2 - Color imaging device and color imaging device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a color image pickup device capable of preventing the occurrence of color moire and a color image pickup apparatus using the color image pickup device.
[0002]
[Prior art]
In general, image pickup devices represented by an image pickup tube and a solid-state image pickup device are widely used in image pickup apparatuses. In particular, a color imaging device used in a single tube or single sensor (Single Sensor) color imaging device can form a color imaging device with a single imaging device, so that a color separation prism is not required and the lens can be miniaturized. In addition, it has many features such as no need for multi-plate type adjustments represented by registration, low power consumption, and has contributed much to the miniaturization and power consumption of color imaging devices. Single-plate color cameras using a color CCD image sensor, which is a solid-state image sensor, have become the mainstream of imaging devices.
[0003]
Each of the color imaging devices performs color modulation (color coding) in the light receiving plane using a color filter called a stripe filter or a mosaic filter in order to obtain color information on one light receiving surface. That is, for example, by applying RGB three-color filters on each photoelectric conversion element (pixel) in a predetermined regular arrangement, each pixel has different spectral sensitivity. Therefore, since the video signal obtained by subject imaging includes dot-sequential color information according to this filter arrangement, it is separated for each signal corresponding to each color filter according to the predetermined arrangement, Color information can be extracted by extracting the separated signal. Since all RGB information is required to obtain a luminance signal (Y signal), at least three pixels (one for each RGB) are required to obtain luminance information for one pixel, and the luminance resolution is sacrificed. Color imaging can be performed with one imaging element.
[0004]
A variety of color coding patterns such as three primary color filters such as RGB stripes and Bayer type RGB mosaics (various types), complementary color filters such as YeMgCy stripes, YeMgCyW four-color mosaics, and YeMgCyG4 color mosaics are proposed and used in the filter arrangement. It has become.
[0005]
[Problems to be solved by the invention]
The present invention relates to the electrical configuration of the color image sensor (imaging tube or solid-state image sensor, CCD or other type, etc.) and the type of color coding (primary color or complementary color, or three colors or four colors). In the following explanation, since the essential problems are pointed out regardless of the point of view and the means for solving them are indicated, only the example will be taken up and explained in the following explanation unless otherwise specified.
[0006]
An example of the RGB Bayer arrangement in the conventional color coding arrangement will be described with reference to FIGS. In the RGB Bayer array, 2 × 2 4 pixels shown in FIG. 3A are used as a basic array, and the basic array is sequentially arranged as shown in FIG. 3B so as to fill a plane. The feature is that the luminance resolution is increased by increasing the density of G, which has a large contribution to the luminance signal, by setting the distribution ratio of the number of pixels to each color of RGB as 1: 2: 1. Further, since it is isotropically arranged in two vertical and horizontal directions, an isotropic resolution can be obtained unlike a stripe filter. Note that FIG. 3B illustrates an arbitrary 8 × 8 = 64 pixels.
[0007]
However, since the Bayer array uses a regular array as described above, it has a serious problem of generating false resolution based on spatial sampling by the array, that is, so-called color moire. That is, when there is a periodic subject having a luminance change (monochrome pattern) having the same cycle as the arrangement cycle in a monochrome subject that is not originally colored, for example, paying attention to the RG row as one horizontal line, R is white, G is black If there is, a signal equivalent to a signal obtained from a red subject having no luminance change is output, resulting in a color output that does not originally exist. The false color signal, that is, the color moire generated by such a repetitive striped pattern is generated in a low frequency region by so-called frequency folding (aliasing). Therefore, it is also possible to perform electrical filter processing including color band suppression in the subsequent stage. It cannot be removed.
[0008]
For this reason, the conventional single-plate color imaging device requires an optical low-pass filter such as crystal in the optical system in order to ensure image quality, and this is not only a major limitation on miniaturization and cost reduction, but it still remains. Deterioration in image quality due to color moiré is inevitable.
[0009]
The present invention was made to essentially solve the problems associated with the above-described periodic color coding arrangement in a conventional color imaging device, and provides a small and low-cost imaging device capable of high-quality color imaging. An object of the present invention is to provide a color imaging device capable of achieving this. To describe the purpose of each claim, the invention according to claim 1 makes it possible to capture a subject with a random color coding arrangement having no regularity, and has a high resolution equal to or higher than a predetermined value over the entire imaging region. It is an object of the present invention to provide a color image pickup device that can secure the image quality and at the same time, does not substantially cause resolution degradation even in the presence of an isolated pixel defect. The invention according to claim 2 is to provide a color image pickup apparatus that uses the color image pickup element according to claim 1 to perform color separation based on random color coding arrangement information of the image pickup element. The invention according to claim 3 is to provide a color imaging apparatus capable of easily and reliably performing color separation based on random color coding arrangement information. The invention according to claim 4 is to provide a color imaging apparatus provided with means for storing random color coding arrangement data that can be mass-produced at low cost. An object of the present invention is to provide a color imaging apparatus that can easily cope with color separation processing of color imaging elements having different color coding arrangements.
[0010]
[Means for Solving the Problems]
In order to solve the above problems, the invention according to claim 1 is a color imaging device having a pixel group in which a plurality of pixels each including a photoelectric conversion device are arranged, and the color coding arrangement of the color imaging device is: A color image sensor is configured as a three-color random array in which an arbitrary pixel of interest satisfies an array restriction condition adjacent to any of three colors including the color of the pixel of interest and four sides of the pixel of interest. In the color imaging device configured as described above, it is possible to image a subject with a random color coding arrangement having no periodicity, and thus it is possible to prevent occurrence of color moire even for a subject having a periodic luminance change. . Since the color coding arrangement satisfies the arrangement restriction condition that an arbitrary pixel of interest is adjacent to any of three sides including the color of the pixel of interest and four sides of the pixel of interest, all imaging is performed even though it is random. A high resolution equal to or higher than a predetermined value can be ensured over a region, and at the same time, resolution degradation can be substantially prevented even if an isolated pixel defect exists.
[0011]
According to a second aspect of the present invention, there is provided a color separation unit that includes the color image pickup device according to the first aspect and that performs color separation processing based on a random color coding arrangement of the color image pickup device with respect to an output signal of the color image pickup device. A color imaging apparatus is provided. By configuring in this way, a color imaging apparatus capable of reliably performing color separation based on color coding arrangement information of a color imaging element having a high resolution equal to or higher than a predetermined value without generating color moiré is realized. Can do.
[0012]
According to a third aspect of the present invention, in the color imaging apparatus according to the second aspect, the storage unit stores array data relating to a random color coding arrangement of the color imaging device for performing color separation processing by the color separation unit. It is characterized by that. Thus, by providing the storage means for storing the color coding arrangement data of the color imaging device, color separation based on the random color coding arrangement information can be easily and reliably performed.
[0013]
According to a fourth aspect of the present invention, in the color imaging apparatus according to the third aspect, the storage means is constituted by a mask ROM. As described above, by using the mask ROM as the storage means for the color coding arrangement data, it becomes possible to mass-produce the color imaging apparatus by using the storage means at a low cost.
[0014]
According to a fifth aspect of the present invention, in the color imaging apparatus according to the third aspect, the storage means is composed of an EEPROM. As described above, by using the EEPROM as the storage means for the color coding arrangement data, it is possible to easily cope with the color separation processing of the color imaging elements having different color coding arrangements.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Next, embodiments will be described. FIG. 1 is a block configuration diagram showing an embodiment of a color imaging device (digital camera) using a color imaging device according to the present invention. In FIG. 1, 1 is a lens system, 2 is a lens driving mechanism, 3 is an exposure control mechanism, 4 is a CCD imaging device, 5 is a CCD driver, 6 is a preprocess circuit including an A / D converter, and 7 is a digital process circuit. Thus, a memory is included as hardware, and all digital process processing is performed. 8 is a memory card interface, 9 is a memory card, 10 is an LCD image display system, 11 is a system controller including a microcomputer as a main configuration, 12 is an operation switch system, 13 is an operation display system, 14 is a strobe, 15 is a lens driver, 16 is an exposure control driver, and 17 is an EEPROM.
[0016]
FIG. 2 shows an example of a color filter array of a random array of the CCD image sensor 4 used as a color image sensor in the embodiment shown in FIG. The number of pixels of the CCD image pickup device is arbitrary but is assumed to be about 1 million pixels. In FIG. 2, only the filter array corresponding to 8 × 8 = 64 pixels in the central portion is displayed. In the following description, the procedure for obtaining such a random sequence will be specifically described, and the illustration is only for the purpose of assisting this understanding. I will. (In addition, since random arrangement is the essence of the present invention, it is meaningless and impossible to exemplify the pattern of the entire region.) The type of filter in this embodiment uses so-called RGB three primary colors. The coding is RGB random filter coding.
[0017]
Next, an example procedure for obtaining such coding will be described. Since this coding is random coding, it is of course possible to use dice each assigned two faces to RGB in order to determine the color filter of each pixel, but in order to reduce the complexity, spreadsheet software is used. Are used to prepare a table array corresponding to the entire pixel array. A numerical value obtained by assigning a formula MOD (RND / 3) (where RND is a random number function with an appropriate number of digits, and MOD (n / d) is a remainder function obtained by dividing n by d) to each cell of the array. On the other hand, for example, 0 → R, 1 → G, 2 → B may be applied.
[0018]
The array obtained in this way is statistically usually not particularly biased, but the one obtained in a single trial is stochastically low, but the number of pixels by color is extremely low. There is a possibility that there is a lot of color and there is a concentration of a specific color over a large area. Alternatively, the possibility of a pattern having periodicity as in the conventional example is extremely low, but it is not zero. Therefore, after several trials are performed by the above method and a plurality of array samples are obtained, an imaging test by actual shooting (preferably using a simulation in practice) is performed, and a sample having a good evaluation result is adopted. desirable.
[0019]
However, even if such a trial method is unavoidable in the final sequence selection, the fact that all trials are performed from the beginning of the design generally significantly reduces the design efficiency. It is not preferable. Or, in order to evaluate the sequence obtained by trial, there should be objective requirements required for the sequence itself that is essential to obtain good image quality (as an extreme example, all It is self-evident that there should be only one color pixel), and it is extremely effective to find such a condition specifically and adopt it as a limiting condition (determination criterion).
[0020]
Specifically, the RGB three-color random array of the CCD image pickup device according to the present embodiment has any of the three colors including the filter color (self color) of any pixel of interest and any of the four sides of the pixel of interest. Adjacent to each other is adopted as a constraint condition. That is, it is a condition that at least one pixel of RGB is included in the upper, lower, left and right four pixels regardless of the color of the filter of the target pixel (self color). This condition guarantees that the closest pixel information in the color separation processing described later is complemented by any one of the adjacent pixels in the upper, lower, left, and right directions. As a result, a high resolution of a certain value or more is ensured. It is guaranteed. In addition, even if a pixel defect (assuming an isolated one) occurs, the filter color (self color) of the pixel of interest always exists in the upper, lower, left, and right adjacent pixels. If the defect is compensated for by this, the degradation of the resolution is practically not caused. For reference, it can be read from FIG. 2 that each color cannot exist by being isolated by one pixel at this time, and the minimum unit is two pixels. In FIG. 2, some pixels in each peripheral column apparently do not satisfy the above condition, but this satisfies the condition due to the presence of a further outside pixel not shown. For this reason, the photoelectric conversion surface of the image sensor has an allowance of one to several rows (columns) for each of the four circumferences than the effective image area, and is a so-called discard pixel area (which is involved in image signal generation, but is effective. An area that is not an image area) is provided.
[0021]
It should be noted that a sequence satisfying such a constraint condition is prepared in advance by preparing a large number of the above completely random sequences by trial and testing them under the above conditions or when generating a sequence by software processing such as spreadsheet. Either can be obtained by generating after imposing a constraint condition.
[0022]
Now, in a color imaging device (digital camera) using a CCD imaging device 4 having such a random color filter array, a signal is read out and processed in the same manner as a conventional camera, and a captured image is recorded in a memory card 9. Or displayed on the LCD image display system 10. The operation different from the conventional one is color separation processing, which is performed by the digital process circuit 7 under the control of the system controller 11. Of course, the color separation process is basically a signal complementing process using neighboring pixel information for a pixel for which no corresponding color signal exists (for example, an R filter pixel in the B signal generation process). There is no difference from the past. However, the conventional color separation corresponds to the regular color coding of the CCD image sensor, performs regular sampling based on the permutation, and performs simple complementation using a hold circuit or the like, and addition / subtraction between pixels as necessary However, since random coding applied in the present invention has no regularity, such processing cannot be performed (in contrast to various processing such as anagro processing, digital processing, and hybrid processing). Therefore, color separation processing is performed with reference to filter coding data (filter table of all pixels corresponding to FIG. 2 above) for each pixel of the CCD image sensor to be used. This coding data is stored in the EEPROM 17 so that it can cope with the case where the color coding of the CCD image pickup element used is different.
[0023]
A specific color separation (each color signal generation) process of the present embodiment is performed as follows. That is, for the target pixel to be processed, reference is first made to the coding data, and for the color signal related to the color of the filter of the pixel itself, the signal level of the pixel is output as a signal, and the coding data for the other color signals. Based on the above, the closest matching color filter pixel is searched for among the neighboring pixels, and the signal level of the corresponding pixel is output as the color signal.
[0024]
Regarding the case where the filter of the pixel to be processed of interest is R, the color separation process is exemplified as follows.
R signal: Since the reference result of the coding data is R, the signal level of the pixel is output as it is as the R signal.
G signal: Since the reference result is not G, the nearest G pixel is searched for among the neighboring pixels based on the coding data, and the signal level of the corresponding pixel is output as the G signal.
B signal: Since the reference result is not B, the nearest B pixel is searched for among neighboring pixels based on the coding data, and the signal level of the corresponding pixel is output as the B signal.
[0025]
In the above color separation processing example, if (R, G, B, B) filter pixels are lined up (up, down, left, right) of the R filter pixel of interest, for example, the “closest B” pixel is There are two left and right, but in such a case, either one of them may be adopted, or the average value of both may be adopted.
[0026]
The color signal obtained as a result of the color separation process as described above is processed as a synchronized RGB 3 primary color signal for all pixels by a circuit in the subsequent stage in the same manner as the conventional RGB 3 primary color signal, and finally is stored in the memory card 9. Recorded or displayed on the LCD image display system 10. It should be noted that the processing in the subsequent circuit is appropriately used as needed, and is known per se, for example, by color balance processing, conversion to a luminance-color difference signal by matrix calculation or its inverse conversion processing, band limitation, etc. These include false color removal or reduction processing, various non-linear processes represented by γ conversion, various information compression processes, and so on.
[0027]
Considering the false color generated at that time, a subject such as a black and white knife edge or an isolated white point (line) naturally has a false color as in the conventional coding due to the influence of planar color coding. However, these are all isolated (false) color points and color lines, and their main energy is distributed in the high frequency range, so that they can be removed by a conventionally known method such as electrical filtering. It is possible to reduce. And for the imaging of striped repetitive patterns, which was the biggest problem in the past, since the coding is random, at least low-frequency false color (color moire) that turns back to a low frequency does not occur. In this embodiment, the generation of the isolated false color that can be removed or reduced is limited. Therefore, in the present embodiment, despite the fact that the optical low-pass filter that has been essential in this type of digital camera is not used, there is almost no false color that is visually problematic, and high image quality can be obtained. .
[0028]
In addition, since the nearest pixel information in the color separation processing is always complemented by any one of the adjacent pixels in the upper, lower, left, and right directions, the maximum pixel blur, that is, the maximum PSF (point spread function) width is 3 pixels. Compared to the case of using the average value of peripheral pixels instead of one pixel for color separation in the conventional Bayer arrangement, the color resolution is about the same for G and about twice that for RB. can do. Considering the complementation when there is an isolated pixel defect, in the case of the Bayer array, the PSF width must be allowed up to a maximum of 5 pixels, whereas in this embodiment the maximum width of the PSF changes. do not do. Moreover, this is a comparison only of the effect of pure pixel sampling, but since no optical low-pass filter is used, the response is not degraded by this, and the resolution is reduced to the frequency region where resolution was impossible in the past. It becomes possible to expand.
[0029]
Although the present invention has been described based on the above embodiment, various modifications can be considered for the above embodiment. First, in the above-described embodiment, the coding data is stored in the EEPROM 17, and it has been shown that it can cope with the case where the coding of the CCD image sensor to be used is different. Since CCD image pickup devices often require different pixel defect data and the like due to variations in mass production, the use of an EEPROM can be an advantage from the viewpoint of using a memory for this purpose. Usually, there is one kind of image pickup element applied to one image pickup apparatus main body (the same except for mass production variations), and since there is no need to change color coding in particular, all the coding data itself can use the same data. If attention is paid to this point, the EEPROM can be replaced with a mask ROM. When replaced with a mask ROM, it can be configured at a lower cost. In either case, it goes without saying that it can also be used as a microcomputer program storage memory of the system controller 11.
[0030]
In addition, as described in the beginning, the present invention relates to an electrical configuration of an image sensor (such as an image pickup tube or a solid-state image sensor, a CCD or another type), a color coding type (primary color or complementary color, or three colors). Or any other color) is not limited to the above-described embodiment, and is not limited to the above embodiment, and all color image pickup devices including these and colors using the same. Applicable to imaging devices. For example, it may be applied to a multi-panel camera. For example, the present invention can be applied as it is to an R / B sensor of a G, R / B type two-plate color camera.
[0031]
Furthermore, the “procedure for obtaining random coding” shown in the above embodiment is merely an example, and the random coding itself can be obtained by an arbitrary method. That is, the color coding arrangement differs from the conventionally known regular arrangement, and is noticeable regular (periodic) when attention is paid to a predetermined region of at least several pixels to several tens of pixels or more of the photoelectric conversion element arrangement. If it does not have a structure and, as a result, the occurrence of low-frequency false colors for the repeated pattern input of a striped pattern caused by the conventional regular arrangement is reduced, the arrangement becomes the random color coding arrangement in the present invention. .
[0032]
【The invention's effect】
As described above based on the embodiments, according to the present invention, the problem associated with the periodic color coding arrangement is essentially solved, and a small and low cost that does not generate color moiré even in a subject having a periodic luminance change. A color imaging device and a color imaging device capable of color imaging with high image quality can be obtained. In particular, according to the invention according to claim 1, subject imaging by a random color coding arrangement having no periodicity is performed. In addition, since a three-color random array satisfying the array restriction condition that an arbitrary target pixel is adjacent to one of four sides of the target pixel and the three colors including the color of the target pixel is generated, color moiré is generated. In addition, a color image sensor that can ensure a high resolution of a predetermined value or more and at the same time does not substantially cause resolution degradation even if an isolated pixel defect exists. It can be. According to the second aspect of the present invention, the color coding arrangement information of the color imaging device that does not cause color moiré, has a high resolution equal to or higher than a predetermined value, and does not cause resolution deterioration even if an isolated pixel defect exists. Therefore, it is possible to realize a color imaging apparatus that can reliably perform color separation based on the above. According to the third aspect of the invention, since the storage means for storing the random color coding arrangement data of the color image sensor is provided, color separation based on the random color coding arrangement information can be performed easily and reliably. . According to the fourth aspect of the present invention, since the mask ROM is used as the storage means for the color coding arrangement data, it is possible to mass-produce the color imaging device at low cost by using the storage means. According to the fifth aspect of the present invention, since the EEPROM is used as the storage means for the color coding arrangement data, it is possible to easily cope with the color separation processing of the color image pickup elements having different color coding arrangements.
[Brief description of the drawings]
FIG. 1 is a block configuration diagram showing an embodiment of a color imaging device and a color imaging device according to the present invention.
FIG. 2 is a diagram showing an example of a random color filter array of the CCD image pickup device in the embodiment shown in FIG.
FIG. 3 is a diagram illustrating an example of a basic array and an entire array of RGB Bayer arrays.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Lens system 2 Lens drive mechanism 3 Exposure control mechanism 4 CCD image pick-up element 5 CCD driver 6 Preprocess circuit 7 Digital process circuit 8 Memory card interface 9 Memory card
10 LCD image display system
11 System controller
12 Operation switch system
13 Operation display system
14 Strobe
15 Lens driver
16 Exposure control driver
17 EEPROM

Claims (5)

A color imaging element having a pixel group formed by arranging a plurality of pixels each including a photoelectric conversion element, and the color coding arrangement of the color imaging element includes three colors including an arbitrary target pixel including the color of the target pixel and the color A color imaging device, which is a three-color random array that satisfies an adjacent array restriction condition on any one of four sides of a pixel of interest. The color imaging device according to claim 1, further comprising color separation means for performing color separation processing based on a random color coding arrangement of the color imaging device on an output signal of the color imaging device. Color imaging device. 3. The color imaging apparatus according to claim 2, further comprising storage means for storing arrangement data relating to a random color coding arrangement of the color imaging element for performing color separation processing by the color separation means. The color imaging apparatus according to claim 3, wherein the storage unit includes a mask ROM. 4. A color imaging apparatus according to claim 3, wherein the storage means is constituted by an EEPROM.

Images