JP2000299871A - Color image pickup element and color image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an image pickup device small in size, low in cost and capa ble of substantially solving the problem caused by a periodic color coding array and picking up a high quality color image, by making the color coding array of a color image pickup element having a photoelectric element group of a prescribed array into a random array. SOLUTION: This color image pickup device is composed of a CCD image pickup element 4 for making a color coding array into a random array, a digital process circuit 7 for performing color separation on the basis of a random color coding array and a storing means consisting of a mask ROM or an EEPROM 17 for storing array data about the random color coding array. Then, the type of a filter is one using the so-called RGB three primary colors, and its coding is defined as RGB random filter coding. The coding data is stored in the EEPROM 17 and can deal with even the case that the color coding of a used CCD image pickup element 4 is different.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a color image pickup device capable of preventing occurrence of color moire and a color image pickup device using the color image pickup device.

[0002]

2. Description of the Related Art Generally, an image pickup device represented by an image pickup tube and a solid-state image pickup device is widely used in an image pickup apparatus. In particular, a color image pickup device used for a single tube or a single plate (Single Sensor) color image pickup device can constitute a color image pickup device with one image pickup device, so that a color separation prism is not required and the lens can be miniaturized. In addition, there is no need for various adjustments of a multi-plate type represented by registration, and it has many features such as low power consumption, and has made many contributions to miniaturization and power consumption reduction of color imaging devices. A single-chip color camera using a color CCD image pickup device, which is a solid-state image pickup device, has become a mainstream imaging device.

In order to obtain color information on one light receiving surface, each of the above color image pickup devices performs color modulation (color coding) in a light receiving plane using a color filter called a stripe filter or a mosaic filter. ing. That is, for example, by attaching filters of three colors of RGB in a predetermined regular arrangement on each photoelectric conversion element (pixel), each pixel has a different spectral sensitivity. Therefore, since the video signal obtained by subject imaging contains dot-sequential color information according to the filter array, the video signal is separated into signals corresponding to the respective color filters in accordance with the above-described predetermined array. By extracting the separated signal, color information can be extracted. In order to obtain a luminance signal (Y signal), all of the RGB information is required. To obtain the luminance information of one pixel, at least three pixels (one for each of RGB) are required, and the luminance resolution is sacrificed. However, color imaging can be performed with one imaging device.

The above filter arrangement includes three primary color filters such as an RGB stripe, a Bayer type RGB mosaic (various types), a YeMgCy stripe, and a YeMgCyW4.
Various color coding patterns, such as complementary color filters such as a color mosaic and a YeMgCyG four-color mosaic, have been proposed and put to practical use.

[0005]

SUMMARY OF THE INVENTION The present invention relates to an electric configuration of the above-mentioned color image pickup device (such as an image pickup tube or a solid-state image pickup device, or a CCD or another type) and a type of color coding (primary color or complementary color). Or three or four colors), which point out the essential problems that they have, regardless of which one they have, and suggest solutions to them. In the following description, only one example will be used unless otherwise specified. , Will be described.

In the above conventional color coding arrangement, RG
An example of the B-Bayer array will be described with reference to FIGS. The RGB Bayer arrangement corresponds to the 2 shown in FIG.
The basic array is composed of four pixels of × 2, and the basic arrays are sequentially arranged as shown in FIG. 3B so as to fill up the plane, and the distribution ratio of the number of pixels to each color of RGB is determined. The characteristic is that the luminance resolution is increased by increasing the density of G that greatly contributes to the luminance signal as 1: 2: 1. In addition, since it is isotropically arranged in two directions, vertical and horizontal, an isotropic resolution can be obtained unlike a stripe filter. FIG. 3B illustrates an example of arbitrary 8 × 8 = 64 pixels.

However, since the Bayer array uses a regular array as described above, there is a serious problem that false resolution based on spatial sampling by the array, that is, so-called color moire occurs. . That is, when there is a periodic subject having a luminance change (monochrome pattern) having the same cycle as the above-described arrangement cycle in a black and white subject having no color, for example, paying attention to the RG row as one horizontal line, R is white and G is black. If so, a signal equivalent to a signal obtained from a red subject having no change in luminance is output, so that a color output that does not exist originally occurs. Since a false color signal, that is, a color moiré, generated by such a striped repetition pattern is generated in a low frequency region by so-called frequency aliasing (aliasing), the false color signal can also be obtained by an electrical filtering process including color band suppression in a subsequent stage. It cannot be removed.

For this reason, the conventional single-chip color image pickup device is
In order to ensure image quality, an optical low-pass filter such as quartz is essential for the optical system, which not only imposes a great restriction on miniaturization and cost reduction, but also inevitably causes image quality deterioration due to remaining color moiré.

SUMMARY OF THE INVENTION The present invention has been made to substantially solve the problem associated with the above-described periodic color coding arrangement in a conventional color image pickup apparatus. It is an object of the present invention to provide a color image sensor capable of providing the same. According to the object of each claim, the invention according to claim 1 can perform imaging of a subject by a random color coding arrangement having no regularity, and can use a color that does not generate color moire without using an optical low-pass filter. It is an object to provide an imaging device. A second object of the present invention is to provide a color image pickup apparatus capable of performing color separation based on random color coding arrangement information of the image pickup element by using the color image pickup element according to the first embodiment. A third object of the present invention is to provide a color imaging apparatus capable of easily and reliably performing color separation based on random color coding arrangement information. A fourth object of the present invention is to provide a color image pickup apparatus having means for storing random color coding arrangement data which can be mass-produced at low cost. It is another object of the present invention to provide a color imaging apparatus which can easily cope with color separation processing of color imaging elements having different color coding arrangements.

[0010]

According to a first aspect of the present invention, there is provided a color imaging device having a group of photoelectric conversion elements arranged in a predetermined arrangement. The color image sensor is configured as a random array. In the color image sensor configured as described above, it is possible to image a subject using a random color coding arrangement having no regularity, and to perform color imaging even on a subject having a periodic luminance change without using an optical low-pass filter. Color imaging free from moire can be achieved.

According to a second aspect of the present invention, there is provided a color imaging device comprising the color imaging device according to the first aspect, wherein a color separation process is performed on an output signal of the color imaging device based on a random color coding arrangement of the color imaging device. The color image pickup device is constituted by providing a separating means. With this configuration, it is possible to realize a color imaging device that can reliably perform color separation based on color coding arrangement information of a color imaging device that does not generate color moiré.

According to a third aspect of the present invention, in the color imaging apparatus according to the second aspect, storage for storing array data relating to a random color coding array of the color imaging element for performing color separation processing by the color separation means. Means are provided. The provision of the storage means for storing the color coding arrangement data of the color image pickup device makes it possible to easily and reliably perform color separation based on the random color coding arrangement information.

According to a fourth aspect of the present invention, in the color imaging apparatus according to the third aspect, the storage means is constituted by a mask ROM. By using the mask ROM as the storage means of the color coding arrangement data, it becomes possible to mass-produce the storage means and hence the color image pickup device at low cost.

According to a fifth aspect of the present invention, in the color imaging apparatus according to the third aspect, the storage means is constituted by an EEPROM. By using the EEPROM as the storage means of the color coding arrangement data, it is possible to easily cope with the color separation processing of the color imaging devices having different color coding arrangements.

[0015]

Next, an embodiment will be described. FIG. 1 is a block diagram showing an embodiment of a color imaging device (digital camera) using a color imaging device according to the present invention. In FIG. 1, 1 is a lens system, 2
Is a lens driving mechanism, 3 is an exposure control mechanism, 4 is a CCD image sensor, 5 is a CCD driver, 6 is a pre-processing circuit including an A / D converter, 7 is a digital processing circuit, and includes a memory as hardware, It performs digital process processing. 8 is a memory card interface, 9 is a memory card, 10 is an LCD image display system, 11 is a system controller including a microcomputer as a main configuration,
12 is an operation switch system, 13 is an operation display system, 14 is a strobe,
15 is a lens driver, 16 is an exposure control driver, 17 is EE
PROM.

FIG. 2 shows an example of a random arrangement of color filters of the CCD image sensor 4 used as a color image sensor in the embodiment shown in FIG. The number of pixels of the CCD image pickup device is arbitrary, but is assumed to be about 1 million pixels. In FIG. 2, only the filter arrangement corresponding to 8 × 8 = 64 pixels at the center is shown. In the following description, a procedure for obtaining such a random sequence is specifically described, and the illustration is merely to assist the understanding, and the illustration of such a region is sufficiently understood. There will be. The type of filter in the present embodiment uses so-called RGB primary colors.
The coding is RGB random filter coding.

Next, an example of a procedure for obtaining such coding will be described. Since this coding is random coding, it is a matter of course that dice having two planes each assigned to RGB may be used to determine the color filter of each pixel. However, in order to reduce the complexity, spreadsheet software is used. A table array corresponding to the entire pixel array is prepared using the above method. Then, in each cell of the array, the formula RST (RND / 3) (where RND is a random number function having an appropriate number of digits, and RST (n / d) is a remainder function obtained by dividing n by d)
Is assigned to, for example, 0 → R, 1
→ G, 2 → B may be applied.

The sequences obtained in this way usually do not have a particularly large bias statistically, but those obtained by only one trial are extremely stochastically low, although they are extremely low in color. There is a possibility that the number of pixels is large or small and a specific color is concentrated over a large area. Or,
The possibility of a pattern having periodicity as in the conventional example is extremely low, but is not zero. Therefore, after performing a plurality of trials by the above method to obtain a plurality of array samples, it is possible to perform an imaging test (actually, it is preferable to use a simulation) by a real photograph and adopt a sample having a good evaluation result. desirable. Statistically, the above-described special bias arrangement can be eliminated without any problem by performing about 3 to 4 or less trials.

In a color image pickup apparatus (digital camera) using the CCD image pickup device 4 having such a random color filter array, signals are read out and processed in the same manner as in a conventional camera, and the picked-up image is stored in a memory card. 9
Or displayed on the LCD image display system 10. An operation different from the conventional one is a color separation process, which is performed by the digital process circuit 7 under the control of the system controller 11.

In the conventional color separation, regular sampling based on permutation is performed in accordance with regular color coding of a CCD image pickup device, and addition and subtraction between pixels are performed. On the other hand, random coding applied in the present invention does not have regularity, whereas such processing cannot be performed. So, CC to use
Filter coding data for each pixel of the D image sensor (filter table for all pixels corresponding to FIG. 2 above)
, And performs color separation processing. This coding data is stored in the EEPROM 17 and used by the CCD.
It is possible to cope with the case where the color coding of the image sensor is different.

The specific color separation (color signal generation) processing of this embodiment is performed as follows. That is,
"For the processing target pixel of interest, first refer to the coding data. For the color signal related to the color of the filter of the pixel itself, output the signal level of the pixel as it is as a signal, and for the other color signals, based on the coding data. A pixel of the closest corresponding color filter is searched for among the neighboring pixels, and the signal level of the corresponding pixel is output as the color signal. "

An example of the color separation processing when the filter of the target pixel to be processed is R is as follows. R signal: Since the reference result of the coding data is R, the signal level of the pixel is output as it is as an R signal. G signal: Since the reference result is not G, the nearest G pixel is searched for among the neighboring pixels based on the coding data, and the signal level of the corresponding pixel is output as the G signal. B signal: Since the reference result is not B, the nearest B pixel is searched for among the neighboring pixels based on the coding data, and the signal level of the corresponding pixel is output as the B signal.

In the above example of the color separation processing, for example, (R, G,
B, B) Assuming that the filter pixels are arranged, two "nearest B" pixels exist on the left and right. In such a case, only one of the signal levels is adopted. Alternatively, an average value of both signal levels may be adopted.

The color signals obtained as a result of the above-described color separation processing are processed as a synchronized RGB primary color signal for all the pixels by a subsequent circuit in the same manner as the conventional RGB primary color signal, and finally are stored in a memory. Recorded on card 9 or L
The image is displayed on the CD image display system 10. The processing in the circuit at the subsequent stage is appropriately used according to the necessity, and is known per se, for example, by color balance processing, conversion into a luminance-color difference signal by matrix operation or inverse conversion processing thereof, band limitation, and the like. False color removal or reduction processing, various non-linear processing represented by γ conversion, various information compression processing, and the like.

Considering the false color generated at that time, regarding a subject such as a black-and-white knife edge and an isolated white point (line), the false color is naturally affected by the planar color coding as in the conventional coding. Occurs. However, these are all isolated (false) color points and color lines, and their main energy is distributed in a high frequency range. It is possible to reduce. In the imaging of a striped repetitive pattern, which has been the biggest problem in the past, since coding is random, at least a low-frequency false color (color moiré) folded to a low frequency does not occur. In the embodiment, only the occurrence of the above-mentioned isolated false colors that can be reduced is limited. Therefore, in the present embodiment, a high quality image can be obtained with virtually no false colors that are visually problematic despite the fact that an optical low-pass filter, which is conventionally required in this type of digital camera, is not used. .

Although the present invention has been described based on the above embodiment, various modifications of the above embodiment are conceivable. First, in the above embodiment, the coding data is stored in the EEPROM 17 so that it is possible to cope with the case where the coding of the CCD image pickup device used is different. In many cases, CCD image sensors require different pixel defect data due to variations in mass production. Therefore, the use of an EEPROM can be advantageous from the viewpoint of using the memory for this purpose as well. Usually, the type of imaging device applied to one imaging device body is the same (the same except for mass production variation), and there is no need to change color coding.
The coding data itself can use the same data. Focusing on this point, the EEPROM can be replaced with a mask ROM. Mask RO
When replaced with M, it can be configured at lower cost. Further, in any case, it is needless to say that the memory can be used also as the program storage memory of the microcomputer of the system controller 11.

Also, as described at the beginning, the present invention relates to the electrical configuration of the image pickup device (such as an image pickup tube or a solid-state image pickup device, or a CCD or another type) and the type of color coding (primary color or complementary color, Or, three colors or four colors) is intended to address and solve the essential problems irrespective of the color, and the present invention is not limited to the above-described embodiment, and all color imaging devices including these and the It can be applied to the used color imaging device. For example, the present invention may be applied to a multi-panel camera. For example, the present invention can be applied to an R / B sensor of a G, R / B type two-plate color camera as it is.

Furthermore, the “procedure for obtaining random coding” described in the above embodiment is merely an example, and the random coding itself can be obtained by any method. That is, unlike the conventionally known regular arrangement, the color coding arrangement is noticeably regular (periodic) when focusing on a predetermined area of at least several pixels to several tens of pixels of the photoelectric conversion element arrangement. If it does not have a structure and consequently reduces the occurrence of low-pass false color for a striped repetitive pattern input caused by a conventional regular arrangement, the arrangement is a random color coding arrangement as referred to in the present invention. Become.

[0029]

As described above with reference to the embodiments, according to the present invention, the problem associated with the periodic color coding arrangement is essentially solved, and color moiré can be reduced even in a subject having a periodic luminance change. It is possible to obtain a color image pickup device and a color image pickup device capable of performing high-quality color image pickup at a small size and at low cost without generation of a subject. In particular, according to the invention according to claim 1, a subject with a random color coding arrangement having no regularity Imaging can be performed, and a color image pickup device in which color moiré does not occur even in a subject having a periodic luminance change can be realized without using an optical low-pass filter. Further, according to the second aspect of the present invention, it is possible to realize a color imaging apparatus capable of reliably performing color separation based on color coding arrangement information of a color imaging element that does not generate color moire. According to the third aspect of the present invention, since the storage means for storing the random color coding arrangement of the color image pickup device is provided, color separation based on the random color coding arrangement information can be easily and reliably performed. . According to the fourth aspect of the present invention, a mask ROM is used as storage means for storing color coding array data.
Therefore, it is possible to mass-produce the color image pickup device at low cost by using the storage means. According to the fifth aspect of the present invention, since the EEPROM is used as the storage means of the color coding arrangement data, it is possible to easily cope with the color separation processing of the color imaging devices having different color coding arrangements.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of a color imaging device and a color imaging device according to the present invention.

FIG. 2 is a diagram showing an example of a random color filter array of the CCD image sensor in the embodiment shown in FIG.

FIG. 3 is a diagram showing an example of a basic array and an overall array of an RGB Bayer array.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Lens system 2 Lens drive mechanism 3 Exposure control mechanism 4 CCD image sensor 5 CCD driver 6 Pre-process circuit 7 Digital process circuit 8 Memory card interface 9 Memory card 10 LCD image display system 11 System controller 12 Operation switch system 13 Operation display system 14 Strobe 15 Lens driver 16 Exposure control driver 17 EEPROM

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 2H002 DB01 DB19 EB00 FB21 HA00 JA07 JA08 ZA01 ZA02 5C065 AA01 BB13 BB43 CC01 CC09 DD17 DD20 EE03 FF02 FF03 GG18 GG31

Claims (5)

[Claims] 1. A color imaging device having a predetermined arrangement of photoelectric conversion element groups, wherein the color coding arrangement of the color imaging device is a random arrangement. 2. The color image pickup device according to claim 1, further comprising: a color separation unit that performs a color separation process on an output signal of the color image pickup device based on a random color coding arrangement of the color image pickup device. A color imaging device characterized by the above-mentioned. 3. A color according to claim 2, further comprising storage means for storing array data relating to a random color coding array of said color image pickup device for performing color separation processing by said color separation means. Imaging device. 4. The color image pickup apparatus according to claim 3, wherein said storage means comprises a mask ROM. 5. The color imaging apparatus according to claim 3, wherein said storage means is constituted by an EEPROM.