JPH11252576A - Color image data processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize a color image data processing method that generates an RGB signal corresponding to each pixel at a high speed. SOLUTION: Data are image data configured by arranging pluralities of pixels each having only one color signal among RGB three colors as a matrix and the image data including the respective color signals in RGB three colors are given to any of four pixels consisting at of 2-row, 2-column as (m-1, n-1), (m, n-1) (m-1, n) (m, n) and the data are converted into a prescribed digital signal by a lookup table LUT method. The converted digital signal is obtained by arranging the digital signal value in each color of RGB colors in a prescribed form and a most significant bit of a signal value corresponding to a color signal of two pixels in four pixels in each color is set to '0'. Then the converted digital signal values of the 4 pixels are summed and the result is outputted as a digital image signal of the (m, n) pixel.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for processing color image data, and more particularly, to a single-chip color image sensor in which color filters of three primary colors of RGB are arranged on an imaging surface of a solid-state imaging device and an image is captured by one solid-state imaging device. The present invention relates to a data processing method for a video camera, a color scanner, and the like.

[0002]

2. Description of the Related Art In a single-panel color video camera using a solid-state imaging device, as disclosed in Japanese Patent Application Laid-Open No. 8-237672, each of pixels arranged in a matrix is provided with one of RGB color filters. It is characterized in that the image pickup unit is configured by arranging the images in a mosaic shape in correspondence with one color.

In such a color video camera, each pixel can obtain only a color signal corresponding to the color of a color filter arranged corresponding to the pixel. Therefore, the other two types of missing color signals need to be generated by interpolation based on the color signals of the peripheral pixels. As described above, a method in which each pixel captures an image of one color at the time of imaging and performs color separation by a processing circuit is widely used in popular video cameras and color scanners.

Conventionally, this color separation is obtained by multiplying each of the color signals of peripheral pixels by a weighting factor, adding the obtained values, and dividing by the sum of all the weighting factors, that is, by averaging. That was common.

[0005]

In this method, the number of pixels increases, that is, as the image resolution increases, the amount of calculation to be processed increases, and the color resolution increases, that is, the number of reproducible colors. , The more memory is required. The method of imaging one color at each pixel is advantageous in that the imaging unit is simpler than the method of capturing a full-color image at each pixel, and excessively costs color separation for performing color signal processing. In this case, the merit of the full-color imaging method is weakened, which is not practical. Also, when the amount of calculation increases, it becomes difficult to output images in real time.

SUMMARY OF THE INVENTION In view of the above problems, an object of the present invention is to provide a color image data processing method capable of generating an RGB signal corresponding to each pixel at high speed.

[0007]

In order to solve the above-mentioned problems, a color image data processing method according to the present invention is directed to a method for processing color input image data constituted by color information of respective pixels arranged in a matrix. , Each pixel is RGB3
When color information of only one of the colors is provided and m and n are arbitrary integers, positions (m-1, n-1), (m, n-
Image data in which any one of four pixels in two rows and two columns of (1), (m-1, n), and (m, n) always includes color information of three colors of RGB as color input image data An input step, and digital signal values of RGB are arranged in a predetermined format corresponding to each of the pixel data of the color input image data, and two out of these four pixels of the digital signal values of each color are arranged. A conversion step in which the most significant bit of the color digital signal value area in which the pixel has color information is set to 0 and converted to a predetermined digital signal value which is output in advance; and the converted digital signal values of these four pixels are added. And outputting the digital image signal of the pixel at the position (m, n) as a digital image signal.

According to this, the image signal of each pixel of the input image data contains only one color signal of RGB. This signal is converted into a predetermined digital signal representing a color by a combination of RGB color signals. The converted digital signal of each pixel obtained in this way is R
Information of only one color of GB is compressed and stored. That is, the digital values of the other color information are 0. In this converted digital signal of each of the four pixels described above, the digital signal values of the two pixels have the same color information. The upper bits are all 0. Therefore, these four
By adding two digital signals, for each pixel,
An image signal including all RGB color signals can be easily obtained.

In the conversion step, the digital signal values of RGB are arranged in a predetermined format corresponding to each of the pixel data of the color input image data. One of the two pixels having the same color information among the pixels may output a digital signal value in which each bit of the digital signal value area representing the color signal is all zero.

According to this, only one of the four pixels is RG.
A color signal of only one of the B colors is included. Therefore, in the interpolation step, by simply adding the signals of these four pixels, an image signal including all the RGB color signals can be easily obtained for each pixel.

It is preferable that two of the four pixels have G color information. Since the G signal has the largest contribution to the luminance, the resolution degradation of the obtained image is small.

It is preferable that a preset combination of digital signal values corresponding to the color input image data can be changed. According to this, the conditions for correction and interpolation can be changed by an external instruction.

[0013]

Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. In each of the drawings, the same components are denoted by the same reference numerals as much as possible, and redundant description will be omitted.

FIG. 1 is a block diagram showing the overall configuration of an imaging system using a single-chip digital output color video camera using image processing according to the present invention.

This system comprises a camera unit 1 and a personal computer (PC) 3. A frame grabber board 2 for processing an output signal of the camera unit 1 is mounted on the PC 3. Also, the camera unit 1 is provided in the PC 3.
Is connected to a monitor 4 for displaying an image picked up by a keyboard 5 and a mouse 6 for performing various inputs.

Next, the detailed structure of each part will be described. First, a camera unit 1 includes a CCD (Charge Coupled Device) having pixels arranged in a matrix, a MOS
(MetalOxide Semiconductor) or other solid-state imaging device 11
A checker-like color filter 12 in which a color filter of any one of RGB is arranged in each pixel in a mosaic pattern on a pixel surface of the solid-state imaging device 11, and an output signal of the solid-state imaging device 11. CDS (Correlated Double Samplin) that removes noise by correlated double sampling
g) Circuit 13 and amplifier circuit 14 for amplifying the output signal
A for converting an analog image signal into a digital image signal
It comprises a / D converter 15, a digital I / F 16 for inputting and outputting to and from an external circuit, and a timing pulse generator 17 for generating a clock signal for controlling the overall operation.

The frame grabber board 2 has a digital I / F 21 connected to the digital I / F 16 of the camera unit 1.
And a LUT (Look Up Tabl) for converting a digital signal sent from the camera unit 1 into another digital signal of a predetermined format.
e) FIFO (Firs) having a circuit 22 and a buffer function of accumulating the converted signal and outputting it at a predetermined timing
t in First out) circuit 23 and a PCI bus I / F 24 for inputting and outputting data to and from the PC 3. Here, the LUT circuit 22 includes a large number of memories, and has a function of outputting data corresponding to an input value at high speed by storing data to be output in an address corresponding to the input value in advance. Have.

The PC 3 includes a PCI bus I / F 31 for exchanging data with the frame grabber board 2, a CPU 32 for processing input data, a memory 33 for holding input data, and an HD 34.

Next, the operation of the present apparatus will be described. Here, various color arrangements are conceivable for the checkered color filter 12, but in general, an arrangement as shown in FIG. 2A is widely used. This is because it is preferable that the G signal has the largest contribution to the image luminance and that the ratio of the G signal be high. Therefore, a G color filter is arranged in half of the pixels, and the R and B pixels are arranged in the remaining pixels. Are alternately arranged for each row. Hereinafter, an example using the color filters 12 having this arrangement will be described. In order to facilitate the understanding of the description, the description will be made using a pixel array of 4 × 4 pixels, and as shown in FIG. Then, the whole is divided into 2 × 2 pixels,
A number indicating the position of the pixel in × 2 pixels is represented by 0 to 3. Accordingly, when the above 4 × 4 pixels are assigned this pixel position number, the result is as shown in FIG.

The subject image is transmitted through a checkered color filter 12, and each pixel is transmitted with light of only one of the RGB color components, enters the solid-state image sensor 11, and has a specific color at each pixel. It is converted into an electric signal representing the luminance of the signal. This signal is sent to the CDS circuit 13 for each pixel,
After the noise is removed, the signal is amplified by the amplifier circuit 14, and A
The digital signal is converted into a 12-bit digital signal by the / D converter 15 and sent out of the camera unit 1 via the digital I / F 16. Each of these circuits is synchronized by a timing pulse generator, and a digital I / F 1
Reference numeral 6 outputs a synchronizing signal and a position signal indicating the position number of a pixel based on the synchronizing signal.

The signal output from the camera unit 1 is transmitted to the LU via the digital I / F 21 of the frame grabber board 2.
The signal is sent to the T circuit 22, where it is converted into a digital signal of a predetermined format and sent to the FIFO circuit 23 (the details of this conversion operation will be described later). The signal conversion operation of the LUT circuit 22 is controlled by the PC 3 via the PCI bus I / F 24. The converted digital signal is FIFO
At a predetermined timing from the circuit 23, the PCI bus I / F2
The data is sent to the inside of the PC 3 via 4, 31. The data sent to the PC 3 is stored in the memory 33 or the HD 34, subjected to predetermined processing by the CPU 32, and
Displayed above. The operating conditions of the camera unit 1 and the frame grabber board 2 and the processing conditions of the image data in the CPU 32 can be controlled by an operator using the keyboard 5 and the mouse 6.

Next, the processing of image data will be described in detail. This apparatus has two imaging modes: a high-speed imaging mode and a high-definition imaging mode. Hereinafter, each imaging mode will be described separately. First, the high-speed imaging mode will be described.

FIG. 3 shows the arrangement of the color signals represented by the digital image signals output from the camera unit 1 for each pixel when the checker filter 12 shown in FIG. 2A is arranged. FIG. 4 shows the bit arrangement of this signal.
Each pixel outputs a color signal of only one of the RGB colors. As shown in FIG. 4, the digital image data of this color signal is assigned to lower 12 bits (for example, R ₀ (11) to R ₀ (0)) of the 16 bits, The position number shown in 2 (c) is indicated by a binary value. Here, the two bits between them are unused bits, and any of 0 and 1 may be input. Further, it may be used as a parity bit of valid data. The reason why the image data is set to 16 bits is that processing on a PC can be simplified.

This data is input to the LUT circuit 22. Here, the output image data from the LUT circuit 22 is 16-bit data in a format in which 5 bits of an R signal, 6 bits of a G signal, and 5 bits of a B signal are arranged from the higher order.
This is high-color (color resolution 65536 colors) image data used in general DOS / V personal computers and the like. FIG. 5 is a layout diagram showing correspondence between pixels processed and output by the LUT circuit 22 and pixels, and FIG. 6 is a diagram showing a bit layout of this data signal.

Here, R ', G', and B 'are output data corresponding to the input data shown in FIG. 4 and are data obtained by compressing the input data to 5 bits, respectively. It is also possible to perform color correction such as image and white balance. At this time, since the area for storing the G signal has 6 bits, 0 is always entered in the most significant bit. The signal of each pixel obtained in this way is always RGB
Out of the primary colors.

This data signal is sent to PC3,
The data for the pixels are added and output. FIG. 7 (a)
(B) and (c) show the correspondence between the data added and output in this way and the pixels of the RGB color components, and FIG.
The bit arrangement of this data is shown. As is clear from these figures, the values of the RB signals are the same in the four neighboring pixels (however, four pixels having the same R signal value and four pixels having the same B signal value are in one row and one column). Minutes off). And
The G signal has a different value for each pixel. This is G
This is because pixel values of two pixels alone are added. Here, in the data before the addition, the most significant bit of the G signal is set to 0 as shown in FIG. This corresponds to halving the digital value of the G signal. Then, since two pixels are added thereafter, it is the same as calculating the average value. Unlike the case where the average is obtained by halving after addition, the digital value after addition does not protrude from the storage area of the G signal to the storage area of the R signal, and the color resolution is maintained.

Here, an example in which the addition is performed by the PC 3 has been described. However, this addition can be easily realized by hardware if a delay circuit for one row and one pixel is used. You may.

FIG. 9 shows an example of this processing. In the camera unit output, the output value of the R signal is doubled, the output value of the G signal is quadrupled, and the output value of the B signal is doubled. The output data of the LUT circuit 22 when the correction processing for performing the linear correction is also performed, and the final output data are shown together with the output of the camera unit.

As described above, according to the present invention, high-speed interpolation can be performed with a simple configuration. Further, since one color signal among the color signals is different for each pixel, there is no deterioration in resolution.

Next, the high definition imaging mode will be described. Even in the high-definition mode, the input up to the LUT circuit 22 is
Processing is performed in the same manner as in the high-speed mode. That is, the LUT circuit 2
The input data to 2 has the format shown in FIG. In the high-definition mode, the LUT circuit 22 may output the input data without processing it, or may perform a specific correction process without changing the number of bits. Here, additional information other than the digital image data may be removed. The processed or unprocessed data is sent to the PC 3. In this high definition mode,
Various known algorithms can be used for the interpolation processing. Here, a case where an algorithm for interpolating missing color information from 3 × 3 pixel peripheral pixels is used will be described with reference to FIG. .

Based on image data in which each pixel shown in FIG. 10A has only one color signal, 2 ×
FIGS. 10B to 10E show the results of interpolating missing color information of each pixel for two pixels. In this case, RG
Since a different color signal is obtained for each pixel for B, the image resolution of the color signal is further improved as compared with the high-speed mode described above.
However, on the other hand, the number of arithmetic processes increases, and at least three lines need to be stored in the memory. Therefore, for example, when high-speed processing is required, for example, at the time of focusing, the high-speed imaging mode is preferably used. In particular, when a screen resolution is required for each color, it is preferable to use the high-definition imaging mode.

Even if the output of the camera unit is not 12 bits,
Data having a bit length of 14 bits, 16 bits or longer may be used. Output data in high-speed mode is also R,
Various types of data can be used, such as 15-bit data of 5 bits each for G and B, and 24-bit data of 8 bits each.

In addition to the arrangement of the color filters, FIG.
Various sequences shown in (a) to (h) can be used. In any of these cases, when data is converted in the high-speed imaging mode, the data may be converted so that the most significant bit of the data area representing the color signals arranged in two out of four pixels becomes zero. By doing so, it is possible to easily calculate the average without overflow during addition.

Alternatively, in the high-speed imaging mode, outputs of pixels corresponding to a plurality of color filters arranged in a plurality of four pixels are converted to 0 by an LUT circuit, and outputs of the remaining pixels are converted to a data area representing the color signal. You may use it fully and convert. For example, the color signal output from the pixel of position number 1 shown in FIG. In this case, the bit arrangement of the output data of each pixel is as shown in FIG. As a result, the bit arrangement of the final output data is as shown in FIG. This operation is an addition operation.
Since only one pixel has a significant value in each bit among the pixels, the addition can be performed by simply taking a logical sum. Therefore, particularly when the circuit is configured by hardware, the device configuration is simple.

The LUT circuit can be used either with a configuration in which a plurality of conversion sets are built-in in advance, or with a configuration in which the conversion sets can be rewritten by an instruction from a PC.

In the above description, an example was described in which a single-panel digital color camera was used as an image input device. However, the present invention is not limited to this, and various image input systems using color filters, for example, It can be widely used for processing input images such as color scanners and color copiers and image data in which each pixel has image information of only one color.

[0037]

As described above, according to the present invention, since the input image data can be converted at a high speed into a format which can be easily added, the interpolation process can be performed at a high speed. At this time, since the color signal of at least one color differs for each pixel, the resolution information can be maintained.

[Brief description of the drawings]

FIG. 1 is an overall configuration block diagram of an embodiment of the present invention.

FIG. 2 is a diagram showing an arrangement of pixels and color filters of the device of FIG. 1;

FIG. 3 is a diagram illustrating correspondence between input data to a LUT circuit and pixels.

FIG. 4 is a diagram showing a bit arrangement of data in FIG. 3;

FIG. 5 is a diagram illustrating correspondence between output data from a LUT circuit and pixels in a high-speed imaging mode.

FIG. 6 is a diagram showing a bit arrangement of data of FIG. 5;

FIG. 7 is a diagram illustrating correspondence between output data of the present apparatus and pixels in the high-speed imaging mode.

FIG. 8 is a diagram showing a bit arrangement of data in FIG. 7;

FIG. 9 is a diagram illustrating an example of processing in a high-speed imaging mode.

FIG. 10 is a diagram illustrating the correspondence between output data and pixels in a high-definition imaging mode.

FIG. 11 is a diagram showing a different arrangement example of color filters.

FIG. 12 is a diagram showing another bit arrangement of the data of FIG. 5;

FIG. 13 is a diagram showing another bit arrangement of the data in FIG. 7;

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Camera part, 2 ... Frame grabber board, 3 ... Personal computer, 4 ... Monitor, 5 ... Keyboard, 6 ...
Mouse, 11: solid-state image sensor, 12: checkered color filter, 15: A / D converter, 22: LUT circuit.

Claims (4)

[Claims] 1. A method of processing color input image data constituted by color information of pixels arranged in a matrix, wherein each pixel has color information of only one of three colors of RGB, m, n Is an arbitrary integer, the position (m-1, n
-1), (m, n-1), (m-1, n), (m, n)
RGB3 must be assigned to any one of the 4 pixels in 2 rows and 2 columns
Inputting image data containing color information of a color as color input image data; and arranging digital signal values of RGB colors in a predetermined format in correspondence with respective pixel data of the color input image data. The digital signal value of each color is converted into a predetermined digital signal value by setting the most significant bit of a digital signal value area of a color in which two of the four pixels have color information to 0, and outputs the digital signal value. A color image data process comprising: adding a converted digital signal value of each of the four pixels; and outputting a digital image signal of a pixel at a position (m, n). Method. 2. A method of processing color input image data constituted by color information of pixels arranged in a matrix, wherein each pixel has color information of only one of three colors of RGB, m, n Is an arbitrary integer, the position (m-1, n
-1), (m, n-1), (m-1, n), (m, n)
RGB3 must be assigned to any one of the 4 pixels in 2 rows and 2 columns
Inputting image data containing color information of a color as color input image data; and arranging digital signal values of RGB colors in a predetermined format in correspondence with respective pixel data of the color input image data. A digital signal in which each bit of a digital signal value region representing a color signal is 0 for one of two pixels having the same color information among the four pixels with a predetermined digital signal value A conversion step of outputting a value, and an interpolation step of adding the converted digital signal values of the four pixels and outputting as a digital image signal of a pixel at a position (m, n). Image data processing method. 3. The color image data processing method according to claim 1, wherein two of the four pixels have color information of G color. 4. The color image data processing method according to claim 1, wherein a preset combination of digital signal values corresponding to the color input image data can be changed.