JP2000217124A - Image pickup device and image processing method for the image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce false colors by relatively easy constitution by performing conversion on the color space of a different color specification system from the color interpolation of a color plane, acting on the ones other than converted luminance component signals, reducing isolated false color components generated in a color interpolation means and compressing image data for which the isolated false color components are reduced through an isolated point removal means. SOLUTION: For U and V signals which are the output of an RGB/YUV conversion circuit 13, filtering by median filters 17a and 17b is executed. Then, in order to perform JPEG compression, low-pass filtering is applied to color difference information in LPFs 14a and 14b, and the band of color difference signals is limited. Thereafter, thinning to a prescribed format is performed in a thinning circuit 15, and compression is performed in the JPEG compression 16. As isolated point removal filtering, filtering for preserving may an edge may be used, which is other than the median filter.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus such as a digital camera and a digital video, and an image processing method in the image pickup apparatus.

[0002]

2. Description of the Related Art In conventional single-panel color digital cameras, there are almost no cameras that can exchange lenses, and almost all of them are lens-integrated systems. Therefore, an optical low-pass filter, an IR cut filter, and the like are provided before the CCD that generates an image signal by light passing through the lens, and a certain effect against moiré and false colors is obtained. However, in the case of a camera having a replaceable lens, the IR cut filter can be disposed in front of the CCD by forming a thin film on the glass surface of the CCD. However, the optical low-pass filter is C
If a space for arranging in front of the CD is taken, the size of the camera body is substantially increased. In addition, by inserting such an optical low-pass filter, moiré and false colors can be reduced to some extent, but there is a problem that the spatial frequency of the image is lowered, and as a result, sharpness of focus like a silver halide photograph is lost. Would. For that reason, the importance of an optical system without an optical low-pass filter is increasing.

[0003] In addition, even if such an optical low-pass filter is provided, in the case of a single-chip CCD digital camera, as represented by the Bayer arrangement, R (green) is applied to G (green).
There are few images of (red) and B (blue), and the color interval is widened, so that a false color is generated at the time of color interpolation. As a conventional color interpolation method, there is also a method of creating three color planes using a digital filter or the like. However, the number of taps of the filter is limited by hardware, and the original resolution of image data is sufficiently obtained. I couldn't do that.

Therefore, conventionally, US Pat. No. 5,373,322
It has been proposed to obtain image data having a high resolution by performing image processing such as represented by U.S. Pat. Although these techniques do not improve against moiré, they are said to be effective in reducing false colors.

[0005]

However, even when such a conventional method is employed, it is possible to completely eliminate an isolated false color that occurs during color interpolation, especially around a small character. Could not. Further, compression as it is, such as JPEG, may cause block noise or the like. Also, if the frequency band of the color difference signal is forcibly restricted before being compressed by JPEG or the like, the image is blurred,
Although the generation level of the false color is lowered, there is a problem that the false color spreads to surrounding pixels. In order to eliminate such false colors, a three-panel camera using a total of three image sensors for each color component is ideal. However, such a configuration increases the size of the camera itself and increases costs. .

The present invention has been made in view of the above conventional example, and has as its object to provide an image pickup apparatus and an image processing method in the image pickup apparatus which can reduce false colors generated by color interpolation processing.

It is another object of the present invention to provide an image pickup apparatus capable of preventing an increase in the size and cost of the image pickup apparatus and reducing false colors, and an image processing method in the image pickup apparatus.

It is another object of the present invention to provide an imaging apparatus capable of reducing false colors with a relatively simple configuration and an image processing method in the imaging apparatus.

[0009]

In order to achieve the above object, an image pickup apparatus according to the present invention has the following arrangement. That is, an image sensor, an image forming means for forming an image on the image sensor, an A / D converter for converting an image signal output from the image sensor into a digital signal, and the A / D converter Color interpolation means for color-interpolating the digital signal converted by the above to generate image data of a plurality of color planes, and a first color space for converting the color space of the plurality of color planes to another color space of a color system A conversion unit, an isolated point removing unit that acts on a component other than the luminance component signal converted by the first color space conversion unit to reduce an isolated false color component generated by the color interpolation unit, and an isolated point removing unit. Compression means for compressing image data in which isolated false color components have been reduced.

In order to achieve the above object, an image processing method in an image pickup apparatus according to the present invention includes the following steps.
That is, an image processing method in an image pickup apparatus having an image sensor and generating an image signal corresponding to a video image formed on the image sensor, wherein the image signal output from the image sensor is converted into a digital signal An A / D conversion step, a color interpolation step of color-interpolating the digital signal converted in the A / D conversion step to create image data of a plurality of color planes, and another table from the color space of the plurality of color planes First color space conversion means for converting to a color-based color space, and reducing isolated false color components generated in the color interpolation step by acting on a signal other than the luminance component signal converted in the first color space conversion step. The image processing apparatus further includes an isolated point removing step and a compression step of compressing the image data in which the isolated false color components have been reduced in the isolated point removing step.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

Embodiment 1 FIG. 1 shows Embodiment 1 of the present invention.
FIG. 2 is a block diagram mainly showing the configuration of an image processing circuit in the digital camera of FIG.

The light 1 incident on the digital camera of the present embodiment passes through a lens 2, the light amount is adjusted by an aperture 3, and an image pickup device such as a CCD or a CMOS is opened only when a shutter (not shown) is open. (Hereinafter, abbreviated as CCD) 5 is exposed. Before the light 1 is exposed to the CCD 5, the light 1
The region on the long wavelength side is cut by an IR (infrared cut-off) filter 4 so that 5 does not detect light in the infrared region. The light thus exposed on the CCD 5 causes the CCD
5 stores an amount of charge corresponding to the intensity of light. This charge amount is amplified to a predetermined gain by the CDS / AGC 6 and converted to digital data by the A / D converter 7. The image data converted into digital data in this manner is adjusted in RGB gain by the white balance circuit 8 and is generated by the color interpolation circuit 9 into, for example, three color (RGB) planes. The image data of the three color planes of RGB is adjusted by the masking processing circuit 10 with respect to the hue of each color of RGB, and then the gamma conversion circuit 11
Thus, the processing necessary for displaying on a display or the like is performed.

Next, the false color removal processing circuit 12 reduces the false colors generated by the color interpolation in the color interpolation circuit 9. Next, the image data is subjected to a compression process such as JPEG because the number of data is large if the RGB 3 planes remain unchanged. Here, first, the RGB / YUV conversion circuit 13 converts the RGB signals into Y color difference signals, and performs low-pass filtering on each of the color difference signals U and V to limit the band with each of the LPFs 14a and 14b. Do. And Y:
From U: V = 4: 4: 4 to Y: U: V = 4: 4: 2 or Y:
A process adapted to a predetermined compression method such as U: V = 4: 1: 1 is performed by the thinning circuit 15 and finally the JPEG circuit 1
6 is JPEG compressed.

FIG. 2 is a block diagram showing a configuration of the false color removal processing circuit 12 according to the first embodiment.

Here, for example, R
It is assumed that image data of a GB plane has been generated. R
The RGB / XYZ conversion circuit 12a converts the RGB color space to the XYZ color space. This conversion is performed using the following equations (1) to
Performed according to (3).

X = 2.7689 × R + 1.7517 × G + 1.1302 × B (1) Y = 1.0000 × R + 4.5907 × G + 0.0601 × B (2) Z = 0.0000 × R + 0.0565 × G + 5 .5943 × B (3) Then, XYZ / L * a * b * conversion circuit 12b
Convert from YZ color space to L * a * b * color space. The conversion equations in this case are as shown in the following equations (4) to (6).

L * = 116 (Y / Yo) -16 (Y / Yo> 0.008856) (4) a * = 500 [(X / Xo)） 1 / 3− (Y−Yo) ^ 1/3 ] (5) b * = 200 [(Y / Yo) ^ 1/3-(Z-Zo) ^ 1/3] (6) Here, Xo, Yo, and Zo indicate tristimulus values of a perfect diffusion surface. , And “^ 1/3” indicates the (1/3) th power.

Here, a median filter (intermediate value filter) is applied to each of a * and b * of L * a * b * by median filters 12c and 12d as isolated point filters.

FIGS. 3A and 3B are diagrams for explaining the principle of processing in the false color removing units 13a and 13b according to the first embodiment. Here, FIG. 3A shows 3 × 3 pixels around the pixel of interest, and FIG. 3B shows a diagram in which those pixel values are arranged in ascending order.

Here, for example, for the target pixel a * 22, 3 × 3 pixels around the target pixel a * 22 are taken out, and their pixel values (color difference values) are arranged in ascending order. Here, the value “10” of a * 33, which is an almost intermediate value among these nine pixels, is replaced with the value “20” of the pixel of interest. The median filters 13a and 13b perform such processing. This makes it possible to reduce false colors without substantially lowering the resolving power. Incidentally, the configuration of the median filter itself is described in, for example, JP-A-5-233804 and JP-A-5-1243.
As is well known as shown in Japanese Patent Publication No. 7 and the like, the description thereof is omitted.

Then, the L * a * b * / XYZ conversion circuit 12e converts the color space from the L * a * b * color space to the XYZ color space again. Finally, the XYZ / RGB conversion circuit 12f
Returns from the XYZ color space to the RGB color space. As a result, a median filter can be applied only in a color space that has little effect on luminance (resolution), so that a reduction in image resolution can be prevented.

[Second Embodiment] FIG. 4 is a block diagram showing a configuration according to a second embodiment of the present invention.

In the first embodiment, the image data in the RGB color space output from the false color removal processing circuit 12 is
In order to perform compression, the color space was converted from RGB to YUV color space by the RGB / YUV conversion circuit 13 and JPEG compression was performed. This YUV conversion is performed according to the following equations (7) to (9).

Y = 0.29900 × R + 0.58700 × G + 0.11400 × B (7) U = −0.16874 × R−0.33126 × G + 0.50000 × B (8) V = 0.50000 × R−0.41869 × G−0.08131 × B ( 9) In the second embodiment, instead of providing the circuit 12, R
The U / V signals output from the GB / YUV conversion circuit 13 are subjected to median filters (intermediate value filters) 17a,
7b. And JPEG
For compression, the LPFs 14a and 14b apply a low-pass filter to the color difference information to limit the band of the color difference signal. After that, the thinning circuit 15 thins out the YUV 444 into a predetermined format such as YUV 422 or YUV 411, and the JPE
G compression 16 is used for compression.

As the isolated point removing filter, in addition to the above-described median filter, filtering called edge preserving smoothing may be used. That is, a window W (i, j) is set around a certain pixel f (x, y), and an output g (x, y) is calculated. Here, assuming that the size of the window is m × n, g (x,
y) = ΣΣW (i, j) · f (xi, yi). Here, the first Σ indicates the sum of i from 1 to m, 2
The Σ indicates the sum of j from 1 to n. Here, for example, a matrix in which data “1” is arranged in 3 × 3 is represented by M
And W = M / 9. Alternatively, as an isolated point removal filter, another edge-preserving smoothing (Ma
ximum Homogenity Smoothing), which is a smoothing for selecting the most uniform area in the vicinity of a certain pixel. For each block average value μi and variance σi, m (arbitrary), weighting factor Fi = ｛min (Σi)｝ / σi, and the output g (x, y) is g (x, y) = {(Fi m-th power) μi} /｝ Fi m-th power. Here, Σ indicates the sum of i = 1 to 8. In addition, filtering called hysteresis smoothing may be used. This creates a hysteresis characteristic for the data value, thereby absorbing noise. In any of the above filters,
When a certain pixel output is significantly different from a peripheral pixel output, the pixel output may be set as an isolated point and interpolation or smoothing may be performed using the peripheral pixel output.

Even if the present invention is applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), an apparatus (for example, a copying machine, a facsimile machine) comprising one device Etc.).

Another object of the present invention is to provide a storage medium storing a program code of software for realizing the functions of the above-described embodiments to a system or an apparatus, and a computer (or CPU or MPU) of the system or apparatus to supply the storage medium. This is also achieved by reading and executing the program code stored in the storage medium.

In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.

As a storage medium for supplying the program code, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD
-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

When the computer executes the readout program code, not only the functions of the above-described embodiment are realized, but also an OS (Operating System) running on the computer based on the instruction of the program code. ) Performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

Further, after the program code read from the storage medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, based on the instruction of the program code, The case where the CPU provided in the function expansion board or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments is also included.

As described above, according to this embodiment, even if an optical low-pass filter is not provided in front of an image pickup device such as a CCD, L * a * b * and YU
A color space such as V is converted to a color space such as a * and b * signals, or a color difference signal such as a U and V signal is filtered by an isolated point removal filter to reduce false color components. By performing compression such as JPEG, it is possible to obtain an image with less block distortion or the like and reduced false color components.

[0034]

As described above, according to the present invention, it is possible to reduce the false color generated by the color interpolation processing, and to reduce the block noise accompanying the compression.

Further, according to the present invention, it is possible to prevent an increase in the size and cost of the image pickup apparatus and effectively reduce false colors without deteriorating the resolution.

Further, according to the present invention, false colors can be reduced with a relatively simple configuration, so that the optical low-pass filter can be simplified or omitted, and a high-resolution image can be obtained. There is.

[Brief description of the drawings]

FIG. 1 is a block diagram mainly illustrating a configuration of an image processing unit of a digital camera according to Embodiment 1 of the present invention.

FIG. 2 is a block diagram illustrating a configuration of a false color removal processing circuit according to the present embodiment.

FIG. 3 is a conceptual diagram illustrating a false color removal process according to the embodiment of the present invention.

FIG. 4 is a block diagram showing a part of an image processing unit of a digital camera according to Embodiment 2 of the present invention.

Claims (14)

[Claims] An imaging device configured to form an image on the imaging device; an A / D conversion device configured to convert an image signal output from the imaging device into a digital signal; Color interpolation means for color-interpolating the digital signal converted by the D conversion means to create image data of a plurality of color planes; and a color interpolation means for converting the color space of the plurality of color planes to another color space of a color system. One color space conversion means; an isolated point removal means for acting on a signal other than the luminance component signal converted by the first color space conversion means to reduce an isolated false color component generated by the color interpolation means; An image pickup apparatus comprising: compression means for compressing image data in which isolated false color components are reduced by the removal means. 2. The imaging apparatus according to claim 1, wherein the isolated point removing unit includes an isolated point removal filter that replaces a target pixel with a substantially intermediate value of pixel values of peripheral pixels of the target pixel. . 3. The imaging apparatus according to claim 2, wherein the isolated point removal filter includes a median filter or a median filter. 4. The imaging apparatus according to claim 1, wherein said color interpolation means creates image data of an RGB plane. 5. The first color space conversion means converts an RGB color space into an L * a * b * color space, and the isolated point removal means applies the isolated point removal filter to an a * b * signal. The imaging device according to claim 4, wherein the imaging is performed. 6. The imaging apparatus according to claim 1, further comprising a second color space conversion unit that converts a signal that has passed through the isolated point removal unit into a YUV color space. 7. The imaging apparatus according to claim 6, wherein the compression unit compresses the image data converted into a YUV color space. 8. An image processing method in an image pickup apparatus having an image pickup device and generating an image signal corresponding to a video image formed on the image pickup device, wherein the image signal output from the image pickup device is a digital signal. A / D conversion step of converting the digital signal into a color signal, and color-interpolating the digital signal converted in the A / D conversion step.
A color interpolation step of creating image data of a plurality of color planes; a first color space conversion means for converting a color space of the plurality of color planes into a color space of another color system; and a first color space conversion step An isolated point removing step of reducing the isolated false color component generated in the color interpolation step by acting on a signal other than the luminance component signal converted in the above step; and image data in which the isolated false color component is reduced in the isolated point removing step. And a compression step of compressing the image. 9. The image processing method according to claim 8, wherein in the isolated point removing step, the target pixel is replaced with a substantially intermediate value of pixel values of peripheral pixels of the target pixel. 10. The image processing method according to claim 8, wherein the isolated point removing step performs filtering using a median filter or a median filter. 11. The image processing method according to claim 8, wherein in the color interpolation step, image data of an RGB plane is created. 12. The method according to claim 1, wherein in the first color space conversion step, the RGB color space is converted to an L * a * b * color space, and in the isolated point removal step, the filtering is performed on the a * b * signal. The image processing method according to claim 10, wherein 13. The image processing method according to claim 8, further comprising a second color space conversion step of converting the signal processed in the isolated point removal step into a YUV color space. 14. The image processing apparatus according to claim 1, wherein in the compression step, the image data converted into the YUV color space is compressed.
3. The image processing method according to 3.