JP2000232655A - Method and device for image processing and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To apply color conversion processing to an image signal obtained from an image pickup device such as a single CCD camera without causing a noise or an artifact. SOLUTION: A low frequency color information generating means 2 generates a color signal C0 denoting low frequency color information from an image signal S0 obtained by a single CCD image pickup device 1 and a high frequency luminance information generating means 3 generates a luminance signal YH denoting high frequency luminance information. A color conversion means 4 applies color conversion processing to the color signal C0 to obtain a converted color signal C1 and a signal processing means 5 generates a processed signal S1 on the basis of the signal C1 and the luminance signal YH.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] The present invention relates to a method of disposing a plurality of types of photoelectric conversion elements having different spectral sensitivities on a single surface.
An image processing method and apparatus for performing image processing on a color image pickup signal obtained by an image pickup device such as a so-called single-chip CCD so that faithful colors can be reproduced, and a program for causing a computer to execute the image processing method are provided. The present invention relates to a recorded computer-readable recording medium.

[0002]

2. Description of the Related Art A device for reading an image recorded on a photographic negative film or a reversal film by a scanner or the like,
In a color image pickup device such as a digital camera, C
The received light is converted into a digital imaging signal using an imaging device such as a CD. The spectral characteristics of a photoelectric conversion element used in such an imaging device generally have a CIE
Does not coincide with the spectral characteristics of the standard observer, ie, the spectral characteristics of the human eye, and accurate color reproducibility can be obtained by directly using the imaging signal obtained by the imaging device for reproduction. Can not do. For this reason, color conversion according to the color reproduction characteristics of a monitor that reproduces an image is performed by a matrix operation as shown in the following equation (1) on each of the RGB color signals that constitute the imaging signal, and further used in the monitor. There has been proposed an image processing apparatus in which gamma correction is performed on a signal value that has been color-converted in accordance with the color developing characteristics of a fluorescent substance (Japanese Patent Laid-Open No. 6-3115).
No. 23). According to this apparatus, accurate color reproducibility can be obtained when an image pickup signal obtained by the image pickup device is reproduced on a monitor.

[0003]

(Equation 1)

In the above equation (1), since the coefficients c, d, f, and g are generally so small as to be negligible, R '= aR + bG, G' = G, B '
= HG + iB is often substituted. Also, the coefficients a, i
Is a positive value, and the coefficients b and h are negative values. Also, the coefficient b,
Since h is a negative value, when calculating R 'and B', the coefficients a and i have values larger than 1 so as not to be smaller than the original signal values of R and B.

On the other hand, as an imaging device such as a CCD,
It is known that a plurality of types of photoelectric conversion elements having different spectral sensitivities are alternately arranged on the same plane (hereinafter, referred to as a single CCD). Here, a photoelectric conversion element having spectral sensitivity to each of R, G, and B, ie, R, G, and B
In the case of a single CCD in which photoelectric conversion elements of each of the G and B channels are alternately arranged, three R, G and B channels of the
A set of the photoelectric conversion elements constitutes one pixel. However, in such a single-chip CCD, since the R, G, and B values of each pixel cannot be obtained at the same pixel position, a color shift or a false color may occur. Further, since the number of photoelectric conversion elements in each channel is smaller than the total number of elements constituting the single CCD, a high-resolution image cannot be obtained. For example, in a single-plate CCD in which photoelectric conversion elements of R, G, and B channels are alternately arranged, the number of photoelectric conversion elements in each channel is only 1/3 of the total number of elements. In comparison, the resolution is reduced to 1/3. For this reason, a method has been proposed in which the signal values of the R, G, and B channels in the portion where the photoelectric conversion element does not exist are obtained by interpolation processing. May cause color shift. In this case, the occurrence of color shift can be prevented by performing the smoothing, but there is a problem that the resolution is deteriorated when the smoothing is performed.

Here, human visual characteristics are more sensitive to luminance than to color. For this reason, a high-frequency luminance signal representing the luminance of each pixel and a low-frequency color signal by the above-described interpolation processing and smoothing processing are generated from the imaging signal obtained by the single-chip CCD, and the luminance signal and the color signal are generated. There has been proposed a method of reconstructing a color image signal by using the method described in Japanese Patent Application Laid-Open Nos. 10-200906 and 9-2009.
No. 65075). According to this method, more information is given to a luminance component having high sensitivity in human visual characteristics, so that a color image signal capable of reproducing an image having an apparently high resolution can be obtained.

[0007]

As described above, a luminance signal and a color signal are generated from an image pickup signal obtained by a single-chip CCD, and apparent resolution is determined by a color image signal obtained by reconstructing these signals. High images can be reproduced. However, the signal obtained at the CCD is
Since it does not match the spectral characteristics of the human eye, it is necessary to perform a matrix operation as in the method described in JP-A-6-311523 to obtain accurate color reproducibility. In this case, a high-frequency luminance signal and a low-frequency color signal are generated from an imaging signal obtained by a single-chip CCD by a method described in the above-mentioned Japanese Patent Application Laid-Open No. 10-200906, and these signals are obtained by reconstructing them. It is conceivable to perform a matrix operation on the obtained color image signal (method 1). In addition, although the exact pixel positions are different, it is conceivable to perform a matrix operation between adjacent color signals in an imaging signal obtained by a single-chip CCD (method 2). This is because when the photoelectric conversion elements are arranged as shown in FIG. 6, R1 ′ = (R1−αG1) / (1−α) B2 ′ = (B2−βG2) / (1−β) (2 ) And the like.

However, Japanese Patent Application Laid-Open No. H10-2009
In the method described in JP-A No. 06-2006 or the like, since a high-frequency luminance signal is generated by estimating from R, G, and B signal values obtained in alternately arranged photoelectric conversion elements, it is finally obtained. There is a possibility that a luminance component that does not exist in the original image may be included as noise in the obtained color image signal. Further, since the coefficients a and i of the matrix are larger than 1 as shown in the above equation (1), when the matrix operation is performed on the color image signal, noise components not included in the original image are obtained. May be emphasized.

In the case of method 2, since signals obtained at different positions are subjected to matrix calculation assuming that the signals are at the same position, the calculation is actually performed using signal values having different phases. And the signal value obtained by the conversion may represent a color that does not actually exist. Also, since the signals obtained in each photoelectric conversion element have not been subjected to processing such as a low-pass filter,
This signal is still affected by aliasing caused by sampling. For this reason, in the converted signal obtained by performing the matrix operation by the above-described method 2, artifacts are likely to occur due to the influence of aliasing distortion.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and is an image capable of performing color conversion processing on an image pickup signal obtained by an image pickup device such as a single-chip CCD without generating noise or artifacts. It is an object of the present invention to provide a computer-readable recording medium on which a program for causing a computer to execute the processing method and apparatus and the image processing method is recorded.

[0011]

An image processing method according to the present invention performs image processing on a color image pickup signal obtained by an image pickup device in which a plurality of types of photoelectric conversion elements having different spectral sensitivities are arranged on a single plane. In the image processing method to be performed, a color signal representing low-frequency color information is generated based on the image signal, a color conversion process is performed on the color signal to generate a converted color signal, and the color signal is generated based on the image signal. A luminance signal representing high-frequency luminance information is generated, and a processed color signal is generated based on the luminance signal and the converted color signal.

Here, “an imaging device in which a plurality of types of photoelectric conversion elements having different spectral sensitivities are arranged on a single surface”
Means an image pickup device such as the above-mentioned single-plate CCD. In addition, each photoelectric conversion element is R (red), G
Not only (green) and B (blue) but also C (cyan), M (magenta), Y (yellow), or CMYG obtained by adding G (green) to CMY may have spectral sensitivity. The arrangement of these photoelectric conversion elements is not limited to a specific one.

Also, "color signal representing low-frequency color information"
Is obtained by an interpolation process or the like as described in the above-mentioned Japanese Patent Application Laid-Open No. 10-200906 or the like, and further obtained by performing a smoothing process to prevent generation of a false color. For this reason, the resolution of the color signal is smaller than the resolution of the image obtained by all the photoelectric conversion elements, and as a result, the color signal represents low-frequency color information.

Further, "color conversion processing" is described in
Refers to a color conversion process by a matrix operation or the like described in US Pat.

The "luminance signal representing high-frequency luminance information" is obtained by estimating from the signal values of each pixel and pixels around it by the method described in Japanese Patent Application Laid-Open No. 10-200906. It refers to the luminance component of the signal value at that pixel. Since the luminance signal is generated for each photoelectric conversion element, the resolution of the luminance signal matches the resolution of all photoelectric conversion elements.

In the image processing method according to the present invention, it is preferable that the luminance signal is generated based on the converted color signal.

An image processing apparatus according to the present invention is an image processing apparatus for performing image processing on a color imaging signal obtained by an imaging device in which a plurality of types of photoelectric conversion elements having different spectral sensitivities are arranged on a single surface. A low-frequency color information generating unit that generates a color signal representing low-frequency color information based on the imaging signal; a color conversion unit that performs a color conversion process on the color signal to generate a converted color signal; High-frequency luminance information generating means for generating a luminance signal representing high-frequency luminance information based on the imaging signal; andsignal processing means for generating a processed color signal based on the luminance signal and the converted color signal. It is a feature.

In the image processing apparatus according to the present invention, it is preferable that the high-frequency luminance information generating means is means for generating the luminance signal based on the converted color signal.

The image processing method according to the present invention may be provided by being recorded on a computer-readable recording medium as a program for causing a computer to execute the image processing method.

[0020]

According to the present invention, a color signal representing low-frequency color information and a luminance signal representing high-frequency luminance information are generated from a color imaging signal obtained by an imaging device such as a single-chip CCD. Then, a color conversion process is performed only on the color signal to obtain a converted color signal, and a processed color signal is obtained based on the luminance signal and the converted color signal. As described above, in the present invention, since the color conversion process is performed only on the color signal, the high-frequency luminance component estimated as in the above method 1 is not emphasized, and thereby noise is emphasized. A processed color signal can be obtained without any processing. Further, the color signal represents low-frequency color information, and the influence of aliasing has been removed, so that there is no risk of artifacts as in the above method 2. Therefore, it is possible to obtain a processed color signal having accurate color reproducibility and capable of reproducing a high-resolution and high-quality image.

[0021]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention. FIG.
As shown in the image processing apparatus according to the embodiment of the present invention,
Image processing is performed on signal values obtained by the respective photoelectric conversion elements constituting the single-chip CCD 1, and a color signal C0 representing low-frequency color information is generated from an image signal S0 composed of the signal values. Low-frequency color information generating means 2 for generating a high-frequency luminance information YH representing high-frequency luminance information from the image signal S0, and a color signal C0 generated by the low-frequency color information generating means 2. Conversion processing means 4 for performing a color conversion process to obtain a converted color signal C1
And a signal processing means 5 for generating a processed signal S1 from the converted color signal C1 and the luminance signal YH.

The image processing apparatus according to the present invention may be provided in an image pickup apparatus for reading an image from a digital camera or a film, such as a monitor or a printer for reproducing an image signal obtained by the image pickup apparatus. May be provided in the reproducing apparatus of (1). Further, the image processing apparatus may be used alone.

FIG. 2 is a view showing the arrangement of the photoelectric conversion elements of the single-plate CCD 1. FIG. 2A is a diagram in which R, G, and B channel photoelectric conversion elements having spectral sensitivity are alternately arranged in R, G, and B, and FIG.
The channels are arranged in every other row in the horizontal direction, and R,
The B channels are alternately arranged. FIG.
FIG. 2 (c) shows an arrangement in which C, M, Y channel photoelectric conversion elements having spectral sensitivities are alternately arranged in C, M, Y. FIG. The added photoelectric conversion elements are alternately arranged. Here, since RGB and CMY define the chromaticity of each primary color, they can be mutually converted. Therefore, the image processing apparatus according to the present invention may perform image processing on the image signal S0 obtained from the single-plate CCD 1 of any arrangement, but in the present embodiment, it is shown in FIG. Description will be made assuming that processing is performed on an image signal S0 obtained in the single-plate CCD 1 having an array of photoelectric conversion elements.

The low-frequency color information generating means 2 generates a color signal C0 as follows. First, the color signal C0
The generation of the R color signal (R signal) that constitutes is described. In the arrangement of the photoelectric conversion elements shown in FIG.
FIG. 3 shows a state in which only the channel elements are extracted. Since there is no R signal at the position of the element indicated by X in FIG. 3 (hereinafter referred to as pixel position X), the R signal at the pixel position X is converted to the single-chip CCD 1 based on the value of the R signal at the neighboring pixel position. By performing an interpolation operation in the vertical and horizontal directions. As this interpolation operation, in addition to linear interpolation, a higher-order interpolation operation such as a B-spline interpolation operation that emphasizes smoothness and a Cubic spline interpolation operation that emphasizes sharpness can be applied.

Here, the Cubic spline interpolation calculation and the B spline interpolation calculation will be described. The image signal S0 used in the present embodiment is obtained by sampling points (pixels) X _k−2 , X _k−1 , X _k , X _{k + 1} , and X _k arranged in one direction and sampled at equal intervals. the corresponding signal value in the _{k-2 ... (S k-} 2, S k-1, S k, S k + 1 S k + 2 ...) for those having. Cubic spline interpolating operation, the interpolation data in the original sampling points (pixels) X _k to X k of the third order which represents the interpolated data Y 'of + ₁ interpolation point provided between X _p Cubic spline interpolation formula (3) Y _k-1 ,
Interpolation coefficients c respectively corresponding to Y _k , Y _{k + 1} , and Y _{k + 2
k-1} , _ck , _{ck + 1} , _{ck + 2} are obtained by the following calculations.

Y ′ = c _k−1 Y _k−1 + c _k Y _k + c _{k + 1} Y _{k + 1} + c _{k + 2} Y _{k + 2} (3) c _k−1 = (− t ³ + 2t ² −t) ) / 2 _ck = (3t ³ -5t ² +2) / 2 _{ck + 1} = (-3t ³ + 4t ² + t) / 2 _{ck + 2} = (t ³ −t ² ) / 2 (where t ( 0 ≦ t ≦ 1) means that the grid interval is 1 and the pixel X
_The position of the interpolation point _{Xp in} the pixel _{Xk + 1} direction with respect to _k is shown. In the B-spline interpolation operation, the interpolation data Y _k in the cubic B-spline interpolation operation expression (4) representing the interpolation data Y ′ of the interpolation point X _p provided between the original sampling points X _{k to} X _{k + 1 is} calculated. Interpolation coefficients b _k−1 , b _k , b _{k + 1} , and b _{k + 2} corresponding to ₋₁ , Y _k , Y _{k + 1} , and Y _{k + 2} , respectively, are obtained by the following calculations. .

Y ′ = b _k−1 Y _k−1 + b _k Y _k + b _{k + 1} Y _{k + 1} + b _{k + 2} Y _{k + 2} (4) b _k−1 = (− t ³ + 3t ² −3t + 1) _{) / 6 b k = (3t} 3 -6t 2 +4) / 6 b k + 1 = (- 3t 3 + 3t 2 + 3t + 1) / 6 b k + 2 = t 3/6 ( where, t (0 ≦ t ≦ 1 ) Indicates that the grid interval is 1 and the pixel X
_The position of the interpolation point _{Xp in} the pixel _{Xk + 1} direction with respect to _k is shown. For the G signal and the B signal, the signal value at the pixel position X where the signal value does not exist is obtained by the above-described interpolation calculation. Then, the R signal, the G signal, and the B signal at each pixel position obtained by the interpolation processing are output as the color signal C0. By performing the interpolation calculation in this manner, R, G,
Although the signal value of B can be obtained, since it is obtained by the interpolation operation, the substantial information (band) does not increase and the image shows a slightly blurred image. Therefore, the color signal C0 indicates low-frequency color information.

In addition, at the edge portion where the density changes abruptly on the image pickup surface of the single-plate CCD 1, a false color which does not actually exist is generated by performing the interpolation operation, and the processed signal finally obtained is thereby obtained. Artifacts may occur in S1. For this reason, it is preferable that the low-frequency color information generating means 2 performs a smoothing process using a low-pass filter after performing the interpolation operation to make the change in the signal value at the edge portion gentle. In this low-pass filter, when the Nyquist frequency when sampling is performed by all the photoelectric conversion elements of the single-plate CCD 1 is fs, the original R signal, G signal, and B signal each have only 1/3 of all signals. From 1
It is preferable to cut high-frequency components of / 3 fs or more.

The signal value at each photoelectric conversion element position (hereinafter referred to as a pixel position) thus obtained is
The color difference values Cr and Cb may be obtained as the color signal C0 by converting into the YCC luminance color difference space by the following equation (5). Here, since RGB and YCC define the chromaticity of each primary color, they can be mutually converted.

[0030]

(Equation 2)

The high-frequency luminance information generating means 3 generates a luminance signal YH by, for example, a method described in Japanese Patent Application Laid-Open No. 10-200906. That is, the luminance value at each pixel position is estimated from the R, G, and B signal values at each pixel position and the average color difference values Cr and Cb near each pixel position to obtain a high-frequency luminance signal YH. Here, since the luminance signal YH is obtained for each pixel, the luminance signal YH is a signal having the same information (band) as the image signal S0 obtained from all the elements. When the low-frequency color information generating means 2 determines the color signal C0 as the YCC luminance color difference signal, the color difference values Cr and Cb may be used as they are, and the color difference values Cr and Cb constituting the color signal C0 may be used as they are. When the signal C0 is obtained as an RGB color signal, the high-frequency luminance information generating means 3 calculates the color difference values Cr,
What is necessary is just to obtain Cb.

More specifically, the luminance signal YH is calculated by the following operation.
Is generated. Here, R in the luminance signal YH
The case where the luminance value YH0 at the pixel position where the signal is obtained is described. The signal value of the R signal obtained in a certain photoelectric conversion element is R0, and the color difference values Cr and Cb corresponding to the pixel position where the R signal is obtained are known variables, the luminance value YH0 at the pixel position, and the pixel position The G value and the B signal in are set as unknown variables, and the equation (5) showing the relationship between RGB and YCC is solved, and the luminance value YH0 at this pixel position can be obtained by the following equation (6).
Similarly, the luminance value YH0 at the pixel position where the G signal and the B signal are obtained can be obtained by the following equations (7) and (8). Note that, in equations (6) to (8), nij is the element in row i and column j of the matrix A in equation (5).

[0033]

(Equation 3)

The luminance value YH0 of each pixel position estimated in this manner is converted into a luminance signal YH representing high-frequency luminance information.
Output as

It should be noted that a signal obtained by subtracting the luminance value Y obtained in Expression (5) from the luminance value YH0 obtained by Expressions (6) to (8) may be obtained as the luminance signal YH. . Here, since the luminance value YH0 obtained by the above equations (6) to (8) is obtained at all pixel positions, it has a band from 0 to the Nyquist frequency fs, and is obtained by the equation (5). Since the luminance value Y is obtained from the color signal of each pixel obtained by interpolation, it has a band from 0 to 1/3 fs. Therefore, the luminance signal YH obtained by subtracting the luminance value Y from the luminance value YH0 represents a band from 1/3 fs to fs.

When generating the luminance signal YH, the color signal C0 generated by the low-frequency color information generating means 2 may be referred to. That is, a direction vector of a signal value change of each of the RGB color signals is calculated, and a position where the direction vector is approximated in each signal value has a gray density change in which all the RGB colors change in the same direction. Can be considered as Therefore, for this area, the luminance value YH0 is obtained by the above equations (6) to (8), and for the other area, the luminance value Y obtained by the above equation (5) based on the color signal C0 is adopted. The luminance signal YH may be constituted by the values YH0 and YH. As a result, a luminance component that does not exist in the original signal is not generated in an area where there is no change in gray density, and as a result, it is possible to prevent the occurrence of an artifact due to the erroneous generation of the luminance component. it can.

Further, based on the color signal C0, a luminance value is obtained by the above equation (5), and a differential value of the luminance value Y is obtained. Only the position where this differential value is high is calculated by the above equations (6) to (8). YH0 is obtained, and for other positions, the luminance value Y obtained by the above equation (5) is used.
The luminance signal YH may be constituted by the luminance values YH0 and Y.

In the color conversion means 4, the matrix operation shown in the above equation (1) is performed on the color signal C0 generated by the low-frequency color information generation means 2 to convert the color signal C0.
1 is obtained. In this matrix operation, a color conversion process for converting a color signal C0 having a spectral sensitivity depending on the single-plate CCD 1 into a signal having a spectral sensitivity corresponding to human vision is performed. The coefficients of the matrix in equation (1) are set in advance based on experiments. In addition,
The color conversion processing in the color conversion means 4 is not limited to the matrix operation, and color conversion may be performed more strictly using a three-dimensional lookup table.

In the signal processing means 5, the converted color signal C
1 and the luminance signal YH to generate a processed signal S1. Specifically, when the high-frequency luminance information generating means 3 obtains the luminance signal YH in the band from 0 to fs, the luminance value Y is calculated from the converted color signal C1 by the above equation (5). The luminance signal YL including the luminance value Y is subtracted from the luminance signal YH to obtain a differential luminance signal ΔY, and the differential luminance signal ΔY is added to the converted color signal C1 to generate the processed signal S1.

Further, the luminance signal YH is changed from 1/3 fs to fs.
In this case, the luminance signal YH may be simply added to the converted chrominance signal C1. Also, the converted color signal C1
Is converted into a YCC luminance / chrominance space by Expression (5), the luminance value Y obtained by the conversion is replaced with a luminance value YH0 constituting the luminance signal YH, and the substituted YCC luminance / chrominance signal is replaced by Expression (5). An RGB color signal may be obtained by inverse conversion, and this color signal may be used as the processed signal S1. Further, when the color signal C0 is obtained as a YCC luminance color difference signal, RGB is obtained by inverse conversion of equation (5) from the converted color difference values Cr and Cb obtained by the color conversion and the luminance value YH0 constituting the luminance signal YH. May be obtained, and the processed signal S1 may be constituted by this color signal.

Next, the operation of this embodiment will be described. FIG. 4 is a flowchart showing the operation of the present embodiment. First, the image signal S0 obtained in the single-chip CCD 1
, A low-frequency color information generating means 2 generates a color signal C0 representing low-frequency color information (step S1). Similarly, a luminance signal YH representing high-frequency luminance information is generated from the image signal S0 by the high-frequency luminance information generating means 3 (step S2). Then, color conversion processing is performed on the color signal C0 generated by the low-frequency color information generation means 2 by the color conversion means 4 to generate a converted color signal C1 (step S3). Then, the signal processing means 5 generates a processed signal S1 from the converted color signal C1 and the luminance signal YH (step S4), and ends the processing.

The processed signal S1 is reproduced by a reproducing apparatus such as a monitor or a printer. If necessary, color conversion (for example, gamma conversion) suitable for the color reproduction characteristics of the reproducing apparatus is performed.

As described above, in the present embodiment, since the color conversion processing is performed only on the color signal C0, the high-frequency luminance component estimated as in the above method 1 is not emphasized. Thus, the processed signal S1 can be obtained without emphasizing noise. The color signal C
0 represents low-frequency color information, and since the influence of aliasing has been removed, there is no risk of occurrence of artifacts. Therefore, it is possible to obtain the processed signal S1 having accurate color reproducibility and capable of reproducing a high-resolution and high-quality image.

Note that, in the above-described embodiment,
A luminance signal YH representing high-frequency luminance information is generated by the method described in Japanese Patent Application Laid-Open No. H09-200906.
The luminance signal YH can also be generated by the method described in US Pat. Here, the method of generating the luminance signal YH described in JP-A-9-65075 is to determine the luminance value YH0 at a certain pixel position based on the signal value at a pixel position adjacent to this pixel position. Here, the pixel position for obtaining the luminance value YH0 is Xij, and the signal value at the pixel position Xij is Sij (i indicates the pixel position in the vertical direction in FIG. 2 and j indicates the pixel position in the horizontal direction in FIG. 2). , The luminance value YH0 is determined by the following equation (9) as an average value with the signal value Sx at any one of the eight pixels adjacent to the pixel position Xij.

YH0 = １ ／ (Sij + Sx) (9) Also, as shown in FIG. 5, 3 in any direction among the four directions including the pixel position Xij for obtaining the luminance value YH0
The signal value of the pixel may be obtained by weighting and adding according to equation (10).

YH0 = 1/4 (Sa + 2Sij + Sb) (10) where Sa and Sb are signal values at two adjacent pixel positions in the direction shown in FIG. Although the matrix operation described in JP-A-6-31523 is performed, the present invention is not limited to this, and various operations can be applied.

Further, in the above embodiment, the case where the high-frequency luminance information generating means 3 generates the luminance signal YH with reference to the color signal C0 has been described. The converted color signal C1 may be obtained by color conversion, and the luminance signal YH may be generated by referring to the converted color signal C1 instead of the color signal C0.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram showing an arrangement of photoelectric conversion elements.

FIG. 3 is a diagram showing a pixel position of an R signal.

FIG. 4 is a flowchart showing the operation of the embodiment.

FIG. 5 is a diagram for explaining how to obtain a luminance value.

FIG. 6 is a view showing an arrangement of photoelectric conversion elements in a method 2;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Single board CCD 2 Low frequency color information generation means 3 High frequency luminance information generation means 4 Color conversion means 5 Signal processing means

Claims (6)

[Claims] 1. An image processing method for performing image processing on a color imaging signal obtained by an imaging device in which a plurality of types of photoelectric conversion elements having different spectral sensitivities are arranged on a single surface, comprising: Generating a color signal representing low-frequency color information; performing a color conversion process on the color signal to generate a converted color signal; generating a luminance signal representing high-frequency luminance information based on the imaging signal; An image processing method, wherein a processed color signal is generated based on the luminance signal and the converted color signal. 2. The image processing method according to claim 1, wherein the luminance signal is generated based on the converted color signal. 3. An image processing apparatus that performs image processing on a color imaging signal obtained by an imaging device in which a plurality of types of photoelectric conversion elements having different spectral sensitivities are arranged on a single surface, wherein: Low-frequency color information generation means for generating a color signal representing low-frequency color information; color conversion means for performing a color conversion process on the color signal to generate a converted color signal; and high-frequency based on the imaging signal An image processing apparatus comprising: a high-frequency luminance information generating unit that generates a luminance signal representing the luminance information of the image signal; and a signal processing unit that generates a processed color signal based on the luminance signal and the converted color signal. . 4. The image processing apparatus according to claim 3, wherein said high-frequency luminance information generating means is means for generating said luminance signal based on said converted color signal. 5. A program for causing a computer to execute an image processing method for performing image processing on a color imaging signal obtained by an imaging device in which a plurality of types of photoelectric conversion elements having different spectral sensitivities are arranged on a single surface. A computer-readable recording medium on which is stored a computer-readable recording medium, wherein the program generates a color signal representing low-frequency color information based on the imaging signal; and performs a color conversion process on the color signal to convert the color. A step of generating a signal, a step of generating a luminance signal representing high-frequency luminance information based on the imaging signal, and a step of generating a processed color signal based on the luminance signal and the converted color signal. A computer-readable recording medium characterized by the above-mentioned. 6. The computer-readable recording medium according to claim 5, wherein the step of generating the luminance signal is a step of generating the luminance signal based on the converted color signal.