JP5841358B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method, and more particularly to an image processing apparatus and an image processing method for creating a luminance signal and a color signal.

  2. Description of the Related Art Conventionally, a technique for reducing the amount of image data by thinning out color signals by utilizing human visual characteristics that sensitivity to color signals is lower than luminance signals has been widely used. A method of interpolating the thinned color signal based on the color signal of the adjacent pixel so that the color smoothness is not impaired even for the image signal from which the color signal is thinned. It is known (see Patent Document 1).

  In addition, a technique for reducing noise of an original image signal by dividing the image signal into a plurality of frequency bands and synthesizing the image signals processed for each frequency band is known (see Patent Document 2). In Patent Document 2, a noise component is reduced while preserving a wideband edge component by synthesizing a signal with a reduced noise component while preserving the edge component in each of n frequency bands.

JP-A-10-75462 JP 2008-015741 A

  In the technique of Patent Document 2, it is necessary to store image signals in four planes of R / G1 / G2 / B, and further, it is necessary to store image signals of each plane for each frequency band. Therefore, a large-capacity memory is required, which is not preferable in terms of cost. In addition, since the required memory access amount is large, in the case of processing that requires high speed such as continuous shooting, the upper limit of the memory access bandwidth may be reached and the processing speed may be reduced.

  In order to alleviate this problem, for example, it is conceivable to apply a color signal thinning technique as shown in Patent Document 1 to the configuration of Patent Document 2 to reduce the required memory capacity and memory access amount.

  However, in the technique of Patent Document 1, since the thinned color signal is restored by interpolating from the color signal of the adjacent pixel, there is a difference between the frequency band of the color signal and the frequency band of the luminance signal, which exists in the subject. There is a problem that color streaks are not generated.

  Therefore, even when applied to the configuration of Patent Document 2, in all of the image signals divided for each frequency band, color streaks due to the difference between the frequency band of the interpolated color signal and the frequency band of the luminance signal are present. Will occur. As a result, when image signals in each frequency band are synthesized, it is expected that the color streaks generated in the image signal in the low frequency band spread and the image quality is deteriorated.

  The present invention has been made in order to solve the problems expected in such a combination of conventional techniques, and in an image processing apparatus and an image processing method for applying signal processing to color signals that have been thinned and interpolated. An object of the present invention is to realize suppression of image quality degradation.

The above-described object is thinned out by conversion means for converting an image signal obtained from an image sensor having a color filter into a first signal and a second signal, thinning means for thinning the first signal, and thinning means. a first signal, storage means for storing the second signal, interpolation means for interpolating the first signal read from the memorize means, with respect to the second signal read from the storage means, decimating a decimation interpolation means for performing means similar decimation and interpolation and decimation and interpolation performed on a first signal by interpolation means, a first signal interpolated by interpolation means, which is thinned and interpolated at a thinning interpolation means by using the second signal, it is achieved by an image processing apparatus characterized by having a row cormorant signal processing means the signal processing.

  With such a configuration, according to the present invention, in an image processing apparatus and an image processing method that apply signal processing to color signals that have been thinned out and interpolated, it is possible to reduce the amount of processing data and suppress deterioration in image quality. be able to.

FIG. 4A is a block diagram illustrating a configuration of an imaging apparatus as an example of an image processing apparatus according to an embodiment of the present invention, and FIG. The block diagram which shows the structural example of the signal processing circuit of FIG. The figure for demonstrating the example of a structure and operation | movement of RB thinning-out circuit in the signal processing part of FIG. FIG. 3 is a diagram for explaining an example of the configuration and operation of a Y thinning interpolation circuit in the signal processing unit of FIG. 2. The figure which shows the change of the frequency band of Y signal and RB signal in the process in the signal processing part of FIG. FIG. 6 is a block diagram illustrating a configuration example of a color signal processing circuit in an image processing apparatus according to a second embodiment of the present invention.

(First embodiment)
Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.
FIG. 1 is a block diagram illustrating a functional configuration example of an imaging apparatus 200 as an example of an image processing apparatus according to an embodiment of the present invention. In the present invention, a configuration for imaging is not essential, and image data to be processed is acquired directly from a data file or the like stored in advance in a storage device or a storage medium, or indirectly via a network or the like. You can also.

  In FIG. 1A, the imaging optical system 201 includes a lens, a diaphragm, and the like, and stores a subject optical image on the imaging surface of the imaging element 202. The system control unit 207 performs focusing control and diaphragm control of the imaging optical system 201. The image sensor 202 is a CCD image sensor or a CMOS image sensor, and converts the subject optical image collected by the imaging optical system 201 into an analog electrical signal in pixel units. In this embodiment, a Bayer array color filter shown in FIG. 1B is arranged on the imaging surface of the image sensor 202, and each pixel of the image sensor 202 is a pixel corresponding to the color of the corresponding color filter. The value (luminance value) is output.

  The A / D conversion unit 203 converts the analog image signal output from the image sensor 202 into a digital image signal (also referred to as image data). The signal processing unit 204 performs signal processing such as noise reduction for each frequency band on the digital image signal output from the A / D conversion unit 203. The memory control unit 205 writes / reads digital image signals to / from the memory (DRAM) 206. A system control unit 207 controls the overall operation of the imaging apparatus 200. In FIG. 1A, components that are not directly related to the description of the present embodiment among the configurations that the imaging apparatus 200 normally has are not illustrated.

  FIG. 2 is a block diagram illustrating a functional configuration example of the signal processing unit 204. The signal processing unit 204 includes a frequency resolution circuit 101, a buffer 103, and a frequency synthesis circuit 102. Although the buffer 103 may be provided in the signal processing unit 204, in this embodiment, a partial area of the memory 206 is used as the buffer 103. Therefore, writing from the frequency resolving circuit 101 to the buffer 103 and reading from the buffer 103 to the frequency synthesizing circuit 102 are executed through the memory control unit 205.

  The frequency resolving circuit 101 divides A / D converted Bayer array image data into a plurality of predetermined frequency bands, and thins out color signal components. The frequency synthesizer circuit 102 performs signal processing for each frequency band and then synthesizes and outputs the result. The output image data of the frequency synthesis circuit 102 is stored in the memory 206 through the memory control unit 205.

(Configuration of frequency resolving circuit 101)
In the frequency resolution circuit 101, the format conversion circuit 104 converts the image signals in the R (red), G1 (green 1), G2 (green 2), and B (blue) formats shown in FIG. Convert to R, B format. Further, the format conversion circuit 104 performs false color suppression processing. The false color suppression processing method is not particularly limited, and any known method can be employed. The method described in JP-A-2002-300590 is an example of a method that can be adopted.

Specifically, for each pixel of the optical image formed on the image sensor, it is determined whether the correlation in the horizontal direction or the vertical direction is stronger. If the horizontal correlation is strong, (R-G1) is selected, and if the vertical correlation is strong, (R-G2) is selected as the (R-Ga1) signal. Similarly, if the horizontal correlation is strong, (B-G2) is selected, and if the vertical correlation is strong, (B-G1) is selected as the (B-Ga2) signal.
For each pixel, a Gb signal is generated from G1, G2, for example, Gb = (G1 + G2) / 2, and (R−Gb) is generated.

Furthermore, for example for each pixel
SatA = | R-Ga1 | + | B-Ga2 |
SatB = | R-Gb | + | B-Gb |
Saturation values SatA and SatB are obtained as
If SatA <SatB, (R−Ga1) is selected, and if SatA ≧ SatB, (R−Gb) is selected and used as the Cr signal.
If SatA <SatB, (B-Ga2) is selected, and if SatA ≧ SatB, (B-Gb) is selected and set as the Cb signal.
By performing such processing, color difference signals Cr and Cb in which false colors are suppressed are generated.

The luminance signal Y is an R signal interpolated pixel by pixel from a horizontal pixel if the horizontal correlation of the optical image formed on the image sensor is strong, or from a vertical pixel if the vertical correlation is strong. , G signal, B signal, for example, Y = 0.33R + 0.59G + 0.11B
Generate from the operation.

  The format conversion circuit 104 generates Y, R, B format image data as Y = Y, R = Cr + Y, and B = Cb + Y from the Y, Cr, and Cb signals generated by such a method, for example.

The low-pass filters (LPF) 105 to 107 are filters having a transfer function H (z) = ½ (1 + Z ⁻¹ ), for example, both in the horizontal direction and in the vertical direction. The downsampling circuits 108 to 110 downsample the input image signal by half in the horizontal direction and the vertical direction. The phase for downsampling is the left pixel of the two left and right pixels in the horizontal direction, and the upper pixel of the two upper and lower pixels in the vertical direction.

  The LPFs 105 to 107 and the downsampling circuits 108 to 110 are connected alternately and in series. Therefore, the Y / R / B signal input to the LPF 105 is band-limited horizontally and vertically by 1/2, and the number of pixels is also thinned horizontally and vertically by 1/2 by the downsampling circuit.

  Then, the Y / R / B signal thinned out by the downsampling circuit 108 is input to the LPF 106, further band-limited to ½, and thinned down to ½ by the downsampling circuit 109. Similarly, the LPF 107 band-limits the Y / R / B signal thinned out by the downsampling circuit 109 to 1/2, and the downsampling circuit 110 thins the number of pixels to 1/2.

  The Y / R / B signals output from the format conversion circuit 104 and the downsampling circuits 108 to 110 are input to RB thinning circuits 111 to 114 that reduce the resolution of the R signal and the B signal, respectively.

  FIG. 3A is a diagram for explaining a configuration example of the RB thinning circuit 111. FIG. 3B is a diagram for explaining the operation of the RB thinning circuit 111 and the RB interpolation circuit 115 of the frequency synthesis circuit 102. Since the configuration and operation of the RB thinning circuits 111 to 114 are common, the RB thinning circuit 111 will be described here. 3A shows only a circuit for thinning out the R signal (R thinning out circuit) in the configuration of the RB thinning out circuit 111, but a circuit for thinning out the B signal (B thinning out circuit). Has the same configuration.

The low-pass filter 401 is a horizontal one-dimensional low-pass filter having a transfer function H (z) = ½ (1 + Z ⁻¹ ), for example. The selector 402 outputs one of the signal to which the low-pass filter 401 is applied and the signal to which the low-pass filter 401 is not applied. The thinning circuit 403 selects an even phase or an odd phase as necessary, and thins out and outputs the pixels having the selected phase (FIG. 3B).

  Note that the signals selected by the selectors are made equal so that the R and B thinning circuits have the same transfer function H (z). That is, when the selector 402 of the R decimation circuit selects the output of the low pass filter 401, the selector of the B decimation circuit also selects the output of the low pass filter. Further, when even-numbered phases (odd phases) are selected by the R thinning-out circuit, thinning is performed by selecting even-numbered phases (odd-numbered phases) even in the B thinning-out circuit. This point will be described in detail later.

  The frequency resolving circuit 101 having the above configuration outputs a Y / RB signal composed of two planes Y and RB, band-limited for each frequency band. As described above, this Y / RB image signal is stored in the buffer 103 for each frequency band (Y / RB-1 to Y / RB-4). As described above, the frequency resolving circuit 101 not only divides the frequency band, but also converts the image signal in the R / G1 / G2 / B format into a Y / RB 2-plane image signal. Therefore, the required capacity of the buffer 103 can be reduced as compared with the case of using R / G1 / G2 / B 4-plane image signals. Thereby, cost reduction and size reduction of an apparatus are realizable. In addition, since memory access per unit time can be reduced, it is possible to suppress a decrease in processing performance.

(Configuration of frequency synthesis circuit 102)
The frequency synthesis circuit 102 first performs interpolation processing on the R and B signals for each frequency band by the RB interpolation circuits 115 to 118. As shown in FIG. 3B, the RB interpolation circuits 115 to 118 are selected by the RB thinning circuits 111 to 114 for Y / RB-1 to Y / RB-4 from which the R and B signals are thinned, respectively. The interpolation phase (odd phase or even phase) is interpolated according to the phase. That is, if the even phase is thinned out, the even phase is interpolated, and if the odd phase is thinned out, the odd phase is interpolated. FIG. 3B shows an example in which interpolation is performed using the average value of adjacent pixels of the interpolation target pixel. Although only the R pixel signal is shown in FIG. 3 as described above, the same interpolation processing is performed for the B pixel signal.

  Y / R / B signals obtained by interpolating RBs by the RB interpolation circuits 115 to 118 are output to the Y thinning interpolation circuits 119 to 122 and the luminance signal processing circuits 123 to 126.

  4A and 4B are diagrams for explaining an example of the configuration and operation of the Y-thinning interpolation circuit 119. FIG. Since the configuration and operation of the Y-thinning interpolation circuits 119 to 122 are common, only the Y-thinning interpolation circuit 119 will be described here.

The low-pass filter 501 is a horizontal one-dimensional low-pass filter having a transfer function H (z) = 1/2 (1 + Z ⁻¹ ), for example. The selector 502 outputs one of the signal to which the low-pass filter 501 is applied and the signal to which the low-pass filter 501 is not applied. The luminance (Y) decimation circuit 503 selects an even phase or an odd phase from the Y signals before decimation as necessary, and decimates and outputs the selected pixels (FIG. 4B). The luminance (Y) interpolation circuit 504 interpolates an interpolation phase corresponding to the thinned phase with respect to the Y signal thinned in the horizontal direction by the Y thinning circuit 503. That is, if the even phase is thinned out, the even phase is interpolated, and if the odd phase is thinned out, the odd phase is interpolated. FIG. 4B shows an example in which interpolation is performed using the average value of adjacent pixels of the interpolation target pixel.

  When the selector 402 of the RB decimation circuits 111 to 114 selects the output of the low-pass filter 411, the selector 502 of the Y decimation interpolation circuit 119 also selects the output of the low-pass filter 501. Thereby, the transfer function H (z) of the Y decimation interpolator is made equal to the transfer functions H (z) of the R decimation circuit and the B decimation circuit.

  In addition, when the even-numbered phase is selected by the RB thinning circuit and thinning is performed, the even-numbered phase is also selected by the Y thinning circuit 503 and thinned. When the RB thinning circuit selects an odd phase and performs thinning, the Y thinning circuit 503 similarly selects and thins the odd phase. Further, the Y interpolation circuit 504 also selects an interpolation phase according to the phase selected by the RB interpolation circuits 115 to 118.

As I mentioned earlier,
By performing equivalent processing to the RB thinning circuits 111 to 114 and the RB interpolation circuits 115 to 118 by the Y thinning interpolation circuits 119 to 122, the frequency bands of the Y signal and the R and B signals are matched. .
In addition, the RB thinning circuits 111 to 114 and the RB interpolation circuits 115 to 118 perform thinning and interpolation at the same phase, so that the phases of the Y signal and the R and B signals also coincide.

  An example of the band change of the Y signal and the RB signal in the processing in the signal processing unit 204 will be described with reference to FIG. In FIG. 5A to FIG. 5G, the horizontal axis indicates the phase, and the right side is the high frequency component. The vertical axis represents gain. FIGS. 3A and 4A show positions a to g at which signals corresponding to FIGS. 5A to 5G are obtained.

  First, in the state before the RB decimation (FIG. 5A) inputted to the RB decimation circuit, the Y signal band and the R and B signal bands coincide. When the low pass filter process is performed on the R and B signals by the LPF 401, the high frequency components of the R and B signals are suppressed as shown in FIG.

  Next, the R and B signals are thinned out by the thinning circuit 403, and as shown in FIG. 5 (c), the high frequency component is cut and a folded component is generated in the low frequency component. The thinned R and B signals are interpolated by the RB interpolation circuits 115 to 118, and the high frequency components are further suppressed as shown in FIG.

  Next, similarly, when the low pass filter process is performed by the LPF 501 on the Y signal, the high frequency component is suppressed as shown in FIG. Further, when the Y signal is thinned out by the Y thinning circuit 503, as shown in FIG. 5F, the high frequency component is cut and a folding component is generated in the low frequency component. Finally, when the Y signal is interpolated by the Y interpolation circuit 504, the high frequency component is further suppressed as shown in FIG. 5G, and the Y signal has the same band as the R and B signals.

  In this way, by causing the Y signal and the R and B signals to have the same frequency band, the occurrence of color streaks caused by the difference between the frequency band of the Y signal and the R and B signals is suppressed. It is possible.

  Y / R / B signals whose frequency bands are aligned by the Y thinning interpolation circuits 119 to 122 are input to the color signal processing circuits 127 to 130.

The color signal processing circuits 127 to 130 perform the reverse processing of the format conversion circuit 104. That is,
Cr = R−Y
Cb = BY
As described above, a Cr signal and a Cb signal are generated, and noise reduction processing is applied to each frequency band.

  There is no restriction | limiting in the method of the noise reduction process applied here, A well-known and arbitrary method can be used. As an example, for example, the method described in JP 2008-15741 A may be used. Specifically, a low-frequency image and a high-frequency image are generated for each frequency band, and a high-frequency image obtained by removing noise by performing coring processing and a high-frequency image before coring processing are combined with the edge strength of the low-frequency image. It is a method of synthesizing according to. By increasing the weight of the high-frequency image before the coring process as the edge strength of the low-frequency image is larger, noise can be removed while preserving the edge of the original image.

  The color signal processing circuits 127 to 130 output Cr, Cb signals subjected to noise reduction processing to the synthesis circuit 131. The Cr and Cb signals for each frequency band output from the color signal processing circuits 127 to 130 are combined by the combining circuit 131 and output as Cr and Cb color difference signals. The color difference signals of Cr and Cb may be output as one plane color difference signal CrCb with the horizontal resolution reduced to ½, or each of Cr and Cb may be output as one plane.

In addition, the luminance signal processing circuits 123 to 126 apply noise reduction processing to the Y / R / B signals output from the RB interpolation circuits 115 to 118 for each frequency band by the above-described method, for example. The Y signal for each frequency component processed by the luminance signal processing circuits 12 3 to 12 6 is synthesized by the synthesis circuit 131 and output. At this time, since the Y signal of the Y / R / B signal input to the luminance signal processing circuits 123 to 126 is not subjected to the frequency band limiting process as performed by the Y thinning interpolation circuits 119 to 122. , It is possible to hold a high-frequency component for the Y signal.

  As described above, according to the present embodiment, in an image processing apparatus that applies signal processing to a color signal that has been thinned and interpolated, as a luminance signal used for processing of the color signal that has been thinned and interpolated, A luminance signal subjected to thinning and interpolation processing similar to the color signal is used. As a result, the frequency bands of the color signal and the luminance signal used for color signal processing can be made uniform, and the occurrence of color streaks that have occurred in the prior art where only the color signal is thinned out and interpolated can be suppressed.

  As described above, in the present embodiment, in the image processing apparatus that applies signal processing to the color signals that have been thinned and interpolated, it is possible to reduce both the amount of processing data and the suppression of image quality degradation.

  It should be understood that this effect can be obtained without using an image processing apparatus that processes an image signal for each frequency band. However, in an image processing apparatus that divides an image signal into a plurality of frequency bands and combines them by applying signal processing for each frequency band, the effect is greater.

  That is, in such an image processing apparatus, the effect of color streaks generated for each frequency band is added by synthesis, but in this embodiment, image quality degradation is suppressed in order to suppress the generation of color streaks by processing for each frequency band. Great suppression effect. In addition, the effect of reducing the storage capacity of intermediate data and the access to the recording device by thinning out the color signal is also required for image processing that requires intermediate data to be stored for each frequency band and that is synthesized by applying signal processing for each frequency band. It will be appreciated that this is noticeable with the device.

(Second Embodiment)
Next, an image processing apparatus according to the second embodiment of the present invention will be described. The image processing apparatus according to the present embodiment is the same as that of the first embodiment except for the processing of the color signal processing circuits 127 to 130 of the signal processing unit 204. Therefore, the color signal processing circuits 127 to 130 according to the present embodiment are the same. Only explained.

  FIG. 6 is a block diagram illustrating a configuration example of the color signal processing circuit according to the present embodiment. In FIG. 6, the color signal processing circuit 127 is shown as a representative of the color signal processing circuits 127 to 130 having the same configuration. The lateral chromatic aberration correction circuit 701 corrects lateral chromatic aberration caused by the imaging optical system. The noise reduction circuit 702 performs the noise reduction process described in the first embodiment.

  There is no particular limitation on the method of chromatic aberration of magnification performed by the chromatic aberration correction circuit 701 of magnification. For example, as disclosed in Japanese Patent Application Laid-Open No. 2008-15946, the R and B signals should be originally based on the G signal. The lateral chromatic aberration is corrected by replacing the pixel value of the position. However, in this embodiment, the lateral chromatic aberration of the R signal and the B signal is corrected using the Y signal as a reference instead of the G signal. Further, when correcting the lateral chromatic aberration of the R signal, if the position of the pixel that should be originally deviated by less than one pixel from the actual pixel position, it should be interpolated from the pixel values of the surrounding R signal, Calculate the pixel value of the position. The same replacement is performed for the B signal, and the lateral chromatic aberration is corrected by correcting the pixel positions of the R signal and the B signal with respect to the Y signal.

  Since the R signal and the B signal include the G signal in the Bayer array by the processing of the format conversion circuit 104, it is impossible to completely correct the lateral chromatic aberration, but it is possible to reduce the influence of the lateral chromatic aberration. is there.

  The Y / R / B signal corrected for the chromatic aberration of magnification is input to the noise reduction circuit 702. In the noise reduction circuit 702, Cr and Cb signals are generated from the Y signal and the RB signal in which the lateral chromatic aberration is corrected, as in the first embodiment, and noise reduction processing is applied.

  As described above, the present invention is effective not only in noise reduction processing but also in image processing devices that perform magnification chromatic aberration correction processing. In an image processing device that processes image signals for each frequency band, reduction in processing data amount and image quality are achieved. It is possible to achieve both suppression of the decrease.

(Other embodiments)
In the above-described embodiment, the luminance signal processing circuits 123 to 126 are described as performing signal processing on the Y signal using the Y / R / B signals output from the RB interpolation circuits 115 to 118. However, when only the Y signal is used for signal processing, it is not necessary to go through the RB interpolation circuits 115 to 118. Alternatively, the luminance signal processing circuits 123 to 126 use only the Y signal among the Y / R / B signals received from the RB interpolation circuits 115 to 118.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (12)

Conversion means for converting an image signal obtained from an image sensor having a color filter into a first signal and a second signal;
Thinning means for thinning the first signal;
Storage means for storing the first signal thinned out by the thinning means and the second signal;
Interpolation means for interpolating the first signal read from the storage means;
Thinning interpolation means for performing thinning and interpolation similar to the thinning and interpolation performed on the first signal by the thinning means and the interpolation means for the second signal read from the storage means;
Signal processing means for performing signal processing using the first signal interpolated by the interpolation means and the second signal thinned and interpolated by the thinning interpolation means;
An image processing apparatus comprising:   2. The thinning means selects an even phase or an odd phase of the image signal and performs the thinning, and the interpolation means interpolates the even phase or odd phase thinned by the thinning means. An image processing apparatus according to 1.   The image processing apparatus according to claim 1, wherein the first signal is a color signal, and the second signal is a luminance signal.   The image processing apparatus according to claim 1, wherein the image signal includes red (R), green (G), and blue (B) signals. The converting means performs processing for generating an R signal and a B signal from the luminance signal and the color difference signal after converting the image signal into a luminance signal and a color difference signal,
The first signal is the signal of the signal and B of R the generated image processing apparatus according to claim 4, wherein the second signal is the luminance signal.   The image processing apparatus according to claim 5, wherein the signal processing includes chromatic aberration correction processing for magnification.   The signal processing means generates a color difference signal from the R signal and the B signal interpolated by the interpolation means and the luminance signal thinned and interpolated by the thinning interpolation means, and for the color difference signal, The image processing apparatus according to claim 5, wherein the signal processing is performed.   The image processing apparatus according to claim 7, wherein the signal processing includes noise reduction processing. The image processing apparatus further includes a dividing unit that divides the first signal and the second signal converted by the converting unit into a plurality of frequency bands,
The thinning means, the storage means, the interpolation means, the thinning interpolation means, and the signal processing means perform processing on the first signal or the second signal for each of the divided frequency bands,
The said image processing apparatus further has a synthetic | combination means which synthesize | combines and outputs the result of the said signal processing for every said frequency band which the said signal processing means outputs. The image processing apparatus according to item. The image processing apparatus according to claim 1, further comprising second signal processing means for performing noise reduction processing on the second signal read from the storage means. An image processing method executed by an image processing apparatus,
A conversion step of converting an image signal obtained from an image sensor having a color filter into a first signal and a second signal;
A thinning-out step of thinning out the first signal;
A storage step of storing in the storage device the first signal thinned out in the thinning-out step and the second signal;
An interpolation step of interpolating the first signal read from the storage device;
A decimation interpolation step for performing decimation and interpolation similar to the decimation and interpolation performed on the first signal by the decimation step and the interpolation step for the second signal read from the storage device;
A signal processing step of performing signal processing using the first signal interpolated by the interpolation step and the second signal thinned and interpolated by the thinning interpolation step;
An image processing method comprising: The program for functioning a computer as each means which the image processing apparatus of any one of Claims 1 thru | or 10 has.

Images