JP2009253760A - Image processing apparatus, image processing method and electronic device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To output color image data at a higher processing speed without increasing circuit scale or power consumption in an image processing apparatus that applies processing to an input color image signal and outputs the signal. <P>SOLUTION: An image processing apparatus according to one embodiment comprises: an input section (103) for inputting an image; a plurality of high frequency generation sections (104A, 104B) for generating high frequency components at pixel positions different from each other in the input image, respectively; a low frequency generation section (105) for generating a low frequency component image whose number of pixels is less than the number of pixels in the input image; a synthesis section (106) for obtaining a synthesis image by synthesizing the generated high frequency components and the generated low frequency component image; and a control section (111) for operating the plurality of high frequency generation sections (104A, 104B) in parallel. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and an electronic apparatus suitable for a digital still camera, for example.

  An imaging apparatus such as a digital camera is provided with an image processing circuit that converts a signal from the imaging element into a color image. The number of constituent pixels of the image sensor is increasing year by year, and accordingly, the processing speed of the image processing circuit is required to be improved. However, improvement in processing speed by miniaturizing an integrated circuit constituting an image processing circuit is already approaching its limit, and it is required to devise signal processing and image processing algorithms themselves.

  As an example of a method for improving the processing speed, Patent Document 1 discloses a method of outputting a result of a plurality of pixels at a time in a circuit portion that performs an interpolation process, and thereafter operating the process with a double operation clock ( Patent Document 1).

As another method, Japanese Patent Application Laid-Open No. H10-228667 describes a color difference component as R (red) or R, when performing interpolation processing on output data from a single-plate image sensor having a Bayer array color filter in order to reduce the amount of data to be processed. The data amount is reduced to ¼ by making it only at the pixel position of B (blue), and the color difference signal interpolation is performed again when combining with the luminance signal made by all pixels, so that the RGB data of all the pixels is obtained. Is disclosed (Patent Document 2).
JP 2000-341561 A Japanese Patent Laid-Open No. 02-122785

  However, in the method shown in Patent Document 1, since the operation clock is partially doubled, both circuit scale and power consumption increase when a complicated process such as noise reduction processing is provided as post-processing. End up.

  In the method disclosed in Patent Document 2, although the amount of processing data is reduced, the time for generating the luminance components of all the pixels affects the processing speed, and as a result, the processing speed is not improved.

  The present invention has been made in view of the above circumstances, and its object is to quickly generate color image data from the output of the image sensor without significantly increasing the circuit scale or power consumption. An image processing apparatus, an image processing method, and an electronic apparatus are provided.

  One embodiment of the present invention includes an input unit that inputs an image, a plurality of high-frequency generation units that each generate high-frequency components at different pixel positions in the input image, and the number of pixels of the input image A low-frequency generating unit that generates a low-frequency component image having a smaller number of pixels, a combining unit that combines the generated high-frequency component and the generated low-frequency component image to obtain a combined image, and And a control unit that operates the high-frequency generation unit in parallel.

  According to the present invention, it is possible to quickly generate a color image from the output of the image sensor without significantly increasing the circuit scale and power consumption.

(First embodiment)
A first embodiment when the present invention is applied to a digital camera will be described below.
FIG. 1 is a block diagram showing a circuit configuration of image processing mainly for a photographing system of the digital camera 100 according to the present embodiment. In the figure, a light image of a subject incident through the lens optical system 101 is formed on a single-plate CCD (Charge Coupled Device) 102 which is a solid-state image sensor.

  The single-chip CCD 102 has, for example, a primary color filter in a Bayer array, and its output is amplified by an AGC amplifier (not shown) at an amplification factor corresponding to the sensitivity set at that time, and also an A / D (not shown) After being digitized by the converter, it is held in the image buffer 103.

  The image data held in the image buffer 103 is input to a processor 110 as an image processing circuit that generates image data for recording. In the processor 110, a high frequency processing circuit 104, a low frequency generation circuit 105, a synthesis circuit 106, a compression circuit 107, and a control circuit 111 are provided.

  The high frequency processing circuit 104 extracts high-frequency component image data in the image data from the image buffer 103 and outputs the extracted image data to the synthesis circuit 106.

  The low frequency generation circuit 105 reduces the amount of information by reducing the number of pixels in the image data from the image buffer 103, further extracts low-frequency component image data, and outputs the extracted image data to the synthesis circuit 106. To do.

  The synthesizing circuit 106 synthesizes the high-frequency component image data from the high-frequency processing circuit 104 and the low-frequency component image data from the low-frequency generation circuit 105 to generate image data that covers the entire spatial frequency band, and compresses it. Output to the circuit 107.

  The compression circuit 107 performs compression of the image data according to a predetermined data compression method, for example, JPEG (Joint Photographic Experts Group), and outputs the compressed image data to the recording unit 108 located outside the processor 110.

  The recording unit 108 includes a memory card that is a recording medium of the digital camera 100 and its input / output interface, and records image data sent from the compression circuit 107 of the processor 110.

  The control circuit 111 executes a control operation in the processor 110, and additionally operates two systems in the high-frequency processing circuit 104 described later in parallel in time.

Next, the configuration and operation of the high frequency processing circuit 104 will be described with reference to FIGS.
FIG. 2 is a configuration diagram of the high frequency processing circuit 104. The high frequency processing circuit 104 includes an edge extraction circuit 104a, a direction determination circuit 104b, an edge extraction circuit 104c, a noise reduction circuit 104d, and a noise reduction circuit 104e.

  The direction determination circuit 104b receives the image data from the image buffer 103, and outputs a direction determination index, which is an index for determining the direction of the edge included in the image data, to the edge extraction circuit 104a and the edge extraction circuit 104c.

The edge extraction circuit 104a and the edge extraction circuit 104c similarly input image data from the image buffer 103, generate a high luminance signal based on the direction determination index from the direction determination circuit 104b, and output the result as a noise reduction circuit 104d and Output to the noise reduction circuit 104e.
The noise reduction circuit 104d and the noise reduction circuit 104e perform noise reduction processing on the input high luminance signal, and output the results to the synthesis circuit 106 as intermediate outputs O1 and O2, respectively.

  The edge extraction circuit 104a, the direction determination circuit 104b, and the noise reduction circuit 104d form a first high-frequency generation circuit 104A. The edge extraction circuit 104c, the direction determination circuit 104b, and the noise reduction circuit 104e form a second high frequency generation circuit 104B.

  These two high-frequency generation circuits 104A and 104B generate and output high-frequency components at different pixel positions in the image data from the image buffer 103, respectively. As described above, time is controlled under the control of the control circuit 111. In parallel.

  Note that the edge extraction circuit 104a performs processing in the order of P1, P3, and P5 at the positions of the black circle pixels in the column direction shown in FIG. The edge extraction circuit 104c operates in parallel with this, and sequentially performs processing on the white circle pixels P2, P4, and P6 immediately below the black circle pixels processed by the edge extraction circuit 104a.

  Hereinafter, processing in each circuit will be described in more detail.

  The direction determination circuit 104b is an intermediate position between a black circle position in FIG. 6 processed by the edge extraction circuit 104a and a white circle position in FIG. 6 processed by the edge extraction circuit 104c (illustrated by a cross in FIG. 6). ), An edge direction determination index D is calculated.

Note that the direction determination circuit 104b may determine, as a pair, two pixel positions that are adjacent to each other and that are intended to be processed in parallel in time. Here, 5 × 6 (horizontal × vertical) pixel data (indicated by a broken line frame in FIG. 6) near the x mark is read from the image buffer 103, and as shown in FIG. The horizontal and vertical differences Dh and Dv of G pixels of
Dh = (| G1-G2 | + | G3-G4 | + | G4-G5 | + | G6-G7 | +
| G8-G9 | + | G9-G10 | + | G11-G12 | +
| G13-G14 | + | G14-G15 |) / 9
Dv = (| G1-G6 | + | G2-G7 | + | G3-G8 | + | G4-G9 | +
| G5-G10 | + | G6-G11 | + | G7-G12 | + | G8-G13 | +
| G9-G14 | + | G10-G15 |) / 10
And based on these, D
D = (Dh-Dv) / (Dh + Dv + T)
(However, | x |: absolute value of x,
T: constant,
D: takes a value from -1 to 1.

When -1, the inner edge direction is horizontal,
1 means vertical. )
And calculate.

The edge extraction circuit 104a and the edge extraction circuit 104c determine a band-pass filter based on the index D and generate a luminance high-frequency signal. For example, at the pixel position of P1 in FIG. 6, the edge extraction circuit 104a reads out the surrounding 5 × 5 area data from the image buffer 103 and applies a 5 × 5 size band-pass filter.
At the pixel position P2 in FIG. 6, the edge extraction circuit 104c reads the surrounding 5 × 5 area data from the image buffer 103 and applies a 5 × 5 size band-pass filter.

The bandpass filter coefficient B (i, j) is determined in advance by a bandpass filter coefficient Bh (i, j) that emphasizes the horizontal edge and a bandpass filter coefficient Bv (i, j) that emphasizes the vertical edge. ) And the index D,
B (i, j) = ((D + 1) * Bv (i, j) + (D-1) * Bh (i, j)) / 2
(1 <= i, j <= 5)
Is calculated by

The noise reduction circuits 104e and 104f output c (e) that is the result of performing coring processing on the outputs e of the edge extraction circuits 104a and 104c, respectively. That is,
c (e) = eT (when e> = T)
0 (when -T <e <T)
e + T (when e <= -T)
It becomes.

The coring threshold T is calculated from the horizontal difference Dh and the vertical difference Dv obtained from the direction determination circuit 104b as follows.
T = min (| Dh-Dv |, Dh, Dv, T0)
(However, min (a, b, c, d): a function that returns the minimum value of a, b, c, d,
T0: Constant. )
Next, FIG. 3 shows a detailed configuration in the low-frequency generation circuit 105. Image data from the previous image buffer 103 is supplied to the RGB generation circuit 105a. The RGB generation circuit 105a performs interpolation processing on the image data from the image buffer 103 to generate RGB data in which all RGB components are aligned in each pixel, and outputs the RGB data to the color conversion circuit 105b.

The RGB generation circuit 105a sequentially outputs the RGB components at the pixel positions indicated by black circles in FIG. 6 in the column direction and then in the row direction. Further, when processing one pixel, 5 × 5 surrounding pixels in the vicinity thereof are required.
The color conversion circuit 105b performs addition / subtraction conversion on the RGB data of the primary color signal system according to the white balance value set at that time, and outputs the converted RGB data to the gradation conversion circuit 105c.

  The gradation conversion circuit 105c converts the intermediate gradation value into a predetermined value by performing gamma correction processing on each of the RGB data, and outputs it to the YC conversion circuit 105d.

  The YC conversion circuit 105d converts the RGB data of the primary color signal system into YCrCb data of the luminance / color difference signal system by matrix calculation, separates the obtained Y data, Cr data, and Cb data and outputs them to the noise reduction circuits 105e to 105g, respectively. To do.

  These noise reduction circuits 105e to 105g obtain low-frequency noise by performing low-pass filter processing on Y data, Cr data, and Cb data, respectively, to obtain intermediate outputs O3 to O5, and output them to the synthesis circuit 106. .

  Next, FIG. 4 shows the configuration of the synthesis circuit 106. In the synthesis circuit 106, the high frequency component from the high frequency processing circuit 104 and the low frequency component from the low frequency generation circuit 105 are synthesized, and the final signal is output to the compression circuit 107. The input signals are the luminance high-frequency signal YH1 (intermediate output O1), the luminance low-frequency signal YL1 (intermediate output O3), and the color difference low-frequency signals Cr1 (intermediate output O4) and Cr2 (intermediate output O5) at the position of the black circle in FIG. ) And the high luminance signal YH2 (intermediate output O5) at the position of the white circle in FIG. The output signals are the luminance signal Y3 (intermediate output O6) at the position of the black circle in FIG. 6, the color difference signals Cr3 (intermediate output O8) and Cb3 (intermediate output O10), and the luminance signal Y4 at the position of the white circle in FIG. Intermediate output O7), color difference signals Cr4 (intermediate output O9) and Cb4 (intermediate output O11).

The emphasis circuit 106a in the synthesis circuit 106 generates a luminance signal Y3 (intermediate output O6) from the luminance high-frequency signal YH1 (intermediate output O1) and the luminance low-frequency signal YL1 (intermediate output O3) input to itself.
Y3 = YL1 + α * YH1
(However, α: Gain for emphasizing high frequencies.)
Generate by.

In addition, in the interpolation circuit 106c, for example, when the luminance low-frequency signal YL1 (intermediate output O3) of P3 in FIG. 6 existing at the black circle position in FIG. From the luminance low-frequency signal YL1 ′ at the pixel position (P1 in FIG. 6), the luminance low-frequency signal YL2 at the pixel position (P2 in FIG. 6) one pixel above is expressed by the following equation:
YL2 = (YL1 + YL1 ') / 2
Generate by. The immediately preceding input YL ′ is held in the internal memory of the interpolation circuit 106c. The calculated luminance low-frequency signal YL2 is output to the enhancement circuit 106b. The enhancement circuit 106b generates the luminance signal Y4 (intermediate output O7) from the luminance high-frequency signal YH2 (intermediate output O2) and the luminance low-frequency signal YL2. Similar to the emphasis circuit 106a,
Y4 = YL2 + α * YH2
Generate by. The enhancement circuit 106b holds the value of YH2 in the internal memory until YL2 is calculated.

  The color difference low-frequency signals Cr1 (intermediate output O4) and Cr2 (intermediate output O5) are output as they are as the color difference signals Cr3 (intermediate output O9) and Cb3 (intermediate output O11), and Cr1 is interpolated by the interpolation circuit 106d, Cr2 is input to the interpolation circuit 106e, and the color difference signals Cr4 and Cb4 at the white circle positions in FIG. 6 are interpolated and output.

The processing in the interpolation circuits 106d and 106e is similar to the interpolation circuit 106c. For example, the interpolation circuit 106d inputs the color difference signal Cr1 of the pixel corresponding to the black circle in FIG. 6 (for example, P3 in FIG. 6) and immediately before. From the color difference signal Cr1 ′ at the pixel position (P1 in FIG. 6) on the two pixels, the color difference signal Cr4 at the pixel position (P2 in FIG. 6) on the one pixel is expressed by the following equation:
Cr4 = (Cr1 + Cr1 ') / 2
Generate by. The immediately preceding input Cr1 ′ is held in the internal memory of the interpolation circuit 106d.

  FIG. 5 summarizes the time-series contents of each intermediate output obtained as a result of the operation of the circuit described so far. In the parentheses are coordinates for obtaining signals written in the order of columns and rows, and correspond to the coordinates indicated in parentheses in FIG.

  As described above, since the high-frequency processing circuit 104 and the low-frequency generation circuit 105 each require a 5 × 5 area to process one pixel, effective data can be obtained from P1 in FIG. From the position. When the processing is started from the position P1 in FIG. 6, first, the high frequency processing circuit 104 sets the luminance high frequency signal YH (1, 1) and the intermediate output O2 at P1 in FIG. 6 as the intermediate output O1. A high luminance signal YH (1,2) at P2 is generated.

  At the same time, in the low-frequency generation circuit 105, the luminance low-frequency signal YL (1,1), the color difference signals Cr (1,1) and Cb (1,1) at P1 in FIG. Generated as O5. In the synthesis circuit 106, the luminance signal Y (1, 1) at P1 is generated as the final output O6 from the intermediate outputs O1 and O3.

  Further, the color difference signals Cr (1,1) and Cb (1,1) at P1 are output as final outputs O8 and O10 as they are.

  At the next timing, the processing target position of the high frequency processing circuit 104 moves to P3 and P4 in FIG. 6, and the luminance high frequency signal YH (1,3) at P3 and the luminance high frequency signal YH (1,3) at P4. 4) is generated as intermediate outputs O1 and O2.

  The processing target position of the low-frequency generation circuit 105 also moves to P3 in FIG. 6, and the luminance low-frequency signal YL (1, 1), the color difference signals Cr (1, 1), and Cb (1, 1) at P3 are respectively intermediate. Generated as outputs O3, O4 and O5.

  Further, the luminance low-frequency signal YL (1,2) at P2 is interpolated from the outputs at the pixels P1 and P3 adjacent to P2, and is synthesized with the luminance high-frequency signal YH (1,2) to obtain the luminance at P2. Signal Y2 is generated as final output O7. Similarly, the color difference signals Cr (1,2) and Cb (1,2) at P2 are interpolated and output as final outputs O9 and O11, respectively.

  By repeating this operation, a luminance signal and a color difference signal for two pixels are obtained within one clock.

  As described above, the two series of high-frequency generation circuits 104A and 104B in the high-frequency processing circuit 104 as the first and second processing means operate in parallel in time, and the low-frequency generation circuit 105 as the low-frequency processing means. Operates at the same timing.

  Then, the synthesis circuit 406 located in the subsequent stage generates a color image composed of the outputs O6 to O11 by synthesizing the high frequency components while interpolating the low frequency components.

  In this way, it is possible to output a plurality of high-frequency components constituting an edge (contour) of an image at a time, and process by thinning out the low-frequency components. Therefore, it is possible to output processing results for a plurality of pixels at a time as a whole while keeping the operation clock the same while suppressing an increase in circuit scale for processing low frequency components, and the processing speed is more than doubled. Can be improved.

  Further, as described in FIG. 2, the specific high-frequency component of the pixel is the luminance edge component, so that the low-frequency component and high-frequency component of the pixel are completely separated and processed, thereby improving the image processing efficiency. Can do.

  Similarly, as shown by the black spot position including the pixel position P1 in FIG. 6, the part of the pixel positions is at an interval of two pixels with one pixel vertically (or horizontally) and the arrangement direction thereof. It is assumed that a low-frequency component is generated at a pixel position corresponding to a lattice point having a one-pixel interval that is not spaced in the horizontal (or vertical) direction orthogonal to the horizontal axis. As a result, the processing delay is small, and the processing speed can be increased without complicating the circuit scale.

  Further, as shown in FIG. 2, the first and second processing means in the high frequency processing circuit 104, that is, the edge extraction circuit 104a, the direction determination circuit 104b and the noise reduction circuit 104d, the edge extraction circuit 104c, and the direction determination Since the circuit 104b and the noise reduction circuit 104e share the direction determination circuit 104b that determines the luminance edge direction of the image in order to generate the high-frequency component, the circuit scale can be reduced.

In the present embodiment example, several modifications are possible.
For example, the direction determination index output by the direction determination circuit 104b is not limited to the above-described index, and any index indicating a known edge direction, such as an angle between the edge and the horizontal, may be used. Further, the processing of the noise reduction circuits 104d and 104f is not limited to that described above, and may be more complicated such as smoothing along the edge direction indicated by the direction determination index of the direction determination circuit 104b.

  Further, the processing of the edge extraction circuits 104a and 104c need not be a band pass filter for a 5 × 5 block, and a non-linear structure extraction method based on a morphological operation can be used.

  Furthermore, in the example of this embodiment, the bandpass filter is applied to all 5 × 5 block data, but a pseudo-luminance high-frequency signal may be obtained by applying a bandpass filter to the luminance G component. In particular, in the case of an image sensor with a Bayer array, as shown in FIG. 18, since the pixel position to which the filter is applied is inverted by the color component obtained at the center of the neighborhood, P1 and P2 in FIG. Even when going on, separate processing circuits are required. Therefore, there is an advantage that the circuit scale does not increase even if P1 and P2 are processed in parallel.

  Furthermore, the edge extraction circuits 104a and 104c and the noise reduction circuits 104d and 104f do not necessarily share the direction determination circuit 104b, and may perform the direction determination in each circuit. Further, edge extraction may be performed by a single bandpass filter without performing direction determination itself.

  As for noise reduction, instead of performing noise reduction on the outputs of the edge extraction circuits 104a and 104c, noise reduction may be performed on each of O6 and O7 which are final luminance signal outputs. This is effective, for example, when it is desired to apply a noise reduction method based on a rank filter that is not suitable for application to edge components. In this case, the noise reduction circuits 104d and 104f are arranged after the outputs O6 and O7 of the synthesis circuit 106, respectively.

  Further, the pixel positions processed by the high-frequency processing circuit 104 at a time do not have to be aligned in the vertical direction as shown by P1 and P2 in FIG. 6, but in the horizontal direction as shown by P1 and Q1 in FIG. May be adjacent to In this case, the low-frequency generation circuit 105 performs processing at a position where one column is skipped in the horizontal direction, and the low-frequency signal of the skipped column is generated by performing horizontal interpolation within the synthesis circuit 106.

  The thinning-out rate need not be limited to ½ as low-frequency processing, and processing may be performed by skipping two or more rows and interpolating between skipped rows. In that case, the high frequency processing circuit 104 generates a high frequency signal at a skipped position within one clock by a circuit operating in parallel. In this case, it is of course desirable to share a common circuit among circuits operating in parallel, such as a direction determination circuit.

  Further, although the present embodiment has been described as processing the output of the single-color CCD 102 of the primary color Bayer array, by changing the algorithm of the RGB generation circuit 105 and the high frequency processing circuit 104, a color filter array other than the Bayer array, It is also possible to process the output of the complementary color filter.

  Further, even if the image pickup device is not a single-plate CCD 102 but a three-plate image pickup device, the same increase in speed is possible.

  In this case, the RGB generation circuit 105a is not necessary. In the edge extraction circuits 104a and 104c, for each pixel in the 5 × 5 region to be processed, first, a luminance component is calculated from the three color components by a known matrix arithmetic expression, and then the luminance obtained in the 5 × 5 region is obtained. A band pass filter for edge extraction is applied to the component.

  Similarly, even when the image pickup device is not a single CCD 102 having a color filter but a monochrome image pickup device without a color filter, the same speeding up can be achieved. In this case, the RGB generation circuit 105a and the color conversion circuit 105b are unnecessary, and the gradation conversion circuit 105c directly performs gradation conversion on the output of the image sensor. The YC conversion circuit 105d directly outputs the input from the gradation conversion circuit 105c to the noise reduction circuit 105e, and the noise reduction circuits 105f and 105g, the interpolation circuit 106d, and the interpolation circuit 106e become unnecessary.

(Second Embodiment)
A second embodiment when the present invention is applied to a digital camera will be described below.
Note that the image processing circuit configuration of the digital camera 100 ′ according to the present embodiment is basically the same as that shown in FIG. 1, and the same reference numerals are used for the same parts. The illustration and description thereof are omitted.

In this embodiment, a low-frequency generation circuit 205 described later is used instead of the low-frequency generation circuit 105 of FIG. 1, and a synthesis circuit 206 described later is used instead of the synthesis circuit 106.
FIG. 7 shows a detailed configuration in the low-frequency generation circuit 205. Image data from the previous image buffer 103 is given to the RGB generation circuit 205a. The RGB generation circuit 205a performs interpolation processing on the image data from the image buffer 103 to generate RGB data in which RGB components are aligned in each pixel, and outputs the RGB data to the color conversion circuit 205b.

  Note that the RGB generation circuit 205a sequentially outputs RGB components at the pixel positions indicated by “◇” in FIG. Further, when processing one pixel, similar to the first embodiment, surrounding pixel data of 5 × 5 in the vicinity is required.

  The color conversion circuit 205b performs addition / subtraction conversion on the RGB data of the primary color signal system according to the white balance value set at that time, and outputs the converted RGB data to the gradation conversion circuit 205c.

  The gradation conversion circuit 205c converts the intermediate gradation value into a predetermined value by performing gamma correction processing on each of the RGB data, and outputs the converted value to the YC conversion circuit 205d.

  The YC conversion circuit 205d converts the RGB data of the primary color signal system into YCrCb data of the luminance / color difference signal system by matrix calculation, and separates the obtained Y (YL) data, Cr data, and Cb data, and buffers 205h, 205i, And 205j.

  The noise reduction circuit 205e performs low-pass filter processing on Y (YL) data to reduce high-frequency component noise, and repeats the process of returning the output O3 'again through the buffer 205h and reprocessing. Thus, the intermediate output O3 that has undergone a plurality of processes is acquired and output to the synthesis circuit 206.

  Similarly, the noise reduction circuits 205f and 205g also perform low-pass filter processing on the Cr data and Cb data to reduce high-frequency component noise. The outputs O4 'and O5' are fed back again through the buffers 205i and 205j. Then, by repeating the process of reprocessing, the intermediate outputs O4 and O5 that have undergone a plurality of processes are acquired and output together to the synthesis circuit 206.

  In the example of the present embodiment, a known noise reduction method is used in the noise reduction circuits 205e to 205g. However, when processing one pixel, neighboring pixel data of 3 × 3 is necessary. Therefore, the buffers 205h, 205i, and 205j have line memories for three columns each for the first processing and the second processing, and perform processing when processable data is accumulated.

  Next, the configuration and operation of the synthesis circuit 206 will be described using FIGS. 8 to 10 together with the operation timings of the high frequency processing circuit 104 and the low frequency generation circuit 205. FIG.

  FIG. 8 is a configuration diagram of the synthesis circuit 206. FIG. 9 shows the signal contents of the inputs / outputs O1 to O11 of the synthesis circuit and the intermediate outputs O3 ′, O4 ′, and O5 ′ in the low-frequency generation circuit 205 in time series, and processing for one column is defined as one unit period. The contents of the signal are noted. Note that the time required for processing for one column is processed in parallel at the positions of the black and white circles adjacent to the upper and lower sides in FIG. 10 in the high-frequency processing circuit 104, and the processing is performed only at the position marked with ◇ in the low-frequency generation circuit 205. This is half the number of vertical pixels.

As described above, since the noise reduction circuit in the low-frequency circuit 205 requires 3 × 3 neighborhood data to process each pixel, the low-frequency generation circuit 205 first starts with 3, 5, and 7 in FIG. A first noise reduction process is performed on the column, and then a second noise reduction process is performed on the fifth column. The processing is started from the third column because 3 × 3 neighborhood data is required for the first noise reduction processing itself. In the fifth and subsequent columns, a period in which the low-frequency signal in which noise has been reduced twice in the fifth column is obtained is defined as a reference period t.
・ Period (t + 2n + 1):
(2n + 9) row 1st noise reduction processing
・ Period (t + 2n + 2):
(2n + 7) row 2nd noise reduction processing
(n is an integer)
As a result, the low-frequency signals O3 to O5 whose noise has been reduced are output once every two periods.

That is, when the value of the j-th intermediate output Oi in the period t is described as Oi (j; t), as shown in FIG. 10, when n is an even number,
O3 (j; t + n) =
Brightness low-frequency signal with reduced noise twice at coordinates (n + 5,2j + 1)
O4 (j; t + n) =
Cr color difference signal with reduced noise twice at coordinates (n + 5,2j + 1)
O5 (j; t + n) =
Cb color difference signal with reduced noise twice at coordinates (n + 5,2j + 1)
On the other hand,
When n is odd, O3 (j; t + n) to O5 (j; t + n) have no output
It becomes.

The synthesizing circuit 206 outputs final outputs O8 to O11 of luminance and color differences in the column (n + 4) in the period (t + n) from the outputs O3 to O5 of the low frequency generation circuit 205 and the outputs O1 and O2 of the high frequency processing circuit 104. Generate. Since only the low frequency signal at the position ◇ in FIG. 10 is obtained from the outputs O3 to O5, the low frequency signals at other positions are the values of the low frequency signal S at the coordinates (x, y) in FIG. And S (x, y) in FIG.
In the black circle position,
S (2i + 1,2j + 1) = no interpolation required (◇ position)
S (2i, 2j + 1) = (S (2i-1,2j + 1) + S (2i + 1,2j + 1)) / 2
In the white circle position,
S (2i + 1,2j) = (S (2i + 1,2j-1) + S (2i + 1,2j + 1)) / 2
S (2i, 2j) = (S (2i-1,2j-1) + S (2i-1,2j + 1)
+ S (2i + 1,2j-1) + S (2i + 1,2j + 1)) / 4
(However, i and j are integers.)
It generates with the interpolation formula.

This interpolation calculation is performed by the interpolation circuits 206e to 206i. In the interpolation circuits 206d, 206f, and 206h, the low-frequency luminance signal, the Cr color difference signal, and the Cb color difference signal at the black circle positions in FIG. And 206i output a low-frequency luminance signal, a Cr color difference signal, and a Cb color difference signal at the positions of white circles in FIG. The calculation in each circuit is the following, in which the interpolation formula is replaced with a time series signal. (Hereinafter, j and n are integers.)
Interpolator 206f:
The luminance low-pass signal O1 'at the black circle position (2j + 1, n + 4) in FIG.
O1 '(j; t + n) = O3 (j; t + n-1) (when n is an odd number)
O1 '(j; t + n) =
(O3 (j; t + n-2) + O3 (j; t + n)) / 2 (when n is an even number)
And generate.

Interpolator 206f:
Figure 10 Cr color difference signal O8 at black circle position (2j + 1, n + 4)
O8 (j; t + n) = O4 (j; t + n-1) (when n is an odd number)
O8 (j; t + n) =
(O4 (j; t + n-2) + O4 (j; t + n)) / 2 (when n is an even number)
And generate.

Interpolator 206h:
Figure 10 Cb color difference signal O10 at black circle position (2j + 1, n + 4)
O10 (j; t + n) = O5 (j; t + n-1) (when n is an odd number)
O10 (j; t + n) =
(O5 (j; t + n-2) + O4 (j; t + n)) / 2 (when n is an even number)
And generate.

Interpolator 206e:
Fig. 10 Luminance low-frequency signal O2 'at white circle position (2j + 2, n + 4)
O2 '(j; t + n) =
(O3 (j; t + n-1) + O3 (j + 1; t + n-1)) / 2 (when n is an odd number)
O2 '(j; t + n) =
(O3 (j; t + n-2) + O3 (j + 1; t + n-2)
+ O3 (j; t + n) + O3 (j + 1; t + n)) / 4 (when n is an even number)
And generate.

Interpolator 206g:
Fig. 10 Cr color difference signal O9 at white circle position (2j + 2, n + 4)
O9 (j; t + n) =
(O3 (j; t + n-1) + O3 (j + 1; t + n-1)) / 2 (when n is an odd number)
O9 (j; t + n) =
(O3 (j; t + n-2) + O3 (j + 1; t + n-2)
+ O3 (j; t + n) + O3 (j + 1; t + n)) / 4 (when n is an even number)
And generate.

Interpolator 206i:
Figure 10 Cb color difference signal O11 at white circle position (2j + 2, n + 4)
O11 (j; t + n) =
(O3 (j; t + n-1) + O3 (j + 1; t + n-1)) / 2 (when n is an odd number)
O11 (j; t + n) =
(O3 (j; t + n-2) + O3 (j + 1; t + n-2)
+ O3 (j; t + n) + O3 (j + 1; t + n)) / 4 (when n is an even number)
And generate.

In the enhancement circuit 206j, from the luminance low-frequency signal O1 ′ that is the output of the interpolation circuit 206d and the luminance high-frequency signal O1 that is the output of the high-frequency generation circuit, during the period (t + n), (n + 4) at the black circle position in FIG. The luminance signal O6 of the column is calculated by the following formula. That is,
O6 (j; t + n) = α * O1 (j; t + n) + O1 '(j; t + n)
(j is an integer, α is a constant)
Similarly, in the enhancement circuit 206k, from the luminance low-frequency signal O2 ′ that is the output of the interpolation circuit 206e and the luminance high-frequency signal O2 that is the output of the high-frequency processing circuit 104, the position of the white circle in FIG. The luminance signal O7 in the (n + 4) th column is calculated by the following formula. That is,
O7 (j; t + n) = α * O2 (j; t + n) + O2 '(j; t + n)
(j is an integer, α is a constant)
Note that the processing start position and processing start timing of the high frequency processing circuit 104 are adjusted so that the luminance high frequency signal of the (n + 4) th column is obtained in the period (t + n), and the low frequency signal is output in the enhancement circuit 206j. And there is no shift in the high frequency signal.

  As described above, in the present embodiment, the low-frequency image is generated by half-decimation in each of the horizontal and vertical directions, and a plurality of noise reduction circuits 205e to 205g are used for the same pixel data remaining for each color component. It is assumed that the output is performed after performing noise removal processing repeatedly, for example, twice.

  By thinning out low frequency components that do not significantly affect image quality degradation in this way, processing speed decreases even if low frequency component noise reduction processing is repeated multiple times in a cyclic manner. Can be avoided.

  In addition, the low-frequency generation circuit 205 calculates low-frequency components at some pixel positions, for example, as shown in the variation of the pattern in which the data amount is thinned to ¼ at the position marked with “◇” in FIG. Since the low-frequency image is generated, the low-frequency component can be generated with higher image quality.

  In this case, in the thinning pattern shown in FIG. 10, the part of the pixel positions are grid points with a spacing of two pixels with one pixel in between both horizontally and vertically. Can be provided, and the noise reduction processing configuration can be realized in various variations.

  In the example of the present embodiment, the low-frequency generation circuit 205 performs the low-frequency component generation processing at the positions indicated by ◇ in FIG. 10, but before input to the low-frequency generation circuit 205. A single-plate image thinned out in advance may be generated, and the low-frequency generation circuit 205 may perform processing on all the pixels of the image.

  FIG. 11 illustrates a thinning pattern in such a modification. In this modified example, first, the single-plate image U in the buffer 103 is read by skipping two pixels in the vertical and vertical directions, and only the data of the portion illustrated by shading in FIG. Reading is performed to generate a thinned single-plate image V in FIG. Then, a 4 × 4 low-pass filter is applied in the vicinity of each pixel of the thinned veneer image V to generate an RGB signal at the position ◇ in FIG. This corresponds to obtaining the RGB signal at the ◇ position in FIG.

  Thus, by performing low-frequency generation from a single plate image thinned out in advance, the circuit is simplified and the circuit scale can be reduced.

  In this modification, the position where the RGB signal is generated is shifted by a half pixel from the pixel position of the original single-plate image. Therefore, the bandpass filter used in the high-frequency processing circuit 104 is 4 × 4, and the direction determination circuit 103b The position for determining whether or not to change is also changed.

  For example, a high-frequency signal at the position indicated by a circle in FIG. 11A is generated using a 4 × 4 region in the vicinity of the circle, and the direction determination is performed using pixels in the broken line portion in FIG. Use at the position of x mark. The judgment result is shared with the generation of the high frequency signal at the ◇ mark located immediately above the ○ mark.

  Various other modifications can be considered in the example of the present embodiment. For example, the size of the neighborhood data used in the noise reduction processing used in the noise reduction circuits 205e, 205f, and 205g is not limited to 3 × 3, and may be any size. Even in this case, the processing can be performed at the same timing as in FIG. 9 although the delay time becomes long.

  Similar to the example of the first embodiment, even if the image pickup device is not a single-chip CCD 102 but a three-plate image pickup device, the same increase in speed is possible. In this case, the RGB generation circuit 205a is not necessary.

  Further, when the image pickup device is not a single CCD 102 having a color filter but a monochrome image pickup device, the same increase in speed is possible. In this case, the RGB generation circuit 205a and the color conversion circuit 205b are not necessary, and the gradation conversion circuit 205c directly performs gradation conversion on the output of the image sensor. The YC conversion circuit 205d directly outputs the input from the gradation conversion circuit 105c to the noise reduction circuit 105e, and the noise reduction circuits 205f and 205g, the buffer 205i, the buffers 205j, 206b and 206c, the interpolation circuit 206f, the interpolation circuit 206g, and the interpolation circuit. 206h and the interpolation circuit 206i are unnecessary.

(Third embodiment)
A third embodiment when the present invention is applied to a digital camera will be described below.
Note that the circuit configuration of the image processing mainly for photographing system of the digital camera 100 ″ according to the present embodiment is basically the same as that shown in FIG. 1, and the same reference numerals are used for the same parts. The illustration and description thereof are omitted.

Therefore, in the present embodiment, a low-frequency generation circuit 305 described later is used instead of the low-frequency generation circuit 105 of FIG. 1, and a synthesis circuit 306 described later is used instead of the synthesis circuit 106.
FIG. 12 shows a detailed configuration within the low frequency generation circuit 305. The image data from the preceding image buffer 103 is given to the RGB generation circuit 305a. The RGB generation circuit 305a performs interpolation processing on the image data from the image buffer 103 to generate RGB data in which RGB components are aligned in each pixel, and outputs the RGB data to the color conversion circuit 305b.

  The RGB generation circuit 305a sequentially outputs RGB components at pixel positions indicated by black circles in FIG. In the case of processing one pixel, the surrounding pixel data of the neighborhood 5 × 5 is required as in the example of the first embodiment.

  The color conversion circuit 305b performs addition / subtraction conversion on the RGB data of the primary color signal system according to the white balance value set at that time, and outputs the converted RGB data to the gradation conversion circuit 305c.

  The gradation conversion circuit 305c converts the intermediate gradation value into a predetermined value by performing a gamma correction process on each of the RGB data, and outputs the converted value to the YC conversion circuit 305d.

  The YC conversion circuit 305d converts the RGB data of the primary color signal system into YCrCb data of the luminance / chrominance signal system by matrix calculation, separates the obtained Y (YL) data, Cr data, and Cb data, and generates Y (YL) data. Are output to the noise reduction circuit 305e, and Cr data and Cb data are output to the selection circuit 305f.

  The noise reduction circuit 305e performs low-pass filter processing on Y (YL) data to reduce high-frequency component noise, and outputs it as an intermediate output O3 to the next stage synthesis circuit 206.

  On the other hand, the selection circuit 305f that receives Cr data and Cb data from the YC conversion circuit 305d processes the Cr data and processes the black circle positions of the even columns when the black circle positions of the odd columns of FIG. 6 are processed. Cb data is selectively outputted when

  The Cr data and Cb data output from the selection circuit 305f are low-pass filtered by the noise reduction circuit 305g, and after high-frequency component noise is reduced, are output to the next stage synthesis circuit 206 as an intermediate output O4.

  Next, the configuration and operation of the synthesis circuit 306 that synthesizes the outputs of the high frequency processing circuit 104 and the low frequency generation circuit 305 will be described with reference to FIGS. 13 and 14. FIG.

  FIG. 13 is a configuration diagram of the synthesis circuit 306. FIG. 14 shows the signal contents of the inputs / outputs O1 to O11 of the synthesis circuit in time series, and the contents of the signals are described with processing for one column as one unit period. Note that the time required for processing for one column is that the high frequency processing circuit 104 performs processing in parallel at the positions of the black and white circles adjacent in the upper and lower directions in FIG. 6, and the low frequency generation circuit 305 performs processing only at the black circle position. This is half the number of vertical pixels.

In FIG. 14, the low frequency generation circuit 305 outputs the luminance low frequency signal at the black circle position of each column in FIG. 6 to O3 and the color difference signal to O4 in each unit period. That is, if the period during which the luminance low-frequency signal O3 in the first column is obtained is the reference period t, O3 and O4 are
O3 (j; t + n) = YL (n + 1,2j + 1)
O4 (j; t + n) = Cr (n + 1,2j + 1) (when n is an even number)
O4 (j; t + n) = Cb (n + 1,2j + 1) (when n is an odd number)
(Where n, j are integers,
Oi (j; t): value of j-th intermediate output Oi in period t,
S (x, y): the value of the signal S in x columns and y rows. )
The type of signal obtained as the output O4 depends on the selection result in the selection circuit 305f. Outputs O3 and O4 are stored in buffers 306a and 306b, respectively.

In the high frequency processing circuit 104, the luminance high frequency signal at the black circle position in the column in FIG. 6 is obtained as O1, and the luminance high frequency signal at the white circle position is obtained as O2. The output timing is adjusted so that the luminance high-frequency signal in the n-th column is output in the period (t + n). As a result, the processing positions of the low-frequency signal and the high-frequency signal are aligned in the final outputs O7 to O11. It is like that. That is,
O1 (j; t + n) = YH (n, 2j + 1)
O2 (j; t + n) = YH (n, 2j + 2)
The synthesis circuit 306 generates final outputs O8 to O11 of luminance and color differences in the n-th column in the period (t + n) from the outputs O3 and O4 of the low-frequency generation circuit 305 and the outputs O1 and O2 of the high-frequency processing circuit 104. To do. Since the low-frequency signals are obtained only at specific pixels at the outputs O3 and O4, the low-frequency signals at other positions are the values of the low-frequency signal S at the coordinates (x, y) in FIG. As
In the black circle position in FIG.
YL (2i + 1,2j + 1) = no interpolation required
YL (2i, 2j + 1) = no interpolation required
Cr (2i + 1,2j + 1) = no interpolation required
Cr (2i, 2j + 1) =
(Cr (2i-1,2j + 1) + Cr (2i + 1,2j + 1)) / 2
Cb (2i + 1,2j + 1) =
(Cb (2i, 2j + 1) + Cr (2i + 2,2j + 1)) / 2
Cb (2i, 2j + 1) = no interpolation required
In the white circle position in FIG.
YL (2i + 1,2j) = (YL (2i + 1,2j-1) + YL (2i + 1,2j + 1)) / 2
YL (2i, 2j) = (YL (2i, 2j-1) + YL (2i, 2j + 1)) / 2
Cr (2i + 1,2j) = (Cr (2i + 1,2j-1) + Cr (2i + 1,2j + 1)) / 2
Cr (2i, 2j) =
(Cr (2i + 1,2j-1) + Cr (2i + 1,2j + 1) +
Cr (2i-1,2j-1) + Cr (2i-1,2j + 1)) / 4Cb (2i + 1,2j)
= (Cb (2i + 2,2j-1) + Cb (2i + 2,2j + 1) + Cb (2i, 2j-1) +
Cb (2i, 2j + 1)) / 4
Cb (2i, 2j) = (Cb (2i, 2j-1) + Cb (2i, 2j + 1)) / 2
(i and j are integers)
It generates with the interpolation formula.

  This interpolation calculation is performed by the interpolation circuits 306c to 306g. The interpolation circuits 306d and 306f respectively obtain the Cr color difference signal and the Cb color difference signal at the black circle positions in FIG. The interpolation circuits 306c, 306e, and 306g output a low-frequency luminance signal, a Cr color difference signal, and a Cb color difference signal at the white circle positions in FIG. 6, respectively.

The calculation in each circuit is the following, in which the interpolation formula is replaced with a time series signal. (Here, j and n are integers.)
Interpolator 306d:
Cr color difference signal O8 at the black circle position (2j + 1, n) in FIG.
O8 (j; t + n) = O4 (j; t + n-1) (when n is an odd number)
O8 (j; t + n) =
(O4 (j; t + n-2) + O4 (j; t + n)) / 2 (when n is an even number)
And generate.

Interpolator 306f:
The Cb color difference signal O10 at the black circle position (2j + 1, n) in FIG.
O10 (j; t + n) =
(O4 (j; t + n-2) + O4 (j; t + n)) / 2 (when n is an odd number)
O10 (j; t + n) = O4 (j; t + n-1) (when n is an even number)
And generate.

Interpolator 306c:
The low luminance signal O2 'at the white circle position (2j + 2, n) in FIG.
O2 '(j; t + n) =
(O3 (j; t + n-1) + O3 (j + 1; t + n-1)) / 2 (n is an integer)
And generate.

Interpolator 306e:
The Cr color difference signal O9 at the white circle position (2j + 2, n) in FIG.
O9 (j; t + n) =
(O4 (j; t + n-1) + O4 (j + 1; t + n-1)) / 2 (when n is an odd number)
O9 (j; t + n) =
(O4 (j; t + n-2) + O4 (j + 1; t + n-2)
+ O4 (j; t + n) + O4 (j + 1; t + n)) / 4 (when n is an even number)
And generate.

Interpolator 306f:
The Cb color difference signal O11 at the white circle position (2j + 2, n) in FIG.
O11 (j; t + n) =
(O4 (j; t + n-2) + O4 (j + 1; t + n-2)
+ O4 (j; t + n) + O4 (j + 1; t + n)) / 4 (when n is an odd number)
O11 (j; t + n) =
(O4 (j; t + n-1) + O4 (j + 1; t + n-1)) / 2 (when n is an even number)
And generate.

In the enhancement circuit 306h, from the luminance low-frequency signal O3 one unit period before in the buffer 306a and the luminance high-frequency signal O1 that is the output of the high frequency generation circuit, in the period (t + n), the black circle position in FIG. The final luminance signal O6 of the nth column is calculated by the following formula. That is,
O6 (j; t + n) = α * O1 (j; t + n) + O3 (j; t + n-1)
(Where j is an integer and α is a constant)
Similarly, in the enhancement circuit 206k, from the luminance low-frequency signal O2 ′ that is the output of the interpolation circuit 306c and the luminance high-frequency signal O2 that is the output of the high frequency generation circuit, in the period (t + n), the white circle position in FIG. The final luminance signal O7 of the nth column is calculated by the following formula. That is,
O7 (j; t + n) = α * O2 (j; t + n) + O2 '(j; t + n)
(Where j is an integer, α is a constant)
As shown in FIG. 14, the low-frequency generation circuit 305 generates the intermediate output O4 by processing the color difference signals Cr data and Cb data in a time-sharing manner using a single noise reduction circuit 305g.

  Since the noise reduction circuit 305g can be shared in this way, the circuit scale can be reduced. In addition, the sharing is performed by the circuit for Cr data and Cb data that are color difference signals. It can be shared more easily than sharing a noise reduction circuit between other color components.

  15 and 16 both illustrate pixel position patterns when the data amount is thinned to ¼ in the low frequency generation circuit 305 as in FIG. 10, and “X” in the figure is thinned out. Indicates the pixel position.

  FIG. 15A is an example of a pattern in which pixels are thinned out at intervals of 2 pixels in the vertical direction. In the same row in the horizontal direction, the color difference information Cr data and Cb data are alternately arranged, and the vertical direction Then, the Cr data is arranged between the Cr data and the Cb data is arranged in the same column with the thinned pixel positions interposed therebetween.

  FIG. 15B is an example of a pattern in which pixels are thinned out at intervals of two pixels in the vertical direction, and Cr data and Cb data of color difference information are alternately arranged in the same row in the horizontal direction and are also vertically Even in the direction, Cr data and Cb data are alternately arranged in the same row with the pixel positions thinned out.

  In FIG. 16A, the relationship between the Cr data and Cb data and the pixel position (X) to be thinned out, and the relationship between the Cr data and Cb data are arranged in a checkered pattern so that each element in the horizontal and vertical directions. Shows an example of arrangement so that are not directly adjacent to each other.

  In FIG. 16B, the Y (YL) data as luminance information and the pixel position (X) to be thinned are arranged in a checkered pattern so that the pixel positions to be thinned in the horizontal direction and the vertical direction are not directly adjacent to each other. An example is given.

  Various modifications can be made to the example of the present embodiment. In the example shown in FIG. 15B, the selection pattern of the color difference data of the selection circuit 305f is changed. In FIG. 15, X indicates that no signal can be obtained. In contrast to the selection pattern in the example of the present embodiment shown in FIG. 15A, a feature is that a Cr signal and a Cb signal are obtained in each column.

  FIGS. 16A and 16B are examples in which the processing target rows of the low-frequency generation circuit 305 are changed depending on the columns. In the example of this embodiment, only at the positions of the black circles (odd rows) in FIG. The low-frequency signal is obtained, whereas the low-frequency signal is obtained in a checkered pattern.

  In any of these patterns, as in the example of the present embodiment, the missing low frequency signal may be interpolated from the low frequency signal obtained in the vicinity thereof. A more complicated interpolation method may be used, and the size of the neighborhood used for the interpolation is not limited to the 3 × 3 size of the present embodiment, but may be a larger size. In that case, the buffer in the low-frequency generation circuit 305 may be increased so that past data can be accumulated in time series.

  Further, even when the image pickup device is not a single-chip CCD 102 but a three-plate image pickup device, the same increase in speed is possible. In this case, the RGB generation circuit 305a becomes unnecessary.

  In this way, in the low frequency generation circuit 305, as shown in FIGS. 15 and 16, the generated low frequency component image is composed of a plurality of color components, and a specific color component is missing at each pixel position. As a result, processing more suitable for the characteristics of the Bayer arrangement can be performed, and the pixel position from which each color component is obtained is shifted, so that more information can be obtained as spatial information.

  Further, as shown in FIG. 16A, the plurality of color components constituting the generated low-frequency component image are Cr data and Cb data which are color difference components, and pixel positions and Cb data where Cr data is missing. The amount of spatial information obtained is maximized by arranging the pixel positions at which the missing is in a checkered pattern.

  The first to third embodiments have been described with reference to a case where the present invention is applied to a digital camera for taking a still image. However, the present invention is not limited to this, for example, for taking a moving image. The present invention is also applicable to electronic devices such as video movie cameras and mobile phone terminals with camera functions. This electronic device is a device that relies on current or electromagnetic field to operate correctly.

  In each of the above-described embodiments, an example is shown in which two high-frequency generation circuits 104A and 104B are used to form the high-frequency processing circuit 104. However, as a modification, three or more high-frequency generation circuits may be used. Good. In this case, the pixel positions that the first to n-th (n is a natural number of 3 or more) high-frequency generation circuits perform parallel processing at a time are n pixel positions arranged next to each other. For example, if the processing at the pixel position shown in FIG. 6 is performed using three high-frequency generation circuits, the first high-frequency generation circuit has a black circle position, and the second high-frequency generation circuit has the black circle position. The white circle position next to the black circle position, and the star position (not shown) next to the white circle position are the targets of the high frequency component generation processing. In this modification as well, all the high-frequency generation circuits may share one direction determination circuit 104b.

  Further, in each of the above embodiments, each of the plurality of high-frequency generation circuits has at least an edge extraction circuit and a noise reduction circuit. However, as a modification, the noise reduction circuit is excluded and at least an edge extraction circuit is included. It may be.

  In addition, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention at the implementation stage. In addition, the functions executed in the above-described embodiments may be combined as appropriate as possible. The embodiments described above include various stages, and various inventions can be extracted by an appropriate combination of a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, if the effect is obtained, a configuration from which the constituent requirements are deleted can be extracted as an invention.

1 is a block diagram mainly showing a circuit configuration of a photographing system of a digital camera according to a first embodiment of the present invention. It is a figure which concerns on the same embodiment. FIG. 2 is a block diagram showing a detailed configuration in a high frequency generation circuit of FIG. 1. It is a figure which concerns on the same embodiment. FIG. 2 is a block diagram showing a detailed configuration in a low frequency generation circuit of FIG. 1. It is a figure which concerns on the same embodiment. FIG. 2 is a block diagram illustrating a detailed configuration in a synthesis circuit in FIG. 1. 4 is a timing chart showing an output signal of each circuit according to the embodiment together with an operation clock. It is a figure which illustrates the pixel position which performs the thinning process in the image processing process which concerns on the embodiment. It is a block diagram which shows the detailed structure in the low-pass production | generation circuit based on the 2nd Embodiment of this invention. FIG. 3 is a block diagram showing a detailed configuration in a synthesis circuit according to the same embodiment. 4 is a timing chart showing an output signal of each circuit according to the embodiment together with an operation clock. It is a figure which illustrates the pixel position which performs the thinning process in the image processing process which concerns on the embodiment. It is a figure which illustrates the other pixel position which performs the thinning process in the image processing process which concerns on the embodiment. It is a block diagram which shows the detailed structure in the low-pass production | generation circuit based on the 3rd Embodiment of this invention. FIG. 3 is a block diagram showing a detailed configuration in a synthesis circuit according to the same embodiment. 4 is a timing chart showing an output signal of each circuit according to the embodiment together with an operation clock. It is a figure which illustrates the pixel position which performs the thinning process in the image processing process which concerns on the embodiment. It is a figure which illustrates the other pixel position which performs the thinning process in the image processing process which concerns on the embodiment. It is a figure explaining the effect | action of the high region production | generation circuit which concerns on the 1st Embodiment of this invention. It is a figure explaining the modification which concerns on the 1st Embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100,100 ', 100 "... Digital camera, 101 ... Lens optical system, 102 ... Single-plate CCD, 103 ... Image buffer, 104 ... High frequency processing circuit, 104A, 104B ... High frequency generation circuit, 105 ... Low frequency generation circuit , 106 ... synthesis circuit, 107 ... compression circuit, 108 ... recording unit, 110 ... processor, 111 ... control circuit, 205 ... low frequency generation circuit, 206 ... synthesis circuit, 305 ... low frequency generation circuit, 306 ... synthesis circuit, 400 DESCRIPTION OF SYMBOLS ... Digital camera 401 ... Lens optical system 402 ... Single plate CCD, 403 ... Image buffer, 404 ... Y generation circuit, 405 ... Low frequency generation circuit, 406 ... Synthesis circuit, 407 ... Compression circuit, 408 ... Recording part.

Claims (14)

An input unit for inputting an image;
A plurality of high-frequency generators each generating high-frequency components at different pixel positions in the input image;
A low-frequency generation unit that generates a low-frequency component image having a smaller number of pixels than the number of pixels of the input image;
A combining unit that combines the generated high-frequency component and the generated low-frequency component image to obtain a combined image;
An image processing apparatus comprising: a control unit that operates the plurality of high frequency generation units in parallel. The low frequency component image is a color image including a plurality of color components;
The low-frequency generation unit includes a noise reduction unit that reduces noise of each color component of the color image,
The image processing apparatus according to claim 1, wherein the noise reduction unit performs noise reduction processing for a plurality of color components in a time division manner using a common circuit. The low frequency generation unit includes a noise reduction unit that reduces noise of the low frequency component image,
The image processing apparatus according to claim 1, wherein the low-frequency generation unit reduces noise by using the noise reduction unit a plurality of times cyclically. The input image is a color image including a plurality of color components,
The high frequency generation unit generates a high frequency luminance component from the color image,
The image according to claim 1, wherein the synthesis unit obtains a color image including a plurality of color components as the synthesized image as a result of synthesizing the generated high-frequency luminance component and the low-frequency component image. Processing equipment.   The image processing apparatus according to claim 1, wherein the low-frequency generation unit generates the low-frequency component image after thinning the input image.   The low-frequency generation unit has a two-pixel interval with one pixel in the horizontal or vertical direction with respect to the input image, and a one-pixel interval adjacent in the vertical or horizontal direction corresponding to the direction. The image processing apparatus according to claim 1, wherein the low frequency component image is generated by extracting a low frequency component at a pixel position corresponding to a lattice point.   The low-frequency generation unit extracts the low-frequency component at a pixel position corresponding to a lattice point having a two-pixel interval with one pixel between the input image in the horizontal and vertical directions. The image processing apparatus according to claim 1, wherein a frequency component image is generated.   The image processing apparatus according to claim 2, wherein the plurality of color components whose noise is reduced by the noise reduction unit includes at least one of color difference components Cr and Cb.   The low-frequency component image is a color image including a plurality of color components, and in the color image, pixel positions where at least one of the plurality of color components is missing, and specific color components in the plurality of color components are The image processing apparatus according to claim 1, wherein the missing pixel position has an arrangement relationship according to a predetermined rule.   The image processing according to claim 9, wherein the plurality of color components are color difference components Cr and Cb, and a pixel position where the color difference component Cr is missing and a pixel position where the color difference component Cb is missing are in a checkered pattern. apparatus. An edge direction determination unit that determines an edge direction of a high-frequency component in the input image;
The plurality of high frequency generation units share the edge direction determination unit, respectively, and each extract a high frequency component that becomes an edge based on a determination result by the edge direction determination unit. The image processing apparatus described. Each of the plurality of high frequency generation units includes an edge extraction unit that extracts an edge of a high-frequency component in the input image, and a noise reduction unit that reduces noise in the signal including the edge extracted by the edge extraction unit. ,
The image processing apparatus according to claim 11, wherein each of the edge extraction units and the noise reduction unit included in each of the plurality of high frequency generation units shares the edge direction determination unit. An input step for inputting an image;
A plurality of high-frequency generation steps each generating high-frequency components at different pixel positions in the input image;
A low frequency generation step of generating a low frequency component image having a number of pixels smaller than the number of pixels of the input image;
A synthesis step of synthesizing the generated high frequency component and the generated low frequency component image to obtain a composite image;
An image processing method comprising: a control step for operating the plurality of high frequency generation steps in parallel.   An electronic apparatus comprising the image processing apparatus according to claim 1.

Images