JP4176023B2 - Signal processing circuit - Google Patents

Description

  The present invention relates to a signal processing circuit, and more particularly to a signal processing circuit that is applied to, for example, an electronic camera and reduces color noise of an image signal output from an imaging device.

An example of a conventional device of this type is disclosed in Patent Document 1. This prior art is intended to reduce color noise without degrading the apparent resolution by performing a smoothing process only on the color difference signal in the portion where neither the luminance edge nor the color edge appears.
JP 2001-175843 A [G06T 1/00, 5/00, H04N 1/409, 1/46]

  However, since the smoothing process of the prior art is based on the color difference signal that has been subjected to signal processing, there has been a limit to the improvement in image quality.

  Therefore, a main object of the present invention is to provide a signal processing circuit capable of further improving the image quality.

According to a first aspect of the present invention, there is provided a signal processing circuit comprising: a plurality of color components having a plurality of color components per pixel based on raw image data including a plurality of pixel data having a color component of any one of a plurality of colors per pixel; A signal processing circuit for generating desired image data composed of mixed pixel data, wherein a first coefficient group corresponds to each of a first number of pixel data located in a first range from a point of interest in a predetermined region of raw image data A first color separation unit that multiplies the coefficients to perform color separation processing and outputs a plurality of first color separation pixel data each having one color of a plurality of color components; and a predetermined region of the raw image data Color separation processing is performed by multiplying each of the second number of pixel data having a number larger than the first number located in the second range wider than the first range from the point of interest by multiplying the corresponding coefficient of the second coefficient group. each of the plurality of colors of color components The first color separation pixel data and the second color separation pixel data output from the plurality of second color separation second color separating means for outputting pixel data, the first color separating means and the second color separating means having one color Mixing means for generating a mixed pixel data corresponding to a target point of desired image data by mixing each color component in accordance with a plurality of mixing coefficients corresponding to the color components of a plurality of colors , and around a target point of a predetermined region mixing the result coefficient, the more variation is large, a plurality of such mixing ratio of the first color separation pixel data is increased discrimination of the discrimination means, and discriminating means for discriminating the magnitude of variation of each of the plurality of colors of color components in Adjusting means for adjusting each of the above.

Each of the plurality of pixel data constituting the raw image data has one color color information of a plurality of colors. The plurality of pixel signals constituting the raw image data are subjected to color separation according to the first coefficient group by the first color separation means, and color separation according to the second coefficient group is performed by the second color separation means. The first color separation means outputs a plurality of first color separation pixel data each having color information of a plurality of colors. The second color separation means also outputs a plurality of second color separation pixel data each having a plurality of color information.

The mixing unit mixes the first color separation pixel data and the second color separation pixel data for each color component. In mixing, a plurality of mixing coefficients respectively corresponding to a plurality of color components are used. Further, the variation of each of the color components of the plurality of colors around the attention point is determined by the determination unit. Each of the plurality of mixing coefficients is adjusted by the adjusting unit based on the determination result of the determining unit.

  Since the first color separation unit and the second color separation unit perform color separation according to different coefficients, the first color separation pixel signal and the second color separation pixel signal exhibit different frequency characteristics. The first color separation pixel signal and the second color separation pixel signal are mixed for each color component in consideration of the variation of each color component. The image signal based on the mixed pixel signal thus obtained becomes an image signal with reduced color noise without impairing the color resolution.

The determining unit determines each of the plurality of mixing coefficients so that the mixing ratio of the first separated pixel data increases as the variation of each of the color components around the target point increases as a result of the determination by the determining unit. To do.

  The variation of the color component increases around the color edge. Therefore, if the first color separation pixel signal is emphasized as the variation is larger, a mixed pixel signal in which the color edge is emphasized can be obtained.

A signal processing circuit according to a second aspect of the invention is dependent on the first aspect, wherein the determining means calculates an average color component value for each color component for a plurality of pixels distributed around the target point , and Sum total calculating means for calculating the sum of the difference between the average color component value and the color component value of each of the plurality of pixels for each color component is included. This makes it possible to accurately determine the variation of each color component around the attention point .

A signal processing circuit according to a third aspect of the present invention is dependent on the first or second aspect, and includes a first creating unit that creates desired image data based on a plurality of mixed pixel data output from the mixing unit, and a plurality of first processing units. Second creation means for creating contour enhancement data reflecting a difference between the color separation pixel data and the plurality of second color separation pixel data , and contour enhancement means for performing contour enhancement according to the contour enhancement data on desired image data Prepare.

  As described above, the first color separation pixel signal and the second color separation pixel signal have different frequency characteristics. The contour enhancement signal reflects the difference between the first color separation pixel signal and the second color separation pixel signal. As a result, high-quality contour enhancement is realized.

The signal processing circuit according to an invention of claim 4 is according to any one of claims 1 to 3, the raw image data is the image data outputted from the photographing means for photographing an object scene.

An electronic camera according to a fifth aspect of the invention includes the signal processing circuit according to any one of the first to fourth aspects.

  According to the present invention, the first color separation pixel signal and the second color separation pixel signal having different frequency characteristics are mixed in consideration of variations in color components, so that a high-quality mixed pixel signal and thus an image signal are generated. can do.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  Referring to FIG. 1, a digital camera (electronic camera) 10 according to this embodiment includes an image sensor 14 that captures an object scene. As shown in FIG. 2, the imaging surface of the image sensor 14 is covered by a primary color filter 12 in which R (Red) filter elements, G (Green) filter elements, and B (Blue) filter elements are arranged in a mosaic pattern. . The plurality of light receiving elements that form the imaging surface correspond one-to-one with the plurality of filter elements that form the primary color filter 12. Accordingly, the charge generated by photoelectric conversion in each light receiving element, that is, the pixel signal has color information (color component) of any one of R, G, and B.

  When the power is turned on, the CPU 20 instructs a TG (Timing Generator) 18 to repeat pre-exposure and vertical thinning readout in order to output a real-time moving image (through image) of the object scene from the LCD monitor 60. The TG 18 performs pre-exposure and vertical thinning readout on the image sensor 14 in response to a vertical synchronization signal generated once every 1/30 seconds. From the image sensor 14, a raw image signal with reduced vertical resolution is read out in a raster scanning manner. The low-resolution raw image signal to be read has an odd line whose color information changes in the order of R, G, R, G,... And an even line whose color information changes in the order of G, B, G, B,. Including. The read raw image signal is subjected to a series of processes of correlated double sampling, automatic gain adjustment, and A / D conversion by the CDS / AGC / AD circuit 16, thereby generating raw image data as a digital signal. The

  The image sensor 12, the TG 18, and the CDS / AGC / AD circuit 16 form an imaging device. Among the pixel data forming the raw image data, pixel data having R color information is defined as “Rraw data”, pixel data having G color information is defined as “Graw data”, and B data Pixel data having color information is defined as “Braw data”.

  The raw image data output from the CDS / AGC / AD circuit 16 is input from the port P 1 to the ASIC 24 and is supplied to the first color separation circuit 26 and the second color separation circuit 28. When attention is paid to the pixel block of 6 horizontal pixels × vertical 6 pixels shown in FIG. 4A, the first color separation circuit 26 performs the calculation according to the equations 1 to 3, and the second color separation circuit 28 the equations 4 to An operation according to Equation 6 is executed.

  Note that Rraw1 to Rraw8 are Rraw data of the R1 pixel to R8 pixel shown in FIG. Moreover, Graw3, Graw4, Graw7-Graw10, Graw13, and Graw14 are Graw data of G3 pixel, G4 pixel, G7 pixel-G10 pixel, G13 pixel, and G14 pixel shown to FIG. 4 (A), respectively. Further, Braw1 to Braw8 are the Braw data of B1 to B8 pixels shown in FIG.

  The Rsep1 data, the Gsep1 data, and the Bsep1 data obtained by Equations 1, 2, and 3, respectively, are pixel data (first color separation pixel data) at a position (target pixel) indicated by a black circle in FIG. 4B. . Further, the Rsep2 data, Gsep2 data, and Bsep2 data obtained by the equations (4), (5), and (6), respectively, are pixel data (second color separation pixel data) at a position (target pixel) indicated by a black circle in FIG. It is.

  That is, the first color separation circuit 26 performs an operation for color separation using the coefficients shown in FIG. 4B, and the second color separation circuit 28 performs color separation using the coefficients shown in FIG. Perform the operation for. As a result, Rsep1 data, Gsep1 data, and Bsep1 data contain higher frequency components than Rsep2 data, Gsep2 data, and Bsep2 data, respectively.

  For example, if the 17-pixel continuous Rraw data, the Raw data, and the Raw data are changed as shown in FIG. 6, the Rsep1 data and the Rsep2 data are changed in the horizontal direction as shown in FIG. Gsep2 data changes in the manner shown in FIG. 8 in the horizontal direction, and Bsep1 data and Bsep2 data change in the manner shown in FIG. 9 in the horizontal direction.

  Returning to FIG. 1, the Rsep1 data, the Gsep1 data, and the Bsep1 data output from the first color separation circuit 26 are subjected to a calculation according to Equation 7 in the luminance matrix calculation circuit 30. The Rsep2 data, the Gsep2 data, and the Bsep2 data output from the second color separation circuit 28 are subjected to a calculation according to Equation 8 by the luminance matrix calculation circuit 32.

  The first luminance matrix calculation circuit 30 outputs Y1 data, and the second luminance matrix calculation circuit 32 outputs Y2 data. The Y1 data is the luminance data (first luminance pixel data) at the position indicated by the black circle in FIG. 4B, and the Y2 data is the luminance data (second luminance pixel data) at the position indicated by the black circle in FIG. 4C. is there.

  When Rsep1 data, Rsep2 data, Gsep1 data, Gsep2 data, Bsep1 data, and Bsep2 data change as shown in FIGS. 7 to 9, Y1 data and Y2 data change as shown in FIG. In other words, due to the difference in coefficients between the first color separation circuit 26 and the second color separation circuit 28, the Y1 data also includes a higher frequency component than the Y2 data.

  The subtraction circuit 34 performs an operation according to Equation 9 on the Y1 data and Y2 data. From the subtraction circuit 34, EDG data (outline emphasis data) at a position indicated by a black circle in FIG. 4B or FIG. 4C is output.

  When the Y1 data and the Y2 data change as shown in FIG. 10, the EDG data changes as shown in FIG.

  Returning to FIG. 1, the raw image data input from the port P1 is also provided to the average deviation calculation circuits 40, 44 and 48. When attention is paid to the pixel block shown in FIG. 4A, the average deviation calculating circuit 40 executes the calculation according to the equations 10 and 11, the average deviation calculating circuit 44 executes the operation according to the equations 12 and 13, and the average The deviation calculating circuit 48 performs an operation according to the equations 14 and 15.

  The operations of Equations 10, 12 and 14 are executed by average value calculation circuits 40a, 44a and 48a, respectively, and the operations of Equations 11, 13 and 15 are executed by deviation value calculation circuits 40b, 44b and 48b, respectively. Is done.

  The average deviation value Rmd obtained by the average deviation calculating circuit 40 indicates the degree of variation (blurring) of Rraw data belonging to the pixel block shown in FIG. Further, the average deviation value Gmd obtained by the average deviation calculation circuit 44 indicates the degree of variation of the Raw data belonging to the pixel block shown in FIG. Furthermore, the average deviation value Bmd obtained by the average deviation calculation circuit 48 indicates the degree of variation of the Braw data belonging to the pixel block shown in FIG.

  The coefficient determination circuit 42 determines the coefficient Kr based on the average deviation value Rmd, the coefficient determination circuit 46 determines the coefficient Kg based on the average deviation value Gmd, and the coefficient determination circuit 50 determines the coefficient based on the average deviation value Bmd. Kb is determined. Specifically, the coefficients Kr, Kg, and Kb corresponding to the average deviation values Rmd, Gmd, and Bmd are determined according to the graph shown in FIG. According to FIG. 5, the coefficients Kr, Kg, and Kb decrease as the average deviation values Rmd, Gmd, and Bmd increase, respectively. The slope when the average deviation values Rmd, Gmd, and Bmd are below the threshold value TH is “SLP1”, and the slope when the average deviation values Rmd, Gmd, and Bmd are above the threshold value TH is “SLP2”.

  The mixing circuit 36 adds coefficients Kr, Kg and Rsep1 data, Gsep1 data and Bsep1 data output from the first color separation circuit 26, and Rsep2 data, Gsep2 data and Bsep2 data output from the second color separation circuit 28, respectively. Each weighted addition according to Kb is performed.

  Referring to FIG. 3, mixing circuit 36 includes multipliers 36a to 36f and adders 36g to 36i. The multiplier 36a multiplies the Rsep1 data by a coefficient “1-Kr”, and the multiplier 36b multiplies the Rsep2 data by a coefficient Kr. The outputs of the multipliers 36a and 36b are added to each other by an adder 36g, whereby Rsep3 data is obtained.

  The multiplier 36c multiplies the Gsep1 data by a coefficient “1-Kg”, and the multiplier 36d multiplies the Gsep2 data by a coefficient Kg. The outputs of the multipliers 36c and 36d are added to each other by an adder 36h, whereby Gsep3 data is obtained.

  The multiplier 36e multiplies the Bsep1 data by a coefficient “1-Kb”, and the multiplier 36f multiplies the Bsep2 data by a coefficient Kb. The outputs of the multipliers 36e and 36f are added to each other by an adder 36i, whereby Bsep3 data is obtained.

  The Rsep3 data is mixed pixel data having R color information, the Gsep3 data is mixed pixel data having G color information, and the Bsep3 data is mixed pixel data having B color information.

  As a result of the coefficient determination and the weighted addition as described above, the output of the first color separation circuit 26, that is, the high frequency component is emphasized in the portion where the degree of color variation is large, that is, the color contour portion, and the degree of color variation. In the portion where the color is small, that is, the portion where the color outline does not appear, the output of the second color separation circuit 28, that is, the low frequency component is emphasized. As a result, the color outline can be accurately preserved. Further, by performing weighted addition according to the coefficients Kr, Kg, and Kb prepared for each color information, it is possible to reduce color noise without impairing the color resolution.

  Returning to FIG. 1, the preprocessing circuit 38 performs processes such as horizontal thinning, white balance adjustment, and YUV conversion on the Rsep3 data, Gsep3 data, and Bsep3 data output from the mixing circuit 36. From the preprocessing circuit 38, Y data as luminance data and U data and V data as color difference data are output.

  The U data and V data are input to the post-processing circuit 54 as they are, and the Y data is input to the post-processing circuit 54 via the contour emphasizing circuit 52. In the contour emphasizing circuit 52, contour emphasis is performed on the Y data in accordance with the EDG data output from the subtracting circuit 34. Specifically, EDG data is added to Y data with a predetermined weighting amount.

  When the through image is displayed on the LCD monitor 60, the post-processing circuit 54 converts the input Y data, U data, and V data into an NTSC composite video signal. The converted composite video signal is applied to the LCD monitor 60 from the port P3. As a result, a through image is output from the LCD monitor 60.

  When the shutter button 22 is operated, the CPU 20 commands the TG 18 to perform one main exposure and one full pixel readout. The TG 18 performs main exposure on the image sensor 14, and reads out all pixel signals generated thereby, that is, high-resolution raw image signals, from the image sensor 14 in a raster scanning manner. This raw image signal also includes odd lines whose color information changes in the order of R, G, R, G,... And even lines whose color information changes in the order of G, B, G, B,.

  The raw image signal output from the image sensor 14 is converted into raw image data by the CDS / AGC / AD circuit 16, and the converted raw image data is input from the port P1 to the ASIC 24. The input raw image data is subjected to processing similar to that described above. As a result, high-resolution Y data, U data, and V data are provided to the post-processing circuit 54. When the shutter button 22 is operated, the post-processing circuit 54 compresses the input YUV data by the JPEG method, and supplies the compressed YUV data generated thereby to the recording processing circuit 56 via the port P2. The recording processing circuit 56 records the given compressed YUV data on the recording medium 58 in a file format.

  As can be seen from the above description, the pixel data of a plurality of pixels input from the port P1 has color information of any one of R, G, and B. The first color separation circuit 26 performs color separation processing according to the coefficients shown in FIG. 4B on the pixel data, and the second color separation circuit 28 performs color separation according to the coefficients shown in FIG. 4C on the pixel data. Apply processing. From the first color separation circuit 26 and the second color separation circuit 28, the first color separation pixel data and the second color separation pixel data each having all the color information of R, G and B are output, respectively. The subtraction circuit 34 creates contour enhancement data that reflects the difference between the first color separation pixel data and the second color separation pixel data. The luminance data output from the pre-processing circuit 38 is subjected to contour enhancement according to the contour enhancement data by the contour enhancement circuit 52.

  Both the first color separation circuit 26 and the second color separation circuit 28 perform color separation on the pixel data input from the port P1. However, coefficients used for color separation differ between the first color separation circuit 26 and the second color separation circuit 28, and the first color separation pixel data and the second color separation pixel data have different frequency characteristics. The edge enhancement data reflects the difference between the first color separation pixel data and the second color separation pixel data. As a result, high-quality contour enhancement is realized.

  The mixing circuit 36 mixes the first color separation pixel data and the second color separation pixel data corresponding to the same pixel for each color information. In mixing, coefficients Kr, Kg, and Kb corresponding to R, G, and B are used. Further, the variation for each color information of the pixel data belonging to the pixel block shown in FIG. 4A is discriminated by the average deviation calculation circuit 40, 44 or 48. Coefficients Kr, Kg, and Kb are adjusted by coefficient determination circuits 42, 46, and 50 based on the outputs of average deviation calculation circuits 40, 44, and 48, respectively. As a result, the greater the variation, the more important is the first color separation pixel data.

  As described above, the first color separation pixel data and the second color separation pixel data exhibit different frequency characteristics. The first color separation pixel data and the second color separation pixel data are mixed for each color information. The image data based on the mixed pixel data becomes image data in which color noise is reduced without impairing the color resolution. Also, the color variation increases around the color edge. Therefore, the larger the variation, the more important the first color separation pixel data is, so that mixed pixel data in which color edges are stored can be obtained.

  In this embodiment, EDG data is obtained according to Equation 9, but EDG data may be obtained according to Equation 16 instead of Equation 9.

It is a block diagram which shows one Example of this invention. It is an illustration figure which shows an example of the color filter applied to FIG. 1 Example. It is a block diagram which shows an example of the mixing circuit applied to the FIG. 1 Example. (A) is an illustrative view showing an assignment state of color elements on a color filter, (B) is an illustrative view showing an assignment state of coefficients used in the first color separation circuit, and (C) is a second illustration. It is an illustration figure which shows the allocation state of the coefficient used with a color separation circuit. It is a graph which shows the change of the coefficient Kr, Kg, or Kb with respect to the average deviation value Rmd, Gmd, or Bmd. It is a graph which shows an example of change of Raw data, Raw data, and Raw data. It is a graph which shows an example of a change of Rsep1 data and Rsep2 data. It is a graph which shows an example of change of Gsep1 data and Gsep2 data. It is a graph which shows an example of change of Bsep1 data and Bsep2 data. It is a graph which shows an example of change of Y1 data and Y2 data. It is a graph which shows an example of a change of EDG data.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Electronic camera 12 ... Primary color filter 14 ... Image sensor 26 ... 1st color separation circuit 28 ... 2nd color separation circuit 30 ... 1st brightness | luminance matrix calculation circuit 32 ... 2nd brightness | luminance matrix calculation circuit 34 ... Subtraction circuit 36 ... Mixing circuit 40, 44, 48 ... average deviation calculation circuit 42, 46, 50 ... coefficient determination circuit 52 ... contour enhancement circuit

Claims (5)

Desired image data consisting of a plurality of mixed pixel data having a plurality of color components per pixel based on raw image data consisting of a plurality of pixel data having a color component of any one of a plurality of colors per pixel A signal processing circuit to generate,
Each of the first number of pixel data located in the first range from the target point of the predetermined region of the raw image data is multiplied by a corresponding coefficient of the first coefficient group to perform color separation processing, and each of the plurality of colors a plurality of first color separation pixel first color separating means for outputting the data having one color among color components,
Correspondence of the second coefficient group to each of the second number of pixel data having a number larger than the first number located in a second range wider than the first range from the attention point of the predetermined region of the raw image data A second color separation unit that performs a color separation process by multiplying a coefficient to output a plurality of second color separation pixel data each having one of the color components of the plurality of colors,
According to a plurality of mixing coefficients corresponding to each of said first color separating means and said output from each of the second color separating means first color separation pixel data and the second color separation pixel data to said plurality of colors of color components Mixing means for mixing each color component to generate mixed pixel data corresponding to the target point of the desired image data ;
A discriminating unit that discriminates the magnitude of variation of each of the color components of the plurality of colors in the vicinity of the target point of the predetermined area ; and as a result of discrimination by the discriminating unit, the larger the variation, the first color separation pixel data A signal processing circuit comprising adjustment means for adjusting each of the plurality of mixing coefficients so that a mixing ratio of the plurality of mixing coefficients increases . The determining means is an average color component value calculating means for calculating an average color component value for each color component for a plurality of pixels distributed around the point of interest, and the average color component value is calculated for each color of the plurality of pixels. including total sum calculation means for calculating the difference between the sum of the component values for each color component, according to claim 1 Symbol placement of the signal processing circuit. First creation means for creating the desired image data based on a plurality of mixed pixel data output from the mixing means;
Wherein the plurality of first color separation pixel data and the plurality of second color separation pixel data and a second creating means the difference to create a contour enhancement data reflected is, and the desired contour in accordance with the edge enhancement data to the image data further comprising edge enhancement means for performing enhancement, according to claim 1 or 2 signal processing circuit according. The raw image data is the image data outputted from the photographing means for photographing a scene, the signal processing circuit according to any one of claims 1 to 3. It claims 1 to an electronic camera comprising a signal processing circuit according to any one of the 4.

Images