JP2011239060A - Image data processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image data processor capable of improving the quality of a YUV image data.SOLUTION: In this image data processor, a white balance adjustment circuit 52 outputs an RGB image data composed of a plurality of pixels each of which has color information of R, G and B. A YUV conversion circuit 54 mixes the color information of the RGB image data outputted by the white balance adjustment circuit 52 with respect to each pixel in a manner conforming to a mixture coefficient, and generates a YUV image data composed of a plurality of pixels each of which has brightness information. Integrators 60r, 60g and 60b integrate the color information of the RGB image data outputted by the white balance adjustment circuit 52 with respect to each common color, and calculate values of integral Ir, Ig and Ib corresponding to R, G and B respectively. A magnitude of the mixture coefficient referenced by the YUV conversion circuit 54 is adjusted on the basis of the rate of the calculated values of integral Ir, Ig and Ib.

Description

  The present invention relates to an image data processing apparatus, and more particularly to an image data processing apparatus which is applied to a digital camera and processes image data formed by a plurality of pixels each having color information of a plurality of representative colors.

  An example of this type of device is disclosed in Patent Document 1. According to this background art, the R data is corrected by each of the R LUT 1 and the R LUT, the G data is corrected by the G LUT, and the B data is corrected by the B LUT. The discrimination selection circuit discriminates the level of the C component calculated based on the R data, the G data, and the B data, and selects one of the R LUT 1 and the R LUT. Thereby, white balance adjustment in consideration of the level of the C component is realized.

JP 2001-285890 A

  However, in the background art, the level of the C component is not referred to when converting RGB data into Y data, and there is a limit to the improvement in image quality.

  Therefore, a main object of the present invention is to provide an image data processing apparatus capable of improving the image quality.

  An electronic camera according to the present invention (10: reference numeral corresponding to the embodiment; the same applies hereinafter) includes first creation means for creating first image data formed by a plurality of pixels each having color information of a plurality of representative colors. (50, 52), second image data formed by a plurality of pixels each having luminance information by mixing the color information of the first image data created by the first creating means for each pixel in a manner according to the mixing coefficient. A second creating means (54) for creating a color image, and integrating the color information of the first image data created by the first creating means for each representative color to calculate a plurality of integral values respectively corresponding to a plurality of representative colors. Means (60r, 60g, 60b) and adjusting means (S23 to S29) for adjusting the magnitude of the mixing coefficient based on the ratio of the plurality of integral values calculated by the integrating means.

  Preferably, third creation means (92 to 96) for creating a plurality of contour emphasis components respectively corresponding to a plurality of representative colors based on the first image data created by the first creation means, created by the third creation means A plurality of contour emphasizing components mixed in a manner according to the mixing coefficient to create contour emphasizing data (98-104), and contour emphasizing processing with reference to the contour emphasizing data created by the fourth creating means Is further provided with a contour emphasizing means (58) for executing the above on the second image data created by the second creating means.

  Preferably, the plurality of representative colors include red, green, and blue, and the adjusting means includes first dividing means (S25) for dividing the integral value corresponding to red by the integral value corresponding to green, and the integral corresponding to blue Second division means (S27) for dividing the value by the integral value corresponding to green is included.

  More preferably, the second creation means includes a first multiplication means (72) for multiplying the division result of the first division means by the color information corresponding to red, and a color information corresponding to the division result of the second division means for blue. Second multiplying means (78) for multiplying.

  Preferably, the image processing apparatus includes a color filter (16) in which a plurality of filter elements respectively corresponding to a plurality of representative colors are arranged in a mosaic pattern, and an imaging surface covered with the color filters, and any one of the plurality of representative colors. An imaging means (18) for outputting raw image data formed by a plurality of pixels each having color information is further provided, and the first creating means is configured to output the first image data based on the raw image data output from the imaging means. Create

  Preferably, the second image data corresponds to image data in YUV format.

  According to the present invention, the plurality of integral values are calculated by integrating the color information of the first image data for each representative color. Therefore, the calculated ratio of the plurality of integral values reflects the color tone of the image represented by the first image data. The luminance information of the second image data is created with reference to a mixing coefficient indicating a different size according to such a ratio. As a result, the quality of the second image data can be improved.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

It is a block diagram which shows the basic composition of this invention. It is a block diagram which shows the structure of one Example of this invention. It is an illustration figure which shows an example of a structure of the primary color filter applied to the FIG. 2 Example. It is an illustration figure which shows an example of the allocation state of the evaluation area in an imaging surface. FIG. 3 is a block diagram illustrating an example of a configuration of a signal processing circuit applied to the embodiment in FIG. 2. FIG. 6 is a block diagram showing a part of a configuration of a YUV conversion circuit applied to the embodiment in FIG. 5; FIG. 6 is a block diagram showing an example of a configuration of an aperture signal generation circuit applied to the embodiment in FIG. 5. (A) is a wave form diagram which shows an example of the input to the secondary differential filter applied to FIG. 7 Example, (A) is an example of the output from the secondary differential filter applied to FIG. 7 Example. FIG. It is a flowchart which shows a part of operation | movement of CPU applied to the FIG. 2 Example. It is a flowchart which shows a part of other operation | movement of CPU applied to the FIG. 2 Example.

Embodiments of the present invention will be described below with reference to the drawings.
[Basic configuration]

  Referring to FIG. 1, the image data processing apparatus of the present invention is basically configured as follows. The first creation means 1 creates first image data formed by a plurality of pixels each having color information of a plurality of representative colors. The second creating means 2 mixes color information of the first image data created by the first creating means 1 for each pixel in a manner according to the mixing coefficient, and is formed by a plurality of pixels each having luminance information. Create image data. The integrating unit 3 integrates the color information of the first image data created by the first creating unit 1 for each representative color, and calculates a plurality of integrated values respectively corresponding to a plurality of representative colors. The adjusting unit 4 adjusts the size of the mixing coefficient based on the ratio of the plurality of integral values calculated by the integrating unit 3.

The plurality of integral values are calculated by integrating the color information of the first image data for each representative color. Therefore, the calculated ratio of the plurality of integral values reflects the color tone of the image represented by the first image data. The second image data is created with reference to a mixing coefficient indicating a different size according to such a ratio. As a result, the quality of the second image data can be improved.
[Example]

  Referring to FIG. 2, the digital camera 10 of this embodiment includes a focus lens 12 and an aperture mechanism 14 driven by drivers 20a and 20b, respectively. The optical image of the object scene that has passed through the focus lens 12 and the diaphragm mechanism 14 is irradiated onto the imaging surface of the imaging device 18.

  A plurality of light receiving elements (= pixels) are two-dimensionally arranged on the imaging surface, and the imaging surface is covered with a primary color Bayer array color filter 16 shown in FIG. Specifically, the color filter 16 corresponds to a filter in which R (Red) filter elements, G (Green) filter elements, and B (Blue) filter elements are arranged in a mosaic pattern. The light receiving elements arranged on the imaging surface have a one-to-one correspondence with the filter elements constituting the color filter 16, and the amount of charge generated by each light receiving element indicates the intensity of light corresponding to the R, G, or B color. reflect.

  When the power is turned on, the CPU 32 instructs the driver 20c to repeat the exposure operation and the charge readout operation in order to execute the moving image capturing process under the imaging task. In response to a vertical synchronization signal Vsync output from an SG (Signal Generator) (not shown), the driver 20c exposes the imaging surface and reads out the charges generated thereby in a raster scanning manner. From the imaging device 18, raw image data based on the read charges is periodically output. The output raw image data corresponds to image data in which each pixel has color information of any one of R, G, and B.

  The signal processing circuit 22 performs color separation on such raw image data to create RGB image data in which each pixel has all color information of R, G, and B, and adjusts white balance to the created RGB image data. Apply. The signal processing circuit 22 further converts the RGB image data having the adjusted white balance into YUV image data in which each pixel has luminance information (= Y) and color difference information (U or V), and the converted YUV image Perform edge enhancement processing on the data. The YUV image data having the emphasized outline is written into the SDRAM 26 through the memory control circuit 24.

  The LCD driver 28 repeatedly reads out the image data stored in the SDRAM 26 through the memory control circuit 24, and drives the LCD monitor 30 based on the read image data. As a result, a moving image (through image) representing the scene is displayed on the monitor screen.

  Referring to FIG. 4, an evaluation area EVA is assigned to the center of the imaging surface. The evaluation area EVA is divided into 16 in each of the horizontal direction and the vertical direction, and 256 divided areas form the evaluation area EVA.

  In addition to the processing described above, the signal processing circuit 22 integrates R data, G data, and B data forming RGB image data after white balance adjustment for each divided area, and generates Y data that forms YUV image data. Is integrated for each divided area, and the high frequency component of the Y data forming the YUV image data is integrated for each divided area. As a result, 256 R evaluation values, 256 G evaluation values, and 256 B evaluation values are generated based on the RGB image data, 256 luminance evaluation values are generated based on the Y data, and 256. The focus evaluation values are generated based on the high frequency component of the Y data.

  All the integration processes are executed each time the vertical synchronization signal Vsync is generated, and the R evaluation value, the G evaluation value, the B evaluation value, the luminance evaluation value, and the focus evaluation value are signal processing circuits in response to the vertical synchronization signal Vsync. 22 is output.

  When the shutter button 34sh provided in the key input device 34 is in a non-operating state, the CPU 32 repeatedly executes a simple AE process to calculate an appropriate EV value based on the luminance evaluation value output from the signal processing circuit 22, Based on the R evaluation value, the G evaluation value, and the B evaluation value output from the signal processing circuit 22, the simple AWB process is repeatedly executed in order to calculate an appropriate gain for white balance adjustment.

  The aperture amount and the exposure time that define the appropriate EV value calculated by the simple AE process are set in the drivers 18b and 18c, respectively. Further, the appropriate gain calculated by the simple AWB process is set in the signal processing circuit 22. As a result, the brightness and white balance of the through image are appropriately adjusted.

  When the shutter button 34 sh is half-pressed, the CPU 32 executes a strict AE process to calculate the optimum EV value based on the luminance evaluation value output from the signal processing circuit 22, and the R output from the signal processing circuit 22. A strict AWB process is executed to calculate an optimum gain for white balance adjustment based on the evaluation value, the G evaluation value, and the B evaluation value.

  As described above, the aperture amount and the exposure time that define the optimum EV value calculated by the strict AE process are set in the drivers 18b and 18c, respectively. In addition, the optimum gain calculated by the strict AWB process is set in the signal processing circuit 22. As a result, the brightness and white balance of the through image are adjusted to optimum values.

  Subsequently, the CPU 32 executes AF processing. The focus lens 12 is moved in the optical axis direction, and the focus evaluation value output from the signal processing circuit 22 is repeatedly captured in parallel with the movement processing of the focus lens 12. The focal point is searched based on the taken focus evaluation value, and the focus lens 12 is placed at the found focal point. This improves the sharpness of the live view image.

  When the shutter button 34sh is fully pressed, the CPU 32 executes a still image capturing process. One frame of image data representing the object scene at the time when the shutter button 34 sh is fully pressed is saved in a work area (not shown) provided in the SDRAM 26. When the still image capturing process is completed, the CPU 32 activates the I / F 36 for the recording process. The I / F 36 reads the image data saved in the work area through the memory control circuit 24 and records the read image data on the recording medium 38 in a file format.

  The signal processing circuit 22 is configured as shown in FIG. The color separation circuit 50 performs color separation processing on the raw image data input from the imaging device 18 to create RGB image data in which each pixel has all R, G, and B color information. Among the created RGB image data, the R data is supplied to the YUV conversion circuit 54 through the amplifier 52r constituting the white balance adjustment circuit 52, the G data is supplied to the YUV conversion circuit 54 as it is, and the B data is white balance. The signal is supplied to the YUV conversion circuit 54 through the amplifier 52b constituting the adjustment circuit 52.

  The amplifier 52r amplifies the R data according to the gain α, and the amplifier 52b amplifies the B data according to the gain β. Here, the gains α and β are parameters that define an appropriate gain calculated by the simple AWB process or an optimum gain calculated by the strict AWB process. The white balance of the RGB image data is appropriately adjusted by the simple AWB process, and is adjusted to the optimum value by the strict AWB process.

  The YUV conversion circuit 54 converts the RGB image data having the white balance adjusted in this way into YUV image data in which each pixel has luminance information (= Y) and color difference information (U or V). Of the converted YUV image data, U data and V data are output as they are, and Y data is output via the adder 58.

  The aperture signal creation circuit 56 creates aperture data (outline enhancement data) based on the RGB image data output from the white balance adjustment circuit 52. The created aperture data is given to the adder 58. The adder 58 subtracts the aperture data from the Y data in order to execute the contour enhancement process. As a result, the contour of the Y data is emphasized.

  The RGB image data output from the white balance adjustment circuit 52 is also supplied to the integrators 60r, 60b, and 60g. The integrator 60r integrates the R data forming the RGB image data for each divided area in response to the vertical synchronization signal Vsync. The integrator 60g integrates the G data forming the RGB image data for each divided area in response to the vertical synchronization signal Vsync. The integrator 60b integrates the B data forming the RGB image data for each divided area in response to the vertical synchronization signal Vsync.

  As a result, 256 integral values Ir, Ir,... Are output as R evaluation values from the integrator 60r, 256 integral values Ig, Ig,... Are output as G evaluation values from the integrator 60g, and 256 pieces. Are output from the integrator 60b as B evaluation values.

  The HPF 62 extracts the high-frequency component of the Y data output from the adder 58, and the integrator 64 integrates the extracted high-frequency component for each divided area in response to the vertical synchronization signal Vsync. The integrator 66 integrates the Y data output from the adder 58 for each divided area in response to the vertical synchronization signal Vsync. As a result, 256 integral values Ihy, Ihy,... Are output from the integrator 64 as focus evaluation values, and 256 integral values Iy, Iy,.

  The YUV conversion circuit 54 is configured as shown in FIG. 6 in connection with the conversion of RGB image data into Y data. Of the RGB image data, R data is amplified by a factor of three by a multiplier, G data is amplified by a factor of six by a multiplier 74, and B data is amplified by a multiplier 78 by a factor of “Kb_n”. The output of the multiplier 70 is amplified by “Kr_n” times by the multiplier 72, and the output of the multiplier 72 is added to the output of the multiplier 74 by the adder 76. The output of the adder 76 is added to the output of the multiplier 78 by the adder 80. Thus, the Y data is output from the adder 80.

  The aperture signal generation circuit 56 is configured as shown in FIG. Among the RGB image data, the R data is subjected to the second order differential processing by the second order differential filter 92, the G data is subjected to the second order differential processing by the second order differential filter 94, and the B data is processed by the second order differential filter 96. A secondary differentiation process is performed. When the input to the secondary differential filter 92, 94 or 96 changes as shown in FIG. 8A, the output from the secondary differential filter 92, 94 or 96 changes as shown in FIG. 8B. .

  The output of the secondary differential filter 92 is amplified “Kr_n” times by the multiplier 98, and the output of the multiplier 98 is added to the output of the secondary differential filter 94 by the adder 100. The output of the secondary differential filter 96 is amplified by “Kb_n” times by the multiplier 102, and the output of the multiplier 102 is added to the output of the adder 100 by the adder 104. Thus, the aperture signal is output from the adder 104.

  “Kr_n” and “Kb_n” correspond to mixing coefficients, and are adjusted by the coefficient adjustment task of the CPU 32 in the following manner. When the vertical synchronization signal Vsync is generated, the CPU 32 calculates the sum of 256 R evaluation values output from the integrator 60r as “Kr”, and calculates the total of 256 G evaluation values output from the integrator 60g. It is calculated as “Kg”, and the total sum of 256 B evaluation values output from the integrator 60b is calculated as “Kb”. The calculated sums Kr, Kg, and Kb are associated with R, G, and B, whereby the color tone of the object scene image is measured as the RGB ratio. Each of the sums Kr and Kb is normalized with reference to the sum Kg, thereby calculating the mixing coefficients Kr_n and Kb_n.

  The CPU 32 executes a plurality of tasks including the imaging task shown in FIG. 9 and the coefficient adjustment task shown in FIG. 10 in parallel. Control programs corresponding to these tasks are stored in the flash memory 40.

  Referring to FIG. 9, in step S1, a moving image capturing process is executed. As a result, a through image representing the scene is displayed on the LCD monitor 30. In step S3, it is determined whether or not the shutter button 34sh has been operated. If the determination result is NO, the simple AE process and the simple AWB process are repeatedly executed in step S5. As a result, the brightness and white balance of the through image are appropriately adjusted.

  When the determination result in step S3 is updated from NO to YES, the strict AE process and the strict AWB process are executed in step S7, and the AF process is executed in step S9. As a result, the brightness and white balance of the through image are strictly adjusted, and the sharpness of the through image is improved. In step S11, it is determined whether or not the shutter button 34sh has been fully pressed. In step S13, it is determined whether or not the operation of the shutter button 34sh has been released.

  If the decision result in the step S11 is YES, a still image capturing process is executed in a step S15, and a recording process is executed in a step S17. As a result, image data representing the object scene at the time when the shutter button 34 sh is fully pressed is recorded on the recording medium 38 in a file format. When the recording process is completed, the process returns to step S3. If the determination result of step S13 is YES, it will return to step S3, without performing the process of step S15 and S17.

  Referring to FIG. 10, in step S21, it is determined whether or not the vertical synchronization signal Vsync has been generated. If the determination result is updated from NO to YES, the RGB ratio is measured in step S23. Specifically, the sum of 256 R evaluation values output from the integrator 60r is calculated as “Kr”, and the sum of 256 G evaluation values output from the integrator 60g is calculated as “Kg”. The total of 256 B evaluation values output from the integrator 60b is calculated as “Kb”, and the calculated totals Kr, Kg, and Kb are associated with R, G, and B.

  In step S25, the sum Kr is divided by the sum Kg to normalize the R ratio, and in step S27, the sum Kb is divided by the sum Kg to normalize the B ratio. The mixing coefficient Kr_n is calculated by the process of step S25, and the mixing coefficient Kb_n is calculated by the process of step S27. In step S29, the calculated mixing coefficients Kr_n and Kb_n are set in the multipliers 72 and 78 of the YUV conversion circuit 54, respectively. In step S31, the calculated mixing coefficients Kr_n and Kb_n are set in the multipliers 98 and 102 of the aperture signal generation circuit 56, respectively. When the process of step S31 is completed, the process returns to step S21.

  As can be seen from the above description, the white balance adjustment circuit 52 outputs RGB image data formed by a plurality of pixels each having R, G, and B color information. The YUV conversion circuit 54 is formed by a plurality of pixels each having luminance information by mixing the color information of the RGB image data output from the white balance adjustment circuit 52 for each pixel in a manner according to the mixing coefficients Kr_n and Kb_n. YUV image data is created. The integrators 60r, 60g, and 60b integrate the color information of the RGB image data output from the white balance adjustment circuit 52 for each common color, 256 R evaluation values, 256 G evaluation values, and 256 pieces. B evaluation value is calculated. The CPU 36 obtains the sum of the calculated R evaluation value, G evaluation value, and B evaluation value as “Kr”, “Kg”, and “Kb” (S23), and based on the ratio of the sum Kr, Kg, and Kb, the mixing coefficient The magnitudes of Kr_n and Kb_n are adjusted (S25 to S27).

  The R evaluation value, the G evaluation value, and the B evaluation value are calculated by integrating the color information of the RGB image data for each common color. Therefore, the ratio of the sums Kr, Kg, and Kb reflects the color tone of the image represented by the RGB image data. The YUV image data is created with reference to the mixing coefficients Kr_n and Kb_n indicating different sizes according to such a ratio. Thereby, the quality of the YUV image data can be improved.

  In this embodiment, a still camera for recording a still image is assumed. However, the present invention can also be applied to a movie camera for recording a moving image.

DESCRIPTION OF SYMBOLS 10 ... Digital camera 18 ... Imaging device 22 ... Signal processing circuit 32 ... CPU
52 ... White balance adjustment circuit 54 ... YUV conversion circuit 56 ... Aperture signal generation circuit 60r, 60b, 60g ... Integrator

Claims (6)

First creation means for creating first image data formed by a plurality of pixels each having color information of a plurality of representative colors;
A second image data formed by a plurality of pixels each having luminance information by mixing color information of the first image data generated by the first generation means for each pixel in a manner according to a mixing coefficient. Creation means,
Integration means for integrating the color information of the first image data created by the first creation means for each representative color to calculate a plurality of integral values respectively corresponding to the plurality of representative colors; and calculated by the integration means An image data processing apparatus comprising adjustment means for adjusting the size of the mixing coefficient based on a ratio of a plurality of integral values. Third creation means for creating a plurality of edge enhancement components respectively corresponding to the plurality of representative colors based on the first image data created by the first creation means;
A fourth creation means for creating a contour enhancement data by mixing a plurality of contour enhancement components created by the third creation means in a manner according to the mixing coefficient; and refer to the contour enhancement data created by the fourth creation means The image data processing apparatus according to claim 1, further comprising contour enhancement means for executing the contour enhancement processing performed on the second image data created by the second creation means. The plurality of representative colors include red, green and blue,
The adjustment means includes first division means for dividing the integral value corresponding to the red color by the integral value corresponding to the green color, and second division for dividing the integral value corresponding to the blue color by the integral value corresponding to the green color. The image data processing apparatus according to claim 1, comprising means.   The second creating means multiplies the division result of the first division means by the color information corresponding to the red color, and the division result of the second division means is multiplied by the color information corresponding to the blue color. The image data processing apparatus according to claim 3, further comprising a second multiplying unit. A color filter in which a plurality of filter elements respectively corresponding to the plurality of representative colors are arranged in a mosaic pattern, and an imaging surface covered by the color filter, and color information of any one of the plurality of representative colors It further comprises imaging means for outputting raw image data formed by a plurality of pixels each having,
5. The image data processing apparatus according to claim 1, wherein the first creation unit creates the first image data based on raw image data output from the imaging unit.   6. The image data processing apparatus according to claim 1, wherein the second image data corresponds to image data in a YUV format.

Images