JP2004274482A - Image processing apparatus and imaging system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enhance the accuracy of white balance processing by improving the discrimination accuracy of a white object part in an image. <P>SOLUTION: A video camera 10 is provided with: an imaging element 11 for imaging an object to acquire an image; a representative hue parameter calculation section 27 for estimating the color temperature of the image from the acquired image; and a white object part discrimination section 29 for variably setting a white discrimination reference to detect the white object part in the image based on estimated color temperature. The video camera 10 uses the white discrimination reference set by the white object part discrimination section 29 to discriminate the white object part in the image, compares a color difference signal of the white object part with the reference value to obtain white balance correction values kr, kb and applies white balance processing to the image by using the white balance correction values. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a technique for adjusting the white balance of an image, and more particularly to a technique for determining a white subject in an image and performing white balance processing on the image.
[0002]
[Prior art]
When a subject is photographed using an imaging device, for example, when photographing under an incandescent lamp and when photographing under sunlight, the same subject is photographed as a different color due to a difference in color temperature of a light source. Is done. In order to reduce the influence of the color temperature of the light source appearing in such an image, a white balance process is performed on an image obtained by shooting.
[0003]
In the white balance process, a correction value is calculated such that a portion of a white object photographed in an image (hereinafter, referred to as a white object portion) is corrected to an appropriate white, and the entire image is corrected using the correction value. . Therefore, in order to improve the accuracy of the white balance process, it is necessary to accurately detect a white subject portion in an image.
[0004]
In the conventional white balance processing, in order to detect a white subject portion in an image, a parameter of hue of each pixel constituting the image, that is, a parameter (hereinafter, referred to as hue and saturation information) of color information which simultaneously represents hue and saturation information. Hue parameter). The tint parameters of the pixels constituting the white object portion change on the blackbody radiation curve according to the change in the color temperature of the light source. Therefore, in the conventional white balance processing, an area including a high color temperature portion to a low color temperature portion of a blackbody radiation curve is set as a white determination criterion, and the hue parameter of each pixel is included in the white determination criterion. Whether or not each pixel is a pixel constituting a white subject portion is determined based on whether or not the pixel is a white subject portion.
[0005]
As described above, a technique for improving detection accuracy in detecting a white subject portion using a color parameter has been proposed. For example, in the white balance processing described in Patent Document 1, the shooting conditions (EV value, shutter speed, F-number and flash ON / OFF information of the camera) and the shooting conditions (such as subject distance) when shooting the subject are taken. By estimating the color temperature of the light source at the time of photographing and variably setting an area serving as a white determination reference according to the estimated color temperature, the detection accuracy of the white subject portion is improved.
[0006]
[Patent Document 1]
Japanese Patent Application Laid-Open No. H11-262029 (page 12, FIG. 4)
[0007]
[Problems to be solved by the invention]
The color parameter and the blackbody radiation curve used in the above-described conventional white balance processing have the following relationship depending on the color temperature of the light source. When the color temperature of the light source is high, the tint parameter of the white object portion is located in the high color temperature portion of the blackbody radiation curve. At this time, the tint parameter of the subject portion having a low chroma reddish color is located around the low temperature portion of the blackbody radiation curve, and is included in the white determination reference region. When the color temperature of the light source is low, the hue parameter of the white subject portion is located in the low color temperature portion of the blackbody radiation curve. In this case, the tint parameter of the object portion having a low-saturation bluish color is located around the high-temperature portion of the blackbody radiation curve, and is included in the white determination reference region. Therefore, when a region including a high color temperature portion to a low color temperature portion of a black body radiation curve is used as a white determination reference, a portion representing a chromatic subject may be erroneously determined to be a white subject portion.
[0008]
The technique disclosed in Patent Document 1 estimates a color temperature from an imaging condition or a subject condition, and determines only a region corresponding to the estimated color temperature in a blackbody radiation curve as a white determination reference to determine a white subject portion. By performing the determination, the erroneous determination as described above is prevented.
[0009]
However, in this conventional technique, when the flash is off, it is assumed that the image is taken under daylight, and the color temperature is estimated to be high. Is determined, and the color temperature is estimated according to this correspondence. Therefore, it is inevitable that the estimation accuracy is reduced by relying on such an empirical rule. If the estimation accuracy of the color temperature is low, it is not possible to accurately determine the white subject portion, and the accuracy of the white balance processing is reduced. Further, in this conventional technique, since the EV value of the camera, the shutter speed, the ON / OFF of the flash, the subject distance, and the like must be detected in order to estimate the color temperature, the configuration of the apparatus is complicated. Further, according to this conventional technique, an image pickup apparatus without an EV value or shutter speed adjustment function or an image pickup apparatus without a flash or a distance measurement function cannot estimate a color temperature, so that the white discrimination reference is variable. Cannot be set to.
[0010]
The present invention has been made to solve such a problem, and provides a technique for improving the accuracy of white balance processing by improving the accuracy of determining a white subject portion. In particular, the present invention erroneously determines a chromatic subject portion whose hue parameter is located around a black body radiation curve as a white subject portion by variably setting a white determination reference regardless of an imaging condition or a subject condition. It is an object of the present invention to provide a technique for preventing such a situation and improving the accuracy of determining a white subject.
[0011]
[Means for Solving the Problems]
An image processing apparatus according to the present invention includes: an image obtaining unit that obtains an image; and a white subject portion in the image, based on a color temperature of the image estimated from the image obtained by the image obtaining unit. Setting means for variably setting a white discrimination criterion serving as a discrimination criterion, and white balance processing means for performing white balance processing on the image acquired by the image acquisition means using the white discrimination criterion set by the setting means And
[0012]
With this configuration, the white determination criterion is set based on the color temperature estimated from the acquired image itself, so that the white determination criterion can be appropriately set irrespective of imaging conditions and subject conditions, and white balance can be set. Processing accuracy can be improved.
[0013]
Here, the image acquisition unit may be an imaging unit that acquires an image by photographing a subject or a unit that receives an image generated by the imaging unit.
[0014]
Further, in the image processing apparatus according to the present invention, the setting unit uses a representative hue parameter representing the entire hue of the image acquired by the image acquisition unit with one hue parameter as the parameter representing the color temperature.
[0015]
With this configuration, since the entire image is represented by one representative color parameter, the color temperature of the image can be estimated using the representative color parameters.
[0016]
Further, in the image processing apparatus according to the aspect of the invention, the color temperature is estimated from an average of color parameters of pixels included in the image.
[0017]
With this configuration, the color temperature can be appropriately estimated using the information of the image itself.
[0018]
In the image processing device according to the present invention, the color temperature is estimated from an average of color parameters of a plurality of pixels extracted from the image.
[0019]
With this configuration, since the color temperature of an image is estimated using only some of the pixels that form the image, the processing load on the device can be reduced, and the processing time can be reduced.
[0020]
Further, in the image processing apparatus of the present invention, the color temperature is a coordinate of a plurality of pixels excluding a pixel having a color parameter separated by a predetermined distance from a blackbody radiation curve on the coordinate on a coordinate representing the color parameter of the image. Estimated from the average of the color parameters.
[0021]
With this configuration, the color temperature is estimated excluding chromatic colors having high saturation, so that the accuracy of color temperature estimation can be improved.
[0022]
An image processing apparatus according to another aspect of the present invention includes an image acquisition unit that acquires an image, an arithmetic unit that determines a representative tint parameter that is a tint parameter representing the image acquired by the image acquisition unit, and the arithmetic unit. Setting means for variably setting a white determination criterion, which is a reference for determining a white subject portion in the image based on the color tone, based on the representative hue parameter obtained by using the white determination criterion set by the setting means. And white balance processing means for performing white balance processing on the image acquired by the image acquisition means.
[0023]
With this configuration, the white determination criterion is set using the parameters of the acquired image itself, so that the white determination criterion can be appropriately set irrespective of the imaging conditions and the subject conditions, and the accuracy of the white balance processing can be improved. Can be improved.
[0024]
Further, an imaging system according to the present invention includes any one of the above-described image processing apparatuses and an imaging apparatus that captures an image of a subject and generates the image. Then, the image acquisition unit acquires an image generated by the imaging device, and the image processing device performs a white balance process on the image generated by the imaging device.
[0025]
With this configuration, it is possible to improve the accuracy of white balance processing on images captured under various color temperature conditions.
[0026]
The invention is not limited to the device embodiments as described above. Another embodiment of the present invention is an image processing method, a program for causing a computer to execute the image processing method, and a recording medium storing the program.
[0027]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0028]
FIG. 1 is a block diagram illustrating a configuration of a video camera which is an imaging system provided with the image processing apparatus according to the present embodiment. The video camera 10 includes an image sensor 11, a CDS circuit 12, an AGC circuit 13, and an A / D converter 14. The image sensor 11 includes three color filters of R (red), G (green) and B (blue) for each pixel, and converts light passing through these filters into an electric signal and outputs the electric signal. The image sensor 11 converts the optical image of the subject into an electric signal, thereby generating a video signal representing the image of the subject, thereby obtaining the image. The video signal output from the image sensor 11 is subjected to noise removal processing by the CDS circuit 12 and gain control processing by the AGC circuit 13, and is converted into a digital signal by the A / D converter 14.
[0029]
The video camera 10 further includes a color separation circuit 21, an R value multiplier 22, a B value multiplier 23, an R color difference calculator 24, and a B color difference calculator 25. The video signal converted into a digital signal by the A / D converter 14 is color-separated into three colors of R, G, and B by a color separation circuit 21, and the R value, the G value, Each signal of the B value is output. These R value, G value, and B value are color parameters representing the color of each pixel of the image by an additive method.
[0030]
Among the color parameter signals, the R value signal and the B value signal are subjected to gain adjustment by being multiplied by white balance correction values kr and kb in an R value multiplier 22 and a B value multiplier 23, respectively. As a result, the R value and B value signals are signals whose white balance has been adjusted by eliminating the influence of the color temperature of the light source. The R color difference calculator 24 and the B color difference calculator 25 subtract the G value from each of the gain-adjusted R value (kr · R value) and the gain-adjusted B value (kb · B value), thereby obtaining a white value. A signal of an R color difference value (kr · R value−G value) and a B color difference value (kb · B value−G value) whose balance has been adjusted is generated and output.
[0031]
The video camera 10 further includes an adder 26 for estimating a color temperature, a representative hue parameter calculating unit 27, an adder 28 for detecting a white subject portion, a white subject detecting unit 29, an R color difference adder 30, and a B color difference adder 31. , R color difference comparator 32 and B color difference comparator 33. The R value, G value, and B value signals output from the color separation circuit 21 are taken into an adder 26 for color temperature estimation, and are subjected to processing in a representative hue parameter calculation unit 27. The processing in the adder 26 and the representative hue parameter calculator 27 will be described later in detail.
[0032]
Each signal of the R value, the G value, and the B value output from the color separation circuit 21 is input to an adder 28 for detecting a white subject portion. The adder 28 divides one image into 48 blocks of 8 × 6, and adds and averages each of the R value, the G value, and the B value for each block. The adder 28 thus calculates the average value of the R block, the average value of the G block, and the average value of the B block.
[0033]
The white subject detection unit 29 calculates the average value of the R block / the average value of the G block and the average value of the B block / the average value of the G block using the average value of the R block, the average value of the G block, and the average value of the B block. The average value of the R block / the average value of the G block and the average value of the B block / the average value of the G block are parameters representing the color of each block, and correspond to the color parameters of the present invention. The white subject detection unit 29 determines whether each block is a white subject portion based on whether the hue parameter of each block satisfies the white determination criterion.
[0034]
FIG. 2 is a diagram illustrating a process of determining a white italic part by the white subject detection unit 29. FIG. 2 shows the coordinates with the horizontal axis and the vertical axis respectively of the R value / G value and the B value / G value (hereinafter referred to as R / G and B / G), which are the tint parameters. The white subject detection unit 29 variably sets the white determination criterion based on the color temperature estimated by the representative hue parameter calculation unit 27, as described later. Here, this white determination criterion setting processing is performed. The white determination process after the white determination reference is set will be described, and the white determination reference setting process will be described later.
[0035]
In the present embodiment, the white determination reference is the white determination area WA on the coordinates in FIG. In these coordinates, the tint parameter of the reddish color is plotted at a position where the value of R / G is large and the value of B / G is small. Further, the hue parameter of the bluish color is plotted at a position where the value of R / G is small and the value of B / G is large. Further, the tint parameter of the green color is plotted at a position where both the values of R / G and B / G are small. Then, the tint parameter of the white subject portion in the image is plotted on the black body radiation curve 41 at a position corresponding to the color temperature of the light source.
[0036]
In the example of FIG. 2, the white determination area WA defined by the determination formulas of R / G> X0, B / G> X1, R / G + B / G> X2, and R / G + B / G <X3 is the white determination reference. Have been. When (R / G, B / G) = (R block average / G block average, B block average / G block average), when a certain block satisfies the above discriminant, (Value / G block average value, B block average value / G block average value) are included in the white determination area WA on the coordinates in FIG. 2, the block is determined to be a white subject portion. The white determination area WA is not limited to a trapezoidal area as shown in FIG. 2 and may be a triangle, a rectangle, or an area surrounded by a curve.
[0037]
The white subject detection unit 29 outputs a white discrimination signal indicating whether or not the block is a white subject portion by the above-described white discrimination processing. However, in the present embodiment, a signal indicating that the block is not a white subject portion is output as a white determination signal. Based on the white determination signal, white balance processing is performed in a subsequent stage based on the video signal of the white subject portion.
[0038]
Returning to FIG. 1, the R color difference adder 30 and the B color difference adder 31 respectively provide the R color difference value and the B color difference value of all pixels of one image output from the R color difference calculator 24 and the B color difference calculator 25. Capture the signal of The R color difference adder 30 and the B color difference adder 31 receive a white discrimination signal from the white subject detection unit 27. As described above, the white determination signal is a signal indicating a block determined not to be a white subject portion. An R color difference adder 30 and a B color difference adder 31 add up the R color difference signal and the B color difference signal of only the white subject portion, excluding the color difference signals of the portions other than the white subject portion, according to the white discrimination signal, and average them. I do. The R color difference adder 30 and the B color difference adder 31 output a signal of the average of the R color difference value and the B color difference value of the white subject portion calculated as described above (referred to as an R white portion color difference average value and a B white portion color difference average value). Is output.
[0039]
The R color difference comparator 32 compares the average value of the R white partial color differences output from the R color difference adder 30 with an R reference value stored in advance. By this comparison, an R white balance correction value kr for adjusting the gain of the R value signal is obtained and output to the R multiplier 22. Similarly, the B color difference comparator 33 compares the B white partial color difference average value with the B reference value, and outputs the B white balance correction value kb to the B value multiplier 23. Here, the R reference value and the B reference value are target values of the R color difference signal and the B color difference signal of the white subject portion after the white balance processing. The R white balance correction value kr and the B white balance correction value kb are, for example, R reference value / R white partial color difference average value and B reference value / B white partial color difference average value.
[0040]
As described above, the video camera 10 according to the present embodiment detects a white subject portion in an image, obtains a correction value for correcting the white subject portion to ideal white, and uses the correction value to generate a video signal. Apply white balance processing. This function of the video camera 10 implements the white balance processing means of the present invention.
[0041]
Next, the white determination reference setting processing by the white subject detection unit 29 will be described in detail. As described above, the white subject detection unit 29 sets the white determination criterion based on the color temperature estimated by the representative hue parameter calculation unit 27. Therefore, hereinafter, first, the color temperature estimation processing by the adder 26 and the representative hue parameter calculation unit 27 will be described, and then the white determination reference setting processing by the white subject detection unit 29 will be described.
[0042]
As shown in FIG. 1, an adder 26 for estimating a color temperature takes in the R, G, and B value signals output from the color separation circuit 21. Then, the adder 26 adds and averages each of the R value, the G value, and the B value for one image, and outputs each data of the R average value, the G average value, and the B average value to the representative hue parameter calculation unit 27. I do. The representative color parameter calculation unit 27 calculates an R average value / G average value and a B average value / G average value using the R average value, the G average value, and the B average value. The R average value / G average value and the B average value / G average value are color parameters that represent the color of the entire image with one parameter, that is, color parameters representing the image. Is equivalent to The representative hue parameter calculation unit 27 corresponds to an operation unit for obtaining a representative hue parameter of an image according to the present invention.
[0043]
In the present embodiment, the representative hue parameter calculation unit 27 further obtains a color temperature corresponding to the representative hue parameter, and outputs information on the color temperature to the white subject detection unit 29.
[0044]
FIG. 3 and FIG. 4 are diagrams for explaining the process of calculating the color temperature corresponding to the representative hue parameter in the representative hue parameter calculation unit 27. In these figures, as in FIG. 2, the horizontal axis indicates R / G and the vertical axis indicates B / G. Hereinafter, two examples of obtaining the color temperature corresponding to the representative hue parameter will be described. Here, the representative hue parameters (R average value / G average value, B average value / G average value) represent the entire image with one hue parameter, and therefore the hue parameters shown in FIGS. Is generally located near the blackbody radiation curve 41.
[0045]
FIG. 3 is a diagram for explaining an example of the processing of the representative hue parameter calculation unit 27. The representative color parameter calculation unit 27 sets the color temperature corresponding to the intersection point when a perpendicular line is drawn from the point representing the representative color parameter to the blackbody radiation curve as the color temperature corresponding to the representative color parameter. For example, as shown in FIG. 3, when the representative hue parameter A is obtained, the representative hue parameter calculation unit 27 determines the perpendicular and the black body when the perpendicular is drawn from the point A to the black body radiation curve 41. The color temperature corresponding to the intersection XA with the radiation curve is defined as the color temperature corresponding to the representative hue parameter A.
[0046]
FIG. 4 is a diagram for explaining another example of the process of the representative hue parameter calculation unit. The tint parameter calculation unit 27 stores a plurality of small areas 43 obtained by dividing an area 42 including a high color temperature part to a low color temperature part of the blackbody radiation curve 41 along the blackbody radiation curve. The representative hue parameter calculation unit 27 further stores, as a candidate color temperature, a color temperature corresponding to a point 44 located substantially at the center of the black body radiation curve 41 included in each small area 43. Then, the hue parameter calculator 27 sets the candidate color temperature of the small area 43 to which the point representing the representative hue parameter belongs as the color temperature corresponding to the representative hue parameter. For example, as shown in FIG. 4, when the representative hue parameter B is obtained, the representative hue parameter calculation unit 27 sets the candidate color temperature of the small area 43 to which the point B belongs to the color temperature corresponding to the representative hue parameter B. I do.
[0047]
In each of the examples of FIG. 3 and FIG. 4, when the representative color parameter calculation unit 27 obtains the color temperature corresponding to the representative color parameter, it outputs information on the color temperature to the white subject detection unit 29.
[0048]
Next, the white determination reference setting process in the white subject detection unit 29 will be described.
[0049]
FIG. 5 is a diagram for explaining the white determination reference setting process. The coordinates shown in FIG. 5 are the same as the coordinates shown in FIG. The tint parameter of the white subject portion in the image is plotted at some position on the blackbody radiation curve 41 according to the color temperature of the image. That is, when the color temperature is low, the hue parameter of the white object portion is located in a portion where the R / G is large and B / G is small (low color temperature portion) on the blackbody radiation curve 41, and when the color temperature is high, It is located in a portion where the R / G is small and the B / G is large (high color temperature portion) on the blackbody radiation curve 41.
[0050]
On the other hand, when the color temperature of the image is high, the tint parameter of the light reddish subject portion is located at or around the low color temperature portion of the blackbody radiation curve 41, and when the color temperature of the image is low, the blackbody The hue parameter of the pale blue subject portion is located in the high color temperature portion of the radiation curve 41 and its periphery. Therefore, assuming that a region including the high color temperature portion to the low color temperature portion of the black body radiation curve 41 is a white determination region as in the related art, a light red or light blue subject portion is detected as a white subject portion. Sometimes.
[0051]
Therefore, in the present embodiment, the white subject detection unit 29 is configured to use the representative hue parameter calculation unit so that the chromatic subject located on or around the black body radiation curve 41 is not erroneously determined to be a white subject. In accordance with the color temperature of the image estimated at 27, a peripheral area of the color temperature is set as a white determination reference. At this time, the white subject detection unit 29 functions as a setting unit that variably sets the white determination range based on the representative hue parameter of the present invention.
[0052]
Specifically, the white subject detection unit 29 acquires the information of the color temperature from the representative hue parameter calculation unit 27, and includes a point on the black body radiation curve 41 corresponding to the color temperature substantially at the center. A white determination area WA having a predetermined size is set. FIG. 5 shows an example in which the white determination areas WA1 to WA3 are set in accordance with the color temperature information CT1 to CT3 obtained from the representative hue parameter calculator 27.
[0053]
Here, as shown in FIG. 5, the white subject detection unit 29 stores an area 45 including a high color temperature portion to a low color temperature portion of the black body radiation curve 41 along the black body radiation curve 41. I have. Then, the white determination area WA is set by cutting out a partial area centered on a point on the black body radiation curve 41 corresponding to the color temperature information CT from the area 45. Note that a candidate area may be stored in advance for each predetermined color temperature range, and the white determination area WA may be set by selecting one of the candidate areas based on the color temperature information CT.
[0054]
Next, the operation of the video camera 10 configured as described above when actually performing white balance processing on an image will be described.
[0055]
First, the image sensor 11 generates a video signal for 30 images per second, for example, by capturing a moving image of a subject. The video signal continuously generated by the image sensor 11 is input to the color separation circuit 21 in order for each image via the CDS circuit 12, the AGC circuit 13, and the A / D converter 14. The color separation circuit 21 separates the video signal into R, G, and B colors, and outputs signals of R, G, and B values.
[0056]
Of these signals, each of the R value and B value signals is subjected to gain adjustment using the white balance correction value in the R multiplier 22 and the B multiplier 23, respectively. The correction value is updated once for every five images. Therefore, in the video camera 10 of the present embodiment, each time the signal for five images is output from the color separation circuit 21, the white balance is calculated using the R value, G value, and B value signals for one image. Processing for obtaining a correction value is executed.
[0057]
The adder 26 acquires the R value, G value, and B value signals for one image each time the signal for five images is output from the color separation circuit 21, and outputs the R value, G value, and B value. Are averaged by adding for all pixels for each pixel to obtain an R average value, a G average value, and a B average value. The representative hue parameter calculator 27 estimates the color temperature of the image using the R average value, the G average value, and the B average value. Then, based on the color temperature estimated by the representative hue parameter calculation unit 27, the white subject detection unit 29 sets an area around the color temperature as a white determination area.
[0058]
On the other hand, the adder 28 obtains each signal of the R value, the G value, and the B value at the same timing as when the adder 26 obtains the signal output from the color separation circuit 21, and obtains an R block average value, The G block average value and the B block average value are obtained. The white subject detection unit 29 calculates the R block average value / G block average value and B block average value / G block average value, which are the hue parameters of each block, from the R block average value, G block average value, and B block average value of each block. Find the value. The white subject detection unit 29 determines whether or not the hue parameter of each block is included in the previously set white determination area, and outputs a white determination signal.
[0059]
The R color difference adder 30 and the B color difference adder 31 obtain the signals of the R color difference value and the B color difference value output from the R color difference calculator 24 and the B color difference calculator 25, respectively, and output the signals from the white subject detection unit 29. In accordance with the white determination signal thus obtained, only the signals of the color difference values of the blocks whose hue parameters are included in the white determination area are added and averaged, and the signals of the R and B color difference average values are output. The R color difference comparator 32 and the B color difference signal comparator 33 compare the R color difference average value and the B color difference average value with the R reference value and the B reference value, respectively, to thereby obtain the R white balance correction value kr and the B white balance correction value kb. And outputs them to the R value multiplier 22 and the B value multiplier 23, respectively.
[0060]
Upon acquiring the R white balance correction value kr and the B white balance correction value kb, respectively, the R value multiplier 22 and the B value multiplier 23 update the previously set respective white balance correction values. Then, the R-value multiplier 22 and the B-value multiplier 23 perform the operations until the next white balance correction value is updated, that is, the next five images of the images previously taken into the adders 26 and 28. On the other hand, the gain of the R value and the B value is adjusted using the updated white balance correction value. The R color difference calculator 24 and the B color difference calculator 25 generate and output signals of the R color difference value and the B color difference value by subtracting the G value from the gain adjusted R value and B value.
[0061]
In this manner, by adjusting the gain of the R value and B value signals in the video signal and then generating the R color difference value and B color difference value signals, it is possible to generate a color difference signal with an adjusted white balance. it can.
[0062]
In the above-described embodiment, the white balance processing is performed on the moving image captured by the image sensor 11, but the white balance processing may be performed on one still image. In this case, a white determination criterion is set by the above processing using the target still image, a white subject portion of the still image is detected using the white determination criterion, and a white balance correction value is obtained. Then, the still image is subjected to white balance processing using the white balance correction value.
[0063]
Further, in the above embodiment, the white subject detection unit 29 calculates the hue parameter for each block obtained by dividing one image into 48 blocks, but the hue parameter R value / G value and B value / G value are obtained in pixel units. Of course, the white subject portion determination processing may be performed.
[0064]
Further, in the above embodiment, the representative color parameter is calculated using the average value of the R, G, and B values of all pixels in one image. However, the present invention is not limited to this. If it is a parameter, it may be calculated by another method. For example, the average value of the R value, the G value, and the B value is obtained using pixels extracted by thinning out all the pixels constituting the image at appropriate intervals, and the representative hue parameter is calculated using such an average value. It may be calculated. As described above, if only some of the pixels extracted from all the pixels are used, the processing load for obtaining the representative hue parameters is reduced, and the processing time is also reduced.
[0065]
Further, the representative hue parameter may be calculated using an average value of the R value, the G value, and the B value excluding pixels with extremely high saturation. On the coordinates of the hue parameter, the saturation increases as the distance from the blackbody radiation curve increases, so by excluding pixels having a hue parameter that is a predetermined distance away from the blackbody radiation curve, excluding pixels with high saturation An average can be determined. In this case, the image processing apparatus stores an adopted area including the blackbody radiation curve. Then, the adder 26 for estimating the color temperature adds only the pixels included in the adopted area to calculate an R average value, a G average value, and a B average value. At this time, the adoption area may be an area that is sufficiently wider than the area 45 illustrated in FIG. As described above, if the representative hue parameters are obtained by excluding the pixels having extremely high saturation, the representative hue parameters approach the blackbody radiation curve, and the white determination reference can be set more accurately.
[0066]
In the above embodiment, the R value / G value and the B value / G value are used as the hue parameters, but the present invention is not limited to this. The hue parameter may be another parameter as long as the parameter simultaneously represents hue and saturation information of the color information. For example, the R value / Y value and the B value / Y value may be used as the hue parameters. The R value / Y value and the B value / Y value are typical parameters that simultaneously represent hue and saturation information of the color information. Here, the Y value is a parameter representing luminance information of the color information, and is a value that can be obtained by weighting and adding the R value, the G value, and the B value. Similarly, for the parameters representing the color of the image, the R average value / Y average value and the B average value / Y average value may be used as the representative color parameters.
[0067]
Further, in the above embodiment, the R value-G value and the B value-G value are treated as the R color difference value and the B color difference value, respectively. However, other values such as the R value-Y value and the B value-Y value are used. May be used as the color difference value.
[0068]
In the above-described embodiment, the color temperature corresponding to the representative hue parameter is determined by the representative hue parameter calculation unit 27, and the white subject detection unit 29 determines the white determination as a white determination criterion according to the color temperature information. Although the area WA is set, the setting of the white determination reference is not limited to this method. Other processing for variably setting the white determination criterion based on the color temperature may be performed. For example, the white subject detection unit 29 may directly set the white determination criterion from the representative hue parameter. At this time, the representative hue parameter is used as information indicating the color temperature.
[0069]
In this case, in the image processing apparatus, for example, an appropriate white determination criterion is prepared in advance in association with each of various values of the representative hue parameter, and stored in a storage unit (not shown). The representative hue parameter calculation unit 27 obtains a representative hue parameter in the same manner as described above, and outputs this to the white subject detection unit 29. Then, the white subject detection unit 29 selects the white determination area WA corresponding to the representative hue parameter. Note that, also in this example, the representative parameter calculating unit 27 corresponds to the calculating unit of the present invention, and the white subject detecting unit 29 corresponds to the setting unit of the present invention.
[0070]
As described above, even in an example in which the white determination reference is variably set without obtaining the color temperature corresponding to the representative color parameter, the color temperature of the image is estimated using the acquired color parameters of the image itself, and then the white determination reference is determined. Is set, the white determination criterion can be appropriately set without using information on the imaging condition and the subject condition, and the accuracy of the white balance process can be improved.
[0071]
Further, in the above embodiment, the color temperature estimation process and the white subject portion detection process are performed on the same image, but both processes may be performed on different images. Further, for example, the intervals of both processes may be different from each other, such that the color temperature estimation process is performed for every 15 images and the white subject portion detection process is performed for every 5 images. Of course, the color temperature estimation processing and the white subject part detection processing may be applied to all the images.
[0072]
Further, some functions of the video camera 10 may be realized by software. In this case, a computer including a CPU, a RAM, a ROM, and the like can realize the function by executing the program. Furthermore, the video camera 10 includes the image sensor 11 and obtains a target image of the white balance process by photographing the subject. However, the video camera 10 may obtain the target image from outside.
[0073]
According to the video camera 10 described above, the white subject portion is determined after the white determination standard for determining the white subject portion is variably set so as to prevent erroneous determination. Can be determined more accurately. As a result, a more accurate white balance correction value can be obtained, and the accuracy of the white balance processing is improved.
[0074]
In the video camera 10 described above, the representative hue parameter calculator 27 estimates the color temperature of the image from the information of the image itself, that is, the color parameters of the image. Since a certain white determination area is set, a white balance correction value can be suitably obtained without depending on imaging conditions or subject conditions. Then, by performing white balance processing on the image using the white balance correction value, the accuracy of the white balance processing can be improved.
[0075]
In the video camera 10 described above, the representative color parameter calculation unit 27 uses the representative color parameter representing the color of the entire image with one color parameter as the parameter representing the color temperature of the image. I can do it.
[0076]
Further, the use of the color parameters of all the pixels constituting the image improves the accuracy of color temperature estimation. On the other hand, by estimating the color temperature using only the pixels extracted by thinning out some of the pixels constituting the image, the processing load for color temperature estimation is reduced, and the processing time is reduced. it can. In addition, on the coordinate table representing the color parameters of the image, the color temperature is estimated by excluding the color parameters separated from the blackbody radiation curve by a predetermined distance, that is, the color parameters having extremely high saturation, thereby obtaining the color temperature. The estimation accuracy is further improved.
[0077]
【The invention's effect】
As described above, according to the present invention, the white balance correction value can be preferably obtained by variably setting the white determination criterion based on the color temperature estimated from the image itself. An image processing apparatus having an excellent effect of improving image quality can be provided.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of an imaging system according to an embodiment of the present invention.
FIG. 2 is a diagram for explaining white determination processing according to the embodiment of the present invention;
FIG. 3 is a diagram for explaining a color temperature estimation process according to the embodiment of the present invention;
FIG. 4 is a diagram for explaining color temperature estimation processing according to the embodiment of the present invention;
FIG. 5 is a view for explaining white determination reference setting processing according to the embodiment of the present invention;
[Explanation of symbols]
10 Imaging system
11 Image sensor
22 R value multiplier
23 B value multiplier
27 Representative color parameter calculator
29 White subject detector
41 Blackbody radiation curve
WA white determination area

Claims (10)

Image acquisition means for acquiring an image,
Setting means for variably setting a white determination criterion serving as a criterion for determining a white subject portion in the image based on a hue based on a color temperature of the image estimated from the image obtained by the image obtaining means,
Using a white determination criterion set by the setting unit, a white balance processing unit that performs white balance processing on the image acquired by the image acquisition unit,
An image processing apparatus comprising: 2. The image according to claim 1, wherein the setting unit uses a representative hue parameter representing a whole hue of the image acquired by the image acquisition unit with one hue parameter as a parameter representing the color temperature. 3. Processing equipment. The image processing apparatus according to claim 1, wherein the color temperature is estimated from an average of color parameters of pixels included in the image. The image processing apparatus according to claim 3, wherein the color temperature is estimated from an average of color parameters of a plurality of pixels extracted from the image. The color temperature is estimated on a coordinate representing a color parameter of the image from an average of color parameters of a plurality of pixels excluding a pixel having a color parameter separated by a predetermined distance from a blackbody radiation curve on the coordinate. The image processing apparatus according to claim 3, wherein: Image acquisition means for acquiring an image,
Calculating means for calculating a representative color parameter which is a color parameter representative of the image acquired by the image acquisition means;
Setting means for variably setting a white determination criterion serving as a reference for determining a white subject portion in the image based on the color tone based on the representative color parameter obtained by the arithmetic means;
Using a white determination criterion set by the setting unit, a white balance processing unit that performs white balance processing on the image acquired by the image acquisition unit,
An image processing apparatus comprising: An image processing apparatus according to claim 1,
An imaging device for imaging the subject to generate the image,
The image acquisition unit acquires an image generated by the imaging device,
The image processing system, wherein the image processing device performs a white balance process on an image generated by the image capturing device. An image acquisition step of acquiring an image;
A setting step of variably setting a white determination criterion serving as a criterion for determining a white subject portion in the image based on a tint based on a color temperature of the image estimated from the image obtained in the image obtaining step; ,
Using a white determination criterion set in the setting step, a white balance processing step of performing white balance processing on the image obtained in the image obtaining step,
An image processing method comprising: An image acquisition step of acquiring an image;
A calculating step of obtaining a representative hue parameter which is a hue parameter representing the image obtained in the image obtaining step;
A setting step of variably setting a white determination criterion that is a reference for determining a white subject portion in the image based on the color tone based on the representative color parameter obtained in the calculation step;
Using a white determination criterion set in the setting step, a white balance processing step of performing white balance processing on the image obtained in the image obtaining step,
An image processing method comprising: A program for causing a computer to execute the image processing method according to claim 8.

Images