JP2006229990A - Video apparatus for automatically adjusting white balance and white balance adjustment method therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video apparatus exactly and automatically adjusting white balance. <P>SOLUTION: The video apparatus has: a color space conversion part 100 for converting input video data sensed by an image sensor into data of a color space having chromaticity components; a gray area extraction part 200 for extracting a gray area of the input video data based on gray areas of respective light source detected using many light sources and a predetermined color checker; a detection part 300 which uses the average value and the median of the input video data in a color gamut to extract initial light source information and which uses the fact that luminance difference exists between gray components and the chromaticity components to adjust the initial light source information so that the chromaticity components flowing in the extracted gray area are excluded; and a gain control part 600 for calculating gain applied to scaling of the input video data on the basis of the adjusted light source information and applying the calculated gain to the input video data to adjust the white balance. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention minimizes the chromaticity component flowing into the gray area of the video device extracted in the chromatic color space and detects an accurate light source, and then uses the detected light source information to perform white balance. The present invention relates to a video apparatus that automatically adjusts white balance that can be automatically adjusted, and a white balance adjustment method thereof.

  White balance adjustment is to detect the white color that has the largest change in object color due to the light source in the input video data, determine the color temperature based on the detected RGB composition ratio, and then use the detected white color as a reference. This refers to a color correction function that shifts the overall color sense by correcting R (Red) and B (Blue) depending on the temperature to achieve color balance.

  In general, when the same subject is imaged using a video device such as a digital still camera or a digital video camera, the image is captured depending on the conditions of various light sources such as an indoor white lamp, a fluorescent lamp, or sunlight. The color looks different. The human eye has color constancy that adapts to each light source and feels white as white. On the other hand, the video apparatus reflects RGB components included in light sources having different color temperatures. For example, when the color temperature of the light source is high, white including blue is reproduced, and when the color temperature is low, white including red is reproduced.

  Therefore, when the color temperature varies depending on the light source, white balance adjustment is necessary to make white appear white. For example, it is necessary to adjust to increase the gain of R and decrease the gain of B for white including blue, and to adjust the gain of B for white including red. It is necessary to adjust so as to increase and decrease the gain of R.

  In order to accurately adjust the white balance in this way, the reference white should be accurately detected. For this reason, the white balance can be adjusted based on the gray world assumption that “if the entire color component included in the captured screen is integrated, it is zero, that is, close to an achromatic color”. In particular, white balance adjustment under the assumption of a gray world can effectively detect white when the input image includes various colors and the color distribution is wide. When adjusting the white balance under the assumption of the gray world, accurate light source information can be obtained even when dominant colors (colors) exist in the input video or when chromaticity components flow into the extracted gray area. Therefore, it is necessary to perform white balance accurately.

  The present invention has been devised to solve the above-described problems of the prior art, and its purpose is to minimize the chromaticity component flowing into the gray area extracted in the chromaticity color space, and to input video. An image device for automatically adjusting white balance, which can accurately adjust white balance using light source information detected by removing dominant colors existing in the image, and a method for adjusting the white balance are provided.

  In order to achieve the above object, a video apparatus for automatically adjusting white balance according to the present invention includes a color space conversion unit that converts input video data sensed by an image sensor into data of a color space having a chromaticity component. A gray area extracting unit for extracting a gray area of the input video data based on a gray area of each light source detected using a plurality of light sources and a predetermined color checker; and an average value and a center value in the color gamut of the input video data To extract the initial light source information and eliminate the chromaticity component that flows into the extracted gray area using the difference in luminance between the gray component and the chromatic component. A detector that adjusts the initial light source information, and a gain applied to the scaling of the input video data based on the adjusted light source information. A gain control unit that adjusts white balance by applying the calculated gain to the input video data is included.

  Here, it is preferable to further include a signal processing unit that performs signal processing so that the input video with the adjusted white balance is displayed.

  At this time, the color space consists of a color space consisting of a luminance (Y) axis, an R / (R + G + B) axis, and a B / (R + G + B) axis, or a luminance (Y) axis, an R / G axis, and a B / G axis. A color space is preferred.

  Preferably, the gray area extraction unit detects a white point of each light source using an average value of gray patch data acquired using a plurality of light sources, and then determines a predetermined range centered on the white point of each light source. An area including all the gray areas of each light source is extracted as a gray area of the input video data.

  Preferably, the detection unit calculates a weighted average value that reflects the average value within the color gamut using the average value and center value within the color gamut of the input video, and then extracts initial light source information based on the weighted average value. After dividing the gray area into a first area having a low luminance and a second area having a high luminance using the light source information extraction unit and the presence of a luminance difference between the gray component and the chromaticity component, the first area And an adjustment unit that adjusts the initial light source information based on the average value of the first region and the average value of the second region.

  Preferably, the light source information extraction unit vertically maps the weighted average value to the central axis connecting the white points of each light source, and then maps the weighted average value for each light source to the white point of each light source. Based on the motion that can minimize the error, the weighted average value on the mapped central axis is corrected and extracted as initial light source information.

The weighted average value is preferably calculated by the following formula (1).
Here, r _weighted is a weighted average R / (R + G + B) value, b _weighted is a weighted average B / (R + G + B) value, α and β are arbitrary constants, and r _ave is within the color gamut. R / (R + G + B) value of the average value, b _ave indicates B / (R + G + B) value of the average value in the color gamut, r _gm is the R / (R + G + B) value of the center value of the color gamut, and b _gm is the color gamut B / (R + G + B) value of the center value of.

  Preferably, the adjustment unit sets the gray region as the first region and the second region based on the threshold luminance value determined using the average value of the maximum value and the minimum value in the gray region and the average value of the gray region. After dividing into regions, the initial light source information is adjusted using the average value of the first region and the average value of the second region.

At this time, the threshold luminance value is preferably calculated by the following equation (2).
Here, Y _threshold is a threshold luminance value, Y _region is an average value of the average value of the first region and the average value of the second region, Y _ave is an average value of the gray region, and α and β are respectively Indicates an arbitrary constant.

  Preferably, the adjustment unit detects, as adjusted light source information, an average value having the shortest distance from the initial light source information among the average value of the first region and the average value of the second region.

At this time, the adjusted light source information is preferably calculated by the following equation (3).
Here, r _wht and b _wht are the R / (R + G + B) value and B / (R + G + B) value of the adjusted light source information, respectively, and r _down and b _down are R / of the average value of the first region, respectively. (R + G + B) value, B / (R + G + B) value, r _up and b _up indicate the R / (R + G + B) value and B / (R + G + B) value of the color gamut average value of the second region, respectively, and Dist _down is The distance between the initial light source information and the average value of the first area is indicated, and Dist _up indicates the distance between the initial light source information and the average value of the second area.

The gain is preferably calculated by the following equation (4).
Here, R _gain and B _gain are a gain value applied to red (R) and a gain value applied to blue (B), respectively, and f _scaling is a scaling factor for scaling an input image. _Yes , r _wht and b _wht are the adjusted R / (R + G + B) value and B / (R + G + B) value of the light source information, respectively, and g _wht is the G value of the light source information.

  Preferably, the gain control unit multiplies the red (R) of the input image by the reciprocal of the calculated scale factor and the red (R) gain value used for the calculation, and the reciprocal of the scale factor, By multiplying the blue (B) of the input image by the blue (B) gain value used for the calculation, the white balance of the input image is adjusted.

  In order to achieve the above object, a method for automatically adjusting the white balance of a video apparatus according to the present invention converts input video data sensed by an image sensor into data in a color space having a chromaticity component. Extracting a gray area of the input video data based on a gray area of each light source detected using a plurality of light sources and a predetermined color checker, a light source using an average value and a center value in the color gamut of the input video data Extract the information and adjust the initial light source information so that the chromaticity component flowing into the extracted gray area is excluded by using the brightness difference between the gray component and the chromaticity component. Extracting the final light source information and calculating the gain applied to the scaling of the input video data based on the final light source information, and using the calculated gain as the input video data And use, including the step of adjusting the white balance.

  Here, it is preferable that the method further includes a step of performing signal processing so that the input image with the adjusted white balance is displayed.

  At this time, the color space is a color space composed of a luminance (Y) axis, an R / (R + G + B) axis, a B / (R + G + B) axis, or a luminance (Y) axis, an R / G axis, and a B / G axis. Is preferably a color space.

  Preferably, the step of extracting the gray region is performed by detecting a white point of each light source using an average value of gray patch data acquired using a plurality of light sources, and centering on the white point of each light source. Detecting a predetermined range as a gray region of each light source and extracting a region including all the gray regions of each light source as a gray region of the input video data.

  Preferably, the step of extracting the final light source information includes calculating a weighted average value that reflects the average value within the color gamut using the average value within the color gamut and the center value of the input video, and then initializing based on the weighted average value. After the light source information is extracted and the gray region is divided into the low-brightness first region and the high-brightness second region using the difference in luminance between the gray component and the chromaticity component, And adjusting the initial light source information based on the average value of the first region and the average value of the second region.

  Preferably, the step of extracting the initial light source information includes mapping the weighted average value vertically to the central axis connecting the white points of each light source, and then mapping the weighted average value for each light source to the white point of each light source. Sometimes, the weighted average value on the central axis mapped based on the motion that can minimize the error is corrected and extracted as initial light source information.

At this time, the weighted average value is preferably calculated by the following equation (1).
Here, r _weighted is a weighted average R / (R + G + B) value, b _weighted is a weighted average B / (R + G + B) value, α and β are arbitrary constants, and r _ave is within the color gamut. R / (R + G + B) value of the average value, b _ave indicates B / (R + G + B) value of the average value in the color gamut, r _gm is the R / (R + G + B) value of the center value of the color gamut, and b _gm is the color gamut B / (R + G + B) value of the center value of.

  Preferably, the step of adjusting the initial light source information may be performed by adjusting the gray region based on the threshold luminance value determined using the average value of the maximum value and the minimum value in the gray region and the average value of the gray region. After dividing into 1 area | region and 2nd area | region, initial light source information is adjusted using the average value of a 1st area | region, and the average value of a 2nd area | region.

At this time, the threshold luminance value is preferably calculated by the following equation (2).
Here, Y _threshold is a threshold luminance value, Y _region is an average value of the average value of the first region and the average value of the second region, Y _ave is an average value of the gray region, and α and β are respectively Indicates an arbitrary constant.

  Preferably, the step of adjusting the initial light source information detects the average value having the shortest distance from the initial light source information among the average value of the first region and the average value of the second region as the adjusted light source information. .

At this time, the adjusted light source information is preferably calculated by the following equation (3).
Here, r _wht and b _wht are the R / (R + G + B) value and B / (R + G + B) value of the adjusted light source information, respectively, and r _down and b _down are R / of the average value of the first region, respectively. (R + G + B) value and B / (R + G + B) value, where r _up and b _up are the average R / (R + G + B) value and B / (R + G + B) value of the second region, respectively, and Dist _down is the initial light source The distance between the information and the average value of the first area is indicated, and Dist _up indicates the distance between the initial light source information and the average value of the second area.

At this time, the gain is preferably calculated by the following (4).
Here, R _gain and B _gain are a gain value applied to red (R) and a gain value applied to blue (B), respectively, and f _scaling is a scaling factor for scaling an input image. _Yes , r _wht and b _wht are the adjusted R / (R + G + B) value and B / (R + G + B) value of the light source information, respectively, and g _wht is the G value of the light source information.

  Preferably, the step of adjusting the white balance includes multiplying the red (R) of the input image by the reciprocal of the scale factor and the red (R) gain value used for the calculation, and the reciprocal of the scale factor; By multiplying the blue (B) of the input image by the blue (B) gain value used for the calculation, the white balance of the input image is adjusted.

  As described above, according to the present invention, the accuracy of white balancing can be improved by detecting the light source information in which the influence of the chromaticity component flowing into the gray region is minimized and adjusting the white balance. And by calculating the color gamut average value based on the color gamut center value at the time of gray area detection used to detect the light source information, the light source information is inaccurate due to the presence of dominant colors in the color gamut. Can be prevented.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
FIG. 1 is a block diagram of a white balance adjusting apparatus according to an embodiment of a video apparatus of the present invention.

  As shown in FIG. 1, the white balance adjustment device of the video apparatus according to the present embodiment includes a color space conversion unit 100, a gray region detection unit 200, a detection unit 300, and a gain control unit 600. Here, the detection unit 300 includes a light source information extraction unit 400 and an adjustment unit 500.

  The color space conversion unit 100 converts input video data into chromaticity color space data. Of the actual color components (color components), the color (color) that largely reflects the influence of the light source change is gray color (gray), so the input video data is converted to a chromaticity color space where it is easy to detect the gray area. . That is, in order to adjust the white balance of the CCD output RGB value (CCD data), which is the output value of the image sensor, the CCD output RGB value is converted into data in a chromaticity color space that allows easy gray detection.

  In this case, the chromaticity color space is, for example, a color space composed of the luminance axis, R / (R + G + B) axis, B / (R + G + B) axis, a color space composed of the luminance axis, R / G axis, B / G axis, etc. It is. This is because when the color cast is generated by the light source, the ratio between the components of the RGB component of the gray color changes, so the ratio between the components of the RGB component of the CCD output RGB value is used as the coordinate axis. This is because the input video data is converted to the chromaticity color space.

  The gray area extraction unit 200 extracts a gray area range based on gray patch data acquired using a large number of light sources. In this case, the multiple light sources are, for example, the standard light source D65, the standard light source D50, the standard light source CWF, the standard light source A, and the like. Also, the accuracy for gray region extraction can be increased by increasing the number of light sources.

  The gray area extraction unit 200 acquires gray patch data (gray patch data) of a Macbeth Color Checker for a large number of light sources, and determines an average value of the acquired gray patch data as white of each light source. . Then, using the distribution characteristics for each light source, a predetermined area centering on the white point of each light source is determined as the gray area of each light source, and the area including all the determined gray areas of each light source is defined as the gray area of the input video Extract.

  The light source information extraction unit 400 extracts initial light source information using the center value and the average value in the color gamut of the input video. Calculates the center value and the average value in the color gamut, and if the difference between the average value and the center value is large, calculates the weighted average value obtained by adding the weight value to the calculated average value, and extracts the initial light source information Used to do. The light source information extraction unit 400 performs vertical mapping of the calculated weighted average value on a light locus that is a central axis connecting the white points of the respective light sources determined by the gray region extraction unit 200, and performs vertical mapping. The initial light source information is finally extracted on the light trajectory.

  The adjustment unit 500 adjusts the initial light source information to extract the final light source information so that the chromaticity component that has flowed into the gray region extracted by the gray region extraction unit 200 is excluded. At this time, the extraction of the final light source information is performed using the fact that there is a luminance difference between the gray component and the chromaticity component. That is, the adjustment unit 500 divides the gray area based on the threshold luminance value, and then determines a final white point using each average value in the divided gray area. Then, the adjustment unit 500 extracts an average value in a region corresponding to a region that does not include a chromaticity component in the divided gray region as adjusted light source information.

  Further, the adjusting unit 500 can finely divide the gray area in order to extract accurate light source information. In addition, when an image having a single background color is input, the subdivided gray area, which is the divided gray area, is divided again to detect the highlight area, and then the average value of the highlight area is determined when adjusting the light source information. Accurate light source information can be extracted as reflected.

  The gain control unit 600 calculates the gain for adjusting the input video data using the final light source information determined by the adjustment unit 500, and then adjusts the white balance by applying the calculated gain to each input video data. To do. Note that the video apparatus according to the present embodiment further includes a signal processing unit that performs signal processing so that an input video whose white balance has been adjusted is displayed after the white balance adjustment device (after the gain control unit 600). It is preferable to provide. In this case, the signal processing unit performs signal processing for generating image display data that can be displayed on a display unit such as a CRT or a liquid crystal display panel.

  2A and 2B are diagrams for explaining the operation of the color space conversion unit 100 shown in FIG. FIG. 2A illustrates the color components R, G, and R in the chromaticity color space of the R / G and B / G planes where the luminance is constant (the horizontal axis indicates R / G and the vertical axis indicates B / G). It is the figure which showed distribution of the video data in case B component is increased uniformly. FIG. 2B shows the chromaticity color of the R / (R + G + B) and B / (R + G + B) planes with constant luminance (the horizontal axis indicates R / (R + G + B) and the vertical axis indicates B / (R + G + B)). It is the figure which showed distribution of the video data in the case of making the R, G, B component of a color component increase uniformly in space.

  In general, the video device does not use the entire CCD (Charge Coupled Device) data, and after sampling the CCD data at a predetermined interval, the patching data based on the average value of the CCD data in the window defined for the sampled data ( patching data). Since the average value of the gray area is seen as the white point, the video apparatus first extracts the gray area in order to extract the light source information, that is, the white point, using such patching data.

  When the color cast is generated by the light source, the ratio between the RGB components of the gray color changes, so that the chromaticities of the R / G and B / G planes as shown in FIG. 2A A color space can be used.

  On the other hand, when the R, G, and B components are uniformly increased for each color component, the distribution of the video data in the chromaticity color space shown in FIG. 2B is the distribution of the video data in the chromaticity color space shown in FIG. 2A. Since the distribution is relatively uniform compared to the distribution, the chromaticity color space shown in FIG. 2B can improve the accuracy of gray region extraction. Therefore, in the following, as shown in FIG. 2B, information on the light source is extracted using a chromaticity color space of R / (R + G + B) and B / (R + G + B) planes with constant luminance.

  3A to 3C are diagrams for explaining the operation of the gray region extraction unit 200 of FIG. FIG. 3A is a diagram illustrating color gamuts of four general light sources, and FIG. 3B is a diagram illustrating gray regions of four light sources. FIG. 3C is a diagram for explaining that a chromatic component flows into the gray area of the detected video apparatus. Here, Gr indicates a gray area of the video apparatus.

Referring to FIG. 3A, the color gamut of each light source may be detected based on video data for a color checker acquired using each light source. At this time, the four light sources (D65, D50, CWF, A) are indicated by “D ₆₅ ”, “D ₅₀ ”, “CWF”, and “A”. Furthermore, a color gamut of the input video can be extracted by adding other light sources other than the four light sources.

  Referring to FIG. 3B, as shown in FIG. 3A, a gray region for each light source can be detected using gray patch data of a color checker in a color gamut detected by four light sources. The average value of the gray patch data of the color checker detected by each light source corresponds to the white point. Therefore, after detecting the white point of each light source, a gray patch distribution range is formed around the white point of each light source. Then, the gray region of the input image can be modeled through the four straight lines so as to include all the gray distribution ranges for the four light sources.

  Referring to FIG. 3C, the gray area of the input video data set by using the color gamut information of the color checker and the four light sources as shown in FIGS. 3A and 3B has a chromaticity component due to the change of the light source. Can be introduced. That is, as shown in FIG. 3B, the gray area of the input video data is extracted by different light sources, so that a gray area wider than the gray area distribution range of the light source actually used for the input video is extracted. A chromaticity component such as (cyan) can flow into the gray region. As shown in FIG. 3C, yellow (Y) and cyan (C) chromaticity components are blue (B), red (R), magenta (M: magenta), and green (G) chromaticity components. A relatively large amount of gas flowed into the gray region (Gr). Accordingly, it is necessary to adjust the extracted light source information in order to exclude the chromaticity component from flowing into the gray region (Gr).

FIG. 4 is a block diagram of the detection unit 300 of FIG.
Referring to FIG. 4, the detection unit 300 includes a light source information extraction unit 400 and an adjustment unit 500. The light source information extraction unit 400 includes a gamut center calculation unit 410, a gamut average calculation unit. 420 and a weighted average value extraction unit 430.

  The color gamut center calculation unit 410 models the light locus that is the central axis connecting the white points of the respective light sources calculated by the gray region extraction unit 200, and then the extension line of the modeled central axis and R / (R + G + B). A color gamut center value is calculated based on the two outermost points of the color gamut detected using the respective intersections of the axis and the B / (R + G + B) axis.

The color gamut average calculation unit 420 calculates an average value within the color gamut in the color gamut of the input video.
The weighted average value extraction unit 430 calculates a weighted average value using the color gamut center value calculated by the color gamut center calculation unit 410 and the in-gamut average value calculated by the color gamut average calculation unit 420. Here, the weighted average value is an average value calculated by assigning a weight value to the in-gamut average value based on the difference between the calculated in-gamut average value and the gamut center value.

  The adjustment unit 500 detects the gray region using different light sources in the initial light source information extracted by the light source information extraction unit 400, thereby eliminating the chromaticity component flowing into the gray region and obtaining the final light source information ( Final light source information) is determined.

  5A to 5C are diagrams for explaining the operation of the light source information extraction unit 400 of FIG. FIG. 5A is a diagram for explaining the operation of the color gamut center calculation unit 410, and FIG. 5B is a diagram for explaining the operation of the weighted average value extraction unit 430. FIG. 5C is a diagram for explaining an operation of acquiring initial light source information determined using the weighted average value calculated by the weighted average value extraction unit 430.

  Referring to FIG. 5A, the color gamut center calculation unit 410 first extracts two outermost points of the color gamut of the input image in order to obtain the color gamut center. Then, the center of the color gamut is obtained by calculating the average value of the two outermost contour points.

  Specifically, the color gamut center calculation unit 410 detects a central axis connecting the white points of the respective light sources calculated by the gray region extraction unit 200 in the chromaticity color space, and then, an extension line of the central axis, R / ( The X0 point and the Y0 point that are the intersections of the R + G + B) axis and the B / (R + G + B) axis are detected. The points that are the minimum distance from the X0 point to the central axis and the points that are the minimum distance from the Y0 point to the central axis are the two outermost points of the color gamut of the input video. That is, the outermost point is the point X1 which is the minimum distance from the point X0 to the central axis and the point Y1 which is the minimum distance from the point Y0 to the central axis. Therefore, the color gamut center value is gm, which is an intermediate value between the X1 point and the Y1 point.

  Referring to FIG. 5B, when the average value calculated by the color gamut average calculation unit 420 and the center value of the color gamut calculated by the color gamut center calculation unit 410 are different from each other, the dominant color in the color gamut (Color) may violate the gray world assumption that the average color is gray. Therefore, since the calculated color gamut average value may not indicate accurate white information, the weighted average value extraction unit 430 determines a weighted average value that reflects the color gamut center value at the time of white information calculation. The weighted average value is obtained by the following equation (1).

Here, r _weighted is a weighted average R / (R + G + B) value, b _weighted is a weighted average B / (R + G + B) value, and α and β are arbitrary constants. In addition, r _ave is the R / (R + G + B) value of the average value within the color gamut, b _ave is the B / (R + G + B) value of the average value within the color gamut, r _gm is the R / (R + G + B) value of the color gamut center value, b _gm represents the B / (R + G + B) value of the color gamut center value.

  As shown in the equation (1), the weighted average value extraction unit 430 calculates the R / (R + G + B) value and the B / (R + G + B) value of the weighted average value by the color gamut average calculation unit 420. The color gamut center value calculated by the color gamut center calculation unit 410 is reflected in the in-gamut average value. That is, the weighted average value extraction unit 430 determines that the in-gamut average value and the gamut center value are moved in the direction of the gamut center value when the in-gamut average value is different from the gamut center value. By assigning a weight value to the gamut and preventing the average value in the color gamut from being biased in a certain direction due to the dominant color of the color gamut, accurate initial light source information can be extracted.

  For example, when the dominant color in the color gamut is yellow as shown in FIG. 5B, the average value in the color gamut calculated by the color gamut average calculating unit 420 has a value closer to yellow instead of gray. Therefore, the weighted average value is extracted by reflecting the color gamut center value (gm) in the calculated color gamut average value and moving the color gamut average value in the direction of the color gamut center value. Therefore, if there is a dominant color in the gamut rather than uniformly different colors in the gamut, incorrect initial light source information will be extracted for the dominant color. Can be prevented.

  Referring to FIG. 5C, point a represents the weighted average of the color gamut extracted by the weighted average value extraction unit 430, and point b represents light that is the central axis obtained by connecting the white points of the respective light sources. A point mapped on the trajectory is indicated, and a point c indicates a light source adjusted (corrected) based on the mapped point b.

  The extraction of the initial light source information is as follows. First, the weighted average value (a) is vertically mapped onto the light locus which is the central axis connecting the white points of the respective light sources. That is, the intersection point of the straight line perpendicular to the light locus including the weighted average value (a) and the light locus is determined as the mapping point. Then, when the weighted average value for each light source is mapped with the white point of the original illuminant on the light locus, the weighted average value mapped to the light locus is corrected so that an error can be minimized. The initial light source information shown in FIG.

Here, r _initial and b _initial are initial light source information values, r _weighted is a weighted average R / (R + G + B) value, and b _weighted is a weighted average B / (R + G + B) value. L, m, n and l ′, m ′, n ′ are arbitrary constants. At this time, in order to apply the equation (5) to correct the weighted average value mapped on the light locus, the distance between the white points of the light sources located in the vicinity of the weighted average value mapped on the light locus. It can be applied when is below a predetermined value.

6A and 6B are diagrams for explaining the operation of the adjustment unit 500. FIG.
FIG. 6A is a diagram for explaining that the adjustment unit 500 adjusts the initial light source information according to the luminance. FIG. 6B is a diagram for describing an operation in which the adjustment unit 500 extracts final light source information using an average value of regions divided based on luminance.

  Referring to FIG. 6A, the horizontal axis (C) represents a chromaticity component, and the vertical axis (Y) represents a luminance component. The adjustment unit 500 divides the gray region based on the luminance component to extract light source information from which the chromaticity component that has flowed into the gray region is removed, and then uses the color gamut average value for each divided region. The initial light source information extracted by the light source information extraction unit 400 is adjusted.

  At this time, when the initial light source information is adjusted based on the luminance component, as the chromaticity value of the video data increases, the closer to the color gamut boundary, as shown in FIG. 6A. On the other hand, the larger the gray value of the video data, the closer to the luminance axis (Y). Accordingly, since a difference in luminance value occurs between such a gray component and a chromaticity component, an area including the chromaticity component and an area not including the chromaticity component are defined based on the luminance. The initial light source information extracted is adjusted using the gray region into which the chromaticity component has been introduced.

  For example, as shown in FIG. 6A, Color2 has a larger chromaticity value than Color1, and is closer to the color gamut boundary. That is, Color1 has a larger gray value than Color2, and is positioned relatively near the luminance axis. At this time, a luminance difference is generated between Color1 and Color2, and the generated luminance difference is l2-l1.

  Referring to FIG. 6B, as described above with reference to FIG. 6A, a difference in luminance value occurs between the gray component and the chromaticity component. The area is divided into a small first area and a second area having a large luminance value. Here, the horizontal axis is R / (R + G + B), and the vertical axis is the luminance Y. Although FIG. 6B shows that the gray area is divided based on the luminance with respect to the R / (R + G + B) value, the R / of the light source information adjusted by dividing the gray area based on the luminance with respect to the B / (R + G + B) value. The (R + G + B) value and the B / (R + G + B) value can also be obtained.

  At this time, a threshold luminance value serving as a reference for dividing the first area and the second area is obtained by the following equation (2).

Here, Y _threshold is a threshold luminance value used as a reference for gray region division, Y _region is an average value of respective average values in the divided gray region, and Y _ave is an average value of the gray region. Α and β are arbitrary constants.

As shown in equation (2), the luminance value that is the reference for gray region division is the average value (Y _region ) of the average value of the maximum luminance value and the minimum luminance value in the divided gray region, and the color of the entire gray region. It is obtained by the area average value (Y _ave ).

Based on the calculated threshold luminance value Y _threshold , it can be divided into a first region having a low luminance value and a second region having a high luminance value. The light source information is adjusted is determined using a white point wp_ _down a gamut average value of the first region, the average value of the white point wp_ _{Stay up-a} gamut average value of the second region. That is, the light source information adjusted by the adjustment unit 500 is obtained by the following equation (3).

Here, r _wht and b _wht are the R / (R + G + B) value and B / (R + G + B) value of the white point value, which are adjusted light source information, respectively. R _down and b _down are the R / (R + G + B) value and B / (R + G + B) value of the average value of the first region, respectively. R _up and b _up are the R / (R + G + B) value of the average value of the second region, respectively. (R + G + B) value, B / (R + G + B) value. At this time, Dist _up and Dist _down are respectively the points detected as the initial light source information, that is, the distance between the point c in FIG. 5C and the average value of the second region, and the average value of the first region. Show. Further, the distance between the initial light source information value and the average value of the second region, and the distance between the average value of the first region can be expressed by the following equation (6), respectively.

Here, Dist _up and Dist _down indicate the distance between the initial light source information value and the average value of the second region, and the distance between the average value of the first region, and r _initial and b _initial are the light source information extraction unit. This is the initial light source information value extracted at 400 (see the above-described equation (5)). R _down and b _down are the R / (R + G + B) value and B / (R + G + B) value of the average value of the first region, respectively. R _up and b _up are the R / (R + G + B) value of the average value of the second region, respectively. (R + G + B) value, B / (R + G + B) value.

The adjusted light source information is an average value in which the distance between the average value of the first area and the average value of the second area shown in Expression (6) is the shortest distance between the initial light source information values.
The adjustment unit 500 uses the calculated luminance threshold value to divide the luminance value into a second region having a high luminance value and a first region having a low luminance value. After the gray area is divided into the areas that do not flow in, the white point of the area including the initial light source is determined as adjusted light source information. Therefore, by determining the white point in the gray area where no chromaticity component is introduced as final light source information, the light source information excluding the chromaticity component is extracted from the initial light source information determined by the chromaticity component introduced into the gray area. can do.

On the other hand, when the input video is a video having a single background color, since the highlight area is included in the second area, which is the high brightness area, the adjustment unit 500 reflects the brightness value of the highlight area to reflect the light source information. So that accurate light source information can be extracted.
In addition, a “divided gray area” that does not include a chromaticity component, that is, a “divided gray area” in which the initial light source information and the white point of the divided area have a minimum value is a high luminance area. In this case, the adjustment unit 500 can determine that the “divided gray area” includes a highlight area.

  Therefore, the adjustment unit 500 may again divide the second area, which is a high-luminance area among the gray areas divided based on the threshold luminance value, into a region including the highlight region and a region not including the highlight region. Good. Here, the area including the highlight area can be an area corresponding to the upper 20% of the luminance of the second area.

  At this time, the adjustment unit 500 may first determine whether or not the highlight area is generated by the input video itself, and the white point of the highlight area may be reflected in the light source information to be adjusted. . Whether or not the highlight area is generated by the input video itself is, for example, whether or not the average luminance value of the highlight area is smaller than the luminance value obtained by multiplying (multiplying) the average luminance value of the entire gray area by 3.5. Judge with.

  When the average luminance value of the highlight region is smaller than the luminance value obtained by multiplying the average luminance value of the entire gray region by 3.5, the adjustment unit 500 sets the average value of the highlight region as the adjusted light source information. On the other hand, when the average luminance value of the highlight area is larger than the luminance value obtained by multiplying the average luminance value of the entire gray area by 3.5, the average value of the second area is set as the adjusted light source information.

  FIG. 7 is a flowchart for explaining a white balance adjustment method of the video apparatus according to the embodiment of the present invention.

  Referring to FIG. 7, first, input video data is converted into color space data having a chromaticity component (S901). Here, as a color space, in order to extract light source information and adjust white balance, a color space that divides video data into a chromatic component and a non-chromatic component is used. Can be used. Further, in the color space of the conversion destination, when color cast is generated by the light source, the property that the RGB component ratio of the gray color is different is used.

  As such a color space, a plane composed of the R / G axis and the B / G axis with constant luminance, or a plane composed of the R / (R + G + B) axis and the B / (R + G + B) axis with constant luminance is used. it can. Hereinafter, an operation for adjusting white balance using light source information extracted by converting input video data into a plane composed of the R / (R + G + B) axis and the B / (R + G + B) axis having a constant luminance will be described.

  Here, the video data is obtained by sampling the data sensed by the image sensor at regular intervals, and the patching data based on the average value of the in-window data determined for the sampled data Can be used.

  Subsequent to S901, the gray area of the video apparatus is extracted using a number of light sources and a color checker in the converted color space (S903). Here, as a large number of light sources, D65, D50, CWF, and A light sources that are generally used as light sources in an imaging apparatus can be used. As the color checker, a Macbeth color checker can be used.

  Further, in this gray area extraction, after extracting the gray area of each light source based on the video data acquired from the Macbeth color checker using each light source, the gray area of the video device including all the extracted gray areas is extracted. . Specifically, after extracting the gray region of each light source based on the average value of the gray patch data of the Macbeth color checker, four straight lines are formed so as to include all of the extracted gray regions of each light source, and the gray region of the video apparatus Can be extracted. At this time, the gray area of the video apparatus is increased by using a non-linear method using a look-up table or the like instead of increasing the number of light sources and using four straight lines set to include the gray area of each light source. The accuracy of extraction can also be increased.

  Subsequent to S903, initial light source information is extracted using the color gamut center and color gamut average of the input video (S905). To avoid violating the gray world assumption that “the average of the input video is gray” when there is a dominant color in the input video, Calculate to reflect the center value of the color gamut. If there are dominant colors in the input video, the difference between the average value of the color gamut of the input video and the central value of the color gamut of the input video is large, so the calculated average value in the color gamut and the central value of the color gamut A weighted average value is calculated by assigning a weight value to.

  Then, the calculated weighted average value is vertically mapped to the central axis connecting the white points of the light sources in the gray region extracted in step S903. That is, the weighted average value is mapped to a point where a straight line including the weighted average value among straight lines perpendicular to the central axis meets the central axis. Then, when mapping the weighted average value for each light source to the white point of the original light source (original illuminant) on the light locus, the weighted average value mapped to the light locus is corrected to correct the initial light source information. Extract.

  Next, the extracted initial light source information is adjusted using the gray region into which the chromaticity component has been introduced (S907). Here, since the gray area of the input video is extracted using the gray areas of a large number of light sources, the area into which the chromaticity component has flowed in is different from the gray area of the actually used light source. May be used to extract light source information. Therefore, it is necessary to adjust the extracted initial light source information using the gray region into which the chromaticity component has been introduced.

  In order to adjust the initial light source information, it is used that there is a luminance difference between the gray component and the chromaticity component. As described above with reference to FIG. 6A, when the luminances of the gray component and the chromaticity component are compared on the plane composed of the chromaticity axis (C) and the luminance axis (Y), the chromaticity and the luminance are proportional. In the region, the gray component has a luminance value lower than that of the chromaticity component, and in the region where chromaticity and luminance are inversely proportional, the gray component has a luminance value larger than that of the chromaticity component.

  Therefore, since there is a luminance difference between the gray component and the chromaticity component, after dividing the gray region based on the threshold luminance value, the position close to the initial light source information in the average value of the divided region The average value existing in (shortest distance) is determined as the adjusted light source information. Using the threshold luminance value as a reference, after dividing the gray area including the chromaticity component and the gray area not including the chromaticity component, the average value of the divided area adjacent to the initial light source information is set as the light source information. Then, the average value in the gray area not including the chromaticity component is extracted as the adjusted light source information. Accordingly, it is possible to extract light source information from which chromaticity components flowing into the gray region are excluded.

  On the other hand, when an input image having a single background color is input, the high brightness area is divided again, instead of dividing the gray area into two areas by one threshold brightness value, and corresponds to the highlight area. The initial light source information can be adjusted so that the average luminance value is reflected.

  Next, the gain applied to the input image is calculated using the adjusted light source information (S909). In order to eliminate the color temperature change due to the light source, the RGB data of each input video is scaled in order to eliminate the color temperature change due to the light source using the extracted light source information. The gain used for the scale of the RGB data of the input video is obtained by the following equation (4).

Here, R _gain and B _gain are a gain value applied to red (R) and a gain value applied to blue (B), respectively. F _scaling is a scaling factor for scaling the input image, and r _wht and b _wht are the R / (R + G + B) value of the white point value and B / (R + G + B), which are adjusted light source information, respectively. ) Value, and g _wht is the G value of the light source information.

Next, the image data is adjusted by applying the calculated gain (S911). The white balance can be adjusted by multiplying (multiplying) R _gain and B _gain calculated using Expression (4) by R and B, respectively. That is, the white balance of each input image can be adjusted by R ′ = R × R _gain / f _scaling , G ′ = G, and B ′ = B × B _gain / f _scaling . Note that the video device according to the present embodiment performs a signal processing stage so that the input video with the adjusted white balance is displayed after the video data is adjusted by applying the calculated gain (S911). Further, it is preferable to execute. In this case, in this signal processing stage, signal processing for generating image display data that can be displayed on a display unit such as a CRT or a liquid crystal display panel is performed.

  8A and 8B are diagrams showing histograms for patches. FIG. 8A is a diagram showing a histogram for a patch to which a conventional white balance adjustment method is applied, and FIG. 8B is a diagram showing a histogram for a patch to which a white balance adjustment method according to the present invention is applied.

Referring to FIGS. 8A and 8B, the white balance adjustment is not performed accurately in the histogram distribution for the gray patch of the resultant image for the Macbeth color checker to which the conventional white balance adjustment method is applied. That is, as shown in FIG. 8A, the distribution of the histogram between RGB components is shifted. On the other hand, as shown in FIG. 8B, the image to which the white balance adjustment method according to the present invention is applied is balanced between RGB components. This difference, for example, for example, is remarkable when the D _50.

  In the above, preferred embodiments of the present invention have been shown and described. However, the present invention is not limited to the specific examples described above, and the technical scope of the present invention does not depart from the gist of the present invention claimed in the claims. Anyone having ordinary knowledge in the field can implement various modifications, and such modifications are within the scope of the appended claims.

1 is a block diagram of a white balance adjustment device according to an embodiment of a video device of the present invention. It is a figure for demonstrating operation | movement of the color space conversion part shown in FIG. It is a figure for demonstrating operation | movement of the color space conversion part shown in FIG. It is a figure for demonstrating operation | movement of the gray area extraction part shown in FIG. It is a figure for demonstrating operation | movement of the gray area extraction part shown in FIG. It is a figure for demonstrating operation | movement of the gray area extraction part shown in FIG. It is a block diagram of the detection part shown in FIG. It is a figure for demonstrating operation | movement of the light source information extraction part shown in FIG. It is a figure for demonstrating operation | movement of the light source information extraction part shown in FIG. It is a figure for demonstrating operation | movement of the light source information extraction part shown in FIG. It is a figure for demonstrating operation | movement of the adjustment part shown in FIG. It is a figure for demonstrating operation | movement of the adjustment part shown in FIG. 5 is a flowchart for explaining a white balance adjustment method of the video apparatus according to the embodiment of the video apparatus of the present invention. It is the figure which showed the histogram with respect to the gray patch after adjusting white balance with the conventional method. It is the figure which showed the histogram with respect to the gray patch after adjusting white balance with the method which concerns on this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Color space conversion part 200 Gray area extraction part 300 Detection part 400 Light source information extraction part 410 Color gamut center calculation part 420 Color gamut average calculation part 430 Weighted average value extraction part 500 Adjustment part 600 Gain control part

Claims (26)

A color space conversion unit that converts input video data sensed by an image sensor into data of a color space having a chromaticity component;
A gray area extracting unit that extracts a gray area of the input video data based on a gray area of each light source detected using a plurality of light sources and a predetermined color checker;
The initial light source information is extracted using the average value and the center value in the color gamut of the input video data, and flows into the extracted gray region using the fact that a luminance difference exists between the gray component and the chromaticity component. A detection unit that adjusts the initial light source information so as to eliminate chromaticity components;
A gain control unit that calculates a gain to be applied to scaling of the input video data based on the adjusted light source information, and adjusts a white balance by applying the calculated gain to the input video data;
A video device for automatically adjusting white balance, characterized by comprising:   The video apparatus for automatically adjusting white balance according to claim 1, further comprising: a signal processing unit that performs signal processing so that the input video with the adjusted white balance is displayed. The color space is
A color space consisting of the luminance (Y) axis, R / (R + G + B) axis, B / (R + G + B) axis, or a color space consisting of the luminance (Y) axis, R / G axis, and B / G axis An image apparatus for automatically adjusting white balance according to claim 1 or 2. The gray area extraction unit includes:
After detecting the white point of each light source using the average value of the gray patch data acquired using the multiple light sources, a predetermined range centering on the white point of each light source is detected as the gray region of each light source. 4. The video apparatus for automatically adjusting white balance according to claim 1, wherein a region including all gray regions of each light source is extracted as a gray region of the input video data. 5. . The detector is
A light source information extraction unit that extracts the initial light source information based on the weighted average value after calculating a weighted average value that reflects the average value in the color gamut using the average value and center value in the color gamut of the input video;
The gray region is divided into a low-brightness first region and a high-brightness second region by using a luminance difference between the gray component and the chromaticity component, and then the average of the first region An adjustment unit that adjusts the initial light source information based on a value and an average value of the second region;
5. The video apparatus for automatically adjusting white balance according to claim 1, comprising: The light source information extraction unit
After the weighted average value is vertically mapped to the central axis connecting the white points of each light source, and when the weighted average value for each light source is mapped to the white point of each light source, the mapping is performed based on the movement that can minimize the error. 6. The video apparatus for automatically adjusting white balance according to claim 5, wherein the weighted average value on the central axis is corrected and extracted as the initial light source information. 6. The video apparatus for automatically adjusting white balance according to claim 5, wherein the weighted average value is calculated by the following equation (1).
Here, r _weighted is the R / (R + G + B) value of the weighted average value, b _weighted is the B / (R + G + B) value of the weighted average value, α and β are arbitrary constants, and r _ave is R / (R + G + B) value of the average value in the color gamut, b _ave indicates the B / (R + G + B) value of the average value in the color gamut, r _gm is an R / (R + G + B) value of the center value of the color gamut, b _gm Indicates the B / (R + G + B) value of the center value of the color gamut. The adjustment unit is
The gray region is divided into the first region and the second region based on the average value of the maximum value and the minimum value in the gray region and the threshold luminance value determined using the average value of the gray region. 6. The video apparatus for automatically adjusting white balance according to claim 5, wherein the initial light source information is adjusted using an average value of the first area and an average value of the second area. The video apparatus for automatically adjusting white balance according to claim 8, wherein the threshold luminance value is calculated by the following equation (2).
Here, Y _threshold is the threshold luminance value, Y _region is the average value of the average value of the first region and the average value of the second region, Y _ave is the average value of the gray region, and α and β represents an arbitrary constant. The adjustment unit is
The average value having the shortest distance from the initial light source information among the average value of the first region and the average value of the second region is detected as adjusted light source information. A video device that automatically adjusts white balance. The video apparatus for automatically adjusting white balance according to claim 10, wherein the adjusted light source information is calculated by the following equation (3).
Here, r _wht and b _wht are the R / (R + G + B) value and B / (R + G + B) value of the adjusted light source information, respectively, and r _down and b _down are the average values of the first region, respectively. R / (R + G + B) value, B / (R + G + B) value, and r _up and b _up indicate the average R / (R + G + B) value and B / (R + G + B) value of the second region, respectively, and Dist _down Indicates the distance between the initial light source information and the average value of the first area, and Dist _up indicates the distance between the initial light source information and the average value of the second area. The video apparatus for automatically adjusting white balance according to claim 1, wherein the gain is calculated by the following expression (4).
Here, R _gain and B _gain are a gain value applied to red (R) and a gain value applied to blue (B), respectively, and f _scaling is a scaling factor for scaling the input image. R _wht and b _wht are the R / (R + G + B) value and B / (R + G + B) value of the adjusted light source information, respectively, and g _wht is the G value of the light source information. The gain controller is
The red (R) of the input image is multiplied by the reciprocal of the scale factor and the red (R) gain value used in the calculation, and the reciprocal of the scale factor and the blue (B) used in the calculation. 13. The video apparatus for automatically adjusting white balance according to claim 12, wherein a white balance of the input video is adjusted by multiplying a blue value (B) of the input video by a gain value of the input video. Converting input video data sensed by an image sensor into data of a color space having a chromaticity component;
Extracting a gray region of the input video data based on a gray region of each light source detected using a plurality of light sources and a predetermined color checker;
The initial light source information is extracted using the average value and the center value in the color gamut of the input video data, and flows into the extracted gray region using the fact that a luminance difference exists between the gray component and the chromaticity component. Adjusting the initial light source information so as to eliminate the chromaticity component and extracting the final light source information;
Calculating a gain to be applied to scaling of the input video data based on the final light source information, and applying the calculated gain to the input video data to adjust white balance;
Signal processing so that the input image with the white balance adjusted is displayed;
A method for automatically adjusting the white balance of a video apparatus, comprising:   15. The method of claim 14, further comprising: performing signal processing so that the input image with the adjusted white balance is displayed. The color space is
A color space consisting of the luminance (Y) axis, R / (R + G + B) axis, B / (R + G + B) axis, or a color space consisting of the luminance (Y) axis, R / G axis, and B / G axis 16. A method for automatically adjusting white balance of a video apparatus according to claim 14 or 15. Extracting the gray region comprises:
Detecting a white point of each light source using an average value of gray patch data acquired using the plurality of light sources;
Detecting a predetermined range centered on the white point of each light source as a gray region of each light source and extracting the region including all the gray regions of each light source as a gray region of the input video data; and
The method for automatically adjusting the white balance of the video apparatus according to any one of claims 14 to 16, further comprising: Extracting the final light source information comprises:
Calculating a weighted average value in which the average value in the color gamut is reflected using the average value and the center value in the color gamut of the input video, and then extracting the initial light source information based on the weighted average value;
The gray region is divided into a low-brightness first region and a high-brightness second region by using a luminance difference between the gray component and the chromaticity component, and then the average of the first region Adjusting the initial light source information based on a value and an average value of the second region;
The method for automatically adjusting the white balance of the video apparatus according to claim 14, further comprising: Extracting the initial light source information comprises:
After the weighted average value is vertically mapped to the central axis connecting the white points of each light source, and when the weighted average value for each light source is mapped to the white point of each light source, the mapping is performed based on the movement that can minimize the error. 19. The method for automatically adjusting white balance of a video apparatus according to claim 18, wherein the weighted average value on the central axis is corrected and extracted as the initial light source information. The method according to claim 18, wherein the weighted average value is calculated by the following equation (1).
Here, r _weighted is the R / (R + G + B) value of the weighted average value, b _weighted is the B / (R + G + B) value of the weighted average value, α and β are arbitrary constants, and r _ave is R / (R + G + B) value of the average value in the color gamut, b _ave indicates the B / (R + G + B) value of the average value in the color gamut, r _gm is an R / (R + G + B) value of the center value of the color gamut, b _gm Indicates the B / (R + G + B) value of the center value of the color gamut. Adjusting the initial light source information comprises:
The gray area is divided into the first area and the second area on the basis of an average value of the maximum value and the minimum value in the gray area and a threshold luminance value determined using the average value of the gray area. 19. The method of automatically adjusting white balance of an image apparatus according to claim 18, wherein the initial light source information is adjusted using an average value of the first area and an average value of the second area. . The method of claim 21, wherein the threshold luminance value is calculated according to the following equation (2).
Here, Y _threshold is the threshold luminance value, Y _region is the average value of the average value of the first region and the average value of the second region, Y _ave is the average value of the gray region, and α and β represents an arbitrary constant. Adjusting the initial light source information comprises:
The average value having the shortest distance from the initial light source information among the average value of the first region and the average value of the second region is detected as adjusted light source information. A method for automatically adjusting the white balance of video equipment. The method of claim 23, wherein the adjusted light source information is calculated according to the following equation (3).
Here, r _wht and b _wht are the R / (R + G + B) value and B / (R + G + B) value of the adjusted light source information, respectively, and r _down and b _down are the average values of the first region, respectively. R / (R + G + B) value, B / (R + G + B) value, and r _up and b _up indicate the average R / (R + G + B) value and B / (R + G + B) value of the second region, respectively, and Dist _down Indicates the distance between the initial light source information and the average value of the first area, and Dist _up indicates the distance between the initial light source information and the average value of the second area. 16. The method for automatically adjusting white balance of a video apparatus according to claim 14 or 15, wherein the gain is calculated by the following (4).
Here, R _gain and B _gain are a gain value applied to red (R) and a gain value applied to blue (B), respectively, and f _scaling is a scaling factor for scaling the input image. R _wht and b _wht are the R / (R + G + B) value and B / (R + G + B) value of the adjusted light source information, respectively, and g _wht is the G value of the light source information. The step of adjusting the white balance includes
The red (R) of the input image is multiplied by the reciprocal of the scale factor and the red (R) gain value used in the calculation, and the reciprocal of the scale factor and the blue (B) used in the calculation. 26. The method of claim 25, wherein the white balance of the input image is adjusted by multiplying the input image blue by the blue value (B) of the input image.