WO2017043811A1

WO2017043811A1 - Image processing apparatus and image processing method

Info

Publication number: WO2017043811A1
Application number: PCT/KR2016/009833
Authority: WO
Inventors: 모리야츠요시
Original assignee: 삼성전자 주식회사
Priority date: 2015-09-09
Filing date: 2016-09-02
Publication date: 2017-03-16

Abstract

The present invention pertains to an image processing apparatus for correcting white balance by properly estimating a light source and suppressing a color failure, and an image processing method, the image processing apparatus comprising: a shading estimation unit for estimating the shading intensity of an image; a light source color estimation unit for configuring, by a color space, an achromatic color determination range corresponding to the estimated shading intensity; a correcting coefficient calculation unit for calculating a white balance gain; and an image signal processing circuit for correcting the white balance.

Description

Image processing apparatus and image processing method

The present invention relates to an image processing apparatus and an image processing method.

(Prior art document)

Publication No. 2013-115571 (Patent Document 1)

Japanese Patent Application Laid-Open No. 2015-095811 (Patent Document 2)

Image processing apparatuses such as digital still cameras adjust the white balance to accurately reproduce the subject color regardless of the light source. For example, Patent Document 1 describes that an image processing apparatus estimates a light source illuminating a subject of a captured image, calculates reliability of the light source estimation, and corrects white balance of the captured image while suppressing color failure. . Patent document 2 also describes that the image processing apparatus corrects the white balance of the captured image.

The white balance can be appropriately corrected by the image processing apparatus according to the background art. However, in the image processing apparatus according to Patent Document 1, since the reliability of the light source estimation is accumulated and processed by a plurality of frames, color deterioration may not be suppressed immediately after starting the apparatus.

SUMMARY OF THE INVENTION The present invention has been made to solve this problem, and an image processing apparatus and image processing method capable of correcting a white balance by appropriately estimating a light source illuminating a subject of the image using an image of a target to be corrected and suppressing color decay. The purpose is to provide.

An image acquisition device according to the present invention includes an image acquisition unit for acquiring an image, and a control unit for storing in advance achromatic determination range for each shading intensity and determining achromatic determination range corresponding to the shading intensity of the acquired image as a color space. It is.

An image acquisition device according to the present invention includes a control unit for calculating a white balance gain based on a pixel value of a pixel having a color coordinate included in the determined achromatic color determination range, and correcting the white balance of the image using the calculated white balance gain. It is.

The image acquisition device according to the present invention includes a control unit that calculates the reliability of the shading estimation based on the flatness of the image and sets the predetermined achromatic determination range when the reliability is less than or equal to the predetermined value to the color space.

The image acquisition device according to the present invention selects a plurality of achromatic determination ranges, generates a second achromatic determination range based on the plurality of achromatic determination ranges, and when the object luminance of the image is within a predetermined range, the second achromatic determination range Is set to a color space as the achromatic color determination range.

The image acquisition device according to the present invention applies a plurality of shading estimation coefficients to an image, selects a shading estimation coefficient to correct the image, stores achromatic determination range for each shading estimation coefficient, and achromatic determination range corresponding to the selected shading estimation coefficient. It has a control unit for setting the.

An image acquisition device according to the present invention comprises a control unit for dividing an image into a plurality of blocks and calculating block statistical values of pixel values for each block, wherein the shading estimation coefficients have coefficient values corresponding to the blocks, respectively, and the block statistical values Selects a shading estimation coefficient for correcting the image by multiplying the coefficient and the coefficient value, calculates a color coordinate of the block, and calculates a white balance gain based on the block statistical value of the block having the color coordinate included in the set achromatic determination range. It is to be provided.

The image processing apparatus according to the present invention stores in advance a shading estimating unit for estimating the shading intensity of an image, an achromatic determination range for each shading intensity, and sets an achromatic determination range corresponding to the estimated shading intensity in a color space. An image using a correction coefficient calculator that calculates a color coordinate of a pixel in the image, calculates a white balance gain based on a pixel value of a pixel having a color coordinate included in a set achromatic determination range, and the calculated white balance gain And a correction unit for correcting the white balance.

An image processing apparatus according to the present invention includes a shading estimating unit for calculating the reliability of the shading estimation based on the flatness of the image, and a light source color estimating unit for setting a predetermined achromatic determination range when the reliability is less than or equal to the predetermined value to the color space. will be.

The image processing apparatus according to the present invention selects a plurality of achromatic determination ranges, generates a second achromatic determination range based on the plurality of achromatic determination ranges, and a second achromatic determination when the object luminance of the image is within a predetermined range. It is provided with the light source color estimation part which sets a range to a color space as an achromatic color judgment range.

An image processing apparatus according to the present invention includes a shading estimating unit for selecting a shading estimation coefficient for applying a plurality of shading estimation coefficients to an image and correcting the image, and storing an achromatic determination range for each shading estimation coefficient and corresponding to the selected shading estimation coefficient. And a light source color estimating unit that sets an achromatic color determination range.

The image processing apparatus according to the present invention further includes a statistical value calculating unit for dividing an image into a plurality of blocks and calculating block statistical values of pixel values for each block, wherein the shading estimation coefficients have coefficient values corresponding to the blocks, respectively. Selects a shading estimation coefficient that multiplies the block statistics with the coefficients to correct the image, and the correction coefficient calculator calculates the color coordinates of the block and based on the block statistical values of the block having the color coordinates included in the set achromatic determination range. And a shading estimator for calculating the white balance gain.

In addition, the image acquisition method according to the present invention stores the image acquisition step, the shading estimation step, the achromatic determination range for each shading intensity in advance, and achromatic color determination range corresponding to the shading intensity of the acquired image. It is to have a step to determine.

An image acquisition method according to the present invention comprises the steps of calculating a white balance gain based on a pixel value of a pixel having a color coordinate included in the determined achromatic color determination range, and correcting the white balance of the image using the calculated white balance gain. To have.

The image acquisition method according to the present invention includes calculating a white balance gain based on a pixel value of a pixel having a color coordinate included in the determined achromatic determination range and correcting the white balance of the image using the calculated white balance gain. To have.

Estimating the shading intensity of the image acquisition method according to the present invention includes calculating the reliability of the shading estimation based on the flatness of the image, and setting the achromatic determination range to the color space has a reliability less than or equal to a predetermined value. Setting the predetermined achromatic determination range as the color space.

The step of setting the achromatic determination range of the image acquisition method according to the present invention as the color space includes selecting a plurality of achromatic determination ranges when the object luminance of the image is within a predetermined range, and based on the plurality of achromatic determination ranges, the second achromatic color determination range. Generating a determination range; And setting the second achromatic determination range as the achromatic determination range as the color space.

Estimating the shading intensity of the image acquisition method according to the present invention includes applying a plurality of shading estimation coefficients to the image to select the shading estimation coefficient to correct the image, and setting the achromatic determination range to the color space. And storing the achromatic determination range for each shading estimation coefficient, and setting the achromatic determination range corresponding to the selected shading estimation coefficient.

The image acquisition method according to the present invention further comprises the step of dividing the image into a plurality of blocks and calculating block statistical values of pixel values for each block, wherein the shading estimation coefficients have coefficient values corresponding to the blocks, respectively, and the shading intensity Estimating includes selecting a shading estimation coefficient for multiplying the block statistical value and the coefficient value to correct the image, and calculating the white balance gain includes calculating a color coordinate of the block and setting the achromatic color determination. Calculating a white balance gain based on block statistical values of blocks having color coordinates included in the range.

The image processing method according to the present invention includes the steps of storing the achromatic determination range for each shading intensity, estimating the shading intensity of the image, setting the achromatic determination range corresponding to the estimated shading intensity as the color space, Calculating the color coordinate of the pixel in the image, calculating the white balance gain based on the pixel value of the pixel having the color coordinate included in the set achromatic determination range, and correcting the white balance of the image using the calculated white balance gain. To have a step.

Estimating the shading intensity of the image processing method according to the present invention includes calculating the reliability of the shading estimation based on the flatness of the image, and setting the achromatic determination range to the color space has a reliability equal to or less than a predetermined value. Setting the predetermined achromatic determination range as the color space.

The step of setting the achromatic determination range of the image processing method according to the present invention as the color space includes selecting a plurality of achromatic determination ranges when the object luminance of the image is within a predetermined range, and based on the plurality of achromatic determination ranges, the second achromatic color determination range. Generating a determination range and setting the second achromatic determination range as the achromatic determination range to the color space.

Estimating the shading intensity of the image processing method according to the present invention includes applying a plurality of shading estimation coefficients to the image to select a shading estimation coefficient for correcting the image, and setting the achromatic determination range to the color space. And storing the achromatic determination range for each shading estimation coefficient, and setting the achromatic determination range corresponding to the selected shading estimation coefficient.

The image processing method according to the present invention has a step of dividing an image into a plurality of blocks, calculating block statistical values of pixel values for each block, and the shading estimation coefficients have coefficient values corresponding to the blocks, respectively, to estimate the shading intensity. The step of multiplying the block statistics and the coefficient value to select the shading estimation coefficient for correcting the image, the step of calculating the white balance gain, the step of calculating the color coordinate of the block and the set achromatic color determination range Calculating a white balance gain based on the block statistic value of the block having the color coordinates included.

According to the present invention, it is possible to provide an image processing apparatus and an image processing method for properly estimating a light source, suppressing color decay, and correcting white balance.

1 is a block diagram showing the configuration of an image acquisition device 100 according to an embodiment.

2 is a flowchart showing a processing procedure of an image acquisition method according to the embodiment.

3 is a block diagram showing the configuration of a digital still camera 300 according to the embodiment.

4 is a diagram for explaining color temperature and shading intensity of a representative light source.

FIG. 5 is a diagram showing a result of shading estimation of an image captured by a natural image photograph as a subject under each light source.

6 is a flowchart showing a processing procedure of a normalization gain calculation method according to the embodiment.

7 is a flowchart showing a processing procedure of a block weight Bw calculation method according to the embodiment.

8 is a diagram for explaining a G level weight according to the embodiment.

It is a figure for demonstrating the histogram and histogram weight of the block color evaluation value Hb which concerns on embodiment.

10 is a diagram for explaining a block weight Bw according to the embodiment.

11 is a flowchart showing a processing procedure of an image processing method according to the embodiment.

12 is a diagram for explaining a light source estimation method according to the embodiment.

It is a figure which shows the achromatic determination range for every estimated light source which concerns on embodiment.

It is a figure for demonstrating the generation method of the 2nd achromatic determination range which concerns on embodiment.

EMBODIMENT OF THE INVENTION Hereinafter, with reference to drawings, the image acquisition apparatus, image processing apparatus, image acquisition method, and image processing method which concern on this embodiment are demonstrated.

The image acquisition device, the image processing device, the image acquisition method, and the image processing method according to the present embodiment estimate a light source that illuminates the subject from the shading intensity of the picked-up image by using a different shading intensity of the picked-up image according to the light source at the time of picking-up. In this case, the color reduction judgment range when calculating the white balance gain is changed to suppress the color decay.

1 is a block diagram illustrating a configuration of an image acquisition apparatus 100 according to an exemplary embodiment.

Referring to FIG. 1, the image acquisition apparatus 100 may include an image acquisition unit 110 and a control unit 120. The image acquisition device 100 may include, for example, a digital still camera that is an electronic imaging device.

The image acquisition unit 110 may include an image pickup device, and may form an image of a subject on an image pickup surface such as an image pickup device, and convert the image signal into digital signals by A / D conversion.

The controller 120 determines an achromatic determination range corresponding to the shading intensity of the image acquired through the image acquisition unit 110, and adjusts the white balance gain based on the pixel value of the pixel having the color coordinate included in the determined achromatic determination range. The white balance of the image can be corrected using the calculated white balance gain.

FIG. 2 is a flowchart for describing a method of operating the image capturing apparatus 100 illustrated in FIG. 1, according to an exemplary embodiment.

In operation 101, the image obtaining unit 110 of the image obtaining apparatus 100 obtains an image. (Step S101)

In operation 102, the control unit 120 of the image acquisition apparatus 100 determines the achromatic determination range corresponding to the shading intensity of the acquired image. (Step S102)

According to an exemplary embodiment, the controller 120 may calculate the reliability of the shading estimation based on the obtained flatness of the image, and set a predetermined achromatic determination range as the color space when the reliability is less than a predetermined value.

According to an embodiment, when the object luminance of the image is within a predetermined range, the controller 120 selects a plurality of the achromatic determination ranges, and generates a second achromatic determination range based on the plurality of achromatic determination ranges. A second achromatic determination range can be set to the color space as the achromatic determination range.

According to an exemplary embodiment, the controller 120 selects a shading estimation coefficient for correcting the image by applying a plurality of shading estimation coefficients to an image, stores a colorless determination range for each shading estimation coefficient, and corresponds to the selected shading estimation coefficient. An achromatic judgment range can be set.

In operation 103, the controller 120 of the image acquisition apparatus 100 may calculate the white balance gain based on the pixel value of the pixel having the color coordinate included in the determined achromatic color determination range. (Step S103)

According to an exemplary embodiment, the controller 120 may divide an image into a plurality of blocks, and calculate block statistical values of pixel values for each block. Shading estimation coefficients may have coefficient values corresponding to the blocks, respectively. The control unit 120 multiplies the block statistic value with the coefficient value, selects a shading estimation coefficient for correcting the image, calculates a color coordinate of the block, and the block of the block having the color coordinate included in the set achromatic determination range. The white balance gain may be calculated based on a statistical value.

In operation 104, the controller 120 may correct the white balance of the image by using the calculated white balance gain. (Step S104)

Next, the structure of the image processing apparatus which concerns on this embodiment is demonstrated using the structure of the digital still camera 300 which is an example of an image processing apparatus.

3 is a block diagram showing the configuration of the digital still camera 300 according to the present embodiment.

The digital still camera 300, which is an electronic imaging device, includes a lens optical system 302, an imaging device 304, an AFE circuit 306, an image signal processing circuit 308, an image display unit 310, an image recording unit 312, and a driver. 314, a timing generator (TG) 316, an image block statistics circuit 318, a shading control unit 320, a shading correction coefficient storage unit 322, a white balance control unit 330, and the like.

The lens optical system 302 has a lens, an iris, a shutter, and the like, and an image of a subject can be formed on the imaging surface of the imaging device 304. The imaging element 304 is an image sensor such as a CCD or a CMOS, and an infrared cut filter (not shown) may be mounted on the lens optical system 302 side of the imaging element 304. The image pickup device 304 can photoelectrically convert the subject image to acquire an image signal (RGB color signal). The AFE circuit 306 can A / D convert an image signal acquired by the imaging element 304 and subjected to signal processing by a CDS circuit (not shown) to be a digital signal.

The image signal processing circuit 308 can perform demosaicing processing, edge emphasis processing, shading correction processing, white balance (WB) correction processing, gamma correction processing, and the like on the image signal output from the AFE circuit 306. The image display unit 310 is a liquid crystal display (LCD) or the like, and can display an image signal subjected to various processes in the image signal processing circuit 308. The image recording unit 312 can record an image signal subjected to various processes in the image signal processing circuit 308 as a memory.

The driver 314 may drive a lens, an iris, and a shutter of the lens optical system 302. The timing generator 316 may generate timing for driving the imaging device 304.

The image block statistics circuit 318 divides the picked-up image region or the partial region of the picked-up image region of the image signal that has become a digital signal in the AFE circuit 306 into a grid to form a plurality of blocks, and calculates a block statistical value for each block. can do. The image block statistics circuit 318 may calculate pixel integrated values for RGB or pixel average values for RGB in each block as block statistics.

In addition, the image block statistics circuit 318 can calculate the object luminance, that is, the average luminance of the captured image region. As an average brightness calculation method, the method of patent document 1 can be employ | adopted, for example.

The shading controller 320 may include a shading estimator 324, a shading correction coefficient calculator 326, and the like. The shading estimating unit 324 estimates the shading intensity based on the block statistical value calculated by the image block statistics circuit 318 and the shading estimation coefficient stored in the shading correction coefficient storage unit 322, and estimates the shading suitable for the image. The coefficient can be selected. In the shading correction coefficient storage unit 322, shading estimation coefficients may be prepared for each light source and for each R value, G value, and B value. The shading estimation coefficient may have a coefficient value for each block.

The shading estimation coefficient can be calculated using the image of the white chart image | photographed under each light source. In the white chart, it is preferable to use a uniformly diffuse reflecting surface having a spectral reflectance of a standard white reflector or the like being constant over the target wavelength region and 90% or more. For a light source such as a light bulb containing a lot of infrared light, the shading estimation coefficient for strongly correcting the R signal in the image periphery can be calculated. For a light source such as a fluorescent lamp that does not contain much infrared light, a shading estimation coefficient for weakly correcting the R signal at the periphery of the image may be calculated.

In addition, the shading estimator 324 may calculate the reliability of the shading estimation. The selected shading estimation coefficient and the calculated reliability of the shading estimation may be output to the white balance controller 330.

The shading correction coefficient calculation unit 326 selects the shading correction coefficient corresponding to the shading estimation coefficient selected by the shading estimation unit 324 from the shading correction coefficient storage unit 322, and the image signal processing circuit 308 selects the shading correction coefficient. Shading can be corrected. In the shading correction coefficient storage unit 322, shading correction coefficients may be prepared for each light source and for each R value, G value, and B value. The shading correction coefficient may have a coefficient value for each pixel or for each of the plurality of pixels.

The shading correction coefficient storage unit 322 may store a set of shading estimation coefficients and shading correction coefficients for each light source such as sunlight, a light bulb, a fluorescent lamp, and the like. In most cases, the shading correction coefficient and the shading estimation coefficient are in a one-to-one set, but are not limited thereto. The shading estimation coefficient and the shading correction coefficient may be calculated by an external device such as a PC. The shading correction coefficients may be the same as the shading estimation coefficients and different from the shading estimation coefficients.

The white balance controller 330 may include a light source color estimator 332, a white balance correction coefficient calculator 334, and the like.

The light source color estimator 332 may estimate a light source or a light source color that illuminates a subject of the captured image based on the shading estimation result, and select and set an achromatic determination range that defines a color coordinate range on the color space. In detail, the light source color estimating unit 332 stores the shading estimation coefficient, the light source (light source color), and the achromatic determination range in association with each other, and may select an achromatic determination range corresponding to the shading estimation coefficient selected by the shading estimation unit 324.

For this reason, the light source color estimation part 332 may have the achromatic determination range memory | storage part (not shown) which stores the achromatic determination range.

The white balance correction coefficient calculation unit 334 may convert the block statistical value calculated by the image block statistics circuit 318 into the color coordinates in the above-described color space.

Then, it is determined whether the color coordinate of each block is included in the achromatic determination range selected by the light source color estimation unit 332, and the white balance gain is obtained by using the block statistical value of the block included in the achromatic determination range, that is, the block determined as achromatic. Can be calculated.

In addition, each component realized by the shading control part 320 and the white balance control part 330 is controlled by the computing device (not shown) which the shading control part 320 and the white balance control part 330 which are computers are equipped, for example. This can be achieved by executing a program.

More specifically, the shading control unit 320 and the white balance control unit 330 can load the program stored in the storage unit (not shown) into the main storage device (not shown), and execute the program under the control of the computing device to realize it. have. In addition, each component is not limited to software implemented by a program, and may be implemented by any combination of hardware, firmware, and software.

The program described above may be stored and supplied to a computer using various types of non-transitory computer readable medium. The non-transitory computer readable medium may include a tangible storage medium having various types of entities.

Examples of non-transitory computer readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (eg, magneto-optical disks), CD-ROMs (Read Only Memory), CD- R, CD-R / W, semiconductor memory (for example, mask ROM, programmable ROM (PROM), erasable PROM (EPROM), flash ROM, random access memory (RAM)).

In addition, the program may be supplied to the shading control unit 320 and the white balance control unit 330 by various types of temporary computer readable medium. Examples of transitory computer readable media may include electrical signals, optical signals, and electromagnetic waves. The temporary computer readable medium can supply a program to a computer through a wired communication path such as wires and optical fibers or a wireless communication path.

Next, an operation of the digital still camera 300 according to the present embodiment, that is, an image processing method will be described.

First, the outline | summary of the light source estimation process of the digital still camera 300 which concerns on this embodiment can be demonstrated.

4 is a diagram for describing color temperature and shading intensity of a representative light source. The horizontal axis represents color temperature and the vertical axis represents shading intensity.

As for the color temperature, the bulb has a low color temperature and daylight has a high color temperature. However, fluorescent lamps and LEDs have a wide range of color temperatures: bulb color (L), white (W), and main white (N, D). For this reason, even if it has the same color temperature, it may be another light source, and it is difficult to estimate a light source from a color temperature.

On the other hand, in terms of shading intensity, shading is strongly generated in a captured image in a light source such as a light bulb or a daylight containing a large amount of near infrared rays. In addition, in order to increase the luminous efficiency, shading is weakly generated in the captured image in a light source such as a fluorescent lamp or an LED containing almost no near infrared light. For this reason, if the shading intensity of the picked-up image is within the interval α shown in Fig. 4, the light source is a light bulb, if it is within the interval β, it is a light bulb or a chief light, if it is within the interval γ, it is a primary light, and if it is within the interval δ, the shading of the captured image The light source can be estimated from the intensity.

In the light source estimation process according to the present embodiment, the light source can be estimated from the shading intensity of the captured image in this manner.

In some captured images, this correspondence between the shading intensity and the light source is not well established. For example, the corresponding relationship is not well established in the captured image when a plurality of light sources such as a light bulb and a fluorescent lamp are present or the subject is a blue sky or a red object under daylight.

FIG. 5 is a diagram showing a result of shading estimation of an image captured by a natural image photograph as a subject under each light source. Shading estimation results are obtained for images taken using 75 natural image photographs having various color schemes under four kinds of light sources (daylight, white fluorescent light, natural fluorescent light, and electric bulb) as subjects. The horizontal axis represents the kind of shading estimation coefficient (number of shading estimation coefficient), and the vertical axis represents the number of images in which the shading estimation coefficient of the number is selected as a result of estimating the shading.

For example, referring to 510 of FIG. 5, when the light source is daylight, the shading estimation coefficient of the number 6 for the near 40 images can be selected as appropriate to correct the shading of the image. In this case, the larger the number of the shading estimation coefficient is, the stronger the shading correction can be in response to the image in which the occurrence of shading is more pronounced (the image with larger shading intensity).

In addition, hatching is attached | subjected to the bar corresponding to the estimation result (number of shading estimation coefficient) when the shading is estimated about the image image | photographed using the white plate as a subject under each light source. For example, referring to 520 of FIG. 5, since the shading estimation coefficient of No. 1 corrects the shading most appropriately for an image photographed with a white plate under a white fluorescent lamp, the hatching of the shading estimation coefficient of No. 1 is hatched. have.

Similarly, referring to 530 of FIG. 5, when the light source is a natural fluorescent lamp, the shading estimation coefficient of No. 2 properly corrects the shading of the image, and therefore hatching is given to the bar of the estimation coefficient of No. 2. FIG. Finally, referring to 540 of FIG. 5, since the shading estimation coefficient of No. 7 corrects the shading most appropriately for the image of the white plate photographed under the bulb, the hatching of the shading estimation coefficient of No. 7 is given.

Since shading varies in characteristics depending on the light source and the subject color, about half of the shading estimation results of the natural image photographed image match the shading estimation result of the image photographing the white plate in any light source. .

Therefore, if the shading estimation result matches the shading estimation result of the image picked up the white plate as the reliability of the shading estimation result, the light source can be estimated more appropriately.

In general, the result of the shading estimation of an image having a large area of the same color distribution and the result of the shading estimation of an image photographing a white plate coincide.

Therefore, in the image processing method according to the present embodiment, it is determined by calculating the block weight Bw as follows to determine whether the captured image is an image in which a subject having the same color distribution exists in a large area as the reliability of the shading estimation result. As described above, the shading estimation unit 324 may calculate the reliability of the shading estimation result.

The shading estimating unit 324 uses the result of multiplying the block statistical values of the image of the target to be corrected by the white balance by the coefficient values of the shading estimation coefficients (hereinafter, referred to as "shading estimation initial coefficients") for the lowest shading correction. Normalization gain is calculated, the block color evaluation value Hb is calculated using the normalization gain, and the block weight Bw is calculated using the block color evaluation value Hb and the G level weight. have.

6 is a flowchart showing the processing procedure of the normalization gain calculation method according to the present embodiment.

The shading estimator 324 may reduce the shading of the target image and calculate the normalization gain by multiplying the block statistic before shading correction by the coefficient value of the initial coefficient of shading to normalize the R, G, and B values of the block to be evaluated. have.

That is, the block detection data can be multiplied by the shading estimation initial coefficient (step S201). When the block statistic values are Ri, Gi, and Bi, and the coefficient values of the shading estimation initial coefficients are Nr, Ng, and Nb, the block statistics values Rn (N, M) and Gn (N after multiplying the shading estimation initial coefficients , M), and Bn (N, M)) are represented by the formula (1).

Next, the block statistical values (Rn (N, M), Gn (N, M), Bn (N, M)) after multiplying the shading estimation initial coefficients are integrated as a whole block (steps S202, S203), and the block statistics Normalization gain can be calculated from the ratio of values. Here, the normalization gain of the G pixel is set to 1.0, and the normalization gain of the R pixel and the B pixel can be obtained.

Specifically, the normalization gain can be obtained by using Equation (2).

That is, it is determined whether the integrated value of Rn (N, M) or Bn (N, M) is 0 (steps S204, S207), and when 0, normalization gain R / Average Gain B of R pixels or B pixels. ) Can be set to 1.0 (steps S206, S209). On the other hand, when the integrated value of Rn (N, M) or Bn (N, M) is not 0, the normalized gain of the R pixel = G integrated value / the R integrated value or the normalized gain of the B pixel = G integrated value / B integrated The value can be set (steps S205 and S208). In order to prevent the overflow, the normalized gain of the R pixel or the B pixel can be clipped with the upper limit to the value of the normalization gain upper limit (R, B) (it can be set as the upper limit).

7 is a flowchart showing the processing procedure of the block weight Bw calculation method according to the present embodiment.

Subsequently to the operation shown in FIG. 6, the shading estimating unit 324 may calculate the G level weight for the purpose of excluding a block near the dark portion or the saturation from the evaluation target (step S210).

8 is a diagram for explaining a G level weight according to the present embodiment.

The G level weight refers to a block between the upper and lower threshold values set in advance by referring to the level of the G value of the block statistical value after multiplying the block data by the shading estimation coefficient to 1.0, and to a block outside the threshold range to 0.0. can do. In addition, a block out of a range in which any one of an R pixel value, a G pixel value, and a B pixel value is preset may be excluded from the evaluation target block.

Next, the block color evaluation value Hb of all the blocks can be calculated (step S211). As in Eq. (3), the normalized gain is multiplied by the block statistical values (Gn (N, M), Bn (N, M)) after multiplying the initial shading estimation coefficients obtained in Equations (1) and (2). The block color evaluation value Hb (N, M) can be calculated.

Next, a histogram of the block color evaluation value Hb is generated (step S212), and the histogram weight can be calculated for the purpose of excluding blocks separated from the mode (steps S213 to S215 and S217).

9 is a diagram for explaining the histogram and the histogram weight of the block color evaluation value Hb according to the present embodiment. The histogram is an example when the block color evaluation value Hb = 0.65 ~ 1.35 is set in 0.05 unit division.

Specifically, the mode of the histogram is searched (step S213), and for each block, it is determined whether the block color evaluation value Hb is within the valid range from the histogram mode (step S214), and is within the valid range as shown in FIG. The histogram weight can be set to 1 (step S215), and the histogram weight can be set to 0 when outside the effective range (step S217). And a block of histogram weight = 1 can be selected as the evaluation target block.

Next, the block weight Bw may be generated by multiplying the G level weight and the histogram weight (step S216).

10 is a diagram for explaining a block weight Bw according to the present embodiment. The block weight Bw of the image peripheral block may be one.

These processes (steps S214 to S218) can be repeated to calculate the block weight Bw for the entire block.

And the ratio of the number of blocks which became block weight Bw = 1 with respect to the total number of blocks of a picked-up image can be calculated. As the ratio becomes larger, it is possible to determine that the captured image is an image in which a subject having the same color distribution exists in a large area, and this ratio is used as the reliability of the shading estimation result, and the reliability and threshold are compared to determine the reliability height of the shading estimation result. can do.

In addition, it may be determined using another method whether the captured image is an image in which a subject having the same color distribution exists in a large area. For example, the determination may be made using the degree of dispersion of all block statistics.

That is, based on the flatness of the picked-up image, such as the ratio of blocks having a block weight Bw = 1, the degree of dispersion of all block statistics, and the like, the picked-up image may be determined to be an image having a large area of the same color distribution. Can be.

In the light source estimation process according to the present embodiment, the probability of the shading estimation result can be obtained in this way and the probability of the light source estimation can be obtained.

Next, the white balance gain calculation processing of the digital still camera 300 according to the present embodiment will be described.

11 is a flowchart showing the processing procedure of the image processing method according to the present embodiment.

When the digital still camera 300 picks up the subject, the image block statistics circuit 318 can calculate the subject luminance, the block statistics value, and the like of the picked-up image as described above.

Then, the shading estimating unit 324 can estimate the shading of the captured image using the block statistics value, the shading estimation coefficient, and the like (step S10). About this shading estimation process, the processing method described in patent document 2, Unexamined-Japanese-Patent No. 2013-198041, etc. can be employ | adopted, and detailed description can be abbreviate | omitted here. The shading estimator 324 may output a shading estimation coefficient suitable for the captured image, a reliability of the shading estimation result, and the like as a result of the shading estimation.

Next, the light source color estimating unit 332 may estimate the light source based on the object brightness, the result of the shading estimation, and the like, and select and set the achromatic color determination range (step S20).

12 is a diagram for explaining a light source estimation method according to the present embodiment. The light source color estimating unit 332 previously stores a table indicating the relationship between the object luminance, the shading estimation coefficient number, and the estimated light source as shown in FIG. In the table, a mark indicates an option when the shading estimation reliability is low. The larger the shading estimation coefficient number is, the more strongly the shading of the image can be corrected in response to the image with the higher shading intensity.

Fig. 13 is a diagram illustrating the achromatic color determination range for each of the estimated light sources according to the present embodiment. The color coordinate range of the achromatic determination range for each estimated light source set in the G / R-G / B color space is shown.

The light source color estimating unit 332 may estimate the light source with reference to the table shown in FIG. 12, and select the achromatic color determination range shown in FIG. 13.

For example, the light source color estimator 332 estimates that the light source is sunlight + light bulb HL when the subject luminance is low, the shading estimation coefficient selected as a result of the shading estimation is number 5, and the reliability of the shading estimation is high. As a colorless determination range, the one corresponding to sunlight + light bulb (HL) can be selected.

Further, the light source color estimating unit 332 estimates that the light source is the entire light source A regardless of the shading estimation result when the subject luminance is low, the shading estimation coefficient of the shading estimation result is number 5, and the reliability of the shading estimation is low. (A) corresponding to all the light sources can be selected as the achromatic color determination range.

In addition, the light source color estimating unit 332 has a brightness of the subject, the shading estimation coefficient of the shading estimation result is number 5, and when the reliability of the shading estimation is high, the light source is sunlight + light bulb HL or sunlight H. It is estimated that the achromatic color determination range corresponds to sunlight + light bulb (HL) and sunlight (H), that is, two ranges.

Further, the light source color estimating unit 332 estimates that the light source is sunlight (H) irrespective of the result of shading estimation and the reliability of the shading estimation when the object luminance is high, and corresponds to sunlight as an achromatic determination range (H). Can be selected.

In addition, the brightness of the subject can be determined, for example, low brightness, medium brightness, and high brightness with respect to 4.0 and 8.0 of the Bv value (Brightness value).

Next, the light source color estimating unit 332 generates a new range (hereinafter referred to as a "second achromatic determination range") using these ranges when the object luminance is the medium luminance and two achromatic determination ranges are selected in step S20. (Step S30).

14 is a diagram for explaining a method for generating the second achromatic determination range according to the present embodiment. In step S20, a method of generating a second achromatic determination range S when selecting a fluorescent lamp, an M corresponding to an LED, and an H corresponding to sunlight (H) is selected as the achromatic determination range.

That is, a plurality of corresponding points a1 to a6 and corresponding points b1 to b6 are defined in advance on the boundary between the two achromatic determination ranges M and H, and the interpolation points between the corresponding points according to the subject luminance (Bv value) ( c1 to c6), and the second achromatic determination range can be generated by connecting the interpolation points c1 to c6 with lines. At this time, if the subject luminance is close to the low luminance side, that is, the Bv value is close to 4, the interpolation point c1 is generated on the side near the corresponding point a1 of the achromatic determination range M on the low luminance side, and the subject If the luminance is close to the high luminance side, that is, the Bv value is close to 8, the interpolation point c1 can be generated on the side near the corresponding point b1 of the achromatic color determination range H on the high luminance side. For example, when the Bv value is 5, the interpolation point c1 may be generated at a position where the correspondence point a1 and the correspondence point b1 are internally 1: 3.

If the achromatic determination range is defined as an n-square, the process of generating the second achromatic determination range by connecting the interpolation points with a line can be simplified.

Next, the white balance correction coefficient calculation unit 334 calculates the color coordinates for each block, determines whether the color coordinates fall within the achromatic determination range selected in step S20 or the second achromatic determination range generated in step S30, so that the color coordinates are achromatic. The block included in the determination range or the second achromatic determination range may be determined to be achromatic, and the white balance gain may be calculated using the achromatic block (step S40).

In the image processing method according to the present embodiment, the calculation method itself of the white balance gain can be used using the calculation method of the prior art. For example, you may calculate a white balance gain based on the pixel average value of the block (achromatic block) considered to be a light source color.

Then, the white balance gain calculated by the white balance correction coefficient calculation unit 334 is input to the image signal processing circuit 308, and white balance correction can be performed by the image signal processing circuit 308.

As described above, the image processing apparatus or the image processing method according to the present embodiment estimates the light source that illuminates the subject from the shading intensity of the captured image by using a different shading intensity of the captured image according to the light source at the time of imaging, and then obtains the white balance gain. The color reduction judgment range at the time of calculation is changed to suppress the color decay.

In the image processing apparatus or the image processing method according to the present embodiment, the light source is estimated according to the type of the shading estimation coefficient and the achromatic determination range set for each light source is selected, but the achromatic determination range is set for each type of the shading estimation coefficient. For example, the achromatic determination range may be selected according to the type of the shading estimation coefficient. In other words, the procedure for estimating the light source may be omitted.

In the image processing apparatus or the image processing method according to the present embodiment, the shading intensity is estimated according to the type of the shading estimation coefficient, but the ratio of the pixel value at the center of the captured image to the pixel value at the periphery of the image, or the image The shading intensity may be estimated from the ratio of the block statistics at the center and the block statistics at the periphery of the image. Then, an estimated light source or an achromatic determination range is set in advance for each range of the ratio of the pixel values or for each ratio of the ratio of the block statistical values, and the achromatic determination range is selected according to the ratio of the measured pixel values or the ratio of the block statistical values. It can be done.

In the image processing apparatus or the image processing method according to the present embodiment, the light source is estimated and the achromatic determination range is selected using the subject luminance, the result of the shading estimation, and the reliability of the shading estimation. In order to prioritize the estimation of the light source using the subject luminance and the shading estimation result without using the reliability of the shading estimation, and to select the achromatic determination range, the result of the shading estimation without using the reliability of the subject luminance and the shading estimation Only the light source may be used to estimate the achromatic determination range. That is, the type of the shading estimation coefficient, the estimated light source and the achromatic determination range may be stored in association with each other, and the light source may be estimated and the achromatic determination range may be selected based on the type of the selected shading estimation coefficient.

In addition, in the image processing apparatus or the image processing method according to the present embodiment, the achromatic determination range is set using the G / RG / B color space, but a different color space may be used, and the achromatic determination range according to the color space may be used. May be set.

In addition, in the image processing apparatus or the image processing method according to the present embodiment, the color coordinates are calculated for each block and determined to be included in the achromatic color determination range, thereby calculating the white balance gain. However, the color coordinates are calculated for each pixel, and the color coordinates of the pixel are achromatic color determination range. The white balance gain may be calculated using the pixel value by determining whether it is included in the. In other words, instead of processing each step of the image processing in units of blocks, the processing may be performed in units of pixels.

Further, in the image processing apparatus or the image processing method according to the present embodiment, the two achromatic determination ranges are generated and set by selecting two achromatic determination ranges when the subject luminance is the medium luminance, but by selecting two or more achromatic determination ranges, The second achromatic determination range may be generated, or one achromatic determination range may be selected and set.

As described above, the digital still camera 300 according to the present embodiment stores in advance the shading estimating unit 324 for estimating the shading intensity of the image and the achromatic determination range for each shading intensity, and corresponds to the estimated shading intensity. A color coordinate of the light source color estimating unit 332 for setting the achromatic color determination range to be the color space, and a pixel value of the pixel having the color coordinates included in the set achromatic color determination range, and calculating the white balance gain. The correction coefficient calculation unit 334 and the image signal processing circuit 308 for correcting the white balance of the image using the calculated white balance gain are provided.

With such a configuration, it is possible to properly estimate the light source, suppress the color decay, and correct the white balance.

Further, the digital still camera 300 according to the present embodiment calculates the reliability of the shading estimation based on the flatness of the image by the shading estimating unit 324, and the light source color estimation unit 332 has the reliability lower than or equal to the predetermined value. It is preferable to set a predetermined achromatic determination range as the color space.

With such a configuration, it is possible to more appropriately estimate the light source based on the reliability of the shading estimation to further suppress the color decay.

Further, in the digital still camera 300 according to the present embodiment, the light source color estimating unit 332 selects a plurality of achromatic determination ranges when the object brightness of the image is within a predetermined range, and based on the plurality of achromatic determination ranges, It is preferable to generate an achromatic determination range and set the second achromatic determination range as the colorless determination range to the color space.

By such a configuration, the white achromatic determination range can be set more appropriately as the second achromatic determination range, and the white balance gain can be calculated with higher accuracy.

In addition, in the digital still camera 300 according to the present embodiment, the shading estimating unit 324 applies a plurality of shading estimation coefficients to the image to select the shading estimation coefficient to properly correct the image, and the light source color estimation unit 332 It is preferable to store the achromatic determination range for each shading estimation coefficient and to set the achromatic determination range corresponding to the selected shading estimation coefficient.

With such a configuration, it is possible to easily and easily estimate the light source based on the selected shading estimation coefficient to suppress the color decay.

In addition, the digital still camera 300 according to the present embodiment further includes an image block statistics circuit 318 for dividing an image into a plurality of blocks and calculating block statistical values for which the pixel values are statistically calculated for each block, and the shading estimation The coefficients each have a coefficient value corresponding to the block, the shading estimator 324 multiplies the block statistics with the coefficient value and selects the shading estimation coefficients to properly correct the image, and the correction coefficient calculator 334 selects the It is preferable to calculate a color coordinate and calculate a white balance gain based on the block statistical value of the block which has a color coordinate contained in the set achromatic determination range.

With such a configuration, it is possible to appropriately estimate the light source and appropriately correct the white balance while reducing the total calculation amount from the shading estimation to the white balance gain calculation.

Further, the image processing method according to the present embodiment sets a color space for storing the achromatic determination range for each shading intensity, step S10 for estimating the shading intensity of the image, and achromatic determination range corresponding to the estimated shading intensity. And calculating the white balance gain based on the pixel coordinates of the pixels having the color coordinates included in the set achromatic determination range, and calculating the color coordinates of the pixels in the image. To correct the white balance of the image.

Claims

In the image acquisition device,

An image acquisition unit which acquires an image;

And a control unit which stores in advance the achromatic determination range for each shading intensity, and determines the achromatic determination range corresponding to the shading intensity of the obtained image as the color space.
The method of claim 1,

And the controller calculates a white balance gain based on pixel values of pixels having color coordinates included in the determined achromatic color determination range, and corrects the white balance of the image using the calculated white balance gain.
The method of claim 2,

And the control unit calculates the reliability of the shading estimation based on the flatness of the image, and sets a predetermined achromatic determination range when the reliability is less than or equal to the predetermined value to the color space.
The method of claim 2,

The control unit selects a plurality of the achromatic determination ranges when the object luminance of the image is within a predetermined range, generates a second achromatic determination range based on the plurality of achromatic determination ranges, and generates the second achromatic determination range. An image acquisition device, which sets the color space as the achromatic color determination range.
The method of claim 2,

The control unit selects a shading estimation coefficient for correcting the image by applying a plurality of shading estimation coefficients to the image, stores the achromatic determination range for each shading estimation coefficient, and determines the achromatic determination range corresponding to the selected shading estimation coefficient. An image acquisition device to set.
The method of claim 5,

The control unit divides the image into a plurality of blocks, calculates a block statistical value of pixel values for each block,

The shading estimation coefficients each have a coefficient value corresponding to the block,

The control unit selects a shading estimation coefficient for correcting the image by multiplying the block statistical value and the coefficient value, calculating a color coordinate of the block, and the block of the block having color coordinates included in the set achromatic determination range. And calculating the white balance gain based on a statistical value.
In the image acquisition method,

Obtaining an image;

Estimating the shading intensity; And

And storing the achromatic determination range for each shading intensity in advance and determining the achromatic determination range corresponding to the shading intensity of the obtained image as the color space.
The method of claim 7, wherein

Calculating a white balance gain based on pixel values of pixels having color coordinates included in the determined achromatic color determination range; And

Correcting the white balance of the image using the calculated white balance gain.
The method of claim 8,

Estimating the shading intensity includes calculating a reliability of the shading estimate based on the flatness of the image,

And setting the achromatic color determination range to a color space includes setting a predetermined achromatic color determination range to the color space when the reliability is less than or equal to a predetermined value.
The method of claim 8,

The setting of the achromatic color determination range to a color space may include:

Selecting a plurality of said achromatic determination ranges, and generating a second achromatic determination range based on said plurality of achromatic determination ranges, when a subject luminance of said image is within a predetermined range; And

And setting the second achromatic determination range as the achromatic determination range as the color space.
The method of claim 8,

Estimating the shading intensity includes applying a plurality of shading estimation coefficients to the image to select a shading estimation coefficient to correct the image,

The setting of the achromatic determination range as a color space includes storing the achromatic determination range for each shading estimation coefficient and setting an achromatic determination range corresponding to the selected shading estimation coefficient.
The method of claim 11,

Dividing the image into a plurality of blocks, and calculating block statistical values of pixel values for each block;

The shading estimation coefficients each have a coefficient value corresponding to the block,

Estimating the shading intensity comprises multiplying the block statistics with the coefficients to select a shading estimation coefficient to correct the image,

Calculating the white balance gain,

Calculating color coordinates of the block; And

And calculating the white balance gain based on the block statistical value of the block having color coordinates included in the set achromatic color determination range.
A computer-readable recording medium having recorded thereon a program for executing the method of claim 7 to a computer.